summaryrefslogtreecommitdiffstats
path: root/src/tools/vppapigen/vppapigen.py
AgeCommit message (Collapse)AuthorFilesLines
2020-10-13stats: counters data modelOle Troan1-8/+97
This adds a new data model for counters. Specifying the errors severity and unit. A later patch will update vpp_get_stats to take advantage of this. Only the map plugin is updates as an example. New .api language: A new "counters" keyword to define counter sets. counters map { none { severity info; type counter64; units "packets"; description "valid MAP packets"; }; bad_protocol { severity error; type counter64; units "packets"; description "bad protocol"; }; }; Each counter has 4 keywords. severity, which is one of error, info or warn. A type, which is one of counter64 or gauge64. units, which is a text field using units from YANG. paths { "/err/ip4-map" "map"; "/err/ip6-map" "map"; "/err/ip4-t-map" "map"; "/err/ip6-t-map" "map"; }; A new paths keyword that maps the counter-set to a path in the stats segment KV store. Updated VPP CLI to include severity so user can see error counter severity. DBGvpp# show errors Count Node Reason Severity 13 ethernet-input no error error Type: feature Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: Ib2177543f49d4c3aef4d7fa72476cff2068f7771 Signed-off-by: Ole Troan <ot@cisco.com>
2020-10-06feature: Add packet trace APIJon Loeliger1-4/+10
Also spiffed up the vpp_api_test plugin loader so it executes VLIB_INIT_FUNCTIONs and VLIB_API_INIT_FUNCTIONs. Type: feature Change-Id: Id9a4f455d73738c41bcfea220df2112bb9679681 Signed-off-by: Jon Loeliger <jdl@netgate.com> Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Dave Barach <dave@barachs.net>
2020-09-11build: fix build for Debian 9 and Debian 10Benoît Ganne1-4/+5
Type: fix Change-Id: Ic07d0ae313b32e420ba93693cb75960a86f752a9 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2020-09-01vppapigen: supports backwards compatible marking of enumsOle Troan1-18/+38
enum bar_enum { BAR1 = 0, BAR2, BAR3 [backwards_compatible], BAR4 = 9 [backwards_compatible], }; This allows adding backwards compatible (as guaranteed by the developer) enums. The enums marked backwards compatible are not considered in the CRC calculation. Type: improvement Change-Id: I6fc0c21b19e1a02cff7f5d279a0f3a32d2f8b630 Signed-off-by: Ole Troan <ot@cisco.com>
2020-08-13vppapigen: remove unreachable codeOle Troan1-1/+0
Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: Ifd81acaa020f6ebff20ec4eff4370d8a18323860
2020-08-11vppapigen: add support for empty optionsOle Troan1-3/+7
Type: improvement Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I59323447bee7bb4f0563251a5df80cbefb9bd89e Signed-off-by: Ole Troan <ot@cisco.com>
2020-07-23vppapigen: missing crcs in user-defined typesOle Troan1-1/+296
make the change backwards compatible Change-Id: I379691aa5972f99b2144deda4b7ef9e6ba4af67d Type: fix Signed-off-by: Ole Troan <ot@cisco.com>
2020-05-25api: add new stream message conventionOle Troan1-1/+6
Instead of having to wrap dump/detail calls in control ping, send details messages in between a normal reply / request pair. As expressed in the below service statement. Example: service { rpc map_domains_gets returns map_domains_get_reply stream map_domain_details; }; define map_domains_get { u32 client_index; u32 context; u32 cursor; }; define map_domains_get_reply { u32 context; i32 retval; u32 cursor; }; To avoid blocking the main thread for too long, the replies are now sent in client message queue size chunks. The reply message returns VNET_API_ERROR_EAGAIN when there is more to read. The API handler must also include a "cursor" that is used to the next call to the get function. API handler example: REPLY_AND_DETAILS_MACRO (VL_API_MAP_DOMAINS_GET_REPLY, mm->domains, ({ send_domain_details (cursor, rp, mp->context); })); The macro starts from cursor and iterates through the pool until vl_api_process_may_suspend() returns true or the iteration reaches the end of the list. Client Example: cursor = 0 d = [] while True: rv, details = map_domains_get(cursor=cursor) d += details if rv.retval == 0 or rv.retval != -165: break cursor = rv.cursor or the convenience iterator: for x in vpp.details_iter(vpp.api.map_domains_get): pass or list(details_iter(map_domains_get)) Change-Id: Iad9f6b41b0ef886adb584c97708dd91cf552749e Type: feature Signed-off-by: Ole Troan <ot@cisco.com>
2020-05-19vppapigen: handle new api file for crcchekcerOle Troan1-2/+9
Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I1b3ae8b348eb3cba628a55ea3e72e85b9a4288d5
2020-05-16vppapigen: add assert for python versionOle Troan1-2/+4
The recent introduction of f-strings require >=Python3.6 Fix a couple of pycodestyle warnings. Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: Id9e375bb8febb157d7f9d7fa21808724770df6f8
2020-05-09vppapigen: api crc checkerOle Troan1-23/+51
crcchecker is a tool for enforcement of the binary API. 1. Production APIs should never change. 2. An API can be deprecated across three release cycles. Release 1: API is marked as deprecated. option deprecated="vyy.mm"; option replaced_by="new_api_2"; Release 2: both old and new APIs are supported Release 3: the deprecated API is deleted. 3. APIs that are experimental / not released are not checked. An API message can be individually marked as in progress, by: option status="in_progress"; In the API definition. The definition of a "production API" is if the major version in the API file is > 0. extras/scripts/crcchecker.py --check-patchset # returns -1 if backwards incompatible extras/scripts/crcchecker.py --dump-manifest extras/scripts/crcchecker.py --git-revision v20.01 <files> extras/scripts/crcchecker.py -- diff <oldfile> <newfile> This patch integrates the tool in "make checkstyle-api". A future patch is required to integrate the tool in the verify job. I.e. this patch does not enable enforcement through Jenkins. Change-Id: I5f13c0976d8a12a58131b3e270f2dc9c00dc7d8c Type: feature Signed-off-by: Ole Troan <ot@cisco.com>
2020-04-06misc: fix python sonarcloud BLOCKER level issuesPaul Vinciguerra1-1/+1
Fix of the top 11 python issues flagged as BLOCKER. Ticket: VPP-1856 Type: fix Change-Id: Icf4691e62f4a69d6ee196b6d6e2ab52d961b5c76 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2020-03-04vppapigen: crc is a negative value for some messages when using python 2.7Mark Nelson1-2/+2
Type: fix Signed-off-by: Mark Nelson <manelso2@cisco.com> Change-Id: I4f121e49d3c05c21eed3fed2469bd88fc84e2271
2020-02-04vppapigen: fix options representationOle Troan1-1/+1
Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: Ief77110160120ea0e1291cc79949a73404cdbfda
2019-09-25api: split api generated filesOle Troan1-2/+7
Split generation of API message enum and the typedefs into separate files, so that the type file can be included from elsewhere. Generate a C file for VPP that contains the API registration, this was previously done via X macros by the C pre- processor. This allows deleting lots of skeleton/copy paste code for each feature. plugins/flowprobe plugins/map examples/sample-plugin vnet/ipip used as Guinea pigs. Generate a C Test file for VAT, that does the same for VAT plugins. Also add support for a per-message CLI option, that is currently limited to VAT help text. option vat_help = "<help text>"; Type: refactor Change-Id: I245e3104bb76f7e1fb69a59ab20cc7c8dfcdd460 Signed-off-by: Ole Troan <ot@cisco.com>
2019-09-19api: split vl_api_prefix into twoOle Troan1-3/+4
One type for address with prefix and one type for prefix. Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: Icfec51d9b7d5cde1d69fbecdd97498688ab7b295 Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Klement Sekera <ksekera@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com>
2019-09-18vppapigen: fix missing vla check for union classOle Troan1-51/+32
Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: Ie775cf3469d761847ac39cf0d80a3ec6463b7928
2019-09-18vppapigen: fix tests and run on verifyPaul Vinciguerra1-14/+35
- changes vppapigen to only process an import once. Type: fix Change-Id: Ifcbcfcc69fdfb80d63195a17701762d0c239d7b4 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com> Signed-off-by: Ole Troan <ot@cisco.com>
2019-09-16api: autogenerate api trace print/endianOle Troan1-9/+70
In addition to the external vppapitrace tool, VPP itself supports dumping of API trace files. In two formats, "custom-dump" and "dump". "dump" gives a human friendly list, and "custom-dump" is meant to give a list of commands that can be fed to VAT. This patch only deals with "dump". Prior to this fix, auto-generation was only done for the basic types. This fix adds support for any type, including lists, and supports pretty-printing of enums, strings, IP addresses, MAC addresses and so on. Usage: api trace dump <api-trace-file> For example Change-Id: I4e485680e6dcfce7489299ae6cf31d835071ac40 ---------- trace 48 ----------- vl_api_sw_interface_set_flags_t: _vl_msg_id: 75 client_index: 0 context: 10 sw_if_index: 1 flags: IF_STATUS_API_FLAG_ADMIN_UP ---------- trace 49 ----------- vl_api_sw_interface_add_del_address_t: _vl_msg_id: 88 client_index: 0 context: 11 sw_if_index: 1 is_add: 1 del_all: 0 prefix: 172.16.1.1/24 ---------- trace 51 ----------- vl_api_cli_inband_t: _vl_msg_id: 819 client_index: 0 context: 13 cmd: packet-generator capture pg0 pcap /tmp/vpp-unittest-TestMAP-YhcmDX/pg0_out.pcap disable ---------- trace 58 ----------- vl_api_ip_neighbor_add_del_t: _vl_msg_id: 199 client_index: 0 context: 20 is_add: 1 neighbor: sw_if_index: 2 flags: IP_API_NEIGHBOR_FLAG_NONE mac_address: 0202.0000.ff02 ip_address: fd01:2::2 Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I5556d06008de2762e7c2d35a8b0963ae670b3db1 Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com>
2019-09-10misc: fix shebang with missing envNathan Skrzypczak1-1/+1
This is needed for macos where /usr/bin/python[23] does not exist and /usr/bin is write protected Type: fix Change-Id: Ia1b3c732248c78482993de532fa2ea96e376d90a Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com>
2019-09-03api: enforce vla is last and fixed string typeOle Troan1-9/+65
Enforce that variable length fields are the last element of API messages. Add a 'fixed' version of string type, since dealing with multiple variable length strings turned out too painful for the C language bindings. The string type is now: { string name[64]; // NUL terminated C-string. Essentially decays to u8 name[64] string name[]; // Variable length string with embedded len field (vl_api_string_t) }; The latter notation could be made available to other types as well. e.g. { vl_api_address_t addresses[]; } instead of { u32 n_addr; vl_api_address_t addresses[n_addr]; }; Type: fix Change-Id: I18fa17ef47227633752ab50453e8d20a652a9f9b Signed-off-by: Ole Troan <ot@cisco.com>
2019-08-20vppapigen: remove support for legacy typedefsPaul Vinciguerra1-1/+3
vppapigen has remapped legacy to typedefs behind the scenes for some time now. - update .api files to use new style typedefs. - issue error on 'typeonly define' in .api files - remove unneeded macros redefining vl_noop_handler Type: refactor Change-Id: I7a8c4a6dafacee6a131f95cd0e9b03a8c60dea8b Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-08-20vppapigen: remove python2 supportPaul Vinciguerra1-28/+30
vppapigen has been running under python3 for months. Remove the python2 conditional code. In case of an error, explicitly log and return non-0. Type: refactor Change-Id: I20a73e0f2cb875aab6441959a85a35e88c9f8250 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-08-19vppapigen map: raise ValueError when fieldname is python keywordPaul Vinciguerra1-0/+4
When working on the lb api, one of the field names was chosen as 'as' (application server). Since 'as' is a python keyword, the field was renamed to _1 in vpp_papi. This changeset instead fails early with a descriptive message, hopefully saving others time troubleshooting the issue. ValueError: Fieldname 'as' is a python keyword and is not accessible via the python API. Type: feature Change-Id: Ib048d97de0e392645540092e356cf8989848c947 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-08-01vppapigen: revert "implement reversible repr's"Vratko Polak1-34/+15
The previous change seems to affect all CRCs. Type: fix Fixes: b504777e7f1c9728e65b874284b4dfd39359c8a8. Change-Id: I65e27ce22020e5ebc764b5a51b5fc84992ddb40a Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-07-31vppapigen: implement reversible repr'sPaul Vinciguerra1-15/+34
Type: feature Change-Id: I92757f041cde399229c42e34515ace0fcd37908b Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-07-03vppapigen: allow decimal number in NUM tokenPaul Vinciguerra1-2/+6
Needed to set f64 default values. Type: feature Change-Id: Ic58ebc0d9d890bf0e7821894285e61a5bee13199 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-06-20vppapigen: allow negative number in NUM tokenOle Troan1-1/+1
Change-Id: I964cb44f8604187390009dcef7dd766b3804c3ca Type: make Signed-off-by: Ole Troan <ot@cisco.com>
2019-06-18fib: fib api updatesNeale Ranns1-2/+0
Enhance the route add/del APIs to take a set of paths rather than just one. Most unicast routing protocols calcualte all the available paths in one run of the algorithm so updating all the paths at once is beneficial for the client. two knobs control the behaviour: is_multipath - if set the the set of paths passed will be added to those that already exist, otherwise the set will replace them. is_add - add or remove the set is_add=0, is_multipath=1 and an empty set, results in deleting the route. It is also considerably faster to add multiple paths at once, than one at a time: vat# ip_add_del_route 1.1.1.1/32 count 100000 multipath via 10.10.10.11 100000 routes in .572240 secs, 174751.80 routes/sec vat# ip_add_del_route 1.1.1.1/32 count 100000 multipath via 10.10.10.12 100000 routes in .528383 secs, 189256.54 routes/sec vat# ip_add_del_route 1.1.1.1/32 count 100000 multipath via 10.10.10.13 100000 routes in .757131 secs, 132077.52 routes/sec vat# ip_add_del_route 1.1.1.1/32 count 100000 multipath via 10.10.10.14 100000 routes in .878317 secs, 113854.12 routes/sec vat# ip_route_add_del 1.1.1.1/32 count 100000 multipath via 10.10.10.11 via 10.10.10.12 via 10.10.10.13 via 10.10.10.14 100000 routes in .900212 secs, 111084.93 routes/sec Change-Id: I416b93f7684745099c1adb0b33edac58c9339c1a Signed-off-by: Neale Ranns <neale.ranns@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-06-07API: Add support for "defaults"Ole Troan1-4/+17
Add support in the API language for specifying a field default. Add default support in Python binding. define foo { u16 mtu [default = 1500]; }; This is client side only. I.e. if the mtu argument is not passed to the foo function, the client language binding will set it default to 1500. Change-Id: I5df43f3cd87cb300b40ca38e15dcab25b40e424a Signed-off-by: Ole Troan <ot@cisco.com>
2019-06-07vppapigen: Fold up CRC from dependent types.Ole Troan1-26/+41
Change-Id: Id51f26f225cd567ca19efc2301e94fa88840ae8f Signed-off-by: Ole Troan <ot@cisco.com> Type: fix Signed-off-by: Ole Troan <ot@cisco.com>
2019-04-29API: Add support for limits to language.Ole Troan1-5/+11
string name [limit = 64]; Meta-data to do argument validation. Change-Id: I1f3e0f09b2d5285224399413d25206f77bd3f4b1 Signed-off-by: Ole Troan <ot@cisco.com>
2019-02-18Resolve vppapigen DeprecationWarning.Paul Vinciguerra1-6/+7
/vpp/src/tools/vppapigen/vppapigen:823: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses Change-Id: If7729778374e9193f6381c8bd2ed34c875db3f1e Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-02-02Python3: Move vppapigen to python3.Paul Vinciguerra1-7/+12
Change-Id: I26846d0c12211a29ccfca7c269b9094f6fdbd95c Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2018-11-29vppapigen: Fix python3 compatibilityOle Troan1-24/+40
Change-Id: I9124fcb755ba43fd8f44712f0940f351c460c5b0 Signed-off-by: Ole Troan <ot@cisco.com>
2018-11-29API: Add support for type aliasesOle Troan1-1/+28
Previously all types are compound. This adds support for aliases, so one can do things like: typedef u32 interface_index; or typedef u8 ip4_address[4]; Change-Id: I0455cad0123fc88acb491d2a3ea2725426bdb246 Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Klement Sekera <ksekera@cisco.com>
2018-09-13Fix: vppapigen make build fails on fresh installPaul Vinciguerra1-2/+2
Steps to reproduce: vagrant@localhost:/vagrant$ build-root/vagrant/build.sh ... @@@@ Building vpp in /vagrant/build-root/build-vpp-native/vpp @@@@ [51/1169] Generating API header /vagrant/build-root/build-vpp-native/vpp/vlibmemory/memclnt.api.json FAILED: cd /vagrant/build-root/build-vpp-native/vpp/vlibmemory && mkdir -p /vagrant/build-root/build-vpp-native/vpp/vlibmemory && /vagrant/src/tools/vppapigen/vppapigen --includedir /vagrant/src --input /vagrant/src/vlibmemory/memclnt.api JSON --output /vagrant/build-root/build-vpp-native/vpp/vlibmemory/memclnt.api.json AttributeError: 'module' object has no attribute 'dumps' This seems to be due to JSON.py namespace colliding with the standard lib json.py Change-Id: If389e4e05ef0c166b0c2b3bef7ec0185298679a8 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2018-06-29CSIT-1148: VPPAPIGEN 2nd level import of .api ignoredOle Troan1-4/+5
Change-Id: I73dcab0c5ff2405b1fcba25975955e1267a4ac4e Signed-off-by: Ole Troan <ot@cisco.com>
2018-05-23VPPAPIGEN: Add union and enum support and IP4/IP6 address type.Ole Troan1-46/+75
Note: The Python, Java and C/C++ bindings must be updated before ip/ip_types.api can be used. ip_types.api: typedef ip4_address { u8 address[4]; }; typedef ip6_address { u8 address[16]; }; enum address_family { ADDRESS_IP4 = 0, ADDRESS_IP6, }; union address_union { vl_api_ip4_address_t ip4; vl_api_ip6_address_t ip6; }; typedef address { vl_api_address_family_t af; vl_api_address_union_t un; }; Change-Id: I22f67092f24db5bd650a03c6f446a84cd9fd1074 Signed-off-by: Ole Troan <ot@cisco.com>
2018-04-17ACL based forwardingAndrew Yourtchenko1-1/+1
A poor man's flow switching or policy based rounting. An ACL is used to match packets and is associated with a [set of] forwarding paths that determine how to forward matched packets - collectively this association is a 'policy'. Policies are then 'attached', in a priority order, to an interface when thaey are encountered as an input feature. If a packet matches no policies it is forwarded normally in the IP FIB. This commit is used to test the "ACL-as-a-service" functionality, which currently compiles, and the existing traffic ACL tests pass in both hash and linear modes. Change-Id: I0b274ec9f2e645352fa898b43eb54c457e195964 Signed-off-by: Neale Ranns <nranns@cisco.com> Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com> Signed-off-by: Ole Troan <ot@cisco.com>
2018-04-11VPPAPIGEN: Consistent CRC32 on complete file.Ole Troan1-17/+25
Prior to this fix vppapigen would just do a crc32 on the Python representation of the file as a set of dictionaries. That of course was not a good idea. Change-Id: Ie454736ffec02fa4679ab27e684b1d6c6406a0f1 Signed-off-by: Ole Troan <ot@cisco.com>
2018-03-08vppapigen: require reply/details messages for requests/dumpsMarek Gradzki1-6/+6
Throw exception if details/reply is not defined instead of logger warning. Change-Id: I6eb39c51eecca6521e5f563ba09dfd62311b45ec Signed-off-by: Marek Gradzki <mgradzki@cisco.com>
2018-03-07VPPAPIGEN: Run tool directly from source tree.Ole Troan1-1/+4
This ensures that tool changes do not require re-bootstrap. Disabled generation of temporary files (gentab / .pyc) to avoid polluting the source tree. Change-Id: I4d6bc035fbb46550fa8f4e99f4091eef90e2d86c Signed-off-by: Ole Troan <ot@cisco.com>
2018-03-06vppapigen: require service definition for singleton messagesMarek Gradzki1-7/+2
Change-Id: Ie7b1c5e50588f65657c3c0900e5914ea82ebb50f Signed-off-by: Marek Gradzki <mgradzki@cisco.com>
2018-03-06vppapigen: do not allow to define message as both request and replyMarek Gradzki1-0/+4
Change-Id: I0738fac32dd0c5d927c52d2eb2c1100d14c147cf Signed-off-by: Marek Gradzki <mgradzki@cisco.com>
2018-03-06vppapigen: require reply ID different than caller IDMarek Gradzki1-0/+4
Change-Id: I316dc99881bce6a36904863d3c1c049b4f5cf658 Signed-off-by: Marek Gradzki <mgradzki@cisco.com>
2018-03-06API: Add service definitions for events and singleton messages (second attempt)Marek Gradzki1-17/+30
Based on https://gerrit.fd.io/r/#/c/10920/ Updates service definition in stats.api with correct reply message names. Change-Id: I3282bee5304e667e23bc1fab3f43d967a50d880d Signed-off-by: Marek Gradzki <mgradzki@cisco.com>
2018-03-05Revert "API: Add service definitions for events and singleton messages."Ole Trøan1-30/+17
This reverts commit f7b7fa53b7eaec81d8c00c1023fb7d01f1f9761f. Change-Id: I87496342943248e94f01ada31459f387c0a3a610 Signed-off-by: Ole Troan <ot@cisco.com>
2018-03-05API: Add service definitions for events and singleton messages.Ole Troan1-17/+30
Change-Id: I7de987c30b263d43521e6280c5273f30b5f6e11c Signed-off-by: Ole Troan <ot@cisco.com>
2018-03-03VPPAPIGEN: Improve output module finding for out-of-tree builds.Ole Troan1-2/+12
Change-Id: I3b646d84586dd61018f71cbf21c971c97fda75c1 Signed-off-by: Ole Troan <ot@cisco.com>