aboutsummaryrefslogtreecommitdiffstats
path: root/src/tools
AgeCommit message (Collapse)AuthorFilesLines
2020-08-11vppapigen: add support for empty optionsOle Troan2-3/+17
Type: improvement Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I59323447bee7bb4f0563251a5df80cbefb9bd89e Signed-off-by: Ole Troan <ot@cisco.com>
2020-08-06misc: harmonize namesDave Barach2-10/+10
Type: fix Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: Ibad744788e200ce012ad88ff59c2c34920742454
2020-07-23vppapigen: missing crcs in user-defined typesOle Troan2-3/+403
make the change backwards compatible Change-Id: I379691aa5972f99b2144deda4b7ef9e6ba4af67d Type: fix Signed-off-by: Ole Troan <ot@cisco.com>
2020-06-17docs: add more FEATURE.yaml documentationDave Barach1-0/+12
For src/tools/g2, src/vlibapi, and src/vlibmemory Type: docs Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I800c3e3a1ecb011d5e692d78a015e78342706786
2020-05-27api: make vpp api handlers endian independentOle Troan1-4/+15
Add a new boolean to signal that the API infrastructure should performan any required endian conversions for the API handler. am->is_autoendian[mm->msg_id_base + VL_API_MAP_ADD_DOMAIN] = 1; Similarly add new REPLY_ macros that perform endian conversion. These changes do not change the on-the-wire encoding of the API messages, and therefore the API CRC is not changed. Type: feature Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I7588f8ccb38b2d1e8d85ea17be99bac43f756267 Signed-off-by: Ole Troan <ot@cisco.com>
2020-05-25api: add new stream message conventionOle Troan2-1/+8
Instead of having to wrap dump/detail calls in control ping, send details messages in between a normal reply / request pair. As expressed in the below service statement. Example: service { rpc map_domains_gets returns map_domains_get_reply stream map_domain_details; }; define map_domains_get { u32 client_index; u32 context; u32 cursor; }; define map_domains_get_reply { u32 context; i32 retval; u32 cursor; }; To avoid blocking the main thread for too long, the replies are now sent in client message queue size chunks. The reply message returns VNET_API_ERROR_EAGAIN when there is more to read. The API handler must also include a "cursor" that is used to the next call to the get function. API handler example: REPLY_AND_DETAILS_MACRO (VL_API_MAP_DOMAINS_GET_REPLY, mm->domains, ({ send_domain_details (cursor, rp, mp->context); })); The macro starts from cursor and iterates through the pool until vl_api_process_may_suspend() returns true or the iteration reaches the end of the list. Client Example: cursor = 0 d = [] while True: rv, details = map_domains_get(cursor=cursor) d += details if rv.retval == 0 or rv.retval != -165: break cursor = rv.cursor or the convenience iterator: for x in vpp.details_iter(vpp.api.map_domains_get): pass or list(details_iter(map_domains_get)) Change-Id: Iad9f6b41b0ef886adb584c97708dd91cf552749e Type: feature Signed-off-by: Ole Troan <ot@cisco.com>
2020-05-19vppapigen: handle new api file for crcchekcerOle Troan1-2/+9
Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I1b3ae8b348eb3cba628a55ea3e72e85b9a4288d5
2020-05-16vppapigen: add assert for python versionOle Troan1-2/+4
The recent introduction of f-strings require >=Python3.6 Fix a couple of pycodestyle warnings. Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: Id9e375bb8febb157d7f9d7fa21808724770df6f8
2020-05-15g2: fix the g2 build for Ubuntu 20.04Dave Barach1-2/+1
Shut off deprecated declaration warnings Type: fix Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I65ea4bbc4d5ee5a11d4e8f554f414f57944c7e1c
2020-05-09vppapigen: api crc checkerOle Troan2-23/+67
crcchecker is a tool for enforcement of the binary API. 1. Production APIs should never change. 2. An API can be deprecated across three release cycles. Release 1: API is marked as deprecated. option deprecated="vyy.mm"; option replaced_by="new_api_2"; Release 2: both old and new APIs are supported Release 3: the deprecated API is deleted. 3. APIs that are experimental / not released are not checked. An API message can be individually marked as in progress, by: option status="in_progress"; In the API definition. The definition of a "production API" is if the major version in the API file is > 0. extras/scripts/crcchecker.py --check-patchset # returns -1 if backwards incompatible extras/scripts/crcchecker.py --dump-manifest extras/scripts/crcchecker.py --git-revision v20.01 <files> extras/scripts/crcchecker.py -- diff <oldfile> <newfile> This patch integrates the tool in "make checkstyle-api". A future patch is required to integrate the tool in the verify job. I.e. this patch does not enable enforcement through Jenkins. Change-Id: I5f13c0976d8a12a58131b3e270f2dc9c00dc7d8c Type: feature Signed-off-by: Ole Troan <ot@cisco.com>
2020-05-08build: various improvementsDamjan Marion1-1/+1
- add option to install only host tools - add option to specify lib and runtime dir Type: improvement Change-Id: I6356b52df459120fc9b0127948bae7679fb10e52 Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-05-07misc: deprecate elftoolDamjan Marion3-500/+0
Type: make Change-Id: I34fec1c40ef660177c266517eaf41d60827609f4 Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-04-06misc: fix python sonarcloud BLOCKER level issuesPaul Vinciguerra1-1/+1
Fix of the top 11 python issues flagged as BLOCKER. Ticket: VPP-1856 Type: fix Change-Id: Icf4691e62f4a69d6ee196b6d6e2ab52d961b5c76 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2020-04-04misc: strcpy be goneDave Barach2-22/+20
Causes static analysis "vulnerability" warnings Type: fix Ticket: VPP-1837 Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I272fa69251d70f62178e6dff0423c16f99937af1
2020-04-04misc: sprintf be goneDave Barach7-164/+163
Along with related static analysis warnings... Type: fix Ticket: VPP-1837 Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I2c6949c7a2250b8f76a63508c7c210daecfe0f91
2020-03-04vppapigen: crc is a negative value for some messages when using python 2.7Mark Nelson1-2/+2
Type: fix Signed-off-by: Mark Nelson <manelso2@cisco.com> Change-Id: I4f121e49d3c05c21eed3fed2469bd88fc84e2271
2020-02-26vppapigen: list imports in JSONOndrej Fabry1-0/+7
Type: fix Change-Id: If50477f2487d8c13f91c119a1492370e6e937904 Signed-off-by: Ondrej Fabry <ofabry@cisco.com>
2020-02-26api: improve api string safetyJakub Grajciar1-3/+2
- Remove vl_api_from_api_string to prevent use of not nul-terminated strings. - Rename vl_api_from_api_to_vec -> vl_api_from_api_to_new_vec to imply a new vector is created. NOT nul terminated. - Add vl_api_from_api_to_new_c_string. Returns nul terminated string in a new vector. - Add vl_api_c_string_to_api_string. Convert nul terminated string to vl_api_string_t - Add vl_api_vec_to_api_string. Convert NON nul terminated vector to vl_api_string_t Type: fix Signed-off-by: Jakub Grajciar <jgrajcia@cisco.com> Change-Id: Iadd59b612c0d960a34ad0dd07a9d17f56435c6ea Signed-off-by: Jakub Grajciar <jgrajcia@cisco.com>
2020-02-04vppapigen: fix options representationOle Troan1-1/+1
Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: Ief77110160120ea0e1291cc79949a73404cdbfda
2020-01-30vppapigen: update markdown documentationPaul Vinciguerra1-7/+10
- Add newly added typedefs. - Update string examples. Change-Id: I1e7ee7cbf5901ba97302472521bf1f42a14765ea Type: docs Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-12-10api: multiple connections per processDave Barach1-1/+1
Type: feature Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I2272521d6e69edcd385ef684af6dd4eea5eaa953
2019-12-07lacp: fix control_ping from pluginsOle Troan1-0/+4
Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I030f75234a346ef6a20640c1a0ed65f4934867f2
2019-11-24vppapigen: clean up typos in c generated codePaul Vinciguerra1-4/+4
Type: style Change-Id: Ib8febf8ea83f7ce1aeb93f227d00d3111fbef4ab Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-11-24vppapigen: fix i64 format stringPaul Vinciguerra1-1/+1
Type: fix Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com> Change-Id: I6b42bb9fb9d2e57f523022e321770bb1ff7415f3
2019-11-24vppapigen: fix typo on f64 endian_stringPaul Vinciguerra1-4/+4
Type: fix Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com> Change-Id: Id47e3f2c348f07c84d9c2499122fa25d6c74fb13
2019-10-10acl: remove api boilerplateOle Troan1-0/+3
Type: refactor Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: Idc9c508eb0e3d4b0c3908360fb0771012a8e2bc0
2019-10-10memif: remove api boilerplateOle Troan1-0/+2
Type: refactor Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: Icab769607c4a5890dc80b8f2661faa7e297c2a27
2019-10-09api: autogenerate event handler functions for *_test.cOle Troan1-0/+17
Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: If75f63244f37ae3312832ca015f738c51ee7da1f Signed-off-by: Ole Troan <ot@cisco.com>
2019-09-25api: split api generated filesOle Troan3-74/+261
Split generation of API message enum and the typedefs into separate files, so that the type file can be included from elsewhere. Generate a C file for VPP that contains the API registration, this was previously done via X macros by the C pre- processor. This allows deleting lots of skeleton/copy paste code for each feature. plugins/flowprobe plugins/map examples/sample-plugin vnet/ipip used as Guinea pigs. Generate a C Test file for VAT, that does the same for VAT plugins. Also add support for a per-message CLI option, that is currently limited to VAT help text. option vat_help = "<help text>"; Type: refactor Change-Id: I245e3104bb76f7e1fb69a59ab20cc7c8dfcdd460 Signed-off-by: Ole Troan <ot@cisco.com>
2019-09-19api: split vl_api_prefix into twoOle Troan3-47/+39
One type for address with prefix and one type for prefix. Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: Icfec51d9b7d5cde1d69fbecdd97498688ab7b295 Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Klement Sekera <ksekera@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com>
2019-09-18vppapigen: fix missing vla check for union classOle Troan2-52/+87
Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: Ie775cf3469d761847ac39cf0d80a3ec6463b7928
2019-09-18vppapigen: fix tests and run on verifyPaul Vinciguerra2-26/+56
- changes vppapigen to only process an import once. Type: fix Change-Id: Ifcbcfcc69fdfb80d63195a17701762d0c239d7b4 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com> Signed-off-by: Ole Troan <ot@cisco.com>
2019-09-16api: autogenerate api trace print/endianOle Troan4-86/+434
In addition to the external vppapitrace tool, VPP itself supports dumping of API trace files. In two formats, "custom-dump" and "dump". "dump" gives a human friendly list, and "custom-dump" is meant to give a list of commands that can be fed to VAT. This patch only deals with "dump". Prior to this fix, auto-generation was only done for the basic types. This fix adds support for any type, including lists, and supports pretty-printing of enums, strings, IP addresses, MAC addresses and so on. Usage: api trace dump <api-trace-file> For example Change-Id: I4e485680e6dcfce7489299ae6cf31d835071ac40 ---------- trace 48 ----------- vl_api_sw_interface_set_flags_t: _vl_msg_id: 75 client_index: 0 context: 10 sw_if_index: 1 flags: IF_STATUS_API_FLAG_ADMIN_UP ---------- trace 49 ----------- vl_api_sw_interface_add_del_address_t: _vl_msg_id: 88 client_index: 0 context: 11 sw_if_index: 1 is_add: 1 del_all: 0 prefix: 172.16.1.1/24 ---------- trace 51 ----------- vl_api_cli_inband_t: _vl_msg_id: 819 client_index: 0 context: 13 cmd: packet-generator capture pg0 pcap /tmp/vpp-unittest-TestMAP-YhcmDX/pg0_out.pcap disable ---------- trace 58 ----------- vl_api_ip_neighbor_add_del_t: _vl_msg_id: 199 client_index: 0 context: 20 is_add: 1 neighbor: sw_if_index: 2 flags: IP_API_NEIGHBOR_FLAG_NONE mac_address: 0202.0000.ff02 ip_address: fd01:2::2 Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I5556d06008de2762e7c2d35a8b0963ae670b3db1 Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com>
2019-09-10misc: fix shebang with missing envNathan Skrzypczak1-1/+1
This is needed for macos where /usr/bin/python[23] does not exist and /usr/bin is write protected Type: fix Change-Id: Ia1b3c732248c78482993de532fa2ea96e376d90a Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com>
2019-09-03api: enforce vla is last and fixed string typeOle Troan2-11/+74
Enforce that variable length fields are the last element of API messages. Add a 'fixed' version of string type, since dealing with multiple variable length strings turned out too painful for the C language bindings. The string type is now: { string name[64]; // NUL terminated C-string. Essentially decays to u8 name[64] string name[]; // Variable length string with embedded len field (vl_api_string_t) }; The latter notation could be made available to other types as well. e.g. { vl_api_address_t addresses[]; } instead of { u32 n_addr; vl_api_address_t addresses[n_addr]; }; Type: fix Change-Id: I18fa17ef47227633752ab50453e8d20a652a9f9b Signed-off-by: Ole Troan <ot@cisco.com>
2019-08-23vppapitrace: add text outputOle Troan1-8/+65
Add new argument --todump, to mimic the VPP api trace dump output. Type: feature Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: Ia8c845b321ef9d55dd6e2f388165ca5498406ab6
2019-08-20vppapigen: remove support for legacy typedefsPaul Vinciguerra1-1/+3
vppapigen has remapped legacy to typedefs behind the scenes for some time now. - update .api files to use new style typedefs. - issue error on 'typeonly define' in .api files - remove unneeded macros redefining vl_noop_handler Type: refactor Change-Id: I7a8c4a6dafacee6a131f95cd0e9b03a8c60dea8b Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-08-20vppapigen: remove python2 supportPaul Vinciguerra1-28/+30
vppapigen has been running under python3 for months. Remove the python2 conditional code. In case of an error, explicitly log and return non-0. Type: refactor Change-Id: I20a73e0f2cb875aab6441959a85a35e88c9f8250 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-08-19vppapigen map: raise ValueError when fieldname is python keywordPaul Vinciguerra1-0/+4
When working on the lb api, one of the field names was chosen as 'as' (application server). Since 'as' is a python keyword, the field was renamed to _1 in vpp_papi. This changeset instead fails early with a descriptive message, hopefully saving others time troubleshooting the issue. ValueError: Fieldname 'as' is a python keyword and is not accessible via the python API. Type: feature Change-Id: Ib048d97de0e392645540092e356cf8989848c947 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-08-08api: vppapitrace JSON/API trace converterOle Troan2-0/+435
usage: vppapitrace.py [-h] [--debug] [--apidir APIDIR] {convert,replay} ... optional arguments: -h, --help show this help message and exit --debug enable debug mode --apidir APIDIR Location of JSON API definitions subcommands: valid subcommands {convert,replay} additional help convert Convert API trace to JSON or Python and back replay Replay messages to running VPP instance To convert an API trace file to JSON: vppapitrace convert /tmp/api.trace trace.json To convert an (edited) JSON file back to API trace for replay: vppapitrace convert trace.json api-edited.trace To generate a Python file that can be replayed: vppapitrace convert /tmp/api.trace trace.py vppapitrace convert trace.json trace.py Replay it to a running VPP instance: vppapitrace replay --socket /tmp/api.trace In VPP that file can be replayed with: vpp# api trace replay api-edited.trace This patch also modifies the API binary trace format, to include the message id to message name table. Change-Id: Ie6441efb53c1c93c9f778f6ae9c1758bccc8dd87 Type: refactor Signed-off-by: Ole Troan <ot@cisco.com>
2019-08-01vppapigen: revert "implement reversible repr's"Vratko Polak1-34/+15
The previous change seems to affect all CRCs. Type: fix Fixes: b504777e7f1c9728e65b874284b4dfd39359c8a8. Change-Id: I65e27ce22020e5ebc764b5a51b5fc84992ddb40a Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-07-31vppapigen: implement reversible repr'sPaul Vinciguerra1-15/+34
Type: feature Change-Id: I92757f041cde399229c42e34515ace0fcd37908b Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-07-31vppapigen: add endian_string for f64Paul Vinciguerra1-0/+1
Type: fix Change-Id: I35838baea21ead4a3f45d998ff225a513781d7ee Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-07-24vapi: add python scripts to vpp-dev packageVratko Polak1-1/+10
As requested in https://lists.fd.io/g/vpp-api-dev/message/18 three vapi scrips are packaged, destination to share/vpp/. Also: + Add "vapi" as a separate component to maintainers file. + Add also vppapigen/generate_json.py to share/vpp/. + Improve CMakeLists.txt indentation. Type: feature Change-Id: Ia06715621aa344e8ee759410b293509a54f81fdd Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-07-18build: add more src dirs for generate_json.pyVratko Polak1-4/+7
Because file vpe.api is in src/vpp/api/ and memclnt.api is in src/vlibmemory/. Also removed api_types, as iteration can be done over output_dir_map. Type: fix Fixes: 9529feb4525dfd13e5636640083361256121d275 Ticket: VPP-1715 Change-Id: I021afeafdf98904d076953f0b09d1e3587fd3100 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-07-17build: add targets for json api filesPaul Vinciguerra1-0/+94
Type: make Ticket: VPP-1715 Change-Id: I78497d679d9e793b47a06a0c5cb3b12d86b08489 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-07-03vppapigen: allow decimal number in NUM tokenPaul Vinciguerra1-2/+6
Needed to set f64 default values. Type: feature Change-Id: Ic58ebc0d9d890bf0e7821894285e61a5bee13199 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-06-20vppapigen: allow negative number in NUM tokenOle Troan1-1/+1
Change-Id: I964cb44f8604187390009dcef7dd766b3804c3ca Type: make Signed-off-by: Ole Troan <ot@cisco.com>
2019-06-18fib: fib api updatesNeale Ranns1-2/+0
Enhance the route add/del APIs to take a set of paths rather than just one. Most unicast routing protocols calcualte all the available paths in one run of the algorithm so updating all the paths at once is beneficial for the client. two knobs control the behaviour: is_multipath - if set the the set of paths passed will be added to those that already exist, otherwise the set will replace them. is_add - add or remove the set is_add=0, is_multipath=1 and an empty set, results in deleting the route. It is also considerably faster to add multiple paths at once, than one at a time: vat# ip_add_del_route 1.1.1.1/32 count 100000 multipath via 10.10.10.11 100000 routes in .572240 secs, 174751.80 routes/sec vat# ip_add_del_route 1.1.1.1/32 count 100000 multipath via 10.10.10.12 100000 routes in .528383 secs, 189256.54 routes/sec vat# ip_add_del_route 1.1.1.1/32 count 100000 multipath via 10.10.10.13 100000 routes in .757131 secs, 132077.52 routes/sec vat# ip_add_del_route 1.1.1.1/32 count 100000 multipath via 10.10.10.14 100000 routes in .878317 secs, 113854.12 routes/sec vat# ip_route_add_del 1.1.1.1/32 count 100000 multipath via 10.10.10.11 via 10.10.10.12 via 10.10.10.13 via 10.10.10.14 100000 routes in .900212 secs, 111084.93 routes/sec Change-Id: I416b93f7684745099c1adb0b33edac58c9339c1a Signed-off-by: Neale Ranns <neale.ranns@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-06-14g2: clean up compile/link errorsDave Barach3-35/+33
Many missing "extern" declarations, it's hard to imagine why the related compile/link errors just showed up. Type: fix Change-Id: I0878bf5fdb10ffdd8e0c49c9e5153cddaf3bc1fd Signed-off-by: Dave Barach <dave@barachs.net>