summaryrefslogtreecommitdiffstats
path: root/src/tools
AgeCommit message (Collapse)AuthorFilesLines
2020-11-21vppapigen: move import processing logic to individual pluginsPaul Vinciguerra4-37/+50
Vppapigen currently embeds the import following control logic in the runner. This change delegates the control to the plugins. Type: refactor Change-Id: Iad20341bc9d652bedb71ca7037d3957fe60c7a0d Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2020-10-13stats: counters data modelOle Troan3-12/+162
This adds a new data model for counters. Specifying the errors severity and unit. A later patch will update vpp_get_stats to take advantage of this. Only the map plugin is updates as an example. New .api language: A new "counters" keyword to define counter sets. counters map { none { severity info; type counter64; units "packets"; description "valid MAP packets"; }; bad_protocol { severity error; type counter64; units "packets"; description "bad protocol"; }; }; Each counter has 4 keywords. severity, which is one of error, info or warn. A type, which is one of counter64 or gauge64. units, which is a text field using units from YANG. paths { "/err/ip4-map" "map"; "/err/ip6-map" "map"; "/err/ip4-t-map" "map"; "/err/ip6-t-map" "map"; }; A new paths keyword that maps the counter-set to a path in the stats segment KV store. Updated VPP CLI to include severity so user can see error counter severity. DBGvpp# show errors Count Node Reason Severity 13 ethernet-input no error error Type: feature Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: Ib2177543f49d4c3aef4d7fa72476cff2068f7771 Signed-off-by: Ole Troan <ot@cisco.com>
2020-10-06feature: Add packet trace APIJon Loeliger1-4/+10
Also spiffed up the vpp_api_test plugin loader so it executes VLIB_INIT_FUNCTIONs and VLIB_API_INIT_FUNCTIONs. Type: feature Change-Id: Id9a4f455d73738c41bcfea220df2112bb9679681 Signed-off-by: Jon Loeliger <jdl@netgate.com> Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Dave Barach <dave@barachs.net>
2020-09-18vat: add infrastructure to align vnet test code and plugin test codeOle Troan1-10/+11
Split vat/api_format.c also for VNET features. Use auto-generated VAT test code and support dynamic message ID allocation as for plugins. The arp and geneve features as Guinea pigs. Type: improvement Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I461591496766bdf10c5f950fd30f1a0ae05909da Signed-off-by: Ole Troan <ot@cisco.com>
2020-09-11build: fix build for Debian 9 and Debian 10Benoît Ganne1-4/+5
Type: fix Change-Id: Ic07d0ae313b32e420ba93693cb75960a86f752a9 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2020-09-01vppapigen: supports backwards compatible marking of enumsOle Troan1-18/+38
enum bar_enum { BAR1 = 0, BAR2, BAR3 [backwards_compatible], BAR4 = 9 [backwards_compatible], }; This allows adding backwards compatible (as guaranteed by the developer) enums. The enums marked backwards compatible are not considered in the CRC calculation. Type: improvement Change-Id: I6fc0c21b19e1a02cff7f5d279a0f3a32d2f8b630 Signed-off-by: Ole Troan <ot@cisco.com>
2020-08-25api: register endian handlers for reply messagesOle Troan1-0/+13
Endian handlers was not registered for reply messages. Causing endian-neutral handlers to crash. Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: Id14173300ee1ab2601e92d58c70d2fa260814b69
2020-08-21vppapigen: Fix missing api dirNathan Skrzypczak1-0/+1
Type: fix pci_types.api lives in vlib/pci which wasn't included in the generate_json.py script Change-Id: I15649786282584b7aefb9ace83c5154eb697b50b Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com>
2020-08-19vppinfra: minor tweaks for cgo interoperationDave Barach2-11/+11
'type' is a keyword in golang, so s/type/event_type/ in elog.h and elsewhere. Add vec_len_not_inline(...), elog_write_file_not_inline(...) and elog_read_file_not_inline(...) since the inline forms aren't usable. More such tweaks may follow. Type: improvement Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I9a80a6afa635f5cdedee554ee9abe400fafc1cb6
2020-08-17vppapigen: generate define for module_semverOle Troan1-0/+7
Generate a semver define for API modules in the VPP side C generator. E.g in map.api_types.h \#define VL_API_MAP_API_VERSION_MAJOR 4 \#define VL_API_MAP_API_VERSION_MINOR 2 \#define VL_API_MAP_API_VERSION_PATCH 1 Type: improvement Change-Id: I2ab86154e85e26fb075c4e91a01ff3692640654b Signed-off-by: Ole Troan <ot@cisco.com>
2020-08-13vppapigen: remove unreachable codeOle Troan1-1/+0
Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: Ifd81acaa020f6ebff20ec4eff4370d8a18323860
2020-08-11vppapigen: add support for empty optionsOle Troan2-3/+17
Type: improvement Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I59323447bee7bb4f0563251a5df80cbefb9bd89e Signed-off-by: Ole Troan <ot@cisco.com>
2020-08-06misc: harmonize namesDave Barach2-10/+10
Type: fix Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: Ibad744788e200ce012ad88ff59c2c34920742454
2020-07-23vppapigen: missing crcs in user-defined typesOle Troan2-3/+403
make the change backwards compatible Change-Id: I379691aa5972f99b2144deda4b7ef9e6ba4af67d Type: fix Signed-off-by: Ole Troan <ot@cisco.com>
2020-06-17docs: add more FEATURE.yaml documentationDave Barach1-0/+12
For src/tools/g2, src/vlibapi, and src/vlibmemory Type: docs Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I800c3e3a1ecb011d5e692d78a015e78342706786
2020-05-27api: make vpp api handlers endian independentOle Troan1-4/+15
Add a new boolean to signal that the API infrastructure should performan any required endian conversions for the API handler. am->is_autoendian[mm->msg_id_base + VL_API_MAP_ADD_DOMAIN] = 1; Similarly add new REPLY_ macros that perform endian conversion. These changes do not change the on-the-wire encoding of the API messages, and therefore the API CRC is not changed. Type: feature Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I7588f8ccb38b2d1e8d85ea17be99bac43f756267 Signed-off-by: Ole Troan <ot@cisco.com>
2020-05-25api: add new stream message conventionOle Troan2-1/+8
Instead of having to wrap dump/detail calls in control ping, send details messages in between a normal reply / request pair. As expressed in the below service statement. Example: service { rpc map_domains_gets returns map_domains_get_reply stream map_domain_details; }; define map_domains_get { u32 client_index; u32 context; u32 cursor; }; define map_domains_get_reply { u32 context; i32 retval; u32 cursor; }; To avoid blocking the main thread for too long, the replies are now sent in client message queue size chunks. The reply message returns VNET_API_ERROR_EAGAIN when there is more to read. The API handler must also include a "cursor" that is used to the next call to the get function. API handler example: REPLY_AND_DETAILS_MACRO (VL_API_MAP_DOMAINS_GET_REPLY, mm->domains, ({ send_domain_details (cursor, rp, mp->context); })); The macro starts from cursor and iterates through the pool until vl_api_process_may_suspend() returns true or the iteration reaches the end of the list. Client Example: cursor = 0 d = [] while True: rv, details = map_domains_get(cursor=cursor) d += details if rv.retval == 0 or rv.retval != -165: break cursor = rv.cursor or the convenience iterator: for x in vpp.details_iter(vpp.api.map_domains_get): pass or list(details_iter(map_domains_get)) Change-Id: Iad9f6b41b0ef886adb584c97708dd91cf552749e Type: feature Signed-off-by: Ole Troan <ot@cisco.com>
2020-05-19vppapigen: handle new api file for crcchekcerOle Troan1-2/+9
Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I1b3ae8b348eb3cba628a55ea3e72e85b9a4288d5
2020-05-16vppapigen: add assert for python versionOle Troan1-2/+4
The recent introduction of f-strings require >=Python3.6 Fix a couple of pycodestyle warnings. Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: Id9e375bb8febb157d7f9d7fa21808724770df6f8
2020-05-15g2: fix the g2 build for Ubuntu 20.04Dave Barach1-2/+1
Shut off deprecated declaration warnings Type: fix Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I65ea4bbc4d5ee5a11d4e8f554f414f57944c7e1c
2020-05-09vppapigen: api crc checkerOle Troan2-23/+67
crcchecker is a tool for enforcement of the binary API. 1. Production APIs should never change. 2. An API can be deprecated across three release cycles. Release 1: API is marked as deprecated. option deprecated="vyy.mm"; option replaced_by="new_api_2"; Release 2: both old and new APIs are supported Release 3: the deprecated API is deleted. 3. APIs that are experimental / not released are not checked. An API message can be individually marked as in progress, by: option status="in_progress"; In the API definition. The definition of a "production API" is if the major version in the API file is > 0. extras/scripts/crcchecker.py --check-patchset # returns -1 if backwards incompatible extras/scripts/crcchecker.py --dump-manifest extras/scripts/crcchecker.py --git-revision v20.01 <files> extras/scripts/crcchecker.py -- diff <oldfile> <newfile> This patch integrates the tool in "make checkstyle-api". A future patch is required to integrate the tool in the verify job. I.e. this patch does not enable enforcement through Jenkins. Change-Id: I5f13c0976d8a12a58131b3e270f2dc9c00dc7d8c Type: feature Signed-off-by: Ole Troan <ot@cisco.com>
2020-05-08build: various improvementsDamjan Marion1-1/+1
- add option to install only host tools - add option to specify lib and runtime dir Type: improvement Change-Id: I6356b52df459120fc9b0127948bae7679fb10e52 Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-05-07misc: deprecate elftoolDamjan Marion3-500/+0
Type: make Change-Id: I34fec1c40ef660177c266517eaf41d60827609f4 Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-04-06misc: fix python sonarcloud BLOCKER level issuesPaul Vinciguerra1-1/+1
Fix of the top 11 python issues flagged as BLOCKER. Ticket: VPP-1856 Type: fix Change-Id: Icf4691e62f4a69d6ee196b6d6e2ab52d961b5c76 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2020-04-04misc: strcpy be goneDave Barach2-22/+20
Causes static analysis "vulnerability" warnings Type: fix Ticket: VPP-1837 Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I272fa69251d70f62178e6dff0423c16f99937af1
2020-04-04misc: sprintf be goneDave Barach7-164/+163
Along with related static analysis warnings... Type: fix Ticket: VPP-1837 Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I2c6949c7a2250b8f76a63508c7c210daecfe0f91
2020-03-04vppapigen: crc is a negative value for some messages when using python 2.7Mark Nelson1-2/+2
Type: fix Signed-off-by: Mark Nelson <manelso2@cisco.com> Change-Id: I4f121e49d3c05c21eed3fed2469bd88fc84e2271
2020-02-26vppapigen: list imports in JSONOndrej Fabry1-0/+7
Type: fix Change-Id: If50477f2487d8c13f91c119a1492370e6e937904 Signed-off-by: Ondrej Fabry <ofabry@cisco.com>
2020-02-26api: improve api string safetyJakub Grajciar1-3/+2
- Remove vl_api_from_api_string to prevent use of not nul-terminated strings. - Rename vl_api_from_api_to_vec -> vl_api_from_api_to_new_vec to imply a new vector is created. NOT nul terminated. - Add vl_api_from_api_to_new_c_string. Returns nul terminated string in a new vector. - Add vl_api_c_string_to_api_string. Convert nul terminated string to vl_api_string_t - Add vl_api_vec_to_api_string. Convert NON nul terminated vector to vl_api_string_t Type: fix Signed-off-by: Jakub Grajciar <jgrajcia@cisco.com> Change-Id: Iadd59b612c0d960a34ad0dd07a9d17f56435c6ea Signed-off-by: Jakub Grajciar <jgrajcia@cisco.com>
2020-02-04vppapigen: fix options representationOle Troan1-1/+1
Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: Ief77110160120ea0e1291cc79949a73404cdbfda
2020-01-30vppapigen: update markdown documentationPaul Vinciguerra1-7/+10
- Add newly added typedefs. - Update string examples. Change-Id: I1e7ee7cbf5901ba97302472521bf1f42a14765ea Type: docs Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-12-10api: multiple connections per processDave Barach1-1/+1
Type: feature Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I2272521d6e69edcd385ef684af6dd4eea5eaa953
2019-12-07lacp: fix control_ping from pluginsOle Troan1-0/+4
Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I030f75234a346ef6a20640c1a0ed65f4934867f2
2019-11-24vppapigen: clean up typos in c generated codePaul Vinciguerra1-4/+4
Type: style Change-Id: Ib8febf8ea83f7ce1aeb93f227d00d3111fbef4ab Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-11-24vppapigen: fix i64 format stringPaul Vinciguerra1-1/+1
Type: fix Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com> Change-Id: I6b42bb9fb9d2e57f523022e321770bb1ff7415f3
2019-11-24vppapigen: fix typo on f64 endian_stringPaul Vinciguerra1-4/+4
Type: fix Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com> Change-Id: Id47e3f2c348f07c84d9c2499122fa25d6c74fb13
2019-10-10acl: remove api boilerplateOle Troan1-0/+3
Type: refactor Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: Idc9c508eb0e3d4b0c3908360fb0771012a8e2bc0
2019-10-10memif: remove api boilerplateOle Troan1-0/+2
Type: refactor Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: Icab769607c4a5890dc80b8f2661faa7e297c2a27
2019-10-09api: autogenerate event handler functions for *_test.cOle Troan1-0/+17
Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: If75f63244f37ae3312832ca015f738c51ee7da1f Signed-off-by: Ole Troan <ot@cisco.com>
2019-09-25api: split api generated filesOle Troan3-74/+261
Split generation of API message enum and the typedefs into separate files, so that the type file can be included from elsewhere. Generate a C file for VPP that contains the API registration, this was previously done via X macros by the C pre- processor. This allows deleting lots of skeleton/copy paste code for each feature. plugins/flowprobe plugins/map examples/sample-plugin vnet/ipip used as Guinea pigs. Generate a C Test file for VAT, that does the same for VAT plugins. Also add support for a per-message CLI option, that is currently limited to VAT help text. option vat_help = "<help text>"; Type: refactor Change-Id: I245e3104bb76f7e1fb69a59ab20cc7c8dfcdd460 Signed-off-by: Ole Troan <ot@cisco.com>
2019-09-19api: split vl_api_prefix into twoOle Troan3-47/+39
One type for address with prefix and one type for prefix. Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: Icfec51d9b7d5cde1d69fbecdd97498688ab7b295 Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Klement Sekera <ksekera@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com>
2019-09-18vppapigen: fix missing vla check for union classOle Troan2-52/+87
Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: Ie775cf3469d761847ac39cf0d80a3ec6463b7928
2019-09-18vppapigen: fix tests and run on verifyPaul Vinciguerra2-26/+56
- changes vppapigen to only process an import once. Type: fix Change-Id: Ifcbcfcc69fdfb80d63195a17701762d0c239d7b4 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com> Signed-off-by: Ole Troan <ot@cisco.com>
2019-09-16api: autogenerate api trace print/endianOle Troan4-86/+434
In addition to the external vppapitrace tool, VPP itself supports dumping of API trace files. In two formats, "custom-dump" and "dump". "dump" gives a human friendly list, and "custom-dump" is meant to give a list of commands that can be fed to VAT. This patch only deals with "dump". Prior to this fix, auto-generation was only done for the basic types. This fix adds support for any type, including lists, and supports pretty-printing of enums, strings, IP addresses, MAC addresses and so on. Usage: api trace dump <api-trace-file> For example Change-Id: I4e485680e6dcfce7489299ae6cf31d835071ac40 ---------- trace 48 ----------- vl_api_sw_interface_set_flags_t: _vl_msg_id: 75 client_index: 0 context: 10 sw_if_index: 1 flags: IF_STATUS_API_FLAG_ADMIN_UP ---------- trace 49 ----------- vl_api_sw_interface_add_del_address_t: _vl_msg_id: 88 client_index: 0 context: 11 sw_if_index: 1 is_add: 1 del_all: 0 prefix: 172.16.1.1/24 ---------- trace 51 ----------- vl_api_cli_inband_t: _vl_msg_id: 819 client_index: 0 context: 13 cmd: packet-generator capture pg0 pcap /tmp/vpp-unittest-TestMAP-YhcmDX/pg0_out.pcap disable ---------- trace 58 ----------- vl_api_ip_neighbor_add_del_t: _vl_msg_id: 199 client_index: 0 context: 20 is_add: 1 neighbor: sw_if_index: 2 flags: IP_API_NEIGHBOR_FLAG_NONE mac_address: 0202.0000.ff02 ip_address: fd01:2::2 Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I5556d06008de2762e7c2d35a8b0963ae670b3db1 Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com>
2019-09-10misc: fix shebang with missing envNathan Skrzypczak1-1/+1
This is needed for macos where /usr/bin/python[23] does not exist and /usr/bin is write protected Type: fix Change-Id: Ia1b3c732248c78482993de532fa2ea96e376d90a Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com>
2019-09-03api: enforce vla is last and fixed string typeOle Troan2-11/+74
Enforce that variable length fields are the last element of API messages. Add a 'fixed' version of string type, since dealing with multiple variable length strings turned out too painful for the C language bindings. The string type is now: { string name[64]; // NUL terminated C-string. Essentially decays to u8 name[64] string name[]; // Variable length string with embedded len field (vl_api_string_t) }; The latter notation could be made available to other types as well. e.g. { vl_api_address_t addresses[]; } instead of { u32 n_addr; vl_api_address_t addresses[n_addr]; }; Type: fix Change-Id: I18fa17ef47227633752ab50453e8d20a652a9f9b Signed-off-by: Ole Troan <ot@cisco.com>
2019-08-23vppapitrace: add text outputOle Troan1-8/+65
Add new argument --todump, to mimic the VPP api trace dump output. Type: feature Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: Ia8c845b321ef9d55dd6e2f388165ca5498406ab6
2019-08-20vppapigen: remove support for legacy typedefsPaul Vinciguerra1-1/+3
vppapigen has remapped legacy to typedefs behind the scenes for some time now. - update .api files to use new style typedefs. - issue error on 'typeonly define' in .api files - remove unneeded macros redefining vl_noop_handler Type: refactor Change-Id: I7a8c4a6dafacee6a131f95cd0e9b03a8c60dea8b Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-08-20vppapigen: remove python2 supportPaul Vinciguerra1-28/+30
vppapigen has been running under python3 for months. Remove the python2 conditional code. In case of an error, explicitly log and return non-0. Type: refactor Change-Id: I20a73e0f2cb875aab6441959a85a35e88c9f8250 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-08-19vppapigen map: raise ValueError when fieldname is python keywordPaul Vinciguerra1-0/+4
When working on the lb api, one of the field names was chosen as 'as' (application server). Since 'as' is a python keyword, the field was renamed to _1 in vpp_papi. This changeset instead fails early with a descriptive message, hopefully saving others time troubleshooting the issue. ValueError: Fieldname 'as' is a python keyword and is not accessible via the python API. Type: feature Change-Id: Ib048d97de0e392645540092e356cf8989848c947 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>