summaryrefslogtreecommitdiffstats
path: root/src/vpp-api/python/vpp_papi
AgeCommit message (Collapse)AuthorFilesLines
2023-03-21papi: vla list of fixed stringsOle Troan2-4/+28
Handle a variable length array of fixed strings. Like: fixed_string = VPPType("fixed_string", [["string", "data", 32]]) s = VPPType("string_vla", [["u32", "length"], ["fixed_string", "services", 0, "length"]]) Previously instead of packing and unpacking as strings, exception packed as u8 instead of list. Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I501a8a4755828042e1539fd5a54eacec21c5e364 Signed-off-by: Ole Troan <ot@cisco.com>
2022-12-12papi: fix VPP_API_DIRPim van Pelt1-1/+4
Docstring in VPP Python API says that find_api_dir() will search for environment variable VPP_API_DIR first and foremost, except it doesn't. Prepend VPP_API_DIR if it exists, and allow dirs to be omitted in case it will be the empty list [] Type: fix Signed-off-by: pim@ipng.nl Change-Id: Ic892e4bb7d8ff50f18e64ddfd2a61343883f07ea
2022-12-07papi: export packed message structuresOle Troan1-0/+16
Use the Python API binding to generate a set of API messages in binary format, that can later be replayed independently of the Python API. Type: improvement Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: Iaab6ca31fd2809193e461ab53f7cc7332a231eb5 Signed-off-by: Ole Troan <ot@cisco.com>
2022-12-07tests: multiple apidir locationsOle Troan1-1/+6
To support testing of external plugins, add support to the test framework and PAPI for specifying a list of locations to look for api.json files. Type: improvement Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I128a306e3c091dc8ef994801b1470b82d2f4595d Signed-off-by: Ole Troan <ot@cisco.com>
2022-12-07papi: fix async support for socket transportOle Troan2-4/+5
Async use of the API is much faster than blocking calls. Seemed like it only worked over shared memory transport. This patches re-enables support for async calls over socket transport. Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I05f3b362035ce0a1c16788ba9003a35601ddb04e Signed-off-by: Ole Troan <ot@cisco.com>
2022-06-01papi: vpp_serializer.py - replace slow bytes() with fast bytearray()Viktor Velichkin1-8/+8
https://docs.python.org/3/library/stdtypes.html "if concatenating bytes objects, you can similarly use bytes.join() or io.BytesIO, or you can do in-place concatenation with a bytearray object. bytearray objects are mutable and have an efficient overallocation mechanism" Type: improvement Signed-off-by: Viktor Velichkin <avisom@yandex.ru> Change-Id: Id20d337f909cce83fcd9e08e8049bb0bf5970fbc
2022-05-10tests: replace pycodestyle with blackKlement Sekera11-993/+1089
Drop pycodestyle for code style checking in favor of black. Black is much faster, stable PEP8 compliant code style checker offering also automatic formatting. It aims to be very stable and produce smallest diffs. It's used by many small and big projects. Running checkstyle with black takes a few seconds with a terse output. Thus, test-checkstyle-diff is no longer necessary. Expand scope of checkstyle to all python files in the repo, replacing test-checkstyle with checkstyle-python. Also, fixstyle-python is now available for automatic style formatting. Note: python virtualenv has been consolidated in test/Makefile, test/requirements*.txt which will eventually be moved to a central location. This is required to simply the automated generation of docker executor images in the CI. Type: improvement Change-Id: I022a326603485f58585e879ac0f697fceefbc9c8 Signed-off-by: Klement Sekera <klement.sekera@gmail.com> Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2022-03-31stats: convert error counters to normal countersDamjan Marion1-61/+15
Change-Id: I9794da718805b40cc922e4f3cf316255398029a9 Type: improvement Signed-off-by: Damjan Marion <damarion@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com>
2021-08-31stats: check epoch in python vpp_stats lsOle Troan1-0/+3
The VPP Stats Python binding was missing checking epoch. Resulting it would not pick up changes in the directory. Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I3226d71c77be2e80e24b5cd48fc9820833f7d30e Signed-off-by: Ole Troan <ot@cisco.com>
2021-07-12papi: remove shared memory transportOle Troan2-152/+4
This patch removes the papi transport shared memory plugin. It also removes any dependency on CFFI. Type: feature Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: Ia81701c0dc506871e511495d837e41420e1fdf72 Signed-off-by: Ole Troan <ot@cisco.com>
2021-06-25stats: reverts part of a fix on Python clientArthur de Kerhor1-12/+4
Puts return statements back inside code blocks under the lock Type: fix Change-Id: I76d426f336200035026b92bcb0ffe2b472a3142d Signed-off-by: Arthur de Kerhor <arthurdekerhor@gmail.com>
2021-06-25stats: fix race conditions in vpp-api stats clientArthur de Kerhor1-4/+12
Type: fix Signed-off-by: Arthur de Kerhor <arthurdekerhor@gmail.com> Signed-off-by: Dave Wallace <dwallacelf@gmail.com> Change-Id: Ie5c197f6ec0d41d5e405b22662701d83ad94d29e
2021-05-21stats: catch stat segment overrun in retry loopOle Troan1-3/+3
Raise IOError so that the blocking code retries not only on optimistic locking failures but also on data segment illegal reads. Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I6bb250e239486b60192004271c1690e790513318
2021-05-04stats: adding symlinks for nodes and interfaces in the stat segmentArthur de Kerhor1-40/+88
A given interface counter (e.g rx) can be accessed via /interfaces/<interface_name>/<counter_name>. Same goes with nodes: /nodes/<node_name>/<counter_name> As interfaces may contain '/' in their names, these are replaced by '_' in symlinks Also added 2 tests for the stat segment Type: feature Signed-off-by: Arthur de Kerhor <arthurdekerhor@gmail.com> Change-Id: I27da252e7b3dc177815616ca46b5c200a456bf0f Signed-off-by: Ole Troan <ot@cisco.com>
2021-04-28papi: change default to use socket transport instead of shared memory transportOle Troan1-2/+2
In preparation for removing Python shared memory support, change the default to use sockets. This may affect users of PAPI. E.g. if running against instances of VPP where the API socket is in a different location or disabled. Type: improvement Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I96308ed70b9ff314c9b487722174f5e4b14efdd2 Signed-off-by: Ole Troan <ot@cisco.com>
2021-04-15papi: fix ubuntu 1804 make test socket.close errorTianyu Li1-1/+1
make test failed on ubuntu 18.04 File "/vpp/src/vpp-api/python/vpp_papi/vpp_stats.py", line 135, in connect socket.close(mfd) AttributeError: module 'socket' has no attribute 'close' Due to ubuntu 18.04 using python3.6 and socket.close() is introduced since python3.7 Using os.close to replace socket.close as mfd is a memory fd Type: fix Signed-off-by: Tianyu Li <tianyu.li@arm.com> Change-Id: I6d980fc87ae6c77bbed416879f9b2fcd0a0abe6a
2021-03-30stats: python: handle when pattern is not list in lsOle Troan1-0/+9
The reimplementation of python stats module mishandled the case where pattern to ls was not a list. Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I9ba189423a76f2fd4298c4c4b19a0875f705d719
2021-03-25stats: python vpp_stats rewrite to access stat segment directlyOle Troan1-296/+482
This module implement Python access to the VPP statistics segment. It accesses the data structures directly in shared memory. VPP uses optimistic locking, so data structures may change underneath us while we are reading. Data is copied out and it's important to spend as little time as possible "holding the lock". Counters are stored in VPP as a two dimensional array. Index by thread and index (typically sw_if_index). Simple counters count only packets, Combined counters count packets and octets. Counters can be accessed in either dimension. stat['/if/rx'] - returns 2D lists stat['/if/rx'][0] - returns counters for all interfaces for thread 0 stat['/if/rx'][0][1] - returns counter for interface 1 on thread 0 stat['/if/rx'][0][1]['packets'] - returns the packet counter for interface 1 on thread 0 stat['/if/rx'][:, 1] - returns the counters for interface 1 on all threads stat['/if/rx'][:, 1].packets() - returns the packet counters for interface 1 on all threads stat['/if/rx'][:, 1].sum_packets() - returns the sum of packet counters for interface 1 on all threads stat['/if/rx-miss'][:, 1].sum() - returns the sum of packet counters for interface 1 on all threads for simple counters Type: refactor Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I1fe7f7c7d11378d06be8276db5e1900ecdb8f515 Signed-off-by: Ole Troan <ot@cisco.com>
2021-03-16api: allow specifying no timeoutKlement Sekera1-1/+1
This functionality is used in make test when DEBUG=gdb is used. Type: fix Signed-off-by: Klement Sekera <ksekera@cisco.com> Change-Id: I1339e6460aa624a3bcb4b03db46991590e126f92
2021-03-15tests: use socket transport instead of shared memoryOle Troan2-7/+2
Type: improvement Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I9e65c94a5a05047a5104e9361ea36eac77b40442 Signed-off-by: Ole Troan <ot@cisco.com>
2021-02-15papi: add method to retrieve field optionsPaul Vinciguerra1-0/+9
Sample usage: cls.MEMIF_DEFAULT_BUFFER_SIZE = cls.vapi.vpp.get_field_options( 'memif_create', 'buffer_size')['default'] Type: improvement Change-Id: I298f4687623003a78c93a703d32f59a937e37bc2 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2021-02-09papi: expose vpp_papi version to clientPaul Vinciguerra1-0/+6
root@ae2a2e85c5d4:/vpp/src/vpp-api/python# python3 Python 3.6.9 (default, Oct 8 2020, 12:12:24) [GCC 8.4.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import vpp_papi >>> vpp_papi.__version__ '1.6.2' Type: feature Change-Id: I73025427a58214a842245fceaa77daa7acd3e6f8 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2021-02-05papi: remove unused name from __all__Paul Vinciguerra1-1/+1
missed in https://gerrit.fd.io/r/c/vpp/+/30238 Type: fix Change-Id: I342868a77d7ca87896d97847ca6525ad1abcd1b6 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2020-12-18papi: remove import vpp_transport_socketPaul Vinciguerra1-2/+2
See discussion in: https://gerrit.fd.io/r/c/vpp/+/19472 Type: refactor Change-Id: Icbf686f615f13898d12fd211f68c3498d373a5d7 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2020-12-08papi: remove old client referencePaul Vinciguerra2-4/+1
deprecated in 3/19 Type: refactor Change-Id: Ia8eebaa0cd89d1e95166a62645400946d8db2f75 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2020-12-04papi: add support for enumflag part 1 of 2Paul Vinciguerra5-19/+160
Allow for papi to accept the new enumflag type. Change-Id: I8d8dc8fdb122e9a1b1881f5b2558635c75e4a299 Type: feature Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com> (cherry picked from commit 299abebe2942b4c78b85f9f3b8843f8213bf1efe)
2020-12-03papi: remove python2 patternsPaul Vinciguerra6-40/+18
Type: refactor Change-Id: I541759b4b788acf08599661eb9e7abc58e9283d2 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2020-12-03papi: allow client control over loggersPaul Vinciguerra6-8/+56
This change enables a client to set debug levels globally as well as individually. exposes loggers as vpp_papi vpp_papi.serializer vpp_papi.transport Type: improvement Change-Id: Ib6bd1a1f552b51a22c9fe3de819a5fb970963ae5 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2020-12-02papi: improve unit testabilityPaul Vinciguerra3-23/+133
refactor the code so that snippets of json can be used to test vpp_papi example unit test provided Type: improvement Change-Id: Ibec608fd2e5b12515aa4db17d85d4319134c22ea Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2020-12-02papi: fix copy/paste errorPaul Vinciguerra1-2/+2
Type: fix Change-Id: I8bd20fb38e132f6ab8cbc0e73095b649b5946498 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2020-11-17papi: remove dependency on aenumPaul Vinciguerra1-11/+2
remove the dependency on the aenum package which was used to provide Enum.IntFlag which is now available in the python stdlib. aenum is not provided as a .deb and causes issues in packaging. Type: fix Change-Id: Ie45ec2130a767345f0aad038451780a5ddc7e8db Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2020-10-07api: add heap alloc to vpp statsOle Troan2-1/+3
The Python VPP Stats module also used the VPP heap. Fix so it now explicitly allocates a heap. Fixes: f68fccfe7e188fec2c9f91da38ca9acf6f67d811 Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I0bd4ae64d6c89cdf634d8d9a91c23ab38017c5cc Signed-off-by: Ole Troan <ot@cisco.com>
2020-09-28api: remove clib_mem_init from vppapiclient contructorOle Troan1-1/+4
Having the constructor in the vppapiclient library led to conflicts with applications wanting to allocate their own heap. Note: Change of behaviour, applications that do not use a CLIB heap must now call vac_mem_init() before using any functions from vppapiclient. Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: Ib155a54579ea5a0dbc26cb4b6daca1274e1dfdfa
2020-08-13stats: remove offsets on vpp sideOle Troan1-5/+4
Represent pointers directly in shared memory and require clients to adjust for shared memory segment being mapped at different base address. Deprecated: stat_segment_pointer() / stat_segment_offset() Added: stat_segment_adjust() Bumped the stat segment version to 2. Type: refactor Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I33e756187b8903b45dcd353e6c1a101b7a4acb79
2020-08-11papi: allow unknown address familyOle Troan2-10/+71
In unions all representations of the union are decoded. Which means trying to decode something that isn't an address might have invalid address family types. Type: fix Ticket: VPP-1884 Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: Id3381ef8cc885952c1eb488ebc70e276eaceb366
2020-05-25api: add new stream message conventionOle Troan1-14/+36
Instead of having to wrap dump/detail calls in control ping, send details messages in between a normal reply / request pair. As expressed in the below service statement. Example: service { rpc map_domains_gets returns map_domains_get_reply stream map_domain_details; }; define map_domains_get { u32 client_index; u32 context; u32 cursor; }; define map_domains_get_reply { u32 context; i32 retval; u32 cursor; }; To avoid blocking the main thread for too long, the replies are now sent in client message queue size chunks. The reply message returns VNET_API_ERROR_EAGAIN when there is more to read. The API handler must also include a "cursor" that is used to the next call to the get function. API handler example: REPLY_AND_DETAILS_MACRO (VL_API_MAP_DOMAINS_GET_REPLY, mm->domains, ({ send_domain_details (cursor, rp, mp->context); })); The macro starts from cursor and iterates through the pool until vl_api_process_may_suspend() returns true or the iteration reaches the end of the list. Client Example: cursor = 0 d = [] while True: rv, details = map_domains_get(cursor=cursor) d += details if rv.retval == 0 or rv.retval != -165: break cursor = rv.cursor or the convenience iterator: for x in vpp.details_iter(vpp.api.map_domains_get): pass or list(details_iter(map_domains_get)) Change-Id: Iad9f6b41b0ef886adb584c97708dd91cf552749e Type: feature Signed-off-by: Ole Troan <ot@cisco.com>
2020-04-28tests: implement ipaddress convenience methodsPaul Vinciguerra2-1/+34
Add vpp specific properties to ip addresses for use in the api. .vapi_af -- returns [ADDRESS_IP4, ADDRESS_IP6] .vapi_af_name -- returns the string ['ip4', 'ip6'] Update tests to demonstrate usage. Type: feature Change-Id: I43447a1522769d99f89debdc714c51700068d771 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-12-19papi: lazily initialize stats clientPaul Vinciguerra1-5/+25
remove wait-loop on stats socket from test framework. Type: refactor Change-Id: I5bb95a7c597707a87f9d9a471215c4b4af1a2280 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-12-05papi: add call statsOle Troan1-1/+23
Type: feature Change-Id: Ic6d44122d3e62e09402e3f1946f7e57e9b5e7c5f Signed-off-by: Ole Troan <ot@cisco.com>
2019-12-03papi: add a per-call _timeout optionPaul Vinciguerra3-8/+14
add the ability to override the default timeout value on a per-call basis. Use: rv = self.vapi.papi.cli_inband(cmd='wait 10', _timeout=15) Type: feature Change-Id: Ia90a58586a1f63e02118599a2a4b7141e5a0b90d Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-11-27papi: fix typo in reprPaul Vinciguerra1-1/+1
Reported by Vratko's review. (Thanks for the review) Fixes: 14b0b4791c0b8c886e7b5c9ca667d060f0bada0b Type: fix Change-Id: I9c080c0c40060cc77977e76edae03d60eb393ce2 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-11-24papi: fix papi default handlingOle Troan1-17/+18
Type: fix Change-Id: I91063e2096fb09c34898a611184c8381fccdb333 Signed-off-by: Ole Troan <ot@cisco.com>
2019-11-24papi: add missing base types to serializerPaul Vinciguerra1-2/+4
File "/vpp/src/vpp-api/python/vpp_papi/vpp_serializer.py", line 512, in __init__ 'Unknown message type {}'.format(f_type)) vpp_papi.vpp_serializer.VPPSerializerValueError: Unknown message type i16 Type: fix Change-Id: Ibf73dc8df90153db586afe614e47be49739bac2f Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-11-23papi: add repr to packer types for troubleshootingPaul Vinciguerra1-0/+47
Type: feature Change-Id: Id3cd89eca0deddb70f506239f9d0543fc28cf7f4 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-11-22papi: support default for type alias decaying to basetypeOle Troan1-18/+63
Add PAPI support for VppTypeAlias decaying to BaseType. E.g vl_api_interface_index_t sw_if_index [default=0xffffffff] Type: feature Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I2061392157c9c11fbb0ff9e5406ea65489b017e9
2019-11-13papi: enhance MACAddress() equalityPaul Vinciguerra1-1/+7
Allows for comparison without needing str(MACAddress()) Traceback (most recent call last): File "/vpp/test/test_ip6.py", line 1074, in test_icmpv6_echo self.assertEqual(ether.dst, self.pg0.remote_mac) AssertionError: '02:01:00:00:ff:02' != MACAddress(02:01:00:00:ff:02) Type: feature Change-Id: Ife1cbfc74d477695d15b33a19da7dd2fa241a348 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-11-12papi: add wrapper to validate crc manifestOle Troan1-0/+28
If a client application is built against 19.08, it can dump the "manifest" of API signatures. Either the all APIs (--dump) or the APIs it is interested in (--dumpfiltered). When the developers of said client application wants to verify that it works with VPP 20.01. It can connect to VPP and --validate the old mainfest file, and will be told a list of messages (both request and reply) that has changed. import argparse from vpp_papi import VPP import sys import argparse parser = argparse.ArgumentParser() group = parser.add_mutually_exclusive_group() group.add_argument("--dump", action="store_true") group.add_argument("--dumpfiltered", action="store_true") group.add_argument("--validate", action="store_true") args = parser.parse_args() vpp = VPP(use_socket=True) vpp.connect(name='apimanifest') if args.validate: # Verify manifest message_table = eval(sys.stdin.read()) missing = vpp.validate_message_table(message_table) print ('Changed message signatures: {}'.format(missing)) elif args.dump: # Output manifest to stdout print('{}'.format(vpp.dump_message_table())) elif args.dumpfiltered: # Output manifest to stdout filterlist = eval(sys.stdin.read()) print('{}'.format(vpp.dump_message_table_filtered(filterlist))) vpp.disconnect() Type: feature Change-Id: I7e708b36f599ed88e4864970c8593cc2fe5fbf61 Signed-off-by: Ole Troan <ot@cisco.com>
2019-11-05misc: Fix python scripts shebang lineRenato Botelho do Couto4-4/+4
Type: fix Since CentOS 8, RPM build script doesn't accept '#!/usr/bin/env python' as a valid shebang line. It requires scripts to explicitly chose between python2 or python3. Change all to use python3 as suggested by Paul Vinciguerra. Depends-On: https://gerrit.fd.io/r/23170 Signed-off-by: Renato Botelho do Couto <renato@netgate.com> Change-Id: Ie72af9f60fd0609e07f05b70f8d96e738b2754d1
2019-10-23papi: fix default handlingOle Troan2-40/+51
The BaseTypers object were reused, so a default for anyother mesage would be inherited in new messages. Type: fix Fixes: 85465588b18fef9c4712f864f512e00741e2d4f2 Change-Id: Ie1efb85a76b088653eb9ea4b88540c98b6b0aad0 Signed-off-by: Ole Troan <ot@cisco.com>
2019-10-21bier: tests support python3Ole Troan1-8/+9
Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I3cf5295f1a85579a66ba38ca1f74678b45474959