aboutsummaryrefslogtreecommitdiffstats
path: root/src/vpp-api/python/vpp_papi/vpp_papi.py
AgeCommit message (Collapse)AuthorFilesLines
2024-01-29api: provide api definition over apiOle Troan1-19/+65
This patch allows a client to bootstrap itself by downloading the JSON API definitions over the API itself. This patch enables it for Python (probably need a dynamic language). Call VPPApiClient with the new bootstrapapi=True parameter. Example (Python): from vpp_papi import VPPApiClient vpp = VPPApiClient(bootstrapapi=True) rv = vpp.connect("foobar") assert rv == 0 print(f'SHOW VERSION: {vpp.api.show_version()}') vpp.disconnect() Type: feature Change-Id: Id903fdccc82b2e22aa1994331d2c150253f2ccae Signed-off-by: Ole Troan <otroan@employees.org>
2024-01-23papi: support old style of providing apidirVratko Polak1-1/+4
77caeb1b193404e76beb27a1aa81321e8eb1cf1f has changed the behavior in an incompatible way, breaking users (including CSIT). The new behavior is more pythonic, but the old behavior has to be supported at least one release after publicly deprecating it. Type: fix Change-Id: I9dfdd2229065010216e49db80b14b856c545965c Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2024-01-18tests: preload api filesMaxime Peim1-29/+39
When sanity test is not done, API files are not loaded until the first test case is run. Hence, it is not possible to use enums, etc. outside of a test class. By preloading API files before running any tests, it prevents its issue. Type: fix Change-Id: I8730150374e6c5f8d6933ec037811372ac2a8da0 Signed-off-by: Maxime Peim <mpeim@cisco.com>
2022-12-12papi: fix VPP_API_DIRPim van Pelt1-1/+4
Docstring in VPP Python API says that find_api_dir() will search for environment variable VPP_API_DIR first and foremost, except it doesn't. Prepend VPP_API_DIR if it exists, and allow dirs to be omitted in case it will be the empty list [] Type: fix Signed-off-by: pim@ipng.nl Change-Id: Ic892e4bb7d8ff50f18e64ddfd2a61343883f07ea
2022-12-07papi: export packed message structuresOle Troan1-0/+16
Use the Python API binding to generate a set of API messages in binary format, that can later be replayed independently of the Python API. Type: improvement Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: Iaab6ca31fd2809193e461ab53f7cc7332a231eb5 Signed-off-by: Ole Troan <ot@cisco.com>
2022-12-07tests: multiple apidir locationsOle Troan1-1/+6
To support testing of external plugins, add support to the test framework and PAPI for specifying a list of locations to look for api.json files. Type: improvement Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I128a306e3c091dc8ef994801b1470b82d2f4595d Signed-off-by: Ole Troan <ot@cisco.com>
2022-12-07papi: fix async support for socket transportOle Troan1-1/+1
Async use of the API is much faster than blocking calls. Seemed like it only worked over shared memory transport. This patches re-enables support for async calls over socket transport. Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I05f3b362035ce0a1c16788ba9003a35601ddb04e Signed-off-by: Ole Troan <ot@cisco.com>
2022-05-10tests: replace pycodestyle with blackKlement Sekera1-161/+189
Drop pycodestyle for code style checking in favor of black. Black is much faster, stable PEP8 compliant code style checker offering also automatic formatting. It aims to be very stable and produce smallest diffs. It's used by many small and big projects. Running checkstyle with black takes a few seconds with a terse output. Thus, test-checkstyle-diff is no longer necessary. Expand scope of checkstyle to all python files in the repo, replacing test-checkstyle with checkstyle-python. Also, fixstyle-python is now available for automatic style formatting. Note: python virtualenv has been consolidated in test/Makefile, test/requirements*.txt which will eventually be moved to a central location. This is required to simply the automated generation of docker executor images in the CI. Type: improvement Change-Id: I022a326603485f58585e879ac0f697fceefbc9c8 Signed-off-by: Klement Sekera <klement.sekera@gmail.com> Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2021-07-12papi: remove shared memory transportOle Troan1-9/+4
This patch removes the papi transport shared memory plugin. It also removes any dependency on CFFI. Type: feature Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: Ia81701c0dc506871e511495d837e41420e1fdf72 Signed-off-by: Ole Troan <ot@cisco.com>
2021-04-28papi: change default to use socket transport instead of shared memory transportOle Troan1-2/+2
In preparation for removing Python shared memory support, change the default to use sockets. This may affect users of PAPI. E.g. if running against instances of VPP where the API socket is in a different location or disabled. Type: improvement Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I96308ed70b9ff314c9b487722174f5e4b14efdd2 Signed-off-by: Ole Troan <ot@cisco.com>
2021-03-15tests: use socket transport instead of shared memoryOle Troan1-5/+0
Type: improvement Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I9e65c94a5a05047a5104e9361ea36eac77b40442 Signed-off-by: Ole Troan <ot@cisco.com>
2021-02-15papi: add method to retrieve field optionsPaul Vinciguerra1-0/+9
Sample usage: cls.MEMIF_DEFAULT_BUFFER_SIZE = cls.vapi.vpp.get_field_options( 'memif_create', 'buffer_size')['default'] Type: improvement Change-Id: I298f4687623003a78c93a703d32f59a937e37bc2 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2021-02-05papi: remove unused name from __all__Paul Vinciguerra1-1/+1
missed in https://gerrit.fd.io/r/c/vpp/+/30238 Type: fix Change-Id: I342868a77d7ca87896d97847ca6525ad1abcd1b6 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2020-12-08papi: remove old client referencePaul Vinciguerra1-3/+0
deprecated in 3/19 Type: refactor Change-Id: Ia8eebaa0cd89d1e95166a62645400946d8db2f75 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2020-12-04papi: add support for enumflag part 1 of 2Paul Vinciguerra1-5/+19
Allow for papi to accept the new enumflag type. Change-Id: I8d8dc8fdb122e9a1b1881f5b2558635c75e4a299 Type: feature Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com> (cherry picked from commit 299abebe2942b4c78b85f9f3b8843f8213bf1efe)
2020-12-03papi: remove python2 patternsPaul Vinciguerra1-18/+8
Type: refactor Change-Id: I541759b4b788acf08599661eb9e7abc58e9283d2 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2020-12-03papi: allow client control over loggersPaul Vinciguerra1-3/+3
This change enables a client to set debug levels globally as well as individually. exposes loggers as vpp_papi vpp_papi.serializer vpp_papi.transport Type: improvement Change-Id: Ib6bd1a1f552b51a22c9fe3de819a5fb970963ae5 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2020-12-02papi: improve unit testabilityPaul Vinciguerra1-21/+64
refactor the code so that snippets of json can be used to test vpp_papi example unit test provided Type: improvement Change-Id: Ibec608fd2e5b12515aa4db17d85d4319134c22ea Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2020-05-25api: add new stream message conventionOle Troan1-14/+36
Instead of having to wrap dump/detail calls in control ping, send details messages in between a normal reply / request pair. As expressed in the below service statement. Example: service { rpc map_domains_gets returns map_domains_get_reply stream map_domain_details; }; define map_domains_get { u32 client_index; u32 context; u32 cursor; }; define map_domains_get_reply { u32 context; i32 retval; u32 cursor; }; To avoid blocking the main thread for too long, the replies are now sent in client message queue size chunks. The reply message returns VNET_API_ERROR_EAGAIN when there is more to read. The API handler must also include a "cursor" that is used to the next call to the get function. API handler example: REPLY_AND_DETAILS_MACRO (VL_API_MAP_DOMAINS_GET_REPLY, mm->domains, ({ send_domain_details (cursor, rp, mp->context); })); The macro starts from cursor and iterates through the pool until vl_api_process_may_suspend() returns true or the iteration reaches the end of the list. Client Example: cursor = 0 d = [] while True: rv, details = map_domains_get(cursor=cursor) d += details if rv.retval == 0 or rv.retval != -165: break cursor = rv.cursor or the convenience iterator: for x in vpp.details_iter(vpp.api.map_domains_get): pass or list(details_iter(map_domains_get)) Change-Id: Iad9f6b41b0ef886adb584c97708dd91cf552749e Type: feature Signed-off-by: Ole Troan <ot@cisco.com>
2020-04-28tests: implement ipaddress convenience methodsPaul Vinciguerra1-1/+29
Add vpp specific properties to ip addresses for use in the api. .vapi_af -- returns [ADDRESS_IP4, ADDRESS_IP6] .vapi_af_name -- returns the string ['ip4', 'ip6'] Update tests to demonstrate usage. Type: feature Change-Id: I43447a1522769d99f89debdc714c51700068d771 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-12-05papi: add call statsOle Troan1-1/+23
Type: feature Change-Id: Ic6d44122d3e62e09402e3f1946f7e57e9b5e7c5f Signed-off-by: Ole Troan <ot@cisco.com>
2019-12-03papi: add a per-call _timeout optionPaul Vinciguerra1-4/+6
add the ability to override the default timeout value on a per-call basis. Use: rv = self.vapi.papi.cli_inband(cmd='wait 10', _timeout=15) Type: feature Change-Id: Ia90a58586a1f63e02118599a2a4b7141e5a0b90d Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-11-12papi: add wrapper to validate crc manifestOle Troan1-0/+28
If a client application is built against 19.08, it can dump the "manifest" of API signatures. Either the all APIs (--dump) or the APIs it is interested in (--dumpfiltered). When the developers of said client application wants to verify that it works with VPP 20.01. It can connect to VPP and --validate the old mainfest file, and will be told a list of messages (both request and reply) that has changed. import argparse from vpp_papi import VPP import sys import argparse parser = argparse.ArgumentParser() group = parser.add_mutually_exclusive_group() group.add_argument("--dump", action="store_true") group.add_argument("--dumpfiltered", action="store_true") group.add_argument("--validate", action="store_true") args = parser.parse_args() vpp = VPP(use_socket=True) vpp.connect(name='apimanifest') if args.validate: # Verify manifest message_table = eval(sys.stdin.read()) missing = vpp.validate_message_table(message_table) print ('Changed message signatures: {}'.format(missing)) elif args.dump: # Output manifest to stdout print('{}'.format(vpp.dump_message_table())) elif args.dumpfiltered: # Output manifest to stdout filterlist = eval(sys.stdin.read()) print('{}'.format(vpp.dump_message_table_filtered(filterlist))) vpp.disconnect() Type: feature Change-Id: I7e708b36f599ed88e4864970c8593cc2fe5fbf61 Signed-off-by: Ole Troan <ot@cisco.com>
2019-11-05misc: Fix python scripts shebang lineRenato Botelho do Couto1-1/+1
Type: fix Since CentOS 8, RPM build script doesn't accept '#!/usr/bin/env python' as a valid shebang line. It requires scripts to explicitly chose between python2 or python3. Change all to use python3 as suggested by Paul Vinciguerra. Depends-On: https://gerrit.fd.io/r/23170 Signed-off-by: Renato Botelho do Couto <renato@netgate.com> Change-Id: Ie72af9f60fd0609e07f05b70f8d96e738b2754d1
2019-10-09papi: introduce read_blockingVratko Polak1-3/+29
Previously, self.transport.q got the messages (at least for socket transport), stored in the encoded (packed) form. In order to avoid accessing internals for async reads, a new method is introduced, to perform blocking reads of decoded (unpacked) messages. The method is also used in _call_vpp(), so sync and async reads are kept compatible. Type: feature Change-Id: Id49792dfa57c00b1a14a198031c5398d09a9ba20 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-09-25papi: truncate long logger messagesKlement Sekera1-1/+4
Dumping whole cli_inband output causes huge unformatted messages written to logger, so truncate these to avoid that. Type: fix Change-Id: I59565a98e3595cbfe4971cc346e104cb198d8f24 Signed-off-by: Klement Sekera <ksekera@cisco.com>
2019-09-10papi: let async calls return contextVratko Polak1-1/+6
Callback receives "reply" messages containing context, but previously there was no easy way to get the automatically generated context value of the originally sent "command" message. With this, the caller can store the contexts, so the callback knows which command got replied to. Type: feature Change-Id: I58ca812d20b03916f74096c396126710115a747c Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-09-04api: memclnt api use string type.Ole Troan1-3/+3
Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: Idad79286b7730c8e85202c4b8e675ea50f8bbc48 Signed-off-by: Ole Troan <ot@cisco.com>
2019-08-08api: vppapitrace JSON/API trace converterOle Troan1-141/+152
usage: vppapitrace.py [-h] [--debug] [--apidir APIDIR] {convert,replay} ... optional arguments: -h, --help show this help message and exit --debug enable debug mode --apidir APIDIR Location of JSON API definitions subcommands: valid subcommands {convert,replay} additional help convert Convert API trace to JSON or Python and back replay Replay messages to running VPP instance To convert an API trace file to JSON: vppapitrace convert /tmp/api.trace trace.json To convert an (edited) JSON file back to API trace for replay: vppapitrace convert trace.json api-edited.trace To generate a Python file that can be replayed: vppapitrace convert /tmp/api.trace trace.py vppapitrace convert trace.json trace.py Replay it to a running VPP instance: vppapitrace replay --socket /tmp/api.trace In VPP that file can be replayed with: vpp# api trace replay api-edited.trace This patch also modifies the API binary trace format, to include the message id to message name table. Change-Id: Ie6441efb53c1c93c9f778f6ae9c1758bccc8dd87 Type: refactor Signed-off-by: Ole Troan <ot@cisco.com>
2019-08-08api: api socket respect unix runtime directoryOle Troan1-1/+1
socksvr {} should respect the unix runtime directory. Default is now /run/vpp/api.sock The 'default' keyword is deprecated. The user is responsible for creating directories outside of the unix runtime directory. Not yet done: Exit VPP when socket cannot be opened. (currently only process exits). Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I6dd66ed58a3d7e02674dfa16d72c1d7bba07b79e
2019-07-16papi: use the injected logger wherever possibleVratko Polak1-18/+6
As the injected logger is already expected to be used everywhere, this is a fix. The few lines in vpp_serializer.py are not fixed, but they are not encountered in CSIT testing. Functions call_logger and return_logger have single call site each (and confusing names, as they do not log), so saved few lines by inlining them. Type: fix Change-Id: I7dd1e610ef6b885943708bf78bddedfbcf4daa1a Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-06-20misc: papi - add __repr__ to FuncWrapperPaul Vinciguerra1-0/+3
Eases troubleshooting. Changes: <FuncWrapper object at 0x7fa7402de790> to <FuncWrapper(func=<want_ip6_ra_events(u16 _vl_msg_id, u32 client_index, u32 context, u8 enable_disable, u32 pid)>)> Type: refactor Change-Id: I3a893090e024a63a0b3b0de51cb3bb2d2a1678c5 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-06-18misc: vpp_papi- add tests, clean up pep8Paul Vinciguerra1-4/+6
Type: test Change-Id: Ic81bd4a9eba3c89a746e7a9b9e471b59cd87fa40 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-06-17misc: vpp-api - add __repr__ to VPPApiClient.Paul Vinciguerra1-0/+13
Show the constructor of the object instead of the memory location. api client: <VPPApiClient apifiles=None, testmode=False, async_thread=True, logger=<logging.Logger object at 0x7f35d1bd2590>, read_timeout=1, use_socket=False, server_address='/tmp/vpp-unittest-TestGbpApi-pOKbWg/api.sock'> vs. api client: <vpp_papi.vpp_papi.VPPApiClient object at 0x7fa0003df750> Type: feature Change-Id: I1460d76f6796776dd525c0844e9e426ec2d52398 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-06-07vpp_papi: Context_id allocator for running forked.Paul Vinciguerra1-5/+7
When running forked, distinct copies of the 'get_context' singleton are created for each process. To run under forked processes, (as with make test TEST_JOBS=10), we need to use a shared memory value across the processes. Type: fix Change-Id: I9eab8ce46ec23584e5bd651735ad75fd3f018e1a Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-06-07vpp_papi: Fix vapi.disconnect exception.Paul Vinciguerra1-0/+1
Traceback (most recent call last): File "/vpp/test/framework.py", line 521, in setUpClass cls.quit() File "/vpp/test/framework.py", line 556, in quit cls.vapi.disconnect() File "/vpp/test/vpp_papi_provider.py", line 308, in disconnect self.vpp.disconnect() File "/vpp/src/vpp-api/python/vpp_papi/vpp_papi.py", line 500, in disconnect if self.event_thread is not None: AttributeError: 'VPPApiClient' object has no attribute 'event_thread' Type: fix Change-Id: I7a8db08b0922bf92ab42df121b1707073af9cedf Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-05-28papi: avoid IOError on disconnectVratko Polak1-1/+4
Change-Id: I331efb20b98a7e3c507d9158d0221ee7d5353b18 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-05-23vpp_papi: Do not raise before find_api_dir()Vratko Polak1-4/+1
The change https://gerrit.fd.io/r/18352 has introduced a check which raises when neither apifiles nor apidir are set. This change removes that check, as it broke CSIT. If such a check is intended, find_api_dir() should be removed. Also, make sure you do not break CSIT next time. Change-Id: I0faab7c41d199e3aa5d5bc6aab0defd1ab9e10da Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-05-15vpp_papi: remove dependency on environment var. VPP_API_DIR.Paul Vinciguerra1-6/+9
Change-Id: I9e3af8674e8aae27079fd03f6286f165d777814f Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-03-20vpp_papi: Expose the list of field attributes to the client.Paul Vinciguerra1-0/+1
Change-Id: I7aba1760f68798fc5bea1244ce798e4b528a4df4 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-03-19vpp_papi: specify encoding for .encode()/.decode()Paul Vinciguerra1-4/+5
py2 defaults to 'ascii', py3 defaults to 'utf-8'. Change-Id: Ib76d0ec56f24535e4bafd397cc6fb5e85967d508 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-03-04test framework: add factory function and default parametersOle Troan1-0/+2
This is the first step to be able to remove all the API message wrappers in vpp_papi_provider.py. This allows to remove all functions that do not override parameters (different from zero), and a separate dictionary for messages requiring different defaults. The general requirement is that all new tests should use named arguments directly. Not positional arguments through the wrapper. Note when removing functions, the calls in vpp_papi_provider wrappers do not necessarily follow message order. Change-Id: If64916c07f8622c138db3a9d7c4a98b93a058e68 Signed-off-by: Ole Troan <ot@cisco.com>
2019-02-28vpp_papi.py: Change VppEnum from explicit metaclass to metaclass ↵Paul Vinciguerra1-4/+11
wrapper/decorator. Change-Id: Iab41bb972e3c7ec005a1a13d5a25f654ae8c3932 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2018-12-18PAPI: Add MACAddress object wrapper for vl_api_mac_address_tOle Troan1-0/+2
Change the definition of vl_api_mac_address_t to an aliased type. Change-Id: I1434f316d0fad6a099592f39bceeb8faeaf1d134 Signed-off-by: Ole Troan <ot@cisco.com>
2018-12-13API: Use string type instead of u8.Ole Troan1-3/+43
The new string type is modelled after string in proto3. It is always variable length. Change-Id: I64884067e28a80072c8dac31b7c7c82d6e306051 Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Michal Cmarada <mcmarada@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com>
2018-12-12PAPI: Allow ipaddress object as argument and return values from API callsOle Troan1-10/+12
The API calls that use any of vl_api_address_t, vl_api_ip4_address, vl_api_ip6_address_t, vl_api_prefix_t, vl_api_ip4_prefix_t, vl_api_ip6_prefix_t now accepts either the old style dictionary, a text string (2001:db8::/32) or an ipaddress ojbect. Unless it is called with '_no_type_conversion':True, it will also return an appropriate ipaddress object. Change-Id: I84e4a1577bd57f6b5ae725f316a523988b6a955b Signed-off-by: Ole Troan <ot@cisco.com>
2018-12-10Python API: Use services to determine stream RPCOle Troan1-4/+11
The services {} definitions contain which APIs are streaming. In addition only create function definition for the client side (not for reply messages). Change-Id: I8e83d35386cdd9ebee83d4571eaebdc6dff19e82 Signed-off-by: Ole Troan <ot@cisco.com>
2018-11-29vpp_papi: Add custom exceptions.Paul Vinciguerra1-12/+37
This patchset adds and raises the following custom exception classes: * class VPPApiError(Exception): * class VPPNotImplementedError(NotImplementedError): * class VPPIOError(IOError): * class VPPRuntimeError(RuntimeError): * class VPPValueError(ValueError): * class VPPSerializerValueError(ValueError): * class VPPStatsIOError(IOError): * class VPPStatsClientLoadError(RuntimeError): * class VppTransportShmemIOError(IOError): * class VppTransportSocketIOError(IOError) Change-Id: Ia40900fd2dcef148d01125d6c691329fc666901e Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2018-11-29API: Add support for type aliasesOle Troan1-2/+9
Previously all types are compound. This adds support for aliases, so one can do things like: typedef u32 interface_index; or typedef u8 ip4_address[4]; Change-Id: I0455cad0123fc88acb491d2a3ea2725426bdb246 Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Klement Sekera <ksekera@cisco.com>
2018-11-28vpp_papi: Use new style classes.Paul Vinciguerra1-2/+2
Python2 defaults to old style classes to maintain compatability with python 2.1. Moving to new style classes will ensure consistent behavior across interpreters. Change-Id: I89493d608d1edb63989000c17a9566a97785a4aa Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>