summaryrefslogtreecommitdiffstats
path: root/src/vpp-api/python
AgeCommit message (Collapse)AuthorFilesLines
2019-10-18ipsec: make tests support python3Ole Troan1-2/+6
Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I3255702e7c562c8d04a91a095e245756c6443a9e
2019-10-14papi: fix socket sendall callsVratko Polak1-4/+6
No point in checking the return value, as .sendall() raises on error (and the previous check was missing "not"). Type: fix Change-Id: I9e07709ddd7093f91ffef87808abbab264b8aa5a Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-10-11papi: harden socket handlingVratko Polak1-18/+24
In the previous implementation of socket transport for PAPI, socket methods .send and .recv_into were used. But they are not guaranteed to send/receive all the data for the full message. The receive part contained a loop, but it handled only the main message, not the header. This patch replaces .send with .sendall and uses newly defined _read_fixed method. Also, removed Paul from maintainers, as he is not active much, lately. Type: fix Change-Id: Iae1a68bf8f9e666856b7c7d62ebfe22defc5dfe1 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-10-09papi: introduce read_blockingVratko Polak1-3/+29
Previously, self.transport.q got the messages (at least for socket transport), stored in the encoded (packed) form. In order to avoid accessing internals for async reads, a new method is introduced, to perform blocking reads of decoded (unpacked) messages. The method is also used in _call_vpp(), so sync and async reads are kept compatible. Type: feature Change-Id: Id49792dfa57c00b1a14a198031c5398d09a9ba20 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-09-25papi: truncate long logger messagesKlement Sekera1-1/+4
Dumping whole cli_inband output causes huge unformatted messages written to logger, so truncate these to avoid that. Type: fix Change-Id: I59565a98e3595cbfe4971cc346e104cb198d8f24 Signed-off-by: Klement Sekera <ksekera@cisco.com>
2019-09-19api: split vl_api_prefix into twoOle Troan3-4/+182
One type for address with prefix and one type for prefix. Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: Icfec51d9b7d5cde1d69fbecdd97498688ab7b295 Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Klement Sekera <ksekera@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com>
2019-09-16api: autogenerate api trace print/endianOle Troan1-2/+2
In addition to the external vppapitrace tool, VPP itself supports dumping of API trace files. In two formats, "custom-dump" and "dump". "dump" gives a human friendly list, and "custom-dump" is meant to give a list of commands that can be fed to VAT. This patch only deals with "dump". Prior to this fix, auto-generation was only done for the basic types. This fix adds support for any type, including lists, and supports pretty-printing of enums, strings, IP addresses, MAC addresses and so on. Usage: api trace dump <api-trace-file> For example Change-Id: I4e485680e6dcfce7489299ae6cf31d835071ac40 ---------- trace 48 ----------- vl_api_sw_interface_set_flags_t: _vl_msg_id: 75 client_index: 0 context: 10 sw_if_index: 1 flags: IF_STATUS_API_FLAG_ADMIN_UP ---------- trace 49 ----------- vl_api_sw_interface_add_del_address_t: _vl_msg_id: 88 client_index: 0 context: 11 sw_if_index: 1 is_add: 1 del_all: 0 prefix: 172.16.1.1/24 ---------- trace 51 ----------- vl_api_cli_inband_t: _vl_msg_id: 819 client_index: 0 context: 13 cmd: packet-generator capture pg0 pcap /tmp/vpp-unittest-TestMAP-YhcmDX/pg0_out.pcap disable ---------- trace 58 ----------- vl_api_ip_neighbor_add_del_t: _vl_msg_id: 199 client_index: 0 context: 20 is_add: 1 neighbor: sw_if_index: 2 flags: IP_API_NEIGHBOR_FLAG_NONE mac_address: 0202.0000.ff02 ip_address: fd01:2::2 Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I5556d06008de2762e7c2d35a8b0963ae670b3db1 Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com>
2019-09-10papi: let async calls return contextVratko Polak1-1/+6
Callback receives "reply" messages containing context, but previously there was no easy way to get the automatically generated context value of the originally sent "command" message. With this, the caller can store the contexts, so the callback knows which command got replied to. Type: feature Change-Id: I58ca812d20b03916f74096c396126710115a747c Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-09-04api: memclnt api use string type.Ole Troan3-6/+6
Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: Idad79286b7730c8e85202c4b8e675ea50f8bbc48 Signed-off-by: Ole Troan <ot@cisco.com>
2019-09-03api: enforce vla is last and fixed string typeOle Troan2-16/+70
Enforce that variable length fields are the last element of API messages. Add a 'fixed' version of string type, since dealing with multiple variable length strings turned out too painful for the C language bindings. The string type is now: { string name[64]; // NUL terminated C-string. Essentially decays to u8 name[64] string name[]; // Variable length string with embedded len field (vl_api_string_t) }; The latter notation could be made available to other types as well. e.g. { vl_api_address_t addresses[]; } instead of { u32 n_addr; vl_api_address_t addresses[n_addr]; }; Type: fix Change-Id: I18fa17ef47227633752ab50453e8d20a652a9f9b Signed-off-by: Ole Troan <ot@cisco.com>
2019-08-09stats: create /run/vpp before stat socket bind()YohanPipereau1-1/+1
When VPP tries to bind to stats.sock it will complain about non-existing /run/vpp directory. /run/vpp is created before cli socket operations are performed. The same should be done for stat socket. Ticket: VPP-1708 Type: fix Change-Id: I53d70939c8125d04a365ac51a6cbf8926dc52adf Signed-off-by: YohanPipereau <ypiperea@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com>
2019-08-08api: vppapitrace JSON/API trace converterOle Troan2-141/+153
usage: vppapitrace.py [-h] [--debug] [--apidir APIDIR] {convert,replay} ... optional arguments: -h, --help show this help message and exit --debug enable debug mode --apidir APIDIR Location of JSON API definitions subcommands: valid subcommands {convert,replay} additional help convert Convert API trace to JSON or Python and back replay Replay messages to running VPP instance To convert an API trace file to JSON: vppapitrace convert /tmp/api.trace trace.json To convert an (edited) JSON file back to API trace for replay: vppapitrace convert trace.json api-edited.trace To generate a Python file that can be replayed: vppapitrace convert /tmp/api.trace trace.py vppapitrace convert trace.json trace.py Replay it to a running VPP instance: vppapitrace replay --socket /tmp/api.trace In VPP that file can be replayed with: vpp# api trace replay api-edited.trace This patch also modifies the API binary trace format, to include the message id to message name table. Change-Id: Ie6441efb53c1c93c9f778f6ae9c1758bccc8dd87 Type: refactor Signed-off-by: Ole Troan <ot@cisco.com>
2019-08-08api: api socket respect unix runtime directoryOle Troan1-1/+1
socksvr {} should respect the unix runtime directory. Default is now /run/vpp/api.sock The 'default' keyword is deprecated. The user is responsible for creating directories outside of the unix runtime directory. Not yet done: Exit VPP when socket cannot be opened. (currently only process exits). Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I6dd66ed58a3d7e02674dfa16d72c1d7bba07b79e
2019-07-31api papi: add alias for timestamp(datetime)/timedeltaPaul Vinciguerra1-1/+8
Now that we have support for f64: - create explicit types for timestamp(datetime)/timedelta - update log_details to use timestamp and remove redundant string representation. If you need the string representation, in python do str(timestamp). If you prefer the raw f64 value, the client can pass in the _no_type_conversion option. Type: feature Change-Id: I547b5fa7122d2afa12628b7db0192c23babbbae8 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-07-24papi: add additional types to vpp_serializerPaul Vinciguerra1-0/+3
vppapigen supports additional types not handled by the papi seializer. This adds the types for the sake of parity. Change-Id: Id8efad7f169b5023879935575ace2cc3e9c291d4 Type: feature Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-07-16papi: use the injected logger wherever possibleVratko Polak1-18/+6
As the injected logger is already expected to be used everywhere, this is a fix. The few lines in vpp_serializer.py are not fixed, but they are not encountered in CSIT testing. Functions call_logger and return_logger have single call site each (and confusing names, as they do not log), so saved few lines by inlining them. Type: fix Change-Id: I7dd1e610ef6b885943708bf78bddedfbcf4daa1a Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-07-12papi: fix vpp_format from change in vl_api_prefix_tPaul Vinciguerra2-12/+10
cherry picking fix for bug introduced by https://gerrit.fd.io/r/#/c/20011/ - also fixes unit tests. Type: fix Fixes: ab05508e1eb96749b68de8ccd2f6f88ff3e64fad Change-Id: I8287385f094911ea70de4751a716a7e0e6521b64 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-07-09papi: update VPPEnumType for python3Paul Vinciguerra1-1/+4
Python3 uses __bool__ instead of __nonzero__ Type: fix Depends-on: https://gerrit.fd.io/r/#/c/20484/ Change-Id: I7dd13d0508ab18d6c50c235f4186006799e92b45 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-07-08map gbp papi: match endianess of f64Paul Vinciguerra1-1/+1
clib_net_to_host_f64, clib_host_to_net_f64 are now implemented as '=', https://gerrit.fd.io/r/#/c/20406/ set papi to match. - all f64 api references are now wrapped with clib_net_to_host_f64 or clib_host_to_net_f64. IEEE f64 endianess is not defined. If clib_net_to_host_f64 and clib_host_to_net_f64 are later defined in VPP as big-endian, it is a single character change in the papi vpp_serializer. Note: This breaks the api in a manner that would not be detected by the flag day initiative. The scope is small. This only impacts map.api, which applied the u64 transformation, while the gbp api uses '='. The implementation of "=" raises issues for the papi socket implementation if used between systems of differing endianess. See Vratko's comments. - Added get_f64_endian_value() to api to allow client to verify endianess of f64's. Type: fix Depends-on: https://gerrit.fd.io/r/#/c/20484/ Change-Id: I00fc64a6557ba0190398df211aa0ea5c7eb101df Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-07-01papi: refactor set_errors_strPaul Vinciguerra1-3/+3
Assuming 2500 50 char lines: string_concat elapsed: 5.919933e-04 string_join elapsed: 2.830029e-04 string_concat size: 156,312,500 string_join size: 125,000 Type: refactor Change-Id: I0a400265426bde492e9a651d24996ddcfe954aa1 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-06-26papi: prevent message_table inconsistenciesVratko Polak1-0/+2
Socket transport is maintaining message_table to map message name to index. After disconnect and re-connect, the VPP may have been restarted with different set of plugins, so message_table has to be wiped. + Edited MAINTAINERS to recognize "papi" as a separate component. Type: fix Change-Id: I1f16ad7ee0886e03bbc6a17bcddbfcaa322354e5 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-06-26api: refactor format_vl_api_prefix_t return keysPaul Vinciguerra2-13/+13
format_vl_api_prefix_t returns a dict with keys 'address' and 'address_length', but other format_vl_api_prefix functions return a dict with 'prefix', and 'len'. Refactor all format_vl_api_prefix_t to return consistent keys 'address' and 'len'. Type: refactor Change-Id: I5f9558fc2da8742a303266e011102f5b2db80aad Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-06-25api: fix vac_read return codesPaul Vinciguerra1-2/+9
- vac_read was returning the same code for multiple errors. - Refactor VppTransportShmemIOError to capture rv and description. Type: refactor Change-Id: Ifb8ca5ff679f658bcd92a43ecddaffd2fc18dbd5 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-06-20misc: papi - add __repr__ to FuncWrapperPaul Vinciguerra1-0/+3
Eases troubleshooting. Changes: <FuncWrapper object at 0x7fa7402de790> to <FuncWrapper(func=<want_ip6_ra_events(u16 _vl_msg_id, u32 client_index, u32 context, u8 enable_disable, u32 pid)>)> Type: refactor Change-Id: I3a893090e024a63a0b3b0de51cb3bb2d2a1678c5 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-06-18misc: vpp_papi- add tests, clean up pep8Paul Vinciguerra7-18/+202
Type: test Change-Id: Ic81bd4a9eba3c89a746e7a9b9e471b59cd87fa40 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-06-17misc: vpp-api - add __repr__ to VPPApiClient.Paul Vinciguerra1-0/+13
Show the constructor of the object instead of the memory location. api client: <VPPApiClient apifiles=None, testmode=False, async_thread=True, logger=<logging.Logger object at 0x7f35d1bd2590>, read_timeout=1, use_socket=False, server_address='/tmp/vpp-unittest-TestGbpApi-pOKbWg/api.sock'> vs. api client: <vpp_papi.vpp_papi.VPPApiClient object at 0x7fa0003df750> Type: feature Change-Id: I1460d76f6796776dd525c0844e9e426ec2d52398 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-06-14stats: add version field to stat segment base headerOle Troan1-0/+3
Add a version in the base header of the stat segment To make support for multiple reader implementations safer. Change-Id: I6816e2a51a98c2df1e621e80d4ef0b4ba4e9f47b Type: feature Signed-off-by: Ole Troan <ot@cisco.com>
2019-06-12papi: make socket disconnect more robustVratko Polak1-1/+1
Previusly, read error could happen before message_thread starts, so do not attempt to join in that case. Change-Id: Ie119916831e320e38eb02e48515b4c6a9cb937e5 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-06-07API: Add support for "defaults"Ole Troan4-28/+82
Add support in the API language for specifying a field default. Add default support in Python binding. define foo { u16 mtu [default = 1500]; }; This is client side only. I.e. if the mtu argument is not passed to the foo function, the client language binding will set it default to 1500. Change-Id: I5df43f3cd87cb300b40ca38e15dcab25b40e424a Signed-off-by: Ole Troan <ot@cisco.com>
2019-06-07vpp_papi: Context_id allocator for running forked.Paul Vinciguerra2-5/+60
When running forked, distinct copies of the 'get_context' singleton are created for each process. To run under forked processes, (as with make test TEST_JOBS=10), we need to use a shared memory value across the processes. Type: fix Change-Id: I9eab8ce46ec23584e5bd651735ad75fd3f018e1a Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-06-07vpp_papi: Fix missing dependency.Paul Vinciguerra1-2/+8
Since we test vpp_papi as part of tests, the need for ipaddress under python2.7 is masked. Add ipaddress to setup.py for python <=3.3. Change-Id: I01c2f5560eeb740e546024e84028d5a2fb2ace45 Type: fix Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-06-07vpp_papi: Fix format_vl_api_address_t under python3.Paul Vinciguerra1-1/+3
Under PY3, inet_pton returns OSError, not socket.error. Type: fix Change-Id: Id270a684c0ab124cbe1ddcb7123e14e85af844b8 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-06-07vpp_papi: Accept ipaddress.Network objects on prefix_t methods.Paul Vinciguerra1-1/+16
vl_api_prefix_t returns ipaddress.<IPv[46]>Network objects. Accept Network objects as well as existing string format. Type: refactor. Change-Id: Iba5403724a7b3e1da3b3740027fccd43631bf31e Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-06-07vpp_papi: Fix vapi.disconnect exception.Paul Vinciguerra1-0/+1
Traceback (most recent call last): File "/vpp/test/framework.py", line 521, in setUpClass cls.quit() File "/vpp/test/framework.py", line 556, in quit cls.vapi.disconnect() File "/vpp/test/vpp_papi_provider.py", line 308, in disconnect self.vpp.disconnect() File "/vpp/src/vpp-api/python/vpp_papi/vpp_papi.py", line 500, in disconnect if self.event_thread is not None: AttributeError: 'VPPApiClient' object has no attribute 'event_thread' Type: fix Change-Id: I7a8db08b0922bf92ab42df121b1707073af9cedf Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-06-04stats: removing empty object in stats vectorOle Troan1-1/+3
Type: fix Change-Id: I9b6bdacdb9a0750834de9a93d8c3f7ed827ce3c8 Signed-off-by: Ole Troan <ot@cisco.com>
2019-05-28papi: avoid IOError on disconnectVratko Polak2-6/+18
Change-Id: I331efb20b98a7e3c507d9158d0221ee7d5353b18 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-05-23vpp_papi: Do not raise before find_api_dir()Vratko Polak1-4/+1
The change https://gerrit.fd.io/r/18352 has introduced a check which raises when neither apifiles nor apidir are set. This change removes that check, as it broke CSIT. If such a check is intended, find_api_dir() should be removed. Also, make sure you do not break CSIT next time. Change-Id: I0faab7c41d199e3aa5d5bc6aab0defd1ab9e10da Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-05-22stats: support multiple works for error countersOle Troan1-4/+15
The current code only allowed access to the main thread error counters. That is not so useful for a multi worker instance. No return a vector indexed by thread of counter_t values. Type: fix Change-Id: Ie322c8889c0c8175e1116e71de04a2cf453b9ed7 Signed-off-by: Ole Troan <ot@cisco.com>
2019-05-15vpp_papi: remove dependency on environment var. VPP_API_DIR.Paul Vinciguerra1-6/+9
Change-Id: I9e3af8674e8aae27079fd03f6286f165d777814f Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-05-10vpp_transport_socket: make connect more resilientVratko Polak1-11/+32
This should make connect() partially retriable, at least against "No such file or directory on socket", and when disconnect() is called before retrying connect(). Added TODOs for future improvements. Change-Id: I5ee727fbc17d66446589b8c781f270055187db68 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-05-09Improve exceptions in vpp_transport_socket.pyVratko Polak1-2/+4
Added TODOs for further improvements. Change-Id: I6724dd8b3de5537518a5c6e0ee9abe72f57b273e Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-05-09Fix copypaste in vpp_papi/vpp_transport_socket.pyVratko Polak1-3/+2
Change-Id: I80e6ccd6f63e3eca2ba723f3b39a47475ae9aed0 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-05-08Fix Exception typo in vpp_papi/vpp_transport_socket.py.Paul Vinciguerra1-1/+1
Change-Id: I80ef74abcb14136984a4c94ba60414a1348015d3 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-05-07Allow repeated connects on PAPI socket transportVratko Polak1-1/+10
The previous implementation fails with RuntimeError("cannot set daemon status of active thread") when a second connect() follows first disconnect(). Change-Id: I671005923eb020130a57b0b484a3386a05d7be33 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-04-29API: Add support for limits to language.Ole Troan2-7/+40
string name [limit = 64]; Meta-data to do argument validation. Change-Id: I1f3e0f09b2d5285224399413d25206f77bd3f4b1 Signed-off-by: Ole Troan <ot@cisco.com>
2019-04-23API: Python and Unix domain socket improvementOle Troan1-27/+19
Handle the case where buffer overflows. Then SOCK_SEQPACKET assumption that multiple API messages are not returned by recv() is broken. Use SOCK_STREAM for API exchanges instead. Add support for running tests over sockets. make test SOCKET=1 Change-Id: Ibe5fd69b1bf617de4c7ba6cce0a7c2b3f97a2821 Signed-off-by: Ole Troan <ot@cisco.com>
2019-04-16stats: Add name vectors to Python clientOle Troan1-3/+21
Change-Id: Ic62dfa0bf3e082a0b999026830c64a9c543da586 Signed-off-by: Ole Troan <ot@cisco.com>
2019-03-22vppapigen: allow for enum size other than u32Andrew Yourtchenko1-3/+5
Change-Id: If20d2fbab9b854b7db276c81918fdff6abcb8385 Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>
2019-03-20vpp_papi: Expose the list of field attributes to the client.Paul Vinciguerra1-0/+1
Change-Id: I7aba1760f68798fc5bea1244ce798e4b528a4df4 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-03-19vpp_papi: specify encoding for .encode()/.decode()Paul Vinciguerra3-9/+10
py2 defaults to 'ascii', py3 defaults to 'utf-8'. Change-Id: Ib76d0ec56f24535e4bafd397cc6fb5e85967d508 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>