summaryrefslogtreecommitdiffstats
path: root/src/vlibmemory
AgeCommit message (Collapse)AuthorFilesLines
2020-08-18api: retry sending fd on EAGAINFlorin Coras1-1/+2
Type: improvement Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I04117636ad95b706e3dd9331f00bd80e57d26d7f
2020-07-19api: call api reaper callbacks for socket clientsDave Barach2-5/+7
Add a callback to clear the per-client packet trace buffer cache. Save the packet trace dump pg setup script. Type: improvement Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I252be911b5f937ece0da5dca152263ece3d52963
2020-06-17docs: add more FEATURE.yaml documentationDave Barach1-0/+13
For src/tools/g2, src/vlibapi, and src/vlibmemory Type: docs Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I800c3e3a1ecb011d5e692d78a015e78342706786
2020-05-27dhcp: vat support for the dhcp_client_dump APIDave Barach2-0/+12
Also: permanently solve ordering issues with the vpp builtin vat plugin loader, by explicitly loading vat plugins once we're sure that all data plane plugins have registered their APIs / API message handlers. Fix compilation / link errors when the vpp builtin vat plugin loader is disbled by cmake configuration. Type: fix Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: Id553c63ae860ebfc196c5ad4b55c19e08fec2c9e Signed-off-by: Ole Troan <ot@cisco.com>
2020-05-25api: add new stream message conventionOle Troan1-0/+22
Instead of having to wrap dump/detail calls in control ping, send details messages in between a normal reply / request pair. As expressed in the below service statement. Example: service { rpc map_domains_gets returns map_domains_get_reply stream map_domain_details; }; define map_domains_get { u32 client_index; u32 context; u32 cursor; }; define map_domains_get_reply { u32 context; i32 retval; u32 cursor; }; To avoid blocking the main thread for too long, the replies are now sent in client message queue size chunks. The reply message returns VNET_API_ERROR_EAGAIN when there is more to read. The API handler must also include a "cursor" that is used to the next call to the get function. API handler example: REPLY_AND_DETAILS_MACRO (VL_API_MAP_DOMAINS_GET_REPLY, mm->domains, ({ send_domain_details (cursor, rp, mp->context); })); The macro starts from cursor and iterates through the pool until vl_api_process_may_suspend() returns true or the iteration reaches the end of the list. Client Example: cursor = 0 d = [] while True: rv, details = map_domains_get(cursor=cursor) d += details if rv.retval == 0 or rv.retval != -165: break cursor = rv.cursor or the convenience iterator: for x in vpp.details_iter(vpp.api.map_domains_get): pass or list(details_iter(map_domains_get)) Change-Id: Iad9f6b41b0ef886adb584c97708dd91cf552749e Type: feature Signed-off-by: Ole Troan <ot@cisco.com>
2020-05-15misc: removed executable bits from source filesRay Kinsella1-0/+0
Identified and removed executable bit from source files in the tree. find . -perm 755 -name *.[ch] -exec chmod a-x {} \; Type: improvement Signed-off-by: Ray Kinsella <mdr@ashroe.eu> Change-Id: I00710d59fcc46ce5be5233109af4c8077daff74b
2020-05-12api: use malloc for rx thread arg instead of heapFlorin Coras1-2/+2
Type: fix Avoids issues if thread with non-zero __os_thread_index attaches to binary api. Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: Ib981f12c867fdee7590391ec43128145bb1abce6
2020-04-22api: 'api trace' CLI consumes a line of inputNeale Ranns1-16/+22
Type: improvement allows the CLI command to be used from a config file Change-Id: Id9e7ad71b208317a65b1ed4065b0cb8777aee6ec Signed-off-by: Neale Ranns <nranns@cisco.com>
2020-04-22misc: asan: do not poison memory after munmap()Benoît Ganne1-5/+0
It is a bad idea to poison memory after munmap because the address space can be reused (eg. for global data of dlopen()ed object) and ASan model allows access by default. Moreover, access to a stale address space will fault. Type: fix Change-Id: I356de422f255447d9d50a3a71fb0c2eaa790d731 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2020-02-12api: do not truncate api dump file sizeBenoît Ganne1-1/+1
Type: fix Change-Id: I5c81d2f55057f5fba780cb12154a3fb1aef79f20 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2020-02-11misc: fix coverity warningsDave Barach1-2/+4
Type: fix Ticket: VPP-1837 Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I6b1ea13fc83460bf4ee75cb9249d83dddaa64ded
2020-02-05api: fix vl_api_clnt_node process stack overflowChenmin Sun1-0/+1
Type: fix Some simple settings(e.g. bringing up an i40e/ice interface) through vnat consume more than the currently available stack space. This root cause of this issue is same with commit b2dbb36fc265b8996fc7fa310dda447d5b0479cb "vlib: fix startup-config \ -process stack overflow" and commit 2fd44a00aa26188ca75f0accd734f2 \ 1758c199bf "vlib: fix cli process stack overflow" Signed-off-by: Chenmin Sun <chenmin.sun@intel.com> Change-Id: I312e4fed96a679aa68b859e28a90a2a4b6eb0c6e
2020-01-23api: mark api_trace_command_fn thread-safeDave Barach1-1/+20
Binary API trace replay with multiple worker threads depends in many cases on worker thread graph replica maintenance. If we (implicitly) assert a worker thread barrier at the debug CLI level, all graph replica changes are deferred until the replay operation completes. If an interface is deleted, the wheels may fall off. Type: fix Ticket: VPP-1824 Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I9b07d43f8501caa5519e5ff9ae4c19dc2661cc84
2019-12-17misc: address sanitizer: fix instrumentationBenoît Ganne1-0/+8
Type: fix Change-Id: I99e3951f8cfb7ab9d2f0a7dcee92199eab29043c Signed-off-by: Benoît Ganne <bganne@cisco.com>
2019-12-12vcl: fix disconnect from binary apiFlorin Coras1-1/+1
Type: fix Change-Id: I4398d26879b5efd932fa1d9ae232aa918ec736d6 Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-12-11vcl: separate binary api connections per thread workerFlorin Coras1-4/+27
Type: fix Change-Id: I2d72efc74a3b0a5b9e4da265475b1b01bf361125 Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-12-11api: Use vl_msg_push/pop_heapNathan Skrzypczak3-55/+26
Type: refactor Change-Id: I0eb46676fc22ce6825b2d879498df344b5a855e8 Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com>
2019-12-10api: pass api and client main to rx threadFlorin Coras2-1/+28
Type: fix Change-Id: Ib8313e87a89c80045edd897924917a88b98d1937 Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-12-10api: multiple connections per processDave Barach8-88/+105
Type: feature Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I2272521d6e69edcd385ef684af6dd4eea5eaa953
2019-12-09api: fix sock reg passing on read eventFlorin Coras3-26/+24
Type: fix Change-Id: I383242e04a114b69fe247d912842be3560e96c10 Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-12-07vlib: use explicit types in apiOle Troan1-1/+1
Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: Ib3a5d9fc36692553eb109976e9365dc7d82911ab
2019-12-06api: avoid swapping vlib_rp before barrier syncFlorin Coras3-61/+46
Type: fix Change-Id: I9868d13e827c6f5aa5535a38f629efb62ff12dbc Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com> Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-12-06api: fix free socket process argsFlorin Coras1-0/+1
Type: fix Change-Id: I910be067de6ed65790d25cc95a3d8b5b66680567 Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-12-06api: multiple socket connections per single processFlorin Coras2-31/+164
Type: feature Change-Id: Idea1857eb2225881d2982a7aa2ae7a79536b3f33 Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-11-27misc: add address sanitizer heap instrumentationBenoît Ganne3-5/+27
Introduce AddressSanitizer support: https://github.com/google/sanitizers/ This starts with heap instrumentation. vlib_buffer, bihash and stack instrumentation should follow. Type: feature Change-Id: I7f20e235b2f79db72efd0e756f22c75f717a9884 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2019-11-18api: configurable binary api client rx pthread fcnDave Barach2-19/+39
Add vl_client_connect_to_vlib_thread_fn (...) and export the memory_client_main_t definition. If you use this new API, make sure not to miss the setjmp / longjmp dance shown in .../src/vlibmemory/memory_client.c:rx_thread_fn(...), which is required for the rx pthread to terminate cleanly; please process client delete reply messages in the rx thread... Type: refactor Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: Ia67038ec59e2e14b174c67afca15b2c3f67a4d20
2019-11-07api: fix dead client scan heap issueNathan Skrzypczak1-0/+4
Type: fix On multiworker setup when an app client dies, the vec_reset_length call fails the assert in clib_mem_is_heap_object. Same thing might happen for the clib_warnings Change-Id: I369f9d2dbe60407c84994a4e8d25f6df7848ca93 Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com>
2019-10-25dhcp: fix crash on unicast renewal sendNeale Ranns1-0/+1
Type: fix - when the addresses were learnt a copy of the client was sent to the main thread, this meant the unicast adjacecny was saved on the copy not on the original. - Add logging. - Improve the proxy-node that hands the clint packets so the DHCP packets are traced. - allow a renewal to configure new address data Change-Id: I6ab0afcccbc4a1cdefdd1b8beeda8fc7ba20ec1f Signed-off-by: Neale Ranns <nranns@cisco.com>
2019-10-09api: comment, simplify and fix api socket readVratko Polak1-48/+80
The function vl_socket_read_ready did contain some comments already, but as they stated, the logic has to be tricky to cover multiple cases. Comment: + Add function-level comment + Add comments to describe some of local variables + Add many comments to describe internal state at particular lines. Simplify: + Remov mbp_set as it is never needed. + Replace msg_len with msgbuf_len to save "+ sizeof (msgbuf_t)". Improve: + Early exit on EAGAIN. Fix: + "n" now only tracks input_buffer. Previously, it was entering the detection of additional messages even for unprocessed_input. + Set up msg_buffer (including appending to unprocessed_input) outside full-message-detection loop now, so it cannot be executed multiple times as before. Type: fix Ticket: VPP-1785 Change-Id: I256e34b435be06844458744a13ea37a0e86a96f9 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-10-08api: add bapi thread handle to api main structure.IJsbrand Wijnands1-1/+9
Adding the thread handle to the api main structure allows the client process of the bin api to manage the thread, like setting the thread name for example. Type: feature Change-Id: I38b58ddc83d5958c4bda76eadd371eee1545724b Signed-off-by: IJsbrand Wijnands <ice@cisco.com>
2019-10-02vlib: improve summary vector-rate statisticsDave Barach1-1/+1
Type: refactor Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I4b77879b0a84fdec3c1518a972cf003d5135222d Signed-off-by: Ole Troan <ot@cisco.com>
2019-09-30api: fix use-after-freeBenoît Ganne1-3/+3
Type: fix Change-Id: I2a599c22d5770587fdd419e5d212c8527054d2e9 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2019-09-16api: autogenerate api trace print/endianOle Troan1-1/+2
In addition to the external vppapitrace tool, VPP itself supports dumping of API trace files. In two formats, "custom-dump" and "dump". "dump" gives a human friendly list, and "custom-dump" is meant to give a list of commands that can be fed to VAT. This patch only deals with "dump". Prior to this fix, auto-generation was only done for the basic types. This fix adds support for any type, including lists, and supports pretty-printing of enums, strings, IP addresses, MAC addresses and so on. Usage: api trace dump <api-trace-file> For example Change-Id: I4e485680e6dcfce7489299ae6cf31d835071ac40 ---------- trace 48 ----------- vl_api_sw_interface_set_flags_t: _vl_msg_id: 75 client_index: 0 context: 10 sw_if_index: 1 flags: IF_STATUS_API_FLAG_ADMIN_UP ---------- trace 49 ----------- vl_api_sw_interface_add_del_address_t: _vl_msg_id: 88 client_index: 0 context: 11 sw_if_index: 1 is_add: 1 del_all: 0 prefix: 172.16.1.1/24 ---------- trace 51 ----------- vl_api_cli_inband_t: _vl_msg_id: 819 client_index: 0 context: 13 cmd: packet-generator capture pg0 pcap /tmp/vpp-unittest-TestMAP-YhcmDX/pg0_out.pcap disable ---------- trace 58 ----------- vl_api_ip_neighbor_add_del_t: _vl_msg_id: 199 client_index: 0 context: 20 is_add: 1 neighbor: sw_if_index: 2 flags: IP_API_NEIGHBOR_FLAG_NONE mac_address: 0202.0000.ff02 ip_address: fd01:2::2 Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I5556d06008de2762e7c2d35a8b0963ae670b3db1 Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com>
2019-09-04api: memclnt api use string type.Ole Troan1-6/+6
Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: Idad79286b7730c8e85202c4b8e675ea50f8bbc48 Signed-off-by: Ole Troan <ot@cisco.com>
2019-08-27api: revert use string type for strings in memclnt.apiOle Troan6-35/+28
This reverts commit 2959d42feb576c0e00c28c4e27658b25f6c783e9. Lacks client side fixes. Type: fix Change-Id: Ib94b18e74325cede41ed1733e57896f17a952526 Signed-off-by: Ole Troan <ot@cisco.com>
2019-08-27api: use string type for strings in memclnt.apiOle Troan6-28/+35
Explicitly using string type in API allows for autogenerating tools to print strings instead of hex-dumping byte strings. Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Klement Sekera <ksekera@cisco.com> Change-Id: I573962d6b34d5d10aab9dc6a5fdf101c9b12a6a6 Signed-off-by: Ole Troan <ot@cisco.com>
2019-08-20vppapigen: remove support for legacy typedefsPaul Vinciguerra1-1/+1
vppapigen has remapped legacy to typedefs behind the scenes for some time now. - update .api files to use new style typedefs. - issue error on 'typeonly define' in .api files - remove unneeded macros redefining vl_noop_handler Type: refactor Change-Id: I7a8c4a6dafacee6a131f95cd0e9b03a8c60dea8b Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-08-08api: vppapitrace JSON/API trace converterOle Troan3-56/+38
usage: vppapitrace.py [-h] [--debug] [--apidir APIDIR] {convert,replay} ... optional arguments: -h, --help show this help message and exit --debug enable debug mode --apidir APIDIR Location of JSON API definitions subcommands: valid subcommands {convert,replay} additional help convert Convert API trace to JSON or Python and back replay Replay messages to running VPP instance To convert an API trace file to JSON: vppapitrace convert /tmp/api.trace trace.json To convert an (edited) JSON file back to API trace for replay: vppapitrace convert trace.json api-edited.trace To generate a Python file that can be replayed: vppapitrace convert /tmp/api.trace trace.py vppapitrace convert trace.json trace.py Replay it to a running VPP instance: vppapitrace replay --socket /tmp/api.trace In VPP that file can be replayed with: vpp# api trace replay api-edited.trace This patch also modifies the API binary trace format, to include the message id to message name table. Change-Id: Ie6441efb53c1c93c9f778f6ae9c1758bccc8dd87 Type: refactor Signed-off-by: Ole Troan <ot@cisco.com>
2019-08-08api: api socket respect unix runtime directoryOle Troan2-20/+12
socksvr {} should respect the unix runtime directory. Default is now /run/vpp/api.sock The 'default' keyword is deprecated. The user is responsible for creating directories outside of the unix runtime directory. Not yet done: Exit VPP when socket cannot be opened. (currently only process exits). Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I6dd66ed58a3d7e02674dfa16d72c1d7bba07b79e
2019-08-02vppinfra: Expose function setting __os_thread_indexNathan Skrzypczak1-20/+1
Type: feature This is needed when creating pthreads in client applications, they need a way to set __os_thread_index per thread that does not conflict with the binary API thread index. If __os_thread_index is left to 0 in two client pthreads and they call vl_msg_api_alloc and vec_resize at the same time it can fail due to them sharing (and push/poping) the same clib_per_cpu_mheaps slot. Change-Id: I85d4248a39b641a4d3ad5a1c1bd6e0db5875fab6 Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com>
2019-07-24api: Disable tracing of memclnt_keepalive messagesDave Barach1-6/+11
A waste of binary API trace space; an otherwise idle control-plane will eventually fill the api trace buffer with them. Type: fix Ticket: VPP-1725 Change-Id: Id8338ea4070cd76481595005986efc558f0694e6 Signed-off-by: Dave Barach <dave@barachs.net> (cherry picked from commit 996a757ff93233379bf0a464dce6d99f5d622ca4)
2019-07-16api: enable binary API event logging in vatDave Barach1-1/+22
Cleaned up a few instances of side-bet elog_string hash table usage. Elog_string handles that problem itself. Add cli commands to vat to initialize, enable/disable, and save an event log. Event logging at the same time in both vpp and vat yields a pair of event logs which can be merged by the "test_elog" tool. Type: refactor Change-Id: I8d6a72206f2309c967ea1630077fba31aef47f93 Signed-off-by: Dave Barach <dave@barachs.net>
2019-07-03api: remove garbage from sockclnt_create replyVratko Polak3-2/+25
The fix uses memset to zero after alloc, as sizing of source string is not obvious. Function vl_msg_api_alloc_zero is added (and used), so similar bugs can be fixed easily. Type: fix Ticket: VPP-1716 Change-Id: I3b20040d0de4222686c58779f2c0af78c5543504 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-06-14api: add mp-safe/barrier-sync indication to elogsDave Barach1-0/+1
Costs nothing, and leaves nothing to the imagination. Type: fix Change-Id: I7c9f9fb9325475c268eca062da7bbbf014438cfc Signed-off-by: Dave Barach <dave@barachs.net>
2019-06-12infra: fix minor memory leak in "api trace..."Dave Barach1-5/+8
Build api trace message print fns into the built-in copy of api_format.c Optimize memory allocator behavior when the api trace wraps. Type: Fix Change-Id: If799d8784a459f981fc9ee3a3ca03d3f63b2bcd0 Signed-off-by: Dave Barach <dave@barachs.net>
2019-05-16init / exit function orderingDave Barach3-10/+7
The vlib init function subsystem now supports a mix of procedural and formally-specified ordering constraints. We should eliminate procedural knowledge wherever possible. The following schemes are *roughly* equivalent: static clib_error_t *init_runs_first (vlib_main_t *vm) { clib_error_t *error; ... do some stuff... if ((error = vlib_call_init_function (init_runs_next))) return error; ... } VLIB_INIT_FUNCTION (init_runs_first); and static clib_error_t *init_runs_first (vlib_main_t *vm) { ... do some stuff... } VLIB_INIT_FUNCTION (init_runs_first) = { .runs_before = VLIB_INITS("init_runs_next"), }; The first form will [most likely] call "init_runs_next" on the spot. The second form means that "init_runs_first" runs before "init_runs_next," possibly much earlier in the sequence. Please DO NOT construct sets of init functions where A before B actually means A *right before* B. It's not necessary - simply combine A and B - and it leads to hugely annoying debugging exercises when trying to switch from ad-hoc procedural ordering constraints to formal ordering constraints. Change-Id: I5e4353503bf43b4acb11a45fb33c79a5ade8426c Signed-off-by: Dave Barach <dave@barachs.net>
2019-05-09bapi: separate init shm reply from fd exchangeFlorin Coras1-1/+17
Wait for init shm reply to be consumed before sending fd over the socket. Change-Id: I5cd0246d8debcc3c8a163b7e2dd400ca0050a2fb Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-04-23API: Python and Unix domain socket improvementOle Troan2-4/+2
Handle the case where buffer overflows. Then SOCK_SEQPACKET assumption that multiple API messages are not returned by recv() is broken. Use SOCK_STREAM for API exchanges instead. Add support for running tests over sockets. make test SOCKET=1 Change-Id: Ibe5fd69b1bf617de4c7ba6cce0a7c2b3f97a2821 Signed-off-by: Ole Troan <ot@cisco.com>
2019-04-16API: Handle pthread cond signal case where signal is missed.Ole Troan1-2/+2
Especially on a single core setup the RX thread could signal the main thread that it was done, before main thread listened to the signal. Change-Id: Ib70337b21bcf77787ce4ee0aa9cf80c6da2215af Signed-off-by: Ole Troan <ot@cisco.com>
2019-04-10API: Fix shared memory only action handlers.Ole Troan2-1/+1
Some API action handlers called vl_msg_ai_send_shmem() directly. That breaks Unix domain socket API transport. A couple (bond / vhost) also tried to send a sw_interface_event directly, but did not send the message to all that had registred interest. That scheme never worked correctly. Refactored and improved the interface event code. Change-Id: Idb90edfd8703c6ae593b36b4eeb4d3ed7da5c808 Signed-off-by: Ole Troan <ot@cisco.com>