aboutsummaryrefslogtreecommitdiffstats
path: root/src/vlibmemory
AgeCommit message (Collapse)AuthorFilesLines
2021-09-28api: API trace improvementsFilip Tehlar8-155/+485
Type: improvement * add support for JSON format in API trace * add ability to replay JSON API trace in both VPP and VAT2 * use CRC for backward compatibility check during JSON API replay * fix API trace CLI (and remove duplicits) * remove custom dump * remove vppapitrace.py * update docs accordingly Change-Id: I5294f68bebe6cbe738630f457f3a87720e06486b Signed-off-by: Filip Tehlar <ftehlar@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com>
2021-09-28misc: vpe.api messages dynamically allocatedOle Troan2-1/+9
This is the last in the series of moving API messages from vpp/api/vpe.api to vlibmemory/memclnt.api. This patch makes the remaining vpe.api messages dynamic, to help VAT2 binary-api command. Moves the VAT test code to a separate file and removes the now unnused API meta files. Type: improvement Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I01dd78eaff1d3715dff17d2643bf0f7f0089935b Signed-off-by: Ole Troan <ot@cisco.com>
2021-09-27misc: api move continuedFlorin Coras3-0/+61
Move control ping and change dependencies from vpe.api_types to memclnt.api_types Type: refactor Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I9f8bc442e28738c48d64d1f6794082c8c4f5725b
2021-09-27misc: move part of vpe apis to vlibmemoryFlorin Coras5-601/+1674
VPE apis are actually vlib apis. This moves those that are not tightly coupled with vapi to vlib_api Type: refactor Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I456a64ce49a0cdeff4a0931c6ea513cb639f683e Signed-off-by: Ole Troan <ot@cisco.com>
2021-07-14api: fix memory error in multi-thread environmentXiaoming Jiang1-1/+7
When reading vm->pending_rpc_requests in main thread, the content may be changed by other workers. Type: fix Signed-off-by: Xiaoming Jiang <jiangxiaoming@outlook.com> Change-Id: I27e9d357b5ecec0f97cd8b950019b35f72fc5f76
2021-06-22api: asan: use vec_set_len() so ASan can keep track of buffer lenBenoît Ganne1-6/+6
ASan poison/unpoison vectors based on the vector length, but _vec_len() bypass ASan annotations, contrary to vec_set_len(). Type: fix Change-Id: I5265a5adcddef1aa11c77e0c3827346b1a66c306 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2021-06-16api: remove custom dumpFilip Tehlar1-32/+3
Type: improvement Change-Id: I4b9b2be8817be10e46accc19219deb2b544f266b Signed-off-by: Filip Tehlar <ftehlar@cisco.com>
2021-05-26api: socket client connect set to blockingFlorin Coras1-1/+1
Binary api client must otherwise check the returned error and if it was EAGAIN/EINPROGRESS poll for connect completion. Type: improvement Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I89845b1a59b9fa2ab0968029435ceb203bfa8f6c
2021-03-26vlib: split vlib_main_t into global and per-threadDamjan Marion1-2/+3
Type: refactor Change-Id: I8b273bc3bf16aa360f031f1b2692f766e5fc4613 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-03-26vlib: introduce vlib_get_elog_main()Damjan Marion1-11/+16
Type: improvement Change-Id: I73383eb15186021cd6527d112da8443a0082f129 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-03-26vlib: introduce vlib_get_main_by_index(), vlib_get_n_threads()Damjan Marion1-2/+2
Type: improvement Change-Id: If3da7d4338470912f37ff1794620418d928fb77f Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-03-19api: Avoid the usage of the freed registration by the API callsAndrew Yourtchenko1-23/+51
This issue happens if: - the API client connects via Unix socket - the client issues the *_dump API call and immediately disconnects What happens after is that the API handler keeps sending the *_details messages, however at some point the write fails, and the socket is deleted. The attempt of a use of the registration pointer results in interpreting the socket as a shared memory socket. This results in a crash, because the data in this structure then does not make sense, like the below: | |Thread 1 "vpp_main" received signal SIGSEGV, Segmentation fault. |__GI___pthread_mutex_lock (mutex=0x0) at ../nptl/pthread_mutex_lock.c:67 |67 ../nptl/pthread_mutex_lock.c: No such file or directory. |(gdb) bt |#0 __GI___pthread_mutex_lock (mutex=0x0) at ../nptl/pthread_mutex_lock.c:67 |#1 0x00007ffff500f957 in svm_queue_lock (q=0x0) at /home/ubuntu/vpp/src/svm/queue.c:101 |#2 svm_queue_add (q=0x0, elem=0x7fffa76c2de0 "\210\365\006\060\001", nowait=0) at /home/ubuntu/vpp/src/svm/queue.c:274 |#3 0x00007ffff6e131e3 in vl_api_send_msg (rp=<optimized out>, elem=<optimized out>) at /home/ubuntu/vpp/src/vlibmemory/api.h:43 |#4 send_sw_interface_details (am=<optimized out>, rp=<optimized out>, swif=0x7fffb957a0bc, interface_name=<optimized out>, context=<optimized out>) | at /home/ubuntu/vpp/src/vnet/interface_api.c:353 |#5 0x00007ffff6e0edeb in vl_api_sw_interface_dump_t_handler (mp=<optimized out>) at /home/ubuntu/vpp/src/vnet/interface_api.c:412 |#6 0x00007ffff7daeb48 in msg_handler_internal (am=<optimized out>, the_msg=0x7fffb839a5e0, trace_it=<optimized out>, do_it=1, free_it=0) | at /home/ubuntu/vpp/src/vlibapi/api_shared.c:501 |#7 vl_msg_api_socket_handler (the_msg=0x7fffb839a5e0) at /home/ubuntu/vpp/src/vlibapi/api_shared.c:790 |#8 0x00007ffff7d7c608 in vl_socket_process_api_msg (rp=<optimized out>, input_v=0x7fffa76c2de0 "\210\365\006\060\001") at /home/ubuntu/vpp/src/vlibmemory/socket_api.c:212 |#9 0x00007ffff7d89ff1 in vl_api_clnt_process (vm=<optimized out>, node=<optimized out>, f=<optimized out>) at /home/ubuntu/vpp/src/vlibmemory/vlib_api.c:405 |#10 0x00007ffff53bf9a7 in vlib_process_bootstrap (_a=<optimized out>) at /home/ubuntu/vpp/src/vlib/main.c:1490 |#11 0x00007ffff4da0b2c in clib_calljmp () from /home/ayourtch/vpp/build-root/install-vpp-native/vpp/lib/libvppinfra.so.21.06 |#12 0x00007fffa99a4d90 in ?? () |#13 0x00007ffff53b6cb2 in vlib_process_startup (vm=0x7ffff56a9880 <vlib_global_main>, p=0x7fffb5d41380, f=0x0) at /home/ubuntu/vpp/src/vlib/main.c:1515 |#14 dispatch_process (vm=0x7ffff56a9880 <vlib_global_main>, p=0x7fffb5d41380, f=0x0, last_time_stamp=<optimized out>) at /home/ubuntu/vpp/src/vlib/main.c:1571 |#15 0x0000000000000000 in ?? () |(gdb) frame 3 |#3 0x00007ffff6e131e3 in vl_api_send_msg (rp=<optimized out>, elem=<optimized out>) at /home/ubuntu/vpp/src/vlibmemory/api.h:43 |43 vl_msg_api_send_shmem (rp->vl_input_queue, (u8 *) & elem); |(gdb) l |38 { |39 vl_socket_api_send (rp, elem); |40 } |41 else |42 { |43 vl_msg_api_send_shmem (rp->vl_input_queue, (u8 *) & elem); |44 } |45 } |46 |47 always_inline int |(gdb) | The approach in this change is to avoid the closing operations "here and now", but instead mark the the registration as a zombie and place a forced RPC towards a callback that does the actual cleanup work. Forced RPC is handled via the API processing loop with barrier sync, so we are guaranteed not to have any API processing in-process. Type: fix Change-Id: I1972d42da620bdb4fd773c83262863c2781d9005 Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>
2021-03-03api: fix crash when cf removedwanghanlin1-2/+16
cf may be removed when: 1. linux_epoll_input_inline process two EPOLLIN events, firstly a normal message, secondly reading 0 bytes because of socket client crash, then cf removed without clear message added to pending event data vectors before 2. clib_file_write called Type: fix Signed-off-by: wanghanlin <wanghanlin@corp.netease.com> Change-Id: I4523e9bb322e98357575925f3113f710d70dd679
2021-01-20vlib: fix build with muslNathan Moos1-0/+1
Type: fix When building with musl, some transitive includes from glibc are not present. This patch adds direct includes for the necessary types, and it also defines _GNU_SOURCE to ensure that the socket types are complete. Change-Id: Ic69c307b9515fec764c32906b5bc7f1fb34f2525 Signed-off-by: Nathan Moos <nmoos@cisco.com>
2020-12-28vlib: add missing file template descriptionsPaul Vinciguerra1-0/+2
Add descriptions to clib_file_t template structures so that sockets can be identified via the 'show unix file' cli command. Type: fix Change-Id: Ibf82d55aa6c7b1126bd252b76d0dc8b7076f5046 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2020-12-14misc: move to new pool_foreach macrosDamjan Marion3-13/+13
Type: refactor Change-Id: Ie67dc579e88132ddb1ee4a34cb69f96920101772 Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-10-21svm: fix ASAN annotations for external chunksBenoît Ganne1-0/+5
Chunks can be allocated from another process. We need to manually mark them as accessible for ASAN. Type: fix Change-Id: Ifbeef3346e9cee2c1231f80cbcf7f9673b5b54be Signed-off-by: Benoît Ganne <bganne@cisco.com>
2020-09-30svm: harmonize ssvm namesFlorin Coras2-3/+3
Type: refactor Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I26e4ad6bfd9e0be7745f6ba948bf51550fd4215e
2020-08-18api: retry sending fd on EAGAINFlorin Coras1-1/+2
Type: improvement Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I04117636ad95b706e3dd9331f00bd80e57d26d7f
2020-07-19api: call api reaper callbacks for socket clientsDave Barach2-5/+7
Add a callback to clear the per-client packet trace buffer cache. Save the packet trace dump pg setup script. Type: improvement Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I252be911b5f937ece0da5dca152263ece3d52963
2020-06-17docs: add more FEATURE.yaml documentationDave Barach1-0/+13
For src/tools/g2, src/vlibapi, and src/vlibmemory Type: docs Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I800c3e3a1ecb011d5e692d78a015e78342706786
2020-05-27dhcp: vat support for the dhcp_client_dump APIDave Barach2-0/+12
Also: permanently solve ordering issues with the vpp builtin vat plugin loader, by explicitly loading vat plugins once we're sure that all data plane plugins have registered their APIs / API message handlers. Fix compilation / link errors when the vpp builtin vat plugin loader is disbled by cmake configuration. Type: fix Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: Id553c63ae860ebfc196c5ad4b55c19e08fec2c9e Signed-off-by: Ole Troan <ot@cisco.com>
2020-05-25api: add new stream message conventionOle Troan1-0/+22
Instead of having to wrap dump/detail calls in control ping, send details messages in between a normal reply / request pair. As expressed in the below service statement. Example: service { rpc map_domains_gets returns map_domains_get_reply stream map_domain_details; }; define map_domains_get { u32 client_index; u32 context; u32 cursor; }; define map_domains_get_reply { u32 context; i32 retval; u32 cursor; }; To avoid blocking the main thread for too long, the replies are now sent in client message queue size chunks. The reply message returns VNET_API_ERROR_EAGAIN when there is more to read. The API handler must also include a "cursor" that is used to the next call to the get function. API handler example: REPLY_AND_DETAILS_MACRO (VL_API_MAP_DOMAINS_GET_REPLY, mm->domains, ({ send_domain_details (cursor, rp, mp->context); })); The macro starts from cursor and iterates through the pool until vl_api_process_may_suspend() returns true or the iteration reaches the end of the list. Client Example: cursor = 0 d = [] while True: rv, details = map_domains_get(cursor=cursor) d += details if rv.retval == 0 or rv.retval != -165: break cursor = rv.cursor or the convenience iterator: for x in vpp.details_iter(vpp.api.map_domains_get): pass or list(details_iter(map_domains_get)) Change-Id: Iad9f6b41b0ef886adb584c97708dd91cf552749e Type: feature Signed-off-by: Ole Troan <ot@cisco.com>
2020-05-15misc: removed executable bits from source filesRay Kinsella1-0/+0
Identified and removed executable bit from source files in the tree. find . -perm 755 -name *.[ch] -exec chmod a-x {} \; Type: improvement Signed-off-by: Ray Kinsella <mdr@ashroe.eu> Change-Id: I00710d59fcc46ce5be5233109af4c8077daff74b
2020-05-12api: use malloc for rx thread arg instead of heapFlorin Coras1-2/+2
Type: fix Avoids issues if thread with non-zero __os_thread_index attaches to binary api. Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: Ib981f12c867fdee7590391ec43128145bb1abce6
2020-04-22api: 'api trace' CLI consumes a line of inputNeale Ranns1-16/+22
Type: improvement allows the CLI command to be used from a config file Change-Id: Id9e7ad71b208317a65b1ed4065b0cb8777aee6ec Signed-off-by: Neale Ranns <nranns@cisco.com>
2020-04-22misc: asan: do not poison memory after munmap()Benoît Ganne1-5/+0
It is a bad idea to poison memory after munmap because the address space can be reused (eg. for global data of dlopen()ed object) and ASan model allows access by default. Moreover, access to a stale address space will fault. Type: fix Change-Id: I356de422f255447d9d50a3a71fb0c2eaa790d731 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2020-02-12api: do not truncate api dump file sizeBenoît Ganne1-1/+1
Type: fix Change-Id: I5c81d2f55057f5fba780cb12154a3fb1aef79f20 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2020-02-11misc: fix coverity warningsDave Barach1-2/+4
Type: fix Ticket: VPP-1837 Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I6b1ea13fc83460bf4ee75cb9249d83dddaa64ded
2020-02-05api: fix vl_api_clnt_node process stack overflowChenmin Sun1-0/+1
Type: fix Some simple settings(e.g. bringing up an i40e/ice interface) through vnat consume more than the currently available stack space. This root cause of this issue is same with commit b2dbb36fc265b8996fc7fa310dda447d5b0479cb "vlib: fix startup-config \ -process stack overflow" and commit 2fd44a00aa26188ca75f0accd734f2 \ 1758c199bf "vlib: fix cli process stack overflow" Signed-off-by: Chenmin Sun <chenmin.sun@intel.com> Change-Id: I312e4fed96a679aa68b859e28a90a2a4b6eb0c6e
2020-01-23api: mark api_trace_command_fn thread-safeDave Barach1-1/+20
Binary API trace replay with multiple worker threads depends in many cases on worker thread graph replica maintenance. If we (implicitly) assert a worker thread barrier at the debug CLI level, all graph replica changes are deferred until the replay operation completes. If an interface is deleted, the wheels may fall off. Type: fix Ticket: VPP-1824 Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I9b07d43f8501caa5519e5ff9ae4c19dc2661cc84
2019-12-17misc: address sanitizer: fix instrumentationBenoît Ganne1-0/+8
Type: fix Change-Id: I99e3951f8cfb7ab9d2f0a7dcee92199eab29043c Signed-off-by: Benoît Ganne <bganne@cisco.com>
2019-12-12vcl: fix disconnect from binary apiFlorin Coras1-1/+1
Type: fix Change-Id: I4398d26879b5efd932fa1d9ae232aa918ec736d6 Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-12-11vcl: separate binary api connections per thread workerFlorin Coras1-4/+27
Type: fix Change-Id: I2d72efc74a3b0a5b9e4da265475b1b01bf361125 Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-12-11api: Use vl_msg_push/pop_heapNathan Skrzypczak3-55/+26
Type: refactor Change-Id: I0eb46676fc22ce6825b2d879498df344b5a855e8 Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com>
2019-12-10api: pass api and client main to rx threadFlorin Coras2-1/+28
Type: fix Change-Id: Ib8313e87a89c80045edd897924917a88b98d1937 Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-12-10api: multiple connections per processDave Barach8-88/+105
Type: feature Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I2272521d6e69edcd385ef684af6dd4eea5eaa953
2019-12-09api: fix sock reg passing on read eventFlorin Coras3-26/+24
Type: fix Change-Id: I383242e04a114b69fe247d912842be3560e96c10 Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-12-07vlib: use explicit types in apiOle Troan1-1/+1
Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: Ib3a5d9fc36692553eb109976e9365dc7d82911ab
2019-12-06api: avoid swapping vlib_rp before barrier syncFlorin Coras3-61/+46
Type: fix Change-Id: I9868d13e827c6f5aa5535a38f629efb62ff12dbc Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com> Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-12-06api: fix free socket process argsFlorin Coras1-0/+1
Type: fix Change-Id: I910be067de6ed65790d25cc95a3d8b5b66680567 Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-12-06api: multiple socket connections per single processFlorin Coras2-31/+164
Type: feature Change-Id: Idea1857eb2225881d2982a7aa2ae7a79536b3f33 Signed-off-by: Florin Coras <fcoras@cisco.com>
2019-11-27misc: add address sanitizer heap instrumentationBenoît Ganne3-5/+27
Introduce AddressSanitizer support: https://github.com/google/sanitizers/ This starts with heap instrumentation. vlib_buffer, bihash and stack instrumentation should follow. Type: feature Change-Id: I7f20e235b2f79db72efd0e756f22c75f717a9884 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2019-11-18api: configurable binary api client rx pthread fcnDave Barach2-19/+39
Add vl_client_connect_to_vlib_thread_fn (...) and export the memory_client_main_t definition. If you use this new API, make sure not to miss the setjmp / longjmp dance shown in .../src/vlibmemory/memory_client.c:rx_thread_fn(...), which is required for the rx pthread to terminate cleanly; please process client delete reply messages in the rx thread... Type: refactor Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: Ia67038ec59e2e14b174c67afca15b2c3f67a4d20
2019-11-07api: fix dead client scan heap issueNathan Skrzypczak1-0/+4
Type: fix On multiworker setup when an app client dies, the vec_reset_length call fails the assert in clib_mem_is_heap_object. Same thing might happen for the clib_warnings Change-Id: I369f9d2dbe60407c84994a4e8d25f6df7848ca93 Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com>
2019-10-25dhcp: fix crash on unicast renewal sendNeale Ranns1-0/+1
Type: fix - when the addresses were learnt a copy of the client was sent to the main thread, this meant the unicast adjacecny was saved on the copy not on the original. - Add logging. - Improve the proxy-node that hands the clint packets so the DHCP packets are traced. - allow a renewal to configure new address data Change-Id: I6ab0afcccbc4a1cdefdd1b8beeda8fc7ba20ec1f Signed-off-by: Neale Ranns <nranns@cisco.com>
2019-10-09api: comment, simplify and fix api socket readVratko Polak1-48/+80
The function vl_socket_read_ready did contain some comments already, but as they stated, the logic has to be tricky to cover multiple cases. Comment: + Add function-level comment + Add comments to describe some of local variables + Add many comments to describe internal state at particular lines. Simplify: + Remov mbp_set as it is never needed. + Replace msg_len with msgbuf_len to save "+ sizeof (msgbuf_t)". Improve: + Early exit on EAGAIN. Fix: + "n" now only tracks input_buffer. Previously, it was entering the detection of additional messages even for unprocessed_input. + Set up msg_buffer (including appending to unprocessed_input) outside full-message-detection loop now, so it cannot be executed multiple times as before. Type: fix Ticket: VPP-1785 Change-Id: I256e34b435be06844458744a13ea37a0e86a96f9 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-10-08api: add bapi thread handle to api main structure.IJsbrand Wijnands1-1/+9
Adding the thread handle to the api main structure allows the client process of the bin api to manage the thread, like setting the thread name for example. Type: feature Change-Id: I38b58ddc83d5958c4bda76eadd371eee1545724b Signed-off-by: IJsbrand Wijnands <ice@cisco.com>
2019-10-02vlib: improve summary vector-rate statisticsDave Barach1-1/+1
Type: refactor Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I4b77879b0a84fdec3c1518a972cf003d5135222d Signed-off-by: Ole Troan <ot@cisco.com>
2019-09-30api: fix use-after-freeBenoît Ganne1-3/+3
Type: fix Change-Id: I2a599c22d5770587fdd419e5d212c8527054d2e9 Signed-off-by: Benoît Ganne <bganne@cisco.com>