aboutsummaryrefslogtreecommitdiffstats
path: root/src/vlib/threads.c
AgeCommit message (Collapse)AuthorFilesLines
2021-07-27vlib: don't ASSERT(vm) in worker thread bootstrapDave Barach1-2/+4
Otherwise, threads declared with .no_data_structure_clone=1 crash on startup. Type: fix Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I5dcb25d1b61330fc9eee5427b815fcfcb9bf2153
2021-05-27vlib: handoff reworkDamjan Marion1-27/+4
Type: improvement Change-Id: I1e199ae31e969154319e94c5cd286b8d8adc6660 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-05-14vlib: remove unused codeDamjan Marion1-121/+0
Type: refactor Change-Id: Ia5f670541dd43a4f6d731dda84b2d898195643fc Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-05-06stats: memory heap countersOle Troan1-0/+1
- Add counters for the main-heap - Add additional counters per heap: STAT_MEM_TOTAL STAT_MEM_USED, STAT_MEM_FREE, STAT_MEM_USED_MMAP, STAT_MEM_TOTAL_ALLOC, STAT_MEM_FREE_CHUNKS, STAT_MEM_RELEASABLE, The per-heap counters are organised as a two dimensional vector. total, used and free are directly available via symlinks. vpp_get_stats ls "^/mem/" /mem/stat segment /mem/stat segment/total /mem/stat segment/used /mem/stat segment/free /mem/main heap /mem/main heap/total /mem/main heap/used /mem/main heap/free vpp_get_stats dump "^/mem/main\ heap$" [0 @ 0]: 1073741776 packets /mem/main heap [1 @ 0]: 91586688 packets /mem/main heap [2 @ 0]: 982155088 packets /mem/main heap [3 @ 0]: 0 packets /mem/main heap [4 @ 0]: 1073741776 packets /mem/main heap [5 @ 0]: 433 packets /mem/main heap [6 @ 0]: 981708688 packets /mem/main heap Type: feature Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I36725dde3b4b3befd27a8b4d3ba931f2d3b627cc
2021-05-05vlib: fix adaptive mode flag unset on nodes reforkingMaxime Peim1-0/+4
The adaptive mode flag is being unset on nodes reforking. The flag is copied in order to avoid this issue. Old flags are copied for all nodes. Type: fix Signed-off-by: Maxime Peim <mpeim@cisco.com> Change-Id: Ic29b20fc0e4454ce4b3765c478f55bf650b4572a
2021-05-03vlib: multiarch vlib_frame_queue_dequeue()Damjan Marion1-139/+0
Fact that it needs to copy buffer indices justifies this move. Type: improvement Change-Id: I5eb815ccc4cca0ef70b092eb83a49b713efdcbeb Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-04-20vlib: remove unused codeDamjan Marion1-30/+0
Type: refactor Change-Id: I01dba12a7f8aa2fa6d2e4113c91dc97e638aca77 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-04-19vppinfra: momve clib_sysfs_list_to_bitmap to sysfs.cDamjan Marion1-27/+0
Type: refactor Change-Id: Ia5a7ebd5c8492326f5bf2f1264f1c5ef38ccb4dc Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-03-26vlib: split vlib_main_t into global and per-threadDamjan Marion1-28/+35
Type: refactor Change-Id: I8b273bc3bf16aa360f031f1b2692f766e5fc4613 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-03-26vlib: introduce vlib_get_elog_main()Damjan Marion1-3/+3
Type: improvement Change-Id: I73383eb15186021cd6527d112da8443a0082f129 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-03-26vlib: introduce vlib_get_main_by_index(), vlib_get_n_threads()Damjan Marion1-26/+29
Type: improvement Change-Id: If3da7d4338470912f37ff1794620418d928fb77f Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-03-22vlib: graphviz upgrade to allow filtersArthur de Kerhor1-5/+0
Possibility to draw only the active nodes on the graph. These are scaled and colored according to their utilization. Type: improvement Signed-off-by: Arthur de Kerhor <arthurdekerhor@gmail.com> Change-Id: I7ddb7b62b3a141cb03750dca24f044138fcc577f
2021-03-11misc: fix gcc-11 buildDamjan Marion1-28/+20
Type: fix Change-Id: Ia17c8255806a2575bb75ed37050b47ddb347050c Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-01-21interface: rx queue infra rework, part oneDamjan Marion1-0/+7
Type: improvement Change-Id: I4008cadfd5141f921afbdc09a3ebcd1dcf88eb29 Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-12-14misc: refactor clib_bitmap_foreach macroDamjan Marion1-6/+6
Type: refactor Change-Id: I077110e1a422722e20aa546a6f3224c06ab0cde5 Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-10-06vppinfra: add heap header in front of dlmalloc spaceDamjan Marion1-1/+1
This patch adds smal header in front of dlmalloc space, and it stores some additional information about the heap. Immediate benefit of this patch is that we know the underlying page size si we can display heap page statistics / real memory usage. Type: improvement Change-Id: Ibd6989cc2f2f64630ab08734c9552e15029c5f3f Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-10-06misc: add xxx_not_inlineDave Barach1-0/+6
vlib_main_t *vlib_get_main_not_inline(void) vlib_thread_main_t *vlib_get_thread_main_not_inline(void) elog_main_t *vlib_get_elog_main_not_inline() Type: refactor Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I6de306d567283ad28ef34c9be0cf27452aecbf6c
2020-10-02vlib: always replace bootstrap vlib_mainsDave Barach1-8/+7
Fixes an obscure issue when spinning up pure pthreads which don't have vlib_main_t clones. VLIB_REGISTER_THREAD () = { <snip> .use_pthreads = 1, .no_data_structure_clone = 1, } Type: improvement Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: Ifae5a2a35865de057eea7782ee71b1011a21c88e
2020-10-02vppinfra: use clib_mem_create_heap() to create numa heap(s)Damjan Marion1-2/+5
Type: improvement Change-Id: Ie04302c576869bc7bfaa9f13ed2ea8a403a393d4 Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-09-28vppinfra: don't call dlmalloc API directly from the codeDamjan Marion1-4/+8
- it is confusing from end consumer perspective that some thing is somewhere called heap and somewhere mspace - this is base for additional work where heap pointer is not the same thing like mspace Type: improvement Change-Id: I644d5a0de17690d65d164d8cec3c5654571629ef Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-09-16vppinfra: introduce clib_mem_mainDamjan Marion1-3/+4
To hold more data later... Type: improvement Change-Id: I4006d22dcacd788988c4907f2c263fd4e4a9d398 Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-09-02fib: IPv6 lookup data structure MP safe when prefixes changeNeale Ranns1-0/+47
Type: fix adding routes should be MP safe. When new prefixes with differrent prefix lengths are added, adjust the sorted list in an MP safe way. Change-Id: Ib73a3c84d01eb86d17f8e79ea2bd2505dd9afb3d Signed-off-by: Neale Ranns <nranns@cisco.com>
2020-07-16misc: add callback hooks and refactor pmcTom Seidenberg1-0/+16
Callbacks for monitoring and performance measurement: - Add new callback list type, with context - Add callbacks for API, CLI, and barrier sync - Modify node dispatch callback to pass plugin-specific context - Modify perfmon plugin to keep PMC samples local to the plugin - Include process nodes in dispatch callback - Pass dispatch function return value to callback Type: refactor Signed-off-by: Tom Seidenberg <tseidenb@cisco.com> Change-Id: I28b06c58490611e08d76ff5b01b2347ba2109b22
2020-06-16vlib: address sanitizer support for stack switch, enable clangDamjan Marion1-0/+3
Type: improvement Change-Id: I81df4b61d1f0b8c1df77c1ee9bebcb491e155b69 Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-04-27vlib: deprecate i2c and cjDave Barach1-4/+0
i2c follows its only use case - the original 82599 driver - into extras/deprecated. cj is/was an emergency debug tool unused in several years. Move to extras/deprecated/vlib Type: refactor Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: Ib55b65373f62630db295c562974bd8f2456c3107
2020-04-21vlib: queue_hi_thresh fix to avoid deadlockElias Rudberg1-2/+4
Adapt queue_hi_thresh value using num_threads to avoid risk of deadlock between threads which could happen for example when different NAT threads try to handoff work to each other at the same time when their frame queues are congested. This change ensures that each thread can reserve a queue entry without causing problems even in the most extreme case when all threads attempt to add to the same queue simultaneously when the queue is nearly full. Type: fix Signed-off-by: Elias Rudberg <elias.rudberg@bahnhof.net> Change-Id: I9e02f753bd00833d8dd500d181b0d4f9a454d703
2020-03-21vlib: allow using core 0 as a last resortVladimir Isaev1-15/+12
Right now even following configuration fails: cpu { corelist-workers 0 } I think we should allow user to use CPU 0 if he explicitly configured it or as the last automatically assigned core. Type: improvement Signed-off-by: Vladimir Isaev <visaev@netgate.com> Change-Id: I041c30349bace558c673475e9a0cffccae5821f7
2020-03-21vlib: complain if workers are configured twiceVladimir Isaev1-1/+10
Right now following configuration leads to crash: cpu { corelist-workers 2 workers 2 } because threads count will be set to 2, but we have only one core in coremask. Type: fix Signed-off-by: Vladimir Isaev <visaev@netgate.com> Change-Id: Ia93b892733971e7c8ddfceaaec5f4eb8bf9063ac
2020-03-12vppinfra: refactor clib_timebase_tDave Barach1-1/+8
Add a clib_time_t * argument to clib_timebase_init(...), to encourage client code to share the vlib_main_t's clib_time_t object. Display the current day / date in GMT via the "show time" debug CLI. Fix the test framework so it processes the new "show time" output format. Type: refactor Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I5e52d57eb164b7cdb6355362d520df6928491711
2020-02-17vlib: fix code of getting numa node with specific cpu_idLijian.Zhang1-3/+18
Use below sysfs files to check which numa node a specific cpu_id belongs to. /sys/devices/system/node/online /sys/devices/system/node/node0/cpulist /sys/devices/system/node/node1/cpulist Type: fix Change-Id: I124b80b1fd4a20dd7bd76f0ae27d5ab23a3a8ff1 Signed-off-by: Lijian Zhang <Lijian.Zhang@arm.com>
2020-02-11vppinfra: remove the historical mheap memory allocatorDave Barach1-14/+2
The mheap allocator has been turned off for several releases. This commit removes the cmake config parameter, parallel support for dlmalloc and mheap, and the mheap allocator itself. Type: refactor Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I104f88a1f06e47e90e5f7fb3e11cd1ca66467903
2020-02-11vlib: Skip core 0 if workers are configured.Jon Loeliger1-0/+15
According to the description in the startup.conf, the assignment of worker threads starts with the lcore following the main_lcore. A non-zero skip_cores will correctly achieve this assignment. However, prior to this patch when workers are assigned, the code picks up and assigns core 0 even thought it shouldn't. This patch determins if a non-zero number of workers are desired and if so, marks CPU unavailable for a worker assignment. Type: fix Change-Id: I1fdf73a6f218dcbf146fda2efc90c553f7cd6d20 Signed-off-by: Jon Loeliger <jdl@netgate.com>
2020-02-10vppinfra: use vm memory allocator for numa mappingFlorin Coras1-5/+3
Type: refactor Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I13b239cd572ae6dfaec07019d3d9b7c0ed3edcfa
2020-02-10vlib: properly configure main thread's core idJon Loeliger1-0/+2
Prior to this patch, the main thread's core id was always reported as 0. This patch makes a call to vlib_get_thread_core_socket() on the main thread so that its core id is set properly. Type: fix Fixes: 5d64c7868f67749a6c99eb4ee5998b518ab6c71c Change-Id: I019cb95eec031da25197e48d956038c4bd6b5040 Signed-off-by: Jon Loeliger <jdl@netgate.com>
2020-02-05vppinfra: numa vector placement supportDave Barach1-6/+29
Type: feature Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I7e7d95a089dd849c1f01ecea84529d8dbf239f21
2020-01-30vppinfra: improve clocks_per_second convergenceDave Barach1-3/+7
Apply exponential smoothing to the clock rate update calculation in clib_time_verify_frequency(), with a half-life of 1 minute and a sampling frequency of 16 seconds. Within 5 minutes or so, the calculation converges With each rate recalculation: reset total_cpu_time based on the kernel timebase delta since vpp started, and the new clock rate Improve the "show clock [verbose]" debug CLI command. BFD echo + echo fail tests marked off until the BFD code can be reworked a bit. Type: fix Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I24e88a78819b12867736c875067b386ef6115c5c
2019-10-02vlib: improve summary vector-rate statisticsDave Barach1-1/+1
Type: refactor Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I4b77879b0a84fdec3c1518a972cf003d5135222d Signed-off-by: Ole Troan <ot@cisco.com>
2019-09-30vlib: fix old nodes vector overflow in reforkBenoît Ganne1-5/+2
Type: fix Change-Id: I81c4cf0ce87288bb2d3c7b9f31e9419290d588b4 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2019-07-23vlib: address vlib_error_t scaling issueDave Barach1-0/+2
Encoding the vpp node index into the vlib_error_t as a 10-bit quantity limits us to 1K graph nodes. Unfortunately, a few nodes need 6 bit per-node error codes. Only a very few nodes have so many counters. It turns out that there are about 2K total error counters in the system, which is (approximately) the maximum error heap index. The current (index,code) encoding limits the number of interfaces to around 250, since each interface has two associated graph nodes and we have about 500 "normal, interior" graph node This patch adds an error-index to node-index map, so we can store error heap indices directly in the vlib_buffer_t. Type: refactor Change-Id: I28101cad3d8750819e27b8785fc0cf71ff54f79a Signed-off-by: Dave Barach <dave@barachs.net>
2019-07-16api: enable binary API event logging in vatDave Barach1-26/+4
Cleaned up a few instances of side-bet elog_string hash table usage. Elog_string handles that problem itself. Add cli commands to vat to initialize, enable/disable, and save an event log. Event logging at the same time in both vpp and vat yields a pair of event logs which can be merged by the "test_elog" tool. Type: refactor Change-Id: I8d6a72206f2309c967ea1630077fba31aef47f93 Signed-off-by: Dave Barach <dave@barachs.net>
2019-06-24vlib: packet tracer support for pkt thread handoffsDave Barach1-0/+6
Type: feature Change-Id: Ia3d9a47679202c2a47cd3746b50e86c6b8627ef6 Signed-off-by: Dave Barach <dave@barachs.net>
2019-06-08trace frame-queue on trigger out of memorydongjuan1-2/+1
FRAME_QUEUE_NELTS is 64 in thread.c Change-Id: Ie7e5962afe05dfc7f38e3d597dabc74dcc2dab8d Signed-off-by: dongjuan <dong.juan1@zte.com.cn>
2019-06-04sort worker-thread init functions in advanceDave Barach1-5/+30
Otherwise, all N worker threads try to sort the list at the same time: a good way to have a bad day. This approach performs *far* better than maintaing order by adding a spin-lock. By direct measurement w/ elog + g2: 11 threads execute the per-thread init function list in 22us, vs. 50ms with a CLIB_PAUSE() enabled spin-lock. Change-Id: I1745f2a213c0561260139a60114dcb981e0c64e5 Signed-off-by: Dave Barach <dave@barachs.net>
2019-05-22stats: support multiple works for error countersOle Troan1-13/+14
The current code only allowed access to the main thread error counters. That is not so useful for a multi worker instance. No return a vector indexed by thread of counter_t values. Type: fix Change-Id: Ie322c8889c0c8175e1116e71de04a2cf453b9ed7 Signed-off-by: Ole Troan <ot@cisco.com>
2019-05-16init / exit function orderingDave Barach1-1/+1
The vlib init function subsystem now supports a mix of procedural and formally-specified ordering constraints. We should eliminate procedural knowledge wherever possible. The following schemes are *roughly* equivalent: static clib_error_t *init_runs_first (vlib_main_t *vm) { clib_error_t *error; ... do some stuff... if ((error = vlib_call_init_function (init_runs_next))) return error; ... } VLIB_INIT_FUNCTION (init_runs_first); and static clib_error_t *init_runs_first (vlib_main_t *vm) { ... do some stuff... } VLIB_INIT_FUNCTION (init_runs_first) = { .runs_before = VLIB_INITS("init_runs_next"), }; The first form will [most likely] call "init_runs_next" on the spot. The second form means that "init_runs_first" runs before "init_runs_next," possibly much earlier in the sequence. Please DO NOT construct sets of init functions where A before B actually means A *right before* B. It's not necessary - simply combine A and B - and it leads to hugely annoying debugging exercises when trying to switch from ad-hoc procedural ordering constraints to formal ordering constraints. Change-Id: I5e4353503bf43b4acb11a45fb33c79a5ade8426c Signed-off-by: Dave Barach <dave@barachs.net>
2019-05-14Preallocate mhash key_tmps vectorDave Barach1-15/+10
Fix os_get_nthreads() so that it starts returning the correct answer as early as possible. Change-Id: Id5292262f2c3f521b07ffbe6a9f6748dcc4dcb7d Signed-off-by: Dave Barach <dave@barachs.net>
2019-04-24Clean up multi-thread barrier-sync hold-down timerDave Barach1-10/+30
Main thread: don't bother with the barrier sync hold-down timer if none of the worker threads are busy. Worker threads: avoid epoll_pwait (10ms timeout) when the control-plane has been active in the last half-second. Change-Id: I82008d09968c65e2a4af0ebb7887389992e60603 Signed-off-by: Dave Barach <dave@barachs.net>
2019-03-10Perf tune get_frame_size_infoDave Barach1-2/+4
It turns out that for scalar sizes 0..24, frames are always the same size. That range includes all current use-cases - and then some - so get rid of the hash table. Old code preserved under #ifdef VLIB_SUPPORTS_ARBITRARY_SCALAR_SIZES. Change-Id: Ic005c7143c9639f77d1a0fadd2fc0e90dccb68c1 Signed-off-by: Dave Barach <dbarach@cisco.com>
2019-02-27Fix crash in barrier sync when vlib_worker_threads is zeroDamjan Marion1-1/+2
Change-Id: I6819dd9dbfc15c17740bdb98b51bdd639ef8c4d2 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-02-20calculate per-thread time offsetDave Barach1-0/+47
The main thread squirrels away vlib_time_now (&vlib_global_main), worker threads use it to calculate an offset in f64 seconds from their own vlib_time_now(vm) value. We use that offset until the next barrier sync. Thanks to Damjan for the suggestion. Change-Id: If56cdfe68e5ad8ac3b0d0fc885dc3ba556cd1215 Signed-off-by: Dave Barach <dave@barachs.net>