summaryrefslogtreecommitdiffstats
path: root/src/vlib/main.c
AgeCommit message (Collapse)AuthorFilesLines
2019-06-04sort worker-thread init functions in advanceDave Barach1-0/+11
Otherwise, all N worker threads try to sort the list at the same time: a good way to have a bad day. This approach performs *far* better than maintaing order by adding a spin-lock. By direct measurement w/ elog + g2: 11 threads execute the per-thread init function list in 22us, vs. 50ms with a CLIB_PAUSE() enabled spin-lock. Change-Id: I1745f2a213c0561260139a60114dcb981e0c64e5 Signed-off-by: Dave Barach <dave@barachs.net>
2019-05-24Add callback multiplex supportDave Barach1-6/+5
Change-Id: Iddeb3a1b0e20706e72ec8f74dabc60b342f003ba Signed-off-by: Dave Barach <dave@barachs.net>
2019-05-01Add node, frame to vlib main loop perf analysis callback argumentsDave Barach1-5/+10
Change-Id: Iaa5cd89791b0dfdb56a75009c564581d10696d83 Signed-off-by: Dave Barach <dave@barachs.net>
2019-03-21Ignore SIGTERM during the vpp boot sequenceDave Barach1-11/+11
Call setjmp and mark the setjmp context valid just prior to entering the vpp main loop. Change-Id: I26d5cd6a624cb2a497d81eb85a62365621b3b469 Signed-off-by: Dave Barach <dave@barachs.net>
2019-03-12Perf tune handoff queue pollingDave Barach1-2/+21
Change-Id: I5cfa0f6eee67156bf87907fcf8a39f16d68a0905 Signed-off-by: Dave Barach <dave@barachs.net>
2019-03-10Perf tune get_frame_size_infoDave Barach1-1/+7
It turns out that for scalar sizes 0..24, frames are always the same size. That range includes all current use-cases - and then some - so get rid of the hash table. Old code preserved under #ifdef VLIB_SUPPORTS_ARBITRARY_SCALAR_SIZES. Change-Id: Ic005c7143c9639f77d1a0fadd2fc0e90dccb68c1 Signed-off-by: Dave Barach <dbarach@cisco.com>
2019-02-27VPP-1576: fix Coverity issuesDave Barach1-27/+17
Change-Id: I8b59b2e1c0525abf4b0492e50a7af57df4cd3ce2 Signed-off-by: Dave Barach <dave@barachs.net>
2019-02-22stats: add buffer gaugesFilip Tehlar1-3/+3
Change-Id: I7f7a459f25d64ea5fa36e30d7dccc667bc19c5a9 Signed-off-by: Filip Tehlar <ftehlar@cisco.com>
2019-02-22Add no-append flag to vlib_frame_tDamjan Marion1-4/+6
Change-Id: I01c4f5755d579282773ac227b0bc24f8ddbb2bd1 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-02-02CLI control of graph dispatch elogsDave Barach1-124/+170
Change-Id: I195c8eabc0ee67880f1e85fc7594b00be6b563e3 Signed-off-by: Dave Barach <dave@barachs.net>
2019-01-31Add 'show vlib graphviz' commandBenoît Ganne1-16/+3
Add a new command to dump vlib graph as graphviz/dot file Change-Id: I43fc072cff8153ac500e5fbc6641a3705c2e995e Signed-off-by: Benoît Ganne <bganne@cisco.com>
2019-01-30buffers: major cleanup and improvementsDamjan Marion1-6/+3
This patch introduces following changes: - deprecated free lists which are not used and not compatible with external buffer managers (i.e. DPDK) - introduces native support for per-numa buffer pools - significantly improves performance of buffer alloc and free Change-Id: I4a8e723ae47056717afd6cac0efe87cb731b5be7 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-28Less intrusive way to capture numa_node and cpu_id changesDamjan Marion1-6/+3
Change-Id: I3a33fb81f31ed473811e9e7a6197b81135913865 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-27Fix issue with cpu_id and numa_code captured too earlyDamjan Marion1-0/+6
Change-Id: I79b213b34c6071d14acf1922f89037a4a5a36c45 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-27perfmon: collect data on selected thread(s)Dave Barach1-8/+0
Add missing pre-input node runtime fork and refork code. unix-epoll-input runs on all threads; each instance needs its own runtime stats. Change-Id: I16b02e42d0c95f863161176c4bb9f9917bef809d Signed-off-by: Dave Barach <dave@barachs.net>
2019-01-24perfmon plugin: 2-way parallel stat collectionDave Barach1-38/+48
As a FUD reduction measure, this patch implements 2-way parallel counter collection. Synthetic stat component counter pairs run at the same time. Running two counters (of any kind) at the same time naturally reduces the aggregate time required by an approximate factor-of-2, depending on whether an even or odd number of stats have been requested. I don't completely buy the argument that computing synthetic stats such as instructions-per-clock will be inaccurate if component counter values are collected sequentially. Given uniform traffic pattern, it must make no difference. As the collection interval increases, the difference between serial and parallel component counter collection will approach zero, see also the Central Limit theorem. Change-Id: I36ebdcf125e8882cca8a1929ec58f17fba1ad8f1 Signed-off-by: Dave Barach <dave@barachs.net>
2019-01-23buffers: wrap vlib_buffer_t to union and expose vector typesDamjan Marion1-3/+2
Change-Id: I1c12e2941cae198ededbb65eb5be51a4eabe2c1b Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-20buffers: remove VLIB_BUFFER_DEFAULT_FREE_LIST macro and fl->n_data_bytesDamjan Marion1-2/+1
Change-Id: I0ba5175be077c40556f2a3ce629c5bbcd71e0a81 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-20buffers: keep buffer_main in vlib_main_tDamjan Marion1-1/+1
Change-Id: I3bb1d9f83dd08f4b93acd4a281bfec0674e39c2e Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-19buffers: remove free-list information from buffer metadataDamjan Marion1-1/+0
Change-Id: I6048c6a51efa826ac333f7d15919cb87dd766d74 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-09Use the official libpcap file typeDave Barach1-1/+1
Change-Id: Ia34a4278eedc8cf450688b1fa0291e1f976868d3 Signed-off-by: Dave Barach <dave@barachs.net>
2018-11-30Metadata / opaque formatting belongs in vppDave Barach1-57/+90
VPP graph dispatch trace record description: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Major Version | Minor Version | NStrings | ProtoHint | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Buffer index (big endian) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + VPP graph node name ... ... | NULL octet | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Buffer Metadata ... ... | NULL octet | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Buffer Opaque ... ... | NULL octet | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Buffer Opaque 2 ... ... | NULL octet | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | VPP ASCII packet trace (if NStrings > 4) | NULL octet | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Packet data (up to 16K) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Graph dispatch records comprise a version stamp, an indication of how many NULL-terminated strings will follow the record header, and a protocol hint. The buffer index allows downstream consumers of these data to easily filter/track single packets as they traverse the forwarding graph. FWIW, the 32-bit buffer index is stored in big endian format. As of this writing, major version = 1, minor version = 0. Nstrings will be either 4 or 5. Here is the current set of protocol hints: typedef enum { VLIB_NODE_PROTO_HINT_NONE = 0, VLIB_NODE_PROTO_HINT_ETHERNET, VLIB_NODE_PROTO_HINT_IP4, VLIB_NODE_PROTO_HINT_IP6, VLIB_NODE_PROTO_HINT_TCP, VLIB_NODE_PROTO_HINT_UDP, VLIB_NODE_N_PROTO_HINTS, } vlib_node_proto_hint_t; Example: VLIB_NODE_PROTO_HINT_IP6 means that the first octet of packet data SHOULD be 0x60, and should begin an ipv6 packet header. Change-Id: Idf310bad80cc0e4207394c80f18db5f77c378741 Signed-off-by: Dave Barach <dave@barachs.net>
2018-11-20Add buffer tracing to the dispatch tracerDave Barach1-6/+79
Change-Id: I56f25d653b71a25c70e6c5c1a93dd9c5158f2079 Signed-off-by: Dave Barach <dave@barachs.net>
2018-11-20vlib: reset frame flags when frame is reusedDamjan Marion1-0/+1
Change-Id: I8f4843e7a961a1e6c3fd057554b31ae49fc9b328 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-11-17pcap-based dispatch tracerDave Barach1-1/+305
To facilitate dispatch trajectory tracing, vlib_buffer_t decoding, etc. through Wireshark Change-Id: I31356b9fa1f40cba8830aaf10a86a9fbb7546438 Signed-off-by: Dave Barach <dave@barachs.net>
2018-11-14Remove c-11 memcpy checks from perf-critical codeDave Barach1-4/+4
Change-Id: Id4f37f5d4a03160572954a416efa1ef9b3d79ad1 Signed-off-by: Dave Barach <dave@barachs.net>
2018-11-13vlib rename vlib_frame_args(...) to vlib_frame_scalar_args(..)Damjan Marion1-0/+1
Typically we have scalar_size == 0, so it doesn't matter but vlib_frame_args was providing pointer to scalar frame data, not vector data. To avoid future confusion function is renamed to vlib_frame_scalar_args(...) Change-Id: I48b75523b46d487feea24f3f3cb10c528dde516f Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-11-08vlib: use index to free suspended frameFlorin Coras1-4/+4
Avoids crash if suspended_process_frames grows. Change-Id: Id26ef0dd0dd001b997c531c4dec004e7e7989670 Signed-off-by: Florin Coras <fcoras@cisco.com>
2018-11-01Move RPC calls off the binary API input queueDave Barach1-1/+6
Change-Id: I2476e3e916a42b41d1e66bfc1ec4f8c4264c1720 Signed-off-by: Dave Barach <dbarach@cisco.com>
2018-10-23Numa-aware, growable physical memory allocator (pmalloc)Damjan Marion1-1/+1
Change-Id: Ic4c46bc733afae8bf0d8146623ed15633928de30 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-10-23c11 safe string handling supportDave Barach1-1/+1
Change-Id: Ied34720ca5a6e6e717eea4e86003e854031b6eab Signed-off-by: Dave Barach <dave@barachs.net>
2018-10-22X86_64 perf counter pluginDave Barach1-16/+67
Change-Id: Ie5a00c15ee9536cc61afab57f6cadc1aa1972f3c Signed-off-by: Dave Barach <dave@barachs.net>
2018-09-14vlib: introduce user flags in vlib_frame_tDamjan Marion1-15/+16
Those flags have local significance and they can be used for sending hints to the next node. Change-Id: Ic2596ee81c64cd16f96344365370e8fcdc465354 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-08-14Run interior graph nodes before process nodesDave Barach1-14/+7
Reduces the chance of tripping over vectors in flight, especially in single-core cases. Change-Id: I132cdd3689f8e634f9a983af57219503817b8560 Signed-off-by: Dave Barach <dave@barachs.net>
2018-06-14Use __attribute__((weak)) references where necessaryDave Barach1-4/+17
It should be possible to use vlib without the vlibmemory library, etc. Change-Id: Ic2316b93d7dbb728fb4ff42a3ca8b0d747c9425e Signed-off-by: Dave Barach <dave@barachs.net>
2018-06-08export counters in a memfd segmentDave Barach1-0/+24
also export per-node error counters directory entries implement object types Change-Id: I8ce8e0a754e1be9de895c44ed9be6533b4ecef0f Signed-off-by: Dave Barach <dave@barachs.net>
2018-05-18session: add session process nodeFlorin Coras1-1/+3
Add a session process node that handles main thread tx and retransmit in order to avoid having a polling input node. Change-Id: I3357e987c023a84b533b32793e37ab4204420f64 Signed-off-by: Florin Coras <fcoras@cisco.com>
2018-03-14vlib: internal buffer manager reworkDamjan Marion1-4/+3
- buffer_main is no longer part of vlib_main_t - pool of free lists is still part of vlib_main_t - mheap is not used anymore for buffer allocation - simple bitmap bassed buffer alloc scheme is introduced Change-Id: I3e1e6d00e2c8122293ed0a741245eb841315a1ff Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-02-06vlib: epoll on worker threadsDamjan Marion1-7/+6
This patch teaches worer threads to sleep and to be waken up by kernel if there is activity on file desctiptors assigned to that thread. It also adds counters to epoll file descriptors and new debug cli 'show unix file'. Change-Id: Iaf67869f4aa88ff5b0a08982e1c08474013107c4 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-01-19Fix MP hazard in interrupt node schedulingDave Barach1-2/+8
Change-Id: Iff14ed6ffd822eb2286aac6af467d9c8660e3d81 Signed-off-by: Dave Barach <dave@barachs.net>
2017-12-15VPP-1102: fix dangling references in RPC handlingDave Barach1-0/+13
Queue RPC calls and send them from the main dispatch loop. As things stood, if the vpp main input queue filled, worker threads could enter a barrier-sync spin-wait in the middle of processing a frame. If thread 0 decided to recreate worker thread data structures, the worker thread(s) could easily crash. Legislate the problem out of existence by enqueueing RPC messages only from the main dispatch loop. At that point, doing a barrier-sync wait is perfectly OK. Change-Id: I18da3e44bb1f29a63fe5f30cf11de732ecfd5bf7 Signed-off-by: Dave Barach <dave@barachs.net>
2017-10-19VPP-1024: rewrite buffer trajectory tracerDave Barach1-41/+18
Use a proper u16 * vector to capture node indices, since vpp w/ plugins now exceeds 255 graph nodes Change-Id: Ic48cad676fa3a6116413ddf08c083dd9660783f1 Signed-off-by: Dave Barach <dave@barachs.net>
2017-09-07vlib physmem reworkDamjan Marion1-13/+7
This patch adds supprot support for multiple numa-aware physmem regions. Change-Id: I5c69a6f4da33c8ee21bdb8604d52fd2886f2327e Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-08-25TCP horizontal scalingDave Barach1-8/+1
- Remove frame handoff support machinery. We haven't used it in a long time. - Configuration support for the local endpoints bihash table - Drop lookup failure packets in tcp46_syn_sent Change-Id: Icd51e6785f74661c741e76fac23d21c4cc998d17 Signed-off-by: Dave Barach <dave@barachs.net>
2017-07-10vlib: store buffer memory information in the buffer_mainDamjan Marion1-2/+18
Currently, buffer index is calculated as a offset to the physmem region shifted by log2_cacheline size. When DPDK is used we "hack" physmem data with information taken from dpdk mempool. This makes physmem code not usable with DPDK. This change makes buffer memory start and size independent of physmem basically allowing physmem to be used when DPDK plugin is loaded. Change-Id: Ieb399d398f147583b9baab467152a352d58c9c31 Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-06-28switch vlib process model to tw_timer_template timer implDave Barach1-23/+39
Change-Id: I36bb47faea55a6fea7af7ee58d87d8f6dd28f93d Signed-off-by: Dave Barach <dave@barachs.net>
2017-06-07VPP-873: fix vector expansion bug in dispatch_pending_nodeDave Barach1-13/+27
The main interior graph-node dispatch loop had a longstanding dangling vector element reference: for (i = 0; i < _vec_len (nm->pending_frames); i++) cpu_time_now = dispatch_pending_node (vm, nm->pending_frames + i, cpu_time_now); Passing a pointer to a vector element (nm->pending_frames + i) has considerable comedic potential if there's any chance that the vector could expand. dispatch_pending_node() calls dispatch_node(), and indirectly any interior graph node dispatch function. If that node happens to expand nm->pending_frames by filling in a new frame, nm->pending_frames can expand. After calling the node dispatch function, dispatch_node() does the following: nf = vec_elt_at_index (nm->next_frames, p->next_frame_index); If nm->pending_frames expands during dispatch function execution, p is a dangling reference to freed memory. By luck, the TCP stack managed to allocate a fresh frame which included "old-p," which caused p->next_frame_index to be filled with the new-frame poison pattern 0xfefefefe. This has been broken from day 1, summer 2007, first use of the third-generation vector processing library. Change-Id: Ideb6363bb060c4e8bf9b901882c318bd83853121 Signed-off-by: Dave Barach <dave@barachs.net>
2017-05-22vhost: migrate to use device infra for worker thread assignment, rx-mode.Steven1-10/+1
and add adaptive mode support to receive queue - Migrate vhost to use device infra which does the interface/queue to worker thread assignment. - Retire vhost thread CLI and corresponding code which assigns interface/queue to worker thread. set interface placement should be used instead to customize the interface/queue to worker thread assignment. - Retire vhost interrupt/polling option when creating vhost-user interface. Instead, set interface rx-mode should be used. - Add code in vnet_device_input_unassign_thread to change the node state to interrupt if the last polling interface has left the worker thread for the device of the corresponding interface/queue. - Add adaptive mode support. The node state is set to interrupt initially. When the scheduler detects a burst of traffic, it switches the input node to polling. Then we inform the device that we don't need interrupt notification. When the traffic subsides, the scheduler switches the input node back to interrupt. Then we immediately tell the driver that we want interrupt notification again. - Remove some duplicate code in vlib/main.c Change-Id: Id19bb1b9e50e6521c6464f470f5825c26924d3a8 Signed-off-by: Steven <sluong@cisco.com>
2017-05-16VPP-845: add configurable elog post-mortem dumpDave Barach1-0/+20
Off by default. Enable via cmdline "... vlib { elog-post-mortem-dump } ..." Change-Id: I2056b9de9b37475f2bfeeb5404da838f1b42645a Signed-off-by: Dave Barach <dave@barachs.net>
2017-04-06Use thread local storage for thread indexDamjan Marion1-7/+7
This patch deprecates stack-based thread identification, Also removes requirement that thread stacks are adjacent. Finally, possibly annoying for some folks, it renames all occurences of cpu_index and cpu_number with thread index. Using word "cpu" is misleading here as thread can be migrated ti different CPU, and also it is not related to linux cpu index. Change-Id: I68cdaf661e701d2336fc953dcb9978d10a70f7c1 Signed-off-by: Damjan Marion <damarion@cisco.com>