aboutsummaryrefslogtreecommitdiffstats
path: root/src/vlib/buffer.c
AgeCommit message (Collapse)AuthorFilesLines
2024-02-12buffers: bring back cache occupancy improvementVratko Polak1-0/+4
The improvement was removed in 40129, causing 5-40% regressions in AVF tests. There is a memory-speed trade-off, this change prefers speed over memory efficiency. Ideally, the choice should be configurable, but that is not easy to achieve, considering how early is vlib_buffer_main_init called. Type: fix Fixes: 038dad7ef29b0b724071edb5f8cc7a9845584454 Change-Id: I4746f3634abe6d233c9d092a372de05b3d1ae4b6 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2024-01-20buffers: add compile-time option to set buffer alignmentDamjan Marion1-5/+1
Type: improvement Change-Id: I88c4c45bed0bdd8686e17e4f77a7d32a08c995aa Signed-off-by: Damjan Marion <damarion@cisco.com>
2023-10-25buffers: buffer name is vector, not c stringDamjan Marion1-6/+7
Type: fix Fixes: 8973b07 Change-Id: Iba6e29bab35be0148b469965c3af8391b5ab9140 Signed-off-by: Damjan Marion <damarion@cisco.com>
2023-10-17buffers: introduce vlib_buffer_template_tDamjan Marion1-4/+1
Type: improvement Change-Id: Ie86a5edf2ada21355543e9a0382052b16ff86927 Signed-off-by: Damjan Marion <damarion@cisco.com>
2023-10-09buffers: buffer allocation improvementsDamjan Marion1-36/+44
- pass buffer pool name trough va - make buffers naturaly aligned - fix calculation of total number of buffers Type: improvement Change-Id: I6aebf249ebd67823b4632ac08905bfa3aa7d1ee5 Signed-off-by: Damjan Marion <damarion@cisco.com>
2022-06-01vlib: add VLIB_NUM_WORKERS_CHANGE_FN() handlerDamjan Marion1-10/+4
Allows features to update their data structures after change in number of worker threads. Type: improvement Change-Id: Icd4d197e28608f5bbb1edd13eb624cd98e33cafe Signed-off-by: Damjan Marion <damarion@cisco.com>
2022-03-09stats: refactorDamjan Marion1-34/+25
Type: refactor Change-Id: Ifd533a095d979dc55bfbe5fac7e0b7510a4d900c Signed-off-by: Damjan Marion <damarion@cisco.com>
2022-01-11vlib: fix the total len not including first buffer formatingMohsin Kazmi1-1/+1
Type: fix total len not including first buffer is in vlib_buffer_t second cacheline. It is not reset after the buffer has been consumed. It leads to printing garbage in packet trace for subsequent use of this buffer. This patch fixes the issue to only print when VLIB_BUFFER_NEXT_PRESENT flag is set. Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com> Change-Id: Ied72308bdb907a5e1ca16d181f2add062807e968
2021-08-27bufmon: add buffer monitoring pluginBenoît Ganne1-7/+27
This plugin allow to keep track of buffer usage in VPP graph nodes. The main use is to detect buffer leakages. Type: feature Change-Id: Iadcf4ab98207fab6e2fa375060879bc2a25b711e Signed-off-by: Benoît Ganne <bganne@cisco.com>
2021-07-02vlib: fix buffer pool alignment sizeTianyu Li1-1/+1
Alignment size should be CLIB_CACHE_LINE_BYTES(64) instead of CLIB_LOG2_CACHE_LINE_BYTES(6) Type: fix Signed-off-by: Tianyu Li <tianyu.li@arm.com> Change-Id: If2d5ae324093be64454377866297f5e76ccddc93
2021-03-26vlib: split vlib_main_t into global and per-threadDamjan Marion1-23/+0
Type: refactor Change-Id: I8b273bc3bf16aa360f031f1b2692f766e5fc4613 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-03-26vlib: introduce vlib_get_main_by_index(), vlib_get_n_threads()Damjan Marion1-2/+2
Type: improvement Change-Id: If3da7d4338470912f37ff1794620418d928fb77f Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-12-14misc: refactor clib_bitmap_foreach macroDamjan Marion1-4/+4
Type: refactor Change-Id: I077110e1a422722e20aa546a6f3224c06ab0cde5 Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-11-24buffers: add page-size configNathan Skrzypczak1-41/+63
Type: feature Add a `buffers {page-size}` parameter to specify page size for buffers. This also fixes an issue with the parsing in unformat_log2_page_size. Change-Id: I7d7b1fa0bb7febaa7509cf2c625882f07eeafaad Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com>
2020-09-04buffers: improve cache occupancyDamjan Marion1-2/+8
Adjust buffer allocation so it always have odd number of cache lines. That should result in better distribution of cachelines among cache sets. Type: improvement Change-Id: I0d39d4cf01cff36ad6f70a700730823a96448c22 Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-06-15buffers: fix non-default sized buffers initializationBenoît Ganne1-5/+21
Type: fix Change-Id: I4a93e1d9936414c514cb237a22624986b3ef5b3d Signed-off-by: Benoît Ganne <bganne@cisco.com>
2020-04-13buffers: configurable buffer fault injectorDave Barach1-0/+26
When configured at compile time via the cmake VPP_BUFFER_FAULT_INJECTOR option, the buffer allocator will appear to fail a certain fraction of the time. By default, the allocator succeeds 80% of the time. Detailed command line configuration options are available, but only when the image has been compiled with cmake option described above: vlib { buffer-alloc-success-rate [0.0 ... 1.0] buffer-alloc-success-seed <nnnn> } Modify vlib_buffer_pool_create(...) so 0 is always an invalid buffer index. Debug images: add checks for bad buffer index enqueues, and also verify that f->n_vectors doesn't accidentally map one or more instances of the frame poison pattern 0xfefefefe. Type: improvement Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: Iab939858014463d1e664682805013d334d6fcbe5
2019-10-31vlib: only dump 1st buffer in chain by defaultBenoît Ganne1-2/+13
Several nodes include buffers in their traces, but only the 1st. When formatting the trace we must not try to iterate through all chained buffers. Default to display only the 1st buffer. Type: fix Change-Id: Ib3c668bbf4ab70ae68eba2ac402c7b7329825b70 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2019-10-28vlib: don't use vector for keeping buffer indices in the poolDamjan Marion1-11/+11
Type: refactor Change-Id: I72221b97d7e0bf5c93e20bbda4473ca67bfcdeb4 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-08-19vlib: fix vlib_buffer_main_init_numa_node memory leak.Guanghua Zhang1-3/+8
Type: fix Signed-off-by: Guanghua Zhang <ghzhang@fiberhome.com> Change-Id: I8252ed2555f5af6db2f12dc7c30e41cc1ec7dde0
2019-06-24vlib: packet tracer support for pkt thread handoffsDave Barach1-1/+1
Type: feature Change-Id: Ia3d9a47679202c2a47cd3746b50e86c6b8627ef6 Signed-off-by: Dave Barach <dave@barachs.net>
2019-03-28Typos. A bunch of typos I've been collecting.Paul Vinciguerra1-1/+1
Change-Id: I53ab8d17914e6563110354e4052109ac02bf8f3b Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-03-06vlib: coverity woesSteven Luong1-2/+1
Remove logically dead code to keep coverity from complaining Change-Id: If27d6684d19ab3c8886732a67922c86e5f0b3554 Signed-off-by: Steven Luong <sluong@cisco.com>
2019-03-04buffers: support cases where numa node memory is not availableDamjan Marion1-37/+77
Change-Id: I550ef893e41d86310d4e5af16d6100b8e557b68a Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-03-04stats: fix crash on early initFilip Tehlar1-4/+18
Change-Id: I911b5070f544c3556ad1ca584d42689c00cc38f3 Signed-off-by: Filip Tehlar <ftehlar@cisco.com>
2019-02-26stats: buffer gauge callbacks use index instead of name.Ole Troan1-26/+20
Change-Id: I500b8e72d7e021b9564dad0ffc087cb0e1ceb96f Signed-off-by: Ole Troan <ot@cisco.com>
2019-02-22stats: add buffer gaugesFilip Tehlar1-1/+87
Change-Id: I7f7a459f25d64ea5fa36e30d7dccc667bc19c5a9 Signed-off-by: Filip Tehlar <ftehlar@cisco.com>
2019-02-09buffers: fix typoDamjan Marion1-6/+8
Change-Id: I4e836244409c98739a13092ee252542a2c5fe259 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-02-06buffers: make buffer data size configurable from startup configDamjan Marion1-16/+25
Example: buffers { default data-size 1536 } Change-Id: I5b4436850ca18025c9fdcfc7ed648c2c2732d660 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-02-06buffers: improve buffer init performanceDamjan Marion1-17/+14
Change-Id: Ib59a3c32af754a898ade17c42e60a88f48b797ff Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-31assert on double-free in debug buildKlement Sekera1-4/+2
The assert allows easier debugging in gdb by aborting instead of exiting cleanly. Also a core is generated so a stack trace can be obtained. Change-Id: I2d05488c4522f4d4570fdfe0283130eb4c853d2a Signed-off-by: Klement Sekera <ksekera@cisco.com>
2019-01-31buffers: reinitialize metadata, add additional validationDamjan Marion1-0/+1
- DPDK overwrites metadata as part of rte_pktmbuf_init(...) so we need reinitialize it - additional checks added to ensure ref_count is never < 1 Change-Id: Ida336f81c4723e8f2e0ad4a70cb7b1ecfff978a0 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-30buffers: major cleanup and improvementsDamjan Marion1-375/+224
This patch introduces following changes: - deprecated free lists which are not used and not compatible with external buffer managers (i.e. DPDK) - introduces native support for per-numa buffer pools - significantly improves performance of buffer alloc and free Change-Id: I4a8e723ae47056717afd6cac0efe87cb731b5be7 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-20buffers: remove VLIB_BUFFER_DEFAULT_FREE_LIST macro and fl->n_data_bytesDamjan Marion1-17/+10
Change-Id: I0ba5175be077c40556f2a3ce629c5bbcd71e0a81 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-20buffers: remove vlib_buffer_delete_free_listDamjan Marion1-43/+0
Change-Id: I5fe01e918b534d1ac1d47f0d7610536f45c3158c Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-20buffers: don't init metadata, as it is already initializedDamjan Marion1-1/+0
Change-Id: Ia083050389853c25b069f0f8286d50d3f4aef527 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-20buffers: keep buffer_main in vlib_main_tDamjan Marion1-16/+17
Change-Id: I3bb1d9f83dd08f4b93acd4a281bfec0674e39c2e Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-19buffers: remove free-list information from buffer metadataDamjan Marion1-14/+7
Change-Id: I6048c6a51efa826ac333f7d15919cb87dd766d74 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-18buffers: remove freelist argument from vlib_buffer_add_dataDamjan Marion1-9/+5
Change-Id: Idb224b807373128b78dc46f177b2d2e0ed288349 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-07Handle buffer alloc failure in vlib_buffer_add_dataDave Barach1-6/+7
It's not OK to crash due to a transient buffer allocation failure. Return 1 if the requested operation failed, otherwise 0. Buffer index parameter change to a value-result, so the caller can differentiate between partial and complete allocation failure: callers which request an initial allocation (inbound bi = ~0) need to check the (out) value to decide whether or not to call vlib_buffer_free(...). Change-Id: I03029d7f2714c17dca4630dfd95a1eb578b68384 Signed-off-by: Dave Barach <dave@barachs.net>
2019-01-01buffers: remove unused codeDamjan Marion1-63/+1
Change-Id: If2bbfbc52994f5de0879763e0b7a7864498debb6 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-11-25vlib:init free list buffers vecEyal Bari1-0/+4
fixes a problem seen when dhcp proxy uses vlib_buffer_chain_linearize function which tries to vlib_buffer_alloc zero buffers - which succeeds if the buffers vector is initialized but otherwise crashes when trying to update the _vec_len in vlib_buffer_alloc_from_free_list solved by initializing the free_list buffers vec Change-Id: I1186d7aac05e54864d72f3f144e1bef8064f8efa Signed-off-by: Eyal Bari <ebari@cisco.com>
2018-11-14Remove c-11 memcpy checks from perf-critical codeDave Barach1-9/+10
Change-Id: Id4f37f5d4a03160572954a416efa1ef9b3d79ad1 Signed-off-by: Dave Barach <dave@barachs.net>
2018-11-13vlib:remove unused argumentEyal Bari1-1/+1
Change-Id: I88c3d3e516401bb1c84991515cd701c156ae19dd Signed-off-by: Eyal Bari <ebari@cisco.com>
2018-10-25pmalloc: support for 4K pagesDamjan Marion1-4/+24
Change-Id: Iecceffe06a92660976ebb58cd3cbec4be8931db0 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-10-23physmem coverity issuesDamjan Marion1-1/+1
Change-Id: Ie9ff9b751190632dfc4576e5cbb1987a4142af5e Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-10-23vlib: alloc buffers on local numa, not on numa 1Damjan Marion1-1/+2
Change-Id: Icb8172238f735fd0825e474e16a006f1435e175c Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-10-23vlib: reduce default buffer memory size to be below RLIMIT_MEMLOCKDamjan Marion1-1/+5
Change-Id: If36be24fa08e21c7718a1dced506a7f254dfb5cf Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-10-23Numa-aware, growable physical memory allocator (pmalloc)Damjan Marion1-54/+21
Change-Id: Ic4c46bc733afae8bf0d8146623ed15633928de30 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-10-23c11 safe string handling supportDave Barach1-4/+4
Change-Id: Ied34720ca5a6e6e717eea4e86003e854031b6eab Signed-off-by: Dave Barach <dave@barachs.net>