summaryrefslogtreecommitdiffstats
path: root/src/vlib/buffer.c
AgeCommit message (Collapse)AuthorFilesLines
2021-07-02vlib: fix buffer pool alignment sizeTianyu Li1-1/+1
Alignment size should be CLIB_CACHE_LINE_BYTES(64) instead of CLIB_LOG2_CACHE_LINE_BYTES(6) Type: fix Signed-off-by: Tianyu Li <tianyu.li@arm.com> Change-Id: If2d5ae324093be64454377866297f5e76ccddc93
2021-03-26vlib: split vlib_main_t into global and per-threadDamjan Marion1-23/+0
Type: refactor Change-Id: I8b273bc3bf16aa360f031f1b2692f766e5fc4613 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-03-26vlib: introduce vlib_get_main_by_index(), vlib_get_n_threads()Damjan Marion1-2/+2
Type: improvement Change-Id: If3da7d4338470912f37ff1794620418d928fb77f Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-12-14misc: refactor clib_bitmap_foreach macroDamjan Marion1-4/+4
Type: refactor Change-Id: I077110e1a422722e20aa546a6f3224c06ab0cde5 Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-11-24buffers: add page-size configNathan Skrzypczak1-41/+63
Type: feature Add a `buffers {page-size}` parameter to specify page size for buffers. This also fixes an issue with the parsing in unformat_log2_page_size. Change-Id: I7d7b1fa0bb7febaa7509cf2c625882f07eeafaad Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com>
2020-09-04buffers: improve cache occupancyDamjan Marion1-2/+8
Adjust buffer allocation so it always have odd number of cache lines. That should result in better distribution of cachelines among cache sets. Type: improvement Change-Id: I0d39d4cf01cff36ad6f70a700730823a96448c22 Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-06-15buffers: fix non-default sized buffers initializationBenoît Ganne1-5/+21
Type: fix Change-Id: I4a93e1d9936414c514cb237a22624986b3ef5b3d Signed-off-by: Benoît Ganne <bganne@cisco.com>
2020-04-13buffers: configurable buffer fault injectorDave Barach1-0/+26
When configured at compile time via the cmake VPP_BUFFER_FAULT_INJECTOR option, the buffer allocator will appear to fail a certain fraction of the time. By default, the allocator succeeds 80% of the time. Detailed command line configuration options are available, but only when the image has been compiled with cmake option described above: vlib { buffer-alloc-success-rate [0.0 ... 1.0] buffer-alloc-success-seed <nnnn> } Modify vlib_buffer_pool_create(...) so 0 is always an invalid buffer index. Debug images: add checks for bad buffer index enqueues, and also verify that f->n_vectors doesn't accidentally map one or more instances of the frame poison pattern 0xfefefefe. Type: improvement Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: Iab939858014463d1e664682805013d334d6fcbe5
2019-10-31vlib: only dump 1st buffer in chain by defaultBenoît Ganne1-2/+13
Several nodes include buffers in their traces, but only the 1st. When formatting the trace we must not try to iterate through all chained buffers. Default to display only the 1st buffer. Type: fix Change-Id: Ib3c668bbf4ab70ae68eba2ac402c7b7329825b70 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2019-10-28vlib: don't use vector for keeping buffer indices in the poolDamjan Marion1-11/+11
Type: refactor Change-Id: I72221b97d7e0bf5c93e20bbda4473ca67bfcdeb4 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-08-19vlib: fix vlib_buffer_main_init_numa_node memory leak.Guanghua Zhang1-3/+8
Type: fix Signed-off-by: Guanghua Zhang <ghzhang@fiberhome.com> Change-Id: I8252ed2555f5af6db2f12dc7c30e41cc1ec7dde0
2019-06-24vlib: packet tracer support for pkt thread handoffsDave Barach1-1/+1
Type: feature Change-Id: Ia3d9a47679202c2a47cd3746b50e86c6b8627ef6 Signed-off-by: Dave Barach <dave@barachs.net>
2019-03-28Typos. A bunch of typos I've been collecting.Paul Vinciguerra1-1/+1
Change-Id: I53ab8d17914e6563110354e4052109ac02bf8f3b Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2019-03-06vlib: coverity woesSteven Luong1-2/+1
Remove logically dead code to keep coverity from complaining Change-Id: If27d6684d19ab3c8886732a67922c86e5f0b3554 Signed-off-by: Steven Luong <sluong@cisco.com>
2019-03-04buffers: support cases where numa node memory is not availableDamjan Marion1-37/+77
Change-Id: I550ef893e41d86310d4e5af16d6100b8e557b68a Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-03-04stats: fix crash on early initFilip Tehlar1-4/+18
Change-Id: I911b5070f544c3556ad1ca584d42689c00cc38f3 Signed-off-by: Filip Tehlar <ftehlar@cisco.com>
2019-02-26stats: buffer gauge callbacks use index instead of name.Ole Troan1-26/+20
Change-Id: I500b8e72d7e021b9564dad0ffc087cb0e1ceb96f Signed-off-by: Ole Troan <ot@cisco.com>
2019-02-22stats: add buffer gaugesFilip Tehlar1-1/+87
Change-Id: I7f7a459f25d64ea5fa36e30d7dccc667bc19c5a9 Signed-off-by: Filip Tehlar <ftehlar@cisco.com>
2019-02-09buffers: fix typoDamjan Marion1-6/+8
Change-Id: I4e836244409c98739a13092ee252542a2c5fe259 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-02-06buffers: make buffer data size configurable from startup configDamjan Marion1-16/+25
Example: buffers { default data-size 1536 } Change-Id: I5b4436850ca18025c9fdcfc7ed648c2c2732d660 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-02-06buffers: improve buffer init performanceDamjan Marion1-17/+14
Change-Id: Ib59a3c32af754a898ade17c42e60a88f48b797ff Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-31assert on double-free in debug buildKlement Sekera1-4/+2
The assert allows easier debugging in gdb by aborting instead of exiting cleanly. Also a core is generated so a stack trace can be obtained. Change-Id: I2d05488c4522f4d4570fdfe0283130eb4c853d2a Signed-off-by: Klement Sekera <ksekera@cisco.com>
2019-01-31buffers: reinitialize metadata, add additional validationDamjan Marion1-0/+1
- DPDK overwrites metadata as part of rte_pktmbuf_init(...) so we need reinitialize it - additional checks added to ensure ref_count is never < 1 Change-Id: Ida336f81c4723e8f2e0ad4a70cb7b1ecfff978a0 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-30buffers: major cleanup and improvementsDamjan Marion1-375/+224
This patch introduces following changes: - deprecated free lists which are not used and not compatible with external buffer managers (i.e. DPDK) - introduces native support for per-numa buffer pools - significantly improves performance of buffer alloc and free Change-Id: I4a8e723ae47056717afd6cac0efe87cb731b5be7 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-20buffers: remove VLIB_BUFFER_DEFAULT_FREE_LIST macro and fl->n_data_bytesDamjan Marion1-17/+10
Change-Id: I0ba5175be077c40556f2a3ce629c5bbcd71e0a81 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-20buffers: remove vlib_buffer_delete_free_listDamjan Marion1-43/+0
Change-Id: I5fe01e918b534d1ac1d47f0d7610536f45c3158c Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-20buffers: don't init metadata, as it is already initializedDamjan Marion1-1/+0
Change-Id: Ia083050389853c25b069f0f8286d50d3f4aef527 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-20buffers: keep buffer_main in vlib_main_tDamjan Marion1-16/+17
Change-Id: I3bb1d9f83dd08f4b93acd4a281bfec0674e39c2e Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-19buffers: remove free-list information from buffer metadataDamjan Marion1-14/+7
Change-Id: I6048c6a51efa826ac333f7d15919cb87dd766d74 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-18buffers: remove freelist argument from vlib_buffer_add_dataDamjan Marion1-9/+5
Change-Id: Idb224b807373128b78dc46f177b2d2e0ed288349 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-07Handle buffer alloc failure in vlib_buffer_add_dataDave Barach1-6/+7
It's not OK to crash due to a transient buffer allocation failure. Return 1 if the requested operation failed, otherwise 0. Buffer index parameter change to a value-result, so the caller can differentiate between partial and complete allocation failure: callers which request an initial allocation (inbound bi = ~0) need to check the (out) value to decide whether or not to call vlib_buffer_free(...). Change-Id: I03029d7f2714c17dca4630dfd95a1eb578b68384 Signed-off-by: Dave Barach <dave@barachs.net>
2019-01-01buffers: remove unused codeDamjan Marion1-63/+1
Change-Id: If2bbfbc52994f5de0879763e0b7a7864498debb6 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-11-25vlib:init free list buffers vecEyal Bari1-0/+4
fixes a problem seen when dhcp proxy uses vlib_buffer_chain_linearize function which tries to vlib_buffer_alloc zero buffers - which succeeds if the buffers vector is initialized but otherwise crashes when trying to update the _vec_len in vlib_buffer_alloc_from_free_list solved by initializing the free_list buffers vec Change-Id: I1186d7aac05e54864d72f3f144e1bef8064f8efa Signed-off-by: Eyal Bari <ebari@cisco.com>
2018-11-14Remove c-11 memcpy checks from perf-critical codeDave Barach1-9/+10
Change-Id: Id4f37f5d4a03160572954a416efa1ef9b3d79ad1 Signed-off-by: Dave Barach <dave@barachs.net>
2018-11-13vlib:remove unused argumentEyal Bari1-1/+1
Change-Id: I88c3d3e516401bb1c84991515cd701c156ae19dd Signed-off-by: Eyal Bari <ebari@cisco.com>
2018-10-25pmalloc: support for 4K pagesDamjan Marion1-4/+24
Change-Id: Iecceffe06a92660976ebb58cd3cbec4be8931db0 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-10-23physmem coverity issuesDamjan Marion1-1/+1
Change-Id: Ie9ff9b751190632dfc4576e5cbb1987a4142af5e Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-10-23vlib: alloc buffers on local numa, not on numa 1Damjan Marion1-1/+2
Change-Id: Icb8172238f735fd0825e474e16a006f1435e175c Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-10-23vlib: reduce default buffer memory size to be below RLIMIT_MEMLOCKDamjan Marion1-1/+5
Change-Id: If36be24fa08e21c7718a1dced506a7f254dfb5cf Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-10-23Numa-aware, growable physical memory allocator (pmalloc)Damjan Marion1-54/+21
Change-Id: Ic4c46bc733afae8bf0d8146623ed15633928de30 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-10-23c11 safe string handling supportDave Barach1-4/+4
Change-Id: Ied34720ca5a6e6e717eea4e86003e854031b6eab Signed-off-by: Dave Barach <dave@barachs.net>
2018-10-16Fix coverity issue for potentially overflowing of page sizeHaiyang Tan1-1/+1
Change-Id: I2779626d745badb63386efcf729da7a094a4f297 Signed-off-by: Haiyang Tan <haiyangtan@tencent.com>
2018-09-03Deprecate old buffer replication schemeDamjan Marion1-49/+14
Change-Id: I1f54b994425c58776e1445c8d9fe142e7a644d3d Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-03-16Fix a long-latent bi=0 bug in vlib_buffer_add_dataDave Barach1-1/+1
Change vlib_buffer_add_data() so it interprets ~0 to mean please allocate a new buffer, instead of 0. Fixed a couple of calls to pass ~0 instead of 0. Zero has always been a valid buffer index, we never happened to actually use it until recent buffer allocator changes. The presenting symptom: ASSERT failure when running "make TEST=test_mpls test-debug" Change-Id: Ic909913c1d464b3434d6d47e0c58f978806854d5 Signed-off-by: Dave Barach <dave@barachs.net>
2018-03-14vlib: internal buffer manager reworkDamjan Marion1-163/+112
- buffer_main is no longer part of vlib_main_t - pool of free lists is still part of vlib_main_t - mheap is not used anymore for buffer allocation - simple bitmap bassed buffer alloc scheme is introduced Change-Id: I3e1e6d00e2c8122293ed0a741245eb841315a1ff Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-03-05vlib: vlib_buffer_free_internal optimizationsDamjan Marion1-53/+88
Change-Id: Ia819b630b824002c59680836a8833df7e5af90e9 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-03-05vlib: vfio code reworkDamjan Marion1-8/+10
Change-Id: I99cf3e7cc991aa7d32385a155c707a6516516117 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-02-07Refactor vlib_buffer flagsDamjan Marion1-17/+30
Change-Id: I853386aebfe488ebb10328435b81b6e3403c5dd0 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-01-31vlib: allocating buffers on thread-x and freeing them on thread-y causesSteven1-6/+5
a crash on debug image (VPP-1151) In debug image, there is extra code to validate the buffer when it is freed. It uses the hash table to lookup the buffer index with spinlock to prevent contention. However, there is one spinlock for each worker thread. So allocating the buffer on thread-x and freeing the same buffer on thread-y causes the validation to fail on thread-y. The fix is to have only one spinlock, stored in vlib_global_main. Change-Id: Ic383846cefe84a3e262255afcf82276742f0f62e Signed-off-by: Steven <sluong@cisco.com> (cherry picked from commit a7effa1b072463f12305a474f082aeaffb7ada4b)
2017-11-27vlib: make vlib_buffer_alloc inline functionDamjan Marion1-63/+8
Currently, every alloc involves callback function call. With this change callback function is called only if there is no empty buffers on the free list. Change-Id: I2238c19ece7ce182c49ba0f2485add52f335f3b6 Signed-off-by: Damjan Marion <damarion@cisco.com>