summaryrefslogtreecommitdiffstats
path: root/src/vlib/buffer.c
AgeCommit message (Collapse)AuthorFilesLines
2019-02-26stats: buffer gauge callbacks use index instead of name.Ole Troan1-26/+20
Change-Id: I500b8e72d7e021b9564dad0ffc087cb0e1ceb96f Signed-off-by: Ole Troan <ot@cisco.com>
2019-02-22stats: add buffer gaugesFilip Tehlar1-1/+87
Change-Id: I7f7a459f25d64ea5fa36e30d7dccc667bc19c5a9 Signed-off-by: Filip Tehlar <ftehlar@cisco.com>
2019-02-09buffers: fix typoDamjan Marion1-6/+8
Change-Id: I4e836244409c98739a13092ee252542a2c5fe259 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-02-06buffers: make buffer data size configurable from startup configDamjan Marion1-16/+25
Example: buffers { default data-size 1536 } Change-Id: I5b4436850ca18025c9fdcfc7ed648c2c2732d660 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-02-06buffers: improve buffer init performanceDamjan Marion1-17/+14
Change-Id: Ib59a3c32af754a898ade17c42e60a88f48b797ff Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-31assert on double-free in debug buildKlement Sekera1-4/+2
The assert allows easier debugging in gdb by aborting instead of exiting cleanly. Also a core is generated so a stack trace can be obtained. Change-Id: I2d05488c4522f4d4570fdfe0283130eb4c853d2a Signed-off-by: Klement Sekera <ksekera@cisco.com>
2019-01-31buffers: reinitialize metadata, add additional validationDamjan Marion1-0/+1
- DPDK overwrites metadata as part of rte_pktmbuf_init(...) so we need reinitialize it - additional checks added to ensure ref_count is never < 1 Change-Id: Ida336f81c4723e8f2e0ad4a70cb7b1ecfff978a0 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-30buffers: major cleanup and improvementsDamjan Marion1-375/+224
This patch introduces following changes: - deprecated free lists which are not used and not compatible with external buffer managers (i.e. DPDK) - introduces native support for per-numa buffer pools - significantly improves performance of buffer alloc and free Change-Id: I4a8e723ae47056717afd6cac0efe87cb731b5be7 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-20buffers: remove VLIB_BUFFER_DEFAULT_FREE_LIST macro and fl->n_data_bytesDamjan Marion1-17/+10
Change-Id: I0ba5175be077c40556f2a3ce629c5bbcd71e0a81 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-20buffers: remove vlib_buffer_delete_free_listDamjan Marion1-43/+0
Change-Id: I5fe01e918b534d1ac1d47f0d7610536f45c3158c Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-20buffers: don't init metadata, as it is already initializedDamjan Marion1-1/+0
Change-Id: Ia083050389853c25b069f0f8286d50d3f4aef527 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-20buffers: keep buffer_main in vlib_main_tDamjan Marion1-16/+17
Change-Id: I3bb1d9f83dd08f4b93acd4a281bfec0674e39c2e Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-19buffers: remove free-list information from buffer metadataDamjan Marion1-14/+7
Change-Id: I6048c6a51efa826ac333f7d15919cb87dd766d74 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-18buffers: remove freelist argument from vlib_buffer_add_dataDamjan Marion1-9/+5
Change-Id: Idb224b807373128b78dc46f177b2d2e0ed288349 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-07Handle buffer alloc failure in vlib_buffer_add_dataDave Barach1-6/+7
It's not OK to crash due to a transient buffer allocation failure. Return 1 if the requested operation failed, otherwise 0. Buffer index parameter change to a value-result, so the caller can differentiate between partial and complete allocation failure: callers which request an initial allocation (inbound bi = ~0) need to check the (out) value to decide whether or not to call vlib_buffer_free(...). Change-Id: I03029d7f2714c17dca4630dfd95a1eb578b68384 Signed-off-by: Dave Barach <dave@barachs.net>
2019-01-01buffers: remove unused codeDamjan Marion1-63/+1
Change-Id: If2bbfbc52994f5de0879763e0b7a7864498debb6 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-11-25vlib:init free list buffers vecEyal Bari1-0/+4
fixes a problem seen when dhcp proxy uses vlib_buffer_chain_linearize function which tries to vlib_buffer_alloc zero buffers - which succeeds if the buffers vector is initialized but otherwise crashes when trying to update the _vec_len in vlib_buffer_alloc_from_free_list solved by initializing the free_list buffers vec Change-Id: I1186d7aac05e54864d72f3f144e1bef8064f8efa Signed-off-by: Eyal Bari <ebari@cisco.com>
2018-11-14Remove c-11 memcpy checks from perf-critical codeDave Barach1-9/+10
Change-Id: Id4f37f5d4a03160572954a416efa1ef9b3d79ad1 Signed-off-by: Dave Barach <dave@barachs.net>
2018-11-13vlib:remove unused argumentEyal Bari1-1/+1
Change-Id: I88c3d3e516401bb1c84991515cd701c156ae19dd Signed-off-by: Eyal Bari <ebari@cisco.com>
2018-10-25pmalloc: support for 4K pagesDamjan Marion1-4/+24
Change-Id: Iecceffe06a92660976ebb58cd3cbec4be8931db0 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-10-23physmem coverity issuesDamjan Marion1-1/+1
Change-Id: Ie9ff9b751190632dfc4576e5cbb1987a4142af5e Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-10-23vlib: alloc buffers on local numa, not on numa 1Damjan Marion1-1/+2
Change-Id: Icb8172238f735fd0825e474e16a006f1435e175c Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-10-23vlib: reduce default buffer memory size to be below RLIMIT_MEMLOCKDamjan Marion1-1/+5
Change-Id: If36be24fa08e21c7718a1dced506a7f254dfb5cf Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-10-23Numa-aware, growable physical memory allocator (pmalloc)Damjan Marion1-54/+21
Change-Id: Ic4c46bc733afae8bf0d8146623ed15633928de30 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-10-23c11 safe string handling supportDave Barach1-4/+4
Change-Id: Ied34720ca5a6e6e717eea4e86003e854031b6eab Signed-off-by: Dave Barach <dave@barachs.net>
2018-10-16Fix coverity issue for potentially overflowing of page sizeHaiyang Tan1-1/+1
Change-Id: I2779626d745badb63386efcf729da7a094a4f297 Signed-off-by: Haiyang Tan <haiyangtan@tencent.com>
2018-09-03Deprecate old buffer replication schemeDamjan Marion1-49/+14
Change-Id: I1f54b994425c58776e1445c8d9fe142e7a644d3d Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-03-16Fix a long-latent bi=0 bug in vlib_buffer_add_dataDave Barach1-1/+1
Change vlib_buffer_add_data() so it interprets ~0 to mean please allocate a new buffer, instead of 0. Fixed a couple of calls to pass ~0 instead of 0. Zero has always been a valid buffer index, we never happened to actually use it until recent buffer allocator changes. The presenting symptom: ASSERT failure when running "make TEST=test_mpls test-debug" Change-Id: Ic909913c1d464b3434d6d47e0c58f978806854d5 Signed-off-by: Dave Barach <dave@barachs.net>
2018-03-14vlib: internal buffer manager reworkDamjan Marion1-163/+112
- buffer_main is no longer part of vlib_main_t - pool of free lists is still part of vlib_main_t - mheap is not used anymore for buffer allocation - simple bitmap bassed buffer alloc scheme is introduced Change-Id: I3e1e6d00e2c8122293ed0a741245eb841315a1ff Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-03-05vlib: vlib_buffer_free_internal optimizationsDamjan Marion1-53/+88
Change-Id: Ia819b630b824002c59680836a8833df7e5af90e9 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-03-05vlib: vfio code reworkDamjan Marion1-8/+10
Change-Id: I99cf3e7cc991aa7d32385a155c707a6516516117 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-02-07Refactor vlib_buffer flagsDamjan Marion1-17/+30
Change-Id: I853386aebfe488ebb10328435b81b6e3403c5dd0 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-01-31vlib: allocating buffers on thread-x and freeing them on thread-y causesSteven1-6/+5
a crash on debug image (VPP-1151) In debug image, there is extra code to validate the buffer when it is freed. It uses the hash table to lookup the buffer index with spinlock to prevent contention. However, there is one spinlock for each worker thread. So allocating the buffer on thread-x and freeing the same buffer on thread-y causes the validation to fail on thread-y. The fix is to have only one spinlock, stored in vlib_global_main. Change-Id: Ic383846cefe84a3e262255afcf82276742f0f62e Signed-off-by: Steven <sluong@cisco.com> (cherry picked from commit a7effa1b072463f12305a474f082aeaffb7ada4b)
2017-11-27vlib: make vlib_buffer_alloc inline functionDamjan Marion1-63/+8
Currently, every alloc involves callback function call. With this change callback function is called only if there is no empty buffers on the free list. Change-Id: I2238c19ece7ce182c49ba0f2485add52f335f3b6 Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-10-25vlib: add support for multiple buffer poolsDamjan Marion1-12/+24
Change-Id: Icaf7d7ad47284aea7a56e8006b69f45874d64202 Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-10-04[aarch64] Fixes CLI crashes on dpaa2 platform.Christophe Fontaine1-1/+1
- always use 'va_args' as pointer in all format_* functions - u32 for all 'indent' params as it's declaration was inconsistent Change-Id: Ic5799309a6b104c9b50fec309cba789c8da99e79 Signed-off-by: Christophe Fontaine <christophe.fontaine@enea.com>
2017-09-15dpdk: cli to check for buffer leakageFlorin Coras1-1/+1
Use buffer pre_data and existing buffer trace trajectory code to find out dpdk buffer leakages. Change-Id: I26a5d8bd2f23d01cb6070ffc3ddcc6d3d863b575 Signed-off-by: Florin Coras <fcoras@cisco.com>
2017-09-07vlib physmem reworkDamjan Marion1-7/+65
This patch adds supprot support for multiple numa-aware physmem regions. Change-Id: I5c69a6f4da33c8ee21bdb8604d52fd2886f2327e Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-08-30Thread safe internal buffer manager, take twoDamjan Marion1-0/+19
First attempt to make internal buffer manager thread safe was not succesfull, so trying again. This time with more testing. Change-Id: I01b8385a9c26d233934a3339255ea4bd31c865ac Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-07-27Thread safe internal buffer managerDamjan Marion1-6/+1
Change-Id: I45845b952aa42a854e1c2c396b85f905de987020 Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-07-14Introduce l{2,3,4}_hdr_offset fields in the buffer metadataDamjan Marion1-6/+10
To save space in the first cacheline following is changed: - total_length_not_including_first_buffer moved to the 2nd cacheline. This field is used only when VLIB_BUFFER_TOTAL_LENGTH_VALID and VLIB_BUFFER_NEXT_PRESENT are both set. - free_list_index is now stored in 4bits inside flags, which allows up to 16 free lists. In case we need more we can store index in the 2nd cachelin Change-Id: Ic8521350819391af470d31d3fa1013e67ecb7681 Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-07-10vlib: store buffer memory information in the buffer_mainDamjan Marion1-24/+33
Currently, buffer index is calculated as a offset to the physmem region shifted by log2_cacheline size. When DPDK is used we "hack" physmem data with information taken from dpdk mempool. This makes physmem code not usable with DPDK. This change makes buffer memory start and size independent of physmem basically allowing physmem to be used when DPDK plugin is loaded. Change-Id: Ieb399d398f147583b9baab467152a352d58c9c31 Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-07-05Buffer name inconsistently used a cstring/vec (VPP-901)Chris Luke1-3/+3
Spotted in the output of CLI command "show buffers", the name field sometimes had trailing garbage, the hall sign of a string not being terminated. In this case it was being inconsistently used as a cstring or a vec. - CLI printf needs %v to print the vec srring - vlib_buffer_create_free_list_helper tried to use clib_mem_is_heap_object() to detect a vec object, wheras it should use clib_mem_is_vec() Change-Id: Ib8b242a0c5a18924b8af7e8e1432784eebcf572c Signed-off-by: Chris Luke <chrisy@flirble.org>
2017-04-06Use thread local storage for thread indexDamjan Marion1-3/+3
This patch deprecates stack-based thread identification, Also removes requirement that thread stacks are adjacent. Finally, possibly annoying for some folks, it renames all occurences of cpu_index and cpu_number with thread index. Using word "cpu" is misleading here as thread can be migrated ti different CPU, and also it is not related to linux cpu index. Change-Id: I68cdaf661e701d2336fc953dcb9978d10a70f7c1 Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-03-10Retire vpp_liteDamjan Marion1-0/+6
vpp_lite platform is not needed anymore as same efect can be achieved with following startup.conf config: plugins { plugin dpdk_plugin.so { disable } } Change-Id: I690ea8ceb1c6e1fe32e01e7da54e9958019a93bf Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-03-09vlib_mains == 0 special cases be goneDave Barach1-3/+24
Clean up spurious binary API client link dependency on libvlib.so, which managed to hide behind vlib_mains == 0 checks reached by VLIB_xxx_FUNCTION macros. Change-Id: I5df1f8ab07dca1944250e643ccf06e60a8462325 Signed-off-by: Dave Barach <dave@barachs.net>
2017-03-01VPP-598: tcp stack initial commitDave Barach1-1/+1
Change-Id: I49e5ce0aae6e4ff634024387ceaf7dbc432a0351 Signed-off-by: Dave Barach <dave@barachs.net> Signed-off-by: Florin Coras <fcoras@cisco.com>
2017-02-28vlib: add buffer cloning supportDamjan Marion1-196/+58
Change-Id: I50070611af15b2b4cc29664a8bee4f821ac3c835 Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-02-06vlib: remove algned/unaligned buffers schemeDamjan Marion1-195/+25
Change-Id: I4433eaed3f4e201edc329c4842cbbf74beb19a9a Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-01-17dpdk: remove duplicate code in buffers.cDamjan Marion1-52/+21
Change-Id: Idc17b4a32d40012556d5d8550942db0372ebf23d Signed-off-by: Damjan Marion <damarion@cisco.com>