summaryrefslogtreecommitdiffstats
path: root/src/vlib/buffer_funcs.h
AgeCommit message (Collapse)AuthorFilesLines
2020-04-13buffers: configurable buffer fault injectorDave Barach1-0/+11
When configured at compile time via the cmake VPP_BUFFER_FAULT_INJECTOR option, the buffer allocator will appear to fail a certain fraction of the time. By default, the allocator succeeds 80% of the time. Detailed command line configuration options are available, but only when the image has been compiled with cmake option described above: vlib { buffer-alloc-success-rate [0.0 ... 1.0] buffer-alloc-success-seed <nnnn> } Modify vlib_buffer_pool_create(...) so 0 is always an invalid buffer index. Debug images: add checks for bad buffer index enqueues, and also verify that f->n_vectors doesn't accidentally map one or more instances of the frame poison pattern 0xfefefefe. Type: improvement Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: Iab939858014463d1e664682805013d334d6fcbe5
2020-04-07l2: handle complete clone fail in l2_floodDave Barach1-2/+2
vlib_buffer_clone(...) may not manage to produce any buffer clones at all. vlib_buffer_clone_256 should not smash the original buffer reference count if no clones are produced. Type: fix Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I14d9d53637a220485c7a0036cfc75a4149b264ea
2020-03-20rdma: ring db only once per vector on txBenoît Ganne1-0/+18
Type: improvement Change-Id: I09b017d60bc38ff8a4a7a73aff42b27611bd8955 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2020-03-02rdma: improve rx loopDamjan Marion1-0/+18
Type: improvement Change-Id: If81847bc0c92d167ce03e1e94a2f8e18f8154af2 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-11-26misc: add gdb helpers for vlib buffersBenoît Ganne1-0/+7
Type: feature Change-Id: I7e619040857310fffd33c355f6c491a55f7a4a02 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2019-10-31vlib: only dump 1st buffer in chain by defaultBenoît Ganne1-1/+1
Several nodes include buffers in their traces, but only the 1st. When formatting the trace we must not try to iterate through all chained buffers. Default to display only the 1st buffer. Type: fix Change-Id: Ib3c668bbf4ab70ae68eba2ac402c7b7329825b70 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2019-10-28vlib: don't use vector for keeping buffer indices in the poolDamjan Marion1-22/+40
Type: refactor Change-Id: I72221b97d7e0bf5c93e20bbda4473ca67bfcdeb4 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-08-15vlib: copy trace_handle in vlib_buffer_copy/clone() functionsJohn Lo1-0/+2
Since vlib_buffer_copy() and vlib_buffer_clone() both preserve VLIB_BUFFER_IS_TRACED bit in flags field, it should also copy trace_handle which would add minimal overhead. Thus, callers of these functions do not have to call vlib_buffer_copy_trace_flags() to copy trace_handle. Type: refactor Signed-off-by: John Lo <loj@cisco.com> Change-Id: Iff6a3f81660dd62b36a2966033eb380305340310
2019-08-09vlib: fix vlib_buffer_copy to preserve buffer flags bitJohn Lo1-3/+8
Make vlib_buffer_copy() preserve buffer flags bit the same way as that of vlib_buffer_clone() so both are consistent. Type: fix Signed-off-by: John Lo <loj@cisco.com> Change-Id: I6c32aa1e88724b482ce2439d82019e690311b664
2019-06-20buffers: fix crashKlement Sekera1-3/+9
this change is being made to fix a crash when current_data < 0 in buffer linearization function Ticket: N/A Type: fix Fixes: f883f6a1132ad4bb7aa9d9a79d420274fbcf3b64 Change-Id: Ia4ede823f673780e0c30d075b091db42e183650d Signed-off-by: Klement Sekera <ksekera@cisco.com>
2019-06-12buffers: fix vlib_buffer_free_no_nextDamjan Marion1-1/+1
Type: fix Fixes: 910d369 Change-Id: I0e8380cd2b0dc038a028d9cf2568741059de460f Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-04-01Enhance vlib_buffer_clone to change headroom space for packetJohn Lo1-7/+66
Add function vlib_buffer_clone_at_offset() so that the cloned packets' 1st buffer will have the original packet header copied at the specified offset (instead of current_data). This can be used to increase headroom in the cloned packet to allow large header encaps in case the stdandard 128B predata is not enough. The original vlib_buffer_clone() still works the same as before. Change-Id: I3b50c8ad7e3952568bc141710567c99a3dacddce Signed-off-by: John Lo <loj@cisco.com>
2019-03-20Re-enable aarch64 neon instruction in vlib_buffer_free_inlineLijian.Zhang1-3/+3
int vaddvq_u8 (uint8x16_t __a) is not appropriate to implement xxx_is_all_zero, as there may be overflow causing incorrect return value. Here's an example. u8x16 x = {0 <repeats 12 times>, 1, 255, 0, 0}; Change-Id: Ia6a10bdf8da360dec12db902d028751a1a77e9a4 Signed-off-by: Lijian Zhang <Lijian.Zhang@arm.com> Reviewed-by: Sirshak Das <Sirshak.Das@arm.com>
2019-03-13buffers: don't use clib_memcpy for copying buffer indicesDamjan Marion1-1/+37
Should be faster this way if n_indices is not constant value Change-Id: I6c34fd313daa2392199f3b9bd20d0cd6cf9ae21b Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-03-13buffers: fix ref_count mask in vlib_buffer_free_inlineDamjan Marion1-1/+1
Credits to: Lollita Liu <lollita.liu@ericsson.com> Change-Id: I40f7ad9cc5220db7ac4571cb14feaf22fc02aa2e Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-03-12ICMP46 error: Clone first buffer instead of "truncating" original bufferOle Troan1-0/+21
Previous code was walked buffer chain, effectively trying to "truncate" the chain, reset the length of first buffer and reused that as the ICMP error message. That could have issues in cases there were other users of the buffer chain. Update to clone the first buffer in chain, and use that for the ICMP error message instead. Change-Id: Ibc1a0bf2d854dae41874808c8297028ed93dd69d Signed-off-by: Ole Troan <ot@cisco.com>
2019-03-04buffers: support cases where numa node memory is not availableDamjan Marion1-2/+2
Change-Id: I550ef893e41d86310d4e5af16d6100b8e557b68a Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-02-25buffer chain linearizationKlement Sekera1-97/+120
Rewrite vlib_buffer_chain_linearize function so that it works as intended. Linearize buffer chains coming out of reassembly to work around some dpdk-tx issues. Note that this is not a complete workaround as a sufficiently large packet will still cause the resulting chain to be too long. Drop features from reassembly code which relies on knowing which and how many buffers were freed during linearization, buffer counts and tracing capabilities for these cases. Change-Id: Ic65de53ecb5c78cd96b178033f6a576ab4060ed1 Signed-off-by: Klement Sekera <ksekera@cisco.com>
2019-02-14buffer: bug fix vlib_buffer_free_inlineLollita Liu1-6/+7
buffer_pool_index, in case of buffer_pool_index change, should free buffers in queue before setting buffer_pool_index again Change-Id: I846ead947a7b2d940c8fc747976239d608597391 Signed-off-by: Lollita Liu <lollita.liu@ericsson.com>
2019-02-09buffers: fix typoDamjan Marion1-4/+4
Change-Id: I4e836244409c98739a13092ee252542a2c5fe259 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-02-06buffers: make buffer data size configurable from startup configDamjan Marion1-9/+10
Example: buffers { default data-size 1536 } Change-Id: I5b4436850ca18025c9fdcfc7ed648c2c2732d660 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-31buffers: vallidate that buffer is allocated during buffer pool putDamjan Marion1-8/+5
Change-Id: I8044b34a37fe1994a8dfa1ca89929f3642c72e8d Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-31buffers: reinitialize metadata, add additional validationDamjan Marion1-13/+43
- DPDK overwrites metadata as part of rte_pktmbuf_init(...) so we need reinitialize it - additional checks added to ensure ref_count is never < 1 Change-Id: Ida336f81c4723e8f2e0ad4a70cb7b1ecfff978a0 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-30buffers: major cleanup and improvementsDamjan Marion1-187/+329
This patch introduces following changes: - deprecated free lists which are not used and not compatible with external buffer managers (i.e. DPDK) - introduces native support for per-numa buffer pools - significantly improves performance of buffer alloc and free Change-Id: I4a8e723ae47056717afd6cac0efe87cb731b5be7 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-29Fix error while compiling using 'vlib/vlib.h' dependencie.Dmitry Vakhrushev1-0/+2
This variable without any needed 'define' becoming unused. Change-Id: I661a75a78dba03abb861ed918ad1e634a8ecd0af Signed-off-by: Dmitry Vakhrushev <dmitry@netgate.com>
2019-01-24buffers: fix vector typesDamjan Marion1-1/+3
Change-Id: I9df96264e30806ac3daf7121f314f34f06232413 Signed-off-by: Damjan Marion <damjan.marion@gmail.com>
2019-01-23buffers: wrap vlib_buffer_t to union and expose vector typesDamjan Marion1-1/+12
Change-Id: I1c12e2941cae198ededbb65eb5be51a4eabe2c1b Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-20buffers: remove VLIB_BUFFER_DEFAULT_FREE_LIST macro and fl->n_data_bytesDamjan Marion1-33/+3
Change-Id: I0ba5175be077c40556f2a3ce629c5bbcd71e0a81 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-20buffers: remove vlib_buffer_delete_free_listDamjan Marion1-16/+0
Change-Id: I5fe01e918b534d1ac1d47f0d7610536f45c3158c Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-20buffers: don't init metadata, as it is already initializedDamjan Marion1-38/+1
Change-Id: Ia083050389853c25b069f0f8286d50d3f4aef527 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-20buffers: keep buffer_main in vlib_main_tDamjan Marion1-14/+22
Change-Id: I3bb1d9f83dd08f4b93acd4a281bfec0674e39c2e Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-19buffers: remove free-list information from buffer metadataDamjan Marion1-45/+6
Change-Id: I6048c6a51efa826ac333f7d15919cb87dd766d74 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-18buffers: remove freelist argument from vlib_buffer_add_dataDamjan Marion1-3/+2
Change-Id: Idb224b807373128b78dc46f177b2d2e0ed288349 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-18Add vlib_buffer_copy_indices inline functionDamjan Marion1-2/+8
This reverts commit 1e59f9ddbdda14591967e1d66eab8623f9ba58e4. Change-Id: Iae1d372b887e170d28cac2fe4c61325ee5a5894a Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-18deprecate clib_memcpy64_x4Damjan Marion1-6/+7
Storing buffer in local template seems to be better option.... Change-Id: I1a2fdd68cb956f99a5b36d2cd810fc623e089bcf Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-07Handle buffer alloc failure in vlib_buffer_add_dataDave Barach1-2/+2
It's not OK to crash due to a transient buffer allocation failure. Return 1 if the requested operation failed, otherwise 0. Buffer index parameter change to a value-result, so the caller can differentiate between partial and complete allocation failure: callers which request an initial allocation (inbound bi = ~0) need to check the (out) value to decide whether or not to call vlib_buffer_free(...). Change-Id: I03029d7f2714c17dca4630dfd95a1eb578b68384 Signed-off-by: Dave Barach <dave@barachs.net>
2019-01-01buffers: remove unused codeDamjan Marion1-77/+2
Change-Id: If2bbfbc52994f5de0879763e0b7a7864498debb6 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-12-13fix vlib_buffer_chain_compress assert in debugKlement Sekera1-1/+2
replace vlib_buffer_advance with its code to work around assert Change-Id: I6e332527f5e0c10d23305b6fc2e837d8b4f99e78 Signed-off-by: Klement Sekera <ksekera@cisco.com>
2018-11-20dhcp4:(VPP-1483) linearize chained packets before handlingEyal Bari1-0/+64
dhcp packets might (when flooded) arrive in chains of cloned buffers Change-Id: Ifddecd656b6a5d6ba8cd94184f5c021684e35548 Signed-off-by: Eyal Bari <ebari@cisco.com>
2018-11-14Remove c-11 memcpy checks from perf-critical codeDave Barach1-22/+22
Change-Id: Id4f37f5d4a03160572954a416efa1ef9b3d79ad1 Signed-off-by: Dave Barach <dave@barachs.net>
2018-11-13vlib:remove unused argumentEyal Bari1-3/+1
Change-Id: I88c3d3e516401bb1c84991515cd701c156ae19dd Signed-off-by: Eyal Bari <ebari@cisco.com>
2018-10-23Numa-aware, growable physical memory allocator (pmalloc)Damjan Marion1-4/+1
Change-Id: Ic4c46bc733afae8bf0d8146623ed15633928de30 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-10-22vlib: introduce vlib_buffer_get_{pa,va,current_va,current_pa} inlinesDamjan Marion1-5/+8
Change-Id: I0b42ac6b05bc9910904a97924ea4bebc84507d4d Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-10-19vppinfra: add atomic macros for __sync builtinsSirshak Das1-3/+3
This is first part of addition of atomic macros with only macros for __sync builtins. - Based on earlier patch by Damjan (https://gerrit.fd.io/r/#/c/10729/) Additionally - clib_atomic_release macro added and used in the absence of any memory barrier. - clib_atomic_bool_cmp_and_swap added Change-Id: Ie4e48c1e184a652018d1d0d87c4be80ddd180a3b Original-patch-by: Damjan Marion <damarion@cisco.com> Signed-off-by: Sirshak Das <sirshak.das@arm.com> Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com> Reviewed-by: Ola Liljedahl <ola.liljedahl@arm.com> Reviewed-by: Steve Capper <steve.capper@arm.com>
2018-10-01API / CLI event-log tracingDave Barach1-0/+1
Add an "elog trace [api][cli][barrier]" debug CLI command. Removed the barrier elog test command. Remove unused reliable multicast code. Change-Id: Ib3ecde901b7c49fe92b313d0087cd7e776adcdce Signed-off-by: Dave Barach <dave@barachs.net>
2018-09-12Add and enable u32x4_extend_to_u64x2_high for aarch64 NEON intrinsics.Sirshak Das1-1/+5
This is the high version of extendto. This function accomplishes the same task as both shuffling and extending done by SSE intrinsics. This enables the NEON version for buffer indexes to buffer pointer translation. Change-Id: I52d7bbf3d76ba69c9acb0e518ff4bc6abf3bbbd4 Signed-off-by: Sirshak Das <sirshak.das@arm.com> Reviewed-by: Steve Capper <steve.capper@arm.com> Reviewed-by: Yi He <yi.he@arm.com> Verified-by: Lijian Zhang <lijian.zhang@arm.com>
2018-07-27128-bit SIMD version of vlib_get_buffersDamjan Marion1-0/+8
Change-Id: I1a28ddf535c80ecf4ba4bf31659ff2fead1d8a64 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-07-23Buffer clone: copy opaque2Neale Ranns1-0/+2
Change-Id: I06e1dbfa0782bc03f2d0e77295b38b4fde7df35e Signed-off-by: Neale Ranns <nranns@cisco.com>
2018-06-09Fix bug in vlib_buffer_free_from_ring_no_nextDamjan Marion1-1/+1
Change-Id: I332bb4578d1a3c79770985bf1f315d2ed823a3e5 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-05-31fix missing void pointer explicit type castZhiyong Yang1-1/+1
Change-Id: I5098e5d966abc44db045783883424b63083f08ee Signed-off-by: Zhiyong Yang <zhiyong.yang@intel.com>