aboutsummaryrefslogtreecommitdiffstats
path: root/src/plugins/dpdk/buffer.c
AgeCommit message (Collapse)AuthorFilesLines
2019-11-06dpdk: conditionally revert rte_vfio_dma_map patchDamjan Marion1-0/+5
DPDK have bug which will be fixed in 19.11. Type: fix Change-Id: I6c0058928e5991d61b3c5fcba706f35e6886b0f2 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-10-29dpdk: replace deprecated VFIO API invocationDamjan Marion1-1/+2
Type: refactor Change-Id: I1712729b3f5d4ab659f51a2d3bf798db16b50820 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-10-28vlib: don't use vector for keeping buffer indices in the poolDamjan Marion1-11/+7
Type: refactor Change-Id: I72221b97d7e0bf5c93e20bbda4473ca67bfcdeb4 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-09-16dpdk: initialize rte_mbuf during mempool dequeueDamjan Marion1-1/+44
In some cases it may happen that buffer is allocated by DPDK, and freed by VPP native code. In such cases dpdk metadata is not reset, so we need to do that during mempool dequeue. Template approach is taken to reduce cost of that operation. Type: fix Fixes: 910d369 Change-Id: Ic239007cfc8fbceb965021c56963cda9d53f63be Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-06-18dpdk: check result of rte_mempool_create_emptyLijian.Zhang1-0/+15
Check the returned pointer, report error and return in case of failure. This avoids crashing without any useful clues or debug messages. Type: fix Change-Id: I15d0735a531c2d9a8b6f67b7d6fe326b98c963c3 Signed-off-by: Lijian Zhang <Lijian.Zhang@arm.com> Reviewed-by: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>
2019-02-09buffers: fix typoDamjan Marion1-1/+1
Change-Id: I4e836244409c98739a13092ee252542a2c5fe259 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-02-06buffers: make buffer data size configurable from startup configDamjan Marion1-1/+1
Example: buffers { default data-size 1536 } Change-Id: I5b4436850ca18025c9fdcfc7ed648c2c2732d660 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-31buffers: vallidate that buffer is allocated during buffer pool putDamjan Marion1-1/+1
Change-Id: I8044b34a37fe1994a8dfa1ca89929f3642c72e8d Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-31dpdk: populate mempool mem_listDamjan Marion1-2/+16
Mellanox driver needs it Change-Id: I28423805f977089d18d32e85cf08e9e7e1fcf5b2 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-31dpdk: store buffer poool index in mempool pool_idDamjan Marion1-22/+10
Change-Id: I2c225a4932dcf326e10fe93de3ee4fdaef4dd3d0 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-31buffers: reinitialize metadata, add additional validationDamjan Marion1-13/+25
- DPDK overwrites metadata as part of rte_pktmbuf_init(...) so we need reinitialize it - additional checks added to ensure ref_count is never < 1 Change-Id: Ida336f81c4723e8f2e0ad4a70cb7b1ecfff978a0 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-30buffers: major cleanup and improvementsDamjan Marion1-479/+271
This patch introduces following changes: - deprecated free lists which are not used and not compatible with external buffer managers (i.e. DPDK) - introduces native support for per-numa buffer pools - significantly improves performance of buffer alloc and free Change-Id: I4a8e723ae47056717afd6cac0efe87cb731b5be7 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-20dpdk: add buffer.hDamjan Marion1-0/+1
Change-Id: I998658ad7860b23425444e218ce2e1ec655b885a Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-20buffers: remove vlib_buffer_delete_free_listDamjan Marion1-47/+0
Change-Id: I5fe01e918b534d1ac1d47f0d7610536f45c3158c Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-20buffers: don't init metadata, as it is already initializedDamjan Marion1-1/+0
Change-Id: Ia083050389853c25b069f0f8286d50d3f4aef527 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-20buffers: keep buffer_main in vlib_main_tDamjan Marion1-2/+2
Change-Id: I3bb1d9f83dd08f4b93acd4a281bfec0674e39c2e Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-19buffers: remove free-list information from buffer metadataDamjan Marion1-2/+1
Change-Id: I6048c6a51efa826ac333f7d15919cb87dd766d74 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-18deprecate clib_memcpy64_x4Damjan Marion1-8/+9
Storing buffer in local template seems to be better option.... Change-Id: I1a2fdd68cb956f99a5b36d2cd810fc623e089bcf Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-01buffers: remove unused codeDamjan Marion1-18/+0
Change-Id: If2bbfbc52994f5de0879763e0b7a7864498debb6 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-10-25pmalloc: support for 4K pagesDamjan Marion1-1/+1
Change-Id: Iecceffe06a92660976ebb58cd3cbec4be8931db0 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-10-23Numa-aware, growable physical memory allocator (pmalloc)Damjan Marion1-19/+15
Change-Id: Ic4c46bc733afae8bf0d8146623ed15633928de30 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-10-23c11 safe string handling supportDave Barach1-3/+3
Change-Id: Ied34720ca5a6e6e717eea4e86003e854031b6eab Signed-off-by: Dave Barach <dave@barachs.net>
2018-10-22dpdk: use rte_vfio_dma_map API instead of digging for vfio fdDamjan Marion1-63/+17
This is new API introduced in DPDK 18.08. Change-Id: I66d49fe54a6abf2af621b597e8d4535e29cecec4 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-10-16Fix coverity issue for potentially overflowing of page sizeHaiyang Tan1-3/+3
Change-Id: I2779626d745badb63386efcf729da7a094a4f297 Signed-off-by: Haiyang Tan <haiyangtan@tencent.com>
2018-09-03Deprecate old buffer replication schemeDamjan Marion1-33/+2
Change-Id: I1f54b994425c58776e1445c8d9fe142e7a644d3d Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-08-13dpdk: support for DPDK 18.08Damjan Marion1-15/+15
Change-Id: If1b93341c222160b9a08f127620c024620e55c37 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-07-18Add config option to use dlmalloc instead of mheapDave Barach1-2/+2
Configure w/ --enable-dlmalloc, see .../build-data/platforms/vpp.mk src/vppinfra/dlmalloc.[ch] are slightly modified versions of the well-known Doug Lea malloc. Main advantage: dlmalloc mspaces have no inherent size limit. Change-Id: I19b3f43f3c65bcfb82c1a265a97922d01912446e Signed-off-by: Dave Barach <dave@barachs.net>
2018-06-07Add support for DPDK 18.05Damjan Marion1-13/+20
Change-Id: I205932bc727c990011bbbe1dc6c0cf5349d19806 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-06-03dpdk: buffer free optimizationsDamjan Marion1-76/+61
~5 clocks/packet improvement... Change-Id: I1a78fa24dcd1b3ab7f45e10b9ded50f79517114a Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-06-03dpdk: improve buffer alloc perfomanceDamjan Marion1-70/+56
This is ~50% improvement in buffer alloc performance. For a 256 buffer allocation, it was ~10 clocks/buffer, now is < 5 clocks. Change-Id: I97590e240a79a42bcab5eb26587fc2d11e6eb163 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-05-30vppinfra: explicitely state for signed types that they are signedDamjan Marion1-2/+2
This fixes some compilation warnings with clang on AArch64. Change-Id: Idb941944e3f199f483c80e143a9e5163a031c4aa Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-05-29Revert "dpdk: set dmamap iova address value according to eal_iova_mode"Damjan Marion1-4/+1
This breaks VFIO operation. This reverts commit d3b3baa4f8e9e4d95264aff16fe85434ef8061bd. Change-Id: I2482e0da2d1ebfc365d13668c4b992b040f561b4 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-05-29Add VLIB_NODE_FN() macro to simplify multiversioning of node functionsDamjan Marion1-2/+2
Change-Id: Ibab5e27277f618ceb2d543b9d6a1a5f191e7d1db Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-05-29dpdk: mempool priv intialization must be done before releasing buffers to poolSachin Saxena1-8/+7
- Currently mempool priv size is getting initialized after releasing buffers to pool. This is causing mismatch in expected & real metadata size value and buffers are getting released with wrong offset. (when metadata offset is in use for a given platform) - Since private data size is 0 initially, metadata size don't include space for VLIB_BUFFER_HDR. Change-Id: I780c4d518104631a3dcf192185bacf58b3598e65 Signed-off-by: Sachin Saxena <sachin.saxena@nxp.com>
2018-05-28dpdk: set dmamap iova address value according to eal_iova_modeSachin Saxena1-1/+4
- Fix the issue where eal iova mode is Virtual Address (RTE_IOVA_VA) but setting DMA iova address to Physical address value always. Change-Id: Ib1e9c1596d95885c7eff11723338121627203e61 Signed-off-by: Sachin Saxena <sachin.saxena@nxp.com>
2018-04-30plugins: dpdk: fix check which makes not sense, likely a typoSzymon Sliwa1-1/+1
Change-Id: If33854f9c32736edf571fb66cdfa759db1c9de25 Signed-off-by: Szymon Sliwa <szs@semihalf.com>
2018-03-14vlib: internal buffer manager reworkDamjan Marion1-30/+18
- buffer_main is no longer part of vlib_main_t - pool of free lists is still part of vlib_main_t - mheap is not used anymore for buffer allocation - simple bitmap bassed buffer alloc scheme is introduced Change-Id: I3e1e6d00e2c8122293ed0a741245eb841315a1ff Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-03-09dpdk: move DPDK vfio hack to dpdk pluginDamjan Marion1-2/+67
Change-Id: I806cbf8c6c49643fe6c317bcceab93c1b9d441ab Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-03-05vlib: vfio code reworkDamjan Marion1-1/+3
Change-Id: I99cf3e7cc991aa7d32385a155c707a6516516117 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-02-15dpdk: add support for DPDK 18.02, deprecate 17.08Damjan Marion1-10/+0
17.11 is still default. Change-Id: I524d232579db8a59c717c5d760398b6b7f811d03 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-02-07Refactor vlib_buffer flagsDamjan Marion1-4/+6
Change-Id: I853386aebfe488ebb10328435b81b6e3403c5dd0 Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-11-27dpdk: fix trajectory compilationFlorin Coras1-1/+1
Change-Id: I53f917cd58101a14667641ee9cecd1eabf3f71d9 Signed-off-by: Florin Coras <fcoras@cisco.com>
2017-11-27vlib: make vlib_buffer_alloc inline functionDamjan Marion1-73/+11
Currently, every alloc involves callback function call. With this change callback function is called only if there is no empty buffers on the free list. Change-Id: I2238c19ece7ce182c49ba0f2485add52f335f3b6 Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-11-24dpdk: avoid false sharing of dpdk_buffer_per_thread_dataDamjan Marion1-0/+1
Change-Id: Iaae69a1219ccaedbfee5c3075c41c7b31c6e2b70 Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-11-23dpdk: prefetch both cachelines during buffer freeDamjan Marion1-1/+1
Change-Id: I4ae65494b8f9bc51521add5e66a51b594aad6716 Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-11-22dpdk: fix potential crash in buffer free functionDamjan Marion1-18/+49
Original code was not thread safe. Change-Id: I3d473b976d2b9ff62f42955c5c2a7fd6b4990cc2 Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-11-20dpdk: add support for DPDK 17.11Damjan Marion1-0/+10
Also remove DPDK 17.05 support. Change-Id: I4f96cb3f002cd90b12d800d6904f2364d7c4e270 Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-11-16dpdk/ipsec: use physmem when creating poolsSergio Gonzalez Monroy1-47/+61
Change-Id: Ic4f797cea6fa21fb29d646256210357cf5267b38 Signed-off-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
2017-11-14Fix typos in configure.ac and dpdk/buffer.cDamjan Marion1-8/+8
Change-Id: I74ff693315a3ffc7aa2640f25d906ca0d6da6bc5 Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-11-13dpdk: introduce AVX512 variants of node functionsDamjan Marion1-12/+52
Change-Id: If581feca0d51d0420c971801aecdf9250c671b36 Signed-off-by: Damjan Marion <damarion@cisco.com>