summaryrefslogtreecommitdiffstats
path: root/src/plugins/dpdk/buffer.c
AgeCommit message (Collapse)AuthorFilesLines
2019-01-31buffers: vallidate that buffer is allocated during buffer pool putDamjan Marion1-1/+1
Change-Id: I8044b34a37fe1994a8dfa1ca89929f3642c72e8d Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-31dpdk: populate mempool mem_listDamjan Marion1-2/+16
Mellanox driver needs it Change-Id: I28423805f977089d18d32e85cf08e9e7e1fcf5b2 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-31dpdk: store buffer poool index in mempool pool_idDamjan Marion1-22/+10
Change-Id: I2c225a4932dcf326e10fe93de3ee4fdaef4dd3d0 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-31buffers: reinitialize metadata, add additional validationDamjan Marion1-13/+25
- DPDK overwrites metadata as part of rte_pktmbuf_init(...) so we need reinitialize it - additional checks added to ensure ref_count is never < 1 Change-Id: Ida336f81c4723e8f2e0ad4a70cb7b1ecfff978a0 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-30buffers: major cleanup and improvementsDamjan Marion1-479/+271
This patch introduces following changes: - deprecated free lists which are not used and not compatible with external buffer managers (i.e. DPDK) - introduces native support for per-numa buffer pools - significantly improves performance of buffer alloc and free Change-Id: I4a8e723ae47056717afd6cac0efe87cb731b5be7 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-20dpdk: add buffer.hDamjan Marion1-0/+1
Change-Id: I998658ad7860b23425444e218ce2e1ec655b885a Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-20buffers: remove vlib_buffer_delete_free_listDamjan Marion1-47/+0
Change-Id: I5fe01e918b534d1ac1d47f0d7610536f45c3158c Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-20buffers: don't init metadata, as it is already initializedDamjan Marion1-1/+0
Change-Id: Ia083050389853c25b069f0f8286d50d3f4aef527 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-20buffers: keep buffer_main in vlib_main_tDamjan Marion1-2/+2
Change-Id: I3bb1d9f83dd08f4b93acd4a281bfec0674e39c2e Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-19buffers: remove free-list information from buffer metadataDamjan Marion1-2/+1
Change-Id: I6048c6a51efa826ac333f7d15919cb87dd766d74 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-18deprecate clib_memcpy64_x4Damjan Marion1-8/+9
Storing buffer in local template seems to be better option.... Change-Id: I1a2fdd68cb956f99a5b36d2cd810fc623e089bcf Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-01-01buffers: remove unused codeDamjan Marion1-18/+0
Change-Id: If2bbfbc52994f5de0879763e0b7a7864498debb6 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-10-25pmalloc: support for 4K pagesDamjan Marion1-1/+1
Change-Id: Iecceffe06a92660976ebb58cd3cbec4be8931db0 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-10-23Numa-aware, growable physical memory allocator (pmalloc)Damjan Marion1-19/+15
Change-Id: Ic4c46bc733afae8bf0d8146623ed15633928de30 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-10-23c11 safe string handling supportDave Barach1-3/+3
Change-Id: Ied34720ca5a6e6e717eea4e86003e854031b6eab Signed-off-by: Dave Barach <dave@barachs.net>
2018-10-22dpdk: use rte_vfio_dma_map API instead of digging for vfio fdDamjan Marion1-63/+17
This is new API introduced in DPDK 18.08. Change-Id: I66d49fe54a6abf2af621b597e8d4535e29cecec4 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-10-16Fix coverity issue for potentially overflowing of page sizeHaiyang Tan1-3/+3
Change-Id: I2779626d745badb63386efcf729da7a094a4f297 Signed-off-by: Haiyang Tan <haiyangtan@tencent.com>
2018-09-03Deprecate old buffer replication schemeDamjan Marion1-33/+2
Change-Id: I1f54b994425c58776e1445c8d9fe142e7a644d3d Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-08-13dpdk: support for DPDK 18.08Damjan Marion1-15/+15
Change-Id: If1b93341c222160b9a08f127620c024620e55c37 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-07-18Add config option to use dlmalloc instead of mheapDave Barach1-2/+2
Configure w/ --enable-dlmalloc, see .../build-data/platforms/vpp.mk src/vppinfra/dlmalloc.[ch] are slightly modified versions of the well-known Doug Lea malloc. Main advantage: dlmalloc mspaces have no inherent size limit. Change-Id: I19b3f43f3c65bcfb82c1a265a97922d01912446e Signed-off-by: Dave Barach <dave@barachs.net>
2018-06-07Add support for DPDK 18.05Damjan Marion1-13/+20
Change-Id: I205932bc727c990011bbbe1dc6c0cf5349d19806 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-06-03dpdk: buffer free optimizationsDamjan Marion1-76/+61
~5 clocks/packet improvement... Change-Id: I1a78fa24dcd1b3ab7f45e10b9ded50f79517114a Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-06-03dpdk: improve buffer alloc perfomanceDamjan Marion1-70/+56
This is ~50% improvement in buffer alloc performance. For a 256 buffer allocation, it was ~10 clocks/buffer, now is < 5 clocks. Change-Id: I97590e240a79a42bcab5eb26587fc2d11e6eb163 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-05-30vppinfra: explicitely state for signed types that they are signedDamjan Marion1-2/+2
This fixes some compilation warnings with clang on AArch64. Change-Id: Idb941944e3f199f483c80e143a9e5163a031c4aa Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-05-29Revert "dpdk: set dmamap iova address value according to eal_iova_mode"Damjan Marion1-4/+1
This breaks VFIO operation. This reverts commit d3b3baa4f8e9e4d95264aff16fe85434ef8061bd. Change-Id: I2482e0da2d1ebfc365d13668c4b992b040f561b4 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-05-29Add VLIB_NODE_FN() macro to simplify multiversioning of node functionsDamjan Marion1-2/+2
Change-Id: Ibab5e27277f618ceb2d543b9d6a1a5f191e7d1db Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-05-29dpdk: mempool priv intialization must be done before releasing buffers to poolSachin Saxena1-8/+7
- Currently mempool priv size is getting initialized after releasing buffers to pool. This is causing mismatch in expected & real metadata size value and buffers are getting released with wrong offset. (when metadata offset is in use for a given platform) - Since private data size is 0 initially, metadata size don't include space for VLIB_BUFFER_HDR. Change-Id: I780c4d518104631a3dcf192185bacf58b3598e65 Signed-off-by: Sachin Saxena <sachin.saxena@nxp.com>
2018-05-28dpdk: set dmamap iova address value according to eal_iova_modeSachin Saxena1-1/+4
- Fix the issue where eal iova mode is Virtual Address (RTE_IOVA_VA) but setting DMA iova address to Physical address value always. Change-Id: Ib1e9c1596d95885c7eff11723338121627203e61 Signed-off-by: Sachin Saxena <sachin.saxena@nxp.com>
2018-04-30plugins: dpdk: fix check which makes not sense, likely a typoSzymon Sliwa1-1/+1
Change-Id: If33854f9c32736edf571fb66cdfa759db1c9de25 Signed-off-by: Szymon Sliwa <szs@semihalf.com>
2018-03-14vlib: internal buffer manager reworkDamjan Marion1-30/+18
- buffer_main is no longer part of vlib_main_t - pool of free lists is still part of vlib_main_t - mheap is not used anymore for buffer allocation - simple bitmap bassed buffer alloc scheme is introduced Change-Id: I3e1e6d00e2c8122293ed0a741245eb841315a1ff Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-03-09dpdk: move DPDK vfio hack to dpdk pluginDamjan Marion1-2/+67
Change-Id: I806cbf8c6c49643fe6c317bcceab93c1b9d441ab Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-03-05vlib: vfio code reworkDamjan Marion1-1/+3
Change-Id: I99cf3e7cc991aa7d32385a155c707a6516516117 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-02-15dpdk: add support for DPDK 18.02, deprecate 17.08Damjan Marion1-10/+0
17.11 is still default. Change-Id: I524d232579db8a59c717c5d760398b6b7f811d03 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-02-07Refactor vlib_buffer flagsDamjan Marion1-4/+6
Change-Id: I853386aebfe488ebb10328435b81b6e3403c5dd0 Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-11-27dpdk: fix trajectory compilationFlorin Coras1-1/+1
Change-Id: I53f917cd58101a14667641ee9cecd1eabf3f71d9 Signed-off-by: Florin Coras <fcoras@cisco.com>
2017-11-27vlib: make vlib_buffer_alloc inline functionDamjan Marion1-73/+11
Currently, every alloc involves callback function call. With this change callback function is called only if there is no empty buffers on the free list. Change-Id: I2238c19ece7ce182c49ba0f2485add52f335f3b6 Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-11-24dpdk: avoid false sharing of dpdk_buffer_per_thread_dataDamjan Marion1-0/+1
Change-Id: Iaae69a1219ccaedbfee5c3075c41c7b31c6e2b70 Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-11-23dpdk: prefetch both cachelines during buffer freeDamjan Marion1-1/+1
Change-Id: I4ae65494b8f9bc51521add5e66a51b594aad6716 Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-11-22dpdk: fix potential crash in buffer free functionDamjan Marion1-18/+49
Original code was not thread safe. Change-Id: I3d473b976d2b9ff62f42955c5c2a7fd6b4990cc2 Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-11-20dpdk: add support for DPDK 17.11Damjan Marion1-0/+10
Also remove DPDK 17.05 support. Change-Id: I4f96cb3f002cd90b12d800d6904f2364d7c4e270 Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-11-16dpdk/ipsec: use physmem when creating poolsSergio Gonzalez Monroy1-47/+61
Change-Id: Ic4f797cea6fa21fb29d646256210357cf5267b38 Signed-off-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
2017-11-14Fix typos in configure.ac and dpdk/buffer.cDamjan Marion1-8/+8
Change-Id: I74ff693315a3ffc7aa2640f25d906ca0d6da6bc5 Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-11-13dpdk: introduce AVX512 variants of node functionsDamjan Marion1-12/+52
Change-Id: If581feca0d51d0420c971801aecdf9250c671b36 Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-11-11dpdk: optimize buffer alloc/freeDamjan Marion1-49/+118
This reverts commit 45a588fa3efaaf52360986360ab1f6827bae3164. Change-Id: I7e541545791f7743ee827bdec8b6fc46cbb0938f Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-10-25vlib: add support for multiple buffer poolsDamjan Marion1-11/+11
Change-Id: Icaf7d7ad47284aea7a56e8006b69f45874d64202 Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-10-14plugins/dpdk: align memory to avoid potential segfault and false sharingGeorgina Sheehan1-1/+1
Made Update to src/plugins/dpdk/buffer.c Change-Id: I87bb8f38974a7be274c1b1d205f5513e7d068e48 Signed-off-by: Georgina <georgina.sheehan@intel.com>
2017-10-10dpdk: fix mempool size calculationDamjan Marion1-2/+3
Change-Id: I5b48310c46ca8a2143b2132110240d7e9a52c25d Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-10-04dpdk: use vpp physmem allocator for dpdk buffersDamjan Marion1-19/+66
This allows us to have single contignuous allocation for DPDK buffers with single mmap FD, so buffer memory can be easily shared with diffrent process. As a consequence dpdk socket-mem is no longer in charge for allocating buffer memory, but still we need some space allocated for dpdk structures so default socket-mem is reduced form 256 to 64 MB. For a default of 16K buffers per numa node, physmem allocation is now 40MB, so basically this change reduces footprint from 256MB per socket to 48 (64 + 40). Change-Id: Ic8cfe83930a18411545b37a12b14aac89affd04f Signed-off-by: Damjan Marion <damarion@cisco.com> Signed-off-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com> Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-09-15dpdk: cli to check for buffer leakageFlorin Coras1-1/+62
Use buffer pre_data and existing buffer trace trajectory code to find out dpdk buffer leakages. Change-Id: I26a5d8bd2f23d01cb6070ffc3ddcc6d3d863b575 Signed-off-by: Florin Coras <fcoras@cisco.com>
2017-08-25dpdk: bump to dpdk 17.08, remove support for dpdk 17.02Damjan Marion1-17/+0
Change-Id: I674fb1212e48693939045523df085326a4dd1809 Signed-off-by: Damjan Marion <damarion@cisco.com>