aboutsummaryrefslogtreecommitdiffstats
path: root/src/plugins/dpdk/cryptodev/cryptodev.c
AgeCommit message (Collapse)AuthorFilesLines
2023-09-28dpdk-cryptodev: improve dequeue behavior, fix cache stats loggingPiotr Bronowski1-25/+48
This patch provides minor improvements to the logic governing dequeuing from the ring. Previously whenever a frame was dequeued we've been trying to dequeue from the ring another one till inflight == 0. Now threshold is set for 8 frames pending in the cache to be consumed by the vnet. This threshold has been chosen based on cache ring stats observation in the system under load. Some unnecessary logic for setting deq_tail has been removed. Also logging has been corrected, and cache ring logic simplied. Type: improvement Signed-off-by: Piotr Bronowski <piotrx.bronowski@intel.com> Change-Id: I19f3daf5913006e9cb23e142a163f596e85f5bda
2023-09-07dpdk-cryptodev: fix cache ring stats cli commandPiotr Bronowski1-14/+12
The logic for calcuating processed elements in the cache ring was broken. In case tail and deq_tail equals and frame element pointed by the tile is not NULL it means there is exactly one processed element in the ring. Type: fix Signed-off-by: Piotr Bronowski <piotrx.bronowski@intel.com> Change-Id: I69c978334fc952049393214ccc9cc5245351f7f7
2023-08-18dpdk-cryptodev: improve cryptodev cache ring implementationPiotr Bronowski1-13/+42
Sw ring is renamed to the cache ring. This name better reflects the puropse of this ring. We've introduced push/pop functions, as well as other utility functions which remove code repetition. Error handlig is improved: previously in case of an error all frame elements were marked as bad, now only these for which errors occured have the error status set. Unnecessary stats counters have been removed. Type: improvement Signed-off-by: Piotr Bronowski <piotrx.bronowski@intel.com> Change-Id: I2fd42a529ac84ce5ad260611d6b35a861d441c79
2023-06-19dpdk-cryptodev: enq/deq scheme reworkPiotr Bronowski1-8/+8
This rework tries to address issues found on SPR QAT, for traffic reaching max possible throughoutput for single QAT PF packet drops were observed. Fix changes enq/deq scheme by utilizing software ring in enq call from VNET but enq and deq to QAT happens only in deq callback function what should enable better utlization of hardware resources. Type: improvement Signed-off-by: Dastin Wilski <dastin.wilski@gmail.com> Signed-off-by: Piotr Bronowski <piotrx.bronowski@intel.com> Change-Id: I2e8c473d20a269fd5e93f0c8d1f8c8aa193712bd
2023-06-13dpdk-cryptodev: introduce sw_ringPiotr Bronowski1-0/+34
This patch introduces sw_ring. This ring is used in next set of patchas and plays role of a buffer for QAT, allowing collecting frame elements in case QAT queue is fully utilized, and assembling frame from QAT dequeued elements. Type: improvement Signed-off-by: Piotr Bronowski <piotrx.bronowski@intel.com> Signed-off-by: Dastin Wilski <dastin.wilski@gmail.com> Change-Id: I20718e200986ab4dba5cbc31c05a904072a6981a
2023-04-25dpdk: code preparation for bumping to DPDK 22.11Xinyao Cai1-9/+77
This patch prepares code for bumping DPDK version to 22.11, but the DPDK version of this patch keeps at 22.07 for compatibility. the "no-dsa" parameter in DPDK configuration is removed, the "blacklist" parameter can be used to block the related DSA devices. Type: feature Signed-off-by: Xinyao Cai <xinyao.cai@intel.com> Change-Id: I08787c6584bba66383fc0a784963f33171196910
2023-03-29dpdk-cryptodev: fix name formatting of session poolsGabriel Oginski1-2/+2
Originally the name for each session pool is incorrectly prepared. It doesn't have right length. It is not null terminated. The fix corrects the name formatting for each session pool. Type: fix Signed-off-by: Gabriel Oginski <gabrielx.oginski@intel.com> Change-Id: I67da3d64702ccb27a5907825528f8c95d91040bb
2023-03-24dpdk-cryptodev: fix formatting name of poolsGabriel Oginski1-2/+2
Originally the name for each session pool can be incorrect prepared. The fix changes formatting for name for each session pool. Type: fix Signed-off-by: Gabriel Oginski <gabrielx.oginski@intel.com> Change-Id: I42e0752f9f46c5a42524ec7b863a7c9dd3c23110
2022-09-07dpdk-cryptodev: reduce request to enable asyncGabriel Oginski1-1/+0
Originally initialization cryptodev device(s) calls double request to enabled async mode and increased ref count twice for async mode. Due to this cannot be change any assigned async handlers to other async crypto engine. The fixes reduce double request to enable async mode in initialization cryptodev device(s) and VPP can be change assigned async handlers to other crypto engine after disabled all async feature, for example: ipsec, wireguard. Type: fix Signed-off-by: Gabriel Oginski <gabrielx.oginski@intel.com> Change-Id: If22e682c3c10de781d05c2e09b5420f75be151c3
2022-01-26dpdk: not having cryptodev resources should not produce warningsDamjan Marion1-4/+1
Type: fix Change-Id: Ifb2e4d93dcf8648b1bd66f4c0ee937295683bd87 Signed-off-by: Damjan Marion <damarion@cisco.com>
2022-01-24dpdk-cryptodev: add support chacha20-poly1305Gabriel Oginski1-4/+11
Originally cryptodev doesn't support chacha20-poly1305 with aad length 0. This patch add support in cryptodev for chacha20-poly1305 with aad length 0. This length is using in Wireguard. Type: improvement Signed-off-by: Gabriel Oginski <gabrielx.oginski@intel.com> Change-Id: I0608920bb557d7d071e7f9f37c80cf50bad81dcc
2022-01-12dpdk: bump to DPDK v21.11Damjan Marion1-5/+6
Type: feature This patch bumps dpdk version from 21.08 to 21.11 Change-Id: Id37fdba75f1ea4f4eac3c92226f3b1c539e1daca Signed-off-by: Dastin Wilski <dastin.wilski@gmail.com> Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-11-30dpdk-cryptodev: scalable session countDastin Wilski1-95/+135
Originally cryptodev allocates mempools for seesion and session private data during its initialization. Moreover the size of these mempools are fixed resulting in limited session count (up to value specified in CRYPTODEV_NB_SESSION macro). This patch allows for session count to scale up by allocating new mempools as they are needed during session creation. Type: improvement Signed-off-by: Dastin Wilski <dastin.wilski@gmail.com> Change-Id: I6ae240b474d3089d3ff50ca5bc7ff48f149983db
2021-06-04dpdk: silence coverity warning on use of uninitialized valuepibr1-0/+1
Type: fix Signed-off-by: Piotr Bronowski <piotrx.bronowski@intel.com> Change-Id: I76923ad5035498aae821db4fd42a127617476fbb
2021-05-07dpdk: fix cryptodev session handlerFan Zhang1-0/+4
Type: fix This patch fixes the possible segmentation fault in DPDK cryptodev when deleting session. Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com> Change-Id: Ie81de09f7250eb5c1ed6ee471363137d8372fe27
2021-04-09dpdk: selection of cryptodev engine data-pathFan Zhang1-847/+564
Type: improvement This patch combined cryptodev op and cryptodev raw API data paths into one and makes the engine run-timely select which data path is used: if all cryptodev devices support RTE_CRYPTODEV_FF_SYM_RAW_DP feature flag, the raw data path API is used, otherwise the traditional data path is used. Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com> Change-Id: Ibbd7c4405288bd9a48a34878954fd3040df7b4ad
2021-03-31dpdk: fix packet offset for GCM crypto opsRobert Shearman1-0/+1
The crypto op data offset passed into DPDK is relative to the mbuf buffer address plus the mbuf data offset, therefore the mbuf data offset needs to be set rather than left at whatever previous value it was at, which is likely to be incorrect and result in the wrong portion of the packet being encrypted/decrypted for GCM. The fe->crypto_start_offset field is relative to the start of the vlib buffer (as opposed to the current data pointer), so set the mbuf data_off field to VLIB_BUFFER_PRE_DATA_SIZE when performing a GCM crypto op enqueue to match the crypto_start_offset semantics. This then matches the behaviour in the non-GCM case. Type: fix Change-Id: I0ac2a44139387158765a3e04cfcaa5ee6f11d395 Signed-off-by: Robert Shearman <robertshearman@gmail.com>
2021-03-04dpdk: deprecate ipsec backendFan Zhang1-196/+233
Type: refactor DPDK crypto devices are now accessible via the async infra, so there is no need for the DPDK ipsec plugin. In addition this patch fixes the problem that cryptodev backend not working when master core and worker cores lies in different numa nodes. Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com> Signed-off-by: Neale Ranns <neale@graphiant.com> Change-Id: Ie8516bea706248c7bc25abac53a9c656bb8247d9
2020-12-04crypto: fixed async frame enqueue race conditionPiotrX Kleski1-1/+0
Type: fix To avoid race condition happening in async crypto engines, async frame state and thread index set should happen before enqueue. In addition as the enqueue handler already returns the enqueue status, when an enqueue is failed, the async crypto engine shall not worry about setting the async frame state but let the submit_open_frame function to do just that. Signed-off-by: PiotrX Kleski <piotrx.kleski@intel.com> Reviewed-by: Fan Zhang <roy.fan.zhang@intel.com> Change-Id: Ic1b0c94478b3cfd5fab98657218bbd70c46a220a
2020-10-21vlib: print logs to stderr if interactive or nosyslog setDamjan Marion1-4/+1
If VPP is started in interactive mode, instead of sending logs to syslog server we print them directly to stderr. Output is colorized, but that can be turned off with unix { nocolor } Type: improvement Change-Id: I9a0f0803e4cba2849a6efa0b6a86b9614ed33ced Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-09-29dpdk: fix compileFan Zhang1-49/+13
Type: fix Since DPDK is now compiled by meson but some compiles in VPP is missing. This patch fixes that. - Fixes QAT PMD not compiled. QAT meson compile, even for sym crypto PMD, is happened in drive/compress/qat. Originally all PMDs in compressdev is disabled by default. This patch fixes that. - Fixes DPDK plugin version detection. DPDK meson build generates rte_build_config.h, which containing all version information in build-dpdk instead of rte_config.h in make. This patch uses the file to detect version data. - Removed SW crypto PMD auto-creation in cryptodev engine. In case the AESNI-MB PMD required shared library is missing. Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com> Change-Id: I7cd91abb5de303ff5e4c55cd05e011b57f883524
2020-09-03crypto: SW scheduler async crypto enginePiotrX Kleski1-2/+4
Type: feature This patch adds new sw_scheduler async crypto engine. The engine transforms async frames info sync crypto ops and delegates them to active sync engines. With the patch it is possible to increase the single worker crypto throughput by offloading the crypto workload to multiple workers. By default all workers in the system will attend the crypto workload processing. However a worker's available cycles are limited. To avail more cycles to one worker to process other workload (e.g. the worker core that handles the RX/TX and IPSec stack processing), a useful cli command is added to remove itself (or add it back later) from the heavy crypto workload but only let other workers to process the crypto. The command is: - set sw_scheduler worker <idx> crypto <on|off> It also adds new interrupt mode to async crypto dispatch node. This mode signals the node when new frames are enqueued as opposed to polling mode that continuously calls dispatch node. New cli commands: - set crypto async dispatch [polling|interrupt] - show crypto async status (displays mode and nodes' states) Signed-off-by: PiotrX Kleski <piotrx.kleski@intel.com> Signed-off-by: DariuszX Kazimierski <dariuszx.kazimierski@intel.com> Reviewed-by: Fan Zhang <roy.fan.zhang@intel.com> Change-Id: I332655f347bb9e3bc9c64166e86e393e911bdb39
2020-07-23dpdk: device_id sorted order for cryptodevVladimir Ratnikov1-0/+16
By default, VPP automatically assignes for each tunnel next available QAT device by order dev_id-que-pair. In most cases we have more than one device and it can greatly increase ipsec perfomance without any actions with configuration from user if we use all the devices first and first que-pairs Type: feature Signed-off-by: Vladimir Ratnikov <vratnikov@netgate.com> Change-Id: Iac9fe74768775459e22f69bb3706b542090a9375
2020-07-16crypto: fix coverity issue for cryptodevFan Zhang1-65/+103
- Fixes coverity issue #210160. - Fixes the possible issue in cryptodev when input node does not update mbuf, such as avf-input. - Fixes GCM ESN packet incorrect tag. - Code clean up to reduce binary size. Type: fix Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com> Signed-off-by: Dariusz Kazimierski <dariuszx.kazimierski@intel.com> Signed-off-by: Piotr Kleski <piotrx.kleski@intel.com> Change-Id: Ic05ae29855ac1f7a62e4af5831a4ed9faa8f561a
2020-06-15dpdk: cryptodev: fix non-null terminated stringsBenoît Ganne1-4/+4
Type: fix Change-Id: Ib6f423e24f1a8d8439cd7e8893e4605e10984d48 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2020-05-27dpdk: bump to DPDK v20.05Damjan Marion1-0/+7
Type: improvement Change-Id: I5f89fc3d994bd85d2c5138069ea2c58661814228 Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-05-06crypto: fix coverity issuesFan Zhang1-5/+11
Fix coverity issues in crypto framework and cryptodev engine. Type: fix Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com> Change-Id: Ib261da0163c8182c803600db22c5a6dad5a19999
2020-04-30crypto: introduce async crypto infraFan Zhang1-0/+1377
Type: feature Signed-off-by: Damjan Marion <damarion@cisco.com> Signed-off-by: Filip Tehlar <ftehlar@cisco.com> Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com> Signed-off-by: Piotr Bronowski <piotrx.bronowski@intel.com> Signed-off-by: Dariusz Kazimierski <dariuszx.kazimierski@intel.com> Signed-off-by: Piotr Kleski <piotrx.kleski@intel.com> Change-Id: I4c3fcccf55c36842b7b48aed260fef2802b5c54b