aboutsummaryrefslogtreecommitdiffstats
path: root/src/plugins/dpdk/cryptodev
AgeCommit message (Collapse)AuthorFilesLines
2022-01-05crypto: encrypt/decrypt queues sw_schedulerJakub Wysocki2-20/+29
Type: improvement Previously multiple sw crypto scheduler queues per core design caused unaverage frame processing rate for each async op ID – the lower the op ID is the highly likely they are processed first. For example, when a RX core is feeding both encryption and decryption jobs of the same crypto algorithm to the queues at a high rate, in the mean time the crypto cores have no enough cycles to process all: the jobs in the decryption queue are less likely being processed, causing packet drop. To improve the situation this patch makes every core only owning a two queues, one for encrypt operations and one for decrypt. The queue is changed either after checking each core or after founding a frame to process. All crypto jobs with different algorithm are pushed to thoses queues and are treated evenly. In addition, the crypto async infra now uses unified dequeue handler, one per engine. Only the active engine will be registered its dequeue handler in crypto main. Signed-off-by: DariuszX Kazimierski <dariuszx.kazimierski@intel.com> Signed-off-by: PiotrX Kleski <piotrx.kleski@intel.com> Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com> Signed-off-by: Jakub Wysocki <jakubx.wysocki@intel.com> Change-Id: I517ee8e31633980de5e0dd4b05e1d5db5dea760e
2021-11-30dpdk: cryptodev AES-CTR+SHA1 support addedPiotrX Kleski1-1/+4
Type: improvement This patch adds AES-CTR-128/192/256 + SHA1 linked algo support to dpdk cryptodev. Signed-off-by: PiotrX Kleski <piotrx.kleski@intel.com> Change-Id: Idc162b29f4075ef8be9577abd3daf6de05f84faa
2021-11-30dpdk-cryptodev: scalable session countDastin Wilski2-96/+143
Originally cryptodev allocates mempools for seesion and session private data during its initialization. Moreover the size of these mempools are fixed resulting in limited session count (up to value specified in CRYPTODEV_NB_SESSION macro). This patch allows for session count to scale up by allocating new mempools as they are needed during session creation. Type: improvement Signed-off-by: Dastin Wilski <dastin.wilski@gmail.com> Change-Id: I6ae240b474d3089d3ff50ca5bc7ff48f149983db
2021-09-13dpdk: fix prefetch assert on ArmTianyu Li1-4/+4
CLIB_PREFETCH (cop[1], CLIB_CACHE_LINE_BYTES * 3, STORE); Note on 64 bytes cache line size arm machines, CLIB_CACHE_LINE_BYTES 128 CLIB_CACHE_PREFETCH_BYTES 6 above CLIB_PREFETCH () macro will be expand to ASSERT ((size) <= 4 * CLIB_CACHE_PREFETCH_BYTES); it will hit assert due to size (i.e. 3 * 128) > 4 * 64 Solution: Change to CLIB_PREFETCH (cop[1], sizeof(*cop[1]), STORE); Type: fix Signed-off-by: Tianyu Li <tianyu.li@arm.com> Reviewed-by: Lijian Zhang <lijian.zhang@arm.com> Change-Id: Id0981fd5bd2b25ff71db4197b25578d0b7a9803e
2021-07-15misc: replace CLIB_PREFETCH with clib_prefetch_{load,store}Damjan Marion2-9/+9
Type: refactor Change-Id: Id10cbf52e8f2dd809080a228d8fa282308be84ac Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-06-04dpdk: silence coverity warning on use of uninitialized valuepibr1-0/+1
Type: fix Signed-off-by: Piotr Bronowski <piotrx.bronowski@intel.com> Change-Id: I76923ad5035498aae821db4fd42a127617476fbb
2021-05-27dpdk: bump to 21.05Fan Zhang1-0/+13
Type: feature This patch bumps DPDK version to 21.05 and updated VPP to accomodate the changes in DPDK latest version. Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com> Change-Id: If217441f70c9ab531196dca7ec7a486ec9931cff
2021-05-26dpdk: fix missing symbolFan Zhang1-2/+1
Type: fix This patch fixes the missing symbol of dpdk_plugin.so when creating symmetric key. The solution is to add dependency of libssl to dpdk cryptodev and disable cryptodev engine when libssl is not presented. Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com> Change-Id: I30aa6e3e3af1faefa82883bad613e1d82235a2ec
2021-05-07dpdk: fix cryptodev session handlerFan Zhang1-0/+4
Type: fix This patch fixes the possible segmentation fault in DPDK cryptodev when deleting session. Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com> Change-Id: Ie81de09f7250eb5c1ed6ee471363137d8372fe27
2021-04-23dpdk: support md5 async handler in dpdk cryptodevAlexander Chernavin1-0/+3
With this change, enable MD5 with AES-CBC support in dpdk cryptodev. Type: improvement Signed-off-by: Alexander Chernavin <achernavin@netgate.com> Change-Id: Ic587aaa1fa0dc102e36eb34f329ef21a16156f26
2021-04-09dpdk: selection of cryptodev engine data-pathFan Zhang5-2727/+2074
Type: improvement This patch combined cryptodev op and cryptodev raw API data paths into one and makes the engine run-timely select which data path is used: if all cryptodev devices support RTE_CRYPTODEV_FF_SYM_RAW_DP feature flag, the raw data path API is used, otherwise the traditional data path is used. Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com> Change-Id: Ibbd7c4405288bd9a48a34878954fd3040df7b4ad
2021-03-31dpdk: fix packet offset for GCM crypto opsRobert Shearman1-0/+1
The crypto op data offset passed into DPDK is relative to the mbuf buffer address plus the mbuf data offset, therefore the mbuf data offset needs to be set rather than left at whatever previous value it was at, which is likely to be incorrect and result in the wrong portion of the packet being encrypted/decrypted for GCM. The fe->crypto_start_offset field is relative to the start of the vlib buffer (as opposed to the current data pointer), so set the mbuf data_off field to VLIB_BUFFER_PRE_DATA_SIZE when performing a GCM crypto op enqueue to match the crypto_start_offset semantics. This then matches the behaviour in the non-GCM case. Type: fix Change-Id: I0ac2a44139387158765a3e04cfcaa5ee6f11d395 Signed-off-by: Robert Shearman <robertshearman@gmail.com>
2021-03-26vlib: introduce vlib_get_main_by_index(), vlib_get_n_threads()Damjan Marion1-1/+1
Type: improvement Change-Id: If3da7d4338470912f37ff1794620418d928fb77f Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-03-17dpdk: cryptodev session managementpibr1-108/+377
Ensure that dpdk_cryptodev engine async handlers are registered for the set of algorithms supported by all cryptodevices in the system. Type: improvement Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com> Signed-off-by: Piotr Bronowski <piotrx.bronowski@intel.com> Change-Id: I8293d87c6f07e935a1a0637704f24918cd3e132a
2021-03-04dpdk: deprecate ipsec backendFan Zhang2-467/+578
Type: refactor DPDK crypto devices are now accessible via the async infra, so there is no need for the DPDK ipsec plugin. In addition this patch fixes the problem that cryptodev backend not working when master core and worker cores lies in different numa nodes. Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com> Signed-off-by: Neale Ranns <neale@graphiant.com> Change-Id: Ie8516bea706248c7bc25abac53a9c656bb8247d9
2021-02-19dpdk: fix cryptodev offset updateFan Zhang1-11/+8
Type: fix This patch fixes the missed crypto and integ offset update for every packet. Previously the offset is updated only when the key is changed. This is ok for encryption but not always true for decryption. Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com> Change-Id: Iccd0011f4ae488746ce487a14b94ddd24fb0c07c
2020-12-18dpdk: rebase cryptodev engine for dpdk 20.11Fan Zhang1-120/+126
Type: feature This patch rebase cryptodev engine for the new cryptodev raw APIs introduced in DPDK 20.11. Signed-off-by: Piotr Bronowski <PiotrX.Bronowski@intel.com> Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com> Change-Id: I4da335379c5dfeb358017092086d305a01b652dc
2020-12-04crypto: fixed async frame enqueue race conditionPiotrX Kleski1-1/+0
Type: fix To avoid race condition happening in async crypto engines, async frame state and thread index set should happen before enqueue. In addition as the enqueue handler already returns the enqueue status, when an enqueue is failed, the async crypto engine shall not worry about setting the async frame state but let the submit_open_frame function to do just that. Signed-off-by: PiotrX Kleski <piotrx.kleski@intel.com> Reviewed-by: Fan Zhang <roy.fan.zhang@intel.com> Change-Id: Ic1b0c94478b3cfd5fab98657218bbd70c46a220a
2020-10-21vlib: print logs to stderr if interactive or nosyslog setDamjan Marion2-8/+2
If VPP is started in interactive mode, instead of sending logs to syslog server we print them directly to stderr. Output is colorized, but that can be turned off with unix { nocolor } Type: improvement Change-Id: I9a0f0803e4cba2849a6efa0b6a86b9614ed33ced Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-09-29dpdk: fix compileFan Zhang1-49/+13
Type: fix Since DPDK is now compiled by meson but some compiles in VPP is missing. This patch fixes that. - Fixes QAT PMD not compiled. QAT meson compile, even for sym crypto PMD, is happened in drive/compress/qat. Originally all PMDs in compressdev is disabled by default. This patch fixes that. - Fixes DPDK plugin version detection. DPDK meson build generates rte_build_config.h, which containing all version information in build-dpdk instead of rte_config.h in make. This patch uses the file to detect version data. - Removed SW crypto PMD auto-creation in cryptodev engine. In case the AESNI-MB PMD required shared library is missing. Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com> Change-Id: I7cd91abb5de303ff5e4c55cd05e011b57f883524
2020-09-09crypto: change cryptodev with new cryptodev APIFan Zhang1-0/+1536
Type: feature This patch updateds cryptodev engine uses new DPDK Cryptodev API planned to be upstreamed in DPDK 20.11. Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com> Signed-off-by: Piotr Bronowski <piotrX.bronowski@intel.com> Change-Id: I8dd1a8ac643f1e952deb787e466b76ea7aa5f420
2020-09-03crypto: SW scheduler async crypto enginePiotrX Kleski1-2/+4
Type: feature This patch adds new sw_scheduler async crypto engine. The engine transforms async frames info sync crypto ops and delegates them to active sync engines. With the patch it is possible to increase the single worker crypto throughput by offloading the crypto workload to multiple workers. By default all workers in the system will attend the crypto workload processing. However a worker's available cycles are limited. To avail more cycles to one worker to process other workload (e.g. the worker core that handles the RX/TX and IPSec stack processing), a useful cli command is added to remove itself (or add it back later) from the heavy crypto workload but only let other workers to process the crypto. The command is: - set sw_scheduler worker <idx> crypto <on|off> It also adds new interrupt mode to async crypto dispatch node. This mode signals the node when new frames are enqueued as opposed to polling mode that continuously calls dispatch node. New cli commands: - set crypto async dispatch [polling|interrupt] - show crypto async status (displays mode and nodes' states) Signed-off-by: PiotrX Kleski <piotrx.kleski@intel.com> Signed-off-by: DariuszX Kazimierski <dariuszx.kazimierski@intel.com> Reviewed-by: Fan Zhang <roy.fan.zhang@intel.com> Change-Id: I332655f347bb9e3bc9c64166e86e393e911bdb39
2020-07-23dpdk: device_id sorted order for cryptodevVladimir Ratnikov1-0/+16
By default, VPP automatically assignes for each tunnel next available QAT device by order dev_id-que-pair. In most cases we have more than one device and it can greatly increase ipsec perfomance without any actions with configuration from user if we use all the devices first and first que-pairs Type: feature Signed-off-by: Vladimir Ratnikov <vratnikov@netgate.com> Change-Id: Iac9fe74768775459e22f69bb3706b542090a9375
2020-07-16crypto: fix coverity issue for cryptodevFan Zhang1-65/+103
- Fixes coverity issue #210160. - Fixes the possible issue in cryptodev when input node does not update mbuf, such as avf-input. - Fixes GCM ESN packet incorrect tag. - Code clean up to reduce binary size. Type: fix Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com> Signed-off-by: Dariusz Kazimierski <dariuszx.kazimierski@intel.com> Signed-off-by: Piotr Kleski <piotrx.kleski@intel.com> Change-Id: Ic05ae29855ac1f7a62e4af5831a4ed9faa8f561a
2020-06-15dpdk: cryptodev: fix non-null terminated stringsBenoît Ganne1-4/+4
Type: fix Change-Id: Ib6f423e24f1a8d8439cd7e8893e4605e10984d48 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2020-05-27dpdk: bump to DPDK v20.05Damjan Marion1-0/+7
Type: improvement Change-Id: I5f89fc3d994bd85d2c5138069ea2c58661814228 Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-05-06crypto: fix coverity issuesFan Zhang1-5/+11
Fix coverity issues in crypto framework and cryptodev engine. Type: fix Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com> Change-Id: Ib261da0163c8182c803600db22c5a6dad5a19999
2020-04-30crypto: introduce async crypto infraFan Zhang2-0/+1402
Type: feature Signed-off-by: Damjan Marion <damarion@cisco.com> Signed-off-by: Filip Tehlar <ftehlar@cisco.com> Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com> Signed-off-by: Piotr Bronowski <piotrx.bronowski@intel.com> Signed-off-by: Dariusz Kazimierski <dariuszx.kazimierski@intel.com> Signed-off-by: Piotr Kleski <piotrx.kleski@intel.com> Change-Id: I4c3fcccf55c36842b7b48aed260fef2802b5c54b