Age | Commit message (Collapse) | Author | Files | Lines |
|
This patch addresses coverity issues CID 322716 and CID 322717.
Type: fix
Signed-off-by: Piotr Bronowski <piotrx.bronowski@intel.com>
Change-Id: I59d6f40c1af8e829d8cb3c042a52e144aeaf1e6b
|
|
Sw ring is renamed to the cache ring. This name better reflects the
puropse of this ring. We've introduced push/pop functions, as well as
other utility functions which remove code repetition. Error handlig
is improved: previously in case of an error all frame elements were
marked as bad, now only these for which errors occured have the error
status set.
Unnecessary stats counters have been removed.
Type: improvement
Signed-off-by: Piotr Bronowski <piotrx.bronowski@intel.com>
Change-Id: I2fd42a529ac84ce5ad260611d6b35a861d441c79
|
|
This patch introduces sw_ring to the crypto op data path implementation,
so that raw data path and crypto op data path use same mechanism of processing
async frames. Crypto op ring has been removed from the implementation.
Type: improvement
Signed-off-by: Piotr Bronowski <piotrx.bronowski@intel.com>
Change-Id: Id823f80a88cfa0ff40252616a36de8bb044c7f45
|
|
When vlib buffer is processed on vnet side its length is corrected by
cipher padding and icv_sz. These changes need to be reflected in
the mbuf internals.
Type: fix
Signed-off-by: Piotr Bronowski <piotrx.bronowski@intel.com>
Change-Id: I0aa03f67f556dfc8f9a577ca1967210527221e02
|
|
This rework tries to address issues found on SPR QAT, for traffic
reaching max possible throughoutput for single QAT PF packet drops were
observed.
Fix changes enq/deq scheme by utilizing software ring in enq call from
VNET but enq and deq to QAT happens only in deq callback function what
should enable better utlization of hardware resources.
Type: improvement
Signed-off-by: Dastin Wilski <dastin.wilski@gmail.com>
Signed-off-by: Piotr Bronowski <piotrx.bronowski@intel.com>
Change-Id: I2e8c473d20a269fd5e93f0c8d1f8c8aa193712bd
|
|
This patch introduces sw_ring. This ring is used in next set of patchas
and plays role of a buffer for QAT, allowing collecting frame elements
in case QAT queue is fully utilized, and assembling frame
from QAT dequeued elements.
Type: improvement
Signed-off-by: Piotr Bronowski <piotrx.bronowski@intel.com>
Signed-off-by: Dastin Wilski <dastin.wilski@gmail.com>
Change-Id: I20718e200986ab4dba5cbc31c05a904072a6981a
|
|
This patch can make crypto dispatch node adaptively switching
between pooling and interrupt mode, and improve vpp overall
performance.
Type: improvement
Signed-off-by: Xiaoming Jiang <jiangxiaoming@outlook.com>
Change-Id: I845ed1d29ba9f3c507ea95a337f6dca7f8d6e24e
|
|
This patch prepares code for bumping DPDK version to 22.11, but the DPDK version of this patch keeps at 22.07 for compatibility.
the "no-dsa" parameter in DPDK configuration is removed, the "blacklist" parameter can be used to block the related DSA devices.
Type: feature
Signed-off-by: Xinyao Cai <xinyao.cai@intel.com>
Change-Id: I08787c6584bba66383fc0a784963f33171196910
|
|
Originally the name for each session pool is incorrectly prepared.
It doesn't have right length. It is not null terminated.
The fix corrects the name formatting for each session pool.
Type: fix
Signed-off-by: Gabriel Oginski <gabrielx.oginski@intel.com>
Change-Id: I67da3d64702ccb27a5907825528f8c95d91040bb
|
|
Originally the name for each session pool can be incorrect prepared.
The fix changes formatting for name for each session pool.
Type: fix
Signed-off-by: Gabriel Oginski <gabrielx.oginski@intel.com>
Change-Id: I42e0752f9f46c5a42524ec7b863a7c9dd3c23110
|
|
Originally initialization cryptodev device(s) calls double request
to enabled async mode and increased ref count twice for async mode.
Due to this cannot be change any assigned async handlers to other
async crypto engine.
The fixes reduce double request to enable async mode in initialization
cryptodev device(s) and VPP can be change assigned async handlers
to other crypto engine after disabled all async feature, for example:
ipsec, wireguard.
Type: fix
Signed-off-by: Gabriel Oginski <gabrielx.oginski@intel.com>
Change-Id: If22e682c3c10de781d05c2e09b5420f75be151c3
|
|
Type: fix
Change-Id: Ifb2e4d93dcf8648b1bd66f4c0ee937295683bd87
Signed-off-by: Damjan Marion <damarion@cisco.com>
|
|
Originally cryptodev doesn't support chacha20-poly1305 with aad length
0.
This patch add support in cryptodev for chacha20-poly1305 with aad
length 0. This length is using in Wireguard.
Type: improvement
Signed-off-by: Gabriel Oginski <gabrielx.oginski@intel.com>
Change-Id: I0608920bb557d7d071e7f9f37c80cf50bad81dcc
|
|
Type: feature
This patch bumps dpdk version from 21.08 to 21.11
Change-Id: Id37fdba75f1ea4f4eac3c92226f3b1c539e1daca
Signed-off-by: Dastin Wilski <dastin.wilski@gmail.com>
Signed-off-by: Damjan Marion <damarion@cisco.com>
|
|
Type: fix
Change-Id: I7aa172e58c970c4971db6ef2ff5b199b7f3c0b99
Signed-off-by: Dastin Wilski <dastin.wilski@gmail.com>
|
|
Type: improvement
Previously multiple sw crypto scheduler queues per core design
caused unaverage frame processing rate for each async op ID –
the lower the op ID is the highly likely they are processed first.
For example, when a RX core is feeding both encryption and
decryption jobs of the same crypto algorithm to the queues at a
high rate, in the mean time the crypto cores have no enough
cycles to process all: the jobs in the decryption queue are less
likely being processed, causing packet drop.
To improve the situation this patch makes every core only owning
a two queues, one for encrypt operations and one for decrypt.
The queue is changed either after checking each core
or after founding a frame to process.
All crypto jobs with different algorithm are pushed to
thoses queues and are treated evenly.
In addition, the crypto async infra now uses unified dequeue handler,
one per engine. Only the active engine will be registered its
dequeue handler in crypto main.
Signed-off-by: DariuszX Kazimierski <dariuszx.kazimierski@intel.com>
Signed-off-by: PiotrX Kleski <piotrx.kleski@intel.com>
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Signed-off-by: Jakub Wysocki <jakubx.wysocki@intel.com>
Change-Id: I517ee8e31633980de5e0dd4b05e1d5db5dea760e
|
|
Type: improvement
This patch adds AES-CTR-128/192/256 + SHA1 linked algo support to dpdk
cryptodev.
Signed-off-by: PiotrX Kleski <piotrx.kleski@intel.com>
Change-Id: Idc162b29f4075ef8be9577abd3daf6de05f84faa
|
|
Originally cryptodev allocates mempools for seesion and session private
data during its initialization. Moreover the size of these mempools are
fixed resulting in limited session count (up to value specified in
CRYPTODEV_NB_SESSION macro).
This patch allows for session count to scale up by allocating new
mempools as they are needed during session creation.
Type: improvement
Signed-off-by: Dastin Wilski <dastin.wilski@gmail.com>
Change-Id: I6ae240b474d3089d3ff50ca5bc7ff48f149983db
|
|
CLIB_PREFETCH (cop[1], CLIB_CACHE_LINE_BYTES * 3, STORE);
Note on 64 bytes cache line size arm machines,
CLIB_CACHE_LINE_BYTES 128
CLIB_CACHE_PREFETCH_BYTES 6
above CLIB_PREFETCH () macro will be expand to
ASSERT ((size) <= 4 * CLIB_CACHE_PREFETCH_BYTES);
it will hit assert due to size (i.e. 3 * 128) > 4 * 64
Solution:
Change to CLIB_PREFETCH (cop[1], sizeof(*cop[1]), STORE);
Type: fix
Signed-off-by: Tianyu Li <tianyu.li@arm.com>
Reviewed-by: Lijian Zhang <lijian.zhang@arm.com>
Change-Id: Id0981fd5bd2b25ff71db4197b25578d0b7a9803e
|
|
Type: refactor
Change-Id: Id10cbf52e8f2dd809080a228d8fa282308be84ac
Signed-off-by: Damjan Marion <damarion@cisco.com>
|
|
Type: fix
Signed-off-by: Piotr Bronowski <piotrx.bronowski@intel.com>
Change-Id: I76923ad5035498aae821db4fd42a127617476fbb
|
|
Type: feature
This patch bumps DPDK version to 21.05 and updated VPP to
accomodate the changes in DPDK latest version.
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Change-Id: If217441f70c9ab531196dca7ec7a486ec9931cff
|
|
Type: fix
This patch fixes the missing symbol of dpdk_plugin.so when
creating symmetric key. The solution is to add dependency
of libssl to dpdk cryptodev and disable cryptodev engine
when libssl is not presented.
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Change-Id: I30aa6e3e3af1faefa82883bad613e1d82235a2ec
|
|
Type: fix
This patch fixes the possible segmentation fault in DPDK
cryptodev when deleting session.
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Change-Id: Ie81de09f7250eb5c1ed6ee471363137d8372fe27
|
|
With this change, enable MD5 with AES-CBC support in dpdk cryptodev.
Type: improvement
Signed-off-by: Alexander Chernavin <achernavin@netgate.com>
Change-Id: Ic587aaa1fa0dc102e36eb34f329ef21a16156f26
|
|
Type: improvement
This patch combined cryptodev op and cryptodev raw API data
paths into one and makes the engine run-timely select which
data path is used: if all cryptodev devices support
RTE_CRYPTODEV_FF_SYM_RAW_DP feature flag, the raw data path
API is used, otherwise the traditional data path is used.
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Change-Id: Ibbd7c4405288bd9a48a34878954fd3040df7b4ad
|
|
The crypto op data offset passed into DPDK is relative to the mbuf
buffer address plus the mbuf data offset, therefore the mbuf data
offset needs to be set rather than left at whatever previous value it
was at, which is likely to be incorrect and result in the wrong
portion of the packet being encrypted/decrypted for GCM.
The fe->crypto_start_offset field is relative to the start of the vlib
buffer (as opposed to the current data pointer), so set the mbuf
data_off field to VLIB_BUFFER_PRE_DATA_SIZE when performing a GCM
crypto op enqueue to match the crypto_start_offset semantics. This
then matches the behaviour in the non-GCM case.
Type: fix
Change-Id: I0ac2a44139387158765a3e04cfcaa5ee6f11d395
Signed-off-by: Robert Shearman <robertshearman@gmail.com>
|
|
Type: improvement
Change-Id: If3da7d4338470912f37ff1794620418d928fb77f
Signed-off-by: Damjan Marion <damarion@cisco.com>
|
|
Ensure that dpdk_cryptodev engine async handlers are registered for the set of algorithms supported by all cryptodevices in the system.
Type: improvement
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Signed-off-by: Piotr Bronowski <piotrx.bronowski@intel.com>
Change-Id: I8293d87c6f07e935a1a0637704f24918cd3e132a
|
|
Type: refactor
DPDK crypto devices are now accessible via the async infra, so
there is no need for the DPDK ipsec plugin.
In addition this patch fixes the problem that cryptodev backend
not working when master core and worker cores lies in different
numa nodes.
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Signed-off-by: Neale Ranns <neale@graphiant.com>
Change-Id: Ie8516bea706248c7bc25abac53a9c656bb8247d9
|
|
Type: fix
This patch fixes the missed crypto and integ offset update for
every packet. Previously the offset is updated only when the
key is changed. This is ok for encryption but not always true
for decryption.
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Change-Id: Iccd0011f4ae488746ce487a14b94ddd24fb0c07c
|
|
Type: feature
This patch rebase cryptodev engine for the new cryptodev
raw APIs introduced in DPDK 20.11.
Signed-off-by: Piotr Bronowski <PiotrX.Bronowski@intel.com>
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Change-Id: I4da335379c5dfeb358017092086d305a01b652dc
|
|
Type: fix
To avoid race condition happening in async crypto engines,
async frame state and thread index set should happen before enqueue.
In addition as the enqueue handler already returns the enqueue status,
when an enqueue is failed, the async crypto engine shall not worry
about setting the async frame state but let the submit_open_frame function
to do just that.
Signed-off-by: PiotrX Kleski <piotrx.kleski@intel.com>
Reviewed-by: Fan Zhang <roy.fan.zhang@intel.com>
Change-Id: Ic1b0c94478b3cfd5fab98657218bbd70c46a220a
|
|
If VPP is started in interactive mode, instead of sending logs to syslog
server we print them directly to stderr.
Output is colorized, but that can be turned off with unix { nocolor }
Type: improvement
Change-Id: I9a0f0803e4cba2849a6efa0b6a86b9614ed33ced
Signed-off-by: Damjan Marion <damarion@cisco.com>
|
|
Type: fix
Since DPDK is now compiled by meson but some compiles in VPP is
missing. This patch fixes that.
- Fixes QAT PMD not compiled. QAT meson compile, even for sym
crypto PMD, is happened in drive/compress/qat. Originally all
PMDs in compressdev is disabled by default. This patch fixes
that.
- Fixes DPDK plugin version detection. DPDK meson build
generates rte_build_config.h, which containing all version
information in build-dpdk instead of rte_config.h in make.
This patch uses the file to detect version data.
- Removed SW crypto PMD auto-creation in cryptodev engine. In
case the AESNI-MB PMD required shared library is missing.
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Change-Id: I7cd91abb5de303ff5e4c55cd05e011b57f883524
|
|
Type: feature
This patch updateds cryptodev engine uses new DPDK Cryptodev
API planned to be upstreamed in DPDK 20.11.
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Signed-off-by: Piotr Bronowski <piotrX.bronowski@intel.com>
Change-Id: I8dd1a8ac643f1e952deb787e466b76ea7aa5f420
|
|
Type: feature
This patch adds new sw_scheduler async crypto engine.
The engine transforms async frames info sync crypto ops and
delegates them to active sync engines. With the patch it
is possible to increase the single worker crypto throughput
by offloading the crypto workload to multiple workers.
By default all workers in the system will attend the crypto
workload processing. However a worker's available cycles
are limited. To avail more cycles to one worker to process
other workload (e.g. the worker core that handles the RX/TX
and IPSec stack processing), a useful cli command is added
to remove itself (or add it back later) from the heavy
crypto workload but only let other workers to process the
crypto. The command is:
- set sw_scheduler worker <idx> crypto <on|off>
It also adds new interrupt mode to async crypto dispatch node.
This mode signals the node when new frames are enqueued
as opposed to polling mode that continuously calls dispatch node.
New cli commands:
- set crypto async dispatch [polling|interrupt]
- show crypto async status (displays mode and nodes' states)
Signed-off-by: PiotrX Kleski <piotrx.kleski@intel.com>
Signed-off-by: DariuszX Kazimierski <dariuszx.kazimierski@intel.com>
Reviewed-by: Fan Zhang <roy.fan.zhang@intel.com>
Change-Id: I332655f347bb9e3bc9c64166e86e393e911bdb39
|
|
By default, VPP automatically assignes for each tunnel
next available QAT device by order dev_id-que-pair.
In most cases we have more than one device and it can
greatly increase ipsec perfomance without any actions
with configuration from user if we use all the
devices first and first que-pairs
Type: feature
Signed-off-by: Vladimir Ratnikov <vratnikov@netgate.com>
Change-Id: Iac9fe74768775459e22f69bb3706b542090a9375
|
|
- Fixes coverity issue #210160.
- Fixes the possible issue in cryptodev when input node does
not update mbuf, such as avf-input.
- Fixes GCM ESN packet incorrect tag.
- Code clean up to reduce binary size.
Type: fix
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Signed-off-by: Dariusz Kazimierski <dariuszx.kazimierski@intel.com>
Signed-off-by: Piotr Kleski <piotrx.kleski@intel.com>
Change-Id: Ic05ae29855ac1f7a62e4af5831a4ed9faa8f561a
|
|
Type: fix
Change-Id: Ib6f423e24f1a8d8439cd7e8893e4605e10984d48
Signed-off-by: Benoît Ganne <bganne@cisco.com>
|
|
Type: improvement
Change-Id: I5f89fc3d994bd85d2c5138069ea2c58661814228
Signed-off-by: Damjan Marion <damarion@cisco.com>
|
|
Fix coverity issues in crypto framework and cryptodev
engine.
Type: fix
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Change-Id: Ib261da0163c8182c803600db22c5a6dad5a19999
|
|
Type: feature
Signed-off-by: Damjan Marion <damarion@cisco.com>
Signed-off-by: Filip Tehlar <ftehlar@cisco.com>
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Signed-off-by: Piotr Bronowski <piotrx.bronowski@intel.com>
Signed-off-by: Dariusz Kazimierski <dariuszx.kazimierski@intel.com>
Signed-off-by: Piotr Kleski <piotrx.kleski@intel.com>
Change-Id: I4c3fcccf55c36842b7b48aed260fef2802b5c54b
|