aboutsummaryrefslogtreecommitdiffstats
path: root/src/plugins/crypto_sw_scheduler/main.c
AgeCommit message (Collapse)AuthorFilesLines
2023-09-12crypto-sw-scheduler: improve function indentationVratko Polak1-159/+156
The checkstyle --fix command remains confused around the def/foreach/undef usage in convert_async_crypto_id, but at least the other functions now look correctly indented to me. Type: style Change-Id: Ic8f7b580267386b7a6b07d33d9ba7ae9787c0e0a Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2023-08-10crypto-sw-scheduler: avoid crypto work on vpp_mainVratko Polak1-16/+18
+ Checkstyle demanded indentation edits. Type: fix Ticket: VPP-2083 Fixes: 9a9604b09f15691d7c4ddf29afd99a31e7e31eed Change-Id: Ie2d33d290330247d36435a073675b732bb64ae93 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2023-06-01crypto: make crypto-dispatch node working in adaptive modeXiaoming Jiang1-12/+2
This patch can make crypto dispatch node adaptively switching between pooling and interrupt mode, and improve vpp overall performance. Type: improvement Signed-off-by: Xiaoming Jiang <jiangxiaoming@outlook.com> Change-Id: I845ed1d29ba9f3c507ea95a337f6dca7f8d6e24e
2023-04-25crypto-sw-scheduler: fix interrupt modeAlexander Skorichenko1-0/+10
Type: fix Currently sw_scheduler runs interchangeably over queues of one selected type either ENCRYPT or DECRYPT. Then switches the type for the next run. This works fine in polling mode as missed frames get processed on the next run. In interrupt mode if all of the workers miss a frame on the first run the interrupt flag is lowered so the frame remains pending in queues waiting for another crypto event to raise the interrupt. With this fix force sw_scheduler in interrupt mode check the second half of the queues if the first pass returned no results. This guarantees a pending frame gets into processing before interrupt is reset. Change-Id: I7e91d125702336eba72c6a3abaeabcae010d396a Signed-off-by: Alexander Skorichenko <askorichenko@netgate.com>
2023-03-06crypto: remove VNET_CRYPTO_OP_FLAG_INIT_IV flagBenoît Ganne1-1/+1
IV requirements vary wildly with the selected mode of operation. For example, for AES-CBC the IV must be unpredictable whereas for AES counter mode (CTR or GCM), it can be predictable but reusing an IV with the same key material is catastrophic. Because of that, it is hard to generate IV in a generic way, and it is better left to the crypto user (eg. IPsec). Type: improvement Change-Id: I32689c591d8c6572b8d37c4d24f175ea6132d3ec Signed-off-by: Benoît Ganne <bganne@cisco.com>
2022-11-18ipsec: Failure at the start of the batch should not invalidate the rest of ↵Neale Ranns1-14/+24
the batch Type: fix Signed-off-by: Neale Ranns <neale@graphiant.com> Change-Id: Icd1e43a5764496784c355c93066273435f16dd35
2022-10-20crypto-sw-scheduler: fix queue iteratorVladimir Ratnikov1-0/+10
When there are several workers, iterator can and will skip head iterator and it will last until BARRIER_SYNC_TIMEOUT won't expire and will cause SIGABRT with `worker thread deadlock` Type: fix Signed-off-by: Vladimir Ratnikov <vratnikov@netgate.com> Change-Id: Id4def4d5894e077ae27592367b141ecd822e86af Signed-off-by: Matthew Smith <mgsmith@netgate.com>
2022-01-05crypto: encrypt/decrypt queues sw_schedulerJakub Wysocki1-214/+247
Type: improvement Previously multiple sw crypto scheduler queues per core design caused unaverage frame processing rate for each async op ID – the lower the op ID is the highly likely they are processed first. For example, when a RX core is feeding both encryption and decryption jobs of the same crypto algorithm to the queues at a high rate, in the mean time the crypto cores have no enough cycles to process all: the jobs in the decryption queue are less likely being processed, causing packet drop. To improve the situation this patch makes every core only owning a two queues, one for encrypt operations and one for decrypt. The queue is changed either after checking each core or after founding a frame to process. All crypto jobs with different algorithm are pushed to thoses queues and are treated evenly. In addition, the crypto async infra now uses unified dequeue handler, one per engine. Only the active engine will be registered its dequeue handler in crypto main. Signed-off-by: DariuszX Kazimierski <dariuszx.kazimierski@intel.com> Signed-off-by: PiotrX Kleski <piotrx.kleski@intel.com> Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com> Signed-off-by: Jakub Wysocki <jakubx.wysocki@intel.com> Change-Id: I517ee8e31633980de5e0dd4b05e1d5db5dea760e
2021-07-15misc: replace CLIB_PREFETCH with clib_prefetch_{load,store}Damjan Marion1-2/+2
Type: refactor Change-Id: Id10cbf52e8f2dd809080a228d8fa282308be84ac Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-07-02crypto: fix sw async crypto with chained buffersBenoît Ganne1-31/+30
When a buffer in the chain comes with a negative current_data offset, the conversion to sgl will skip it because of resetting offset to 0. Moreover, crypto_start_offset is relative to the 1st buffer data pointer so we should not check it against subsequent buffers anyway. Type: fix Change-Id: Id177a90bfda242a5372c7e8836cf6668e98c780e Signed-off-by: Benoît Ganne <bganne@cisco.com>
2021-04-29crypto: fix chained buffer integrity supportBenoît Ganne1-1/+15
Type: fix Change-Id: I984a3e577a4209e41d046eaf3a8eef8986dc6147 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2021-04-07crypto: fix ops flags in crypto sw schedulerBenoît Ganne1-7/+2
The crypto op flags must be reset to frame flags minus invalid values depending of the operation, instead of forcing them to specific values. Type: fix Change-Id: Ib02c2a738bbca6962394b3c03088d516d0da56a0 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2021-03-05crypto: revert "fix ops flags in crypto sw scheduler"Damjan Marion1-2/+7
This reverts commit 30ad571cc35e4dc6d4d7e50b81b97f83f8770eea. Type: fix Change-Id: If8c6e388e732d2a1b5efd0677d9528a646365f94 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-03-04crypto: fix ops flags in crypto sw schedulerBenoît Ganne1-7/+2
The sw crypto scheduler converts crypto frames to individual crypto operations. This is done by reusing per-thread vectors for crypto, integrity and chained operations. The crypto op flags must be reset to frame flags minus invalid values depending of the operation. The previous tentative also cleared the chained buffer flag, breaking jumbo support. Type: fix Change-Id: Icce6887a9e0dae8c300c56e97b977e203e784713 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2021-02-09crypto: fix bad-hmac in sw scheduler if async modeAlexander Chernavin1-0/+1
When IPsec async mode is enabled, packets don't pass through the tunnel if ciphers other than AES GCM are used for child SAs. An error that arises is "bad-hmac" in the "crypto-dispatch" node. On the encryption stage, the VNET_CRYPTO_OP_FLAG_HMAC_CHECK flag is set for the integrity crypto operation when it's not supposed to. It seems that the flag remains from the previous operation. With this change, zero flags of crypto operations in the SW scheduler during operation filling. Type: fix Signed-off-by: Alexander Chernavin <achernavin@netgate.com> Change-Id: Iabac253474e95cb01f9ec0933f3c4860f8a5289c
2020-12-04crypto: fixed async frame enqueue race conditionPiotrX Kleski1-2/+0
Type: fix To avoid race condition happening in async crypto engines, async frame state and thread index set should happen before enqueue. In addition as the enqueue handler already returns the enqueue status, when an enqueue is failed, the async crypto engine shall not worry about setting the async frame state but let the submit_open_frame function to do just that. Signed-off-by: PiotrX Kleski <piotrx.kleski@intel.com> Reviewed-by: Fan Zhang <roy.fan.zhang@intel.com> Change-Id: Ic1b0c94478b3cfd5fab98657218bbd70c46a220a
2020-09-15crypto: Crypto SW Scheduler Coverity WarningsNeale Ranns1-1/+1
Type: fix Signed-off-by: Neale Ranns <nranns@cisco.com> Change-Id: I7f98d0c7847ecc40b90b78e5ae83f320575be310
2020-09-03crypto: Add async crypto APIsNathan Skrzypczak1-0/+2
Type: feature This adds api calls for the following CLIs: * set sw_scheuduler worker <N> crypto on|off * set crypto async dispatch polling|interrupt * set crypto handler * set crypto async handler Change-Id: Ic701d149c440e42ea4575da42b9f69e4c8759602 Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com>
2020-09-03crypto: SW scheduler async crypto enginePiotrX Kleski1-0/+713
Type: feature This patch adds new sw_scheduler async crypto engine. The engine transforms async frames info sync crypto ops and delegates them to active sync engines. With the patch it is possible to increase the single worker crypto throughput by offloading the crypto workload to multiple workers. By default all workers in the system will attend the crypto workload processing. However a worker's available cycles are limited. To avail more cycles to one worker to process other workload (e.g. the worker core that handles the RX/TX and IPSec stack processing), a useful cli command is added to remove itself (or add it back later) from the heavy crypto workload but only let other workers to process the crypto. The command is: - set sw_scheduler worker <idx> crypto <on|off> It also adds new interrupt mode to async crypto dispatch node. This mode signals the node when new frames are enqueued as opposed to polling mode that continuously calls dispatch node. New cli commands: - set crypto async dispatch [polling|interrupt] - show crypto async status (displays mode and nodes' states) Signed-off-by: PiotrX Kleski <piotrx.kleski@intel.com> Signed-off-by: DariuszX Kazimierski <dariuszx.kazimierski@intel.com> Reviewed-by: Fan Zhang <roy.fan.zhang@intel.com> Change-Id: I332655f347bb9e3bc9c64166e86e393e911bdb39