summaryrefslogtreecommitdiffstats
path: root/src/vnet/crypto
AgeCommit message (Collapse)AuthorFilesLines
2023-09-12crypto: allow changing dispatch modeVratko Polak4-2/+51
This change aims to affect crypto_sw_scheduler behavior, but all the edits end up in vnet/crypto. Previous release CSIT tests were testing async crypto in polling mode. After 9a9604b09f15691d7c4ddf29afd99a31e7e31eed introduced adaptive mode for crypto dispatch, the CSIT performance got way worse. Possibly, there is another VPP bug related to adaptive mode (it should not lose as many packets as seen in CSIT), but the next release is too close for trying to fix that. This change (instead of fixing adaptive mode) allows CSIT to continue testing polling mode (after explicit API call), while keeping the adaptive mode as default behavior. The deprecated crypto_set_async_dispatch always disable adaptive mode, crypto_set_async_dispatch_v2 has parameter to enable or disable it. The mode parameter is still used for the inital state of adaptive mode. Type: feature Change-Id: Ib98080eefb4be291207af543884f2c3837f92f59 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2023-08-08ipsec: add support for RFC-4543 ENCR_NULL_AUTH_AES_GMACBenoît Ganne2-6/+15
Type: improvement Change-Id: I830f7a2ea3ac0aff5185698b9fa7a278c45116b0 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2023-06-08crypto: use fixed crypto frame poolgaoginskx2-8/+11
The async frames pool may be resized once drained. This will cause 2 problems: original pool pointer is invalidated and pool size changed, both problems will confuse the crypto infra user graph nodes (like IPsec and Wireguard) and crypto engines if they expect the pool pointers always valid and the pool size never changed (for performance reason). This patch introduces fixed size of the async frames pool. This helps zeroing surprise to the components shown above and avoiding segmentation fault when pool resizing happened. In addition, the crypto engine may take advantage of the feature to sync its own pool/vector with crypto infra. Type: improvement Signed-off-by: Gabriel Oginski <gabrielx.oginski@intel.com> Signed-off-by: Piotr Bronowski <piotrx.bronowski@intel.com> Change-Id: I2a71783b90149fa376848b9c4f84ce8c6c034bef
2023-06-01crypto: make crypto-dispatch node working in adaptive modeXiaoming Jiang6-187/+27
This patch can make crypto dispatch node adaptively switching between pooling and interrupt mode, and improve vpp overall performance. Type: improvement Signed-off-by: Xiaoming Jiang <jiangxiaoming@outlook.com> Change-Id: I845ed1d29ba9f3c507ea95a337f6dca7f8d6e24e
2023-03-06crypto: remove VNET_CRYPTO_OP_FLAG_INIT_IV flagBenoît Ganne1-3/+2
IV requirements vary wildly with the selected mode of operation. For example, for AES-CBC the IV must be unpredictable whereas for AES counter mode (CTR or GCM), it can be predictable but reusing an IV with the same key material is catastrophic. Because of that, it is hard to generate IV in a generic way, and it is better left to the crypto user (eg. IPsec). Type: improvement Change-Id: I32689c591d8c6572b8d37c4d24f175ea6132d3ec Signed-off-by: Benoît Ganne <bganne@cisco.com>
2023-03-06crypto: make it easier to diagnose keys use-after-freeBenoît Ganne1-2/+2
Type: improvement Change-Id: Ib98eba146e24e659acf3b9a228b81fcd641f4c67 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2023-02-08ipsec: fix AES CBC IV generation (CVE-2022-46397)Benoît Ganne1-2/+2
For AES-CBC, the IV must be unpredictable (see NIST SP800-38a Appendix C). Chaining IVs like is done by ipsecmb and native backends for the VNET_CRYPTO_OP_FLAG_INIT_IV is fully predictable. Encrypt a counter as part of the message, making the (predictable) counter-generated IV unpredictable. Fixes: VPP-2037 Type: fix Change-Id: If4f192d62bf97dda553e7573331c75efa11822ae Signed-off-by: Benoît Ganne <bganne@cisco.com>
2022-10-24crypto: fix dead nn foreachDmitry Valter1-5/+5
Pass vec_foreach args in the right order Type: fix Signed-off-by: Dmitry Valter <d-valter@yandex-team.ru> Change-Id: I90d3dfb70ee2dbd9a48ae9ecfd2c1526a74d0786
2022-03-31crypto: drop the frame if there is no handlerDastin Wilski1-0/+6
If async engines are disbaled and async is turned on vpp tries to enqueue frame with nonexisting handler which leads to segfault. This patch checks for handler and drops the frame in case it doesn't exist. Type: fix Signed-off-by: Dastin Wilski <dastin.wilski@gmail.com> Change-Id: I67211867ee29dc41cc9f0733e8e0b3ea86677f85
2022-03-18vppinfra: refactor *_will_expand() functionsDamjan Marion1-2/+1
Type: refactor Change-Id: I3625eacf9e04542ca8778df5d46075a8654642c7 Signed-off-by: Damjan Marion <damarion@cisco.com>
2022-02-16crypto: Fix for the crash in cryptomgovind1-0/+4
Fix for the crash when both crypto_native_plugin and DPDK QAT are enabled in startup conf. Type: fix Signed-off-by: mgovind <govindarajan.mohandoss@arm.com> Change-Id: Ib020ed7130a99080a093c70c06d47bcacd6d23b1
2022-01-21wireguard: add async mode for encryption packetsGabriel Oginski1-9/+10
Originally wireguard doesn't support async mode for encryption packets. This patch add async mode for encryption in wireguard and also adds support chacha20-poly1305 algorithm in cryptodev for async handler. In addition it contains new command line to activate async mode for wireguard: set wireguard async mode on|off and also add new command to check active mode for wireguard: show wireguard mode Type: improvement Signed-off-by: Gabriel Oginski <gabrielx.oginski@intel.com> Change-Id: I141d48b42ee8dbff0112b8542ab5205268089da6
2022-01-12crypto: omit loop iterationDastin Wilski1-4/+6
This fix adds check that will omit loop iteration in case dequeue handler is zero. Type: fix Signed-off-by: Dastin Wilski <dastin.wilski@gmail.com> Change-Id: I7526e3fe7d8c8da9662b4e9204efd5e2d8be1908
2022-01-05crypto: encrypt/decrypt queues sw_schedulerJakub Wysocki4-34/+99
Type: improvement Previously multiple sw crypto scheduler queues per core design caused unaverage frame processing rate for each async op ID – the lower the op ID is the highly likely they are processed first. For example, when a RX core is feeding both encryption and decryption jobs of the same crypto algorithm to the queues at a high rate, in the mean time the crypto cores have no enough cycles to process all: the jobs in the decryption queue are less likely being processed, causing packet drop. To improve the situation this patch makes every core only owning a two queues, one for encrypt operations and one for decrypt. The queue is changed either after checking each core or after founding a frame to process. All crypto jobs with different algorithm are pushed to thoses queues and are treated evenly. In addition, the crypto async infra now uses unified dequeue handler, one per engine. Only the active engine will be registered its dequeue handler in crypto main. Signed-off-by: DariuszX Kazimierski <dariuszx.kazimierski@intel.com> Signed-off-by: PiotrX Kleski <piotrx.kleski@intel.com> Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com> Signed-off-by: Jakub Wysocki <jakubx.wysocki@intel.com> Change-Id: I517ee8e31633980de5e0dd4b05e1d5db5dea760e
2021-10-26crypto: add barrier in crypto key addGabriel Oginski1-0/+12
Originally the pool of keys can be expand and cache with pointer for key can be invalid. For example in Wireguard during handshake process this pool can be expand and pointer for these keys in cache can be invalid for workers or can has incorrect value (poison memory). The fixes add barrier if the pool needs be to expand to ensure that cache in function will be valid and avoid situation when cache has invalid pointer for these keys. Type: fix Signed-off-by: Gabriel Oginski <gabrielx.oginski@intel.com> Change-Id: Ida8f300213dfebb91ecaf1937fb08de81c20ba7b
2021-09-09vlib: fix vlib_buffer_enqueue_to_next() overflowBenoît Ganne1-4/+4
vlib_buffer_enqueue_to_next() requires to allow overflow of up to 63 elements of 'buffer' and 'nexts' array. - add helper to compute the minimum size - fix occurences in session and async crypto Type: fix Change-Id: If8d7eebc5bf9beba71ba194aec0f79b8eb6d5843 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2021-05-13tests: move test source to vpp/testDave Wallace1-28/+0
- Generate copyright year and version instead of using hard-coded data Type: refactor Signed-off-by: Dave Wallace <dwallacelf@gmail.com> Change-Id: I6058f5025323b3aa483f5df4a2c4371e27b5914e
2021-05-05crypto crypto-openssl: support hashing operationsFilip Tehlar2-15/+52
Type: feature Change-Id: I36041fe5c5f0ff129aee42516189807e96f62123 Signed-off-by: Filip Tehlar <ftehlar@cisco.com>
2021-04-20crypto: support async handlers for 3des and md5Alexander Chernavin2-3/+12
With this change, add support for 3DES and MD5 in IPsec async mode. After changes in foreach_crypto_link_async_alg, the last combination in the list (aes-256-cbc-hmac-sha-512) started to fail during decription. That was also fixed by proper vector size validation. Type: improvement Signed-off-by: Alexander Chernavin <achernavin@netgate.com> Change-Id: I660657bdab62ea9cf031c3e43d99f2317e5f74d7
2021-04-09dpdk: selection of cryptodev engine data-pathFan Zhang2-2/+4
Type: improvement This patch combined cryptodev op and cryptodev raw API data paths into one and makes the engine run-timely select which data path is used: if all cryptodev devices support RTE_CRYPTODEV_FF_SYM_RAW_DP feature flag, the raw data path API is used, otherwise the traditional data path is used. Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com> Change-Id: Ibbd7c4405288bd9a48a34878954fd3040df7b4ad
2021-03-26vlib: introduce vlib_get_main_by_index(), vlib_get_n_threads()Damjan Marion4-13/+15
Type: improvement Change-Id: If3da7d4338470912f37ff1794620418d928fb77f Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-03-05ipsec: Submit fuller async framesNeale Ranns1-53/+29
Type: improvement In the current scheme an async frame is submitted each time the crypto op changes. thus happens each time a different SA is used and thus potentially many times per-node. thi can lead to the submision of many partially filled frames. change the scheme to construct as many full frames as possible in the node and submit them all at the end. the frame owner ship is passed to the user so that there can be more than one open frame per-op at any given time. Signed-off-by: Neale Ranns <neale@graphiant.com> Change-Id: Ic2305581d7b5aa26133f52115e0cd28ba956ed55
2021-03-04crypto: add support for aes-ctr+sha-1 chainsBenoît Ganne1-16/+19
Type: feature Change-Id: I9d4f90bc701d2b9b903a018f8d27cec5e129d7be Signed-off-by: Benoît Ganne <bganne@cisco.com>
2021-02-26crypto: fix coverity issue 218445Piotr Bronowski1-3/+0
Fixes coverity issue CID 218445 (#1 of 1): Logically dead code (DEADCODE) dead_error_line: Execution cannot reach this statement: return 4294967295U;. Type: fix Signed-off-by: Piotr Bronowski <piotrx.bronowski@intel.com> Change-Id: Ibf8ee0458320d20c3adca2efa2a4bfad7c190dbe
2021-02-24crypto: A more memory efficient layout of the frame element structNeale Ranns2-18/+26
Type: improvement Also: - state as enum so my GDB life is easier - typo; s/indice/indices/; Signed-off-by: Neale Ranns <neale@graphiant.com> Change-Id: I3320f5ef1ccd7d042071ef336488a41adfad7463
2021-01-20crypto: fixed vnet_crypto_is_set_handler()PiotrX Kleski1-1/+10
Type: fix vnet_crypto_is_set_handler() was checking if op handler is set by using alg instead of op id. This fix changes the check to use op ids from crypto_main's alg_data for provided alg. Signed-off-by: PiotrX Kleski <piotrx.kleski@intel.com> Change-Id: I5f0f00a889ddada44d9912af175ebad66677037a
2020-12-14misc: refactor clib_bitmap_foreach macroDamjan Marion1-2/+2
Type: refactor Change-Id: I077110e1a422722e20aa546a6f3224c06ab0cde5 Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-12-04crypto: fixed vnet_crypto_async_reset_frame assertPiotrX Kleski1-1/+2
Type: fix Changed vnet_crypto_async_reset_frame assert to expect also ERROR state frames. Signed-off-by: PiotrX Kleski <piotrx.kleski@intel.com> Change-Id: I3abc29f3f9642027aee38a59a932e54c90da859d
2020-12-04crypto: fixed async frame enqueue race conditionPiotrX Kleski1-2/+8
Type: fix To avoid race condition happening in async crypto engines, async frame state and thread index set should happen before enqueue. In addition as the enqueue handler already returns the enqueue status, when an enqueue is failed, the async crypto engine shall not worry about setting the async frame state but let the submit_open_frame function to do just that. Signed-off-by: PiotrX Kleski <piotrx.kleski@intel.com> Reviewed-by: Fan Zhang <roy.fan.zhang@intel.com> Change-Id: Ic1b0c94478b3cfd5fab98657218bbd70c46a220a
2020-11-17tests: move crypto tests to src/vnet/crypto/testDave Wallace1-0/+28
- Refactor make test code to be co-located with the vpp feature source code Type: test Signed-off-by: Dave Wallace <dwallacelf@gmail.com> Change-Id: I089bda44c31cbb217132e5b385cd9ea96ea5239e
2020-09-21crypto: revert crypto set handler all APIYulong Pei1-36/+3
This reverts commit 8c91b2ae2b32d428ef35605707788fe064621cb3, but keep a comment fix. Type: fix Signed-off-by: Yulong Pei <yulong.pei@intel.com> Change-Id: Ia66941bf18d3efac96f41bdf905d877cfb3ab211
2020-09-16crypto: Crypto set handler API to support set all as CLIYulong Pei2-4/+37
Type: improvement Signed-off-by: Yulong Pei <yulong.pei@intel.com> Change-Id: I43556f8c76c7aae64d9c927e1fda3c1774d7e49d
2020-09-09crypto: change cryptodev with new cryptodev APIFan Zhang1-1/+1
Type: feature This patch updateds cryptodev engine uses new DPDK Cryptodev API planned to be upstreamed in DPDK 20.11. Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com> Signed-off-by: Piotr Bronowski <piotrX.bronowski@intel.com> Change-Id: I8dd1a8ac643f1e952deb787e466b76ea7aa5f420
2020-09-03crypto: Add async crypto APIsNathan Skrzypczak5-5/+172
Type: feature This adds api calls for the following CLIs: * set sw_scheuduler worker <N> crypto on|off * set crypto async dispatch polling|interrupt * set crypto handler * set crypto async handler Change-Id: Ic701d149c440e42ea4575da42b9f69e4c8759602 Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com>
2020-09-03crypto: SW scheduler async crypto enginePiotrX Kleski4-69/+232
Type: feature This patch adds new sw_scheduler async crypto engine. The engine transforms async frames info sync crypto ops and delegates them to active sync engines. With the patch it is possible to increase the single worker crypto throughput by offloading the crypto workload to multiple workers. By default all workers in the system will attend the crypto workload processing. However a worker's available cycles are limited. To avail more cycles to one worker to process other workload (e.g. the worker core that handles the RX/TX and IPSec stack processing), a useful cli command is added to remove itself (or add it back later) from the heavy crypto workload but only let other workers to process the crypto. The command is: - set sw_scheduler worker <idx> crypto <on|off> It also adds new interrupt mode to async crypto dispatch node. This mode signals the node when new frames are enqueued as opposed to polling mode that continuously calls dispatch node. New cli commands: - set crypto async dispatch [polling|interrupt] - show crypto async status (displays mode and nodes' states) Signed-off-by: PiotrX Kleski <piotrx.kleski@intel.com> Signed-off-by: DariuszX Kazimierski <dariuszx.kazimierski@intel.com> Reviewed-by: Fan Zhang <roy.fan.zhang@intel.com> Change-Id: I332655f347bb9e3bc9c64166e86e393e911bdb39
2020-09-01crypto: add chacha20-poly1305 algoArtem Glazychev1-2/+5
Type: feature Signed-off-by: Artem Glazychev <artem.glazychev@xored.com> Change-Id: I3697cf7fab7abb7c3d2f61ef326c9116bc1eed66
2020-05-06crypto: fix coverity issuesFan Zhang1-20/+1
Fix coverity issues in crypto framework and cryptodev engine. Type: fix Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com> Change-Id: Ib261da0163c8182c803600db22c5a6dad5a19999
2020-04-30crypto: introduce async crypto infraFan Zhang5-32/+1025
Type: feature Signed-off-by: Damjan Marion <damarion@cisco.com> Signed-off-by: Filip Tehlar <ftehlar@cisco.com> Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com> Signed-off-by: Piotr Bronowski <piotrx.bronowski@intel.com> Signed-off-by: Dariusz Kazimierski <dariuszx.kazimierski@intel.com> Signed-off-by: Piotr Kleski <piotrx.kleski@intel.com> Change-Id: I4c3fcccf55c36842b7b48aed260fef2802b5c54b
2020-02-26crypto: fix set crypto handlersFilip Tehlar1-1/+1
Type: fix Change-Id: I9e168a87aff1882636840eb94e5509cd08885fb5 Signed-off-by: Filip Tehlar <ftehlar@cisco.com>
2020-02-18crypto: show crypto handlers re-designFilip Tehlar2-24/+37
Type: improvement Show simple and chained handler details on a single line. Change-Id: I5ad807a4d3608fc38c1134f564755d5751c30070 Signed-off-by: Filip Tehlar <ftehlar@cisco.com>
2020-02-11ipsec: add support for chained buffersFilip Tehlar4-57/+256
Type: feature Change-Id: Ie072a7c2bbb1e4a77f7001754f01897efd30fc53 Signed-off-by: Filip Tehlar <ftehlar@cisco.com>
2019-11-13crypto: do not crash if no crypto engine loadedBenoît Ganne1-1/+1
Do not overflow ops_handlers vector. Type: fix Change-Id: I8d5e7fb8125a7bd87ecfe6f4f1390fb9f43dad8f Signed-off-by: Benoît Ganne <bganne@cisco.com>
2019-11-12crypto: not use vec api with opt_data[VNET_CRYPTO_N_OP_IDS]Lijian Zhang3-3/+4
opt_data is defined as a array, while in some code, e.g., function vnet_crypto_get_op_type, it's used as vec. vec api is not applicable to static arraies. src/vnet/crypto/crypto.h:234:70: error: address of array 'cm->opt_data' will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion] vnet_crypto_op_data_t *od = ({ do { if ((0 > 0) && ! ((id) < ((cm->opt_data) ? (((vec_header_t *) (cm->opt_data) - 1)->len) : 0))) Type: fix Change-Id: I0b6754406e4216ca975bc1da4b5d4ce293a9bb45 Signed-off-by: Lijian Zhang <Lijian.Zhang@arm.com>
2019-07-24ipsec: GCM, Anti-replay and ESN fixessNeale Ranns1-2/+1
Type: fix Several Fixes: 1 - Anti-replay did not work with GCM becuase it overwrote the sequence number in the ESP header. To fix i added the seq num to the per-packet data so it is preserved 2 - The high sequence number was not byte swapped during ESP encrypt. 3 - openssl engine was the only one to return FAIL_DECRYPT for bad GCM the others return BAD_HMAC. removed the former 4 - improved tracing to show the low and high seq numbers 5 - documented the anti-replay window checks 6 - fixed scapy patch for ESN support for GCM 7 - tests for anti-reply (w/ and w/o ESN) for each crypto algo Change-Id: Id65d96b6d1d4dd821b2ab557e87468fff6d70e5b Signed-off-by: Neale Ranns <nranns@cisco.com>
2019-06-25ipsec: return error if the engine backend has no handler for theNeale Ranns2-0/+9
requested alogrithm. Type: feature Change-Id: I19a9c14b2bb52ba2fc66246845b7ada73d5095d1 Signed-off-by: Neale Ranns <nranns@cisco.com>
2019-06-07ipsec: remove the set_key APINeale Ranns2-24/+0
there's no use case to just change the key of an SA. instead the SA should be renegociated and the new SA applied to the existing SPD entry or tunnel. the set_key functions were untested. Type: refactor Change-Id: Ib096eebaafb20be7b5501ece5a24aea038373002 Signed-off-by: Neale Ranns <nranns@cisco.com>
2019-06-05IPSEC: some CLI fixesNeale Ranns2-3/+3
Change-Id: I45618347e37440263270baf07b2f82f653f754a5 Signed-off-by: Neale Ranns <nranns@cisco.com>
2019-04-30crypto: enforce per-alg crypto key lengthBenoît Ganne2-16/+50
Crypto algorithms have different requirements on key length. As we do not support key stretching (eg. PBKDF2), user must provide the exact key length used by the algorithm. Failing that means low-level crypto functions might read garbage (eg. aes128_key_expand() will read 16-bytes, regardless of the key provided by the user). Change-Id: I347a1ea7a59720a1ed07ceaad8b00a31f78458c9 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2019-04-26crypto, ipsec: change GCM IV handlingDamjan Marion1-1/+1
- nonce construction out of salt and iv is ipsec specific so it should be handled in ipsec code - fixes GCM unit tests - GCM IV is constructed out of simple counter, per RFC4106 section 3.1 Change-Id: Ib7712cc9612830daa737f5171d8384f1d361bb61 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-04-25crypto: AES GCM IV length is always 12Damjan Marion1-1/+1
... at least for use cases we are interested in Change-Id: I1156ff354635e8f990ce2664ebc8dcd3786ddca5 Signed-off-by: Damjan Marion <damarion@cisco.com>