summaryrefslogtreecommitdiffstats
path: root/src/vppinfra
AgeCommit message (Collapse)AuthorFilesLines
2022-01-05vppinfra: few bitops inlinesDamjan Marion1-0/+26
Change-Id: I9aa01fafd5329a399aa9d72a6e2cc6c9d65dc85c Type: improvement Signed-off-by: Damjan Marion <damarion@cisco.com>
2022-01-01vppinfra: fix bitmap can't get correct next clear indexjiangxiaoming1-2/+1
when bitmap vec len is 1 and all the bits has been occupied, clib_bitmap_next_clear(ai, 63) will return result:65 which should be 64. It will lead to missing iteration for pool_foreach. Type: fix Signed-off-by: jiangxiaoming <jiangxiaoming@outlook.com> Change-Id: Iadac7e6f6b4da357943c4c9d50bf22353c4a8408
2021-12-22vppinfra: add format_u64_bitmap functionDamjan Marion2-0/+34
Sample output: 60 56 52 48 44 40 36 32 28 24 20 16 12 8 4 0 0x0020 .... .... .... .... .... .... .... .... .... .... .... .... ..11 .... ...1 1... 0x0000000000003018 0x0018 1111 1111 1111 1111 1111 1111 1111 1111 1111 1111 1111 1111 1111 1111 111. .1.1 0xffffffffffffffe5 0x0010 .... .... .... .... .... .... ..1. ...1 .1.1 1... 11.. 1... .... .1.. 1111 ..11 0x0000002158c804f3 0x0008 .... .... .... .... .... .... .... .... .... .... .... .... .... .... ..1. 11.. 0x000000000000002c 0x0000 .... .... .... .... .... .... .... .... .... .... .... .... .... .... .... ...1 0x0000000000000001 Type: improvement Change-Id: Ib99129866ad5a1d2d52be6855406e2829aa9ec3e Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-12-22vppinfra: add bihash 12_4 templateDamjan Marion2-0/+85
Type: feature Change-Id: I8a6b2c7d7682a01c472d7375d444e8b43eb5db04 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-12-22vppinfra: fix map_lock issueliuyacan1-2/+2
The release of the map_lock is a bit too early, it needs to wrap mprotect(), otherwise the access rights of hdr may be deprived of other worker threads. Type: fix Signed-off-by: liuyacan <liuyacan@corp.netease.com> Change-Id: I9cb89d49a097ec6d5ae9fd2af2bddf967ec946e9
2021-12-17vppinfra: missing __clib_exportbenker1-1/+1
Type: fix hash_resize is available in hash.h file, but missing __clib_export in hash.c Signed-off-by: Leung Lai Yung <benkerbuild@gmail.com> Change-Id: Ibb741b532cd1080ec5d8314aae8dbbca87f42502
2021-12-16vppinfra: clang doesn't know about attribute nocloneDamjan Marion1-0/+4
Type: fix Fixes: 88019c40 Change-Id: I219597c95883f49b6ff51e65e6c6c80e3c1518b7 Signed-off-by: Damjan Marion <dmarion@me.com>
2021-12-16vppinfra: fix clib_fifo_elt_at_indexFlorin Coras1-1/+1
Type: fix Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I4924420b827bdf7d161a1f723c2bfd5370d43772
2021-12-15vppinfra: toeplitz hash four in parallelDamjan Marion6-76/+622
Type: improvement Change-Id: Icb3f39f42d01c51d7b03543cb7d6b5dabad11866 Signed-off-by: Damjan Marion <dmarion@me.com>
2021-12-14vppinfra: toeplitz hashDamjan Marion5-0/+704
Type: feature Change-Id: I0ccf280c4b769ee624cfb780146b847a0e8e4942 Signed-off-by: Damjan Marion <dmarion@me.com>
2021-12-14vppinfra: display cpu frequency in test_vector_funcs perf testDamjan Marion2-8/+24
Type: improvement Change-Id: I350a30eca0d3d7ec8816e4e7c30889621029c694 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-12-14interface: Add interface monitor cliNathan Skrzypczak2-0/+21
This adds an interface CLI 'monitor interface <interface>' that periodically reports rx/tx pps & bandwidth for a given interface. Type: feature Change-Id: Ia9d59b3443913520a52b38d7bda012190be6f167 Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com>
2021-12-13vppinfra: multiple perf bundle support in test_vector_funcsDamjan Marion2-10/+79
Type: improvement Change-Id: I93be722f4dabc2b49a5e83f61f4eb1d72f5f7d45 Signed-off-by: Damjan Marion <dmarion@me.com>
2021-12-06vppinfra: add 'filter' option to test_vector_funcsDamjan Marion2-1/+9
Type: improvement Change-Id: I5ed9029e5dc1368b7debbef04a3ac439a61af9ea Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-12-06vppinfra: sha2 testsDamjan Marion4-3/+345
Type: improvement Change-Id: I8a5d8d5db4e4a8ee3a1164bfbe91badff181d06a Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-12-02vppinfra: add perf testing to test_vector_funcDamjan Marion9-23/+386
Type: improvement Change-Id: I7aacd58d113c13036c15655817400032dd8d1932 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-12-02vppinfra: vector shuffle cleanupDamjan Marion5-36/+62
Type: refactor Change-Id: I8b3fc2ce30df313467274a174c5ac6adbf296153 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-12-02vppinfra: add SIMD masked bitwise opsDamjan Marion1-0/+28
Type: improvement Change-Id: Id73b00a8cb3af799f9f97949bc1302f162a91a57 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-11-30vppinfra: remove duplicate api function definitionFlorin Coras2-13/+0
vl_api_c_string_to_api_string declared in api_types.h and defined in api_shared.c. Remove from jsonformat Type: improvement Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I994c8c13b7246a857464c1f879432f73692c6d1f
2021-11-25vppinfra: will_expand for pool_put and bitmapStanislav Zaikin2-0/+27
Macro for pool_put and put a barrier inside load_balance_destroy when bitmap is actually growing. Type: improvement Signed-off-by: Stanislav Zaikin <zstaseg@gmail.com> Change-Id: Ief2912e8efd744289ebed68373fa6fd0ee83118e
2021-11-23vppinfra: fallback to builtin memcpy if vector code is not enabledDamjan Marion1-3/+2
Type: fix Fixes: 56f54af Change-Id: I678ff3622640c911eefb57a9288ef95ed859eeb9 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-11-23vppinfra: update CPU listDamjan Marion1-47/+52
Type: improvement Change-Id: Ibe1f23801d5eac6d1dae61c6df1a47d59455d42e Signed-off-by: Damjan Marion <dmarion@me.com>
2021-11-22vppinfra: annotate pool_dup() for asanBenoît Ganne1-22/+24
When using AddressSanitizer, pool objects are poisoned, but pool_dup() deep copy is ok. Type: fix Change-Id: Id801c4ab35666474a297d885c4b92878e44feab6 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2021-11-22vppinfra: fix the memory leak while doing clib_bihash_freeHan Wu1-0/+1
Type: fix Signed-off-by: Han Wu <wuhan9084@163.com> Change-Id: I6ad3c313dcab5ed193c135676c1cebd0a141fea1
2021-11-15vppinfra: fix masks in AVX512 clib_count_equal_*Dmitry Valter2-31/+40
Mask result of uAxB_is_equal_mask when buffer is masked. Otherwise it return vector length B as a result for zeroed words. This bug caused crashes in error_drop in tests on Ice Lake. Type: fix Fixes: 7459be1b3626b608e60df574343a1432a068ebce Change-Id: I56183e77f8a8ab6c530e79b465067958de84dceb Signed-off-by: Dmitry Valter <d-valter@yandex-team.ru>
2021-11-12hash: refactor crc32_5tupleDamjan Marion2-40/+54
Type: improvement Change-Id: I31cae2367e2ec7fc89991ca0df994a73da93aaed Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-11-10vppinfra: new vectorized ip checksum functions incl. csum_and_copyDamjan Marion4-0/+473
Type: improvement Change-Id: Id5810b7f4a6d6e4ce16b73c235b50db5d475ebf7 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-11-10vppinfra: deprecate test_mheap.cDave Barach1-286/+0
The original mheap allocator found the exit a long time ago, move test_mheap.c to .../extras/deprecated/vppinfra Type: test Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: Ib1ed40ac605ad76f33c22a681a6df867985bbbba
2021-11-10vppinfra: new memcpy for x86_64Damjan Marion10-938/+786
Change-Id: I5a5055580479960ac53e3f989aa188faf57fb05d Type: improvement Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-11-08vppinfra: vectorized index to pointer functionDamjan Marion3-0/+314
Type: improvement Change-Id: I05e1a8fa31761b113355123429d72da18881d4b0 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-11-08vppinfra: clib_count_equal_* testsDamjan Marion2-1/+101
Type: improvement Change-Id: I8f75cd9ce78ce686985e65c75dcddf498cef7621 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-11-08vppinfra: AVX512 in clib_count_equal_*Damjan Marion2-13/+98
Type: improvement Change-Id: I8105d396cfc984e00cf5137bc57122510f5e6437 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-11-06vppinfra: move clib_count_equal_* codeDamjan Marion3-235/+241
Type: refactor Change-Id: Ib9e8abdbf745ad6563fc79c9ebb6b2ea65917d08 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-11-05vppinfra: remove dead code in elf.cDamjan Marion1-2/+0
Type: refactor Change-Id: Ia47644ca5fb7c848c0de7e7c3ed2c69e8d5cb80f Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-11-05unittest: gcc-11 errors for clib_strcpy, clib_strstr, clib_strcat, and ↵Steven Luong1-45/+0
clib_strncat There are 3 versions of the string functions. For example, for strcpy, they are 1. strcpy(dst, src) -- the legacy unsafe version 2. strcpy_s(dst, dmax, src) -- C11 safeC version which has an addition argument named dmax. 3. clib_strcpy(dst,src) -- clib version to enable legacy code that uses strcpy to make use of strcpy_s without adding the additional argument, dmax, which is required by the C11 safeC version. The implementation for the clib version is to artificially provide dmax to strcpy_s. In this case, it uses 4096 which assumes that if the legacy code works without blowing up, it is likely to work with the clib version without problem. gcc-11 is getting smarter by checking if dmax is within the object's boundary. When the object is declared as static array, it will flag a warning/error if dmax is out of bound for the object since the real size of dst can be determined at compile time. There is no way to find the real size of dst if the object is dynamically allocated at compile time. For this reason, we simply can't provide support for the clib version of the function anymore. If any code is using the clib version, the choice is to migrate to the safeC version. Type: fix Fixes: b0598497afde60146fe8480331c9f96e7a79475a Signed-off-by: Steven Luong <sluong@cisco.com> Change-Id: I99fa59c878331f995b734588cca3906a1d4782f5
2021-11-03vppinfra: strstr_s_inline checks string unterminated wrongSteven Luong1-1/+1
When checking whether s2 is unterminated, it uses s1max. It should use s2max. Type: fix Fixes: b0598497afde60146fe8480331c9f96e7a79475a Signed-off-by: Steven Luong <sluong@cisco.com> Change-Id: I8a0b8ceebc2dd18402a87274add528c3d085a95a
2021-11-02vppinfra: initial RISC-V supportDamjan Marion4-1/+65
Type: feature Change-Id: I1314d60fc420366526efaddd1ed215cf5f8b75dd Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-11-02vppinfra: add CLIB_ASSUME() macroDamjan Marion1-0/+7
This macro privides a way to tell compiler that it is safe to assume that specified expression is true so it can optimize code accordingly. i.e. CLIB_ASSUME (n < 3); while (n) { /* ... */ } Will tell compiler that n is never going to be >= 3 so instead of creating loop it will just unroll loop 2 times. Type: improvement Change-Id: I718a9b95ff7980d7ac68eb9a88357a4ab6eee74a Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-11-02vppinfra: fix calljmp testDamjan Marion1-14/+12
Change-Id: Id1b380880c6509d983727f6fb57e7db97e66655a Type: fix Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-10-28vppinfra: A bitmap copy functionNeale Ranns1-0/+18
Type: improvement Signed-off-by: Neale Ranns <neale@graphiant.com> Change-Id: Id7a7788b41dbcf280e025e5256c41729b0c95f39
2021-10-27vppinfra: improve clib_array_mask_u32Damjan Marion1-33/+52
Type: improvement Change-Id: Ibdb79a0a1c3ba56f9b2f0f2536aafcdeda5cb6d6 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-10-27vppinfra: use bzhi in pow2_mask when availableDamjan Marion1-0/+3
Saves one clock.... Type: improvement Change-Id: I43da40fb4887b77ac851f759c50a7ca2814f8f40 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-10-16perfmon: topdown lvl 2 support on sapphire rapidsRay Kinsella1-0/+1
Added topdown level 2 support on sapphire rapids, including ability to indentify a sapphire rapids cpu. Type: improvement Signed-off-by: Ray Kinsella <mdr@ashroe.eu> Change-Id: I9f99a92fa0886b98bb5185cff32bebd5a094f329
2021-10-13dpdk: fix vmbus device name parsingBenoît Ganne2-3/+4
unformat_init_vector() expects a vector, not a NULL-terminated C-string. Type: fix Change-Id: I20a266243f63d94b0c6fe24e25ee8346c08c8ff2 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2021-10-13vppinfra: fix socket init netns coverityFlorin Coras1-0/+3
Type: fix Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I4f37c6601ace08ae886b08d2284b413d457e4eae
2021-10-13vppinfra: fix format_table coverity warningFlorin Coras1-1/+1
Type: fix Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: Ida114ba35227f70ddd87cad791a21f186be1cba8
2021-10-12vppinfra: use unaligned non-vector load/stores in x86 memcpyDamjan Marion4-21/+28
Type: fix Change-Id: I54ef23a52f05cc95210a736f84b927dd69b8a6f7 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-10-12ipsec: Performance improvement of ipsec4_output_node using flow cacheGovindarajan Mohandoss1-0/+2
Adding flow cache support to improve outbound IPv4/IPSec SPD lookup performance. Details about flow cache: Mechanism: 1. First packet of a flow will undergo linear search in SPD table. Once a policy match is found, a new entry will be added into the flow cache. From 2nd packet onwards, the policy lookup will happen in flow cache. 2. The flow cache is implemented using bihash without collision handling. This will avoid the logic to age out or recycle the old flows in flow cache. Whenever a collision occurs, old entry will be overwritten by the new entry. Worst case is when all the 256 packets in a batch result in collision and fall back to linear search. Average and best case will be O(1). 3. The size of flow cache is fixed and decided based on the number of flows to be supported. The default is set to 1 million flows. This can be made as a configurable option as a next step. 4. Whenever a SPD rule is added/deleted by the control plane, the flow cache entries will be completely deleted (reset) in the control plane. The assumption here is that SPD rule add/del is not a frequent operation from control plane. Flow cache reset is done, by putting the data plane in fall back mode, to bypass flow cache and do linear search till the SPD rule add/delete operation is complete. Once the rule is successfully added/deleted, the data plane will be allowed to make use of the flow cache. The flow cache will be reset only after flushing out the inflight packets from all the worker cores using vlib_worker_wait_one_loop(). Details about bihash usage: 1. A new bihash template (16_8) is added to support IPv4 5 tuple. BIHASH_KVP_PER_PAGE and BIHASH_KVP_AT_BUCKET_LEVEL are set to 1 in the new template. It means only one KVP is supported per bucket. 2. Collision handling is avoided by calling BV (clib_bihash_add_or_overwrite_stale) function. Through the stale callback function pointer, the KVP entry will be overwritten during collision. 3. Flow cache reset is done using BV (clib_bihash_foreach_key_value_pair) function. Through the callback function pointer, the KVP value is reset to ~0ULL. MRR performance numbers with 1 core, 1 ESP Tunnel, null-encrypt, 64B for different SPD policy matching indices: SPD Policy index : 1 10 100 1000 Throughput : MPPS/MPPS MPPS/MPPS MPPS/MPPS KPPS/MPPS (Baseline/Optimized) ARM Neoverse N1 : 5.2/4.84 4.55/4.84 2.11/4.84 329.5/4.84 ARM TX2 : 2.81/2.6 2.51/2.6 1.27/2.6 176.62/2.6 INTEL SKX : 4.93/4.48 4.29/4.46 2.05/4.48 336.79/4.47 Next Steps: Following can be made as a configurable option through startup conf at IPSec level: 1. Enable/Disable Flow cache. 2. Bihash configuration like number of buckets and memory size. 3. Dual/Quad loop unroll can be applied around bihash to further improve the performance. 4. The same flow cache logic can be applied for IPv6 as well as in IPSec inbound direction. A deeper and wider flow cache using bihash_40_8 can replace existing bihash_16_8, to make it common for both IPv4 and IPv6 in both outbound and inbound directions. Following changes are made based on the review comments: 1. ON/OFF flow cache through startup conf. Default: OFF 2. Flow cache stale entry detection using epoch counter. 3. Avoid host order endianness conversion during flow cache lookup. 4. Move IPSec startup conf to a common file. 5. Added SPD flow cache unit test case 6. Replaced bihash with vectors to implement flow cache. 7. ipsec_add_del_policy API is not mpsafe. Cleaned up inflight packets check in control plane. Type: improvement Signed-off-by: mgovind <govindarajan.Mohandoss@arm.com> Signed-off-by: Zachary Leaf <zachary.leaf@arm.com> Tested-by: Jieqiang Wang <jieqiang.wang@arm.com> Change-Id: I62b4d6625fbc6caf292427a5d2046aa5672b2006
2021-10-07vppinfra: asan: improve overflow semanticBenoît Ganne2-33/+43
Type: improvement Change-Id: Ia63899b82e34f179f9efa921e4630b598f2a86cb Signed-off-by: Benoît Ganne <bganne@cisco.com>
2021-10-06vlib: doc nitfixesNathan Skrzypczak1-1/+1
Type: improvement Change-Id: I9e761f908d9d2becbc61eb0515dc6b7c1e1e036f Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com>