summaryrefslogtreecommitdiffstats
path: root/src/vppinfra
AgeCommit message (Collapse)AuthorFilesLines
2021-11-15vppinfra: fix masks in AVX512 clib_count_equal_*Dmitry Valter2-31/+40
Mask result of uAxB_is_equal_mask when buffer is masked. Otherwise it return vector length B as a result for zeroed words. This bug caused crashes in error_drop in tests on Ice Lake. Type: fix Fixes: 7459be1b3626b608e60df574343a1432a068ebce Change-Id: I56183e77f8a8ab6c530e79b465067958de84dceb Signed-off-by: Dmitry Valter <d-valter@yandex-team.ru>
2021-11-12hash: refactor crc32_5tupleDamjan Marion2-40/+54
Type: improvement Change-Id: I31cae2367e2ec7fc89991ca0df994a73da93aaed Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-11-10vppinfra: new vectorized ip checksum functions incl. csum_and_copyDamjan Marion4-0/+473
Type: improvement Change-Id: Id5810b7f4a6d6e4ce16b73c235b50db5d475ebf7 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-11-10vppinfra: deprecate test_mheap.cDave Barach1-286/+0
The original mheap allocator found the exit a long time ago, move test_mheap.c to .../extras/deprecated/vppinfra Type: test Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: Ib1ed40ac605ad76f33c22a681a6df867985bbbba
2021-11-10vppinfra: new memcpy for x86_64Damjan Marion10-938/+786
Change-Id: I5a5055580479960ac53e3f989aa188faf57fb05d Type: improvement Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-11-08vppinfra: vectorized index to pointer functionDamjan Marion3-0/+314
Type: improvement Change-Id: I05e1a8fa31761b113355123429d72da18881d4b0 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-11-08vppinfra: clib_count_equal_* testsDamjan Marion2-1/+101
Type: improvement Change-Id: I8f75cd9ce78ce686985e65c75dcddf498cef7621 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-11-08vppinfra: AVX512 in clib_count_equal_*Damjan Marion2-13/+98
Type: improvement Change-Id: I8105d396cfc984e00cf5137bc57122510f5e6437 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-11-06vppinfra: move clib_count_equal_* codeDamjan Marion3-235/+241
Type: refactor Change-Id: Ib9e8abdbf745ad6563fc79c9ebb6b2ea65917d08 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-11-05vppinfra: remove dead code in elf.cDamjan Marion1-2/+0
Type: refactor Change-Id: Ia47644ca5fb7c848c0de7e7c3ed2c69e8d5cb80f Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-11-05unittest: gcc-11 errors for clib_strcpy, clib_strstr, clib_strcat, and ↵Steven Luong1-45/+0
clib_strncat There are 3 versions of the string functions. For example, for strcpy, they are 1. strcpy(dst, src) -- the legacy unsafe version 2. strcpy_s(dst, dmax, src) -- C11 safeC version which has an addition argument named dmax. 3. clib_strcpy(dst,src) -- clib version to enable legacy code that uses strcpy to make use of strcpy_s without adding the additional argument, dmax, which is required by the C11 safeC version. The implementation for the clib version is to artificially provide dmax to strcpy_s. In this case, it uses 4096 which assumes that if the legacy code works without blowing up, it is likely to work with the clib version without problem. gcc-11 is getting smarter by checking if dmax is within the object's boundary. When the object is declared as static array, it will flag a warning/error if dmax is out of bound for the object since the real size of dst can be determined at compile time. There is no way to find the real size of dst if the object is dynamically allocated at compile time. For this reason, we simply can't provide support for the clib version of the function anymore. If any code is using the clib version, the choice is to migrate to the safeC version. Type: fix Fixes: b0598497afde60146fe8480331c9f96e7a79475a Signed-off-by: Steven Luong <sluong@cisco.com> Change-Id: I99fa59c878331f995b734588cca3906a1d4782f5
2021-11-03vppinfra: strstr_s_inline checks string unterminated wrongSteven Luong1-1/+1
When checking whether s2 is unterminated, it uses s1max. It should use s2max. Type: fix Fixes: b0598497afde60146fe8480331c9f96e7a79475a Signed-off-by: Steven Luong <sluong@cisco.com> Change-Id: I8a0b8ceebc2dd18402a87274add528c3d085a95a
2021-11-02vppinfra: initial RISC-V supportDamjan Marion4-1/+65
Type: feature Change-Id: I1314d60fc420366526efaddd1ed215cf5f8b75dd Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-11-02vppinfra: add CLIB_ASSUME() macroDamjan Marion1-0/+7
This macro privides a way to tell compiler that it is safe to assume that specified expression is true so it can optimize code accordingly. i.e. CLIB_ASSUME (n < 3); while (n) { /* ... */ } Will tell compiler that n is never going to be >= 3 so instead of creating loop it will just unroll loop 2 times. Type: improvement Change-Id: I718a9b95ff7980d7ac68eb9a88357a4ab6eee74a Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-11-02vppinfra: fix calljmp testDamjan Marion1-14/+12
Change-Id: Id1b380880c6509d983727f6fb57e7db97e66655a Type: fix Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-10-28vppinfra: A bitmap copy functionNeale Ranns1-0/+18
Type: improvement Signed-off-by: Neale Ranns <neale@graphiant.com> Change-Id: Id7a7788b41dbcf280e025e5256c41729b0c95f39
2021-10-27vppinfra: improve clib_array_mask_u32Damjan Marion1-33/+52
Type: improvement Change-Id: Ibdb79a0a1c3ba56f9b2f0f2536aafcdeda5cb6d6 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-10-27vppinfra: use bzhi in pow2_mask when availableDamjan Marion1-0/+3
Saves one clock.... Type: improvement Change-Id: I43da40fb4887b77ac851f759c50a7ca2814f8f40 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-10-16perfmon: topdown lvl 2 support on sapphire rapidsRay Kinsella1-0/+1
Added topdown level 2 support on sapphire rapids, including ability to indentify a sapphire rapids cpu. Type: improvement Signed-off-by: Ray Kinsella <mdr@ashroe.eu> Change-Id: I9f99a92fa0886b98bb5185cff32bebd5a094f329
2021-10-13dpdk: fix vmbus device name parsingBenoît Ganne2-3/+4
unformat_init_vector() expects a vector, not a NULL-terminated C-string. Type: fix Change-Id: I20a266243f63d94b0c6fe24e25ee8346c08c8ff2 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2021-10-13vppinfra: fix socket init netns coverityFlorin Coras1-0/+3
Type: fix Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I4f37c6601ace08ae886b08d2284b413d457e4eae
2021-10-13vppinfra: fix format_table coverity warningFlorin Coras1-1/+1
Type: fix Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: Ida114ba35227f70ddd87cad791a21f186be1cba8
2021-10-12vppinfra: use unaligned non-vector load/stores in x86 memcpyDamjan Marion4-21/+28
Type: fix Change-Id: I54ef23a52f05cc95210a736f84b927dd69b8a6f7 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-10-12ipsec: Performance improvement of ipsec4_output_node using flow cacheGovindarajan Mohandoss1-0/+2
Adding flow cache support to improve outbound IPv4/IPSec SPD lookup performance. Details about flow cache: Mechanism: 1. First packet of a flow will undergo linear search in SPD table. Once a policy match is found, a new entry will be added into the flow cache. From 2nd packet onwards, the policy lookup will happen in flow cache. 2. The flow cache is implemented using bihash without collision handling. This will avoid the logic to age out or recycle the old flows in flow cache. Whenever a collision occurs, old entry will be overwritten by the new entry. Worst case is when all the 256 packets in a batch result in collision and fall back to linear search. Average and best case will be O(1). 3. The size of flow cache is fixed and decided based on the number of flows to be supported. The default is set to 1 million flows. This can be made as a configurable option as a next step. 4. Whenever a SPD rule is added/deleted by the control plane, the flow cache entries will be completely deleted (reset) in the control plane. The assumption here is that SPD rule add/del is not a frequent operation from control plane. Flow cache reset is done, by putting the data plane in fall back mode, to bypass flow cache and do linear search till the SPD rule add/delete operation is complete. Once the rule is successfully added/deleted, the data plane will be allowed to make use of the flow cache. The flow cache will be reset only after flushing out the inflight packets from all the worker cores using vlib_worker_wait_one_loop(). Details about bihash usage: 1. A new bihash template (16_8) is added to support IPv4 5 tuple. BIHASH_KVP_PER_PAGE and BIHASH_KVP_AT_BUCKET_LEVEL are set to 1 in the new template. It means only one KVP is supported per bucket. 2. Collision handling is avoided by calling BV (clib_bihash_add_or_overwrite_stale) function. Through the stale callback function pointer, the KVP entry will be overwritten during collision. 3. Flow cache reset is done using BV (clib_bihash_foreach_key_value_pair) function. Through the callback function pointer, the KVP value is reset to ~0ULL. MRR performance numbers with 1 core, 1 ESP Tunnel, null-encrypt, 64B for different SPD policy matching indices: SPD Policy index : 1 10 100 1000 Throughput : MPPS/MPPS MPPS/MPPS MPPS/MPPS KPPS/MPPS (Baseline/Optimized) ARM Neoverse N1 : 5.2/4.84 4.55/4.84 2.11/4.84 329.5/4.84 ARM TX2 : 2.81/2.6 2.51/2.6 1.27/2.6 176.62/2.6 INTEL SKX : 4.93/4.48 4.29/4.46 2.05/4.48 336.79/4.47 Next Steps: Following can be made as a configurable option through startup conf at IPSec level: 1. Enable/Disable Flow cache. 2. Bihash configuration like number of buckets and memory size. 3. Dual/Quad loop unroll can be applied around bihash to further improve the performance. 4. The same flow cache logic can be applied for IPv6 as well as in IPSec inbound direction. A deeper and wider flow cache using bihash_40_8 can replace existing bihash_16_8, to make it common for both IPv4 and IPv6 in both outbound and inbound directions. Following changes are made based on the review comments: 1. ON/OFF flow cache through startup conf. Default: OFF 2. Flow cache stale entry detection using epoch counter. 3. Avoid host order endianness conversion during flow cache lookup. 4. Move IPSec startup conf to a common file. 5. Added SPD flow cache unit test case 6. Replaced bihash with vectors to implement flow cache. 7. ipsec_add_del_policy API is not mpsafe. Cleaned up inflight packets check in control plane. Type: improvement Signed-off-by: mgovind <govindarajan.Mohandoss@arm.com> Signed-off-by: Zachary Leaf <zachary.leaf@arm.com> Tested-by: Jieqiang Wang <jieqiang.wang@arm.com> Change-Id: I62b4d6625fbc6caf292427a5d2046aa5672b2006
2021-10-07vppinfra: asan: improve overflow semanticBenoît Ganne2-33/+43
Type: improvement Change-Id: Ia63899b82e34f179f9efa921e4630b598f2a86cb Signed-off-by: Benoît Ganne <bganne@cisco.com>
2021-10-06vlib: doc nitfixesNathan Skrzypczak1-1/+1
Type: improvement Change-Id: I9e761f908d9d2becbc61eb0515dc6b7c1e1e036f Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com>
2021-10-05build: don't hardcode triplet, allow specifying custom lib dirDamjan Marion1-1/+1
Type: fix Change-Id: I33f364fda88914f88f9b976cb83e6d3ff466f0bb Signed-off-by: Damjan Marion <damarion@cisco.com> Signed-off-by: Ray Kinsella <mdr@ashroe.eu>
2021-10-04vppinfra: fix potential memory access error in _pool_init_fixedJieqiang Wang1-1/+1
_pool_init_fixed uses mmap to initialize a fixed-size and preallocated pool, whose size is the sum of vector_size and free_index_size with alignment to the CLIB_CACHE_LINE_BYTES and page size. In this way vector_size equals to pool_header_t + vec_header_t + elt_size * max_elts so moving to the end of the pool space should be pool_header_t pointer + vector_size, instead of vec_header_t pointer + vector_size. Simple code to reproduce this error: u64 *pool; pool_init_fixed(pool, 2042); Improve unit test to cover this case Type: fix Signed-off-by: Jieqiang Wang <jieqiang.wang@arm.com> Reviewed-by: Lijian Zhang <lijian.zhang@arm.com> Reviewed-by: Tianyu Li <tianyu.li@arm.com> Change-Id: If088ef89b3dcb2d874ee837ae9da60983b14615c Signed-off-by: Dave Barach <dave@barachs.net>
2021-09-28api: API trace improvementsFilip Tehlar5-37/+694
Type: improvement * add support for JSON format in API trace * add ability to replay JSON API trace in both VPP and VAT2 * use CRC for backward compatibility check during JSON API replay * fix API trace CLI (and remove duplicits) * remove custom dump * remove vppapitrace.py * update docs accordingly Change-Id: I5294f68bebe6cbe738630f457f3a87720e06486b Signed-off-by: Filip Tehlar <ftehlar@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com>
2021-09-23classify: use AVX-512 to calculate hash on x86Damjan Marion2-0/+6
Type:improvement Change-Id: I9f9f16eabf64203db11cd4338948d76ca5e0ef12 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-09-22vppinfra: make default hugepage size configurableDamjan Marion2-37/+19
i.e. memory { default-hugepage-size 1G } Type: improvement Change-Id: I822afb51712ae92f4e4992b8ffa33dcb15ccaef1 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-09-20vppinfra: format table improvementsNathan Skrzypczak2-10/+32
This adds a way to define default fg, bg colors and alignement for cell tables. It also allows removing the table title by not setting it. It also removes the trailing newline, for usage inside a format("%U", format_table, ...) Type: improvement Change-Id: I27d7a04c4a919b34d0170b04e24a56831f581ea1 Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com>
2021-09-17build: use GNUInstallDirs install destinationsNick Brown1-1/+1
So as to be compliant with distribution layouts, as recommend by: https://cmake.org/cmake/help/latest/command/install.html#installing-files Type: make Change-Id: Ic46ace4f26aab1aa4902cbd013c40a92c480680d Signed-off-by: Nick Brown <nickbroon@gmail.com>
2021-09-17vppinfra: move format_table from perfmonNathan Skrzypczak3-0/+374
This code seems really usefull for reuse in other plugins, for pretty table formatting Type: feature Change-Id: Ib5784a0dfc81b7d5a5d1f5ccdd02072e460a50fb Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com>
2021-08-27vppinfra: add compress functions for u64, u16 and u8Mohsin Kazmi3-4/+352
Type: improvement Change-Id: I2640148b8959f9a8303520ba2815fe02f1e47928 Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com>
2021-08-20vppinfra: fix clib_random_buffer_get_data cachingBenoît Ganne1-2/+2
When using cached bytes: - do not overflow - do not return the same bytes twice Type: fix Change-Id: I2a87b47a79300e56a2201b8fc3cb6cb15b592e28 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2021-08-10vppinfra: Fix bihash del warningNathan Skrzypczak2-0/+7
Type: fix When freeing an uninstantiated bihash created with dont_add_to_all_bihash_list = 1 we get a warning. This removes the warning & the search for the bihash on cleanup. Change-Id: Iac50ce7e30b97925768f7ad3cb1d30af14686e21 Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com>
2021-08-04vppinfra: fix the array mask functionMohsin Kazmi2-2/+20
Type: fix Fixes: 0ec7dad7a00852663eb88554561347987f87bb53 Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com> Change-Id: I7fab80b3c7e86ac712a34c24ea3e526b0b5bb7ad
2021-07-27vppinfra: introduce CLIB_CACHE_PREFETCH_BYTESDamjan Marion1-31/+29
Type: improvement Change-Id: Ic07010f11ef303f5213a33b0faf24aaedb62f110 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-07-27vppinfra: fix sock init netnsNathan Skrzypczak1-1/+5
Type: fix Change-Id: I0ce8183ded601bdab031c9689ca361414fed165f Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com>
2021-07-22vppinfra: add abstract socket & netns fnsNathan Skrzypczak5-3/+140
* Add clib_socket_init support for abstract sockets if name starts with an '@' * Add clib_socket_init_netns to open socket in netns * Add clib_netns_open Type: feature Change-Id: I89637ad657c702ec38ddecb5c03a1673d0dfb104 Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com>
2021-07-16vppinfra: remove pool_foreach_old, pool_foreach_index_old, ↵Damjan Marion2-28/+0
clib_bitmap_foreach_old Type: refactor Change-Id: Ifacdd001bdeb5d609d495406f53546090b86476d Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-07-16vppinfra: add array mask funcMohsin Kazmi3-0/+185
Type: feature Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com> Change-Id: I6869221917f30f7e59709e20571b4615bc68dc8c
2021-07-15vppinfra: fix the vector funcs test for march variantsMohsin Kazmi1-1/+12
Type: fix Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com> Change-Id: I4208c2622817eb51a4b192cf420f9f1b5f193eef
2021-07-15misc: replace CLIB_PREFETCH with clib_prefetch_{load,store}Damjan Marion1-16/+19
Type: refactor Change-Id: Id10cbf52e8f2dd809080a228d8fa282308be84ac Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-07-14vppinfra: remove old and unused vector macros and inlinesDamjan Marion3-257/+0
Type: refactor Change-Id: Ic504bcfca6e7fbd85e858c3bc7a4f5e72d931789 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-07-13vppinfra: fix saturate add/sub NEON wrappersLijian.Zhang1-40/+60
Fix the saturate add/sub wrappers in vector_neon.h by using the correct intrinsics. Type: fix Signed-off-by: Lijian Zhang <Lijian.Zhang@arm.com> Reviewed-by: Tianyu Li <Tianyu.Li@arm.com> Change-Id: I38a85633948472d4bdb1c199a806633d3070013f
2021-07-13vppinfra: put each vector function into own fileDamjan Marion7-79/+90
Type: refactor Change-Id: I2dd9a18497992ac7e2686c14f5d17eccccda0cda Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-07-13vppinfra: fix SIGSEGV in clib_mem_vm_unmapchenqijun1-4/+6
while one mprotect PROT_NONE on hdr->next or hdr->prev, the other one with the PROT_NONE is unmap at the same time, cause SIGSEGV. Type: fix Signed-off-by: arikachen <eaglesora@gmail.com> Change-Id: I21c0497da140c9654b566e47f767a90346715ed8
2021-06-21vppinfra: asan: fix overflow supportBenoît Ganne3-14/+89
In some part of VPP we are accessing memory past what was allocated for efficiency when we know it is safe to do so. We need to tell ASan about it so it does not complain. The initial attempt was too simple and could not manage cases where the poisoned memory was scattered. This new attempt is more robust as we save and restore the full shadow state. Note it will still not work properly if we poison/unpoison memory while in an overflow context, but this should not be a big issue as overflow should only be temporary. Type: fix Change-Id: I636f44eb59aa8455a4d13929a3791b56619af7b4 Signed-off-by: Benoît Ganne <bganne@cisco.com>