summaryrefslogtreecommitdiffstats
path: root/src/vppinfra
AgeCommit message (Collapse)AuthorFilesLines
2021-11-02vppinfra: initial RISC-V supportDamjan Marion4-1/+65
Type: feature Change-Id: I1314d60fc420366526efaddd1ed215cf5f8b75dd Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-11-02vppinfra: add CLIB_ASSUME() macroDamjan Marion1-0/+7
This macro privides a way to tell compiler that it is safe to assume that specified expression is true so it can optimize code accordingly. i.e. CLIB_ASSUME (n < 3); while (n) { /* ... */ } Will tell compiler that n is never going to be >= 3 so instead of creating loop it will just unroll loop 2 times. Type: improvement Change-Id: I718a9b95ff7980d7ac68eb9a88357a4ab6eee74a Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-11-02vppinfra: fix calljmp testDamjan Marion1-14/+12
Change-Id: Id1b380880c6509d983727f6fb57e7db97e66655a Type: fix Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-10-28vppinfra: A bitmap copy functionNeale Ranns1-0/+18
Type: improvement Signed-off-by: Neale Ranns <neale@graphiant.com> Change-Id: Id7a7788b41dbcf280e025e5256c41729b0c95f39
2021-10-27vppinfra: improve clib_array_mask_u32Damjan Marion1-33/+52
Type: improvement Change-Id: Ibdb79a0a1c3ba56f9b2f0f2536aafcdeda5cb6d6 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-10-27vppinfra: use bzhi in pow2_mask when availableDamjan Marion1-0/+3
Saves one clock.... Type: improvement Change-Id: I43da40fb4887b77ac851f759c50a7ca2814f8f40 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-10-16perfmon: topdown lvl 2 support on sapphire rapidsRay Kinsella1-0/+1
Added topdown level 2 support on sapphire rapids, including ability to indentify a sapphire rapids cpu. Type: improvement Signed-off-by: Ray Kinsella <mdr@ashroe.eu> Change-Id: I9f99a92fa0886b98bb5185cff32bebd5a094f329
2021-10-13dpdk: fix vmbus device name parsingBenoît Ganne2-3/+4
unformat_init_vector() expects a vector, not a NULL-terminated C-string. Type: fix Change-Id: I20a266243f63d94b0c6fe24e25ee8346c08c8ff2 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2021-10-13vppinfra: fix socket init netns coverityFlorin Coras1-0/+3
Type: fix Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I4f37c6601ace08ae886b08d2284b413d457e4eae
2021-10-13vppinfra: fix format_table coverity warningFlorin Coras1-1/+1
Type: fix Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: Ida114ba35227f70ddd87cad791a21f186be1cba8
2021-10-12vppinfra: use unaligned non-vector load/stores in x86 memcpyDamjan Marion4-21/+28
Type: fix Change-Id: I54ef23a52f05cc95210a736f84b927dd69b8a6f7 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-10-12ipsec: Performance improvement of ipsec4_output_node using flow cacheGovindarajan Mohandoss1-0/+2
Adding flow cache support to improve outbound IPv4/IPSec SPD lookup performance. Details about flow cache: Mechanism: 1. First packet of a flow will undergo linear search in SPD table. Once a policy match is found, a new entry will be added into the flow cache. From 2nd packet onwards, the policy lookup will happen in flow cache. 2. The flow cache is implemented using bihash without collision handling. This will avoid the logic to age out or recycle the old flows in flow cache. Whenever a collision occurs, old entry will be overwritten by the new entry. Worst case is when all the 256 packets in a batch result in collision and fall back to linear search. Average and best case will be O(1). 3. The size of flow cache is fixed and decided based on the number of flows to be supported. The default is set to 1 million flows. This can be made as a configurable option as a next step. 4. Whenever a SPD rule is added/deleted by the control plane, the flow cache entries will be completely deleted (reset) in the control plane. The assumption here is that SPD rule add/del is not a frequent operation from control plane. Flow cache reset is done, by putting the data plane in fall back mode, to bypass flow cache and do linear search till the SPD rule add/delete operation is complete. Once the rule is successfully added/deleted, the data plane will be allowed to make use of the flow cache. The flow cache will be reset only after flushing out the inflight packets from all the worker cores using vlib_worker_wait_one_loop(). Details about bihash usage: 1. A new bihash template (16_8) is added to support IPv4 5 tuple. BIHASH_KVP_PER_PAGE and BIHASH_KVP_AT_BUCKET_LEVEL are set to 1 in the new template. It means only one KVP is supported per bucket. 2. Collision handling is avoided by calling BV (clib_bihash_add_or_overwrite_stale) function. Through the stale callback function pointer, the KVP entry will be overwritten during collision. 3. Flow cache reset is done using BV (clib_bihash_foreach_key_value_pair) function. Through the callback function pointer, the KVP value is reset to ~0ULL. MRR performance numbers with 1 core, 1 ESP Tunnel, null-encrypt, 64B for different SPD policy matching indices: SPD Policy index : 1 10 100 1000 Throughput : MPPS/MPPS MPPS/MPPS MPPS/MPPS KPPS/MPPS (Baseline/Optimized) ARM Neoverse N1 : 5.2/4.84 4.55/4.84 2.11/4.84 329.5/4.84 ARM TX2 : 2.81/2.6 2.51/2.6 1.27/2.6 176.62/2.6 INTEL SKX : 4.93/4.48 4.29/4.46 2.05/4.48 336.79/4.47 Next Steps: Following can be made as a configurable option through startup conf at IPSec level: 1. Enable/Disable Flow cache. 2. Bihash configuration like number of buckets and memory size. 3. Dual/Quad loop unroll can be applied around bihash to further improve the performance. 4. The same flow cache logic can be applied for IPv6 as well as in IPSec inbound direction. A deeper and wider flow cache using bihash_40_8 can replace existing bihash_16_8, to make it common for both IPv4 and IPv6 in both outbound and inbound directions. Following changes are made based on the review comments: 1. ON/OFF flow cache through startup conf. Default: OFF 2. Flow cache stale entry detection using epoch counter. 3. Avoid host order endianness conversion during flow cache lookup. 4. Move IPSec startup conf to a common file. 5. Added SPD flow cache unit test case 6. Replaced bihash with vectors to implement flow cache. 7. ipsec_add_del_policy API is not mpsafe. Cleaned up inflight packets check in control plane. Type: improvement Signed-off-by: mgovind <govindarajan.Mohandoss@arm.com> Signed-off-by: Zachary Leaf <zachary.leaf@arm.com> Tested-by: Jieqiang Wang <jieqiang.wang@arm.com> Change-Id: I62b4d6625fbc6caf292427a5d2046aa5672b2006
2021-10-07vppinfra: asan: improve overflow semanticBenoît Ganne2-33/+43
Type: improvement Change-Id: Ia63899b82e34f179f9efa921e4630b598f2a86cb Signed-off-by: Benoît Ganne <bganne@cisco.com>
2021-10-06vlib: doc nitfixesNathan Skrzypczak1-1/+1
Type: improvement Change-Id: I9e761f908d9d2becbc61eb0515dc6b7c1e1e036f Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com>
2021-10-05build: don't hardcode triplet, allow specifying custom lib dirDamjan Marion1-1/+1
Type: fix Change-Id: I33f364fda88914f88f9b976cb83e6d3ff466f0bb Signed-off-by: Damjan Marion <damarion@cisco.com> Signed-off-by: Ray Kinsella <mdr@ashroe.eu>
2021-10-04vppinfra: fix potential memory access error in _pool_init_fixedJieqiang Wang1-1/+1
_pool_init_fixed uses mmap to initialize a fixed-size and preallocated pool, whose size is the sum of vector_size and free_index_size with alignment to the CLIB_CACHE_LINE_BYTES and page size. In this way vector_size equals to pool_header_t + vec_header_t + elt_size * max_elts so moving to the end of the pool space should be pool_header_t pointer + vector_size, instead of vec_header_t pointer + vector_size. Simple code to reproduce this error: u64 *pool; pool_init_fixed(pool, 2042); Improve unit test to cover this case Type: fix Signed-off-by: Jieqiang Wang <jieqiang.wang@arm.com> Reviewed-by: Lijian Zhang <lijian.zhang@arm.com> Reviewed-by: Tianyu Li <tianyu.li@arm.com> Change-Id: If088ef89b3dcb2d874ee837ae9da60983b14615c Signed-off-by: Dave Barach <dave@barachs.net>
2021-09-28api: API trace improvementsFilip Tehlar5-37/+694
Type: improvement * add support for JSON format in API trace * add ability to replay JSON API trace in both VPP and VAT2 * use CRC for backward compatibility check during JSON API replay * fix API trace CLI (and remove duplicits) * remove custom dump * remove vppapitrace.py * update docs accordingly Change-Id: I5294f68bebe6cbe738630f457f3a87720e06486b Signed-off-by: Filip Tehlar <ftehlar@cisco.com> Signed-off-by: Ole Troan <ot@cisco.com>
2021-09-23classify: use AVX-512 to calculate hash on x86Damjan Marion2-0/+6
Type:improvement Change-Id: I9f9f16eabf64203db11cd4338948d76ca5e0ef12 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-09-22vppinfra: make default hugepage size configurableDamjan Marion2-37/+19
i.e. memory { default-hugepage-size 1G } Type: improvement Change-Id: I822afb51712ae92f4e4992b8ffa33dcb15ccaef1 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-09-20vppinfra: format table improvementsNathan Skrzypczak2-10/+32
This adds a way to define default fg, bg colors and alignement for cell tables. It also allows removing the table title by not setting it. It also removes the trailing newline, for usage inside a format("%U", format_table, ...) Type: improvement Change-Id: I27d7a04c4a919b34d0170b04e24a56831f581ea1 Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com>
2021-09-17build: use GNUInstallDirs install destinationsNick Brown1-1/+1
So as to be compliant with distribution layouts, as recommend by: https://cmake.org/cmake/help/latest/command/install.html#installing-files Type: make Change-Id: Ic46ace4f26aab1aa4902cbd013c40a92c480680d Signed-off-by: Nick Brown <nickbroon@gmail.com>
2021-09-17vppinfra: move format_table from perfmonNathan Skrzypczak3-0/+374
This code seems really usefull for reuse in other plugins, for pretty table formatting Type: feature Change-Id: Ib5784a0dfc81b7d5a5d1f5ccdd02072e460a50fb Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com>
2021-08-27vppinfra: add compress functions for u64, u16 and u8Mohsin Kazmi3-4/+352
Type: improvement Change-Id: I2640148b8959f9a8303520ba2815fe02f1e47928 Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com>
2021-08-20vppinfra: fix clib_random_buffer_get_data cachingBenoît Ganne1-2/+2
When using cached bytes: - do not overflow - do not return the same bytes twice Type: fix Change-Id: I2a87b47a79300e56a2201b8fc3cb6cb15b592e28 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2021-08-10vppinfra: Fix bihash del warningNathan Skrzypczak2-0/+7
Type: fix When freeing an uninstantiated bihash created with dont_add_to_all_bihash_list = 1 we get a warning. This removes the warning & the search for the bihash on cleanup. Change-Id: Iac50ce7e30b97925768f7ad3cb1d30af14686e21 Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com>
2021-08-04vppinfra: fix the array mask functionMohsin Kazmi2-2/+20
Type: fix Fixes: 0ec7dad7a00852663eb88554561347987f87bb53 Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com> Change-Id: I7fab80b3c7e86ac712a34c24ea3e526b0b5bb7ad
2021-07-27vppinfra: introduce CLIB_CACHE_PREFETCH_BYTESDamjan Marion1-31/+29
Type: improvement Change-Id: Ic07010f11ef303f5213a33b0faf24aaedb62f110 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-07-27vppinfra: fix sock init netnsNathan Skrzypczak1-1/+5
Type: fix Change-Id: I0ce8183ded601bdab031c9689ca361414fed165f Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com>
2021-07-22vppinfra: add abstract socket & netns fnsNathan Skrzypczak5-3/+140
* Add clib_socket_init support for abstract sockets if name starts with an '@' * Add clib_socket_init_netns to open socket in netns * Add clib_netns_open Type: feature Change-Id: I89637ad657c702ec38ddecb5c03a1673d0dfb104 Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com>
2021-07-16vppinfra: remove pool_foreach_old, pool_foreach_index_old, ↵Damjan Marion2-28/+0
clib_bitmap_foreach_old Type: refactor Change-Id: Ifacdd001bdeb5d609d495406f53546090b86476d Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-07-16vppinfra: add array mask funcMohsin Kazmi3-0/+185
Type: feature Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com> Change-Id: I6869221917f30f7e59709e20571b4615bc68dc8c
2021-07-15vppinfra: fix the vector funcs test for march variantsMohsin Kazmi1-1/+12
Type: fix Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com> Change-Id: I4208c2622817eb51a4b192cf420f9f1b5f193eef
2021-07-15misc: replace CLIB_PREFETCH with clib_prefetch_{load,store}Damjan Marion1-16/+19
Type: refactor Change-Id: Id10cbf52e8f2dd809080a228d8fa282308be84ac Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-07-14vppinfra: remove old and unused vector macros and inlinesDamjan Marion3-257/+0
Type: refactor Change-Id: Ic504bcfca6e7fbd85e858c3bc7a4f5e72d931789 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-07-13vppinfra: fix saturate add/sub NEON wrappersLijian.Zhang1-40/+60
Fix the saturate add/sub wrappers in vector_neon.h by using the correct intrinsics. Type: fix Signed-off-by: Lijian Zhang <Lijian.Zhang@arm.com> Reviewed-by: Tianyu Li <Tianyu.Li@arm.com> Change-Id: I38a85633948472d4bdb1c199a806633d3070013f
2021-07-13vppinfra: put each vector function into own fileDamjan Marion7-79/+90
Type: refactor Change-Id: I2dd9a18497992ac7e2686c14f5d17eccccda0cda Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-07-13vppinfra: fix SIGSEGV in clib_mem_vm_unmapchenqijun1-4/+6
while one mprotect PROT_NONE on hdr->next or hdr->prev, the other one with the PROT_NONE is unmap at the same time, cause SIGSEGV. Type: fix Signed-off-by: arikachen <eaglesora@gmail.com> Change-Id: I21c0497da140c9654b566e47f767a90346715ed8
2021-06-21vppinfra: asan: fix overflow supportBenoît Ganne3-14/+89
In some part of VPP we are accessing memory past what was allocated for efficiency when we know it is safe to do so. We need to tell ASan about it so it does not complain. The initial attempt was too simple and could not manage cases where the poisoned memory was scattered. This new attempt is more robust as we save and restore the full shadow state. Note it will still not work properly if we poison/unpoison memory while in an overflow context, but this should not be a big issue as overflow should only be temporary. Type: fix Change-Id: I636f44eb59aa8455a4d13929a3791b56619af7b4 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2021-06-08vppinfra: pool_free_elts() now supports fixed-size poolsDave Barach1-29/+36
Test added to the unittest plugin / test_vlib.py Type: improvement Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I73445e57918347c102ff6f5e8c9ddb9bd96f1407
2021-05-26vppinfra: explicit blocking mode for sock connectsFlorin Coras2-2/+4
Type: fix Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I3a33230de13fef613dc9523cf24a9968d200c2e5
2021-05-26api: socket client connect set to blockingFlorin Coras1-0/+8
Binary api client must otherwise check the returned error and if it was EAGAIN/EINPROGRESS poll for connect completion. Type: improvement Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I89845b1a59b9fa2ab0968029435ceb203bfa8f6c
2021-05-19vppinfra: add more llist macrosFlorin Coras1-0/+38
Type: improvement Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: Ife79871c6530d2cd485928fee465baf2c8957e11
2021-05-13tests: move test source to vpp/testDave Wallace2-114/+0
- Generate copyright year and version instead of using hard-coded data Type: refactor Signed-off-by: Dave Wallace <dwallacelf@gmail.com> Change-Id: I6058f5025323b3aa483f5df4a2c4371e27b5914e
2021-05-11vppinfra: Add missing %oNathan Skrzypczak1-0/+4
This is used in vppinfra/socket.c:495 Type: feature Change-Id: I89b409ae7abb01723108ae3e6c55bb1675db50ee Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com>
2021-05-07vppinfra: add universal array mask_compare and compress funcsDamjan Marion8-17/+467
Type: improvement Change-Id: I6d812339f626ea630ad9354632d2f9506122d379 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-05-06vppinfra: fix testsDamjan Marion17-87/+78
Type: fix Change-Id: If59a66aae658dd35dbcb4987ab00c306b3c6e2e2 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-05-06stats: memory heap countersOle Troan1-7/+10
- Add counters for the main-heap - Add additional counters per heap: STAT_MEM_TOTAL STAT_MEM_USED, STAT_MEM_FREE, STAT_MEM_USED_MMAP, STAT_MEM_TOTAL_ALLOC, STAT_MEM_FREE_CHUNKS, STAT_MEM_RELEASABLE, The per-heap counters are organised as a two dimensional vector. total, used and free are directly available via symlinks. vpp_get_stats ls "^/mem/" /mem/stat segment /mem/stat segment/total /mem/stat segment/used /mem/stat segment/free /mem/main heap /mem/main heap/total /mem/main heap/used /mem/main heap/free vpp_get_stats dump "^/mem/main\ heap$" [0 @ 0]: 1073741776 packets /mem/main heap [1 @ 0]: 91586688 packets /mem/main heap [2 @ 0]: 982155088 packets /mem/main heap [3 @ 0]: 0 packets /mem/main heap [4 @ 0]: 1073741776 packets /mem/main heap [5 @ 0]: 433 packets /mem/main heap [6 @ 0]: 981708688 packets /mem/main heap Type: feature Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I36725dde3b4b3befd27a8b4d3ba931f2d3b627cc
2021-05-05vppinfra: enfoce function pointer cast in CLIB_MARCH_FN_POINTERDamjan Marion1-2/+2
Type: improvement Change-Id: I6ba9f9467a3990f1436a60b1dbc6cb795fd18ba9 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-05-05vppinfra: fix x86 packs / packus wrappersDamjan Marion4-94/+52
They both take signed value as input. Type: fix Change-Id: If3d8ec4e0b1c02d7d65262bdd9db49ff7fbfef39 Signed-off-by: Damjan Marion <damarion@cisco.com>
2021-04-30misc: experimental configure scriptDamjan Marion1-2/+2
Type: make Change-Id: Iaeb9d22eec9a7a763b63899814a44e78c8050f1f Signed-off-by: Damjan Marion <damarion@cisco.com>