summaryrefslogtreecommitdiffstats
path: root/src/vppinfra
AgeCommit message (Collapse)AuthorFilesLines
2020-05-07vppinfra: loop counter off by 1 in search_free_list()Steven Luong1-1/+1
In search_free_list(), we have this do while loop. do { l--; f_index = h->free_lists[b][l]; f = elt_at (h, f_index); f_size = heap_elt_size (v, f); if ((s = f_size - size) >= 0) break; } while (l >= 0); When (l == 0), we still go back up to execute l--. Then l become -1. The next statement is we index h->free_lists[b][-1]. After that, elt_at() would probably cause a crash in the ASSERT. Type: fix Ticket: VPPSUPP-63 Signed-off-by: Steven Luong <sluong@cisco.com> Change-Id: I617d122aa221cfdfe38f8be50f4e0f0e76e11bb5
2020-05-07rdma: fix alignment issueDamjan Marion1-0/+3
Type: fix Change-Id: I0ff7a6f3354066c2252c1ebd0d43ac59db278bf0 Signed-off-by: Damjan Marion <damarion@cisco.com> Signed-off-by: Elias Rudberg <elias.rudberg@bahnhof.net>
2020-05-06docs: clean up make docs jobPaul Vinciguerra1-1/+2
Type: docs Change-Id: I9b5e5137eb4c1e89f6e8d7a278cd11a0fd496471 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2020-04-30build: rework x86 CPU variantsDamjan Marion3-6/+20
Type: improvement Change-Id: Ief243f88e654e578ef9b8060fcf535b364aececb Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-04-30build: support arch-specific compiling for Neoverse N1Lijian.Zhang1-0/+11
Enable arch-specific compiling and dynamic optimal function selection for Arm Neoverse-N1. Support for -march=armv8.2-a+crc+crypto -mtune=neoverse-n1 for Neoverse-N1 is added starting from gcc-9.2.0 without change with change improvement [L2 - 1x flows] 11.00 Mpps 11.46 Mpps 4% [L2 - 10Kx flows] 6.83 Mpps 7.17 Mpps 5% [L3 - 1x flows] 10.39 Mpps 10.78 Mpps 3.7% [L3 - 10Kx flows] 6.67 Mpps 7.19 Mpps 7.8% Type: feature Change-Id: I5d24d17820b3dd6909b913202e8c31fc7d48650f Signed-off-by: Lijian Zhang <Lijian.Zhang@arm.com> Reviewed-by: Jieqiang Wang <Jieqiang.Wang@arm.com> Reviewed-by: Govindarajan Mohandoss <Govindarajan.Mohandoss@arm.com>
2020-04-28vppinfra: type prove vec_new and vec_resizeAndreas Schultz1-6/+10
Some vector functions (e.g. vec_new) pass the vector pointer through vec_resize. This turn the pointer from a real type into a void pointer. Explicitly cast the pointer back to its original type to catch type mismatches. Type: improvement Signed-off-by: Andreas Schultz <andreas.schultz@travelping.com> Change-Id: Id13e52d4f038af2cee28e5abf1aca720f8909378
2020-04-28vppinfra: improve test coverageDave Barach1-1/+1
Bonus corner-case bugfix in bitmap.h, found during the exercise. Issue dates from 2001 or thereabouts. Please review this specific change carefully. lcov_post: filter system include directories and generated files in build-root Type: improvement Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: Iaa0b63e9dc571dfe3d992197ac49ba4d93403c61
2020-04-27vppinfra: selectively disable false-positive GCC-10 warningsBenoît Ganne3-0/+24
GCC-10 increase overflows-related warnings but is confused by SIMD operations. Type: fix Change-Id: Iafde754c2fbec60e2d0a328f295b1f5c156d8234 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2020-04-27vppinfra: selectively disable false-positive GCC-10 warningsBenoît Ganne1-0/+9
GCC-10 increase overflows-related warnings but failed to infer that b->n_cached_bytes is always < sizeof(uword). Type: fix Change-Id: I956ae609abc9e39d4a932e5801510999d7d27b79 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2020-04-27build: add vppinfra/warnings.h to exported headers listBenoît Ganne1-0/+1
To allow the use of WARN_ON/OFF macros to selectively disable build warnings. Type: fix Change-Id: Iceb9d28b2b80c373afb51900880c23041be836db Signed-off-by: Benoît Ganne <bganne@cisco.com>
2020-04-24vppinfra: finish deprecating qsort.cDave Barach2-272/+7
Minor change to vec_sort_with_function(...): don't depend on the qsort implementation to deal with null, zero-long, or 1-long vectors Type: fix Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I7bd7b0421673d2a025363089562aa7c6266fba66
2020-04-23nat: add/del ed_ext_ports only if the table is instantiatedDave Barach1-0/+3
Add a suitable ASSERT in the bihash template in case this happens again. Type: fix Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: Ib370d4238f6bae2995bc30fd17fad5c41053c3d1
2020-04-23vppinfra: more bihash optimizatonsDamjan Marion3-66/+67
* Avoid doing expensive bit extraction for most likely case where bucket .log2_page_size == 0 and .linear_search == 0, saves 3-5 cycles for lookup, data_prefetch and add operation * use bextr instruction when available (x86 BMI instruction set) Type: improvement Change-Id: I163df36a29287482c5f133be8b21d62a2f7440de Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-04-22vppinfra: move unused code to extras/deprecated/vppinfraDave Barach37-12383/+1
Type: improvement Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: Id28299a188feefa1899d835fd499f018af95d81b
2020-04-22misc: fix coverity warningsDave Barach1-1/+1
Type: fix Ticket: VPP-1837 Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I9ec87d2293d8f92c3e488a0f61083cf815ac496c
2020-04-22misc: asan: mark parsed elf memory as readable for ASanBenoît Ganne1-0/+2
Type: fix Change-Id: Ic7441a09bab2cabc7632ee502368584ac022f997 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2020-04-22misc: asan: do not poison memory after munmap()Benoît Ganne1-1/+0
It is a bad idea to poison memory after munmap because the address space can be reused (eg. for global data of dlopen()ed object) and ASan model allows access by default. Moreover, access to a stale address space will fault. Type: fix Change-Id: I356de422f255447d9d50a3a71fb0c2eaa790d731 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2020-04-22vppinfra: fix potential race in bihash bucket lockDamjan Marion1-11/+9
Type: improvement Change-Id: Ia04bd26ecd689894753e036e52920316de611910 Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-04-22vppinfra: improve bihash add/del performanceDamjan Marion2-26/+36
Measured improvement is from 439 to 167 clocks for add operation in 16_8 case... Type: improvement Change-Id: I975ff46ff30b983a3ec80a5cde25ccb68d7fa03b Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-04-21vppinfra: bihash improvementsDave Barach15-50/+272
Template instances can allocate BIHASH_KVP_PER_PAGE data records tangent to the bucket, to remove a dependent read / prefetch. Template instances can ask for immediate memory allocation, to avoid several branches in the lookup path. Clean up l2 fib, gpb plugin codes: use clib_bihash_get_bucket(...) Use hugepages for bihash allocation arenas Type: improvement Signed-off-by: Dave Barach <dave@barachs.net> Signed-off-by: Damjan Marion <damarion@cisco.com> Change-Id: I92fc11bc58e48d84e2d61f44580916dd1c56361c
2020-04-17vppinfra: install missing tw_timer_2t_2w_512sl header fileYu Sun1-0/+1
Type: improvement Signed-off-by: Yu Sun <yusun2@cisco.com> Change-Id: I68aea7c5776c5b31081c98388df4133d2062218a
2020-04-15vppinfra: don't use memcmp to compare keys in cuckooDamjan Marion4-10/+1
Type: improvement Change-Id: I7e11bf72be5fad5967724c038eb649a261294ca0 Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-04-15vppinfra: delay bucket2 calc in cuckoo searchDamjan Marion1-28/+25
There is no need to calculate bucket2 if there is hit on bucket1 Type: improvement Change-Id: Id01c37963497668c0160068501294568a181d011 Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-04-15vppinfra: add clib_cuckoo_search_inline_with_hash functionDamjan Marion1-3/+10
Type: improvement Change-Id: I547263ae954506f11101666ff768524fbfdb579e Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-04-15vppinfra: install missing cuckoo hash header filesDamjan Marion1-0/+2
Type: improvement Change-Id: Ifb4eec00fd4f1d19e4b0af802d015a35e402e0af Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-04-01vppinfra: fix vppinfra test code buildDave Barach3-4/+6
Type: fix Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I921adae4ad797bf80cfcdb05d2a89ace9183a89a
2020-04-01vppinfra: add tw_timer_2t_2w_512sl variantDave Barach4-3/+187
Type: feature Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I72cacfb5696dca74335f31415c0df795467615a5
2020-03-30vppinfra: add support for avx512 alignment version of load and storeZhiyong Yang1-0/+8
Type: improvement Signed-off-by: Zhiyong Yang <zhiyong.yang@intel.com> Change-Id: Idfec9cb9370a8cf4966d3fdfa440496f21e17005
2020-03-27vppinfra: add clib_bihash_get_bucketDamjan Marion2-20/+13
Type: improvement Change-Id: I073bb7bea2a55eabbb6c253b003966f0a821e4a3 Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-03-22vppinfra: fix typo in dlmalloc.cDave Barach1-1/+1
Fix libffi package name for Ubuntu 20.04 Type: fix Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: Idc567717494b4c40c307f20a40d5e10cd26b0a46
2020-03-21vppinfra: fix minor cuckoo bugs and add cuckoo_16_8Klement Sekera5-26/+170
Type: improvement Change-Id: If1164d2eb81e9d4748436cb1bb8b164857d70565 Signed-off-by: Klement Sekera <ksekera@cisco.com>
2020-03-16rdma: introduce direct verb for Cx4/5 txBenoît Ganne1-0/+8
Direct Verb allows for direct access to NIC HW rx/tx rings. This patch introduce TX direct verb support for Mellanox ConnectX-4/5 adapters. 'dv' mode must be explicitely selected at interface creation to benefit from this. Type: feature Change-Id: If830ba9f33db73299acdbddc68b5c09eaf6add98 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2020-03-16rdma: add Mellanox mlx5 Direct Verbs receive supportDamjan Marion2-0/+16
Type: feature Change-Id: I3f287ab536a482c366ad7df47e1c04e640992ebc Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-03-12vppinfra: refactor clib_timebase_tDave Barach3-12/+18
Add a clib_time_t * argument to clib_timebase_init(...), to encourage client code to share the vlib_main_t's clib_time_t object. Display the current day / date in GMT via the "show time" debug CLI. Fix the test framework so it processes the new "show time" output format. Type: refactor Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I5e52d57eb164b7cdb6355362d520df6928491711
2020-03-12vppinfra: remove time jump workaroundDave Barach1-14/+0
A partial revert of gerrit 25729. The last_run_time == 0.0 check is necessary and remains in place. Type: fix Fixes: 3d9f134 Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I3d2c9f90b2bc867f02c4749a5b19f997b84185b9
2020-03-11vppinfra: minimize tw verbosity on timebase jumpFlorin Coras1-2/+3
Type: improvement Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I5db3457a9fed11d6ecf6eaabcdf8f1d1177b2a9f
2020-03-10vppinfra: improve timebase resilienceDave Barach1-3/+20
Deal with arbitrary kernel reference time changes: for example, yanking the kernel reference clock back to a time before vpp started. Best practice involves aligning the kernel reference clock with reality prior to starting apps which use 10us granularity timers. Compute change in the reference and cpu clocks. Recompute the vpp start time reference and and total cpu clock count, using the current clock tick rate. Next, compute a new clock rate sample. If the sample seems sane, factor it into the exponentially smoothed clock rate and recalculate total cpu ticks based on the new clock rate. Type: fix Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: Ib6132ffbbe0e6d140725676de5e35be112a31dfe
2020-03-09vppinfra: fix corner cases in tw_timer_expireDave Barach1-1/+22
Type: fix Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I4b3ff6e9c8e1d76037b168aeab36dcb5b4482260
2020-03-08vppinfra: fix 32-bit overflow in os_cpu_clock_frequencyDamjan Marion1-1/+1
Type: fix Change-Id: I23250fcbc70086584b5448baec9af9a1528992f5 Signed-off-by: Damjan Marion <damarion@cisco.com> Signed-off-by: Dave Barach <dave@barachs.net>
2020-02-25svm: refactor fifoFlorin Coras2-10/+21
Type: refactor Switch from a wrapped byte space to a "continuous" one wherein fifo chunks are appended to the fifo as more data is enqueued and chunks are removed as data is dequeued. The fifo is still subject to a maximum size, i.e., maximum number of bytes that can be enqueued, so the max number of chunks associated to the fifo is also constrained. When enqueueing data, which must fit within the available free space, if not enough "supporting" chunk memory is available, the fifo asks the fifo segment for enough chunk memory to ensure that the write can succeed. To avoid allocating large amounts of small chunks due to small writes, if possible, the size of the chunks requested is lower capped by min_alloc. When dequeuing data, all the chunks that have been completely drained, i.e., head moved beyond the chunks’ end bytes, are unlinked from the fifo and returned to the fifo segment. The one exception to this is the last chunk which is never unlinked. Change-Id: I98c1dbd9135fb79650365c7e40c29238b96cd4ee Signed-off-by: Florin Coras <fcoras@cisco.com>
2020-02-25svm: refactor fifo chunk trackingFlorin Coras1-0/+1
Avoid tracking with rbtrees all of the chunks associated to a fifo. Instead, only track chunks when doing out-of-order operations (peek or ooo enqueue). Type: refactor Change-Id: I9f8bd266211746637d98e6a12ffc4b2d6346950a Signed-off-by: Florin Coras <fcoras@cisco.com>
2020-02-25crypto-native: GCM implementation with vector AESNI instructionsDamjan Marion1-0/+48
Introduced on intel IceLake uarch. Type: feature Change-Id: I1514c76c34e53ce0577666caf32a50f95eb6548f Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-02-24vppinfra: correct fixed pool header size calculationDave Barach1-3/+1
Remove duplicate space allocation for the pool header. Not significant w/ CLIB_CACHE_LINE_BYTES >= 64 since the code rounds the size of the pool header to an even multiple of the cache line size. Type: fix Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I923f2a60e7565cf2dfbc18d78264bf82ff30c926
2020-02-23vppinfra: deprecate CLIB_VEC64Dave Barach3-382/+0
Type: refactor Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: Id1e7c0926036db4601c91438397ceed22381fc07
2020-02-18vppinfra: fix debug image builds on armDamjan Marion1-11/+2
vextq_u8(...) reuqires constant value so instead of inline function we need to use macro. Type: fix Signed-off-by: Damjan Marion <dmarion@me.com> Change-Id: I9c1d878c9ec750f0ed5b5eac4dffde50e97e7357
2020-02-17misc: fix coverity warningsDave Barach1-0/+29
Add an ALWAYS_ASSERT (...) macro, to (a) shut up coverity, and (b) check the indicated condition in production images. As in: p = hash_get(...); ALWAYS_ASSERT(p) /* was ASSERT(p) */ elt = pool_elt_at_index(pool, p[0]); This may not be the best way to handle a specific case, but failure to check return values at all followed by e.g. a pointer dereference isn't ok. Type: fix Ticket: VPP-1837 Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: Ia97c641cefcfb7ea7d77ea5a55ed4afea0345acb
2020-02-17crypto-native: calculate ghash using vpclmulqdq instructionsDamjan Marion1-0/+15
vpclmulqdq is introduced on intel icelake architecture and allows computing 4 carry-less multiplications in paralled by using 512-bit SIMD registers Type: feature Change-Id: Idb09d6f51ba6f116bba11649b2d99f649356d449 Signed-off-by: Damjan Marion <damjan.marion@gmail.com>
2020-02-14crypto-native: refactor CBC codeDamjan Marion1-0/+6
Type: refactor Change-Id: I61e25942de318d03fb3d75689259709d687479bc Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-02-13vppinfra: add 128-bit and 512-bit a ^ b ^ c shortcutDamjan Marion3-0/+27
This allows us to combine 2 XOR operations into signle instruction which makes difference in crypto op: - in x86, by using ternary logic instruction - on ARM, by using EOR3 instruction (available with sha3 feature) Type: refactor Change-Id: Ibdf9001840399d2f838d491ca81b57cbd8430433 Signed-off-by: Damjan Marion <damjan.marion@gmail.com>
2020-02-13crypto-native: add AArch64 AES-GCM native implementationDamjan Marion2-0/+18
Type: feature Change-Id: I4f96b0af13b875d491704b010328a1814e1dbda1 Signed-off-by: Damjan Marion <dmarion@me.com>