summaryrefslogtreecommitdiffstats
path: root/src/vppinfra
AgeCommit message (Collapse)AuthorFilesLines
2020-03-16rdma: introduce direct verb for Cx4/5 txBenoît Ganne1-0/+8
Direct Verb allows for direct access to NIC HW rx/tx rings. This patch introduce TX direct verb support for Mellanox ConnectX-4/5 adapters. 'dv' mode must be explicitely selected at interface creation to benefit from this. Type: feature Change-Id: If830ba9f33db73299acdbddc68b5c09eaf6add98 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2020-03-16rdma: add Mellanox mlx5 Direct Verbs receive supportDamjan Marion2-0/+16
Type: feature Change-Id: I3f287ab536a482c366ad7df47e1c04e640992ebc Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-03-12vppinfra: refactor clib_timebase_tDave Barach3-12/+18
Add a clib_time_t * argument to clib_timebase_init(...), to encourage client code to share the vlib_main_t's clib_time_t object. Display the current day / date in GMT via the "show time" debug CLI. Fix the test framework so it processes the new "show time" output format. Type: refactor Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I5e52d57eb164b7cdb6355362d520df6928491711
2020-03-12vppinfra: remove time jump workaroundDave Barach1-14/+0
A partial revert of gerrit 25729. The last_run_time == 0.0 check is necessary and remains in place. Type: fix Fixes: 3d9f134 Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I3d2c9f90b2bc867f02c4749a5b19f997b84185b9
2020-03-11vppinfra: minimize tw verbosity on timebase jumpFlorin Coras1-2/+3
Type: improvement Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I5db3457a9fed11d6ecf6eaabcdf8f1d1177b2a9f
2020-03-10vppinfra: improve timebase resilienceDave Barach1-3/+20
Deal with arbitrary kernel reference time changes: for example, yanking the kernel reference clock back to a time before vpp started. Best practice involves aligning the kernel reference clock with reality prior to starting apps which use 10us granularity timers. Compute change in the reference and cpu clocks. Recompute the vpp start time reference and and total cpu clock count, using the current clock tick rate. Next, compute a new clock rate sample. If the sample seems sane, factor it into the exponentially smoothed clock rate and recalculate total cpu ticks based on the new clock rate. Type: fix Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: Ib6132ffbbe0e6d140725676de5e35be112a31dfe
2020-03-09vppinfra: fix corner cases in tw_timer_expireDave Barach1-1/+22
Type: fix Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I4b3ff6e9c8e1d76037b168aeab36dcb5b4482260
2020-03-08vppinfra: fix 32-bit overflow in os_cpu_clock_frequencyDamjan Marion1-1/+1
Type: fix Change-Id: I23250fcbc70086584b5448baec9af9a1528992f5 Signed-off-by: Damjan Marion <damarion@cisco.com> Signed-off-by: Dave Barach <dave@barachs.net>
2020-02-25svm: refactor fifoFlorin Coras2-10/+21
Type: refactor Switch from a wrapped byte space to a "continuous" one wherein fifo chunks are appended to the fifo as more data is enqueued and chunks are removed as data is dequeued. The fifo is still subject to a maximum size, i.e., maximum number of bytes that can be enqueued, so the max number of chunks associated to the fifo is also constrained. When enqueueing data, which must fit within the available free space, if not enough "supporting" chunk memory is available, the fifo asks the fifo segment for enough chunk memory to ensure that the write can succeed. To avoid allocating large amounts of small chunks due to small writes, if possible, the size of the chunks requested is lower capped by min_alloc. When dequeuing data, all the chunks that have been completely drained, i.e., head moved beyond the chunks’ end bytes, are unlinked from the fifo and returned to the fifo segment. The one exception to this is the last chunk which is never unlinked. Change-Id: I98c1dbd9135fb79650365c7e40c29238b96cd4ee Signed-off-by: Florin Coras <fcoras@cisco.com>
2020-02-25svm: refactor fifo chunk trackingFlorin Coras1-0/+1
Avoid tracking with rbtrees all of the chunks associated to a fifo. Instead, only track chunks when doing out-of-order operations (peek or ooo enqueue). Type: refactor Change-Id: I9f8bd266211746637d98e6a12ffc4b2d6346950a Signed-off-by: Florin Coras <fcoras@cisco.com>
2020-02-25crypto-native: GCM implementation with vector AESNI instructionsDamjan Marion1-0/+48
Introduced on intel IceLake uarch. Type: feature Change-Id: I1514c76c34e53ce0577666caf32a50f95eb6548f Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-02-24vppinfra: correct fixed pool header size calculationDave Barach1-3/+1
Remove duplicate space allocation for the pool header. Not significant w/ CLIB_CACHE_LINE_BYTES >= 64 since the code rounds the size of the pool header to an even multiple of the cache line size. Type: fix Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I923f2a60e7565cf2dfbc18d78264bf82ff30c926
2020-02-23vppinfra: deprecate CLIB_VEC64Dave Barach3-382/+0
Type: refactor Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: Id1e7c0926036db4601c91438397ceed22381fc07
2020-02-18vppinfra: fix debug image builds on armDamjan Marion1-11/+2
vextq_u8(...) reuqires constant value so instead of inline function we need to use macro. Type: fix Signed-off-by: Damjan Marion <dmarion@me.com> Change-Id: I9c1d878c9ec750f0ed5b5eac4dffde50e97e7357
2020-02-17misc: fix coverity warningsDave Barach1-0/+29
Add an ALWAYS_ASSERT (...) macro, to (a) shut up coverity, and (b) check the indicated condition in production images. As in: p = hash_get(...); ALWAYS_ASSERT(p) /* was ASSERT(p) */ elt = pool_elt_at_index(pool, p[0]); This may not be the best way to handle a specific case, but failure to check return values at all followed by e.g. a pointer dereference isn't ok. Type: fix Ticket: VPP-1837 Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: Ia97c641cefcfb7ea7d77ea5a55ed4afea0345acb
2020-02-17crypto-native: calculate ghash using vpclmulqdq instructionsDamjan Marion1-0/+15
vpclmulqdq is introduced on intel icelake architecture and allows computing 4 carry-less multiplications in paralled by using 512-bit SIMD registers Type: feature Change-Id: Idb09d6f51ba6f116bba11649b2d99f649356d449 Signed-off-by: Damjan Marion <damjan.marion@gmail.com>
2020-02-14crypto-native: refactor CBC codeDamjan Marion1-0/+6
Type: refactor Change-Id: I61e25942de318d03fb3d75689259709d687479bc Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-02-13vppinfra: add 128-bit and 512-bit a ^ b ^ c shortcutDamjan Marion3-0/+27
This allows us to combine 2 XOR operations into signle instruction which makes difference in crypto op: - in x86, by using ternary logic instruction - on ARM, by using EOR3 instruction (available with sha3 feature) Type: refactor Change-Id: Ibdf9001840399d2f838d491ca81b57cbd8430433 Signed-off-by: Damjan Marion <damjan.marion@gmail.com>
2020-02-13crypto-native: add AArch64 AES-GCM native implementationDamjan Marion2-0/+18
Type: feature Change-Id: I4f96b0af13b875d491704b010328a1814e1dbda1 Signed-off-by: Damjan Marion <dmarion@me.com>
2020-02-13vppinfra: add cmake option to grow vectors by 1Dave Barach3-0/+13
For debugging. Do not set this option in production. Type: feature Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I5e59671c4932e064bc087b85bf9c62c6f3bf48cf
2020-02-12vppinfra: add clib_prefetch_load and clib_prefetch_storeDamjan Marion1-0/+12
For people tired of typen CLIB_CACHE_LINE_BYTES.... Type: improvement Change-Id: I7658a8525ff6e3edc81a29b05a6fda33e537806e Signed-off-by: Damjan Marion <dmarion@me.com>
2020-02-12vppinfra: add ARM NEON implementation of u8x16_word_shift_{left,right}Damjan Marion1-0/+12
Type: improvement Change-Id: I310e421513e9d3f96ad7debc72c9407e231962b8 Signed-off-by: Damjan Marion <dmarion@me.com>
2020-02-11vppinfra: remove the historical mheap memory allocatorDave Barach6-1955/+6
The mheap allocator has been turned off for several releases. This commit removes the cmake config parameter, parallel support for dlmalloc and mheap, and the mheap allocator itself. Type: refactor Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I104f88a1f06e47e90e5f7fb3e11cd1ca66467903
2020-02-11misc: fix coverity warningsDave Barach1-1/+1
Type: fix Ticket: VPP-1837 Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I6b1ea13fc83460bf4ee75cb9249d83dddaa64ded
2020-02-11vppinfra: add ARM cpu typesDamjan Marion1-0/+4
Type: improvement Change-Id: Ib2cb708fdcb14fdea9298c10d67f8fe73887f18b Signed-off-by: Damjan Marion <dmarion@me.com>
2020-02-11vppinfra: remove unused variable from timing wheelKlement Sekera2-4/+0
Type: fix Change-Id: I77b03efcac04cc46550d03657464ab8de5d7da78 Signed-off-by: Klement Sekera <ksekera@cisco.com>
2020-02-11svm: numa awareness for ssvm segmentsFlorin Coras2-1/+40
Type: feature Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I999836a7893a89aac5243b111eac35fddd03e2a6
2020-02-10vppinfra: use vm memory allocator for numa mappingFlorin Coras4-45/+17
Type: refactor Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I13b239cd572ae6dfaec07019d3d9b7c0ed3edcfa
2020-02-07 misc: address sanitizer: fix vm instrumentationBenoît Ganne2-0/+9
Type: fix Change-Id: Ifd61c0683c85fe7340965c225ed23e46ec88e01a Signed-off-by: Benoît Ganne <bganne@cisco.com>
2020-02-05vppinfra: numa vector placement supportDave Barach11-60/+282
Type: feature Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I7e7d95a089dd849c1f01ecea84529d8dbf239f21
2020-02-05vppinfra: deal with 0 return from os_cpu_clock_frequency()Dave Barach1-0/+16
Sporadic reports of os_cpu_clock_frequency() returning 0.0 in highly parallel container environments. To avoid immediate division by zero: Step 1: try estimate_clock_frequency(1e-3). Step 2: give up. Pretend we have a 2gHz clock. Type: fix Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I19d0fe5259b757ab778599c7026ce485153b43fa
2020-02-03vppinfra: fix typo in tw_timer_template.cDave Barach1-1/+1
Fix minor memory leak Type: fix Ticket: VPP-1833 Fixes: 4af9ba1dab Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: Id10fba70471ca78f73f14146054f6b12c5d4431f
2020-01-31crypto-native: add ARMv8 AES-CBC implementationDamjan Marion1-0/+6
Type: feature Change-Id: I32256061b9509880eec843db2f918879cdafbe47 Signed-off-by: Damjan Marion <dmarion@me.com>
2020-01-30vppinfra: improve clocks_per_second convergenceDave Barach2-54/+76
Apply exponential smoothing to the clock rate update calculation in clib_time_verify_frequency(), with a half-life of 1 minute and a sampling frequency of 16 seconds. Within 5 minutes or so, the calculation converges With each rate recalculation: reset total_cpu_time based on the kernel timebase delta since vpp started, and the new clock rate Improve the "show clock [verbose]" debug CLI command. BFD echo + echo fail tests marked off until the BFD code can be reworked a bit. Type: fix Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I24e88a78819b12867736c875067b386ef6115c5c
2020-01-27vppinfra: use CPUID provided base frequency if availableDamjan Marion1-0/+32
Type: fix Change-Id: Ifb007207be97119e07c3a0eba4714eb519de043c Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-01-27vppinfra: add x86 CPU definitionsDamjan Marion1-0/+8
Type: feature Change-Id: I9d1f9f00ac011a93709850186dcf4cf5ea3bf88a Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-01-16vppinfra: fixing compilation issues in 32-bitVijayabhaskar Katamreddy1-2/+3
Fixing compilation issuues for 32-bit also setting init flag for shm based bihash Type: fix Signed-off-by: Vijayabhaskar Katamreddy <vkatamre@cisco.com> Change-Id: Ic2072c5ba7fc77d061ca9f1b844a71f6e22e58b2
2020-01-08vppinfra: retry socket connect on EAGAINFlorin Coras1-4/+6
Type: fix Signed-off-by: Florin Coras <fcoras@cisco.com> Change-Id: I93577acf559a8fa639aab7ec3f7cdbe7df9a248d
2020-01-07vppinfra: hash [un]set malloc/free takes a const keyNeale Ranns1-5/+5
Type: refactor the key is not modified by these functions Change-Id: I578f054355fca69e8a086bb69013155a01ed759f Signed-off-by: Neale Ranns <nranns@cisco.com>
2019-12-17pg: don't leak open files in packet-generatorChristian E. Hopps1-0/+1
Fix pg code to close it's open file descriptors before zero'ing the pcap_main structure for re-use. Ticket: VPP-1780 Type: fix Signed-off-by: Christian E. Hopps <chopps@chopps.org> Change-Id: I32945c6476ae83b8d210ee67ac78db3e8f786f46
2019-12-17misc: address sanitizer: do not instrument dlmalloc internalsBenoît Ganne1-0/+4
Type: fix Change-Id: I2b8273666db864d80012c39623ae866ac3527426 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2019-12-17misc: address sanitizer: fix instrumentationBenoît Ganne2-1/+4
Type: fix Change-Id: I99e3951f8cfb7ab9d2f0a7dcee92199eab29043c Signed-off-by: Benoît Ganne <bganne@cisco.com>
2019-12-17vppinfra: fix cpu flag string overflowBenoît Ganne1-7/+5
Type: fix Change-Id: Idb1fff8a172034044bb33d5b271a84d1fd672ef5 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2019-12-16vppinfra: bihash walk cb typedef and continue/stop controlsNeale Ranns3-7/+24
Type: feature Change-Id: I28f7a658be3f3beec9ea32635b60d1d3a10d9b06 Signed-off-by: Neale Ranns <nranns@cisco.com>
2019-12-02vppinfra: preserve total_cpu_time across clock freq adjustmentDave Barach2-4/+17
If clib_time_verify_frequency() adjusts the clock frequency, transform total_cpu_time to the new time coordinate space. Otherwise, we break comparisons with previous clib_time_now() value. Without this correction, time jumps in one direction or the other depending on the sign of the frequency change. Reasonably harmless in most cases, but under perfect storm conditions the wheels fall off. Type: fix Signed-off-by: Dave Barach <dave@barachs.net> Change-Id: I21802c2630e2c87ff817cd732b7d78bc022cd2d7
2019-11-27misc: add address sanitizer heap instrumentationBenoît Ganne10-33/+145
Introduce AddressSanitizer support: https://github.com/google/sanitizers/ This starts with heap instrumentation. vlib_buffer, bihash and stack instrumentation should follow. Type: feature Change-Id: I7f20e235b2f79db72efd0e756f22c75f717a9884 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2019-11-26build: remove valgrind leftoversBenoît Ganne5-4360/+0
Valgrind never really worked well with VPP. Remove the partial support. Type: refactor Change-Id: Ic09773fd85f904fdd2240bc161e23a4c2b196cf6 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2019-11-22vppinfra: fix coverity issue 205691 in pmallocAndrew Yourtchenko1-0/+1
set the address to MMAP_FAILED if mmap has failed, so that we do not attempt to free it in the error handling path. Change-Id: I6e6b51a365fb68086dc20aa40a676a36af59a3ba Type: fix Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>
2019-11-07pmalloc: always lock pagesDamjan Marion1-25/+11
Type: feature Change-Id: I5bbf37969c9c51e40a013d1fc3ab966838eeb80d Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-11-05session tcp: refactor pacer timeFlorin Coras1-0/+7
Type: refactor Change-Id: Ic1c3e1f7987702cd88972acc34849dc1f585d5fe Signed-off-by: Florin Coras <fcoras@cisco.com>