summaryrefslogtreecommitdiffstats
path: root/src/vppinfra
AgeCommit message (Collapse)AuthorFilesLines
2018-07-26vppinfra: add SSE4.2 version of u8x16_shuffleDamjan Marion1-0/+6
Change-Id: I4bf1cfe5a9492092a7362675079c47629b6f0ee8 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-07-25Generate vppinfra/config.h from config.in script (same as we do for ↵Neale Ranns1-0/+23
vlib/config.h) Change-Id: I55549b589e34a62d3704f788fce801392de22f46 Signed-off-by: Neale Ranns <nranns@cisco.com>
2018-07-20bihash: give hint to CPU that we are spinlockingDamjan Marion2-4/+6
Change-Id: I78c0a6da5d8fc63c1ced43589c42abc15ab12b16 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-07-20Fine-grained add / delete lockingDave Barach9-354/+234
Add a bucket-level lock bit. Use a spinlock only when actually allocating, freeing, or splitting a bucket. Should improve multi-thread add/del performance. Change-Id: I3e40e2a8371685457f340d6584dea14e3207f2b0 Signed-off-by: Dave Barach <dave@barachs.net>
2018-07-19Fix coverity warning / legitimate minor bugDave Barach1-1/+1
Of the form "if ((foo & 0x10) == 1)", as opposed to "if (foo & 0x10)" Change-Id: I6a6d276aeed4af7c1c6c78546ee68d598d54d7db Signed-off-by: Dave Barach <dave@barachs.net>
2018-07-18Add config option to use dlmalloc instead of mheapDave Barach11-8/+7149
Configure w/ --enable-dlmalloc, see .../build-data/platforms/vpp.mk src/vppinfra/dlmalloc.[ch] are slightly modified versions of the well-known Doug Lea malloc. Main advantage: dlmalloc mspaces have no inherent size limit. Change-Id: I19b3f43f3c65bcfb82c1a265a97922d01912446e Signed-off-by: Dave Barach <dave@barachs.net>
2018-07-18vppinfra: increase max bihash arena size to 512GBDamjan Marion2-11/+21
Change-Id: Ic636297df4c03303fdcb176669f0268d80e22123 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-07-16vppinfra: AVX2 interleave functionsDamjan Marion1-3/+14
Change-Id: I8688f700fccd87484da3e202ca3a070cc14eb267 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-07-12Revert "vppinfra: AVX2 blend"Dave Barach1-6/+0
Causes clang validation failures. The patch did not actually pass validation; unfortunately it received a +1 from fd.io JJB - presumably due to a race condition This reverts commit 779c865cc6c7af5bb435d8b3465d80685370edb2. Change-Id: Ica3697f8f90e67d3eae4debc597f27d7d512004a Signed-off-by: Dave Barach <dbarach@cisco.com>
2018-07-12vppinfra: AVX2 blendDamjan Marion1-0/+6
Change-Id: Ie7a64318f10ebb535c98aff4e25cdfc48f60ff33 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-07-09IGMP improvementsNeale Ranns1-0/+21
- Enable/Disable an interface for IGMP - improve logging - refactor common code - no orphaned timers - IGMP state changes in main thread only - Large groups split over multiple state-change reports - SSM range configuration API. - more tests Change-Id: If5674f1044e7e97274a711f47807c9ba689d7b9a Signed-off-by: Neale Ranns <nranns@cisco.com>
2018-07-04Fix clib_bitmap_next_clear() function when no clear bits leftJohn Lo1-0/+3
If the bitmap has no bit clear after the input bit position i, the function will return i even if its bit is set. Fix is to return the next bit just beyond the free bitmap. This can cause IP neighbor scan crash in ip_neighbor_scan() with a debug image. With production image, ip_neighbor_scan() may still function, AFAICT, with extra neighbor delete attempts for entries already deleted, until these entries are reused for new neighbors. Change-Id: If6422ef6f63908ea39651de4ccbd8cb0b294bd69 Signed-off-by: John Lo <loj@cisco.com>
2018-07-03tcp: update snd_una_max for retranmsitsFlorin Coras1-1/+1
Change-Id: I0573d0aff39581bba96e610228a10ae923a8ca06 Signed-off-by: Florin Coras <fcoras@cisco.com>
2018-07-02Add per-numa page allocation info to 'show memory'Damjan Marion2-0/+64
Change-Id: I64e4e3d68c0f3958323f30b12a26cfaafa8bad85 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-06-30bitmap: add nocheck variants for bit opsFlorin Coras2-20/+54
Change-Id: Ifd155e2980a9f8e6af9bb6b08619c15b2bf18ef1 Signed-off-by: Florin Coras <fcoras@cisco.com>
2018-06-29bihash key compare improvementsDamjan Marion3-12/+10
Looks like CPU doesn't like overlaping loads. This new codes in some cases shows 3-4 clock improvements. Change-Id: Ia1b49976ad95140c573f892fdc0a32eebbfa06c8 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-06-28Fix mheap_get_aligned() performance jackpotDave Barach2-3/+64
If non-trivial alignment (e.g. 64) requested, and the object size (e.g. 16) is smaller than (alignment_request - MHEAP_ELT_OVERHEAD_BYTES), round up the size request. This avoids creating remainder chunks, which are false-cache-line-sharing bait to begin with. Change-Id: Ie1a21286d29557d125bb346254b1be2def868b1a Signed-off-by: Dave Barach <dave@barachs.net>
2018-06-28ip: vectorized ip checksumDamjan Marion1-0/+28
Change-Id: Ida678e6f31daa8decb18189da712a350336326e2 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-06-27Tune pool_get / pool_putDave Barach2-7/+56
Stop spending cycles repeatedly tail-trimming the pool free element bitmap; possibly at the expense of slightly hurting pool_foreach peformance. Change-Id: I8a7f3e7b26c71d7496ba9393b2a167dc7f538355 Signed-off-by: Dave Barach <dave@barachs.net>
2018-06-27vppinfra: add vector horizontal add and byte swap (SSE4.2 & AVX2)Damjan Marion2-0/+31
Change-Id: I4e0fd487970796f0153a5b16333827d23b57deac Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-06-26Fix load_unaligned undefined and other possible build failuresSirshak Das1-26/+40
Add aarch64 neon intrinsics to fix build failures similar to this: error: implicit declaration of function ‘u64x2_load_unaligned’ Change-Id: I6178504a48242742df3f7d75abdaf108796cf73f Signed-off-by: Sirshak Das <sirshak.das@arm.com>
2018-06-26We don't have (yet) 128-bit unaligned load/store on ARMDamjan Marion1-2/+2
Change-Id: I16395bbf843e338cdd366d85bb4df3de95d9b265 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-06-26add backtrace in unix_signal_handlerKingwel Xie1-21/+14
crash stack backtrace will be directed to syslog 1. make use of glic backtrace in execinfo.h. the old clib_backtrace is removed 2. install SIGABRT in signal handler, but have to remove it when backtrace is done. reason is to capture stack trace caused by SIGABRT. vPP ASSERT always call os_exit then abort(). we definitely want to know the trace of this situation. It is a little tricky to avoid SIGABRT infinite loop 3. always load symbols by calling clib_elf_main_init () in main(). Otherwise, PC addresses instead of symbols will be displayed. Change-Id: I150e15b94a4620b2ea4f08c73dc3e6ad1856de1e Signed-off-by: Kingwel Xie <kingwel.xie@ericsson.com>
2018-06-26SIMD optimized linear search in clib_bitmap_first_setDamjan Marion1-2/+23
Change-Id: Ib3a55598a83cc99485b40e38e7c406ecb126fd42 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-06-25tw: add light weight timer update functionFlorin Coras4-31/+172
Because it avoids pool putting/getting the timer, this function is somewhat faster than stopping and restarting a timer. Change-Id: Id99ed9d356b0b1f7e12facfe8da193e1cd30b3ec Signed-off-by: Florin Coras <fcoras@cisco.com>
2018-06-14Add clib_bihash_search_inline_2_with_hash to bihash templateAndrew Yourtchenko1-5/+15
Use similar approach as in the clib_bihash_search_inline_with_hash to be able to do the hash calculation and lookup separately. Change-Id: Ief79aa0f9f1e42b0af88be4807ca01fac30a80d7 Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>
2018-06-13Disable bihash bucket-level cachingDave Barach3-3/+3
It'll be interesting to see what the perf trend job says about this change. Change-Id: I66307a19a865011ac9660108098874fa1481c895 Signed-off-by: Dave Barach <dave@barachs.net>
2018-06-08Time range support for vppinfraDave Barach3-0/+781
Change-Id: I2356b1e05fd868b46b4d26ade760900a5739ca4d Signed-off-by: Dave Barach <dave@barachs.net>
2018-06-05VPP API: Memory traceOle Troan4-3/+27
if you plan to put a hash into shared memory, the key sum and key equal functions MUST be set to constants such as KEY_FUNC_STRING, KEY_FUNC_MEM, etc. -lvppinfra is PIC, which means that the process which set up the hash won't have the same idea where the key sum and key compare functions live in other processes. Change-Id: Ib3b5963a0d2fb467b91e1f16274df66ac74009e9 Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Dave Barach <dave@barachs.net> Signed-off-by: Ole Troan <ot@cisco.com>
2018-06-04Configure or deduce CLIB_LOG2_CACHE_LINE_BYTES (VPP-1064)Dave Barach1-1/+4
Added configure argument "--with-log2-cache-line-bytes=5|6|7|auto" AKA 32, 64, or 128 bytes, or use the inferred value from the build host. produces build-xxx/vpp/vppinfra/config.h, which .../src/vppinfra/cache.h Kernels which implement the following pseudo-file (aka x86_64) are easy: /sys/devices/system/cpu/cpu0/cache/index0/coherency_line_size Otherwise, extract the cpuid from /proc/cpuinfo and map it to the cache line size. Change-Id: I7ff861e042faf82c3901fa1db98864fbdea95b74 Signed-off-by: Dave Barach <dave@barachs.net> Signed-off-by: Nitin Saxena <nitin.saxena@cavium.com>
2018-06-02AVF input node reworkDamjan Marion1-0/+3
Change-Id: Ib121b24935d5c706cfba6e4b6d321086a38cad91 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-05-30Fix clang compilation on aarch64: value size does not match register size.Sirshak Das1-1/+1
Fixes clang error: value size does not match register size specified by the constraint and modifier Change-Id: I83e69445eacd6570607334e086a8582addb5bdfc Signed-off-by: Sirshak Das <sirshak.das@arm.com> Reviewed-by: Brian Brooks <brian.brooks@arm.com> Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
2018-05-30vppinfra: explicitely state for signed types that they are signedDamjan Marion2-9/+9
This fixes some compilation warnings with clang on AArch64. Change-Id: Idb941944e3f199f483c80e143a9e5163a031c4aa Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-05-29Add VLIB_NODE_FN() macro to simplify multiversioning of node functionsDamjan Marion1-2/+25
Change-Id: Ibab5e27277f618ceb2d543b9d6a1a5f191e7d1db Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-05-28Change optimizaton level from tree-vectorize to O3Damjan Marion1-1/+1
Change-Id: Ia1b49d7fd5f32d9a5139df5df636b46264003a63 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-05-28Fix flowhash size computation for very large hash tablesPierre Pfister1-1/+1
Change-Id: Ieae4ff6429fc5bdcf0e243db40ab7ec00c30730a Signed-off-by: Pierre Pfister <ppfister@cisco.com>
2018-05-25bond: performance harvestingSteven2-0/+71
- hash is great. But it is a bit too slow for the DP. Use direct array indexing to quickly retrieve the slave interface. - the algorithm used by flow hash is great. But it is a bit too slow for the DP. Use l2_hash_hash() extracted from lb_hash.h which ECMP is using. It makes use of intrinsic crc32 instruction set. - shortcut modulo arithmetic when the operand is 2**x (where x up to 4) to avoid division instruction. - special case for link count == 1 in bond_tx_fn() - use clib_mem_unaligned to access data for the packet to avoid alignment error - Fix some typos for packet tracing. Change-Id: I8eae3ad497061c5473aa675ba894ee0211120d25 Signed-off-by: Steven <sluong@cisco.com>
2018-05-25Vectorized bihash_{48,40,24,16}_8 key compareDamjan Marion6-24/+83
bihash_48_8 case: Scalar code: 6 clocks SSE4.2 code: 3 clocks AVX2 code: 2.27 clocks AVX512 code: 1.5 clocks Change-Id: I40700175835a1e7321276e47eadbf9771d3c5a68 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-05-22vppinfra: add clib_count_equal_uXX and clib_memset_uXX functionsDamjan Marion4-5/+339
Change-Id: I56782652d8ef10304900cc293cfc0502689d800e Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-05-20vector functions cleanup and improvementsDamjan Marion7-93/+97
Remove functions which have native C equivalent (i.e. _is_equal can be replaced with ==, _add with +) Add SSE4.2, AVX-512 implementations of splat, load_unaligned, store_unaligned, is_all_zero, is_equal, is_all_equal Change-Id: Ie80b0e482e7a76248ad79399c2576468532354cd Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-05-19Disable vector code in vlib_buffer_enqueue_to_next if no msb mask functionDamjan Marion1-0/+2
This fixes ARM64 build where we dont have defined u16x8_msb_mask(...) Change-Id: I864f5134a0d951601810c800f587d173b3b7ef41 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-05-18Add vlib_buffer_enqueue_to_next inline functionDamjan Marion3-1/+22
Change-Id: I1042c0fe179b57a00ce99c8d62cb1bdbe24d9184 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-05-17Add buffer pointer-to-index and index-to-pointer array functionsDamjan Marion1-0/+22
Change-Id: Ib3fcc3ceb7f315389bcdecbb7d9632540a5dd6ba Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-05-11Periodic scan and probe of IP neighbors to maintain neighbor poolsJohn Lo2-0/+6
Scan IPv4 and IPv6 neigbor pool entries once a minute to keep them up to date. The neighbor of an entry is probed if its time-stamp is older than 1 minute. If the neighbor respond, its time-stamp will be updated. If there is no response from a neighbor, its entry will be deleted when the time-stamp of the entry become more than 4 minutes old. Static neighbor entries are not probed nor deleted. Implemented CLI and API to enable and disable priodic scan of IPv4, IPv6 or both types of IP neighbors. CLI is "ip scan-neighbor" and API is "ip_scan_neighbor_enable_disable". Other IP neighbor scan parameters can also be changed from their defaults via the CLI/API. Change-Id: Id1a0a934ace15d03db845aa698bcbb9cdabebfcd Signed-off-by: John Lo <loj@cisco.com>
2018-05-11VPP-1275 Fix memory leaks in IPsec CLIKlement Sekera1-1/+1
Change-Id: I1f7c634328f25b33580a215af2daeb498cd3b181 Signed-off-by: Klement Sekera <ksekera@cisco.com>
2018-05-10vppinfra: use count_trailing_zeros in sparse_vec_indexDamjan Marion3-68/+30
It is much cheaper to use ctzll than to do shift,subtract and mask in likely case when we are looking for 1st set bit in the uword. Change-Id: I31954081571978878c7098bafad0c85a91755fa2 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-05-10vppinfra: use popcnt instruction when availableDamjan Marion1-0/+8
Change-Id: Id02d613b8613a2d448840fe2d6a5e3b168a3c563 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-05-09dpdk: tx code reworkDamjan Marion1-0/+12
Change-Id: Ifea9c772e8784642433b92091f5769eb9ec06890 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-05-05autodetect alignment during _vec_resizeDamjan Marion5-8/+12
Change-Id: I2896dbde78b5d58dc706756f4c76632c303557ae Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-05-04Harmonize vec/pool_get_aligned object sizes and alignment requestsDave Barach2-0/+4
Object sizes must evenly divide alignment requests, or vice versa. Otherwise, only the first object will be aligned as requested. Three choices: add CLIB_CACHE_LINE_ALIGN_MARK(align_me) at the end of structures, manually pad to an even divisor or multiple of the alignment request, or use plain vectors/pools. static assert for enforcement. Change-Id: I41aa6ff1a58267301d32aaf4b9cd24678ac1c147 Signed-off-by: Dave Barach <dbarach@cisco.com>