summaryrefslogtreecommitdiffstats
path: root/src/vppinfra
AgeCommit message (Collapse)AuthorFilesLines
2018-06-14Add clib_bihash_search_inline_2_with_hash to bihash templateAndrew Yourtchenko1-5/+15
Use similar approach as in the clib_bihash_search_inline_with_hash to be able to do the hash calculation and lookup separately. Change-Id: Ief79aa0f9f1e42b0af88be4807ca01fac30a80d7 Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>
2018-06-13Disable bihash bucket-level cachingDave Barach3-3/+3
It'll be interesting to see what the perf trend job says about this change. Change-Id: I66307a19a865011ac9660108098874fa1481c895 Signed-off-by: Dave Barach <dave@barachs.net>
2018-06-08Time range support for vppinfraDave Barach3-0/+781
Change-Id: I2356b1e05fd868b46b4d26ade760900a5739ca4d Signed-off-by: Dave Barach <dave@barachs.net>
2018-06-05VPP API: Memory traceOle Troan4-3/+27
if you plan to put a hash into shared memory, the key sum and key equal functions MUST be set to constants such as KEY_FUNC_STRING, KEY_FUNC_MEM, etc. -lvppinfra is PIC, which means that the process which set up the hash won't have the same idea where the key sum and key compare functions live in other processes. Change-Id: Ib3b5963a0d2fb467b91e1f16274df66ac74009e9 Signed-off-by: Ole Troan <ot@cisco.com> Signed-off-by: Dave Barach <dave@barachs.net> Signed-off-by: Ole Troan <ot@cisco.com>
2018-06-04Configure or deduce CLIB_LOG2_CACHE_LINE_BYTES (VPP-1064)Dave Barach1-1/+4
Added configure argument "--with-log2-cache-line-bytes=5|6|7|auto" AKA 32, 64, or 128 bytes, or use the inferred value from the build host. produces build-xxx/vpp/vppinfra/config.h, which .../src/vppinfra/cache.h Kernels which implement the following pseudo-file (aka x86_64) are easy: /sys/devices/system/cpu/cpu0/cache/index0/coherency_line_size Otherwise, extract the cpuid from /proc/cpuinfo and map it to the cache line size. Change-Id: I7ff861e042faf82c3901fa1db98864fbdea95b74 Signed-off-by: Dave Barach <dave@barachs.net> Signed-off-by: Nitin Saxena <nitin.saxena@cavium.com>
2018-06-02AVF input node reworkDamjan Marion1-0/+3
Change-Id: Ib121b24935d5c706cfba6e4b6d321086a38cad91 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-05-30Fix clang compilation on aarch64: value size does not match register size.Sirshak Das1-1/+1
Fixes clang error: value size does not match register size specified by the constraint and modifier Change-Id: I83e69445eacd6570607334e086a8582addb5bdfc Signed-off-by: Sirshak Das <sirshak.das@arm.com> Reviewed-by: Brian Brooks <brian.brooks@arm.com> Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
2018-05-30vppinfra: explicitely state for signed types that they are signedDamjan Marion2-9/+9
This fixes some compilation warnings with clang on AArch64. Change-Id: Idb941944e3f199f483c80e143a9e5163a031c4aa Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-05-29Add VLIB_NODE_FN() macro to simplify multiversioning of node functionsDamjan Marion1-2/+25
Change-Id: Ibab5e27277f618ceb2d543b9d6a1a5f191e7d1db Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-05-28Change optimizaton level from tree-vectorize to O3Damjan Marion1-1/+1
Change-Id: Ia1b49d7fd5f32d9a5139df5df636b46264003a63 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-05-28Fix flowhash size computation for very large hash tablesPierre Pfister1-1/+1
Change-Id: Ieae4ff6429fc5bdcf0e243db40ab7ec00c30730a Signed-off-by: Pierre Pfister <ppfister@cisco.com>
2018-05-25bond: performance harvestingSteven2-0/+71
- hash is great. But it is a bit too slow for the DP. Use direct array indexing to quickly retrieve the slave interface. - the algorithm used by flow hash is great. But it is a bit too slow for the DP. Use l2_hash_hash() extracted from lb_hash.h which ECMP is using. It makes use of intrinsic crc32 instruction set. - shortcut modulo arithmetic when the operand is 2**x (where x up to 4) to avoid division instruction. - special case for link count == 1 in bond_tx_fn() - use clib_mem_unaligned to access data for the packet to avoid alignment error - Fix some typos for packet tracing. Change-Id: I8eae3ad497061c5473aa675ba894ee0211120d25 Signed-off-by: Steven <sluong@cisco.com>
2018-05-25Vectorized bihash_{48,40,24,16}_8 key compareDamjan Marion6-24/+83
bihash_48_8 case: Scalar code: 6 clocks SSE4.2 code: 3 clocks AVX2 code: 2.27 clocks AVX512 code: 1.5 clocks Change-Id: I40700175835a1e7321276e47eadbf9771d3c5a68 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-05-22vppinfra: add clib_count_equal_uXX and clib_memset_uXX functionsDamjan Marion4-5/+339
Change-Id: I56782652d8ef10304900cc293cfc0502689d800e Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-05-20vector functions cleanup and improvementsDamjan Marion7-93/+97
Remove functions which have native C equivalent (i.e. _is_equal can be replaced with ==, _add with +) Add SSE4.2, AVX-512 implementations of splat, load_unaligned, store_unaligned, is_all_zero, is_equal, is_all_equal Change-Id: Ie80b0e482e7a76248ad79399c2576468532354cd Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-05-19Disable vector code in vlib_buffer_enqueue_to_next if no msb mask functionDamjan Marion1-0/+2
This fixes ARM64 build where we dont have defined u16x8_msb_mask(...) Change-Id: I864f5134a0d951601810c800f587d173b3b7ef41 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-05-18Add vlib_buffer_enqueue_to_next inline functionDamjan Marion3-1/+22
Change-Id: I1042c0fe179b57a00ce99c8d62cb1bdbe24d9184 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-05-17Add buffer pointer-to-index and index-to-pointer array functionsDamjan Marion1-0/+22
Change-Id: Ib3fcc3ceb7f315389bcdecbb7d9632540a5dd6ba Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-05-11Periodic scan and probe of IP neighbors to maintain neighbor poolsJohn Lo2-0/+6
Scan IPv4 and IPv6 neigbor pool entries once a minute to keep them up to date. The neighbor of an entry is probed if its time-stamp is older than 1 minute. If the neighbor respond, its time-stamp will be updated. If there is no response from a neighbor, its entry will be deleted when the time-stamp of the entry become more than 4 minutes old. Static neighbor entries are not probed nor deleted. Implemented CLI and API to enable and disable priodic scan of IPv4, IPv6 or both types of IP neighbors. CLI is "ip scan-neighbor" and API is "ip_scan_neighbor_enable_disable". Other IP neighbor scan parameters can also be changed from their defaults via the CLI/API. Change-Id: Id1a0a934ace15d03db845aa698bcbb9cdabebfcd Signed-off-by: John Lo <loj@cisco.com>
2018-05-11VPP-1275 Fix memory leaks in IPsec CLIKlement Sekera1-1/+1
Change-Id: I1f7c634328f25b33580a215af2daeb498cd3b181 Signed-off-by: Klement Sekera <ksekera@cisco.com>
2018-05-10vppinfra: use count_trailing_zeros in sparse_vec_indexDamjan Marion3-68/+30
It is much cheaper to use ctzll than to do shift,subtract and mask in likely case when we are looking for 1st set bit in the uword. Change-Id: I31954081571978878c7098bafad0c85a91755fa2 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-05-10vppinfra: use popcnt instruction when availableDamjan Marion1-0/+8
Change-Id: Id02d613b8613a2d448840fe2d6a5e3b168a3c563 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-05-09dpdk: tx code reworkDamjan Marion1-0/+12
Change-Id: Ifea9c772e8784642433b92091f5769eb9ec06890 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-05-05autodetect alignment during _vec_resizeDamjan Marion5-8/+12
Change-Id: I2896dbde78b5d58dc706756f4c76632c303557ae Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-05-04Harmonize vec/pool_get_aligned object sizes and alignment requestsDave Barach2-0/+4
Object sizes must evenly divide alignment requests, or vice versa. Otherwise, only the first object will be aligned as requested. Three choices: add CLIB_CACHE_LINE_ALIGN_MARK(align_me) at the end of structures, manually pad to an even divisor or multiple of the alignment request, or use plain vectors/pools. static assert for enforcement. Change-Id: I41aa6ff1a58267301d32aaf4b9cd24678ac1c147 Signed-off-by: Dave Barach <dbarach@cisco.com>
2018-04-30Remove historical README fileDave Barach1-43/+0
Change-Id: I54a00686a7f3a61f583a5f701a0ab6c5480a455b Signed-off-by: Dave Barach <dave@barachs.net>
2018-04-25dpdk: complete rework of the dpdk-input nodeDamjan Marion4-5/+149
Change-Id: If174d189de40e6f9ffae99997bba93a2519d9fda Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-04-18vppinfra: make set_mempolicy failure non-critical unless NUMA_FORCE is setDamjan Marion1-1/+2
Change-Id: I6c1c855cf5fc2ee06f1c7ddd6576ca16cd556fdd Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-04-11Clean up temp dir in failure casesDave Barach1-0/+3
Change-Id: Icfb99a09726c01e96ff14967afbafa4116e02eff Signed-off-by: Dave Barach <dbarach@cisco.com>
2018-03-22Add circular loggingDave Barach3-21/+85
Change-Id: Ide8bf41e24a427643a3a17b1c9089993790c12a6 Signed-off-by: Dave Barach <dave@barachs.net>
2018-03-12Remove md5.[ch] from vppinfraDave Barach3-515/+0
Removed the sole use of it from ip6_neighbor.c Change-Id: Ie53cb3b6a3a41ec0917ec2042e5006d0cfaefc01 Signed-off-by: Dave Barach <dave@barachs.net>
2018-03-09Correct address calculation for VPP-1168Lee Roberts1-1/+1
Use (u64) cast to ensure proper address calculations. Change-Id: I6bad50010b140189f1b0af177e55da0045bd7a93 Signed-off-by: Lee Roberts <lee.roberts@hpe.com>
2018-03-06glibc 2.27 fixMarco Varlese1-0/+2
With glibc 2.27 the memfd_create has been added to the devel libraries. That's causing the internally defined static function to clash with the system wide one. This patch addresses that issue on systems with latest glibc libraries. Change-Id: I788bf49b23d5b5f1cb1c0374e243d8a429178a71 Signed-off-by: Marco Varlese <marco.varlese@suse.com>
2018-03-04vppinfra: fix clib_mem_vm_ext_alloc non-shared allocationsDamjan Marion2-3/+9
Change-Id: I6d049c0875b91f67f008dc04ae7efe2f8ddc276e Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-02-26Added u8x16,u32x4,u64x2 variants of _zero_byte_mask(x) for ARM/NEON ↵Adrian Oanca1-0/+20
platform. VPP-1129 Change-Id: I954acb56d901e42976e71534317f38d7c4359bcf Signed-off-by: Adrian Oanca <adrian.oanca@enea.com>
2018-02-24u8x16_compare_byte_mask - optimize to use 128bit registers as suggested by ↵Adrian Oanca1-24/+9
Nintin Change-Id: I88aabd34ef385d620695ac17ec3fe2f4a5177ada Signed-off-by: Adrian Oanca <adrian.oanca@enea.com>
2018-02-23Add prefetch inlines, update bi-hash doc tagsDave Barach2-12/+90
Change-Id: I2e9d01ccba5288e89b886464436097d3cb7d2d18 Signed-off-by: Dave Barach <dave@barachs.net>
2018-02-22bihash table size perf/scale improvementsDave Barach3-41/+73
Directly allocate and carve cache-line-aligned chunks of virtual memory. To a first approximation, bihash wasn't using clib_mem_free(...). We eliminate mheap object header/trailers, which improves space efficiency. We also eliminate the 4gb bihash table size limit. An 8_8 bihash w/ 100 million random entries uses 3.8 Gbytes. Change-Id: Icf925fdf99bce7d6ac407ac4edd30560b8f04808 Signed-off-by: Dave Barach <dave@barachs.net>
2018-02-21vppinfra: change default cache line size 64 bytesDamjan Marion1-10/+2
This change only affects Aarch64 where previously we were using 128 bytes. Change-Id: I52a3f2f3ff8c06abe8ae3933bc0d7a2a7749dd8a Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-02-21vppinfra: remove vector_iwmmxt.hDamjan Marion2-126/+0
Change-Id: I71fac34b071a07b4331cb5c900c3b8667c1fc114 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-02-21add 'is_all_zero(x)' for NEONAdrian Oanca1-0/+24
Change-Id: I5045e0f3ac4698e820b69ad46b96763e404e6fe4 Signed-off-by: Adrian Oanca <adrian.oanca@enea.com>
2018-02-20vppinfra: autogerate vector typedefs and basic inline functionsDamjan Marion5-379/+83
Change-Id: Ie9f611fa6a962b0937245f5cc949571ba11c5604 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-02-20Reset expired timer vector length after callbackDave Barach1-1/+4
Otherwise, in a "catch-up / multiple tick" case, the code will repeatedly hand previously-processed expired timer handles to the user callback. Change-Id: Idef4f242279ea41cb557bb6cff5984de02a6503d Signed-off-by: Dave Barach <dave@barachs.net>
2018-02-20vppinfra: CLIB_HAVE_VEC128 mandates SSE4.2Damjan Marion2-9/+3
Change-Id: I6511110d0472203498a4f8741781eeeeb4f90844 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-02-19Use neutral vector code for ethernet_frame_is_taggedDamjan Marion1-0/+25
Also it removes ethernet_frame_is_any_taged implemebntation which seems to be equally costly compared to two invocations of ethernet_frame_is_tagged. Change-Id: If1c95f8267cd34b807ec07e0d675cbd0db2fdf9f Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-02-18fix format_hexdumpDamjan Marion1-1/+2
Change-Id: I9710be2e722d716e22d989b3417fb49d2db0848a Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-02-14vppinfra: Remove empty fileMohsin Kazmi1-0/+0
Change-Id: I3908cc112b40d4bb52da18e7c3ac5ae0af455f87 Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com>
2018-02-09GBP pluginNeale Ranns1-0/+0
Group Base Policy (GBP) defines: - endpoints: typically a VM or container that is connected to the virtual switch/router (i.e. to VPP) - endpoint-group: (EPG) a collection of endpoints - policy: rules determining which traffic can pass between EPGs a.k.a a 'contract' Here, policy is implemented via an ACL. EPG classification for transit packets is determined by: - source EPG: from the packet's input interface - destination EPG: from the packet's destination IP address. Change-Id: I7b983844826b5fc3d49e21353ebda9df9b224e25 Signed-off-by: Neale Ranns <neale.ranns@cisco.com>
2018-02-08add CLIB_HAVE_VEC128 with NEON intrinsics (VPP-1127)Gabriel Ganne2-8/+64
Enable CLIB_HAVE_VEC128 if both aarch64 and __ARM_NEON ie. armv8 only, not armv7 Add more neon compare intrinsics wrappers. I only add simple intrinsics wrappers. More complex ones can be added later as they are needed, with performance tests on the corresponding feature to back them up. Remove wrongly added 128bits definitions defined on both armv7 and armv8 without concern for NEON instructions presence. Notable correspondinf code activations: * MHEAP_FLAG_SMALL_OBJECT_CACHE in mheap.c * ip4 fib mtrie leaves access * enable ixge plugin compilation for aarch64 (conf still disables it by default) Change-Id: I99953823627bdff6f222d232c78aa7b655aaf77a Signed-off-by: Gabriel Ganne <gabriel.ganne@enea.com>
2018-02-08Minimize bihash memory consumptionDave Barach3-118/+207
Reference-count the number of entries in each bucket. If the reference count goes to zero, free the backing store. Add long-term churn-testing to test_bihash_template.c, thanks to Andrew Yourtchenko for the initial implementation. Change-Id: I4fbd9229cacfaba8027a85cbf87b74afdead6e39 Signed-off-by: Dave Barach <dave@barachs.net>