aboutsummaryrefslogtreecommitdiffstats
path: root/resources/libraries
AgeCommit message (Collapse)AuthorFilesLines
2020-11-16Ansible: Remove vpp_device snergster dependencypmikus2-9/+9
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I145a4b5511141f1e2b4e387daa358e32dd2c8015
2020-11-10TCP CPS: Count NAT sessionsVratko Polak1-2/+4
The previous code counted full TCP connections, which need one more packet, leading to worse results. Change-Id: Ifcf78356b6ed54819ea0bf5aa069d7d9cb951183 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-11-10Speed up test archive compressionVratko Polak1-1/+1
16 times faster setting at cost of 50% larger archive. Before: $ time ( cat tests.tar | xz -9e > tests.9e.tar.xz ) real 0m41.640s user 0m41.009s sys 0m0.602s $ wc -c tests.9e.tar.xz 913432 tests.9e.tar.xz After: $ time ( cat tests.tar | xz -3 > tests.3.tar.xz ) real 0m2.422s user 0m2.294s sys 0m0.161s $ wc -c tests.3.tar.xz 1556080 tests.3.tar.xz Change-Id: Ia5c86183d4e19db3cbf95dd84cf86ee720ef4051 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-10-29Support existing test types with ASTFVratko Polak17-1091/+2048
+ Add UDP_CPS, TCP_CPS, UDP_PPS and TCP_PPS suites. + Update existing cps traffic profiles. + Add missing traffic profiles. + UDP: + Single burst of 32 packets was confirmed as safe enough for TRex. + Maybe 64 could work, but not enough testing for that. + Multiple bursts have lead to reduced TRex performance, as overlaping bursts (from different client instances) tend to fill up the buffers. + TCP: + Data size set to 11111 bytes, completely arbitrarily. + Results look reasonable, so I have kept that. - MSS not set at all - No tested support for frame size other than 64B. - Frame size does not even factor into TCP profiles. + So other frame sizes are skipped in autogen. + Update tags in related suites. - HOSTS_{n} and SRC_USER_{n} should be unified. - Questionable clarification on difference between IP4BASE and SCALE. + Add NAT state resetters to tests that need them. + Resetter is called (if set) before each measurement. + If ramp-up is detected, resetter is not set. + Rename "mult" argument to "multiplier". + Abstracted from packets to transactions. + Transaction corresponds to profile. + TRex multiplier argument sets target rate in transactions per second. + The familiar STL traffic: + Bidirectional is considered to be 2 packets per transaction. + Unidirectional is considered to be 1 packet per transaction. + The newer ASTF traffic: + 4 subtypes, each has different number of packets per transaction. + For max rate computation: + Packets in the more numerous direction are considered. + Rely on TRex reported traffic duration for ASTF: + Use the server side value. - Client side value is higher by an overhead. - TRex is not sending traffic during that time. + Remove delays from traffic profiles. - Those delays would increase the reprted traffic time. + Support for scale lmited trials. + Only for ASTF profiles, each ASTF profile has limited scale. + Scale defined in suite variables. + For TRex to send all transactions provided duration value is ignored. + The appropriate value is computed in TrafficGenerator. + An ad-hoc time constant is added to match the TRex client side time overhead. + The profile driver receives the computed duration. + Measurement for PLRsearch add a sleep if the computed duration is smaller. + Alternative argument for search algos if scale is limited. + Both need higher timeout to accomodate big scales. + MLRsearch can afford fewer phases. + Added a parameter to optionally shorten the duration. + Use short duration for runtime stats trial and failure stats trial. + Use very large keepalive values in udp profiles to avoid ka packets. + No polling in ASTF profile driver. - Polling could eliminate the time overhead value. + But polling proved to introduce some loss, affecting the results. + Handle duration stretching in ASTF by stopping traffic. + The stop has several steps so that: + The traffic is really stopped entirely. + Late packets do not count (maybe as errors). + Stats are preserved to read for results (and cleared afterwards). + Several quantities added to ReceiveRateMeasurement: + Original target duration is preserved (algos need that). + Input estimate (tps) for early search iterations. + Output estimate (maybe pps) for MRR output. + Strict result (unsent counts as loss) for NDR. + Use L2 counters (opackets, ipackets) where possible. - TRex has trouble processing packets for the L7 ones at high loads. + Remove warmup from profile drivers and keywords. + Suites should call "Send ramp-up traffic" explicitly if needed. + Added parsing for few more counters. + Both to use in formulas or just for debug purposes. - Only 64B cases in autogen, framesize support to be added later. + Latency streams during search can be enabled via PERF_USE_LATENCY env var. + MLRsearch improvments: + Rename argument names to min_rate and max_rate. + Use relative receive rate in initial phase. + PLRsearch improvements: + Careful computation when output (pps) does not match input (tps). + Use geometric distribution (instead of Poisson). + Helps agains math errors. + This should improve estimate stability. - But in practice big losses still lead to significant jumps. + Traffic generator improvements: + send_traffic_on_tg now calls the full set_rate_provider_defaults. + _send_traffic_on_tg_internal for the logic without provider defaults. + As the internal function is re-used by measure() without affecting defaults. + Move _parse_traffic_results just before get_measurement_result. + As the latter uses fields set bu the former, it is now easier to read. + Multiple sources for approximate duration. + Tried from more precise to more available. + Includes logic for _pps tests (added in later change). + Move explicit type conversions to earlier occurences. + Profile driver output field uses semicolons to simplify parsing. + Performance Robot lib file split to several smaller ones. + performance_actions.robot: + Hosts Additional Statistics Action For * keywords. + performance_display.robot: + Hosts keyword for displaying and verifying results. + Change test message to use the correct unit (pps or cps). + performance_limits.robot renamed to performance_vars.robot + Added many keywords, mostly for accessing test variables. + Moved variables for Policer into a new keyword there. + Some keywords need sophisticated logic. - Other are basically Get Variable Value. + But in future more logic can be added, without editing callers. + Documentation for the new keywords acts as a documentation for test variables. + performance_utils.robot has the rest. + Eliminated arguments if the value is in test variable. + Small improvements to documentation. - Still not enough cleanup with respect to arguments and test variables. + Keywords are sorted alphabetically now in each one. + Suites: + Unified variables table: + No colons in comments. + ${n_hosts}, ${n_ports} and use them instead hardcoded numbers. + Add -cps to existing cps suite names. + Remove "trial data overwrite". + Compute max rate as in STL suites. + Each NAT suite has ip4base suite to compare results to. - Those act as indirect TRex calibration. - VPP does not lose packets in those. + Latency in ASTF suites is disabled hard. - As we do not support latency in ASTF profiles yet. + Unidirectional tests governed by suite variable, not an argument. + Write long argument lists vertically. + Prefer to use argument names. + In Python, also the last argument is followed by comma. + It makes renaming and reordering easier. + Similarly applies to prints with long lists of values. + A TODO to update api crc file comments. Change-Id: I84729355edbec051298a9de1162107f88ff5737d Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-10-29tests: Add GSO enabled testsMartin Balaz9-25/+151
Change-Id: I409b060f64ae7c6787448ae519fd76d8384e9ffb Signed-off-by: Martin Balaz <martin.balaz@pantheon.tech>
2020-10-23CSIT-1597 NAT44 API: dynamic configJan Gelety2-1/+70
- cover API changes in VPP: https://gerrit.fd.io/r/c/vpp/+/29463 - update vpp stable to version 21.01-rc0~283-g5f4f2081c Change-Id: I079c10e4537448c3b078f22c3fe4ed266a5e2e2c Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-10-20FIX: Namespace deletionpmikus2-4/+13
+ Not sure who was doing code. + Set interface up should have namespace version Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I1103cddb9dd11e25a8ab0ce3f5126b2db9444749
2020-10-19FIX: disable sending IPv6 RA messages from VMsJuraj Linkeš5-19/+20
There were occasional Router Advertisement packets received in a Traffic Generator in VM testcases. Fix by adding config that disables sending IPv6 RA messages on interfaces in VM VPP config before enabling them. Also rename related keywords and change descriptions based on https://docs.fd.io/vpp/21.01/db/d4c/clicmd_src_vnet_ip6-nd.html#clicmd_ip6_nd Change-Id: I1c2a33337ac91f6039d287b4c2aac388e3a52383 Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2020-10-19Tests: Remove GBPpmikus3-447/+0
- Specs - Tests - Library - TODO: PAL Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I8890940d56c7a9924d4a5f9365b6bb881610db14
2020-10-14vpp_device: fix shim hostnameJuraj Linkeš1-5/+4
Replace the hacky grep of /etc/resolv.conf with default docker gateway IP. Change-Id: Iec3a4658826f2ba871acb14d511e9c79a1273290 Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2020-10-12Infra: Enable AMDpmikus1-0/+4
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I3954157e91aafd370c3ac0843708903d19b84936
2020-10-07vpp_device: updates for 1n-tx2 testbedsJuraj Linkeš1-11/+7
One ThunderX2 9975 server (.69) was replaced with two ThunderX2 9980 (.70, .71) servers. Move the .69 server under ansible perf section in anticipation of repurposing it for that purpose. Update the ansible scripts with .70 and .71 config and rename port names in device.sh lib to reflect the NIC differences between .69 and .70 (and .71). Change-Id: I88b75648735243e5559175d3192ffcc8fc70071c Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2020-09-12Test: Add ramp-up phase to nat44ed-udir testsJan Gelety2-3/+61
Change-Id: I3aa50ec1ef9b0445014daa31e767323060f4a03f Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-09-30Framework: Add possibility to send ramp-up trafficJan Gelety2-22/+43
Change-Id: Ie24184ca4ac2d6c7abc32f0f103e10bc402ad93b Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-09-30test: compute max_translations_per_thread based on worker threads numberJan Gelety1-0/+16
Change-Id: I1c638aef886bf37a9feb4a29e4949c7c8f19b717 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-09-30Framework: main heap size value multiplicationJan Gelety2-1/+5
- default value for main heap size set to 2G - allow allocation of n x 2G in tests that need it Change-Id: I02dafe6bc61649d2823486a3e983f66f5a213a88 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-09-28Constants: Lower Mellanox pps limitVratko Polak1-1/+7
The previous 60 Mpps stil leads to ~30% duration stretching. + Add comment on why 36 Mpps was chosen as the new limit. Change-Id: Ic11e8ece03939bdc8680cd7bc4122373583a2f17 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-09-25FIX: Workaround AVF interface down until VPP-1934pmikus1-1/+7
- Follow up https://jira.fd.io/browse/VPP-1934 Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: Id0a26c5f67f229480332530a8531401d954f4422
2020-09-24test: nat44det - add session number checkJan Gelety4-5/+70
- some tests need to reduce rate for ramp-up phase - some tests need to extend trail duration in ramp-up phase - removed 2n1l-10ge2p1x710-ethip4udp-nat44det-h1-p63-s63 suite as nat out ports are randomly selected from available port range so T-Rex stateless is not able to provide required out2in traffic Change-Id: I1145496610d202f81d911e68aa819844d7600918 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-09-23FIX: Race conditionpmikus2-0/+14
avf tests works, but vfio-pci tests are following, the interfaces must be in down state (ideally unbind, as vpp cannot pick them). Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I77af85ec4239059a5455ef68683ca129548bd7bd
2020-09-23FIX: Mellanox handlingpmikus1-2/+2
- From 2n-zn2 testing Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I9b1f0916f0f1d90a223918cfe48409d29f2ee773
2020-09-23Infra: Add 2n-xn2pmikus1-7/+10
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I2ff6526b5209dabf70c6593f60162d9b8c147014
2020-09-23Framework: AVFpmikus1-0/+22
Fix 1/3: Explicitly put PF interface up (this patch) Fix 2/3: Done on TB Fix 3/3: VPP bug displaying VF up if underlying PF is not up Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I45d66986ec76e6e14eebaad6828ef72724c626ab
2020-09-22Framework: Heapsize configurationpmikus5-25/+36
- https://gerrit.fd.io/r/c/vpp/+/28829 - https://gerrit.fd.io/r/c/vpp/+/28896 - ip6 to be added later Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I7b03b8ace5c5c7582d4a2f66bfd788a3ec80a916
2020-09-21FIX: Unknown trigger filespmikus1-5/+5
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I8d6108af943d729fecbcfe4867ea820a69b4eb1e
2020-09-11Add ipsec async mode performance test casesYulong Pei2-0/+27
In VPP 20.05, vpp added async crypto engine that support to use QAT hardware to do encryption and decryption, vnet/ipsec enabled async mode to use async crypto engine. Current async crypto engine also use dpdk_cryptodev as async handlers, in the future it may add other native QAT driver as async handlers. Note that async crypto engine is to support vnet/ipsec, it is different with current existing dpdk backend which itself has ESP implementation in plugins/dpdk/ipsec. Change-Id: I4e6eaa7ca1eddb8b1c45212de0684fb26907119b Signed-off-by: Yulong Pei <yulong.pei@intel.com>
2020-09-07VPP-DEV API COV: Add NAT44-ED testsJan Gelety2-1/+54
Jira: CSIT-1755 Change-Id: I34baa22a49f44da3fa80d91fa2f4132c982fe610 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-09-03Framework: Code aligmentspmikus7-30/+302
+ Unifying code structures - To easily plug another DUT + New PCI PassThrough templates + Improved perf stat on cores allocated in test. Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I325f17b977314f93cb91818feddfddf3e607eb8a
2020-09-03Framework: Bump DPDK 20.08pmikus6-99/+35
+ DPDK 20.08 + Migrate make -> meson + Fix all trending issues Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I31dcb22627c0f8d17ec63c5b138a2da958b006f4
2020-08-24T-Rex: 2.82, core pin, 8 workerspmikus1-2/+2
+ Bump T-Rex version. We need new features for ASTF test. + Apply core pining. Results in a more stable performance. + Tweak the number of T-Rex workers. + We need an even value to achieve ymmetric performance with pinning. + Value 8 was selected as a best compromise. This is a combination of 3 commits. This is the 1st commit message: T-Rex: 2.82 This is the commit message #2: Change Trex to CORE_MASK_PIN mode to improve performance https://trex-tgn.cisco.com/trex/doc/trex_stateless.html#_core_masking_per_interface Above link have below explaination, "When the profile is symmetric, performance can be improved by pinning half of the cores to port 0, and half of the cores to port 1, thus avoiding cache trashing and bouncing." The reason to change this is that to run CSIT with 100G NIC often failed with "TRex stateless runtime error timeout", it caused by Trex can not send enough traffic within the fixed duration. by change to CORE_MASK_PIN mode fix the issue. Not editing ASTF, as that supports different options. This is the commit message #3: Experiment: Vary number of TRex workers With CORE_MASK_PIN, we can get more predictable time distribution. Decided to use 8 workers, that gives good results both for high end (RDMA-core l2patch) and low end (vhost) tests. Change-Id: I5c61127799e0624464e960fcb980ad1c4058e744 Signed-off-by: pmikus <pmikus@cisco.com> Signed-off-by: Yulong Pei <yulong.pei@intel.com> Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-08-24Framework: Qemu alignmentspmikus2-17/+28
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I2781e85f44acffb4f8d7f02326ba2ca668dad0c5
2020-08-08Fix: NAT44 deterministic moved to separate det44 pluginJan Gelety3-91/+184
- align CSIT code with VPP code changes for NAT44 deterministic (DET44) feature - align test names according to snat44ed tests - remove obsolete 3-node nat tests - remove 2n1l-10ge2p1x710-ethip4udp-snat44det-h1048576-p63-s66060288 tests (not enough memory for such high number of sessions) Change-Id: I9a22b99b4cfa56d18e9c7ef9c58296e202567d42 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-08-20Fix: Use new-style interface variablesVratko Polak2-6/+6
The old-style variables contain None in some places, mainly in SRv6 proxy tests. Change-Id: If3887a7dba051454c504b345a6a316d5d69d0139 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-08-20Framework: use 'stl' in trex stateless profile namesJan Gelety1-3/+2
Change-Id: I74641cc89d2f25d50b67d51bf2567082b420aabb Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-08-19Add Intel E810CQ 100G NIC configurationYulong Pei2-2/+7
for Intel E810CQ 100G NIC, kernel driver of PF is ice, kernel driver of VF is iavf, its VF hardware support VPP native avf driver. Signed-off-by: Yulong Pei <yulong.pei@intel.com> Change-Id: Ic8d86e5ee00057bbbcd09df619a38bd1371c8fd7
2020-08-07Perf: NAT44 endpoint-dependent mode - udp, part IJan Gelety11-235/+525
- continuation of https://gerrit.fd.io/r/c/csit/+/26898 as there was reached limit of changes (1000) Jira: CSIT-1711 - udp synthetic profiles w/o data packets - udp cps perf tests, phase I (no special "search cps" KW) Part I means that we are using MRR tests to collect traffic data until there is ready new CPS test type with corresponding algorithm. Change-Id: I0d30feb9ecf1d0bff937152656f8eb422f831378 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-08-18Framework: Qemu alignmentspmikus1-8/+21
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I20098fca8fb513accef3edc9a72bfd3c56bf9be2
2020-08-18Framework: Alignmentspmikus1-2/+2
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I1b535ea61ab68f6e37989ffc942979cdfd24f55e
2020-08-17Framework: Profiles loadpmikus4-140/+264
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: Iab84aff31a23bb9d8e1165f5314004803fd8a501
2020-08-17Measure latency only in special casesVratko Polak2-7/+15
+ Measure latency in 90/50/10/0% PDR loads in ndrpdr tests. + Do not measure latency anywhere else. - Needs manual editing to re-enable in soak tests. Change-Id: I69fa11bfcf71012f683061c5effea52a1be91620 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-08-06Limit user triggers: no ORVratko Polak1-0/+7
To avoid runs such as: https://jenkins.fd.io/job/vpp-csit-verify-perf-master-3n-hsw/340/ Change-Id: I1b30d5f440ddf8ff32b11265b2ac2176f4b9a665 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-07-23T-Rex: Add advanced stateful modeJan Gelety5-178/+383
- provide base routines to run T-Rex in advanced stateful mode Change-Id: Ib0dc5f2919c370753335f6446860683dc4b12d93 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-07-31Framework: Perf stat capturingPeter Mikus4-10/+84
Change-Id: I3bbe1fe0073ddeead5219993675f24955e8c3dfd Signed-off-by: Peter Mikus <pmikus@cisco.com>
2020-07-30Introduce per DUT configurable statspmikus2-17/+68
- test/suite/global - binary logic is not working Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: Ia3d81cbf2c5f04d1093a0a408c84a9ffc6f3eef0
2020-07-28Soak: Avoid a possible deadlock.Vratko Polak1-3/+5
Change-Id: I31c2d7744b5cd3021132fb188480b8edec74986c Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-07-24NDRPDR: Allow smaller min_rate, 9001 ppsVratko Polak1-12/+16
+ Latency measurements need more than 9000 pps. - Previously 0% measurement used 9500 pps. Change-Id: Ic0841de096dfa8a61329f98aa1ba6d3f0ce60c66 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-07-23FIX: Log required cli_cmd not "cli_inband"Jan Gelety1-2/+2
Change-Id: If0975b1d54882390c5be418927e2961d0f4c8429 Signed-off-by: Jan Gelety <jgelety@cisco.com> Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-07-23perpatch: Echo MAKE_PARALLEL_* var before buildJuraj Linkeš1-1/+15
VPP uses MAKE_PARALLEL_FLAGS or MAKE_PARALLEL_JOBS to limit the number of cpus to use during build, so emit a line on stdout if it's used. Change-Id: I669398d474d172abb6c848a45f24f1bdd56990d8 Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2020-07-16Remove remains of WRK testsTibor Frank7-277/+0
Change-Id: If61783fb717757c6189f06924412bd079e15a08f Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-07-13Soak: Do not allow negative lossVratko Polak2-2/+11
Change-Id: I6da359d25edc415e44263d3f85f166369e564987 Signed-off-by: Vratko Polak <vrpolak@cisco.com>