aboutsummaryrefslogtreecommitdiffstats
path: root/tests/vpp
AgeCommit message (Collapse)AuthorFilesLines
2022-06-30job_specs: add wireguard perf tests to regressionYulong Pei1-1/+1
Signed-off-by: xinfeng zhao <xinfengx.zhao@intel.com> Signed-off-by: Yulong Pei <yulong.pei@intel.com> Change-Id: I9e499df973fd9990ab02fe7008d12e78b2ab089f (cherry picked from commit 325cef50672aed697e277405fe9bb9b797d02701)
2022-06-30Add Test Suite for VPP WireGuardCui,Cheng1-0/+168
Signed-off-by: Cui,Cheng <cheng.cui@intel.com> Change-Id: I71e257e5f16a887a40ff83d782a9ebdee9d05c34 Signed-off-by: Cui,Cheng <cheng.cui@intel.com> (cherry picked from commit 75ef277536c43096e7646430568d97d2cc489238)
2022-06-28feat(telemetry): ReworkPeter Mikus81-147/+147
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I0a84ce88b318c488ba2d519b20237c88b9f9f1e6 (cherry picked from commit b33b1461fe815b1a09267118644538a5b9351c60)
2022-06-21fix(hoststack): distinguish two tcp tag typesVratko Polak4-8/+8
Previous change did not consider TCP_PPS and TCP_CPS robot tags are used by two different suite types (hoststack or ASTF). This fixes the unintended impact on hoststack. + Add HOSTSTACK tag to VSAP suites. - They could also get VSAP tag, but not needed for this Change. Fixes: 1daa6fdc0bae284dee1b61f34534e59b60b7526a Change-Id: Ic583b5ae336c9b74794706fefc232f221a243c87 Signed-off-by: Vratko Polak <vrpolak@cisco.com> (cherry picked from commit f3a55b00e3f9e3418426c295cc6f9810a6eb137e)
2022-06-17Add ipsec policy test suites with flow director or rss acceleratedYulong Pei3-0/+535
The test scenario is about two private networks which communication was protected by ipsec. The test suites are to show performance impovement about ipsec esp flow lookup that offloaded by NIC flow director and rss. Verified on 3n-clx and Intel E810 NIC environment, with 64B ipsec packet flow, performance improved ~31% with 1C2T, ~110% with 2C4T, ~250% with 4C8T. Signed-off-by: xinfeng zhao <xinfengx.zhao@intel.com> Signed-off-by: Yulong Pei <yulong.pei@intel.com> Change-Id: I30aec8c5115e5a6fbef88c11d1bef2624029d1b9 (cherry picked from commit f7ee014dee09b4089924342b0fe5654b2fe91e90)
2022-06-16feat(tests): Remove PPS tests.Vratko Polak20-1779/+120
+ Delete nat44 pps suites. + Rename ip4base suites, testcases, suite tags (also tg) to -tput. + Also robot tags like UDP_PPS and TCP_PPS. - Keep traffic types and traffic profile names with pps. + They can be renamed in a different change if needed. + Update job specs. - Do not update methodology yet (avoid conflict with another Change). Ticket: CSIT-1829 Change-Id: Ic0dac441f6e8253edce88e057ab7fad1d3814c24 Signed-off-by: Vratko Polak <vrpolak@cisco.com> (cherry picked from commit 158f54ff4d416c9bd282ffd0b8e812ec4e8214f4)
2022-05-24fix(uti): Add TG type exportPeter Mikus14-28/+28
- This is actually bug not a feature. - AB to be added later - Tested on TREX and iPerf3 Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Ib6f2d13e3b9401a9fb5759e42a8a310ee11b9d41
2022-05-18Add gtpu flow offload test suiteYulong Pei2-4/+150
The implementation of GTPU offload rx is to use ip4_gtpu flow MARK action of NIC and vpp flow REDIRECT_TO_NODE and BUFFER_ADVANCE fuction to direct received gtpu flow to gtpu4-flow-input graph node, skipped ethernet-input, ip4-input, ip4-lookup, ip4-local, ip4-udp-lookup normal graph node processing. Verified on 3n-clx and Intel E810 NIC environment, single core with 64B packet, performance improve ~33% that compare with pure software way. Signed-off-by: xinfeng zhao <xinfengx.zhao@intel.com> Signed-off-by: Yulong Pei <yulong.pei@intel.com> Change-Id: I2af4589448bdb1729e4ce206a8cf3a1239c61af8 Signed-off-by: Yulong Pei <yulong.pei@intel.com>
2022-05-03feat(astf): Support framesizes for ASTFVratko Polak30-340/+1075
- No support for IMIX. + Fix a bad bug in padding (most ASTF profiles had wrong frame sizes). + Fix a big typo in TCP PPS profiles (s->c was not data, just RST). + Control transaction size via ASTF_N_DATA_FRAMES env variable. - Default value 5 leads to transactions smaller than before. + It ensures transaction is one burst (per direction) even for jumbo. + Edit autogen to set supported frame sizes based on suite id. + Both TCP and UDP use the same values: + 64B for CPS (exact for UDP, nominal for TCP). + 100B, 1518B and 9000B for TPUT and PPS. - TCP TPUT achievable minimum is 70B. + Used 100B to leave room for possible IPv6 ASTF tests. + Separate function for code reused by vpp and trex tests. - I do not really like the new "copy and edit" approach added here. + But it is a quick edit, better autogen refactor is low priority. + Consider both established and transitory sessions as valid. - Mostly for compatibility with 2202 behavior and to avoid ramp-ups. - Assuming both session states have similar enough VPP CPU overhead. + Added a TODO to investigate and maybe reconsider later. + Update the state timeout value to 240s. + That is the default for TCP (for transitory state). - UDP could keep using 300s. + But I prefer UDP and TCP to behave as similarly as possible. + Use TRex tunables to get the exact frame size (for data packets). - It is not clear why the recipe for MSS has to be this complicated. + Move code away from profile init, as frame size is not known there. + Change internal profile API, so values related to MSS are passed. + Lower ramp-up rate for TCP TPUT tests. + Because without lower rate, jumbo fails on packet loss in ramp-up. + UDP TPUT ramp-up rate also lowered (just to keep suites more similar). + Distinguish one-direction and aggregated average frame size. + Update keyword documentation where the distiction matters. + One-direction is needed for turning bandwidth limit to TPS limit. + Aggregated is needed for correct NDRPDR bandwidth result value. - TCP TPUT will always be few percent below bidirectional maximum. + That is unavoidable, as one direction sends more control packets. + Add runtime consistency checks so future refactors are safer. + Fail if padding requested would be negative. + Fail if suite claims unexpected values for packets per transaction. + Edit the 4 types of ASTF profiles to keep them similar to each other. + Move UDP TPUT limit value from a field back to direct argument. + Stop pretending first UDP packet is not data. + Apply small improvements where convenient. + Replace "aggregate" with "aggregated" where possible. + To lower probability of any future typos in variable names. + Avoid calling Set Numeric Frame Sizes twice. + Code formatting, keyword documentation, code comments, ... + Add TODOs for less important code quality improvements. - Postpone updating of methodology pages to a subsequent change. Change-Id: I4b381e5210e69669f972326202fdcc5a2c9c923b Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2022-03-31fix(geneve): Restore session multiplicationVratko Polak5-10/+10
Previous edit was restoring the correct formatting for bold text in suite documentation by a mass replace of single asterisk to double asterisk. But in geneve suites, one usage of single asterisk was numeric multiplication, which was errorneously replaces with double asterisk (power operation). This restores the multiplication. Fixes: e82ebbd96e2ba73276e2f1b6d7f9c2d8a9442c3f Change-Id: I537ebe59dd8c04768a90a5b61a348c57e3dfd149 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2022-03-22fix(nat44det): reduce ramp-up rateVratko Polak6-18/+18
The previous values were too high for 9000b bandwidth. In order to avoid any packet loss, the new values are low enough even for x710 (10 Gbps). The ramp-up rate is also considerably lowered for lower scales, to keep the rate constant. Ramp-up duration slightly increased above the minimum needed to avoid time overheads, duration stretching and rounding errors. Ticket: CSIT-1811 Change-Id: Iefa8d4d9344e3876014dc5415a4acfedc5abf429 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2022-02-04Fix: Typo in Perf.NfvDensity.DcrMemif.ChainIpsecTibor Frank35-35/+35
Change-Id: I901ef0afce782bf69b7169e0dd08ce06b6f074b0 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2022-01-17fix(IPsec): fix policy testsJuraj Linkeš28-58/+58
Replace the hardcoded SPD inbound/outbound ranges with values derived from test inputs. Add the necessary routes now that the tunnel endpoints are not in the same subnet. Also add ip neighbor entry on DUT2 for the same reason. Also replace ipsec sa dump with show ipsec all in teardown of tests where both SAs and SPDs are configured to improve troubleshooting. Change-Id: I7d89a99fcf457a701c87bf6ac07364b62802677d Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2021-12-22add 2-node ipsec hwasync casesxinfeng zhao11-0/+1837
Signed-off-by: xinfeng zhao <xinfengx.zhao@intel.com> Change-Id: I1590d438216e4c886aae03f83364837153623dbc
2021-12-15UTI: Export resultsVratko Polak2-4/+12
+ Model version 1.0.0. - Only some result types are exported. + MRR, NDRPDR and SOAK. - Other result types to be added later. + In contrast, all test types are detected. + Convert custom classes to JSON-serializable equivalents. + Sort dict keys before converting to JSON. + Override the order for some known keys. + Export sets as sorted arrays. + Convert to info content from serialized raw content. + Also export outputs for suite setups and teardowns. + Info files for setup/teardown exist only temporarily. + The data is merged into suite.info.json file. + This simplifies presentation of total suite duration. + Define model via JSON schema: - Just test case, suite setup/teardown/suite to be added later. - Just info, raw to be added later. + Proper descriptions. + Json is generated from yaml. + This is a convenience for maintainers. + The officially used schema is the .json one. + TODOs written into a separate .txt file. + Validate exported instance against the schema. + Include format checking. + Update CSIT requirements for validation dependencies. + This needs python-dateutil==2.8.2, only a patch bump. + Compute bandwidth also for soak tests. + This unifies with NDRPDR to simplify schema definition. - PAL may need an update for parsing soak test message. + Include SSH log items, raw output only. + Generate all outputs in a single filesystem tree. + Move raw outputs into test_output_raw.tar.xz. + Rename existing tar with suites to generated_robot_files.tar.xz. Change-Id: I69ff7b330ed1a14dc435fd0ef008e753c0d7f78c Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-12-15docs: small update structure fixViliam Luc9-22/+19
Signed-off-by: Viliam Luc <vluc@cisco.com> Change-Id: Ib0a853f7bd8cdbb0dee747e4f601bc4e54d006e9
2021-12-13doc: fixing documentation structureViliam Luc661-13348/+16658
When building documentation using sphinx we see ~1200 similar warnings [0] [0] - https://s3-logs.fd.io/vex-yul-rot-jenkins-1/csit-verify-tox-master-ubuntu2004-x86_64/3289/doc_verify.log.gz These warning are harmless and can be fixed later Signed-off-by: Viliam Luc <vluc@cisco.com> Change-Id: I1ac1099d38935971d47491dde905715345d3935c
2021-11-24NAT44det: Decrease ramp-up rate at big scalesVratko Polak2-4/+4
Some testbeds occasionally miss one session with the previous value. + Increase ramp-up duration proportionally. Change-Id: I5300127d7e8fbc670c37ae392ab514b077b3c9b1 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-11-03Re-adressing ipsec int tests to fit deployment in AWSMaros Mullner91-385/+385
Signed-off-by: Maros Mullner <maros.mullner@pantheon.tech> Change-Id: I83b00d56a50d98c36547621cb2f41cb3b2f35741
2021-11-03perf: remove depracted ipsec scale reconf test suitesMaciek Konstantynowicz30-5075/+0
Remaining ipsec reconf test suites for 40 tunnels include: - int-aes128cbc-hmac512sha - int-aes256gcm Change-Id: I9c5f549368595bb0cbd0d7e9918b7695184d31c4 Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
2021-11-03fix(Tests): Hoststack driverspmikus4-8/+0
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: If98f575503a59106fe8b94d5c683881412ea4f1c
2021-11-02crypto: remove deprecated ipsec scale perf test suitesMaciek Konstantynowicz146-23984/+0
Remove following test suites: - ipsec tests - ethip4ipsectptlispgpe. - policy-aes128gcm. - policy-aes128cbc-hmac256sha. - policy-aes128cbc-hmac512sha. - int-aes128cbc-hmac256sha. - scale of - 400tnlsw. - 5000tnlsw. - 20000tnlsw. - 60000tnlsw. Change-Id: Ib4915213f6c440c70f9b17cbaac834245e51bd85 Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
2021-10-25IPsec: add nth SPD entry outbound flow cache TCsJuraj Linkeš8-0/+1200
Add 1, 10, 100 and 1000 SPD entries using IPv4 outbound flow cache optimization. Change-Id: I7abb65a82454c17ef754cb11386186610f0c27e8 Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2021-10-20doc gen: fix doc generatorViliam Luc4-36/+36
Change-Id: I75e836ce728fd333764f17d69813093cb947ef4f Signed-off-by: Viliam Luc <vluc@cisco.com>
2021-10-08Add flow test suitesxinfeng zhao13-0/+1482
The comms DDP package needs to be installed to support all flow protocols Signed-off-by: xinfeng zhao <xinfengx.zhao@intel.com> Change-Id: I6ab1bd8beb9edb8c9889a0ed9a999080ca3cef3d
2021-10-06docstring: Docstring warnings fixed.Viliam Luc120-120/+120
Change-Id: I8679c819b7c6ecd9e680bdc6edeafd2124def0b5 Signed-off-by: Viliam Luc <vluc@cisco.com>
2021-10-05GSO: Fix the add workers keywordVratko Polak4-4/+4
The old one got deduplicated in https://gerrit.fd.io/r/c/csit/+/32687 Change-Id: Ie82ba36a0805154f17e9fcde0efafb93a686c686 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-09-17Fix in_mask values in nat44 udir suitesVratko Polak4-4/+4
New IPUtil.Vpp_Route_Add implementation detects wrong values, as it refuses start addresses with host bits set. Fixed by copying in_mask values from nearby ASTF suites. Change-Id: I768c023bd004ce97b74199d70e5768c5e948d429 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-08-24Telemetry: vpp devicepmikus54-156/+157
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I80efb278cf824dd8b1d38bb3ff3df43366e7295a
2021-08-20FIX: job specspmikus4-4/+4
+ this is not checked by suite generator Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: Ib21cd50d6cc5e3d808fa34a3bb270277c7f36f88
2021-08-19vpp_device: Multiple NICs (suitegen)pmikus72-437/+436
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I81958fbf6ef240d53a0fb8708ca882baf02f606c
2021-08-19vpp_device: Multiple NICs (infra)pmikus56-112/+112
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: Ie8662b6a485374b5df68dec9c82065cf59f28c8f
2021-08-16Unmark 64B-avf-ethipv4-l2bdbasemaclrn-devVratko Polak1-1/+1
It is no longer expected to fail. Change-Id: I04e7f5ed3940ed93c253b5c31e2295a611858925 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-07-28IPsec: add nth matching SPD entry outbound TCsJuraj Linkeš8-0/+1176
Add testcases with plain ipv4 forwarding with 1, 10, 100 and 1000 SPD entries on outbound traffic in both directions both directions. Only match the last SPD entry and process others before the matching entry. Add testcases only without flow cache optimization. Refactor the Python functions that add SPD entries: - Unify the args in functions that add one and multiple entries. - For multiple entries, add the ability to pass an object that will handle how values in each iteration (i.e. for each entry) are modified. Change-Id: I061922eec6acc75a4e115202c07e72d89bf1f4d3 Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2021-07-15FIX: Typo's in perf suitespmikus28-28/+28
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I770bb2114d92bda4ab2408be9b075263c1cd2223
2021-06-24Perf actions: Skip vpp-runtime in stat_pre_trialVratko Polak1-1/+1
The list of actions stat_pre_trial is executed after the list stat_runtime, which already contains vpp-runtime. Change-Id: If9b403f38b2f811d77f36d9cae973130f30015da Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-06-17Core: Rework CPU allocationpmikus40-100/+104
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I6826add7b3032041632c3952c45a3c64409400b0
2021-06-16ip6lisp: Do not fail on string overhead valueVratko Polak2-2/+2
+ Set overhead in those suites to numeric values. + Change the library to tolerate string representations anyway. Change-Id: Ic6215840f7797801c994a38db5637999eb85a034 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-06-10FIX: Pylint reducepmikus16-16/+16
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I909942dbb920df7f0fe15c0c92cda92c3cd8d8ad
2021-06-10Infra: vpp_device -> Ubuntu 20.04pmikus1-20/+19
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: If24c385b93e1ba310ea258984c97494b4cab467c
2021-06-10Add gtpu sw performance test casesxinfeng zhao1-0/+138
Signed-off-by: xinfeng zhao <xinfengx.zhao@intel.com> Reviewed-by: Yulong Pei <yulong.pei@intel.com> Change-Id: Ibe4420529192559a3cdbb84e0a67859750a9e0d5
2021-05-28Framework: Telemetry retakepmikus2-33/+24
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I2f019a083916aec9f7816266f6ad5b92dcc31fa0
2021-05-26rxq_ratio default value changed to 1viliam.luc10-10/+0
Change-Id: I0c973ceffc13f4f3c5f959d56cb0890b87091a82 Signed-off-by: Viliam Luc <viliam.luc@pantheon.tech>
2021-05-26Perf: Add perfmon_plugin for telemetrypmikus803-1276/+1557
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I710af90aa5474381a82095c8a11ecf6c01c2483b
2021-05-20hoststack perf: iperf3 udp hoststack performance tests.Dave Wallace2-0/+141
Change-Id: Ie2e735971934d8aeac2d436505c120f09ce67ac7 Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2021-05-20perf: add TCP Nginx+LDPRELOAD suitesxizhanx4-0/+484
1. Suite steup add download nginx 2. Add nginx-1.14.2/1.15.0 ldp test suite 3. Add NginxUtils,NginxConfigGenerator method 4. Taskset the PID of nginx to the unused cores in VPP and these cores are under NIC's NUMA ID 5. cleanup add Kill Processes - nohup Signed-off-by: xizhanx <xix.zhang@intel.com> Change-Id: Idbf0e4ec3bf63e88281a8e3e34f52e00a6801c85 Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2021-04-01IPsec: add 4, 40 and 400 tunnel policy testsJuraj Linkeš24-0/+3852
Add more granularity test policy tests. Mirror the number of tunnels in other IPsec tests under 1000 tunnels. Change-Id: I9bde7447a5d809bab05db132bf6cb524e97e19b3 Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2021-03-31IPsec: add 2n crypto policy udir perf testsJuraj Linkeš8-0/+1276
Add 2n1l udir 1 and 1000 tunnel sw policy IPsec tests to mirror the existing 3n tests. Add static ip neighbor mac entries which can't be retrieved in 2node setups. Change-Id: I13dd557cbeed7f907fa9b4c21e4e245d48916513 Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2021-03-31Device: Add stats test with AVFVratko Polak2-21/+88
+ Extract the stats keyword into a library. Change-Id: Icfaaa32c2a97d479ba0339acf34b865006949a89 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-03-02Fix: Async scheduler tests are not hwVratko Polak12-12/+12
Suite name part is correct: swasync. This fixes the tag, from IPSECHW to IPSECSW. Change-Id: I13a5201dade7edf180d315416c82c15d69c1bfce Signed-off-by: Vratko Polak <vrpolak@cisco.com>