aboutsummaryrefslogtreecommitdiffstats
path: root/tests
AgeCommit message (Collapse)AuthorFilesLines
2023-02-09Add 1M flows test suites of VPP IP4 and VPP IP6Yulong Pei4-0/+579
VPP routing with one million fib rules also is a key indicator for performance benchmark Signed-off-by: xinfeng zhao <xinfengx.zhao@intel.com> Signed-off-by: Yulong Pei <yulong.pei@intel.com> Change-Id: I19b52f6b96bdc5ed1e76305d258825fb17bd3af9 (cherry picked from commit 5a2822a899e51d3af7b588f3c88b46150ab52cd5)
2023-02-02fix(perf/hoststack/quic): enable plugin_crypto_openssl.soDave Wallace4-9/+9
- convert hoststack tests to use app-api-socket instead of vpp-api-socket Change-Id: Ibcd87f3624e33168ed3afcb30267262efc927e7c Signed-off-by: Dave Wallace <dwallacelf@gmail.com> (cherry picked from commit 61dae95975d881bc3f9cef16f1aee47f649374d8) (cherry picked from commit 2e4982841af54ea2e0367af6b28a4551db821b7f)
2023-01-24Tune host stack performance testsFilip Tehlar2-16/+26
* remove evt_qs_memfd_seg and evt_qs_seg_size as they are obsolete * use app socket api instead of vpp's api * apply rx and tx descriptors constraints * increase event queue size for nginx tests Signed-off-by: Filip Tehlar <ftehlar@cisco.com> Change-Id: Ie7ac12b8f587e0b1962884127e8eb68a4e588aee Signed-off-by: Dave Wallace <dwallacelf@gmail.com> (cherry picked from commit 752538617f43672a0a9eeb93432929032cca05d3)
2022-12-14nginx: bump down to working 1_21_5 versionViliam Luc2-20/+20
Signed-off-by: Viliam Luc <vluc@cisco.com> Change-Id: I9ff7abda9961d92c98ab08c77e79406644c115fc
2022-12-06feat(model): Cleanup and introduce telemetrypmikus1-2/+2
- Due to divergence from original design path the RAW was never consumed. It adds too much code complexity and requires processing on both storage and compute. Removing entirely to make modeling efficient. - log (apparently SSH) section will never be consumed in the way it is coded in model. This section is also not part of model schema itself due to the point above. - Introducing telemetry section that is going to carry telemetry items required for CDash. Signed-off-by: pmikus <peter.mikus@protonmail.ch> Change-Id: I7e0256c6c9715de8ee559eed29dce96329aac97d
2022-11-25nginx: new stable 1.22.1 versionViliam Luc4-266/+20
- removed old 1_15_0 - removed old 1_14_2 + updated job_specs Signed-off-by: Viliam Luc <vluc@cisco.com> Change-Id: Ie75cde28f485b809fe08f808f4fa7d31d52c3929
2022-11-14TAGS: minor update in TRex tests tagsViliam Luc12-25/+24
- fix ip6 scale tag - unified order of tags Signed-off-by: Viliam Luc <vluc@cisco.com> Change-Id: I173bd8369167f07a118f607a9c1957e354f772b2
2022-11-14CSIT-1879: Enable vxlan plugin in all vxlan test suitesSteven Luong37-40/+43
vxlan is going to a plugin. https://gerrit.fd.io/r/c/vpp/+/37511 Need to add vxlan_plugin.so to all vxlan test cases Signed-off-by: Steven Luong <sluong@cisco.com> Change-Id: Ie7f9c6c61eac4b85aeca769d467ddd0f7d5118f0 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2022-11-14CSIT-1880 - Enable vhost plugin in all vhost test suitesSteven Luong105-194/+204
vhost is going to a plugin. https://gerrit.fd.io/r/c/vpp/+/37488 Need to add vhost_plugin.so to all vhost test cases Signed-off-by: Steven Luong <sluong@cisco.com> Change-Id: I5c79795c1ba5ffcdc9d10697faffbc0b49b22abb
2022-10-03Add 10k tunnels IPSec policy mode test suitesYulong Pei4-0/+696
Also add 10k tunnels IPSec policy mode test with spd fast path enabled. spd fast path is to reduce policy adding time, refer to patchset [1] & [2] for details. [1] https://gerrit.fd.io/r/c/vpp/+/36095 [2] https://gerrit.fd.io/r/c/vpp/+/36097 Signed-off-by: xinfeng zhao <xinfengx.zhao@intel.com> Signed-off-by: yulong pei <yulong.pei@intel.com> Change-Id: I211b38a60d273c46e68a5b5f712090037977c6e6
2022-09-30Add VPP wireguard async mode test suiteYulong Pei34-61/+1081
Add VPP wireguard async mode test suite to use QAT device for crypto. Also change keyword ipsechw to cryptohw in suite_setup.robot since currently crypto device is not only used by IPSec. Signed-off-by: Yulong Pei <yulong.pei@intel.com> Signed-off-by: xinfeng zhao <xinfengx.zhao@intel.com> Change-Id: Ibdadb3b09c04b7181415ffd4a248abddc6289075
2022-09-30Add wireguard multiple tunnels test suitesYulong Pei6-4/+849
Signed-off-by: Yulong Pei <yulong.pei@intel.com> Change-Id: I7abd546e67fdbe481b204bb6a1ec7e9c654dcdae
2022-09-28telemetry: linux telemetry with perf-statViliam Luc1-2/+2
Signed-off-by: Viliam Luc <vluc@cisco.com> Change-Id: I17ced17a309cc0ac21c5fc94e570c89a456339e2
2022-07-27fix: start testpmd and l3fwd in 3 cyclesViliam Luc4-12/+16
Signed-off-by: Viliam Luc <vluc@cisco.com> Change-Id: I935ee5cf5bd7e760ef29fdc338f981e4e7cc333f
2022-06-30job_specs: add wireguard perf tests to regressionYulong Pei1-1/+1
Signed-off-by: xinfeng zhao <xinfengx.zhao@intel.com> Signed-off-by: Yulong Pei <yulong.pei@intel.com> Change-Id: I9e499df973fd9990ab02fe7008d12e78b2ab089f
2022-06-28feat(telemetry): ReworkPeter Mikus81-147/+147
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I0a84ce88b318c488ba2d519b20237c88b9f9f1e6
2022-06-17Add ipsec policy test suites with flow director or rss acceleratedYulong Pei3-0/+535
The test scenario is about two private networks which communication was protected by ipsec. The test suites are to show performance impovement about ipsec esp flow lookup that offloaded by NIC flow director and rss. Verified on 3n-clx and Intel E810 NIC environment, with 64B ipsec packet flow, performance improved ~31% with 1C2T, ~110% with 2C4T, ~250% with 4C8T. Signed-off-by: xinfeng zhao <xinfengx.zhao@intel.com> Signed-off-by: Yulong Pei <yulong.pei@intel.com> Change-Id: I30aec8c5115e5a6fbef88c11d1bef2624029d1b9
2022-06-15feat(tests): Remove PPS tests.Vratko Polak24-1803/+144
+ Delete nat44 pps suites. + Rename ip4base suites, testcases, suite tags (also tg) to -tput. + Also robot tags like UDP_PPS and TCP_PPS. - Keep traffic types and traffic profile names with pps. + They can be renamed in a different change if needed. + Update job specs. - Do not update methodology yet (avoid conflict with another Change). Ticket: CSIT-1829 Change-Id: Ic0dac441f6e8253edce88e057ab7fad1d3814c24 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2022-06-10fix(hoststack): distinguish two tcp tag typesVratko Polak4-8/+8
Previous change did not consider TCP_PPS and TCP_CPS robot tags are used by two different suite types (hoststack or ASTF). This fixes the unintended impact on hoststack. + Add HOSTSTACK tag to VSAP suites. - They could also get VSAP tag, but not needed for this Change. Fixes: 1daa6fdc0bae284dee1b61f34534e59b60b7526a Change-Id: Ic583b5ae336c9b74794706fefc232f221a243c87 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2022-06-10fix(wireguard): add missing test teardownVratko Polak1-0/+1
Fixes: 75ef277536c43096e7646430568d97d2cc489238 Change-Id: I94575bee0aa77907cc900b58b186c3612fb18d85 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2022-06-09Add Test Suite for VPP WireGuardCui,Cheng1-0/+168
Signed-off-by: Cui,Cheng <cheng.cui@intel.com> Change-Id: I71e257e5f16a887a40ff83d782a9ebdee9d05c34 Signed-off-by: Cui,Cheng <cheng.cui@intel.com>
2022-05-24fix(uti): Add TG type exportPeter Mikus14-28/+28
- This is actually bug not a feature. - AB to be added later - Tested on TREX and iPerf3 Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Ib6f2d13e3b9401a9fb5759e42a8a310ee11b9d41
2022-05-18Add gtpu flow offload test suiteYulong Pei2-4/+150
The implementation of GTPU offload rx is to use ip4_gtpu flow MARK action of NIC and vpp flow REDIRECT_TO_NODE and BUFFER_ADVANCE fuction to direct received gtpu flow to gtpu4-flow-input graph node, skipped ethernet-input, ip4-input, ip4-lookup, ip4-local, ip4-udp-lookup normal graph node processing. Verified on 3n-clx and Intel E810 NIC environment, single core with 64B packet, performance improve ~33% that compare with pure software way. Signed-off-by: xinfeng zhao <xinfengx.zhao@intel.com> Signed-off-by: Yulong Pei <yulong.pei@intel.com> Change-Id: I2af4589448bdb1729e4ce206a8cf3a1239c61af8 Signed-off-by: Yulong Pei <yulong.pei@intel.com>
2022-05-03feat(astf): Support framesizes for ASTFVratko Polak34-356/+1125
- No support for IMIX. + Fix a bad bug in padding (most ASTF profiles had wrong frame sizes). + Fix a big typo in TCP PPS profiles (s->c was not data, just RST). + Control transaction size via ASTF_N_DATA_FRAMES env variable. - Default value 5 leads to transactions smaller than before. + It ensures transaction is one burst (per direction) even for jumbo. + Edit autogen to set supported frame sizes based on suite id. + Both TCP and UDP use the same values: + 64B for CPS (exact for UDP, nominal for TCP). + 100B, 1518B and 9000B for TPUT and PPS. - TCP TPUT achievable minimum is 70B. + Used 100B to leave room for possible IPv6 ASTF tests. + Separate function for code reused by vpp and trex tests. - I do not really like the new "copy and edit" approach added here. + But it is a quick edit, better autogen refactor is low priority. + Consider both established and transitory sessions as valid. - Mostly for compatibility with 2202 behavior and to avoid ramp-ups. - Assuming both session states have similar enough VPP CPU overhead. + Added a TODO to investigate and maybe reconsider later. + Update the state timeout value to 240s. + That is the default for TCP (for transitory state). - UDP could keep using 300s. + But I prefer UDP and TCP to behave as similarly as possible. + Use TRex tunables to get the exact frame size (for data packets). - It is not clear why the recipe for MSS has to be this complicated. + Move code away from profile init, as frame size is not known there. + Change internal profile API, so values related to MSS are passed. + Lower ramp-up rate for TCP TPUT tests. + Because without lower rate, jumbo fails on packet loss in ramp-up. + UDP TPUT ramp-up rate also lowered (just to keep suites more similar). + Distinguish one-direction and aggregated average frame size. + Update keyword documentation where the distiction matters. + One-direction is needed for turning bandwidth limit to TPS limit. + Aggregated is needed for correct NDRPDR bandwidth result value. - TCP TPUT will always be few percent below bidirectional maximum. + That is unavoidable, as one direction sends more control packets. + Add runtime consistency checks so future refactors are safer. + Fail if padding requested would be negative. + Fail if suite claims unexpected values for packets per transaction. + Edit the 4 types of ASTF profiles to keep them similar to each other. + Move UDP TPUT limit value from a field back to direct argument. + Stop pretending first UDP packet is not data. + Apply small improvements where convenient. + Replace "aggregate" with "aggregated" where possible. + To lower probability of any future typos in variable names. + Avoid calling Set Numeric Frame Sizes twice. + Code formatting, keyword documentation, code comments, ... + Add TODOs for less important code quality improvements. - Postpone updating of methodology pages to a subsequent change. Change-Id: I4b381e5210e69669f972326202fdcc5a2c9c923b Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2022-04-11feat(telemetry): T-Rex runtimePeter Mikus1-2/+2
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I2829b5931e5a63a33f524e1d14ffb20c27f4ce5a
2022-03-31fix(geneve): Restore session multiplicationVratko Polak5-10/+10
Previous edit was restoring the correct formatting for bold text in suite documentation by a mass replace of single asterisk to double asterisk. But in geneve suites, one usage of single asterisk was numeric multiplication, which was errorneously replaces with double asterisk (power operation). This restores the multiplication. Fixes: e82ebbd96e2ba73276e2f1b6d7f9c2d8a9442c3f Change-Id: I537ebe59dd8c04768a90a5b61a348c57e3dfd149 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2022-03-30feat(terraform): 1n_aws_c5nPeter Mikus13-26/+39
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I12d2d58de01693b18628cd73b5d8c36467506f32
2022-03-22fix(nat44det): reduce ramp-up rateVratko Polak6-18/+18
The previous values were too high for 9000b bandwidth. In order to avoid any packet loss, the new values are low enough even for x710 (10 Gbps). The ramp-up rate is also considerably lowered for lower scales, to keep the rate constant. Ramp-up duration slightly increased above the minimum needed to avoid time overheads, duration stretching and rounding errors. Ticket: CSIT-1811 Change-Id: Iefa8d4d9344e3876014dc5415a4acfedc5abf429 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2022-02-04Fix: Typo in Perf.NfvDensity.DcrMemif.ChainIpsecTibor Frank35-35/+35
Change-Id: I901ef0afce782bf69b7169e0dd08ce06b6f074b0 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2022-01-17fix(IPsec): fix policy testsJuraj Linkeš28-58/+58
Replace the hardcoded SPD inbound/outbound ranges with values derived from test inputs. Add the necessary routes now that the tunnel endpoints are not in the same subnet. Also add ip neighbor entry on DUT2 for the same reason. Also replace ipsec sa dump with show ipsec all in teardown of tests where both SAs and SPDs are configured to improve troubleshooting. Change-Id: I7d89a99fcf457a701c87bf6ac07364b62802677d Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2021-12-22add 2-node ipsec hwasync casesxinfeng zhao11-0/+1837
Signed-off-by: xinfeng zhao <xinfengx.zhao@intel.com> Change-Id: I1590d438216e4c886aae03f83364837153623dbc
2021-12-15UTI: Export resultsVratko Polak22-9/+83
+ Model version 1.0.0. - Only some result types are exported. + MRR, NDRPDR and SOAK. - Other result types to be added later. + In contrast, all test types are detected. + Convert custom classes to JSON-serializable equivalents. + Sort dict keys before converting to JSON. + Override the order for some known keys. + Export sets as sorted arrays. + Convert to info content from serialized raw content. + Also export outputs for suite setups and teardowns. + Info files for setup/teardown exist only temporarily. + The data is merged into suite.info.json file. + This simplifies presentation of total suite duration. + Define model via JSON schema: - Just test case, suite setup/teardown/suite to be added later. - Just info, raw to be added later. + Proper descriptions. + Json is generated from yaml. + This is a convenience for maintainers. + The officially used schema is the .json one. + TODOs written into a separate .txt file. + Validate exported instance against the schema. + Include format checking. + Update CSIT requirements for validation dependencies. + This needs python-dateutil==2.8.2, only a patch bump. + Compute bandwidth also for soak tests. + This unifies with NDRPDR to simplify schema definition. - PAL may need an update for parsing soak test message. + Include SSH log items, raw output only. + Generate all outputs in a single filesystem tree. + Move raw outputs into test_output_raw.tar.xz. + Rename existing tar with suites to generated_robot_files.tar.xz. Change-Id: I69ff7b330ed1a14dc435fd0ef008e753c0d7f78c Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-12-15docs: small update structure fixViliam Luc9-22/+19
Signed-off-by: Viliam Luc <vluc@cisco.com> Change-Id: Ib0a853f7bd8cdbb0dee747e4f601bc4e54d006e9
2021-12-13doc: fixing documentation structureViliam Luc678-13631/+17013
When building documentation using sphinx we see ~1200 similar warnings [0] [0] - https://s3-logs.fd.io/vex-yul-rot-jenkins-1/csit-verify-tox-master-ubuntu2004-x86_64/3289/doc_verify.log.gz These warning are harmless and can be fixed later Signed-off-by: Viliam Luc <vluc@cisco.com> Change-Id: I1ac1099d38935971d47491dde905715345d3935c
2021-11-29Remove executable flag from suite filesVratko Polak4-0/+0
If ./{filename} does not execute, {filename} should not be flagged as executable. No .robot file is executable. Change-Id: I7add9d223ad0e587bc041a4cca6bac222a89e078 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-11-24NAT44det: Decrease ramp-up rate at big scalesVratko Polak2-4/+4
Some testbeds occasionally miss one session with the previous value. + Increase ramp-up duration proportionally. Change-Id: I5300127d7e8fbc670c37ae392ab514b077b3c9b1 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-11-03Re-adressing ipsec int tests to fit deployment in AWSMaros Mullner91-385/+385
Signed-off-by: Maros Mullner <maros.mullner@pantheon.tech> Change-Id: I83b00d56a50d98c36547621cb2f41cb3b2f35741
2021-11-03perf: remove depracted ipsec scale reconf test suitesMaciek Konstantynowicz30-5075/+0
Remaining ipsec reconf test suites for 40 tunnels include: - int-aes128cbc-hmac512sha - int-aes256gcm Change-Id: I9c5f549368595bb0cbd0d7e9918b7695184d31c4 Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
2021-11-03fix(Tests): Hoststack driverspmikus4-8/+0
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: If98f575503a59106fe8b94d5c683881412ea4f1c
2021-11-02crypto: remove deprecated ipsec scale perf test suitesMaciek Konstantynowicz146-23984/+0
Remove following test suites: - ipsec tests - ethip4ipsectptlispgpe. - policy-aes128gcm. - policy-aes128cbc-hmac256sha. - policy-aes128cbc-hmac512sha. - int-aes128cbc-hmac256sha. - scale of - 400tnlsw. - 5000tnlsw. - 20000tnlsw. - 60000tnlsw. Change-Id: Ib4915213f6c440c70f9b17cbaac834245e51bd85 Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
2021-10-25IPsec: add nth SPD entry outbound flow cache TCsJuraj Linkeš8-0/+1200
Add 1, 10, 100 and 1000 SPD entries using IPv4 outbound flow cache optimization. Change-Id: I7abb65a82454c17ef754cb11386186610f0c27e8 Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2021-10-20doc gen: fix doc generatorViliam Luc4-36/+36
Change-Id: I75e836ce728fd333764f17d69813093cb947ef4f Signed-off-by: Viliam Luc <vluc@cisco.com>
2021-10-08Add flow test suitesxinfeng zhao13-0/+1482
The comms DDP package needs to be installed to support all flow protocols Signed-off-by: xinfeng zhao <xinfengx.zhao@intel.com> Change-Id: I6ab1bd8beb9edb8c9889a0ed9a999080ca3cef3d
2021-10-06back-to-back: trex suites add 3n tagViliam Luc13-37/+40
Change-Id: I96a031aaf246c2f79c9d90c04038a51915ded4ea Signed-off-by: Viliam Luc <vluc@cisco.com>
2021-10-06docstring: Docstring warnings fixed.Viliam Luc120-120/+120
Change-Id: I8679c819b7c6ecd9e680bdc6edeafd2124def0b5 Signed-off-by: Viliam Luc <vluc@cisco.com>
2021-10-05GSO: Fix the add workers keywordVratko Polak4-4/+4
The old one got deduplicated in https://gerrit.fd.io/r/c/csit/+/32687 Change-Id: Ie82ba36a0805154f17e9fcde0efafb93a686c686 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-09-28Trex suites: fix TREX tagVratko Polak3-3/+3
Change-Id: I2193cc971a827d73965695c251fb00496fe3d2c7 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-09-17N2n: Fix min frame size for ip6Vratko Polak3-7/+7
Change-Id: Iee3c8ce2079e58da400e52dc4cf6633f90a71f57 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-09-17Fix in_mask values in nat44 udir suitesVratko Polak4-4/+4
New IPUtil.Vpp_Route_Add implementation detects wrong values, as it refuses start addresses with host bits set. Fixed by copying in_mask values from nearby ASTF suites. Change-Id: I768c023bd004ce97b74199d70e5768c5e948d429 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-09-14back-to-back: add tg n2n suitesViliam Luc15-2/+1031
Change-Id: I911e121475221fbf5449ed49dc6236caf13742b0 Signed-off-by: Viliam Luc <vluc@cisco.com>