aboutsummaryrefslogtreecommitdiffstats
path: root/tests
AgeCommit message (Collapse)AuthorFilesLines
2021-12-22add 2-node ipsec hwasync casesxinfeng zhao11-0/+1837
Signed-off-by: xinfeng zhao <xinfengx.zhao@intel.com> Change-Id: I1590d438216e4c886aae03f83364837153623dbc
2021-12-15UTI: Export resultsVratko Polak22-9/+83
+ Model version 1.0.0. - Only some result types are exported. + MRR, NDRPDR and SOAK. - Other result types to be added later. + In contrast, all test types are detected. + Convert custom classes to JSON-serializable equivalents. + Sort dict keys before converting to JSON. + Override the order for some known keys. + Export sets as sorted arrays. + Convert to info content from serialized raw content. + Also export outputs for suite setups and teardowns. + Info files for setup/teardown exist only temporarily. + The data is merged into suite.info.json file. + This simplifies presentation of total suite duration. + Define model via JSON schema: - Just test case, suite setup/teardown/suite to be added later. - Just info, raw to be added later. + Proper descriptions. + Json is generated from yaml. + This is a convenience for maintainers. + The officially used schema is the .json one. + TODOs written into a separate .txt file. + Validate exported instance against the schema. + Include format checking. + Update CSIT requirements for validation dependencies. + This needs python-dateutil==2.8.2, only a patch bump. + Compute bandwidth also for soak tests. + This unifies with NDRPDR to simplify schema definition. - PAL may need an update for parsing soak test message. + Include SSH log items, raw output only. + Generate all outputs in a single filesystem tree. + Move raw outputs into test_output_raw.tar.xz. + Rename existing tar with suites to generated_robot_files.tar.xz. Change-Id: I69ff7b330ed1a14dc435fd0ef008e753c0d7f78c Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-12-15docs: small update structure fixViliam Luc9-22/+19
Signed-off-by: Viliam Luc <vluc@cisco.com> Change-Id: Ib0a853f7bd8cdbb0dee747e4f601bc4e54d006e9
2021-12-13doc: fixing documentation structureViliam Luc678-13631/+17013
When building documentation using sphinx we see ~1200 similar warnings [0] [0] - https://s3-logs.fd.io/vex-yul-rot-jenkins-1/csit-verify-tox-master-ubuntu2004-x86_64/3289/doc_verify.log.gz These warning are harmless and can be fixed later Signed-off-by: Viliam Luc <vluc@cisco.com> Change-Id: I1ac1099d38935971d47491dde905715345d3935c
2021-11-29Remove executable flag from suite filesVratko Polak4-0/+0
If ./{filename} does not execute, {filename} should not be flagged as executable. No .robot file is executable. Change-Id: I7add9d223ad0e587bc041a4cca6bac222a89e078 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-11-24NAT44det: Decrease ramp-up rate at big scalesVratko Polak2-4/+4
Some testbeds occasionally miss one session with the previous value. + Increase ramp-up duration proportionally. Change-Id: I5300127d7e8fbc670c37ae392ab514b077b3c9b1 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-11-03Re-adressing ipsec int tests to fit deployment in AWSMaros Mullner91-385/+385
Signed-off-by: Maros Mullner <maros.mullner@pantheon.tech> Change-Id: I83b00d56a50d98c36547621cb2f41cb3b2f35741
2021-11-03perf: remove depracted ipsec scale reconf test suitesMaciek Konstantynowicz30-5075/+0
Remaining ipsec reconf test suites for 40 tunnels include: - int-aes128cbc-hmac512sha - int-aes256gcm Change-Id: I9c5f549368595bb0cbd0d7e9918b7695184d31c4 Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
2021-11-03fix(Tests): Hoststack driverspmikus4-8/+0
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: If98f575503a59106fe8b94d5c683881412ea4f1c
2021-11-02crypto: remove deprecated ipsec scale perf test suitesMaciek Konstantynowicz146-23984/+0
Remove following test suites: - ipsec tests - ethip4ipsectptlispgpe. - policy-aes128gcm. - policy-aes128cbc-hmac256sha. - policy-aes128cbc-hmac512sha. - int-aes128cbc-hmac256sha. - scale of - 400tnlsw. - 5000tnlsw. - 20000tnlsw. - 60000tnlsw. Change-Id: Ib4915213f6c440c70f9b17cbaac834245e51bd85 Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
2021-10-25IPsec: add nth SPD entry outbound flow cache TCsJuraj Linkeš8-0/+1200
Add 1, 10, 100 and 1000 SPD entries using IPv4 outbound flow cache optimization. Change-Id: I7abb65a82454c17ef754cb11386186610f0c27e8 Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2021-10-20doc gen: fix doc generatorViliam Luc4-36/+36
Change-Id: I75e836ce728fd333764f17d69813093cb947ef4f Signed-off-by: Viliam Luc <vluc@cisco.com>
2021-10-08Add flow test suitesxinfeng zhao13-0/+1482
The comms DDP package needs to be installed to support all flow protocols Signed-off-by: xinfeng zhao <xinfengx.zhao@intel.com> Change-Id: I6ab1bd8beb9edb8c9889a0ed9a999080ca3cef3d
2021-10-06back-to-back: trex suites add 3n tagViliam Luc13-37/+40
Change-Id: I96a031aaf246c2f79c9d90c04038a51915ded4ea Signed-off-by: Viliam Luc <vluc@cisco.com>
2021-10-06docstring: Docstring warnings fixed.Viliam Luc120-120/+120
Change-Id: I8679c819b7c6ecd9e680bdc6edeafd2124def0b5 Signed-off-by: Viliam Luc <vluc@cisco.com>
2021-10-05GSO: Fix the add workers keywordVratko Polak4-4/+4
The old one got deduplicated in https://gerrit.fd.io/r/c/csit/+/32687 Change-Id: Ie82ba36a0805154f17e9fcde0efafb93a686c686 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-09-28Trex suites: fix TREX tagVratko Polak3-3/+3
Change-Id: I2193cc971a827d73965695c251fb00496fe3d2c7 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-09-17N2n: Fix min frame size for ip6Vratko Polak3-7/+7
Change-Id: Iee3c8ce2079e58da400e52dc4cf6633f90a71f57 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-09-17Fix in_mask values in nat44 udir suitesVratko Polak4-4/+4
New IPUtil.Vpp_Route_Add implementation detects wrong values, as it refuses start addresses with host bits set. Fixed by copying in_mask values from nearby ASTF suites. Change-Id: I768c023bd004ce97b74199d70e5768c5e948d429 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-09-14back-to-back: add tg n2n suitesViliam Luc15-2/+1031
Change-Id: I911e121475221fbf5449ed49dc6236caf13742b0 Signed-off-by: Viliam Luc <vluc@cisco.com>
2021-09-10back-to-back tests: add TG testsViliam Luc3-0/+143
Change-Id: I9d028294deb1e31b6d185deb1c7523e0226a0ada Signed-off-by: Viliam Luc <vluc@cisco.com>
2021-08-24Telemetry: vpp devicepmikus54-156/+157
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I80efb278cf824dd8b1d38bb3ff3df43366e7295a
2021-08-20FIX: job specspmikus4-4/+4
+ this is not checked by suite generator Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: Ib21cd50d6cc5e3d808fa34a3bb270277c7f36f88
2021-08-19vpp_device: Multiple NICs (suitegen)pmikus72-437/+436
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I81958fbf6ef240d53a0fb8708ca882baf02f606c
2021-08-19vpp_device: Multiple NICs (infra)pmikus56-112/+112
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: Ie8662b6a485374b5df68dec9c82065cf59f28c8f
2021-08-16Unmark 64B-avf-ethipv4-l2bdbasemaclrn-devVratko Polak1-1/+1
It is no longer expected to fail. Change-Id: I04e7f5ed3940ed93c253b5c31e2295a611858925 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-07-28IPsec: add nth matching SPD entry outbound TCsJuraj Linkeš8-0/+1176
Add testcases with plain ipv4 forwarding with 1, 10, 100 and 1000 SPD entries on outbound traffic in both directions both directions. Only match the last SPD entry and process others before the matching entry. Add testcases only without flow cache optimization. Refactor the Python functions that add SPD entries: - Unify the args in functions that add one and multiple entries. - For multiple entries, add the ability to pass an object that will handle how values in each iteration (i.e. for each entry) are modified. Change-Id: I061922eec6acc75a4e115202c07e72d89bf1f4d3 Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2021-07-15FIX: Typo's in perf suitespmikus28-28/+28
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I770bb2114d92bda4ab2408be9b075263c1cd2223
2021-06-24Perf actions: Skip vpp-runtime in stat_pre_trialVratko Polak1-1/+1
The list of actions stat_pre_trial is executed after the list stat_runtime, which already contains vpp-runtime. Change-Id: If9b403f38b2f811d77f36d9cae973130f30015da Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-06-17Core: Rework CPU allocationpmikus40-100/+104
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I6826add7b3032041632c3952c45a3c64409400b0
2021-06-16ip6lisp: Do not fail on string overhead valueVratko Polak2-2/+2
+ Set overhead in those suites to numeric values. + Change the library to tolerate string representations anyway. Change-Id: Ic6215840f7797801c994a38db5637999eb85a034 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-06-10FIX: Pylint reducepmikus17-17/+17
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I909942dbb920df7f0fe15c0c92cda92c3cd8d8ad
2021-06-10Infra: vpp_device -> Ubuntu 20.04pmikus1-20/+19
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: If24c385b93e1ba310ea258984c97494b4cab467c
2021-06-10Add gtpu sw performance test casesxinfeng zhao1-0/+138
Signed-off-by: xinfeng zhao <xinfengx.zhao@intel.com> Reviewed-by: Yulong Pei <yulong.pei@intel.com> Change-Id: Ibe4420529192559a3cdbb84e0a67859750a9e0d5
2021-05-28Framework: Telemetry retakepmikus3-46/+33
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I2f019a083916aec9f7816266f6ad5b92dcc31fa0
2021-05-26rxq_ratio default value changed to 1viliam.luc10-10/+0
Change-Id: I0c973ceffc13f4f3c5f959d56cb0890b87091a82 Signed-off-by: Viliam Luc <viliam.luc@pantheon.tech>
2021-05-26Perf: Add perfmon_plugin for telemetrypmikus808-1281/+1562
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I710af90aa5474381a82095c8a11ecf6c01c2483b
2021-05-20hoststack perf: iperf3 udp hoststack performance tests.Dave Wallace2-0/+141
Change-Id: Ie2e735971934d8aeac2d436505c120f09ce67ac7 Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2021-05-20perf: add TCP Nginx+LDPRELOAD suitesxizhanx4-0/+484
1. Suite steup add download nginx 2. Add nginx-1.14.2/1.15.0 ldp test suite 3. Add NginxUtils,NginxConfigGenerator method 4. Taskset the PID of nginx to the unused cores in VPP and these cores are under NIC's NUMA ID 5. cleanup add Kill Processes - nohup Signed-off-by: xizhanx <xix.zhang@intel.com> Change-Id: Idbf0e4ec3bf63e88281a8e3e34f52e00a6801c85 Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2021-04-01IPsec: add 4, 40 and 400 tunnel policy testsJuraj Linkeš24-0/+3852
Add more granularity test policy tests. Mirror the number of tunnels in other IPsec tests under 1000 tunnels. Change-Id: I9bde7447a5d809bab05db132bf6cb524e97e19b3 Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2021-03-31IPsec: add 2n crypto policy udir perf testsJuraj Linkeš8-0/+1276
Add 2n1l udir 1 and 1000 tunnel sw policy IPsec tests to mirror the existing 3n tests. Add static ip neighbor mac entries which can't be retrieved in 2node setups. Change-Id: I13dd557cbeed7f907fa9b4c21e4e245d48916513 Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2021-03-31Device: Add stats test with AVFVratko Polak2-21/+88
+ Extract the stats keyword into a library. Change-Id: Icfaaa32c2a97d479ba0339acf34b865006949a89 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-03-02Fix: Async scheduler tests are not hwVratko Polak12-12/+12
Suite name part is correct: swasync. This fixes the tag, from IPSECHW to IPSECSW. Change-Id: I13a5201dade7edf180d315416c82c15d69c1bfce Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-02-26IPsec: add 2n crypto udir perf testsJuraj Linkeš96-0/+15736
Add unidirectional 2n crypto tests. Only one direction can be tested on a 2 node topology, since we can't use the same interface for both encrypted and unencrypted traffic. Add the following tests: * {n}tnlsw-ip4base-int ndrpdr tests * {n}tnlsw-1atnl-ip4base-int reconf tests * {n}tnlswasync-scheduler-ip4base-int ndrpdr tests Where n is the number of tunnels: 1, 4, 40, 400, 1000, 5000, 10000, 20000, 40000, 60000 for the first two 1, 2, 4, 8 for the async scheduler tests All of these with the following ecryption-auth algorithms: aes128gcm aes256gcm aes128cbc-hmac256sha aes128cbc-hmac512sha Also add the corresponding trex profiles: trex-stl-2n-ethip4-ip4dst{n}-udir.py Where n is the number of tunnels listed above. The profiles are shared among the tests. Change-Id: I22bb46e6ad59801581a78aa19310bee8a5293e56 Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2021-02-22Ipsec: Always generate ikeyVratko Polak2-10/+10
Some tests use a crypto algorithm with no integrity algorithm. Generate empty binary strings as fake integrity keys to keep return values of low level methods consistent. + Add return_keys argument to avoid returning long lists. + Improve various docstrings. Change-Id: Idae1877bdde32d194ce4e3bb3053c8dba39d377a Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-02-19Add test suites for crypto sw scheduler engineYulong Pei16-0/+2704
This patch is to add test suites for vpp plugin crypto_sw_scheduler, IPsec sync mode is to do crypto and packet forward work in same worker cores, crypto_sw_scheduler can schedule crypto work to other async crypto cores to improve whole crypto processing capability. This test suites configure fixed 1 rx queues per port, then measure IPsec performance with 1, 2, 3 crypto cores. This patchset include 1, 2, 4, 8 ipsec tunnels test cases. +Vratko help to change to count total physical cores instead of previous only count crypto cores in test cases. Change-Id: I0e67182e3d13273890a23703d838101900e25126 Signed-off-by: Yulong Pei <yulong.pei@intel.com> Signed-off-by: Vratko Polak <vrpolak@cisco.com> Signed-off-by: pmikus <pmikus@cisco.com>
2021-02-19FIX: Naming convention compliancepmikus4-16/+16
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: Idcee623abbc5568fc6ca49113bc2ff8add179046
2021-02-19Performance: Add GSO tests to report executionpmikus4-4/+4
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I35b2f786105103f19907d74c4f6cc77d7cd3a5bb
2021-02-19Framework: GSO testpmikus4-4/+4
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I874568aebbe50c38e634438eb1201ca737bd18ad
2021-02-05NAT44ed: Add pure throuhput bidir testsVratko Polak10-0/+1409
+ With ramp-up, without reset, with session verification. + Uses the same profile as pps tests. + Ramp up duration is not specified, as duration is computed. + Timeout tracking with automated ramp-up. + Correct computation of next duration. + Checking both early and late sessions. + No loss measurement also acts as a ramp-up. + Return ReceiveRateMeasurement from send_traffic_on_tg_internal, as that is needed for detecting whether trial is ok as ramp up. - Udp needs quite low ramp-up rate after recent regression. - Max scale has higher rate (so failing) to avoid session timeouts. + Bump copyright year. Change-Id: I50c928659cd5b985b490a2e5fb69c5cd790600b0 Signed-off-by: Vratko Polak <vrpolak@cisco.com>