aboutsummaryrefslogtreecommitdiffstats
path: root/tests/vpp/perf
AgeCommit message (Collapse)AuthorFilesLines
2020-01-28Ipsec: Unify first line of Local Template docVratko Polak62-94/+90
Seeing differences when diff-ing between suites is distracting. + Bump copyright year, even for files with no change. Change-Id: Iaca79647821dd8233bdbe6b0ac8b14fdb04060a8 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-28perf: hoststack iperf3 test tuningDave Wallace1-18/+11
Change-Id: I53425f57fe9ecef9cff2c94642cc7cb24537a961 Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2020-01-28Fix two auth_alg valuesVratko Polak2-2/+2
Change-Id: I0e85fc958779df3d5dbacf1ad1e3898268a832ec Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-28Update overheads for IPsec CBC testsVratko Polak32-64/+64
Updated to the values as seen in packet trace. Even if VPP creates wrongly sized packets (compared to RFCs), the overhead should correspond to the actual packet size present, in order to correctly prevent linerate on DUT-DUT link. The new overhead values are 62 (256SHA) and 78 (512SHA). The GCM value is already correct, at 54 bytes, so density tests are ok. - The lispgpe test is not updated, as it currently fails. We will update overhead there in (or after) 24578. Change-Id: I5cc6920205f37ddc80e76804fabd90b67174addf Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-28FIX: dot1qip4vxlan testsPeter Mikus24-24/+24
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Ieb6e6010108987f99f55730149e3e4b7f1a7fc21
2020-01-18FIX: VTS testsJan Gelety3-6/+6
Change-Id: I594d248c58dcdcdeceea57af2dd25e2b2e08247f Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-01-23FIX: nfv_densityJan Gelety185-370/+370
- use correct osi_layer=L2 (so L2 spoofing check is switched off in case of avf driver) - add pci address information to eth interface in topology file - nfv_density chain_ipsec tests work only with DPDK in current implementation Change-Id: I233c6e5634a14581960c7459b87f11fcee8365bd Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-01-17perf: add TCP Iperf3+LDPRELOAD test suiteDave Wallace1-0/+71
Change-Id: Icff49fb31cce342a2a4ae799e844ec91f9e5e366 Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2020-01-10Trending: new daily setJan Gelety42-353/+777
New daily sets are prepared based on information in https://gerrit.fd.io/r/c/csit/+/24073/1/docs/job_specs/perf_tests_job_specs.md and previous test set definitions in docs/job_specs/test_select_list_[2n|3n]_[clx|skx|hsw|tsh|dnv].md files. - mrr-daily-2n-clx: 510 TCs (incl. nfv_density), expected exec. time 8:50h - mrr-daily-2n-skx: 525 TCs (incl. nfv_density), expected exec. time 7:55h - mrr-daily-3n-skx: 393 TCs (incl. nfv_density), expected exec. time 11:00h - mrr-daily-3n-hsw: 177 TCs (incl. nfv_density), expected exec. time 7:10h - mrr-daily-3n-tsh: 204 TCs, expected exec. time 21:00h - mrr-daily-2n-dnv: 84 TCs, expected exec. time 2:25h - mrr-daily-3n-dnv: 144 TCs, expected exec. time 6:35h + add some missing test suites + add trex-sl-2n-ethip4udp-1000u15p.py T-Rex traffic profile + correction of TS and TC names and tags in directory tests/vpp/perf/nfv_density/chain_ipsec Change-Id: Icfc86e9af97ed8dd8ccd2a34355c99aad69a28c0 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-01-10Support suite tags in autogenVratko Polak457-0/+457
+ Include a script to add suite tags to many suites at once. + Add suite tags also to device tests (not covered by autogen). Change-Id: I514ee6178e22999b43460028fe2696738b012f04 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-10Autogen: Generate also NIC drivers.Vratko Polak197-3907/+184
+ Disallowed -avf- (or -rdma-) as "template" suites. + GBP suite switched to DPDK driver in repo. + Each NIC has its own list of supported drivers, in Constants. + Updated tag expressions for daily jobs: + Feature, ipsec, memif, scale, srv6, tunnels, vhost and vts are tested only with vfio-pci. + Other (base, dot1q, dot1ad) tested with all drivers. + Setup actions currently depend on driver, generated. - The performance_rdma action is trivial for now. - Several tests fail, to be fixed later, e.g. by performance_rdma. + Reconf tests are also supported. + Added DRV_VFIO_PCI tags missing, mainly in density tests. - Vhost suites (density, reconf) are failing, but suites look good. - TCP suites do not support NIC drivers yet. - DPDK obviously not supported. + Use Python 3 in regenerate scripts. + Fix typos binded => bound. + File open modes set either u"rt" or u"wt" everywhere. + Remove a trailing space in an environment variable name. Change-Id: I290470675dc5c9e88b2eaa5ab6285ecd9ed7827a Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-15Hoststack perf infrastructure refactoringDave Wallace3-6/+10
- DUT only topology (hoststack test apps are co-located with vpp) - Make vpp app specific keywords generic where applicable - Add IP4 Prefix to topology file - Support running wrk in linux namespace - Refactor namespace cleanup - Remove redundant namespace creation code - Refactor test/keyword dirs: tcp -> hoststack - Add hoststack utility keywords - Refactor wrk suite setup/teardown - Update tests with recent perf infra changes Change-Id: Ia1cf07978d579393eef94923819a87c8c1f36f34 Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2019-12-20VTS: Unify the testsPeter Mikus3-58/+36
- Converting to 2n as they were always 2n (with l2xc on 3rd node) - Removing KW and converting to layered approach Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Ie349c50f72eb362815e7c5ede076d421ab386e76
2019-12-16FIX: NAT44 bugPeter Mikus1-1/+2
+ From trending ... Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I0ebb5429f0ad731d42aa43855cd99ae73b1461fc
2019-12-13Python3: refactor ':FOR' statementsDave Wallace2-2/+4
Signed-off-by: Dave Wallace <dwallacelf@gmail.com> Change-Id: I76835e3d3acf6955e328f30427f9dd0098947e41
2019-12-11Introduce VPP-IPsec container tests.Ludovit Mikula36-0/+5268
Change-Id: Ie64d662e81879bd52785e0188450d998bf056bda Signed-off-by: Ludovit Mikula <ludovit.mikula@pantheon.tech>
2019-12-05Python3: IPv4 FIB scale tests with random flowsBalaji Venkatraman3-0/+400
Signed-off-by: Balaji Venkatraman <balajiv@cisco.com> Change-Id: Iadd9e8da655f3ccbd36167c1bbee37a774c3bc1f Signed-off-by: Balaji Venkatraman <balajiv@cisco.com>
2019-12-01Convert SRv6 from LXC to DockerPeter Mikus3-9/+9
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I7c948be40aaf2eb03434cec5548a89aaf0f1eca6
2019-11-28Python3: resources and librariesJan Gelety464-4145/+4148
Change-Id: I1392c06b1d64f62b141d24c0d42a8e36913b15e2 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-11-14FIX: AVF IP6basePeter Mikus1-1/+1
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Ibc3350acab54100f03f540397bfcbd6e7455ea2c
2019-11-13Add vpp loadbalancer l3dsr/nat4 mode test suiteshaiyanx1.zhang3-1/+262
Change-Id: I03603108fe7128fb4bde4abddb6a0959dd5b5c73 Signed-off-by: haiyanx1.zhang <haiyanx1.zhang@intel.com>
2019-11-11Add accidentally deleted NAT setup commandVratko Polak1-0/+1
One suite was missing "And Add NAT to all DUTs". Change-Id: I6e9c3de9f236a080cfe829db2d155a6f6ba3b3f4 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-11-11FIX: 2lbvpplacpPeter Mikus4-4/+0
- These testcases are non standard so they need to be refactored to be able to use the interface layers Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I9400846ff04f1864babd6d7fa153f0c3708cb105
2019-10-30FIX: IPSecHWPeter Mikus8-16/+0
- IPsec proto is not for GCM Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: If6a9721574a72dacb4486c837100e4f2ed2bab19
2019-10-25Introduce pre-initialize driver layerPeter Mikus445-2324/+3011
+ This will help unify driver base differences between TCs. + Decrease amount of logic needed for suite generator. Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I1e84ba361dc1e829f0612c58a61096e2633ce0c5
2019-10-19Tests: Introduce driver layerPeter Mikus445-437/+1294
+ Small bugfixes in various suites Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I051af0a6da7340fe3f617400370c3846ca5241ab
2019-10-09Add vpp loadbalancer maglev mode test suitehaiyanX1.zhang2-0/+145
Change-Id: I61555ba566efef0a2151db9a30bf7f5d9ccad1df Signed-off-by: haiyanx1.zhang <haiyanx1.zhang@intel.com>
2019-10-04FIX: IPv6 AVF dot1qPeter Mikus2-2/+4
Change-Id: Id199eb774850153fb225434d4a3d194706fdfdb3 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-09-26CSIT-1471: Policer - VAT API to PAPIpremkumarv6-18/+36
Change-Id: I6c7eff8d9cbb196ccb684a45579ac59942351cfe Signed-off-by: premkumarv <preym17@gmail.com>
2019-09-17Update CRC list and support 21997/7Vratko Polak1-0/+6
- Attempt to repair IPsec LispGpe perf suite. + Collection name to reflect the current stable vpp. + Add messages found in CSIT L1 keywords. - Uncommented (instead of deleted) untestable messages. Reasons: - Honeycomb. - Messages used by unused keywords. + Listed reasons. Honeycomb not mentioned if both reasons apply. + Delete CRC items for commands not found in keywords anymore. + Add CRCs from .json.api files (as teardown is hard to execute). + Define and restore alphabetical order. + Add hints to find used API commands (not entirely reliable). + Move used commands to "cmd = " form so hints find them. + Argument to run_cli_command changed from "cmd" to "cli_cmd". + Except also struct.error where IOError is excepted. Change-Id: I61058dbe1e33296908aabd0c13433bb16cfa6adf Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-09-13Add: AVF IPv6Peter Mikus18-22/+1326
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I6cb4ef356047bdc341a70a4e7b8d0b3f54eb7f9d
2019-09-05FIX: Use vhost macs when testpmd_mac forwarding used in VMJan Gelety2-2/+2
Change-Id: I291c3dc852e7c1bde6fb6102a9168cfec9992682 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-09-02Add: Testpmd VM testPeter Mikus28-36/+2033
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Id39667515e304f4d92f230f82d3f1dbc1e22ac32
2019-08-27- Removed 9000B test cases from ipsec test suites.Ludovit Mikula65-780/+0
- Added new traffic profiles. Signed-off-by: Ludovit Mikula <ludovit.mikula@pantheon.tech> Change-Id: Ie06b7a1cf4d9df403d098b7a495219fd95e5869e
2019-08-25FIX: Copy-paste typo in ↵Jan Gelety1-1/+1
2n-10ge2p1x710-eth-l2bd-6ch-48vh-24vm1t-vppip4-ndrpdr.robot Change-Id: Ibe231547e26d9b379554eec8ca6a9b4d7bc1224b Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-08-23Introduce reconfig suites, for dot1q+ip4+vxlanVratko Polak24-24/+1944
Ticket: CSIT-1551 + Config copied from other suites. + Layer keywords prepared for additional chains. + Renamed "create" argument to "vlan_per_chain". + TrafficGenerator result extraction improvements. + Also unified TG type and subtype checking. + Throughput estimated as PDR for zero PLR. + Actual Qemu start is skipped. + Parse xstats instead of stats in trex stop scipt. - Because stats are unconditionally cleared on each script connect. + Remember start values to get better ReceiveMeasurementResult at stop. + Expose loss count and time as appropriate test message. + Autogen support. + Multiple chain amounts and core densities. - Only 1 additional chain. + Proper TAGs. Documented. + Clarified min framesize tags are for TG-DUT. + Added a TODO concerning overhead values. + Added 118B tag. + Robot performance keywords improvements. + Added missing return value documentation. + Frame size as required test variable, not an explicit argument. + MLRsearch keyword to store result (not into test message). - TODO: De-duplicate: main keyword to measure, minor to set test message. - But implementation uses the fact default arguments are different. + Keywords for starting and stopping background traffic. - No checks whether start-stop pair is complete. + Multiple minor indentation and logging improvements. - Missing / TODOs: - Support also vlan_per_chain=True? - Support dot1q without vxlan? - Support also deletion? - How to package config steps so -reconf can be generated from -ndrpdr? Change-Id: If0270697f9ffa837c34ef802917220a53f156703 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-08-23Update CRC list commentsVratko Polak16-16/+16
+ Add testcases and tag expressions for perf occurrences. + Add TNL_1 tag for single tunnel IPsec suites. - Some CRCs were missing before, but form a pair with a listed CRC. - Probably the other half of the pair is not used. - Will be fixed in subsequent Change. Change-Id: I5754210755c13c41226b71045813714a65a1cdf3 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-08-21Add: 2-node ip4 oacl testsJan Gelety78-12/+372
Change-Id: I786fe9f75777e1cd0daee70bc350185dd7fd17e1 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-08-21Add: avf-dot1q testcasesPeter Mikus10-67/+469
+ Few fixes in doc + Align testcases Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I5575db8255a74fd46f37d5cc2637b9b4448bcd7f
2019-08-20Align NF_denisty testsPeter Mikus86-170/+508
+ Apply layer approach for interfaces Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I6c1898b9833f9f42b8912f1b521e30093360d7ea
2019-08-20Add: NF_density combinationsPeter Mikus12-6/+894
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I9ba859cf8ecb2d65c90ec7c6f75dc3e353e74de9
2019-08-19Autogen: Create NICs for TCP suites.Vratko Polak3-18/+18
+ Put Regenerator internal functions ourside class. + Duplicate into _default_ and _tcp_ functions. + Rework Testcase to be a single class with two factories. + Improve suite and testcase naming. Change-Id: I1f835b576c476c1a40c8c52469b1876695f9ae73 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-08-14FIX: Remove eth_avf and portX_vifY interfaces from topo in TC/TS tear downJan Gelety15-15/+15
Change-Id: I90901c4ab14dfa76dc3e1f786a4b986479e0ba47 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-08-14VAT-to-PAPI: Fix HTTP/TCP testsDave Wallace2-60/+116
Change-Id: Ie728bc73eb7429d246e82dd195fc63191c3ac51d Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2019-08-12Clean: Various librariesPeter Mikus43-83/+125
- Remove translate TC as they have no perf opposite - Remove unused keywords - Merge similar keywords / rename - Move keywords from perf library to categories Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I6433006136e2497ad8fb17193aa20c578d836999
2019-08-09Clean: L2 bridge domainPeter Mikus10-10/+20
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I3721900b91a9c8e9db8766bdcb9771b6bcf95bca
2019-08-08FIX: Various typos in nf_density + missing KWPeter Mikus4-5/+5
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I73878542672ab2dcfa1db046fbca4b7d931e744c
2019-08-07Add: AVF ip4scalePeter Mikus12-9/+916
+ Few corrections + Aligning 3n/2n Change-Id: If0918c0df38c5532899bec9ae7a7019117fff477 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-08-06Add: Dot1Q + L2BD + GBPPeter Mikus2-2/+142
Change-Id: I0050f715011ecfa92b3ee88b301809a56abb7946 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-07-26Move: 100tnl tests to 400tnlPeter Mikus4-56/+56
- Just bcause they wanted. Change-Id: I46a83924d61e9191bb02cf9dcfb761156686d545 Signed-off-by: Peter Mikus <pmikus@cisco.com>