aboutsummaryrefslogtreecommitdiffstats
path: root/tests
AgeCommit message (Collapse)AuthorFilesLines
2020-01-28Ipsec: Unify first line of Local Template docVratko Polak62-94/+90
Seeing differences when diff-ing between suites is distracting. + Bump copyright year, even for files with no change. Change-Id: Iaca79647821dd8233bdbe6b0ac8b14fdb04060a8 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-28perf: hoststack iperf3 test tuningDave Wallace1-18/+11
Change-Id: I53425f57fe9ecef9cff2c94642cc7cb24537a961 Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2020-01-28Fix two auth_alg valuesVratko Polak2-2/+2
Change-Id: I0e85fc958779df3d5dbacf1ad1e3898268a832ec Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-28Update overheads for IPsec CBC testsVratko Polak32-64/+64
Updated to the values as seen in packet trace. Even if VPP creates wrongly sized packets (compared to RFCs), the overhead should correspond to the actual packet size present, in order to correctly prevent linerate on DUT-DUT link. The new overhead values are 62 (256SHA) and 78 (512SHA). The GCM value is already correct, at 54 bytes, so density tests are ok. - The lispgpe test is not updated, as it currently fails. We will update overhead there in (or after) 24578. Change-Id: I5cc6920205f37ddc80e76804fabd90b67174addf Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-28FIX: dot1qip4vxlan testsPeter Mikus24-24/+24
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Ieb6e6010108987f99f55730149e3e4b7f1a7fc21
2020-01-18FIX: VTS testsJan Gelety3-6/+6
Change-Id: I594d248c58dcdcdeceea57af2dd25e2b2e08247f Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-01-23FIX: nfv_densityJan Gelety185-370/+370
- use correct osi_layer=L2 (so L2 spoofing check is switched off in case of avf driver) - add pci address information to eth interface in topology file - nfv_density chain_ipsec tests work only with DPDK in current implementation Change-Id: I233c6e5634a14581960c7459b87f11fcee8365bd Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-01-17perf: add TCP Iperf3+LDPRELOAD test suiteDave Wallace1-0/+71
Change-Id: Icff49fb31cce342a2a4ae799e844ec91f9e5e366 Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2020-01-15Remove everything related to TLDKVratko Polak18-409/+0
- Leftovers from kubernetes found, but not removed here. Change-Id: If8cb9269d0f3e69f642d7fe02c59122e17925a4d Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-10Trending: new daily setJan Gelety42-353/+777
New daily sets are prepared based on information in https://gerrit.fd.io/r/c/csit/+/24073/1/docs/job_specs/perf_tests_job_specs.md and previous test set definitions in docs/job_specs/test_select_list_[2n|3n]_[clx|skx|hsw|tsh|dnv].md files. - mrr-daily-2n-clx: 510 TCs (incl. nfv_density), expected exec. time 8:50h - mrr-daily-2n-skx: 525 TCs (incl. nfv_density), expected exec. time 7:55h - mrr-daily-3n-skx: 393 TCs (incl. nfv_density), expected exec. time 11:00h - mrr-daily-3n-hsw: 177 TCs (incl. nfv_density), expected exec. time 7:10h - mrr-daily-3n-tsh: 204 TCs, expected exec. time 21:00h - mrr-daily-2n-dnv: 84 TCs, expected exec. time 2:25h - mrr-daily-3n-dnv: 144 TCs, expected exec. time 6:35h + add some missing test suites + add trex-sl-2n-ethip4udp-1000u15p.py T-Rex traffic profile + correction of TS and TC names and tags in directory tests/vpp/perf/nfv_density/chain_ipsec Change-Id: Icfc86e9af97ed8dd8ccd2a34355c99aad69a28c0 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-01-10Support suite tags in autogenVratko Polak498-1/+499
+ Include a script to add suite tags to many suites at once. + Add suite tags also to device tests (not covered by autogen). Change-Id: I514ee6178e22999b43460028fe2696738b012f04 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-10Autogen: Generate also NIC drivers.Vratko Polak198-3908/+185
+ Disallowed -avf- (or -rdma-) as "template" suites. + GBP suite switched to DPDK driver in repo. + Each NIC has its own list of supported drivers, in Constants. + Updated tag expressions for daily jobs: + Feature, ipsec, memif, scale, srv6, tunnels, vhost and vts are tested only with vfio-pci. + Other (base, dot1q, dot1ad) tested with all drivers. + Setup actions currently depend on driver, generated. - The performance_rdma action is trivial for now. - Several tests fail, to be fixed later, e.g. by performance_rdma. + Reconf tests are also supported. + Added DRV_VFIO_PCI tags missing, mainly in density tests. - Vhost suites (density, reconf) are failing, but suites look good. - TCP suites do not support NIC drivers yet. - DPDK obviously not supported. + Use Python 3 in regenerate scripts. + Fix typos binded => bound. + File open modes set either u"rt" or u"wt" everywhere. + Remove a trailing space in an environment variable name. Change-Id: I290470675dc5c9e88b2eaa5ab6285ecd9ed7827a Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-15Hoststack perf infrastructure refactoringDave Wallace3-6/+10
- DUT only topology (hoststack test apps are co-located with vpp) - Make vpp app specific keywords generic where applicable - Add IP4 Prefix to topology file - Support running wrk in linux namespace - Refactor namespace cleanup - Remove redundant namespace creation code - Refactor test/keyword dirs: tcp -> hoststack - Add hoststack utility keywords - Refactor wrk suite setup/teardown - Update tests with recent perf infra changes Change-Id: Ia1cf07978d579393eef94923819a87c8c1f36f34 Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2020-01-09DMM: RemoveTibor Frank2-66/+0
Change-Id: Ibbfbed79e473c804390802ae1ecd737b50c06aa3 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-01-07Update of VPP_STABLE_VER filesJan Gelety1-1/+1
- use new vpp ref build - ubuntu 18.04: 20.01-rc0~983-g78565f38e - use new vpp ref build - centos7: 20.01-rc0~983_g78565f3~b8651 + remove EXPECTED_FAILING tag from tc01-64B-ethip4-l2patch-dev test Change-Id: Iab47a66003926024f87e028b1b1d9136b8fb4ec4 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-01-02Revert "L2Patch: Remove EXPECTED_FAILING"Peter Mikus1-1/+1
This reverts commit 09c5a6b8e1c6efed8826ef34aa64809226e80edb. Reason for revert: CSIT has not yet the latest VPP version Change-Id: Ibaa2c00c639bacef1561898daf9485c3a68efec4 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2020-01-02L2Patch: Remove EXPECTED_FAILINGPeter Mikus1-1/+1
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Ie950370c47403872597f8857edd651df2552ccb2
2019-12-20VTS: Unify the testsPeter Mikus3-58/+36
- Converting to 2n as they were always 2n (with l2xc on 3rd node) - Removing KW and converting to layered approach Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Ie349c50f72eb362815e7c5ede076d421ab386e76
2019-12-16FIX: NAT44 bugPeter Mikus1-1/+2
+ From trending ... Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I0ebb5429f0ad731d42aa43855cd99ae73b1461fc
2019-12-13Python3: refactor ':FOR' statementsDave Wallace2-2/+4
Signed-off-by: Dave Wallace <dwallacelf@gmail.com> Change-Id: I76835e3d3acf6955e328f30427f9dd0098947e41
2019-12-01CSIT-VPP-DEV: move tap tests back to critical poolJan Gelety3-3/+1
Change-Id: Ic99e828588d561a07c169502dff8ca19ac98400f Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-12-11Introduce VPP-IPsec container tests.Ludovit Mikula36-0/+5268
Change-Id: Ie64d662e81879bd52785e0188450d998bf056bda Signed-off-by: Ludovit Mikula <ludovit.mikula@pantheon.tech>
2019-12-10VPPD: Add L2patchPeter Mikus1-0/+75
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Iebdc03e56cf9ecf0322ff5c2c8698e9e3daf4407
2019-12-10CSIT-1597 API cleanup: classifyJan Gelety3-1/+3
- cover API changes in VPP: https://gerrit.fd.io/r/c/vpp/+/21551 - move vpp stable build to 20.01-rc0~821 - tap tests tagged 'EXPECTED_FAILING' until necessary vpp api change https://gerrit.fd.io/r/c/vpp/+/21706 and csit code is adapted Change-Id: If4b9b7b5bc207612d08aa8aa4469a5361e058fc3 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-12-05Python3: IPv4 FIB scale tests with random flowsBalaji Venkatraman3-0/+400
Signed-off-by: Balaji Venkatraman <balajiv@cisco.com> Change-Id: Iadd9e8da655f3ccbd36167c1bbee37a774c3bc1f Signed-off-by: Balaji Venkatraman <balajiv@cisco.com>
2019-12-03Remove: VIRLPeter Mikus2-107/+0
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I2b1672412e19f15cb46246fffa931351cd3d518b
2019-12-01Convert SRv6 from LXC to DockerPeter Mikus3-9/+9
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I7c948be40aaf2eb03434cec5548a89aaf0f1eca6
2019-11-28Python3: resources and librariesJan Gelety509-4512/+4524
Change-Id: I1392c06b1d64f62b141d24c0d42a8e36913b15e2 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-11-21NSH_SFC: RemoveTibor Frank10-651/+0
Change-Id: Ib7f9dff7bede4f8ec3148f234109132be920de82 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-11-19HONEYCOMB: RemovePeter Mikus32-6019/+0
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I60d80564050cb786c7b1e0a08f31f8292f82197e
2019-11-14FIX: AVF IP6basePeter Mikus1-1/+1
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Ibc3350acab54100f03f540397bfcbd6e7455ea2c
2019-11-13LIGATO: RemovePeter Mikus12-1760/+0
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I0c9b8b07ed3a113e24a38faaddd5ed06fae67768
2019-11-13Add vpp loadbalancer l3dsr/nat4 mode test suiteshaiyanx1.zhang3-1/+262
Change-Id: I03603108fe7128fb4bde4abddb6a0959dd5b5c73 Signed-off-by: haiyanx1.zhang <haiyanx1.zhang@intel.com>
2019-11-11Add accidentally deleted NAT setup commandVratko Polak1-0/+1
One suite was missing "And Add NAT to all DUTs". Change-Id: I6e9c3de9f236a080cfe829db2d155a6f6ba3b3f4 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-11-11FIX: do not use ping in vpp-device testsJan Gelety19-149/+131
Change-Id: Ic11f863e7ffad70d45e6e8dc39c70efc62cecc96 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-11-11FIX: 2lbvpplacpPeter Mikus4-4/+0
- These testcases are non standard so they need to be refactored to be able to use the interface layers Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I9400846ff04f1864babd6d7fa153f0c3708cb105
2019-10-30FIX: IPSecHWPeter Mikus8-16/+0
- IPsec proto is not for GCM Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: If6a9721574a72dacb4486c837100e4f2ed2bab19
2019-10-25Introduce pre-initialize driver layerPeter Mikus481-2468/+3223
+ This will help unify driver base differences between TCs. + Decrease amount of logic needed for suite generator. Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I1e84ba361dc1e829f0612c58a61096e2633ce0c5
2019-10-19Tests: Introduce driver layerPeter Mikus481-493/+1400
+ Small bugfixes in various suites Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I051af0a6da7340fe3f617400370c3846ca5241ab
2019-10-16VPPD: GBP test casePeter Mikus1-0/+88
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I020cdb6ced70a9c22773dfbe1662aecd2b744d2d
2019-10-09Add vpp loadbalancer maglev mode test suitehaiyanX1.zhang2-0/+145
Change-Id: I61555ba566efef0a2151db9a30bf7f5d9ccad1df Signed-off-by: haiyanx1.zhang <haiyanx1.zhang@intel.com>
2019-10-04FIX: IPv6 AVF dot1qPeter Mikus2-2/+4
Change-Id: Id199eb774850153fb225434d4a3d194706fdfdb3 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-09-26CSIT-1471: Policer - VAT API to PAPIpremkumarv10-27/+69
Change-Id: I6c7eff8d9cbb196ccb684a45579ac59942351cfe Signed-off-by: premkumarv <preym17@gmail.com>
2019-09-19LISP: 3-node (func) to 2-node (dev) converstionNidhy12-575/+417
Change-Id: I0fbe4d9ef1b30a1ac1382ea43b5b52aeb8a7d39f Signed-off-by: Nidhy <nidhyanandhan.a@hcl.com>
2019-09-17Update CRC list and support 21997/7Vratko Polak1-0/+6
- Attempt to repair IPsec LispGpe perf suite. + Collection name to reflect the current stable vpp. + Add messages found in CSIT L1 keywords. - Uncommented (instead of deleted) untestable messages. Reasons: - Honeycomb. - Messages used by unused keywords. + Listed reasons. Honeycomb not mentioned if both reasons apply. + Delete CRC items for commands not found in keywords anymore. + Add CRCs from .json.api files (as teardown is hard to execute). + Define and restore alphabetical order. + Add hints to find used API commands (not entirely reliable). + Move used commands to "cmd = " form so hints find them. + Argument to run_cli_command changed from "cmd" to "cli_cmd". + Except also struct.error where IOError is excepted. Change-Id: I61058dbe1e33296908aabd0c13433bb16cfa6adf Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-09-13Add: AVF IPv6Peter Mikus18-22/+1326
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I6cb4ef356047bdc341a70a4e7b8d0b3f54eb7f9d
2019-09-06Read environment variables in Constants.pyVratko Polak1-40/+0
Instead of using EnsureGlobalVariable, which is clunky to use from Python. As a consequence, all caps variables from Constants.py are used directly and tests/__init__.robot and robot_enhancements.robot are deleted. + Rename the CRC global kill switch based on ci-man review. Change-Id: I10723792475bc83352bf8c3b7f7946ecf885a194 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-09-05Allow CRC checking to be controlled via env varVratko Polak1-0/+5
The plan is to override the default in some (but not all) jobs. The csit-vpp jobs shall do the checking, but vpp-csit jobs shall not (except api-crc job of course) in order to confirm benign changes. Change-Id: If59d17991de3fb0e847113a87e0533a8ee62af7f Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-09-05FIX: Use vhost macs when testpmd_mac forwarding used in VMJan Gelety2-2/+2
Change-Id: I291c3dc852e7c1bde6fb6102a9168cfec9992682 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-09-02Add: Testpmd VM testPeter Mikus28-36/+2033
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Id39667515e304f4d92f230f82d3f1dbc1e22ac32