aboutsummaryrefslogtreecommitdiffstats
path: root/tests/vpp/perf/ip4
AgeCommit message (Collapse)AuthorFilesLines
2020-01-10Trending: new daily setJan Gelety1-0/+134
New daily sets are prepared based on information in https://gerrit.fd.io/r/c/csit/+/24073/1/docs/job_specs/perf_tests_job_specs.md and previous test set definitions in docs/job_specs/test_select_list_[2n|3n]_[clx|skx|hsw|tsh|dnv].md files. - mrr-daily-2n-clx: 510 TCs (incl. nfv_density), expected exec. time 8:50h - mrr-daily-2n-skx: 525 TCs (incl. nfv_density), expected exec. time 7:55h - mrr-daily-3n-skx: 393 TCs (incl. nfv_density), expected exec. time 11:00h - mrr-daily-3n-hsw: 177 TCs (incl. nfv_density), expected exec. time 7:10h - mrr-daily-3n-tsh: 204 TCs, expected exec. time 21:00h - mrr-daily-2n-dnv: 84 TCs, expected exec. time 2:25h - mrr-daily-3n-dnv: 144 TCs, expected exec. time 6:35h + add some missing test suites + add trex-sl-2n-ethip4udp-1000u15p.py T-Rex traffic profile + correction of TS and TC names and tags in directory tests/vpp/perf/nfv_density/chain_ipsec Change-Id: Icfc86e9af97ed8dd8ccd2a34355c99aad69a28c0 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-01-10Support suite tags in autogenVratko Polak69-0/+69
+ Include a script to add suite tags to many suites at once. + Add suite tags also to device tests (not covered by autogen). Change-Id: I514ee6178e22999b43460028fe2696738b012f04 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-10Autogen: Generate also NIC drivers.Vratko Polak12-1338/+2
+ Disallowed -avf- (or -rdma-) as "template" suites. + GBP suite switched to DPDK driver in repo. + Each NIC has its own list of supported drivers, in Constants. + Updated tag expressions for daily jobs: + Feature, ipsec, memif, scale, srv6, tunnels, vhost and vts are tested only with vfio-pci. + Other (base, dot1q, dot1ad) tested with all drivers. + Setup actions currently depend on driver, generated. - The performance_rdma action is trivial for now. - Several tests fail, to be fixed later, e.g. by performance_rdma. + Reconf tests are also supported. + Added DRV_VFIO_PCI tags missing, mainly in density tests. - Vhost suites (density, reconf) are failing, but suites look good. - TCP suites do not support NIC drivers yet. - DPDK obviously not supported. + Use Python 3 in regenerate scripts. + Fix typos binded => bound. + File open modes set either u"rt" or u"wt" everywhere. + Remove a trailing space in an environment variable name. Change-Id: I290470675dc5c9e88b2eaa5ab6285ecd9ed7827a Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-12-16FIX: NAT44 bugPeter Mikus1-1/+2
+ From trending ... Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I0ebb5429f0ad731d42aa43855cd99ae73b1461fc
2019-12-05Python3: IPv4 FIB scale tests with random flowsBalaji Venkatraman3-0/+400
Signed-off-by: Balaji Venkatraman <balajiv@cisco.com> Change-Id: Iadd9e8da655f3ccbd36167c1bbee37a774c3bc1f Signed-off-by: Balaji Venkatraman <balajiv@cisco.com>
2019-11-28Python3: resources and librariesJan Gelety77-712/+712
Change-Id: I1392c06b1d64f62b141d24c0d42a8e36913b15e2 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-11-11Add accidentally deleted NAT setup commandVratko Polak1-0/+1
One suite was missing "And Add NAT to all DUTs". Change-Id: I6e9c3de9f236a080cfe829db2d155a6f6ba3b3f4 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-10-25Introduce pre-initialize driver layerPeter Mikus76-437/+576
+ This will help unify driver base differences between TCs. + Decrease amount of logic needed for suite generator. Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I1e84ba361dc1e829f0612c58a61096e2633ce0c5
2019-10-19Tests: Introduce driver layerPeter Mikus76-76/+218
+ Small bugfixes in various suites Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I051af0a6da7340fe3f617400370c3846ca5241ab
2019-09-26CSIT-1471: Policer - VAT API to PAPIpremkumarv4-10/+20
Change-Id: I6c7eff8d9cbb196ccb684a45579ac59942351cfe Signed-off-by: premkumarv <preym17@gmail.com>
2019-08-21Add: 2-node ip4 oacl testsJan Gelety42-8/+336
Change-Id: I786fe9f75777e1cd0daee70bc350185dd7fd17e1 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-08-21Add: avf-dot1q testcasesPeter Mikus4-2/+274
+ Few fixes in doc + Align testcases Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I5575db8255a74fd46f37d5cc2637b9b4448bcd7f
2019-08-14FIX: Remove eth_avf and portX_vifY interfaces from topo in TC/TS tear downJan Gelety8-8/+8
Change-Id: I90901c4ab14dfa76dc3e1f786a4b986479e0ba47 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-08-12Clean: Various librariesPeter Mikus40-80/+120
- Remove translate TC as they have no perf opposite - Remove unused keywords - Merge similar keywords / rename - Move keywords from perf library to categories Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I6433006136e2497ad8fb17193aa20c578d836999
2019-08-07Add: AVF ip4scalePeter Mikus12-9/+916
+ Few corrections + Aligning 3n/2n Change-Id: If0918c0df38c5532899bec9ae7a7019117fff477 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-07-12VPPD: Cop, iACL, PolicerPeter Mikus6-9/+267
- Alignment with PERF Change-Id: Id7b2cb91c88e44cf9265c3376330f6aa579abf7a Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-06-28VAT-to-PAPI: ClassifyTibor Frank1-6/+4
Change-Id: Ic06a0a65429680e6ecdc3f5288d091c2c2630921 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-06-25Align suite/test teardown/setupPeter Mikus63-489/+388
+ Phase II - aligning FUNC/PERF setup. + 2n/3n compatibility of suite setup. + Same structure of testcase for DEV/PERF. + DEVICE tests are now running VPP startup config in same way as PERF. + DEVICE is using COREKEEPER! - DEVICE tests are running topology compute twice for backward compatibility. TODO to remove. - Still phase III needed to align TAGS, LIBRARIES and remove OLD setup phase. Change-Id: Ib6e1b0ebf4abebf3cbe73b4fda5cac953c2a7270 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-06-21Allow 9000b tests for AVFVratko Polak1-0/+12
See https://gerrit.fd.io/r/20148 Change-Id: I22856b14e3608348f2ccecb1d99d87209db2d776 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-06-21Align suite/test teardown/setupPeter Mikus63-126/+126
+ Phase I - aligning FUNC/PERF Change-Id: I46b1e43687ddb29f5ed3c6335fe1baf21ec02822 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-04-08Upgrade autogen to NICs and search typesVratko Polak184-18195/+1879
+ All perf suites updated, as autogen change is backward incompatible. + Only x710 -ndrpdr suites remain in git repository. + Removed retry feature of run_tests. + Renamed topology_type and traffic_type to traffic_profile. + Renamed framesize to frame_size; nic_model to nic_name. + Reordered Variables table to start with nic_name and overhead. + Fixed wrong overhead value in some cbc-sha1 suites. + Fixed some suite tags. ++ Introduced tags to distinguish suites, such as IPSECINT. + Bound crypto hardware to NIC name. + Implemented NIC limit hiding. + Also search and teardown related arguments are now hidden. + Main measurement keywords updated to new arg handling. + Max rate related keywords moved to performance_limits.robot + Teardowns unified. + Generated tests are archived. ++ Generated directory is .gitignore-d. + Regenerator raises an exception on seeing non-compatible suite. ++ Relatively helpful message should be seen in exception. + Suite and template Documentation has generated parts. +- With short NIC names only. + Autogen checker also upgraded. + Tag expressions replaced with long files to ensure analogous tests. Change-Id: I60e9a999187e7da1f60d0eb4fb02afa14682aa46 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-04-05Fixes for Taishan testbedjuraj.linkes22-3006/+0
* rename NICs in Taishan testbed to Intel-X520-DA2 * fix testbed NIC selection * remove unnecessary 82599es tests * blacklist unsupported tests on Taishan testbed * lowercase previously created bash local variables Change-Id: I3b7f59e2edc00929fab2f5a0fce483f236f83075 Signed-off-by: juraj.linkes <juraj.linkes@pantheon.tech>
2019-03-01Add ARM Taishan testbed33 perftestStanislav Chlebec22-0/+3006
Change-Id: Iaa0e7fbca7de26c7dce8b389cc3494f628a2ecd0 Signed-off-by: Stanislav Chlebec <stanislav.chlebec@pantheon.tech> Signed-off-by: juraj.linkes <juraj.linkes@pantheon.tech>
2019-01-29PLRsearch: Initial implementation and suitesVratko Polak1-0/+74
Missing bits: - Add up-to-date .rst document (in child Change). - Prepare for releasing to PyPI.org (in child Change): -- Either copy dependencies from MLRsearch, or list in requirements. -- Perhaps move common dependencies to separate package for both search to depend on. -- All the other metadata stuff. Jira: CSIT-1276 Change-Id: I277efdb63dbb90b30e11f4e30a82e2130ac8efc3 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-12-11Add missing = to all suite VariablesVratko Polak26-52/+52
Use the following if we ever decide '} =' is better than '}='. $ cd csit/tests $ find . -type f -print0 | xargs -0 sed -i -E 's/^\| \$\{(.*)\}= \|([^|])/\| \$\{\1\} = \|\2/g' FIXME: Add a corresponding check to https://gerrit.fd.io/r/13450 Change-Id: I73c516a0d7e475ff0986fd6515aaae10accd1a3a Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-10-10CSIT-1297: Modify the bootstrap.sh to run only selected mrr suitesTibor Frank8-8/+8
Change-Id: I2f5ac334e09bc35bfaefc112cf0bb7f57290a732 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2018-09-28CSIT-1321 Add IP4base tests using AVF driverPeter Mikus4-0/+484
Change-Id: I960c2b4170efa06597851853cfc15bda1c60fe94 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-09-20Create basic 2-node topology performance tests for Intel-X553 NICYulong Pei2-0/+254
Intel-X553 is Denverton (Intel Atom cpu based SOC) integrated NIC, add below basic 2-node topology performance test suites for it, the test suites were changed from the existing Intel-x710 test suites, - eth-l2patch-mrr - eth-l2patch-ndrpdr - eth-l2xcbase-mrr - eth-l2xcbase-ndrpdr - eth-l2bdbasemaclrn-mrr - eth-l2bdbasemaclrn-ndrpdr - ethip4-ip4base-mrr - ethip4-ip4base-ndrpdr - ethip6-ip6base-mrr - ethip6-ip6base-ndrpdr Change-Id: Ib6d3655a6e14ae275c197fd7b39d59e17a233326 Signed-off-by: Yulong Pei <yulong.pei@intel.com>
2018-09-18Add dot1q-ip6base-[ndrpdr|mrr] perf tests for 2-node topologyJan Gelety4-4/+4
Jira: CSIT-1283 Change-Id: I0f6841bafdb47bdbc3efe29801fe66a0d4c388ca Signed-off-by: Jan Gelety <jgelety@cisco.com>
2018-09-14Add dot1q-ip4base-[ndrpdr|mrr] perf tests for 2-node topologyJan Gelety8-0/+1106
Jira: CSIT-1278 Change-Id: I3cc929222d7fd58f38e33917e84675e9c0b55bbd Signed-off-by: Jan Gelety <jgelety@cisco.com>
2018-09-05Create ip4scale tests for 2-node topologyJan Gelety24-138/+1722
Jira: CSIT-1174 Change-Id: I46944a3bc0f508d80c8c944dab0190925ebe21f5 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2018-08-28Create ip4-feature perf tests for 2-node topologyJan Gelety144-846/+4337
- ACL tests - NAT44 tests - iPoliceMarker tests Jira: CSIT-1175, CSIT-1154 Change-Id: Ife7e7df50f7c1c8033bacdb7512e78be6bcdf4eb Signed-off-by: Jan Gelety <jgelety@cisco.com>
2018-08-27FIX: Correction of ACL perf test suite documentationJan Gelety72-905/+840
Change-Id: I01cc87f5f5409346566ded52da9d384fdd3fa2c4 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2018-08-03Fix ommissions from recent ChangesVratko Polak1-0/+1
+ Insert missing "Add worker threads"... keyword to udpsrcscale15 suite. + Add MRR for 40ge2p1xl710-ethip4ipsectptlispgpe-ip4base-cbc-sha1 suite. Change-Id: I467e5469a1421caa25373ab91010bd4d824fb32f Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-07-25Add MTU handling into perf testsPeter Mikus1-1/+1
- Add ability to configure MTU on interfaces. Put the MTU into configuration keywors for all perf testcases. Change-Id: I364b4bc26b26f2f66f350949c0aaa2a2aa675682 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-07-24CSIT-1097: Unify max NIC bandwidth in ip4Peter Mikus2-5/+5
Also, minor fix in ip6. Change-Id: I3a23e49d2162e34a1d2a8671ae9f7e74b832315c Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-07-24CSIT-1097: Migrate ip4 to NDRPDR and edit MRRVratko Polak152-6875/+12460
Change-Id: I9e3b2973c14f818247658a2ae34456c9fc2bae4e Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-07-16Add L2xc/L2bd/Ip4/Ip6 2-node tests for x710Peter Mikus4-0/+526
- CSIT-1161 Add 2-node L2XC tests - CSIT-1160 Add 2-node L2BD tests - CSIT-1157 Add 2-node IP4 tests - CSIT-1153 Add 2-node IP6 KW - CSIT-1158 Add 2-node IP6 tests Change-Id: I7c8fed7018bb355c0de73f6a8bc3a8ec80366972 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-07-13Refactor perf test casesPeter Mikus79-7237/+1515
- Refactor Performance TCs due to changes in Hyperthreading and RXQ allocations. - Simplify the structure of suite to avoid large refactors in future. Change-Id: Ifb1b9719391f26745f075a900bd9fb39b6cb809c Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-07-11CSIT-1178: Prepare for bursty MRRVratko Polak27-27/+27
+ All suites and tests still refer to MRR instead of BMRR (or FRMOBL). + API change: Traffic should pass with maximum rate first argument got a default value, thus it needs to be moved towards the end of argument list. + Consequently, the value for it has been deleted from suites. + Test message is now compatible only with BMRR parsing. - Reporting of transmit rate is left as TODO. + Global test __init__ with new way of giving defaults to global variables. + The new keyword is put into robot_enhancements.robot resource. + Number of trials and trial duration are now configurable. - Bootstrap scripts have no support for overriding the defaults yet. Change-Id: I4b108cc9c256730e62c03fbacba0a1c314f8ae88 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-07-09CSIT-1142 Change thread perf test TAGsPeter Mikus79-1609/+1609
Due to automatization of SMT detection this change is suppose to remove static thread/core tags in favor of dynamic one. Leaving the static tags for number of physical cores to be able to select TCs. Change-Id: I7f99f605821f363e45c333f46d1dea786693521b Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-07-09CSIT-1142 2-node topology - keywordsPeter Mikus79-138/+138
- Clean up of 2-node setup KWs Change-Id: I59fc901fb57544eceb6a041b56b515cda8babf85 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-06-13FIX: TAG processingPeter Mikus2-2/+2
- Improve TAG post processing Change-Id: I5a36f5874ea747cbcae2562fc58c9f5abe9bac02 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-05-04CSIT-992: Add two suites using optimized searchVratko Polak1-0/+142
+ Add 10ge2p1x520-ethip4-ip4base-ndrpdr.robot + Add 10ge2p1x520-eth-l2bdbasemaclrn-eth-2vhostvr1024-1vm-ndrpdr.robot + Make sure both IMIX and 9000B test cases are there. + Document NDRPDR test tag. + Execute some of NDRPDR tests daily. Change-Id: I66427d49cb6b8304fde4e8d91606de33b624186a Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-05-02Remove obsolete NDRPDR checksPeter Mikus9-1695/+0
Change-Id: If495493f4dce8d2eecfb8c06bcd587f315b70b1d Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-04-06CSIT-1020: MRR Day 2 - x710Jan Gelety14-1/+3048
- CSIT-1029: x710 MRR tests - IP4 - ip4 -10ge2p1x710-ethip4-ip4base-copwhtlistbase-mrr.robot -10ge2p1x710-ethip4-ip4base-iacldstbase-mrr.robot -10ge2p1x710-ethip4-ip4base-ipolicemarkbase-mrr.robot -10ge2p1x710-ethip4-ip4base-mrr.robot -10ge2p1x710-ethip4-ip4scale200k-mrr.robot -10ge2p1x710-ethip4-ip4scale20k-mrr.robot -10ge2p1x710-ethip4-ip4scale2m-mrr.robot -10ge2p1x710-ethip4udp-ip4base-iacl1sf-10kflows-mrr.robot -10ge2p1x710-ethip4udp-ip4base-iacl1sl-10kflows-mrr.robot -10ge2p1x710-ethip4udp-ip4base-iacl50sf-10kflows-mrr.robot -10ge2p1x710-ethip4udp-ip4base-iacl50sl-10kflows-mrr.robot -10ge2p1x710-ethip4udp-ip4base-nat44-mrr.robot - 10ge2p1x710-ethip4udp-ip4scale1000-udpsrcscale15-nat44-mrr.robot - ip4_tunnels -10ge2p1x710-ethip4lispip4-ip4base-mrr.robot -10ge2p1x710-ethip4vxlan-l2bdbasemaclrn-mrr.robot - 10ge2p1x710-ethip4vxlan-l2xcbase-mrr.robot Change-Id: I141bbd8039f4a5993c77758b1c410dcbd203ab97 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2018-03-21CSIT-998: MRR testsJan Gelety14-19/+3113
- CSIT-1001: MRR tests: IP4 - ip4 - 10ge2p1x520-ethip4-ip4base-copwhtlistbase-mrr.robot - 10ge2p1x520-ethip4-ip4base-iacldstbase-mrr.robot - 10ge2p1x520-ethip4-ip4base-ipolicemarkbase-mrr.robot - 10ge2p1x520-ethip4-ip4scale200k-mrr.robot - 10ge2p1x520-ethip4-ip4scale20k-mrr.robot - 10ge2p1x520-ethip4-ip4scale2m-mrr.robot - 10ge2p1x520-ethip4udp-ip4base-iacl1sf-10kflows-mrr.robot - 10ge2p1x520-ethip4udp-ip4base-iacl1sl-10kflows-mrr.robot - 10ge2p1x520-ethip4udp-ip4base-iacl50sf-10kflows-mrr.robot - 10ge2p1x520-ethip4udp-ip4base-iacl50sl-10kflows-mrr.robot - 10ge2p1x520-ethip4udp-ip4base-nat44-mrr.robot - 10ge2p1x520-ethip4udp-ip4scale1000-udpsrcscale15-nat44-mrr.robot - 40ge2p1xl710-ethip4-ip4base-mrr.robot - ip4_tunnels - 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-mrr.robot - 10ge2p1x520-ethip4vxlan-l2xcbase-mrr.robot - 10ge2p1x520-ethip4lispip4-ip4base-mrr.robot Change-Id: Ibfd03aea9b97d29a8fa05e2956f6df54603a2208 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2018-03-05Convert behavior-driven to data-driven IPv4Peter Mikus9-2071/+577
- Align test cases to data driven style for IPv4 Change-Id: Ie8fafaf7ba585e89feee417af96aa95d7ba4b4f9 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-02-23Raw results chk keywords and initial TCPeter Mikus1-0/+172
- Implementation of Display raw results - Implementation of Traffic should pass with maximum rate - Code optimizations - L2BD, L2XC, IP4, IP6 TC Change-Id: I0b50e995a853ed3c8f88323ccc650e47b8ed2496 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-02-09Disable all plugins except DPDK by defaultJan Gelety43-86/+172
- enable corresponding plugin for ACL, NAT, MEMIF tests Change-Id: Ie8bfdcbf1734283b6412ef3015215a00ce3ebdf6 Signed-off-by: Jan Gelety <jgelety@cisco.com>