aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2020-01-10Trending: new daily setJan Gelety57-619/+3935
New daily sets are prepared based on information in https://gerrit.fd.io/r/c/csit/+/24073/1/docs/job_specs/perf_tests_job_specs.md and previous test set definitions in docs/job_specs/test_select_list_[2n|3n]_[clx|skx|hsw|tsh|dnv].md files. - mrr-daily-2n-clx: 510 TCs (incl. nfv_density), expected exec. time 8:50h - mrr-daily-2n-skx: 525 TCs (incl. nfv_density), expected exec. time 7:55h - mrr-daily-3n-skx: 393 TCs (incl. nfv_density), expected exec. time 11:00h - mrr-daily-3n-hsw: 177 TCs (incl. nfv_density), expected exec. time 7:10h - mrr-daily-3n-tsh: 204 TCs, expected exec. time 21:00h - mrr-daily-2n-dnv: 84 TCs, expected exec. time 2:25h - mrr-daily-3n-dnv: 144 TCs, expected exec. time 6:35h + add some missing test suites + add trex-sl-2n-ethip4udp-1000u15p.py T-Rex traffic profile + correction of TS and TC names and tags in directory tests/vpp/perf/nfv_density/chain_ipsec Change-Id: Icfc86e9af97ed8dd8ccd2a34355c99aad69a28c0 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-01-10Support suite tags in autogenVratko Polak501-23/+674
+ Include a script to add suite tags to many suites at once. + Add suite tags also to device tests (not covered by autogen). Change-Id: I514ee6178e22999b43460028fe2696738b012f04 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-10Autogen: Generate also NIC drivers.Vratko Polak223-4089/+465
+ Disallowed -avf- (or -rdma-) as "template" suites. + GBP suite switched to DPDK driver in repo. + Each NIC has its own list of supported drivers, in Constants. + Updated tag expressions for daily jobs: + Feature, ipsec, memif, scale, srv6, tunnels, vhost and vts are tested only with vfio-pci. + Other (base, dot1q, dot1ad) tested with all drivers. + Setup actions currently depend on driver, generated. - The performance_rdma action is trivial for now. - Several tests fail, to be fixed later, e.g. by performance_rdma. + Reconf tests are also supported. + Added DRV_VFIO_PCI tags missing, mainly in density tests. - Vhost suites (density, reconf) are failing, but suites look good. - TCP suites do not support NIC drivers yet. - DPDK obviously not supported. + Use Python 3 in regenerate scripts. + Fix typos binded => bound. + File open modes set either u"rt" or u"wt" everywhere. + Remove a trailing space in an environment variable name. Change-Id: I290470675dc5c9e88b2eaa5ab6285ecd9ed7827a Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-15Report: Current vs. Previous Release - NFV TestsTibor Frank2-0/+202
Change-Id: I00cbde38b8f900efe3e3123a942de2288381ea3b Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-01-15Hoststack perf infrastructure refactoringDave Wallace19-80/+1067
- DUT only topology (hoststack test apps are co-located with vpp) - Make vpp app specific keywords generic where applicable - Add IP4 Prefix to topology file - Support running wrk in linux namespace - Refactor namespace cleanup - Remove redundant namespace creation code - Refactor test/keyword dirs: tcp -> hoststack - Add hoststack utility keywords - Refactor wrk suite setup/teardown - Update tests with recent perf infra changes Change-Id: Ia1cf07978d579393eef94923819a87c8c1f36f34 Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2020-01-14FIX: VXLAN scalePeter Mikus1-3/+5
+ Another stupid arp -> neighbor rename Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Ia873d6c2a485659033f71cdf546782db74787d13
2020-01-14FIX: NF_density testsPeter Mikus1-2/+2
+ ARP -> Neighbor Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I74044bd7db6cb4b1bd3648a424e9c8f1473be8d1
2020-01-14Update comments in crc file.Vratko Polak1-5/+4
+ L2patch is already covered. + RDMA is not device testable yet. Change-Id: I1dcccfcb340a8fe61c20d3516340e26db293cfaa Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-14Report: Add VNF Service Chains TunnelsTibor Frank6-333/+989
Change-Id: Iafdcb543f6adc583567a05aa5a882dce17b58f03 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-01-13FIX: NF_density testsPeter Mikus5-9/+9
+ Because arp->neigbor was so important to do. Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I552e175b7555ebf5053d7994c0c9173c0c96fc58
2020-01-13Report: Move VTS tests from 3n to 2nTibor Frank18-808/+203
Change-Id: Ie2faa6cb611d372c4a20584f7b40f3281660f2b1 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-01-13CSIT-1597 API cleanup: rdmaJan Gelety4-247/+21
- cover API changes in VPP: https://gerrit.fd.io/r/c/vpp/+/24225 - update vpp stable to version 20.01-rc0~1011 Change-Id: Ibbfdbe867cf89715619efc5c5507859983025ece Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-01-10Report: Add comparison tablesTibor Frank6-212/+1035
CSIT-1654 Change-Id: I280195b295d4fc717cb355e78c5f1731bb1eabe5 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-01-10Report: Add 2n-clxTibor Frank113-76/+7624
Change-Id: I34dc11c28d21f0b9fed87d76942fe8b8cdd86c88 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-01-09FIX: Container testPeter Mikus2-9/+15
+ We should check if container is UP + We should adjust privileges. Change-Id: I3f604b7b9740943b4fd3b0574c18d2729e88b816 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2020-01-09Report: Configure Report 2001Tibor Frank12-93/+25
Change-Id: I5ef3c37cf329b76a1a7f94d60c3563872eef3c5d Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-01-09DMM: RemoveTibor Frank27-1302/+7
Change-Id: Ibbfbed79e473c804390802ae1ecd737b50c06aa3 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-01-08API: Add collection for 24225/3Jan Gelety1-8/+8
Change-Id: I5c4593d05dbcb4d918611dc7520432a446dbf664 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-01-08PAL: Set y-axis for HDRHistogramTibor Frank1-3/+4
Change-Id: Ibdee625d6d9215b18771a6348735dc6a07e9ae58 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-01-08Track real(er) duration of trafficVratko Polak1-0/+7
+ Print duration. - Missing: Parse duration. - We do not have good formula to compensate, yet. Change-Id: I1c76ac35ce531b1994480d6aa0638f16992231f5 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-07T-Rex: Upgrade to 2.73 with DDP profile fixPeter Mikus6-12/+7
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Icaa99e0a71a19ba23697c21a4b5aa046813a718a
2020-01-07Vhost: Add GSO optionPeter Mikus5-14/+39
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Iba9af71c293645b480203af72fcf940cbe9ccb3a
2020-01-07Fix Trex start configurationVratko Polak2-12/+21
Since https://gerrit.fd.io/r/c/csit/+/24056 csit-vpp perf tests are failing. Turns out the reason is too large limit_memory value, but other improvements applied during investigation are included as well. + Cat trex.log if trex fails to start. + Improve trex startup command formatting (remove double spaces). + Reorder imports. Change-Id: I2c1106ea6f88a1a275682e73eba212d08c7947c8 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-07Remove leftovers related to XenialVratko Polak2-2/+1
+ File VPP_STABLE_VER_UBUNTU. + Subtree docs/report/vpp_functional_tests/. Change-Id: Ifd9dd922152199de9f1352da709b1533d6c15aba Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-07Remove VIRL Func tests from ReportTibor Frank15-2768/+0
Change-Id: Ic6edf1f6f6c12d8e76da76617dbf1c4e8e1e2952 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-01-07Update of VPP_STABLE_VER filesJan Gelety3-3/+3
- use new vpp ref build - ubuntu 18.04: 20.01-rc0~983-g78565f38e - use new vpp ref build - centos7: 20.01-rc0~983_g78565f3~b8651 + remove EXPECTED_FAILING tag from tc01-64B-ethip4-l2patch-dev test Change-Id: Iab47a66003926024f87e028b1b1d9136b8fb4ec4 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-01-07Trending: Move VTS tests from 3n to 2nTibor Frank5-443/+80
Change-Id: I420cef699d42973602f1290ccd0cc5e378d42492 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-01-03Add new NIC limitsPeter Mikus1-0/+2
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I2b1a0d1455ae74a2ad4fdb553286aef5942f2a99
2020-01-02Revert "L2Patch: Remove EXPECTED_FAILING"Peter Mikus1-1/+1
This reverts commit 09c5a6b8e1c6efed8826ef34aa64809226e80edb. Reason for revert: CSIT has not yet the latest VPP version Change-Id: Ibaa2c00c639bacef1561898daf9485c3a68efec4 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2020-01-02L2Patch: Remove EXPECTED_FAILINGPeter Mikus1-1/+1
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Ie950370c47403872597f8857edd651df2552ccb2
2020-01-02TREX: Ability to configure from outside TrafficGenerator.pyPeter Mikus2-11/+19
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Ia9832e5e65dbcf2d4f99c52fd0570d456262ab9a
2019-12-20VTS: Unify the testsPeter Mikus4-130/+36
- Converting to 2n as they were always 2n (with l2xc on 3rd node) - Removing KW and converting to layered approach Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Ie349c50f72eb362815e7c5ede076d421ab386e76
2019-12-20Add VPP NSIM Plugin KeywordsDave Wallace2-0/+144
Change-Id: I1e080a50a68ded8eebc5e6093c55d06281726684 Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2019-12-18python3: fix RF 'only continuation marker' warningDave Wallace1-5/+5
Change-Id: I250ffd196045bc623d5d38f30f16a08bcaea137c Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2019-12-18Trending: Fix input data processingTibor Frank1-1/+1
Change-Id: I79cecd07c933cc46beb09de1ee2b4a0db5dd8a79 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-12-17Telemetry: Add more operational dataPeter Mikus6-138/+84
+ Add both NDR and PDR telemetry capture + Speedup sockets + Adjust privileges Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Ia6fd5d405e6fb410651d8b705c921653753aea10
2019-12-13FIX: eth interfaces need numa_node information in topology fileJan Gelety4-28/+44
Change-Id: I9a3c6947da8ab5033e29b39833c97bcdafdc3ca9 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-12-17CTPA: Improve handling pps vs MppsVratko Polak1-22/+25
+ Jumpavg assumes precision is around 1, so gives wrong classifications when fed Mpps numbers, and the classification has to be done on pps values. + Hover should display Mpps values properly, e.g. as floats. + Decided to show 3 digits after decimal point. + Better distinguishing between keys and indices. Change-Id: I99bdad5b401839889463c7cf6f8b0930ec65b457 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-12-16python3: fix wrk traffic profile YAML loadDave Wallace1-2/+2
Signed-off-by: Dave Wallace <dwallacelf@gmail.com> Change-Id: I3b7493221f3a0378b7f6cb0b0a21e8f18a0cc937
2019-12-16Vagrant: Introduce CentOS8Peter Mikus6-103/+139
+ Full ansible compatibility - Docker currently not working in Centos8 as it was not yet released Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I837091621db6bbebc9fee729496384cabe2f5357
2019-12-16Perpatch compare: pylint fixes.Vratko Polak1-95/+92
+ Do not execute on import. + Replace hack() with sorted(). Change-Id: Ic439a799ab29d42ae21faf70b2aea6dce6e89510 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-12-16Do not execute crc check on importVratko Polak1-39/+57
Other guilty scripts already have open changes that fix them. Change-Id: Ife75e999d6d3f3f506b0f5dcdb679ce1f1e6bdbe Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-12-16FIX: NAT44 bugPeter Mikus1-1/+2
+ From trending ... Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I0ebb5429f0ad731d42aa43855cd99ae73b1461fc
2019-12-16FIX PIP requirement packagesPeter Mikus5-23/+6
+ CSIT will only use packages installed from master PIP requirement.txt file + All CSIT subsequent code must use single function and PIP list for virtualenv - Report/PAL/doc/trending will follow in next patch Change-Id: Ia9150fb05a144182a5a163178d553370ede55cdd Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-12-16Remove supervisord dependency from containersPeter Mikus2-128/+35
+ Startup config stream-line Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Ia12e48202a5fc88461d3d24ac9b9599e7175b1d8
2019-12-13Reduce spam from device bootstrapVratko Polak1-0/+6
It uses "declare -f" to paste the complete common.sh file. Twice. The output from ssh remote side still remains visible, but avoiding the spam should ease scrolling to Robot console output. Change-Id: I746484ada7650a6dfa6794e76550757af5413a82 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-12-12API: Add collection for 22970/5Jan Gelety1-0/+237
Change-Id: If865526d833a267eba85e6ae4107e4d0da0dfd0b Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-12-13Python3: refactor ':FOR' statementsDave Wallace3-3/+6
Signed-off-by: Dave Wallace <dwallacelf@gmail.com> Change-Id: I76835e3d3acf6955e328f30427f9dd0098947e41
2019-12-12CSIT-1597, CSIT-1647 API cleanup: gbpJan Gelety4-246/+9
- cover API changes in VPP: https://gerrit.fd.io/r/c/vpp/+/23835 - update vpp stable to version 20.01-rc0~843 Change-Id: Idb1299cbbba578c09542c23858fb1fc55d090f26 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-12-12API: Add collection for 23835/1Jan Gelety3-14/+14
- update vpp stable to version 20.01-rc0~841 Change-Id: I05fc3878a606852c8348b61b40d8000411709d33 Signed-off-by: Jan Gelety <jgelety@cisco.com>