aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2018-09-14Add dot1q-ip4base-[ndrpdr|mrr] perf tests for 2-node topologyJan Gelety17-142/+1443
Jira: CSIT-1278 Change-Id: I3cc929222d7fd58f38e33917e84675e9c0b55bbd Signed-off-by: Jan Gelety <jgelety@cisco.com>
2018-09-13Add coding guidelines documentVratko Polak1-0/+322
Started as a quite verbatim paste from: https://wiki.fd.io/view/CSIT/Documentation#CSIT_Test_Code_Guidelines then re-formatted, and new ideas added. Change-Id: I13fc3d267f07d88cf22402051480613e55123d44 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-09-13CSIT-1282 Migrate from Nexus.fd.io to packagecloud.ioPeter Mikus10-210/+185
Change-Id: Ib4e72dc8e3c0fedfcd702f4f97bc4f26cbc642e9 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-09-13CSIT-1280 Single memif container test failingPeter Mikus1-5/+35
- FIX Change-Id: I4b62d08b140dc73eb950d21bc85dc41ad638bbd2 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-09-12Fix: L2 scale tests are not baselineVratko Polak1-1/+1
This fixes select_tags function in common.sh to not include scale l2bd tests in default tag list. This is needed because "L2BDBASE" tag selects also scale tests (contrary to similar IP4BASE and IP6BASE tags). (Line length over 80 characters is left there as an indirect TODO to improve the tags.) Change-Id: Ic1d4a046f327fdd1b8b08eb20f1cdb4b12f337c9 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-09-12Fix: trap for unreserve before cleanupVratko Polak1-3/+3
Change-Id: I978a056ffdfd6daa00a0a1c587774b4a7204b434 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-09-12CSIT-1269: Add VTS tests to trendingTibor Frank5-1/+102
Change-Id: I9ddbad8e4c3f64553c72f77f93eb9d6c21602eca Signed-off-by: Tibor Frank <tifrank@cisco.com>
2018-09-10Create l2bdscale tests for 2-node topologyJan Gelety45-423/+2425
Jira: CSIT-1268 Change-Id: I13bf2e570a7d7231ba19c2c41db2b44c0b8d215a Signed-off-by: Jan Gelety <jgelety@cisco.com>
2018-09-10CSIT-1179: Switch MRR to multiple trialsVratko Polak2-4/+8
But keep the detection in PAL using only average, not size nor stdev. Change-Id: I07092bdeb892c871550265757fd2846a36fbb352 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-09-07CSIT-1270: Split feature tests in trendingTibor Frank14-70/+608
Change-Id: Iecc841737552fea07b7fcf297c478479f7872ae7 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2018-09-07CSIT-1271: Add VTS tests to MRR jobsTibor Frank1-0/+3
Change-Id: I5c0be60a1819c5c46225b4468ba9c88068231f08 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2018-09-07CSIT-1272: Fix VTS testsTibor Frank6-6/+6
Change-Id: Ibc4ae346032b2b711e4dcd1dbc9c0873e8464b19 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2018-09-07CSIT-1262: Add 2n/3n-skx to trendingTibor Frank125-3362/+9228
CSIT-1263: Edit Trending static content Change-Id: Ief60f3bea75fb9fd153f7abb0a5933f1a3e317aa Signed-off-by: Tibor Frank <tifrank@cisco.com>
2018-09-07Create ip6scale tests for 2-node topologyJan Gelety28-181/+2062
Jira: CSIT-1176 Change-Id: I78f4e9cb24a7de0bde24a78d1ca26607598a62b2 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2018-09-06Add INFO.yaml fileVanessa Rene Valderrama1-0/+86
Add INFO.yaml to list: - Project description - Properties - Issue Tracking - Contacts - PTL information - Meeting information - Committer information Change-Id: I02ef3fca1bef20ab5497b632c1995019bd7d1074 Signed-off-by: Vanessa Rene Valderrama <vvalderrama@linuxfoundation.org>
2018-09-06Fix autogen violationsVratko Polak4-76/+124
Four AVF suites had test cases in different order. Change-Id: I12fbe3508016a79a2153cc004c654a21ee4279bf Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-09-06CSIT-1264: Add oper branch wrapper entry scriptVratko Polak3-1/+133
Change-Id: I569b089b711f89fecca8dd31d6906ef311982877 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-09-06fix CSIT broken issue when numa_node value is -1 on single Socket platformYulong Pei1-2/+5
Single Socket platform e.g. Intel Atom cpu based SOC platform with Ubuntu 16.04.4(kernel 4.13.0-36-generic) or Centos 7.5 (kernel 3.10.0-862.el7), value of /sys/bus/pci/devices/<pci_device_id>/numa_node is -1, this will break CSIT performance test running, but for this kind of SOC platform, it can consider that is not NUMA based platform, numa_node=-1 is reasonable, so fix it at CSIT side, when numa_node=1 and the system's NUMA node count is 1, set numa_node=0. DPDK also did it as this way. Change-Id: I9ac23d3cece2f1489e38f05b50a462bb2ad9f661 Signed-off-by: Yulong Pei <yulong.pei@intel.com>
2018-09-05Create ip4scale tests for 2-node topologyJan Gelety28-151/+2035
Jira: CSIT-1174 Change-Id: I46944a3bc0f508d80c8c944dab0190925ebe21f5 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2018-09-05CSIT-1205 Create AVF driver testPeter Mikus13-132/+1028
- Add L1 KWs for SR-IOV handling (init Vfs, remove Vfs, ...) - Cleanup L1 KWs for SR-IOV bind/unbind/pci_get/... - Add L2 KWs for Test Setup/Teardown, L2patch, Create AVF interface - Add sample L2patch test fox x710, xxv710 Change-Id: If17077877455a14043617d8ea0d06cbe47b469e3 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-09-05Add Report historyTibor Frank2-1/+2
Change-Id: I5c7e594bcce266d765d2a517adcbec8e88da7c48 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2018-09-05Add known issues to report 18.07Tibor Frank4-31/+62
Change-Id: Ide101c80e2ef13021f25096da6afd9e4364ee263 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2018-09-05Fix various pylint violationsVratko Polak19-106/+111
+ SchedUtils.py: Do not use `len(SEQUENCE)` to determine if a sequence is empty + VatHistory.py: Do not use `len(SEQUENCE)` to determine if a sequence is empty + VppCounters.py: Do not use `len(SEQUENCE)` to determine if a sequence is empty + Memif.py: ++ Do not use `len(SEQUENCE)` to determine if a sequence is empty ++ Either all return statements in a function should return an expression, or none of them should. ++ Update :return: on possible None. + Classify.py: Unnecessary "else" after "return" + ContainerUtils.py: Useless super delegation in method '__init__' + CpuUtils.py: Do not use `len(SEQUENCE)` to determine if a sequence is empty + DropRateSearch.py: Either all return statements in a function should return an expression, or none of them should. + IPv4NodeAddress.py: Do not use `len(SEQUENCE)` to determine if a sequence is empty ++ Also improve docstrings. + IPv4Setup.py: Useless super delegation in method '__init__' + IPv6Setup.py: Do not use `len(SEQUENCE)` to determine if a sequence is empty ++ Also improve docstrings. + IPv6Setup.py: standard import "from ipaddress import IPv6Network" should be placed before "from robot.api import logger" + MacSwap.py: Trailing newlines + NATUtil.py: Do not use `len(SEQUENCE)` to determine if a sequence is empty + NodePath.py: Unnecessary "else" after "return" + Tap.py: Do not use `len(SEQUENCE)` to determine if a sequence is empty + topology.py: Either all return statements in a function should return an expression, or none of them should. + topology.py: Unnecessary "else" after "return" ++ Do not use `len(SEQUENCE)` to determine if a sequence is empty ++ Improve docstrings + DUTSetup.py: Do not use `len(SEQUENCE)` to determine if a sequence is empty ++ Also do not compare int(ret_code) just to access zero-ness. + ssh.py: Do not use `len(SEQUENCE)` to determine if a sequence is empty + InterfaceUtil.py: Unnecessary "else" after "return" Change-Id: Iba4244aa79661ee7df15fed5c7c6dbf04dfa88b2 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-09-05Add eth-ip4base-2memif[ndrpdr|mrr] perf tests for 2-node topologyJan Gelety12-0/+1611
- added also 3-node test cases for comparison Jira: CSIT-1261 Change-Id: I1bfcc184e925e969d4a610920c4e49af23047507 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2018-09-04Fix pylint part 1Peter Mikus9-19/+57
- Decrease pylint errors. Part: easy Change-Id: I452e5b5a11e9b78c03cd173a3848babe21b93c73 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-08-30CSIT-1135: Scripts for VPP per-patch testingVratko Polak10-396/+1413
+ Scripts do not rely on (other) bootstraps. + Perf verify bootstrap is also the new style sript now. + Scripts are divided to functions for better re-use. + Functions are sourced from small number of large "library" files. - Still using jumpavg from pipy. - Perpatch has specific simplified parsing (instead of the PAL one). - Bash style document is in a separate Change. Change-Id: If88fa528ce155ea86b614e3d77c0550b91bbdf11 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-08-28Create ip4-feature perf tests for 2-node topologyJan Gelety150-917/+4695
- ACL tests - NAT44 tests - iPoliceMarker tests Jira: CSIT-1175, CSIT-1154 Change-Id: Ife7e7df50f7c1c8033bacdb7512e78be6bcdf4eb Signed-off-by: Jan Gelety <jgelety@cisco.com>
2018-08-28Update of VPP_STABLE_VER and DPDK_STABLE_VER filesJan Gelety3-3/+3
- use new vpp ref build - ubuntu 16.04: 18.10-rc0~242-g0219b8d~b4906 - use new vpp ref build - centos7: 18.10-rc0~242_g0219b8d~b4824 - use new dpdk version: 18.08-vpp1 Change-Id: I1c8a12d3481108ceff8da1f4e31431fbdd9a7f1f Signed-off-by: Jan Gelety <jgelety@cisco.com>
2018-08-27Insert warning against editing in requirements.txtVratko Polak1-0/+34
Change-Id: Ifed10692e122c3a42d75282c92476e508cec597d Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-08-27FIX: Correction of ACL perf test suite documentationJan Gelety72-905/+840
Change-Id: I01cc87f5f5409346566ded52da9d384fdd3fa2c4 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2018-08-24add the expected prompt for ubuntu root userYulong Pei2-3/+3
if ssh use root account to login ubuntu os, its prompt is ":~# ", so add it to let csit test can run with root account. Change-Id: I21f5380245324c8e48099433e3d374188e9001d9 Signed-off-by: Yulong Pei <yulong.pei@intel.com>
2018-08-24Adjust TEST_TAGs for timed jobsPeter Mikus3-39/+3
+ remove obsolete bootstraps Change-Id: Iee1fe404106f2fa0c438ff30526e756f037ce676 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-08-24VPP link bonding perf tests with two links between DUTsJan Gelety16-35/+946
Jira: CSIT-1247 Change-Id: Ia6a3f7682eb9a35040682913c0e33a1d61d1e6fa Signed-off-by: Jan Gelety <jgelety@cisco.com>
2018-08-23report 18.07: added 2 known issues identified in CSIT-18.07 for vpp performance.Maciek Konstantynowicz1-0/+6
Change-Id: I220084bb5b5fd4909c8f1e2bef60e98594958e45 Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
2018-08-22CSIT-1222: Edit MLRsearch readme for 0.2.0 releaseVratko Polak1-2/+9
Change-Id: Ib2349f574720c61040f824f87bce01b5d68a2598 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-08-22CSIT-944: PAL Extensions: Mapping tableTibor Frank6-27/+459
Change-Id: I3bebf9ad849c056d2bbac476d22ef5e077af05b3 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2018-08-22CSIT-1164 Create VM vhost tests for 2-node topologyPeter Mikus37-1/+4947
Change-Id: I955d3170da94cf35a2fda03259bcdc020ac2b652 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-08-22CSIT-1222: Do two doublings in external MLRsearchVratko Polak8-23/+75
Make number of doublings configurable, keep Python default at 1, set Robot default to 2. Also make docstring types unique (pylint was complaining about classes and modules having the same name). Increase MLRsearch version to 0.2.0. Change-Id: Ib846032e79ff52994503c0cfef2f86655502c275 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-08-21HC Report: update list of known issuesMarek Gradzki1-11/+3
Current list was based on Jira, which turned out to be outdated. Change-Id: I216616cda766075b8b78f7c3f2f7a6775a622e23 Signed-off-by: Marek Gradzki <mgradzki@cisco.com>
2018-08-21HC Tests: reenable VxLaN GPE ipv6 testsMarek Gradzki1-4/+2
The issue is no longer present in VPP master branch (1810). Change-Id: I64f4fa8544ace8dc3686db9a1a3f3897ba333c61 Signed-off-by: Marek Gradzki <mgradzki@cisco.com>
2018-08-21FIX: Use correct interface type in x710_1lbvpplacp perf testsJan Gelety4-4/+4
Change-Id: I68db0b0441ffc7d9b5e04048688711915ab24726 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2018-08-21Add MRR tests for kubernetesVratko Polak41-61/+3454
Fix number of flows in Documentation. Change-Id: I0db62ab79c44db0ea55e000f8ef9cb8ee28a07fe Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-08-21FIX: Boostrap-perf array assignmentPeter Mikus1-3/+3
Change-Id: I4d7c27dd182ae8e7b68b5f6342c163ac3615cad2 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-08-20HC Tests: mark BGP v6 tests as criticalMarek Gradzki1-2/+0
TC13 of BGP tests was marked as EXPECTED_FAILING due to HONEYCOMB-403, but the error does not occur on 1807 nor master for some time. Change-Id: I3d8f4ee78fed699927fed05e8fa070cfa95b2166 Signed-off-by: Marek Gradzki <mgradzki@cisco.com>
2018-08-20Refactor VHOST codePeter Mikus108-2237/+1422
CSIT-1164 Create VM vhost tests for 2-node topology CSIT-1173 Implement VM vhost KWs for 2-node topology - Cleanup QemuUtils.py Library (PEP8, some TODOs, readability, multi queue, ring size, qemu version detection) - Cleanup VHOST KWs perf (Unify running QEMU on N-node topology) - Add ability to configure Queues, Hyperthreading, automatically compute number of CPU for VM/Testpmd and RXQ for Testpmd workload. - Partial cleanup of configuration KWs (l2xc, l2bd) for 2-node - Create IPv4 2-node KW Change-Id: I8aae1355bafa651de715cd438fe706c443ea1d88 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-08-20Update of VPP_STABLE_VER filesJan Gelety2-2/+2
- use new vpp ref build - ubuntu 16.04: 18.10-rc0~193-g612dd6a~b4857 - use new vpp ref build - centos7: 18.10-rc0~193_g612dd6a~b4774 Change-Id: I2bb408d0299d8e6bad0ad8e6b37eccf27d3f6de9 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2018-08-20PAL improvement: Log repr() of PresentationErrorVratko Polak2-3/+5
Change-Id: I491e3a77dda06e6a1caa34f70baaef9751674fbd Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-08-20FIX: PAL: Keep increasing index on errorVratko Polak1-0/+1
The previous fix has added KeyError reporting and recovery, but the reported index got stuck on value 15. Change-Id: I2cec0f678a79ba3e5ca425c92c605e00cf551a35 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-08-18report 18.07: fixed link mapping for generated report pdf file.Maciek Konstantynowicz1-1/+2
Change-Id: I2e23fe3b6f03055beab536e6534dd43d229b7024 Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
2018-08-18release 1807: fixed directory nesting for perf comparison files.Maciek Konstantynowicz4-38/+38
Change-Id: I14e1a53543ab3b03b3136920a67ab594a22167cf Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>