aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2018-03-30Change the default plugin behavior in perf testsPeter Mikus6-61/+21
Currently we are selectively disabling plugins by creating list of disabled plugins for performance tests. Thanks to the new API in VPP we can revert the logic to selectivly enable plugins. This is more stable for future onboarding of new plugins into VPP. Change-Id: Ibdd56d6097230d095a304bbb6984b565d9428f51 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-03-29Add IPSEC MRR tests to VPP verify jobsPeter Mikus1-0/+4
- Add IPSEC MRR tests to VPP verify jobs on IPSEC trigger Change-Id: I39f79fab266e86fbc8d15859b1aaa26e15f26cbd Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-03-29Add IMIX for Vhost and Memif into MRR trendingPeter Mikus1-0/+6
- Add IMIX for Vhost and Memif into MRR trending Change-Id: Ifab55b14a97f742e801ddc6f15388f3a134db799 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-03-28PAL Trending: Data in txt table in MppsTibor Frank2-3/+3
Change-Id: I3b27f95c899e2a904fe806de24f7a0d5440cb4ff Signed-off-by: Tibor Frank <tifrank@cisco.com>
2018-03-28PAL Trending: Data in txt table in MppsTibor Frank1-2/+10
Change-Id: I5427e14479b4403319b817e75d83b95b64bcddb6 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2018-03-28PAL Trending: Static contentTibor Frank13-551/+110
Change-Id: I4c46a0b06c5d726f7bfb5705d153f101eb46796f Signed-off-by: Tibor Frank <tifrank@cisco.com>
2018-03-28Perf bootstrap cleanupPeter Mikus3-135/+101
- Add archiving of all artifacts to logs.fd.io - Remove obsolete test tags - Sync all bootstrap and prepare for future merge-into-single bootstrap. Change-Id: I023df6964bb049df3ccffdffa31a93b667e1587a Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-03-27Trending: Set the start build number to 15.Tibor Frank1-2/+2
Change-Id: If1603fb8d39c339caf9c49015a63e2910f98e292 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2018-03-27PAL trending: Trending lineTibor Frank1-3/+4
Change-Id: I46309f53365b9335fc93bcecdf293bca07a0e51d Signed-off-by: Tibor Frank <tifrank@cisco.com>
2018-03-26FIX: Catch exeption in start-testcasePeter Mikus1-3/+3
Catch all exceptions by default if removing scratch directory is not successful. This prevents further confusion about NFS permissions as this is not the root cause of script failure. Change-Id: I0f775bb38e98cf69b4ce277b6e0dda4dad0a8732 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-03-26Report: Fix packet_throughput_graphs/ip4Tibor Frank3-11/+34
Change-Id: I3943d02dcca95ed31baaa104648589c42ef479a7 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2018-03-26Report: Detect outliers for the comparative tablesTibor Frank3-45/+46
Change-Id: Ia1be590a720df71ff51e8a7ca910ed27a4642209 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2018-03-26Report: Detect outliers for the comparative tablesTibor Frank2-45/+41
Change-Id: I2bbb2c4e18f456da9b8d62b6654d7e23fe36ee58 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2018-03-26Report: Detect outliers for the comparative tablesTibor Frank1-13/+9
Change-Id: I87f7f239560544261e21d36d020c56dda0e601fd Signed-off-by: Tibor Frank <tifrank@cisco.com>
2018-03-26Add IPSec testsTibor Frank4-84/+105
Change-Id: Ie1ddc6189a68a299207fe9158d1be8409ec2bc6a Signed-off-by: Tibor Frank <tifrank@cisco.com>
2018-03-26Optimize Qemu installation to speed up vhost testsPeter Mikus52-521/+203
Currently Qemu is being installed if there is a change of qsz parameter between tests/suites. Qemu is installed always into the same directory. This patch changes the default behavior to install qemu over and install pathced version to separate directory. It also disables force install. Change-Id: I0d7493a02b026a6ae4a5ea8bacf54656de9db567 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-03-26CSIT-919: Jenkins jobTibor Frank3-14/+10
Change-Id: I8ed4a8e135ebbebc7799a633872a50293eb410af Signed-off-by: Tibor Frank <tifrank@cisco.com>
2018-03-26Add lastCompletedBuild option to the specTibor Frank3-47/+86
Change-Id: I4432e626a7bc81be2cede56da6a3660450f09c01 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2018-03-26CSIT-919: Jenkons jobTibor Frank1-5/+3
- return correct return value Change-Id: I761593b90336ef3926e157c068982713ea3fe71f Signed-off-by: Tibor Frank <tifrank@cisco.com>
2018-03-23CSIT-1004: Generate csv and txt tables with trending and statistical dataTibor Frank6-8/+56
Change-Id: I50ba76c06621a9a293a5327b5b7709e4df6a4756 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2018-03-23CSIT-949: Data modelsTibor Frank7-16/+269
Change-Id: I07d0b122632c15fa07a5871dc268d654f410ef4f Signed-off-by: Tibor Frank <tifrank@cisco.com>
2018-03-22CSIT-1005: Add ipsec MRR testsVratko Polak20-180/+1569
+ 40ge2p1xl710-ethip4ipsecbasetnl-ip4base-int-aes-gcm-mrr.robot + 40ge2p1xl710-ethip4ipsecbasetnl-ip4base-int-cbc-sha1-mrr.robot + 40ge2p1xl710-ethip4ipsecbasetnlsw-ip4base-int-aes-gcm-mrr.robot + 40ge2p1xl710-ethip4ipsecbasetnlsw-ip4base-int-cbc-sha1-mrr.robot + 40ge2p1xl710-ethip4ipsecscale1000tnl-ip4base-int-aes-gcm-mrr.robot + 40ge2p1xl710-ethip4ipsecscale1000tnl-ip4base-int-cbc-sha1-mrr.robot Do not add test cases for 9000B framesize yet. Improve other suites to keep the style consistent. Change-Id: I03a58e8d6745083e267b9d53bb3b85efcb2ea827 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-03-22CSIT-913: Continuous Trending, Analysis and Change DetectionTibor Frank9-20/+802
- CSIT-949: Add new plots - CSIT 951: Add new plots Change-Id: I09440df5209e3207339a8a9509c084da16f06356 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2018-03-22Update sr_steer_add_del.vat templateJan Gelety1-1/+1
Change-Id: I7b7fd3a8815cf33fa50ca5f3f7a9128edef8b827 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2018-03-22FIX: check OS on DUT instead on hostJan Gelety1-1/+3
CSIT-1010 Change-Id: I21303c22a50c0fe7914f769aca61ef350287ccb5 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2018-03-22Add 78B tests to MRR trending dailyTibor Frank1-0/+3
Change-Id: I73b9c505dcd28361978a075bc81db3112e308b9c Signed-off-by: Tibor Frank <tifrank@cisco.com>
2018-03-21HC Tests: adapt sub-interface tests to recent api changes (HC2VPP-289)Marek Gradzki4-9/+38
Follows: https://gerrit.fd.io/r/#/c/11228/ Change-Id: If1a281fc04c3b8015519c97788e1dae213e74b0e Signed-off-by: Marek Gradzki <mgradzki@cisco.com>
2018-03-21CSIT-913: Continuous Trending, Analysis and Change DetectionTibor Frank22-154/+1836
- CSIT-915: LLD - CSIT-917: Functions to evaluate the results according to the PASS / FAIL criteria - CSIT-918: Sphinx configuration - CSIT-948: Statistical functions - CSIT-949: Data models for trending plots - CSIT-950: Code trending plots - CSIT-951: Static content - CSIT-984: PAL Specification file - CSIT-996: Download data from nexus Change-Id: Icb9305945bb0f142135bb177cb8781ba0096280e Signed-off-by: Tibor Frank <tifrank@cisco.com>
2018-03-21CSIT-998: MRR testsJan Gelety18-21/+3811
- CSIT-1001: MRR tests: IP4 - ip4 - 10ge2p1x520-ethip4-ip4base-copwhtlistbase-mrr.robot - 10ge2p1x520-ethip4-ip4base-iacldstbase-mrr.robot - 10ge2p1x520-ethip4-ip4base-ipolicemarkbase-mrr.robot - 10ge2p1x520-ethip4-ip4scale200k-mrr.robot - 10ge2p1x520-ethip4-ip4scale20k-mrr.robot - 10ge2p1x520-ethip4-ip4scale2m-mrr.robot - 10ge2p1x520-ethip4udp-ip4base-iacl1sf-10kflows-mrr.robot - 10ge2p1x520-ethip4udp-ip4base-iacl1sl-10kflows-mrr.robot - 10ge2p1x520-ethip4udp-ip4base-iacl50sf-10kflows-mrr.robot - 10ge2p1x520-ethip4udp-ip4base-iacl50sl-10kflows-mrr.robot - 10ge2p1x520-ethip4udp-ip4base-nat44-mrr.robot - 10ge2p1x520-ethip4udp-ip4scale1000-udpsrcscale15-nat44-mrr.robot - 40ge2p1xl710-ethip4-ip4base-mrr.robot - ip4_tunnels - 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-mrr.robot - 10ge2p1x520-ethip4vxlan-l2xcbase-mrr.robot - 10ge2p1x520-ethip4lispip4-ip4base-mrr.robot Change-Id: Ibfd03aea9b97d29a8fa05e2956f6df54603a2208 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2018-03-21Fix bug in dpdk-testpmd-mac KWTibor Frank1-6/+0
Change-Id: I6711e186062a72700552418eeee11b031e8c8ce4 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2018-03-21Fix for pypcap import problem.Thomas F Herbert1-1/+1
JIRA: CSIT-991 This fix forces the downloads of the complete package for all the pip python packages to be installed on Centos regardless of the Python metadata on the host. Change-Id: I9f5b9faf75306c2b0ae34ca640bc5f36811f900c Signed-off-by: Thomas F Herbert <therbert@redhat.com>
2018-03-21MRR Daily: Run only 64B testsTibor Frank1-1/+3
Change-Id: I61bb0c785fcc1407b80ce8019df69e64a3e4eb9e Signed-off-by: Tibor Frank <tifrank@cisco.com>
2018-03-20Add max-pkt-len parameter to dpdk testpmd startTibor Frank18-70/+172
Change-Id: I114c187f007c10223502669c2ba85c63d09cd84a Signed-off-by: Tibor Frank <tifrank@cisco.com>
2018-03-20HC Tests: move honeycomb tests out of vpp directoryMarek Gradzki37-7/+15
https://gerrit.fd.io/r/#/c/9257/ moved VPP instalation to vpp/func/__init__.robot, which is run before Honeycomb suite. Instalation process starts with removing all vpp packages, which fails because of honeycomb dependency installed by bootstrap script. This patch fixes HC func jobs by moving them to separate dir. The honeycomb/func/__init__.robot was updated to include previous content of vpp/func/__init__.robot. HC perf jobs were also moved, but they may require additional care (CSIT-1006). Change-Id: I99d94272c80a4c57c85ec5cf99cddfbeab7de663 Signed-off-by: Marek Gradzki <mgradzki@cisco.com>
2018-03-20Update the list of disabled plugginsTibor Frank2-8/+46
Change-Id: I089416953f5c3292358e443c806987dddce1c0d3 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2018-03-19No overlapping subnets between DUT/TG and DUT/VMNeale Ranns5-6/+6
Change-Id: Ib6a2921070f3524b001ecea2467b28264288fee3 Signed-off-by: Neale Ranns <nranns@cisco.com>
2018-03-19CSIT-935: Add mrr 9000B and 4t4c testsTibor Frank20-330/+1250
CSIT-1002: container_memif - 10ge2p1x520-eth-l2bdbasemaclrn-eth-2memif-1lxc-mrr.robot - 10ge2p1x520-eth-l2xcbase-eth-2memif-1dcr-mrr.robot - 10ge2p1x520-eth-l2xcbase-eth-2memif-1lxc-mrr.robot - 40ge2p1xl710-eth-l2bdbasemaclrn-eth-2memif-1lxc-mrr.robot - 40ge2p1xl710-eth-l2xcbase-eth-2memif-1lxc-mrr.robot CSIT-1003: vm vhost - 10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhostvr1024-1vm-mrr.robot - 10ge2p1x520-dot1q-l2xcbase-eth-2vhostvr1024-1vm-mrr.robot - 10ge2p1x520-eth-l2bdbasemaclrn-eth-2vhostvr1024-1vm-mrr.robot - 10ge2p1x520-eth-l2bdbasemaclrn-eth-2vhostvr256-1vm-mrr.robot - 10ge2p1x520-eth-l2bdbasemaclrn-eth-4vhostvr1024-2vm-mrr.robot - 10ge2p1x520-eth-l2xcbase-eth-2vhostvr1024-1vm-mrr.robot - 10ge2p1x520-eth-l2xcbase-eth-2vhostvr256-1vm-mrr.robot - 10ge2p1x520-eth-l2xcbase-eth-4vhostvr1024-2vm-mrr.robot - 10ge2p1x520-ethip4-ip4base-eth-2vhostvr1024-1vm-mrr.robot - 10ge2p1x520-ethip4-ip4base-eth-2vhostvr256-1vm-mrr.robot - 10ge2p1x520-ethip4-ip4base-eth-4vhostvr1024-2vm-mrr.robot - 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhostvr1024-1vm-mrr.robot - 40ge2p1x710-eth-l2bdbasemaclrn-eth-2vhostvr1024-1vm-mrr.robot - 40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhostvr1024-1vm-mrr.robot - 40ge2p1xl710-eth-l2bdbasemaclrn-eth-4vhostvr1024-2vm-mrr.robot Change-Id: Idd049db6134179f0f9ed603569ec8512faca1ab9 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2018-03-19FIX: remove all vpp packages installedJan Gelety1-10/+10
Change-Id: I1a2e51fada2f16fb236499a5cbff735f5e08aae2 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2018-03-19Update of VPP_STABLE_VER filesJan Gelety2-2/+2
- use new vpp ref build - ubuntu 16.04: 18.04-rc0~425-g039cbfe~b3921_amd64 - use new vpp ref build - centos7: 18.04-rc0~425_g039cbfe~b3910.x86_64 Change-Id: Id8c7e767b67b9dc8ad7c5ae129279df5d13f6466 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2018-03-19CSIT-998: MRR testsTibor Frank21-20/+4273
- CSIT-999: MRR tests: L2 - 10ge2p1x520-eth-l2xcbase-mrr.robot - 10ge2p1x520-eth-l2bdbasemaclrn-mrr.robot - 10ge2p1x520-dot1q-l2bdbasemaclrn-mrr.robot - 10ge2p1x520-dot1q-l2xcbase-mrr.robot - 10ge2p1x520-eth-l2bdbasemaclrn-iacl1sf-10kflows-mrr.robot - 10ge2p1x520-eth-l2bdbasemaclrn-iacl1sl-10kflows-mrr.robot - 10ge2p1x520-eth-l2bdbasemaclrn-iacl50sf-10kflows-mrr.robot - 10ge2p1x520-eth-l2bdbasemaclrn-iacl50sl-10kflows-mrr.robot - 10ge2p1x520-eth-l2bdscale100kmaclrn-mrr.robot - 10ge2p1x520-eth-l2bdscale10kmaclrn-mrr.robot - 10ge2p1x520-eth-l2bdscale1mmaclrn-mrr.robot - 10ge2p1x710-eth-l2bdbasemaclrn-ndrpdrdisc.robot - 40ge2p1xl710-eth-l2bdbasemaclrn-mrr.robot - 40ge2p1xl710-eth-l2xcbase-mrr.robot - CSIT-1000: MRR tests: IPv6 ─ 10ge2p1x520-ethip6-ip6base-copwhtlistbase-mrr.robot - 10ge2p1x520-ethip6-ip6base-iacldstbase-mrr.robot ─ 10ge2p1x520-ethip6-ip6scale200k-mrr.robot ─ 10ge2p1x520-ethip6-ip6scale20k-mrr.robot ─ 10ge2p1x520-ethip6-ip6scale2m-mrr.robot ─ 40ge2p1xl710-ethip6-ip6base-mrr.robot Change-Id: Ie42b08151b5da3ba32b972af8b6bedf313c6d29c Signed-off-by: Tibor Frank <tifrank@cisco.com>
2018-03-16Add vhost and memif MRR suitePeter Mikus21-8/+3438
Change-Id: Iaded13af82383955c2ff093ddc96e09682d95776 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-03-16VPP install and verify in __init__.robotAndrej Kilvady5-53/+212
Move VPP installation to separate test in test suite setup phase to clearly indicate any issue with VPP installation. Added test to check VPP responsiveness after installation. Change-Id: Idc2c78152e23aa7301bb5dbf9b1b6f4b639c3e84 Signed-off-by: Andrej Kilvady <andrej.kilvady@pantheon.tech>
2018-03-14Robot logs archiving for trending jobsPeter Mikus2-8/+15
- Add output.xml auto-archiving to logs.fd.io. - Cleanup from obsolete code. Change-Id: Iafa9ef802264d2ad36ca0f556cb14d2b3ca0f179 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-03-13HC Tests: disable NSH pluginMarek Gradzki6-16/+26
NSH artifacts in Nexus are outdated and crash VPP. Reenable NSH after it is fixed (CSIT-994). Change-Id: I3a0367682d62d5ccdbf008336ed682e7ab9032a0 Signed-off-by: Marek Gradzki <mgradzki@cisco.com>
2018-03-12Optimize VPP restart after startup.conf changePeter Mikus1-7/+4
Change-Id: Iabe7de5d598f904bd7e691c3ffed1d07ed0c7f68 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-03-12Update of VPP_STABLE_VER files and DPDK_STABLE_VER fileJan Gelety3-3/+3
- use new vpp ref build - ubuntu 16.04: 18.04-rc0~377-g03f47f1~b3869_amd64 - use new vpp ref build - centos7: 18.04-rc0~377_g03f47f1~b3862.x86_64 - use new dpdk version: 18.02-vpp1 Change-Id: Ic2bf4850caec109f7d588afedb8eec0c76ab8d3a Signed-off-by: Jan Gelety <jgelety@cisco.com>
2018-03-08CSIT-989 Fix: Reflect latest memif API changesPeter Mikus13-471/+533
Change-Id: I0c9cb073a42a8b4b7e573023e784ab9e72d0ad55 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-03-08CSIT-988: Add libmbedcrypto0 as a new vpp dependencyPeter Mikus4-3/+13
Add libmbedcrypto0 to the list of installed packages on Physical testbeds + VIRL images Change-Id: Ied68b79a36d0b6c8619aa22effeac30e5f3c0cf6 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-03-07FIX: Bootstrap.sh for functional VIRL jobsPeter Mikus1-5/+3
- Fix the path of temporary result files to workspace instead of /run/shm Change-Id: I4819ef5d6371412a91b6736aa36b223df38eacac Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-03-07CSIT-988: Fix functional bootstrapPeter Mikus2-3/+7
Change-Id: I6fb16e799e991ce01e9f997a05c4be72a26ce3af Signed-off-by: Peter Mikus <pmikus@cisco.com>