aboutsummaryrefslogtreecommitdiffstats
path: root/resources/libraries/bash/function
AgeCommit message (Collapse)AuthorFilesLines
2021-01-07fix: use csit lts branch in vpp-csit lts jobsJan Gelety1-1/+5
Change-Id: Iebd62fd6b0c798f7b4dd1f3b093c156e533b3900 Signed-off-by: Jan Gelety <jgelety@cisco.com> (cherry picked from commit a5b3a8b91d8e6c9baa4361d70b96769a98bfc454)
2021-01-04FIX: AWK NIC replace in case of x520pmikus1-1/+1
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I7d83ba048e0609d6b8623fab5c2960e48a37c023 (cherry picked from commit 3c3930b9f6f9a40d6b30f6e56e8c40279e34f650)
2020-12-18FIX: Backward compatibility include selectionpmikus1-2/+9
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: Ieff60a44d42d66acee8ba1680e7e285d6cd01bc9 (cherry picked from commit e62ade12b7c9918cda2d363387ed0f517aa9840d)
2020-12-17Switch include to testpmikus1-22/+36
+ Reduce time overhead when parsing --include vs --test + Input files will remain the same + 3n-hsw 150include ~24min, 150test ~5min + 2n-clx 489include ~61min, 489test ~9min Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: Ia453b1bc1d1862bfc378a7611064a67ee564e2f2
2020-12-04vpp_device: fix vpp device kernel driver bindingJuraj Linkeš1-9/+5
Attempt to unbind a driver from a device only if it is bound to a driver. Remove the dynamic addition of an existing device ID to a driver. From the docs [0]: Writing a device ID to this file will attempt to dynamically add a new device ID to a PCI device driver. Since we assume the VFs are bound to the kernel driver when VPP Device topology creation is done, it implies that the kernel driver supports the device ID of those VFs, removing the need to add the support. [0]: https://www.kernel.org/doc/Documentation/ABI/testing/sysfs-bus-pci Change-Id: I20f3ca071a5a84a06ff358ba514532248a8f9ad0 Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech> (cherry picked from commit 71d7150a65a7c006bf46b2c1001dbaa00b5681fb)
2020-12-03Avoid ci-management archiving files twiceVratko Polak2-57/+45
This is a follow-up to https://gerrit.fd.io/r/c/csit/+/20394 https://gerrit.fd.io/r/20119 has changed the way archival works, everything should now go to logs.fd.io (instead of appearing on run page in jenkins.fd.io). The glob pattern for archiving is quite eager, doing recursive search. That is good, as it can find also misplaced useful outputs. But it also means our usage of copy_archives function creates two copies of archived directtories, usually archives/ and archives/archive/. This change renames copy_archives to move_archives, with few workarounds to support multiple calls. I also renamed ARCHIVE_DIR value from $CSIT_DIR/archive to $CSIT_DIR/archives to make "move" operation look natural. Finally, download_builds function is being removed, as after recent improvements to VPP compilation speed nobody seems to be using it. Change-Id: I19a429e1dfdfaab7fcf32a9609963b1aebd33c6c Signed-off-by: Vratko Polak <vrpolak@cisco.com> (cherry picked from commit 523c6e6e24101206ff1318ca17c310dff8b3c9d2)
2020-12-01Tolerate base64 encoded Gerrit commentsVratko Polak1-2/+8
+ Move "|| true" to a place that really aviods errors. + Attemp to parse decoded string if trigger is not found in plain one. Change-Id: If3587229ec588f9ad41acb3050add1142032d2d8 Signed-off-by: Vratko Polak <vrpolak@cisco.com> (cherry picked from commit 4582f0f408616cdff8e606ac3abfe154f8f0514b) (cherry picked from commit f89bc7a87e0b6015e50de3557a1724c8aaafbf60)
2020-11-162n-zn2 Epyc backportpmikus1-8/+15
- Better to get it fully aligned then cherry-pick spaghetti. Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: If223ef3f0247413d53225eb57f8903a7675632e3
2020-10-14vpp_device: fix shim hostnameJuraj Linkeš1-5/+4
Replace the hacky grep of /etc/resolv.conf with default docker gateway IP. Change-Id: Iec3a4658826f2ba871acb14d511e9c79a1273290 Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2020-10-14vpp_device: updates for 1n-tx2 testbedsJuraj Linkeš1-11/+7
One ThunderX2 9975 server (.69) was replaced with two ThunderX2 9980 (.70, .71) servers. Move the .69 server under ansible perf section in anticipation of repurposing it for that purpose. Update the ansible scripts with .70 and .71 config and rename port names in device.sh lib to reflect the NIC differences between .69 and .70 (and .71). Change-Id: I88b75648735243e5559175d3192ffcc8fc70071c Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2020-09-21FIX: Unknown trigger filespmikus1-5/+5
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I8d6108af943d729fecbcfe4867ea820a69b4eb1e
2020-09-03Framework: Bump DPDK 20.08pmikus1-88/+22
+ DPDK 20.08 + Migrate make -> meson + Fix all trending issues Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I31dcb22627c0f8d17ec63c5b138a2da958b006f4
2020-08-06Limit user triggers: no ORVratko Polak1-0/+7
To avoid runs such as: https://jenkins.fd.io/job/vpp-csit-verify-perf-master-3n-hsw/340/ Change-Id: I1b30d5f440ddf8ff32b11265b2ac2176f4b9a665 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-07-23perpatch: Echo MAKE_PARALLEL_* var before buildJuraj Linkeš1-1/+15
VPP uses MAKE_PARALLEL_FLAGS or MAKE_PARALLEL_JOBS to limit the number of cpus to use during build, so emit a line on stdout if it's used. Change-Id: I669398d474d172abb6c848a45f24f1bdd56990d8 Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2020-06-30make graph node variant can be configured from gerrit triggerYulong Pei1-0/+13
for example, below will trigger VPP with default code path test in CSIT, "csit-2n-clx-perftest mrrAND64bANDnic_intel-xxv710ANDeth-l2bdbasemaclrnNOTdrv_avf" and below will trigger VPP with avx512 code path test in CSIT, "csit-2n-clx-perftest icl mrrAND64bANDnic_intel-xxv710ANDeth-l2bdbasemaclrnNOTdrv_avf" Change-Id: Id3abf932203ce00f34f8643c5bc4d5fc4d3e6efb Signed-off-by: Yulong Pei <yulong.pei@intel.com>
2020-06-11Update selection of report-iterative test setsTibor Frank1-1/+4
- Create the same directory structure for report-iterative as it is for report-coverage. - Move files from report-iterative directory to created sub-directories. - Rename the moved files to keep the convention defined in report-coverage. - Update the corresponding functionality in resources/libraries/bash/function/common.sh The patches: - https://gerrit.fd.io/r/c/csit/+/27486 - https://gerrit.fd.io/r/c/csit/+/27484 - https://gerrit.fd.io/r/c/csit/+/27485 must be updated. Change-Id: I1aaf22690e3df5abf56893bb1d2d380debc54ddd Signed-off-by: Tibor Frank <tifrank@cisco.com>
2020-06-03Fix DPDK build on armJuraj Linkeš1-0/+6
Directive #include <file> in GCC implementation searches for files in a standard list of system directories, which leads to a build error on Taishan arm machines: /tmp/openvpp-testing/dpdk/lib/librte_ethdev/rte_ethdev.h:4287:10: fatal error: rte_ethdev_core.h: No such file or directory #include <rte_ethdev_core.h> Fix this by replacing the directive with #include "file" which searches for files first in the directory containing the current file and other directories only after that. Change-Id: If283b81c992c001a2f980312cdd99053c3d703d9 Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2020-05-27FIX: check if testpmd is aliveJan Gelety1-3/+5
Change-Id: I12f21a4a7ae7c8593c42fa8e6cc4d65e36d41270 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-05-28Remove hugepage dirs when killing dpdkJuraj Linkeš1-1/+1
There could be directories under /dev/hugepages/ so remove those as well. Change-Id: I8ecdde159b8e0049460d0bcdc8b1a2c85b1ed25f Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2020-05-26FIX: add dpdk mrr_weekly and report_iterative job specsJan Gelety1-4/+5
- split job specs to directories Change-Id: I9a71621216ab0bdf85d147ce03f473e7cbb0b266 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-05-26Add csit-vpp/dpdk-perf-report-coverage job and job specificationsJan Gelety1-0/+8
- note: currently dpdk report-coverge test sets are equal to dpdk report-iterative test sets Change-Id: Ib9a718bfea4b24842742f49ee2eca022aa8a08e5 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-05-18Add csit-vpp-perf-report-iterative jobJan Gelety1-0/+5
- job will be used to collect vpp performance iterative data for csit report Change-Id: I4cff5caa0a97d08b11f3eb9b72963deaf1b04c73 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-05-14FIX: Download only vpp packagesJan Gelety1-1/+1
Signed-off-by: Jan Gelety <jgelety@cisco.com> Change-Id: I88682b972947d73ae8433b550bd5e96b0b7c2c76 (cherry picked from commit 36237cf89a092fee57faa6b2b1726d40df493845)
2020-05-13Infra: PIP installation freezepmikus1-3/+3
+ Waiting for Docker image to be updated. Then next step I will disable any update attempt. Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I92d4e27be0537a95473cd723a45e6185b26ce599
2020-05-13job-specs: new job test spec files including test count and durationsJan Gelety12-3152/+15
Change-Id: I4ad87092bff1c072051ca1c7243d8dedb1c4576e Signed-off-by: Jan Gelety <jgelety@cisco.com> Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
2020-05-04Print more info when checking out oper branchVratko Polak1-0/+7
Change-Id: Ic8d1080edacea1d27eb80e61f48cd4cd4430c139 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-04-23Performance: DPDK refactorPeter Mikus1-0/+362
+ Rework BASH scripts (more code to python). + Move BASH into libraries. + Allows RDMA usage. + Fix 9000B tests. + Rename confusing l2fwd -> testpmd. + Fix suite setup. + Fix PCI whitelist to not accidentally pickup wrong interface. + Fix deprecated DPDK cli arguments. - MLX5 jumbo are disabled on NIC (i will increase separately). https://jenkins.fd.io/job/csit-dpdk-perf-verify-master-2n-clx/6/console (l3fwd still broken) - MLX5 IMIX seems to be some TRex issue with IMIX for mlx5 (i will handle separately) Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I31d1b67305fa247cb5e1f57e739d3ef30dc1a04b
2020-04-06Ansible: Speed up by pipeliningPeter Mikus1-2/+6
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I3ad5675cda017b3dcf8cba88d56d5d6c11184e20
2020-04-02Fix: select correct oper branch checkout_csit_for_vppJan Gelety1-1/+4
Signed-off-by: Jan Gelety <jgelety@cisco.com> Change-Id: I2e72b7e136d3907a038cb7729c11d155a12c486f (cherry picked from commit 9e2e4eaa2c6854eee47f7057d16b75339f453f89)
2020-02-28bash: tolerate -dev as a package nameVratko Polak1-2/+2
Misbehaving projects can create it, and apt thinks it is a cli option. Change-Id: Icc42231e2d498f976281ce725373c6679fe61fa4 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-02-18Download latest VPP packages for hourly jobsjuraj.linkes1-3/+4
1n-tx2 vpp-device tests are supposed to give us a view of how VPP patches affect VPP device tests, so download the latest packages for them as well. Also replace round brackets with curly brackets in the condition to make it faster according to https://github.com/koalaman/shellcheck/wiki/SC2235. Change-Id: I9caafea3590d04af9cb34d6bb536ff786fe08411 Signed-off-by: juraj.linkes <juraj.linkes@pantheon.tech>
2020-02-16daily: Do not run nf_density chain_ipsec tests on 3n-hswJan Gelety1-26/+0
Change-Id: I9a1afaf3654b96161c4c1de483c99c1f849b6980 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-02-04Ansible: Cleanup and speedupPeter Mikus2-12/+36
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Ia2ce482798204ad426f93f334c97e28eb51139fd
2020-01-23FIX: nfv_densityJan Gelety1-32/+32
- use correct osi_layer=L2 (so L2 spoofing check is switched off in case of avf driver) - add pci address information to eth interface in topology file - nfv_density chain_ipsec tests work only with DPDK in current implementation Change-Id: I233c6e5634a14581960c7459b87f11fcee8365bd Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-01-22Generate tests in deterministic orderVratko Polak1-1/+2
Change-Id: I47e45f395976fd65a8da0e93bc33588cc213c79e Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-20reduce test cases for 2n-dnv and 3n-dnv mrr daily testYulong Pei2-202/+2
since dnv platform is to use by low end market product, no need care more about too much flows performance test. Change-Id: Id7204b0353cbf968e641f14ed980f01af75971b2 Signed-off-by: Yulong Pei <yulong.pei@intel.com>
2020-01-16FIX: remove 4drc nfv_density chain ipsec tests from 3n-hsw daily jobJan Gelety1-12/+0
- not enough cpu cores reported Change-Id: If3ddbbf4f5af741f328c45ac6b8c874e61d348ab Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-01-10Trending: new daily setJan Gelety13-258/+2994
New daily sets are prepared based on information in https://gerrit.fd.io/r/c/csit/+/24073/1/docs/job_specs/perf_tests_job_specs.md and previous test set definitions in docs/job_specs/test_select_list_[2n|3n]_[clx|skx|hsw|tsh|dnv].md files. - mrr-daily-2n-clx: 510 TCs (incl. nfv_density), expected exec. time 8:50h - mrr-daily-2n-skx: 525 TCs (incl. nfv_density), expected exec. time 7:55h - mrr-daily-3n-skx: 393 TCs (incl. nfv_density), expected exec. time 11:00h - mrr-daily-3n-hsw: 177 TCs (incl. nfv_density), expected exec. time 7:10h - mrr-daily-3n-tsh: 204 TCs, expected exec. time 21:00h - mrr-daily-2n-dnv: 84 TCs, expected exec. time 2:25h - mrr-daily-3n-dnv: 144 TCs, expected exec. time 6:35h + add some missing test suites + add trex-sl-2n-ethip4udp-1000u15p.py T-Rex traffic profile + correction of TS and TC names and tags in directory tests/vpp/perf/nfv_density/chain_ipsec Change-Id: Icfc86e9af97ed8dd8ccd2a34355c99aad69a28c0 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2020-01-10Autogen: Generate also NIC drivers.Vratko Polak8-112/+89
+ Disallowed -avf- (or -rdma-) as "template" suites. + GBP suite switched to DPDK driver in repo. + Each NIC has its own list of supported drivers, in Constants. + Updated tag expressions for daily jobs: + Feature, ipsec, memif, scale, srv6, tunnels, vhost and vts are tested only with vfio-pci. + Other (base, dot1q, dot1ad) tested with all drivers. + Setup actions currently depend on driver, generated. - The performance_rdma action is trivial for now. - Several tests fail, to be fixed later, e.g. by performance_rdma. + Reconf tests are also supported. + Added DRV_VFIO_PCI tags missing, mainly in density tests. - Vhost suites (density, reconf) are failing, but suites look good. - TCP suites do not support NIC drivers yet. - DPDK obviously not supported. + Use Python 3 in regenerate scripts. + Fix typos binded => bound. + File open modes set either u"rt" or u"wt" everywhere. + Remove a trailing space in an environment variable name. Change-Id: I290470675dc5c9e88b2eaa5ab6285ecd9ed7827a Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-07Remove leftovers related to XenialVratko Polak1-1/+1
+ File VPP_STABLE_VER_UBUNTU. + Subtree docs/report/vpp_functional_tests/. Change-Id: Ifd9dd922152199de9f1352da709b1533d6c15aba Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-12-16Vagrant: Introduce CentOS8Peter Mikus1-2/+2
+ Full ansible compatibility - Docker currently not working in Centos8 as it was not yet released Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I837091621db6bbebc9fee729496384cabe2f5357
2019-12-16FIX PIP requirement packagesPeter Mikus2-5/+1
+ CSIT will only use packages installed from master PIP requirement.txt file + All CSIT subsequent code must use single function and PIP list for virtualenv - Report/PAL/doc/trending will follow in next patch Change-Id: Ia9150fb05a144182a5a163178d553370ede55cdd Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-12-13Reduce spam from device bootstrapVratko Polak1-0/+6
It uses "declare -f" to paste the complete common.sh file. Twice. The output from ssh remote side still remains visible, but avoiding the spam should ease scrolling to Robot console output. Change-Id: I746484ada7650a6dfa6794e76550757af5413a82 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-12-05Improve quoting of asterisksVratko Polak1-3/+3
When quoted, the asterisks are understood as literal characters. When unquoted, so they act as globs patterns. When used to match files already present in the filesystem, the asterisks should be unquoted. When used to match a package name (not a file), the asterisks should be quoted to prevent the bash expansion, as programs such as apt-get and yum do their own expasion afterwards. Unquoted asterisks for matching package names may happen to work when there are no files to match the bash expansion, but quoted asterisks are sure to avoid accidents. Also, we want an asterisk both before and after "vpp", due to package names such as libvppinfra-dev and python3-vpp-api. Change-Id: I657a24c24a71d865915aba201bd0d3276cb33281 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-12-05FIX: Some leftovers + csit_sut image syncPeter Mikus1-4/+3
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I9b47f9a31c11f155ba5b4676ccf0fc4c413a3942
2019-12-02Refactor jumpavg to be more readable and usableVratko Polak1-3/+4
This is the python3 change, the python2 is still used for testing bisect. + New version is 0.2.0 due to amount o API changes. + Jumpavg is now part of CSIT resource package tree. + Perpatch migrated to new jumpavg. - PAL NOT updated (the update moved to a different Change). Change-Id: I7d7a8bf8a411196c20c2a40a8c64478d6709bc07 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-12-02FIX: Perpatch PY3Peter Mikus1-2/+2
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Iae97cf4490fea38f5f03bdaa120ee7ae48b941ab
2019-11-29Python3: PIP requirementPeter Mikus1-5/+6
- Install via PIP3 - Use virtualenv Python3 interpret Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Iad22b344b418a1d91774d23d5ea17bfd48d394b3
2019-11-19HONEYCOMB: RemovePeter Mikus1-122/+0
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: I60d80564050cb786c7b1e0a08f31f8292f82197e
2019-11-18Improve test tag string parsingVratko Polak1-21/+12
Instead of just adding the missing -clx, the new way aims to be forward compatible. The new implementation is quite permissive, ignoring all "word" chars sticking to the trigger word and returning everything after that and space up to the endline. + Unified perftest and devicetest handling. + Consequently, triggers like "devicetest-1n-skx GBP" will work now. Change-Id: I3bda6105195180d35a589a75318f569e36f03461 Signed-off-by: Vratko Polak <vrpolak@cisco.com>