aboutsummaryrefslogtreecommitdiffstats
path: root/resources/libraries/bash
AgeCommit message (Collapse)AuthorFilesLines
2021-11-08vpp_device: bind to vfio-pci before running testsJuraj Linkeš1-15/+101
In rare cases, binding the whole /dev/vfio folder will result in unusable VFs: notice dpdk EAL: Cannot open /dev/vfio/151: Device or resource busy [0], section 4.3.1. provides some clues as to what's going on and how to avoid the failure. Mounting /dev/vfio reset the file descriptors of all devices under /dev/vfio. Vfio-pci creates a device when an interface is bound to it. The rare failure then occurs when /dev/vfio is mounted while a process is using the file descriptors result in that process using invalid file descriptors (or file descriptors belonging to a different VF). Fix the issue by binding i40e and ice VFs to vfio-pci before containers are created and make sure that the VFs are not unbound later in testing. Only bind DUT VFs since the TG uses the kernel driver. [0]: https://connect.redhat.com/sites/default/files/2021-03/Cloud Native Network Function Requirements.pdf Ticket: CSIT-1794 Change-Id: I83db91b29d16669fb034b141ad247f6f796fdf64 Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2021-08-19Migrate report/trending/docs to bash frameworkpmikus4-0/+349
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: Idf8462f15a2dd76b42dcaea9df6a0fc15a45503f (cherry picked from commit 8079ed2ff7e2fdd41474d286b98cd9be19481f7a)
2021-08-13Tox: Move logs to archive also on -1Vratko Polak1-3/+4
Change-Id: Id29925da9c8e0bc0a28a9f4d38c332852c687317 Signed-off-by: Vratko Polak <vrpolak@cisco.com> (cherry picked from commit 8f9290d1594d35eb838c0c4c9d754b80a35e4a3a)
2021-08-13CI-MAN: Unify backupspmikus1-0/+6
+ tox must follow the $WORKSPACE/archives rule Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I717afded894c3a5468bbf5013b242be5e1a9cb6e (cherry picked from commit 88483ff0a6b4b5ff5b0390d8ff2adeb9a857c1c3)
2021-08-11Infra: Add terraform no-color output.pmikus1-2/+2
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I851e979c9b0db41c83facb92010c1b16885ee5a0 (cherry picked from commit 6de5c7fc4524c5121ed00dc2fa3ab34301d45215) (cherry picked from commit 29d8ac190da8ff0a975614e3aad2f56ebe1b5efd)
2021-08-09FIX: Terraform installationpmikus1-2/+4
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I2963520237c0bcc30836df3722a945060bd438d3 (cherry picked from commit 2674845e28c3f36e6290c78bd73c0a43b0ecca15) (cherry picked from commit 0e5b90cc8dad50365de53fa80db9817066ac3f82)
2021-08-09FIX: Terraform installpmikus1-0/+2
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I1bc29338bf677c5777474ea07954df4ca0ed0a31 (cherry picked from commit 13f4ec1dc98710d2cf226de3643dc5a0a6c5ac67) (cherry picked from commit dbace27a91928d386f79943a2faa13411d14ff63)
2021-08-09Infra: Install terraformpmikus1-1/+3
+ Unless docker images gets updated Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I9d09b24ca7bef71dad5774ae2b5e2e67e7b8f285 (cherry picked from commit ddd2cc047694396928086482339588966a858fba) (cherry picked from commit 62420b659ffa75edbfe7c8ac4f480dff5b01c27e)
2021-07-27FIX: Executable bit for ansible pluginpmikus1-3/+5
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I2416a9682710441d098acfff50e943ac3f788500 (cherry picked from commit f7c3f3918646a7bb157e648ed96beb9bb5c232ea)
2021-07-26Infra: Terraform add ansible plugin initpmikus1-0/+113
+ Manual override Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: If296d28dc3eaa023c935a0887a14ac10e83076ba (cherry picked from commit b27f2c7cbfe08712a00959c8cc6a2f618a25b3e7)
2021-07-19Report: Add rls dataTibor Frank1-0/+1
Change-Id: I9c81563c904b02c5ca6ede5dc3d0922851c11ae4 Signed-off-by: Tibor Frank <tifrank@cisco.com> (cherry picked from commit 39957a024d1fff87403aacc34aaaa5a922b6639f)
2021-07-13job-spec: Add hoststack and vsapTibor Frank1-0/+1
+ hoststack + vsap (cps, rps) + add e810cq to common.sh Change-Id: I2e897597d13e488f7324a7b91826ca32fd93c84f Signed-off-by: Tibor Frank <tifrank@cisco.com> (cherry picked from commit e77fa027ad2a541e53d92743d8477d3aecf944e2)
2021-06-10FIX: Pylint reducepmikus26-29/+29
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I909942dbb920df7f0fe15c0c92cda92c3cd8d8ad (cherry picked from commit 7829fea4a2c8936513fa95215b7d84997f814a69)
2021-06-10Infra: vpp_device -> Ubuntu 20.04pmikus1-2/+4
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: If24c385b93e1ba310ea258984c97494b4cab467c (cherry picked from commit f754eac4faf76fdbffac648769c03f5668f9f6a2)
2021-06-01Tox: Run pylint also for GPLVratko Polak1-2/+2
Change-Id: Ic6c5d35dc602638b06962fcad30d8517c9553552 Signed-off-by: Vratko Polak <vrpolak@cisco.com> (cherry picked from commit 1d8887789f98551aac51571b8c4c0c3a18612203)
2021-05-20perf: add TCP Nginx+LDPRELOAD suitesxizhanx3-3/+177
1. Suite steup add download nginx 2. Add nginx-1.14.2/1.15.0 ldp test suite 3. Add NginxUtils,NginxConfigGenerator method 4. Taskset the PID of nginx to the unused cores in VPP and these cores are under NIC's NUMA ID 5. cleanup add Kill Processes - nohup Signed-off-by: xizhanx <xix.zhang@intel.com> Change-Id: Idbf0e4ec3bf63e88281a8e3e34f52e00a6801c85 Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2021-04-27Infra: AWS - add CSIT parameters to bootstrapTomas Alexy3-2/+29
Signed-off-by: Tomas Alexy <tomas.alexy@pantheon.tech> Change-Id: I41b5905bcaab303a74d9985283e469922e5fba94
2021-04-20tox: Add job-spec checkerTibor Frank1-0/+60
Change-Id: I2bbdc28431b6271d1950ec0e74bf708cae841146 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2021-04-15common.sh - fix 3n AWS topology tag selectionTomas Alexy1-2/+2
Signed-off-by: Tomas Alexy <tomas.alexy@pantheon.tech> Change-Id: Ic6dd4a14887d572487d7d4efcca8a437cf140ca9
2021-04-14Infra: common.sh - add ENA 50g NICTomas Alexy1-0/+1
Signed-off-by: Tomas Alexy <tomas.alexy@pantheon.tech> Change-Id: Idc672f6086ab9d4ad38b52947a52c667d0908d8f
2021-04-12Infra: AWS bootstrap integrationTomas Alexy2-3/+45
Signed-off-by: Tomas Alexy <tomas.alexy@pantheon.tech> Change-Id: I254c79201a198ee054851c934167a24e9e8803cb
2021-04-08Ansible git movepmikus1-4/+4
+ Better accessibility + Compliant with fdio.infra._function_ - function [pxe|terraform|ansible|vagrant] + dill==0.3.3 also applied on TBs - ci-man to follow today - Docs to be updated in separate patch Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: Iff9eaa29d63044188cc8160db2d9b44b7635782a
2021-04-06DPDK buid: Fix sed commandVratko Polak1-1/+1
The previous way of concatenating strings would not work even in Python. Change-Id: Ice06f4a06fefbc4e42ee2572786244207dfefa11 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-03-29FIX: VPP version file name based on OSpmikus1-13/+13
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: Icab8b87498e8487268857bb04c5ba7069d9297e3
2021-03-26ansible: update TX2 VPP Device hugepages and ifsJuraj Linkeš1-8/+2
These interfaces are not used, so remove them until we actually use them. Use less amount of hugepages since we don't need as much and the rest of the memory is more useful for build/host processes. Change-Id: I52b2d6e2812e5cadeab9e51a1bae3688794f414a Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2021-03-23Infra: fix NUMA count in Arm DPDK buildJuraj Linkeš1-3/+6
The default NUMA value for Taishan build is insufficient and results in failed test runs. Fix by increasing the amount to what's on the device. Change-Id: Ia7c761e81ee9063d7129472d9caa8cebbbb7e1cf Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2021-03-20Tox: Increase verbosityVratko Polak1-2/+3
CSIT is becoming stricter in avoiding downloads during run, as everything should be pre-cached in executor image. This change makes it easier for human to detect downloads. Change-Id: If7085f96ef1033f7f9405a86232a3d39a239d3e1 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-03-12Allow IPSECSW on 2node testbedsVratko Polak1-5/+5
Change-Id: I71d72699a7fa99749e2554fcb2252f8bd86bfdfa Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-03-10Infra: Do not strict check keys in Ansiblepmikus1-7/+9
- not until we migrate to SSH keys in ubuntu 2004 Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I796f86922c672ba053d64f317cf6adb46b845cac
2021-02-08FIX: Nic selectionpmikus1-2/+2
- Buggy unnoticed for quite some time Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I7d2387186e0250b80d664e2f004f909fd80c7bfa
2021-01-11Infra: Increase shm-sizepmikus1-2/+2
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: Ib9dfb8623757fd23fd8b49b5e04ed86e202ada66
2021-01-07fix: use csit lts branch in vpp-csit lts jobsJan Gelety1-1/+5
Change-Id: Iebd62fd6b0c798f7b4dd1f3b093c156e533b3900 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2021-01-04FIX: AWK NIC replace in case of x520pmikus1-1/+1
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I7d83ba048e0609d6b8623fab5c2960e48a37c023
2020-12-21Framework: add 2n-tx2 perf testbedJuraj Linkeš1-1/+12
Add testbed file and make code changes in common.sh. Change-Id: I493bb92db3c66358afa81ec81bcd99b296f49c03 Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2020-12-18FIX: Backward compatibility include selectionpmikus1-2/+9
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: Ieff60a44d42d66acee8ba1680e7e285d6cd01bc9
2020-12-17Switch include to testpmikus1-22/+36
+ Reduce time overhead when parsing --include vs --test + Input files will remain the same + 3n-hsw 150include ~24min, 150test ~5min + 2n-clx 489include ~61min, 489test ~9min Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I7e5957c32385aecbc4551bd124da41afa5bc63c9
2020-12-09Tox: Add GPL license checkerVratko Polak2-1/+74
A common mistage is to add a file into GPL directory, with Apache license only. This adds a tox checker to vote -1 on such changes, gpl_license.log contains the list of affected files. There are TODOs related to unlicensed files or unconventional GPL licenses. Change-Id: I267e4fb98afd199d5acc7d211068b28c97b31dc9 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-12-07Fix vpp-csit device scriptVratko Polak1-1/+0
In Gerrit 30041 that was not tested properly. Change-Id: I42f57f1933b0e7d029030f4ec502f7268780827b Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-12-04vpp_device: fix vpp device kernel driver bindingJuraj Linkeš1-9/+5
Attempt to unbind a driver from a device only if it is bound to a driver. Remove the dynamic addition of an existing device ID to a driver. From the docs [0]: Writing a device ID to this file will attempt to dynamically add a new device ID to a PCI device driver. Since we assume the VFs are bound to the kernel driver when VPP Device topology creation is done, it implies that the kernel driver supports the device ID of those VFs, removing the need to add the support. [0]: https://www.kernel.org/doc/Documentation/ABI/testing/sysfs-bus-pci Change-Id: I20f3ca071a5a84a06ff358ba514532248a8f9ad0 Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2020-12-02Avoid ci-management archiving files twiceVratko Polak6-70/+54
This is a follow-up to https://gerrit.fd.io/r/c/csit/+/20394 https://gerrit.fd.io/r/20119 has changed the way archival works, everything should now go to logs.fd.io (instead of appearing on run page in jenkins.fd.io). The glob pattern for archiving is quite eager, doing recursive search. That is good, as it can find also misplaced useful outputs. But it also means our usage of copy_archives function creates two copies of archived directtories, usually archives/ and archives/archive/. This change renames copy_archives to move_archives, with few workarounds to support multiple calls. I also renamed ARCHIVE_DIR value from $CSIT_DIR/archive to $CSIT_DIR/archives to make "move" operation look natural. Finally, download_builds function is being removed, as after recent improvements to VPP compilation speed nobody seems to be using it. Change-Id: I19a429e1dfdfaab7fcf32a9609963b1aebd33c6c Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-12-01Tolerate base64 encoded Gerrit commentsVratko Polak1-2/+8
+ Move "|| true" to a place that really aviods errors. + Attemp to parse decoded string if trigger is not found in plain one. Change-Id: If3587229ec588f9ad41acb3050add1142032d2d8 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-11-26vpp_device: disable IPv6 in containersJuraj Linkeš1-0/+3
Even with IPv6 nd disabled in VMs, there are IPv6 packets coming from DuT: --dut_if2_mac ba:dc:0f:fe:02:05 ... dst = 33:33:00:00:00:01 src = ba:dc:0f:fe:02:05 Docker disables IPv6 by default on interfaces it manages, which doesn't include VF interfaces. Disabling IPv6 in containers using sysctl may fix this. Change-Id: Idba506653442a3da2cd82e69cedddb3294d92788 Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2020-11-10Speed up test archive compressionVratko Polak1-1/+1
16 times faster setting at cost of 50% larger archive. Before: $ time ( cat tests.tar | xz -9e > tests.9e.tar.xz ) real 0m41.640s user 0m41.009s sys 0m0.602s $ wc -c tests.9e.tar.xz 913432 tests.9e.tar.xz After: $ time ( cat tests.tar | xz -3 > tests.3.tar.xz ) real 0m2.422s user 0m2.294s sys 0m0.161s $ wc -c tests.3.tar.xz 1556080 tests.3.tar.xz Change-Id: Ia5c86183d4e19db3cbf95dd84cf86ee720ef4051 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-10-14vpp_device: fix shim hostnameJuraj Linkeš1-5/+4
Replace the hacky grep of /etc/resolv.conf with default docker gateway IP. Change-Id: Iec3a4658826f2ba871acb14d511e9c79a1273290 Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2020-10-12Infra: Enable AMDpmikus1-0/+4
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I3954157e91aafd370c3ac0843708903d19b84936
2020-10-07vpp_device: updates for 1n-tx2 testbedsJuraj Linkeš1-11/+7
One ThunderX2 9975 server (.69) was replaced with two ThunderX2 9980 (.70, .71) servers. Move the .69 server under ansible perf section in anticipation of repurposing it for that purpose. Update the ansible scripts with .70 and .71 config and rename port names in device.sh lib to reflect the NIC differences between .69 and .70 (and .71). Change-Id: I88b75648735243e5559175d3192ffcc8fc70071c Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2020-09-23Infra: Add 2n-xn2pmikus1-7/+10
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I2ff6526b5209dabf70c6593f60162d9b8c147014
2020-09-21FIX: Unknown trigger filespmikus1-5/+5
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I8d6108af943d729fecbcfe4867ea820a69b4eb1e
2020-09-03Framework: Bump DPDK 20.08pmikus1-88/+22
+ DPDK 20.08 + Migrate make -> meson + Fix all trending issues Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I31dcb22627c0f8d17ec63c5b138a2da958b006f4
2020-08-06Limit user triggers: no ORVratko Polak1-0/+7
To avoid runs such as: https://jenkins.fd.io/job/vpp-csit-verify-perf-master-3n-hsw/340/ Change-Id: I1b30d5f440ddf8ff32b11265b2ac2176f4b9a665 Signed-off-by: Vratko Polak <vrpolak@cisco.com>