aboutsummaryrefslogtreecommitdiffstats
path: root/resources/libraries
AgeCommit message (Collapse)AuthorFilesLines
2019-07-17Use PapiSocketProvider for most PAPI callsVratko Polak27-476/+787
Ticket: CSIT-1541 Ticket: VPP-1722 Ticket: CSIT-1546 + Increase timeout to hide x520 slownes of show hardware detail. - Install sshpass and update ssh client in virl bootstrap. + Added TODOs to remove when CSIT-1546 is fixed. + Enable default socksvr on any startup conf. + Improve OptionString init and repr. - The non-socket executor still kept for stats. + Remove everything unrelated to stats from non-socket executor. - Remove some debug-loooking calls to avoid failures. TODO: Introduce proper parsing to the affected keywords. + Reduce logging from PAPI code to level INFO. - Needs https://gerrit.fd.io/r/20660 to fully work. + Change default values for LocalExecution.run() + Return code check enabled by default. Code is more readable when rc!=0 is allowed explicitly, and the test code will now detect unexpected failures. + Logging disabled by default. Output XML is large already. Important logging can be enabled explicitly. + Restore alphabetical order in common.sh functions. Change-Id: I05882cb6b620ad14638f7404b5ad38c7a5de9e6c Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-07-17Refactor VPP Device VM vhost testsjuraj.linkes9-176/+238
* replace the current VM image with kernel img * rework keyword usage to make it consistent with performance tests * remove resources/libraries/robot/shared/qemu.robot as it's not used anywhere anymore Change-Id: Ia5bc19e9e6ed9af031e4d9b5c0c89431fb49fd33 Signed-off-by: juraj.linkes <juraj.linkes@pantheon.tech>
2019-07-17FIX: Move per-patch building mechanicsPeter Mikus1-5/+2
- Use make pkg-verify Change-Id: Id0f95c9261f1a1b5fa017b0823f87aeb5d6bb6d1 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-07-12VPPD: Cop, iACL, PolicerPeter Mikus4-139/+113
- Alignment with PERF Change-Id: Id7b2cb91c88e44cf9265c3376330f6aa579abf7a Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-07-12PapiExecutor always verifiesVratko Polak20-765/+457
Do not support returning unverified replies anymore. Basically, ".get_replies().verify_replies()" is now just ".get_replies()". This allows fairly large simplifications both at call sites and in PapiExecutor.py + Rename get_dumps to get_details. + Introduce get_reply and get_sw_if_index. + Rename variables holding get_*() value, + e.g. get_stats() value is stored to variable named "stats". + Rename "item" of subsequent loop to hint the type instead. + Rename "details" function argument to "verbose". + Process reply details in place, instead of building new list. - Except hybrid blocks which can return both list or single item. - Except human readable text building blocks. + Rename most similar names to sw_if_index. - Except "vpp_sw_index" and some function names. + Use single run_cli_cmd from PapiExecutor. + Do not chain methods over multiple lines. + Small space gain is not worth readability loss. + Include minor code and docstrings improvement. + Add some TODOs. Change-Id: Ib2110a3d2101a74d5837baab3a58dc46aafc6ce3 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-07-12Avoid ci-management archiving files twiceVratko Polak5-31/+1
https://gerrit.fd.io/r/20119 has changed the way archival works, everything should now go to logs.fd.io (instead of appearing on run page in jenkins.fd.io). The glob pattern for archiving is quite eager, doing recursive search. That is good, as it can find also misplaced useful outputs. But it also means our usage of copy_archives function creates two copies of archived directories, usually archives/ and archives/archive/. This change simply removes copy_archives, relying on the new recursive search. A solution with changing ARCHIVE_DIR and moving if needed would be better, but looking at amout of jobs and various func bootstraps, it is not worth the effort. Change-Id: Iad2bc09455944307e59eecafa64d1db6dc8befe8 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-07-12Add per arch mrr daily test for denverton platformYulong Pei7-1/+17
Change-Id: Ica7d6bf0dbdfde3de70eabd131c79cadd8af3a16 Signed-off-by: Yulong Pei <yulong.pei@intel.com>
2019-07-11CSIT-1477: add 1n_tx2 VPP Devicejuraj.linkes5-45/+64
- add 1n_tx2 testbed - update VF reservation to meet 1n_tx2 testbed needs - update ansible with 1n_tx2 Change-Id: Ia075a913d4859f537fd0e6bff731ea88aff01dd9 Signed-off-by: juraj.linkes <juraj.linkes@pantheon.tech>
2019-07-11Aligning FUNC/PERF keywords - Data drivenPeter Mikus1-4/+18
Change-Id: I233c8908c724556db09c60de6efa9237068fb4c3 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-07-10Fix few pylint violationsVratko Polak4-11/+9
+ Stop attempting to check test/ as module inits are not there. Change-Id: Ia4e498061be3e3118b07e98c9c2f761f2454653e Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-07-10Tolerate failures when setting MTUVratko Polak1-3/+8
Because some NICs are known to not allow jumbo frames. Change-Id: I45fe636e3a880c6640922468c170f50e85dd2ba2 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-07-10FIX: use correct slicing of classify mask/match stringJan Gelety1-28/+75
Change-Id: I11cc8ddc6925b44a596aa03d71318efd76232d6d Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-07-10Bash functions style cleanupVratko Polak8-132/+205
+ Update rst documentation for bash style + Command substitution: + Clarify when to use backticks. + Recommend avoiding nested command substitution. + Do not recommend putting command substitution results into quotes. + Function definition content: + Move "set -exuo pipefail" after comment only blocks. + Other set flags allowed for functions with good reasons. + Apply the new recommendations. - Blank lines unified in code but no written recommendation in rst. + Add missing references to functions called, variables read or set. + Add TODOs to where lists would be long. + Minor improvements to function descriptions. + Make "if" expressions more python-like. + Add missing "|| die" (or "|| true") where spotted. + Downgrade DEFAULT_NIC to a local variable. + Add TODO to list reasons for blacklisted tags. Change-Id: I05dce030a8c2cb1b3a242d8b977e8fe150d8ee20 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-07-09Add scale ipsec tunnel tests 20KPeter Mikus1-15/+28
Change-Id: Ib52e1735b6b82ea9fea44c06c379f117068e94c1 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-07-09CSIT-1539 Aligning FUNC/PERF keywords - Phase IIIPeter Mikus7-164/+160
Change-Id: Ibd3e99390bf004173d3d126946b95ffe9ba8f196 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-06-19FIX: VXLAN scale testsJan Gelety3-297/+438
Change-Id: I9287d5ed6d7876b98add3fb3d523e9764094843b Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-06-18FIX: ip scale needs sw_if_index in VAT commandJan Gelety1-1/+19
Change-Id: Icb3e22a84af10b98edf83084991b8f2607c95798 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-07-05Remove pkt_trace as global variablePeter Mikus2-4/+4
- Use explicit parameters Change-Id: I21cbc8cf13ec5e455b20c685c4427a53a655f71f Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-07-05Add Additional Setup action for scapyPeter Mikus1-0/+7
Change-Id: I82f70d7c80b921da03553f6c1351a9a2f786ff0e Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-07-03FIX: use len parameter name in prefix structure in ip_route_add_delJan Gelety1-47/+93
+ new vpp stabel build with fixed ipv6 address incrementation Change-Id: Id3f4104a48e45b6fd70ccbd47ad9e3c6925220fd Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-07-02Replace license checker with global noticesVratko Polak1-106/+0
Caring about per-file notices is too distracting. Change-Id: I3f07a9fd1d3a4350135dd3d14f4944647af39a0b Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-07-01FIX: re-introduce VAT for IP scale testsJan Gelety1-0/+20
Change-Id: I864118ee67628ea81f2f89dab5890df8fe15c4bc Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-06-28Add copyright checker to toxVratko Polak1-0/+106
+ Only looks at files edited since HEAD~. + Only checks files of whitelisted extensions. + Distinguishes missing and outdated copyrights. + Any line with "Copyright" substring can work. + Many copyright lines are tolerated, if at least one has correct year. + Voting. Change-Id: I2cda0459cb191eeec7aada69c508973568039d87 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-06-28PAPI-to-VAT: MACSwapTibor Frank1-61/+0
- not used, not needed, deleted Change-Id: I9b4c78b7ba3fbac3805d06a65f2799ed212770b3 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-06-28VAT-to-PAPI: ClassifyTibor Frank3-396/+618
Change-Id: Ic06a0a65429680e6ecdc3f5288d091c2c2630921 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-06-27VAT-to-PAPI: IPFIX and SPANTibor Frank5-375/+22
Change-Id: I34bdc17d6350e5a441dabd9154620627780f4c12 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-06-27VAT-to-PAPI: TraceTibor Frank1-9/+8
Change-Id: Id0b6943eec7e8a15cb2db5c89f93a38f21d686c7 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-06-16FIX: IPUtil after vpp api changesJan Gelety5-134/+191
+ do not log all ip_route_add_del api calls to PAPI history in case of scale tests + fix for SRv6 proxy tests + support of more complex structures in api calls/replies in PapiExecutor + update of vpp stable build to version 19.08-rc0~487-gb98dbb1 Change-Id: I9972e126b68ce1cb57f781e3d90ea9b3914de69e Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-06-26Introduce EXPECTED_FAILINGPeter Mikus1-7/+1
Change-Id: I6c578e9126778c584ccb57a4a2c76d96dea8fb35 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-06-25Align suite/test teardown/setupPeter Mikus13-838/+461
+ Phase II - aligning FUNC/PERF setup. + 2n/3n compatibility of suite setup. + Same structure of testcase for DEV/PERF. + DEVICE tests are now running VPP startup config in same way as PERF. + DEVICE is using COREKEEPER! - DEVICE tests are running topology compute twice for backward compatibility. TODO to remove. - Still phase III needed to align TAGS, LIBRARIES and remove OLD setup phase. Change-Id: Ib6e1b0ebf4abebf3cbe73b4fda5cac953c2a7270 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-06-24Store "tag" and "url" in files in reservation dirVratko Polak1-1/+8
String "Unknown" is used if test run tag or url is not supplied, so this change should be backward compatible. + Used "runtag", as both "test" and "tag" start with taken "-t". + Fixed some pylint violations. + Docstring explaining the steps taken and reasoning behind two files. + Other minor improvements. Change-Id: If704d02e51374087fe39ad0f69432477d1b310d3 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-06-21Allow 9000b tests for AVFVratko Polak1-4/+0
See https://gerrit.fd.io/r/20148 Change-Id: I22856b14e3608348f2ccecb1d99d87209db2d776 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-06-21Align suite/test teardown/setupPeter Mikus8-435/+232
+ Phase I - aligning FUNC/PERF Change-Id: I46b1e43687ddb29f5ed3c6335fe1baf21ec02822 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-06-21SetupFramework: log thread error to consoleVratko Polak3-103/+205
Because logger.error from a non-main thread is lost. + Increase verbosity of virtualenv exception. + Introduce LocalExecution.py library. + Include multiple other style improvements. + Add some ssh.py improvements: + Accept also commands of OptionString type. + Introduce a limited form of raise_from for exception chaining. + Add include_reason argument to exec_cmd_no_error. Change-Id: I0cd92bcdaa1a5c4cc20cb99067593fc3b40d2c31 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-06-20Add 2n-dnv and 3n-dnv configuration to bootstrap-verify-perf.shYulong Pei1-0/+33
Change-Id: I90427de5b611e432e5e46ab8758a31e376215914 Signed-off-by: Yulong Pei <yulong.pei@intel.com>
2019-06-19Add continue/fail option for enable tracesTibor Frank2-7/+24
Change-Id: I675f9664391c9f18f312773812f765f345983cc1 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-06-19VAT-to-PAPI: VPPCountersTibor Frank4-81/+155
Change-Id: I5954fdd61ead9b4143f8cc4e20294cf8e199e0af Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-06-19VAT-to-PAPI: NATUtilTibor Frank3-278/+108
Change-Id: Id64bbf899e78c7c55410a8ce608393c22fa596ab Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-06-18CSIT-1459: Migrate IP libraries from VAT to PAPIJan Gelety23-2150/+795
Change-Id: Id84cf9c3afd53260b5bbf658a6a3870aa0c91333 Signed-off-by: Jan Gelety <jgelety@cisco.com>
2019-06-19VAT-to-PAPI: VPPUtilsTibor Frank3-30/+50
Change-Id: If4eb592e2718c4564f1f7929ca6ff91c02f074a6 Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-06-19Add scale ipsec tunnel testsPeter Mikus1-1/+0
- 5K - 10K Change-Id: Iaa2c77d07c3686e09fe327c663d7d7fc42024034 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-06-18Fix exit from reservation loopVratko Polak1-1/+5
Break from the while loop is on both success and no testbeds. Change-Id: Iea368034927965c7f99f5502240bd768e3f62eec Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-06-18Continue reservation when cleanup failsVratko Polak3-8/+34
+ Only on not-failed-yet testbeds. + If any, else give up. Change-Id: I3209507d3d57a77ef6afad3184ca25e098910311 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-06-18PAPI: Reduce the amount of logged informationTibor Frank2-9/+24
Change-Id: I5cfd19a9ecca6891232d92d232a159422c0a835e Signed-off-by: Tibor Frank <tifrank@cisco.com>
2019-06-17Add more tests to daily jobsVratko Polak5-1352/+6
+ Add baseline tests for all NICs, only min size. + Add scale tests for x710 NIC, only min size and without 4c. New test counts: csit-vpp-perf-mrr-daily-master-2n-skx: 96 csit-vpp-perf-mrr-daily-master-3n-skx: 211 csit-vpp-perf-mrr-daily-master: 223 Change-Id: I14c0699dd8277b6e59cdfa3e41585f50ec75dd29 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-06-15FIX: IPsec TNL mode funcPeter Mikus1-0/+4
Change-Id: I4fdcf48d8c60d728b0fe1b44baf0d2042185dd42 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-06-14Add hsw drivers to topology and check against AVFVratko Polak1-0/+5
Change-Id: Ifcc1dcdb3a0b4a7c62e4a41e5b3d016b276e93be Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-06-14Add comments for some tag exclusionsVratko Polak1-0/+7
Change-Id: I9daa70e433a2942d29c4e64856cb76d31f94a6f3 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-06-14FIX: IPsec TNL modePeter Mikus1-32/+6
Change-Id: Ide82ae5fa03d3fec8f4db9db7634be0a1e339cd1 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-06-14FIX: IPsec INT modePeter Mikus1-7/+4
Change-Id: I286490280b6e62f9f212831a5bf1d14db1838fa7 Signed-off-by: Peter Mikus <pmikus@cisco.com>