summaryrefslogtreecommitdiffstats
path: root/test/framework.py
AgeCommit message (Collapse)AuthorFilesLines
2023-11-03tests: refactor asf framework codeDave Wallace1-1553/+9
- Make framework.py classes a subset of asfframework.py classes - Remove all packet related code from asfframework.py - Add test class and test case set up debug output to log - Repatriate packet tests from asf to test directory - Remove non-packet related code from framework.py and inherit them from asfframework.py classes - Clean up unused import variables - Re-enable BFD tests on Ubuntu 22.04 and fix intermittent test failures in echo_looped_back testcases (where # control packets verified but not guaranteed to be received during test) - Re-enable Wireguard tests on Ubuntu 22.04 and fix intermittent test failures in handshake ratelimiting testcases and event testcase - Run Wiregard testcase suites solo - Improve debug output in log.txt - Increase VCL/LDP post sleep timeout to allow iperf server to finish cleanly. - Fix pcap history files to be sorted by suite and testcase and ensure order/timestamp is correct based on creation in the testcase. - Decode pcap files for each suite and testcase for all errors or if configured via comandline option / env var - Improve vpp corefile detection to allow complete corefile generation - Disable vm vpp interfaces testcases on debian11 - Clean up failed unittest dir when retrying failed testcases and unify testname directory and failed linknames into framwork functions Type: test Change-Id: I0764f79ea5bb639d278bf635ed2408d4d5220e1e Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2023-08-17tests: more descriptive error messageKlement Sekera1-1/+5
Type: improvement Change-Id: Icf8a5dc711e1e11ec919b515d7af2f487a1c04c2 Signed-off-by: Klement Sekera <klement.sekera@gmail.com> Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2023-08-16tests docs: update python3 venv packagesDave Wallace1-2/+1
- Package update performed by 1. updating pip, pip-tools, setuptools 2. 'make test-refresh-deps' on ubuntu 22.04 3. fixing 'make test' and 'make docs' issues on ubuntu 22.04 4. 'make test-refresh-deps' on ubuntu 20.04 - Add dependency for 'make test-refresh-deps' to insure python venv is set up. - Update of python formatter, black, caused reformating of 41 python code files. Type: make Change-Id: I7cafdf4b5189065ac57cb6b254937f6e0897a924 Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2023-05-20tests: support for expected failuresKlement Sekera1-46/+64
- Add support for @unittest.expectedFailure decorator. Type: improvement Signed-off-by: Klement Sekera <klement.sekera@gmail.com> Change-Id: I761751cda505e962225dc680b97c1fffa96f5176 Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2023-05-20tests: enhance counter comparison error messageKlement Sekera1-2/+4
- Make error message more human readable. Type: improvement Signed-off-by: Klement Sekera <klement.sekera@gmail.com> Change-Id: Iefc276b3a85ff82b927028a72bb91ed87ebd04ba Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2023-05-19tests: refactor extra_vpp_punt_configKlement Sekera1-3/+3
Rename extra_vpp_punt_config to a more generic name extra_vpp_config to better fit its purpose. It's fit for general use and already used that way by quic and vcl tests anyway. Type: refactor Signed-off-by: Klement Sekera <klement.sekera@gmail.com> Change-Id: Ib0a5789b0dbb3a8c3cae654dea4e32ac5e56dd41
2023-02-14tests: support tmp-dir on different filesystemDmitry Valter1-1/+1
Support running tests with `--tmp-dir` on a filesystem different from /tmp. os.rename withs only within a single FS whereas shutil.move works accross different filesystems. Type: improvement Signed-off-by: Dmitry Valter <d-valter@yandex-team.ru> Change-Id: I5371f5d75386bd2b82a75b3e6c1f2c850bc62356
2023-01-18tests: improve packet checksum functionsKlement Sekera1-6/+21
Fool-proof assert_checksum_valid so that one does not verify checksum on wrong layer (because of how scapy internally works). Make assert_packet_checksums_valid start checksum checking at inner layers and outwards to make it more obvious where the error is. With old behaviour, if one received an ICMP packet carrying a truncated TCP packet, an error would be raised for ICMP checksum, as that one would be the first to be wrong after recalculating all packet checksums, while the real issue is TCP header being truncated and thus unsuitable for use with this function. Type: improvement Signed-off-by: Klement Sekera <klement.sekera@gmail.com> Change-Id: I39a2b50ec5610f969cfde9796416ee3a50ae0ba3
2022-10-07tests: disable broken wireguard tests on vpp_debug imageDave Wallace1-0/+11
Type: test Signed-off-by: Dave Wallace <dwallacelf@gmail.com> Change-Id: I3a53d57e42f4c1f5ba0de6d2b181c7f2ad083a3a
2022-09-27tests: disable failing tests on ubuntu-22.04 debian-11Dave Wallace1-1/+57
Type: test Change-Id: I7b2314a731c83b3dcd69c999edb8ebed53839724 Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2022-09-20tests: run tests against a running VPPNaveen Joy1-3/+14
Usage: test/run.py -r -t {test_filter} Instead of starting a new instance of VPP, when the -r argument is provided, test is run against a running VPP instance. Optionally, one can also set the VPP socket directory using the -d argument. The default location for socket files is /var/run/user/${uid}/vpp and /var/run/vpp if VPP is started as root. Type: improvement Change-Id: I05e57a067fcb90fb49973f8159fc17925b741f1a Signed-off-by: Naveen Joy <najoy@cisco.com>
2022-09-19tests: skip tests failing on ubuntu 22.04Dave Wallace1-0/+26
Type: test Signed-off-by: Dave Wallace <dwallacelf@gmail.com> Change-Id: I218059de5d05680d661f302293475b6c2a7bf81d
2022-08-05tests: fix node variant selectionBenoît Ganne1-1/+1
Type: fix Fixes: 4830e4f78fb8e46b23a1a0711cd06969a77d8d95 Change-Id: Iddc73dbda633acd72bd82e52f8ae83c17e3940f6 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2022-05-10tests: replace pycodestyle with blackKlement Sekera1-356/+524
Drop pycodestyle for code style checking in favor of black. Black is much faster, stable PEP8 compliant code style checker offering also automatic formatting. It aims to be very stable and produce smallest diffs. It's used by many small and big projects. Running checkstyle with black takes a few seconds with a terse output. Thus, test-checkstyle-diff is no longer necessary. Expand scope of checkstyle to all python files in the repo, replacing test-checkstyle with checkstyle-python. Also, fixstyle-python is now available for automatic style formatting. Note: python virtualenv has been consolidated in test/Makefile, test/requirements*.txt which will eventually be moved to a central location. This is required to simply the automated generation of docker executor images in the CI. Type: improvement Change-Id: I022a326603485f58585e879ac0f697fceefbc9c8 Signed-off-by: Klement Sekera <klement.sekera@gmail.com> Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2022-03-22tests: fix DEBUG=attach functionalityKlement Sekera1-4/+7
Make make test-start-vpp-in-gdb work again. Fix incorrect temp directory when using DEBUG=attach. Type: fix Fixes: b23ffd7ef216463c35b75c831e6a27e58971f4ec Change-Id: Ie98b637acbbe0221606ccdc7b54f63885e5951a4 Signed-off-by: Klement Sekera <klement.sekera@gmail.com>
2022-03-04ip: rate-limit the sending of ICMP error messagesNeale Ranns1-0/+11
Type: improvement For error conditions, such as TTL expired, dest unreach, etc, Rate limit the sending of ICMP error messages. The rate limiting is done based on src,dst IP address of the received packet. the rate limit has been chosen, somewhat arbitrarily, to be 1e-3. This is the same limit as the ARP throttling. Signed-off-by: Neale Ranns <neale@graphiant.com> Change-Id: I4a0b791cde8c941a9bf37de6aa5da56779d3cef4
2022-02-24tests: better reporting for unexpected packetsKlement Sekera1-4/+2
Raise a new UnexpectedPacketErrror, when a packet is captured unexpectedly. This pretty-prints a terse description of said packet. Type: improvement Signed-off-by: Klement Sekera <klement.sekera@gmail.com> Change-Id: Ibac19fc5bbd82a150fec3c90940a37af6344fd4f
2022-02-21tests: add enhanced packet counter verificationKlement Sekera1-12/+74
Add support for inline packet counter verification to send_and_* functions. Diff dictionary is a dictionary of dictionaries of interesting stats: diff_dictionary = { "err" : { '/error/counter1' : 4, }, sw_if_index1 : { '/stat/segment/counter1' : 5, '/stat/segment/counter2' : 6, }, sw_if_index2 : { '/stat/segment/counter1' : 7, }, } It describes a per sw-if-index diffset, where each key is stat segment path and value is the expected change for that counter for sw-if-index. Special case string "err" is used for error counters. This then allows more precise packet counter verification by first defining a "zero" dictionary, e.g. for ED NAT: cls.no_diff = StatsDiff({ pg.sw_if_index: { '/nat44-ed/in2out/fastpath/tcp': 0, '/nat44-ed/in2out/fastpath/udp': 0, '/nat44-ed/in2out/fastpath/icmp': 0, '/nat44-ed/in2out/fastpath/drops': 0, '/nat44-ed/in2out/slowpath/tcp': 0, '/nat44-ed/in2out/slowpath/udp': 0, '/nat44-ed/in2out/slowpath/icmp': 0, '/nat44-ed/in2out/slowpath/drops': 0, '/nat44-ed/in2out/fastpath/tcp': 0, '/nat44-ed/in2out/fastpath/udp': 0, '/nat44-ed/in2out/fastpath/icmp': 0, '/nat44-ed/in2out/fastpath/drops': 0, '/nat44-ed/in2out/slowpath/tcp': 0, '/nat44-ed/in2out/slowpath/udp': 0, '/nat44-ed/in2out/slowpath/icmp': 0, '/nat44-ed/in2out/slowpath/drops': 0, } for pg in cls.pg_interfaces }) and then to specify only changed counters directly when calling one of send_and_* functions: self.send_and_assert_no_replies( self.pg0, pkts, msg="i2o pkts", stats_diff=self.no_diff | { "err": { '/err/nat44-ed-in2out-slowpath/out of ports': len(pkts), }, self.pg0.sw_if_index: { '/nat44-ed/in2out/slowpath/drops': len(pkts), }, } ) operator | is overloaded by StatsDiff class to perform a deep merge operation, so in above case, dictionaries for "err" and self.pg0.sw_if_index do not overwrite whole sub-dictionaries, rather the contents are merged, assuring that all the remaining counters are verified to be zero. Type: improvement Signed-off-by: Klement Sekera <klement.sekera@gmail.com> Change-Id: I2b87f7bd58a7d4b34ee72344e2f871b2f372e2d9
2022-02-21tests: add a generalised counter assert functionKlement Sekera1-2/+11
Type: improvement Signed-off-by: Klement Sekera <klement.sekera@gmail.com> Change-Id: I5e0fd1019372df0cd403725b8cac52363af13718
2022-02-21tests: allow not removing vpp objects after testKlement Sekera1-1/+3
Some tests might want to keep vpp config in between test functions. Add a flag to allow that. Type: improvement Signed-off-by: Klement Sekera <klement.sekera@gmail.com> Change-Id: I90243cd667dce922b43b381f3d52f4ac0f6bf3a7
2022-02-18tests: Consolidate the implementations of send_and_expect_XNeale Ranns1-0/+12
Type: test there were multiple instances of send_and_expect_load_balancing and a send_and_expect_one_itf which has the same functionality as send_and_expect_one. Put one implementation of both in framework.py (where the other send_and_X functions reside). Signed-off-by: Neale Ranns <neale@graphiant.com> Change-Id: I7f629d440220bee29368067f475059322e1134f7
2022-02-17tests: make tests less make dependentKlement Sekera1-100/+49
Implement command line argument parsing instead of passing arguments via environment variables. Add script for running tests without having to invoke make. Deprecate running tests via make. Type: improvement Change-Id: I2e3054a61a2ae25d460e9be00be7d7705fbf943e Signed-off-by: Klement Sekera <ksekera@cisco.com> Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2022-02-05tests: mark the test suites broken when ASan is enabledNaveen Joy1-0/+16
Mark broken test suites using @tag_fixme_asan.The main issue is that some tests do not pass with ASan. These will need to be fixed, but it takes time and in the meantime, new issues are introduced that trip up ASan. When this tag is set and Asan is enabled, failing ASan tests are skipped, so incremental fixes can pass the CI Type: improvement Change-Id: I02602eb74234c25a4c701279e14704b81d4c5b71 Signed-off-by: Naveen Joy <najoy@cisco.com>
2022-01-24nat: TCP state tracking based on RFC 7857/RFC 6146Klement Sekera1-2/+2
Implement proper state machine based on above RFCs. ACKs to SYNs/FINs are no longer required/tracked. This is more friendly to peers and accounts for lost packets and retransmits. This change also means that all traffic is translated and forwarded while in transitory timeout, which helps delivering e.g. retransmitted FINs, FINACKs and other messages. Also support reopening a session in transitory timeout after seeing both FINs by seeing both SYNs again. This helps quick connection reestablishment if the peers want to. Type: improvement Signed-off-by: Klement Sekera <ksekera@cisco.com> Signed-off-by: Miklos Tirpak <miklos.tirpak@gmail.com> Change-Id: Ibf521c79463472db97e593bfa02b32b4a06dfd2a
2021-12-10ip: reassembly: handle atomic fragments correctlyKlement Sekera1-1/+4
If a fragment arrives with fragment offset = 0 and M = 0, it means that this is actually a complete packet and per RFC 8200, it should be treated independently from other fragments. This patch does that. Fragmentation header is stripped and fragment is forwarded irregardles of other existing reassemblies in case of full reassembly and treated the same way as regular packet in shallow virtual reassembly. Type: improvement Change-Id: If3322d5e3160cd755b8465a642702a9166d46cc2 Signed-off-by: Klement Sekera <ksekera@cisco.com>
2021-11-22vlib: add virtual time supportBenoît Ganne1-0/+4
Type: feature Change-Id: Iabd76558e9c72ed8286cfeeb1fbaa4fde4832a90 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2021-11-02build: remove unnecessary executable bitsRay Kinsella1-0/+0
Run 'find src -executable -type f', remove unnecessary executable bits from the source tree. Type: refactor Signed-off-by: Ray Kinsella <mdr@ashroe.eu> Change-Id: I070e22c8fb2ef8712bc3ea620727ee5ab3d9a9fb
2021-10-14nat: static mappings in flow hashKlement Sekera1-0/+2
Put static mappings in flow hash, drop existing hash tables used for static mappings. Drop refcount variables and use hash table as a single point of truth. Allow creating a static mapping conflicting with dynamic mapping, which will take precedence after dynamic mapping is freed, so that the existing flow can finish transferring data. Type: fix Signed-off-by: Klement Sekera <ksekera@cisco.com> Change-Id: Idfde8efabc09971be38921d4b0ca5ccf4e9fe412
2021-10-12Revert "nat: static mappings in flow hash"Ole Troan1-2/+0
This reverts commit 69b7599e4b061a8996205f0304232ede84cb70d4. Type: fix Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: If531b122ae5a9f91c2fe6eaa0da69922a91f16d3
2021-10-12nat: static mappings in flow hashKlement Sekera1-0/+2
Put static mappings in flow hash, drop existing hash tables used for static mappings. Drop refcount variables and use hash table as a single point of truth. Allow creating a static mapping conflicting with dynamic mapping, which will take precedence after dynamic mapping is freed, so that the existing flow can finish transferring data. Type: fix Signed-off-by: Klement Sekera <ksekera@cisco.com> Change-Id: Ieeba691d83a83887d0a0baccd5f3832f66126096
2021-10-05build: don't hardcode triplet, allow specifying custom lib dirDamjan Marion1-16/+4
Type: fix Change-Id: I33f364fda88914f88f9b976cb83e6d3ff466f0bb Signed-off-by: Damjan Marion <damarion@cisco.com> Signed-off-by: Ray Kinsella <mdr@ashroe.eu>
2021-09-27classify: Large and nested classifer unit testsRay Kinsella1-8/+16
Type: test Large and nested unit tests to test AVX-512 optimized versions of the classify hash and match algorithims. Signed-off-by: Ray Kinsella <mdr@ashroe.eu> Change-Id: Ie423fee5e0fd1cb4bdf3bec8e0230a5f7cfc75fc
2021-06-16api: remove custom dumpFilip Tehlar1-2/+0
Type: improvement Change-Id: I4b9b2be8817be10e46accc19219deb2b544f266b Signed-off-by: Filip Tehlar <ftehlar@cisco.com>
2021-06-14pg: A Tunnel mode variant of a pg interfaceNeale Ranns1-2/+28
Type: feature this allows VPP to simulate linux tun devices. Signed-off-by: Neale Ranns <neale@graphiant.com> Change-Id: I3adf38b49a254804370f78edd5d275d192fd00a6
2021-05-21tests: add role to VCLAppWorkerDave Wallace1-2/+3
- Improve readability of log output with respect to worker role Type: test Signed-off-by: Dave Wallace <dwallacelf@gmail.com> Change-Id: I1a5787421b31483fa1a2558951568417dfc111d9
2021-05-20tests: don't buffer worker stdin/stdoutDave Wallace1-2/+3
- Required to get VCLAppWorker stdin / stdout data into log.txt when a test fails. Type: test Change-Id: I4db467986940f701750aec6d842a4d9aee22b4e6 Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2021-05-04stats: adding symlinks for nodes and interfaces in the stat segmentArthur de Kerhor1-1/+3
A given interface counter (e.g rx) can be accessed via /interfaces/<interface_name>/<counter_name>. Same goes with nodes: /nodes/<node_name>/<counter_name> As interfaces may contain '/' in their names, these are replaced by '_' in symlinks Also added 2 tests for the stat segment Type: feature Signed-off-by: Arthur de Kerhor <arthurdekerhor@gmail.com> Change-Id: I27da252e7b3dc177815616ca46b5c200a456bf0f Signed-off-by: Ole Troan <ot@cisco.com>
2021-04-28tests: add worker count to class descriptionKlement Sekera1-1/+9
While running test: ============================================================================== NAT44ED Test Case ============================================================================== ==> ============================================================================== NAT44ED Test Case [main thread only] ============================================================================== ============================================================================== NAT44ED Test Case [1 worker thread] ============================================================================== ============================================================================== NAT44ED Test Case [4 worker threads] ============================================================================== Type: improvement Signed-off-by: Klement Sekera <ksekera@cisco.com> Change-Id: I61904b411ff1b58700564698079dc4c07cea5c5e
2021-04-16tests: cpus awarenessKlement Sekera1-60/+75
Introduce MAX_CPUS parameters to control maximum number of CPUs used by VPP(s) during testing, with default value 'auto' corresponding to all CPUs available. Calculate test CPU requirements by taking into account the number of workers, so a test requires 1 (main thread) + # of worker CPUs. When running tests, keep track of both running test jobs (controlled by TEST_JOBS parameter) and free CPUs. This then causes two limits in the system - to not exceed number of jobs in parallel but also to not exceed number of CPUs available. Skip tests which require more CPUs than are available in system (or more than MAX_CPUS) and print a warning message. Type: improvement Change-Id: Ib8fda54e4c6a36179d64160bb87fbd3a0011762d Signed-off-by: Klement Sekera <ksekera@cisco.com>
2021-04-12tests: support attaching to existing vppKlement Sekera1-29/+70
Introduce a new option DEBUG=attach to run a test against existing already running vpp. A new target 'make test-start-gdb' will spawn VPP in gdb for this purpose. Customization options explained in test-help. Type: improvement Change-Id: Ia160a85b33da3b2df292d44bb95729af9dd9da96 Signed-off-by: Klement Sekera <ksekera@cisco.com>
2021-04-09tests: improve namingKlement Sekera1-12/+12
Change capture to pcap to remove confusion in framework.py Type: improvement Signed-off-by: Klement Sekera <ksekera@cisco.com> Change-Id: Idf8bd61f174b8b2df3bbd13908e54d597da4a83b
2021-04-01tests: support injecting multiple worker pcaps on one PGKlement Sekera1-4/+11
This change allows one to inject multiple streams for different workers on the same PG interface at the same time. Type: improvement Change-Id: I29d80369aabada261eda466e5a5d8d3518bb8bc8 Signed-off-by: Klement Sekera <ksekera@cisco.com>
2021-03-25stats: python vpp_stats rewrite to access stat segment directlyOle Troan1-1/+1
This module implement Python access to the VPP statistics segment. It accesses the data structures directly in shared memory. VPP uses optimistic locking, so data structures may change underneath us while we are reading. Data is copied out and it's important to spend as little time as possible "holding the lock". Counters are stored in VPP as a two dimensional array. Index by thread and index (typically sw_if_index). Simple counters count only packets, Combined counters count packets and octets. Counters can be accessed in either dimension. stat['/if/rx'] - returns 2D lists stat['/if/rx'][0] - returns counters for all interfaces for thread 0 stat['/if/rx'][0][1] - returns counter for interface 1 on thread 0 stat['/if/rx'][0][1]['packets'] - returns the packet counter for interface 1 on thread 0 stat['/if/rx'][:, 1] - returns the counters for interface 1 on all threads stat['/if/rx'][:, 1].packets() - returns the packet counters for interface 1 on all threads stat['/if/rx'][:, 1].sum_packets() - returns the sum of packet counters for interface 1 on all threads stat['/if/rx-miss'][:, 1].sum() - returns the sum of packet counters for interface 1 on all threads for simple counters Type: refactor Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I1fe7f7c7d11378d06be8276db5e1900ecdb8f515 Signed-off-by: Ole Troan <ot@cisco.com>
2021-03-20tests: add support for worker awarenessKlement Sekera1-25/+33
VppTestCase now has vpp_worker_count property set to number of workers. This can be overriden by child classes. Also overriden by VPP_WORKER_CONFIG variable for legacy reasons. Type: improvement Change-Id: Ic328bacb9003ddf9e92815767653bd362aa7f086 Signed-off-by: Klement Sekera <ksekera@cisco.com>
2021-03-15tests: use socket transport instead of shared memoryOle Troan1-12/+16
Type: improvement Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I9e65c94a5a05047a5104e9361ea36eac77b40442 Signed-off-by: Ole Troan <ot@cisco.com>
2021-03-15tests: fix NoneType printing if VPP died earlyKlement Sekera1-12/+15
Make error message more meaningful. Type: fix Change-Id: I3c49cb179c3ee7a59657b3ae9a06311f57dc52ac Signed-off-by: Klement Sekera <ksekera@cisco.com>
2021-03-02classify: fix multiple filters supportBenoît Ganne1-7/+9
This fix the classify filter if we attach several different filters. This also fix some issues with l3 and l4 parsing. Type: fix Change-Id: I9dc6c55049a3bbc0110d1097b40d9da27633626b Signed-off-by: Benoît Ganne <bganne@cisco.com>
2021-02-15tests: explicitly close subprocess filesPaul Vinciguerra1-0/+2
Resolve the following error: /vpp/test/framework.py:657: ResourceWarning: unclosed file <_io.BufferedReader name=6> del cls.vpp Object allocated at (most recent call last): File "/usr/lib/python3.8/subprocess.py", lineno 844 self.stdout = io.open(c2pread, 'rb', bufsize) Type: test Change-Id: Ia2974da594f0582dcff1f4bdf40d25475769c46c Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2021-02-08tests: allow for externally supplied VPP workers config for testsAndrew Yourtchenko1-1/+17
Allow to supply the external VPP worker config for tests which do not specify the workers config explicitly, and use the tags infra to flag those that need attention in this configuration. This commit shows one example use of such a tag, there will be a separate commit with the rest of the places needing it, since that change is rather mechanical. Thus, the assumption is that the test should by default be agnostic of the VPP configuration, unless it explicitly specifies so. Type: test Change-Id: I3c0077e4e22a75cb9561fb98d3b783b93486b2be Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>
2021-01-22tests: add generalized tags for tests, use them for run-solo testsAndrew Yourtchenko1-3/+29
We have accumulated several scenarios in prod or wishlists where it would be useful to have a general infra to say yes/no about a certain test, and potentially make decisions based on that, for example: - runs solo (aka 'time-dependent') - (wishlist) part of quick smoke-test set - (wishlist) intermittent failure unrelated to timing - (wishlist) test broken with a multi-worker config in vpp Refactor the current "run-solo" code to allow for this extension. Type: test Change-Id: Ia5b3810e57c0543753c8e0dc4dc0cfb4a30b36ac Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com> Signed-off-by: Klement Sekera <ksekera@cisco.com>