aboutsummaryrefslogtreecommitdiffstats
path: root/test/config.py
AgeCommit message (Collapse)AuthorFilesLines
2024-07-15tests: more options for decoding pcapsKlement Sekera1-2/+3
Introduce "none", "all" and "failed" options for --decode-pcaps parameter. Keep "failed" as default to be consistent with current behaviour. Add missing documentation to test/Makefile and passthrough to Makefile. Rationale: running tshark binary takes about 100-150ms and if there are thousands of pcap files, it takes minutes to decode them. This might not be desirable if rerunning the tests repeatedly during development. Type: improvement Change-Id: Ie033521d51d18b9d499b9bc40fe6eff21c94622d Signed-off-by: Klement Sekera <klement.sekera@gmail.com>
2024-01-18tests: preload api filesMaxime Peim1-0/+1
When sanity test is not done, API files are not loaded until the first test case is run. Hence, it is not possible to use enums, etc. outside of a test class. By preloading API files before running any tests, it prevents its issue. Type: fix Change-Id: I8730150374e6c5f8d6933ec037811372ac2a8da0 Signed-off-by: Maxime Peim <mpeim@cisco.com>
2023-11-03tests: allow explicit defaults for arg typesDmitry Valter1-2/+2
Allow settings default values explicitly in positive_int_or_default and positive_float_or_default. It allows setting setting default 0 test retries explicitly despite it being not positive. Type: improvement Signed-off-by: Dmitry Valter <d-valter@yandex-team.com> Change-Id: Id23a9fdae0ef174eea8992c1f9fc2530aade6194
2023-11-03tests: refactor asf framework codeDave Wallace1-0/+8
- Make framework.py classes a subset of asfframework.py classes - Remove all packet related code from asfframework.py - Add test class and test case set up debug output to log - Repatriate packet tests from asf to test directory - Remove non-packet related code from framework.py and inherit them from asfframework.py classes - Clean up unused import variables - Re-enable BFD tests on Ubuntu 22.04 and fix intermittent test failures in echo_looped_back testcases (where # control packets verified but not guaranteed to be received during test) - Re-enable Wireguard tests on Ubuntu 22.04 and fix intermittent test failures in handshake ratelimiting testcases and event testcase - Run Wiregard testcase suites solo - Improve debug output in log.txt - Increase VCL/LDP post sleep timeout to allow iperf server to finish cleanly. - Fix pcap history files to be sorted by suite and testcase and ensure order/timestamp is correct based on creation in the testcase. - Decode pcap files for each suite and testcase for all errors or if configured via comandline option / env var - Improve vpp corefile detection to allow complete corefile generation - Disable vm vpp interfaces testcases on debian11 - Clean up failed unittest dir when retrying failed testcases and unify testname directory and failed linknames into framwork functions Type: test Change-Id: I0764f79ea5bb639d278bf635ed2408d4d5220e1e Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2023-10-03build: add ability to disable some plugins from packaging and testsAndrew Yourtchenko1-0/+9
When custom-packaging the VPP artifacts, it can be useful to exclude some of the core plugins from packaging/testing, for some reasons. A removal of a plugin(s) from the worktree needs to be tracked as a separate change, and thus is tricky from the maintenance point of view. This change adds the ability to "pretend they do not exist" - plugins which are added to the comma-separated environment variable "VPP_EXCLUDED_PLUGINS" will not be added to the build process and not packaged. The tests do not have the 1:1 relationship as plugins, so they might need to be modified separately. This change includes some of these modifications as an example. Type: feature Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com> Change-Id: Id31562d00a01ced1acbb4996a633517cbd6f09d8
2023-06-21tests: do not run qemu interface tests if the environment does not allow itAndrew Yourtchenko1-0/+10
cdf73b973181ff4c67147900408216e37bae897a has added the qemu tests as part of the default test run, which results in "make test" failure in more restricted environments which do not allow the namespace creation. Add a config flag to skip those tests, and skip them if the namespace creation fails. Type: test Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com> Change-Id: Ie631f7fb2a80864f77c79619eba4a43712e950e5
2023-04-25tests: support multiple filter expressionsKlement Sekera1-1/+4
Support multiple comma-delimited filter expressions, e.g. to run both bfd and ip4 tests, it's now possible to do: make test TEST=bfd,ip4 Same goes for wildcards, e.g.: make test TEST=bfd,..test_longest_prefix_match,..test_icmp_error Type: improvement Change-Id: I0cceaa443cb612dca955f301c7407959f9a71a6e Signed-off-by: Klement Sekera <klement.sekera@gmail.com>
2022-12-07tests: multiple apidir locationsOle Troan1-0/+8
To support testing of external plugins, add support to the test framework and PAPI for specifying a list of locations to look for api.json files. Type: improvement Signed-off-by: Ole Troan <ot@cisco.com> Change-Id: I128a306e3c091dc8ef994801b1470b82d2f4595d Signed-off-by: Ole Troan <ot@cisco.com>
2022-09-20tests: run tests against a running VPPNaveen Joy1-0/+23
Usage: test/run.py -r -t {test_filter} Instead of starting a new instance of VPP, when the -r argument is provided, test is run against a running VPP instance. Optionally, one can also set the VPP socket directory using the -d argument. The default location for socket files is /var/run/user/${uid}/vpp and /var/run/vpp if VPP is started as root. Type: improvement Change-Id: I05e57a067fcb90fb49973f8159fc17925b741f1a Signed-off-by: Naveen Joy <najoy@cisco.com>
2022-08-16tests: move "venv" to "build-root" directory from "test" directorySaima Yunus1-1/+1
Type: refactor - refactored VPP test code to remove "ignore_path" variable from "discover_tests" function and "run_test" code - configured VPP test makefile, config file, and 'run.sh' shell script to move "venv" directory from "test" dir to "build-root" dir Signed-off-by: Saima Yunus <yunus.saima.234@gmail.com> Change-Id: Id2beecbb99f24ce13ed118a1869c5adbef247e50
2022-05-24tests: fix default failed dir settingKlement Sekera1-4/+5
When running tests via run.sh, default setting of None would cause failed directory symlink to appear in vpp workspace with an ugly name. This patch places the symlink in temporary directory. Type: fix Fixes: b23ffd7ef216463c35b75c831e6a27e58971f4ec Signed-off-by: Klement Sekera <klement.sekera@gmail.com> Change-Id: Ic1715eba7ac1f82f71855e2aeb9b659d27bbb3af
2022-05-10tests: replace pycodestyle with blackKlement Sekera1-125/+238
Drop pycodestyle for code style checking in favor of black. Black is much faster, stable PEP8 compliant code style checker offering also automatic formatting. It aims to be very stable and produce smallest diffs. It's used by many small and big projects. Running checkstyle with black takes a few seconds with a terse output. Thus, test-checkstyle-diff is no longer necessary. Expand scope of checkstyle to all python files in the repo, replacing test-checkstyle with checkstyle-python. Also, fixstyle-python is now available for automatic style formatting. Note: python virtualenv has been consolidated in test/Makefile, test/requirements*.txt which will eventually be moved to a central location. This is required to simply the automated generation of docker executor images in the CI. Type: improvement Change-Id: I022a326603485f58585e879ac0f697fceefbc9c8 Signed-off-by: Klement Sekera <klement.sekera@gmail.com> Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2022-03-18tests: fix the RND_SEED parsingAndrew Yourtchenko1-1/+13
The random seed is not an integer, so the current code does not allow reproducing a test run by running e.g. RND_SEED=1647595144.0940742 make test Solution: make the random seed a positive float. Also, add the missing positiveness check to the positive_integer function. Type: fix Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com> Change-Id: I858bab0a9b828b99c20a2252aeecb9e2dda4ee21
2022-02-17tests: make tests less make dependentKlement Sekera1-0/+290
Implement command line argument parsing instead of passing arguments via environment variables. Add script for running tests without having to invoke make. Deprecate running tests via make. Type: improvement Change-Id: I2e3054a61a2ae25d460e9be00be7d7705fbf943e Signed-off-by: Klement Sekera <ksekera@cisco.com> Signed-off-by: Dave Wallace <dwallacelf@gmail.com>