summaryrefslogtreecommitdiffstats
path: root/test/run_tests.py
AgeCommit message (Collapse)AuthorFilesLines
2018-09-06Fix test summary and retriesjuraj.linkes1-22/+35
There was an issue when tests crashed and weren't properly retried. Change-Id: Id5ef828ecc9a8dc0f08c50183721db06e162e6c3 Signed-off-by: juraj.linkes <juraj.linkes@pantheon.tech>
2018-08-30Fix hanging test runner when child process diesjuraj.linkes1-50/+59
When fixing the test summary after a test run doesn't finish properly I introduced a bug where child process which died would leave the whole run hanging. This patch fixed the bug while still having the correct test summary. Change-Id: I206b1a7dab4032d24cbc50667b8dd0bdcebb67a6 Signed-off-by: juraj.linkes <juraj.linkes@pantheon.tech>
2018-08-25Fix logging to file and summary for crashed testsjuraj.linkes1-84/+59
Logging was not being logged to log.txt Crashed tests were not reported properly when running tests in one process Change-Id: Ica2d703d88351cc8a94bad86764ae4376dc79590 Signed-off-by: juraj.linkes <juraj.linkes@pantheon.tech>
2018-08-23CSIT-1139: Implement parallel test executionjuraj.linkes1-158/+631
The implementation of parallel test execution in VPP Test Framework. - VPPTestCase test methods are grouped together - tests are running in separate processes - VPP instances spawned by tests are assigned to different cores - output from these processes is redirected through pipes and printed out testcase by testcase - TEST_JOBS env var is used to specify the number of parallel processes - improved test summary - a bit of code cleanup Change-Id: I9ca93904d9fe2c3daf980500c64a8611838ae28c Signed-off-by: juraj.linkes <juraj.linkes@pantheon.tech>
2018-06-27make test: easy profiling via make test PROFILE=1Klement Sekera1-1/+7
Change-Id: Ib845578485f523b7f14e98c83d05f78db382ecde Signed-off-by: Klement Sekera <ksekera@cisco.com>
2018-06-25make test: fix RETRIES when setUpClass throwsKlement Sekera1-1/+4
This change adds handling of special case when setUpClass throws. In this case TestResults receives a mock object called _ErrorHolder. By parsing its description, we find test class name and use it to lookup the test class in test suite to be able to add it to the list of failures for re-running. Change-Id: I656f21e38aa450fc567cdcbcf6e586967f947a64 Signed-off-by: Klement Sekera <ksekera@cisco.com>
2018-03-31make test: print a warning in case a core_pattern contains a filter programAndrew Yourtchenko1-0/+2
The default config on Ubuntu 16.04.4 desktop results in truncated cores when running make test which coredumps. Uninstalling the filter program (apport) makes the corefiles normal size. Print a warning about that fact, so the others potentially affected didn't have to wonder. Change-Id: Iba4b0a2765a25100d6e24fd7f4de0e0339efd835 Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>
2018-03-24make test: enhance core-file informationKlement Sekera1-15/+20
Change-Id: I1283960a9a49f6d70b9d7b7793cfb345c22ccdea Signed-off-by: Klement Sekera <ksekera@cisco.com>
2018-03-23make test: code cleanupKlement Sekera1-5/+8
Change-Id: Ic689de569e5b6e6209d16d6acdb13c489daca1f5 Signed-off-by: Klement Sekera <ksekera@cisco.com>
2018-03-14make test: early core detection, code cleanupKlement Sekera1-2/+24
Change-Id: I503d69f902f1c8d3107fb3f2b9f31b0083ebb747 Signed-off-by: Klement Sekera <ksekera@cisco.com>
2018-03-14make test: handle multiple failures in one caseKlement Sekera1-1/+4
Change-Id: I5451ddec4bfdede12653415f90bf991e81494033 Signed-off-by: Klement Sekera <ksekera@cisco.com>
2018-02-17make test: detect child crashKlement Sekera1-7/+17
This change causes parent process to detect child crash in seconds instead of waiting for timeout to pass. Change-Id: Ib45e86b7fc97e687d99c554be4212aaeea781dcc Signed-off-by: Klement Sekera <ksekera@cisco.com>
2017-10-10make test: add RETRIES optionKlement Sekera1-8/+53
Change-Id: Ibe31e932bc997f0101a8947e01df90a90d1f100f Signed-off-by: Klement Sekera <ksekera@cisco.com>
2017-10-03make test: Copy api_post_mortem.$$ file tmp test dir for archiving.Dave Wallace1-2/+8
Change-Id: I4baf89ef383dbc2f309081a6b56b13ebcb8fc2df Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2017-09-30make test: Create link to failed test dir on timeout.Dave Wallace1-0/+6
- Also change default coredump configuration from "coredump-size unlimited" to "full-coredump" Change-Id: Iefedc2636f2d9696b7575b34e91dd7be49f601fa Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2017-08-25make test: separate test discovery codeKlement Sekera1-34/+11
Separating test discovery code to it's own script file has the advantage of easily doing e.g. listing of all existing tests. Change-Id: I80dc280263cc7e33e7e13cb0d48b39bf08ece24d Signed-off-by: Klement Sekera <ksekera@cisco.com>
2017-08-12make test: fix broken DEBUG=gdb* optionsKlement Sekera1-42/+50
Change-Id: I5d80982eeab78a629760f567eda3b1539d96e3a8 Signed-off-by: Klement Sekera <ksekera@cisco.com>
2017-08-10make test: bump default test timeout to 10 minutesKlement Sekera1-1/+1
Change-Id: I25d88966376d712ff61f29227a45880a59e8ecf2 Signed-off-by: Klement Sekera <ksekera@cisco.com>
2017-08-10make test: detect hung testsKlement Sekera1-3/+69
Run tests in a forked process with a set of pipes to communicate keep-alives and overall result. This allows us to detect when e.g. vpp dies mid-API call causing the test to hang waiting for response (which will never come since vpp died). Support setting a (per test case) TIMEOUT make test option to set timeout, with a default timeout of 120 seconds. Example - fail the test suite if any test-case fails to finish within 300s: make TIMEOUT=300 test Change-Id: I0d04f26a7232968f4bf043adf5d5b508f5018717 Signed-off-by: Klement Sekera <ksekera@cisco.com>
2017-03-27make test: properly propagate exit statusKlement Sekera1-1/+2
Change-Id: Ie9f48a0d5e0a9cd08eb8f07d49149eee40f04131 Signed-off-by: Klement Sekera <ksekera@cisco.com>
2017-03-22make test: support out-of-tree testsKlement Sekera1-1/+50
env EXTERN_TESTS="/path/to/extra/tests" make test causes to run the default test set and tests collected from test_*.py files under subtree specified in EXTERN_TESTS. Change-Id: I58c5471dd6010730278a5b47d4318737d920bc28 Signed-off-by: Klement Sekera <ksekera@cisco.com>
2016-10-03test: new test infrastructureDamjan Marion1-0/+12
Change-Id: I73ca19c431743f6b39669c583d9222a6559346ef Signed-off-by: Jan Gelety <jgelety@cisco.com> Signed-off-by: Juraj Sloboda <jsloboda@cisco.com> Signed-off-by: Stefan Kobza <skobza@cisco.com> Signed-off-by: Matej Klotton <mklotton@cisco.com> Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com> Signed-off-by: Damjan Marion <damarion@cisco.com>