tests: "force solo" testcase support

Some of the tests are time-sensitive, and at present require a non-trivial
modification in order to run at high concurrency.

Without these modifications, they intermittently fail, and require
the test retries.

Rather than setting them to the extended tests and forgetting
about them, put them into a "solo" set, which gets run in a
single-threaded mode after the rest of the tests are done.

Mark a few of the tests that showed errors during TEST_JOBS=48
as forced-solo.

Also, give a better diagnostic if the testcase misses a docstring
needed to represent it in the diagnostic outputs.

Type: fix

Change-Id: I33fe62eb17edc1885bd2c3523892051d52da6546
Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>

1 files changed, 24 insertions, 0 deletions

diff --git a/test/test_bfd.py b/test/test_bfd.py
index 67b62766c21..f66f75a3408 100644
--- a/test/test_bfd.py
+++ b/test/test_bfd.py
@@ -686,6 +686,10 @@ class BFD4TestCase(VppTestCase):
     test_session = None
 
     @classmethod
+    def force_solo(cls):
+        return True
+
+    @classmethod
     def setUpClass(cls):
         super(BFD4TestCase, cls).setUpClass()
         cls.vapi.cli("set log class bfd level debug")
@@ -1494,6 +1498,10 @@ class BFD6TestCase(VppTestCase):
     test_session = None
 
     @classmethod
+    def force_solo(cls):
+        return True
+
+    @classmethod
     def setUpClass(cls):
         super(BFD6TestCase, cls).setUpClass()
         cls.vapi.cli("set log class bfd level debug")
@@ -1705,6 +1713,10 @@ class BFDFIBTestCase(VppTestCase):
     test_session = None
 
     @classmethod
+    def force_solo(cls):
+        return True
+
+    @classmethod
     def setUpClass(cls):
         super(BFDFIBTestCase, cls).setUpClass()
 
@@ -1893,6 +1905,10 @@ class BFDSHA1TestCase(VppTestCase):
     test_session = None
 
     @classmethod
+    def force_solo(cls):
+        return True
+
+    @classmethod
     def setUpClass(cls):
         super(BFDSHA1TestCase, cls).setUpClass()
         cls.vapi.cli("set log class bfd level debug")
@@ -2123,6 +2139,10 @@ class BFDAuthOnOffTestCase(VppTestCase):
     test_session = None
 
     @classmethod
+    def force_solo(cls):
+        return True
+
+    @classmethod
     def setUpClass(cls):
         super(BFDAuthOnOffTestCase, cls).setUpClass()
         cls.vapi.cli("set log class bfd level debug")
@@ -2332,6 +2352,10 @@ class BFDCLITestCase(VppTestCase):
     pg0 = None
 
     @classmethod
+    def force_solo(cls):
+        return True
+
+    @classmethod
     def setUpClass(cls):
         super(BFDCLITestCase, cls).setUpClass()
         cls.vapi.cli("set log class bfd level debug")