tests: "force solo" testcase support

Some of the tests are time-sensitive, and at present require a non-trivial
modification in order to run at high concurrency.

Without these modifications, they intermittently fail, and require
the test retries.

Rather than setting them to the extended tests and forgetting
about them, put them into a "solo" set, which gets run in a
single-threaded mode after the rest of the tests are done.

Mark a few of the tests that showed errors during TEST_JOBS=48
as forced-solo.

Also, give a better diagnostic if the testcase misses a docstring
needed to represent it in the diagnostic outputs.

Type: fix

Change-Id: I33fe62eb17edc1885bd2c3523892051d52da6546
Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>

1 files changed, 16 insertions, 1 deletions

diff --git a/test/framework.py b/test/framework.py
index c9d72a768bb..ba5f4014798 100644
--- a/test/framework.py
+++ b/test/framework.py
@@ -299,6 +299,11 @@ class VppTestCase(unittest.TestCase):
             return 0
 
     @classmethod
+    def force_solo(cls):
+        """ if the test case class is timing-sensitive - return true """
+        return False
+
+    @classmethod
     def instance(cls):
         """Return the instance of this testcase"""
         return cls.test_instance
@@ -1412,9 +1417,19 @@ class VppTestResult(unittest.TestResult):
         """
 
         def print_header(test):
+            test_doc = getdoc(test)
+            if not test_doc:
+                raise Exception("No doc string for test '%s'" % test.id())
+            test_title = test_doc.splitlines()[0]
+            test_title_colored = colorize(test_title, GREEN)
+            if test.force_solo():
+                # long live PEP-8 and 80 char width limitation...
+                c = YELLOW
+                test_title_colored = colorize("SOLO RUN: " + test_title, c)
+
             if not hasattr(test.__class__, '_header_printed'):
                 print(double_line_delim)
-                print(colorize(getdoc(test).splitlines()[0], GREEN))
+                print(test_title_colored)
                 print(double_line_delim)
             test.__class__._header_printed = True