tests: "force solo" testcase support

Some of the tests are time-sensitive, and at present require a non-trivial
modification in order to run at high concurrency.

Without these modifications, they intermittently fail, and require
the test retries.

Rather than setting them to the extended tests and forgetting
about them, put them into a "solo" set, which gets run in a
single-threaded mode after the rest of the tests are done.

Mark a few of the tests that showed errors during TEST_JOBS=48
as forced-solo.

Also, give a better diagnostic if the testcase misses a docstring
needed to represent it in the diagnostic outputs.

Type: fix

Change-Id: I33fe62eb17edc1885bd2c3523892051d52da6546
Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>

4 files changed, 16 insertions, 0 deletions

diff --git a/src/plugins/dhcp/test/test_dhcp.py b/src/plugins/dhcp/test/test_dhcp.py
index 065683fbcda..266932c3830 100644
--- a/src/plugins/dhcp/test/test_dhcp.py
+++ b/src/plugins/dhcp/test/test_dhcp.py
@@ -36,6 +36,10 @@ class TestDHCP(VppTestCase):
     """ DHCP Test Case """
 
     @classmethod
+    def force_solo(cls):
+        return True
+
+    @classmethod
     def setUpClass(cls):
         super(TestDHCP, cls).setUpClass()
 
diff --git a/src/plugins/dhcp/test/test_dhcp6.py b/src/plugins/dhcp/test/test_dhcp6.py
index 7254496f48f..ebd43729e4b 100644
--- a/src/plugins/dhcp/test/test_dhcp6.py
+++ b/src/plugins/dhcp/test/test_dhcp6.py
@@ -224,6 +224,10 @@ class TestDHCPv6IANAControlPlane(VppTestCase):
     """ DHCPv6 IA NA Control Plane Test Case """
 
     @classmethod
+    def force_solo(cls):
+        return True
+
+    @classmethod
     def setUpClass(cls):
         super(TestDHCPv6IANAControlPlane, cls).setUpClass()
 
diff --git a/src/plugins/flowprobe/test/test_flowprobe.py b/src/plugins/flowprobe/test/test_flowprobe.py
index 092e8d3951d..4cf019ad47e 100644
--- a/src/plugins/flowprobe/test/test_flowprobe.py
+++ b/src/plugins/flowprobe/test/test_flowprobe.py
@@ -347,6 +347,10 @@ class Flowprobe(MethodHolder):
     """Template verification, timer tests"""
 
     @classmethod
+    def force_solo(cls):
+        return True
+
+    @classmethod
     def setUpClass(cls):
         super(Flowprobe, cls).setUpClass()
 
diff --git a/src/plugins/memif/test/test_memif.py b/src/plugins/memif/test/test_memif.py
index f11dd89f975..244818cfc3a 100644
--- a/src/plugins/memif/test/test_memif.py
+++ b/src/plugins/memif/test/test_memif.py
@@ -17,6 +17,10 @@ class TestMemif(VppTestCase):
     """ Memif Test Case """
 
     @classmethod
+    def force_solo(cls):
+        return True
+
+    @classmethod
     def setUpClass(cls):
         # fork new process before client connects to VPP
         cls.remote_test = RemoteClass(RemoteVppTestCase)