Report: Configure report

+ 2101 --> 2106
+ remove 3n-hsw
+ unify data sets
+ configure data sets
- implement changes from job-specs

Change-Id: I200e7a80ebf88864ce4a7a2134f5251b557f6a3a
Signed-off-by: Tibor Frank <tifrank@cisco.com>

5 files changed, 2 insertions, 158 deletions

diff --git a/docs/report/vpp_performance_tests/comparisons/2n-clx_vs_3n-hsw_testbeds.rst b/docs/report/vpp_performance_tests/comparisons/2n-clx_vs_3n-hsw_testbeds.rst
deleted file mode 100644
index 3e641b6762..0000000000
--- a/docs/report/vpp_performance_tests/comparisons/2n-clx_vs_3n-hsw_testbeds.rst
+++ /dev/null
@@ -1,55 +0,0 @@
-2n-Clx vs 3n-Hsw Testbeds
--------------------------
-
-Relative comparison of |vpp-release| packet throughput (NDR, PDR and
-MRR) is calculated for the same tests executed on 3-Node Skylake (3n-
-skx) and 3-Node Haswell (3n-hsw) physical testbed types, in 1-core,
-2-core and 4-core configurations.
-
-.. note::
-
-    Test results have been generated by
-    `FD.io test executor vpp performance job 3n-hsw`_ and
-    `FD.io test executor vpp performance job 2n-clx`_
-    with RF result
-    files csit-vpp-perf-|srelease|-\*.zip
-    `archived here <../../_static/archive/>`_.
-
-NDR Comparison
-~~~~~~~~~~~~~~
-
-Comparison tables in HTML, ASCII and CSV formats:
-
-  - `HTML 1c NDR comparison <performance-compare-testbeds-3n-hsw-2n-clx-1c-ndr.html>`_
-  - `HTML 2c NDR comparison <performance-compare-testbeds-3n-hsw-2n-clx-2c-ndr.html>`_
-  - `ASCII 1c NDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-1c-ndr.txt>`_
-  - `ASCII 2c NDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-2c-ndr.txt>`_
-  - `CSV 1c NDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-1c-ndr-csv.csv>`_
-  - `CSV 2c NDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-2c-ndr-csv.csv>`_
-
-PDR Comparison
-~~~~~~~~~~~~~~
-
-Comparison tables in HTML, ASCII and CSV formats:
-
-  - `HTML 1c PDR comparison <performance-compare-testbeds-3n-hsw-2n-clx-1c-pdr.html>`_
-  - `HTML 2c PDR comparison <performance-compare-testbeds-3n-hsw-2n-clx-2c-pdr.html>`_
-  - `ASCII 1c PDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-1c-pdr.txt>`_
-  - `ASCII 2c PDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-2c-pdr.txt>`_
-  - `CSV 1c PDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-1c-pdr-csv.csv>`_
-  - `CSV 2c PDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-2c-pdr-csv.csv>`_
-
-MRR Comparison
-~~~~~~~~~~~~~~
-
-Comparison tables in HTML, ASCII and CSV formats:
-
-  - `HTML 1c MRR comparison <performance-compare-testbeds-3n-hsw-2n-clx-1c-mrr.html>`_
-  - `HTML 2c MRR comparison <performance-compare-testbeds-3n-hsw-2n-clx-2c-mrr.html>`_
-  - `HTML 4c MRR comparison <performance-compare-testbeds-3n-hsw-2n-clx-4c-mrr.html>`_
-  - `ASCII 1c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-1c-mrr.txt>`_
-  - `ASCII 2c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-2c-mrr.txt>`_
-  - `ASCII 4c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-4c-mrr.txt>`_
-  - `CSV 1c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-1c-mrr-csv.csv>`_
-  - `CSV 2c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-2c-mrr-csv.csv>`_
-  - `CSV 4c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-4c-mrr-csv.csv>`_
diff --git a/docs/report/vpp_performance_tests/comparisons/3n-skx_vs_3n-hsw_testbeds.rst b/docs/report/vpp_performance_tests/comparisons/3n-skx_vs_3n-hsw_testbeds.rst
deleted file mode 100644
index 2eb552e882..0000000000
--- a/docs/report/vpp_performance_tests/comparisons/3n-skx_vs_3n-hsw_testbeds.rst
+++ /dev/null
@@ -1,55 +0,0 @@
-3n-Skx vs 3n-Hsw Testbeds
--------------------------
-
-Relative comparison of |vpp-release| packet throughput (NDR, PDR and
-MRR) is calculated for the same tests executed on 3-Node Skylake (3n-
-skx) and 3-Node Haswell (3n-hsw) physical testbed types, in 1-core,
-2-core and 4-core configurations.
-
-.. note::
-
-    Test results have been generated by
-    `FD.io test executor vpp performance job 3n-hsw`_ and
-    `FD.io test executor vpp performance job 3n-skx`_
-    with RF result
-    files csit-vpp-perf-|srelease|-\*.zip
-    `archived here <../../_static/archive/>`_.
-
-NDR Comparison
-~~~~~~~~~~~~~~
-
-Comparison tables in HTML, ASCII and CSV formats:
-
-  - `HTML 1c NDR comparison <performance-compare-testbeds-3n-hsw-3n-skx-1c-ndr.html>`_
-  - `HTML 2c NDR comparison <performance-compare-testbeds-3n-hsw-3n-skx-2c-ndr.html>`_
-  - `ASCII 1c NDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-1c-ndr.txt>`_
-  - `ASCII 2c NDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-2c-ndr.txt>`_
-  - `CSV 1c NDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-1c-ndr-csv.csv>`_
-  - `CSV 2c NDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-2c-ndr-csv.csv>`_
-
-PDR Comparison
-~~~~~~~~~~~~~~
-
-Comparison tables in HTML, ASCII and CSV formats:
-
-  - `HTML 1c PDR comparison <performance-compare-testbeds-3n-hsw-3n-skx-1c-pdr.html>`_
-  - `HTML 2c PDR comparison <performance-compare-testbeds-3n-hsw-3n-skx-2c-pdr.html>`_
-  - `ASCII 1c PDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-1c-pdr.txt>`_
-  - `ASCII 2c PDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-2c-pdr.txt>`_
-  - `CSV 1c PDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-1c-pdr-csv.csv>`_
-  - `CSV 2c PDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-2c-pdr-csv.csv>`_
-
-MRR Comparison
-~~~~~~~~~~~~~~
-
-Comparison tables in HTML, ASCII and CSV formats:
-
-  - `HTML 1c MRR comparison <performance-compare-testbeds-3n-hsw-3n-skx-1c-mrr.html>`_
-  - `HTML 2c MRR comparison <performance-compare-testbeds-3n-hsw-3n-skx-2c-mrr.html>`_
-  - `HTML 4c MRR comparison <performance-compare-testbeds-3n-hsw-3n-skx-4c-mrr.html>`_
-  - `ASCII 1c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-1c-mrr.txt>`_
-  - `ASCII 2c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-2c-mrr.txt>`_
-  - `ASCII 4c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-4c-mrr.txt>`_
-  - `CSV 1c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-1c-mrr-csv.csv>`_
-  - `CSV 2c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-2c-mrr-csv.csv>`_
-  - `CSV 4c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-4c-mrr-csv.csv>`_
diff --git a/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst b/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst
index 3cfd5ae3f8..e251b68954 100644
--- a/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst
+++ b/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst
@@ -8,7 +8,7 @@ Relative comparison of VPP packet throughput (NDR, PDR and MRR) between
 |vpp-release| and |vpp-release-1| (measured for |csit-release| and
 |csit-release-1| respectively) is calculated from results of tests
 running on 2-node Intel Xeon Skylake (2n-skx), 3-node Intel Xeon Skylake
-(3n-skx), 3-Node Intel Xeon Haswell (3n-hsw), 2-node Intel Atom Denverton
+(3n-skx), 2-node Intel Atom Denverton
 (2n-dnv), 3-node Intel Atom Denverton (3n-dnv), 3-node Arm TaiShan (3n-tsh)
 testbeds, in 1-core, 2-core and 4-core (MRR only) configurations.
 
@@ -29,7 +29,6 @@ results repeatability vary between the test cases.
     - `FD.io test executor vpp performance job 3n-skx`_,
     - `FD.io test executor vpp performance job 2n-clx`_,
     - `FD.io test executor vpp performance job 2n-zn2`_,
-    - `FD.io test executor vpp performance job 3n-hsw`_,
     - `FD.io test executor vpp performance job 2n-dnv`_,
     - `FD.io test executor vpp performance job 3n-dnv`_,
     - `FD.io test executor vpp performance job 3n-tsh`_,
@@ -248,48 +247,6 @@ Comparison tables in HTML, ASCII and CSV formats:
   - `CSV 4t2c MRR comparison <../../_static/vpp/performance-changes-2n-zn2-xxv710-4t2c-mrr-csv.csv>`_
   - `CSV 8t4c MRR comparison <../../_static/vpp/performance-changes-2n-zn2-xxv710-8t4c-mrr-csv.csv>`_
 
-3n-hsw
-~~~~~~
-
-NDR Comparison
-``````````````
-
-Comparison tables in HTML, ASCII and CSV formats:
-
-  - `HTML 1t1c NDR comparison <performance-changes-3n-hsw-1t1c-ndr.html>`_
-  - `HTML 2t2c NDR comparison <performance-changes-3n-hsw-2t2c-ndr.html>`_
-  - `ASCII 1t1c NDR comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-ndr.txt>`_
-  - `ASCII 2t2c NDR comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-ndr.txt>`_
-  - `CSV 1t1c NDR comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-ndr-csv.csv>`_
-  - `CSV 2t2c NDR comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-ndr-csv.csv>`_
-
-PDR Comparison
-``````````````
-
-Comparison tables in HTML, ASCII and CSV formats:
-
-  - `HTML 1t1c PDR comparison <performance-changes-3n-hsw-1t1c-pdr.html>`_
-  - `HTML 2t2c PDR comparison <performance-changes-3n-hsw-2t2c-pdr.html>`_
-  - `ASCII 1t1c PDR comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-pdr.txt>`_
-  - `ASCII 2t2c PDR comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-pdr.txt>`_
-  - `CSV 1t1c PDR comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-pdr-csv.csv>`_
-  - `CSV 2t2c PDR comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-pdr-csv.csv>`_
-
-MRR Comparison
-``````````````
-
-Comparison tables in HTML, ASCII and CSV formats:
-
-  - `HTML 1t1c MRR comparison <performance-changes-3n-hsw-1t1c-mrr.html>`_
-  - `HTML 2t2c MRR comparison <performance-changes-3n-hsw-2t2c-mrr.html>`_
-  - `HTML 4t4c MRR comparison <performance-changes-3n-hsw-4t4c-mrr.html>`_
-  - `ASCII 1t1c MRR comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-mrr.txt>`_
-  - `ASCII 2t2c MRR comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-mrr.txt>`_
-  - `ASCII 4t4c MRR comparison <../../_static/vpp/performance-changes-3n-hsw-4t4c-mrr.txt>`_
-  - `CSV 1t1c MRR comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-mrr-csv.csv>`_
-  - `CSV 2t2c MRR comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-mrr-csv.csv>`_
-  - `CSV 4t4c MRR comparison <../../_static/vpp/performance-changes-3n-hsw-4t4c-mrr-csv.csv>`_
-
 2n-dnv
 ~~~~~~
 
diff --git a/docs/report/vpp_performance_tests/comparisons/index.rst b/docs/report/vpp_performance_tests/comparisons/index.rst
index 1a05947685..551ec9159d 100644
--- a/docs/report/vpp_performance_tests/comparisons/index.rst
+++ b/docs/report/vpp_performance_tests/comparisons/index.rst
@@ -11,9 +11,7 @@ Comparisons
 .. toctree::
 
     current_vs_previous_release
-    3n-skx_vs_3n-hsw_testbeds
     2n-skx_vs_2n-clx_testbeds
-    2n-clx_vs_3n-hsw_testbeds
     3n-skx_vs_2n-skx_testbeds
     nic_comparison
     soak_vs_ndr
diff --git a/docs/report/vpp_performance_tests/comparisons/nic_comparison.rst b/docs/report/vpp_performance_tests/comparisons/nic_comparison.rst
index 51905c9d0e..9d9dd8eec0 100644
--- a/docs/report/vpp_performance_tests/comparisons/nic_comparison.rst
+++ b/docs/report/vpp_performance_tests/comparisons/nic_comparison.rst
@@ -6,8 +6,7 @@ NICs Comparison
 
 Relative comparison of VPP packet throughput (NDR, PDR and MRR) between
 NICs (measured for |csit-release) is calculated from results of tests
-running on 3-Node Intel Xeon Haswell testbeds (3n-hsw) in 1-core, 2-core
-and 4-core (MRR only) configurations.
+running on 3n-skx, 2n-skx testbeds.
 
 Listed mean and standard deviation values are computed based on a series
 of the same tests executed against respective VPP releases to verify