CSIT-1395: Create comparison tables for NICs

Change-Id: Ib8583041a0d16a6242160b4f51eae973ef33504f
Signed-off-by: Tibor Frank <tifrank@cisco.com>

3 files changed, 142 insertions, 3 deletions

diff --git a/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst b/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst
index 6545b5ec1f..f15f6932aa 100644
--- a/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst
+++ b/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst
@@ -22,9 +22,12 @@ results repeatability vary between the test cases.
 .. note::
 
     Test results have been generated by
-    `FD.io test executor vpp performance job 3n-hsw`_,
-    with RF result
-    files csit-vpp-perf-|srelease|-\*.zip
+
+    - `FD.io test executor vpp performance job 3n-hsw`_,
+    - `FD.io test executor vpp performance job 3n-skx`_,
+    - `FD.io test executor vpp performance job 2n-skx`_
+
+    with RF result files csit-vpp-perf-|srelease|-\*.zip
     `archived here <../../_static/archive/>`_.
 
 3n-hsw
diff --git a/docs/report/vpp_performance_tests/comparisons/index.rst b/docs/report/vpp_performance_tests/comparisons/index.rst
index d706d33581..8c502c49d7 100644
--- a/docs/report/vpp_performance_tests/comparisons/index.rst
+++ b/docs/report/vpp_performance_tests/comparisons/index.rst
@@ -13,4 +13,5 @@ Comparisons
     current_vs_previous_release
     3n-skx_vs_3n-hsw_testbeds
     3n-skx_vs_2n-skx_testbeds
+    nic_comparison
 
diff --git a/docs/report/vpp_performance_tests/comparisons/nic_comparison.rst b/docs/report/vpp_performance_tests/comparisons/nic_comparison.rst
new file mode 100644
index 0000000000..6db2756c38
--- /dev/null
+++ b/docs/report/vpp_performance_tests/comparisons/nic_comparison.rst
@@ -0,0 +1,135 @@
+
+.. _vpp_compare_nics_release:
+
+NICs Comparison
+---------------
+
+Relative comparison of VPP packet throughput (NDR, PDR and MRR) between
+NICs (measured for |csit-release) is calculated from results of tests
+running on 3-Node Intel Xeon Haswell testbeds (3n-hsw) in 1-core, 2-core
+and 4-core (MRR only) configurations.
+
+Listed mean and standard deviation values are computed based on a series
+of the same tests executed against respective VPP releases to verify
+test results repeatability, with percentage change calculated for mean
+values. Note that the standard deviation is quite high for a small
+number of packet throughput tests, what indicates poor test results
+repeatability and makes the relative change of mean throughput value not
+fully representative for these tests. The root causes behind poor
+results repeatability vary between the test cases.
+
+.. note::
+
+    Test results have been generated by
+
+    - `FD.io test executor vpp performance job 3n-hsw`_,
+    - `FD.io test executor vpp performance job 3n-skx`_,
+    - `FD.io test executor vpp performance job 2n-skx`_
+
+    with RF result files csit-vpp-perf-|srelease|-\*.zip
+    `archived here <../../_static/archive/>`_.
+
+3n-hsw
+~~~~~~
+
+NDR Comparison
+``````````````
+
+Comparison tables in ASCII and CSV formats:
+
+  - `ASCII 1t1c NDR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-nics-ndr.txt>`_
+  - `ASCII 2t2c NDR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-nics-ndr.txt>`_
+  - `CSV 1t1c NDR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-nics-ndr.csv>`_
+  - `CSV 2t2c NDR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-nics-ndr.csv>`_
+
+PDR Comparison
+``````````````
+
+Comparison tables in ASCII and CSV formats:
+
+  - `ASCII 1t1c PDR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-nics-pdr.txt>`_
+  - `ASCII 2t2c PDR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-nics-pdr.txt>`_
+  - `CSV 1t1c PDR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-nics-pdr.csv>`_
+  - `CSV 2t2c PDR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-nics-pdr.csv>`_
+
+MRR Comparison
+``````````````
+
+Comparison tables in ASCII and CSV formats:
+
+  - `ASCII 1t1c MRR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-nics-mrr.txt>`_
+  - `ASCII 2t2c MRR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-nics-mrr.txt>`_
+  - `ASCII 4t4c MRR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-4t4c-nics-mrr.txt>`_
+  - `CSV 1t1c MRR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-nics-mrr.csv>`_
+  - `CSV 2t2c MRR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-nics-mrr.csv>`_
+  - `CSV 4t4c MRR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-4t4c-nics-mrr.csv>`_
+
+3n-skx
+~~~~~~
+
+NDR Comparison
+``````````````
+
+Comparison tables in ASCII and CSV formats:
+
+  - `ASCII 2t1c NDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-nics-ndr.txt>`_
+  - `ASCII 4t2c NDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-nics-ndr.txt>`_
+  - `CSV 2t1c NDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-nics-ndr.csv>`_
+  - `CSV 4t2c NDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-nics-ndr.csv>`_
+
+PDR Comparison
+``````````````
+
+Comparison tables in ASCII and CSV formats:
+
+  - `ASCII 2t1c PDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-nics-pdr.txt>`_
+  - `ASCII 4t2c PDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-nics-pdr.txt>`_
+  - `CSV 2t1c PDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-nics-pdr.csv>`_
+  - `CSV 4t2c PDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-nics-pdr.csv>`_
+
+MRR Comparison
+``````````````
+
+Comparison tables in ASCII and CSV formats:
+
+  - `ASCII 2t1c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-nics-mrr.txt>`_
+  - `ASCII 4t2c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-nics-mrr.txt>`_
+  - `ASCII 8t4c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-8t4c-nics-mrr.txt>`_
+  - `CSV 2t1c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-nics-mrr.csv>`_
+  - `CSV 4t2c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-nics-mrr.csv>`_
+  - `CSV 8t4c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-8t4c-nics-mrr.csv>`_
+
+2n-skx
+~~~~~~
+
+NDR Comparison
+``````````````
+
+Comparison tables in ASCII and CSV formats:
+
+  - `ASCII 2t1c NDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-nics-ndr.txt>`_
+  - `ASCII 4t2c NDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-nics-ndr.txt>`_
+  - `CSV 2t1c NDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-nics-ndr.csv>`_
+  - `CSV 4t2c NDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-nics-ndr.csv>`_
+
+PDR Comparison
+``````````````
+
+Comparison tables in ASCII and CSV formats:
+
+  - `ASCII 2t1c PDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-nics-pdr.txt>`_
+  - `ASCII 4t2c PDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-nics-pdr.txt>`_
+  - `CSV 2t1c PDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-nics-pdr.csv>`_
+  - `CSV 4t2c PDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-nics-pdr.csv>`_
+
+MRR Comparison
+``````````````
+
+Comparison tables in ASCII and CSV formats:
+
+  - `ASCII 2t1c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-nics-mrr.txt>`_
+  - `ASCII 4t2c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-nics-mrr.txt>`_
+  - `ASCII 8t4c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-8t4c-nics-mrr.txt>`_
+  - `CSV 2t1c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-nics-mrr.csv>`_
+  - `CSV 4t2c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-nics-mrr.csv>`_
+  - `CSV 8t4c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-8t4c-nics-mrr.csv>`_