aboutsummaryrefslogtreecommitdiffstats
path: root/docs/report/vpp_performance_tests
diff options
context:
space:
mode:
authorTibor Frank <tifrank@cisco.com>2019-01-18 13:19:01 +0100
committerTibor Frank <tifrank@cisco.com>2019-01-21 11:52:03 +0000
commitbec963c7d97302d7a0e41d38a5eec1ff5e518067 (patch)
treee8e345578f2f609104cd9f5fd296cdd4af06fdbe /docs/report/vpp_performance_tests
parentd7a10b0bb4133a32884079a146b6876506b99054 (diff)
CSIT-1395: Create comparison tables for NICs
Change-Id: Ib8583041a0d16a6242160b4f51eae973ef33504f Signed-off-by: Tibor Frank <tifrank@cisco.com>
Diffstat (limited to 'docs/report/vpp_performance_tests')
-rw-r--r--docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst9
-rw-r--r--docs/report/vpp_performance_tests/comparisons/index.rst1
-rw-r--r--docs/report/vpp_performance_tests/comparisons/nic_comparison.rst135
3 files changed, 142 insertions, 3 deletions
diff --git a/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst b/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst
index 6545b5ec1f..f15f6932aa 100644
--- a/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst
+++ b/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst
@@ -22,9 +22,12 @@ results repeatability vary between the test cases.
.. note::
Test results have been generated by
- `FD.io test executor vpp performance job 3n-hsw`_,
- with RF result
- files csit-vpp-perf-|srelease|-\*.zip
+
+ - `FD.io test executor vpp performance job 3n-hsw`_,
+ - `FD.io test executor vpp performance job 3n-skx`_,
+ - `FD.io test executor vpp performance job 2n-skx`_
+
+ with RF result files csit-vpp-perf-|srelease|-\*.zip
`archived here <../../_static/archive/>`_.
3n-hsw
diff --git a/docs/report/vpp_performance_tests/comparisons/index.rst b/docs/report/vpp_performance_tests/comparisons/index.rst
index d706d33581..8c502c49d7 100644
--- a/docs/report/vpp_performance_tests/comparisons/index.rst
+++ b/docs/report/vpp_performance_tests/comparisons/index.rst
@@ -13,4 +13,5 @@ Comparisons
current_vs_previous_release
3n-skx_vs_3n-hsw_testbeds
3n-skx_vs_2n-skx_testbeds
+ nic_comparison
diff --git a/docs/report/vpp_performance_tests/comparisons/nic_comparison.rst b/docs/report/vpp_performance_tests/comparisons/nic_comparison.rst
new file mode 100644
index 0000000000..6db2756c38
--- /dev/null
+++ b/docs/report/vpp_performance_tests/comparisons/nic_comparison.rst
@@ -0,0 +1,135 @@
+
+.. _vpp_compare_nics_release:
+
+NICs Comparison
+---------------
+
+Relative comparison of VPP packet throughput (NDR, PDR and MRR) between
+NICs (measured for |csit-release) is calculated from results of tests
+running on 3-Node Intel Xeon Haswell testbeds (3n-hsw) in 1-core, 2-core
+and 4-core (MRR only) configurations.
+
+Listed mean and standard deviation values are computed based on a series
+of the same tests executed against respective VPP releases to verify
+test results repeatability, with percentage change calculated for mean
+values. Note that the standard deviation is quite high for a small
+number of packet throughput tests, what indicates poor test results
+repeatability and makes the relative change of mean throughput value not
+fully representative for these tests. The root causes behind poor
+results repeatability vary between the test cases.
+
+.. note::
+
+ Test results have been generated by
+
+ - `FD.io test executor vpp performance job 3n-hsw`_,
+ - `FD.io test executor vpp performance job 3n-skx`_,
+ - `FD.io test executor vpp performance job 2n-skx`_
+
+ with RF result files csit-vpp-perf-|srelease|-\*.zip
+ `archived here <../../_static/archive/>`_.
+
+3n-hsw
+~~~~~~
+
+NDR Comparison
+``````````````
+
+Comparison tables in ASCII and CSV formats:
+
+ - `ASCII 1t1c NDR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-nics-ndr.txt>`_
+ - `ASCII 2t2c NDR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-nics-ndr.txt>`_
+ - `CSV 1t1c NDR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-nics-ndr.csv>`_
+ - `CSV 2t2c NDR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-nics-ndr.csv>`_
+
+PDR Comparison
+``````````````
+
+Comparison tables in ASCII and CSV formats:
+
+ - `ASCII 1t1c PDR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-nics-pdr.txt>`_
+ - `ASCII 2t2c PDR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-nics-pdr.txt>`_
+ - `CSV 1t1c PDR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-nics-pdr.csv>`_
+ - `CSV 2t2c PDR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-nics-pdr.csv>`_
+
+MRR Comparison
+``````````````
+
+Comparison tables in ASCII and CSV formats:
+
+ - `ASCII 1t1c MRR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-nics-mrr.txt>`_
+ - `ASCII 2t2c MRR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-nics-mrr.txt>`_
+ - `ASCII 4t4c MRR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-4t4c-nics-mrr.txt>`_
+ - `CSV 1t1c MRR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-nics-mrr.csv>`_
+ - `CSV 2t2c MRR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-nics-mrr.csv>`_
+ - `CSV 4t4c MRR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-4t4c-nics-mrr.csv>`_
+
+3n-skx
+~~~~~~
+
+NDR Comparison
+``````````````
+
+Comparison tables in ASCII and CSV formats:
+
+ - `ASCII 2t1c NDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-nics-ndr.txt>`_
+ - `ASCII 4t2c NDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-nics-ndr.txt>`_
+ - `CSV 2t1c NDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-nics-ndr.csv>`_
+ - `CSV 4t2c NDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-nics-ndr.csv>`_
+
+PDR Comparison
+``````````````
+
+Comparison tables in ASCII and CSV formats:
+
+ - `ASCII 2t1c PDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-nics-pdr.txt>`_
+ - `ASCII 4t2c PDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-nics-pdr.txt>`_
+ - `CSV 2t1c PDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-nics-pdr.csv>`_
+ - `CSV 4t2c PDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-nics-pdr.csv>`_
+
+MRR Comparison
+``````````````
+
+Comparison tables in ASCII and CSV formats:
+
+ - `ASCII 2t1c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-nics-mrr.txt>`_
+ - `ASCII 4t2c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-nics-mrr.txt>`_
+ - `ASCII 8t4c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-8t4c-nics-mrr.txt>`_
+ - `CSV 2t1c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-nics-mrr.csv>`_
+ - `CSV 4t2c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-nics-mrr.csv>`_
+ - `CSV 8t4c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-8t4c-nics-mrr.csv>`_
+
+2n-skx
+~~~~~~
+
+NDR Comparison
+``````````````
+
+Comparison tables in ASCII and CSV formats:
+
+ - `ASCII 2t1c NDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-nics-ndr.txt>`_
+ - `ASCII 4t2c NDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-nics-ndr.txt>`_
+ - `CSV 2t1c NDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-nics-ndr.csv>`_
+ - `CSV 4t2c NDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-nics-ndr.csv>`_
+
+PDR Comparison
+``````````````
+
+Comparison tables in ASCII and CSV formats:
+
+ - `ASCII 2t1c PDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-nics-pdr.txt>`_
+ - `ASCII 4t2c PDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-nics-pdr.txt>`_
+ - `CSV 2t1c PDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-nics-pdr.csv>`_
+ - `CSV 4t2c PDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-nics-pdr.csv>`_
+
+MRR Comparison
+``````````````
+
+Comparison tables in ASCII and CSV formats:
+
+ - `ASCII 2t1c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-nics-mrr.txt>`_
+ - `ASCII 4t2c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-nics-mrr.txt>`_
+ - `ASCII 8t4c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-8t4c-nics-mrr.txt>`_
+ - `CSV 2t1c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-nics-mrr.csv>`_
+ - `CSV 4t2c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-nics-mrr.csv>`_
+ - `CSV 8t4c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-8t4c-nics-mrr.csv>`_