From bec963c7d97302d7a0e41d38a5eec1ff5e518067 Mon Sep 17 00:00:00 2001 From: Tibor Frank Date: Fri, 18 Jan 2019 13:19:01 +0100 Subject: CSIT-1395: Create comparison tables for NICs Change-Id: Ib8583041a0d16a6242160b4f51eae973ef33504f Signed-off-by: Tibor Frank --- .../comparisons/current_vs_previous_release.rst | 9 +- .../vpp_performance_tests/comparisons/index.rst | 1 + .../comparisons/nic_comparison.rst | 135 +++++++++++++++++++++ 3 files changed, 142 insertions(+), 3 deletions(-) create mode 100644 docs/report/vpp_performance_tests/comparisons/nic_comparison.rst (limited to 'docs/report/vpp_performance_tests') diff --git a/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst b/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst index 6545b5ec1f..f15f6932aa 100644 --- a/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst +++ b/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst @@ -22,9 +22,12 @@ results repeatability vary between the test cases. .. note:: Test results have been generated by - `FD.io test executor vpp performance job 3n-hsw`_, - with RF result - files csit-vpp-perf-|srelease|-\*.zip + + - `FD.io test executor vpp performance job 3n-hsw`_, + - `FD.io test executor vpp performance job 3n-skx`_, + - `FD.io test executor vpp performance job 2n-skx`_ + + with RF result files csit-vpp-perf-|srelease|-\*.zip `archived here <../../_static/archive/>`_. 3n-hsw diff --git a/docs/report/vpp_performance_tests/comparisons/index.rst b/docs/report/vpp_performance_tests/comparisons/index.rst index d706d33581..8c502c49d7 100644 --- a/docs/report/vpp_performance_tests/comparisons/index.rst +++ b/docs/report/vpp_performance_tests/comparisons/index.rst @@ -13,4 +13,5 @@ Comparisons current_vs_previous_release 3n-skx_vs_3n-hsw_testbeds 3n-skx_vs_2n-skx_testbeds + nic_comparison diff --git a/docs/report/vpp_performance_tests/comparisons/nic_comparison.rst b/docs/report/vpp_performance_tests/comparisons/nic_comparison.rst new file mode 100644 index 0000000000..6db2756c38 --- /dev/null +++ b/docs/report/vpp_performance_tests/comparisons/nic_comparison.rst @@ -0,0 +1,135 @@ + +.. _vpp_compare_nics_release: + +NICs Comparison +--------------- + +Relative comparison of VPP packet throughput (NDR, PDR and MRR) between +NICs (measured for |csit-release) is calculated from results of tests +running on 3-Node Intel Xeon Haswell testbeds (3n-hsw) in 1-core, 2-core +and 4-core (MRR only) configurations. + +Listed mean and standard deviation values are computed based on a series +of the same tests executed against respective VPP releases to verify +test results repeatability, with percentage change calculated for mean +values. Note that the standard deviation is quite high for a small +number of packet throughput tests, what indicates poor test results +repeatability and makes the relative change of mean throughput value not +fully representative for these tests. The root causes behind poor +results repeatability vary between the test cases. + +.. note:: + + Test results have been generated by + + - `FD.io test executor vpp performance job 3n-hsw`_, + - `FD.io test executor vpp performance job 3n-skx`_, + - `FD.io test executor vpp performance job 2n-skx`_ + + with RF result files csit-vpp-perf-|srelease|-\*.zip + `archived here <../../_static/archive/>`_. + +3n-hsw +~~~~~~ + +NDR Comparison +`````````````` + +Comparison tables in ASCII and CSV formats: + + - `ASCII 1t1c NDR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-nics-ndr.txt>`_ + - `ASCII 2t2c NDR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-nics-ndr.txt>`_ + - `CSV 1t1c NDR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-nics-ndr.csv>`_ + - `CSV 2t2c NDR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-nics-ndr.csv>`_ + +PDR Comparison +`````````````` + +Comparison tables in ASCII and CSV formats: + + - `ASCII 1t1c PDR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-nics-pdr.txt>`_ + - `ASCII 2t2c PDR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-nics-pdr.txt>`_ + - `CSV 1t1c PDR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-nics-pdr.csv>`_ + - `CSV 2t2c PDR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-nics-pdr.csv>`_ + +MRR Comparison +`````````````` + +Comparison tables in ASCII and CSV formats: + + - `ASCII 1t1c MRR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-nics-mrr.txt>`_ + - `ASCII 2t2c MRR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-nics-mrr.txt>`_ + - `ASCII 4t4c MRR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-4t4c-nics-mrr.txt>`_ + - `CSV 1t1c MRR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-nics-mrr.csv>`_ + - `CSV 2t2c MRR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-nics-mrr.csv>`_ + - `CSV 4t4c MRR Intel-x520 and Intel-x710 comparison <../../_static/vpp/performance-changes-3n-hsw-4t4c-nics-mrr.csv>`_ + +3n-skx +~~~~~~ + +NDR Comparison +`````````````` + +Comparison tables in ASCII and CSV formats: + + - `ASCII 2t1c NDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-nics-ndr.txt>`_ + - `ASCII 4t2c NDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-nics-ndr.txt>`_ + - `CSV 2t1c NDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-nics-ndr.csv>`_ + - `CSV 4t2c NDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-nics-ndr.csv>`_ + +PDR Comparison +`````````````` + +Comparison tables in ASCII and CSV formats: + + - `ASCII 2t1c PDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-nics-pdr.txt>`_ + - `ASCII 4t2c PDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-nics-pdr.txt>`_ + - `CSV 2t1c PDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-nics-pdr.csv>`_ + - `CSV 4t2c PDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-nics-pdr.csv>`_ + +MRR Comparison +`````````````` + +Comparison tables in ASCII and CSV formats: + + - `ASCII 2t1c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-nics-mrr.txt>`_ + - `ASCII 4t2c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-nics-mrr.txt>`_ + - `ASCII 8t4c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-8t4c-nics-mrr.txt>`_ + - `CSV 2t1c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-nics-mrr.csv>`_ + - `CSV 4t2c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-nics-mrr.csv>`_ + - `CSV 8t4c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-8t4c-nics-mrr.csv>`_ + +2n-skx +~~~~~~ + +NDR Comparison +`````````````` + +Comparison tables in ASCII and CSV formats: + + - `ASCII 2t1c NDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-nics-ndr.txt>`_ + - `ASCII 4t2c NDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-nics-ndr.txt>`_ + - `CSV 2t1c NDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-nics-ndr.csv>`_ + - `CSV 4t2c NDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-nics-ndr.csv>`_ + +PDR Comparison +`````````````` + +Comparison tables in ASCII and CSV formats: + + - `ASCII 2t1c PDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-nics-pdr.txt>`_ + - `ASCII 4t2c PDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-nics-pdr.txt>`_ + - `CSV 2t1c PDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-nics-pdr.csv>`_ + - `CSV 4t2c PDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-nics-pdr.csv>`_ + +MRR Comparison +`````````````` + +Comparison tables in ASCII and CSV formats: + + - `ASCII 2t1c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-nics-mrr.txt>`_ + - `ASCII 4t2c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-nics-mrr.txt>`_ + - `ASCII 8t4c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-8t4c-nics-mrr.txt>`_ + - `CSV 2t1c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-nics-mrr.csv>`_ + - `CSV 4t2c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-nics-mrr.csv>`_ + - `CSV 8t4c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-8t4c-nics-mrr.csv>`_ -- cgit 1.2.3-korg