From c10980893443b59c464deb6cd0d66d5be972593f Mon Sep 17 00:00:00 2001 From: Tibor Frank Date: Mon, 31 May 2021 10:35:20 +0200 Subject: Report: Configure report + 2101 --> 2106 + remove 3n-hsw + unify data sets + configure data sets - implement changes from job-specs Change-Id: I200e7a80ebf88864ce4a7a2134f5251b557f6a3a Signed-off-by: Tibor Frank --- .../comparisons/2n-clx_vs_3n-hsw_testbeds.rst | 55 ---------------------- .../comparisons/3n-skx_vs_3n-hsw_testbeds.rst | 55 ---------------------- .../comparisons/current_vs_previous_release.rst | 45 +----------------- .../vpp_performance_tests/comparisons/index.rst | 2 - .../comparisons/nic_comparison.rst | 3 +- 5 files changed, 2 insertions(+), 158 deletions(-) delete mode 100644 docs/report/vpp_performance_tests/comparisons/2n-clx_vs_3n-hsw_testbeds.rst delete mode 100644 docs/report/vpp_performance_tests/comparisons/3n-skx_vs_3n-hsw_testbeds.rst (limited to 'docs/report/vpp_performance_tests/comparisons') diff --git a/docs/report/vpp_performance_tests/comparisons/2n-clx_vs_3n-hsw_testbeds.rst b/docs/report/vpp_performance_tests/comparisons/2n-clx_vs_3n-hsw_testbeds.rst deleted file mode 100644 index 3e641b6762..0000000000 --- a/docs/report/vpp_performance_tests/comparisons/2n-clx_vs_3n-hsw_testbeds.rst +++ /dev/null @@ -1,55 +0,0 @@ -2n-Clx vs 3n-Hsw Testbeds -------------------------- - -Relative comparison of |vpp-release| packet throughput (NDR, PDR and -MRR) is calculated for the same tests executed on 3-Node Skylake (3n- -skx) and 3-Node Haswell (3n-hsw) physical testbed types, in 1-core, -2-core and 4-core configurations. - -.. note:: - - Test results have been generated by - `FD.io test executor vpp performance job 3n-hsw`_ and - `FD.io test executor vpp performance job 2n-clx`_ - with RF result - files csit-vpp-perf-|srelease|-\*.zip - `archived here <../../_static/archive/>`_. - -NDR Comparison -~~~~~~~~~~~~~~ - -Comparison tables in HTML, ASCII and CSV formats: - - - `HTML 1c NDR comparison `_ - - `HTML 2c NDR comparison `_ - - `ASCII 1c NDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-1c-ndr.txt>`_ - - `ASCII 2c NDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-2c-ndr.txt>`_ - - `CSV 1c NDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-1c-ndr-csv.csv>`_ - - `CSV 2c NDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-2c-ndr-csv.csv>`_ - -PDR Comparison -~~~~~~~~~~~~~~ - -Comparison tables in HTML, ASCII and CSV formats: - - - `HTML 1c PDR comparison `_ - - `HTML 2c PDR comparison `_ - - `ASCII 1c PDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-1c-pdr.txt>`_ - - `ASCII 2c PDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-2c-pdr.txt>`_ - - `CSV 1c PDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-1c-pdr-csv.csv>`_ - - `CSV 2c PDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-2c-pdr-csv.csv>`_ - -MRR Comparison -~~~~~~~~~~~~~~ - -Comparison tables in HTML, ASCII and CSV formats: - - - `HTML 1c MRR comparison `_ - - `HTML 2c MRR comparison `_ - - `HTML 4c MRR comparison `_ - - `ASCII 1c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-1c-mrr.txt>`_ - - `ASCII 2c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-2c-mrr.txt>`_ - - `ASCII 4c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-4c-mrr.txt>`_ - - `CSV 1c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-1c-mrr-csv.csv>`_ - - `CSV 2c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-2c-mrr-csv.csv>`_ - - `CSV 4c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-4c-mrr-csv.csv>`_ diff --git a/docs/report/vpp_performance_tests/comparisons/3n-skx_vs_3n-hsw_testbeds.rst b/docs/report/vpp_performance_tests/comparisons/3n-skx_vs_3n-hsw_testbeds.rst deleted file mode 100644 index 2eb552e882..0000000000 --- a/docs/report/vpp_performance_tests/comparisons/3n-skx_vs_3n-hsw_testbeds.rst +++ /dev/null @@ -1,55 +0,0 @@ -3n-Skx vs 3n-Hsw Testbeds -------------------------- - -Relative comparison of |vpp-release| packet throughput (NDR, PDR and -MRR) is calculated for the same tests executed on 3-Node Skylake (3n- -skx) and 3-Node Haswell (3n-hsw) physical testbed types, in 1-core, -2-core and 4-core configurations. - -.. note:: - - Test results have been generated by - `FD.io test executor vpp performance job 3n-hsw`_ and - `FD.io test executor vpp performance job 3n-skx`_ - with RF result - files csit-vpp-perf-|srelease|-\*.zip - `archived here <../../_static/archive/>`_. - -NDR Comparison -~~~~~~~~~~~~~~ - -Comparison tables in HTML, ASCII and CSV formats: - - - `HTML 1c NDR comparison `_ - - `HTML 2c NDR comparison `_ - - `ASCII 1c NDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-1c-ndr.txt>`_ - - `ASCII 2c NDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-2c-ndr.txt>`_ - - `CSV 1c NDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-1c-ndr-csv.csv>`_ - - `CSV 2c NDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-2c-ndr-csv.csv>`_ - -PDR Comparison -~~~~~~~~~~~~~~ - -Comparison tables in HTML, ASCII and CSV formats: - - - `HTML 1c PDR comparison `_ - - `HTML 2c PDR comparison `_ - - `ASCII 1c PDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-1c-pdr.txt>`_ - - `ASCII 2c PDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-2c-pdr.txt>`_ - - `CSV 1c PDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-1c-pdr-csv.csv>`_ - - `CSV 2c PDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-2c-pdr-csv.csv>`_ - -MRR Comparison -~~~~~~~~~~~~~~ - -Comparison tables in HTML, ASCII and CSV formats: - - - `HTML 1c MRR comparison `_ - - `HTML 2c MRR comparison `_ - - `HTML 4c MRR comparison `_ - - `ASCII 1c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-1c-mrr.txt>`_ - - `ASCII 2c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-2c-mrr.txt>`_ - - `ASCII 4c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-4c-mrr.txt>`_ - - `CSV 1c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-1c-mrr-csv.csv>`_ - - `CSV 2c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-2c-mrr-csv.csv>`_ - - `CSV 4c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-4c-mrr-csv.csv>`_ diff --git a/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst b/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst index 3cfd5ae3f8..e251b68954 100644 --- a/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst +++ b/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst @@ -8,7 +8,7 @@ Relative comparison of VPP packet throughput (NDR, PDR and MRR) between |vpp-release| and |vpp-release-1| (measured for |csit-release| and |csit-release-1| respectively) is calculated from results of tests running on 2-node Intel Xeon Skylake (2n-skx), 3-node Intel Xeon Skylake -(3n-skx), 3-Node Intel Xeon Haswell (3n-hsw), 2-node Intel Atom Denverton +(3n-skx), 2-node Intel Atom Denverton (2n-dnv), 3-node Intel Atom Denverton (3n-dnv), 3-node Arm TaiShan (3n-tsh) testbeds, in 1-core, 2-core and 4-core (MRR only) configurations. @@ -29,7 +29,6 @@ results repeatability vary between the test cases. - `FD.io test executor vpp performance job 3n-skx`_, - `FD.io test executor vpp performance job 2n-clx`_, - `FD.io test executor vpp performance job 2n-zn2`_, - - `FD.io test executor vpp performance job 3n-hsw`_, - `FD.io test executor vpp performance job 2n-dnv`_, - `FD.io test executor vpp performance job 3n-dnv`_, - `FD.io test executor vpp performance job 3n-tsh`_, @@ -248,48 +247,6 @@ Comparison tables in HTML, ASCII and CSV formats: - `CSV 4t2c MRR comparison <../../_static/vpp/performance-changes-2n-zn2-xxv710-4t2c-mrr-csv.csv>`_ - `CSV 8t4c MRR comparison <../../_static/vpp/performance-changes-2n-zn2-xxv710-8t4c-mrr-csv.csv>`_ -3n-hsw -~~~~~~ - -NDR Comparison -`````````````` - -Comparison tables in HTML, ASCII and CSV formats: - - - `HTML 1t1c NDR comparison `_ - - `HTML 2t2c NDR comparison `_ - - `ASCII 1t1c NDR comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-ndr.txt>`_ - - `ASCII 2t2c NDR comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-ndr.txt>`_ - - `CSV 1t1c NDR comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-ndr-csv.csv>`_ - - `CSV 2t2c NDR comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-ndr-csv.csv>`_ - -PDR Comparison -`````````````` - -Comparison tables in HTML, ASCII and CSV formats: - - - `HTML 1t1c PDR comparison `_ - - `HTML 2t2c PDR comparison `_ - - `ASCII 1t1c PDR comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-pdr.txt>`_ - - `ASCII 2t2c PDR comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-pdr.txt>`_ - - `CSV 1t1c PDR comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-pdr-csv.csv>`_ - - `CSV 2t2c PDR comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-pdr-csv.csv>`_ - -MRR Comparison -`````````````` - -Comparison tables in HTML, ASCII and CSV formats: - - - `HTML 1t1c MRR comparison `_ - - `HTML 2t2c MRR comparison `_ - - `HTML 4t4c MRR comparison `_ - - `ASCII 1t1c MRR comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-mrr.txt>`_ - - `ASCII 2t2c MRR comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-mrr.txt>`_ - - `ASCII 4t4c MRR comparison <../../_static/vpp/performance-changes-3n-hsw-4t4c-mrr.txt>`_ - - `CSV 1t1c MRR comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-mrr-csv.csv>`_ - - `CSV 2t2c MRR comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-mrr-csv.csv>`_ - - `CSV 4t4c MRR comparison <../../_static/vpp/performance-changes-3n-hsw-4t4c-mrr-csv.csv>`_ - 2n-dnv ~~~~~~ diff --git a/docs/report/vpp_performance_tests/comparisons/index.rst b/docs/report/vpp_performance_tests/comparisons/index.rst index 1a05947685..551ec9159d 100644 --- a/docs/report/vpp_performance_tests/comparisons/index.rst +++ b/docs/report/vpp_performance_tests/comparisons/index.rst @@ -11,9 +11,7 @@ Comparisons .. toctree:: current_vs_previous_release - 3n-skx_vs_3n-hsw_testbeds 2n-skx_vs_2n-clx_testbeds - 2n-clx_vs_3n-hsw_testbeds 3n-skx_vs_2n-skx_testbeds nic_comparison soak_vs_ndr diff --git a/docs/report/vpp_performance_tests/comparisons/nic_comparison.rst b/docs/report/vpp_performance_tests/comparisons/nic_comparison.rst index 51905c9d0e..9d9dd8eec0 100644 --- a/docs/report/vpp_performance_tests/comparisons/nic_comparison.rst +++ b/docs/report/vpp_performance_tests/comparisons/nic_comparison.rst @@ -6,8 +6,7 @@ NICs Comparison Relative comparison of VPP packet throughput (NDR, PDR and MRR) between NICs (measured for |csit-release) is calculated from results of tests -running on 3-Node Intel Xeon Haswell testbeds (3n-hsw) in 1-core, 2-core -and 4-core (MRR only) configurations. +running on 3n-skx, 2n-skx testbeds. Listed mean and standard deviation values are computed based on a series of the same tests executed against respective VPP releases to verify -- cgit 1.2.3-korg