From 6c913634f0d70d64b2c88122bbe9eb911f29e417 Mon Sep 17 00:00:00 2001 From: Tibor Frank Date: Fri, 10 Jan 2020 13:22:00 +0100 Subject: Report: Add comparison tables CSIT-1654 Change-Id: I280195b295d4fc717cb355e78c5f1731bb1eabe5 Signed-off-by: Tibor Frank --- .../comparisons/current_vs_previous_release.rst | 252 +++++++++++---------- 1 file changed, 128 insertions(+), 124 deletions(-) (limited to 'docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst') diff --git a/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst b/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst index 29abb0ac1a..cd0af1f914 100644 --- a/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst +++ b/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst @@ -7,8 +7,10 @@ Current vs. Previous Release Relative comparison of VPP packet throughput (NDR, PDR and MRR) between |vpp-release| and |vpp-release-1| (measured for |csit-release| and |csit-release-1| respectively) is calculated from results of tests -running on 3-Node Intel Xeon Haswell testbeds (3n-hsw) in 1-core, 2-core -and 4-core (MRR only) configurations. +running on 2-node Intel Xeon Skylake (2n-skx), 3-node Intel Xeon Skylake +(3n-skx), 3-Node Intel Xeon Haswell (3n-hsw), 2-node Intel Atom Denverton +(2n-dnv), 3-node Intel Atom Denverton (3n-dnv), 3-node Arm TaiShan (3n-tsh) +testbeds, in 1-core, 2-core and 4-core (MRR only) configurations. Listed mean and standard deviation values are computed based on a series of the same tests executed against respective VPP releases to verify @@ -23,9 +25,12 @@ results repeatability vary between the test cases. Test results have been generated by - - `FD.io test executor vpp performance job 3n-hsw`_, + - `FD.io test executor vpp performance job 2n-skx`_, - `FD.io test executor vpp performance job 3n-skx`_, - - `FD.io test executor vpp performance job 2n-skx`_ + - `FD.io test executor vpp performance job 3n-hsw`_, + - `FD.io test executor vpp performance job 2n-dnv`_, + - `FD.io test executor vpp performance job 3n-dnv`_, + - `FD.io test executor vpp performance job 3n-tsh`_ with RF result files csit-vpp-perf-|srelease|-\*.zip `archived here <../../_static/archive/>`_. @@ -156,129 +161,128 @@ Comparison tables in HTML, ASCII and CSV formats: - `CSV 2t2c MRR comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-mrr.csv>`_ - `CSV 4t4c MRR comparison <../../_static/vpp/performance-changes-3n-hsw-4t4c-mrr.csv>`_ -.. - 3n-tsh - ~~~~~~ +2n-dnv +~~~~~~ + +NDR Comparison +`````````````` + +Comparison tables in HTML, ASCII and CSV formats: - NDR Comparison - `````````````` + - `HTML 1t1c NDR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-ndr.html>`_ + - `HTML 2t2c NDR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-ndr.html>`_ + - `ASCII 1t1c NDR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-ndr.txt>`_ + - `ASCII 2t2c NDR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-ndr.txt>`_ + - `CSV 1t1c NDR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-ndr.csv>`_ + - `CSV 2t2c NDR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-ndr.csv>`_ - Comparison tables in HTML, ASCII and CSV formats: +PDR Comparison +`````````````` - - `HTML 1t1c NDR comparison <../../_static/vpp/performance-changes-3n-tsh-1t1c-ndr.html>`_ - - `HTML 2t2c NDR comparison <../../_static/vpp/performance-changes-3n-tsh-2t2c-ndr.html>`_ - - `ASCII 1t1c NDR comparison <../../_static/vpp/performance-changes-3n-tsh-1t1c-ndr.txt>`_ - - `ASCII 2t2c NDR comparison <../../_static/vpp/performance-changes-3n-tsh-2t2c-ndr.txt>`_ - - `CSV 1t1c NDR comparison <../../_static/vpp/performance-changes-3n-tsh-1t1c-ndr.csv>`_ - - `CSV 2t2c NDR comparison <../../_static/vpp/performance-changes-3n-tsh-2t2c-ndr.csv>`_ +Comparison tables in HTML, ASCII and CSV formats: - PDR Comparison - `````````````` + - `HTML 1t1c PDR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-pdr.html>`_ + - `HTML 2t2c PDR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-pdr.html>`_ + - `ASCII 1t1c PDR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-pdr.txt>`_ + - `ASCII 2t2c PDR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-pdr.txt>`_ + - `CSV 1t1c PDR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-pdr.csv>`_ + - `CSV 2t2c PDR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-pdr.csv>`_ + +MRR Comparison +`````````````` + +Comparison tables in HTML, ASCII and CSV formats: + + - `HTML 1t1c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-mrr.html>`_ + - `HTML 2t2c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-mrr.html>`_ + - `HTML 4t4c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-4t4c-mrr.html>`_ + - `ASCII 1t1c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-mrr.txt>`_ + - `ASCII 2t2c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-mrr.txt>`_ + - `ASCII 4t4c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-4t4c-mrr.txt>`_ + - `CSV 1t1c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-mrr.csv>`_ + - `CSV 2t2c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-mrr.csv>`_ + - `CSV 4t4c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-4t4c-mrr.csv>`_ + +3n-dnv +~~~~~~ + +NDR Comparison +`````````````` + +Comparison tables in HTML, ASCII and CSV formats: + + - `HTML 1t1c NDR comparison <../../_static/vpp/performance-changes-3n-dnv-1t1c-ndr.html>`_ + - `HTML 2t2c NDR comparison <../../_static/vpp/performance-changes-3n-dnv-2t2c-ndr.html>`_ + - `ASCII 1t1c NDR comparison <../../_static/vpp/performance-changes-3n-dnv-1t1c-ndr.txt>`_ + - `ASCII 2t2c NDR comparison <../../_static/vpp/performance-changes-3n-dnv-2t2c-ndr.txt>`_ + - `CSV 1t1c NDR comparison <../../_static/vpp/performance-changes-3n-dnv-1t1c-ndr.csv>`_ + - `CSV 2t2c NDR comparison <../../_static/vpp/performance-changes-3n-dnv-2t2c-ndr.csv>`_ + +PDR Comparison +`````````````` - Comparison tables in HTML, ASCII and CSV formats: +Comparison tables in HTML, ASCII and CSV formats: + + - `HTML 1t1c PDR comparison <../../_static/vpp/performance-changes-3n-dnv-1t1c-pdr.html>`_ + - `HTML 2t2c PDR comparison <../../_static/vpp/performance-changes-3n-dnv-2t2c-pdr.html>`_ + - `ASCII 1t1c PDR comparison <../../_static/vpp/performance-changes-3n-dnv-1t1c-pdr.txt>`_ + - `ASCII 2t2c PDR comparison <../../_static/vpp/performance-changes-3n-dnv-2t2c-pdr.txt>`_ + - `CSV 1t1c PDR comparison <../../_static/vpp/performance-changes-3n-dnv-1t1c-pdr.csv>`_ + - `CSV 2t2c PDR comparison <../../_static/vpp/performance-changes-3n-dnv-2t2c-pdr.csv>`_ + +MRR Comparison +`````````````` + +Comparison tables in HTML, ASCII and CSV formats: + + - `HTML 1t1c MRR comparison <../../_static/vpp/performance-changes-3n-dnv-1t1c-mrr.html>`_ + - `HTML 2t2c MRR comparison <../../_static/vpp/performance-changes-3n-dnv-2t2c-mrr.html>`_ + - `HTML 4t4c MRR comparison <../../_static/vpp/performance-changes-3n-dnv-4t4c-mrr.html>`_ + - `ASCII 1t1c MRR comparison <../../_static/vpp/performance-changes-3n-dnv-1t1c-mrr.txt>`_ + - `ASCII 2t2c MRR comparison <../../_static/vpp/performance-changes-3n-dnv-2t2c-mrr.txt>`_ + - `ASCII 4t4c MRR comparison <../../_static/vpp/performance-changes-3n-dnv-4t4c-mrr.txt>`_ + - `CSV 1t1c MRR comparison <../../_static/vpp/performance-changes-3n-dnv-1t1c-mrr.csv>`_ + - `CSV 2t2c MRR comparison <../../_static/vpp/performance-changes-3n-dnv-2t2c-mrr.csv>`_ + - `CSV 4t4c MRR comparison <../../_static/vpp/performance-changes-3n-dnv-4t4c-mrr.csv>`_ + +3n-tsh +~~~~~~ + +NDR Comparison +`````````````` + +Comparison tables in HTML, ASCII and CSV formats: + + - `HTML 1t1c NDR comparison <../../_static/vpp/performance-changes-3n-tsh-1t1c-ndr.html>`_ + - `HTML 2t2c NDR comparison <../../_static/vpp/performance-changes-3n-tsh-2t2c-ndr.html>`_ + - `ASCII 1t1c NDR comparison <../../_static/vpp/performance-changes-3n-tsh-1t1c-ndr.txt>`_ + - `ASCII 2t2c NDR comparison <../../_static/vpp/performance-changes-3n-tsh-2t2c-ndr.txt>`_ + - `CSV 1t1c NDR comparison <../../_static/vpp/performance-changes-3n-tsh-1t1c-ndr.csv>`_ + - `CSV 2t2c NDR comparison <../../_static/vpp/performance-changes-3n-tsh-2t2c-ndr.csv>`_ + +PDR Comparison +`````````````` + +Comparison tables in HTML, ASCII and CSV formats: + + - `HTML 1t1c PDR comparison <../../_static/vpp/performance-changes-3n-tsh-1t1c-pdr.html>`_ + - `HTML 2t2c PDR comparison <../../_static/vpp/performance-changes-3n-tsh-2t2c-pdr.html>`_ + - `ASCII 1t1c PDR comparison <../../_static/vpp/performance-changes-3n-tsh-1t1c-pdr.txt>`_ + - `ASCII 2t2c PDR comparison <../../_static/vpp/performance-changes-3n-tsh-2t2c-pdr.txt>`_ + - `CSV 1t1c PDR comparison <../../_static/vpp/performance-changes-3n-tsh-1t1c-pdr.csv>`_ + - `CSV 2t2c PDR comparison <../../_static/vpp/performance-changes-3n-tsh-2t2c-pdr.csv>`_ + +MRR Comparison +`````````````` + +Comparison tables in HTML, ASCII and CSV formats: - - `HTML 1t1c PDR comparison <../../_static/vpp/performance-changes-3n-tsh-1t1c-pdr.html>`_ - - `HTML 2t2c PDR comparison <../../_static/vpp/performance-changes-3n-tsh-2t2c-pdr.html>`_ - - `ASCII 1t1c PDR comparison <../../_static/vpp/performance-changes-3n-tsh-1t1c-pdr.txt>`_ - - `ASCII 2t2c PDR comparison <../../_static/vpp/performance-changes-3n-tsh-2t2c-pdr.txt>`_ - - `CSV 1t1c PDR comparison <../../_static/vpp/performance-changes-3n-tsh-1t1c-pdr.csv>`_ - - `CSV 2t2c PDR comparison <../../_static/vpp/performance-changes-3n-tsh-2t2c-pdr.csv>`_ - - MRR Comparison - `````````````` - - Comparison tables in HTML, ASCII and CSV formats: - - - `HTML 1t1c MRR comparison <../../_static/vpp/performance-changes-3n-tsh-1t1c-mrr.html>`_ - - `HTML 2t2c MRR comparison <../../_static/vpp/performance-changes-3n-tsh-2t2c-mrr.html>`_ - - `HTML 4t4c MRR comparison <../../_static/vpp/performance-changes-3n-tsh-4t4c-mrr.html>`_ - - `ASCII 1t1c MRR comparison <../../_static/vpp/performance-changes-3n-tsh-1t1c-mrr.txt>`_ - - `ASCII 2t2c MRR comparison <../../_static/vpp/performance-changes-3n-tsh-2t2c-mrr.txt>`_ - - `ASCII 4t4c MRR comparison <../../_static/vpp/performance-changes-3n-tsh-4t4c-mrr.txt>`_ - - `CSV 1t1c MRR comparison <../../_static/vpp/performance-changes-3n-tsh-1t1c-mrr.csv>`_ - - `CSV 2t2c MRR comparison <../../_static/vpp/performance-changes-3n-tsh-2t2c-mrr.csv>`_ - - `CSV 4t4c MRR comparison <../../_static/vpp/performance-changes-3n-tsh-4t4c-mrr.csv>`_ - - 2n-dnv - ~~~~~~ - - NDR Comparison - `````````````` - - Comparison tables in HTML, ASCII and CSV formats: - - - `HTML 1t1c NDR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-ndr.html>`_ - - `HTML 2t2c NDR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-ndr.html>`_ - - `ASCII 1t1c NDR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-ndr.txt>`_ - - `ASCII 2t2c NDR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-ndr.txt>`_ - - `CSV 1t1c NDR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-ndr.csv>`_ - - `CSV 2t2c NDR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-ndr.csv>`_ - - PDR Comparison - `````````````` - - Comparison tables in HTML, ASCII and CSV formats: - - - `HTML 1t1c PDR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-pdr.html>`_ - - `HTML 2t2c PDR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-pdr.html>`_ - - `ASCII 1t1c PDR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-pdr.txt>`_ - - `ASCII 2t2c PDR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-pdr.txt>`_ - - `CSV 1t1c PDR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-pdr.csv>`_ - - `CSV 2t2c PDR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-pdr.csv>`_ - - MRR Comparison - `````````````` - - Comparison tables in HTML, ASCII and CSV formats: - - - `HTML 1t1c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-mrr.html>`_ - - `HTML 2t2c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-mrr.html>`_ - - `HTML 4t4c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-4t4c-mrr.html>`_ - - `ASCII 1t1c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-mrr.txt>`_ - - `ASCII 2t2c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-mrr.txt>`_ - - `ASCII 4t4c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-4t4c-mrr.txt>`_ - - `CSV 1t1c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-mrr.csv>`_ - - `CSV 2t2c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-mrr.csv>`_ - - `CSV 4t4c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-4t4c-mrr.csv>`_ - - 3n-dnv - ~~~~~~ - - NDR Comparison - `````````````` - - Comparison tables in HTML, ASCII and CSV formats: - - - `HTML 1t1c NDR comparison <../../_static/vpp/performance-changes-3n-dnv-1t1c-ndr.html>`_ - - `HTML 2t2c NDR comparison <../../_static/vpp/performance-changes-3n-dnv-2t2c-ndr.html>`_ - - `ASCII 1t1c NDR comparison <../../_static/vpp/performance-changes-3n-dnv-1t1c-ndr.txt>`_ - - `ASCII 2t2c NDR comparison <../../_static/vpp/performance-changes-3n-dnv-2t2c-ndr.txt>`_ - - `CSV 1t1c NDR comparison <../../_static/vpp/performance-changes-3n-dnv-1t1c-ndr.csv>`_ - - `CSV 2t2c NDR comparison <../../_static/vpp/performance-changes-3n-dnv-2t2c-ndr.csv>`_ - - PDR Comparison - `````````````` - - Comparison tables in HTML, ASCII and CSV formats: - - - `HTML 1t1c PDR comparison <../../_static/vpp/performance-changes-3n-dnv-1t1c-pdr.html>`_ - - `HTML 2t2c PDR comparison <../../_static/vpp/performance-changes-3n-dnv-2t2c-pdr.html>`_ - - `ASCII 1t1c PDR comparison <../../_static/vpp/performance-changes-3n-dnv-1t1c-pdr.txt>`_ - - `ASCII 2t2c PDR comparison <../../_static/vpp/performance-changes-3n-dnv-2t2c-pdr.txt>`_ - - `CSV 1t1c PDR comparison <../../_static/vpp/performance-changes-3n-dnv-1t1c-pdr.csv>`_ - - `CSV 2t2c PDR comparison <../../_static/vpp/performance-changes-3n-dnv-2t2c-pdr.csv>`_ - - MRR Comparison - `````````````` - - Comparison tables in HTML, ASCII and CSV formats: - - - `HTML 1t1c MRR comparison <../../_static/vpp/performance-changes-3n-dnv-1t1c-mrr.html>`_ - - `HTML 2t2c MRR comparison <../../_static/vpp/performance-changes-3n-dnv-2t2c-mrr.html>`_ - - `HTML 4t4c MRR comparison <../../_static/vpp/performance-changes-3n-dnv-4t4c-mrr.html>`_ - - `ASCII 1t1c MRR comparison <../../_static/vpp/performance-changes-3n-dnv-1t1c-mrr.txt>`_ - - `ASCII 2t2c MRR comparison <../../_static/vpp/performance-changes-3n-dnv-2t2c-mrr.txt>`_ - - `ASCII 4t4c MRR comparison <../../_static/vpp/performance-changes-3n-dnv-4t4c-mrr.txt>`_ - - `CSV 1t1c MRR comparison <../../_static/vpp/performance-changes-3n-dnv-1t1c-mrr.csv>`_ - - `CSV 2t2c MRR comparison <../../_static/vpp/performance-changes-3n-dnv-2t2c-mrr.csv>`_ - - `CSV 4t4c MRR comparison <../../_static/vpp/performance-changes-3n-dnv-4t4c-mrr.csv>`_ + - `HTML 1t1c MRR comparison <../../_static/vpp/performance-changes-3n-tsh-1t1c-mrr.html>`_ + - `HTML 2t2c MRR comparison <../../_static/vpp/performance-changes-3n-tsh-2t2c-mrr.html>`_ + - `HTML 4t4c MRR comparison <../../_static/vpp/performance-changes-3n-tsh-4t4c-mrr.html>`_ + - `ASCII 1t1c MRR comparison <../../_static/vpp/performance-changes-3n-tsh-1t1c-mrr.txt>`_ + - `ASCII 2t2c MRR comparison <../../_static/vpp/performance-changes-3n-tsh-2t2c-mrr.txt>`_ + - `ASCII 4t4c MRR comparison <../../_static/vpp/performance-changes-3n-tsh-4t4c-mrr.txt>`_ + - `CSV 1t1c MRR comparison <../../_static/vpp/performance-changes-3n-tsh-1t1c-mrr.csv>`_ + - `CSV 2t2c MRR comparison <../../_static/vpp/performance-changes-3n-tsh-2t2c-mrr.csv>`_ + - `CSV 4t4c MRR comparison <../../_static/vpp/performance-changes-3n-tsh-4t4c-mrr.csv>`_ -- cgit 1.2.3-korg