diff options
| author |   Tibor Frank <tifrank@cisco.com> | 2020-07-07 09:35:59 +0200 |
|---|---|---|
| committer | Tibor Frank <tifrank@cisco.com> | 2020-07-08 07:52:54 +0200 |
| commit | ff8c8cf6083f64fd337115fec20501f7b247540e (patch) | |
| tree | 326e09da7eea2493b5d95d708344cc4d7dfa21ef | |
| parent | 9439b30ba91e3532c66548c34bade4469fdfab37 (diff) | |
PAL: Improve comparison tables
- place RCAs to the columns with diffs
CSIT-1737
Change-Id: I3ccf5ee510796684e3e09dbdc6264b05f391f633
Signed-off-by: Tibor Frank <tifrank@cisco.com>
8 files changed, 439 insertions, 724 deletions
diff --git a/docs/report/vpp_performance_tests/comparisons/2n-clx_vs_3n-hsw_testbeds.rst b/docs/report/vpp_performance_tests/comparisons/2n-clx_vs_3n-hsw_testbeds.rst index 4837b6409b..3e641b6762 100644 --- a/docs/report/vpp_performance_tests/comparisons/2n-clx_vs_3n-hsw_testbeds.rst +++ b/docs/report/vpp_performance_tests/comparisons/2n-clx_vs_3n-hsw_testbeds.rst @@ -24,8 +24,8 @@ Comparison tables in HTML, ASCII and CSV formats: - `HTML 2c NDR comparison <performance-compare-testbeds-3n-hsw-2n-clx-2c-ndr.html>`_ - `ASCII 1c NDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-1c-ndr.txt>`_ - `ASCII 2c NDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-2c-ndr.txt>`_ - - `CSV 1c NDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-1c-ndr.csv>`_ - - `CSV 2c NDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-2c-ndr.csv>`_ + - `CSV 1c NDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-1c-ndr-csv.csv>`_ + - `CSV 2c NDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-2c-ndr-csv.csv>`_ PDR Comparison ~~~~~~~~~~~~~~ @@ -36,8 +36,8 @@ Comparison tables in HTML, ASCII and CSV formats: - `HTML 2c PDR comparison <performance-compare-testbeds-3n-hsw-2n-clx-2c-pdr.html>`_ - `ASCII 1c PDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-1c-pdr.txt>`_ - `ASCII 2c PDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-2c-pdr.txt>`_ - - `CSV 1c PDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-1c-pdr.csv>`_ - - `CSV 2c PDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-2c-pdr.csv>`_ + - `CSV 1c PDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-1c-pdr-csv.csv>`_ + - `CSV 2c PDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-2c-pdr-csv.csv>`_ MRR Comparison ~~~~~~~~~~~~~~ @@ -50,6 +50,6 @@ Comparison tables in HTML, ASCII and CSV formats: - `ASCII 1c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-1c-mrr.txt>`_ - `ASCII 2c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-2c-mrr.txt>`_ - `ASCII 4c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-4c-mrr.txt>`_ - - `CSV 1c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-1c-mrr.csv>`_ - - `CSV 2c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-2c-mrr.csv>`_ - - `CSV 4c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-4c-mrr.csv>`_ + - `CSV 1c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-1c-mrr-csv.csv>`_ + - `CSV 2c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-2c-mrr-csv.csv>`_ + - `CSV 4c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-2n-clx-4c-mrr-csv.csv>`_ diff --git a/docs/report/vpp_performance_tests/comparisons/2n-skx_vs_2n-clx_testbeds.rst b/docs/report/vpp_performance_tests/comparisons/2n-skx_vs_2n-clx_testbeds.rst index 3b9abb80f4..464a8024f8 100644 --- a/docs/report/vpp_performance_tests/comparisons/2n-skx_vs_2n-clx_testbeds.rst +++ b/docs/report/vpp_performance_tests/comparisons/2n-skx_vs_2n-clx_testbeds.rst @@ -23,8 +23,8 @@ Comparison tables in HTML, ASCII and CSV formats: - `HTML 2c NDR comparison <performance-compare-testbeds-2n-skx-2n-clx-4t2c-ndr.html>`_ - `ASCII 1c NDR comparison <../../_static/vpp/performance-compare-testbeds-2n-skx-2n-clx-2t1c-ndr.txt>`_ - `ASCII 2c NDR comparison <../../_static/vpp/performance-compare-testbeds-2n-skx-2n-clx-4t2c-ndr.txt>`_ - - `CSV 1c NDR comparison <../../_static/vpp/performance-compare-testbeds-2n-skx-2n-clx-2t1c-ndr.csv>`_ - - `CSV 2c NDR comparison <../../_static/vpp/performance-compare-testbeds-2n-skx-2n-clx-4t2c-ndr.csv>`_ + - `CSV 1c NDR comparison <../../_static/vpp/performance-compare-testbeds-2n-skx-2n-clx-2t1c-ndr-csv.csv>`_ + - `CSV 2c NDR comparison <../../_static/vpp/performance-compare-testbeds-2n-skx-2n-clx-4t2c-ndr-csv.csv>`_ PDR Comparison ~~~~~~~~~~~~~~ @@ -35,8 +35,8 @@ Comparison tables in HTML, ASCII and CSV formats: - `HTML 2c PDR comparison <performance-compare-testbeds-2n-skx-2n-clx-4t2c-pdr.html>`_ - `ASCII 1c PDR comparison <../../_static/vpp/performance-compare-testbeds-2n-skx-2n-clx-2t1c-pdr.txt>`_ - `ASCII 2c PDR comparison <../../_static/vpp/performance-compare-testbeds-2n-skx-2n-clx-4t2c-pdr.txt>`_ - - `CSV 1c PDR comparison <../../_static/vpp/performance-compare-testbeds-2n-skx-2n-clx-2t1c-pdr.csv>`_ - - `CSV 2c PDR comparison <../../_static/vpp/performance-compare-testbeds-2n-skx-2n-clx-4t2c-pdr.csv>`_ + - `CSV 1c PDR comparison <../../_static/vpp/performance-compare-testbeds-2n-skx-2n-clx-2t1c-pdr-csv.csv>`_ + - `CSV 2c PDR comparison <../../_static/vpp/performance-compare-testbeds-2n-skx-2n-clx-4t2c-pdr-csv.csv>`_ MRR Comparison ~~~~~~~~~~~~~~ @@ -49,6 +49,6 @@ Comparison tables in HTML, ASCII and CSV formats: - `ASCII 1c MRR comparison <../../_static/vpp/performance-compare-testbeds-2n-skx-2n-clx-2t1c-mrr.txt>`_ - `ASCII 2c MRR comparison <../../_static/vpp/performance-compare-testbeds-2n-skx-2n-clx-4t2c-mrr.txt>`_ - `ASCII 4c MRR comparison <../../_static/vpp/performance-compare-testbeds-2n-skx-2n-clx-8t4c-mrr.txt>`_ - - `CSV 1c MRR comparison <../../_static/vpp/performance-compare-testbeds-2n-skx-2n-clx-2t1c-mrr.csv>`_ - - `CSV 2c MRR comparison <../../_static/vpp/performance-compare-testbeds-2n-skx-2n-clx-4t2c-mrr.csv>`_ - - `CSV 4c MRR comparison <../../_static/vpp/performance-compare-testbeds-2n-skx-2n-clx-8t4c-mrr.csv>`_ + - `CSV 1c MRR comparison <../../_static/vpp/performance-compare-testbeds-2n-skx-2n-clx-2t1c-mrr-csv.csv>`_ + - `CSV 2c MRR comparison <../../_static/vpp/performance-compare-testbeds-2n-skx-2n-clx-4t2c-mrr-csv.csv>`_ + - `CSV 4c MRR comparison <../../_static/vpp/performance-compare-testbeds-2n-skx-2n-clx-8t4c-mrr-csv.csv>`_ diff --git a/docs/report/vpp_performance_tests/comparisons/3n-skx_vs_2n-skx_testbeds.rst b/docs/report/vpp_performance_tests/comparisons/3n-skx_vs_2n-skx_testbeds.rst index bb4a23ef5a..bfbc9ae3df 100644 --- a/docs/report/vpp_performance_tests/comparisons/3n-skx_vs_2n-skx_testbeds.rst +++ b/docs/report/vpp_performance_tests/comparisons/3n-skx_vs_2n-skx_testbeds.rst @@ -27,8 +27,8 @@ Comparison tables in HTML, ASCII and CSV formats: - `HTML 2c NDR comparison <performance-compare-topologies-3n-skx-2n-skx-2c-ndr.html>`_ - `ASCII 1c NDR comparison <../../_static/vpp/performance-compare-topologies-3n-skx-2n-skx-1c-ndr.txt>`_ - `ASCII 2c NDR comparison <../../_static/vpp/performance-compare-topologies-3n-skx-2n-skx-2c-ndr.txt>`_ - - `CSV 1c NDR comparison <../../_static/vpp/performance-compare-topologies-3n-skx-2n-skx-1c-ndr.csv>`_ - - `CSV 2c NDR comparison <../../_static/vpp/performance-compare-topologies-3n-skx-2n-skx-2c-ndr.csv>`_ + - `CSV 1c NDR comparison <../../_static/vpp/performance-compare-topologies-3n-skx-2n-skx-1c-ndr-csv.csv>`_ + - `CSV 2c NDR comparison <../../_static/vpp/performance-compare-topologies-3n-skx-2n-skx-2c-ndr-csv.csv>`_ PDR Comparison ~~~~~~~~~~~~~~ @@ -39,8 +39,8 @@ Comparison tables in HTML, ASCII and CSV formats: - `HTML 2c PDR comparison <performance-compare-topologies-3n-skx-2n-skx-2c-pdr.html>`_ - `ASCII 1c PDR comparison <../../_static/vpp/performance-compare-topologies-3n-skx-2n-skx-1c-pdr.txt>`_ - `ASCII 2c PDR comparison <../../_static/vpp/performance-compare-topologies-3n-skx-2n-skx-2c-pdr.txt>`_ - - `CSV 1c PDR comparison <../../_static/vpp/performance-compare-topologies-3n-skx-2n-skx-1c-pdr.csv>`_ - - `CSV 2c PDR comparison <../../_static/vpp/performance-compare-topologies-3n-skx-2n-skx-2c-pdr.csv>`_ + - `CSV 1c PDR comparison <../../_static/vpp/performance-compare-topologies-3n-skx-2n-skx-1c-pdr-csv.csv>`_ + - `CSV 2c PDR comparison <../../_static/vpp/performance-compare-topologies-3n-skx-2n-skx-2c-pdr-csv.csv>`_ MRR Comparison ~~~~~~~~~~~~~~ @@ -53,6 +53,6 @@ Comparison tables in HTML, ASCII and CSV formats: - `ASCII 1c MRR comparison <../../_static/vpp/performance-compare-topologies-3n-skx-2n-skx-1c-mrr.txt>`_ - `ASCII 2c MRR comparison <../../_static/vpp/performance-compare-topologies-3n-skx-2n-skx-2c-mrr.txt>`_ - `ASCII 4c MRR comparison <../../_static/vpp/performance-compare-topologies-3n-skx-2n-skx-4c-mrr.txt>`_ - - `CSV 1c MRR comparison <../../_static/vpp/performance-compare-topologies-3n-skx-2n-skx-1c-mrr.csv>`_ - - `CSV 2c MRR comparison <../../_static/vpp/performance-compare-topologies-3n-skx-2n-skx-2c-mrr.csv>`_ - - `CSV 4c MRR comparison <../../_static/vpp/performance-compare-topologies-3n-skx-2n-skx-4c-mrr.csv>`_ + - `CSV 1c MRR comparison <../../_static/vpp/performance-compare-topologies-3n-skx-2n-skx-1c-mrr-csv.csv>`_ + - `CSV 2c MRR comparison <../../_static/vpp/performance-compare-topologies-3n-skx-2n-skx-2c-mrr-csv.csv>`_ + - `CSV 4c MRR comparison <../../_static/vpp/performance-compare-topologies-3n-skx-2n-skx-4c-mrr-csv.csv>`_ diff --git a/docs/report/vpp_performance_tests/comparisons/3n-skx_vs_3n-hsw_testbeds.rst b/docs/report/vpp_performance_tests/comparisons/3n-skx_vs_3n-hsw_testbeds.rst index 003f92cea8..2eb552e882 100644 --- a/docs/report/vpp_performance_tests/comparisons/3n-skx_vs_3n-hsw_testbeds.rst +++ b/docs/report/vpp_performance_tests/comparisons/3n-skx_vs_3n-hsw_testbeds.rst @@ -24,8 +24,8 @@ Comparison tables in HTML, ASCII and CSV formats: - `HTML 2c NDR comparison <performance-compare-testbeds-3n-hsw-3n-skx-2c-ndr.html>`_ - `ASCII 1c NDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-1c-ndr.txt>`_ - `ASCII 2c NDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-2c-ndr.txt>`_ - - `CSV 1c NDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-1c-ndr.csv>`_ - - `CSV 2c NDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-2c-ndr.csv>`_ + - `CSV 1c NDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-1c-ndr-csv.csv>`_ + - `CSV 2c NDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-2c-ndr-csv.csv>`_ PDR Comparison ~~~~~~~~~~~~~~ @@ -36,8 +36,8 @@ Comparison tables in HTML, ASCII and CSV formats: - `HTML 2c PDR comparison <performance-compare-testbeds-3n-hsw-3n-skx-2c-pdr.html>`_ - `ASCII 1c PDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-1c-pdr.txt>`_ - `ASCII 2c PDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-2c-pdr.txt>`_ - - `CSV 1c PDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-1c-pdr.csv>`_ - - `CSV 2c PDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-2c-pdr.csv>`_ + - `CSV 1c PDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-1c-pdr-csv.csv>`_ + - `CSV 2c PDR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-2c-pdr-csv.csv>`_ MRR Comparison ~~~~~~~~~~~~~~ @@ -50,6 +50,6 @@ Comparison tables in HTML, ASCII and CSV formats: - `ASCII 1c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-1c-mrr.txt>`_ - `ASCII 2c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-2c-mrr.txt>`_ - `ASCII 4c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-4c-mrr.txt>`_ - - `CSV 1c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-1c-mrr.csv>`_ - - `CSV 2c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-2c-mrr.csv>`_ - - `CSV 4c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-4c-mrr.csv>`_ + - `CSV 1c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-1c-mrr-csv.csv>`_ + - `CSV 2c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-2c-mrr-csv.csv>`_ + - `CSV 4c MRR comparison <../../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-4c-mrr-csv.csv>`_ diff --git a/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst b/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst index a65e5d164c..42c5ff9f41 100644 --- a/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst +++ b/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst @@ -67,8 +67,8 @@ Comparison tables in HTML, ASCII and CSV formats: - `HTML 4t2c NDR comparison <performance-changes-2n-skx-4t2c-ndr.html>`_ - `ASCII 2t1c NDR comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-ndr.txt>`_ - `ASCII 4t2c NDR comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-ndr.txt>`_ - - `CSV 2t1c NDR comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-ndr.csv>`_ - - `CSV 4t2c NDR comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-ndr.csv>`_ + - `CSV 2t1c NDR comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-ndr-csv.csv>`_ + - `CSV 4t2c NDR comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-ndr-csv.csv>`_ .. - `HTML 2t1c NFV Service Density NDR comparison <performance-changes-nfv-2n-skx-2t1c-ndr.html>`_ @@ -77,9 +77,9 @@ Comparison tables in HTML, ASCII and CSV formats: - `ASCII 2t1c NFV Service Density NDR comparison <../../_static/vpp/performance-changes-nfv-2n-skx-2t1c-ndr.txt>`_ - `ASCII 4t2c NFV Service Density NDR comparison <../../_static/vpp/performance-changes-nfv-2n-skx-4t2c-ndr.txt>`_ - `ASCII 8t4c NFV Service Density NDR comparison <../../_static/vpp/performance-changes-nfv-2n-skx-8t4c-ndr.txt>`_ - - `CSV 2t1c NFV Service Density NDR comparison <../../_static/vpp/performance-changes-nfv-2n-skx-2t1c-ndr.csv>`_ - - `CSV 4t2c NFV Service Density NDR comparison <../../_static/vpp/performance-changes-nfv-2n-skx-4t2c-ndr.csv>`_ - - `CSV 8t4c NFV Service Density NDR comparison <../../_static/vpp/performance-changes-nfv-2n-skx-8t4c-ndr.csv>`_ + - `CSV 2t1c NFV Service Density NDR comparison <../../_static/vpp/performance-changes-nfv-2n-skx-2t1c-ndr-csv.csv>`_ + - `CSV 4t2c NFV Service Density NDR comparison <../../_static/vpp/performance-changes-nfv-2n-skx-4t2c-ndr-csv.csv>`_ + - `CSV 8t4c NFV Service Density NDR comparison <../../_static/vpp/performance-changes-nfv-2n-skx-8t4c-ndr-csv.csv>`_ PDR Comparison `````````````` @@ -90,8 +90,8 @@ Comparison tables in HTML, ASCII and CSV formats: - `HTML 4t2c PDR comparison <performance-changes-2n-skx-4t2c-pdr.html>`_ - `ASCII 2t1c PDR comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-pdr.txt>`_ - `ASCII 4t2c PDR comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-pdr.txt>`_ - - `CSV 2t1c PDR comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-pdr.csv>`_ - - `CSV 4t2c PDR comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-pdr.csv>`_ + - `CSV 2t1c PDR comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-pdr-csv.csv>`_ + - `CSV 4t2c PDR comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-pdr-csv.csv>`_ .. - `HTML 2t1c NFV Service Density PDR comparison <performance-changes-nfv-2n-skx-2t1c-pdr.html>`_ @@ -100,9 +100,9 @@ Comparison tables in HTML, ASCII and CSV formats: - `ASCII 2t1c NFV Service Density PDR comparison <../../_static/vpp/performance-changes-nfv-2n-skx-2t1c-pdr.txt>`_ - `ASCII 4t2c NFV Service Density PDR comparison <../../_static/vpp/performance-changes-nfv-2n-skx-4t2c-pdr.txt>`_ - `ASCII 8t4c NFV Service Density PDR comparison <../../_static/vpp/performance-changes-nfv-2n-skx-8t4c-pdr.txt>`_ - - `CSV 2t1c NFV Service Density PDR comparison <../../_static/vpp/performance-changes-nfv-2n-skx-2t1c-pdr.csv>`_ - - `CSV 4t2c NFV Service Density PDR comparison <../../_static/vpp/performance-changes-nfv-2n-skx-4t2c-pdr.csv>`_ - - `CSV 8t4c NFV Service Density PDR comparison <../../_static/vpp/performance-changes-nfv-2n-skx-8t4c-pdr.csv>`_ + - `CSV 2t1c NFV Service Density PDR comparison <../../_static/vpp/performance-changes-nfv-2n-skx-2t1c-pdr-csv.csv>`_ + - `CSV 4t2c NFV Service Density PDR comparison <../../_static/vpp/performance-changes-nfv-2n-skx-4t2c-pdr-csv.csv>`_ + - `CSV 8t4c NFV Service Density PDR comparison <../../_static/vpp/performance-changes-nfv-2n-skx-8t4c-pdr-csv.csv>`_ MRR Comparison `````````````` @@ -115,9 +115,9 @@ Comparison tables in HTML, ASCII and CSV formats: - `ASCII 2t1c MRR comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-mrr.txt>`_ - `ASCII 4t2c MRR comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-mrr.txt>`_ - `ASCII 8t4c MRR comparison <../../_static/vpp/performance-changes-2n-skx-8t4c-mrr.txt>`_ - - `CSV 2t1c MRR comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-mrr.csv>`_ - - `CSV 4t2c MRR comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-mrr.csv>`_ - - `CSV 8t4c MRR comparison <../../_static/vpp/performance-changes-2n-skx-8t4c-mrr.csv>`_ + - `CSV 2t1c MRR comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-mrr-csv.csv>`_ + - `CSV 4t2c MRR comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-mrr-csv.csv>`_ + - `CSV 8t4c MRR comparison <../../_static/vpp/performance-changes-2n-skx-8t4c-mrr-csv.csv>`_ .. - `HTML 2t1c NFV Service Density MRR comparison <performance-changes-nfv-2n-skx-2t1c-mrr.html>`_ @@ -126,9 +126,9 @@ Comparison tables in HTML, ASCII and CSV formats: - `ASCII 2t1c NFV Service Density MRR comparison <../../_static/vpp/performance-changes-nfv-2n-skx-2t1c-mrr.txt>`_ - `ASCII 4t2c NFV Service Density MRR comparison <../../_static/vpp/performance-changes-nfv-2n-skx-4t2c-mrr.txt>`_ - `ASCII 8t4c NFV Service Density MRR comparison <../../_static/vpp/performance-changes-nfv-2n-skx-8t4c-mrr.txt>`_ - - `CSV 2t1c NFV Service Density MRR comparison <../../_static/vpp/performance-changes-nfv-2n-skx-2t1c-mrr.csv>`_ - - `CSV 4t2c NFV Service Density MRR comparison <../../_static/vpp/performance-changes-nfv-2n-skx-4t2c-mrr.csv>`_ - - `CSV 8t4c NFV Service Density MRR comparison <../../_static/vpp/performance-changes-nfv-2n-skx-8t4c-mrr.csv>`_ + - `CSV 2t1c NFV Service Density MRR comparison <../../_static/vpp/performance-changes-nfv-2n-skx-2t1c-mrr-csv.csv>`_ + - `CSV 4t2c NFV Service Density MRR comparison <../../_static/vpp/performance-changes-nfv-2n-skx-4t2c-mrr-csv.csv>`_ + - `CSV 8t4c NFV Service Density MRR comparison <../../_static/vpp/performance-changes-nfv-2n-skx-8t4c-mrr-csv.csv>`_ 3n-skx ~~~~~~ @@ -161,8 +161,8 @@ Comparison tables in HTML, ASCII and CSV formats: - `HTML 4t2c NDR comparison <performance-changes-3n-skx-4t2c-ndr.html>`_ - `ASCII 2t1c NDR comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-ndr.txt>`_ - `ASCII 4t2c NDR comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-ndr.txt>`_ - - `CSV 2t1c NDR comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-ndr.csv>`_ - - `CSV 4t2c NDR comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-ndr.csv>`_ + - `CSV 2t1c NDR comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-ndr-csv.csv>`_ + - `CSV 4t2c NDR comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-ndr-csv.csv>`_ PDR Comparison `````````````` @@ -173,8 +173,8 @@ Comparison tables in HTML, ASCII and CSV formats: - `HTML 4t2c PDR comparison <performance-changes-3n-skx-4t2c-pdr.html>`_ - `ASCII 2t1c PDR comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-pdr.txt>`_ - `ASCII 4t2c PDR comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-pdr.txt>`_ - - `CSV 2t1c PDR comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-pdr.csv>`_ - - `CSV 4t2c PDR comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-pdr.csv>`_ + - `CSV 2t1c PDR comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-pdr-csv.csv>`_ + - `CSV 4t2c PDR comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-pdr-csv.csv>`_ MRR Comparison `````````````` @@ -187,9 +187,9 @@ Comparison tables in HTML, ASCII and CSV formats: - `ASCII 2t1c MRR comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-mrr.txt>`_ - `ASCII 4t2c MRR comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-mrr.txt>`_ - `ASCII 8t4c MRR comparison <../../_static/vpp/performance-changes-3n-skx-8t4c-mrr.txt>`_ - - `CSV 2t1c MRR comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-mrr.csv>`_ - - `CSV 4t2c MRR comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-mrr.csv>`_ - - `CSV 8t4c MRR comparison <../../_static/vpp/performance-changes-3n-skx-8t4c-mrr.csv>`_ + - `CSV 2t1c MRR comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-mrr-csv.csv>`_ + - `CSV 4t2c MRR comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-mrr-csv.csv>`_ + - `CSV 8t4c MRR comparison <../../_static/vpp/performance-changes-3n-skx-8t4c-mrr-csv.csv>`_ 2n-clx-xxv710 ~~~~~~~~~~~~~ @@ -203,8 +203,8 @@ Comparison tables in HTML, ASCII and CSV formats: - `HTML 4t2c NDR comparison <performance-changes-2n-clx-xxv710-4t2c-ndr.html>`_ - `ASCII 2t1c NDR comparison <../../_static/vpp/performance-changes-2n-clx-xxv710-2t1c-ndr.txt>`_ - `ASCII 4t2c NDR comparison <../../_static/vpp/performance-changes-2n-clx-xxv710-4t2c-ndr.txt>`_ - - `CSV 2t1c NDR comparison <../../_static/vpp/performance-changes-2n-clx-xxv710-2t1c-ndr.csv>`_ - - `CSV 4t2c NDR comparison <../../_static/vpp/performance-changes-2n-clx-xxv710-4t2c-ndr.csv>`_ + - `CSV 2t1c NDR comparison <../../_static/vpp/performance-changes-2n-clx-xxv710-2t1c-ndr-csv.csv>`_ + - `CSV 4t2c NDR comparison <../../_static/vpp/performance-changes-2n-clx-xxv710-4t2c-ndr-csv.csv>`_ .. - `HTML 2t1c NFV Service Density NDR comparison <performance-changes-nfv-2n-clx-xxv710-2t1c-ndr.html>`_ @@ -213,9 +213,9 @@ Comparison tables in HTML, ASCII and CSV formats: - `ASCII 2t1c NFV Service Density NDR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-xxv710-2t1c-ndr.txt>`_ - `ASCII 4t2c NFV Service Density NDR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-xxv710-4t2c-ndr.txt>`_ - `ASCII 8t4c NFV Service Density NDR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-xxv710-8t4c-ndr.txt>`_ - - `CSV 2t1c NFV Service Density NDR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-xxv710-2t1c-ndr.csv>`_ - - `CSV 4t2c NFV Service Density NDR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-xxv710-4t2c-ndr.csv>`_ - - `CSV 8t4c NFV Service Density NDR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-xxv710-8t4c-ndr.csv>`_ + - `CSV 2t1c NFV Service Density NDR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-xxv710-2t1c-ndr-csv.csv>`_ + - `CSV 4t2c NFV Service Density NDR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-xxv710-4t2c-ndr-csv.csv>`_ + - `CSV 8t4c NFV Service Density NDR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-xxv710-8t4c-ndr-csv.csv>`_ PDR Comparison `````````````` @@ -226,8 +226,8 @@ Comparison tables in HTML, ASCII and CSV formats: - `HTML 4t2c PDR comparison <performance-changes-2n-clx-xxv710-4t2c-pdr.html>`_ - `ASCII 2t1c PDR comparison <../../_static/vpp/performance-changes-2n-clx-xxv710-2t1c-pdr.txt>`_ - `ASCII 4t2c PDR comparison <../../_static/vpp/performance-changes-2n-clx-xxv710-4t2c-pdr.txt>`_ - - `CSV 2t1c PDR comparison <../../_static/vpp/performance-changes-2n-clx-xxv710-2t1c-pdr.csv>`_ - - `CSV 4t2c PDR comparison <../../_static/vpp/performance-changes-2n-clx-xxv710-4t2c-pdr.csv>`_ + - `CSV 2t1c PDR comparison <../../_static/vpp/performance-changes-2n-clx-xxv710-2t1c-pdr-csv.csv>`_ + - `CSV 4t2c PDR comparison <../../_static/vpp/performance-changes-2n-clx-xxv710-4t2c-pdr-csv.csv>`_ .. - `HTML 2t1c NFV Service Density PDR comparison <performance-changes-nfv-2n-clx-xxv710-2t1c-pdr.html>`_ @@ -236,9 +236,9 @@ Comparison tables in HTML, ASCII and CSV formats: - `ASCII 2t1c NFV Service Density PDR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-xxv710-2t1c-pdr.txt>`_ - `ASCII 4t2c NFV Service Density PDR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-xxv710-4t2c-pdr.txt>`_ - `ASCII 8t4c NFV Service Density PDR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-xxv710-8t4c-pdr.txt>`_ - - `CSV 2t1c NFV Service Density PDR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-xxv710-2t1c-pdr.csv>`_ - - `CSV 4t2c NFV Service Density PDR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-xxv710-4t2c-pdr.csv>`_ - - `CSV 8t4c NFV Service Density PDR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-xxv710-8t4c-pdr.csv>`_ + - `CSV 2t1c NFV Service Density PDR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-xxv710-2t1c-pdr-csv.csv>`_ + - `CSV 4t2c NFV Service Density PDR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-xxv710-4t2c-pdr-csv.csv>`_ + - `CSV 8t4c NFV Service Density PDR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-xxv710-8t4c-pdr-csv.csv>`_ MRR Comparison `````````````` @@ -251,9 +251,9 @@ Comparison tables in HTML, ASCII and CSV formats: - `ASCII 2t1c MRR comparison <../../_static/vpp/performance-changes-2n-clx-xxv710-2t1c-mrr.txt>`_ - `ASCII 4t2c MRR comparison <../../_static/vpp/performance-changes-2n-clx-xxv710-4t2c-mrr.txt>`_ - `ASCII 8t4c MRR comparison <../../_static/vpp/performance-changes-2n-clx-xxv710-8t4c-mrr.txt>`_ - - `CSV 2t1c MRR comparison <../../_static/vpp/performance-changes-2n-clx-xxv710-2t1c-mrr.csv>`_ - - `CSV 4t2c MRR comparison <../../_static/vpp/performance-changes-2n-clx-xxv710-4t2c-mrr.csv>`_ - - `CSV 8t4c MRR comparison <../../_static/vpp/performance-changes-2n-clx-xxv710-8t4c-mrr.csv>`_ + - `CSV 2t1c MRR comparison <../../_static/vpp/performance-changes-2n-clx-xxv710-2t1c-mrr-csv.csv>`_ + - `CSV 4t2c MRR comparison <../../_static/vpp/performance-changes-2n-clx-xxv710-4t2c-mrr-csv.csv>`_ + - `CSV 8t4c MRR comparison <../../_static/vpp/performance-changes-2n-clx-xxv710-8t4c-mrr-csv.csv>`_ .. - `HTML 2t1c NFV Service Density MRR comparison <performance-changes-nfv-2n-clx-xxv710-2t1c-mrr.html>`_ @@ -262,9 +262,9 @@ Comparison tables in HTML, ASCII and CSV formats: - `ASCII 2t1c NFV Service Density MRR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-xxv710-2t1c-mrr.txt>`_ - `ASCII 4t2c NFV Service Density MRR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-xxv710-4t2c-mrr.txt>`_ - `ASCII 8t4c NFV Service Density MRR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-xxv710-8t4c-mrr.txt>`_ - - `CSV 2t1c NFV Service Density MRR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-xxv710-2t1c-mrr.csv>`_ - - `CSV 4t2c NFV Service Density MRR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-xxv710-4t2c-mrr.csv>`_ - - `CSV 8t4c NFV Service Density MRR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-xxv710-8t4c-mrr.csv>`_ + - `CSV 2t1c NFV Service Density MRR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-xxv710-2t1c-mrr-csv.csv>`_ + - `CSV 4t2c NFV Service Density MRR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-xxv710-4t2c-mrr-csv.csv>`_ + - `CSV 8t4c NFV Service Density MRR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-xxv710-8t4c-mrr-csv.csv>`_ 2n-clx-x556 ~~~~~~~~~~~ @@ -278,8 +278,8 @@ Comparison tables in HTML, ASCII and CSV formats: - `HTML 4t2c NDR comparison <performance-changes-2n-clx-x556-4t2c-ndr.html>`_ - `ASCII 2t1c NDR comparison <../../_static/vpp/performance-changes-2n-clx-x556-2t1c-ndr.txt>`_ - `ASCII 4t2c NDR comparison <../../_static/vpp/performance-changes-2n-clx-x556-4t2c-ndr.txt>`_ - - `CSV 2t1c NDR comparison <../../_static/vpp/performance-changes-2n-clx-x556-2t1c-ndr.csv>`_ - - `CSV 4t2c NDR comparison <../../_static/vpp/performance-changes-2n-clx-x556-4t2c-ndr.csv>`_ + - `CSV 2t1c NDR comparison <../../_static/vpp/performance-changes-2n-clx-x556-2t1c-ndr-csv.csv>`_ + - `CSV 4t2c NDR comparison <../../_static/vpp/performance-changes-2n-clx-x556-4t2c-ndr-csv.csv>`_ .. - `HTML 2t1c NFV Service Density NDR comparison <performance-changes-nfv-2n-clx-x556-2t1c-ndr.html>`_ @@ -288,9 +288,9 @@ Comparison tables in HTML, ASCII and CSV formats: - `ASCII 2t1c NFV Service Density NDR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-x556-2t1c-ndr.txt>`_ - `ASCII 4t2c NFV Service Density NDR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-x556-4t2c-ndr.txt>`_ - `ASCII 8t4c NFV Service Density NDR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-x556-8t4c-ndr.txt>`_ - - `CSV 2t1c NFV Service Density NDR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-x556-2t1c-ndr.csv>`_ - - `CSV 4t2c NFV Service Density NDR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-x556-4t2c-ndr.csv>`_ - - `CSV 8t4c NFV Service Density NDR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-x556-8t4c-ndr.csv>`_ + - `CSV 2t1c NFV Service Density NDR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-x556-2t1c-ndr-csv.csv>`_ + - `CSV 4t2c NFV Service Density NDR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-x556-4t2c-ndr-csv.csv>`_ + - `CSV 8t4c NFV Service Density NDR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-x556-8t4c-ndr-csv.csv>`_ PDR Comparison `````````````` @@ -301,8 +301,8 @@ Comparison tables in HTML, ASCII and CSV formats: - `HTML 4t2c PDR comparison <performance-changes-2n-clx-x556-4t2c-pdr.html>`_ - `ASCII 2t1c PDR comparison <../../_static/vpp/performance-changes-2n-clx-x556-2t1c-pdr.txt>`_ - `ASCII 4t2c PDR comparison <../../_static/vpp/performance-changes-2n-clx-x556-4t2c-pdr.txt>`_ - - `CSV 2t1c PDR comparison <../../_static/vpp/performance-changes-2n-clx-x556-2t1c-pdr.csv>`_ - - `CSV 4t2c PDR comparison <../../_static/vpp/performance-changes-2n-clx-x556-4t2c-pdr.csv>`_ + - `CSV 2t1c PDR comparison <../../_static/vpp/performance-changes-2n-clx-x556-2t1c-pdr-csv.csv>`_ + - `CSV 4t2c PDR comparison <../../_static/vpp/performance-changes-2n-clx-x556-4t2c-pdr-csv.csv>`_ .. - `HTML 2t1c NFV Service Density PDR comparison <performance-changes-nfv-2n-clx-x556-2t1c-pdr.html>`_ @@ -311,9 +311,9 @@ Comparison tables in HTML, ASCII and CSV formats: - `ASCII 2t1c NFV Service Density PDR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-x556-2t1c-pdr.txt>`_ - `ASCII 4t2c NFV Service Density PDR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-x556-4t2c-pdr.txt>`_ - `ASCII 8t4c NFV Service Density PDR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-x556-8t4c-pdr.txt>`_ - - `CSV 2t1c NFV Service Density PDR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-x556-2t1c-pdr.csv>`_ - - `CSV 4t2c NFV Service Density PDR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-x556-4t2c-pdr.csv>`_ - - `CSV 8t4c NFV Service Density PDR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-x556-8t4c-pdr.csv>`_ + - `CSV 2t1c NFV Service Density PDR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-x556-2t1c-pdr-csv.csv>`_ + - `CSV 4t2c NFV Service Density PDR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-x556-4t2c-pdr-csv.csv>`_ + - `CSV 8t4c NFV Service Density PDR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-x556-8t4c-pdr-csv.csv>`_ MRR Comparison `````````````` @@ -326,9 +326,9 @@ Comparison tables in HTML, ASCII and CSV formats: - `ASCII 2t1c MRR comparison <../../_static/vpp/performance-changes-2n-clx-x556-2t1c-mrr.txt>`_ - `ASCII 4t2c MRR comparison <../../_static/vpp/performance-changes-2n-clx-x556-4t2c-mrr.txt>`_ - `ASCII 8t4c MRR comparison <../../_static/vpp/performance-changes-2n-clx-x556-8t4c-mrr.txt>`_ - - `CSV 2t1c MRR comparison <../../_static/vpp/performance-changes-2n-clx-x556-2t1c-mrr.csv>`_ - - `CSV 4t2c MRR comparison <../../_static/vpp/performance-changes-2n-clx-x556-4t2c-mrr.csv>`_ - - `CSV 8t4c MRR comparison <../../_static/vpp/performance-changes-2n-clx-x556-8t4c-mrr.csv>`_ + - `CSV 2t1c MRR comparison <../../_static/vpp/performance-changes-2n-clx-x556-2t1c-mrr-csv.csv>`_ + - `CSV 4t2c MRR comparison <../../_static/vpp/performance-changes-2n-clx-x556-4t2c-mrr-csv.csv>`_ + - `CSV 8t4c MRR comparison <../../_static/vpp/performance-changes-2n-clx-x556-8t4c-mrr-csv.csv>`_ .. - `HTML 2t1c NFV Service Density MRR comparison <performance-changes-nfv-2n-clx-x556-2t1c-mrr.html>`_ @@ -337,9 +337,9 @@ Comparison tables in HTML, ASCII and CSV formats: - `ASCII 2t1c NFV Service Density MRR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-x556-2t1c-mrr.txt>`_ - `ASCII 4t2c NFV Service Density MRR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-x556-4t2c-mrr.txt>`_ - `ASCII 8t4c NFV Service Density MRR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-x556-8t4c-mrr.txt>`_ - - `CSV 2t1c NFV Service Density MRR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-x556-2t1c-mrr.csv>`_ - - `CSV 4t2c NFV Service Density MRR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-x556-4t2c-mrr.csv>`_ - - `CSV 8t4c NFV Service Density MRR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-x556-8t4c-mrr.csv>`_ + - `CSV 2t1c NFV Service Density MRR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-x556-2t1c-mrr-csv.csv>`_ + - `CSV 4t2c NFV Service Density MRR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-x556-4t2c-mrr-csv.csv>`_ + - `CSV 8t4c NFV Service Density MRR comparison <../../_static/vpp/performance-changes-nfv-2n-clx-x556-8t4c-mrr-csv.csv>`_ 3n-hsw ~~~~~~ @@ -353,8 +353,8 @@ Comparison tables in HTML, ASCII and CSV formats: - `HTML 2t2c NDR comparison <performance-changes-3n-hsw-2t2c-ndr.html>`_ - `ASCII 1t1c NDR comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-ndr.txt>`_ - `ASCII 2t2c NDR comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-ndr.txt>`_ - - `CSV 1t1c NDR comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-ndr.csv>`_ - - `CSV 2t2c NDR comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-ndr.csv>`_ + - `CSV 1t1c NDR comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-ndr-csv.csv>`_ + - `CSV 2t2c NDR comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-ndr-csv.csv>`_ PDR Comparison `````````````` @@ -365,8 +365,8 @@ Comparison tables in HTML, ASCII and CSV formats: - `HTML 2t2c PDR comparison <performance-changes-3n-hsw-2t2c-pdr.html>`_ - `ASCII 1t1c PDR comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-pdr.txt>`_ - `ASCII 2t2c PDR comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-pdr.txt>`_ - - `CSV 1t1c PDR comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-pdr.csv>`_ - - `CSV 2t2c PDR comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-pdr.csv>`_ + - `CSV 1t1c PDR comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-pdr-csv.csv>`_ + - `CSV 2t2c PDR comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-pdr-csv.csv>`_ MRR Comparison `````````````` @@ -379,9 +379,9 @@ Comparison tables in HTML, ASCII and CSV formats: - `ASCII 1t1c MRR comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-mrr.txt>`_ - `ASCII 2t2c MRR comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-mrr.txt>`_ - `ASCII 4t4c MRR comparison <../../_static/vpp/performance-changes-3n-hsw-4t4c-mrr.txt>`_ - - `CSV 1t1c MRR comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-mrr.csv>`_ - - `CSV 2t2c MRR comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-mrr.csv>`_ - - `CSV 4t4c MRR comparison <../../_static/vpp/performance-changes-3n-hsw-4t4c-mrr.csv>`_ + - `CSV 1t1c MRR comparison <../../_static/vpp/performance-changes-3n-hsw-1t1c-mrr-csv.csv>`_ + - `CSV 2t2c MRR comparison <../../_static/vpp/performance-changes-3n-hsw-2t2c-mrr-csv.csv>`_ + - `CSV 4t4c MRR comparison <../../_static/vpp/performance-changes-3n-hsw-4t4c-mrr-csv.csv>`_ 2n-dnv ~~~~~~ @@ -395,8 +395,8 @@ Comparison tables in HTML, ASCII and CSV formats: - `HTML 2t2c NDR comparison <performance-changes-2n-dnv-2t2c-ndr.html>`_ - `ASCII 1t1c NDR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-ndr.txt>`_ - `ASCII 2t2c NDR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-ndr.txt>`_ - - `CSV 1t1c NDR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-ndr.csv>`_ - - `CSV 2t2c NDR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-ndr.csv>`_ + - `CSV 1t1c NDR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-ndr-csv.csv>`_ + - `CSV 2t2c NDR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-ndr-csv.csv>`_ PDR Comparison `````````````` @@ -407,8 +407,8 @@ Comparison tables in HTML, ASCII and CSV formats: - `HTML 2t2c PDR comparison <performance-changes-2n-dnv-2t2c-pdr.html>`_ - `ASCII 1t1c PDR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-pdr.txt>`_ - `ASCII 2t2c PDR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-pdr.txt>`_ - - `CSV 1t1c PDR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-pdr.csv>`_ - - `CSV 2t2c PDR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-pdr.csv>`_ + - `CSV 1t1c PDR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-pdr-csv.csv>`_ + - `CSV 2t2c PDR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-pdr-csv.csv>`_ MRR Comparison `````````````` @@ -421,9 +421,9 @@ Comparison tables in HTML, ASCII and CSV formats: - `ASCII 1t1c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-mrr.txt>`_ - `ASCII 2t2c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-mrr.txt>`_ - `ASCII 4t4c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-4t4c-mrr.txt>`_ - - `CSV 1t1c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-mrr.csv>`_ - - `CSV 2t2c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-mrr.csv>`_ - - `CSV 4t4c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-4t4c-mrr.csv>`_ + - `CSV 1t1c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-1t1c-mrr-csv.csv>`_ + - `CSV 2t2c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-2t2c-mrr-csv.csv>`_ + - `CSV 4t4c MRR comparison <../../_static/vpp/performance-changes-2n-dnv-4t4c-mrr-csv.csv>`_ 3n-dnv ~~~~~~ @@ -437,8 +437,8 @@ Comparison tables in HTML, ASCII and CSV formats: - `HTML 2t2c NDR comparison <performance-changes-3n-dnv-2t2c-ndr.html>`_ - `ASCII 1t1c NDR comparison <../../_static/vpp/performance-changes-3n-dnv-1t1c-ndr.txt>`_ - `ASCII 2t2c NDR comparison <../../_static/vpp/performance-changes-3n-dnv-2t2c-ndr.txt>`_ - - `CSV 1t1c NDR comparison <../../_static/vpp/performance-changes-3n-dnv-1t1c-ndr.csv>`_ - - `CSV 2t2c NDR comparison <../../_static/vpp/performance-changes-3n-dnv-2t2c-ndr.csv>`_ + - `CSV 1t1c NDR comparison <../../_static/vpp/performance-changes-3n-dnv-1t1c-ndr-csv.csv>`_ + - `CSV 2t2c NDR comparison <../../_static/vpp/performance-changes-3n-dnv-2t2c-ndr-csv.csv>`_ PDR Comparison `````````````` @@ -449,8 +449,8 @@ Comparison tables in HTML, ASCII and CSV formats: - `HTML 2t2c PDR comparison <performance-changes-3n-dnv-2t2c-pdr.html>`_ - `ASCII 1t1c PDR comparison <../../_static/vpp/performance-changes-3n-dnv-1t1c-pdr.txt>`_ - `ASCII 2t2c PDR comparison <../../_static/vpp/performance-changes-3n-dnv-2t2c-pdr.txt>`_ - - `CSV 1t1c PDR comparison <../../_static/vpp/performance-changes-3n-dnv-1t1c-pdr.csv>`_ - - `CSV 2t2c PDR comparison <../../_static/vpp/performance-changes-3n-dnv-2t2c-pdr.csv>`_ + - `CSV 1t1c PDR comparison <../../_static/vpp/performance-changes-3n-dnv-1t1c-pdr-csv.csv>`_ + - `CSV 2t2c PDR comparison <../../_static/vpp/performance-changes-3n-dnv-2t2c-pdr-csv.csv>`_ MRR Comparison `````````````` @@ -463,9 +463,9 @@ Comparison tables in HTML, ASCII and CSV formats: - `ASCII 1t1c MRR comparison <../../_static/vpp/performance-changes-3n-dnv-1t1c-mrr.txt>`_ - `ASCII 2t2c MRR comparison <../../_static/vpp/performance-changes-3n-dnv-2t2c-mrr.txt>`_ - `ASCII 4t4c MRR comparison <../../_static/vpp/performance-changes-3n-dnv-4t4c-mrr.txt>`_ - - `CSV 1t1c MRR comparison <../../_static/vpp/performance-changes-3n-dnv-1t1c-mrr.csv>`_ - - `CSV 2t2c MRR comparison <../../_static/vpp/performance-changes-3n-dnv-2t2c-mrr.csv>`_ - - `CSV 4t4c MRR comparison <../../_static/vpp/performance-changes-3n-dnv-4t4c-mrr.csv>`_ + - `CSV 1t1c MRR comparison <../../_static/vpp/performance-changes-3n-dnv-1t1c-mrr-csv.csv>`_ + - `CSV 2t2c MRR comparison <../../_static/vpp/performance-changes-3n-dnv-2t2c-mrr-csv.csv>`_ + - `CSV 4t4c MRR comparison <../../_static/vpp/performance-changes-3n-dnv-4t4c-mrr-csv.csv>`_ 3n-tsh ~~~~~~ @@ -479,8 +479,8 @@ Comparison tables in HTML, ASCII and CSV formats: - `HTML 2t2c NDR comparison <performance-changes-3n-tsh-2t2c-ndr.html>`_ - `ASCII 1t1c NDR comparison <../../_static/vpp/performance-changes-3n-tsh-1t1c-ndr.txt>`_ - `ASCII 2t2c NDR comparison <../../_static/vpp/performance-changes-3n-tsh-2t2c-ndr.txt>`_ - - `CSV 1t1c NDR comparison <../../_static/vpp/performance-changes-3n-tsh-1t1c-ndr.csv>`_ - - `CSV 2t2c NDR comparison <../../_static/vpp/performance-changes-3n-tsh-2t2c-ndr.csv>`_ + - `CSV 1t1c NDR comparison <../../_static/vpp/performance-changes-3n-tsh-1t1c-ndr-csv.csv>`_ + - `CSV 2t2c NDR comparison <../../_static/vpp/performance-changes-3n-tsh-2t2c-ndr-csv.csv>`_ PDR Comparison `````````````` @@ -491,8 +491,8 @@ Comparison tables in HTML, ASCII and CSV formats: - `HTML 2t2c PDR comparison <performance-changes-3n-tsh-2t2c-pdr.html>`_ - `ASCII 1t1c PDR comparison <../../_static/vpp/performance-changes-3n-tsh-1t1c-pdr.txt>`_ - `ASCII 2t2c PDR comparison <../../_static/vpp/performance-changes-3n-tsh-2t2c-pdr.txt>`_ - - `CSV 1t1c PDR comparison <../../_static/vpp/performance-changes-3n-tsh-1t1c-pdr.csv>`_ - - `CSV 2t2c PDR comparison <../../_static/vpp/performance-changes-3n-tsh-2t2c-pdr.csv>`_ + - `CSV 1t1c PDR comparison <../../_static/vpp/performance-changes-3n-tsh-1t1c-pdr-csv.csv>`_ + - `CSV 2t2c PDR comparison <../../_static/vpp/performance-changes-3n-tsh-2t2c-pdr-csv.csv>`_ MRR Comparison `````````````` @@ -505,6 +505,6 @@ Comparison tables in HTML, ASCII and CSV formats: - `ASCII 1t1c MRR comparison <../../_static/vpp/performance-changes-3n-tsh-1t1c-mrr.txt>`_ - `ASCII 2t2c MRR comparison <../../_static/vpp/performance-changes-3n-tsh-2t2c-mrr.txt>`_ - `ASCII 4t4c MRR comparison <../../_static/vpp/performance-changes-3n-tsh-4t4c-mrr.txt>`_ - - `CSV 1t1c MRR comparison <../../_static/vpp/performance-changes-3n-tsh-1t1c-mrr.csv>`_ - - `CSV 2t2c MRR comparison <../../_static/vpp/performance-changes-3n-tsh-2t2c-mrr.csv>`_ - - `CSV 4t4c MRR comparison <../../_static/vpp/performance-changes-3n-tsh-4t4c-mrr.csv>`_ + - `CSV 1t1c MRR comparison <../../_static/vpp/performance-changes-3n-tsh-1t1c-mrr-csv.csv>`_ + - `CSV 2t2c MRR comparison <../../_static/vpp/performance-changes-3n-tsh-2t2c-mrr-csv.csv>`_ + - `CSV 4t4c MRR comparison <../../_static/vpp/performance-changes-3n-tsh-4t4c-mrr-csv.csv>`_ diff --git a/docs/report/vpp_performance_tests/comparisons/nic_comparison.rst b/docs/report/vpp_performance_tests/comparisons/nic_comparison.rst index 5e5a4d3ba1..51905c9d0e 100644 --- a/docs/report/vpp_performance_tests/comparisons/nic_comparison.rst +++ b/docs/report/vpp_performance_tests/comparisons/nic_comparison.rst @@ -40,8 +40,8 @@ Comparison tables in HTML, ASCII and CSV formats: - `HTML 4t2c NDR Intel-x710 and Intel-xxv710 comparison <performance-changes-3n-skx-4t2c-nics-ndr.html>`_ - `ASCII 2t1c NDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-nics-ndr.txt>`_ - `ASCII 4t2c NDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-nics-ndr.txt>`_ - - `CSV 2t1c NDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-nics-ndr.csv>`_ - - `CSV 4t2c NDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-nics-ndr.csv>`_ + - `CSV 2t1c NDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-nics-ndr-csv.csv>`_ + - `CSV 4t2c NDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-nics-ndr-csv.csv>`_ PDR Comparison `````````````` @@ -52,8 +52,8 @@ Comparison tables in HTML, ASCII and CSV formats: - `HTML 4t2c PDR Intel-x710 and Intel-xxv710 comparison <performance-changes-3n-skx-4t2c-nics-pdr.html>`_ - `ASCII 2t1c PDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-nics-pdr.txt>`_ - `ASCII 4t2c PDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-nics-pdr.txt>`_ - - `CSV 2t1c PDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-nics-pdr.csv>`_ - - `CSV 4t2c PDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-nics-pdr.csv>`_ + - `CSV 2t1c PDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-nics-pdr-csv.csv>`_ + - `CSV 4t2c PDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-nics-pdr-csv.csv>`_ MRR Comparison `````````````` @@ -66,9 +66,9 @@ Comparison tables in HTML, ASCII and CSV formats: - `ASCII 2t1c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-nics-mrr.txt>`_ - `ASCII 4t2c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-nics-mrr.txt>`_ - `ASCII 8t4c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-8t4c-nics-mrr.txt>`_ - - `CSV 2t1c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-nics-mrr.csv>`_ - - `CSV 4t2c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-nics-mrr.csv>`_ - - `CSV 8t4c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-8t4c-nics-mrr.csv>`_ + - `CSV 2t1c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-2t1c-nics-mrr-csv.csv>`_ + - `CSV 4t2c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-4t2c-nics-mrr-csv.csv>`_ + - `CSV 8t4c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-3n-skx-8t4c-nics-mrr-csv.csv>`_ 2n-skx ~~~~~~ @@ -82,8 +82,8 @@ Comparison tables in HTML, ASCII and CSV formats: - `HTML 4t2c NDR Intel-x710 and Intel-xxv710 comparison <performance-changes-2n-skx-4t2c-nics-ndr.html>`_ - `ASCII 2t1c NDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-nics-ndr.txt>`_ - `ASCII 4t2c NDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-nics-ndr.txt>`_ - - `CSV 2t1c NDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-nics-ndr.csv>`_ - - `CSV 4t2c NDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-nics-ndr.csv>`_ + - `CSV 2t1c NDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-nics-ndr-csv.csv>`_ + - `CSV 4t2c NDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-nics-ndr-csv.csv>`_ PDR Comparison `````````````` @@ -94,8 +94,8 @@ Comparison tables in HTML, ASCII and CSV formats: - `HTML 4t2c PDR Intel-x710 and Intel-xxv710 comparison <performance-changes-2n-skx-4t2c-nics-pdr.html>`_ - `ASCII 2t1c PDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-nics-pdr.txt>`_ - `ASCII 4t2c PDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-nics-pdr.txt>`_ - - `CSV 2t1c PDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-nics-pdr.csv>`_ - - `CSV 4t2c PDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-nics-pdr.csv>`_ + - `CSV 2t1c PDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-nics-pdr-csv.csv>`_ + - `CSV 4t2c PDR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-nics-pdr-csv.csv>`_ MRR Comparison `````````````` @@ -108,6 +108,6 @@ Comparison tables in HTML, ASCII and CSV formats: - `ASCII 2t1c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-nics-mrr.txt>`_ - `ASCII 4t2c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-nics-mrr.txt>`_ - `ASCII 8t4c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-8t4c-nics-mrr.txt>`_ - - `CSV 2t1c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-nics-mrr.csv>`_ - - `CSV 4t2c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-nics-mrr.csv>`_ - - `CSV 8t4c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-8t4c-nics-mrr.csv>`_ + - `CSV 2t1c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-nics-mrr-csv.csv>`_ + - `CSV 4t2c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-nics-mrr-csv.csv>`_ + - `CSV 8t4c MRR Intel-x710 and Intel-xxv710 comparison <../../_static/vpp/performance-changes-2n-skx-8t4c-nics-mrr-csv.csv>`_ diff --git a/resources/tools/presentation/generator_tables.py b/resources/tools/presentation/generator_tables.py index 29a398a19f..2d4f87fc95 100644 --- a/resources/tools/presentation/generator_tables.py +++ b/resources/tools/presentation/generator_tables.py @@ -1641,6 +1641,7 @@ def table_comparison(table, input_data): tbl_lst.append(row) comparisons = table.get(u"comparisons", None) + rcas = list() if comparisons and isinstance(comparisons, list): for idx, comp in enumerate(comparisons): try: @@ -1650,13 +1651,33 @@ def table_comparison(table, input_data): logging.warning(u"Comparison: No references defined! Skipping.") comparisons.pop(idx) continue - if not (0 < col_ref <= len(cols) and - 0 < col_cmp <= len(cols)) or \ - col_ref == col_cmp: + if not (0 < col_ref <= len(cols) and 0 < col_cmp <= len(cols) or + col_ref == col_cmp): logging.warning(f"Wrong values of reference={col_ref} " f"and/or compare={col_cmp}. Skipping.") comparisons.pop(idx) continue + rca_file_name = comp.get(u"rca-file", None) + if rca_file_name: + try: + with open(rca_file_name, u"r") as file_handler: + rcas.append( + { + u"title": f"RCA{idx + 1}", + u"data": load(file_handler, Loader=FullLoader) + } + ) + except (YAMLError, IOError) as err: + logging.warning( + f"The RCA file {rca_file_name} does not exist or " + f"it is corrupted!" + ) + logging.debug(repr(err)) + rcas.append(None) + else: + rcas.append(None) + else: + comparisons = None tbl_cmp_lst = list() if comparisons: @@ -1679,11 +1700,10 @@ def table_comparison(table, input_data): ) if delta is None: break - else: - new_row.append({ - u"mean": delta * 1e6, - u"stdev": d_stdev * 1e6 - }) + new_row.append({ + u"mean": delta * 1e6, + u"stdev": d_stdev * 1e6 + }) else: break else: @@ -1695,25 +1715,6 @@ def table_comparison(table, input_data): except TypeError as err: logging.warning(f"Empty data element in table\n{tbl_cmp_lst}\n{err}") - rcas = list() - rca_in = table.get(u"rca", None) - if rca_in and isinstance(rca_in, list): - for idx, itm in enumerate(rca_in): - try: - with open(itm.get(u"data", u""), u"r") as rca_file: - rcas.append( - { - u"title": itm.get(u"title", f"RCA{idx}"), - u"data": load(rca_file, Loader=FullLoader) - } - ) - except (YAMLError, IOError) as err: - logging.warning( - f"The RCA file {itm.get(u'data', u'')} does not exist or " - f"it is corrupted!" - ) - logging.debug(repr(err)) - tbl_for_csv = list() for line in tbl_cmp_lst: row = [line[0], ] @@ -1727,6 +1728,8 @@ def table_comparison(table, input_data): row.append(round(float(itm[u'mean']) / 1e6, 3)) row.append(round(float(itm[u'stdev']) / 1e6, 3)) for rca in rcas: + if rca is None: + continue rca_nr = rca[u"data"].get(row[0], u"-") row.append(f"[{rca_nr}]" if rca_nr != u"-" else u"-") tbl_for_csv.append(row) @@ -1742,7 +1745,9 @@ def table_comparison(table, input_data): header_csv.append( f"Stdev({comp.get(u'title', u'')})" ) - header_csv.extend([rca[u"title"] for rca in rcas]) + for rca in rcas: + if rca: + header_csv.append(rca[u"title"]) legend_lst = table.get(u"legend", None) if legend_lst is None: @@ -1751,10 +1756,11 @@ def table_comparison(table, input_data): legend = u"\n" + u"\n".join(legend_lst) + u"\n" footnote = u"" - if rcas: - footnote += u"\nRCA:\n" + if rcas and any(rcas): + footnote += u"\nRoot Cause Analysis:\n" for rca in rcas: - footnote += rca[u"data"].get(u"footnote", u"") + if rca: + footnote += f"{rca[u'data'].get(u'footnote', u'')}\n" csv_file_name = f"{table[u'output-file']}-csv.csv" with open(csv_file_name, u"wt", encoding='utf-8') as file_handler: @@ -1800,6 +1806,10 @@ def table_comparison(table, input_data): tbl_tmp.append(row) + header = [u"Test Case", ] + header.extend([col[u"title"] for col in cols]) + header.extend([comp.get(u"title", u"") for comp in comparisons]) + tbl_final = list() for line in tbl_tmp: row = [line[0], ] @@ -1810,18 +1820,27 @@ def table_comparison(table, input_data): itm_lst = itm.rsplit(u"\u00B1", 1) itm_lst[-1] = \ f"{u' ' * (max_lens[idx] - len(itm_lst[-1]))}{itm_lst[-1]}" - row.append(u"\u00B1".join(itm_lst)) - for rca in rcas: - rca_nr = rca[u"data"].get(row[0], u"-") - row.append(f"[{rca_nr}]" if rca_nr != u"-" else u"-") - + itm_str = u"\u00B1".join(itm_lst) + + if idx >= len(cols): + # Diffs + rca = rcas[idx - len(cols)] + if rca: + # Add rcas to diffs + rca_nr = rca[u"data"].get(row[0], None) + if rca_nr: + hdr_len = len(header[idx + 1]) - 1 + if hdr_len < 19: + hdr_len = 19 + rca_nr = f"[{rca_nr}]" + itm_str = ( + f"{u' ' * (4 - len(rca_nr))}{rca_nr}" + f"{u' ' * (hdr_len - 4 - len(itm_str))}" + f"{itm_str}" + ) + row.append(itm_str) tbl_final.append(row) - header = [u"Test Case", ] - header.extend([col[u"title"] for col in cols]) - header.extend([comp.get(u"title", u"") for comp in comparisons]) - header.extend([rca[u"title"] for rca in rcas]) - # Generate csv tables: csv_file_name = f"{table[u'output-file']}.csv" logging.info(f" Writing the file {csv_file_name}") diff --git a/resources/tools/presentation/specification.yaml b/resources/tools/presentation/specification.yaml index b3e949ecd0..d8e6b6837a 100644 --- a/resources/tools/presentation/specification.yaml +++ b/resources/tools/presentation/specification.yaml @@ -3332,23 +3332,18 @@ - title: "2001.0-4 vs 2001.0-2" reference: 2 compare: 3 + rca-file: "rca/rca-2005-3n-hsw-1t1c-pdr-rca1.yaml" - title: "2005.0-4 vs 2001.0-4" reference: 3 compare: 4 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-3n-hsw-1t1c-pdr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-3n-hsw-1t1c-pdr-rca2.yaml" + rca-file: "rca/rca-2005-3n-hsw-1t1c-pdr-rca2.yaml" legend: - '' - 'Test Case: Test cases executed for: PDR throughput on 3n-hsw with xl710.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' - '2001.0-4 vs 2001.0-2: Compares CSIT environment current vs previous with previous VPP code.' - '2005.0-4 vs 2001.0-4: Compares VPP code current vs previous in the current CSIT environment.' -# - 'RCA1: Root cause analysis reference for 2001.0-4 vs 2001.0-2.' -# - 'RCA2: Root cause analysis reference for 2005.0-4 vs 2001.0-4.' # VPP Performance Changes 3n-hsw 2t2c pdr - type: "table" @@ -3370,23 +3365,18 @@ - title: "2001.0-4 vs 2001.0-2" reference: 2 compare: 3 + rca-file: "rca/rca-2005-3n-hsw-2t2c-pdr-rca1.yaml" - title: "2005.0-4 vs 2001.0-4" reference: 3 compare: 4 -# rca: -# - title: "RCA2" -# data: "rca/rca-2005-3n-hsw-2t2c-pdr-rca1.yaml" -# - title: "RCA3" -# data: "rca/rca-2005-3n-hsw-2t2c-pdr-rca2.yaml" + rca-file: "rca/rca-2005-3n-hsw-2t2c-pdr-rca2.yaml" legend: - '' - 'Test Case: Test cases executed for: PDR throughput on 3n-hsw with xl710.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' - '2001.0-4 vs 2001.0-2: Compares CSIT environment current vs previous with previous VPP code.' - '2005.0-4 vs 2001.0-4: Compares VPP code current vs previous in the current CSIT environment.' -# - 'RCA1: Root cause analysis reference for 2001.0-4 vs 2001.0-2.' -# - 'RCA2: Root cause analysis reference for 2005.0-4 vs 2001.0-4.' # VPP Performance Changes 3n-hsw 1t1c ndr - type: "table" @@ -3408,19 +3398,16 @@ - title: "2001.0-4 vs 2001.0-2" reference: 2 compare: 3 + rca-file: "rca/rca-2005-3n-hsw-1t1c-ndr-rca2.yaml" - title: "2005.0-4 vs 2001.0-4" reference: 3 compare: 4 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-3n-hsw-1t1c-ndr-rca2.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-3n-hsw-1t1c-ndr-rca3.yaml" + rca-file: "rca/rca-2005-3n-hsw-1t1c-ndr-rca3.yaml" legend: - '' - 'Test Case: Test cases executed for: NDR throughput on 3n-hsw with xl710.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' - '2001.0-4 vs 2001.0-2: Compares CSIT environment current vs previous with previous VPP code.' - '2005.0-4 vs 2001.0-4: Compares VPP code current vs previous in the current CSIT environment.' # - 'RCA1: Root cause analysis reference for 2001.0-4 vs 2001.0-2.' @@ -3446,23 +3433,18 @@ - title: "2001.0-4 vs 2001.0-2" reference: 2 compare: 3 + rca-file: "rca/rca-2005-3n-hsw-2t2c-ndr-rca1.yaml" - title: "2005.0-4 vs 2001.0-4" reference: 3 compare: 4 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-3n-hsw-2t2c-ndr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-3n-hsw-2t2c-ndr-rca2.yaml" + rca-file: "rca/rca-2005-3n-hsw-2t2c-ndr-rca2.yaml" legend: - '' - 'Test Case: Test cases executed for: NDR throughput on 3n-hsw with xl710.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' - '2001.0-4 vs 2001.0-2: Compares CSIT environment current vs previous with previous VPP code.' - '2005.0-4 vs 2001.0-4: Compares VPP code current vs previous in the current CSIT environment.' -# - 'RCA1: Root cause analysis reference for 2001.0-4 vs 2001.0-2.' -# - 'RCA2: Root cause analysis reference for 2005.0-4 vs 2001.0-4.' # VPP Performance Changes 3n-hsw 1t1c pdr-full - type: "table" @@ -3486,30 +3468,23 @@ - title: "2005.0-4 vs 2005.0-3" reference: 4 compare: 5 + rca-file: "rca/rca-2005-3n-hsw-1t1c-pdr-rca1.yaml" - title: "2001.0-4 vs 2001.0-2" reference: 2 compare: 3 + rca-file: "rca/rca-2005-3n-hsw-1t1c-pdr-rca2.yaml" - title: "2005.0-4 vs 2001.0-4" reference: 3 compare: 5 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-3n-hsw-1t1c-pdr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-3n-hsw-1t1c-pdr-rca2.yaml" -# - title: "RCA3" -# data: "rca/rca-2005-3n-hsw-1t1c-pdr-rca3.yaml" + rca-file: "rca/rca-2005-3n-hsw-1t1c-pdr-rca3.yaml" legend: - '' - 'Test Case: Test cases executed for: PDR throughput on 3n-hsw with xl710.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' - '2005.0-4 vs 2005.0-3: Compares CSIT environments "4" to "3". "3" was discarded as it substantially changed TRex behaviour due to higher number of TRex workers used.' - '2001.0-4 vs 2001.0-2: Compares CSIT environment current vs previous with previous VPP code.' - '2005.0-4 vs 2001.0-4: Compares VPP code current vs previous in the current CSIT environment.' -# - 'RCA1: Root cause analysis reference for 2005.0-4 vs 2005.0-3.' -# - 'RCA2: Root cause analysis reference for 2001.0-4 vs 2001.0-2.' -# - 'RCA3: Root cause analysis reference for 2005.0-4 vs 2001.0-4.' # VPP Performance Changes 3n-hsw 2t2c pdr-full - type: "table" @@ -3533,30 +3508,23 @@ - title: "2005.0-4 vs 2005.0-3" reference: 4 compare: 5 + rca-file: "rca/rca-2005-3n-hsw-2t2c-pdr-rca1.yaml" - title: "2001.0-4 vs 2001.0-2" reference: 2 compare: 3 + rca-file: "rca/rca-2005-3n-hsw-2t2c-pdr-rca2.yaml" - title: "2005.0-4 vs 2001.0-4" reference: 3 compare: 5 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-3n-hsw-2t2c-pdr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-3n-hsw-2t2c-pdr-rca2.yaml" -# - title: "RCA3" -# data: "rca/rca-2005-3n-hsw-2t2c-pdr-rca3.yaml" + rca-file: "rca/rca-2005-3n-hsw-2t2c-pdr-rca3.yaml" legend: - '' - 'Test Case: Test cases executed for: PDR throughput on 3n-hsw with xl710.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' - '2005.0-4 vs 2005.0-3: Compares CSIT environments "4" to "3". "3" was discarded as it substantially changed TRex behaviour due to higher number of TRex workers used.' - '2001.0-4 vs 2001.0-2: Compares CSIT environment current vs previous with previous VPP code.' - '2005.0-4 vs 2001.0-4: Compares VPP code current vs previous in the current CSIT environment.' -# - 'RCA1: Root cause analysis reference for 2005.0-4 vs 2005.0-3.' -# - 'RCA2: Root cause analysis reference for 2001.0-4 vs 2001.0-2.' -# - 'RCA3: Root cause analysis reference for 2005.0-4 vs 2001.0-4.' # VPP Performance Changes 3n-hsw 1t1c ndr-full - type: "table" @@ -3580,30 +3548,23 @@ - title: "2005.0-4 vs 2005.0-3" reference: 4 compare: 5 + rca-file: "rca/rca-2005-3n-hsw-1t1c-ndr-rca1.yaml" - title: "2001.0-4 vs 2001.0-2" reference: 2 compare: 3 + rca-file: "rca/rca-2005-3n-hsw-1t1c-ndr-rca2.yaml" - title: "2005.0-4 vs 2001.0-4" reference: 3 compare: 5 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-3n-hsw-1t1c-ndr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-3n-hsw-1t1c-ndr-rca2.yaml" -# - title: "RCA3" -# data: "rca/rca-2005-3n-hsw-1t1c-ndr-rca3.yaml" + rca-file: "rca/rca-2005-3n-hsw-1t1c-ndr-rca3.yaml" legend: - '' - 'Test Case: Test cases executed for: NDR throughput on 3n-hsw with xl710.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' - '2005.0-4 vs 2005.0-3: Compares CSIT environments "4" to "3". "3" was discarded as it substantially changed TRex behaviour due to higher number of TRex workers used.' - '2001.0-4 vs 2001.0-2: Compares CSIT environment current vs previous with previous VPP code.' - '2005.0-4 vs 2001.0-4: Compares VPP code current vs previous in the current CSIT environment.' -# - 'RCA1: Root cause analysis reference for 2005.0-4 vs 2005.0-3.' -# - 'RCA2: Root cause analysis reference for 2001.0-4 vs 2001.0-2.' -# - 'RCA3: Root cause analysis reference for 2005.0-4 vs 2001.0-4.' # VPP Performance Changes 3n-hsw 2t2c ndr-full - type: "table" @@ -3627,30 +3588,23 @@ - title: "2005.0-4 vs 2005.0-3" reference: 4 compare: 5 + rca-file: "rca/rca-2005-3n-hsw-2t2c-ndr-rca1.yaml" - title: "2001.0-4 vs 2001.0-2" reference: 2 compare: 3 + rca-file: "rca/rca-2005-3n-hsw-2t2c-ndr-rca2.yaml" - title: "2005.0-4 vs 2001.0-4" reference: 3 compare: 5 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-3n-hsw-2t2c-ndr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-3n-hsw-2t2c-ndr-rca2.yaml" -# - title: "RCA3" -# data: "rca/rca-2005-3n-hsw-2t2c-ndr-rca3.yaml" + rca-file: "rca/rca-2005-3n-hsw-2t2c-ndr-rca3.yaml" legend: - '' - 'Test Case: Test cases executed for: NDR throughput on 3n-hsw with xl710.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' - '2005.0-4 vs 2005.0-3: Compares CSIT environments "4" to "3". "3" was discarded as it substantially changed TRex behaviour due to higher number of TRex workers used.' - '2001.0-4 vs 2001.0-2: Compares CSIT environment current vs previous with previous VPP code.' - '2005.0-4 vs 2001.0-4: Compares VPP code current vs previous in the current CSIT environment.' -# - 'RCA1: Root cause analysis reference for 2005.0-4 vs 2005.0-3.' -# - 'RCA2: Root cause analysis reference for 2001.0-4 vs 2001.0-2.' -# - 'RCA3: Root cause analysis reference for 2005.0-4 vs 2001.0-4.' # VPP Performance Changes 3n-skx 2t1c pdr - type: "table" @@ -3672,23 +3626,18 @@ - title: "2001.0-4 vs 2001.0-2" reference: 2 compare: 3 + rca-file: "rca/rca-2005-3n-skx-2t1c-pdr-rca1.yaml" - title: "2005.0-4 vs 2001.0-4" reference: 3 compare: 4 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-3n-skx-2t1c-pdr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-3n-skx-2t1c-pdr-rca2.yaml" + rca-file: "rca/rca-2005-3n-skx-2t1c-pdr-rca2.yaml" legend: - '' - 'Test Case: Test cases executed for: PDR throughput on 3n-skx with xxv710.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' - '2001.0-4 vs 2001.0-2: Compares CSIT environment current vs previous with previous VPP code.' - '2005.0-4 vs 2001.0-4: Compares VPP code current vs previous in the current CSIT environment.' -# - 'RCA1: Root cause analysis reference for 2001.0-4 vs 2001.0-2.' -# - 'RCA2: Root cause analysis reference for 2005.0-4 vs 2001.0-4.' # VPP Performance Changes 3n-skx 4t2c pdr - type: "table" @@ -3710,23 +3659,18 @@ - title: "2001.0-4 vs 2001.0-2" reference: 2 compare: 3 + rca-file: "rca/rca-2005-3n-skx-4t2c-pdr-rca1.yaml" - title: "2005.0-4 vs 2001.0-4" reference: 3 compare: 4 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-3n-skx-4t2c-pdr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-3n-skx-4t2c-pdr-rca2.yaml" + rca-file: "rca/rca-2005-3n-skx-4t2c-pdr-rca2.yaml" legend: - '' - 'Test Case: Test cases executed for: PDR throughput on 3n-skx with xxv710.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' - '2001.0-4 vs 2001.0-2: Compares CSIT environment current vs previous with previous VPP code.' - '2005.0-4 vs 2001.0-4: Compares VPP code current vs previous in the current CSIT environment.' -# - 'RCA1: Root cause analysis reference for 2001.0-4 vs 2001.0-2.' -# - 'RCA2: Root cause analysis reference for 2005.0-4 vs 2001.0-4.' # VPP Performance Changes 3n-skx 2t1c ndr - type: "table" @@ -3748,23 +3692,18 @@ - title: "2001.0-4 vs 2001.0-2" reference: 2 compare: 3 + rca-file: "rca/rca-2005-3n-skx-2t1c-ndr-rca1.yaml" - title: "2005.0-4 vs 2001.0-4" reference: 3 compare: 4 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-3n-skx-2t1c-ndr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-3n-skx-2t1c-ndr-rca2.yaml" + rca-file: "rca/rca-2005-3n-skx-2t1c-ndr-rca2.yaml" legend: - '' - 'Test Case: Test cases executed for: NDR throughput on 3n-skx with xxv710.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' - '2001.0-4 vs 2001.0-2: Compares CSIT environment current vs previous with previous VPP code.' - '2005.0-4 vs 2001.0-4: Compares VPP code current vs previous in the current CSIT environment.' -# - 'RCA1: Root cause analysis reference for 2001.0-4 vs 2001.0-2.' -# - 'RCA2: Root cause analysis reference for 2005.0-4 vs 2001.0-4.' # VPP Performance Changes 3n-skx 4t2c ndr - type: "table" @@ -3786,23 +3725,18 @@ - title: "2001.0-4 vs 2001.0-2" reference: 2 compare: 3 + rca-file: "rca/rca-2005-3n-skx-4t2c-ndr-rca1.yaml" - title: "2005.0-4 vs 2001.0-4" reference: 3 compare: 4 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-3n-skx-4t2c-ndr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-3n-skx-4t2c-ndr-rca2.yaml" + rca-file: "rca/rca-2005-3n-skx-4t2c-ndr-rca2.yaml" legend: - '' - 'Test Case: Test cases executed for: NDR throughput on 3n-skx with xxv710.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' - '2001.0-4 vs 2001.0-2: Compares CSIT environment current vs previous with previous VPP code.' - '2005.0-4 vs 2001.0-4: Compares VPP code current vs previous in the current CSIT environment.' -# - 'RCA1: Root cause analysis reference for 2001.0-4 vs 2001.0-2.' -# - 'RCA2: Root cause analysis reference for 2005.0-4 vs 2001.0-4.' # VPP Performance Changes 3n-skx 2t1c pdr-full - type: "table" @@ -3826,30 +3760,23 @@ - title: "2005.0-4 vs 2005.0-3" reference: 4 compare: 5 + rca-file: "rca/rca-2005-3n-skx-2t1c-pdr-rca1.yaml" - title: "2001.0-4 vs 2001.0-2" reference: 2 compare: 3 + rca-file: "rca/rca-2005-3n-skx-2t1c-pdr-rca2.yaml" - title: "2005.0-4 vs 2001.0-4" reference: 3 compare: 5 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-3n-skx-2t1c-pdr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-3n-skx-2t1c-pdr-rca2.yaml" -# - title: "RCA3" -# data: "rca/rca-2005-3n-skx-2t1c-pdr-rca3.yaml" + rca-file: "rca/rca-2005-3n-skx-2t1c-pdr-rca3.yaml" legend: - '' - 'Test Case: Test cases executed for: PDR throughput on 3n-skx with xxv710.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' - '2005.0-4 vs 2005.0-3: Compares CSIT environments "4" to "3". "3" was discarded as it substantially changed TRex behaviour due to higher number of TRex workers used.' - '2001.0-4 vs 2001.0-2: Compares CSIT environment current vs previous with previous VPP code.' - '2005.0-4 vs 2001.0-4: Compares VPP code current vs previous in the current CSIT environment.' -# - 'RCA1: Root cause analysis reference for 2005.0-4 vs 2005.0-3.' -# - 'RCA2: Root cause analysis reference for 2001.0-4 vs 2001.0-2.' -# - 'RCA3: Root cause analysis reference for 2005.0-4 vs 2001.0-4.' # VPP Performance Changes 3n-skx 4t2c pdr-full - type: "table" @@ -3873,30 +3800,23 @@ - title: "2005.0-4 vs 2005.0-3" reference: 4 compare: 5 + rca-file: "rca/rca-2005-3n-skx-4t2c-pdr-rca1.yaml" - title: "2001.0-4 vs 2001.0-2" reference: 2 compare: 3 + rca-file: "rca/rca-2005-3n-skx-4t2c-pdr-rca2.yaml" - title: "2005.0-4 vs 2001.0-4" reference: 3 compare: 5 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-3n-skx-4t2c-pdr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-3n-skx-4t2c-pdr-rca2.yaml" -# - title: "RCA3" -# data: "rca/rca-2005-3n-skx-4t2c-pdr-rca3.yaml" + rca-file: "rca/rca-2005-3n-skx-4t2c-pdr-rca3.yaml" legend: - '' - 'Test Case: Test cases executed for: PDR throughput on 3n-skx with xxv710.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' - '2005.0-4 vs 2005.0-3: Compares CSIT environments "4" to "3". "3" was discarded as it substantially changed TRex behaviour due to higher number of TRex workers used.' - '2001.0-4 vs 2001.0-2: Compares CSIT environment current vs previous with previous VPP code.' - '2005.0-4 vs 2001.0-4: Compares VPP code current vs previous in the current CSIT environment.' -# - 'RCA1: Root cause analysis reference for 2005.0-4 vs 2005.0-3.' -# - 'RCA2: Root cause analysis reference for 2001.0-4 vs 2001.0-2.' -# - 'RCA3: Root cause analysis reference for 2005.0-4 vs 2001.0-4.' # VPP Performance Changes 3n-skx 2t1c ndr-full - type: "table" @@ -3920,30 +3840,23 @@ - title: "2005.0-4 vs 2005.0-3" reference: 4 compare: 5 + rca-file: "rca/rca-2005-3n-skx-2t1c-ndr-rca1.yaml" - title: "2001.0-4 vs 2001.0-2" reference: 2 compare: 3 + rca-file: "rca/rca-2005-3n-skx-2t1c-ndr-rca2.yaml" - title: "2005.0-4 vs 2001.0-4" reference: 3 compare: 5 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-3n-skx-2t1c-ndr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-3n-skx-2t1c-ndr-rca2.yaml" -# - title: "RCA3" -# data: "rca/rca-2005-3n-skx-2t1c-ndr-rca3.yaml" + rca-file: "rca/rca-2005-3n-skx-2t1c-ndr-rca3.yaml" legend: - '' - 'Test Case: Test cases executed for: NDR throughput on 3n-skx with xxv710.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' - '2005.0-4 vs 2005.0-3: Compares CSIT environments "4" to "3". "3" was discarded as it substantially changed TRex behaviour due to higher number of TRex workers used.' - '2001.0-4 vs 2001.0-2: Compares CSIT environment current vs previous with previous VPP code.' - '2005.0-4 vs 2001.0-4: Compares VPP code current vs previous in the current CSIT environment.' -# - 'RCA1: Root cause analysis reference for 2005.0-4 vs 2005.0-3.' -# - 'RCA2: Root cause analysis reference for 2001.0-4 vs 2001.0-2.' -# - 'RCA3: Root cause analysis reference for 2005.0-4 vs 2001.0-4.' # VPP Performance Changes 3n-skx 4t2c ndr-full - type: "table" @@ -3967,30 +3880,23 @@ - title: "2005.0-4 vs 2005.0-3" reference: 4 compare: 5 + rca-file: "rca/rca-2005-3n-skx-4t2c-ndr-rca1.yaml" - title: "2001.0-4 vs 2001.0-2" reference: 2 compare: 3 + rca-file: "rca/rca-2005-3n-skx-4t2c-ndr-rca2.yaml" - title: "2005.0-4 vs 2001.0-4" reference: 3 compare: 5 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-3n-skx-4t2c-ndr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-3n-skx-4t2c-ndr-rca2.yaml" -# - title: "RCA3" -# data: "rca/rca-2005-3n-skx-4t2c-ndr-rca3.yaml" + rca-file: "rca/rca-2005-3n-skx-4t2c-ndr-rca3.yaml" legend: - '' - 'Test Case: Test cases executed for: NDR throughput on 3n-skx with xxv710.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' - '2005.0-4 vs 2005.0-3: Compares CSIT environments "4" to "3". "3" was discarded as it substantially changed TRex behaviour due to higher number of TRex workers used.' - '2001.0-4 vs 2001.0-2: Compares CSIT environment current vs previous with previous VPP code.' - '2005.0-4 vs 2001.0-4: Compares VPP code current vs previous in the current CSIT environment.' -# - 'RCA1: Root cause analysis reference for 2005.0-4 vs 2005.0-3.' -# - 'RCA2: Root cause analysis reference for 2001.0-4 vs 2001.0-2.' -# - 'RCA3: Root cause analysis reference for 2005.0-4 vs 2001.0-4.' # VPP Performance Changes 2n-skx 2t1c pdr - type: "table" @@ -4012,23 +3918,18 @@ - title: "2001.0-4 vs 2001.0-2" reference: 2 compare: 3 + rca-file: "rca/rca-2005-2n-skx-2t1c-pdr-rca1.yaml" - title: "2005.0-4 vs 2001.0-4" reference: 3 compare: 4 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-2n-skx-2t1c-pdr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-2n-skx-2t1c-pdr-rca2.yaml" + rca-file: "rca/rca-2005-2n-skx-2t1c-pdr-rca2.yaml" legend: - '' - 'Test Case: Test cases executed for: PDR throughput on 2n-skx with xxv710.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' - '2001.0-4 vs 2001.0-2: Compares CSIT environment current vs previous with previous VPP code.' - '2005.0-4 vs 2001.0-4: Compares VPP code current vs previous in the current CSIT environment.' -# - 'RCA1: Root cause analysis reference for 2001.0-4 vs 2001.0-2.' -# - 'RCA2: Root cause analysis reference for 2005.0-4 vs 2001.0-4.' # VPP Performance Changes 2n-skx 4t2c pdr - type: "table" @@ -4050,23 +3951,18 @@ - title: "2001.0-4 vs 2001.0-2" reference: 2 compare: 3 + rca-file: "rca/rca-2005-2n-skx-4t2c-pdr-rca1.yaml" - title: "2005.0-4 vs 2001.0-4" reference: 3 compare: 4 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-2n-skx-4t2c-pdr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-2n-skx-4t2c-pdr-rca2.yaml" + rca-file: "rca/rca-2005-2n-skx-4t2c-pdr-rca2.yaml" legend: - '' - 'Test Case: Test cases executed for: PDR throughput on 2n-skx with xxv710.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' - '2001.0-4 vs 2001.0-2: Compares CSIT environment current vs previous with previous VPP code.' - '2005.0-4 vs 2001.0-4: Compares VPP code current vs previous in the current CSIT environment.' -# - 'RCA1: Root cause analysis reference for 2001.0-4 vs 2001.0-2.' -# - 'RCA2: Root cause analysis reference for 2005.0-4 vs 2001.0-4.' # VPP Performance Changes 2n-skx 2t1c ndr - type: "table" @@ -4088,23 +3984,18 @@ - title: "2001.0-4 vs 2001.0-2" reference: 2 compare: 3 + rca-file: "rca/rca-2005-2n-skx-2t1c-ndr-rca1.yaml" - title: "2005.0-4 vs 2001.0-4" reference: 3 compare: 4 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-2n-skx-2t1c-ndr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-2n-skx-2t1c-ndr-rca2.yaml" + rca-file: "rca/rca-2005-2n-skx-2t1c-ndr-rca2.yaml" legend: - '' - 'Test Case: Test cases executed for: NDR throughput on 2n-skx with xxv710.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' - '2001.0-4 vs 2001.0-2: Compares CSIT environment current vs previous with previous VPP code.' - '2005.0-4 vs 2001.0-4: Compares VPP code current vs previous in the current CSIT environment.' -# - 'RCA1: Root cause analysis reference for 2001.0-4 vs 2001.0-2.' -# - 'RCA2: Root cause analysis reference for 2005.0-4 vs 2001.0-4.' # VPP Performance Changes 2n-skx 4t2c ndr - type: "table" @@ -4126,23 +4017,18 @@ - title: "2001.0-4 vs 2001.0-2" reference: 2 compare: 3 + rca-file: "rca/rca-2005-2n-skx-4t2c-ndr-rca1.yaml" - title: "2005.0-4 vs 2001.0-4" reference: 3 compare: 4 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-2n-skx-4t2c-ndr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-2n-skx-4t2c-ndr-rca2.yaml" + rca-file: "rca/rca-2005-2n-skx-4t2c-ndr-rca2.yaml" legend: - '' - 'Test Case: Test cases executed for: NDR throughput on 2n-skx with xxv710.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' - '2001.0-4 vs 2001.0-2: Compares CSIT environment current vs previous with previous VPP code.' - '2005.0-4 vs 2001.0-4: Compares VPP code current vs previous in the current CSIT environment.' -# - 'RCA1: Root cause analysis reference for 2001.0-4 vs 2001.0-2.' -# - 'RCA2: Root cause analysis reference for 2005.0-4 vs 2001.0-4.' # VPP Performance Changes 2n-skx 2t1c pdr-full - type: "table" @@ -4166,30 +4052,24 @@ - title: "2005.0-4 vs 2005.0-3" reference: 4 compare: 5 + rca-file: "rca/rca-2005-2n-skx-2t1c-pdr-rca1.yaml" - title: "2001.0-4 vs 2001.0-2" reference: 2 compare: 3 + rca-file: "rca/rca-2005-2n-skx-2t1c-pdr-rca2.yaml" - title: "2005.0-4 vs 2001.0-4" reference: 3 compare: 5 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-2n-skx-2t1c-pdr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-2n-skx-2t1c-pdr-rca2.yaml" -# - title: "RCA3" -# data: "rca/rca-2005-2n-skx-2t1c-pdr-rca3.yaml" + rca-file: "rca/rca-2005-2n-skx-2t1c-pdr-rca3.yaml" + legend: - '' - 'Test Case: Test cases executed for: PDR throughput on 2n-skx with xxv710.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' - '2005.0-4 vs 2005.0-3: Compares CSIT environments "4" to "3". "3" was discarded as it substantially changed TRex behaviour due to higher number of TRex workers used.' - '2001.0-4 vs 2001.0-2: Compares CSIT environment current vs previous with previous VPP code.' - '2005.0-4 vs 2001.0-4: Compares VPP code current vs previous in the current CSIT environment.' -# - 'RCA1: Root cause analysis reference for 2005.0-4 vs 2005.0-3.' -# - 'RCA2: Root cause analysis reference for 2001.0-4 vs 2001.0-2.' -# - 'RCA3: Root cause analysis reference for 2005.0-4 vs 2001.0-4.' # VPP Performance Changes 2n-skx 4t2c pdr-full - type: "table" @@ -4213,30 +4093,23 @@ - title: "2005.0-4 vs 2005.0-3" reference: 4 compare: 5 + rca-file: "rca/rca-2005-2n-skx-4t2c-pdr-rca1.yaml" - title: "2001.0-4 vs 2001.0-2" reference: 2 compare: 3 + rca-file: "rca/rca-2005-2n-skx-4t2c-pdr-rca2.yaml" - title: "2005.0-4 vs 2001.0-4" reference: 3 compare: 5 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-2n-skx-4t2c-pdr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-2n-skx-4t2c-pdr-rca2.yaml" -# - title: "RCA3" -# data: "rca/rca-2005-2n-skx-4t2c-pdr-rca3.yaml" + rca-file: "rca/rca-2005-2n-skx-4t2c-pdr-rca3.yaml" legend: - '' - 'Test Case: Test cases executed for: PDR throughput on 2n-skx with xxv710.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' - '2005.0-4 vs 2005.0-3: Compares CSIT environments "4" to "3". "3" was discarded as it substantially changed TRex behaviour due to higher number of TRex workers used.' - '2001.0-4 vs 2001.0-2: Compares CSIT environment current vs previous with previous VPP code.' - '2005.0-4 vs 2001.0-4: Compares VPP code current vs previous in the current CSIT environment.' -# - 'RCA1: Root cause analysis reference for 2005.0-4 vs 2005.0-3.' -# - 'RCA2: Root cause analysis reference for 2001.0-4 vs 2001.0-2.' -# - 'RCA3: Root cause analysis reference for 2005.0-4 vs 2001.0-4.' # VPP Performance Changes 2n-skx 2t1c ndr-full - type: "table" @@ -4260,30 +4133,23 @@ - title: "2005.0-4 vs 2005.0-3" reference: 4 compare: 5 + rca-file: "rca/rca-2005-2n-skx-2t1c-ndr-rca1.yaml" - title: "2001.0-4 vs 2001.0-2" reference: 2 compare: 3 + rca-file: "rca/rca-2005-2n-skx-2t1c-ndr-rca2.yaml" - title: "2005.0-4 vs 2001.0-4" reference: 3 compare: 5 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-2n-skx-2t1c-ndr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-2n-skx-2t1c-ndr-rca2.yaml" -# - title: "RCA3" -# data: "rca/rca-2005-2n-skx-2t1c-ndr-rca3.yaml" + rca-file: "rca/rca-2005-2n-skx-2t1c-ndr-rca3.yaml" legend: - '' - 'Test Case: Test cases executed for: NDR throughput on 2n-skx with xxv710.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' - '2005.0-4 vs 2005.0-3: Compares CSIT environments "4" to "3". "3" was discarded as it substantially changed TRex behaviour due to higher number of TRex workers used.' - '2001.0-4 vs 2001.0-2: Compares CSIT environment current vs previous with previous VPP code.' - '2005.0-4 vs 2001.0-4: Compares VPP code current vs previous in the current CSIT environment.' -# - 'RCA1: Root cause analysis reference for 2005.0-4 vs 2005.0-3.' -# - 'RCA2: Root cause analysis reference for 2001.0-4 vs 2001.0-2.' -# - 'RCA3: Root cause analysis reference for 2005.0-4 vs 2001.0-4.' # VPP Performance Changes 2n-skx 4t2c ndr-full - type: "table" @@ -4307,30 +4173,23 @@ - title: "2005.0-4 vs 2005.0-3" reference: 4 compare: 5 + rca-file: "rca/rca-2005-2n-skx-4t2c-ndr-rca1.yaml" - title: "2001.0-4 vs 2001.0-2" reference: 2 compare: 3 + rca-file: "rca/rca-2005-2n-skx-4t2c-ndr-rca2.yaml" - title: "2005.0-4 vs 2001.0-4" reference: 3 compare: 5 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-2n-skx-4t2c-ndr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-2n-skx-4t2c-ndr-rca2.yaml" -# - title: "RCA3" -# data: "rca/rca-2005-2n-skx-4t2c-ndr-rca3.yaml" + rca-file: "rca/rca-2005-2n-skx-4t2c-ndr-rca3.yaml" legend: - '' - 'Test Case: Test cases executed for: NDR throughput on 2n-skx with xxv710.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' - '2005.0-4 vs 2005.0-3: Compares CSIT environments "4" to "3". "3" was discarded as it substantially changed TRex behaviour due to higher number of TRex workers used.' - '2001.0-4 vs 2001.0-2: Compares CSIT environment current vs previous with previous VPP code.' - '2005.0-4 vs 2001.0-4: Compares VPP code current vs previous in the current CSIT environment.' -# - 'RCA1: Root cause analysis reference for 2005.0-4 vs 2005.0-3.' -# - 'RCA2: Root cause analysis reference for 2001.0-4 vs 2001.0-2.' -# - 'RCA3: Root cause analysis reference for 2005.0-4 vs 2001.0-4.' # VPP Performance Changes 2n-clx-xxv710 2t1c pdr - type: "table" @@ -4350,23 +4209,18 @@ - title: "2001.0-4 vs 2001.0-2" reference: 1 compare: 2 + rca-file: "rca/rca-2005-2n-clx-2t1c-pdr-rca1.yaml" - title: "2005.0-4 vs 2001.0-4" reference: 2 compare: 3 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-2n-clx-2t1c-pdr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-2n-clx-2t1c-pdr-rca2.yaml" + rca-file: "rca/rca-2005-2n-clx-2t1c-pdr-rca2.yaml" legend: - '' - 'Test Case: Test cases executed for: PDR throughput on 2n-clx with xxv710.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' - '2001.0-4 vs 2001.0-2: Compares CSIT environment current vs previous with previous VPP code.' - '2005.0-4 vs 2001.0-4: Compares VPP code current vs previous in the current CSIT environment.' -# - 'RCA1: Root cause analysis reference for 2001.0-4 vs 2001.0-2.' -# - 'RCA2: Root cause analysis reference for 2005.0-4 vs 2001.0-4.' # VPP Performance Changes 2n-clx-xxv710 4t2c pdr - type: "table" @@ -4386,23 +4240,18 @@ - title: "2001.0-4 vs 2001.0-2" reference: 1 compare: 2 + rca-file: "rca/rca-2005-2n-clx-4t2c-pdr-rca1.yaml" - title: "2005.0-4 vs 2001.0-4" reference: 2 compare: 3 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-2n-clx-4t2c-pdr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-2n-clx-4t2c-pdr-rca2.yaml" + rca-file: "rca/rca-2005-2n-clx-4t2c-pdr-rca2.yaml" legend: - '' - 'Test Case: Test cases executed for: PDR throughput on 2n-clx with xxv710.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' - '2001.0-4 vs 2001.0-2: Compares CSIT environment current vs previous with previous VPP code.' - '2005.0-4 vs 2001.0-4: Compares VPP code current vs previous in the current CSIT environment.' -# - 'RCA1: Root cause analysis reference for 2001.0-4 vs 2001.0-2.' -# - 'RCA2: Root cause analysis reference for 2005.0-4 vs 2001.0-4.' # VPP Performance Changes 2n-clx-xxv710 2t1c ndr - type: "table" @@ -4422,23 +4271,18 @@ - title: "2001.0-4 vs 2001.0-2" reference: 1 compare: 2 + rca-file: "rca/rca-2005-2n-clx-2t1c-ndr-rca1.yaml" - title: "2005.0-4 vs 2001.0-4" reference: 2 compare: 3 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-2n-clx-2t1c-ndr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-2n-clx-2t1c-ndr-rca2.yaml" + rca-file: "rca/rca-2005-2n-clx-2t1c-ndr-rca2.yaml" legend: - '' - 'Test Case: Test cases executed for: NDR throughput on 2n-clx with xxv710.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' - '2001.0-4 vs 2001.0-2: Compares CSIT environment current vs previous with previous VPP code.' - '2005.0-4 vs 2001.0-4: Compares VPP code current vs previous in the current CSIT environment.' -# - 'RCA1: Root cause analysis reference for 2001.0-4 vs 2001.0-2.' -# - 'RCA2: Root cause analysis reference for 2005.0-4 vs 2001.0-4.' # VPP Performance Changes 2n-clx-xxv710 4t2c ndr - type: "table" @@ -4458,23 +4302,18 @@ - title: "2001.0-4 vs 2001.0-2" reference: 1 compare: 2 + rca-file: "rca/rca-2005-2n-clx-4t2c-ndr-rca1.yaml" - title: "2005.0-4 vs 2001.0-4" reference: 2 compare: 3 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-2n-clx-4t2c-ndr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-2n-clx-4t2c-ndr-rca2.yaml" + rca-file: "rca/rca-2005-2n-clx-4t2c-ndr-rca2.yaml" legend: - '' - 'Test Case: Test cases executed for: NDR throughput on 2n-clx with xxv710.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' - '2001.0-4 vs 2001.0-2: Compares CSIT environment current vs previous with previous VPP code.' - '2005.0-4 vs 2001.0-4: Compares VPP code current vs previous in the current CSIT environment.' -# - 'RCA1: Root cause analysis reference for 2001.0-4 vs 2001.0-2.' -# - 'RCA2: Root cause analysis reference for 2005.0-4 vs 2001.0-4.' # VPP Performance Changes 2n-clx-xxv710 2t1c pdr-full - type: "table" @@ -4496,30 +4335,23 @@ - title: "2005.0-4 vs 2005.0-3" reference: 3 compare: 4 + rca-file: "rca/rca-2005-2n-clx-2t1c-pdr-rca1.yaml" - title: "2001.0-4 vs 2001.0-2" reference: 1 compare: 2 + rca-file: "rca/rca-2005-2n-clx-2t1c-pdr-rca2.yaml" - title: "2005.0-4 vs 2001.0-4" reference: 2 compare: 4 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-2n-clx-2t1c-pdr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-2n-clx-2t1c-pdr-rca2.yaml" -# - title: "RCA3" -# data: "rca/rca-2005-2n-clx-2t1c-pdr-rca3.yaml" + rca-file: "rca/rca-2005-2n-clx-2t1c-pdr-rca3.yaml" legend: - '' - 'Test Case: Test cases executed for: PDR throughput on 2n-clx with xxv710.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' - '2005.0-4 vs 2005.0-3: Compares CSIT environments "4" to "3". "3" was discarded as it substantially changed TRex behaviour due to higher number of TRex workers used.' - '2001.0-4 vs 2001.0-2: Compares CSIT environment current vs previous with previous VPP code.' - '2005.0-4 vs 2001.0-4: Compares VPP code current vs previous in the current CSIT environment.' -# - 'RCA1: Root cause analysis reference for 2005.0-4 vs 2005.0-3.' -# - 'RCA2: Root cause analysis reference for 2001.0-4 vs 2001.0-2.' -# - 'RCA3: Root cause analysis reference for 2005.0-4 vs 2001.0-4.' # VPP Performance Changes 2n-clx-xxv710 4t2c pdr-full - type: "table" @@ -4541,30 +4373,23 @@ - title: "2005.0-4 vs 2005.0-3" reference: 3 compare: 4 + rca-file: "rca/rca-2005-2n-clx-4t2c-pdr-rca1.yaml" - title: "2001.0-4 vs 2001.0-2" reference: 1 compare: 2 + rca-file: "rca/rca-2005-2n-clx-4t2c-pdr-rca2.yaml" - title: "2005.0-4 vs 2001.0-4" reference: 2 compare: 4 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-2n-clx-4t2c-pdr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-2n-clx-4t2c-pdr-rca2.yaml" -# - title: "RCA3" -# data: "rca/rca-2005-2n-clx-4t2c-pdr-rca3.yaml" + rca-file: "rca/rca-2005-2n-clx-4t2c-pdr-rca3.yaml" legend: - '' - 'Test Case: Test cases executed for: PDR throughput on 2n-clx with xxv710.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' - '2005.0-4 vs 2005.0-3: Compares CSIT environments "4" to "3". "3" was discarded as it substantially changed TRex behaviour due to higher number of TRex workers used.' - '2001.0-4 vs 2001.0-2: Compares CSIT environment current vs previous with previous VPP code.' - '2005.0-4 vs 2001.0-4: Compares VPP code current vs previous in the current CSIT environment.' -# - 'RCA1: Root cause analysis reference for 2005.0-4 vs 2005.0-3.' -# - 'RCA2: Root cause analysis reference for 2001.0-4 vs 2001.0-2.' -# - 'RCA3: Root cause analysis reference for 2005.0-4 vs 2001.0-4.' # VPP Performance Changes 2n-clx-xxv710 2t1c ndr-full - type: "table" @@ -4586,30 +4411,23 @@ - title: "2005.0-4 vs 2005.0-3" reference: 3 compare: 4 + rca-file: "rca/rca-2005-2n-clx-2t1c-ndr-rca1.yaml" - title: "2001.0-4 vs 2001.0-2" reference: 1 compare: 2 + rca-file: "rca/rca-2005-2n-clx-2t1c-ndr-rca2.yaml" - title: "2005.0-4 vs 2001.0-4" reference: 2 compare: 4 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-2n-clx-2t1c-ndr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-2n-clx-2t1c-ndr-rca2.yaml" -# - title: "RCA3" -# data: "rca/rca-2005-2n-clx-2t1c-ndr-rca3.yaml" + rca-file: "rca/rca-2005-2n-clx-2t1c-ndr-rca3.yaml" legend: - '' - 'Test Case: Test cases executed for: NDR throughput on 2n-clx with xxv710.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' - '2005.0-4 vs 2005.0-3: Compares CSIT environments "4" to "3". "3" was discarded as it substantially changed TRex behaviour due to higher number of TRex workers used.' - '2001.0-4 vs 2001.0-2: Compares CSIT environment current vs previous with previous VPP code.' - '2005.0-4 vs 2001.0-4: Compares VPP code current vs previous in the current CSIT environment.' -# - 'RCA1: Root cause analysis reference for 2005.0-4 vs 2005.0-3.' -# - 'RCA2: Root cause analysis reference for 2001.0-4 vs 2001.0-2.' -# - 'RCA3: Root cause analysis reference for 2005.0-4 vs 2001.0-4.' # VPP Performance Changes 2n-clx-xxv710 4t2c ndr-full - type: "table" @@ -4631,30 +4449,23 @@ - title: "2005.0-4 vs 2005.0-3" reference: 3 compare: 4 + rca-file: "rca/rca-2005-2n-clx-4t2c-ndr-rca1.yaml" - title: "2001.0-4 vs 2001.0-2" reference: 1 compare: 2 + rca-file: "rca/rca-2005-2n-clx-4t2c-ndr-rca2.yaml" - title: "2005.0-4 vs 2001.0-4" reference: 2 compare: 4 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-2n-clx-4t2c-ndr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-2n-clx-4t2c-ndr-rca2.yaml" -# - title: "RCA3" -# data: "rca/rca-2005-2n-clx-4t2c-ndr-rca3.yaml" + rca-file: "rca/rca-2005-2n-clx-4t2c-ndr-rca3.yaml" legend: - '' - 'Test Case: Test cases executed for: NDR throughput on 2n-clx with xxv710.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' - '2005.0-4 vs 2005.0-3: Compares CSIT environments "4" to "3". "3" was discarded as it substantially changed TRex behaviour due to higher number of TRex workers used.' - '2001.0-4 vs 2001.0-2: Compares CSIT environment current vs previous with previous VPP code.' - '2005.0-4 vs 2001.0-4: Compares VPP code current vs previous in the current CSIT environment.' -# - 'RCA1: Root cause analysis reference for 2005.0-4 vs 2005.0-3.' -# - 'RCA2: Root cause analysis reference for 2001.0-4 vs 2001.0-2.' -# - 'RCA3: Root cause analysis reference for 2005.0-4 vs 2001.0-4.' # VPP Performance Changes 2n-clx-x556 2t1c pdr - type: "table" @@ -4676,8 +4487,8 @@ - '' - 'Test Case: Test cases executed for: PDR throughput on 2n-clx with x556.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' - - '2005.0-4 vs 2001.0-2: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' + - '2005.0-4 vs 2001.0-2: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' # - ' RCA1: root cause analysis reference for 2001.0-4 vs 2001.0-2.' # VPP Performance Changes 2n-clx-x556 4t2c pdr @@ -4700,8 +4511,8 @@ - '' - 'Test Case: Test cases executed for: PDR throughput on 2n-clx with x556.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' - - '2005.0-4 vs 2001.0-2: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' + - '2005.0-4 vs 2001.0-2: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' # - ' RCA1: root cause analysis reference for 2001.0-4 vs 2001.0-2.' # VPP Performance Changes 2n-clx-x556 2t1c ndr @@ -4724,8 +4535,8 @@ - '' - 'Test Case: Test cases executed for: NDR throughput on 2n-clx with x556.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' - - '2005.0-4 vs 2001.0-2: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' + - '2005.0-4 vs 2001.0-2: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' # - ' RCA1: root cause analysis reference for 2001.0-4 vs 2001.0-2.' # VPP Performance Changes 2n-clx-x556 4t2c ndr @@ -4748,8 +4559,8 @@ - '' - 'Test Case: Test cases executed for: NDR throughput on 2n-clx with x556.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' - - '2005.0-4 vs 2001.0-2: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' + - '2005.0-4 vs 2001.0-2: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' # - ' RCA1: root cause analysis reference for 2001.0-4 vs 2001.0-2.' # VPP Performance Changes 2n-dnv 1t1c pdr @@ -4956,23 +4767,18 @@ - title: "2001.0-4 vs 2001.0-2" reference: 2 compare: 3 + rca-file: "rca/rca-2005-3n-tsh-1t1c-pdr-rca1.yaml" - title: "2005.0-4 vs 2001.0-4" reference: 3 compare: 4 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-3n-tsh-1t1c-pdr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-3n-tsh-1t1c-pdr-rca2.yaml" + rca-file: "rca/rca-2005-3n-tsh-1t1c-pdr-rca2.yaml" legend: - '' - 'Test Case: Test cases executed for: PDR throughput on 3n-tsh with x520.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' - '2001.0-4 vs 2001.0-2: Compares CSIT environment current vs previous with previous VPP code.' - '2005.0-4 vs 2001.0-4: Compares VPP code current vs previous in the current CSIT environment.' -# - 'RCA1: Root cause analysis reference for 2001.0-4 vs 2001.0-2.' -# - 'RCA2: Root cause analysis reference for 2005.0-4 vs 2001.0-4.' # VPP Performance Changes 3n-tsh 2t2c pdr - type: "table" @@ -4994,23 +4800,18 @@ - title: "2001.0-4 vs 2001.0-2" reference: 2 compare: 3 + rca-file: "rca/rca-2005-3n-tsh-2t2c-pdr-rca1.yaml" - title: "2005.0-4 vs 2001.0-4" reference: 3 compare: 4 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-3n-tsh-2t2c-pdr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-3n-tsh-2t2c-pdr-rca2.yaml" + rca-file: "rca/rca-2005-3n-tsh-2t2c-pdr-rca2.yaml" legend: - '' - 'Test Case: Test cases executed for: PDR throughput on 3n-tsh with x520.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' - '2001.0-4 vs 2001.0-2: Compares CSIT environment current vs previous with previous VPP code.' - '2005.0-4 vs 2001.0-4: Compares VPP code current vs previous in the current CSIT environment.' -# - 'RCA1: Root cause analysis reference for 2001.0-4 vs 2001.0-2.' -# - 'RCA2: Root cause analysis reference for 2005.0-4 vs 2001.0-4.' # VPP Performance Changes 3n-tsh 1t1c ndr - type: "table" @@ -5032,23 +4833,18 @@ - title: "2001.0-4 vs 2001.0-2" reference: 2 compare: 3 + rca-file: "rca/rca-2005-3n-tsh-1t1c-ndr-rca1.yaml" - title: "2005.0-4 vs 2001.0-4" reference: 3 compare: 4 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-3n-tsh-1t1c-ndr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-3n-tsh-1t1c-ndr-rca2.yaml" + rca-file: "rca/rca-2005-3n-tsh-1t1c-ndr-rca2.yaml" legend: - '' - 'Test Case: Test cases executed for: NDR throughput on 3n-tsh with x520.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' - '2001.0-4 vs 2001.0-2: Compares CSIT environment current vs previous with previous VPP code.' - '2005.0-4 vs 2001.0-4: Compares VPP code current vs previous in the current CSIT environment.' -# - 'RCA1: Root cause analysis reference for 2001.0-4 vs 2001.0-2.' -# - 'RCA2: Root cause analysis reference for 2005.0-4 vs 2001.0-4.' # VPP Performance Changes 3n-tsh 2t2c ndr - type: "table" @@ -5070,23 +4866,18 @@ - title: "2001.0-4 vs 2001.0-2" reference: 2 compare: 3 + rca-file: "rca/rca-2005-3n-tsh-2t2c-ndr-rca1.yaml" - title: "2005.0-4 vs 2001.0-4" reference: 3 compare: 4 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-3n-tsh-2t2c-ndr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-3n-tsh-2t2c-ndr-rca2.yaml" + rca-file: "rca/rca-2005-3n-tsh-2t2c-ndr-rca2.yaml" legend: - '' - 'Test Case: Test cases executed for: NDR throughput on 3n-tsh with x520.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' - '2001.0-4 vs 2001.0-2: Compares CSIT environment current vs previous with previous VPP code.' - '2005.0-4 vs 2001.0-4: Compares VPP code current vs previous in the current CSIT environment.' -# - 'RCA1: Root cause analysis reference for 2001.0-4 vs 2001.0-2.' -# - 'RCA2: Root cause analysis reference for 2005.0-4 vs 2001.0-4.' # VPP Performance Changes 3n-tsh 1t1c pdr-full - type: "table" @@ -5110,30 +4901,23 @@ - title: "2005.0-4 vs 2005.0-3" reference: 4 compare: 5 + rca-file: "rca/rca-2005-3n-tsh-1t1c-pdr-rca1.yaml" - title: "2001.0-4 vs 2001.0-2" reference: 2 compare: 3 + rca-file: "rca/rca-2005-3n-tsh-1t1c-pdr-rca2.yaml" - title: "2005.0-4 vs 2001.0-4" reference: 3 compare: 5 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-3n-tsh-1t1c-pdr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-3n-tsh-1t1c-pdr-rca2.yaml" -# - title: "RCA3" -# data: "rca/rca-2005-3n-tsh-1t1c-pdr-rca3.yaml" + rca-file: "rca/rca-2005-3n-tsh-1t1c-pdr-rca3.yaml" legend: - '' - 'Test Case: Test cases executed for: PDR throughput on 3n-tsh with x520.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' - '2005.0-4 vs 2005.0-3: Compares CSIT environments "4" to "3". "3" was discarded as it substantially changed TRex behaviour due to higher number of TRex workers used.' - '2001.0-4 vs 2001.0-2: Compares CSIT environment current vs previous with previous VPP code.' - '2005.0-4 vs 2001.0-4: Compares VPP code current vs previous in the current CSIT environment.' -# - 'RCA1: Root cause analysis reference for 2005.0-4 vs 2005.0-3.' -# - 'RCA2: Root cause analysis reference for 2001.0-4 vs 2001.0-2.' -# - 'RCA3: Root cause analysis reference for 2005.0-4 vs 2001.0-4.' # VPP Performance Changes 3n-tsh 2t2c pdr-full - type: "table" @@ -5157,30 +4941,23 @@ - title: "2005.0-4 vs 2005.0-3" reference: 4 compare: 5 + rca-file: "rca/rca-2005-3n-tsh-2t2c-pdr-rca1.yaml" - title: "2001.0-4 vs 2001.0-2" reference: 2 compare: 3 + rca-file: "rca/rca-2005-3n-tsh-2t2c-pdr-rca2.yaml" - title: "2005.0-4 vs 2001.0-4" reference: 3 compare: 5 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-3n-tsh-2t2c-pdr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-3n-tsh-2t2c-pdr-rca2.yaml" -# - title: "RCA3" -# data: "rca/rca-2005-3n-tsh-2t2c-pdr-rca3.yaml" + rca-file: "rca/rca-2005-3n-tsh-2t2c-pdr-rca3.yaml" legend: - '' - 'Test Case: Test cases executed for: PDR throughput on 3n-tsh with x520.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' - '2005.0-4 vs 2005.0-3: Compares CSIT environments "4" to "3". "3" was discarded as it substantially changed TRex behaviour due to higher number of TRex workers used.' - '2001.0-4 vs 2001.0-2: Compares CSIT environment current vs previous with previous VPP code.' - '2005.0-4 vs 2001.0-4: Compares VPP code current vs previous in the current CSIT environment.' -# - 'RCA1: Root cause analysis reference for 2005.0-4 vs 2005.0-3.' -# - 'RCA2: Root cause analysis reference for 2001.0-4 vs 2001.0-2.' -# - 'RCA3: Root cause analysis reference for 2005.0-4 vs 2001.0-4.' # VPP Performance Changes 3n-tsh 1t1c ndr-full - type: "table" @@ -5204,30 +4981,23 @@ - title: "2005.0-4 vs 2005.0-3" reference: 4 compare: 5 + rca-file: "rca/rca-2005-3n-tsh-1t1c-ndr-rca1.yaml" - title: "2001.0-4 vs 2001.0-2" reference: 2 compare: 3 + rca-file: "rca/rca-2005-3n-tsh-1t1c-ndr-rca2.yaml" - title: "2005.0-4 vs 2001.0-4" reference: 3 compare: 5 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-3n-tsh-1t1c-ndr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-3n-tsh-1t1c-ndr-rca2.yaml" -# - title: "RCA3" -# data: "rca/rca-2005-3n-tsh-1t1c-ndr-rca3.yaml" + rca-file: "rca/rca-2005-3n-tsh-1t1c-ndr-rca3.yaml" legend: - '' - 'Test Case: Test cases executed for: NDR throughput on 3n-tsh with x520.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' - '2005.0-4 vs 2005.0-3: Compares CSIT environments "4" to "3". "3" was discarded as it substantially changed TRex behaviour due to higher number of TRex workers used.' - '2001.0-4 vs 2001.0-2: Compares CSIT environment current vs previous with previous VPP code.' - '2005.0-4 vs 2001.0-4: Compares VPP code current vs previous in the current CSIT environment.' -# - 'RCA1: Root cause analysis reference for 2005.0-4 vs 2005.0-3.' -# - 'RCA2: Root cause analysis reference for 2001.0-4 vs 2001.0-2.' -# - 'RCA3: Root cause analysis reference for 2005.0-4 vs 2001.0-4.' # VPP Performance Changes 3n-tsh 2t2c ndr-full - type: "table" @@ -5251,30 +5021,23 @@ - title: "2005.0-4 vs 2005.0-3" reference: 4 compare: 5 + rca-file: "rca/rca-2005-3n-tsh-2t2c-ndr-rca1.yaml" - title: "2001.0-4 vs 2001.0-2" reference: 2 compare: 3 + rca-file: "rca/rca-2005-3n-tsh-2t2c-ndr-rca2.yaml" - title: "2005.0-4 vs 2001.0-4" reference: 3 compare: 5 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-3n-tsh-2t2c-ndr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-3n-tsh-2t2c-ndr-rca2.yaml" -# - title: "RCA3" -# data: "rca/rca-2005-3n-tsh-2t2c-ndr-rca3.yaml" + rca-file: "rca/rca-2005-3n-tsh-2t2c-ndr-rca3.yaml" legend: - '' - 'Test Case: Test cases executed for: NDR throughput on 3n-tsh with x520.' - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base.' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' - '2005.0-4 vs 2005.0-3: Compares CSIT environments "4" to "3". "3" was discarded as it substantially changed TRex behaviour due to higher number of TRex workers used.' - '2001.0-4 vs 2001.0-2: Compares CSIT environment current vs previous with previous VPP code.' - '2005.0-4 vs 2001.0-4: Compares VPP code current vs previous in the current CSIT environment.' -# - 'RCA1: Root cause analysis reference for 2005.0-4 vs 2005.0-3.' -# - 'RCA2: Root cause analysis reference for 2001.0-4 vs 2001.0-2.' -# - 'RCA3: Root cause analysis reference for 2005.0-4 vs 2001.0-4.' # VPP Comparison Across Testbeds 3n-hsw to 3n-skx 1c ndr - type: "table" @@ -5648,22 +5411,18 @@ - title: "2005.0-4 vs 2005.0-3" reference: 3 compare: 4 + rca-file: "rca/rca-2005-3n-hsw-1t1c-mrr-rca1.yaml" - title: "2005.0-4 vs 2001.0-2" reference: 2 compare: 4 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-3n-hsw-1t1c-mrr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-3n-hsw-1t1c-mrr-rca2.yaml" + rca-file: "rca/rca-2005-3n-hsw-1t1c-mrr-rca2.yaml" legend: - '' - - ' Test Case: Test cases executed for: PDR throughput on 3n-hsw with xl710.' - - ' YYMM.V-E: [Mpps +- StDev] results for VPP version (YY year, MM month, V minor version) tested in CSIT environment version (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - '2005.0-4 vs 2005.0-3: [% +- StDev] Relative difference, latter is the base. Environment version "3" was found to be incompatible with "2" due to higher number of TRex workers used.' - - '2005.0-4 vs 2001.0-2: [% +- StDev] Relative difference, latter is the base. Used as a reference to compare benchmark results of VPP current release v20.05.0 vs previous release v20.01.0.' -# - ' RCA1: root cause analysis reference for 2005.0-4 vs 2005.0-3.' -# - ' RCA2: root cause analysis reference for 2005.0-4 vs 2001.0-2.' + - 'Test Case: Test cases executed for: MRR throughput on 3n-hsw with xl710.' + - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' + - '2005.0-4 vs 2005.0-3: Environment version "3" was found to be incompatible with "2" due to higher number of TRex workers used.' + - '2005.0-4 vs 2001.0-2: Used as a reference to compare benchmark results of VPP current release v20.05.0 vs previous release v20.01.0.' # VPP Performance Changes 3n-hsw 2t2c mrr - type: "table" @@ -5685,22 +5444,18 @@ - title: "2005.0-4 vs 2005.0-3" reference: 3 compare: 4 + rca-file: "rca/rca-2005-3n-hsw-2t2c-mrr-rca1.yaml" - title: "2005.0-4 vs 2001.0-2" reference: 2 compare: 4 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-3n-hsw-2t2c-mrr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-3n-hsw-2t2c-mrr-rca2.yaml" + rca-file: "rca/rca-2005-3n-hsw-2t2c-mrr-rca2.yaml" legend: - '' - - ' Test Case: Test cases executed for: PDR throughput on 3n-hsw with xl710.' - - ' YYMM.V-E: [Mpps +- StDev] results for VPP version (YY year, MM month, V minor version) tested in CSIT environment version (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - '2005.0-4 vs 2005.0-3: [% +- StDev] Relative difference, latter is the base. Environment version "3" was found to be incompatible with "2" due to higher number of TRex workers used.' - - '2005.0-4 vs 2001.0-2: [% +- StDev] Relative difference, latter is the base. Used as a reference to compare benchmark results of VPP current release v20.05.0 vs previous release v20.01.0.' -# - ' RCA1: root cause analysis reference for 2005.0-4 vs 2005.0-3.' -# - ' RCA2: root cause analysis reference for 2005.0-4 vs 2001.0-2.' + - 'Test Case: Test cases executed for: MRR throughput on 3n-hsw with xl710.' + - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' + - '2005.0-4 vs 2005.0-3: Environment version "3" was found to be incompatible with "2" due to higher number of TRex workers used.' + - '2005.0-4 vs 2001.0-2: Used as a reference to compare benchmark results of VPP current release v20.05.0 vs previous release v20.01.0.' # VPP Performance Changes 3n-hsw 4t4c mrr - type: "table" @@ -5722,23 +5477,18 @@ - title: "2005.0-4 vs 2005.0-3" reference: 3 compare: 4 + rca-file: "rca/rca-2005-3n-hsw-4t4c-mrr-rca1.yaml" - title: "2005.0-4 vs 2001.0-2" reference: 2 compare: 4 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-3n-hsw-4t4c-mrr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-3n-hsw-4t4c-mrr-rca2.yaml" + rca-file: "rca/rca-2005-3n-hsw-4t4c-mrr-rca2.yaml" legend: - '' - - ' Test Case: Test cases executed for: PDR throughput on 3n-hsw with xl710.' - - ' YYMM.V-E: [Mpps +- StDev] results for VPP version (YY year, MM month, V minor version) tested in CSIT environment version (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - '2005.0-4 vs 2005.0-3: [% +- StDev] Relative difference, latter is the base. Environment version "3" was found to be incompatible with "2" due to higher number of TRex workers used.' - - '2005.0-4 vs 2001.0-2: [% +- StDev] Relative difference, latter is the base. Used as a reference to compare benchmark results of VPP current release v20.05.0 vs previous release v20.01.0.' -# - ' RCA1: root cause analysis reference for 2005.0-4 vs 2005.0-3.' -# - ' RCA2: root cause analysis reference for 2005.0-4 vs 2001.0-2.' - + - 'Test Case: Test cases executed for: MRR throughput on 3n-hsw with xl710.' + - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' + - '2005.0-4 vs 2005.0-3: Environment version "3" was found to be incompatible with "2" due to higher number of TRex workers used.' + - '2005.0-4 vs 2001.0-2: Used as a reference to compare benchmark results of VPP current release v20.05.0 vs previous release v20.01.0.' # VPP Performance Changes 3n-skx 2t1c mrr - type: "table" @@ -5748,8 +5498,6 @@ include-tests: "MRR" # "PDR" | "NDR" | "MRR" filter: "'2T1C' and 'MRR' and 'NIC_Intel-XXV710' and not 'NF_DENSITY'" columns: -# - title: "1908" -# data-set: "vpp-performance-changes-3n-skx-mrr-h1" - title: "2001.0-2" data-set: "vpp-performance-changes-3n-skx-mrr-ref" - title: "2005.0-3" @@ -5760,22 +5508,18 @@ - title: "2005.0-4 vs 2005.0-3" reference: 2 compare: 3 + rca-file: "rca/rca-2005-3n-skx-2t1c-mrr-rca1.yaml" - title: "2005.0-4 vs 2001.0-2" reference: 1 compare: 3 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-3n-skx-2t1c-mrr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-3n-skx-2t1c-mrr-rca2.yaml" + rca-file: "rca/rca-2005-3n-skx-2t1c-mrr-rca2.yaml" legend: - '' - - ' Test Case: Test cases executed for: PDR throughput on 3n-skx with xxv710.' - - ' YYMM.V-E: [Mpps +- StDev] results for VPP version (YY year, MM month, V minor version) tested in CSIT environment version (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - '2005.0-4 vs 2005.0-3: [% +- StDev] Relative difference, latter is the base. Environment version "3" was found to be incompatible with "2" due to higher number of TRex workers used.' - - '2005.0-4 vs 2001.0-2: [% +- StDev] Relative difference, latter is the base. Used as a reference to compare benchmark results of VPP current release v20.05.0 vs previous release v20.01.0.' -# - ' RCA1: root cause analysis reference for 2005.0-4 vs 2005.0-3.' -# - ' RCA2: root cause analysis reference for 2005.0-4 vs 2001.0-2.' + - 'Test Case: Test cases executed for: MRR throughput on 3n-skx with xxv710.' + - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' + - '2005.0-4 vs 2005.0-3: Environment version "3" was found to be incompatible with "2" due to higher number of TRex workers used.' + - '2005.0-4 vs 2001.0-2: Used as a reference to compare benchmark results of VPP current release v20.05.0 vs previous release v20.01.0.' # VPP Performance Changes 3n-skx 4t2c mrr - type: "table" @@ -5785,8 +5529,6 @@ include-tests: "MRR" # "PDR" | "NDR" | "MRR" filter: "'4T2C' and 'MRR' and 'NIC_Intel-XXV710' and not 'NF_DENSITY'" columns: -# - title: "1908" -# data-set: "vpp-performance-changes-3n-skx-mrr-h1" - title: "2001.0-2" data-set: "vpp-performance-changes-3n-skx-mrr-ref" - title: "2005.0-3" @@ -5797,22 +5539,18 @@ - title: "2005.0-4 vs 2005.0-3" reference: 2 compare: 3 + rca-file: "rca/rca-2005-3n-skx-4t2c-mrr-rca1.yaml" - title: "2005.0-4 vs 2001.0-2" reference: 1 compare: 3 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-3n-skx-4t2c-mrr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-3n-skx-4t2c-mrr-rca2.yaml" + rca-file: "rca/rca-2005-3n-skx-4t2c-mrr-rca2.yaml" legend: - '' - - ' Test Case: Test cases executed for: PDR throughput on 3n-skx with xxv710.' - - ' YYMM.V-E: [Mpps +- StDev] results for VPP version (YY year, MM month, V minor version) tested in CSIT environment version (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - '2005.0-4 vs 2005.0-3: [% +- StDev] Relative difference, latter is the base. Environment version "3" was found to be incompatible with "2" due to higher number of TRex workers used.' - - '2005.0-4 vs 2001.0-2: [% +- StDev] Relative difference, latter is the base. Used as a reference to compare benchmark results of VPP current release v20.05.0 vs previous release v20.01.0.' -# - ' RCA1: root cause analysis reference for 2005.0-4 vs 2005.0-3.' -# - ' RCA2: root cause analysis reference for 2005.0-4 vs 2001.0-2.' + - 'Test Case: Test cases executed for: MRR throughput on 3n-skx with xxv710.' + - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' + - '2005.0-4 vs 2005.0-3: Environment version "3" was found to be incompatible with "2" due to higher number of TRex workers used.' + - '2005.0-4 vs 2001.0-2: Used as a reference to compare benchmark results of VPP current release v20.05.0 vs previous release v20.01.0.' # VPP Performance Changes 3n-skx 8t4c mrr - type: "table" @@ -5822,8 +5560,6 @@ include-tests: "MRR" # "PDR" | "NDR" | "MRR" filter: "'8T4C' and 'MRR' and 'NIC_Intel-XXV710' and not 'NF_DENSITY'" columns: -# - title: "1908" -# data-set: "vpp-performance-changes-3n-skx-mrr-h1" - title: "2001.0-2" data-set: "vpp-performance-changes-3n-skx-mrr-ref" - title: "2005.0-3" @@ -5834,22 +5570,18 @@ - title: "2005.0-4 vs 2005.0-3" reference: 2 compare: 3 + rca-file: "rca/rca-2005-3n-skx-8t4c-mrr-rca1.yaml" - title: "2005.0-4 vs 2001.0-2" reference: 1 compare: 3 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-3n-skx-8t4c-mrr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-3n-skx-8t4c-mrr-rca2.yaml" + rca-file: "rca/rca-2005-3n-skx-8t4c-mrr-rca2.yaml" legend: - '' - - ' Test Case: Test cases executed for: PDR throughput on 3n-skx with xxv710.' - - ' YYMM.V-E: [Mpps +- StDev] results for VPP version (YY year, MM month, V minor version) tested in CSIT environment version (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - '2005.0-4 vs 2005.0-3: [% +- StDev] Relative difference, latter is the base. Environment version "3" was found to be incompatible with "2" due to higher number of TRex workers used.' - - '2005.0-4 vs 2001.0-2: [% +- StDev] Relative difference, latter is the base. Used as a reference to compare benchmark results of VPP current release v20.05.0 vs previous release v20.01.0.' -# - ' RCA1: root cause analysis reference for 2005.0-4 vs 2005.0-3.' -# - ' RCA2: root cause analysis reference for 2005.0-4 vs 2001.0-2.' + - 'Test Case: Test cases executed for: MRR throughput on 3n-skx with xxv710.' + - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' + - '2005.0-4 vs 2005.0-3: Environment version "3" was found to be incompatible with "2" due to higher number of TRex workers used.' + - '2005.0-4 vs 2001.0-2: Used as a reference to compare benchmark results of VPP current release v20.05.0 vs previous release v20.01.0.' # VPP Performance Changes 2n-skx 2t1c mrr - type: "table" @@ -5869,22 +5601,18 @@ - title: "2005.0-4 vs 2005.0-3" reference: 2 compare: 3 + rca-file: "rca/rca-2005-2n-skx-2t1c-mrr-rca1.yaml" - title: "2005.0-4 vs 2001.0-2" reference: 1 compare: 3 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-2n-skx-2t1c-mrr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-2n-skx-2t1c-mrr-rca2.yaml" + rca-file: "rca/rca-2005-2n-skx-2t1c-mrr-rca2.yaml" legend: - '' - - ' Test Case: Test cases executed for: PDR throughput on 2n-skx with xxv710.' - - ' YYMM.V-E: [Mpps +- StDev] results for VPP version (YY year, MM month, V minor version) tested in CSIT environment version (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - '2005.0-4 vs 2005.0-3: [% +- StDev] Relative difference, latter is the base. Environment version "3" was found to be incompatible with "2" due to higher number of TRex workers used.' - - '2005.0-4 vs 2001.0-2: [% +- StDev] Relative difference, latter is the base. Used as a reference to compare benchmark results of VPP current release v20.05.0 vs previous release v20.01.0.' -# - ' RCA1: root cause analysis reference for 2005.0-4 vs 2005.0-3.' -# - ' RCA2: root cause analysis reference for 2005.0-4 vs 2001.0-2.' + - 'Test Case: Test cases executed for: MRR throughput on 2n-skx with xxv710.' + - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' + - '2005.0-4 vs 2005.0-3: Environment version "3" was found to be incompatible with "2" due to higher number of TRex workers used.' + - '2005.0-4 vs 2001.0-2: Used as a reference to compare benchmark results of VPP current release v20.05.0 vs previous release v20.01.0.' # VPP Performance Changes 2n-skx 4t2c mrr - type: "table" @@ -5904,22 +5632,18 @@ - title: "2005.0-4 vs 2005.0-3" reference: 2 compare: 3 + rca-file: "rca/rca-2005-2n-skx-4t2c-mrr-rca1.yaml" - title: "2005.0-4 vs 2001.0-2" reference: 1 compare: 3 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-2n-skx-4t2c-mrr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-2n-skx-4t2c-mrr-rca2.yaml" + rca-file: "rca/rca-2005-2n-skx-4t2c-mrr-rca2.yaml" legend: - '' - - ' Test Case: Test cases executed for: PDR throughput on 2n-skx with xxv710.' - - ' YYMM.V-E: [Mpps +- StDev] results for VPP version (YY year, MM month, V minor version) tested in CSIT environment version (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - '2005.0-4 vs 2005.0-3: [% +- StDev] Relative difference, latter is the base. Environment version "3" was found to be incompatible with "2" due to higher number of TRex workers used.' - - '2005.0-4 vs 2001.0-2: [% +- StDev] Relative difference, latter is the base. Used as a reference to compare benchmark results of VPP current release v20.05.0 vs previous release v20.01.0.' -# - ' RCA1: root cause analysis reference for 2005.0-4 vs 2005.0-3.' -# - ' RCA2: root cause analysis reference for 2005.0-4 vs 2001.0-2.' + - 'Test Case: Test cases executed for: MRR throughput on 2n-skx with xxv710.' + - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' + - '2005.0-4 vs 2005.0-3: Environment version "3" was found to be incompatible with "2" due to higher number of TRex workers used.' + - '2005.0-4 vs 2001.0-2: Used as a reference to compare benchmark results of VPP current release v20.05.0 vs previous release v20.01.0.' # VPP Performance Changes 2n-skx 8t4c mrr - type: "table" @@ -5939,22 +5663,18 @@ - title: "2005.0-4 vs 2005.0-3" reference: 2 compare: 3 + rca-file: "rca/rca-2005-2n-skx-8t4c-mrr-rca1.yaml" - title: "2005.0-4 vs 2001.0-2" reference: 1 compare: 3 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-2n-skx-8t4c-mrr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-2n-skx-8t4c-mrr-rca2.yaml" + rca-file: "rca/rca-2005-2n-skx-8t4c-mrr-rca2.yaml" legend: - '' - - ' Test Case: Test cases executed for: PDR throughput on 2n-skx with xxv710.' - - ' YYMM.V-E: [Mpps +- StDev] results for VPP version (YY year, MM month, V minor version) tested in CSIT environment version (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - '2005.0-4 vs 2005.0-3: [% +- StDev] Relative difference, latter is the base. Environment version "3" was found to be incompatible with "2" due to higher number of TRex workers used.' - - '2005.0-4 vs 2001.0-2: [% +- StDev] Relative difference, latter is the base. Used as a reference to compare benchmark results of VPP current release v20.05.0 vs previous release v20.01.0.' -# - ' RCA1: root cause analysis reference for 2005.0-4 vs 2005.0-3.' -# - ' RCA2: root cause analysis reference for 2005.0-4 vs 2001.0-2.' + - 'Test Case: Test cases executed for: MRR throughput on 2n-skx with xxv710.' + - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' + - '2005.0-4 vs 2005.0-3: Environment version "3" was found to be incompatible with "2" due to higher number of TRex workers used.' + - '2005.0-4 vs 2001.0-2: Used as a reference to compare benchmark results of VPP current release v20.05.0 vs previous release v20.01.0.' # VPP Performance Changes 2n-clx-xxv710 2t1c mrr - type: "table" @@ -5974,22 +5694,18 @@ - title: "2005.0-4 vs 2005.0-3" reference: 2 compare: 3 + rca-file: "rca/rca-2005-2n-clx-2t1c-mrr-rca1.yaml" - title: "2005.0-4 vs 2001.0-2" reference: 1 compare: 3 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-2n-clx-2t1c-mrr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-2n-clx-2t1c-mrr-rca2.yaml" + rca-file: "rca/rca-2005-2n-clx-2t1c-mrr-rca2.yaml" legend: - '' - - ' Test Case: Test cases executed for: PDR throughput on 2n-clx with xxv710.' - - ' YYMM.V-E: [Mpps +- StDev] results for VPP version (YY year, MM month, V minor version) tested in CSIT environment version (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - '2005.0-4 vs 2005.0-3: [% +- StDev] Relative difference, latter is the base. Environment version "3" was found to be incompatible with "2" due to higher number of TRex workers used.' - - '2005.0-4 vs 2001.0-2: [% +- StDev] Relative difference, latter is the base. Used as a reference to compare benchmark results of VPP current release v20.05.0 vs previous release v20.01.0.' -# - ' RCA1: root cause analysis reference for 2005.0-4 vs 2005.0-3.' -# - ' RCA2: root cause analysis reference for 2005.0-4 vs 2001.0-2.' + - 'Test Case: Test cases executed for: MRR throughput on 2n-clx with xxv710.' + - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' + - '2005.0-4 vs 2005.0-3: Environment version "3" was found to be incompatible with "2" due to higher number of TRex workers used.' + - '2005.0-4 vs 2001.0-2: Used as a reference to compare benchmark results of VPP current release v20.05.0 vs previous release v20.01.0.' # VPP Performance Changes 2n-clx-xxv710 4t2c mrr - type: "table" @@ -6009,22 +5725,18 @@ - title: "2005.0-4 vs 2005.0-3" reference: 2 compare: 3 + rca-file: "rca/rca-2005-2n-clx-4t2c-mrr-rca1.yaml" - title: "2005.0-4 vs 2001.0-2" reference: 1 compare: 3 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-2n-clx-4t2c-mrr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-2n-clx-4t2c-mrr-rca2.yaml" + rca-file: "rca/rca-2005-2n-clx-4t2c-mrr-rca2.yaml" legend: - '' - - ' Test Case: Test cases executed for: PDR throughput on 2n-clx with xxv710.' - - ' YYMM.V-E: [Mpps +- StDev] results for VPP version (YY year, MM month, V minor version) tested in CSIT environment version (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - '2005.0-4 vs 2005.0-3: [% +- StDev] Relative difference, latter is the base. Environment version "3" was found to be incompatible with "2" due to higher number of TRex workers used.' - - '2005.0-4 vs 2001.0-2: [% +- StDev] Relative difference, latter is the base. Used as a reference to compare benchmark results of VPP current release v20.05.0 vs previous release v20.01.0.' -# - ' RCA1: root cause analysis reference for 2005.0-4 vs 2005.0-3.' -# - ' RCA2: root cause analysis reference for 2005.0-4 vs 2001.0-2.' + - 'Test Case: Test cases executed for: MRR throughput on 2n-clx with xxv710.' + - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' + - '2005.0-4 vs 2005.0-3: Environment version "3" was found to be incompatible with "2" due to higher number of TRex workers used.' + - '2005.0-4 vs 2001.0-2: Used as a reference to compare benchmark results of VPP current release v20.05.0 vs previous release v20.01.0.' # VPP Performance Changes 2n-clx-xxv710 8t4c mrr - type: "table" @@ -6044,22 +5756,18 @@ - title: "2005.0-4 vs 2005.0-3" reference: 2 compare: 3 + rca-file: "rca/rca-2005-2n-clx-8t4c-mrr-rca1.yaml" - title: "2005.0-4 vs 2001.0-2" reference: 1 compare: 3 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-2n-clx-8t4c-mrr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-2n-clx-8t4c-mrr-rca2.yaml" + rca-file: "rca/rca-2005-2n-clx-8t4c-mrr-rca2.yaml" legend: - '' - - ' Test Case: Test cases executed for: PDR throughput on 2n-clx with xxv710.' - - ' YYMM.V-E: [Mpps +- StDev] results for VPP version (YY year, MM month, V minor version) tested in CSIT environment version (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - '2005.0-4 vs 2005.0-3: [% +- StDev] Relative difference, latter is the base. Environment version "3" was found to be incompatible with "2" due to higher number of TRex workers used.' - - '2005.0-4 vs 2001.0-2: [% +- StDev] Relative difference, latter is the base. Used as a reference to compare benchmark results of VPP current release v20.05.0 vs previous release v20.01.0.' -# - ' RCA1: root cause analysis reference for 2005.0-4 vs 2005.0-3.' -# - ' RCA2: root cause analysis reference for 2005.0-4 vs 2001.0-2.' + - 'Test Case: Test cases executed for: MRR throughput on 2n-clx with xxv710.' + - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' + - '2005.0-4 vs 2005.0-3: Environment version "3" was found to be incompatible with "2" due to higher number of TRex workers used.' + - '2005.0-4 vs 2001.0-2: Used as a reference to compare benchmark results of VPP current release v20.05.0 vs previous release v20.01.0.' # VPP Performance Changes 2n-clx-clx-x556 2t1c mrr - type: "table" @@ -6282,22 +5990,18 @@ - title: "2005.0-4 vs 2005.0-3" reference: 3 compare: 4 + rca-file: "rca/rca-2005-3n-tsh-1t1c-mrr-rca1.yaml" - title: "2005.0-4 vs 2001.0-2" reference: 2 compare: 4 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-3n-tsh-1t1c-mrr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-3n-tsh-1t1c-mrr-rca2.yaml" + rca-file: "rca/rca-2005-3n-tsh-1t1c-mrr-rca2.yaml" legend: - '' - - ' Test Case: Test cases executed for: PDR throughput on 3n-tsh with x520.' - - ' YYMM.V-E: [Mpps +- StDev] results for VPP version (YY year, MM month, V minor version) tested in CSIT environment version (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - '2005.0-4 vs 2005.0-3: [% +- StDev] Relative difference, latter is the base. Environment version "3" was found to be incompatible with "2" due to higher number of TRex workers used.' - - '2005.0-4 vs 2001.0-2: [% +- StDev] Relative difference, latter is the base. Used as a reference to compare benchmark results of VPP current release v20.05.0 vs previous release v20.01.0.' -# - ' RCA1: root cause analysis reference for 2005.0-4 vs 2005.0-3.' -# - ' RCA2: root cause analysis reference for 2005.0-4 vs 2001.0-2.' + - 'Test Case: Test cases executed for: MRR throughput on 3n-tsh with x520.' + - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' + - '2005.0-4 vs 2005.0-3: Environment version "3" was found to be incompatible with "2" due to higher number of TRex workers used.' + - '2005.0-4 vs 2001.0-2: Used as a reference to compare benchmark results of VPP current release v20.05.0 vs previous release v20.01.0.' # VPP Performance Changes 3n-tsh 2t2c mrr - type: "table" @@ -6319,22 +6023,18 @@ - title: "2005.0-4 vs 2005.0-3" reference: 3 compare: 4 + rca-file: "rca/rca-2005-3n-tsh-2t2c-mrr-rca1.yaml" - title: "2005.0-4 vs 2001.0-2" reference: 2 compare: 4 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-3n-tsh-2t2c-mrr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-3n-tsh-2t2c-mrr-rca2.yaml" + rca-file: "rca/rca-2005-3n-tsh-2t2c-mrr-rca2.yaml" legend: - '' - - ' Test Case: Test cases executed for: PDR throughput on 3n-tsh with x520.' - - ' YYMM.V-E: [Mpps +- StDev] results for VPP version (YY year, MM month, V minor version) tested in CSIT environment version (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - '2005.0-4 vs 2005.0-3: [% +- StDev] Relative difference, latter is the base. Environment version "3" was found to be incompatible with "2" due to higher number of TRex workers used.' - - '2005.0-4 vs 2001.0-2: [% +- StDev] Relative difference, latter is the base. Used as a reference to compare benchmark results of VPP current release v20.05.0 vs previous release v20.01.0.' -# - ' RCA1: root cause analysis reference for 2005.0-4 vs 2005.0-3.' -# - ' RCA2: root cause analysis reference for 2005.0-4 vs 2001.0-2.' + - 'Test Case: Test cases executed for: MRR throughput on 3n-tsh with x520.' + - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' + - '2005.0-4 vs 2005.0-3: Environment version "3" was found to be incompatible with "2" due to higher number of TRex workers used.' + - '2005.0-4 vs 2001.0-2: Used as a reference to compare benchmark results of VPP current release v20.05.0 vs previous release v20.01.0.' # VPP Performance Changes 3n-tsh 4t4c mrr - type: "table" @@ -6356,22 +6056,18 @@ - title: "2005.0-4 vs 2005.0-3" reference: 3 compare: 4 + rca-file: "rca/rca-2005-3n-tsh-4t4c-mrr-rca1.yaml" - title: "2005.0-4 vs 2001.0-2" reference: 2 compare: 4 -# rca: -# - title: "RCA1" -# data: "rca/rca-2005-3n-tsh-4t4c-mrr-rca1.yaml" -# - title: "RCA2" -# data: "rca/rca-2005-3n-tsh-4t4c-mrr-rca2.yaml" + rca-file: "rca/rca-2005-3n-tsh-4t4c-mrr-rca2.yaml" legend: - '' - - ' Test Case: Test cases executed for: PDR throughput on 3n-tsh with x520.' - - ' YYMM.V-E: [Mpps +- StDev] results for VPP version (YY year, MM month, V minor version) tested in CSIT environment version (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' - - '2005.0-4 vs 2005.0-3: [% +- StDev] Relative difference, latter is the base. Environment version "3" was found to be incompatible with "2" due to higher number of TRex workers used.' - - '2005.0-4 vs 2001.0-2: [% +- StDev] Relative difference, latter is the base. Used as a reference to compare benchmark results of VPP current release v20.05.0 vs previous release v20.01.0.' -# - ' RCA1: root cause analysis reference for 2005.0-4 vs 2005.0-3.' -# - ' RCA2: root cause analysis reference for 2005.0-4 vs 2001.0-2.' + - 'Test Case: Test cases executed for: MRR throughput on 3n-tsh with x520.' + - 'YYMM.V-E: [Mpps +- StDev] results for VPP (YY year, MM month, V minor version) tested in CSIT (E environment version, "2" for 2020-03-27, "3" for 2020-06-11, "4" for 2020-06-24).' + - 'YYMM.V-E vs YYMM.V-E: [% +- StDev] Relative difference, latter is the base. [N] for Root Cause Analysis (RCA) reference, see RCA list below.' + - '2005.0-4 vs 2005.0-3: Environment version "3" was found to be incompatible with "2" due to higher number of TRex workers used.' + - '2005.0-4 vs 2001.0-2: Used as a reference to compare benchmark results of VPP current release v20.05.0 vs previous release v20.01.0.' # VPP Comparison Across Testbeds 3n-hsw to 3n-skx 1c mrr - type: "table" |