diff options
4 files changed, 88 insertions, 93 deletions
diff --git a/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst b/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst index d329e09150..30b428eb6f 100644 --- a/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst +++ b/docs/report/vpp_performance_tests/comparisons/current_vs_previous_release.rst @@ -56,19 +56,19 @@ Comparison tables in HTML, ASCII and CSV formats: - `ASCII 2t1c PDR RCA <../../_static/vpp/performance-rca-2n-skx-2t1c-pdr.txt>`_ - `CSV 2t1c PDR RCA <../../_static/vpp/performance-rca-2n-skx-2t1c-pdr-csv.csv>`_ -NDR Comparison -`````````````` +.. + ``NDR Comparison + `````````````` -Comparison tables in HTML, ASCII and CSV formats: + Comparison tables in HTML, ASCII and CSV formats: - - `HTML 2t1c NDR comparison <performance-changes-2n-skx-2t1c-ndr.html>`_ - - `HTML 4t2c NDR comparison <performance-changes-2n-skx-4t2c-ndr.html>`_ - - `ASCII 2t1c NDR comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-ndr.txt>`_ - - `ASCII 4t2c NDR comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-ndr.txt>`_ - - `CSV 2t1c NDR comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-ndr.csv>`_ - - `CSV 4t2c NDR comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-ndr.csv>`_ + - `HTML 2t1c NDR comparison <performance-changes-2n-skx-2t1c-ndr.html>`_ + - `HTML 4t2c NDR comparison <performance-changes-2n-skx-4t2c-ndr.html>`_ + - `ASCII 2t1c NDR comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-ndr.txt>`_ + - `ASCII 4t2c NDR comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-ndr.txt>`_ + - `CSV 2t1c NDR comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-ndr.csv>`_ + - `CSV 4t2c NDR comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-ndr.csv>`_ -.. - `HTML 2t1c NFV Service Density NDR comparison <performance-changes-nfv-2n-skx-2t1c-ndr.html>`_ - `HTML 4t2c NFV Service Density NDR comparison <performance-changes-nfv-2n-skx-4t2c-ndr.html>`_ - `HTML 8t4c NFV Service Density NDR comparison <performance-changes-nfv-2n-skx-8t4c-ndr.html>`_ @@ -79,19 +79,18 @@ Comparison tables in HTML, ASCII and CSV formats: - `CSV 4t2c NFV Service Density NDR comparison <../../_static/vpp/performance-changes-nfv-2n-skx-4t2c-ndr.csv>`_ - `CSV 8t4c NFV Service Density NDR comparison <../../_static/vpp/performance-changes-nfv-2n-skx-8t4c-ndr.csv>`_ -PDR Comparison -`````````````` + PDR Comparison + `````````````` -Comparison tables in HTML, ASCII and CSV formats: + Comparison tables in HTML, ASCII and CSV formats: - - `HTML 2t1c PDR comparison <performance-changes-2n-skx-2t1c-pdr.html>`_ - - `HTML 4t2c PDR comparison <performance-changes-2n-skx-4t2c-pdr.html>`_ - - `ASCII 2t1c PDR comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-pdr.txt>`_ - - `ASCII 4t2c PDR comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-pdr.txt>`_ - - `CSV 2t1c PDR comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-pdr.csv>`_ - - `CSV 4t2c PDR comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-pdr.csv>`_ + - `HTML 2t1c PDR comparison <performance-changes-2n-skx-2t1c-pdr.html>`_ + - `HTML 4t2c PDR comparison <performance-changes-2n-skx-4t2c-pdr.html>`_ + - `ASCII 2t1c PDR comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-pdr.txt>`_ + - `ASCII 4t2c PDR comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-pdr.txt>`_ + - `CSV 2t1c PDR comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-pdr.csv>`_ + - `CSV 4t2c PDR comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-pdr.csv>`_ -.. - `HTML 2t1c NFV Service Density PDR comparison <performance-changes-nfv-2n-skx-2t1c-pdr.html>`_ - `HTML 4t2c NFV Service Density PDR comparison <performance-changes-nfv-2n-skx-4t2c-pdr.html>`_ - `HTML 8t4c NFV Service Density PDR comparison <performance-changes-nfv-2n-skx-8t4c-pdr.html>`_ @@ -102,22 +101,22 @@ Comparison tables in HTML, ASCII and CSV formats: - `CSV 4t2c NFV Service Density PDR comparison <../../_static/vpp/performance-changes-nfv-2n-skx-4t2c-pdr.csv>`_ - `CSV 8t4c NFV Service Density PDR comparison <../../_static/vpp/performance-changes-nfv-2n-skx-8t4c-pdr.csv>`_ -MRR Comparison -`````````````` + MRR Comparison + `````````````` -Comparison tables in HTML, ASCII and CSV formats: + Comparison tables in HTML, ASCII and CSV formats: + + - `HTML 2t1c MRR comparison <performance-changes-2n-skx-2t1c-mrr.html>`_ + - `HTML 4t2c MRR comparison <performance-changes-2n-skx-4t2c-mrr.html>`_ + - `HTML 8t4c MRR comparison <performance-changes-2n-skx-8t4c-mrr.html>`_ + - `ASCII 2t1c MRR comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-mrr.txt>`_ + - `ASCII 4t2c MRR comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-mrr.txt>`_ + - `ASCII 8t4c MRR comparison <../../_static/vpp/performance-changes-2n-skx-8t4c-mrr.txt>`_ + - `CSV 2t1c MRR comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-mrr.csv>`_ + - `CSV 4t2c MRR comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-mrr.csv>`_ + - `CSV 8t4c MRR comparison <../../_static/vpp/performance-changes-2n-skx-8t4c-mrr.csv>`_ - - `HTML 2t1c MRR comparison <performance-changes-2n-skx-2t1c-mrr.html>`_ - - `HTML 4t2c MRR comparison <performance-changes-2n-skx-4t2c-mrr.html>`_ - - `HTML 8t4c MRR comparison <performance-changes-2n-skx-8t4c-mrr.html>`_ - - `ASCII 2t1c MRR comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-mrr.txt>`_ - - `ASCII 4t2c MRR comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-mrr.txt>`_ - - `ASCII 8t4c MRR comparison <../../_static/vpp/performance-changes-2n-skx-8t4c-mrr.txt>`_ - - `CSV 2t1c MRR comparison <../../_static/vpp/performance-changes-2n-skx-2t1c-mrr.csv>`_ - - `CSV 4t2c MRR comparison <../../_static/vpp/performance-changes-2n-skx-4t2c-mrr.csv>`_ - - `CSV 8t4c MRR comparison <../../_static/vpp/performance-changes-2n-skx-8t4c-mrr.csv>`_ -.. - `HTML 2t1c NFV Service Density MRR comparison <performance-changes-nfv-2n-skx-2t1c-mrr.html>`_ - `HTML 4t2c NFV Service Density MRR comparison <performance-changes-nfv-2n-skx-4t2c-mrr.html>`_ - `HTML 8t4c NFV Service Density MRR comparison <performance-changes-nfv-2n-skx-8t4c-mrr.html>`_ diff --git a/resources/tools/presentation/generator_tables.py b/resources/tools/presentation/generator_tables.py index 468b9c642d..14130e5059 100644 --- a/resources/tools/presentation/generator_tables.py +++ b/resources/tools/presentation/generator_tables.py @@ -1000,8 +1000,8 @@ def table_perf_comparison(table, input_data): if rca_data: footnote = rca_data.get(u"footnote", u"") if footnote: - txt_file.write(footnote) - txt_file.write(u":END") + txt_file.write(f"\n{footnote}") + txt_file.write(u"\n:END") # Generate html table: _tpc_generate_html_table( @@ -1371,8 +1371,8 @@ def table_perf_comparison_nic(table, input_data): if rca_data: footnote = rca_data.get(u"footnote", u"") if footnote: - txt_file.write(footnote) - txt_file.write(u":END") + txt_file.write(f"\n{footnote}") + txt_file.write(u"\n:END") # Generate html table: _tpc_generate_html_table( @@ -2589,7 +2589,7 @@ def table_comparison(table, input_data): header_csv.append(f"Stdev({col[u'title']})") for comp in comparisons: header_csv.append( - f"Avg({comp.get(u'title', u'')}" + f"Avg({comp.get(u'title', u'')})" ) header_csv.append( f"Stdev({comp.get(u'title', u'')})" @@ -2686,7 +2686,7 @@ def table_comparison(table, input_data): txt_file.write(legend) if footnote: txt_file.write(footnote) - txt_file.write(u":END") + txt_file.write(u"\n:END") # Generate html table: _tpc_generate_html_table( diff --git a/resources/tools/presentation/rca/rca-2n-skx-2t1c-pdr-rca1.yaml b/resources/tools/presentation/rca/rca-2n-skx-2t1c-pdr-rca1.yaml index 45d1ba46c7..f8e93e08ea 100644 --- a/resources/tools/presentation/rca/rca-2n-skx-2t1c-pdr-rca1.yaml +++ b/resources/tools/presentation/rca/rca-2n-skx-2t1c-pdr-rca1.yaml @@ -1,30 +1,24 @@ -64b-2t1c-eth-l2patch: 2 -64b-2t1c-eth-l2xcbase-eth-2memif-1dcr: 3 +64b-2t1c-avf-dot1q-ip4base: 1 +64b-2t1c-avf-dot1q-l2bdbasemaclrn-gbp: 1 64b-2t1c-avf-eth-l2patch: 2 64b-2t1c-avf-eth-l2xcbase: 2 -64b-2t1c-eth-l2bdbasemaclrn-eth-2memif-1dcr: 3 -64b-2t1c-eth-l2bdbasemaclrn-eth-2vhostvr1024-1vm-vppl2xc: 4 -64b-2t1c-dot1q-l2bdbasemaclrn-eth-2memif-1dcr: 3 -64b-2t1c-dot1q-l2xcbase: 5 -64b-2t1c-ethip4-ip4base-eth-2vhostvr1024-1vm-vppip4: 4 +64b-2t1c-avf-ethip4-ip4base: 1 +64b-2t1c-eth-l2patch: 2 64b-2t1c-eth-l2xcbase-eth-2vhostvr1024-1vm-vppl2xc: 4 -64b-2t1c-dot1q-l2bdbasemaclrn-eth-2vhostvr1024-1vm-vppl2xc: 4 -64b-2t1c-ethip4-ip4base-eth-2memif-1dcr: 3 -64b-2t1c-eth-l2bdbasemaclrn-eth-2vhostvr1024-1vm: 4 -64b-2t1c-eth-l2xcbase-eth-2vhostvr1024-1vm: 4 -64b-2t1c-ethip4-ip4base-eth-2vhostvr1024-1vm: 4 -64b-2t1c-avf-dot1q-l2bdbasemaclrn: 1 -64b-2t1c-dot1q-ip4base: 1 64b-2t1c-ethip4-ip4base: 1 64b-2t1c-ethip4udp-ip4base-iacl50sf-10kflows: 1 -64b-2t1c-ethip4udp-ip4base-iacl50sl-10kflows: 1 -64b-2t1c-avf-dot1q-l2bdbasemaclrn-gbp: 1 -64b-2t1c-ethip4-ip4scale200k: 1 -64b-2t1c-ethip4-ip4scale2m: 1 -64b-2t1c-avf-dot1q-ip4base: 1 -64b-2t1c-avf-ethip4-ip4base: 1 -64b-2t1c-avf-ethip4-ip4scale200k: 1 -64b-2t1c-avf-ethip4-ip4scale20k: 1 -64b-2t1c-avf-ethip4-ip4scale2m: 1 -64b-2t1c-ethip4udp-ip4base-nat44: 1 -footnote: "[1] Impact of Skx ucode upgrade from 0x2000043 to 0x2000065 in combination with SuperMicro motherboards/firmware and kernel updates, subject to the ongoing detailed RCA investigation with Intel NPG.\n[2] Applied fix of FVL NIC firmware 6.0.1 for increasing TRex Mpps rate from 27 Mpps to 37 Mpps, [CSIT-1503], [TRex-519].\n[3] Applied VPP PAPI fix to enable memif zero-copy, [CSIT-1592], [VPP-1764].\n[4] To-Be-Investigated, vhost-user avg PDR throughput rate has higher stdev than before.\n[5] To-Be-Investigated, dot1q-l2xc with DPDK FVL.\n" +64b-2t1c-ethip4udp-ip4base-oacl50sl-10kflows: 1 +# Multiline string, empty lines (or \n) are required, see https://yaml-multiline.info/ +footnote: " [1] Impact of Skx ucode upgrade from 0x2000043 to 0x2000065 + in combination with SuperMicro motherboards/firmware and kernel + updates, subjecto to the ongoing detailed RCA investigation with + Intel NPG. + + [2] Applied fix of FVL NIC firmware 6.0.1 for increasing TRex Mpps + rate from 27 Mpps to 37 Mpps, [CSIT-1503], [TRex-519]. + + [3] Applied VPP PAPI fix to enable memif zero-copy, [CSIT-1592], + [VPP-1764]. + + [4] VPP vhost-user (with vpp inside VM) PDR throughput rate has + somewhat higher stdev than before." diff --git a/resources/tools/presentation/rca/rca-2n-skx-2t1c-pdr-rca2.yaml b/resources/tools/presentation/rca/rca-2n-skx-2t1c-pdr-rca2.yaml index 45d1ba46c7..0192d74329 100644 --- a/resources/tools/presentation/rca/rca-2n-skx-2t1c-pdr-rca2.yaml +++ b/resources/tools/presentation/rca/rca-2n-skx-2t1c-pdr-rca2.yaml @@ -1,30 +1,32 @@ -64b-2t1c-eth-l2patch: 2 -64b-2t1c-eth-l2xcbase-eth-2memif-1dcr: 3 -64b-2t1c-avf-eth-l2patch: 2 -64b-2t1c-avf-eth-l2xcbase: 2 -64b-2t1c-eth-l2bdbasemaclrn-eth-2memif-1dcr: 3 -64b-2t1c-eth-l2bdbasemaclrn-eth-2vhostvr1024-1vm-vppl2xc: 4 -64b-2t1c-dot1q-l2bdbasemaclrn-eth-2memif-1dcr: 3 +64b-2t1c-avf-dot1q-ip4base: 10 +64b-2t1c-avf-ethip4-ip4base: 10 +64b-2t1c-avf-ethip4-ip4scale200k: 7 +64b-2t1c-avf-ethip4-ip4scale20k: 7 +64b-2t1c-avf-ethip4-ip4scale2m: 7 +64b-2t1c-dot1q-l2bdbasemaclrn-eth-2vhostvr1024-1vm-vppl2xc: 8 64b-2t1c-dot1q-l2xcbase: 5 -64b-2t1c-ethip4-ip4base-eth-2vhostvr1024-1vm-vppip4: 4 -64b-2t1c-eth-l2xcbase-eth-2vhostvr1024-1vm-vppl2xc: 4 -64b-2t1c-dot1q-l2bdbasemaclrn-eth-2vhostvr1024-1vm-vppl2xc: 4 -64b-2t1c-ethip4-ip4base-eth-2memif-1dcr: 3 -64b-2t1c-eth-l2bdbasemaclrn-eth-2vhostvr1024-1vm: 4 -64b-2t1c-eth-l2xcbase-eth-2vhostvr1024-1vm: 4 -64b-2t1c-ethip4-ip4base-eth-2vhostvr1024-1vm: 4 -64b-2t1c-avf-dot1q-l2bdbasemaclrn: 1 -64b-2t1c-dot1q-ip4base: 1 -64b-2t1c-ethip4-ip4base: 1 -64b-2t1c-ethip4udp-ip4base-iacl50sf-10kflows: 1 -64b-2t1c-ethip4udp-ip4base-iacl50sl-10kflows: 1 -64b-2t1c-avf-dot1q-l2bdbasemaclrn-gbp: 1 -64b-2t1c-ethip4-ip4scale200k: 1 -64b-2t1c-ethip4-ip4scale2m: 1 -64b-2t1c-avf-dot1q-ip4base: 1 -64b-2t1c-avf-ethip4-ip4base: 1 -64b-2t1c-avf-ethip4-ip4scale200k: 1 -64b-2t1c-avf-ethip4-ip4scale20k: 1 -64b-2t1c-avf-ethip4-ip4scale2m: 1 -64b-2t1c-ethip4udp-ip4base-nat44: 1 -footnote: "[1] Impact of Skx ucode upgrade from 0x2000043 to 0x2000065 in combination with SuperMicro motherboards/firmware and kernel updates, subject to the ongoing detailed RCA investigation with Intel NPG.\n[2] Applied fix of FVL NIC firmware 6.0.1 for increasing TRex Mpps rate from 27 Mpps to 37 Mpps, [CSIT-1503], [TRex-519].\n[3] Applied VPP PAPI fix to enable memif zero-copy, [CSIT-1592], [VPP-1764].\n[4] To-Be-Investigated, vhost-user avg PDR throughput rate has higher stdev than before.\n[5] To-Be-Investigated, dot1q-l2xc with DPDK FVL.\n" +64b-2t1c-eth-l2bdbasemaclrn-eth-2vhostvr1024-1vm-vppl2xc: 8 +64b-2t1c-eth-l2xcbase-eth-2memif-1dcr: 11 +64b-2t1c-eth-l2xcbase-eth-2vhostvr1024-1vm-vppl2xc: 8 +64b-2t1c-eth-l2xcbase-eth-2vhostvr1024-1vm: 8 +64b-2t1c-ethip4-ip4base-eth-2memif-1dcr: 11 +64b-2t1c-ethip4udp-ip4base-nat44: 6 +# Multiline string, empty lines (or \n) are required, see https://yaml-multiline.info/ +footnote: "[5] Unknown VPP progression, retro-inspection of weekly + ndrpdr tests points to https://gerrit.fd.io/r/c/vpp/+/22805, + automated bisect script does not work due to frequent API changes. + + [6] Known VPP Regression: + https://gerrit.fd.io/r/c/vpp/+/23963#message-044278e6_752c3327 + + [7] Unknown VPP regression in avf-ip4scale tests. + + [8] VPP vhost-user (with vpp inside VM) PDR throughput rate has + somewhat higher stdev than before. + + [9] Unknown VPP Progression in vhost-user tests (with testpmd inside + VM). + + [10] Unknown VPP regression in avf-ip4base tests. + + [11] Unknown VPP regression in memif tests." |