diff options
| author |   Tibor Frank <tifrank@cisco.com> | 2018-07-30 09:49:06 +0200 |
|---|---|---|
| committer | Tibor Frank <tifrank@cisco.com> | 2018-07-30 13:03:18 +0000 |
| commit | ce1088d88744f2c040801c9852565d522b3feb68 (patch) | |
| tree | 3683c1e740da518fc80d94e8b69e4b2cbad841c1 | |
| parent | 84f46d579b405873ce9746a0bae2e8fdc38bd415 (diff) | |
CSIT-1197: Add Comparison Across Testbeds to the Report
Change-Id: Ibf0c880926b87e781830d7b39438b5145144ef5b
Signed-off-by: Tibor Frank <tifrank@cisco.com>
37 files changed, 963 insertions, 2372 deletions
diff --git a/docs/report/detailed_test_results/vpp_functional_results_centos/index.rst b/docs/report/detailed_test_results/vpp_functional_results_centos/index.rst index 66a5167cdc..e621f2e042 100644 --- a/docs/report/detailed_test_results/vpp_functional_results_centos/index.rst +++ b/docs/report/detailed_test_results/vpp_functional_results_centos/index.rst @@ -4,8 +4,8 @@ CentOS .. note:: Data sources for reported test results: i) `FD.io test executor vpp - functional jobs`_, ii) archived FD.io jobs test result `output files - <../../_static/archive/>`_. + functional jobs using CentOs`_, ii) archived FD.io jobs test result + `output files <../../_static/archive/>`_. .. toctree:: diff --git a/docs/report/detailed_test_results/vpp_functional_results_ubuntu/index.rst b/docs/report/detailed_test_results/vpp_functional_results_ubuntu/index.rst index 4fc7924293..738920e6df 100644 --- a/docs/report/detailed_test_results/vpp_functional_results_ubuntu/index.rst +++ b/docs/report/detailed_test_results/vpp_functional_results_ubuntu/index.rst @@ -4,8 +4,8 @@ Ubuntu .. note:: Data sources for reported test results: i) `FD.io test executor vpp - functional jobs`_, ii) archived FD.io jobs test result `output files - <../../_static/archive/>`_. + functional jobs using Ubuntu`_, ii) archived FD.io jobs test result + `output files <../../_static/archive/>`_. .. toctree:: diff --git a/docs/report/dpdk_performance_tests/csit_release_notes.rst b/docs/report/dpdk_performance_tests/csit_release_notes.rst index 214eb4c4e7..363c2e70f1 100644 --- a/docs/report/dpdk_performance_tests/csit_release_notes.rst +++ b/docs/report/dpdk_performance_tests/csit_release_notes.rst @@ -6,6 +6,100 @@ Changes in |csit-release| No code changes apart from bug fixes. +Performance Changes +------------------- + +Relative performance changes in measured packet throughput in |csit-release| +are calculated against the results from |csit-release-1| +report. Listed mean and standard deviation values are computed based on +a series of the same tests executed against respective VPP releases to +verify test results repeatability, with percentage change calculated for +mean values. Note that the standard deviation is quite high for a small +number of packet throughput tests, what indicates poor test results +repeatability and makes the relative change of mean throughput value not +fully representative for these tests. The root causes behind poor +results repeatability vary between the test cases. + +NDR Changes +~~~~~~~~~~~ + +NDR throughput changes between releases are available in a +CSV and pretty ASCII formats: + + - `csv format for 1t1c <../_static/dpdk/performance-changes-1t1c-ndr.csv>`_, + - `csv format for 2t2c <../_static/dpdk/performance-changes-2t2c-ndr.csv>`_, + - `pretty ASCII format for 1t1c <../_static/dpdk/performance-changes-1t1c-ndr.txt>`_, + - `pretty ASCII format for 2t2c <../_static/dpdk/performance-changes-2t2c-ndr.txt>`_. + +.. note:: + + Test results have been generated by + `FD.io test executor dpdk performance job 3n-hsw`_ + with Robot Framework result + files csit-vpp-perf-|srelease|-\*.zip + `archived here <../_static/archive/>`_. + +PDR Changes +~~~~~~~~~~~ + +PDR throughput changes between releases are available in a +CSV and pretty ASCII formats: + + - `csv format for 1t1c <../_static/dpdk/performance-changes-1t1c-pdr.csv>`_, + - `csv format for 2t2c <../_static/dpdk/performance-changes-2t2c-pdr.csv>`_, + - `pretty ASCII format for 1t1c <../_static/dpdk/performance-changes-1t1c-pdr.txt>`_, + - `pretty ASCII format for 2t2c <../_static/dpdk/performance-changes-2t2c-pdr.txt>`_. + +.. note:: + + Test results have been generated by + `FD.io test executor dpdk performance job 3n-hsw`_ + with Robot Framework result + files csit-vpp-perf-|srelease|-\*.zip + `archived here <../_static/archive/>`_. + +Comparison Across Testbeds +-------------------------- + +Relative performance changes in measured packet throughputon 3-Node Skx testbed +are calculated against the results measured on 3-Node Hsw testbed. + +NDR Changes +~~~~~~~~~~~ + +NDR changes between testbeds are available in a +CSV and pretty ASCII formats: + + - `csv format for ndr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-ndr.csv>`_, + - `pretty ASCII format for ndr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-ndr.txt>`_. + +.. note:: + + Test results have been generated by + `FD.io test executor dpdk performance job 3n-hsw`_ and + `FD.io test executor dpdk performance job 3n-skx`_ + with Robot Framework result + files csit-vpp-perf-|srelease|-\*.zip + `archived here <../_static/archive/>`_. + +PDR Changes +~~~~~~~~~~~ + +PDR throughput changes between testbeds are available in a +CSV and pretty ASCII formats: + + - `csv format for pdr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-pdr.csv>`_, + - `pretty ASCII format for pdr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-pdr.txt>`_. + +.. note:: + + Test results have been generated by + `FD.io test executor dpdk performance job 3n-hsw`_ and + `FD.io test executor dpdk performance job 3n-skx`_ + with Robot Framework result + files csit-vpp-perf-|srelease|-\*.zip + `archived here <../_static/archive/>`_. + Known Issues ------------ diff --git a/docs/report/dpdk_performance_tests/overview.rst b/docs/report/dpdk_performance_tests/overview.rst index 037300b3ee..b38f9595be 100644 --- a/docs/report/dpdk_performance_tests/overview.rst +++ b/docs/report/dpdk_performance_tests/overview.rst @@ -116,7 +116,7 @@ Performance Tests Naming ------------------------ |csit-release| follows a common structured naming convention for all performance -and system functional tests, introduced in CSIT rls1701. +and system functional tests, introduced in CSIT-17.01. The naming should be intuitive for majority of the tests. Complete description of CSIT test naming convention is provided on :ref:`csit_test_naming`. diff --git a/docs/report/dpdk_performance_tests/packet_latency_graphs/index.rst b/docs/report/dpdk_performance_tests/packet_latency_graphs/index.rst index 1e59d1eae2..014d1095a7 100644 --- a/docs/report/dpdk_performance_tests/packet_latency_graphs/index.rst +++ b/docs/report/dpdk_performance_tests/packet_latency_graphs/index.rst @@ -8,7 +8,7 @@ latency per test. *Title of each graph* is a regex (regular expression) matching all throughput test cases plotted on this graph, *X-axis labels* are indices of individual test suites executed by -`FD.io test executor dpdk performance jobs`_ that created result output file +FD.io test executor dpdk performance jobs that created result output file used as data source for the graph, *Y-axis labels* are measured packet Latency [uSec] values, and the *Graph legend* lists the plotted test suites and their indices. Latency is reported for concurrent symmetric bi-directional flows, @@ -19,8 +19,13 @@ TGint2-to-SUT2-to-SUT1-to-TGint1. .. note:: Test results have been generated by - `FD.io test executor dpdk performance jobs`_ with Robot Framework result - files csit-dpdk-perf-\*.zip `archived here <../../_static/archive/>`_. + `FD.io test executor dpdk performance job 3n-hsw`_, + `FD.io test executor dpdk performance job 3n-skx`_ and + `FD.io test executor dpdk performance job 2n-skx`_ with Robot Framework + result files csit-vpp-perf-|srelease|-\*.zip + `archived here <../../_static/archive/>`_. + Plotted data set size per test case is equal to the number of job executions + presented in this report version: **10**. .. toctree:: :maxdepth: 1 diff --git a/docs/report/dpdk_performance_tests/packet_throughput_graphs/index.rst b/docs/report/dpdk_performance_tests/packet_throughput_graphs/index.rst index 42dfe4f572..a243922798 100644 --- a/docs/report/dpdk_performance_tests/packet_throughput_graphs/index.rst +++ b/docs/report/dpdk_performance_tests/packet_throughput_graphs/index.rst @@ -20,7 +20,7 @@ have the same value, only a horizontal line is plotted. *Title of each graph* is a regex (regular expression) matching all throughput test cases plotted on this graph, *X-axis labels* are indices of individual test suites executed by -`FD.io test executor dpdk performance jobs`_ jobs that created result output +FD.io test executor dpdk performance jobs that created result output files used as data sources for the graph, *Y-axis labels* are measured Packets Per Second [pps] values, and the *Graph legend* lists the plotted test suites and their indices. @@ -28,8 +28,11 @@ and their indices. .. note:: Test results have been generated by - `FD.io test executor dpdk performance jobs`_ with Robot Framework result - files csit-dpdk-perf-\*.zip `archived here <../../_static/archive/>`_. + `FD.io test executor dpdk performance job 3n-hsw`_, + `FD.io test executor dpdk performance job 3n-skx`_ and + `FD.io test executor dpdk performance job 2n-skx`_ with Robot Framework + result files csit-vpp-perf-|srelease|-\*.zip + `archived here <../../_static/archive/>`_. Plotted data set size per test case is equal to the number of job executions presented in this report version: **10**. diff --git a/docs/report/honeycomb_functional_tests/overview.rst b/docs/report/honeycomb_functional_tests/overview.rst index 9100ed6e2c..f67f735c9d 100644 --- a/docs/report/honeycomb_functional_tests/overview.rst +++ b/docs/report/honeycomb_functional_tests/overview.rst @@ -179,7 +179,7 @@ the DUT node. Functional Tests Naming ----------------------- -|csit-release| introduced a common structured naming convention for all +CSIT-17.01 introduced a common structured naming convention for all performance and functional tests. This change was driven by substantially growing number and type of CSIT test cases. Firstly, the original practice did not always follow any strict naming convention. Secondly test names did not diff --git a/docs/report/introduction/methodology.rst b/docs/report/introduction/methodology.rst index b5b737ec69..483cbb7295 100644 --- a/docs/report/introduction/methodology.rst +++ b/docs/report/introduction/methodology.rst @@ -190,7 +190,7 @@ List of vpp startup.conf settings applied to all tests: Typically needed for use faster vector PMDs (together with no-multi-seg). #. socket-mem <value>,<value> - memory per numa. (Not required anymore - due to VPP code changes, should be removed in CSIT rls1810.) + due to VPP code changes, should be removed in CSIT-18.10.) Per Test Settings ~~~~~~~~~~~~~~~~~ diff --git a/docs/report/introduction/overview.rst b/docs/report/introduction/overview.rst index e16a6e175f..eed5da44f8 100644 --- a/docs/report/introduction/overview.rst +++ b/docs/report/introduction/overview.rst @@ -84,4 +84,4 @@ the source data files are also provided. #. **CSIT Framework Documentation** - description of the overall CSIT framework design hierarchy, CSIT test naming convention, followed by description of Presentation and Analytics Layer (PAL) introduced in - |csit-release|.
\ No newline at end of file + CSIT-17.07.
\ No newline at end of file diff --git a/docs/report/test_configuration/cot_performance_configuration/index.rst b/docs/report/test_configuration/cot_performance_configuration/index.rst index 3387691067..eb651eea1a 100644 --- a/docs/report/test_configuration/cot_performance_configuration/index.rst +++ b/docs/report/test_configuration/cot_performance_configuration/index.rst @@ -3,8 +3,8 @@ VPP K8s Container Memif .. note:: - Data sources for reported test results: i) `FD.io test executor vpp - functional jobs`_, ii) archived FD.io jobs test result `output files + Data sources for reported test configurations: i) `FD.io test executor + ligato performance jobs`_, ii) archived FD.io jobs test result `output files <../../_static/archive/>`_. .. note:: diff --git a/docs/report/test_configuration/vpp_functional_configuration_centos/index.rst b/docs/report/test_configuration/vpp_functional_configuration_centos/index.rst index f16812a095..c749d4c61c 100644 --- a/docs/report/test_configuration/vpp_functional_configuration_centos/index.rst +++ b/docs/report/test_configuration/vpp_functional_configuration_centos/index.rst @@ -3,7 +3,7 @@ CentOS .. note:: - Data sources for reported test results: i) `FD.io test executor vpp + Data sources for reported test configurations: i) `FD.io test executor vpp functional jobs`_, ii) archived FD.io jobs test result `output files <../../_static/archive/>`_. diff --git a/docs/report/test_configuration/vpp_functional_configuration_ubuntu/index.rst b/docs/report/test_configuration/vpp_functional_configuration_ubuntu/index.rst index f2d403b7e9..7d82cec79c 100644 --- a/docs/report/test_configuration/vpp_functional_configuration_ubuntu/index.rst +++ b/docs/report/test_configuration/vpp_functional_configuration_ubuntu/index.rst @@ -3,9 +3,9 @@ Ubuntu .. note:: - Data sources for reported test results: i) `FD.io test executor vpp - functional jobs`_, ii) archived FD.io jobs test result `output files - <../../_static/archive/>`_. + Data sources for reported test configurations: i) `FD.io test executor vpp + functional jobs using Ubuntu`_, ii) archived FD.io jobs test result + `output files <../../_static/archive/>`_. .. toctree:: diff --git a/docs/report/test_configuration/vpp_mrr_configuration_2n_skx/index.rst b/docs/report/test_configuration/vpp_mrr_configuration_2n_skx/index.rst index 1df3bb374c..7a1d13e6b4 100644 --- a/docs/report/test_configuration/vpp_mrr_configuration_2n_skx/index.rst +++ b/docs/report/test_configuration/vpp_mrr_configuration_2n_skx/index.rst @@ -3,7 +3,7 @@ .. note:: - Data sources for reported test results: i) `FD.io test executor vpp + Data sources for reported test configurations: i) `FD.io test executor vpp performance job 2n-skx`_, ii) archived FD.io jobs test result `output files <../../_static/archive/>`_. diff --git a/docs/report/test_configuration/vpp_mrr_configuration_3n_hsw/index.rst b/docs/report/test_configuration/vpp_mrr_configuration_3n_hsw/index.rst index 2e0c701b82..f7c3a30ef6 100644 --- a/docs/report/test_configuration/vpp_mrr_configuration_3n_hsw/index.rst +++ b/docs/report/test_configuration/vpp_mrr_configuration_3n_hsw/index.rst @@ -3,7 +3,7 @@ .. note:: - Data sources for reported test results: i) `FD.io test executor vpp + Data sources for reported test configurations: i) `FD.io test executor vpp performance job 3n-hsw`_, ii) archived FD.io jobs test result `output files <../../_static/archive/>`_. diff --git a/docs/report/test_configuration/vpp_mrr_configuration_3n_skx/index.rst b/docs/report/test_configuration/vpp_mrr_configuration_3n_skx/index.rst index 38c77bc9c8..cdab8e6b3a 100644 --- a/docs/report/test_configuration/vpp_mrr_configuration_3n_skx/index.rst +++ b/docs/report/test_configuration/vpp_mrr_configuration_3n_skx/index.rst @@ -3,7 +3,7 @@ .. note:: - Data sources for reported test results: i) `FD.io test executor vpp + Data sources for reported test configurations: i) `FD.io test executor vpp performance job 3n-skx`_, ii) archived FD.io jobs test result `output files <../../_static/archive/>`_. diff --git a/docs/report/test_configuration/vpp_performance_configuration_2n_skx/index.rst b/docs/report/test_configuration/vpp_performance_configuration_2n_skx/index.rst index c468270e1a..dbfbc408e1 100644 --- a/docs/report/test_configuration/vpp_performance_configuration_2n_skx/index.rst +++ b/docs/report/test_configuration/vpp_performance_configuration_2n_skx/index.rst @@ -3,7 +3,7 @@ .. note:: - Data sources for reported test results: i) `FD.io test executor vpp + Data sources for reported test configurations: i) `FD.io test executor vpp performance job 2n-skx`_, ii) archived FD.io jobs test result `output files <../../_static/archive/>`_. diff --git a/docs/report/test_configuration/vpp_performance_configuration_3n_hsw/index.rst b/docs/report/test_configuration/vpp_performance_configuration_3n_hsw/index.rst index 3e5b09f641..28ba4fd1b7 100644 --- a/docs/report/test_configuration/vpp_performance_configuration_3n_hsw/index.rst +++ b/docs/report/test_configuration/vpp_performance_configuration_3n_hsw/index.rst @@ -3,7 +3,7 @@ .. note:: - Data sources for reported test results: i) `FD.io test executor vpp + Data sources for reported test configurations: i) `FD.io test executor vpp performance job 3n-hsw`_, ii) archived FD.io jobs test result `output files <../../_static/archive/>`_. diff --git a/docs/report/test_configuration/vpp_performance_configuration_3n_skx/index.rst b/docs/report/test_configuration/vpp_performance_configuration_3n_skx/index.rst index 76c2a9edb6..86fc9a9709 100644 --- a/docs/report/test_configuration/vpp_performance_configuration_3n_skx/index.rst +++ b/docs/report/test_configuration/vpp_performance_configuration_3n_skx/index.rst @@ -3,7 +3,7 @@ .. note:: - Data sources for reported test results: i) `FD.io test executor vpp + Data sources for reported test configurations: i) `FD.io test executor vpp performance job 3n-skx`_, ii) archived FD.io jobs test result `output files <../../_static/archive/>`_. diff --git a/docs/report/test_operational_data/vpp_performance_operational_data_2n_skx/index.rst b/docs/report/test_operational_data/vpp_performance_operational_data_2n_skx/index.rst index e363e0376c..7872b2a13d 100644 --- a/docs/report/test_operational_data/vpp_performance_operational_data_2n_skx/index.rst +++ b/docs/report/test_operational_data/vpp_performance_operational_data_2n_skx/index.rst @@ -3,7 +3,7 @@ .. note:: - Data sources for reported test results: i) `FD.io test executor vpp + Data sources for reported test operational data: i) `FD.io test executor vpp performance job 2n-skx`_, ii) archived FD.io jobs test result `output files <../../_static/archive/>`_. diff --git a/docs/report/test_operational_data/vpp_performance_operational_data_3n_hsw/index.rst b/docs/report/test_operational_data/vpp_performance_operational_data_3n_hsw/index.rst index 530cf853b4..cd941ab978 100644 --- a/docs/report/test_operational_data/vpp_performance_operational_data_3n_hsw/index.rst +++ b/docs/report/test_operational_data/vpp_performance_operational_data_3n_hsw/index.rst @@ -3,7 +3,7 @@ .. note:: - Data sources for reported test results: i) `FD.io test executor vpp + Data sources for reported test operational data: i) `FD.io test executor vpp performance job 3n-hsw`_, ii) archived FD.io jobs test result `output files <../../_static/archive/>`_. diff --git a/docs/report/test_operational_data/vpp_performance_operational_data_3n_skx/index.rst b/docs/report/test_operational_data/vpp_performance_operational_data_3n_skx/index.rst index c82e2c1906..c560eca82e 100644 --- a/docs/report/test_operational_data/vpp_performance_operational_data_3n_skx/index.rst +++ b/docs/report/test_operational_data/vpp_performance_operational_data_3n_skx/index.rst @@ -3,7 +3,7 @@ .. note:: - Data sources for reported test results: i) `FD.io test executor vpp + Data sources for reported test operational data: i) `FD.io test executor vpp performance job 3n-skx`_, ii) archived FD.io jobs test result `output files <../../_static/archive/>`_. diff --git a/docs/report/vpp_functional_tests/overview.rst b/docs/report/vpp_functional_tests/overview.rst index 90afd4ec83..fe58edd92d 100644 --- a/docs/report/vpp_functional_tests/overview.rst +++ b/docs/report/vpp_functional_tests/overview.rst @@ -128,7 +128,7 @@ Functional Tests Naming ----------------------- |csit-release| follows a common structured naming convention for all performance -and system functional tests, introduced in CSIT rls1701. +and system functional tests, introduced in CSIT-17.01. The naming should be intuitive for majority of the tests. Complete description of CSIT test naming convention is provided on :ref:`csit_test_naming`.. diff --git a/docs/report/vpp_performance_tests/csit_release_notes.rst b/docs/report/vpp_performance_tests/csit_release_notes.rst index 4fdee9738c..74e3082f8e 100644 --- a/docs/report/vpp_performance_tests/csit_release_notes.rst +++ b/docs/report/vpp_performance_tests/csit_release_notes.rst @@ -38,21 +38,21 @@ Changes in |csit-release| Performance Changes ------------------- -Relative performance changes in measured packet throughput in CSIT -|release| are calculated against the results from CSIT |release-1| +Relative performance changes in measured packet throughput in |csit-release| +are calculated against the results from |csit-release-1| report. Listed mean and standard deviation values are computed based on a series of the same tests executed against respective VPP releases to -verify test results repeatibility, with percentage change calculated for +verify test results repeatability, with percentage change calculated for mean values. Note that the standard deviation is quite high for a small number of packet throughput tests, what indicates poor test results repeatability and makes the relative change of mean throughput value not fully representative for these tests. The root causes behind poor -results repeatibility vary between the test cases. +results repeatability vary between the test cases. NDR Changes ~~~~~~~~~~~ -NDR small packet throughput changes between releases are available in a +NDR throughput changes between releases are available in a CSV and pretty ASCII formats: - `csv format for 1t1c <../_static/vpp/performance-changes-1t1c-ndr.csv>`_, @@ -63,13 +63,15 @@ CSV and pretty ASCII formats: .. note:: Test results have been generated by - `FD.io test executor vpp performance jobs`_ with Robot Framework result - files csit-vpp-perf-|srelease|-\*.zip `archived here <../_static/archive/>`_. + `FD.io test executor vpp performance job 3n-hsw`_, + with Robot Framework result + files csit-vpp-perf-|srelease|-\*.zip + `archived here <../_static/archive/>`_. PDR Changes ~~~~~~~~~~~ -NDR small packet throughput changes between releases are available in a +PDR throughput changes between releases are available in a CSV and pretty ASCII formats: - `csv format for 1t1c <../_static/vpp/performance-changes-1t1c-pdr.csv>`_, @@ -80,8 +82,10 @@ CSV and pretty ASCII formats: .. note:: Test results have been generated by - `FD.io test executor vpp performance jobs`_ with Robot Framework result - files csit-vpp-perf-|srelease|-\*.zip `archived here <../_static/archive/>`_. + `FD.io test executor vpp performance job 3n-hsw`_, + with Robot Framework result + files csit-vpp-perf-|srelease|-\*.zip + `archived here <../_static/archive/>`_. MRR Changes ~~~~~~~~~~~ @@ -106,15 +110,44 @@ CSV and pretty ASCII formats: Comparison Across Testbeds -------------------------- -.. warning:: +Relative performance changes in measured packet throughputon 3-Node Skx testbed +are calculated against the results measured on 3-Node Hsw testbed. - TODO: Add: +NDR Changes +~~~~~~~~~~~ - Table 1. - Test Case 3-Node Hsw 3-Node Skx Skx vs. Hsw Delta [%] +NDR changes between testbeds are available in a +CSV and pretty ASCII formats: - Table 2. - Test Case 3-Node Skx 2-Node Skx 2-Node vs. 3-Node Delta [%] + - `csv format for ndr <../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-ndr.csv>`_, + - `pretty ASCII format for ndr <../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-ndr.txt>`_. + +.. note:: + + Test results have been generated by + `FD.io test executor vpp performance job 3n-hsw`_ and + `FD.io test executor vpp performance job 3n-skx`_ + with Robot Framework result + files csit-vpp-perf-|srelease|-\*.zip + `archived here <../_static/archive/>`_. + +PDR Changes +~~~~~~~~~~~ + +PDR throughput changes between testbeds are available in a +CSV and pretty ASCII formats: + + - `csv format for pdr <../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-pdr.csv>`_, + - `pretty ASCII format for pdr <../_static/vpp/performance-compare-testbeds-3n-hsw-3n-skx-pdr.txt>`_. + +.. note:: + + Test results have been generated by + `FD.io test executor vpp performance job 3n-hsw`_ and + `FD.io test executor vpp performance job 3n-skx`_ + with Robot Framework result + files csit-vpp-perf-|srelease|-\*.zip + `archived here <../_static/archive/>`_. Throughput Trending ------------------- diff --git a/docs/report/vpp_performance_tests/http_server_performance/index.rst b/docs/report/vpp_performance_tests/http_server_performance/index.rst index 6d40f1ff7c..32f75949dd 100644 --- a/docs/report/vpp_performance_tests/http_server_performance/index.rst +++ b/docs/report/vpp_performance_tests/http_server_performance/index.rst @@ -20,7 +20,7 @@ have the same value, only a horizontal line is plotted. .. note:: Data sources for reported test results: i) `FD.io test executor vpp - performance jobs`_, ii) archived FD.io jobs test result `output files + performance job 3n-hsw`_, ii) archived FD.io jobs test result `output files <../../_static/archive/>`_. CSIT source code for the test cases used for plots can be found in diff --git a/docs/report/vpp_performance_tests/impact_meltdown/index.rst b/docs/report/vpp_performance_tests/impact_meltdown/index.rst deleted file mode 100644 index 35ecb7453c..0000000000 --- a/docs/report/vpp_performance_tests/impact_meltdown/index.rst +++ /dev/null @@ -1,164 +0,0 @@ -Impact of Meltdown Patches -========================== - -Following sections list changes to VPP throughput performance after applying -patches addressing security vulnerabilities referred to as Meltdown -(Variant3: Rogue Data Cache Load). Incremental kernel patches are -applied for Ubuntu 16.04LTS as documented on -`Ubuntu SpectreAndMeltdown page <https://wiki.ubuntu.com/SecurityTeam/KnowledgeBase/SpectreAndMeltdown>`_. -Detailed listing of used software versions and patches is documented in -:ref:`test_environment`. - -NDR and PDR packet throughput results are compared for 1-core/1-thread, -2-cores/2-threads and 4-cores/4-threads VPP configurations, with -reference performance numbers coming from tests without the Meltdown -patches. Tables show test results grouped into Best 20 changes (minimal -performance impact), followed by Worst 20 changes (maximal performance -impact). All results are also provided in downloadable CSV and pretty -ASCII formats. - -NDR Throughput: Best 20 Changes -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. only:: html - - .. csv-table:: - :align: center - :file: ../../../../_build/_static/vpp/meltdown-impact-ndr-1t1c-top.csv - -.. only:: latex - - .. raw:: latex - - \makeatletter - \csvset{ - perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead}, - } - \makeatother - - {\tiny - \csvautobooklongtable[separator=comma, - respect all, - no check column count, - perfimprovements column width=1cm, - late after line={\\\hline}, - late after last line={\end{longtable}} - ]{../_build/_static/vpp/meltdown-impact-ndr-1t1c-top.csv} - } - -NDR Throughput: Worst 20 Changes -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. only:: html - - .. csv-table:: - :align: center - :file: ../../../../_build/_static/vpp/meltdown-impact-ndr-1t1c-bottom.csv - -.. only:: latex - - .. raw:: latex - - \makeatletter - \csvset{ - perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead}, - } - \makeatother - - {\tiny - \csvautobooklongtable[separator=comma, - respect all, - no check column count, - perfimprovements column width=1cm, - late after line={\\\hline}, - late after last line={\end{longtable}} - ]{../_build/_static/vpp/meltdown-impact-ndr-1t1c-bottom.csv} - } - -.. only:: html - - NDR Throughput: All Changes - ~~~~~~~~~~~~~~~~~~~~~~~~~~~ - - Complete results for all NDR tests are available in a CSV and pretty - ASCII formats: - - - `csv format for 1t1c <../../_static/vpp/meltdown-impact-ndr-1t1c-full.csv>`_, - - `csv format for 2t2c <../../_static/vpp/meltdown-impact-ndr-2t2c-full.csv>`_, - - `csv format for 4t4c <../../_static/vpp/meltdown-impact-ndr-4t4c-full.csv>`_, - - `pretty ASCII format for 1t1c <../../_static/vpp/meltdown-impact-ndr-1t1c-full.txt>`_, - - `pretty ASCII format for 2t2c <../../_static/vpp/meltdown-impact-ndr-2t2c-full.txt>`_, - - `pretty ASCII format for 4t4c <../../_static/vpp/meltdown-impact-ndr-4t4c-full.txt>`_. - -PDR Throughput: Best 20 Changes -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. only:: html - - .. csv-table:: - :align: center - :file: ../../../../_build/_static/vpp/meltdown-impact-pdr-1t1c-top.csv - -.. only:: latex - - .. raw:: latex - - \makeatletter - \csvset{ - perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead}, - } - \makeatother - - {\tiny - \csvautobooklongtable[separator=comma, - respect all, - no check column count, - perfimprovements column width=1cm, - late after line={\\\hline}, - late after last line={\end{longtable}} - ]{../_build/_static/vpp/meltdown-impact-pdr-1t1c-top.csv} - } - -PDR Throughput: Worst 20 Changes -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. only:: html - - .. csv-table:: - :align: center - :file: ../../../../_build/_static/vpp/meltdown-impact-pdr-1t1c-bottom.csv - -.. only:: latex - - .. raw:: latex - - \makeatletter - \csvset{ - perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead}, - } - \makeatother - - {\tiny - \csvautobooklongtable[separator=comma, - respect all, - no check column count, - perfimprovements column width=1cm, - late after line={\\\hline}, - late after last line={\end{longtable}} - ]{../_build/_static/vpp/meltdown-impact-pdr-1t1c-bottom.csv} - } - -.. only:: html - -PDR Throughput: All Changes -~~~~~~~~~~~~~~~~~~~~~~~~~~~ - - Complete results for all PDR tests are available in a CSV and pretty - ASCII formats: - - - `csv format for 1t1c <../../_static/vpp/meltdown-impact-pdr-1t1c-full.csv>`_, - - `csv format for 2t2c <../../_static/vpp/meltdown-impact-pdr-2t2c-full.csv>`_, - - `csv format for 4t4c <../../_static/vpp/meltdown-impact-pdr-4t4c-full.csv>`_, - - `pretty ASCII format for 1t1c <../../_static/vpp/meltdown-impact-pdr-1t1c-full.txt>`_, - - `pretty ASCII format for 2t2c <../../_static/vpp/meltdown-impact-pdr-2t2c-full.txt>`_, - - `pretty ASCII format for 4t4c <../../_static/vpp/meltdown-impact-pdr-4t4c-full.txt>`_. diff --git a/docs/report/vpp_performance_tests/impact_spectreandmeltdown/index.rst b/docs/report/vpp_performance_tests/impact_spectreandmeltdown/index.rst deleted file mode 100644 index eec38e8698..0000000000 --- a/docs/report/vpp_performance_tests/impact_spectreandmeltdown/index.rst +++ /dev/null @@ -1,167 +0,0 @@ -Impact of SpectreAndMeltdown Patches -==================================== - -Following sections list changes to VPP throughput performance after -applying patches addressing security vulnerabilities referred to as: -Meltdown (Variant3: Rogue Data Cache Load) and Spectre (Variant1: Bounds -Check Bypass; Variant2: Branch Target Injection) security -vulnerabilities. Incremental kernel patches for Ubuntu 16.04 LTS as -documented on -`Ubuntu SpectreAndMeltdown page <https://wiki.ubuntu.com/SecurityTeam/KnowledgeBase/SpectreAndMeltdown>`_. -For Spectre additional Processor microcode and BIOS firmware changes are -applied. Detailed listing of used software versions and patches is -documented in :ref:`test_environment`. - -NDR and PDR packet throughput results are compared for 1-core/1-thread, -2-cores/2-threads and 4-cores/4-threads VPP configurations, with -reference performance numbers coming from tests without the Meltdown -patches. Tables show test results grouped into Best 20 changes (minimal -performance impact), followed by Worst 20 changes (maximal performance -impact). All results are also provided in downloadable CSV and pretty -ASCII formats. - -NDR Throughput: Best 20 Changes -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. only:: html - - .. csv-table:: - :align: center - :file: ../../../../_build/_static/vpp/meltdown-spectre-impact-ndr-1t1c-top.csv - -.. only:: latex - - .. raw:: latex - - \makeatletter - \csvset{ - perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead}, - } - \makeatother - - {\tiny - \csvautobooklongtable[separator=comma, - respect all, - no check column count, - perfimprovements column width=1cm, - late after line={\\\hline}, - late after last line={\end{longtable}} - ]{../_build/_static/vpp/meltdown-spectre-impact-ndr-1t1c-top.csv} - } - -NDR Throughput: Worst 20 Changes -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. only:: html - - .. csv-table:: - :align: center - :file: ../../../../_build/_static/vpp/meltdown-spectre-impact-ndr-1t1c-bottom.csv - -.. only:: latex - - .. raw:: latex - - \makeatletter - \csvset{ - perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead}, - } - \makeatother - - {\tiny - \csvautobooklongtable[separator=comma, - respect all, - no check column count, - perfimprovements column width=1cm, - late after line={\\\hline}, - late after last line={\end{longtable}} - ]{../_build/_static/vpp/meltdown-spectre-impact-ndr-1t1c-bottom.csv} - } - -.. only:: html - - NDR Throughput: All Changes - ~~~~~~~~~~~~~~~~~~~~~~~~~~~ - - Complete results for all NDR tests are available in a CSV and pretty - ASCII formats: - - - `csv format for 1t1c <../../_static/vpp/meltdown-spectre-impact-ndr-1t1c-full.csv>`_, - - `csv format for 2t2c <../../_static/vpp/meltdown-spectre-impact-ndr-2t2c-full.csv>`_, - - `csv format for 4t4c <../../_static/vpp/meltdown-spectre-impact-ndr-4t4c-full.csv>`_, - - `pretty ASCII format for 1t1c <../../_static/vpp/meltdown-spectre-impact-ndr-1t1c-full.txt>`_, - - `pretty ASCII format for 2t2c <../../_static/vpp/meltdown-spectre-impact-ndr-2t2c-full.txt>`_, - - `pretty ASCII format for 4t4c <../../_static/vpp/meltdown-spectre-impact-ndr-4t4c-full.txt>`_. - -PDR Throughput: Best 20 Changes -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. only:: html - - .. csv-table:: - :align: center - :file: ../../../../_build/_static/vpp/meltdown-spectre-impact-pdr-1t1c-top.csv - -.. only:: latex - - .. raw:: latex - - \makeatletter - \csvset{ - perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead}, - } - \makeatother - - {\tiny - \csvautobooklongtable[separator=comma, - respect all, - no check column count, - perfimprovements column width=1cm, - late after line={\\\hline}, - late after last line={\end{longtable}} - ]{../_build/_static/vpp/meltdown-spectre-impact-pdr-1t1c-top.csv} - } - -PDR Throughput: Worst 20 Changes -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. only:: html - - .. csv-table:: - :align: center - :file: ../../../../_build/_static/vpp/meltdown-spectre-impact-pdr-1t1c-bottom.csv - -.. only:: latex - - .. raw:: latex - - \makeatletter - \csvset{ - perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead}, - } - \makeatother - - {\tiny - \csvautobooklongtable[separator=comma, - respect all, - no check column count, - perfimprovements column width=1cm, - late after line={\\\hline}, - late after last line={\end{longtable}} - ]{../_build/_static/vpp/meltdown-spectre-impact-pdr-1t1c-bottom.csv} - } - -.. only:: html - - PDR Throughput: All Changes - ~~~~~~~~~~~~~~~~~~~~~~~~~~~ - - Complete results for all PDR tests are available in a CSV and pretty - ASCII formats: - - - `csv format for 1t1c <../../_static/vpp/meltdown-spectre-impact-pdr-1t1c-full.csv>`_, - - `csv format for 2t2c <../../_static/vpp/meltdown-spectre-impact-pdr-2t2c-full.csv>`_, - - `csv format for 4t4c <../../_static/vpp/meltdown-spectre-impact-pdr-4t4c-full.csv>`_, - - `pretty ASCII format for 1t1c <../../_static/vpp/meltdown-spectre-impact-pdr-1t1c-full.txt>`_, - - `pretty ASCII format for 2t2c <../../_static/vpp/meltdown-spectre-impact-pdr-2t2c-full.txt>`_, - - `pretty ASCII format for 4t4c <../../_static/vpp/meltdown-spectre-impact-pdr-4t4c-full.txt>`_. diff --git a/docs/report/vpp_performance_tests/overview.rst b/docs/report/vpp_performance_tests/overview.rst index 86116bf7c5..bd0681dc40 100644 --- a/docs/report/vpp_performance_tests/overview.rst +++ b/docs/report/vpp_performance_tests/overview.rst @@ -350,7 +350,7 @@ Performance Tests Naming ------------------------ FD.io |csit-release| follows a common structured naming convention for -all performance and system functional tests, introduced in CSIT rls1701. +all performance and system functional tests, introduced in CSIT-17.01. The naming should be intuitive for majority of the tests. Complete description of FD.io CSIT test naming convention is provided on diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/container_orchestrated.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/container_orchestrated.rst index 222a7d8301..cda1d3698a 100644 --- a/docs/report/vpp_performance_tests/packet_latency_graphs/container_orchestrated.rst +++ b/docs/report/vpp_performance_tests/packet_latency_graphs/container_orchestrated.rst @@ -115,215 +115,3 @@ CSIT source code for the test cases used for plots can be found in \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{k8s-memif-3n-hsw-x710-64b-2t2c-base_and_scale-ndr-lat} \label{fig:k8s-memif-3n-hsw-x710-64b-2t2c-base_and_scale-ndr-lat} \end{figure} - -3n-skx-x710 -~~~~~~~~~~~ - -64b-2t1c-base_and_scale ------------------------ - -.. raw:: html - - <center><b> - -:index:`Latency: k8s-memif-3n-skx-x710-64b-2t1c-base_and_scale-ndr` - -.. raw:: html - - </b> - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/k8s-memif-3n-skx-x710-64b-2t1c-base_and_scale-ndr-lat.html"></iframe> - <p><br><br></p> - </center> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/vpp/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{k8s-memif-3n-skx-x710-64b-2t1c-base_and_scale-ndr-lat} - \label{fig:k8s-memif-3n-skx-x710-64b-2t1c-base_and_scale-ndr-lat} - \end{figure} - -64b-4t2c-base_and_scale ------------------------ - -.. raw:: html - - <center><b> - -:index:`Latency: k8s-memif-3n-skx-x710-64b-4t2c-base_and_scale-ndr` - -.. raw:: html - - </b> - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/k8s-memif-3n-skx-x710-64b-4t2c-base_and_scale-ndr-lat.html"></iframe> - <p><br><br></p> - </center> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/vpp/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{k8s-memif-3n-skx-x710-64b-4t2c-base_and_scale-ndr-lat} - \label{fig:k8s-memif-3n-skx-x710-64b-4t2c-base_and_scale-ndr-lat} - \end{figure} - -3n-skx-xxv710 -~~~~~~~~~~~~~ - -64b-2t1c-base_and_scale ------------------------ - -.. raw:: html - - <center><b> - -:index:`Latency: k8s-memif-3n-skx-xxv710-64b-2t1c-base_and_scale-ndr` - -.. raw:: html - - </b> - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/k8s-memif-3n-skx-xxv710-64b-2t1c-base_and_scale-ndr-lat.html"></iframe> - <p><br><br></p> - </center> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/vpp/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{k8s-memif-3n-skx-xxv710-64b-2t1c-base_and_scale-ndr-lat} - \label{fig:k8s-memif-3n-skx-xxv710-64b-2t1c-base_and_scale-ndr-lat} - \end{figure} - -64b-4t2c-base_and_scale ------------------------ - -.. raw:: html - - <center><b> - -:index:`Latency: k8s-memif-3n-skx-xxv710-64b-4t2c-base_and_scale-ndr` - -.. raw:: html - - </b> - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/k8s-memif-3n-skx-xxv710-64b-4t2c-base_and_scale-ndr-lat.html"></iframe> - <p><br><br></p> - </center> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/vpp/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{k8s-memif-3n-skx-xxv710-64b-4t2c-base_and_scale-ndr-lat} - \label{fig:k8s-memif-3n-skx-xxv710-64b-4t2c-base_and_scale-ndr-lat} - \end{figure} - -2n-skx-x710 -~~~~~~~~~~~ - -64b-2t1c-base_and_scale ------------------------ - -.. raw:: html - - <center><b> - -:index:`Latency: k8s-memif-2n-skx-x710-64b-2t1c-base_and_scale-ndr` - -.. raw:: html - - </b> - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/k8s-memif-2n-skx-x710-64b-2t1c-base_and_scale-ndr-lat.html"></iframe> - <p><br><br></p> - </center> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/vpp/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{k8s-memif-2n-skx-x710-64b-2t1c-base_and_scale-ndr-lat} - \label{fig:k8s-memif-2n-skx-x710-64b-2t1c-base_and_scale-ndr-lat} - \end{figure} - -64b-4t2c-base_and_scale ------------------------ - -.. raw:: html - - <center><b> - -:index:`Latency: k8s-memif-2n-skx-x710-64b-4t2c-base_and_scale-ndr` - -.. raw:: html - - </b> - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/k8s-memif-2n-skx-x710-64b-4t2c-base_and_scale-ndr-lat.html"></iframe> - <p><br><br></p> - </center> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/vpp/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{k8s-memif-2n-skx-x710-64b-4t2c-base_and_scale-ndr-lat} - \label{fig:k8s-memif-2n-skx-x710-64b-4t2c-base_and_scale-ndr-lat} - \end{figure} - -2n-skx-xxv710 -~~~~~~~~~~~~~ - -64b-2t1c-base_and_scale ------------------------ - -.. raw:: html - - <center><b> - -:index:`Latency: k8s-memif-2n-skx-xxv710-64b-2t1c-base_and_scale-ndr` - -.. raw:: html - - </b> - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/k8s-memif-2n-skx-xxv710-64b-2t1c-base_and_scale-ndr-lat.html"></iframe> - <p><br><br></p> - </center> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/vpp/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{k8s-memif-2n-skx-xxv710-64b-2t1c-base_and_scale-ndr-lat} - \label{fig:k8s-memif-2n-skx-xxv710-64b-2t1c-base_and_scale-ndr-lat} - \end{figure} - -64b-4t2c-base_and_scale ------------------------ - -.. raw:: html - - <center><b> - -:index:`Latency: k8s-memif-2n-skx-xxv710-64b-4t2c-base_and_scale-ndr` - -.. raw:: html - - </b> - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/k8s-memif-2n-skx-xxv710-64b-4t2c-base_and_scale-ndr-lat.html"></iframe> - <p><br><br></p> - </center> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/vpp/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{k8s-memif-2n-skx-xxv710-64b-4t2c-base_and_scale-ndr-lat} - \label{fig:k8s-memif-2n-skx-xxv710-64b-4t2c-base_and_scale-ndr-lat} - \end{figure} diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/index.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/index.rst index dd8d320878..ab44222f63 100644 --- a/docs/report/vpp_performance_tests/packet_latency_graphs/index.rst +++ b/docs/report/vpp_performance_tests/packet_latency_graphs/index.rst @@ -8,7 +8,7 @@ latency per test. *Title of each graph* is a regex (regular expression) matching all throughput test cases plotted on this graph, *X-axis labels* are indices of individual test suites executed by -`FD.io test executor vpp performance jobs`_ that created result output file +FD.io test executor vpp performance jobs that created result output file used as data source for the graph, *Y-axis labels* are measured packet Latency [uSec] values, and the *Graph legend* lists the plotted test suites and their indices. Latency is reported for concurrent symmetric bi-directional flows, @@ -19,8 +19,13 @@ TGint2-to-SUT2-to-SUT1-to-TGint1. .. note:: Test results have been generated by - `FD.io test executor vpp performance jobs`_ with Robot Framework result - files csit-vpp-perf-|srelease|-\*.zip `archived here <../../_static/archive/>`_. + `FD.io test executor vpp performance job 3n-hsw`_, + `FD.io test executor vpp performance job 3n-skx`_ and + `FD.io test executor vpp performance job 2n-skx`_ with Robot Framework + result files csit-vpp-perf-|srelease|-\*.zip + `archived here <../../_static/archive/>`_. + Plotted data set size per test case is equal to the number of job executions + presented in this report version: **10**. .. toctree:: :maxdepth: 1 diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/container_orchestrated.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/container_orchestrated.rst index 37397f82f4..e859fb3706 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/container_orchestrated.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/container_orchestrated.rst @@ -203,391 +203,3 @@ CSIT source code for the test cases used for plots can be found in \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{k8s-memif-3n-hsw-x710-64b-2t2c-base_and_scale-pdr} \label{fig:k8s-memif-3n-hsw-x710-64b-2t2c-base_and_scale-pdr} \end{figure} - -3n-skx-x710 -~~~~~~~~~~~ - -64b-2t1c-base_and_scale ------------------------ - -.. raw:: html - - <center><b> - -:index:`Throughput: k8s-memif-3n-skx-x710-64b-2t1c-base_and_scale-ndr` - -.. raw:: html - - </b> - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/k8s-memif-3n-skx-x710-64b-2t1c-base_and_scale-ndr.html"></iframe> - <p><br><br></p> - </center> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/vpp/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{k8s-memif-3n-skx-x710-64b-2t1c-base_and_scale-ndr} - \label{fig:k8s-memif-3n-skx-x710-64b-2t1c-base_and_scale-ndr} - \end{figure} - -.. raw:: html - - <center><b> - -:index:`Throughput: k8s-memif-3n-skx-x710-64b-2t1c-base_and_scale-pdr` - -.. raw:: html - - </b> - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/k8s-memif-3n-skx-x710-64b-2t1c-base_and_scale-pdr.html"></iframe> - <p><br><br></p> - </center> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/vpp/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{k8s-memif-3n-skx-x710-64b-2t1c-base_and_scale-pdr} - \label{fig:k8s-memif-3n-skx-x710-64b-2t1c-base_and_scale-pdr} - \end{figure} - -64b-4t2c-base_and_scale ------------------------ - -.. raw:: html - - <center><b> - -:index:`Throughput: k8s-memif-3n-skx-x710-64b-4t2c-base_and_scale-ndr` - -.. raw:: html - - </b> - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/k8s-memif-3n-skx-x710-64b-4t2c-base_and_scale-ndr.html"></iframe> - <p><br><br></p> - </center> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/vpp/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{k8s-memif-3n-skx-x710-64b-4t2c-base_and_scale-ndr} - \label{fig:k8s-memif-3n-skx-x710-64b-4t2c-base_and_scale-ndr} - \end{figure} - -.. raw:: html - - <center><b> - -:index:`Throughput: k8s-memif-3n-skx-x710-64b-4t2c-base_and_scale-pdr` - -.. raw:: html - - </b> - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/k8s-memif-3n-skx-x710-64b-4t2c-base_and_scale-pdr.html"></iframe> - <p><br><br></p> - </center> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/vpp/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{k8s-memif-3n-skx-x710-64b-4t2c-base_and_scale-pdr} - \label{fig:k8s-memif-3n-skx-x710-64b-4t2c-base_and_scale-pdr} - \end{figure} - -3n-skx-xxv710 -~~~~~~~~~~~~~ - -64b-2t1c-base_and_scale ------------------------ - -.. raw:: html - - <center><b> - -:index:`Throughput: k8s-memif-3n-skx-xxv710-64b-2t1c-base_and_scale-ndr` - -.. raw:: html - - </b> - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/k8s-memif-3n-skx-xxv710-64b-2t1c-base_and_scale-ndr.html"></iframe> - <p><br><br></p> - </center> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/vpp/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{k8s-memif-3n-skx-xxv710-64b-2t1c-base_and_scale-ndr} - \label{fig:k8s-memif-3n-skx-xxv710-64b-2t1c-base_and_scale-ndr} - \end{figure} - -.. raw:: html - - <center><b> - -:index:`Throughput: k8s-memif-3n-skx-xxv710-64b-2t1c-base_and_scale-pdr` - -.. raw:: html - - </b> - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/k8s-memif-3n-skx-xxv710-64b-2t1c-base_and_scale-pdr.html"></iframe> - <p><br><br></p> - </center> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/vpp/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{k8s-memif-3n-skx-xxv710-64b-2t1c-base_and_scale-pdr} - \label{fig:k8s-memif-3n-skx-xxv710-64b-2t1c-base_and_scale-pdr} - \end{figure} - -64b-4t2c-base_and_scale ------------------------ - -.. raw:: html - - <center><b> - -:index:`Throughput: k8s-memif-3n-skx-xxv710-64b-4t2c-base_and_scale-ndr` - -.. raw:: html - - </b> - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/k8s-memif-3n-skx-xxv710-64b-4t2c-base_and_scale-ndr.html"></iframe> - <p><br><br></p> - </center> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/vpp/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{k8s-memif-3n-skx-xxv710-64b-4t2c-base_and_scale-ndr} - \label{fig:k8s-memif-3n-skx-xxv710-64b-4t2c-base_and_scale-ndr} - \end{figure} - -.. raw:: html - - <center><b> - -:index:`Throughput: k8s-memif-3n-skx-xxv710-64b-4t2c-base_and_scale-pdr` - -.. raw:: html - - </b> - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/k8s-memif-3n-skx-xxv710-64b-4t2c-base_and_scale-pdr.html"></iframe> - <p><br><br></p> - </center> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/vpp/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{k8s-memif-3n-skx-xxv710-64b-4t2c-base_and_scale-pdr} - \label{fig:k8s-memif-3n-skx-xxv710-64b-4t2c-base_and_scale-pdr} - \end{figure} - -2n-skx-x710 -~~~~~~~~~~~ - -64b-2t1c-base_and_scale ------------------------ - -.. raw:: html - - <center><b> - -:index:`Throughput: k8s-memif-2n-skx-x710-64b-2t1c-base_and_scale-ndr` - -.. raw:: html - - </b> - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/k8s-memif-2n-skx-x710-64b-2t1c-base_and_scale-ndr.html"></iframe> - <p><br><br></p> - </center> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/vpp/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{k8s-memif-2n-skx-x710-64b-2t1c-base_and_scale-ndr} - \label{fig:k8s-memif-2n-skx-x710-64b-2t1c-base_and_scale-ndr} - \end{figure} - -.. raw:: html - - <center><b> - -:index:`Throughput: k8s-memif-2n-skx-x710-64b-2t1c-base_and_scale-pdr` - -.. raw:: html - - </b> - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/k8s-memif-2n-skx-x710-64b-2t1c-base_and_scale-pdr.html"></iframe> - <p><br><br></p> - </center> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/vpp/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{k8s-memif-2n-skx-x710-64b-2t1c-base_and_scale-pdr} - \label{fig:k8s-memif-2n-skx-x710-64b-2t1c-base_and_scale-pdr} - \end{figure} - -64b-4t2c-base_and_scale ------------------------ - -.. raw:: html - - <center><b> - -:index:`Throughput: k8s-memif-2n-skx-x710-64b-4t2c-base_and_scale-ndr` - -.. raw:: html - - </b> - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/k8s-memif-2n-skx-x710-64b-4t2c-base_and_scale-ndr.html"></iframe> - <p><br><br></p> - </center> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/vpp/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{k8s-memif-2n-skx-x710-64b-4t2c-base_and_scale-ndr} - \label{fig:k8s-memif-2n-skx-x710-64b-4t2c-base_and_scale-ndr} - \end{figure} - -.. raw:: html - - <center><b> - -:index:`Throughput: k8s-memif-2n-skx-x710-64b-4t2c-base_and_scale-pdr` - -.. raw:: html - - </b> - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/k8s-memif-2n-skx-x710-64b-4t2c-base_and_scale-pdr.html"></iframe> - <p><br><br></p> - </center> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/vpp/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{k8s-memif-2n-skx-x710-64b-4t2c-base_and_scale-pdr} - \label{fig:k8s-memif-2n-skx-x710-64b-4t2c-base_and_scale-pdr} - \end{figure} - -2n-skx-xxv710 -~~~~~~~~~~~~~ - -64b-2t1c-base_and_scale ------------------------ - -.. raw:: html - - <center><b> - -:index:`Throughput: k8s-memif-2n-skx-xxv710-64b-2t1c-base_and_scale-ndr` - -.. raw:: html - - </b> - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/k8s-memif-2n-skx-xxv710-64b-2t1c-base_and_scale-ndr.html"></iframe> - <p><br><br></p> - </center> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/vpp/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{k8s-memif-2n-skx-xxv710-64b-2t1c-base_and_scale-ndr} - \label{fig:k8s-memif-2n-skx-xxv710-64b-2t1c-base_and_scale-ndr} - \end{figure} - -.. raw:: html - - <center><b> - -:index:`Throughput: k8s-memif-2n-skx-xxv710-64b-2t1c-base_and_scale-pdr` - -.. raw:: html - - </b> - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/k8s-memif-2n-skx-xxv710-64b-2t1c-base_and_scale-pdr.html"></iframe> - <p><br><br></p> - </center> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/vpp/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{k8s-memif-2n-skx-xxv710-64b-2t1c-base_and_scale-pdr} - \label{fig:k8s-memif-2n-skx-xxv710-64b-2t1c-base_and_scale-pdr} - \end{figure} - -64b-4t2c-base_and_scale ------------------------ - -.. raw:: html - - <center><b> - -:index:`Throughput: k8s-memif-2n-skx-xxv710-64b-4t2c-base_and_scale-ndr` - -.. raw:: html - - </b> - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/k8s-memif-2n-skx-xxv710-64b-4t2c-base_and_scale-ndr.html"></iframe> - <p><br><br></p> - </center> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/vpp/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{k8s-memif-2n-skx-xxv710-64b-4t2c-base_and_scale-ndr} - \label{fig:k8s-memif-2n-skx-xxv710-64b-4t2c-base_and_scale-ndr} - \end{figure} - -.. raw:: html - - <center><b> - -:index:`Throughput: k8s-memif-2n-skx-xxv710-64b-4t2c-base_and_scale-pdr` - -.. raw:: html - - </b> - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/k8s-memif-2n-skx-xxv710-64b-4t2c-base_and_scale-pdr.html"></iframe> - <p><br><br></p> - </center> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/vpp/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{k8s-memif-2n-skx-xxv710-64b-4t2c-base_and_scale-pdr} - \label{fig:k8s-memif-2n-skx-xxv710-64b-4t2c-base_and_scale-pdr} - \end{figure} diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/index.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/index.rst index 78dd34db04..aa03a669f1 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/index.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/index.rst @@ -20,7 +20,7 @@ have the same value, only a horizontal line is plotted. *Title of each graph* is a regex (regular expression) matching all throughput test cases plotted on this graph, *X-axis labels* are indices of individual test suites executed by -`FD.io test executor vpp performance jobs`_ jobs that created result output +FD.io test executor vpp performance jobs that created result output files used as data sources for the graph, *Y-axis labels* are measured Packets Per Second [pps] values, and the *Graph legend* lists the plotted test suites and their indices. @@ -29,8 +29,11 @@ and their indices. .. note:: Test results have been generated by - `FD.io test executor vpp performance jobs`_ with Robot Framework result - files csit-vpp-perf-|srelease|-\*.zip `archived here <../../_static/archive/>`_. + `FD.io test executor vpp performance job 3n-hsw`_, + `FD.io test executor vpp performance job 3n-skx`_ and + `FD.io test executor vpp performance job 2n-skx`_ with Robot Framework + result files csit-vpp-perf-|srelease|-\*.zip + `archived here <../../_static/archive/>`_. Plotted data set size per test case is equal to the number of job executions presented in this report version: **10**. diff --git a/docs/report/vpp_performance_tests/performance_improvements/tmpl_performance_improvements.csv b/docs/report/vpp_performance_tests/performance_improvements/tmpl_performance_improvements.csv deleted file mode 100644 index 691a0c7801..0000000000 --- a/docs/report/vpp_performance_tests/performance_improvements/tmpl_performance_improvements.csv +++ /dev/null @@ -1,59 +0,0 @@ -tests.vpp.perf.l2.10ge2p1x520-eth-l2xcbase-ndrpdrdisc.tc01-64b-1t1c-eth-l2xcbase-ndrdisc,10ge2p1x520: 64B-1t1c-eth-l2xcbase-ndrdisc,9.4,12.7,13.3 -tests.vpp.perf.l2.10ge2p1x520-dot1ad-l2xcbase-ndrpdrdisc.tc01-64b-1t1c-dot1ad-l2xcbase-ndrdisc,10ge2p1x520: 64B-1t1c-dot1ad-l2xcbase-ndrdisc,7.4,8.8,9.3 -tests.vpp.perf.l2.10ge2p1x520-dot1q-l2xcbase-ndrpdrdisc.tc01-64b-1t1c-dot1q-l2xcbase-ndrdisc,10ge2p1x520: 64B-1t1c-dot1q-l2xcbase-ndrdisc,7.5,8.8,9.2 -tests.vpp.perf.ip4 tunnels.10ge2p1x520-ethip4vxlan-l2xcbase-ndrpdrdisc.tc01-64b-1t1c-ethip4vxlan-l2xcbase-ndrdisc,10ge2p1x520: 64B-1t1c-ethip4vxlan-l2xcbase-ndrdisc,5.4,6.5,6.8 -tests.vpp.perf.l2.10ge2p1x520-eth-l2bdbasemaclrn-ndrpdrdisc.tc01-64b-1t1c-eth-l2bdbasemaclrn-ndrdisc,10ge2p1x520: 64B-1t1c-eth-l2bdbasemaclrn-ndrdisc,7.8,10.4,10.8 -tests.vpp.perf.ip4.10ge2p1x520-ethip4-ip4base-ndrpdrdisc.tc01-64b-1t1c-ethip4-ip4base-ndrdisc,10ge2p1x520: 64B-1t1c-ethip4-ip4base-ndrdisc,8.7,9.7,10.6 -tests.vpp.perf.ip4.10ge2p1x520-ethip4-ip4base-iacldstbase-ndrpdrdisc.tc01-64b-1t1c-ethip4-ip4base-iacldstbase-ndrdisc,10ge2p1x520: 64B-1t1c-ethip4-ip4base-iacldstbase-ndrdisc,6.9,7.6,8.3 -tests.vpp.perf.ip4.10ge2p1x520-ethip4-ip4scale200k-ndrpdrdisc.tc01-64b-1t1c-ethip4-ip4scale200k-ndrdisc,10ge2p1x520: 64B-1t1c-ethip4-ip4scale200k-ndrdisc,8.5,9,9.7 -tests.vpp.perf.ip4.10ge2p1x520-ethip4-ip4scale20k-ndrpdrdisc.tc01-64b-1t1c-ethip4-ip4scale20k-ndrdisc,10ge2p1x520: 64B-1t1c-ethip4-ip4scale20k-ndrdisc,8.5,9,9.7 -tests.vpp.perf.ip4.10ge2p1x520-ethip4-ip4base-ipolicemarkbase-ndrpdrdisc.tc01-64b-1t1c-ethip4-ip4base-ipolicemarkbase-ndrdisc,10ge2p1x520: 64B-1t1c-ethip4-ip4base-ipolicemarkbase-ndrdisc,6.9,7.4,8.1 -tests.vpp.perf.ip6.10ge2p1x520-ethip6-ip6scale200k-ndrpdrdisc.tc01-78b-1t1c-ethip6-ip6scale200k-ndrdisc,10ge2p1x520: 78B-1t1c-ethip6-ip6scale200k-ndrdisc,6.5,5.3,5.3 -tests.vpp.perf.ip4 tunnels.10ge2p1x520-ethip4lispip4-ip4base-ndrpdrdisc.tc01-64b-1t1c-ethip4lispip4-ip4base-ndrdisc,10ge2p1x520: 64B-1t1c-ethip4lispip4-ip4base-ndrdisc,4.4,4.8,5.5 -tests.vpp.perf.vm vhost.10ge2p1x520-dot1q-l2xcbase-eth-2vhostvr1024-1vm-ndrpdrdisc.tc01-64b-1t1c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc,10ge2p1x520: 64B-1t1c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc,0.5,2.8,3.2 -tests.vpp.perf.vm vhost.10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhostvr1024-1vm-ndrpdrdisc.tc01-64b-1t1c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc,10ge2p1x710: 64B-1t1c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc,0.4,2.7,3.2 -tests.vpp.perf.vm vhost.10ge2p1x520-ethip4-ip4base-eth-2vhostvr1024-1vm-ndrpdrdisc.tc01-64b-1t1c-ethip4-ip4base-eth-2vhostvr1024-1vm-ndrdisc,10ge2p1x520: 64B-1t1c-ethip4-ip4base-eth-2vhostvr1024-1vm-ndrdisc,0.3,2.6,3.1 -tests.vpp.perf.ip6.10ge2p1x520-ethip6-ip6base-ndrpdrdisc.tc01-78b-1t1c-ethip6-ip6base-ndrdisc,10ge2p1x520: 78B-1t1c-ethip6-ip6base-ndrdisc,3,7.3,8.1 -tests.vpp.perf.ip6.10ge2p1x520-ethip6-ip6base-copwhtlistbase-ndrpdrdisc.tc01-78b-1t1c-ethip6-ip6base-copwhtlistbase-ndrdisc,10ge2p1x520: 78B-1t1c-ethip6-ip6base-copwhtlistbase-ndrdisc,6.1,6.1,6.9 -tests.vpp.perf.ip6.10ge2p1x520-ethip6-ip6base-iacldstbase-ndrpdrdisc.tc01-78b-1t1c-ethip6-ip6base-iacldstbase-ndrdisc,10ge2p1x520: 78B-1t1c-ethip6-ip6base-iacldstbase-ndrdisc,6.5,6.1,6.9 -tests.vpp.perf.ip6.10ge2p1x520-ethip6-ip6scale2m-ndrpdrdisc.tc01-78b-1t1c-ethip6-ip6scale2m-ndrdisc,10ge2p1x520: 78B-1t1c-ethip6-ip6scale2m-ndrdisc,5.3,4.2,4.6 -tests.vpp.perf.l2.40ge2p1xl710-eth-l2xcbase-ndrpdrdisc.tc01-64b-1t1c-eth-l2xcbase-ndrdisc,10ge2p1xl710: 64B-1t1c-eth-l2xcbase-ndrdisc,9.5,12.2,12.4 -tests.vpp.perf.ip4.10ge2p1x520-ethip4-ip4base-copwhtlistbase-ndrpdrdisc.tc01-64b-1t1c-ethip4-ip4base-copwhtlistbase-ndrdisc,10ge2p1x520: 64B-1t1c-ethip4-ip4base-copwhtlistbase-ndrdisc,7.1,8.3,9 -tests.vpp.perf.ip4.10ge2p1x520-ethip4-ip4scale2m-ndrpdrdisc.tc01-64b-1t1c-ethip4-ip4scale2m-ndrdisc,10ge2p1x520: 64B-1t1c-ethip4-ip4scale2m-ndrdisc,8.5,7.8,8.1 -tests.vpp.perf.ip6.10ge2p1x520-ethip6-ip6scale20k-ndrpdrdisc.tc01-78b-1t1c-ethip6-ip6scale20k-ndrdisc,10ge2p1x520: 78B-1t1c-ethip6-ip6scale20k-ndrdisc,6.9,6.5,6.9 -tests.vpp.perf.l2.10ge2p1x520-eth-l2xcbase-ndrpdrdisc.tc02-64b-1t1c-eth-l2xcbase-pdrdisc,10ge2p1x520: 64B-1t1c-eth-l2xcbase-pdrdisc,9.4,12.7,13.4 -tests.vpp.perf.l2.10ge2p1x520-dot1ad-l2xcbase-ndrpdrdisc.tc02-64b-1t1c-dot1ad-l2xcbase-pdrdisc,10ge2p1x520: 64B-1t1c-dot1ad-l2xcbase-pdrdisc,7.4,8.8,9.3 -tests.vpp.perf.l2.10ge2p1x520-dot1q-l2xcbase-ndrpdrdisc.tc02-64b-1t1c-dot1q-l2xcbase-pdrdisc,10ge2p1x520: 64B-1t1c-dot1q-l2xcbase-pdrdisc,7.5,8.8,9.2 -tests.vpp.perf.ip4 tunnels.10ge2p1x520-ethip4vxlan-l2xcbase-ndrpdrdisc.tc02-64b-1t1c-ethip4vxlan-l2xcbase-pdrdisc,10ge2p1x520: 64B-1t1c-ethip4vxlan-l2xcbase-pdrdisc,5.4,6.5,6.8 -tests.vpp.perf.l2.10ge2p1x520-eth-l2bdbasemaclrn-ndrpdrdisc.tc02-64b-1t1c-eth-l2bdbasemaclrn-pdrdisc,10ge2p1x520: 64B-1t1c-eth-l2bdbasemaclrn-pdrdisc,7.8,10.6,10.9 -tests.vpp.perf.ip4.10ge2p1x520-ethip4-ip4base-ndrpdrdisc.tc02-64b-1t1c-ethip4-ip4base-pdrdisc,10ge2p1x520: 64B-1t1c-ethip4-ip4base-pdrdisc,8.7,9.7,10.6 -tests.vpp.perf.ip4.10ge2p1x520-ethip4-ip4base-copwhtlistbase-ndrpdrdisc.tc02-64b-1t1c-ethip4-ip4base-copwhtlistbase-pdrdisc,10ge2p1x520: 64B-1t1c-ethip4-ip4base-copwhtlistbase-pdrdisc,7.1,8.3,9.2 -tests.vpp.perf.ip4.10ge2p1x520-ethip4-ip4base-iacldstbase-ndrpdrdisc.tc02-64b-1t1c-ethip4-ip4base-iacldstbase-pdrdisc,10ge2p1x520: 64B-1t1c-ethip4-ip4base-iacldstbase-pdrdisc,7.1,7.6,8.3 -tests.vpp.perf.ip4.10ge2p1x520-ethip4-ip4scale200k-ndrpdrdisc.tc02-64b-1t1c-ethip4-ip4scale200k-pdrdisc,10ge2p1x520: 64B-1t1c-ethip4-ip4scale200k-pdrdisc,8.5,9,9.7 -tests.vpp.perf.ip4.10ge2p1x520-ethip4-ip4scale20k-ndrpdrdisc.tc02-64b-1t1c-ethip4-ip4scale20k-pdrdisc,10ge2p1x520: 64B-1t1c-ethip4-ip4scale20k-pdrdisc,8.5,9,9.7 -tests.vpp.perf.ip4.10ge2p1x520-ethip4-ip4scale2m-ndrpdrdisc.tc02-64b-1t1c-ethip4-ip4scale2m-pdrdisc,10ge2p1x520: 64B-1t1c-ethip4-ip4scale2m-pdrdisc,8.3,8.1,8.3 -tests.vpp.perf.ip4.10ge2p1x520-ethip4-ip4base-ipolicemarkbase-ndrpdrdisc.tc02-64b-1t1c-ethip4-ip4base-ipolicemarkbase-pdrdisc,10ge2p1x520: 64B-1t1c-ethip4-ip4base-ipolicemarkbase-pdrdisc,7.1,7.4,8.1 -tests.vpp.perf.ip6.10ge2p1x520-ethip6-ip6scale200k-ndrpdrdisc.tc02-78b-1t1c-ethip6-ip6scale200k-pdrdisc,10ge2p1x520: 78B-1t1c-ethip6-ip6scale200k-pdrdisc,6.9,5.3,5.3 -tests.vpp.perf.ip6.10ge2p1x520-ethip6-ip6scale20k-ndrpdrdisc.tc02-78b-1t1c-ethip6-ip6scale20k-pdrdisc,10ge2p1x520: 78B-1t1c-ethip6-ip6scale20k-pdrdisc,6.9,6.5,6.9 -tests.vpp.perf.vm vhost.10ge2p1x520-dot1q-l2xcbase-eth-2vhostvr1024-1vm-ndrpdrdisc.tc02-64b-1t1c-eth-l2xcbase-eth-2vhost-1vm-pdrdisc,10ge2p1x520: 64B-1t1c-eth-l2xcbase-eth-2vhost-1vm-pdrdisc,2.6,3.2,3.2 -tests.vpp.perf.vm vhost.10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhostvr1024-1vm-ndrpdrdisc.tc02-64b-1t1c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc,10ge2p1x520: 64B-1t1c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc,2.1,2.9,3.2 -tests.vpp.perf.vm vhost.10ge2p1x520-ethip4-ip4base-eth-2vhostvr1024-1vm-ndrpdrdisc.tc02-64b-1t1c-ethip4-ip4base-eth-2vhostvr1024-1vm-pdrdisc,10ge2p1x520: 64B-1t1c-ethip4-ip4base-eth-2vhostvr1024-1vm-pdrdisc,2,2.7,3 -tests.vpp.perf.ip4 tunnels.10ge2p1x520-ethip4lispip4-ip4base-ndrpdrdisc.tc02-64b-1t1c-ethip4lispip4-ip4base-pdrdisc,10ge2p1x520: 64B-1t1c-ethip4lispip4-ip4base-pdrdisc,4.6,4.8,5.5 -tests.vpp.perf.ip6.10ge2p1x520-ethip6-ip6base-ndrpdrdisc.tc02-78b-1t1c-ethip6-ip6base-pdrdisc,10ge2p1x520: 78B-1t1c-ethip6-ip6base-pdrdisc,7.7,7.3,8.1 -tests.vpp.perf.ip6.10ge2p1x520-ethip6-ip6base-copwhtlistbase-ndrpdrdisc.tc02-78b-1t1c-ethip6-ip6base-copwhtlistbase-pdrdisc,10ge2p1x520: 78B-1t1c-ethip6-ip6base-copwhtlistbase-pdrdisc,6.1,6.1,6.9 -tests.vpp.perf.ip6.10ge2p1x520-ethip6-ip6base-iacldstbase-ndrpdrdisc.tc02-78b-1t1c-ethip6-ip6base-iacldstbase-pdrdisc,10ge2p1x520: 78B-1t1c-ethip6-ip6base-iacldstbase-pdrdisc,6.5,6.1,6.9 -tests.vpp.perf.ip6.10ge2p1x520-ethip6-ip6scale2m-ndrpdrdisc.tc02-78b-1t1c-ethip6-ip6scale2m-pdrdisc,10ge2p1x520: 78B-1t1c-ethip6-ip6scale2m-pdrdisc,5.3,4.2,4.6 -tests.vpp.perf.container memif.10ge2p1x520-eth-l2bdbasemaclrn-eth-2memif-1lxc-ndrpdrdisc.tc01-64b-1t1c-eth-l2bdbasemaclrn-eth-2memif-1lxc-ndrdisc,10ge2p1x520: 64B-1t1c-eth-l2bdbasemaclrn-eth-2memif-1lxc-ndrdisc,,, -tests.vpp.perf.container memif.10ge2p1x520-eth-l2bdbasemaclrn-eth-2memif-1lxc-ndrpdrdisc.tc02-64b-1t1c-eth-l2bdbasemaclrn-eth-2memif-1lxc-pdrdisc,10ge2p1x520: 64B-1t1c-eth-l2bdbasemaclrn-eth-2memif-1lxc-pdrdisc,,, -tests.vpp.perf.container memif.10ge2p1x520-eth-l2xcbase-eth-2memif-1drc-ndrpdrdisc.tc01-64b-1t1c-eth-l2xcbase-eth-2memif-1dcr-ndrdisc,10ge2p1x520: 64B-1t1c-eth-l2xcbase-eth-2memif-1dcr-ndrdisc,,, -tests.vpp.perf.container memif.10ge2p1x520-eth-l2xcbase-eth-2memif-1drc-ndrpdrdisc.tc02-64b-1t1c-eth-l2xcbase-eth-2memif-1dcr-pdrdisc,10ge2p1x520: 64B-1t1c-eth-l2xcbase-eth-2memif-1dcr-pdrdisc,,, -tests.vpp.perf.container memif.10ge2p1x520-eth-l2xcbase-eth-2memif-1lxc-ndrpdrdisc.tc01-64b-1t1c-eth-l2xcbase-eth-2memif-1lxc-ndrdisc,10ge2p1x520: 64B-1t1c-eth-l2xcbase-eth-2memif-1lxc-ndrdisc,,, -tests.vpp.perf.container memif.10ge2p1x520-eth-l2xcbase-eth-2memif-1lxc-ndrpdrdisc.tc02-64b-1t1c-eth-l2xcbase-eth-2memif-1lxc-pdrdisc,10ge2p1x520: 64B-1t1c-eth-l2xcbase-eth-2memif-1lxc-pdrdisc,,, -tests.kubernetes.perf.container memif.10ge2p1x520-eth-1drcl2bdbasemaclrn-eth-2memif-1drcl2xc-k8s-ndrpdrdisc.tc01-64b-1t1c-eth-1drcl2bdbasemaclrn-eth-2memif-1drcl2xc-k8s-ndrdisc,10ge2p1x520: 64B-1t1c-eth-1drcl2bdbasemaclrn-eth-2memif-1drcl2xc-k8s-ndrdisc,,, -tests.kubernetes.perf.container memif.10ge2p1x520-eth-1drcl2bdbasemaclrn-eth-2memif-1drcl2xc-k8s-ndrpdrdisc.tc02-64b-1t1c-eth-1drcl2bdbasemaclrn-eth-2memif-1drcl2xc-k8s-pdrdisc,10ge2p1x520: 64B-1t1c-eth-1drcl2bdbasemaclrn-eth-2memif-1drcl2xc-k8s-pdrdisc,,, -tests.kubernetes.perf.container memif.10ge2p1x520-eth-1drcl2bdbasemaclrn-eth-4memif-2drcl2xc-k8s-ndrpdrdisc.tc01-64b-1t1c-eth-1drcl2bdbasemaclrn-eth-4memif-2drcl2xc-k8s-ndrdisc,10ge2p1x520: 64B-1t1c-eth-1drcl2bdbasemaclrn-eth-4memif-2drcl2xc-k8s-ndrdisc,,, -tests.kubernetes.perf.container memif.10ge2p1x520-eth-1drcl2bdbasemaclrn-eth-4memif-2drcl2xc-k8s-ndrpdrdisc.tc02-64b-1t1c-eth-1drcl2bdbasemaclrn-eth-4memif-2drcl2xc-k8s-pdrdisc,10ge2p1x520: 64B-1t1c-eth-1drcl2bdbasemaclrn-eth-4memif-2drcl2xc-k8s-pdrdisc,,, -tests.kubernetes.perf.container memif.10ge2p1x520-eth-1drcl2xcbase-eth-2memif-1drcl2xc-k8s-ndrpdrdisc.tc01-64b-1t1c-eth-1drcl2xcbase-eth-2memif-1drcl2xc-k8s-ndrdisc,10ge2p1x520: 64B-1t1c-eth-1drcl2xcbase-eth-2memif-1drcl2xc-k8s-ndrdisc,,, -tests.kubernetes.perf.container memif.10ge2p1x520-eth-1drcl2xcbase-eth-2memif-1drcl2xc-k8s-ndrpdrdisc.tc02-64b-1t1c-eth-1drcl2xcbase-eth-2memif-1drcl2xc-k8s-pdrdisc,10ge2p1x520: 64B-1t1c-eth-1drcl2xcbase-eth-2memif-1drcl2xc-k8s-pdrdisc,,, -tests.kubernetes.perf.container memif.10ge2p1x520-eth-1drcl2xcbase-eth-4memif-2drcl2xc-k8s-ndrpdrdisc.tc01-64b-1t1c-eth-1drcl2xcbase-eth-4memif-2drcl2xc-k8s-ndrdisc,10ge2p1x520: 64B-1t1c-eth-1drcl2xcbase-eth-4memif-2drcl2xc-k8s-ndrdisc,,, -tests.kubernetes.perf.container memif.10ge2p1x520-eth-1drcl2xcbase-eth-4memif-2drcl2xc-k8s-ndrpdrdisc.tc02-64b-1t1c-eth-1drcl2xcbase-eth-4memif-2drcl2xc-k8s-pdrdisc,10ge2p1x520: 64B-1t1c-eth-1drcl2xcbase-eth-4memif-2drcl2xc-k8s-pdrdisc,,, diff --git a/docs/report/vpp_performance_tests/throughput_speedup_multi_core/container_orchestrated.rst b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/container_orchestrated.rst index d3fedbef57..f5934d2619 100644 --- a/docs/report/vpp_performance_tests/throughput_speedup_multi_core/container_orchestrated.rst +++ b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/container_orchestrated.rst @@ -161,203 +161,3 @@ CSIT source code for the test cases used for plots can be found in \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{k8s-memif-3n-hsw-xl710-64b-base_and_scale-pdr-tsa} \label{fig:k8s-memif-3n-hsw-xl710-64b-base_and_scale-pdr-tsa} \end{figure} - -3n-skx-x710 -~~~~~~~~~~~ - -64b-base_and_scale ------------------- - -.. raw:: html - - <center><b> - -:index:`Speedup: k8s-memif-3n-skx-xl710-64b-base_and_scale-ndr` - -.. raw:: html - - </b> - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/k8s-memif-3n-skx-xl710-64b-base_and_scale-ndr-tsa.html"></iframe> - <p><br><br></p> - </center> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/vpp/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{k8s-memif-3n-skx-xl710-64b-base_and_scale-ndr-tsa} - \label{fig:k8s-memif-3n-skx-xl710-64b-base_and_scale-ndr-tsa} - \end{figure} - -.. raw:: html - - <center><b> - -:index:`Speedup: k8s-memif-3n-skx-xl710-64b-base_and_scale-pdr` - -.. raw:: html - - </b> - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/k8s-memif-3n-skx-xl710-64b-base_and_scale-pdr-tsa.html"></iframe> - <p><br><br></p> - </center> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/vpp/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{k8s-memif-3n-skx-xl710-64b-base_and_scale-pdr-tsa} - \label{fig:k8s-memif-3n-skx-xl710-64b-base_and_scale-pdr-tsa} - \end{figure} - -3n-skx-xxv710 -~~~~~~~~~~~~~ - -64b-base_and_scale ------------------- - -.. raw:: html - - <center><b> - -:index:`Speedup: k8s-memif-3n-skx-xxv710-64b-base_and_scale-ndr` - -.. raw:: html - - </b> - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/k8s-memif-3n-skx-xxv710-64b-base_and_scale-ndr-tsa.html"></iframe> - <p><br><br></p> - </center> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/vpp/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{k8s-memif-3n-skx-xxv710-64b-base_and_scale-ndr-tsa} - \label{fig:k8s-memif-3n-skx-xxv710-64b-base_and_scale-ndr-tsa} - \end{figure} - -.. raw:: html - - <center><b> - -:index:`Speedup: k8s-memif-3n-skx-xxv710-64b-base_and_scale-pdr` - -.. raw:: html - - </b> - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/k8s-memif-3n-skx-xxv710-64b-base_and_scale-pdr-tsa.html"></iframe> - <p><br><br></p> - </center> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/vpp/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{k8s-memif-3n-skx-xxv710-64b-base_and_scale-pdr-tsa} - \label{fig:k8s-memif-3n-skx-xxv710-64b-base_and_scale-pdr-tsa} - \end{figure} - -2n-skx-x710 -~~~~~~~~~~~ - -64b-base_and_scale ------------------- - -.. raw:: html - - <center><b> - -:index:`Speedup: k8s-memif-2n-skx-x710-64b-base_and_scale-ndr` - -.. raw:: html - - </b> - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/k8s-memif-2n-skx-x710-64b-base_and_scale-ndr-tsa.html"></iframe> - <p><br><br></p> - </center> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/vpp/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{k8s-memif-2n-skx-x710-64b-base_and_scale-ndr-tsa} - \label{fig:k8s-memif-2n-skx-x710-64b-base_and_scale-ndr-tsa} - \end{figure} - -.. raw:: html - - <center><b> - -:index:`Speedup: k8s-memif-2n-skx-x710-64b-base_and_scale-pdr` - -.. raw:: html - - </b> - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/k8s-memif-2n-skx-x710-64b-base_and_scale-pdr-tsa.html"></iframe> - <p><br><br></p> - </center> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/vpp/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{k8s-memif-2n-skx-x710-64b-base_and_scale-pdr-tsa} - \label{fig:k8s-memif-2n-skx-x710-64b-base_and_scale-pdr-tsa} - \end{figure} - -2n-skx-xxv710 -~~~~~~~~~~~~~ - -64b-base_and_scale ------------------- - -.. raw:: html - - <center><b> - -:index:`Speedup: k8s-memif-2n-skx-xxv710-64b-base_and_scale-ndr` - -.. raw:: html - - </b> - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/k8s-memif-2n-skx-xxv710-64b-base_and_scale-ndr-tsa.html"></iframe> - <p><br><br></p> - </center> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/vpp/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{k8s-memif-2n-skx-xxv710-64b-base_and_scale-ndr-tsa} - \label{fig:k8s-memif-2n-skx-xxv710-64b-base_and_scale-ndr-tsa} - \end{figure} - -.. raw:: html - - <center><b> - -:index:`Speedup: k8s-memif-2n-skx-xxv710-64b-base_and_scale-pdr` - -.. raw:: html - - </b> - <iframe width="700" height="1000" frameborder="0" scrolling="no" src="../../_static/vpp/k8s-memif-2n-skx-xxv710-64b-base_and_scale-pdr-tsa.html"></iframe> - <p><br><br></p> - </center> - -.. raw:: latex - - \begin{figure}[H] - \centering - \graphicspath{{../_build/_static/vpp/}} - \includegraphics[clip, trim=0cm 8cm 5cm 0cm, width=0.70\textwidth]{k8s-memif-2n-skx-xxv710-64b-base_and_scale-pdr-tsa} - \label{fig:k8s-memif-2n-skx-xxv710-64b-base_and_scale-pdr-tsa} - \end{figure} diff --git a/docs/report/vpp_performance_tests/throughput_speedup_multi_core/index.rst b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/index.rst index edaf9ff9cc..2417ce35dd 100644 --- a/docs/report/vpp_performance_tests/throughput_speedup_multi_core/index.rst +++ b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/index.rst @@ -12,8 +12,13 @@ threaded VPP configurations relative to 1-core configurations. .. note:: Test results have been generated by - `FD.io test executor vpp performance jobs`_ with Robot Framework result - files csit-vpp-perf-\*.zip `archived here <../../_static/archive/>`_. + `FD.io test executor vpp performance job 3n-hsw`_, + `FD.io test executor vpp performance job 3n-skx`_ and + `FD.io test executor vpp performance job 2n-skx`_ with Robot Framework + result files csit-vpp-perf-|srelease|-\*.zip + `archived here <../../_static/archive/>`_. + Plotted data set size per test case is equal to the number of job executions + presented in this report version: **10**. .. toctree:: :maxdepth: 1 diff --git a/resources/tools/presentation/conf.py b/resources/tools/presentation/conf.py index 26d291400a..8236cab605 100644 --- a/resources/tools/presentation/conf.py +++ b/resources/tools/presentation/conf.py @@ -62,6 +62,7 @@ rst_epilog = """ .. |release-1| replace:: {prev_release} .. |srelease| replace:: {srelease} .. |csit-release| replace:: CSIT-{csitrelease} +.. |csit-release-1| replace:: CSIT-{csit_prev_release} .. |vpp-release| replace:: VPP-{vpprelease} release .. |vpp-release-1| replace:: VPP-{vpp_prev_release} release .. |dpdk-release| replace:: DPDK {dpdkrelease} @@ -91,10 +92,10 @@ rst_epilog = """ .. _FD.io test executor vpp performance job 3n-skx: https://jenkins.fd.io/view/csit/job/csit-vpp-perf-verify-1807-3n-skx .. _FD.io test executor vpp performance job 2n-skx: https://jenkins.fd.io/view/csit/job/csit-vpp-perf-verify-1807-2n-skx .. _FD.io test executor ligato performance jobs: https://jenkins.fd.io/job/csit-ligato-perf-{srelease}-all -.. _FD.io test executor vpp functional jobs: https://jenkins.fd.io/view/csit/job/csit-vpp-functional-{srelease}-ubuntu1604-virl/lastSuccessfulBuild -.. _FD.io test executor Honeycomb functional jobs: https://jenkins.fd.io/view/csit/job/hc2vpp-csit-integration-{srelease}-ubuntu1604/lastSuccessfulBuild -.. _FD.io test executor honeycomb performance jobs: https://jenkins.fd.io/view/hc2vpp/job/hc2vpp-csit-perf-master-ubuntu1604/lastSuccessfulBuild -.. _FD.io test executor NSH_SFC functional jobs: https://jenkins.fd.io/view/csit/job/csit-nsh_sfc-verify-func-{srelease}-ubuntu1604-virl/lastSuccessfulBuild +.. _FD.io test executor vpp functional jobs using Ubuntu: https://jenkins.fd.io/view/csit/job/csit-vpp-functional-{srelease}-ubuntu1604-virl +.. _FD.io test executor vpp functional jobs using CentOs: https://jenkins.fd.io/view/csit/job/csit-vpp-functional-{srelease}-centos7-virl +.. _FD.io test executor Honeycomb functional jobs: https://jenkins.fd.io/view/csit/job/hc2vpp-csit-integration-{srelease}-ubuntu1604 +.. _FD.io test executor NSH_SFC functional jobs: https://jenkins.fd.io/view/csit/job/csit-nsh_sfc-verify-func-{srelease}-ubuntu1604-virl .. _FD.io VPP compile job: https://jenkins.fd.io/view/vpp/job/vpp-merge-{srelease}-ubuntu1604/ .. _FD.io DPDK compile job: https://jenkins.fd.io/view/deb-dpdk/job/deb_dpdk-merge-{sdpdkrelease}-ubuntu1604/ .. _CSIT Testbed Setup: https://git.fd.io/csit/tree/resources/tools/testbed-setup/README.md?h={release} @@ -103,6 +104,7 @@ rst_epilog = """ prev_release='rls1804', srelease='1807', csitrelease='18.07', + csit_prev_release='18.04', vpprelease='18.07', vpp_prev_release='18.04', dpdkrelease='18.05', diff --git a/resources/tools/presentation/generator_tables.py b/resources/tools/presentation/generator_tables.py index cd61e321f1..6d81f43ba4 100644 --- a/resources/tools/presentation/generator_tables.py +++ b/resources/tools/presentation/generator_tables.py @@ -391,7 +391,7 @@ def table_performance_comparison(table, input_data): "{0} Stdev [Mpps]".format(table["reference"]["title"]), "{0} {1} [Mpps]".format(table["compare"]["title"], hdr_param), "{0} Stdev [Mpps]".format(table["compare"]["title"]), - "Change [%]"]) + "Delta [%]"]) header_str = ",".join(header) + "\n" except (AttributeError, KeyError) as err: logging.error("The model is invalid, missing parameter: {0}". diff --git a/resources/tools/presentation/specification.yaml b/resources/tools/presentation/specification.yaml index ade101d1b3..76cfeb30c8 100644 --- a/resources/tools/presentation/specification.yaml +++ b/resources/tools/presentation/specification.yaml @@ -95,31 +95,11 @@ - type: "configuration" + data-sets: - plot-vpp-http-server-performance: - csit-vpp-perf-1804-all: - - 39 # wrk - - 40 # wrk - - 41 # wrk - - 42 # wrk - - 43 # wrk - - 44 # wrk - - 45 # wrk - - 46 # wrk - - 47 # wrk - - 48 # wrk + + # VPP Performance vpp-performance-changes: - csit-vpp-perf-1710-all: - - 11 - - 12 - - 13 - - 14 - - 15 - - 16 - - 17 - - 18 - - 19 - - 20 csit-vpp-perf-1801-all: - 124 # sel - 127 # sel @@ -144,6 +124,35 @@ - 30 # sel - 31 # sel - 35 # sel + # TODO: Add Data sources + # 1807 + vpp-compare-testbeds: + # TODO: Add Data sources + csit-vpp-perf-1801-all: + - 124 # sel + - 127 # sel + - 128 # sel + - 141 # sel + - 142 # sel + - 143 # sel + - 145 # sel + - 146 # sel + - 162 # sel + - 163 # sel + - 167 # sel + - 172 # sel acl only + # TODO: Add Data sources + csit-vpp-perf-1804-all: + - 21 # sel + - 22 # sel + - 23 # sel + - 24 # sel + - 27 # sel + - 28 # sel + - 29 # sel + - 30 # sel + - 31 # sel + - 35 # sel vpp-performance-changes-mrr: csit-vpp-perf-check-1801: - 3 @@ -167,7 +176,43 @@ - 20 # mrr - sel - 21 # mrr - sel - 22 # mrr - sel + # TODO: Add Data sources + # 1807 + vpp-perf-results-3n-hsw: + # TODO: Add Data sources + csit-vpp-perf-1804-all: + - 19 # full + - 20 # full + - 25 # full + - 49 # full + vpp-perf-results-3n-skx: + # TODO: Add Data sources + csit-vpp-perf-1804-all: + - 19 # full + - 20 # full + - 25 # full + - 49 # full + vpp-perf-results-2n-skx: + # TODO: Add Data sources + csit-vpp-perf-1804-all: + - 19 # full + - 20 # full + - 25 # full + - 49 # full + vpp-mrr-results-3n-hsw: + # TODO: Add Data sources + csit-vpp-perf-check-1804: + - 5 # mrr - full + vpp-mrr-results-3n-skx: + # TODO: Add Data sources + csit-vpp-perf-check-1804: + - 5 # mrr - full + vpp-mrr-results-2n-skx: + # TODO: Add Data sources + csit-vpp-perf-check-1804: + - 5 # mrr - full plot-throughput-speedup-analysis-3n-hsw: + # TODO: Add Data sources csit-vpp-perf-1804-all: - 19 # full - 20 # full @@ -184,6 +229,7 @@ - 31 # sel - 35 # sel plot-throughput-speedup-analysis-3n-skx: + # TODO: Add Data sources csit-vpp-perf-1804-all: - 19 # full - 20 # full @@ -200,6 +246,7 @@ - 31 # sel - 35 # sel plot-throughput-speedup-analysis-2n-skx: + # TODO: Add Data sources csit-vpp-perf-1804-all: - 19 # full - 20 # full @@ -215,100 +262,8 @@ - 30 # sel - 31 # sel - 35 # sel - plot-ligato-throughput-speedup-analysis-3n-hsw: - csit-ligato-perf-1804-all: - - 5 # sel - - 6 # sel - - 7 # sel - - 8 # sel - - 9 # sel - - 10 # sel - - 11 # sel - - 12 # sel - - 13 # sel - - 14 # sel - plot-ligato-throughput-speedup-analysis-3n-skx: - csit-ligato-perf-1804-all: - - 5 # sel - - 6 # sel - - 7 # sel - - 8 # sel - - 9 # sel - - 10 # sel - - 11 # sel - - 12 # sel - - 13 # sel - - 14 # sel - plot-ligato-throughput-speedup-analysis-2n-skx: - csit-ligato-perf-1804-all: - - 5 # sel - - 6 # sel - - 7 # sel - - 8 # sel - - 9 # sel - - 10 # sel - - 11 # sel - - 12 # sel - - 13 # sel - - 14 # sel - vpp-perf-results-3n-hsw: - csit-vpp-perf-1804-all: - - 19 # full - - 20 # full - - 25 # full - - 49 # full - vpp-perf-results-3n-skx: - csit-vpp-perf-1804-all: - - 19 # full - - 20 # full - - 25 # full - - 49 # full - vpp-perf-results-2n-skx: - csit-vpp-perf-1804-all: - - 19 # full - - 20 # full - - 25 # full - - 49 # full - vpp-func-results-ubuntu: - csit-vpp-functional-1807-ubuntu1604-virl: - - "lastSuccessfulBuild" - vpp-func-results-centos: - csit-vpp-functional-1807-centos7-virl: - - "lastSuccessfulBuild" - vpp-mrr-results-3n-hsw: - csit-vpp-perf-check-1804: - - 5 # mrr - full - vpp-mrr-results-3n-skx: - csit-vpp-perf-check-1804: - - 5 # mrr - full - vpp-mrr-results-2n-skx: - csit-vpp-perf-check-1804: - - 5 # mrr - full - ligato-perf-results-3n-hsw: - csit-ligato-perf-1804-all: - - 4 # full - ligato-perf-results-3n-skx: - csit-ligato-perf-1804-all: - - 4 # full - ligato-perf-results-2n-skx: - csit-ligato-perf-1804-all: - - 4 # full - dpdk-perf-results-3n-hsw: - csit-dpdk-perf-verify-1807-3n-hsw: - - 10 - dpdk-perf-results-3n-skx: - csit-dpdk-perf-verify-1807-3n-skx: - - 12 - dpdk-perf-results-2n-skx: - csit-dpdk-perf-verify-1807-2n-skx: - - 10 - hc-func-results: - csit-hc2vpp-verify-func-1807-ubuntu1604: - - "lastSuccessfulBuild" - nsh-func-results: - csit-nsh_sfc-verify-func-1804-ubuntu1604-virl: - - 7 plot-vpp-throughput-latency-3n-hsw: + # TODO: Add Data sources csit-vpp-perf-1804-all: - 19 # full - 20 # full @@ -325,6 +280,7 @@ - 31 # sel - 35 # sel plot-vpp-throughput-latency-3n-skx: + # TODO: Add Data sources csit-vpp-perf-1804-all: - 19 # full - 20 # full @@ -341,6 +297,7 @@ - 31 # sel - 35 # sel plot-vpp-throughput-latency-2n-skx: + # TODO: Add Data sources csit-vpp-perf-1804-all: - 19 # full - 20 # full @@ -356,6 +313,116 @@ - 30 # sel - 31 # sel - 35 # sel + plot-vpp-http-server-performance: + # TODO: Add Data sources + csit-vpp-perf-1804-all: + - 39 # wrk + - 40 # wrk + - 41 # wrk + - 42 # wrk + - 43 # wrk + - 44 # wrk + - 45 # wrk + - 46 # wrk + - 47 # wrk + - 48 # wrk + + # VPP Functional + vpp-func-results-ubuntu: + csit-vpp-functional-1807-ubuntu1604-virl: + # TODO: Add Data sources + - "lastSuccessfulBuild" + vpp-func-results-centos: + csit-vpp-functional-1807-centos7-virl: + # TODO: Add Data sources + - "lastSuccessfulBuild" + + # VPP Ligato + ligato-perf-results-3n-hsw: + # TODO: Add Data sources + csit-ligato-perf-1804-all: + - 4 # full + plot-ligato-throughput-speedup-analysis-3n-hsw: + # TODO: Add Data sources + csit-ligato-perf-1804-all: + - 5 # sel + - 6 # sel + - 7 # sel + - 8 # sel + - 9 # sel + - 10 # sel + - 11 # sel + - 12 # sel + - 13 # sel + - 14 # sel + plot-ligato-throughput-latency-3n-hsw: + # TODO: Add Data sources + csit-ligato-perf-1804-all: + - 5 # sel + - 6 # sel + - 7 # sel + - 8 # sel + - 9 # sel + - 10 # sel + - 11 # sel + - 12 # sel + - 13 # sel + - 14 # sel + + # DPDK Performance + dpdk-performance-changes: + csit-dpdk-perf-1804-all: + - 4 + - 5 + - 6 + - 7 + - 8 + - 9 + - 10 + - 11 + - 12 + - 13 + csit-dpdk-perf-verify-1807-3n-hsw: + - 2 + - 3 + - 4 + - 5 + - 6 + - 7 + - 8 + - 9 + - 10 + dpdk-compare-testbeds: + csit-dpdk-perf-verify-1807-3n-skx: + - 1 + - 2 + - 3 + - 6 + - 7 + - 8 + - 9 + - 10 + - 11 + - 12 + csit-dpdk-perf-verify-1807-3n-hsw: + - 2 + - 3 + - 4 + - 5 + - 6 + - 7 + - 8 + - 9 + - 10 + dpdk-perf-results-3n-hsw: + csit-dpdk-perf-verify-1807-3n-hsw: + - 10 + dpdk-perf-results-3n-skx: + csit-dpdk-perf-verify-1807-3n-skx: + - 12 + dpdk-perf-results-2n-skx: + csit-dpdk-perf-verify-1807-2n-skx: + - 10 plot-dpdk-throughput-latency-3n-hsw: csit-dpdk-perf-verify-1807-3n-hsw: - 2 @@ -367,6 +434,7 @@ - 8 - 9 - 10 + plot-dpdk-throughput-latency-3n-skx: csit-dpdk-perf-verify-1807-3n-skx: - 1 - 2 @@ -390,42 +458,18 @@ - 8 - 9 - 10 - plot-ligato-throughput-latency-3n-hsw: - csit-ligato-perf-1804-all: - - 5 # sel - - 6 # sel - - 7 # sel - - 8 # sel - - 9 # sel - - 10 # sel - - 11 # sel - - 12 # sel - - 13 # sel - - 14 # sel - plot-ligato-throughput-latency-3n-skx: - csit-ligato-perf-1804-all: - - 5 # sel - - 6 # sel - - 7 # sel - - 8 # sel - - 9 # sel - - 10 # sel - - 11 # sel - - 12 # sel - - 13 # sel - - 14 # sel - plot-ligato-throughput-latency-2n-skx: - csit-ligato-perf-1804-all: - - 5 # sel - - 6 # sel - - 7 # sel - - 8 # sel - - 9 # sel - - 10 # sel - - 11 # sel - - 12 # sel - - 13 # sel - - 14 # sel + + # HoneyComb Functional + hc-func-results: + csit-hc2vpp-verify-func-1807-ubuntu1604: + # TODO: Add Data sources + - "lastSuccessfulBuild" + + #NSH SFC Functional + nsh-func-results: + # TODO: Add Data sources + csit-nsh_sfc-verify-func-1804-ubuntu1604-virl: + - 7 plot-layouts: @@ -639,29 +683,16 @@ dst-path: "{DIR[WORKING,SRC]}" - - type: "input" # Ignored in debug mode + type: "input" general: file-name: "robot-plugin.zip" file-format: ".zip" download-path: "{job}/{build}/robot/report/*zip*/{filename}" extract: "robot-plugin/output.xml" + builds: - csit-vpp-perf-1710-all: - - 11 - - 12 - - 13 - - 14 - - 15 - - 16 - - 17 - - 18 - - 19 - - 20 + csit-vpp-perf-1801-all: -# - 122 # full -# - 126 # full -# - 129 # full -# - 140 # full - 124 # sel - 127 # sel - 128 # sel @@ -699,6 +730,13 @@ - 46 # wrk - 47 # wrk - 48 # wrk + # TODO: Add Data sources +# csit-vpp-perf-verify-1807-3n-hsw: +# - +# csit-vpp-perf-verify-1807-3n-skx: +# - +# csit-vpp-perf-verify-1807-2n-skx: +# - csit-vpp-perf-check-1801: - 3 # mrr - 4 # mrr @@ -722,28 +760,8 @@ - 20 # mrr - sel - 21 # mrr - sel - 22 # mrr - sel -# csit-ligato-perf-1710-all: -# - 5 -# - 7 -# - 8 -# - 9 -# - 10 -# - 11 -# - 12 -# - 13 -# - 16 -# - 17 -# csit-ligato-perf-1801-all: -# - 16 # sel -# - 17 # sel -# - 18 # sel -# - 19 # sel -# - 20 # sel -# - 21 # sel -# - 22 # sel -# - 23 # sel -# - 24 # sel -# - 25 # full + # TODO: Add Data sources + # mrr 1807 csit-ligato-perf-1804-all: - 4 # full - 5 # sel @@ -756,6 +774,20 @@ - 12 # sel - 13 # sel - 14 # sel + # TODO: Add Data sources +# csit-ligato-perf-1807-all: +# - + csit-dpdk-perf-1804-all: + - 4 + - 5 + - 6 + - 7 + - 8 + - 9 + - 10 + - 11 + - 12 + - 13 csit-dpdk-perf-verify-1807-3n-hsw: - 2 - 3 @@ -789,12 +821,16 @@ - 9 - 10 csit-vpp-functional-1807-ubuntu1604-virl: + # TODO: Add Data sources - "lastSuccessfulBuild" csit-vpp-functional-1807-centos7-virl: + # TODO: Add Data sources - "lastSuccessfulBuild" + # TODO: Add Data sources csit-nsh_sfc-verify-func-1804-ubuntu1604-virl: - 7 csit-hc2vpp-verify-func-1807-ubuntu1604: + # TODO: Add Data sources - "lastSuccessfulBuild" - @@ -810,6 +846,8 @@ ### T A B L E S ### ################################################################################ +################################################################################ +# VPP - type: "table" title: "VPP Performance Changes 1t1c pdr" @@ -817,38 +855,39 @@ output-file: "{DIR[STATIC,VPP]}/performance-changes-1t1c-pdr" history: - - title: "rls1710" + title: "rls1801" data: - csit-vpp-perf-1710-all: - - 11 - - 12 - - 13 - - 14 - - 15 - - 16 - - 17 - - 18 - - 19 - - 20 + csit-vpp-perf-1801-all: + - 124 # sel + - 127 # sel + - 128 # sel + - 141 # sel + - 142 # sel + - 143 # sel + - 145 # sel + - 146 # sel + - 162 # sel + - 163 # sel + - 167 # sel + - 172 # sel acl only reference: - title: "rls1801" + title: "rls1804" data: - csit-vpp-perf-1801-all: - - 124 # sel - - 127 # sel - - 128 # sel - - 141 # sel - - 142 # sel - - 143 # sel - - 145 # sel - - 146 # sel - - 162 # sel - - 163 # sel - - 167 # sel - - 172 # sel acl only + csit-vpp-perf-1804-all: + - 21 # sel + - 22 # sel + - 23 # sel + - 24 # sel + - 27 # sel + - 28 # sel + - 29 # sel + - 30 # sel + - 31 # sel + - 35 # sel compare: - title: "rls1804" + title: "rls1807" data: + # TODO: Add Data sources csit-vpp-perf-1804-all: - 21 # sel - 22 # sel @@ -875,38 +914,39 @@ output-file: "{DIR[STATIC,VPP]}/performance-changes-2t2c-pdr" history: - - title: "rls1710" + title: "rls1801" data: - csit-vpp-perf-1710-all: - - 11 - - 12 - - 13 - - 14 - - 15 - - 16 - - 17 - - 18 - - 19 - - 20 + csit-vpp-perf-1801-all: + - 124 # sel + - 127 # sel + - 128 # sel + - 141 # sel + - 142 # sel + - 143 # sel + - 145 # sel + - 146 # sel + - 162 # sel + - 163 # sel + - 167 # sel + - 172 # sel acl only reference: - title: "rls1801" + title: "rls1804" data: - csit-vpp-perf-1801-all: - - 124 # sel - - 127 # sel - - 128 # sel - - 141 # sel - - 142 # sel - - 143 # sel - - 145 # sel - - 146 # sel - - 162 # sel - - 163 # sel - - 167 # sel - - 172 # sel acl only + csit-vpp-perf-1804-all: + - 21 # sel + - 22 # sel + - 23 # sel + - 24 # sel + - 27 # sel + - 28 # sel + - 29 # sel + - 30 # sel + - 31 # sel + - 35 # sel compare: - title: "rls1804" + title: "rls1807" data: + # TODO: Add Data sources csit-vpp-perf-1804-all: - 21 # sel - 22 # sel @@ -933,22 +973,126 @@ output-file: "{DIR[STATIC,VPP]}/performance-changes-1t1c-ndr" history: - - title: "rls1710" + title: "rls1801" data: - csit-vpp-perf-1710-all: - - 11 - - 12 - - 13 - - 14 - - 15 - - 16 - - 17 - - 18 - - 19 - - 20 + csit-vpp-perf-1801-all: + - 124 # sel + - 127 # sel + - 128 # sel + - 141 # sel + - 142 # sel + - 143 # sel + - 145 # sel + - 146 # sel + - 162 # sel + - 163 # sel + - 167 # sel + - 172 # sel acl only + reference: + title: "rls1804" + data: + csit-vpp-perf-1804-all: + - 21 # sel + - 22 # sel + - 23 # sel + - 24 # sel + - 27 # sel + - 28 # sel + - 29 # sel + - 30 # sel + - 31 # sel + - 35 # sel + compare: + title: "rls1807" + data: + # TODO: Add Data sources + csit-vpp-perf-1804-all: + - 21 # sel + - 22 # sel + - 23 # sel + - 24 # sel + - 27 # sel + - 28 # sel + - 29 # sel + - 30 # sel + - 31 # sel + - 35 # sel + data: "vpp-performance-changes" + include-tests: "NDR" # "PDR" | "NDR" | "MRR" + filter: "'1T1C' and ('NDRPDR' or 'NDRPDRDISC')" # "('NDRPDR' or 'NDRPDRDISC')" | "'MRR'" + parameters: + - "name" + - "parent" + - "throughput" + +- + type: "table" + title: "VPP Performance Changes 2t2c ndr" + algorithm: "table_performance_comparison" + output-file: "{DIR[STATIC,VPP]}/performance-changes-2t2c-ndr" + history: + - + title: "rls1801" + data: + csit-vpp-perf-1801-all: + - 124 # sel + - 127 # sel + - 128 # sel + - 141 # sel + - 142 # sel + - 143 # sel + - 145 # sel + - 146 # sel + - 162 # sel + - 163 # sel + - 167 # sel + - 172 # sel acl only + reference: + title: "rls1804" + data: + csit-vpp-perf-1804-all: + - 21 # sel + - 22 # sel + - 23 # sel + - 24 # sel + - 27 # sel + - 28 # sel + - 29 # sel + - 30 # sel + - 31 # sel + - 35 # sel + compare: + title: "rls1807" + data: + # TODO: Add Data sources + csit-vpp-perf-1804-all: + - 21 # sel + - 22 # sel + - 23 # sel + - 24 # sel + - 27 # sel + - 28 # sel + - 29 # sel + - 30 # sel + - 31 # sel + - 35 # sel + data: "vpp-performance-changes" + include-tests: "NDR" # "PDR" | "NDR" | "MRR" + filter: "'2T2C' and ('NDRPDR' or 'NDRPDRDISC')" # "('NDRPDR' or 'NDRPDRDISC')" | "'MRR'" + parameters: + - "name" + - "parent" + - "throughput" + +- + type: "table" + title: "VPP Comparison Across Testbeds 3n-hsw to 3n-skx ndr" + algorithm: "table_performance_comparison" + output-file: "{DIR[STATIC,VPP]}/performance-compare-testbeds-3n-hsw-3n-skx-ndr" reference: - title: "rls1801" + title: "3-Node Hsw" data: + # TODO: Add Data sources csit-vpp-perf-1801-all: - 124 # sel - 127 # sel @@ -963,8 +1107,9 @@ - 167 # sel - 172 # sel acl only compare: - title: "rls1804" + title: "3-Node Skx" data: + # TODO: Add Data sources csit-vpp-perf-1804-all: - 21 # sel - 22 # sel @@ -976,9 +1121,9 @@ - 30 # sel - 31 # sel - 35 # sel - data: "vpp-performance-changes" + data: "vpp-compare-testbeds" include-tests: "NDR" # "PDR" | "NDR" | "MRR" - filter: "'1T1C' and ('NDRPDR' or 'NDRPDRDISC')" # "('NDRPDR' or 'NDRPDRDISC')" | "'MRR'" + filter: "'NIC_Intel-X710' and ('NDRPDR' or 'NDRPDRDISC')" # "('NDRPDR' or 'NDRPDRDISC')" | "'MRR'" parameters: - "name" - "parent" @@ -986,27 +1131,13 @@ - type: "table" - title: "VPP Performance Changes 2t2c ndr" + title: "VPP Comparison Across Testbeds 3n-hsw to 3n-skx pdr" algorithm: "table_performance_comparison" - output-file: "{DIR[STATIC,VPP]}/performance-changes-2t2c-ndr" - history: - - - title: "rls1710" - data: - csit-vpp-perf-1710-all: - - 11 - - 12 - - 13 - - 14 - - 15 - - 16 - - 17 - - 18 - - 19 - - 20 + output-file: "{DIR[STATIC,VPP]}/performance-compare-testbeds-3n-hsw-3n-skx-pdr" reference: - title: "rls1801" + title: "3-Node Hsw" data: + # TODO: Add Data sources csit-vpp-perf-1801-all: - 124 # sel - 127 # sel @@ -1021,8 +1152,9 @@ - 167 # sel - 172 # sel acl only compare: - title: "rls1804" + title: "3-Node Skx" data: + # TODO: Add Data sources csit-vpp-perf-1804-all: - 21 # sel - 22 # sel @@ -1034,9 +1166,9 @@ - 30 # sel - 31 # sel - 35 # sel - data: "vpp-performance-changes" - include-tests: "NDR" # "PDR" | "NDR" | "MRR" - filter: "'2T2C' and ('NDRPDR' or 'NDRPDRDISC')" # "('NDRPDR' or 'NDRPDRDISC')" | "'MRR'" + data: "vpp-compare-testbeds" + include-tests: "PDR" # "PDR" | "NDR" | "MRR" + filter: "'NIC_Intel-X710' and ('NDRPDR' or 'NDRPDRDISC')" # "('NDRPDR' or 'NDRPDRDISC')" | "'MRR'" parameters: - "name" - "parent" @@ -1047,23 +1179,39 @@ title: "VPP Performance Changes - 1t1c MRR" algorithm: "table_performance_comparison" output-file: "{DIR[STATIC,VPP]}/performance-changes-1t1c-mrr" + history: + - + title: "rls1801" + data: + csit-vpp-perf-check-1801: + - 3 + - 4 + - 5 + - 6 + - 7 + - 8 + - 9 + - 11 + - 12 + - 13 reference: - title: "Release 1801" + title: "rls1804" data: - csit-vpp-perf-check-1801: - - 3 - - 4 - - 5 - - 6 - - 7 - - 8 - - 9 - - 11 - - 12 - - 13 + csit-vpp-perf-check-1804: + - 6 # mrr - sel + - 13 # mrr - sel + - 14 # mrr - sel + - 15 # mrr - sel + - 16 # mrr - sel + - 17 # mrr - sel + - 19 # mrr - sel + - 20 # mrr - sel + - 21 # mrr - sel + - 22 # mrr - sel compare: - title: "Release 1804" + title: "rls1807" data: + # TODO: Add Data sources csit-vpp-perf-check-1804: - 6 # mrr - sel - 13 # mrr - sel @@ -1088,23 +1236,39 @@ title: "VPP Performance Changes - 2t2c MRR" algorithm: "table_performance_comparison" output-file: "{DIR[STATIC,VPP]}/performance-changes-2t2c-mrr" + history: + - + title: "rls1801" + data: + csit-vpp-perf-check-1801: + - 3 + - 4 + - 5 + - 6 + - 7 + - 8 + - 9 + - 11 + - 12 + - 13 reference: - title: "Release 1801" + title: "rls1804" data: - csit-vpp-perf-check-1801: - - 3 - - 4 - - 5 - - 6 - - 7 - - 8 - - 9 - - 11 - - 12 - - 13 + csit-vpp-perf-check-1804: + - 6 # mrr - sel + - 13 # mrr - sel + - 14 # mrr - sel + - 15 # mrr - sel + - 16 # mrr - sel + - 17 # mrr - sel + - 19 # mrr - sel + - 20 # mrr - sel + - 21 # mrr - sel + - 22 # mrr - sel compare: - title: "Release 1804" + title: "rls1807" data: + # TODO: Add Data sources csit-vpp-perf-check-1804: - 6 # mrr - sel - 13 # mrr - sel @@ -1129,23 +1293,39 @@ title: "VPP Performance Changes - 4t4c MRR" algorithm: "table_performance_comparison" output-file: "{DIR[STATIC,VPP]}/performance-changes-4t4c-mrr" + history: + - + title: "rls1801" + data: + csit-vpp-perf-check-1801: + - 3 + - 4 + - 5 + - 6 + - 7 + - 8 + - 9 + - 11 + - 12 + - 13 reference: - title: "Release 1801" + title: "rls1804" data: - csit-vpp-perf-check-1801: - - 3 - - 4 - - 5 - - 6 - - 7 - - 8 - - 9 - - 11 - - 12 - - 13 + csit-vpp-perf-check-1804: + - 6 # mrr - sel + - 13 # mrr - sel + - 14 # mrr - sel + - 15 # mrr - sel + - 16 # mrr - sel + - 17 # mrr - sel + - 19 # mrr - sel + - 20 # mrr - sel + - 21 # mrr - sel + - 22 # mrr - sel compare: - title: "Release 1804" + title: "rls1807" data: + # TODO: Add Data sources csit-vpp-perf-check-1804: - 6 # mrr - sel - 13 # mrr - sel @@ -1622,6 +1802,248 @@ - "doc" - "msg" +################################################################################ +# DPDK +- + type: "table" + title: "DPDK Performance Changes 1t1c pdr" + algorithm: "table_performance_comparison" + output-file: "{DIR[STATIC,DPDK]}/performance-changes-1t1c-pdr" + reference: + title: "rls1804" + data: + csit-dpdk-perf-1804-all: + - 4 + - 5 + - 6 + - 7 + - 8 + - 9 + - 10 + - 11 + - 12 + - 13 + compare: + title: "rls1807" + data: + csit-dpdk-perf-verify-1807-3n-hsw: + - 2 + - 3 + - 4 + - 5 + - 6 + - 7 + - 8 + - 9 + - 10 + data: "dpdk-performance-changes" + include-tests: "PDR" # "PDR" | "NDR" | "MRR" + filter: "'1T1C' and ('NDRPDR' or 'NDRPDRDISC')" # "('NDRPDR' or 'NDRPDRDISC')" | "'MRR'" + parameters: + - "name" + - "parent" + - "throughput" + +- + type: "table" + title: "DPDK Performance Changes 2t2c pdr" + algorithm: "table_performance_comparison" + output-file: "{DIR[STATIC,DPDK]}/performance-changes-2t2c-pdr" + reference: + title: "rls1804" + data: + csit-dpdk-perf-1804-all: + - 4 + - 5 + - 6 + - 7 + - 8 + - 9 + - 10 + - 11 + - 12 + - 13 + compare: + title: "rls1807" + data: + csit-dpdk-perf-verify-1807-3n-hsw: + - 2 + - 3 + - 4 + - 5 + - 6 + - 7 + - 8 + - 9 + - 10 + data: "dpdk-performance-changes" + include-tests: "PDR" # "PDR" | "NDR" | "MRR" + filter: "'2T2C' and ('NDRPDR' or 'NDRPDRDISC')" # "('NDRPDR' or 'NDRPDRDISC')" | "'MRR'" + parameters: + - "name" + - "parent" + - "throughput" + +- + type: "table" + title: "DPDK Performance Changes 1t1c ndr" + algorithm: "table_performance_comparison" + output-file: "{DIR[STATIC,DPDK]}/performance-changes-1t1c-ndr" + reference: + title: "rls1804" + data: + csit-dpdk-perf-1804-all: + - 4 + - 5 + - 6 + - 7 + - 8 + - 9 + - 10 + - 11 + - 12 + - 13 + compare: + title: "rls1807" + data: + csit-dpdk-perf-verify-1807-3n-hsw: + - 2 + - 3 + - 4 + - 5 + - 6 + - 7 + - 8 + - 9 + - 10 + data: "dpdk-performance-changes" + include-tests: "NDR" # "PDR" | "NDR" | "MRR" + filter: "'1T1C' and ('NDRPDR' or 'NDRPDRDISC')" # "('NDRPDR' or 'NDRPDRDISC')" | "'MRR'" + parameters: + - "name" + - "parent" + - "throughput" + +- + type: "table" + title: "DPDK Performance Changes 2t2c ndr" + algorithm: "table_performance_comparison" + output-file: "{DIR[STATIC,DPDK]}/performance-changes-2t2c-ndr" + reference: + title: "rls1804" + data: + csit-dpdk-perf-1804-all: + - 4 + - 5 + - 6 + - 7 + - 8 + - 9 + - 10 + - 11 + - 12 + - 13 + compare: + title: "rls1807" + data: + csit-dpdk-perf-verify-1807-3n-hsw: + - 2 + - 3 + - 4 + - 5 + - 6 + - 7 + - 8 + - 9 + - 10 + data: "dpdk-performance-changes" + include-tests: "NDR" # "PDR" | "NDR" | "MRR" + filter: "'2T2C' and ('NDRPDR' or 'NDRPDRDISC')" # "('NDRPDR' or 'NDRPDRDISC')" | "'MRR'" + parameters: + - "name" + - "parent" + - "throughput" + +- + type: "table" + title: "DPDK Comparison Across Testbeds 3n-hsw to 3n-skx ndr" + algorithm: "table_performance_comparison" + output-file: "{DIR[STATIC,DPDK]}/performance-compare-testbeds-3n-hsw-3n-skx-ndr" + reference: + title: "3-Node Hsw" + data: + csit-dpdk-perf-verify-1807-3n-hsw: + - 2 + - 3 + - 4 + - 5 + - 6 + - 7 + - 8 + - 9 + - 10 + compare: + title: "3-Node Skx" + data: + csit-dpdk-perf-verify-1807-3n-skx: + - 1 + - 2 + - 3 + - 6 + - 7 + - 8 + - 9 + - 10 + - 11 + - 12 + data: "dpdk-compare-testbeds" + include-tests: "NDR" # "PDR" | "NDR" | "MRR" + filter: "'NIC_Intel-X710' and ('NDRPDR' or 'NDRPDRDISC')" # "('NDRPDR' or 'NDRPDRDISC')" | "'MRR'" + parameters: + - "name" + - "parent" + - "throughput" + +- + type: "table" + title: "DPDK Comparison Across Testbeds 3n-hsw to 3n-skx pdr" + algorithm: "table_performance_comparison" + output-file: "{DIR[STATIC,DPDK]}/performance-compare-testbeds-3n-hsw-3n-skx-pdr" + reference: + title: "3-Node Hsw" + data: + csit-dpdk-perf-verify-1807-3n-hsw: + - 2 + - 3 + - 4 + - 5 + - 6 + - 7 + - 8 + - 9 + - 10 + compare: + title: "3-Node Skx" + data: + csit-dpdk-perf-verify-1807-3n-skx: + - 1 + - 2 + - 3 + - 6 + - 7 + - 8 + - 9 + - 10 + - 11 + - 12 + data: "dpdk-compare-testbeds" + include-tests: "PDR" # "PDR" | "NDR" | "MRR" + filter: "'NIC_Intel-X710' and ('NDRPDR' or 'NDRPDRDISC')" # "('NDRPDR' or 'NDRPDRDISC')" | "'MRR'" + parameters: + - "name" + - "parent" + - "throughput" + - type: "table" title: "Detailed Test Results - DPDK Performance Results 3n-hsw" @@ -1685,6 +2107,9 @@ - "parent" - "msg" + +################################################################################ +# HoneyComb - type: "table" title: "Detailed Test Results - Honeycomb Functional Results" @@ -1711,6 +2136,8 @@ - "doc" - "status" +################################################################################ +# NSH SFC - type: "table" title: "Detailed Test Results - NSH SFC Functional Results" @@ -2011,7 +2438,7 @@ - "doc" - "level" - "parent" - data-start-level: 2 + data-start-level: 4 - type: "file" @@ -2029,7 +2456,7 @@ - "doc" - "level" - "parent" - data-start-level: 2 + data-start-level: 4 - type: "file" @@ -2046,7 +2473,7 @@ - "doc" - "level" - "parent" - data-start-level: 2 + data-start-level: 4 - type: "file" @@ -2064,7 +2491,7 @@ - "doc" - "level" - "parent" - data-start-level: 2 + data-start-level: 4 - type: "file" @@ -4554,426 +4981,6 @@ title: "" # No title layout: "plot-throughput" -# Packet Throughput - VPP Container Orchestrated Memif 3n-skx-x710 -- - type: "plot" - title: "VPP Throughput: k8s-memif-3n-skx-x710-64b-2t1c-base_and_scale-ndr" - algorithm: "plot_performance_box" - output-file-type: ".html" - output-file: "{DIR[STATIC,VPP]}/k8s-memif-3n-skx-x710-64b-2t1c-base_and_scale-ndr" - data: "plot-ligato-throughput-latency-3n-skx" - filter: "'NIC_Intel-X710' and - '64B' and - ('BASE' or 'SCALE') and - 'NDRDISC' and - '2T1C' and - ('L2BDMACSTAT' or 'L2BDMACLRN' or 'L2XCFWD') and - not 'VHOST' and - 'MEMIF'" - parameters: - - "throughput" - - "parent" - traces: - hoverinfo: "x+y" - boxpoints: "outliers" - whiskerwidth: 0 - layout: - title: "" # No title - layout: "plot-throughput" - -- - type: "plot" - title: "VPP Throughput: k8s-memif-3n-skx-x710-64b-2t1c-base_and_scale-pdr" - algorithm: "plot_performance_box" - output-file-type: ".html" - output-file: "{DIR[STATIC,VPP]}/k8s-memif-3n-skx-x710-64b-2t1c-base_and_scale-pdr" - data: "plot-ligato-throughput-latency-3n-skx" - filter: "'NIC_Intel-X710' and - '64B' and - ('BASE' or 'SCALE') and - 'PDRDISC' and not 'NDRDISC' and - '2T1C' and - ('L2BDMACSTAT' or 'L2BDMACLRN' or 'L2XCFWD') and - not 'VHOST' and - 'MEMIF'" - parameters: - - "throughput" - - "parent" - traces: - hoverinfo: "x+y" - boxpoints: "outliers" - whiskerwidth: 0 - layout: - title: "" # No title - layout: "plot-throughput" - -- - type: "plot" - title: "VPP Throughput: k8s-memif-3n-skx-x710-64b-4t2c-base_and_scale-ndr" - algorithm: "plot_performance_box" - output-file-type: ".html" - output-file: "{DIR[STATIC,VPP]}/k8s-memif-3n-skx-x710-64b-4t2c-base_and_scale-ndr" - data: "plot-ligato-throughput-latency-3n-skx" - filter: "'NIC_Intel-X710' and - '64B' and - ('BASE' or 'SCALE') and - 'NDRDISC' and - '4T2C' and - ('L2BDMACSTAT' or 'L2BDMACLRN' or 'L2XCFWD') and - not 'VHOST' and - 'MEMIF'" - parameters: - - "throughput" - - "parent" - traces: - hoverinfo: "x+y" - boxpoints: "outliers" - whiskerwidth: 0 - layout: - title: "" # No title - layout: "plot-throughput" - -- - type: "plot" - title: "VPP Throughput: k8s-memif-3n-skx-x710-64b-4t2c-base_and_scale-pdr" - algorithm: "plot_performance_box" - output-file-type: ".html" - output-file: "{DIR[STATIC,VPP]}/k8s-memif-3n-skx-x710-64b-4t2c-base_and_scale-pdr" - data: "plot-ligato-throughput-latency-3n-skx" - filter: "'NIC_Intel-X710' and - '64B' and - ('BASE' or 'SCALE') and - 'PDRDISC' and not 'NDRDISC' and - '4T2C' and - ('L2BDMACSTAT' or 'L2BDMACLRN' or 'L2XCFWD') and - not 'VHOST' and - 'MEMIF'" - parameters: - - "throughput" - - "parent" - traces: - hoverinfo: "x+y" - boxpoints: "outliers" - whiskerwidth: 0 - layout: - title: "" # No title - layout: "plot-throughput" - -# Packet Throughput - VPP Container Orchestrated Memif 3n-skx-xxv710 -- - type: "plot" - title: "VPP Throughput: k8s-memif-3n-skx-xxv710-64b-2t1c-base_and_scale-ndr" - algorithm: "plot_performance_box" - output-file-type: ".html" - output-file: "{DIR[STATIC,VPP]}/k8s-memif-3n-skx-xxv710-64b-2t1c-base_and_scale-ndr" - data: "plot-ligato-throughput-latency-3n-skx" - filter: "'NIC_Intel-XXV710' and - '64B' and - ('BASE' or 'SCALE') and - 'NDRDISC' and - '2T1C' and - ('L2BDMACSTAT' or 'L2BDMACLRN' or 'L2XCFWD') and - not 'VHOST' and - 'MEMIF'" - parameters: - - "throughput" - - "parent" - traces: - hoverinfo: "x+y" - boxpoints: "outliers" - whiskerwidth: 0 - layout: - title: "" # No title - layout: "plot-throughput" - -- - type: "plot" - title: "VPP Throughput: k8s-memif-3n-skx-xxv710-64b-2t1c-base_and_scale-pdr" - algorithm: "plot_performance_box" - output-file-type: ".html" - output-file: "{DIR[STATIC,VPP]}/k8s-memif-3n-skx-xxv710-64b-2t1c-base_and_scale-pdr" - data: "plot-ligato-throughput-latency-3n-skx" - filter: "'NIC_Intel-XXV710' and - '64B' and - ('BASE' or 'SCALE') and - 'PDRDISC' and not 'NDRDISC' and - '2T1C' and - ('L2BDMACSTAT' or 'L2BDMACLRN' or 'L2XCFWD') and - not 'VHOST' and - 'MEMIF'" - parameters: - - "throughput" - - "parent" - traces: - hoverinfo: "x+y" - boxpoints: "outliers" - whiskerwidth: 0 - layout: - title: "" # No title - layout: "plot-throughput" - -- - type: "plot" - title: "VPP Throughput: k8s-memif-3n-skx-xxv710-64b-4t2c-base_and_scale-ndr" - algorithm: "plot_performance_box" - output-file-type: ".html" - output-file: "{DIR[STATIC,VPP]}/k8s-memif-3n-skx-xxv710-64b-4t2c-base_and_scale-ndr" - data: "plot-ligato-throughput-latency-3n-skx" - filter: "'NIC_Intel-XXV710' and - '64B' and - ('BASE' or 'SCALE') and - 'NDRDISC' and - '4T2C' and - ('L2BDMACSTAT' or 'L2BDMACLRN' or 'L2XCFWD') and - not 'VHOST' and - 'MEMIF'" - parameters: - - "throughput" - - "parent" - traces: - hoverinfo: "x+y" - boxpoints: "outliers" - whiskerwidth: 0 - layout: - title: "" # No title - layout: "plot-throughput" - -- - type: "plot" - title: "VPP Throughput: k8s-memif-3n-skx-xxv710-64b-4t2c-base_and_scale-pdr" - algorithm: "plot_performance_box" - output-file-type: ".html" - output-file: "{DIR[STATIC,VPP]}/k8s-memif-3n-skx-xxv710-64b-4t2c-base_and_scale-pdr" - data: "plot-ligato-throughput-latency-3n-skx" - filter: "'NIC_Intel-XXV710' and - '64B' and - ('BASE' or 'SCALE') and - 'PDRDISC' and not 'NDRDISC' and - '4T2C' and - ('L2BDMACSTAT' or 'L2BDMACLRN' or 'L2XCFWD') and - not 'VHOST' and - 'MEMIF'" - parameters: - - "throughput" - - "parent" - traces: - hoverinfo: "x+y" - boxpoints: "outliers" - whiskerwidth: 0 - layout: - title: "" # No title - layout: "plot-throughput" - -# Packet Throughput - VPP Container Orchestrated Memif 2n-skx-x710 -- - type: "plot" - title: "VPP Throughput: k8s-memif-2n-skx-x710-64b-2t1c-base_and_scale-ndr" - algorithm: "plot_performance_box" - output-file-type: ".html" - output-file: "{DIR[STATIC,VPP]}/k8s-memif-2n-skx-x710-64b-2t1c-base_and_scale-ndr" - data: "plot-ligato-throughput-latency-2n-skx" - filter: "'NIC_Intel-X710' and - '64B' and - ('BASE' or 'SCALE') and - 'NDRDISC' and - '2T1C' and - ('L2BDMACSTAT' or 'L2BDMACLRN' or 'L2XCFWD') and - not 'VHOST' and - 'MEMIF'" - parameters: - - "throughput" - - "parent" - traces: - hoverinfo: "x+y" - boxpoints: "outliers" - whiskerwidth: 0 - layout: - title: "" # No title - layout: "plot-throughput" - -- - type: "plot" - title: "VPP Throughput: k8s-memif-2n-skx-x710-64b-2t1c-base_and_scale-pdr" - algorithm: "plot_performance_box" - output-file-type: ".html" - output-file: "{DIR[STATIC,VPP]}/k8s-memif-2n-skx-x710-64b-2t1c-base_and_scale-pdr" - data: "plot-ligato-throughput-latency-2n-skx" - filter: "'NIC_Intel-X710' and - '64B' and - ('BASE' or 'SCALE') and - 'PDRDISC' and not 'NDRDISC' and - '2T1C' and - ('L2BDMACSTAT' or 'L2BDMACLRN' or 'L2XCFWD') and - not 'VHOST' and - 'MEMIF'" - parameters: - - "throughput" - - "parent" - traces: - hoverinfo: "x+y" - boxpoints: "outliers" - whiskerwidth: 0 - layout: - title: "" # No title - layout: "plot-throughput" - -- - type: "plot" - title: "VPP Throughput: k8s-memif-2n-skx-x710-64b-4t2c-base_and_scale-ndr" - algorithm: "plot_performance_box" - output-file-type: ".html" - output-file: "{DIR[STATIC,VPP]}/k8s-memif-2n-skx-x710-64b-4t2c-base_and_scale-ndr" - data: "plot-ligato-throughput-latency-2n-skx" - filter: "'NIC_Intel-X710' and - '64B' and - ('BASE' or 'SCALE') and - 'NDRDISC' and - '4T2C' and - ('L2BDMACSTAT' or 'L2BDMACLRN' or 'L2XCFWD') and - not 'VHOST' and - 'MEMIF'" - parameters: - - "throughput" - - "parent" - traces: - hoverinfo: "x+y" - boxpoints: "outliers" - whiskerwidth: 0 - layout: - title: "" # No title - layout: "plot-throughput" - -- - type: "plot" - title: "VPP Throughput: k8s-memif-2n-skx-x710-64b-4t2c-base_and_scale-pdr" - algorithm: "plot_performance_box" - output-file-type: ".html" - output-file: "{DIR[STATIC,VPP]}/k8s-memif-2n-skx-x710-64b-4t2c-base_and_scale-pdr" - data: "plot-ligato-throughput-latency-2n-skx" - filter: "'NIC_Intel-X710' and - '64B' and - ('BASE' or 'SCALE') and - 'PDRDISC' and not 'NDRDISC' and - '4T2C' and - ('L2BDMACSTAT' or 'L2BDMACLRN' or 'L2XCFWD') and - not 'VHOST' and - 'MEMIF'" - parameters: - - "throughput" - - "parent" - traces: - hoverinfo: "x+y" - boxpoints: "outliers" - whiskerwidth: 0 - layout: - title: "" # No title - layout: "plot-throughput" - -# Packet Throughput - VPP Container Orchestrated Memif 2n-skx-xxv710 -- - type: "plot" - title: "VPP Throughput: k8s-memif-2n-skx-xxv710-64b-2t1c-base_and_scale-ndr" - algorithm: "plot_performance_box" - output-file-type: ".html" - output-file: "{DIR[STATIC,VPP]}/k8s-memif-2n-skx-xxv710-64b-2t1c-base_and_scale-ndr" - data: "plot-ligato-throughput-latency-2n-skx" - filter: "'NIC_Intel-XXV710' and - '64B' and - ('BASE' or 'SCALE') and - 'NDRDISC' and - '2T1C' and - ('L2BDMACSTAT' or 'L2BDMACLRN' or 'L2XCFWD') and - not 'VHOST' and - 'MEMIF'" - parameters: - - "throughput" - - "parent" - traces: - hoverinfo: "x+y" - boxpoints: "outliers" - whiskerwidth: 0 - layout: - title: "" # No title - layout: "plot-throughput" - -- - type: "plot" - title: "VPP Throughput: k8s-memif-2n-skx-xxv710-64b-2t1c-base_and_scale-pdr" - algorithm: "plot_performance_box" - output-file-type: ".html" - output-file: "{DIR[STATIC,VPP]}/k8s-memif-2n-skx-xxv710-64b-2t1c-base_and_scale-pdr" - data: "plot-ligato-throughput-latency-2n-skx" - filter: "'NIC_Intel-XXV710' and - '64B' and - ('BASE' or 'SCALE') and - 'PDRDISC' and not 'NDRDISC' and - '2T1C' and - ('L2BDMACSTAT' or 'L2BDMACLRN' or 'L2XCFWD') and - not 'VHOST' and - 'MEMIF'" - parameters: - - "throughput" - - "parent" - traces: - hoverinfo: "x+y" - boxpoints: "outliers" - whiskerwidth: 0 - layout: - title: "" # No title - layout: "plot-throughput" - -- - type: "plot" - title: "VPP Throughput: k8s-memif-2n-skx-xxv710-64b-4t2c-base_and_scale-ndr" - algorithm: "plot_performance_box" - output-file-type: ".html" - output-file: "{DIR[STATIC,VPP]}/k8s-memif-2n-skx-xxv710-64b-4t2c-base_and_scale-ndr" - data: "plot-ligato-throughput-latency-2n-skx" - filter: "'NIC_Intel-XXV710' and - '64B' and - ('BASE' or 'SCALE') and - 'NDRDISC' and - '4T2C' and - ('L2BDMACSTAT' or 'L2BDMACLRN' or 'L2XCFWD') and - not 'VHOST' and - 'MEMIF'" - parameters: - - "throughput" - - "parent" - traces: - hoverinfo: "x+y" - boxpoints: "outliers" - whiskerwidth: 0 - layout: - title: "" # No title - layout: "plot-throughput" - -- - type: "plot" - title: "VPP Throughput: k8s-memif-2n-skx-xxv710-64b-4t2c-base_and_scale-pdr" - algorithm: "plot_performance_box" - output-file-type: ".html" - output-file: "{DIR[STATIC,VPP]}/k8s-memif-2n-skx-xxv710-64b-4t2c-base_and_scale-pdr" - data: "plot-ligato-throughput-latency-2n-skx" - filter: "'NIC_Intel-XXV710' and - '64B' and - ('BASE' or 'SCALE') and - 'PDRDISC' and not 'NDRDISC' and - '4T2C' and - ('L2BDMACSTAT' or 'L2BDMACLRN' or 'L2XCFWD') and - not 'VHOST' and - 'MEMIF'" - parameters: - - "throughput" - - "parent" - traces: - hoverinfo: "x+y" - boxpoints: "outliers" - whiskerwidth: 0 - layout: - title: "" # No title - layout: "plot-throughput" - ################################################################################ # Packet Throughput - VPP IPv4 3n-hsw-x520 @@ -11914,202 +11921,6 @@ title: "" # No title layout: "plot-latency" -# Packet Latency - VPP Container Memif 3n-skx-x710 -- - type: "plot" - title: "VPP Latency: k8s-memif-3n-skx-x710-64b-2t1c-base_and_scale-ndr-lat" - algorithm: "plot_latency_box" - output-file-type: ".html" - output-file: "{DIR[STATIC,VPP]}/k8s-memif-3n-skx-x710-64b-2t1c-base_and_scale-ndr-lat" - data: "plot-ligato-throughput-latency-3n-skx" - filter: "'NIC_Intel-X710' and - '64B' and - ('BASE' or 'SCALE') and - 'NDRDISC' and - '2T1C' and - ('L2BDMACSTAT' or 'L2BDMACLRN' or 'L2XCFWD') and - not 'VHOST' and - 'MEMIF'" - parameters: - - "latency" - - "parent" - traces: - boxmean: False - layout: - title: "" # No title - layout: "plot-latency" - -- - type: "plot" - title: "VPP Latency: k8s-memif-3n-skx-x710-64b-4t2c-base_and_scale-ndr-lat" - algorithm: "plot_latency_box" - output-file-type: ".html" - output-file: "{DIR[STATIC,VPP]}/k8s-memif-3n-skx-x710-64b-4t2c-base_and_scale-ndr-lat" - data: "plot-ligato-throughput-latency-3n-skx" - filter: "'NIC_Intel-X710' and - '64B' and - ('BASE' or 'SCALE') and - 'NDRDISC' and - '4T2C' and - ('L2BDMACSTAT' or 'L2BDMACLRN' or 'L2XCFWD') and - not 'VHOST' and - 'MEMIF'" - parameters: - - "latency" - - "parent" - traces: - boxmean: False - layout: - title: "" # No title - layout: "plot-latency" - -# Packet Latency - VPP Container Memif 3n-skx-xxv710 -- - type: "plot" - title: "VPP Latency: k8s-memif-3n-skx-xxv710-64b-2t1c-base_and_scale-ndr-lat" - algorithm: "plot_latency_box" - output-file-type: ".html" - output-file: "{DIR[STATIC,VPP]}/k8s-memif-3n-skx-xxv710-64b-2t1c-base_and_scale-ndr-lat" - data: "plot-ligato-throughput-latency-3n-skx" - filter: "'NIC_Intel-XXV710' and - '64B' and - ('BASE' or 'SCALE') and - 'NDRDISC' and - '2T1C' and - ('L2BDMACSTAT' or 'L2BDMACLRN' or 'L2XCFWD') and - not 'VHOST' and - 'MEMIF'" - parameters: - - "latency" - - "parent" - traces: - boxmean: False - layout: - title: "" # No title - layout: "plot-latency" - -- - type: "plot" - title: "VPP Latency: k8s-memif-3n-skx-xxv710-64b-4t2c-base_and_scale-ndr-lat" - algorithm: "plot_latency_box" - output-file-type: ".html" - output-file: "{DIR[STATIC,VPP]}/k8s-memif-3n-skx-xxv710-64b-4t2c-base_and_scale-ndr-lat" - data: "plot-ligato-throughput-latency-3n-skx" - filter: "'NIC_Intel-XXV710' and - '64B' and - ('BASE' or 'SCALE') and - 'NDRDISC' and - '4T2C' and - ('L2BDMACSTAT' or 'L2BDMACLRN' or 'L2XCFWD') and - not 'VHOST' and - 'MEMIF'" - parameters: - - "latency" - - "parent" - traces: - boxmean: False - layout: - title: "" # No title - layout: "plot-latency" - -# Packet Latency - VPP Container Memif 2n-skx-x710 -- - type: "plot" - title: "VPP Latency: k8s-memif-2n-skx-x710-64b-2t1c-base_and_scale-ndr-lat" - algorithm: "plot_latency_box" - output-file-type: ".html" - output-file: "{DIR[STATIC,VPP]}/k8s-memif-2n-skx-x710-64b-2t1c-base_and_scale-ndr-lat" - data: "plot-ligato-throughput-latency-2n-skx" - filter: "'NIC_Intel-X710' and - '64B' and - ('BASE' or 'SCALE') and - 'NDRDISC' and - '2T1C' and - ('L2BDMACSTAT' or 'L2BDMACLRN' or 'L2XCFWD') and - not 'VHOST' and - 'MEMIF'" - parameters: - - "latency" - - "parent" - traces: - boxmean: False - layout: - title: "" # No title - layout: "plot-latency" - -- - type: "plot" - title: "VPP Latency: k8s-memif-2n-skx-x710-64b-4t2c-base_and_scale-ndr-lat" - algorithm: "plot_latency_box" - output-file-type: ".html" - output-file: "{DIR[STATIC,VPP]}/k8s-memif-2n-skx-x710-64b-4t2c-base_and_scale-ndr-lat" - data: "plot-ligato-throughput-latency-2n-skx" - filter: "'NIC_Intel-X710' and - '64B' and - ('BASE' or 'SCALE') and - 'NDRDISC' and - '4T2C' and - ('L2BDMACSTAT' or 'L2BDMACLRN' or 'L2XCFWD') and - not 'VHOST' and - 'MEMIF'" - parameters: - - "latency" - - "parent" - traces: - boxmean: False - layout: - title: "" # No title - layout: "plot-latency" - -# Packet Latency - VPP Container Memif 2n-skx-xxv710 -- - type: "plot" - title: "VPP Latency: k8s-memif-2n-skx-xxv710-64b-2t1c-base_and_scale-ndr-lat" - algorithm: "plot_latency_box" - output-file-type: ".html" - output-file: "{DIR[STATIC,VPP]}/k8s-memif-2n-skx-xxv710-64b-2t1c-base_and_scale-ndr-lat" - data: "plot-ligato-throughput-latency-2n-skx" - filter: "'NIC_Intel-XXV710' and - '64B' and - ('BASE' or 'SCALE') and - 'NDRDISC' and - '2T1C' and - ('L2BDMACSTAT' or 'L2BDMACLRN' or 'L2XCFWD') and - not 'VHOST' and - 'MEMIF'" - parameters: - - "latency" - - "parent" - traces: - boxmean: False - layout: - title: "" # No title - layout: "plot-latency" - -- - type: "plot" - title: "VPP Latency: k8s-memif-2n-skx-xxv710-64b-4t2c-base_and_scale-ndr-lat" - algorithm: "plot_latency_box" - output-file-type: ".html" - output-file: "{DIR[STATIC,VPP]}/k8s-memif-2n-skx-xxv710-64b-4t2c-base_and_scale-ndr-lat" - data: "plot-ligato-throughput-latency-2n-skx" - filter: "'NIC_Intel-XXV710' and - '64B' and - ('BASE' or 'SCALE') and - 'NDRDISC' and - '4T2C' and - ('L2BDMACSTAT' or 'L2BDMACLRN' or 'L2XCFWD') and - not 'VHOST' and - 'MEMIF'" - parameters: - - "latency" - - "parent" - traces: - boxmean: False - layout: - title: "" # No title - layout: "plot-latency" - ################################################################################ # Packet Latency - VPP IPv4 3n-hsw-x520 @@ -15351,186 +15162,6 @@ title: "" # No title layout: "plot-throughput-speedup-analysis" -# Speedup - VPP Container Memif 3n-skx-x710 -- - type: "plot" - title: "Speedup: k8s-memif-3n-skx-x710-64b-base_and_scale-ndr-tsa" - algorithm: "plot_throughput_speedup_analysis" - output-file-type: ".html" - output-file: "{DIR[STATIC,VPP]}/k8s-memif-3n-skx-x710-64b-base_and_scale-ndr-tsa" - data: "plot-ligato-throughput-speedup-analysis-3n-skx" - filter: "'NIC_Intel-X710' and - '64B' and - ('BASE' or 'SCALE') and - 'NDRDISC' and - ('L2BDMACSTAT' or 'L2BDMACLRN' or 'L2XCFWD') and - not 'VHOST' and - 'MEMIF'" - parameters: - - "throughput" - - "parent" - - "tags" - layout: - title: "" # No title - layout: "plot-throughput-speedup-analysis" - -- - type: "plot" - title: "Speedup: k8s-memif-3n-skx-x710-64b-base_and_scale-pdr-tsa" - algorithm: "plot_throughput_speedup_analysis" - output-file-type: ".html" - output-file: "{DIR[STATIC,VPP]}/k8s-memif-3n-skx-x710-64b-base_and_scale-pdr-tsa" - data: "plot-ligato-throughput-speedup-analysis-3n-skx" - filter: "'NIC_Intel-X710' and - '64B' and - ('BASE' or 'SCALE') and - 'PDRDISC' and not 'NDRDISC' and - ('L2BDMACSTAT' or 'L2BDMACLRN' or 'L2XCFWD') and - not 'VHOST' and - 'MEMIF'" - parameters: - - "throughput" - - "parent" - - "tags" - layout: - title: "" # No title - layout: "plot-throughput-speedup-analysis" - -# Speedup - VPP Container Memif 3n-skx-xxv710 -- - type: "plot" - title: "Speedup: k8s-memif-3n-skx-xxv710-64b-base_and_scale-ndr-tsa" - algorithm: "plot_throughput_speedup_analysis" - output-file-type: ".html" - output-file: "{DIR[STATIC,VPP]}/k8s-memif-3n-skx-xxv710-64b-base_and_scale-ndr-tsa" - data: "plot-ligato-throughput-speedup-analysis-3n-skx" - filter: "'NIC_Intel-XXV710' and - '64B' and - ('BASE' or 'SCALE') and - 'NDRDISC' and - ('L2BDMACSTAT' or 'L2BDMACLRN' or 'L2XCFWD') and - not 'VHOST' and - 'MEMIF'" - parameters: - - "throughput" - - "parent" - - "tags" - layout: - title: "" # No title - layout: "plot-throughput-speedup-analysis" - -- - type: "plot" - title: "Speedup: k8s-memif-3n-skx-xxv710-64b-base_and_scale-pdr-tsa" - algorithm: "plot_throughput_speedup_analysis" - output-file-type: ".html" - output-file: "{DIR[STATIC,VPP]}/k8s-memif-3n-skx-xxv710-64b-base_and_scale-pdr-tsa" - data: "plot-ligato-throughput-speedup-analysis-3n-skx" - filter: "'NIC_Intel-XXV710' and - '64B' and - ('BASE' or 'SCALE') and - 'PDRDISC' and not 'NDRDISC' and - ('L2BDMACSTAT' or 'L2BDMACLRN' or 'L2XCFWD') and - not 'VHOST' and - 'MEMIF'" - parameters: - - "throughput" - - "parent" - - "tags" - layout: - title: "" # No title - layout: "plot-throughput-speedup-analysis" - -# Speedup - VPP Container Memif 2n-skx-x710 -- - type: "plot" - title: "Speedup: k8s-memif-2n-skx-x710-64b-base_and_scale-ndr-tsa" - algorithm: "plot_throughput_speedup_analysis" - output-file-type: ".html" - output-file: "{DIR[STATIC,VPP]}/k8s-memif-2n-skx-x710-64b-base_and_scale-ndr-tsa" - data: "plot-ligato-throughput-speedup-analysis-2n-skx" - filter: "'NIC_Intel-X710' and - '64B' and - ('BASE' or 'SCALE') and - 'NDRDISC' and - ('L2BDMACSTAT' or 'L2BDMACLRN' or 'L2XCFWD') and - not 'VHOST' and - 'MEMIF'" - parameters: - - "throughput" - - "parent" - - "tags" - layout: - title: "" # No title - layout: "plot-throughput-speedup-analysis" - -- - type: "plot" - title: "Speedup: k8s-memif-2n-skx-x710-64b-base_and_scale-pdr-tsa" - algorithm: "plot_throughput_speedup_analysis" - output-file-type: ".html" - output-file: "{DIR[STATIC,VPP]}/k8s-memif-2n-skx-x710-64b-base_and_scale-pdr-tsa" - data: "plot-ligato-throughput-speedup-analysis-2n-skx" - filter: "'NIC_Intel-X710' and - '64B' and - ('BASE' or 'SCALE') and - 'PDRDISC' and not 'NDRDISC' and - ('L2BDMACSTAT' or 'L2BDMACLRN' or 'L2XCFWD') and - not 'VHOST' and - 'MEMIF'" - parameters: - - "throughput" - - "parent" - - "tags" - layout: - title: "" # No title - layout: "plot-throughput-speedup-analysis" - -# Speedup - VPP Container Memif 2n-skx-xxv710 -- - type: "plot" - title: "Speedup: k8s-memif-2n-skx-xxv710-64b-base_and_scale-ndr-tsa" - algorithm: "plot_throughput_speedup_analysis" - output-file-type: ".html" - output-file: "{DIR[STATIC,VPP]}/k8s-memif-2n-skx-xxv710-64b-base_and_scale-ndr-tsa" - data: "plot-ligato-throughput-speedup-analysis-2n-skx" - filter: "'NIC_Intel-XXV710' and - '64B' and - ('BASE' or 'SCALE') and - 'NDRDISC' and - ('L2BDMACSTAT' or 'L2BDMACLRN' or 'L2XCFWD') and - not 'VHOST' and - 'MEMIF'" - parameters: - - "throughput" - - "parent" - - "tags" - layout: - title: "" # No title - layout: "plot-throughput-speedup-analysis" - -- - type: "plot" - title: "Speedup: k8s-memif-2n-skx-xxv710-64b-base_and_scale-pdr-tsa" - algorithm: "plot_throughput_speedup_analysis" - output-file-type: ".html" - output-file: "{DIR[STATIC,VPP]}/k8s-memif-2n-skx-xxv710-64b-base_and_scale-pdr-tsa" - data: "plot-ligato-throughput-speedup-analysis-2n-skx" - filter: "'NIC_Intel-XXV710' and - '64B' and - ('BASE' or 'SCALE') and - 'PDRDISC' and not 'NDRDISC' and - ('L2BDMACSTAT' or 'L2BDMACLRN' or 'L2XCFWD') and - not 'VHOST' and - 'MEMIF'" - parameters: - - "throughput" - - "parent" - - "tags" - layout: - title: "" # No title - layout: "plot-throughput-speedup-analysis" - ################################################################################ # Speedup - VPP IPv4 3n-hsw-x520 |