diff options
6 files changed, 124 insertions, 104 deletions
diff --git a/docs/report/dpdk_performance_tests/packet_latency_graphs/index.rst b/docs/report/dpdk_performance_tests/packet_latency_graphs/index.rst index 52acbab5ad..7f2f237cd5 100644 --- a/docs/report/dpdk_performance_tests/packet_latency_graphs/index.rst +++ b/docs/report/dpdk_performance_tests/packet_latency_graphs/index.rst @@ -6,31 +6,37 @@ Packet Latency ============== -Plotted results are generated from a single execution of CSIT NDR discovery -test. Box plots are used to show the Minimum, Median and Maximum packet +Latency results are generated from a single execution of NDR discovery +test across physical testbeds hosted LF FD.io labs: 3n-hsw, 2n-skx, 2n- +skx. Box plots are used to show the Minimum, Median and Maximum packet latency per test. -*Title of each graph* describes the packet path, testbed topology, processor -architecture, packet size, number of cores and threads used by workers and -DUT configuration, *X-axis labels* are indices of individual test suites -executed by FD.io test executor dpdk performance jobs that created result output -file used as data source for the graph, *Y-axis labels* are measured packet -Latency [uSec] values, and the *Graph legend* lists the plotted test suites and -their indices. Latency is reported for concurrent symmetric bi-directional -flows, separately for each direction: i) West-to-East: -TGint1-to-SUT1-to-SUT2-to-TGint2, and ii) East-to-West: -TGint2-to-SUT2-to-SUT1-to-TGint1. +Additional information about graph data: + +#. **Graph Title**: describes tested packet path, testbed topology, + processor model, NIC model, packet size, number of cores and + threads used by data plane workers and indication of DUT + configuration. + +#. **X-axis Labels**: indices of individual test suites as listed in + Graph Legend and direction of latency flow: + + - West-to-East: TGint1-to-SUT1-to-SUT2-to-TGint2. + - East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1. + +#. **Y-axis Labels**: measured packet latency values in [uSec]. + +#. **Graph Legend**: lists X-axis indices with associated CSIT test + suites executed to generate graphed test results. .. note:: Test results have been generated by `FD.io test executor dpdk performance job 3n-hsw`_, `FD.io test executor dpdk performance job 3n-skx`_ and - `FD.io test executor dpdk performance job 2n-skx`_ with Robot Framework + `FD.io test executor dpdk performance job 2n-skx`_ with RF result files csit-dpdk-perf-|srelease|-\*.zip `archived here <../../_static/archive/>`_. - Plotted data set size per test case is equal to the number of job executions - presented in this report version: **10**. .. toctree:: :maxdepth: 1 diff --git a/docs/report/dpdk_performance_tests/packet_throughput_graphs/index.rst b/docs/report/dpdk_performance_tests/packet_throughput_graphs/index.rst index dbfe204611..37d6d6fc0c 100644 --- a/docs/report/dpdk_performance_tests/packet_throughput_graphs/index.rst +++ b/docs/report/dpdk_performance_tests/packet_throughput_graphs/index.rst @@ -6,40 +6,44 @@ Packet Throughput ================= -Plotted results are generated by multiple executions of the same CSIT -performance tests across three physical testbeds within LF FD.io labs. -To provide a descriptive summary view, Box-and-Whisker plots are used to -display variation in measured throughput values, without making any -assumptions of the underlying statistical distribution. - -For each plotted test case, Box-and-Whisker plots show the quartiles -(Min, 1st quartile / 25th percentile, 2nd quartile / 50th percentile / -mean, 3rd quartile / 75th percentile, Max) across collected data set -(data set size stated in the note below). Outliers are plotted as -individual points. Min and max values are plotted as bottom and top -Whiskers respectively. 2nd and 3rd quartiles are plotted as bottom and -top edge of the box. If multiple samples match only two values, and all -samples fall between them, then no whiskers are plotted. If all samples -have the same value, only a horizontal line is plotted. - -*Title of each graph* describes the packet path, testbed topology, processor -architecture, packet size, number of cores and threads used by workers and -DUT configuration, *X-axis labels* are indices by individual test suites -executed by FD.io test executor dpdk performance jobs that created result output -files used as data sources for the graph, *Y-axis labels* are measured Packets -Per Second [pps] values, and the *Graph legend* lists the plotted test suites -and their indices. +Throughput graphs are generated by multiple executions of the same +performance tests across physical testbeds hosted LF FD.io labs: 3n-hsw, +2n-skx, 2n-skx. Box-and-Whisker plots are used to display variations in +measured throughput values, without making any assumptions of the +underlying statistical distribution. + +For each test case, Box-and-Whisker plots show the quartiles (Min, 1st +quartile / 25th percentile, 2nd quartile / 50th percentile / mean, 3rd +quartile / 75th percentile, Max) across collected data set. Outliers are +plotted as individual points. + +Additional information about graph data: + +#. **Graph Title**: describes tested packet path, testbed topology, + processor model, NIC model, packet size, number of cores and + threads used by data plane workers and indication of DPDK DUT + configuration. + +#. **X-axis Labels**: indices of individual test suites as listed in + Graph Legend. + +#. **Y-axis Labels**: measured Packets Per Second [pps] throughput + values. + +#. **Graph Legend**: lists X-axis indices with associated CSIT test + suites executed to generate graphed test results. .. note:: Test results have been generated by `FD.io test executor dpdk performance job 3n-hsw`_, `FD.io test executor dpdk performance job 3n-skx`_ and - `FD.io test executor dpdk performance job 2n-skx`_ with Robot Framework + `FD.io test executor dpdk performance job 2n-skx`_ with RF result files csit-dpdk-perf-|srelease|-\*.zip `archived here <../../_static/archive/>`_. - Plotted data set size per test case is equal to the number of job executions - presented in this report version: **10**. + Required per test case data set size is **10** and for DPDK tests + this is the actual size, as all scheduled test executions completed + successfully. .. toctree:: :maxdepth: 1 diff --git a/docs/report/vpp_performance_tests/http_server_performance/index.rst b/docs/report/vpp_performance_tests/http_server_performance/index.rst index 6a268628e3..2e2226f2f5 100644 --- a/docs/report/vpp_performance_tests/http_server_performance/index.rst +++ b/docs/report/vpp_performance_tests/http_server_performance/index.rst @@ -6,21 +6,16 @@ HTTP/TCP Performance ==================== -Plotted results are generated by multiple executions of the same CSIT -performance tests across three physical testbeds within LF FD.io labs. -To provide a descriptive summary view, Box-and-Whisker plots are used to -display variation in measured performance values, without making any -assumptions of the underlying statistical distribution. - -For each plotted test case, Box-and-Whisker plots show the quartiles -(Min, 1st quartile / 25th percentile, 2nd quartile / 50th percentile / -mean, 3rd quartile / 75th percentile, Max) across collected data set -(data set size stated in the note below). Outliers are plotted as -individual points. Min and max values are plotted as bottom and top -Whiskers respectively. 2nd and 3rd quartiles are plotted as bottom and -top edge of the box. If multiple samples match only two values, and all -samples fall between them, then no whiskers are plotted. If all samples -have the same value, only a horizontal line is plotted. +Performance graphs are generated by multiple executions of the same +performance tests across physical testbeds hosted LF FD.io labs: 3n-hsw. +Box-and-Whisker plots are used to display variations in measured +throughput values, without making any assumptions of the underlying +statistical distribution. + +For each test case, Box-and-Whisker plots show the quartiles (Min, 1st +quartile / 25th percentile, 2nd quartile / 50th percentile / mean, 3rd +quartile / 75th percentile, Max) across collected data set. Outliers are +plotted as individual points. .. note:: diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/index.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/index.rst index 5825df326c..db1012b98e 100644 --- a/docs/report/vpp_performance_tests/packet_latency_graphs/index.rst +++ b/docs/report/vpp_performance_tests/packet_latency_graphs/index.rst @@ -6,31 +6,37 @@ Packet Latency ============== -Plotted results are generated from a single execution of CSIT NDR discovery -test. Box plots are used to show the Minimum, Median and Maximum packet +Latency results are generated from a single execution of NDR discovery +test across physical testbeds hosted LF FD.io labs: 3n-hsw, 2n-skx, 2n- +skx. Box plots are used to show the Minimum, Median and Maximum packet latency per test. -*Title of each graph* describes the packet path, testbed topology, processor -architecture, packet size, number of cores and threads used by workers and -DUT configuration, *X-axis labels* are indices of individual test suites -executed by FD.io test executor vpp performance jobs that created result output -file used as data source for the graph, *Y-axis labels* are measured packet -Latency [uSec] values, and the *Graph legend* lists the plotted test suites and -their indices. Latency is reported for concurrent symmetric bi-directional -flows, separately for each direction: i) West-to-East: -TGint1-to-SUT1-to-SUT2-to-TGint2, and ii) East-to-West: -TGint2-to-SUT2-to-SUT1-to-TGint1. +Additional information about graph data: + +#. **Graph Title**: describes tested packet path, testbed topology, + processor model, NIC model, packet size, number of cores and + threads used by data plane workers and indication of DUT + configuration. + +#. **X-axis Labels**: indices of individual test suites as listed in + Graph Legend and direction of latency flow: + + - West-to-East: TGint1-to-SUT1-to-SUT2-to-TGint2. + - East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1. + +#. **Y-axis Labels**: measured packet latency values in [uSec]. + +#. **Graph Legend**: lists X-axis indices with associated CSIT test + suites executed to generate graphed test results. .. note:: Test results have been generated by `FD.io test executor vpp performance job 3n-hsw`_, `FD.io test executor vpp performance job 3n-skx`_ and - `FD.io test executor vpp performance job 2n-skx`_ with Robot Framework + `FD.io test executor vpp performance job 2n-skx`_ with RF result files csit-vpp-perf-|srelease|-\*.zip `archived here <../../_static/archive/>`_. - Plotted data set size per test case is equal to the number of job executions - presented in this report version: **10**. .. toctree:: :maxdepth: 1 diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/index.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/index.rst index 8d6ff86504..7be6b0132d 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/index.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/index.rst @@ -6,29 +6,32 @@ Packet Throughput ================= -Plotted results are generated by multiple executions of the same CSIT -performance tests across three physical testbeds within LF FD.io labs. -To provide a descriptive summary view, Box-and-Whisker plots are used to -display variation in measured throughput values, without making any -assumptions of the underlying statistical distribution. - -For each plotted test case, Box-and-Whisker plots show the quartiles -(Min, 1st quartile / 25th percentile, 2nd quartile / 50th percentile / -mean, 3rd quartile / 75th percentile, Max) across collected data set -(data set size stated in the note below). Outliers are plotted as -individual points. Min and max values are plotted as bottom and top -Whiskers respectively. 2nd and 3rd quartiles are plotted as bottom and -top edge of the box. If multiple samples match only two values, and all -samples fall between them, then no whiskers are plotted. If all samples -have the same value, only a horizontal line is plotted. - -*Title of each graph* describes the packet path, testbed topology, processor -architecture, packet size, number of cores and threads used by workers and -DUT configuration, *X-axis labels* are indices of individual test suites -executed by FD.io test executor vpp performance jobs that created result output -files used as data sources for the graph, *Y-axis labels* are measured Packets -Per Second [pps] values, and the *Graph legend* lists the plotted test suites -and their indices. +Throughput graphs are generated by multiple executions of the same +performance tests across physical testbeds hosted LF FD.io labs: 3n-hsw, +2n-skx, 2n-skx. Box-and-Whisker plots are used to display variations in +measured throughput values, without making any assumptions of the +underlying statistical distribution. + +For each test case, Box-and-Whisker plots show the quartiles (Min, 1st +quartile / 25th percentile, 2nd quartile / 50th percentile / mean, 3rd +quartile / 75th percentile, Max) across collected data set. Outliers are +plotted as individual points. + +Additional information about graph data: + +#. **Graph Title**: describes tested packet path, testbed topology, + processor model, NIC model, packet size, number of cores and + threads used by data plane workers and indication of VPP DUT + configuration. + +#. **X-axis Labels**: indices of individual test suites as listed in + Graph Legend. + +#. **Y-axis Labels**: measured Packets Per Second [pps] throughput + values. + +#. **Graph Legend**: lists X-axis indices with associated CSIT test + suites executed to generate graphed test results. .. note:: @@ -36,11 +39,14 @@ and their indices. Test results have been generated by `FD.io test executor vpp performance job 3n-hsw`_, `FD.io test executor vpp performance job 3n-skx`_ and - `FD.io test executor vpp performance job 2n-skx`_ with Robot Framework + `FD.io test executor vpp performance job 2n-skx`_ with RF result files csit-vpp-perf-|srelease|-\*.zip `archived here <../../_static/archive/>`_. - Plotted data set size per test case is equal to the number of job executions - presented in this report version: **10**. + Required per test case data set size is **10**, but for VPP tests + the actual size varies per test case and is <=10. This is due to + not all test executions completing successfully due to a known + `VPP-1361 <https://jira.fd.io/browse/VPP-1361>`_ issue + sporadically impacting physical interface up/down state. .. toctree:: :maxdepth: 1 diff --git a/docs/report/vpp_performance_tests/throughput_speedup_multi_core/index.rst b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/index.rst index b3f95ec6c3..4f3acaa0c5 100644 --- a/docs/report/vpp_performance_tests/throughput_speedup_multi_core/index.rst +++ b/docs/report/vpp_performance_tests/throughput_speedup_multi_core/index.rst @@ -7,10 +7,10 @@ Speedup Multi-Core ================== -Throughput Speedup Multi-Core graphs are generated based on results from -multiple executions of the same CSIT performance tests across the three -physical testbeds within LF FD.io labs. Plotted grouped bars illustrate -the 64B/78B packet throughput speedup ratio for 2- and 4-core multi- +Speedup Multi-Core throughput graphs are generated by multiple +executions of the same performance tests across physical testbeds hosted +LF FD.io labs: 3n-hsw, 2n-skx, 2n-skx. Grouped bars illustrate the +64B/78B packet throughput speedup ratio for 2- and 4-core multi- threaded VPP configurations relative to 1-core configurations. .. note:: @@ -18,11 +18,14 @@ threaded VPP configurations relative to 1-core configurations. Test results have been generated by `FD.io test executor vpp performance job 3n-hsw`_, `FD.io test executor vpp performance job 3n-skx`_ and - `FD.io test executor vpp performance job 2n-skx`_ with Robot Framework + `FD.io test executor vpp performance job 2n-skx`_ with RF result files csit-vpp-perf-|srelease|-\*.zip `archived here <../../_static/archive/>`_. - Plotted data set size per test case is equal to the number of job executions - presented in this report version: **10**. + Required per test case data set size is **10**, but for VPP tests + the actual size varies per test case and is <=10. This is due to + not all test executions completing successfully due to a known + `VPP-1361 <https://jira.fd.io/browse/VPP-1361>`_ issue + sporadically impacting physical interface up/down state. .. toctree:: :maxdepth: 1 |