diff options
Diffstat (limited to 'docs')
21 files changed, 134 insertions, 395 deletions
diff --git a/docs/report/introduction/overview.rst b/docs/report/introduction/overview.rst index d781993198..97b6abc5bb 100644 --- a/docs/report/introduction/overview.rst +++ b/docs/report/introduction/overview.rst @@ -65,3 +65,11 @@ CSIT |release| report contains following main sections and sub-sections: executions using CSIT Robot Framework output files as source data; *VPP
Performance Results*, *Testpmd Performance Results*, *VPP Functional
Results*, *HoneyComb Functional Results*, *VPPtest Functional Results*.
+
+#. **Test Configuration** - auto-generated DUT configuration data from CSIT jobs
+ executions using CSIT Robot Framework output files as source data; *VPP
+ Performance Test Configs*, *VPP Functional Test Configs*.
+
+#. **Test Operational Data** - auto-generated DUT operational data from CSIT jobs
+ executions using CSIT Robot Framework output files as source data; *VPP
+ Performance Operational Data*.
diff --git a/docs/report/testpmd_performance_tests/packet_latency_graphs/index.rst b/docs/report/testpmd_performance_tests/packet_latency_graphs/index.rst index 4402794a1b..fb16912039 100644 --- a/docs/report/testpmd_performance_tests/packet_latency_graphs/index.rst +++ b/docs/report/testpmd_performance_tests/packet_latency_graphs/index.rst @@ -1,6 +1,28 @@ Packet Latency Graphs ===================== +Plotted results are generated from a single execution of CSIT NDR discovery +test. Box plots are used to show the Minimum, Median and Maximum packet +latency per test. + +*Title of each graph* is a regex (regular expression) matching all +throughput test cases plotted on this graph, *X-axis labels* are indices +of individual test suites executed by csit-dpdk-perf-1704-all job that +created result output file used as data source for the graph, *Y-axis +labels* are measured packet Latency [uSec] values, and the *Graph +legend* lists the plotted test suites and their indices. Latency is +reported for concurrent symmetric bi-directional flows, separately for +each direction: i) West-to-East: TGint1-to-SUT1-to-SUT2-to-TGint2, and +ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1. + +.. note:: + + Test results have been generated by FD.io test executor jobs + `csit-dpdk-perf-1704-all + <https://jenkins.fd.io/view/csit/job/csit-dpdk-perf-1704-all/>`_, + with Robot Framework result files csit-dpdk-perf-1704-all-1.zip + `archived here <../../_static/archive/>`_. + .. toctree:: l2 diff --git a/docs/report/testpmd_performance_tests/packet_latency_graphs/l2.rst b/docs/report/testpmd_performance_tests/packet_latency_graphs/l2.rst index 95b82d784d..51ffea21b3 100644 --- a/docs/report/testpmd_performance_tests/packet_latency_graphs/l2.rst +++ b/docs/report/testpmd_performance_tests/packet_latency_graphs/l2.rst @@ -7,28 +7,6 @@ NDR throughput rate. Latency is reported for Testpmd running in multiple configurations of Testpmd pmd thread(s), a.k.a. Testpmd data plane thread(s), and their physical CPU core(s) placement. -Results are generated from a single execution of CSIT NDR discovery -test. Box plots are used to show the Minimum, Average and Maximum packet -latency per test. - -*Title of each graph* is a regex (regular expression) matching all -throughput test cases plotted on this graph, *X-axis labels* are indices -of individual test suites executed by csit-dpdk-perf-1704-all job that -created result output file used as data source for the graph, *Y-axis -labels* are measured packet Latency [uSec] values, and the *Graph -legend* lists the plotted test suites and their indices. Latency is -reported for concurrent symmetric bi-directional flows, separately for -each direction: i) West-to-East: TGint1-to-SUT1-to-SUT2-to-TGint2, and -ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1. - -.. note:: - - Test results have been generated by FD.io test executor jobs - `csit-dpdk-perf-1704-all - <https://jenkins.fd.io/view/csit/job/csit-dpdk-perf-1704-all/>`_, - with Robot Framework result files csit-dpdk-perf-1704-all-<id>.zip - `archived here <../../_static/archive/>`_. - Testpmd packet latency - running in configuration of **one worker thread (1t) on one physical core (1c)** - is presented in the figure below. diff --git a/docs/report/testpmd_performance_tests/packet_throughput_graphs/index.rst b/docs/report/testpmd_performance_tests/packet_throughput_graphs/index.rst index 4e4e87a1ea..d607b6ead6 100644 --- a/docs/report/testpmd_performance_tests/packet_throughput_graphs/index.rst +++ b/docs/report/testpmd_performance_tests/packet_throughput_graphs/index.rst @@ -1,6 +1,39 @@ Packet Throughput Graphs ======================== +Plotted results are generated by multiple executions of the same CSIT +performance tests across three physical testbeds within LF FD.io labs. +To provide a descriptive summary view, Box-and-Whisker plots are used to +display variation in measured throughput values, without making any +assumptions of the underlying statistical distribution. + +For each plotted test case, Box-and-Whisker plots show the quartiles +(Min, 1st quartile / 25th percentile, 2nd quartile / 50th percentile / +mean, 3rd quartile / 75th percentile, Max) across collected data set +(data set size stated in the note below). Outliers are plotted as +individual points. Min and max values are plotted as bottom and top +Whiskers respectively. 2nd and 3rd quartiles are plotted as bottom and +top edge of the box. If multiple samples match only two values, and all +samples fall between them, then no whiskers are plotted. If all samples +have the same value, only a horizontal line is plotted. + +*Title of each graph* is a regex (regular expression) matching all +throughput test cases plotted on this graph, *X-axis labels* are indices +of individual test suites executed by csit-dpdk-perf-1704-all jobs that +created result output files used as data sources for the graph, *Y-axis +labels* are measured Packets Per Second [pps] values, and the *Graph +legend* lists the plotted test suites and their indices. + +.. note:: + + Test results have been generated by FD.io test executor jobs + `csit-dpdk-perf-1704-all + <https://jenkins.fd.io/view/csit/job/csit-dpdk-perf-1704-all/>`_, + with Robot Framework result files csit-dpdk-perf-1704-all-<id>.zip + `archived here <../../_static/archive/>`_. Plotted data set size per + test case is equal to the number of job executions presented in this + report version: **10**. + .. toctree:: l2 diff --git a/docs/report/testpmd_performance_tests/packet_throughput_graphs/l2.rst b/docs/report/testpmd_performance_tests/packet_throughput_graphs/l2.rst index 8b13dc9e50..40f320ec04 100644 --- a/docs/report/testpmd_performance_tests/packet_throughput_graphs/l2.rst +++ b/docs/report/testpmd_performance_tests/packet_throughput_graphs/l2.rst @@ -7,33 +7,6 @@ and PDR throughput (<0.5% packet loss). Performance is reported for Testpmd running in multiple configurations of Testpmd pmd thread(s), a.k.a. Testpmd data plane thread(s), and their physical CPU core(s) placement. -Results are generated by multiple executions of the same CSIT -performance tests across three physical testbeds within LF FD.io labs. -To provide a descriptive summary view, Box-and-Whisker plots are used to -display variation in measured throughput values, without making any -assumptions of the underlying statistical distribution. For each plotted -test case, Box-and-Whisker plots show the quartiles (Min, 1st quartile -/ 25th percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / -75th percentile, Max) across collected data set (data set size stated in -the note below). Outliers are plotted as individual points. - -*Title of each graph* is a regex (regular expression) matching all -throughput test cases plotted on this graph, *X-axis labels* are indices -of individual test suites executed by csit-dpdk-perf-1704-all jobs that -created result output files used as data sources for the graph, *Y-axis -labels* are measured Packets Per Second [pps] values, and the *Graph -legend* lists the plotted test suites and their indices. - -.. note:: - - Test results have been generated by FD.io test executor jobs - `csit-dpdk-perf-1704-all - <https://jenkins.fd.io/view/csit/job/csit-dpdk-perf-1704-all/>`_, - with Robot Framework result files csit-dpdk-perf-1704-all-<id>.zip - `archived here <../../_static/archive/>`_. Plotted data set size per - test case is equal to the number of job executions presented in this - report version: **4**. - NDR Throughput ~~~~~~~~~~~~~~ diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/index.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/index.rst index f333f230f7..f71709453f 100644 --- a/docs/report/vpp_performance_tests/packet_latency_graphs/index.rst +++ b/docs/report/vpp_performance_tests/packet_latency_graphs/index.rst @@ -1,6 +1,28 @@ Packet Latency Graphs ===================== +Plotted results are generated from a single execution of CSIT NDR discovery +test. Box plots are used to show the Minimum, Median and Maximum packet +latency per test. + +*Title of each graph* is a regex (regular expression) matching all +throughput test cases plotted on this graph, *X-axis labels* are indices +of individual test suites executed by csit-vpp-perf-1704-all job that +created result output file used as data source for the graph, *Y-axis +labels* are measured packet Latency [uSec] values, and the *Graph +legend* lists the plotted test suites and their indices. Latency is +reported for concurrent symmetric bi-directional flows, separately for +each direction: i) West-to-East: TGint1-to-SUT1-to-SUT2-to-TGint2, and +ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1. + +.. note:: + + Test results have been generated by FD.io test executor jobs + `csit-vpp-perf-1704-all + <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_, + with Robot Framework result files csit-vpp-perf-1704-all-6.zip + `archived here <../../_static/archive/>`_. + .. toctree:: l2 diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/ipsec.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/ipsec.rst index 0c506f720c..db82797dca 100644 --- a/docs/report/vpp_performance_tests/packet_latency_graphs/ipsec.rst +++ b/docs/report/vpp_performance_tests/packet_latency_graphs/ipsec.rst @@ -10,20 +10,6 @@ for VPP running in multiple configurations of VPP worker thread(s), a.k.a. VPP data plane thread(s), and their physical CPU core(s) placement. -Results are generated from a single execution of CSIT NDR discovery -test. Box plots are used to show the Minimum, Average and Maximum packet -latency per test. - -*Title of each graph* is a regex (regular expression) matching all -throughput test cases plotted on this graph, *X-axis labels* are indices -of individual test suites executed by csit-vpp-perf-1704-all job that -created result output file used as data source for the graph, *Y-axis -labels* are measured packet Latency [uSec] values, and the *Graph -legend* lists the plotted test suites and their indices. Latency is -reported for concurrent symmetric bi-directional flows, separately for -each direction: i) West-to-East: TGint1-to-SUT1-to-SUT2-to-TGint2, and -ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1. - .. note:: Test results have been generated by FD.io test executor jobs diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4.rst index 5ddf813cea..10b22b2cc6 100644 --- a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4.rst +++ b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4.rst @@ -7,28 +7,6 @@ rate. Latency is reported for VPP running in multiple configurations of VPP worker thread(s), a.k.a. VPP data plane thread(s), and their physical CPU core(s) placement. -Results are generated from a single execution of CSIT NDR discovery -test. Box plots are used to show the Minimum, Average and Maximum packet -latency per test. - -*Title of each graph* is a regex (regular expression) matching all -throughput test cases plotted on this graph, *X-axis labels* are indices -of individual test suites executed by csit-vpp-perf-1704-all job that -created result output file used as data source for the graph, *Y-axis -labels* are measured packet Latency [uSec] values, and the *Graph -legend* lists the plotted test suites and their indices. Latency is -reported for concurrent symmetric bi-directional flows, separately for -each direction: i) West-to-East: TGint1-to-SUT1-to-SUT2-to-TGint2, and -ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1. - -.. note:: - - Test results have been generated by FD.io test executor jobs - `csit-vpp-perf-1704-all - <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_, - with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip - `archived here <../../_static/archive/>`_. - VPP packet latency in 1t1c setup (1thread, 1core) is presented in the graph below. .. raw:: html diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4_tunnels.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4_tunnels.rst index ab39a97704..b14b0191de 100644 --- a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4_tunnels.rst +++ b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4_tunnels.rst @@ -7,28 +7,6 @@ rate. Latency is reported for VPP running in multiple configurations of VPP worker thread(s), a.k.a. VPP data plane thread(s), and their physical CPU core(s) placement. -Results are generated from a single execution of CSIT NDR discovery -test. Box plots are used to show the Minimum, Average and Maximum packet -latency per test. - -*Title of each graph* is a regex (regular expression) matching all -throughput test cases plotted on this graph, *X-axis labels* are indices -of individual test suites executed by csit-vpp-perf-1704-all job that -created result output file used as data source for the graph, *Y-axis -labels* are measured packet Latency [uSec] values, and the *Graph -legend* lists the plotted test suites and their indices. Latency is -reported for concurrent symmetric bi-directional flows, separately for -each direction: i) West-to-East: TGint1-to-SUT1-to-SUT2-to-TGint2, and -ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1. - -.. note:: - - Test results have been generated by FD.io test executor jobs - `csit-vpp-perf-1704-all - <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_, - with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip - `archived here <../../_static/archive/>`_. - VPP packet latency in 1t1c setup (1thread, 1core) is presented in the graph below. .. raw:: html diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6.rst index 93c1befe0d..fd556a3134 100644 --- a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6.rst +++ b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6.rst @@ -7,28 +7,6 @@ rate. Latency is reported for VPP running in multiple configurations of VPP worker thread(s), a.k.a. VPP data plane thread(s), and their physical CPU core(s) placement. -Results are generated from a single execution of CSIT NDR discovery -test. Box plots are used to show the Minimum, Average and Maximum packet -latency per test. - -*Title of each graph* is a regex (regular expression) matching all -throughput test cases plotted on this graph, *X-axis labels* are indices -of individual test suites executed by csit-vpp-perf-1704-all job that -created result output file used as data source for the graph, *Y-axis -labels* are measured packet Latency [uSec] values, and the *Graph -legend* lists the plotted test suites and their indices. Latency is -reported for concurrent symmetric bi-directional flows, separately for -each direction: i) West-to-East: TGint1-to-SUT1-to-SUT2-to-TGint2, and -ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1. - -.. note:: - - Test results have been generated by FD.io test executor jobs - `csit-vpp-perf-1704-all - <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_, - with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip - `archived here <../../_static/archive/>`_. - VPP packet latency in 1t1c setup (1thread, 1core) is presented in the graph below. .. raw:: html diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6_tunnels.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6_tunnels.rst index 186bbfd386..c650a53e6c 100644 --- a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6_tunnels.rst +++ b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6_tunnels.rst @@ -7,28 +7,6 @@ rate. Latency is reported for VPP running in multiple configurations of VPP worker thread(s), a.k.a. VPP data plane thread(s), and their physical CPU core(s) placement. -Results are generated from a single execution of CSIT NDR discovery -test. Box plots are used to show the Minimum, Average and Maximum packet -latency per test. - -*Title of each graph* is a regex (regular expression) matching all -throughput test cases plotted on this graph, *X-axis labels* are indices -of individual test suites executed by csit-vpp-perf-1704-all job that -created result output file used as data source for the graph, *Y-axis -labels* are measured packet Latency [uSec] values, and the *Graph -legend* lists the plotted test suites and their indices. Latency is -reported for concurrent symmetric bi-directional flows, separately for -each direction: i) West-to-East: TGint1-to-SUT1-to-SUT2-to-TGint2, and -ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1. - -.. note:: - - Test results have been generated by FD.io test executor jobs - `csit-vpp-perf-1704-all - <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_, - with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip - `archived here <../../_static/archive/>`_. - VPP packet latency in 1t1c setup (1thread, 1core) is presented in the graph below. .. raw:: html diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/l2.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/l2.rst index a017a42773..129470d5f1 100644 --- a/docs/report/vpp_performance_tests/packet_latency_graphs/l2.rst +++ b/docs/report/vpp_performance_tests/packet_latency_graphs/l2.rst @@ -7,28 +7,6 @@ rate. Latency is reported for VPP running in multiple configurations of VPP worker thread(s), a.k.a. VPP data plane thread(s), and their physical CPU core(s) placement. -Results are generated from a single execution of CSIT NDR discovery -test. Box plots are used to show the Minimum, Average and Maximum packet -latency per test. - -*Title of each graph* is a regex (regular expression) matching all -throughput test cases plotted on this graph, *X-axis labels* are indices -of individual test suites executed by csit-vpp-perf-1704-all job that -created result output file used as data source for the graph, *Y-axis -labels* are measured packet Latency [uSec] values, and the *Graph -legend* lists the plotted test suites and their indices. Latency is -reported for concurrent symmetric bi-directional flows, separately for -each direction: i) West-to-East: TGint1-to-SUT1-to-SUT2-to-TGint2, and -ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1. - -.. note:: - - Test results have been generated by FD.io test executor jobs - `csit-vpp-perf-1704-all - <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_, - with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip - `archived here <../../_static/archive/>`_. - VPP packet latency in 1t1c setup (1thread, 1core) is presented in the graph below. .. raw:: html diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/vm_vhost.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/vm_vhost.rst index d5010289d6..eb9732dfe4 100644 --- a/docs/report/vpp_performance_tests/packet_latency_graphs/vm_vhost.rst +++ b/docs/report/vpp_performance_tests/packet_latency_graphs/vm_vhost.rst @@ -8,28 +8,6 @@ for VPP running in multiple configurations of VPP worker thread(s), a.k.a. VPP data plane thread(s), and their physical CPU core(s) placement. -Results are generated from a single execution of CSIT NDR discovery -test. Box plots are used to show the Minimum, Average and Maximum packet -latency per test. - -*Title of each graph* is a regex (regular expression) matching all -throughput test cases plotted on this graph, *X-axis labels* are indices -of individual test suites executed by csit-vpp-perf-1704-all job that -created result output file used as data source for the graph, *Y-axis -labels* are measured packet Latency [uSec] values, and the *Graph -legend* lists the plotted test suites and their indices. Latency is -reported for concurrent symmetric bi-directional flows, separately for -each direction: i) West-to-East: TGint1-to-SUT1-to-SUT2-to-TGint2, and -ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1. - -.. note:: - - Test results have been generated by FD.io test executor jobs - `csit-vpp-perf-1704-all - <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_, - with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip - `archived here <../../_static/archive/>`_. - VPP packet latency in 1t1c setup (1thread, 1core) is presented in the graph below. .. raw:: html diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/index.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/index.rst index 4abdf8eeb9..d744ef09e5 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/index.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/index.rst @@ -1,6 +1,39 @@ Packet Throughput Graphs ======================== +Plotted results are generated by multiple executions of the same CSIT +performance tests across three physical testbeds within LF FD.io labs. +To provide a descriptive summary view, Box-and-Whisker plots are used to +display variation in measured throughput values, without making any +assumptions of the underlying statistical distribution. + +For each plotted test case, Box-and-Whisker plots show the quartiles +(Min, 1st quartile / 25th percentile, 2nd quartile / 50th percentile / +mean, 3rd quartile / 75th percentile, Max) across collected data set +(data set size stated in the note below). Outliers are plotted as +individual points. Min and max values are plotted as bottom and top +Whiskers respectively. 2nd and 3rd quartiles are plotted as bottom and +top edge of the box. If multiple samples match only two values, and all +samples fall between them, then no whiskers are plotted. If all samples +have the same value, only a horizontal line is plotted. + +*Title of each graph* is a regex (regular expression) matching all +throughput test cases plotted on this graph, *X-axis labels* are indices +of individual test suites executed by csit-vpp-perf-1704-all jobs that +created result output files used as data sources for the graph, *Y-axis +labels* are measured Packets Per Second [pps] values, and the *Graph +legend* lists the plotted test suites and their indices. + +.. note:: + + Test results have been generated by FD.io test executor jobs + `csit-vpp-perf-1704-all + <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_, + with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip + `archived here <../../_static/archive/>`_. Plotted data set size per + test case is equal to the number of job executions presented in this + report version: **10**. + .. toctree:: l2 diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipsec.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipsec.rst index ad71a8a07c..a76b78fcdd 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipsec.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipsec.rst @@ -10,32 +10,10 @@ Performance is reported for VPP running in multiple configurations of VPP worker thread(s), a.k.a. VPP data plane thread(s), and their physical CPU core(s) placement. -Results are generated by multiple executions of the same CSIT -performance tests across three physical testbeds within LF FD.io labs. -To provide a descriptive summary view, Box-and-Whisker plots are used to -display variation in measured throughput values, without making any -assumptions of the underlying statistical distribution. For each plotted -test case, Box-and-Whisker plots show the quartiles (Min, 1st quartile -/ 25th percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / -75th percentile, Max) across collected data set (data set size stated in -the note below). Outliers are plotted as individual points. - -*Title of each graph* is a regex (regular expression) matching all -throughput test cases plotted on this graph, *X-axis labels* are indices -of individual test suites executed by csit-vpp-perf-1704-all jobs that -created result output files used as data sources for the graph, *Y-axis -labels* are measured Packets Per Second [pps] values, and the *Graph -legend* lists the plotted test suites and their indices. - -.. note:: - - Test results have been generated by FD.io test executor jobs - `csit-vpp-perf-1704-all - <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_, - with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip - `archived here <../../_static/archive/>`_. Plotted data set size per - test case is equal to the number of job executions presented in this - report version: **5**. +Note: only 1,000 scale IPSec tunnel test cases are plotted. Single IPSec +tunnel test cases are listed in regex outputs, but marked {NOT PLOTTED} +due to the current limitation of auto-plotting scripts. For all test +result data please refere to **Detailed Test Results** section. NDR Throughput ~~~~~~~~~~~~~~ @@ -59,11 +37,11 @@ git repository: 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-aes-gcm-ndrpdrdisc.robot:| tc01-64B-1t1c-ethip4ipsecscale1ip4-ip4base-interfaces-aes-gcm-ndrdisc 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-aes-gcm-ndrpdrdisc.robot:| tc03-64B-1t1c-ethip4ipsecscale1000ip4-ip4base-interfaces-aes-gcm-ndrdisc - 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc01-64B-1t1c-ethip4ipsecscale1ip4-ip4base-interfaces-cbc-sha1-ndrdisc + {NOT PLOTTED} 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc01-64B-1t1c-ethip4ipsecscale1ip4-ip4base-interfaces-cbc-sha1-ndrdisc 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc03-64B-1t1c-ethip4ipsecscale1000ip4-ip4base-interfaces-cbc-sha1-ndrdisc - 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc01-64B-1t1c-ethip4ipsecscale1ip4-ip4base-tunnels-aes-gcm-ndrdisc + {NOT PLOTTED} 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc01-64B-1t1c-ethip4ipsecscale1ip4-ip4base-tunnels-aes-gcm-ndrdisc 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc03-64B-1t1c-ethip4ipsecscale1000ip4-ip4base-tunnels-aes-gcm-ndrdisc - 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc01-64B-1t1c-ethip4ipsecscale1ip4-ip4base-tunnels-cbc-sha1-ndrdisc + {NOT PLOTTED} 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc01-64B-1t1c-ethip4ipsecscale1ip4-ip4base-tunnels-cbc-sha1-ndrdisc 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc03-64B-1t1c-ethip4ipsecscale1000ip4-ip4base-tunnels-cbc-sha1-ndrdisc 40ge2p1xl710-ethip4ipsectptlispgpe-ip4base-cbc-sha1-ndrpdrdisc.robot:| tc01-64B-1t1c-ethip4ipsectptlispgpe-ip4base-cbc-sha1-ndrdisc @@ -87,11 +65,11 @@ git repository: 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-aes-gcm-ndrpdrdisc.robot:| tc13-64B-2t2c-ethip4ipsecscale1ip4-ip4base-interfaces-aes-gcm-ndrdisc 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-aes-gcm-ndrpdrdisc.robot:| tc15-64B-2t2c-ethip4ipsecscale1000ip4-ip4base-interfaces-aes-gcm-ndrdisc - 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc13-64B-2t2c-ethip4ipsecscale1ip4-ip4base-interfaces-cbc-sha1-ndrdisc + {NOT PLOTTED} 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc13-64B-2t2c-ethip4ipsecscale1ip4-ip4base-interfaces-cbc-sha1-ndrdisc 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc15-64B-2t2c-ethip4ipsecscale1000ip4-ip4base-interfaces-cbc-sha1-ndrdisc - 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc13-64B-2t2c-ethip4ipsecscale1ip4-ip4base-tunnels-aes-gcm-ndrdisc + {NOT PLOTTED} 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc13-64B-2t2c-ethip4ipsecscale1ip4-ip4base-tunnels-aes-gcm-ndrdisc 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc15-64B-2t2c-ethip4ipsecscale1000ip4-ip4base-tunnels-aes-gcm-ndrdisc - 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc13-64B-2t2c-ethip4ipsecscale1ip4-ip4base-tunnels-cbc-sha1-ndrdisc + {NOT PLOTTED} 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc13-64B-2t2c-ethip4ipsecscale1ip4-ip4base-tunnels-cbc-sha1-ndrdisc 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc15-64B-2t2c-ethip4ipsecscale1000ip4-ip4base-tunnels-cbc-sha1-ndrdisc 40ge2p1xl710-ethip4ipsectptlispgpe-ip4base-cbc-sha1-ndrpdrdisc.robot:| tc07-64B-2t2c-ethip4ipsectptlispgpe-ip4base-cbc-sha1-ndrdisc @@ -117,11 +95,11 @@ git repository: 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-aes-gcm-ndrpdrdisc.robot:| tc02-64B-1t1c-ethip4ipsecscale1ip4-ip4base-interfaces-aes-gcm-pdrdisc 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-aes-gcm-ndrpdrdisc.robot:| tc04-64B-1t1c-ethip4ipsecscale1000ip4-ip4base-interfaces-aes-gcm-pdrdisc - 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc02-64B-1t1c-ethip4ipsecscale1ip4-ip4base-interfaces-cbc-sha1-pdrdisc + {NOT PLOTTED} 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc02-64B-1t1c-ethip4ipsecscale1ip4-ip4base-interfaces-cbc-sha1-pdrdisc 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc04-64B-1t1c-ethip4ipsecscale1000ip4-ip4base-interfaces-cbc-sha1-pdrdisc - 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc02-64B-1t1c-ethip4ipsecscale1ip4-ip4base-tunnels-aes-gcm-pdrdisc + {NOT PLOTTED} 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc02-64B-1t1c-ethip4ipsecscale1ip4-ip4base-tunnels-aes-gcm-pdrdisc 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc04-64B-1t1c-ethip4ipsecscale1000ip4-ip4base-tunnels-aes-gcm-pdrdisc - 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc02-64B-1t1c-ethip4ipsecscale1ip4-ip4base-tunnels-cbc-sha1-pdrdisc + {NOT PLOTTED} 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc02-64B-1t1c-ethip4ipsecscale1ip4-ip4base-tunnels-cbc-sha1-pdrdisc 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc04-64B-1t1c-ethip4ipsecscale1000ip4-ip4base-tunnels-cbc-sha1-pdrdisc 40ge2p1xl710-ethip4ipsectptlispgpe-ip4base-cbc-sha1-ndrpdrdisc.robot:| tc02-64B-1t1c-ethip4ipsectptlispgpe-ip4base-cbc-sha1-pdrdisc @@ -145,11 +123,11 @@ git repository: 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-aes-gcm-ndrpdrdisc.robot:| tc14-64B-2t2c-ethip4ipsecscale1ip4-ip4base-interfaces-aes-gcm-pdrdisc 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-aes-gcm-ndrpdrdisc.robot:| tc16-64B-2t2c-ethip4ipsecscale1000ip4-ip4base-interfaces-aes-gcm-pdrdisc - 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc14-64B-2t2c-ethip4ipsecscale1ip4-ip4base-interfaces-cbc-sha1-pdrdisc + {NOT PLOTTED} 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc14-64B-2t2c-ethip4ipsecscale1ip4-ip4base-interfaces-cbc-sha1-pdrdisc 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc16-64B-2t2c-ethip4ipsecscale1000ip4-ip4base-interfaces-cbc-sha1-pdrdisc - 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc14-64B-2t2c-ethip4ipsecscale1ip4-ip4base-tunnels-aes-gcm-pdrdisc + {NOT PLOTTED} 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc14-64B-2t2c-ethip4ipsecscale1ip4-ip4base-tunnels-aes-gcm-pdrdisc 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc16-64B-2t2c-ethip4ipsecscale1000ip4-ip4base-tunnels-aes-gcm-pdrdisc - 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc14-64B-2t2c-ethip4ipsecscale1ip4-ip4base-tunnels-cbc-sha1-pdrdisc + {NOT PLOTTED} 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc14-64B-2t2c-ethip4ipsecscale1ip4-ip4base-tunnels-cbc-sha1-pdrdisc 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc16-64B-2t2c-ethip4ipsecscale1000ip4-ip4base-tunnels-cbc-sha1-pdrdisc 40ge2p1xl710-ethip4ipsectptlispgpe-ip4base-cbc-sha1-ndrpdrdisc.robot:| tc08-64B-2t2c-ethip4ipsectptlispgpe-ip4base-cbc-sha1-pdrdisc diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4.rst index cd57cbd94c..bc56f389a8 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4.rst @@ -7,33 +7,6 @@ and PDR throughput (<0.5% packet loss). Performance is reported for VPP running in multiple configurations of VPP worker thread(s), a.k.a. VPP data plane thread(s), and their physical CPU core(s) placement. -Results are generated by multiple executions of the same CSIT -performance tests across three physical testbeds within LF FD.io labs. -To provide a descriptive summary view, Box-and-Whisker plots are used to -display variation in measured throughput values, without making any -assumptions of the underlying statistical distribution. For each plotted -test case, Box-and-Whisker plots show the quartiles (Min, 1st quartile -/ 25th percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / -75th percentile, Max) across collected data set (data set size stated in -the note below). Outliers are plotted as individual points. - -*Title of each graph* is a regex (regular expression) matching all -throughput test cases plotted on this graph, *X-axis labels* are indices -of individual test suites executed by csit-vpp-perf-1704-all jobs that -created result output files used as data sources for the graph, *Y-axis -labels* are measured Packets Per Second [pps] values, and the *Graph -legend* lists the plotted test suites and their indices. - -.. note:: - - Test results have been generated by FD.io test executor jobs - `csit-vpp-perf-1704-all - <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_, - with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip - `archived here <../../_static/archive/>`_. Plotted data set size per - test case is equal to the number of job executions presented in this - report version: **5**. - NDR Throughput ~~~~~~~~~~~~~~ diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4_tunnels.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4_tunnels.rst index 0701ed06ba..8cd9374381 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4_tunnels.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4_tunnels.rst @@ -7,33 +7,6 @@ and PDR throughput (<0.5% packet loss). Performance is reported for VPP running in multiple configurations of VPP worker thread(s), a.k.a. VPP data plane thread(s), and their physical CPU core(s) placement. -Results are generated by multiple executions of the same CSIT -performance tests across three physical testbeds within LF FD.io labs. -To provide a descriptive summary view, Box-and-Whisker plots are used to -display variation in measured throughput values, without making any -assumptions of the underlying statistical distribution. For each plotted -test case, Box-and-Whisker plots show the quartiles (Min, 1st quartile -/ 25th percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / -75th percentile, Max) across collected data set (data set size stated in -the note below). Outliers are plotted as individual points. - -*Title of each graph* is a regex (regular expression) matching all -throughput test cases plotted on this graph, *X-axis labels* are indices -of individual test suites executed by csit-vpp-perf-1704-all jobs that -created result output files used as data sources for the graph, *Y-axis -labels* are measured Packets Per Second [pps] values, and the *Graph -legend* lists the plotted test suites and their indices. - -.. note:: - - Test results have been generated by FD.io test executor jobs - `csit-vpp-perf-1704-all - <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_, - with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip - `archived here <../../_static/archive/>`_. Plotted data set size per - test case is equal to the number of job executions presented in this - report version: **5**. - NDR Throughput ~~~~~~~~~~~~~~ diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6.rst index e2b8554036..6bcd52f33f 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6.rst @@ -7,33 +7,6 @@ and PDR throughput (<0.5% packet loss). Performance is reported for VPP running in multiple configurations of VPP worker thread(s), a.k.a. VPP data plane thread(s), and their physical CPU core(s) placement. -Results are generated by multiple executions of the same CSIT -performance tests across three physical testbeds within LF FD.io labs. -To provide a descriptive summary view, Box-and-Whisker plots are used to -display variation in measured throughput values, without making any -assumptions of the underlying statistical distribution. For each plotted -test case, Box-and-Whisker plots show the quartiles (Min, 1st quartile -/ 25th percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / -75th percentile, Max) across collected data set (data set size stated in -the note below). Outliers are plotted as individual points. - -*Title of each graph* is a regex (regular expression) matching all -throughput test cases plotted on this graph, *X-axis labels* are indices -of individual test suites executed by csit-vpp-perf-1704-all jobs that -created result output files used as data sources for the graph, *Y-axis -labels* are measured Packets Per Second [pps] values, and the *Graph -legend* lists the plotted test suites and their indices. - -.. note:: - - Test results have been generated by FD.io test executor jobs - `csit-vpp-perf-1704-all - <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_, - with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip - `archived here <../../_static/archive/>`_. Plotted data set size per - test case is equal to the number of job executions presented in this - report version: **5**. - NDR Throughput ~~~~~~~~~~~~~~ diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6_tunnels.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6_tunnels.rst index ebc585ea59..0baf6dd782 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6_tunnels.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6_tunnels.rst @@ -7,33 +7,6 @@ and PDR throughput (<0.5% packet loss). Performance is reported for VPP running in multiple configurations of VPP worker thread(s), a.k.a. VPP data plane thread(s), and their physical CPU core(s) placement. -Results are generated by multiple executions of the same CSIT -performance tests across three physical testbeds within LF FD.io labs. -To provide a descriptive summary view, Box-and-Whisker plots are used to -display variation in measured throughput values, without making any -assumptions of the underlying statistical distribution. For each plotted -test case, Box-and-Whisker plots show the quartiles (Min, 1st quartile -/ 25th percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / -75th percentile, Max) across collected data set (data set size stated in -the note below). Outliers are plotted as individual points. - -*Title of each graph* is a regex (regular expression) matching all -throughput test cases plotted on this graph, *X-axis labels* are indices -of individual test suites executed by csit-vpp-perf-1704-all jobs that -created result output files used as data sources for the graph, *Y-axis -labels* are measured Packets Per Second [pps] values, and the *Graph -legend* lists the plotted test suites and their indices. - -.. note:: - - Test results have been generated by FD.io test executor jobs - `csit-vpp-perf-1704-all - <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_, - with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip - `archived here <../../_static/archive/>`_. Plotted data set size per - test case is equal to the number of job executions presented in this - report version: **5**. - NDR Throughput ~~~~~~~~~~~~~~ diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/l2.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/l2.rst index 5e8822c4a5..85309113a0 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/l2.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/l2.rst @@ -7,33 +7,6 @@ and PDR throughput (<0.5% packet loss). Performance is reported for VPP running in multiple configurations of VPP worker thread(s), a.k.a. VPP data plane thread(s), and their physical CPU core(s) placement. -Results are generated by multiple executions of the same CSIT -performance tests across three physical testbeds within LF FD.io labs. -To provide a descriptive summary view, Box-and-Whisker plots are used to -display variation in measured throughput values, without making any -assumptions of the underlying statistical distribution. For each plotted -test case, Box-and-Whisker plots show the quartiles (Min, 1st quartile -/ 25th percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / -75th percentile, Max) across collected data set (data set size stated in -the note below). Outliers are plotted as individual points. - -*Title of each graph* is a regex (regular expression) matching all -throughput test cases plotted on this graph, *X-axis labels* are indices -of individual test suites executed by csit-vpp-perf-1704-all jobs that -created result output files used as data sources for the graph, *Y-axis -labels* are measured Packets Per Second [pps] values, and the *Graph -legend* lists the plotted test suites and their indices. - -.. note:: - - Test results have been generated by FD.io test executor jobs - `csit-vpp-perf-1704-all - <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_, - with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip - `archived here <../../_static/archive/>`_. Plotted data set size per - test case is equal to the number of job executions presented in this - report version: **5**. - NDR Throughput ~~~~~~~~~~~~~~ diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/vm_vhost.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/vm_vhost.rst index 818a9de78c..b932a8e4f1 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/vm_vhost.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/vm_vhost.rst @@ -8,33 +8,6 @@ packet loss). Performance is reported for VPP running in multiple configurations of VPP worker thread(s), a.k.a. VPP data plane thread(s), and their physical CPU core(s) placement. -Results are generated by multiple executions of the same CSIT -performance tests across three physical testbeds within LF FD.io labs. -To provide a descriptive summary view, Box-and-Whisker plots are used to -display variation in measured throughput values, without making any -assumptions of the underlying statistical distribution. For each plotted -test case, Box-and-Whisker plots show the quartiles (Min, 1st quartile -/ 25th percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / -75th percentile, Max) across collected data set (data set size stated in -the note below). Outliers are plotted as individual points. - -*Title of each graph* is a regex (regular expression) matching all -throughput test cases plotted on this graph, *X-axis labels* are indices -of individual test suites executed by csit-vpp-perf-1704-all jobs that -created result output files used as data sources for the graph, *Y-axis -labels* are measured Packets Per Second [pps] values, and the *Graph -legend* lists the plotted test suites and their indices. - -.. note:: - - Test results have been generated by FD.io test executor jobs - `csit-vpp-perf-1704-all - <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_, - with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip - `archived here <../../_static/archive/>`_. Plotted data set size per - test case is equal to the number of job executions presented in this - report version: **5**. - NDR Throughput ~~~~~~~~~~~~~~ |