aboutsummaryrefslogtreecommitdiffstats
path: root/docs/report/vpp_performance_tests
diff options
context:
space:
mode:
Diffstat (limited to 'docs/report/vpp_performance_tests')
-rw-r--r--docs/report/vpp_performance_tests/packet_latency_graphs/index.rst22
-rw-r--r--docs/report/vpp_performance_tests/packet_latency_graphs/ipsec.rst14
-rw-r--r--docs/report/vpp_performance_tests/packet_latency_graphs/ipv4.rst22
-rw-r--r--docs/report/vpp_performance_tests/packet_latency_graphs/ipv4_tunnels.rst22
-rw-r--r--docs/report/vpp_performance_tests/packet_latency_graphs/ipv6.rst22
-rw-r--r--docs/report/vpp_performance_tests/packet_latency_graphs/ipv6_tunnels.rst22
-rw-r--r--docs/report/vpp_performance_tests/packet_latency_graphs/l2.rst22
-rw-r--r--docs/report/vpp_performance_tests/packet_latency_graphs/vm_vhost.rst22
-rw-r--r--docs/report/vpp_performance_tests/packet_throughput_graphs/index.rst33
-rw-r--r--docs/report/vpp_performance_tests/packet_throughput_graphs/ipsec.rst54
-rw-r--r--docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4.rst27
-rw-r--r--docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4_tunnels.rst27
-rw-r--r--docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6.rst27
-rw-r--r--docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6_tunnels.rst27
-rw-r--r--docs/report/vpp_performance_tests/packet_throughput_graphs/l2.rst27
-rw-r--r--docs/report/vpp_performance_tests/packet_throughput_graphs/vm_vhost.rst27
16 files changed, 71 insertions, 346 deletions
diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/index.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/index.rst
index f333f230f7..f71709453f 100644
--- a/docs/report/vpp_performance_tests/packet_latency_graphs/index.rst
+++ b/docs/report/vpp_performance_tests/packet_latency_graphs/index.rst
@@ -1,6 +1,28 @@
Packet Latency Graphs
=====================
+Plotted results are generated from a single execution of CSIT NDR discovery
+test. Box plots are used to show the Minimum, Median and Maximum packet
+latency per test.
+
+*Title of each graph* is a regex (regular expression) matching all
+throughput test cases plotted on this graph, *X-axis labels* are indices
+of individual test suites executed by csit-vpp-perf-1704-all job that
+created result output file used as data source for the graph, *Y-axis
+labels* are measured packet Latency [uSec] values, and the *Graph
+legend* lists the plotted test suites and their indices. Latency is
+reported for concurrent symmetric bi-directional flows, separately for
+each direction: i) West-to-East: TGint1-to-SUT1-to-SUT2-to-TGint2, and
+ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1.
+
+.. note::
+
+ Test results have been generated by FD.io test executor jobs
+ `csit-vpp-perf-1704-all
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_,
+ with Robot Framework result files csit-vpp-perf-1704-all-6.zip
+ `archived here <../../_static/archive/>`_.
+
.. toctree::
l2
diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/ipsec.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/ipsec.rst
index 0c506f720c..db82797dca 100644
--- a/docs/report/vpp_performance_tests/packet_latency_graphs/ipsec.rst
+++ b/docs/report/vpp_performance_tests/packet_latency_graphs/ipsec.rst
@@ -10,20 +10,6 @@ for VPP running in multiple configurations of VPP worker thread(s),
a.k.a. VPP data plane thread(s), and their physical CPU core(s)
placement.
-Results are generated from a single execution of CSIT NDR discovery
-test. Box plots are used to show the Minimum, Average and Maximum packet
-latency per test.
-
-*Title of each graph* is a regex (regular expression) matching all
-throughput test cases plotted on this graph, *X-axis labels* are indices
-of individual test suites executed by csit-vpp-perf-1704-all job that
-created result output file used as data source for the graph, *Y-axis
-labels* are measured packet Latency [uSec] values, and the *Graph
-legend* lists the plotted test suites and their indices. Latency is
-reported for concurrent symmetric bi-directional flows, separately for
-each direction: i) West-to-East: TGint1-to-SUT1-to-SUT2-to-TGint2, and
-ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1.
-
.. note::
Test results have been generated by FD.io test executor jobs
diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4.rst
index 5ddf813cea..10b22b2cc6 100644
--- a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4.rst
+++ b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4.rst
@@ -7,28 +7,6 @@ rate. Latency is reported for VPP running in multiple configurations of
VPP worker thread(s), a.k.a. VPP data plane thread(s), and their
physical CPU core(s) placement.
-Results are generated from a single execution of CSIT NDR discovery
-test. Box plots are used to show the Minimum, Average and Maximum packet
-latency per test.
-
-*Title of each graph* is a regex (regular expression) matching all
-throughput test cases plotted on this graph, *X-axis labels* are indices
-of individual test suites executed by csit-vpp-perf-1704-all job that
-created result output file used as data source for the graph, *Y-axis
-labels* are measured packet Latency [uSec] values, and the *Graph
-legend* lists the plotted test suites and their indices. Latency is
-reported for concurrent symmetric bi-directional flows, separately for
-each direction: i) West-to-East: TGint1-to-SUT1-to-SUT2-to-TGint2, and
-ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1.
-
-.. note::
-
- Test results have been generated by FD.io test executor jobs
- `csit-vpp-perf-1704-all
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_,
- with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip
- `archived here <../../_static/archive/>`_.
-
VPP packet latency in 1t1c setup (1thread, 1core) is presented in the graph below.
.. raw:: html
diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4_tunnels.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4_tunnels.rst
index ab39a97704..b14b0191de 100644
--- a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4_tunnels.rst
+++ b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4_tunnels.rst
@@ -7,28 +7,6 @@ rate. Latency is reported for VPP running in multiple configurations of
VPP worker thread(s), a.k.a. VPP data plane thread(s), and their
physical CPU core(s) placement.
-Results are generated from a single execution of CSIT NDR discovery
-test. Box plots are used to show the Minimum, Average and Maximum packet
-latency per test.
-
-*Title of each graph* is a regex (regular expression) matching all
-throughput test cases plotted on this graph, *X-axis labels* are indices
-of individual test suites executed by csit-vpp-perf-1704-all job that
-created result output file used as data source for the graph, *Y-axis
-labels* are measured packet Latency [uSec] values, and the *Graph
-legend* lists the plotted test suites and their indices. Latency is
-reported for concurrent symmetric bi-directional flows, separately for
-each direction: i) West-to-East: TGint1-to-SUT1-to-SUT2-to-TGint2, and
-ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1.
-
-.. note::
-
- Test results have been generated by FD.io test executor jobs
- `csit-vpp-perf-1704-all
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_,
- with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip
- `archived here <../../_static/archive/>`_.
-
VPP packet latency in 1t1c setup (1thread, 1core) is presented in the graph below.
.. raw:: html
diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6.rst
index 93c1befe0d..fd556a3134 100644
--- a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6.rst
+++ b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6.rst
@@ -7,28 +7,6 @@ rate. Latency is reported for VPP running in multiple configurations of
VPP worker thread(s), a.k.a. VPP data plane thread(s), and their
physical CPU core(s) placement.
-Results are generated from a single execution of CSIT NDR discovery
-test. Box plots are used to show the Minimum, Average and Maximum packet
-latency per test.
-
-*Title of each graph* is a regex (regular expression) matching all
-throughput test cases plotted on this graph, *X-axis labels* are indices
-of individual test suites executed by csit-vpp-perf-1704-all job that
-created result output file used as data source for the graph, *Y-axis
-labels* are measured packet Latency [uSec] values, and the *Graph
-legend* lists the plotted test suites and their indices. Latency is
-reported for concurrent symmetric bi-directional flows, separately for
-each direction: i) West-to-East: TGint1-to-SUT1-to-SUT2-to-TGint2, and
-ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1.
-
-.. note::
-
- Test results have been generated by FD.io test executor jobs
- `csit-vpp-perf-1704-all
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_,
- with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip
- `archived here <../../_static/archive/>`_.
-
VPP packet latency in 1t1c setup (1thread, 1core) is presented in the graph below.
.. raw:: html
diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6_tunnels.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6_tunnels.rst
index 186bbfd386..c650a53e6c 100644
--- a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6_tunnels.rst
+++ b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6_tunnels.rst
@@ -7,28 +7,6 @@ rate. Latency is reported for VPP running in multiple configurations of
VPP worker thread(s), a.k.a. VPP data plane thread(s), and their
physical CPU core(s) placement.
-Results are generated from a single execution of CSIT NDR discovery
-test. Box plots are used to show the Minimum, Average and Maximum packet
-latency per test.
-
-*Title of each graph* is a regex (regular expression) matching all
-throughput test cases plotted on this graph, *X-axis labels* are indices
-of individual test suites executed by csit-vpp-perf-1704-all job that
-created result output file used as data source for the graph, *Y-axis
-labels* are measured packet Latency [uSec] values, and the *Graph
-legend* lists the plotted test suites and their indices. Latency is
-reported for concurrent symmetric bi-directional flows, separately for
-each direction: i) West-to-East: TGint1-to-SUT1-to-SUT2-to-TGint2, and
-ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1.
-
-.. note::
-
- Test results have been generated by FD.io test executor jobs
- `csit-vpp-perf-1704-all
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_,
- with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip
- `archived here <../../_static/archive/>`_.
-
VPP packet latency in 1t1c setup (1thread, 1core) is presented in the graph below.
.. raw:: html
diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/l2.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/l2.rst
index a017a42773..129470d5f1 100644
--- a/docs/report/vpp_performance_tests/packet_latency_graphs/l2.rst
+++ b/docs/report/vpp_performance_tests/packet_latency_graphs/l2.rst
@@ -7,28 +7,6 @@ rate. Latency is reported for VPP running in multiple configurations of
VPP worker thread(s), a.k.a. VPP data plane thread(s), and their
physical CPU core(s) placement.
-Results are generated from a single execution of CSIT NDR discovery
-test. Box plots are used to show the Minimum, Average and Maximum packet
-latency per test.
-
-*Title of each graph* is a regex (regular expression) matching all
-throughput test cases plotted on this graph, *X-axis labels* are indices
-of individual test suites executed by csit-vpp-perf-1704-all job that
-created result output file used as data source for the graph, *Y-axis
-labels* are measured packet Latency [uSec] values, and the *Graph
-legend* lists the plotted test suites and their indices. Latency is
-reported for concurrent symmetric bi-directional flows, separately for
-each direction: i) West-to-East: TGint1-to-SUT1-to-SUT2-to-TGint2, and
-ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1.
-
-.. note::
-
- Test results have been generated by FD.io test executor jobs
- `csit-vpp-perf-1704-all
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_,
- with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip
- `archived here <../../_static/archive/>`_.
-
VPP packet latency in 1t1c setup (1thread, 1core) is presented in the graph below.
.. raw:: html
diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/vm_vhost.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/vm_vhost.rst
index d5010289d6..eb9732dfe4 100644
--- a/docs/report/vpp_performance_tests/packet_latency_graphs/vm_vhost.rst
+++ b/docs/report/vpp_performance_tests/packet_latency_graphs/vm_vhost.rst
@@ -8,28 +8,6 @@ for VPP running in multiple configurations of VPP worker thread(s),
a.k.a. VPP data plane thread(s), and their physical CPU core(s)
placement.
-Results are generated from a single execution of CSIT NDR discovery
-test. Box plots are used to show the Minimum, Average and Maximum packet
-latency per test.
-
-*Title of each graph* is a regex (regular expression) matching all
-throughput test cases plotted on this graph, *X-axis labels* are indices
-of individual test suites executed by csit-vpp-perf-1704-all job that
-created result output file used as data source for the graph, *Y-axis
-labels* are measured packet Latency [uSec] values, and the *Graph
-legend* lists the plotted test suites and their indices. Latency is
-reported for concurrent symmetric bi-directional flows, separately for
-each direction: i) West-to-East: TGint1-to-SUT1-to-SUT2-to-TGint2, and
-ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1.
-
-.. note::
-
- Test results have been generated by FD.io test executor jobs
- `csit-vpp-perf-1704-all
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_,
- with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip
- `archived here <../../_static/archive/>`_.
-
VPP packet latency in 1t1c setup (1thread, 1core) is presented in the graph below.
.. raw:: html
diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/index.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/index.rst
index 4abdf8eeb9..d744ef09e5 100644
--- a/docs/report/vpp_performance_tests/packet_throughput_graphs/index.rst
+++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/index.rst
@@ -1,6 +1,39 @@
Packet Throughput Graphs
========================
+Plotted results are generated by multiple executions of the same CSIT
+performance tests across three physical testbeds within LF FD.io labs.
+To provide a descriptive summary view, Box-and-Whisker plots are used to
+display variation in measured throughput values, without making any
+assumptions of the underlying statistical distribution.
+
+For each plotted test case, Box-and-Whisker plots show the quartiles
+(Min, 1st quartile / 25th percentile, 2nd quartile / 50th percentile /
+mean, 3rd quartile / 75th percentile, Max) across collected data set
+(data set size stated in the note below). Outliers are plotted as
+individual points. Min and max values are plotted as bottom and top
+Whiskers respectively. 2nd and 3rd quartiles are plotted as bottom and
+top edge of the box. If multiple samples match only two values, and all
+samples fall between them, then no whiskers are plotted. If all samples
+have the same value, only a horizontal line is plotted.
+
+*Title of each graph* is a regex (regular expression) matching all
+throughput test cases plotted on this graph, *X-axis labels* are indices
+of individual test suites executed by csit-vpp-perf-1704-all jobs that
+created result output files used as data sources for the graph, *Y-axis
+labels* are measured Packets Per Second [pps] values, and the *Graph
+legend* lists the plotted test suites and their indices.
+
+.. note::
+
+ Test results have been generated by FD.io test executor jobs
+ `csit-vpp-perf-1704-all
+ <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_,
+ with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip
+ `archived here <../../_static/archive/>`_. Plotted data set size per
+ test case is equal to the number of job executions presented in this
+ report version: **10**.
+
.. toctree::
l2
diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipsec.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipsec.rst
index ad71a8a07c..a76b78fcdd 100644
--- a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipsec.rst
+++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipsec.rst
@@ -10,32 +10,10 @@ Performance is reported for VPP running in multiple configurations of
VPP worker thread(s), a.k.a. VPP data plane thread(s), and their
physical CPU core(s) placement.
-Results are generated by multiple executions of the same CSIT
-performance tests across three physical testbeds within LF FD.io labs.
-To provide a descriptive summary view, Box-and-Whisker plots are used to
-display variation in measured throughput values, without making any
-assumptions of the underlying statistical distribution. For each plotted
-test case, Box-and-Whisker plots show the quartiles (Min, 1st quartile
-/ 25th percentile, 2nd quartile / 50th percentile / mean, 3rd quartile /
-75th percentile, Max) across collected data set (data set size stated in
-the note below). Outliers are plotted as individual points.
-
-*Title of each graph* is a regex (regular expression) matching all
-throughput test cases plotted on this graph, *X-axis labels* are indices
-of individual test suites executed by csit-vpp-perf-1704-all jobs that
-created result output files used as data sources for the graph, *Y-axis
-labels* are measured Packets Per Second [pps] values, and the *Graph
-legend* lists the plotted test suites and their indices.
-
-.. note::
-
- Test results have been generated by FD.io test executor jobs
- `csit-vpp-perf-1704-all
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_,
- with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip
- `archived here <../../_static/archive/>`_. Plotted data set size per
- test case is equal to the number of job executions presented in this
- report version: **5**.
+Note: only 1,000 scale IPSec tunnel test cases are plotted. Single IPSec
+tunnel test cases are listed in regex outputs, but marked {NOT PLOTTED}
+due to the current limitation of auto-plotting scripts. For all test
+result data please refere to **Detailed Test Results** section.
NDR Throughput
~~~~~~~~~~~~~~
@@ -59,11 +37,11 @@ git repository:
40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-aes-gcm-ndrpdrdisc.robot:| tc01-64B-1t1c-ethip4ipsecscale1ip4-ip4base-interfaces-aes-gcm-ndrdisc
40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-aes-gcm-ndrpdrdisc.robot:| tc03-64B-1t1c-ethip4ipsecscale1000ip4-ip4base-interfaces-aes-gcm-ndrdisc
- 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc01-64B-1t1c-ethip4ipsecscale1ip4-ip4base-interfaces-cbc-sha1-ndrdisc
+ {NOT PLOTTED} 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc01-64B-1t1c-ethip4ipsecscale1ip4-ip4base-interfaces-cbc-sha1-ndrdisc
40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc03-64B-1t1c-ethip4ipsecscale1000ip4-ip4base-interfaces-cbc-sha1-ndrdisc
- 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc01-64B-1t1c-ethip4ipsecscale1ip4-ip4base-tunnels-aes-gcm-ndrdisc
+ {NOT PLOTTED} 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc01-64B-1t1c-ethip4ipsecscale1ip4-ip4base-tunnels-aes-gcm-ndrdisc
40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc03-64B-1t1c-ethip4ipsecscale1000ip4-ip4base-tunnels-aes-gcm-ndrdisc
- 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc01-64B-1t1c-ethip4ipsecscale1ip4-ip4base-tunnels-cbc-sha1-ndrdisc
+ {NOT PLOTTED} 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc01-64B-1t1c-ethip4ipsecscale1ip4-ip4base-tunnels-cbc-sha1-ndrdisc
40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc03-64B-1t1c-ethip4ipsecscale1000ip4-ip4base-tunnels-cbc-sha1-ndrdisc
40ge2p1xl710-ethip4ipsectptlispgpe-ip4base-cbc-sha1-ndrpdrdisc.robot:| tc01-64B-1t1c-ethip4ipsectptlispgpe-ip4base-cbc-sha1-ndrdisc
@@ -87,11 +65,11 @@ git repository:
40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-aes-gcm-ndrpdrdisc.robot:| tc13-64B-2t2c-ethip4ipsecscale1ip4-ip4base-interfaces-aes-gcm-ndrdisc
40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-aes-gcm-ndrpdrdisc.robot:| tc15-64B-2t2c-ethip4ipsecscale1000ip4-ip4base-interfaces-aes-gcm-ndrdisc
- 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc13-64B-2t2c-ethip4ipsecscale1ip4-ip4base-interfaces-cbc-sha1-ndrdisc
+ {NOT PLOTTED} 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc13-64B-2t2c-ethip4ipsecscale1ip4-ip4base-interfaces-cbc-sha1-ndrdisc
40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc15-64B-2t2c-ethip4ipsecscale1000ip4-ip4base-interfaces-cbc-sha1-ndrdisc
- 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc13-64B-2t2c-ethip4ipsecscale1ip4-ip4base-tunnels-aes-gcm-ndrdisc
+ {NOT PLOTTED} 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc13-64B-2t2c-ethip4ipsecscale1ip4-ip4base-tunnels-aes-gcm-ndrdisc
40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc15-64B-2t2c-ethip4ipsecscale1000ip4-ip4base-tunnels-aes-gcm-ndrdisc
- 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc13-64B-2t2c-ethip4ipsecscale1ip4-ip4base-tunnels-cbc-sha1-ndrdisc
+ {NOT PLOTTED} 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc13-64B-2t2c-ethip4ipsecscale1ip4-ip4base-tunnels-cbc-sha1-ndrdisc
40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc15-64B-2t2c-ethip4ipsecscale1000ip4-ip4base-tunnels-cbc-sha1-ndrdisc
40ge2p1xl710-ethip4ipsectptlispgpe-ip4base-cbc-sha1-ndrpdrdisc.robot:| tc07-64B-2t2c-ethip4ipsectptlispgpe-ip4base-cbc-sha1-ndrdisc
@@ -117,11 +95,11 @@ git repository:
40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-aes-gcm-ndrpdrdisc.robot:| tc02-64B-1t1c-ethip4ipsecscale1ip4-ip4base-interfaces-aes-gcm-pdrdisc
40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-aes-gcm-ndrpdrdisc.robot:| tc04-64B-1t1c-ethip4ipsecscale1000ip4-ip4base-interfaces-aes-gcm-pdrdisc
- 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc02-64B-1t1c-ethip4ipsecscale1ip4-ip4base-interfaces-cbc-sha1-pdrdisc
+ {NOT PLOTTED} 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc02-64B-1t1c-ethip4ipsecscale1ip4-ip4base-interfaces-cbc-sha1-pdrdisc
40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc04-64B-1t1c-ethip4ipsecscale1000ip4-ip4base-interfaces-cbc-sha1-pdrdisc
- 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc02-64B-1t1c-ethip4ipsecscale1ip4-ip4base-tunnels-aes-gcm-pdrdisc
+ {NOT PLOTTED} 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc02-64B-1t1c-ethip4ipsecscale1ip4-ip4base-tunnels-aes-gcm-pdrdisc
40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc04-64B-1t1c-ethip4ipsecscale1000ip4-ip4base-tunnels-aes-gcm-pdrdisc
- 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc02-64B-1t1c-ethip4ipsecscale1ip4-ip4base-tunnels-cbc-sha1-pdrdisc
+ {NOT PLOTTED} 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc02-64B-1t1c-ethip4ipsecscale1ip4-ip4base-tunnels-cbc-sha1-pdrdisc
40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc04-64B-1t1c-ethip4ipsecscale1000ip4-ip4base-tunnels-cbc-sha1-pdrdisc
40ge2p1xl710-ethip4ipsectptlispgpe-ip4base-cbc-sha1-ndrpdrdisc.robot:| tc02-64B-1t1c-ethip4ipsectptlispgpe-ip4base-cbc-sha1-pdrdisc
@@ -145,11 +123,11 @@ git repository:
40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-aes-gcm-ndrpdrdisc.robot:| tc14-64B-2t2c-ethip4ipsecscale1ip4-ip4base-interfaces-aes-gcm-pdrdisc
40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-aes-gcm-ndrpdrdisc.robot:| tc16-64B-2t2c-ethip4ipsecscale1000ip4-ip4base-interfaces-aes-gcm-pdrdisc
- 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc14-64B-2t2c-ethip4ipsecscale1ip4-ip4base-interfaces-cbc-sha1-pdrdisc
+ {NOT PLOTTED} 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc14-64B-2t2c-ethip4ipsecscale1ip4-ip4base-interfaces-cbc-sha1-pdrdisc
40ge2p1xl710-ethip4ipsecscaleip4-ip4base-interfaces-cbc-sha1-ndrpdrdisc.robot:| tc16-64B-2t2c-ethip4ipsecscale1000ip4-ip4base-interfaces-cbc-sha1-pdrdisc
- 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc14-64B-2t2c-ethip4ipsecscale1ip4-ip4base-tunnels-aes-gcm-pdrdisc
+ {NOT PLOTTED} 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc14-64B-2t2c-ethip4ipsecscale1ip4-ip4base-tunnels-aes-gcm-pdrdisc
40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-aes-gcm-ndrpdrdisc.robot:| tc16-64B-2t2c-ethip4ipsecscale1000ip4-ip4base-tunnels-aes-gcm-pdrdisc
- 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc14-64B-2t2c-ethip4ipsecscale1ip4-ip4base-tunnels-cbc-sha1-pdrdisc
+ {NOT PLOTTED} 40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc14-64B-2t2c-ethip4ipsecscale1ip4-ip4base-tunnels-cbc-sha1-pdrdisc
40ge2p1xl710-ethip4ipsecscaleip4-ip4base-tunnels-cbc-sha1-ndrpdrdisc.robot:| tc16-64B-2t2c-ethip4ipsecscale1000ip4-ip4base-tunnels-cbc-sha1-pdrdisc
40ge2p1xl710-ethip4ipsectptlispgpe-ip4base-cbc-sha1-ndrpdrdisc.robot:| tc08-64B-2t2c-ethip4ipsectptlispgpe-ip4base-cbc-sha1-pdrdisc
diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4.rst
index cd57cbd94c..bc56f389a8 100644
--- a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4.rst
+++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4.rst
@@ -7,33 +7,6 @@ and PDR throughput (<0.5% packet loss). Performance is reported for VPP
running in multiple configurations of VPP worker thread(s), a.k.a. VPP
data plane thread(s), and their physical CPU core(s) placement.
-Results are generated by multiple executions of the same CSIT
-performance tests across three physical testbeds within LF FD.io labs.
-To provide a descriptive summary view, Box-and-Whisker plots are used to
-display variation in measured throughput values, without making any
-assumptions of the underlying statistical distribution. For each plotted
-test case, Box-and-Whisker plots show the quartiles (Min, 1st quartile
-/ 25th percentile, 2nd quartile / 50th percentile / mean, 3rd quartile /
-75th percentile, Max) across collected data set (data set size stated in
-the note below). Outliers are plotted as individual points.
-
-*Title of each graph* is a regex (regular expression) matching all
-throughput test cases plotted on this graph, *X-axis labels* are indices
-of individual test suites executed by csit-vpp-perf-1704-all jobs that
-created result output files used as data sources for the graph, *Y-axis
-labels* are measured Packets Per Second [pps] values, and the *Graph
-legend* lists the plotted test suites and their indices.
-
-.. note::
-
- Test results have been generated by FD.io test executor jobs
- `csit-vpp-perf-1704-all
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_,
- with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip
- `archived here <../../_static/archive/>`_. Plotted data set size per
- test case is equal to the number of job executions presented in this
- report version: **5**.
-
NDR Throughput
~~~~~~~~~~~~~~
diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4_tunnels.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4_tunnels.rst
index 0701ed06ba..8cd9374381 100644
--- a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4_tunnels.rst
+++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4_tunnels.rst
@@ -7,33 +7,6 @@ and PDR throughput (<0.5% packet loss). Performance is reported for VPP
running in multiple configurations of VPP worker thread(s), a.k.a. VPP
data plane thread(s), and their physical CPU core(s) placement.
-Results are generated by multiple executions of the same CSIT
-performance tests across three physical testbeds within LF FD.io labs.
-To provide a descriptive summary view, Box-and-Whisker plots are used to
-display variation in measured throughput values, without making any
-assumptions of the underlying statistical distribution. For each plotted
-test case, Box-and-Whisker plots show the quartiles (Min, 1st quartile
-/ 25th percentile, 2nd quartile / 50th percentile / mean, 3rd quartile /
-75th percentile, Max) across collected data set (data set size stated in
-the note below). Outliers are plotted as individual points.
-
-*Title of each graph* is a regex (regular expression) matching all
-throughput test cases plotted on this graph, *X-axis labels* are indices
-of individual test suites executed by csit-vpp-perf-1704-all jobs that
-created result output files used as data sources for the graph, *Y-axis
-labels* are measured Packets Per Second [pps] values, and the *Graph
-legend* lists the plotted test suites and their indices.
-
-.. note::
-
- Test results have been generated by FD.io test executor jobs
- `csit-vpp-perf-1704-all
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_,
- with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip
- `archived here <../../_static/archive/>`_. Plotted data set size per
- test case is equal to the number of job executions presented in this
- report version: **5**.
-
NDR Throughput
~~~~~~~~~~~~~~
diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6.rst
index e2b8554036..6bcd52f33f 100644
--- a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6.rst
+++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6.rst
@@ -7,33 +7,6 @@ and PDR throughput (<0.5% packet loss). Performance is reported for VPP
running in multiple configurations of VPP worker thread(s), a.k.a. VPP
data plane thread(s), and their physical CPU core(s) placement.
-Results are generated by multiple executions of the same CSIT
-performance tests across three physical testbeds within LF FD.io labs.
-To provide a descriptive summary view, Box-and-Whisker plots are used to
-display variation in measured throughput values, without making any
-assumptions of the underlying statistical distribution. For each plotted
-test case, Box-and-Whisker plots show the quartiles (Min, 1st quartile
-/ 25th percentile, 2nd quartile / 50th percentile / mean, 3rd quartile /
-75th percentile, Max) across collected data set (data set size stated in
-the note below). Outliers are plotted as individual points.
-
-*Title of each graph* is a regex (regular expression) matching all
-throughput test cases plotted on this graph, *X-axis labels* are indices
-of individual test suites executed by csit-vpp-perf-1704-all jobs that
-created result output files used as data sources for the graph, *Y-axis
-labels* are measured Packets Per Second [pps] values, and the *Graph
-legend* lists the plotted test suites and their indices.
-
-.. note::
-
- Test results have been generated by FD.io test executor jobs
- `csit-vpp-perf-1704-all
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_,
- with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip
- `archived here <../../_static/archive/>`_. Plotted data set size per
- test case is equal to the number of job executions presented in this
- report version: **5**.
-
NDR Throughput
~~~~~~~~~~~~~~
diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6_tunnels.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6_tunnels.rst
index ebc585ea59..0baf6dd782 100644
--- a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6_tunnels.rst
+++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6_tunnels.rst
@@ -7,33 +7,6 @@ and PDR throughput (<0.5% packet loss). Performance is reported for VPP
running in multiple configurations of VPP worker thread(s), a.k.a. VPP
data plane thread(s), and their physical CPU core(s) placement.
-Results are generated by multiple executions of the same CSIT
-performance tests across three physical testbeds within LF FD.io labs.
-To provide a descriptive summary view, Box-and-Whisker plots are used to
-display variation in measured throughput values, without making any
-assumptions of the underlying statistical distribution. For each plotted
-test case, Box-and-Whisker plots show the quartiles (Min, 1st quartile
-/ 25th percentile, 2nd quartile / 50th percentile / mean, 3rd quartile /
-75th percentile, Max) across collected data set (data set size stated in
-the note below). Outliers are plotted as individual points.
-
-*Title of each graph* is a regex (regular expression) matching all
-throughput test cases plotted on this graph, *X-axis labels* are indices
-of individual test suites executed by csit-vpp-perf-1704-all jobs that
-created result output files used as data sources for the graph, *Y-axis
-labels* are measured Packets Per Second [pps] values, and the *Graph
-legend* lists the plotted test suites and their indices.
-
-.. note::
-
- Test results have been generated by FD.io test executor jobs
- `csit-vpp-perf-1704-all
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_,
- with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip
- `archived here <../../_static/archive/>`_. Plotted data set size per
- test case is equal to the number of job executions presented in this
- report version: **5**.
-
NDR Throughput
~~~~~~~~~~~~~~
diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/l2.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/l2.rst
index 5e8822c4a5..85309113a0 100644
--- a/docs/report/vpp_performance_tests/packet_throughput_graphs/l2.rst
+++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/l2.rst
@@ -7,33 +7,6 @@ and PDR throughput (<0.5% packet loss). Performance is reported for VPP
running in multiple configurations of VPP worker thread(s), a.k.a. VPP
data plane thread(s), and their physical CPU core(s) placement.
-Results are generated by multiple executions of the same CSIT
-performance tests across three physical testbeds within LF FD.io labs.
-To provide a descriptive summary view, Box-and-Whisker plots are used to
-display variation in measured throughput values, without making any
-assumptions of the underlying statistical distribution. For each plotted
-test case, Box-and-Whisker plots show the quartiles (Min, 1st quartile
-/ 25th percentile, 2nd quartile / 50th percentile / mean, 3rd quartile /
-75th percentile, Max) across collected data set (data set size stated in
-the note below). Outliers are plotted as individual points.
-
-*Title of each graph* is a regex (regular expression) matching all
-throughput test cases plotted on this graph, *X-axis labels* are indices
-of individual test suites executed by csit-vpp-perf-1704-all jobs that
-created result output files used as data sources for the graph, *Y-axis
-labels* are measured Packets Per Second [pps] values, and the *Graph
-legend* lists the plotted test suites and their indices.
-
-.. note::
-
- Test results have been generated by FD.io test executor jobs
- `csit-vpp-perf-1704-all
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_,
- with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip
- `archived here <../../_static/archive/>`_. Plotted data set size per
- test case is equal to the number of job executions presented in this
- report version: **5**.
-
NDR Throughput
~~~~~~~~~~~~~~
diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/vm_vhost.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/vm_vhost.rst
index 818a9de78c..b932a8e4f1 100644
--- a/docs/report/vpp_performance_tests/packet_throughput_graphs/vm_vhost.rst
+++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/vm_vhost.rst
@@ -8,33 +8,6 @@ packet loss). Performance is reported for VPP running in multiple
configurations of VPP worker thread(s), a.k.a. VPP data plane thread(s),
and their physical CPU core(s) placement.
-Results are generated by multiple executions of the same CSIT
-performance tests across three physical testbeds within LF FD.io labs.
-To provide a descriptive summary view, Box-and-Whisker plots are used to
-display variation in measured throughput values, without making any
-assumptions of the underlying statistical distribution. For each plotted
-test case, Box-and-Whisker plots show the quartiles (Min, 1st quartile
-/ 25th percentile, 2nd quartile / 50th percentile / mean, 3rd quartile /
-75th percentile, Max) across collected data set (data set size stated in
-the note below). Outliers are plotted as individual points.
-
-*Title of each graph* is a regex (regular expression) matching all
-throughput test cases plotted on this graph, *X-axis labels* are indices
-of individual test suites executed by csit-vpp-perf-1704-all jobs that
-created result output files used as data sources for the graph, *Y-axis
-labels* are measured Packets Per Second [pps] values, and the *Graph
-legend* lists the plotted test suites and their indices.
-
-.. note::
-
- Test results have been generated by FD.io test executor jobs
- `csit-vpp-perf-1704-all
- <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_,
- with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip
- `archived here <../../_static/archive/>`_. Plotted data set size per
- test case is equal to the number of job executions presented in this
- report version: **5**.
-
NDR Throughput
~~~~~~~~~~~~~~