diff options
Diffstat (limited to 'docs')
15 files changed, 95 insertions, 53 deletions
diff --git a/docs/report/vpp_performance_tests/csit_release_notes.rst b/docs/report/vpp_performance_tests/csit_release_notes.rst index 8aa4533972..8ab00d7e8e 100644 --- a/docs/report/vpp_performance_tests/csit_release_notes.rst +++ b/docs/report/vpp_performance_tests/csit_release_notes.rst @@ -74,6 +74,8 @@ Non-Drop Rate Throughput discovery tests: +-------------------+-----------------------------------------------------------------+------------+-----------+-----------+-----------------+
| IPv6 iAcl | 10ge2p1x520: 78B-1t1c-ethip6-ip6base-iacldstbase-ndrdisc | 6.5 | 6.1 | 6.9 | 13% |
+-------------------+-----------------------------------------------------------------+------------+-----------+-----------+-----------------+
+| IPv6 FIB 2M | 10ge2p1x520: 78B-1t1c-ethip6-ip6scale2m-ndrdisc | 5.3 | 4.2 | 4.6 | 10% |
++-------------------+-----------------------------------------------------------------+------------+-----------+-----------+-----------------+
PDR Throughput
~~~~~~~~~~~~~~
@@ -98,6 +100,8 @@ Partial Drop Rate thoughput discovery tests with packet Loss Tolerance of 0.5%: +-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+
| IPv6 iAcl | 10ge2p1x520: 78B-1t1c-ethip6-ip6base-iacldstbase-pdrdisc | 6.5 | 6.1 | 6.9 | 13% |
+-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+
+| IPv6 FIB 2M | 10ge2p1x520: 78B-1t1c-ethip6-ip6scale2m-pdrdisc | 5.3 | 4.2 | 4.6 | 10% |
++-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+
Measured improvements are in line with VPP code optimizations listed in
`VPP-17.04 release notes
@@ -156,8 +160,6 @@ Non-Drop Rate Throughput discovery tests: +-------------------+-----------------------------------------------------------------+------------+-----------+-----------+-----------------+
| IPv6 FIB 20k | 10ge2p1x520: 78B-1t1c-ethip6-ip6scale20k-ndrdisc | 6.9 | 6.5 | 6.9 | 6% |
+-------------------+-----------------------------------------------------------------+------------+-----------+-----------+-----------------+
-| IPv6 FIB 2M | 10ge2p1x520: 78B-1t1c-ethip6-ip6scale2m-ndrdisc | 5.3 | 4.2 | 4.6 | 10% |
-+-------------------+-----------------------------------------------------------------+------------+-----------+-----------+-----------------+
PDR Throughput
~~~~~~~~~~~~~~
@@ -196,8 +198,6 @@ Partial Drop Rate thoughput discovery tests with packet Loss Tolerance of 0.5%: +-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+
| IPv6 FIB 20k | 10ge2p1x520: 78B-1t1c-ethip6-ip6scale20k-pdrdisc | 6.9 | 6.5 | 6.9 | 6% |
+-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+
-| IPv6 FIB 2M | 10ge2p1x520: 78B-1t1c-ethip6-ip6scale2m-pdrdisc | 5.3 | 4.2 | 4.6 | 10% |
-+-------------------+-----------------------------------------------------------------+-----------+-----------+-----------+-----------------+
Known Issues
------------
diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/ipsec.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/ipsec.rst index 82f297ee92..0c506f720c 100644 --- a/docs/report/vpp_performance_tests/packet_latency_graphs/ipsec.rst +++ b/docs/report/vpp_performance_tests/packet_latency_graphs/ipsec.rst @@ -30,7 +30,7 @@ ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1. `csit-vpp-perf-1704-all <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_, with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip - `archived here <../../_static/archive/>`_ + `archived here <../../_static/archive/>`_. VPP packet latency in 1t1c setup (1thread, 1core) is presented in the graph below. diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4.rst index fb85fe9464..5ddf813cea 100644 --- a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4.rst +++ b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4.rst @@ -27,7 +27,7 @@ ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1. `csit-vpp-perf-1704-all <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_, with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip - `archived here <../../_static/archive/>`_ + `archived here <../../_static/archive/>`_. VPP packet latency in 1t1c setup (1thread, 1core) is presented in the graph below. diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4_tunnels.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4_tunnels.rst index 42f5ced1cd..ab39a97704 100644 --- a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4_tunnels.rst +++ b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4_tunnels.rst @@ -27,7 +27,7 @@ ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1. `csit-vpp-perf-1704-all <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_, with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip - `archived here <../../_static/archive/>`_ + `archived here <../../_static/archive/>`_. VPP packet latency in 1t1c setup (1thread, 1core) is presented in the graph below. diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6.rst index 7d4cd70885..93c1befe0d 100644 --- a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6.rst +++ b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6.rst @@ -27,7 +27,7 @@ ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1. `csit-vpp-perf-1704-all <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_, with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip - `archived here <../../_static/archive/>`_ + `archived here <../../_static/archive/>`_. VPP packet latency in 1t1c setup (1thread, 1core) is presented in the graph below. diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6_tunnels.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6_tunnels.rst index c54509d3a3..186bbfd386 100644 --- a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6_tunnels.rst +++ b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6_tunnels.rst @@ -27,7 +27,7 @@ ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1. `csit-vpp-perf-1704-all <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_, with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip - `archived here <../../_static/archive/>`_ + `archived here <../../_static/archive/>`_. VPP packet latency in 1t1c setup (1thread, 1core) is presented in the graph below. diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/l2.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/l2.rst index f7836feeef..a017a42773 100644 --- a/docs/report/vpp_performance_tests/packet_latency_graphs/l2.rst +++ b/docs/report/vpp_performance_tests/packet_latency_graphs/l2.rst @@ -27,7 +27,7 @@ ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1. `csit-vpp-perf-1704-all <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_, with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip - `archived here <../../_static/archive/>`_ + `archived here <../../_static/archive/>`_. VPP packet latency in 1t1c setup (1thread, 1core) is presented in the graph below. diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/vm_vhost.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/vm_vhost.rst index 2e3071d6d8..d5010289d6 100644 --- a/docs/report/vpp_performance_tests/packet_latency_graphs/vm_vhost.rst +++ b/docs/report/vpp_performance_tests/packet_latency_graphs/vm_vhost.rst @@ -28,7 +28,7 @@ ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1. `csit-vpp-perf-1704-all <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_, with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip - `archived here <../../_static/archive/>`_ + `archived here <../../_static/archive/>`_. VPP packet latency in 1t1c setup (1thread, 1core) is presented in the graph below. diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipsec.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipsec.rst index 63580303ff..ad71a8a07c 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipsec.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipsec.rst @@ -10,11 +10,15 @@ Performance is reported for VPP running in multiple configurations of VPP worker thread(s), a.k.a. VPP data plane thread(s), and their physical CPU core(s) placement. -Results are generated by multiple executions of the same CSIT tests. -In order to display variation in measured throughput values, Box-and- -whisker plots are used to show their quartiles (Min, 1st quartile / 25th -percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / 75th -percentile, Max). Outliers are plotted as individual points. +Results are generated by multiple executions of the same CSIT +performance tests across three physical testbeds within LF FD.io labs. +To provide a descriptive summary view, Box-and-Whisker plots are used to +display variation in measured throughput values, without making any +assumptions of the underlying statistical distribution. For each plotted +test case, Box-and-Whisker plots show the quartiles (Min, 1st quartile +/ 25th percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / +75th percentile, Max) across collected data set (data set size stated in +the note below). Outliers are plotted as individual points. *Title of each graph* is a regex (regular expression) matching all throughput test cases plotted on this graph, *X-axis labels* are indices @@ -29,7 +33,9 @@ legend* lists the plotted test suites and their indices. `csit-vpp-perf-1704-all <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_, with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip - `archived here <../../_static/archive/>`_ + `archived here <../../_static/archive/>`_. Plotted data set size per + test case is equal to the number of job executions presented in this + report version: **5**. NDR Throughput ~~~~~~~~~~~~~~ diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4.rst index be4af20e84..cd57cbd94c 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4.rst @@ -7,11 +7,15 @@ and PDR throughput (<0.5% packet loss). Performance is reported for VPP running in multiple configurations of VPP worker thread(s), a.k.a. VPP data plane thread(s), and their physical CPU core(s) placement. -Results are generated by multiple executions of the same CSIT tests. -In order to display variation in measured throughput values, Box-and- -whisker plots are used to show their quartiles (Min, 1st quartile / 25th -percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / 75th -percentile, Max). Outliers are plotted as individual points. +Results are generated by multiple executions of the same CSIT +performance tests across three physical testbeds within LF FD.io labs. +To provide a descriptive summary view, Box-and-Whisker plots are used to +display variation in measured throughput values, without making any +assumptions of the underlying statistical distribution. For each plotted +test case, Box-and-Whisker plots show the quartiles (Min, 1st quartile +/ 25th percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / +75th percentile, Max) across collected data set (data set size stated in +the note below). Outliers are plotted as individual points. *Title of each graph* is a regex (regular expression) matching all throughput test cases plotted on this graph, *X-axis labels* are indices @@ -26,7 +30,9 @@ legend* lists the plotted test suites and their indices. `csit-vpp-perf-1704-all <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_, with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip - `archived here <../../_static/archive/>`_ + `archived here <../../_static/archive/>`_. Plotted data set size per + test case is equal to the number of job executions presented in this + report version: **5**. NDR Throughput ~~~~~~~~~~~~~~ diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4_tunnels.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4_tunnels.rst index f99c0a502e..0701ed06ba 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4_tunnels.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4_tunnels.rst @@ -7,11 +7,15 @@ and PDR throughput (<0.5% packet loss). Performance is reported for VPP running in multiple configurations of VPP worker thread(s), a.k.a. VPP data plane thread(s), and their physical CPU core(s) placement. -Results are generated by multiple executions of the same CSIT tests. -In order to display variation in measured throughput values, Box-and- -whisker plots are used to show their quartiles (Min, 1st quartile / 25th -percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / 75th -percentile, Max). Outliers are plotted as individual points. +Results are generated by multiple executions of the same CSIT +performance tests across three physical testbeds within LF FD.io labs. +To provide a descriptive summary view, Box-and-Whisker plots are used to +display variation in measured throughput values, without making any +assumptions of the underlying statistical distribution. For each plotted +test case, Box-and-Whisker plots show the quartiles (Min, 1st quartile +/ 25th percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / +75th percentile, Max) across collected data set (data set size stated in +the note below). Outliers are plotted as individual points. *Title of each graph* is a regex (regular expression) matching all throughput test cases plotted on this graph, *X-axis labels* are indices @@ -26,7 +30,9 @@ legend* lists the plotted test suites and their indices. `csit-vpp-perf-1704-all <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_, with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip - `archived here <../../_static/archive/>`_ + `archived here <../../_static/archive/>`_. Plotted data set size per + test case is equal to the number of job executions presented in this + report version: **5**. NDR Throughput ~~~~~~~~~~~~~~ diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6.rst index c475ab8f18..e2b8554036 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6.rst @@ -7,11 +7,15 @@ and PDR throughput (<0.5% packet loss). Performance is reported for VPP running in multiple configurations of VPP worker thread(s), a.k.a. VPP data plane thread(s), and their physical CPU core(s) placement. -Results are generated by multiple executions of the same CSIT tests. -In order to display variation in measured throughput values, Box-and- -whisker plots are used to show their quartiles (Min, 1st quartile / 25th -percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / 75th -percentile, Max). Outliers are plotted as individual points. +Results are generated by multiple executions of the same CSIT +performance tests across three physical testbeds within LF FD.io labs. +To provide a descriptive summary view, Box-and-Whisker plots are used to +display variation in measured throughput values, without making any +assumptions of the underlying statistical distribution. For each plotted +test case, Box-and-Whisker plots show the quartiles (Min, 1st quartile +/ 25th percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / +75th percentile, Max) across collected data set (data set size stated in +the note below). Outliers are plotted as individual points. *Title of each graph* is a regex (regular expression) matching all throughput test cases plotted on this graph, *X-axis labels* are indices @@ -26,7 +30,9 @@ legend* lists the plotted test suites and their indices. `csit-vpp-perf-1704-all <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_, with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip - `archived here <../../_static/archive/>`_ + `archived here <../../_static/archive/>`_. Plotted data set size per + test case is equal to the number of job executions presented in this + report version: **5**. NDR Throughput ~~~~~~~~~~~~~~ diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6_tunnels.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6_tunnels.rst index 79e77445dd..ebc585ea59 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6_tunnels.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6_tunnels.rst @@ -7,11 +7,15 @@ and PDR throughput (<0.5% packet loss). Performance is reported for VPP running in multiple configurations of VPP worker thread(s), a.k.a. VPP data plane thread(s), and their physical CPU core(s) placement. -Rresults are generated by multiple executions of the same CSIT tests. -In order to display variation in measured throughput values, Box-and- -whisker plots are used to show their quartiles (Min, 1st quartile / 25th -percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / 75th -percentile, Max). Outliers are plotted as individual points. +Results are generated by multiple executions of the same CSIT +performance tests across three physical testbeds within LF FD.io labs. +To provide a descriptive summary view, Box-and-Whisker plots are used to +display variation in measured throughput values, without making any +assumptions of the underlying statistical distribution. For each plotted +test case, Box-and-Whisker plots show the quartiles (Min, 1st quartile +/ 25th percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / +75th percentile, Max) across collected data set (data set size stated in +the note below). Outliers are plotted as individual points. *Title of each graph* is a regex (regular expression) matching all throughput test cases plotted on this graph, *X-axis labels* are indices @@ -26,7 +30,9 @@ legend* lists the plotted test suites and their indices. `csit-vpp-perf-1704-all <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_, with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip - `archived here <../../_static/archive/>`_ + `archived here <../../_static/archive/>`_. Plotted data set size per + test case is equal to the number of job executions presented in this + report version: **5**. NDR Throughput ~~~~~~~~~~~~~~ diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/l2.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/l2.rst index d9a69015e4..5344dd7d51 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/l2.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/l2.rst @@ -7,11 +7,15 @@ and PDR throughput (<0.5% packet loss). Performance is reported for VPP running in multiple configurations of VPP worker thread(s), a.k.a. VPP data plane thread(s), and their physical CPU core(s) placement. -Results are generated by multiple executions of the same CSIT tests. -In order to display variation in measured throughput values, Box-and- -whisker plots are used to show their quartiles (Min, 1st quartile / 25th -percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / 75th -percentile, Max). Outliers are plotted as individual points. +Results are generated by multiple executions of the same CSIT +performance tests across three physical testbeds within LF FD.io labs. +To provide a descriptive summary view, Box-and-Whisker plots are used to +display variation in measured throughput values, without making any +assumptions of the underlying statistical distribution. For each plotted +test case, Box-and-Whisker plots show the quartiles (Min, 1st quartile +/ 25th percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / +75th percentile, Max) across collected data set (data set size stated in +the note below). Outliers are plotted as individual points. *Title of each graph* is a regex (regular expression) matching all throughput test cases plotted on this graph, *X-axis labels* are indices @@ -26,7 +30,9 @@ legend* lists the plotted test suites and their indices. `csit-vpp-perf-1704-all <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_, with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip - `archived here <../../_static/archive/>`_ + `archived here <../../_static/archive/>`_. Plotted data set size per + test case is equal to the number of job executions presented in this + report version: **5**. NDR Throughput ~~~~~~~~~~~~~~ diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/vm_vhost.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/vm_vhost.rst index e4c6af9f6b..818a9de78c 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/vm_vhost.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/vm_vhost.rst @@ -8,11 +8,15 @@ packet loss). Performance is reported for VPP running in multiple configurations of VPP worker thread(s), a.k.a. VPP data plane thread(s), and their physical CPU core(s) placement. -Results are generated by multiple executions of the same CSIT tests. -In order to display variation in measured throughput values, Box-and- -whisker plots are used to show their quartiles (Min, 1st quartile / 25th -percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / 75th -percentile, Max). Outliers are plotted as individual points. +Results are generated by multiple executions of the same CSIT +performance tests across three physical testbeds within LF FD.io labs. +To provide a descriptive summary view, Box-and-Whisker plots are used to +display variation in measured throughput values, without making any +assumptions of the underlying statistical distribution. For each plotted +test case, Box-and-Whisker plots show the quartiles (Min, 1st quartile +/ 25th percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / +75th percentile, Max) across collected data set (data set size stated in +the note below). Outliers are plotted as individual points. *Title of each graph* is a regex (regular expression) matching all throughput test cases plotted on this graph, *X-axis labels* are indices @@ -27,7 +31,9 @@ legend* lists the plotted test suites and their indices. `csit-vpp-perf-1704-all <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_, with Robot Framework result files csit-vpp-perf-1704-all-<id>.zip - `archived here <../../_static/archive/>`_ + `archived here <../../_static/archive/>`_. Plotted data set size per + test case is equal to the number of job executions presented in this + report version: **5**. NDR Throughput ~~~~~~~~~~~~~~ |