aboutsummaryrefslogtreecommitdiffstats
path: root/docs/report/testpmd_performance_tests
diff options
context:
space:
mode:
authorMaciek Konstantynowicz <mkonstan@cisco.com>2017-04-21 20:42:05 +0100
committerPeter Mikus <pmikus@cisco.com>2017-04-25 16:12:37 +0000
commit02f2a2176ec92efdf63399fb7dba1eb586465f38 (patch)
tree6c830bbc3b01daa38478385a054c7f3f631117cc /docs/report/testpmd_performance_tests
parent0fc813b1694a6ae70b759e7ca96741f21f81b051 (diff)
CSIT rls1704 report - update to handcrafted sections:
- index.rst - vpp_performance_tests/csit_release_notes - vpp_performance_tests/overview.rst - vpp_performance_tests/packet_throughput_graphs/... - testpmd_performance_tests/csit_release_notes.rst - vpp_functional_tests/csit_release_notes.rst - testpmd_performance_tests/packet_latency_graphs/... - testpmd_performance_tests/packet_throughput_graphs/... Change-Id: I06bd27bc30d06d69873da74756ca586e12e55643 Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
Diffstat (limited to 'docs/report/testpmd_performance_tests')
-rw-r--r--docs/report/testpmd_performance_tests/csit_release_notes.rst2
-rw-r--r--docs/report/testpmd_performance_tests/packet_latency_graphs/l2.rst61
-rw-r--r--docs/report/testpmd_performance_tests/packet_throughput_graphs/l2.rst104
3 files changed, 98 insertions, 69 deletions
diff --git a/docs/report/testpmd_performance_tests/csit_release_notes.rst b/docs/report/testpmd_performance_tests/csit_release_notes.rst
index 6d415a1add..ba78a1bc54 100644
--- a/docs/report/testpmd_performance_tests/csit_release_notes.rst
+++ b/docs/report/testpmd_performance_tests/csit_release_notes.rst
@@ -6,5 +6,5 @@ Changes in CSIT |release|
#. Added Testpmd tests
- - new NICs - Intel xl710
+ - new NICs - 2p40GE Intel xl710, 2p10GE Intel x710
diff --git a/docs/report/testpmd_performance_tests/packet_latency_graphs/l2.rst b/docs/report/testpmd_performance_tests/packet_latency_graphs/l2.rst
index bac622674c..ddb18f42d2 100644
--- a/docs/report/testpmd_performance_tests/packet_latency_graphs/l2.rst
+++ b/docs/report/testpmd_performance_tests/packet_latency_graphs/l2.rst
@@ -1,24 +1,33 @@
-L2 Ethernet Switching
-=====================
-
-This section provides a summary of Testpmd Phy-to-Phy L2 Ethernet looping
-performance illustrating packet latency measured at 50% of discovered NDR
-throughput rate. Latency is reported for Testpmd running in multiple
-configurations of Testpmd worker thread(s), a.k.a. Testpmd data plane thread
-(s), and their physical CPU core(s) placement.
-
-*Title of each graph* is a regex (regular expression) matching all plotted
-latency test cases, *X-axis labels* are indeces of csit-dpdk-perf-1704 jobs
-that created result output files used as data sources for the graph,
-*Y-axis labels* are measured packet Latency [uSec] values, and the *graph
-legend* identifes the plotted test suites.
+L2 Ethernet Interface Loop
+==========================
+
+This section includes summary graphs of Testpmd Phy-to-Phy packet
+latency with L2 Ethernet Interface Loop measured at 50% of discovered
+NDR throughput rate. Latency is reported for Testpmd running in multiple
+configurations of Testpmd pmd thread(s), a.k.a. Testpmd data plane
+thread(s), and their physical CPU core(s) placement.
+
+Results are generated from a single execution of CSIT NDR discovery
+test. Box plots are used to show the Minimum, Average and Maximum packet
+latency per test.
+
+*Title of each graph* is a regex (regular expression) matching all
+throughput test cases plotted on this graph, *X-axis labels* are indices
+of individual test suites executed by csit-dpdk-perf-1704-all job that
+created result output file used as data source for the graph, *Y-axis
+labels* are measured packet Latency [uSec] values, and the *Graph
+legend* lists the plotted test suites and their indices. Latency is
+reported for concurrent symmetric bi-directional flows, separately for
+each direction: i) West-to-East: TGint1-to-SUT1-to-SUT2-to-TGint2, and
+ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1.
.. note::
- Data sources for reported test results: i) FD.io test executor jobs
- `csit-dpdk-perf-1704-all <https://jenkins.fd.io/view/csit/job/csit-dpdk-perf-1704-all/>`_ ,
- ii) archived FD.io jobs test result output files
- `csit-dpdk-perf-1704-all#job-number <https://nexus.fd.io/link>`_.
+ Test results have been generated by FD.io test executor jobs
+ `csit-dpdk-perf-1704-all
+ <https://jenkins.fd.io/view/csit/job/csit-dpdk-perf-1704-all/>`_,
+ with Robot Framework result files csit-dpdk-perf-1704-all-<id>.zip
+ `archived here <../../_static/archive/>`_
Testpmd packet latency - running in configuration of **one worker thread (1t) on one
physical core (1c)** - is presented in the figure below.
@@ -34,10 +43,12 @@ git repository by filtering with specified regex as follows:
.. code-block:: bash
- $ csit/tests/perf
- $ grep -E "64B-1t1c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc" *
+ $ $CSIT/dpdk-tests/perf
+ $ grep -E "64B-1t1c-eth-l2xcbase-testpmd-ndrdisc" *
- 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-ndrdisc
+ 10ge2p1x520-eth-l2xcbase-testpmd-ndrpdrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-testpmd-ndrdisc
+ 10ge2p1x710-eth-l2xcbase-testpmd-ndrpdrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-testpmd-ndrdisc
+ 40ge2p1xl710-eth-l2xcbase-testpmd-ndrpdrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-testpmd-ndrdisc
Testpmd packet latency - running in configuration of **two worker threads (2t)
on two physical cores (2c)** - is presented in the figure below.
@@ -53,8 +64,10 @@ git repository by filtering with specified regex as follows:
.. code-block:: bash
- $ csit/tests/perf
- $ grep -E "64B-2t2c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc" *
+ $ $CSIT/dpdk-tests/perf
+ $ grep -E "64B-2t2c-eth-l2xcbase-testpmd-ndrdisc" *
- 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-ndrdisc
+ 10ge2p1x520-eth-l2xcbase-testpmd-ndrpdrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-testpmd-ndrdisc
+ 10ge2p1x710-eth-l2xcbase-testpmd-ndrpdrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-testpmd-ndrdisc
+ 40ge2p1xl710-eth-l2xcbase-testpmd-ndrpdrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-testpmd-ndrdisc
diff --git a/docs/report/testpmd_performance_tests/packet_throughput_graphs/l2.rst b/docs/report/testpmd_performance_tests/packet_throughput_graphs/l2.rst
index 0afb54792d..3601384c99 100644
--- a/docs/report/testpmd_performance_tests/packet_throughput_graphs/l2.rst
+++ b/docs/report/testpmd_performance_tests/packet_throughput_graphs/l2.rst
@@ -1,29 +1,38 @@
-L2 Ethernet Looping
-===================
-
-Following sections provide a summary of VPP Phy-to-Phy L2 Ethernet Looping
-performance illustrating NDR throughput (zero packet loss) and PDR throughput
-(<0.5% packet loss). Performance is reported for Testpmd running in multiple
-Testpmd worker thread (a.k.a. VPP data plane thread) and physical CPU core
-configurations.
-
-*Title of each graph* is a regex (regular expression) matching all plotted
-throughput test cases, *X-axis labels* are indeces of csit-dpdk-perf-1704 jobs
-that created result output files used as data sources for the graph,
-*Y-axis labels* are measured Packets Per Second [pps] values, and the *graph
-legend* identifes the plotted test suites.
+L2 Ethernet Interface Loop
+==========================
+
+Following sections include summary graphs of DPDK Testpmd Phy-to-Phy performance
+with L2 Ethernet Interface Loop, including NDR throughput (zero packet loss)
+and PDR throughput (<0.5% packet loss). Performance is reported for Testpmd
+running in multiple configurations of Testpmd pmd thread(s), a.k.a. Testpmd
+data plane thread(s), and their physical CPU core(s) placement.
+
+Results are generated by multiple executions of the same CSIT tests.
+In order to display variation in measured throughput values, Box-and-
+whisker plots are used to show their quartiles (Min, 1st quartile / 25th
+percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / 75th
+percentile, Max). Outliers are plotted as individual points.
+
+*Title of each graph* is a regex (regular expression) matching all
+throughput test cases plotted on this graph, *X-axis labels* are indices
+of individual test suites executed by csit-dpdk-perf-1704-all jobs that
+created result output files used as data sources for the graph, *Y-axis
+labels* are measured Packets Per Second [pps] values, and the *Graph
+legend* lists the plotted test suites and their indices.
.. note::
- Sources of test results data: i) FD.io executor jobs `csit-dpdk-perf-1704-all
- <https://jenkins.fd.io/view/csit/job/csit-dpdk-perf-1704-all/>`_ ,
- ii) FD.io jobs archieved result files.
+ Test results have been generated by FD.io test executor jobs
+ `csit-dpdk-perf-1704-all
+ <https://jenkins.fd.io/view/csit/job/csit-dpdk-perf-1704-all/>`_,
+ with Robot Framework result files csit-dpdk-perf-1704-all-<id>.zip
+ `archived here <../../_static/archive/>`_
NDR Throughput
~~~~~~~~~~~~~~
-NDR throughput of Testpmd - running in configuration of **one worker thread
-(1t) on one physical core (1c)** - is presented in the figure below.
+Testpmd NDR 64B packet throughput in 1t1c setup (1thread, 1core) is presented
+in the graph below.
.. raw:: html
@@ -32,18 +41,20 @@ NDR throughput of Testpmd - running in configuration of **one worker thread
*Figure 1. Testpmd 1thread 1core - NDR Throughput for Phy-to-Phy L2 Ethernet
Looping.*
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
+CSIT source code for the test cases used for above plots can be found in CSIT
+git repository:
.. code-block:: bash
- $ csit/dpdk-tests/perf
- $ grep -E "64B-1t1c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc" *
+ $ $CSIT/dpdk-tests/perf
+ $ grep -E "64B-1t1c-eth-l2xcbase-testpmd-ndrdisc" *
- 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-ndrdisc
+ 10ge2p1x520-eth-l2xcbase-testpmd-ndrpdrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-testpmd-ndrdisc
+ 10ge2p1x710-eth-l2xcbase-testpmd-ndrpdrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-testpmd-ndrdisc
+ 40ge2p1xl710-eth-l2xcbase-testpmd-ndrpdrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-testpmd-ndrdisc
-NDR throughput of Testpmd - running in configuration of **two worker threads
-(2t) on two physical cores (2c)** - is presented in the figure below.
+Testpmd NDR 64B packet throughput in 2t2c setup (2thread, 2core) is presented
+in the graph below.
.. raw:: html
@@ -52,22 +63,23 @@ NDR throughput of Testpmd - running in configuration of **two worker threads
*Figure 2. Testpmd 2threads 2cores - NDR Throughput for Phy-to-Phy L2 Ethernet
Looping.*
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
+CSIT source code for the test cases used for above plots can be found in CSIT
+git repository:
.. code-block:: bash
- $ csit/dpdk-tests/perf
- $ grep -E "64B-2t2c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc" *
+ $ $CSIT/dpdk-tests/perf
+ $ grep -E "64B-2t2c-eth-l2xcbase-testpmd-ndrdisc" *
- 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-ndrdisc
+ 10ge2p1x520-eth-l2xcbase-testpmd-ndrpdrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-testpmd-ndrdisc
+ 10ge2p1x710-eth-l2xcbase-testpmd-ndrpdrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-testpmd-ndrdisc
+ 40ge2p1xl710-eth-l2xcbase-testpmd-ndrpdrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-testpmd-ndrdisc
PDR Throughput
~~~~~~~~~~~~~~
-PDR throughput of Testpmd - running in configuration of **one worker thread
-(1t) on one physical core (1c)** - is presented in the figure below. PDR at
-below 0.5% packet loss ratio.
+Testpmd PDR 64B packet throughput in 1t1c setup (1thread, 1core) is presented
+in the graph below. PDR measured for 0.5% packet loss ratio.
.. raw:: html
@@ -76,18 +88,20 @@ below 0.5% packet loss ratio.
*Figure 3. Testpmd 1thread 1core - PDR Throughput for Phy-to-Phy L2 Ethernet
Looping.*
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
+CSIT source code for the test cases used for above plots can be found in CSIT
+git repository:
.. code-block:: bash
$ csit/tests/perf
- $ grep -E "64B-1t1c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-pdrdisc" *
+ $ grep -E "64B-1t1c-eth-l2xcbase-testpmd-pdrdisc" *
- 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc02-64B-1t1c-eth-l2xcbase-pdrdisc
+ 10ge2p1x520-eth-l2xcbase-testpmd-ndrpdrdisc.robot:| tc02-64B-1t1c-eth-l2xcbase-testpmd-pdrdisc
+ 10ge2p1x710-eth-l2xcbase-testpmd-ndrpdrdisc.robot:| tc02-64B-1t1c-eth-l2xcbase-testpmd-pdrdisc
+ 40ge2p1xl710-eth-l2xcbase-testpmd-ndrpdrdisc.robot:| tc02-64B-1t1c-eth-l2xcbase-testpmd-pdrdisc
-PDR throughput of Testpmd - running in configuration of **two worker threads
-(2t) on two physical cores (2c)** - is presented in the figure below.
+Testpmd PDR 64B packet throughput in 2t2c setup (2thread, 2core) is presented
+in the graph below. PDR measured for 0.5% packet loss ratio.
.. raw:: html
@@ -96,13 +110,15 @@ PDR throughput of Testpmd - running in configuration of **two worker threads
*Figure 4. Testpmd 2thread 2core - PDR Throughput for Phy-to-Phy L2 Ethernet
Looping.*
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
+CSIT source code for the test cases used for above plots can be found in CSIT
+git repository:
.. code-block:: bash
$ csit/tests/perf
- $ grep -E "64B-2t2c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-pdrdisc" *
+ $ grep -E "64B-2t2c-eth-l2xcbase-testpmd-pdrdisc" *
- 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc08-64B-2t2c-eth-l2xcbase-pdrdisc
+ 10ge2p1x520-eth-l2xcbase-testpmd-ndrpdrdisc.robot:| tc08-64B-2t2c-eth-l2xcbase-testpmd-pdrdisc
+ 10ge2p1x710-eth-l2xcbase-testpmd-ndrpdrdisc.robot:| tc08-64B-2t2c-eth-l2xcbase-testpmd-pdrdisc
+ 40ge2p1xl710-eth-l2xcbase-testpmd-ndrpdrdisc.robot:| tc08-64B-2t2c-eth-l2xcbase-testpmd-pdrdisc