diff options
Diffstat (limited to 'docs/report/testpmd_performance_tests')
3 files changed, 98 insertions, 69 deletions
diff --git a/docs/report/testpmd_performance_tests/csit_release_notes.rst b/docs/report/testpmd_performance_tests/csit_release_notes.rst index 6d415a1add..ba78a1bc54 100644 --- a/docs/report/testpmd_performance_tests/csit_release_notes.rst +++ b/docs/report/testpmd_performance_tests/csit_release_notes.rst @@ -6,5 +6,5 @@ Changes in CSIT |release| #. Added Testpmd tests - - new NICs - Intel xl710 + - new NICs - 2p40GE Intel xl710, 2p10GE Intel x710 diff --git a/docs/report/testpmd_performance_tests/packet_latency_graphs/l2.rst b/docs/report/testpmd_performance_tests/packet_latency_graphs/l2.rst index bac622674c..ddb18f42d2 100644 --- a/docs/report/testpmd_performance_tests/packet_latency_graphs/l2.rst +++ b/docs/report/testpmd_performance_tests/packet_latency_graphs/l2.rst @@ -1,24 +1,33 @@ -L2 Ethernet Switching -===================== - -This section provides a summary of Testpmd Phy-to-Phy L2 Ethernet looping -performance illustrating packet latency measured at 50% of discovered NDR -throughput rate. Latency is reported for Testpmd running in multiple -configurations of Testpmd worker thread(s), a.k.a. Testpmd data plane thread -(s), and their physical CPU core(s) placement. - -*Title of each graph* is a regex (regular expression) matching all plotted -latency test cases, *X-axis labels* are indeces of csit-dpdk-perf-1704 jobs -that created result output files used as data sources for the graph, -*Y-axis labels* are measured packet Latency [uSec] values, and the *graph -legend* identifes the plotted test suites. +L2 Ethernet Interface Loop +========================== + +This section includes summary graphs of Testpmd Phy-to-Phy packet +latency with L2 Ethernet Interface Loop measured at 50% of discovered +NDR throughput rate. Latency is reported for Testpmd running in multiple +configurations of Testpmd pmd thread(s), a.k.a. Testpmd data plane +thread(s), and their physical CPU core(s) placement. + +Results are generated from a single execution of CSIT NDR discovery +test. Box plots are used to show the Minimum, Average and Maximum packet +latency per test. + +*Title of each graph* is a regex (regular expression) matching all +throughput test cases plotted on this graph, *X-axis labels* are indices +of individual test suites executed by csit-dpdk-perf-1704-all job that +created result output file used as data source for the graph, *Y-axis +labels* are measured packet Latency [uSec] values, and the *Graph +legend* lists the plotted test suites and their indices. Latency is +reported for concurrent symmetric bi-directional flows, separately for +each direction: i) West-to-East: TGint1-to-SUT1-to-SUT2-to-TGint2, and +ii) East-to-West: TGint2-to-SUT2-to-SUT1-to-TGint1. .. note:: - Data sources for reported test results: i) FD.io test executor jobs - `csit-dpdk-perf-1704-all <https://jenkins.fd.io/view/csit/job/csit-dpdk-perf-1704-all/>`_ , - ii) archived FD.io jobs test result output files - `csit-dpdk-perf-1704-all#job-number <https://nexus.fd.io/link>`_. + Test results have been generated by FD.io test executor jobs + `csit-dpdk-perf-1704-all + <https://jenkins.fd.io/view/csit/job/csit-dpdk-perf-1704-all/>`_, + with Robot Framework result files csit-dpdk-perf-1704-all-<id>.zip + `archived here <../../_static/archive/>`_ Testpmd packet latency - running in configuration of **one worker thread (1t) on one physical core (1c)** - is presented in the figure below. @@ -34,10 +43,12 @@ git repository by filtering with specified regex as follows: .. code-block:: bash - $ csit/tests/perf - $ grep -E "64B-1t1c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc" * + $ $CSIT/dpdk-tests/perf + $ grep -E "64B-1t1c-eth-l2xcbase-testpmd-ndrdisc" * - 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-ndrdisc + 10ge2p1x520-eth-l2xcbase-testpmd-ndrpdrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-testpmd-ndrdisc + 10ge2p1x710-eth-l2xcbase-testpmd-ndrpdrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-testpmd-ndrdisc + 40ge2p1xl710-eth-l2xcbase-testpmd-ndrpdrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-testpmd-ndrdisc Testpmd packet latency - running in configuration of **two worker threads (2t) on two physical cores (2c)** - is presented in the figure below. @@ -53,8 +64,10 @@ git repository by filtering with specified regex as follows: .. code-block:: bash - $ csit/tests/perf - $ grep -E "64B-2t2c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc" * + $ $CSIT/dpdk-tests/perf + $ grep -E "64B-2t2c-eth-l2xcbase-testpmd-ndrdisc" * - 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-ndrdisc + 10ge2p1x520-eth-l2xcbase-testpmd-ndrpdrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-testpmd-ndrdisc + 10ge2p1x710-eth-l2xcbase-testpmd-ndrpdrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-testpmd-ndrdisc + 40ge2p1xl710-eth-l2xcbase-testpmd-ndrpdrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-testpmd-ndrdisc diff --git a/docs/report/testpmd_performance_tests/packet_throughput_graphs/l2.rst b/docs/report/testpmd_performance_tests/packet_throughput_graphs/l2.rst index 0afb54792d..3601384c99 100644 --- a/docs/report/testpmd_performance_tests/packet_throughput_graphs/l2.rst +++ b/docs/report/testpmd_performance_tests/packet_throughput_graphs/l2.rst @@ -1,29 +1,38 @@ -L2 Ethernet Looping -=================== - -Following sections provide a summary of VPP Phy-to-Phy L2 Ethernet Looping -performance illustrating NDR throughput (zero packet loss) and PDR throughput -(<0.5% packet loss). Performance is reported for Testpmd running in multiple -Testpmd worker thread (a.k.a. VPP data plane thread) and physical CPU core -configurations. - -*Title of each graph* is a regex (regular expression) matching all plotted -throughput test cases, *X-axis labels* are indeces of csit-dpdk-perf-1704 jobs -that created result output files used as data sources for the graph, -*Y-axis labels* are measured Packets Per Second [pps] values, and the *graph -legend* identifes the plotted test suites. +L2 Ethernet Interface Loop +========================== + +Following sections include summary graphs of DPDK Testpmd Phy-to-Phy performance +with L2 Ethernet Interface Loop, including NDR throughput (zero packet loss) +and PDR throughput (<0.5% packet loss). Performance is reported for Testpmd +running in multiple configurations of Testpmd pmd thread(s), a.k.a. Testpmd +data plane thread(s), and their physical CPU core(s) placement. + +Results are generated by multiple executions of the same CSIT tests. +In order to display variation in measured throughput values, Box-and- +whisker plots are used to show their quartiles (Min, 1st quartile / 25th +percentile, 2nd quartile / 50th percentile / mean, 3rd quartile / 75th +percentile, Max). Outliers are plotted as individual points. + +*Title of each graph* is a regex (regular expression) matching all +throughput test cases plotted on this graph, *X-axis labels* are indices +of individual test suites executed by csit-dpdk-perf-1704-all jobs that +created result output files used as data sources for the graph, *Y-axis +labels* are measured Packets Per Second [pps] values, and the *Graph +legend* lists the plotted test suites and their indices. .. note:: - Sources of test results data: i) FD.io executor jobs `csit-dpdk-perf-1704-all - <https://jenkins.fd.io/view/csit/job/csit-dpdk-perf-1704-all/>`_ , - ii) FD.io jobs archieved result files. + Test results have been generated by FD.io test executor jobs + `csit-dpdk-perf-1704-all + <https://jenkins.fd.io/view/csit/job/csit-dpdk-perf-1704-all/>`_, + with Robot Framework result files csit-dpdk-perf-1704-all-<id>.zip + `archived here <../../_static/archive/>`_ NDR Throughput ~~~~~~~~~~~~~~ -NDR throughput of Testpmd - running in configuration of **one worker thread -(1t) on one physical core (1c)** - is presented in the figure below. +Testpmd NDR 64B packet throughput in 1t1c setup (1thread, 1core) is presented +in the graph below. .. raw:: html @@ -32,18 +41,20 @@ NDR throughput of Testpmd - running in configuration of **one worker thread *Figure 1. Testpmd 1thread 1core - NDR Throughput for Phy-to-Phy L2 Ethernet Looping.* -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: +CSIT source code for the test cases used for above plots can be found in CSIT +git repository: .. code-block:: bash - $ csit/dpdk-tests/perf - $ grep -E "64B-1t1c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc" * + $ $CSIT/dpdk-tests/perf + $ grep -E "64B-1t1c-eth-l2xcbase-testpmd-ndrdisc" * - 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-ndrdisc + 10ge2p1x520-eth-l2xcbase-testpmd-ndrpdrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-testpmd-ndrdisc + 10ge2p1x710-eth-l2xcbase-testpmd-ndrpdrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-testpmd-ndrdisc + 40ge2p1xl710-eth-l2xcbase-testpmd-ndrpdrdisc.robot:| tc01-64B-1t1c-eth-l2xcbase-testpmd-ndrdisc -NDR throughput of Testpmd - running in configuration of **two worker threads -(2t) on two physical cores (2c)** - is presented in the figure below. +Testpmd NDR 64B packet throughput in 2t2c setup (2thread, 2core) is presented +in the graph below. .. raw:: html @@ -52,22 +63,23 @@ NDR throughput of Testpmd - running in configuration of **two worker threads *Figure 2. Testpmd 2threads 2cores - NDR Throughput for Phy-to-Phy L2 Ethernet Looping.* -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: +CSIT source code for the test cases used for above plots can be found in CSIT +git repository: .. code-block:: bash - $ csit/dpdk-tests/perf - $ grep -E "64B-2t2c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc" * + $ $CSIT/dpdk-tests/perf + $ grep -E "64B-2t2c-eth-l2xcbase-testpmd-ndrdisc" * - 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-ndrdisc + 10ge2p1x520-eth-l2xcbase-testpmd-ndrpdrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-testpmd-ndrdisc + 10ge2p1x710-eth-l2xcbase-testpmd-ndrpdrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-testpmd-ndrdisc + 40ge2p1xl710-eth-l2xcbase-testpmd-ndrpdrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-testpmd-ndrdisc PDR Throughput ~~~~~~~~~~~~~~ -PDR throughput of Testpmd - running in configuration of **one worker thread -(1t) on one physical core (1c)** - is presented in the figure below. PDR at -below 0.5% packet loss ratio. +Testpmd PDR 64B packet throughput in 1t1c setup (1thread, 1core) is presented +in the graph below. PDR measured for 0.5% packet loss ratio. .. raw:: html @@ -76,18 +88,20 @@ below 0.5% packet loss ratio. *Figure 3. Testpmd 1thread 1core - PDR Throughput for Phy-to-Phy L2 Ethernet Looping.* -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: +CSIT source code for the test cases used for above plots can be found in CSIT +git repository: .. code-block:: bash $ csit/tests/perf - $ grep -E "64B-1t1c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-pdrdisc" * + $ grep -E "64B-1t1c-eth-l2xcbase-testpmd-pdrdisc" * - 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc02-64B-1t1c-eth-l2xcbase-pdrdisc + 10ge2p1x520-eth-l2xcbase-testpmd-ndrpdrdisc.robot:| tc02-64B-1t1c-eth-l2xcbase-testpmd-pdrdisc + 10ge2p1x710-eth-l2xcbase-testpmd-ndrpdrdisc.robot:| tc02-64B-1t1c-eth-l2xcbase-testpmd-pdrdisc + 40ge2p1xl710-eth-l2xcbase-testpmd-ndrpdrdisc.robot:| tc02-64B-1t1c-eth-l2xcbase-testpmd-pdrdisc -PDR throughput of Testpmd - running in configuration of **two worker threads -(2t) on two physical cores (2c)** - is presented in the figure below. +Testpmd PDR 64B packet throughput in 2t2c setup (2thread, 2core) is presented +in the graph below. PDR measured for 0.5% packet loss ratio. .. raw:: html @@ -96,13 +110,15 @@ PDR throughput of Testpmd - running in configuration of **two worker threads *Figure 4. Testpmd 2thread 2core - PDR Throughput for Phy-to-Phy L2 Ethernet Looping.* -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: +CSIT source code for the test cases used for above plots can be found in CSIT +git repository: .. code-block:: bash $ csit/tests/perf - $ grep -E "64B-2t2c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-pdrdisc" * + $ grep -E "64B-2t2c-eth-l2xcbase-testpmd-pdrdisc" * - 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc08-64B-2t2c-eth-l2xcbase-pdrdisc + 10ge2p1x520-eth-l2xcbase-testpmd-ndrpdrdisc.robot:| tc08-64B-2t2c-eth-l2xcbase-testpmd-pdrdisc + 10ge2p1x710-eth-l2xcbase-testpmd-ndrpdrdisc.robot:| tc08-64B-2t2c-eth-l2xcbase-testpmd-pdrdisc + 40ge2p1xl710-eth-l2xcbase-testpmd-ndrpdrdisc.robot:| tc08-64B-2t2c-eth-l2xcbase-testpmd-pdrdisc |