From 5d294b7d4c501d971d67c4e3fe192a9dc504d76e Mon Sep 17 00:00:00 2001 From: Maciek Konstantynowicz Date: Thu, 20 Apr 2017 22:42:01 +0100 Subject: CSIT rls1704 report - update to handcraftec sections: - introduction /overview /general_notes /csit_test_naming - vpp_performance_tests /overview /csit_release_notes /test_environment /documentation /xx_graphs - vpp_functional_tests /overview /csit_release_notes /documentation - testpmd_performance_tests /overview /csit_release_notes /test_environment /documentation /xx_graphs Change-Id: I10593d4155edc663ca07ae5cd29eb679c5f5d85f Signed-off-by: Maciek Konstantynowicz --- .../vpp_performance_tests/csit_release_notes.rst | 28 +------ .../report/vpp_performance_tests/documentation.rst | 2 +- docs/report/vpp_performance_tests/overview.rst | 93 ++++++++++------------ .../packet_latency_graphs/ipv4.rst | 36 +-------- .../packet_latency_graphs/ipv4_tunnels.rst | 32 +------- .../packet_latency_graphs/ipv6.rst | 35 +------- .../packet_latency_graphs/ipv6_tunnels.rst | 31 +------- .../packet_latency_graphs/l2.rst | 37 +-------- .../packet_latency_graphs/vm_vhost.rst | 36 +-------- .../packet_throughput_graphs/ipv4.rst | 67 ++-------------- .../packet_throughput_graphs/ipv4_tunnels.rst | 60 ++------------ .../packet_throughput_graphs/ipv6.rst | 66 ++------------- .../packet_throughput_graphs/ipv6_tunnels.rst | 57 ++----------- .../packet_throughput_graphs/l2.rst | 67 ++-------------- .../packet_throughput_graphs/vm_vhost.rst | 61 ++------------ .../vpp_performance_tests/test_environment.rst | 67 ++++++++-------- 16 files changed, 144 insertions(+), 631 deletions(-) (limited to 'docs/report/vpp_performance_tests') diff --git a/docs/report/vpp_performance_tests/csit_release_notes.rst b/docs/report/vpp_performance_tests/csit_release_notes.rst index 54fa73dfee..4c942e19a4 100644 --- a/docs/report/vpp_performance_tests/csit_release_notes.rst +++ b/docs/report/vpp_performance_tests/csit_release_notes.rst @@ -4,12 +4,10 @@ CSIT Release Notes Changes in CSIT |release| ------------------------- -#. Naming change for all VPP performance test suites and test cases. #. VPP performance test environment changes - - upgrade to Ubuntu 16.04 - - VM and vhost-user test environment optimizations - - addition of HW cryptodev devices in LFD FD.io physical testbed + - further VM and vhost-user test environment optimizations - Qemu virtio queue size increased from default value of 256 to 1024 + - addition of HW cryptodev devices in all three LF FD.io physical testbeds #. Added tests @@ -37,7 +35,7 @@ Performance Improvements Substantial improvements in measured packet throughput, with relative increase of double-digit percentage points, have been observed in a number of CSIT |release| tests listed below. Relative improvements are calculated against the -test results in CSIT rls1609 report. +test results in CSIT rls1701 report. NDR Throughput ~~~~~~~~~~~~~~ @@ -119,7 +117,7 @@ Other Performance Changes Other changes in measured packet throughput, with either minor relative increase or decrease, have been observed in a number of CSIT |release| tests listed below. Relative changes are calculated against the test results in CSIT -rls1609 report. +rls1701 report. NDR Throughput ~~~~~~~~~~~~~~ @@ -208,21 +206,3 @@ Here is the list of known issues in CSIT |release| for VPP performance tests: | | - 2t2c, 4t4c - for some tested NICs. | | +---+-------------------------------------------------+-----------------------------------------------------------------+ -Tests to be Added - Cryptodev ------------------------------ - -DPDK Cryptodev functionality support for both SW and HW crypto devices has -been introduced in VPP-17.01 release. CSIT functional and performance tests -have been also developed and merged. However due to the factors beyond CSIT -project control execution of those tests within the LF FD.io test environment -still need to complete. Once the results become available, they will be -published as an addendum to the current version of CSIT |release| report. - -Tests to be Added - SNAT44 --------------------------- - -VPP SNAT44 functionality has been introduced in VPP-17.01 release. CSIT -performance tests are still in development and integration into LFD FD.io test -environment. Once the tests are fully integrated and results become available, -they will be published as an addendum to the current version of CSIT |release| -report. diff --git a/docs/report/vpp_performance_tests/documentation.rst b/docs/report/vpp_performance_tests/documentation.rst index 5017580ca0..b282c2a9b6 100644 --- a/docs/report/vpp_performance_tests/documentation.rst +++ b/docs/report/vpp_performance_tests/documentation.rst @@ -2,5 +2,5 @@ Documentation ============= `CSIT VPP Performance Tests Documentation -`_ contains detailed +`_ contains detailed functional description and input parameters for each test case. diff --git a/docs/report/vpp_performance_tests/overview.rst b/docs/report/vpp_performance_tests/overview.rst index d0ce5e7bcf..0a5525a6d8 100644 --- a/docs/report/vpp_performance_tests/overview.rst +++ b/docs/report/vpp_performance_tests/overview.rst @@ -52,53 +52,56 @@ Going forward CSIT project will be looking to add more hardware into FD.io performance labs to address larger scale multi-interface and multi-NIC performance testing scenarios. -For test cases that require DUT (VPP) to communicate with VM over vhost-user -interfaces, a VM is created on SUT1 and SUT2. DUT (VPP) test topology with VM -is shown in the figure below including applicable packet flow thru the VM +For test cases that require DUT (VPP) to communicate with VM(s) over vhost-user +interfaces, N of VM instances are created on SUT1 and SUT2. For N=1 DUT (VPP) forwards packets between vhostuser and physical interfaces. For N>1 DUT (VPP) a logical service chain forwarding topology is created on DUT (VPP) by applying L2 or IPv4/IPv6 configuration depending on the test suite. +DUT (VPP) test topology with N VM instances +is shown in the figure below including applicable packet flow thru the DUTs and VMs (marked in the figure with ``***``). :: - +------------------------+ +------------------------+ - | +----------+ | | +----------+ | - | | VM | | | | VM | | - | | ****** | | | | ****** | | - | +--^----^--+ | | +--^----^--+ | - | *| |* | | *| |* | - | +------v----v------+ | | +------v----v------+ | - | | * * |**|***********|**| * * | | - | | ***** *******<----------------->******* ***** | | - | | * DUT1 | | | | DUT2 * | | - | +--^---------------+ | | +---------------^--+ | - | *| | | |* | - | *| SUT1 | | SUT2 |* | - +------------------------+ +------------------^-----+ - *| |* - *| |* - *| +-----------+ |* - *| | | |* - *+------------------> TG <------------------+* - ******************* | |******************** - +-----------+ - -For VM tests, packets are switched by DUT (VPP) twice, hence the -throughput rates measured by TG (and listed in this report) must be multiplied -by two to represent the actual DUT aggregate packet forwarding rate. - -Note that reported VPP performance results are specific to the SUT tested. + +-------------------------+ +-------------------------+ + | +---------+ +---------+ | | +---------+ +---------+ | + | | VM[1] | | VM[N] | | | | VM[1] | | VM[N] | | + | | ***** | | ***** | | | | ***** | | ***** | | + | +--^---^--+ +--^---^--+ | | +--^---^--+ +--^---^--+ | + | *| |* *| |* | | *| |* *| |* | + | +--v---v-------v---v--+ | | +--v---v-------v---v--+ | + | | * * * * |*|***********|*| * * * * | | + | | * ********* ***<-|-----------|->*** ********* * | | + | | * DUT1 | | | | DUT2 * | | + | +--^------------------+ | | +------------------^--+ | + | *| | | |* | + | *| SUT1 | | SUT2 |* | + +-------------------------+ +-------------------------+ + *| |* + *| |* + *| +-----------+ |* + *| | | |* + *+--------------------> TG <--------------------+* + **********************| |********************** + +-----------+ + +For VM tests, packets are switched by DUT (VPP) multiple times: twice for a single VM, three times for two VMs, N+1 times for N VMs. +Hence the external +throughput rates measured by TG and listed in this report must be multiplied +by (N+1) to represent the actual DUT aggregate packet forwarding rate. + +CSIT |release| + +Note that reported VPP performance results are specific to the SUTs tested. Current LF FD.io SUTs are based on Intel XEON E5-2699v3 2.3GHz CPUs. SUTs with other CPUs are likely to yield different results. A good rule of thumb, that can be applied to estimate VPP packet thoughput for Phy-to-Phy (NIC-to-NIC, PCI-to-PCI) topology, is to expect the forwarding performance to be proportional to CPU core frequency, assuming CPU is the only limiting factor -and all other SUT aspects equal to FD.io CSIT environment. The same rule of +and all other SUT parameters equivalent to FD.io CSIT environment. The same rule of thumb can be also applied for Phy-to-VM-to-Phy (NIC-to-VM-to-NIC) topology, -but due to much higher dependency on very high frequency memory operations and +but due to much higher dependency on intensive memory operations and sensitivity to Linux kernel scheduler settings and behaviour, this estimation may not always yield good enough accuracy. -Detailed LF FD.io test bed specification and physical topology are described -in `wiki CSIT LF FDio testbed `_. +For detailed LF FD.io test bed specification and physical topology please refer to `LF FDio CSIT testbed wiki page `_. Performance Tests Coverage -------------------------- @@ -109,10 +112,10 @@ Performance tests are split into the two main categories: in accordance to RFC2544. - NDR - discovery of Non Drop Rate packet throughput, at zero packet loss; - followed by packet one-way latency measurements at 10%, 50% and 100% of + followed by one-way packet latency measurements at 10%, 50% and 100% of discovered NDR throughput. - PDR - discovery of Partial Drop Rate, with specified non-zero packet loss - currently set to 0.5%; followed by packet one-way latency measurements at + currently set to 0.5%; followed by one-way packet latency measurements at 100% of discovered PDR throughput. - Throughput verification - verification of packet forwarding rate against @@ -180,17 +183,10 @@ continuously. Performance Tests Naming ------------------------ -CSIT |release| introduced a common structured naming convention for all -performance and functional tests. This change was driven by substantially -growing number and type of CSIT test cases. Firstly, the original practice did -not always follow any strict naming convention. Secondly test names did not -always clearly capture tested packet encapsulations, and the actual type or -content of the tests. Thirdly HW configurations in terms of NICs, ports and -their locality were not captured either. These were but few reasons that drove -the decision to change and define a new more complete and stricter test naming -convention, and to apply this to all existing and new test cases. - -The new naming should be intuitive for majority of the tests. The complete +CSIT |release| follows a common structured naming convention for all +performance and system functional tests, introduced in CSIT rls1701. + +The naming should be intuitive for majority of the tests. Complete description of CSIT test naming convention is provided on `CSIT test naming wiki `_. @@ -247,9 +243,8 @@ following VPP thread and core configurations: #. 1t1c - 1 VPP worker thread on 1 CPU physical core. #. 2t2c - 2 VPP worker threads on 2 CPU physical cores. -#. 4t4c - 4 VPP threads on 4 CPU physical cores. -Note that in quite a few test cases running VPP on 2 or 4 physical cores hits +Note that in quite a few test cases running VPP on 2 physical cores hits the tested NIC I/O bandwidth or packets-per-second limit. Methodology: Packet Throughput diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4.rst index 6473422316..dffd2a2f78 100644 --- a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4.rst +++ b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4.rst @@ -8,7 +8,7 @@ configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and their physical CPU core(s) placement. *Title of each graph* is a regex (regular expression) matching all plotted -latency test cases, *X-axis labels* are indeces of csit-vpp-perf-1701 jobs +latency test cases, *X-axis labels* are indeces of csit-vpp-perf-1704 jobs that created result output files used as data sources for the graph, *Y-axis labels* are measured packet Latency [uSec] values, and the *graph legend* identifes the plotted test suites. @@ -16,11 +16,9 @@ legend* identifes the plotted test suites. .. note:: Data sources for reported test results: i) FD.io test executor jobs - `csit-vpp-perf-1701-all - `_ and - `csit-vpp-perf-1701-long - `_ - , ii) archived FD.io jobs test result `output files + `csit-vpp-perf-1704-all + `_ , + ii) archived FD.io jobs test result `output files <../../_static/archive/>`_. VPP packet latency - running in configuration of **one worker thread (1t) on one @@ -75,29 +73,3 @@ git repository by filtering with specified regex as follows: 10ge2p1x520-ethip4-ip4scale2m-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4scale2m-ndrdisc 40ge2p1xl710-ethip4-ip4base-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4base-ndrdisc -VPP packet latency - running in configuration of **four worker threads (4t) on four -physical cores (4c)** - is presented in the figure below. - -.. raw:: html - - - -*Figure 3. VPP 4threads 4cores - packet latency for Phy-to-Phy IPv4 Routed-Forwarding.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. code-block:: bash - - $ csit/tests/perf - $ grep -E "64B-4t4c-ethip4-ip4[a-z0-9]+-[a-z-]*ndrdisc" * - - 10ge2p1x520-ethip4-ip4base-copwhtlistbase-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-copwhtlistbase-ndrdisc - 10ge2p1x520-ethip4-ip4base-iacldstbase-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-iacldstbase-ndrdisc - 10ge2p1x520-ethip4-ip4base-ipolicemarkbase-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-ipolicemarkbase-ndrdisc - 10ge2p1x520-ethip4-ip4base-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-ndrdisc - 10ge2p1x520-ethip4-ip4scale200k-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4scale200k-ndrdisc - 10ge2p1x520-ethip4-ip4scale20k-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4scale20k-ndrdisc - 10ge2p1x520-ethip4-ip4scale2m-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4scale2m-ndrdisc - 40ge2p1xl710-ethip4-ip4base-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-ndrdisc - diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4_tunnels.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4_tunnels.rst index a41351403f..decef07375 100644 --- a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4_tunnels.rst +++ b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4_tunnels.rst @@ -8,7 +8,7 @@ configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and their physical CPU core(s) placement. *Title of each graph* is a regex (regular expression) matching all plotted -latency test cases, *X-axis labels* are indeces of csit-vpp-perf-1701 jobs +latency test cases, *X-axis labels* are indeces of csit-vpp-perf-1704 jobs that created result output files used as data sources for the graph, *Y-axis labels* are measured packet Latency [uSec] values, and the *graph legend* identifes the plotted test suites. @@ -16,11 +16,9 @@ legend* identifes the plotted test suites. .. note:: Data sources for reported test results: i) FD.io test executor jobs - `csit-vpp-perf-1701-all - `_ and - `csit-vpp-perf-1701-long - `_ - , ii) archived FD.io jobs test result `output files + `csit-vpp-perf-1704-all + `_ , + ii) archived FD.io jobs test result `output files <../../_static/archive/>`_. VPP packet latency - running in configuration of **one worker thread (1t) on one @@ -67,25 +65,3 @@ git repository by filtering with specified regex as follows: 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrdisc.robot:| tc07-64B-2t2c-ethip4vxlan-l2bdbasemaclrn-ndrdisc 10ge2p1x520-ethip4vxlan-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-ethip4vxlan-l2xcbase-ndrdisc -VPP packet latency - running in configuration of **four worker threads (4t) on four -physical cores (4c)** - is presented in the figure below. - -.. raw:: html - - - -*Figure 3. VPP 4threads 4cores - packet latency for Phy-to-Phy IPv4 Overlay Tunnels.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. code-block:: bash - - $ csit/tests/perf - $ grep -E "64B-4t4c-ethip4[a-z0-9]+-[a-z0-9]*-ndrdisc" * - - 10ge2p1x520-ethip4lispip4-ip4base-ndrdisc.robot:| tc13-64B-4t4c-ethip4lispip4-ip4base-ndrdisc - 10ge2p1x520-ethip4lispip6-ip4base-ndrdisc.robot:| tc13-64B-4t4c-ethip4lispip6-ip4base-ndrdisc - 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-ethip4vxlan-l2bdbasemaclrn-ndrdisc - 10ge2p1x520-ethip4vxlan-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-ethip4vxlan-l2xcbase-ndrdisc - diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6.rst index 762b427349..dd06399861 100644 --- a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6.rst +++ b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6.rst @@ -8,7 +8,7 @@ configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and their physical CPU core(s) placement. *Title of each graph* is a regex (regular expression) matching all plotted -latency test cases, *X-axis labels* are indeces of csit-vpp-perf-1701 jobs +latency test cases, *X-axis labels* are indeces of csit-vpp-perf-1704 jobs that created result output files used as data sources for the graph, *Y-axis labels* are measured packet Latency [uSec] values, and the *graph legend* identifes the plotted test suites. @@ -16,11 +16,9 @@ legend* identifes the plotted test suites. .. note:: Data sources for reported test results: i) FD.io test executor jobs - `csit-vpp-perf-1701-all - `_ and - `csit-vpp-perf-1701-long - `_ - , ii) archived FD.io jobs test result `output files + `csit-vpp-perf-1704-all + `_ , + ii) archived FD.io jobs test result `output files <../../_static/archive/>`_. VPP packet latency - running in configuration of **one worker thread (1t) on one @@ -74,28 +72,3 @@ git repository by filtering with specified regex as follows: 10ge2p1x520-ethip6-ip6scale2m-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6scale2m-ndrdisc 40ge2p1xl710-ethip6-ip6base-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6base-ndrdisc -VPP packet latency - running in configuration of **four worker threads (4t) on four -physical cores (4c)** - is presented in the figure below. - -.. raw:: html - - - -*Figure 3. VPP 4threads 4cores - packet latency for Phy-to-Phy IPv6 Routed-Forwarding.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. code-block:: bash - - $ csit/tests/perf - $ grep -E "78B-4t4c-ethip6-ip6[a-z0-9]+-[a-z-]*ndrdisc" * - - 10ge2p1x520-ethip6-ip6base-copwhtlistbase-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6base-copwhtlistbase-ndrdisc - 10ge2p1x520-ethip6-ip6base-iacldstbase-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6base-iacldstbase-ndrdisc - 10ge2p1x520-ethip6-ip6base-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6base-ndrdisc - 10ge2p1x520-ethip6-ip6scale200k-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6scale200k-ndrdisc - 10ge2p1x520-ethip6-ip6scale20k-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6scale20k-ndrdisc - 10ge2p1x520-ethip6-ip6scale2m-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6scale2m-ndrdisc - 40ge2p1xl710-ethip6-ip6base-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6base-ndrdisc - diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6_tunnels.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6_tunnels.rst index 1b9e5626b6..261f2dfbd4 100644 --- a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6_tunnels.rst +++ b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6_tunnels.rst @@ -8,7 +8,7 @@ configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and their physical CPU core(s) placement. *Title of each graph* is a regex (regular expression) matching all plotted -latency test cases, *X-axis labels* are indeces of csit-vpp-perf-1701 jobs +latency test cases, *X-axis labels* are indeces of csit-vpp-perf-1704 jobs that created result output files used as data sources for the graph, *Y-axis labels* are measured packet Latency [uSec] values, and the *graph legend* identifes the plotted test suites. @@ -16,11 +16,9 @@ legend* identifes the plotted test suites. .. note:: Data sources for reported test results: i) FD.io test executor jobs - `csit-vpp-perf-1701-all - `_ and - `csit-vpp-perf-1701-long - `_ - , ii) archived FD.io jobs test result `output files + `csit-vpp-perf-1704-all + `_ , + ii) archived FD.io jobs test result `output files <../../_static/archive/>`_. VPP packet latency - running in configuration of **one worker thread (1t) on one @@ -63,24 +61,3 @@ git repository by filtering with specified regex as follows: 10ge2p1x520-ethip6lispip4-ip6base-ndrdisc.robot:| tc07-78B-2t2c-ethip6lispip4-ip6base-ndrdisc 10ge2p1x520-ethip6lispip6-ip6base-ndrdisc.robot:| tc07-78B-2t2c-ethip6lispip6-ip6base-ndrdisc -VPP packet latency - running in configuration of **four worker threads (4t) on four -physical cores (4c)** - is presented in the figure below. - -.. raw:: html - - - -*Figure 3. VPP 4threads 4cores - packet latency for Phy-to-Phy IPv6 Overlay Tunnels.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. code-block:: bash - - $ csit/tests/perf - $ grep -E "78B-4t4c-ethip6[a-z0-9]+-[a-z0-9]*-ndrdisc" * - - 10ge2p1x520-ethip6lispip4-ip6base-ndrdisc.robot:| tc13-78B-4t4c-ethip6lispip4-ip6base-ndrdisc - 10ge2p1x520-ethip6lispip6-ip6base-ndrdisc.robot:| tc13-78B-4t4c-ethip6lispip6-ip6base-ndrdisc - - diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/l2.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/l2.rst index 8354ac5582..98b5a8deec 100644 --- a/docs/report/vpp_performance_tests/packet_latency_graphs/l2.rst +++ b/docs/report/vpp_performance_tests/packet_latency_graphs/l2.rst @@ -8,7 +8,7 @@ configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and their physical CPU core(s) placement. *Title of each graph* is a regex (regular expression) matching all plotted -latency test cases, *X-axis labels* are indeces of csit-vpp-perf-1701 jobs +latency test cases, *X-axis labels* are indeces of csit-vpp-perf-1704 jobs that created result output files used as data sources for the graph, *Y-axis labels* are measured packet Latency [uSec] values, and the *graph legend* identifes the plotted test suites. @@ -16,11 +16,9 @@ legend* identifes the plotted test suites. .. note:: Data sources for reported test results: i) FD.io test executor jobs - `csit-vpp-perf-1701-all - `_ and - `csit-vpp-perf-1701-long - `_ - , ii) archived FD.io jobs test result `output files + `csit-vpp-perf-1704-all + `_ , + ii) archived FD.io jobs test result `output files <../../_static/archive/>`_. VPP packet latency - running in configuration of **one worker thread (1t) on one @@ -77,30 +75,3 @@ git repository by filtering with specified regex as follows: 40ge2p1xl710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-ndrdisc 40ge2p1xl710-eth-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-ndrdisc -VPP packet latency - running in configuration of **four worker threads (4t) on four -physical cores (4c)** - is presented in the figure below. - -.. raw:: html - - - -*Figure 3. VPP 4threads 4cores - packet latency for Phy-to-Phy L2 Ethernet Switching.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. code-block:: bash - - $ csit/tests/perf - $ grep -E "64B-4t4c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc" * - - 10ge2p1vic1227-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc - 10ge2p1x520-dot1ad-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-dot1ad-l2xcbase-ndrdisc - 10ge2p1x520-dot1q-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-dot1q-l2xcbase-ndrdisc - 10ge2p1x520-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc - 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-ndrdisc - 10ge2p1x710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc - 40ge2p1vic1385-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc - 40ge2p1xl710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc - 40ge2p1xl710-eth-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-ndrdisc - diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/vm_vhost.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/vm_vhost.rst index 9c070fa3d4..0afd357609 100644 --- a/docs/report/vpp_performance_tests/packet_latency_graphs/vm_vhost.rst +++ b/docs/report/vpp_performance_tests/packet_latency_graphs/vm_vhost.rst @@ -8,7 +8,7 @@ configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and their physical CPU core(s) placement. *Title of each graph* is a regex (regular expression) matching all plotted -latency test cases, *X-axis labels* are indeces of csit-vpp-perf-1701 jobs +latency test cases, *X-axis labels* are indeces of csit-vpp-perf-1704 jobs that created result output files used as data sources for the graph, *Y-axis labels* are measured packet Latency [uSec] values, and the *graph legend* identifes the plotted test suites. @@ -16,11 +16,9 @@ legend* identifes the plotted test suites. .. note:: Data sources for reported test results: i) FD.io test executor jobs - `csit-vpp-perf-1701-all - `_ and - `csit-vpp-perf-1701-long - `_ - , ii) archived FD.io jobs test result `output files + `csit-vpp-perf-1704-all + `_ , + ii) archived FD.io jobs test result `output files <../../_static/archive/>`_. VPP packet latency - running in configuration of **one worker thread (1t) on one @@ -75,29 +73,3 @@ git repository by filtering with specified regex as follows: 10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc 40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc -VPP packet latency - running in configuration of **four worker threads (4t) on four -physical cores (4c)** - is presented in the figure below. - -.. raw:: html - - - -*Figure 3. VPP 4threads 4cores - packet latency for Phy-to-VM-to-Phy VM vhost-user.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. code-block:: bash - - $ csit/tests/perf - $ grep -E "64B-4t4c-.*vhost.*-ndrdisc" * - - 10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc - 10ge2p1x520-dot1q-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc - 10ge2p1x520-ethip4-ip4base-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-eth-2vhost-1vm-ndrdisc - 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc - 10ge2p1x520-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc - 10ge2p1x520-eth-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc - 10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc - 40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc - diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4.rst index 169d07a332..c5cce0d274 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4.rst @@ -8,7 +8,7 @@ configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and their physical CPU core(s) placement. *Title of each graph* is a regex (regular expression) matching all plotted -throughput test cases, *X-axis labels* are indeces of csit-vpp-perf-1701 jobs +throughput test cases, *X-axis labels* are indeces of csit-vpp-perf-1704 jobs that created result output files used as data sources for the graph, *Y-axis labels* are measured Packets Per Second [pps] values, and the *graph legend* identifes the plotted test suites. @@ -16,11 +16,9 @@ legend* identifes the plotted test suites. .. note:: Data sources for reported test results: i) FD.io test executor jobs - `csit-vpp-perf-1701-all - `_ and - `csit-vpp-perf-1701-long - `_ - , ii) archived FD.io jobs test result `output files + `csit-vpp-perf-1704-all + `_ , + ii) archived FD.io jobs test result `output files <../../_static/archive/>`_. NDR Throughput @@ -79,33 +77,6 @@ git repository by filtering with specified regex as follows: 10ge2p1x520-ethip4-ip4scale2m-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4scale2m-ndrdisc 40ge2p1xl710-ethip4-ip4base-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4base-ndrdisc -VPP NDR Throughput - running in configuration of **four worker threads (4t) on -four physical cores (4c)** - is presented in the figure below. - -.. raw:: html - - - -*Figure 3. VPP 4threads 4cores - NDR Throughput for Phy-to-Phy IPv4 -Routed-Forwarding.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. code-block:: bash - - $ csit/tests/perf - $ grep -E "64B-4t4c-ethip4-ip4[a-z0-9]+-[a-z-]*ndrdisc" * - - 10ge2p1x520-ethip4-ip4base-copwhtlistbase-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-copwhtlistbase-ndrdisc - 10ge2p1x520-ethip4-ip4base-iacldstbase-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-iacldstbase-ndrdisc - 10ge2p1x520-ethip4-ip4base-ipolicemarkbase-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-ipolicemarkbase-ndrdisc - 10ge2p1x520-ethip4-ip4base-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-ndrdisc - 10ge2p1x520-ethip4-ip4scale200k-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4scale200k-ndrdisc - 10ge2p1x520-ethip4-ip4scale20k-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4scale20k-ndrdisc - 10ge2p1x520-ethip4-ip4scale2m-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4scale2m-ndrdisc - 40ge2p1xl710-ethip4-ip4base-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-ndrdisc - PDR Throughput ~~~~~~~~~~~~~~ @@ -117,7 +88,7 @@ packet loss ratio. -*Figure 4. VPP 1thread 1core - PDR Throughput for Phy-to-Phy IPv4 +*Figure 3. VPP 1thread 1core - PDR Throughput for Phy-to-Phy IPv4 Routed-Forwarding.* CSIT test cases used to generate results presented above can be found in CSIT @@ -143,7 +114,7 @@ two physical cores (2c)** - is presented in the figure below. -*Figure 5. VPP 2thread 2core - PDR Throughput for Phy-to-Phy IPv4 +*Figure 4. VPP 2thread 2core - PDR Throughput for Phy-to-Phy IPv4 Routed-Forwarding.* CSIT test cases used to generate results presented above can be found in CSIT @@ -162,29 +133,3 @@ git repository by filtering with specified regex as follows: 10ge2p1x520-ethip4-ip4scale20k-ndrdisc.robot:| tc08-64B-2t2c-ethip4-ip4scale20k-pdrdisc 10ge2p1x520-ethip4-ip4scale2m-ndrdisc.robot:| tc08-64B-2t2c-ethip4-ip4scale2m-pdrdisc -VPP PDR Throughput - running in configuration of **four worker threads (4t) on -four physical cores (4c)** - is presented in the figure below. - -.. raw:: html - - - -*Figure 6. VPP 4thread 4core - PDR Throughput for Phy-to-Phy IPv4 -Routed-Forwarding.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. code-block:: bash - - $ csit/tests/perf - $ grep -E "64B-4t4c-ethip4-ip4[a-z0-9]+-[a-z-]*pdrdisc" * - - 10ge2p1x520-ethip4-ip4base-copwhtlistbase-ndrdisc.robot:| tc14-64B-4t4c-ethip4-ip4base-copwhtlistbase-pdrdisc - 10ge2p1x520-ethip4-ip4base-iacldstbase-ndrdisc.robot:| tc14-64B-4t4c-ethip4-ip4base-iacldstbase-pdrdisc - 10ge2p1x520-ethip4-ip4base-ipolicemarkbase-ndrdisc.robot:| tc14-64B-4t4c-ethip4-ip4base-ipolicemarkbase-pdrdisc - 10ge2p1x520-ethip4-ip4base-ndrdisc.robot:| tc14-64B-4t4c-ethip4-ip4base-pdrdisc - 10ge2p1x520-ethip4-ip4scale200k-ndrdisc.robot:| tc14-64B-4t4c-ethip4-ip4scale200k-pdrdisc - 10ge2p1x520-ethip4-ip4scale20k-ndrdisc.robot:| tc14-64B-4t4c-ethip4-ip4scale20k-pdrdisc - 10ge2p1x520-ethip4-ip4scale2m-ndrdisc.robot:| tc14-64B-4t4c-ethip4-ip4scale2m-pdrdisc - diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4_tunnels.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4_tunnels.rst index 131e4e45e2..374505e0db 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4_tunnels.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4_tunnels.rst @@ -8,7 +8,7 @@ configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and their physical CPU core(s) placement. *Title of each graph* is a regex (regular expression) matching all plotted -throughput test cases, *X-axis labels* are indeces of csit-vpp-perf-1701 jobs +throughput test cases, *X-axis labels* are indeces of csit-vpp-perf-1704 jobs that created result output files used as data sources for the graph, *Y-axis labels* are measured Packets Per Second [pps] values, and the *graph legend* identifes the plotted test suites. @@ -16,11 +16,9 @@ legend* identifes the plotted test suites. .. note:: Data sources for reported test results: i) FD.io test executor jobs - `csit-vpp-perf-1701-all - `_ and - `csit-vpp-perf-1701-long - `_ - , ii) archived FD.io jobs test result `output files + `csit-vpp-perf-1704-all + `_ , + ii) archived FD.io jobs test result `output files <../../_static/archive/>`_. NDR Throughput @@ -71,29 +69,6 @@ git repository by filtering with specified regex as follows: 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrdisc.robot:| tc07-64B-2t2c-ethip4vxlan-l2bdbasemaclrn-ndrdisc 10ge2p1x520-ethip4vxlan-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-ethip4vxlan-l2xcbase-ndrdisc -VPP NDR Throughput - running in configuration of **four worker threads (4t) on -four physical cores (4c)** - is presented in the figure below. - -.. raw:: html - - - -*Figure 3. VPP 4threads 4cores - NDR Throughput for Phy-to-Phy IPv4 Overlay -Tunnels.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. code-block:: bash - - $ csit/tests/perf - $ grep -E "64B-4t4c-ethip4[a-z0-9]+-[a-z0-9]*-ndrdisc" * - - 10ge2p1x520-ethip4lispip4-ip4base-ndrdisc.robot:| tc13-64B-4t4c-ethip4lispip4-ip4base-ndrdisc - 10ge2p1x520-ethip4lispip6-ip4base-ndrdisc.robot:| tc13-64B-4t4c-ethip4lispip6-ip4base-ndrdisc - 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-ethip4vxlan-l2bdbasemaclrn-ndrdisc - 10ge2p1x520-ethip4vxlan-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-ethip4vxlan-l2xcbase-ndrdisc - PDR Throughput ~~~~~~~~~~~~~~ @@ -105,7 +80,7 @@ packet loss ratio. -*Figure 4. VPP 1thread 1core - PDR Throughput for Phy-to-Phy IPv4 Overlay +*Figure 3. VPP 1thread 1core - PDR Throughput for Phy-to-Phy IPv4 Overlay Tunnels.* CSIT test cases used to generate results presented above can be found in CSIT @@ -128,7 +103,7 @@ two physical cores (2c)** - is presented in the figure below. -*Figure 5. VPP 2thread 2core - PDR Throughput for Phy-to-Phy IPv4 Overlay Tunnels.* +*Figure 4. VPP 2thread 2core - PDR Throughput for Phy-to-Phy IPv4 Overlay Tunnels.* CSIT test cases used to generate results presented above can be found in CSIT git repository by filtering with specified regex as follows: @@ -143,26 +118,3 @@ git repository by filtering with specified regex as follows: 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrdisc.robot:| tc08-64B-2t2c-ethip4vxlan-l2bdbasemaclrn-pdrdisc 10ge2p1x520-ethip4vxlan-l2xcbase-ndrdisc.robot:| tc08-64B-2t2c-ethip4vxlan-l2xcbase-pdrdisc -VPP PDR Throughput - running in configuration of **four worker threads (4t) on -four physical cores (4c)** - is presented in the figure below. - -.. raw:: html - - - -*Figure 6. VPP 4thread 4core - PDR Throughput for Phy-to-Phy IPv4 Overlay -Tunnels.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. code-block:: bash - - $ csit/tests/perf - $ grep -E "64B-4t4c-ethip4[a-z0-9]+-[a-z0-9]*-pdrdisc" * - - 10ge2p1x520-ethip4lispip4-ip4base-ndrdisc.robot:| tc14-64B-4t4c-ethip4lispip4-ip4base-pdrdisc - 10ge2p1x520-ethip4lispip6-ip4base-ndrdisc.robot:| tc14-64B-4t4c-ethip4lispip6-ip4base-pdrdisc - 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrdisc.robot:| tc14-64B-4t4c-ethip4vxlan-l2bdbasemaclrn-pdrdisc - 10ge2p1x520-ethip4vxlan-l2xcbase-ndrdisc.robot:| tc14-64B-4t4c-ethip4vxlan-l2xcbase-pdrdisc - diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6.rst index 1230bde94c..09f19a876b 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6.rst @@ -8,7 +8,7 @@ configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and their physical CPU core(s) placement. *Title of each graph* is a regex (regular expression) matching all plotted -throughput test cases, *X-axis labels* are indeces of csit-vpp-perf-1701 jobs +throughput test cases, *X-axis labels* are indeces of csit-vpp-perf-1704 jobs that created result output files used as data sources for the graph, *Y-axis labels* are measured Packets Per Second [pps] values, and the *graph legend* identifes the plotted test suites. @@ -16,11 +16,9 @@ legend* identifes the plotted test suites. .. note:: Data sources for reported test results: i) FD.io test executor jobs - `csit-vpp-perf-1701-all - `_ and - `csit-vpp-perf-1701-long - `_ - , ii) archived FD.io jobs test result `output files + `csit-vpp-perf-1704-all + `_ , + ii) archived FD.io jobs test result `output files <../../_static/archive/>`_. NDR Throughput @@ -78,32 +76,6 @@ git repository by filtering with specified regex as follows: 10ge2p1x520-ethip6-ip6scale2m-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6scale2m-ndrdisc 40ge2p1xl710-ethip6-ip6base-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6base-ndrdisc -VPP NDR Throughput - running in configuration of **four worker threads (4t) on -four physical cores (4c)** - is presented in the figure below. - -.. raw:: html - - - -*Figure 3. VPP 4threads 4cores - NDR Throughput for Phy-to-Phy IPv6 -Routed-Forwarding.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. code-block:: bash - - $ csit/tests/perf - $ grep -E "78B-4t4c-ethip6-ip6[a-z0-9]+-[a-z-]*ndrdisc" * - - 10ge2p1x520-ethip6-ip6base-copwhtlistbase-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6base-copwhtlistbase-ndrdisc - 10ge2p1x520-ethip6-ip6base-iacldstbase-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6base-iacldstbase-ndrdisc - 10ge2p1x520-ethip6-ip6base-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6base-ndrdisc - 10ge2p1x520-ethip6-ip6scale200k-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6scale200k-ndrdisc - 10ge2p1x520-ethip6-ip6scale20k-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6scale20k-ndrdisc - 10ge2p1x520-ethip6-ip6scale2m-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6scale2m-ndrdisc - 40ge2p1xl710-ethip6-ip6base-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6base-ndrdisc - PDR Throughput ~~~~~~~~~~~~~~ @@ -115,7 +87,7 @@ packet loss ratio. -*Figure 4. VPP 1thread 1core - PDR Throughput for Phy-to-Phy IPv6 +*Figure 3. VPP 1thread 1core - PDR Throughput for Phy-to-Phy IPv6 Routed-Forwarding.* CSIT test cases used to generate results presented above can be found in CSIT @@ -140,7 +112,7 @@ two physical cores (2c)** - is presented in the figure below. -*Figure 5. VPP 2thread 2core - PDR Throughput for Phy-to-Phy IPv6 +*Figure 4. VPP 2thread 2core - PDR Throughput for Phy-to-Phy IPv6 Routed-Forwarding.* CSIT test cases used to generate results presented above can be found in CSIT @@ -158,29 +130,3 @@ git repository by filtering with specified regex as follows: 10ge2p1x520-ethip6-ip6scale20k-ndrdisc.robot:| tc08-78B-2t2c-ethip6-ip6scale20k-pdrdisc 10ge2p1x520-ethip6-ip6scale2m-ndrdisc.robot:| tc08-78B-2t2c-ethip6-ip6scale2m-pdrdisc -VPP PDR Throughput - running in configuration of **four worker threads (4t) on -four physical cores (4c)** - is presented in the figure below. - -.. raw:: html - - - -*Figure 6. VPP 4thread 4core - PDR Throughput for Phy-to-Phy IPv6 -Routed-Forwarding.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. code-block:: bash - - $ csit/tests/perf - $ grep -E "78B-4t4c-ethip6-ip6[a-z0-9]+-[a-z-]*pdrdisc" * - - 10ge2p1x520-ethip6-ip6base-copwhtlistbase-ndrdisc.robot:| tc14-78B-4t4c-ethip6-ip6base-copwhtlistbase-pdrdisc - 10ge2p1x520-ethip6-ip6base-iacldstbase-ndrdisc.robot:| tc14-78B-4t4c-ethip6-ip6base-iacldstbase-pdrdisc - 10ge2p1x520-ethip6-ip6base-ndrdisc.robot:| tc14-78B-4t4c-ethip6-ip6base-pdrdisc - 10ge2p1x520-ethip6-ip6scale200k-ndrdisc.robot:| tc14-78B-4t4c-ethip6-ip6scale200k-pdrdisc - 10ge2p1x520-ethip6-ip6scale20k-ndrdisc.robot:| tc14-78B-4t4c-ethip6-ip6scale20k-pdrdisc - 10ge2p1x520-ethip6-ip6scale2m-ndrdisc.robot:| tc14-78B-4t4c-ethip6-ip6scale2m-pdrdisc - - diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6_tunnels.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6_tunnels.rst index fcd5bef2ca..34a7df7e70 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6_tunnels.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6_tunnels.rst @@ -8,7 +8,7 @@ configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and their physical CPU core(s) placement. *Title of each graph* is a regex (regular expression) matching all plotted -throughput test cases, *X-axis labels* are indeces of csit-vpp-perf-1701 jobs +throughput test cases, *X-axis labels* are indeces of csit-vpp-perf-1704 jobs that created result output files used as data sources for the graph, *Y-axis labels* are measured Packets Per Second [pps] values, and the *graph legend* identifes the plotted test suites. @@ -16,11 +16,9 @@ legend* identifes the plotted test suites. .. note:: Data sources for reported test results: i) FD.io test executor jobs - `csit-vpp-perf-1701-all - `_ and - `csit-vpp-perf-1701-long - `_ - , ii) archived FD.io jobs test result `output files + `csit-vpp-perf-1704-all + `_ , + ii) archived FD.io jobs test result `output files <../../_static/archive/>`_. NDR Throughput @@ -68,27 +66,6 @@ git repository by filtering with specified regex as follows: 10ge2p1x520-ethip6lispip4-ip6base-ndrdisc.robot:| tc07-78B-2t2c-ethip6lispip4-ip6base-ndrdisc 10ge2p1x520-ethip6lispip6-ip6base-ndrdisc.robot:| tc07-78B-2t2c-ethip6lispip6-ip6base-ndrdisc -VPP NDR Throughput - running in configuration of **four worker threads (4t) on -four physical cores (4c)** - is presented in the figure below. - -.. raw:: html - - - -*Figure 3. VPP 4threads 4cores - NDR Throughput for Phy-to-Phy IPv6 Overlay -Tunnels.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. code-block:: bash - - $ csit/tests/perf - $ grep -E "78B-4t4c-ethip6[a-z0-9]+-[a-z0-9]*-ndrdisc" * - - 10ge2p1x520-ethip6lispip4-ip6base-ndrdisc.robot:| tc13-78B-4t4c-ethip6lispip4-ip6base-ndrdisc - 10ge2p1x520-ethip6lispip6-ip6base-ndrdisc.robot:| tc13-78B-4t4c-ethip6lispip6-ip6base-ndrdisc - PDR Throughput ~~~~~~~~~~~~~~ @@ -100,7 +77,7 @@ packet loss ratio. -*Figure 4. VPP 1thread 1core - PDR Throughput for Phy-to-Phy IPv6 Overlay +*Figure 3. VPP 1thread 1core - PDR Throughput for Phy-to-Phy IPv6 Overlay Tunnels.* CSIT test cases used to generate results presented above can be found in CSIT @@ -121,7 +98,7 @@ two physical cores (2c)** - is presented in the figure below. -*Figure 5. VPP 2thread 2core - PDR Throughput for Phy-to-Phy IPv6 Overlay +*Figure 4. VPP 2thread 2core - PDR Throughput for Phy-to-Phy IPv6 Overlay Tunnels.* CSIT test cases used to generate results presented above can be found in CSIT @@ -134,25 +111,3 @@ git repository by filtering with specified regex as follows: 10ge2p1x520-ethip6lispip4-ip6base-ndrdisc.robot:| tc08-78B-2t2c-ethip6lispip4-ip6base-pdrdisc 10ge2p1x520-ethip6lispip6-ip6base-ndrdisc.robot:| tc08-78B-2t2c-ethip6lispip6-ip6base-pdrdisc - -VPP PDR Throughput - running in configuration of **four worker threads (4t) on -four physical cores (4c)** - is presented in the figure below. - -.. raw:: html - - - -*Figure 6. VPP 4thread 4core - PDR Throughput for Phy-to-Phy IPv6 Overlay -Tunnels.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. code-block:: bash - - $ csit/tests/perf - $ grep -E "78B-4t4c-ethip6[a-z0-9]+-[a-z0-9]*-pdrdisc" * - - 10ge2p1x520-ethip6lispip4-ip6base-ndrdisc.robot:| tc14-78B-4t4c-ethip6lispip4-ip6base-pdrdisc - 10ge2p1x520-ethip6lispip6-ip6base-ndrdisc.robot:| tc14-78B-4t4c-ethip6lispip6-ip6base-pdrdisc - diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/l2.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/l2.rst index e28400ecce..429696bcae 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/l2.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/l2.rst @@ -8,7 +8,7 @@ configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and their physical CPU core(s) placement. *Title of each graph* is a regex (regular expression) matching all plotted -throughput test cases, *X-axis labels* are indeces of csit-vpp-perf-1701 jobs +throughput test cases, *X-axis labels* are indeces of csit-vpp-perf-1704 jobs that created result output files used as data sources for the graph, *Y-axis labels* are measured Packets Per Second [pps] values, and the *graph legend* identifes the plotted test suites. @@ -16,11 +16,9 @@ legend* identifes the plotted test suites. .. note:: Data sources for reported test results: i) FD.io test executor jobs - `csit-vpp-perf-1701-all - `_ and - `csit-vpp-perf-1701-long - `_ - , ii) archived FD.io jobs test result `output files + `csit-vpp-perf-1704-all + `_ , + ii) archived FD.io jobs test result `output files <../../_static/archive/>`_. NDR Throughput @@ -82,34 +80,6 @@ git repository by filtering with specified regex as follows: 40ge2p1xl710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-ndrdisc 40ge2p1xl710-eth-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-ndrdisc -VPP NDR Throughput - running in configuration of **four worker threads (4t) on -four physical cores (4c)** - is presented in the figure below. - -.. raw:: html - - - -*Figure 3. VPP 4threads 4cores - NDR Throughput for Phy-to-Phy L2 Ethernet -Switching.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. code-block:: bash - - $ csit/tests/perf - $ grep -E "64B-4t4c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc" * - - 10ge2p1vic1227-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc - 10ge2p1x520-dot1ad-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-dot1ad-l2xcbase-ndrdisc - 10ge2p1x520-dot1q-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-dot1q-l2xcbase-ndrdisc - 10ge2p1x520-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc - 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-ndrdisc - 10ge2p1x710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc - 40ge2p1vic1385-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc - 40ge2p1xl710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc - 40ge2p1xl710-eth-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-ndrdisc - PDR Throughput ~~~~~~~~~~~~~~ @@ -121,7 +91,7 @@ loss ratio. -*Figure 4. VPP 1thread 1core - PDR Throughput for Phy-to-Phy L2 Ethernet +*Figure 3. VPP 1thread 1core - PDR Throughput for Phy-to-Phy L2 Ethernet Switching.* CSIT test cases used to generate results presented above can be found in CSIT @@ -147,7 +117,7 @@ two physical cores (2c)** - is presented in the figure below. -*Figure 5. VPP 2thread 2core - PDR Throughput for Phy-to-Phy L2 Ethernet +*Figure 4. VPP 2thread 2core - PDR Throughput for Phy-to-Phy L2 Ethernet Switching.* CSIT test cases used to generate results presented above can be found in CSIT @@ -166,28 +136,3 @@ git repository by filtering with specified regex as follows: 10ge2p1x710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc08-64B-2t2c-eth-l2bdbasemaclrn-pdrdisc 40ge2p1vic1385-eth-l2bdbasemaclrn-ndrdisc.robot:| tc08-64B-2t2c-eth-l2bdbasemaclrn-pdrdisc -VPP PDR Throughput - running in configuration of **four worker threads (4t) on -four physical cores (4c)** - is presented in the figure below. - -.. raw:: html - - - -*Figure 6. VPP 4thread 4core - PDR Throughput for Phy-to-Phy L2 Ethernet -Switching.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. code-block:: bash - - $ csit/tests/perf - $ grep -E "64B-4t4c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-pdrdisc" * - - 10ge2p1vic1227-eth-l2bdbasemaclrn-ndrdisc.robot:| tc14-64B-4t4c-eth-l2bdbasemaclrn-pdrdisc - 10ge2p1x520-dot1ad-l2xcbase-ndrdisc.robot:| tc14-64B-4t4c-dot1ad-l2xcbase-pdrdisc - 10ge2p1x520-dot1q-l2xcbase-ndrdisc.robot:| tc14-64B-4t4c-dot1q-l2xcbase-pdrdisc - 10ge2p1x520-eth-l2bdbasemaclrn-ndrdisc.robot:| tc14-64B-4t4c-eth-l2bdbasemaclrn-pdrdisc - 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc14-64B-4t4c-eth-l2xcbase-pdrdisc - 10ge2p1x710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc14-64B-4t4c-eth-l2bdbasemaclrn-pdrdisc - diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/vm_vhost.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/vm_vhost.rst index beadc052d4..9772912d9c 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/vm_vhost.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/vm_vhost.rst @@ -8,7 +8,7 @@ configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and their physical CPU core(s) placement. *Title of each graph* is a regex (regular expression) matching all plotted -throughput test cases, *X-axis labels* are indeces of csit-vpp-perf-1701 jobs +throughput test cases, *X-axis labels* are indeces of csit-vpp-perf-1704 jobs that created result output files used as data sources for the graph, *Y-axis labels* are measured Packets Per Second [pps] values, and the *graph legend* identifes the plotted test suites. @@ -16,11 +16,9 @@ legend* identifes the plotted test suites. .. note:: Data sources for reported test results: i) FD.io test executor jobs - `csit-vpp-perf-1701-all - `_ and - `csit-vpp-perf-1701-long - `_ - , ii) archived FD.io jobs test result `output files + `csit-vpp-perf-1704-all + `_ , + ii) archived FD.io jobs test result `output files <../../_static/archive/>`_. NDR Throughput @@ -87,26 +85,6 @@ four physical cores (4c)** - is presented in the figure below. -*Figure 3. VPP 4threads 4cores - NDR Throughput for Phy-to-VM-to-Phy VM -vhost-user.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. code-block:: bash - - $ csit/tests/perf - $ grep -E "64B-4t4c-.*vhost.*-ndrdisc" * - - 10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc - 10ge2p1x520-dot1q-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc - 10ge2p1x520-ethip4-ip4base-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-eth-2vhost-1vm-ndrdisc - 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc - 10ge2p1x520-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc - 10ge2p1x520-eth-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc - 10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc - 40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc - PDR Throughput ~~~~~~~~~~~~~~ @@ -118,7 +96,7 @@ packet loss ratio. -*Figure 4. VPP 1thread 1core - PDR Throughput for Phy-to-VM-to-Phy VM +*Figure 3. VPP 1thread 1core - PDR Throughput for Phy-to-VM-to-Phy VM vhost-user.* CSIT test cases used to generate results presented above can be found in CSIT @@ -145,7 +123,7 @@ two physical cores (2c)** - is presented in the figure below. -*Figure 5. VPP 2thread 2core - PDR Throughput for Phy-to-VM-to-Phy VM +*Figure 4. VPP 2thread 2core - PDR Throughput for Phy-to-VM-to-Phy VM vhost-user.* CSIT test cases used to generate results presented above can be found in CSIT @@ -165,30 +143,3 @@ git repository by filtering with specified regex as follows: 10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc08-64B-2t2c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc 40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc08-64B-2t2c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc -VPP PDR Throughput - running in configuration of **four worker threads (4t) on -four physical cores (4c)** - is presented in the figure below. - -.. raw:: html - - - -*Figure 6. VPP 4thread 4core - PDR Throughput for Phy-to-VM-to-Phy VM -vhost-user.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. code-block:: bash - - $ csit/tests/perf - $ grep -E "64B-4t4c-.*vhost.*-pdrdisc" * - - 10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc14-64B-4t4c-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc - 10ge2p1x520-dot1q-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc14-64B-4t4c-eth-l2xcbase-eth-2vhost-1vm-pdrdisc - 10ge2p1x520-ethip4-ip4base-eth-2vhost-1vm-ndrdisc.robot:| tc14-64B-4t4c-ethip4-ip4base-eth-2vhost-1vm-pdrdisc - 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc14-64B-4t4c-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc - 10ge2p1x520-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc14-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc - 10ge2p1x520-eth-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc14-64B-4t4c-eth-l2xcbase-eth-2vhost-1vm-pdrdisc - 10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc14-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc - 40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc14-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc - diff --git a/docs/report/vpp_performance_tests/test_environment.rst b/docs/report/vpp_performance_tests/test_environment.rst index 0fd22213b9..c8ea969728 100644 --- a/docs/report/vpp_performance_tests/test_environment.rst +++ b/docs/report/vpp_performance_tests/test_environment.rst @@ -6,20 +6,22 @@ consists of two SUTs and one TG. SUT Configuration - Host HW --------------------------- -Hardware details (CPU, memory, NIC layout) are described in -`CSIT/CSIT_LF_testbed `_ in -summary: +Host hardware details (CPU, memory, NIC layout) and physical topology are +described in detail in +`LF FDio CSIT testbed wiki page `_. + +**Host configuration** - All hosts are Cisco UCS C240-M4 (2x Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz, 18c, 512GB RAM), -- BIOS settings are default except for the following: +- BIOS settings are default except the following: - - Hyperthreading disabled, - - SpeedStep disabled - - TurboBoost disabled - - Power Technology: Performance + - Hyperthreading disabled. + - SpeedStep disabled. + - TurboBoost disabled. + - Power Technology: Performance. -- Hosts run Ubuntu 16.04.1, kernel 4.4.0-42-generic +- Hosts run Ubuntu 16.04.1, kernel 4.4.0-42-generic. - Linux kernel boot command line option "intel_pstate=disable" is applied to both SUTs and TG. In addition, on SUTs, only cores 0 and 18 (the first core on each socket) are available to the Linux operating system and generic tasks, @@ -35,7 +37,7 @@ summary: - This allows for a total of five ring topologies, each using ports on specific NIC model, enabling per NIC model benchmarking. -**NIC types** +**NIC models and placement** - 0a:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01) Subsystem: Intel Corporation Ethernet Server @@ -67,9 +69,10 @@ summary: SUT Configuration - Host OS Linux --------------------------------- -Software details (OS, configuration) is described in -`CSIT/CSIT_LF_testbed `_ in -summary: +Software details (OS, configuration) are described in +`CSIT/CSIT_LF_testbed `_. + +Below a subset of the configuration: .. code-block:: xml @@ -2107,10 +2110,10 @@ summary: 0 @@ -2192,29 +2195,29 @@ summary: dev.cdrom.check_media = 0 dev.cdrom.debug = 0 dev.cdrom.info = CD-ROM information, Id: cdrom.c 3.20 2003/12/17 - dev.cdrom.info = - dev.cdrom.info = drive name: - dev.cdrom.info = drive speed: + dev.cdrom.info = + dev.cdrom.info = drive name: + dev.cdrom.info = drive speed: dev.cdrom.info = drive # of slots: - dev.cdrom.info = Can close tray: - dev.cdrom.info = Can open tray: - dev.cdrom.info = Can lock tray: + dev.cdrom.info = Can close tray: + dev.cdrom.info = Can open tray: + dev.cdrom.info = Can lock tray: dev.cdrom.info = Can change speed: dev.cdrom.info = Can select disk: dev.cdrom.info = Can read multisession: - dev.cdrom.info = Can read MCN: + dev.cdrom.info = Can read MCN: dev.cdrom.info = Reports media changed: - dev.cdrom.info = Can play audio: - dev.cdrom.info = Can write CD-R: + dev.cdrom.info = Can play audio: + dev.cdrom.info = Can write CD-R: dev.cdrom.info = Can write CD-RW: - dev.cdrom.info = Can read DVD: + dev.cdrom.info = Can read DVD: dev.cdrom.info = Can write DVD-R: dev.cdrom.info = Can write DVD-RAM: - dev.cdrom.info = Can read MRW: - dev.cdrom.info = Can write MRW: - dev.cdrom.info = Can write RAM: - dev.cdrom.info = - dev.cdrom.info = + dev.cdrom.info = Can read MRW: + dev.cdrom.info = Can write MRW: + dev.cdrom.info = Can write RAM: + dev.cdrom.info = + dev.cdrom.info = dev.cdrom.lock = 0 dev.hpet.max-user-freq = 64 dev.mac_hid.mouse_button2_keycode = 97 @@ -2276,7 +2279,7 @@ summary: kernel.hardlockup_all_cpu_backtrace = 0 kernel.hardlockup_panic = 0 kernel.hostname = t1-sut1 - kernel.hotplug = + kernel.hotplug = kernel.hung_task_check_count = 4194304 kernel.hung_task_panic = 0 kernel.hung_task_timeout_secs = 120 @@ -2577,7 +2580,7 @@ summary: net.ipv4.ip_forward = 0 net.ipv4.ip_forward_use_pmtu = 0 net.ipv4.ip_local_port_range = 32768 60999 - net.ipv4.ip_local_reserved_ports = + net.ipv4.ip_local_reserved_ports = net.ipv4.ip_no_pmtu_disc = 0 net.ipv4.ip_nonlocal_bind = 0 net.ipv4.ipfrag_high_thresh = 4194304 -- cgit 1.2.3-korg