diff options
| author |   Maciek Konstantynowicz <mkonstan@cisco.com> | 2017-04-20 22:42:01 +0100 |
|---|---|---|
| committer |   Peter Mikus <pmikus@cisco.com> | 2017-04-21 12:28:05 +0000 |
| commit | 182e755776923c81b2e2138125728b592dabe27b (patch) | |
| tree | 03ea3e56470a76f58b45b86489f8fa4ca1441b32 | |
| parent | aef6be686bfb1cdbf85d1fca0e5d28e43650ec0d (diff) | |
CSIT rls1704 report - update to handcraftec sections:
- introduction /overview /general_notes /csit_test_naming
- vpp_performance_tests /overview /csit_release_notes /test_environment /documentation /xx_graphs
- vpp_functional_tests /overview /csit_release_notes /documentation
- testpmd_performance_tests /overview /csit_release_notes /test_environment /documentation /xx_graphs
Change-Id: I10593d4155edc663ca07ae5cd29eb679c5f5d85f
Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
(cherry picked from commit 5d294b7d4c501d971d67c4e3fe192a9dc504d76e)
28 files changed, 224 insertions, 813 deletions
diff --git a/docs/report/introduction/csit_test_naming.rst b/docs/report/introduction/csit_test_naming.rst index 0415e5c5d6..a4de765e0a 100644 --- a/docs/report/introduction/csit_test_naming.rst +++ b/docs/report/introduction/csit_test_naming.rst @@ -4,21 +4,13 @@ CSIT Test Naming Background
----------
-CSIT |release| introduced a common structured naming convention for all
-performance and functional tests. This change was driven by substantially
-growing number and type of CSIT test cases. Firstly, the original practice did
-not always follow any strict naming convention. Secondly test names did not
-always clearly capture tested packet encapsulations, and the actual type or
-content of the tests. Thirdly HW configurations in terms of NICs, ports and
-their locality were not captured either. These were but few reasons that drove
-the decision to change and define a new more complete and stricter test naming
-convention. And to apply it to all existing and new test suites and test
-cases.
-
-The new naming should be intuitive for majority of the tests. Complete
-description of the newly adopted CSIT test naming convention is provided on
-`CSIT test naming page <https://wiki.fd.io/view/CSIT/csit-test-naming>`_. Here
-few illustrative examples of the new naming usage for test suites across CSIT
+CSIT |release| follows a common structured naming convention for all
+performance and system functional tests, introduced in CSIT rls1701.
+
+The naming should be intuitive for majority of the tests. Complete
+description of CSIT test naming convention is provided on
+`CSIT test naming wiki page <https://wiki.fd.io/view/CSIT/csit-test-naming>`_. Below
+few illustrative examples of the naming usage for test suites across CSIT
performance, functional and HoneyComb management test areas.
Naming Convention
@@ -123,4 +115,4 @@ topologies: sub-interface events with HoneyComb NetConf Notifications, functional tests.
For complete description of CSIT test naming convention please refer to `CSIT
-test naming page <https://wiki.fd.io/view/CSIT/csit-test-naming>`_.
+test naming wiki page <https://wiki.fd.io/view/CSIT/csit-test-naming>`_.
diff --git a/docs/report/introduction/general_notes.rst b/docs/report/introduction/general_notes.rst index c05401e4cc..ebb73cdc93 100644 --- a/docs/report/introduction/general_notes.rst +++ b/docs/report/introduction/general_notes.rst @@ -3,7 +3,7 @@ General Notes All CSIT test results listed in this report are sourced and auto-generated
from output.xml Robot Framework (RF) files resulting from LF FD.io Jenkins
-jobs execution against VPP-17.01 release artifacts. References are provided
+jobs execution against VPP-17.04 release artifacts. References are provided
to the original LF FD.io Jenkins job results. However, as LF FD.io Jenkins
infrastructure does not automatically archive all jobs (history record is
provided for the last 30 days or 40 jobs only), additional references are
@@ -12,10 +12,10 @@ storage system. FD.io CSIT project currently covers multiple FD.io system and sub-system
testing areas and this is reflected in this report, where each testing area
-is listed separately, as follows
+is listed separately, as follows:
-#. **VPP Performance Tests** - VPP performance are tests executed in physical
- FD.io testbeds, focusing on VPP network data plane performance at this stage,
+#. **VPP Performance Tests** - VPP performance tests are executed in physical
+ FD.io testbeds, focusing on VPP network data plane performance at this stage,
both for Phy-to-Phy (NIC-to-NIC) and Phy-to-VM-to-Phy (NIC-to-VM-to-NIC)
forwarding topologies. Tested across a range of NICs, 10GE and 40GE
interfaces, range of multi-thread and multi-core configurations. VPP
@@ -40,11 +40,9 @@ is listed separately, as follows functionality of VPP. Tests cover a range of CRUD operations executed
against VPP.
-CSIT |release| report does also include VPP unit test results. These tests
-have been developed within the FD.io VPP project, and not in CSIT, but they do
-complement tests done by CSIT. They are provided mainly as a reference, to
-give the reader a more complete view of automated testing executed against
-VPP-17.01 release.
+In addition to above, CSIT |release| report does also include VPP unit test results. VPP unit tests are developed within the FD.io VPP project and as they complement CSIT system functional tests, they are provided mainly as a reference and to
+provide a more complete view of automated testing executed against
+VPP-17.04 release.
FD.io CSIT system is developed using two main coding platforms: Robot
Framework (RF) and Python. CSIT |release| source code for the executed test
diff --git a/docs/report/introduction/overview.rst b/docs/report/introduction/overview.rst index ee1a91c174..d781993198 100644 --- a/docs/report/introduction/overview.rst +++ b/docs/report/introduction/overview.rst @@ -3,14 +3,14 @@ Overview This is the **F**\ast **D**\ata **I**/**O** Project (**FD.io**) **C**\ontinuous
**S**\ystem **I**\ntegration and **T**\esting (**CSIT**) project report for CSIT
-|release| system testing of VPP-17.01 release.
+|release| system testing of VPP-17.04 release.
The report describes CSIT functional and performance tests and their
continuous execution delivered in CSIT |release|. A high-level overview is
provided for each CSIT test environment running in Linux Foundation (LF) FD.io
Continuous Performance Labs. This is followed by summary of all executed tests
-against the VPP-17.01 release and associated sub-systems (HoneyComb, DPDK),
-CSIT |release| release notes, result highlights and known issues. More
+against the VPP-17.04 release and associated FD.io projects and sub-systems (HoneyComb, DPDK),
+CSIT |release| release notes, result highlights and known issues discovered in CSIT. More
detailed description of each environment, pointers to CSIT test code
documentation and detailed test resuls with links to the source data files are
also provided.
@@ -28,19 +28,18 @@ CSIT |release| report contains following main sections and sub-sections: specifics, methodology for multi-core, packet throughput and latency, and
KVM VM vhost tests; *CSIT Release Notes* - changes in CSIT |release|, added
tests, performance changes, environment or methodology changes, known CSIT
- issues, tests to be added; *Packet Throughput Graphs* and *Packet Latency
+ issues; *Packet Throughput Graphs* and *Packet Latency
Graphs* - plotted NDR, PDR throughput and latency results from multiple
- test job executions; *Test Environment* - environment description ;
- *Documentation* - source code documentation for VPP performance tests.
+ test job executions; *Test Environment* - environment description;
+ *Documentation* - CSIT source code documentation for VPP performance tests.
#. **Testpmd Performance Tests** - Testpmd performance tests executed in
physical FD.io testbeds; *Overview* - tested topologies, test coverage;
*CSIT Release Notes* - changes in CSIT |release|, any known CSIT issues;
- *Tests to Be Added* - performance tests to be added in the next revision of
- CSIT |release| report; *Packet Throughput Graphs* and *Packet Latency Graphs*
+ *Packet Throughput Graphs* and *Packet Latency Graphs*
- plotted NDR, PDR throughput and latency results from multiple test job
executions; *Test Environment* - environment description; *Documentation* -
- source code documentation for Testpmd performance tests.
+ CSIT source code documentation for Testpmd performance tests.
#. **VPP Functional Tests** - VPP functional tests executed in virtual
FD.io testbeds; *Overview* - tested virtual topologies, test coverage and
@@ -52,8 +51,8 @@ CSIT |release| report contains following main sections and sub-sections: #. **HoneyComb Functional Tests** - HoneyComb functional tests executed in
virtual FD.io testbeds; *Overview* - tested virtual topologies, test
coverage and naming specifics; *CSIT Release Notes* - changes in CSIT
- |release|, added tests, environment or methodology changes, known CSIT issues,
- tests to be added; *Test Environment* - environment description ;
+ |release|, added tests, environment or methodology changes, known CSIT issues;
+ *Test Environment* - environment description ;
*Documentation* - source code documentation for Honeycomb functional tests.
#. **VPP Unit Tests** - refers to VPP functional unit tests executed as
diff --git a/docs/report/testpmd_performance_tests/csit_release_notes.rst b/docs/report/testpmd_performance_tests/csit_release_notes.rst index 09d7f727da..6d415a1add 100644 --- a/docs/report/testpmd_performance_tests/csit_release_notes.rst +++ b/docs/report/testpmd_performance_tests/csit_release_notes.rst @@ -6,11 +6,5 @@ Changes in CSIT |release| #. Added Testpmd tests - - new NICs - Intel x520 + - new NICs - Intel xl710 -Tests to be Added - Additional NICs ------------------------------------ - -Adding test cases with more NIC types. Once the results become available, they -will be published as an addendum to the current version of CSIT |release| -report. diff --git a/docs/report/testpmd_performance_tests/documentation.rst b/docs/report/testpmd_performance_tests/documentation.rst index 737e46505b..4a0fe5dce5 100644 --- a/docs/report/testpmd_performance_tests/documentation.rst +++ b/docs/report/testpmd_performance_tests/documentation.rst @@ -2,5 +2,5 @@ Documentation =============
`CSIT Testpmd Performance Tests Documentation
-<https://docs.fd.io/csit/master/doc/tests.perf.html>`_ contains detailed
+<https://docs.fd.io/csit/rls1704/doc/tests.perf.html>`_ contains detailed
functional description and input parameters for each test case.
diff --git a/docs/report/testpmd_performance_tests/overview.rst b/docs/report/testpmd_performance_tests/overview.rst index db1ac7849d..2182a00cd8 100644 --- a/docs/report/testpmd_performance_tests/overview.rst +++ b/docs/report/testpmd_performance_tests/overview.rst @@ -42,8 +42,7 @@ tested NIC models include: #. 2port40GE VIC1385 Cisco. #. 2port40GE XL710 Intel. -Detailed LF FD.io test bed specification and topology is described in -`wiki CSIT LF testbed <https://wiki.fd.io/view/CSIT/CSIT_LF_testbed>`_. +For detailed LF FD.io test bed specification and physical topology please refer to `LF FDio CSIT testbed wiki page <https://wiki.fd.io/view/CSIT/CSIT_LF_testbed>`_. Performance Tests Coverage -------------------------- @@ -71,6 +70,11 @@ CSIT |release| includes following performance test suites, listed per NIC type: - **L2IntLoop** - L2 Interface Loop forwarding any Ethernet frames between two Interfaces. +- 2port40GE XL710 Intel + + - **L2IntLoop** - L2 Interface Loop forwarding any Ethernet frames between + two Interfaces. + Execution of performance tests takes time, especially the throughput discovery tests. Due to limited HW testbed resources available within FD.io labs hosted by Linux Foundation, the number of tests for NICs other than X520 (a.k.a. @@ -94,7 +98,6 @@ following Testpmd thread and core configurations: #. 1t1c - 1 Testpmd pmd thread on 1 CPU physical core. #. 2t2c - 2 Testpmd pmd threads on 2 CPU physical cores. -#. 4t4c - 4 Testpmd pmd threads on 4 CPU physical cores. Note that in many tests running Testpmd reaches tested NIC I/O bandwidth or packets-per-second limit. diff --git a/docs/report/testpmd_performance_tests/packet_latency_graphs/l2.rst b/docs/report/testpmd_performance_tests/packet_latency_graphs/l2.rst index fc2e833967..4ffc9b00ec 100644 --- a/docs/report/testpmd_performance_tests/packet_latency_graphs/l2.rst +++ b/docs/report/testpmd_performance_tests/packet_latency_graphs/l2.rst @@ -8,7 +8,7 @@ configurations of Testpmd worker thread(s), a.k.a. Testpmd data plane thread (s), and their physical CPU core(s) placement.
*Title of each graph* is a regex (regular expression) matching all plotted
-latency test cases, *X-axis labels* are indeces of csit-dpdk-perf-1701 jobs
+latency test cases, *X-axis labels* are indeces of csit-dpdk-perf-1704 jobs
that created result output files used as data sources for the graph,
*Y-axis labels* are measured packet Latency [uSec] values, and the *graph
legend* identifes the plotted test suites.
@@ -16,11 +16,9 @@ legend* identifes the plotted test suites. .. note::
Data sources for reported test results: i) FD.io test executor jobs
- `csit-dpdk-perf-1701-all <https://jenkins.fd.io/view/csit/job/csit-dpdk-perf-1701-all/>`_
- and
- `csit-dpdk-perf-1701-long <https://jenkins.fd.io/view/csit/job/csit-dpdk-perf-1701-long/>`_
- , ii) archived FD.io jobs test result output files
- `csit-dpdk-perf-1701-all#job-number <https://nexus.fd.io/link>`_.
+ `csit-dpdk-perf-1704-all <https://jenkins.fd.io/view/csit/job/csit-dpdk-perf-1704-all/>`_ ,
+ ii) archived FD.io jobs test result output files
+ `csit-dpdk-perf-1704-all#job-number <https://nexus.fd.io/link>`_.
Testpmd packet latency - running in configuration of **one worker thread (1t) on one
physical core (1c)** - is presented in the figure below.
@@ -60,22 +58,3 @@ git repository by filtering with specified regex as follows: 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-ndrdisc
-Testpmd packet latency - running in configuration of **four worker threads (4t)
-on four physical cores (4c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/testpmd/64B-4t4c-l2-ndrdisc-lat50.html"></iframe>
-
-*Figure 3. Testpmd 4thread 4core - packet latency for Phy-to-Phy L2 Ethernet Looping.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-4t4c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc" *
-
- 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-ndrdisc
-
diff --git a/docs/report/testpmd_performance_tests/packet_throughput_graphs/l2.rst b/docs/report/testpmd_performance_tests/packet_throughput_graphs/l2.rst index ee79a4c6d0..90dd456473 100644 --- a/docs/report/testpmd_performance_tests/packet_throughput_graphs/l2.rst +++ b/docs/report/testpmd_performance_tests/packet_throughput_graphs/l2.rst @@ -8,18 +8,16 @@ Testpmd worker thread (a.k.a. VPP data plane thread) and physical CPU core configurations.
*Title of each graph* is a regex (regular expression) matching all plotted
-throughput test cases, *X-axis labels* are indeces of csit-dpdk-perf-1701 jobs
+throughput test cases, *X-axis labels* are indeces of csit-dpdk-perf-1704 jobs
that created result output files used as data sources for the graph,
*Y-axis labels* are measured Packets Per Second [pps] values, and the *graph
legend* identifes the plotted test suites.
.. note::
- Sources of test results data: i) FD.io executor jobs `csit-dpdk-perf-1701-all
- <https://jenkins.fd.io/view/csit/job/csit-dpdk-perf-1701-all/>`_ and
- `csit-dpdk-perf-1701-long
- <https://jenkins.fd.io/view/csit/job/csit-dpdk-perf-1701-long/>`_, ii)
- FD.io jobs archieved result files.
+ Sources of test results data: i) FD.io executor jobs `csit-dpdk-perf-1704-all
+ <https://jenkins.fd.io/view/csit/job/csit-dpdk-perf-1704-all/>`_ ,
+ ii) FD.io jobs archieved result files.
NDR Throughput
~~~~~~~~~~~~~~
@@ -64,26 +62,6 @@ git repository by filtering with specified regex as follows: 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-ndrdisc
-NDR throughput of Testpmd - running in configuration of **four worker threads
-(4t) on four physical cores (4c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/testpmd/64B-4t4c-l2-ndrdisc.html"></iframe>
-
-*Figure 3. Testpmd 4threads 4cores - NDR Throughput for Phy-to-Phy L2 Ethernet
-Looping.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/dpdk-tests/perf
- $ grep -E "64B-4t4c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc" *
-
- 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-ndrdisc
-
PDR Throughput
~~~~~~~~~~~~~~
@@ -95,7 +73,7 @@ below 0.5% packet loss ratio. <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/testpmd/64B-1t1c-l2-pdrdisc.html"></iframe>
-*Figure 4. Testpmd 1thread 1core - PDR Throughput for Phy-to-Phy L2 Ethernet
+*Figure 3. Testpmd 1thread 1core - PDR Throughput for Phy-to-Phy L2 Ethernet
Looping.*
CSIT test cases used to generate results presented above can be found in CSIT
@@ -115,7 +93,7 @@ PDR throughput of Testpmd - running in configuration of **two worker threads <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/testpmd/64B-2t2c-l2-pdrdisc.html"></iframe>
-*Figure 5. Testpmd 2thread 2core - PDR Throughput for Phy-to-Phy L2 Ethernet
+*Figure 4. Testpmd 2thread 2core - PDR Throughput for Phy-to-Phy L2 Ethernet
Looping.*
CSIT test cases used to generate results presented above can be found in CSIT
@@ -128,22 +106,3 @@ git repository by filtering with specified regex as follows: 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc08-64B-2t2c-eth-l2xcbase-pdrdisc
-PDR throughput of Testpmd - running in configuration of **four worker threads
-(4t) on four physical cores (4c)** - is presented in the figure below.
-
-.. raw:: html
-
- <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/testpmd/64B-4t4c-l2-pdrdisc.html"></iframe>
-
-*Figure 6. Testpmd 4thread 4core - PDR Throughput for Phy-to-Phy L2 Ethernet
-Looping.*
-
-CSIT test cases used to generate results presented above can be found in CSIT
-git repository by filtering with specified regex as follows:
-
-.. code-block:: bash
-
- $ csit/tests/perf
- $ grep -E "64B-4t4c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-pdrdisc" *
-
- 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc14-64B-4t4c-eth-l2xcbase-pdrdisc
diff --git a/docs/report/testpmd_performance_tests/test_environment.rst b/docs/report/testpmd_performance_tests/test_environment.rst index e4f301daa0..dd51137b2f 100644 --- a/docs/report/testpmd_performance_tests/test_environment.rst +++ b/docs/report/testpmd_performance_tests/test_environment.rst @@ -6,9 +6,11 @@ consists of two SUTs and one TG. SUT Configuration - Host HW
---------------------------
-Hardware details (CPU, memory, NIC layout) are described in
-`CSIT/CSIT_LF_testbed <https://wiki.fd.io/view/CSIT/CSIT_LF_testbed>`_ in
-summary:
+Host hardware details (CPU, memory, NIC layout) and physical topology are
+described in detail in
+`LF FDio CSIT testbed wiki page <https://wiki.fd.io/view/CSIT/CSIT_LF_testbed>`_.
+
+**Host configuration**
- All hosts are Cisco UCS C240-M4 (2x Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz,
18c, 512GB RAM),
@@ -35,7 +37,7 @@ summary: - This allows for a total of five ring topologies, each using ports on
specific NIC model, enabling per NIC model benchmarking.
-**NIC types**
+**NIC models and placement**
- 0a:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+
Network Connection (rev 01) Subsystem: Intel Corporation Ethernet Server
@@ -67,9 +69,10 @@ summary: SUT Configuration - Host OS Linux
---------------------------------
-Software details (OS, configuration) is described in
-`CSIT/CSIT_LF_testbed <https://wiki.fd.io/view/CSIT/CSIT_LF_testbed>`_ in
-summary:
+Software details (OS, configuration) are described in
+`CSIT/CSIT_LF_testbed <https://wiki.fd.io/view/CSIT/CSIT_LF_testbed>`_.
+
+Below a subset of the configuration:
.. code-block:: xml
@@ -2107,10 +2110,10 @@ summary: <exec_command><![CDATA[lsblk -l]]></exec_command>
<exec_return_code>0</exec_return_code>
<exec_output><![CDATA[NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
- sda 8:0 0 1.8T 0 disk
+ sda 8:0 0 1.8T 0 disk
sda1 8:1 0 243M 0 part /boot
sda2 8:2 0 1.8T 0 part /
- sda3 8:3 0 1K 0 part
+ sda3 8:3 0 1K 0 part
sda5 8:5 0 976M 0 part [SWAP]
]]></exec_output>
</function>
@@ -2192,29 +2195,29 @@ summary: dev.cdrom.check_media = 0
dev.cdrom.debug = 0
dev.cdrom.info = CD-ROM information, Id: cdrom.c 3.20 2003/12/17
- dev.cdrom.info =
- dev.cdrom.info = drive name:
- dev.cdrom.info = drive speed:
+ dev.cdrom.info =
+ dev.cdrom.info = drive name:
+ dev.cdrom.info = drive speed:
dev.cdrom.info = drive # of slots:
- dev.cdrom.info = Can close tray:
- dev.cdrom.info = Can open tray:
- dev.cdrom.info = Can lock tray:
+ dev.cdrom.info = Can close tray:
+ dev.cdrom.info = Can open tray:
+ dev.cdrom.info = Can lock tray:
dev.cdrom.info = Can change speed:
dev.cdrom.info = Can select disk:
dev.cdrom.info = Can read multisession:
- dev.cdrom.info = Can read MCN:
+ dev.cdrom.info = Can read MCN:
dev.cdrom.info = Reports media changed:
- dev.cdrom.info = Can play audio:
- dev.cdrom.info = Can write CD-R:
+ dev.cdrom.info = Can play audio:
+ dev.cdrom.info = Can write CD-R:
dev.cdrom.info = Can write CD-RW:
- dev.cdrom.info = Can read DVD:
+ dev.cdrom.info = Can read DVD:
dev.cdrom.info = Can write DVD-R:
dev.cdrom.info = Can write DVD-RAM:
- dev.cdrom.info = Can read MRW:
- dev.cdrom.info = Can write MRW:
- dev.cdrom.info = Can write RAM:
- dev.cdrom.info =
- dev.cdrom.info =
+ dev.cdrom.info = Can read MRW:
+ dev.cdrom.info = Can write MRW:
+ dev.cdrom.info = Can write RAM:
+ dev.cdrom.info =
+ dev.cdrom.info =
dev.cdrom.lock = 0
dev.hpet.max-user-freq = 64
dev.mac_hid.mouse_button2_keycode = 97
@@ -2276,7 +2279,7 @@ summary: kernel.hardlockup_all_cpu_backtrace = 0
kernel.hardlockup_panic = 0
kernel.hostname = t1-sut1
- kernel.hotplug =
+ kernel.hotplug =
kernel.hung_task_check_count = 4194304
kernel.hung_task_panic = 0
kernel.hung_task_timeout_secs = 120
@@ -2577,7 +2580,7 @@ summary: net.ipv4.ip_forward = 0
net.ipv4.ip_forward_use_pmtu = 0
net.ipv4.ip_local_port_range = 32768 60999
- net.ipv4.ip_local_reserved_ports =
+ net.ipv4.ip_local_reserved_ports =
net.ipv4.ip_no_pmtu_disc = 0
net.ipv4.ip_nonlocal_bind = 0
net.ipv4.ipfrag_high_thresh = 4194304
@@ -3495,7 +3498,7 @@ DUT Configuration - DPDK **DPDK Version**
-16.09
+17.02
**DPDK Compile Parameters**
diff --git a/docs/report/vpp_functional_tests/csit_release_notes.rst b/docs/report/vpp_functional_tests/csit_release_notes.rst index 6db4900046..c87cb46e3c 100644 --- a/docs/report/vpp_functional_tests/csit_release_notes.rst +++ b/docs/report/vpp_functional_tests/csit_release_notes.rst @@ -1,24 +1,10 @@ - -.. |br| raw:: html - - <br /> - CSIT Release Notes ================== Changes in CSIT |release| ------------------------- -#. Naming change for all VPP functional test suites in VIRL - - - VPP functional test case names stayed unchanged - -#. VPP functional test environment changes - - - upgrade to Ubuntu 16.04 - - VM and vhost-user test environment optimizations - -#. Introduction of Centos tests +#. Introduction of Centos7 tests #. Added VPP functional tests @@ -66,11 +52,3 @@ Here is the list of known issues in CSIT |release| for VPP functional tests in V | | | The (Rx) traffic is mirrored to correct destination port. | +---+-------------------------------------------------+-----------------------------------------------------------------+ -Tests to be Added - Centos --------------------------- - -CSIT |release| added Centos functional test execution environment in FD.io -VIRL testbeds.However due to the factors beyond CSIT project control execution -of those tests within the LF FD.io test environment still need to complete. -Once the results become available, they will be published as an addendum to -the current version of CSIT |release| report. diff --git a/docs/report/vpp_functional_tests/documentation.rst b/docs/report/vpp_functional_tests/documentation.rst index e286e6cf19..d34877b0b3 100644 --- a/docs/report/vpp_functional_tests/documentation.rst +++ b/docs/report/vpp_functional_tests/documentation.rst @@ -2,6 +2,6 @@ Documentation =============
`CSIT VPP Functional Tests Documentation
-<https://docs.fd.io/csit/master/doc/tests.func.html>`_ contains detailed
+<https://docs.fd.io/csit/rls1704/doc/tests.func.html>`_ contains detailed
functional description and input parameters for each test case.
diff --git a/docs/report/vpp_functional_tests/overview.rst b/docs/report/vpp_functional_tests/overview.rst index e85bb92f7b..0fa15a5472 100644 --- a/docs/report/vpp_functional_tests/overview.rst +++ b/docs/report/vpp_functional_tests/overview.rst @@ -38,7 +38,7 @@ Both logical test topologies are shown in the figures below. +---------------> <--------------+
| | | | | |
| |------------> DUT1 <-----------+ |
- | | | +--^--^------------+ | | |
+ | | | +------------------+ | | |
| | | | | |
| | | SUT1 | | |
| | +------------------------+ | |
@@ -131,17 +131,10 @@ results listed in this report: Functional Tests Naming
-----------------------
-CSIT |release| introduced a common structured naming convention for all
-performance and functional tests. This change was driven by substantially
-growing number and type of CSIT test cases. Firstly, the original practice did
-not always follow any strict naming convention. Secondly test names did not
-always clearly capture tested packet encapsulations, and the actual type or
-content of the tests. Thirdly HW configurations in terms of NICs, ports and
-their locality were not captured either. These were but few reasons that drove
-the decision to change and define a new more complete and stricter test naming
-convention, and to apply this to all existing and new test cases.
-
-The new naming should be intuitive for majority of the tests. The complete
+CSIT |release| follows a common structured naming convention for all
+performance and system functional tests, introduced in CSIT rls1701.
+
+The naming should be intuitive for majority of the tests. Complete
description of CSIT test naming convention is provided on `CSIT test naming
page <https://wiki.fd.io/view/CSIT/csit-test-naming>`_.
diff --git a/docs/report/vpp_performance_tests/csit_release_notes.rst b/docs/report/vpp_performance_tests/csit_release_notes.rst index 54fa73dfee..4c942e19a4 100644 --- a/docs/report/vpp_performance_tests/csit_release_notes.rst +++ b/docs/report/vpp_performance_tests/csit_release_notes.rst @@ -4,12 +4,10 @@ CSIT Release Notes Changes in CSIT |release|
-------------------------
-#. Naming change for all VPP performance test suites and test cases.
#. VPP performance test environment changes
- - upgrade to Ubuntu 16.04
- - VM and vhost-user test environment optimizations
- - addition of HW cryptodev devices in LFD FD.io physical testbed
+ - further VM and vhost-user test environment optimizations - Qemu virtio queue size increased from default value of 256 to 1024
+ - addition of HW cryptodev devices in all three LF FD.io physical testbeds
#. Added tests
@@ -37,7 +35,7 @@ Performance Improvements Substantial improvements in measured packet throughput, with relative increase
of double-digit percentage points, have been observed in a number of CSIT
|release| tests listed below. Relative improvements are calculated against the
-test results in CSIT rls1609 report.
+test results in CSIT rls1701 report.
NDR Throughput
~~~~~~~~~~~~~~
@@ -119,7 +117,7 @@ Other Performance Changes Other changes in measured packet throughput, with either minor relative
increase or decrease, have been observed in a number of CSIT |release| tests
listed below. Relative changes are calculated against the test results in CSIT
-rls1609 report.
+rls1701 report.
NDR Throughput
~~~~~~~~~~~~~~
@@ -208,21 +206,3 @@ Here is the list of known issues in CSIT |release| for VPP performance tests: | | - 2t2c, 4t4c - for some tested NICs. | |
+---+-------------------------------------------------+-----------------------------------------------------------------+
-Tests to be Added - Cryptodev
------------------------------
-
-DPDK Cryptodev functionality support for both SW and HW crypto devices has
-been introduced in VPP-17.01 release. CSIT functional and performance tests
-have been also developed and merged. However due to the factors beyond CSIT
-project control execution of those tests within the LF FD.io test environment
-still need to complete. Once the results become available, they will be
-published as an addendum to the current version of CSIT |release| report.
-
-Tests to be Added - SNAT44
---------------------------
-
-VPP SNAT44 functionality has been introduced in VPP-17.01 release. CSIT
-performance tests are still in development and integration into LFD FD.io test
-environment. Once the tests are fully integrated and results become available,
-they will be published as an addendum to the current version of CSIT |release|
-report.
diff --git a/docs/report/vpp_performance_tests/documentation.rst b/docs/report/vpp_performance_tests/documentation.rst index 5017580ca0..b282c2a9b6 100644 --- a/docs/report/vpp_performance_tests/documentation.rst +++ b/docs/report/vpp_performance_tests/documentation.rst @@ -2,5 +2,5 @@ Documentation =============
`CSIT VPP Performance Tests Documentation
-<https://docs.fd.io/csit/master/doc/tests.perf.html>`_ contains detailed
+<https://docs.fd.io/csit/rls1704/doc/tests.perf.html>`_ contains detailed
functional description and input parameters for each test case.
diff --git a/docs/report/vpp_performance_tests/overview.rst b/docs/report/vpp_performance_tests/overview.rst index d0ce5e7bcf..0a5525a6d8 100644 --- a/docs/report/vpp_performance_tests/overview.rst +++ b/docs/report/vpp_performance_tests/overview.rst @@ -52,53 +52,56 @@ Going forward CSIT project will be looking to add more hardware into FD.io performance labs to address larger scale multi-interface and multi-NIC
performance testing scenarios.
-For test cases that require DUT (VPP) to communicate with VM over vhost-user
-interfaces, a VM is created on SUT1 and SUT2. DUT (VPP) test topology with VM
-is shown in the figure below including applicable packet flow thru the VM
+For test cases that require DUT (VPP) to communicate with VM(s) over vhost-user
+interfaces, N of VM instances are created on SUT1 and SUT2. For N=1 DUT (VPP) forwards packets between vhostuser and physical interfaces. For N>1 DUT (VPP) a logical service chain forwarding topology is created on DUT (VPP) by applying L2 or IPv4/IPv6 configuration depending on the test suite.
+DUT (VPP) test topology with N VM instances
+is shown in the figure below including applicable packet flow thru the DUTs and VMs
(marked in the figure with ``***``).
::
- +------------------------+ +------------------------+
- | +----------+ | | +----------+ |
- | | VM | | | | VM | |
- | | ****** | | | | ****** | |
- | +--^----^--+ | | +--^----^--+ |
- | *| |* | | *| |* |
- | +------v----v------+ | | +------v----v------+ |
- | | * * |**|***********|**| * * | |
- | | ***** *******<----------------->******* ***** | |
- | | * DUT1 | | | | DUT2 * | |
- | +--^---------------+ | | +---------------^--+ |
- | *| | | |* |
- | *| SUT1 | | SUT2 |* |
- +------------------------+ +------------------^-----+
- *| |*
- *| |*
- *| +-----------+ |*
- *| | | |*
- *+------------------> TG <------------------+*
- ******************* | |********************
- +-----------+
-
-For VM tests, packets are switched by DUT (VPP) twice, hence the
-throughput rates measured by TG (and listed in this report) must be multiplied
-by two to represent the actual DUT aggregate packet forwarding rate.
-
-Note that reported VPP performance results are specific to the SUT tested.
+ +-------------------------+ +-------------------------+
+ | +---------+ +---------+ | | +---------+ +---------+ |
+ | | VM[1] | | VM[N] | | | | VM[1] | | VM[N] | |
+ | | ***** | | ***** | | | | ***** | | ***** | |
+ | +--^---^--+ +--^---^--+ | | +--^---^--+ +--^---^--+ |
+ | *| |* *| |* | | *| |* *| |* |
+ | +--v---v-------v---v--+ | | +--v---v-------v---v--+ |
+ | | * * * * |*|***********|*| * * * * | |
+ | | * ********* ***<-|-----------|->*** ********* * | |
+ | | * DUT1 | | | | DUT2 * | |
+ | +--^------------------+ | | +------------------^--+ |
+ | *| | | |* |
+ | *| SUT1 | | SUT2 |* |
+ +-------------------------+ +-------------------------+
+ *| |*
+ *| |*
+ *| +-----------+ |*
+ *| | | |*
+ *+--------------------> TG <--------------------+*
+ **********************| |**********************
+ +-----------+
+
+For VM tests, packets are switched by DUT (VPP) multiple times: twice for a single VM, three times for two VMs, N+1 times for N VMs.
+Hence the external
+throughput rates measured by TG and listed in this report must be multiplied
+by (N+1) to represent the actual DUT aggregate packet forwarding rate.
+
+CSIT |release|
+
+Note that reported VPP performance results are specific to the SUTs tested.
Current LF FD.io SUTs are based on Intel XEON E5-2699v3 2.3GHz CPUs. SUTs with
other CPUs are likely to yield different results. A good rule of thumb, that
can be applied to estimate VPP packet thoughput for Phy-to-Phy (NIC-to-NIC,
PCI-to-PCI) topology, is to expect the forwarding performance to be
proportional to CPU core frequency, assuming CPU is the only limiting factor
-and all other SUT aspects equal to FD.io CSIT environment. The same rule of
+and all other SUT parameters equivalent to FD.io CSIT environment. The same rule of
thumb can be also applied for Phy-to-VM-to-Phy (NIC-to-VM-to-NIC) topology,
-but due to much higher dependency on very high frequency memory operations and
+but due to much higher dependency on intensive memory operations and
sensitivity to Linux kernel scheduler settings and behaviour, this estimation
may not always yield good enough accuracy.
-Detailed LF FD.io test bed specification and physical topology are described
-in `wiki CSIT LF FDio testbed <https://wiki.fd.io/view/CSIT/CSIT_LF_testbed>`_.
+For detailed LF FD.io test bed specification and physical topology please refer to `LF FDio CSIT testbed wiki page <https://wiki.fd.io/view/CSIT/CSIT_LF_testbed>`_.
Performance Tests Coverage
--------------------------
@@ -109,10 +112,10 @@ Performance tests are split into the two main categories: in accordance to RFC2544.
- NDR - discovery of Non Drop Rate packet throughput, at zero packet loss;
- followed by packet one-way latency measurements at 10%, 50% and 100% of
+ followed by one-way packet latency measurements at 10%, 50% and 100% of
discovered NDR throughput.
- PDR - discovery of Partial Drop Rate, with specified non-zero packet loss
- currently set to 0.5%; followed by packet one-way latency measurements at
+ currently set to 0.5%; followed by one-way packet latency measurements at
100% of discovered PDR throughput.
- Throughput verification - verification of packet forwarding rate against
@@ -180,17 +183,10 @@ continuously. Performance Tests Naming
------------------------
-CSIT |release| introduced a common structured naming convention for all
-performance and functional tests. This change was driven by substantially
-growing number and type of CSIT test cases. Firstly, the original practice did
-not always follow any strict naming convention. Secondly test names did not
-always clearly capture tested packet encapsulations, and the actual type or
-content of the tests. Thirdly HW configurations in terms of NICs, ports and
-their locality were not captured either. These were but few reasons that drove
-the decision to change and define a new more complete and stricter test naming
-convention, and to apply this to all existing and new test cases.
-
-The new naming should be intuitive for majority of the tests. The complete
+CSIT |release| follows a common structured naming convention for all
+performance and system functional tests, introduced in CSIT rls1701.
+
+The naming should be intuitive for majority of the tests. Complete
description of CSIT test naming convention is provided on `CSIT test naming wiki
<https://wiki.fd.io/view/CSIT/csit-test-naming>`_.
@@ -247,9 +243,8 @@ following VPP thread and core configurations: #. 1t1c - 1 VPP worker thread on 1 CPU physical core.
#. 2t2c - 2 VPP worker threads on 2 CPU physical cores.
-#. 4t4c - 4 VPP threads on 4 CPU physical cores.
-Note that in quite a few test cases running VPP on 2 or 4 physical cores hits
+Note that in quite a few test cases running VPP on 2 physical cores hits
the tested NIC I/O bandwidth or packets-per-second limit.
Methodology: Packet Throughput
diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4.rst index 6473422316..dffd2a2f78 100644 --- a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4.rst +++ b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4.rst @@ -8,7 +8,7 @@ configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and their physical CPU core(s) placement. *Title of each graph* is a regex (regular expression) matching all plotted -latency test cases, *X-axis labels* are indeces of csit-vpp-perf-1701 jobs +latency test cases, *X-axis labels* are indeces of csit-vpp-perf-1704 jobs that created result output files used as data sources for the graph, *Y-axis labels* are measured packet Latency [uSec] values, and the *graph legend* identifes the plotted test suites. @@ -16,11 +16,9 @@ legend* identifes the plotted test suites. .. note:: Data sources for reported test results: i) FD.io test executor jobs - `csit-vpp-perf-1701-all - <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and - `csit-vpp-perf-1701-long - <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_ - , ii) archived FD.io jobs test result `output files + `csit-vpp-perf-1704-all + <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_ , + ii) archived FD.io jobs test result `output files <../../_static/archive/>`_. VPP packet latency - running in configuration of **one worker thread (1t) on one @@ -75,29 +73,3 @@ git repository by filtering with specified regex as follows: 10ge2p1x520-ethip4-ip4scale2m-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4scale2m-ndrdisc 40ge2p1xl710-ethip4-ip4base-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4base-ndrdisc -VPP packet latency - running in configuration of **four worker threads (4t) on four -physical cores (4c)** - is presented in the figure below. - -.. raw:: html - - <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-ethip4-ip4-ndrdisc-lat50.html"></iframe> - -*Figure 3. VPP 4threads 4cores - packet latency for Phy-to-Phy IPv4 Routed-Forwarding.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. code-block:: bash - - $ csit/tests/perf - $ grep -E "64B-4t4c-ethip4-ip4[a-z0-9]+-[a-z-]*ndrdisc" * - - 10ge2p1x520-ethip4-ip4base-copwhtlistbase-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-copwhtlistbase-ndrdisc - 10ge2p1x520-ethip4-ip4base-iacldstbase-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-iacldstbase-ndrdisc - 10ge2p1x520-ethip4-ip4base-ipolicemarkbase-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-ipolicemarkbase-ndrdisc - 10ge2p1x520-ethip4-ip4base-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-ndrdisc - 10ge2p1x520-ethip4-ip4scale200k-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4scale200k-ndrdisc - 10ge2p1x520-ethip4-ip4scale20k-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4scale20k-ndrdisc - 10ge2p1x520-ethip4-ip4scale2m-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4scale2m-ndrdisc - 40ge2p1xl710-ethip4-ip4base-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-ndrdisc - diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4_tunnels.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4_tunnels.rst index a41351403f..decef07375 100644 --- a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4_tunnels.rst +++ b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv4_tunnels.rst @@ -8,7 +8,7 @@ configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and their physical CPU core(s) placement. *Title of each graph* is a regex (regular expression) matching all plotted -latency test cases, *X-axis labels* are indeces of csit-vpp-perf-1701 jobs +latency test cases, *X-axis labels* are indeces of csit-vpp-perf-1704 jobs that created result output files used as data sources for the graph, *Y-axis labels* are measured packet Latency [uSec] values, and the *graph legend* identifes the plotted test suites. @@ -16,11 +16,9 @@ legend* identifes the plotted test suites. .. note:: Data sources for reported test results: i) FD.io test executor jobs - `csit-vpp-perf-1701-all - <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and - `csit-vpp-perf-1701-long - <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_ - , ii) archived FD.io jobs test result `output files + `csit-vpp-perf-1704-all + <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_ , + ii) archived FD.io jobs test result `output files <../../_static/archive/>`_. VPP packet latency - running in configuration of **one worker thread (1t) on one @@ -67,25 +65,3 @@ git repository by filtering with specified regex as follows: 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrdisc.robot:| tc07-64B-2t2c-ethip4vxlan-l2bdbasemaclrn-ndrdisc 10ge2p1x520-ethip4vxlan-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-ethip4vxlan-l2xcbase-ndrdisc -VPP packet latency - running in configuration of **four worker threads (4t) on four -physical cores (4c)** - is presented in the figure below. - -.. raw:: html - - <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-ethip4-ndrdisc-lat50.html"></iframe> - -*Figure 3. VPP 4threads 4cores - packet latency for Phy-to-Phy IPv4 Overlay Tunnels.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. code-block:: bash - - $ csit/tests/perf - $ grep -E "64B-4t4c-ethip4[a-z0-9]+-[a-z0-9]*-ndrdisc" * - - 10ge2p1x520-ethip4lispip4-ip4base-ndrdisc.robot:| tc13-64B-4t4c-ethip4lispip4-ip4base-ndrdisc - 10ge2p1x520-ethip4lispip6-ip4base-ndrdisc.robot:| tc13-64B-4t4c-ethip4lispip6-ip4base-ndrdisc - 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-ethip4vxlan-l2bdbasemaclrn-ndrdisc - 10ge2p1x520-ethip4vxlan-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-ethip4vxlan-l2xcbase-ndrdisc - diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6.rst index 762b427349..dd06399861 100644 --- a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6.rst +++ b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6.rst @@ -8,7 +8,7 @@ configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and their physical CPU core(s) placement. *Title of each graph* is a regex (regular expression) matching all plotted -latency test cases, *X-axis labels* are indeces of csit-vpp-perf-1701 jobs +latency test cases, *X-axis labels* are indeces of csit-vpp-perf-1704 jobs that created result output files used as data sources for the graph, *Y-axis labels* are measured packet Latency [uSec] values, and the *graph legend* identifes the plotted test suites. @@ -16,11 +16,9 @@ legend* identifes the plotted test suites. .. note:: Data sources for reported test results: i) FD.io test executor jobs - `csit-vpp-perf-1701-all - <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and - `csit-vpp-perf-1701-long - <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_ - , ii) archived FD.io jobs test result `output files + `csit-vpp-perf-1704-all + <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_ , + ii) archived FD.io jobs test result `output files <../../_static/archive/>`_. VPP packet latency - running in configuration of **one worker thread (1t) on one @@ -74,28 +72,3 @@ git repository by filtering with specified regex as follows: 10ge2p1x520-ethip6-ip6scale2m-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6scale2m-ndrdisc 40ge2p1xl710-ethip6-ip6base-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6base-ndrdisc -VPP packet latency - running in configuration of **four worker threads (4t) on four -physical cores (4c)** - is presented in the figure below. - -.. raw:: html - - <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-4t4c-ethip6-ip6-ndrdisc-lat50.html"></iframe> - -*Figure 3. VPP 4threads 4cores - packet latency for Phy-to-Phy IPv6 Routed-Forwarding.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. code-block:: bash - - $ csit/tests/perf - $ grep -E "78B-4t4c-ethip6-ip6[a-z0-9]+-[a-z-]*ndrdisc" * - - 10ge2p1x520-ethip6-ip6base-copwhtlistbase-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6base-copwhtlistbase-ndrdisc - 10ge2p1x520-ethip6-ip6base-iacldstbase-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6base-iacldstbase-ndrdisc - 10ge2p1x520-ethip6-ip6base-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6base-ndrdisc - 10ge2p1x520-ethip6-ip6scale200k-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6scale200k-ndrdisc - 10ge2p1x520-ethip6-ip6scale20k-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6scale20k-ndrdisc - 10ge2p1x520-ethip6-ip6scale2m-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6scale2m-ndrdisc - 40ge2p1xl710-ethip6-ip6base-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6base-ndrdisc - diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6_tunnels.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6_tunnels.rst index 1b9e5626b6..261f2dfbd4 100644 --- a/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6_tunnels.rst +++ b/docs/report/vpp_performance_tests/packet_latency_graphs/ipv6_tunnels.rst @@ -8,7 +8,7 @@ configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and their physical CPU core(s) placement. *Title of each graph* is a regex (regular expression) matching all plotted -latency test cases, *X-axis labels* are indeces of csit-vpp-perf-1701 jobs +latency test cases, *X-axis labels* are indeces of csit-vpp-perf-1704 jobs that created result output files used as data sources for the graph, *Y-axis labels* are measured packet Latency [uSec] values, and the *graph legend* identifes the plotted test suites. @@ -16,11 +16,9 @@ legend* identifes the plotted test suites. .. note:: Data sources for reported test results: i) FD.io test executor jobs - `csit-vpp-perf-1701-all - <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and - `csit-vpp-perf-1701-long - <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_ - , ii) archived FD.io jobs test result `output files + `csit-vpp-perf-1704-all + <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_ , + ii) archived FD.io jobs test result `output files <../../_static/archive/>`_. VPP packet latency - running in configuration of **one worker thread (1t) on one @@ -63,24 +61,3 @@ git repository by filtering with specified regex as follows: 10ge2p1x520-ethip6lispip4-ip6base-ndrdisc.robot:| tc07-78B-2t2c-ethip6lispip4-ip6base-ndrdisc 10ge2p1x520-ethip6lispip6-ip6base-ndrdisc.robot:| tc07-78B-2t2c-ethip6lispip6-ip6base-ndrdisc -VPP packet latency - running in configuration of **four worker threads (4t) on four -physical cores (4c)** - is presented in the figure below. - -.. raw:: html - - <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-4t4c-ethip6-ndrdisc-lat50.html"></iframe> - -*Figure 3. VPP 4threads 4cores - packet latency for Phy-to-Phy IPv6 Overlay Tunnels.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. code-block:: bash - - $ csit/tests/perf - $ grep -E "78B-4t4c-ethip6[a-z0-9]+-[a-z0-9]*-ndrdisc" * - - 10ge2p1x520-ethip6lispip4-ip6base-ndrdisc.robot:| tc13-78B-4t4c-ethip6lispip4-ip6base-ndrdisc - 10ge2p1x520-ethip6lispip6-ip6base-ndrdisc.robot:| tc13-78B-4t4c-ethip6lispip6-ip6base-ndrdisc - - diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/l2.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/l2.rst index 8354ac5582..98b5a8deec 100644 --- a/docs/report/vpp_performance_tests/packet_latency_graphs/l2.rst +++ b/docs/report/vpp_performance_tests/packet_latency_graphs/l2.rst @@ -8,7 +8,7 @@ configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and their physical CPU core(s) placement. *Title of each graph* is a regex (regular expression) matching all plotted -latency test cases, *X-axis labels* are indeces of csit-vpp-perf-1701 jobs +latency test cases, *X-axis labels* are indeces of csit-vpp-perf-1704 jobs that created result output files used as data sources for the graph, *Y-axis labels* are measured packet Latency [uSec] values, and the *graph legend* identifes the plotted test suites. @@ -16,11 +16,9 @@ legend* identifes the plotted test suites. .. note:: Data sources for reported test results: i) FD.io test executor jobs - `csit-vpp-perf-1701-all - <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and - `csit-vpp-perf-1701-long - <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_ - , ii) archived FD.io jobs test result `output files + `csit-vpp-perf-1704-all + <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_ , + ii) archived FD.io jobs test result `output files <../../_static/archive/>`_. VPP packet latency - running in configuration of **one worker thread (1t) on one @@ -77,30 +75,3 @@ git repository by filtering with specified regex as follows: 40ge2p1xl710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-ndrdisc 40ge2p1xl710-eth-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-ndrdisc -VPP packet latency - running in configuration of **four worker threads (4t) on four -physical cores (4c)** - is presented in the figure below. - -.. raw:: html - - <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-l2-ndrdisc-lat50.html"></iframe> - -*Figure 3. VPP 4threads 4cores - packet latency for Phy-to-Phy L2 Ethernet Switching.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. code-block:: bash - - $ csit/tests/perf - $ grep -E "64B-4t4c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc" * - - 10ge2p1vic1227-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc - 10ge2p1x520-dot1ad-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-dot1ad-l2xcbase-ndrdisc - 10ge2p1x520-dot1q-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-dot1q-l2xcbase-ndrdisc - 10ge2p1x520-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc - 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-ndrdisc - 10ge2p1x710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc - 40ge2p1vic1385-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc - 40ge2p1xl710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc - 40ge2p1xl710-eth-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-ndrdisc - diff --git a/docs/report/vpp_performance_tests/packet_latency_graphs/vm_vhost.rst b/docs/report/vpp_performance_tests/packet_latency_graphs/vm_vhost.rst index 9c070fa3d4..0afd357609 100644 --- a/docs/report/vpp_performance_tests/packet_latency_graphs/vm_vhost.rst +++ b/docs/report/vpp_performance_tests/packet_latency_graphs/vm_vhost.rst @@ -8,7 +8,7 @@ configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and their physical CPU core(s) placement. *Title of each graph* is a regex (regular expression) matching all plotted -latency test cases, *X-axis labels* are indeces of csit-vpp-perf-1701 jobs +latency test cases, *X-axis labels* are indeces of csit-vpp-perf-1704 jobs that created result output files used as data sources for the graph, *Y-axis labels* are measured packet Latency [uSec] values, and the *graph legend* identifes the plotted test suites. @@ -16,11 +16,9 @@ legend* identifes the plotted test suites. .. note:: Data sources for reported test results: i) FD.io test executor jobs - `csit-vpp-perf-1701-all - <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and - `csit-vpp-perf-1701-long - <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_ - , ii) archived FD.io jobs test result `output files + `csit-vpp-perf-1704-all + <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_ , + ii) archived FD.io jobs test result `output files <../../_static/archive/>`_. VPP packet latency - running in configuration of **one worker thread (1t) on one @@ -75,29 +73,3 @@ git repository by filtering with specified regex as follows: 10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc 40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc -VPP packet latency - running in configuration of **four worker threads (4t) on four -physical cores (4c)** - is presented in the figure below. - -.. raw:: html - - <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-vhost-ndrdisc-lat50.html"></iframe> - -*Figure 3. VPP 4threads 4cores - packet latency for Phy-to-VM-to-Phy VM vhost-user.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. code-block:: bash - - $ csit/tests/perf - $ grep -E "64B-4t4c-.*vhost.*-ndrdisc" * - - 10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc - 10ge2p1x520-dot1q-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc - 10ge2p1x520-ethip4-ip4base-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-eth-2vhost-1vm-ndrdisc - 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc - 10ge2p1x520-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc - 10ge2p1x520-eth-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc - 10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc - 40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc - diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4.rst index 169d07a332..c5cce0d274 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4.rst @@ -8,7 +8,7 @@ configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and their physical CPU core(s) placement. *Title of each graph* is a regex (regular expression) matching all plotted -throughput test cases, *X-axis labels* are indeces of csit-vpp-perf-1701 jobs +throughput test cases, *X-axis labels* are indeces of csit-vpp-perf-1704 jobs that created result output files used as data sources for the graph, *Y-axis labels* are measured Packets Per Second [pps] values, and the *graph legend* identifes the plotted test suites. @@ -16,11 +16,9 @@ legend* identifes the plotted test suites. .. note:: Data sources for reported test results: i) FD.io test executor jobs - `csit-vpp-perf-1701-all - <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and - `csit-vpp-perf-1701-long - <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_ - , ii) archived FD.io jobs test result `output files + `csit-vpp-perf-1704-all + <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_ , + ii) archived FD.io jobs test result `output files <../../_static/archive/>`_. NDR Throughput @@ -79,33 +77,6 @@ git repository by filtering with specified regex as follows: 10ge2p1x520-ethip4-ip4scale2m-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4scale2m-ndrdisc 40ge2p1xl710-ethip4-ip4base-ndrdisc.robot:| tc07-64B-2t2c-ethip4-ip4base-ndrdisc -VPP NDR Throughput - running in configuration of **four worker threads (4t) on -four physical cores (4c)** - is presented in the figure below. - -.. raw:: html - - <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-ethip4-ip4-ndrdisc.html"></iframe> - -*Figure 3. VPP 4threads 4cores - NDR Throughput for Phy-to-Phy IPv4 -Routed-Forwarding.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. code-block:: bash - - $ csit/tests/perf - $ grep -E "64B-4t4c-ethip4-ip4[a-z0-9]+-[a-z-]*ndrdisc" * - - 10ge2p1x520-ethip4-ip4base-copwhtlistbase-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-copwhtlistbase-ndrdisc - 10ge2p1x520-ethip4-ip4base-iacldstbase-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-iacldstbase-ndrdisc - 10ge2p1x520-ethip4-ip4base-ipolicemarkbase-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-ipolicemarkbase-ndrdisc - 10ge2p1x520-ethip4-ip4base-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-ndrdisc - 10ge2p1x520-ethip4-ip4scale200k-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4scale200k-ndrdisc - 10ge2p1x520-ethip4-ip4scale20k-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4scale20k-ndrdisc - 10ge2p1x520-ethip4-ip4scale2m-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4scale2m-ndrdisc - 40ge2p1xl710-ethip4-ip4base-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-ndrdisc - PDR Throughput ~~~~~~~~~~~~~~ @@ -117,7 +88,7 @@ packet loss ratio. <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-1t1c-ethip4-ip4-pdrdisc.html"></iframe> -*Figure 4. VPP 1thread 1core - PDR Throughput for Phy-to-Phy IPv4 +*Figure 3. VPP 1thread 1core - PDR Throughput for Phy-to-Phy IPv4 Routed-Forwarding.* CSIT test cases used to generate results presented above can be found in CSIT @@ -143,7 +114,7 @@ two physical cores (2c)** - is presented in the figure below. <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-2t2c-ethip4-ip4-pdrdisc.html"></iframe> -*Figure 5. VPP 2thread 2core - PDR Throughput for Phy-to-Phy IPv4 +*Figure 4. VPP 2thread 2core - PDR Throughput for Phy-to-Phy IPv4 Routed-Forwarding.* CSIT test cases used to generate results presented above can be found in CSIT @@ -162,29 +133,3 @@ git repository by filtering with specified regex as follows: 10ge2p1x520-ethip4-ip4scale20k-ndrdisc.robot:| tc08-64B-2t2c-ethip4-ip4scale20k-pdrdisc 10ge2p1x520-ethip4-ip4scale2m-ndrdisc.robot:| tc08-64B-2t2c-ethip4-ip4scale2m-pdrdisc -VPP PDR Throughput - running in configuration of **four worker threads (4t) on -four physical cores (4c)** - is presented in the figure below. - -.. raw:: html - - <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-ethip4-ip4-pdrdisc.html"></iframe> - -*Figure 6. VPP 4thread 4core - PDR Throughput for Phy-to-Phy IPv4 -Routed-Forwarding.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. code-block:: bash - - $ csit/tests/perf - $ grep -E "64B-4t4c-ethip4-ip4[a-z0-9]+-[a-z-]*pdrdisc" * - - 10ge2p1x520-ethip4-ip4base-copwhtlistbase-ndrdisc.robot:| tc14-64B-4t4c-ethip4-ip4base-copwhtlistbase-pdrdisc - 10ge2p1x520-ethip4-ip4base-iacldstbase-ndrdisc.robot:| tc14-64B-4t4c-ethip4-ip4base-iacldstbase-pdrdisc - 10ge2p1x520-ethip4-ip4base-ipolicemarkbase-ndrdisc.robot:| tc14-64B-4t4c-ethip4-ip4base-ipolicemarkbase-pdrdisc - 10ge2p1x520-ethip4-ip4base-ndrdisc.robot:| tc14-64B-4t4c-ethip4-ip4base-pdrdisc - 10ge2p1x520-ethip4-ip4scale200k-ndrdisc.robot:| tc14-64B-4t4c-ethip4-ip4scale200k-pdrdisc - 10ge2p1x520-ethip4-ip4scale20k-ndrdisc.robot:| tc14-64B-4t4c-ethip4-ip4scale20k-pdrdisc - 10ge2p1x520-ethip4-ip4scale2m-ndrdisc.robot:| tc14-64B-4t4c-ethip4-ip4scale2m-pdrdisc - diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4_tunnels.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4_tunnels.rst index 131e4e45e2..374505e0db 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4_tunnels.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv4_tunnels.rst @@ -8,7 +8,7 @@ configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and their physical CPU core(s) placement. *Title of each graph* is a regex (regular expression) matching all plotted -throughput test cases, *X-axis labels* are indeces of csit-vpp-perf-1701 jobs +throughput test cases, *X-axis labels* are indeces of csit-vpp-perf-1704 jobs that created result output files used as data sources for the graph, *Y-axis labels* are measured Packets Per Second [pps] values, and the *graph legend* identifes the plotted test suites. @@ -16,11 +16,9 @@ legend* identifes the plotted test suites. .. note:: Data sources for reported test results: i) FD.io test executor jobs - `csit-vpp-perf-1701-all - <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and - `csit-vpp-perf-1701-long - <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_ - , ii) archived FD.io jobs test result `output files + `csit-vpp-perf-1704-all + <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_ , + ii) archived FD.io jobs test result `output files <../../_static/archive/>`_. NDR Throughput @@ -71,29 +69,6 @@ git repository by filtering with specified regex as follows: 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrdisc.robot:| tc07-64B-2t2c-ethip4vxlan-l2bdbasemaclrn-ndrdisc 10ge2p1x520-ethip4vxlan-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-ethip4vxlan-l2xcbase-ndrdisc -VPP NDR Throughput - running in configuration of **four worker threads (4t) on -four physical cores (4c)** - is presented in the figure below. - -.. raw:: html - - <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-ethip4-ndrdisc.html"></iframe> - -*Figure 3. VPP 4threads 4cores - NDR Throughput for Phy-to-Phy IPv4 Overlay -Tunnels.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. code-block:: bash - - $ csit/tests/perf - $ grep -E "64B-4t4c-ethip4[a-z0-9]+-[a-z0-9]*-ndrdisc" * - - 10ge2p1x520-ethip4lispip4-ip4base-ndrdisc.robot:| tc13-64B-4t4c-ethip4lispip4-ip4base-ndrdisc - 10ge2p1x520-ethip4lispip6-ip4base-ndrdisc.robot:| tc13-64B-4t4c-ethip4lispip6-ip4base-ndrdisc - 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-ethip4vxlan-l2bdbasemaclrn-ndrdisc - 10ge2p1x520-ethip4vxlan-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-ethip4vxlan-l2xcbase-ndrdisc - PDR Throughput ~~~~~~~~~~~~~~ @@ -105,7 +80,7 @@ packet loss ratio. <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-1t1c-ethip4-pdrdisc.html"></iframe> -*Figure 4. VPP 1thread 1core - PDR Throughput for Phy-to-Phy IPv4 Overlay +*Figure 3. VPP 1thread 1core - PDR Throughput for Phy-to-Phy IPv4 Overlay Tunnels.* CSIT test cases used to generate results presented above can be found in CSIT @@ -128,7 +103,7 @@ two physical cores (2c)** - is presented in the figure below. <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-2t2c-ethip4-pdrdisc.html"></iframe> -*Figure 5. VPP 2thread 2core - PDR Throughput for Phy-to-Phy IPv4 Overlay Tunnels.* +*Figure 4. VPP 2thread 2core - PDR Throughput for Phy-to-Phy IPv4 Overlay Tunnels.* CSIT test cases used to generate results presented above can be found in CSIT git repository by filtering with specified regex as follows: @@ -143,26 +118,3 @@ git repository by filtering with specified regex as follows: 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrdisc.robot:| tc08-64B-2t2c-ethip4vxlan-l2bdbasemaclrn-pdrdisc 10ge2p1x520-ethip4vxlan-l2xcbase-ndrdisc.robot:| tc08-64B-2t2c-ethip4vxlan-l2xcbase-pdrdisc -VPP PDR Throughput - running in configuration of **four worker threads (4t) on -four physical cores (4c)** - is presented in the figure below. - -.. raw:: html - - <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-ethip4-pdrdisc.html"></iframe> - -*Figure 6. VPP 4thread 4core - PDR Throughput for Phy-to-Phy IPv4 Overlay -Tunnels.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. code-block:: bash - - $ csit/tests/perf - $ grep -E "64B-4t4c-ethip4[a-z0-9]+-[a-z0-9]*-pdrdisc" * - - 10ge2p1x520-ethip4lispip4-ip4base-ndrdisc.robot:| tc14-64B-4t4c-ethip4lispip4-ip4base-pdrdisc - 10ge2p1x520-ethip4lispip6-ip4base-ndrdisc.robot:| tc14-64B-4t4c-ethip4lispip6-ip4base-pdrdisc - 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrdisc.robot:| tc14-64B-4t4c-ethip4vxlan-l2bdbasemaclrn-pdrdisc - 10ge2p1x520-ethip4vxlan-l2xcbase-ndrdisc.robot:| tc14-64B-4t4c-ethip4vxlan-l2xcbase-pdrdisc - diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6.rst index 1230bde94c..09f19a876b 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6.rst @@ -8,7 +8,7 @@ configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and their physical CPU core(s) placement. *Title of each graph* is a regex (regular expression) matching all plotted -throughput test cases, *X-axis labels* are indeces of csit-vpp-perf-1701 jobs +throughput test cases, *X-axis labels* are indeces of csit-vpp-perf-1704 jobs that created result output files used as data sources for the graph, *Y-axis labels* are measured Packets Per Second [pps] values, and the *graph legend* identifes the plotted test suites. @@ -16,11 +16,9 @@ legend* identifes the plotted test suites. .. note:: Data sources for reported test results: i) FD.io test executor jobs - `csit-vpp-perf-1701-all - <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and - `csit-vpp-perf-1701-long - <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_ - , ii) archived FD.io jobs test result `output files + `csit-vpp-perf-1704-all + <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_ , + ii) archived FD.io jobs test result `output files <../../_static/archive/>`_. NDR Throughput @@ -78,32 +76,6 @@ git repository by filtering with specified regex as follows: 10ge2p1x520-ethip6-ip6scale2m-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6scale2m-ndrdisc 40ge2p1xl710-ethip6-ip6base-ndrdisc.robot:| tc07-78B-2t2c-ethip6-ip6base-ndrdisc -VPP NDR Throughput - running in configuration of **four worker threads (4t) on -four physical cores (4c)** - is presented in the figure below. - -.. raw:: html - - <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-4t4c-ethip6-ip6-ndrdisc.html"></iframe> - -*Figure 3. VPP 4threads 4cores - NDR Throughput for Phy-to-Phy IPv6 -Routed-Forwarding.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. code-block:: bash - - $ csit/tests/perf - $ grep -E "78B-4t4c-ethip6-ip6[a-z0-9]+-[a-z-]*ndrdisc" * - - 10ge2p1x520-ethip6-ip6base-copwhtlistbase-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6base-copwhtlistbase-ndrdisc - 10ge2p1x520-ethip6-ip6base-iacldstbase-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6base-iacldstbase-ndrdisc - 10ge2p1x520-ethip6-ip6base-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6base-ndrdisc - 10ge2p1x520-ethip6-ip6scale200k-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6scale200k-ndrdisc - 10ge2p1x520-ethip6-ip6scale20k-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6scale20k-ndrdisc - 10ge2p1x520-ethip6-ip6scale2m-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6scale2m-ndrdisc - 40ge2p1xl710-ethip6-ip6base-ndrdisc.robot:| tc13-78B-4t4c-ethip6-ip6base-ndrdisc - PDR Throughput ~~~~~~~~~~~~~~ @@ -115,7 +87,7 @@ packet loss ratio. <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-1t1c-ethip6-ip6-pdrdisc.html"></iframe> -*Figure 4. VPP 1thread 1core - PDR Throughput for Phy-to-Phy IPv6 +*Figure 3. VPP 1thread 1core - PDR Throughput for Phy-to-Phy IPv6 Routed-Forwarding.* CSIT test cases used to generate results presented above can be found in CSIT @@ -140,7 +112,7 @@ two physical cores (2c)** - is presented in the figure below. <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-2t2c-ethip6-ip6-pdrdisc.html"></iframe> -*Figure 5. VPP 2thread 2core - PDR Throughput for Phy-to-Phy IPv6 +*Figure 4. VPP 2thread 2core - PDR Throughput for Phy-to-Phy IPv6 Routed-Forwarding.* CSIT test cases used to generate results presented above can be found in CSIT @@ -158,29 +130,3 @@ git repository by filtering with specified regex as follows: 10ge2p1x520-ethip6-ip6scale20k-ndrdisc.robot:| tc08-78B-2t2c-ethip6-ip6scale20k-pdrdisc 10ge2p1x520-ethip6-ip6scale2m-ndrdisc.robot:| tc08-78B-2t2c-ethip6-ip6scale2m-pdrdisc -VPP PDR Throughput - running in configuration of **four worker threads (4t) on -four physical cores (4c)** - is presented in the figure below. - -.. raw:: html - - <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-4t4c-ethip6-ip6-pdrdisc.html"></iframe> - -*Figure 6. VPP 4thread 4core - PDR Throughput for Phy-to-Phy IPv6 -Routed-Forwarding.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. code-block:: bash - - $ csit/tests/perf - $ grep -E "78B-4t4c-ethip6-ip6[a-z0-9]+-[a-z-]*pdrdisc" * - - 10ge2p1x520-ethip6-ip6base-copwhtlistbase-ndrdisc.robot:| tc14-78B-4t4c-ethip6-ip6base-copwhtlistbase-pdrdisc - 10ge2p1x520-ethip6-ip6base-iacldstbase-ndrdisc.robot:| tc14-78B-4t4c-ethip6-ip6base-iacldstbase-pdrdisc - 10ge2p1x520-ethip6-ip6base-ndrdisc.robot:| tc14-78B-4t4c-ethip6-ip6base-pdrdisc - 10ge2p1x520-ethip6-ip6scale200k-ndrdisc.robot:| tc14-78B-4t4c-ethip6-ip6scale200k-pdrdisc - 10ge2p1x520-ethip6-ip6scale20k-ndrdisc.robot:| tc14-78B-4t4c-ethip6-ip6scale20k-pdrdisc - 10ge2p1x520-ethip6-ip6scale2m-ndrdisc.robot:| tc14-78B-4t4c-ethip6-ip6scale2m-pdrdisc - - diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6_tunnels.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6_tunnels.rst index fcd5bef2ca..34a7df7e70 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6_tunnels.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/ipv6_tunnels.rst @@ -8,7 +8,7 @@ configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and their physical CPU core(s) placement. *Title of each graph* is a regex (regular expression) matching all plotted -throughput test cases, *X-axis labels* are indeces of csit-vpp-perf-1701 jobs +throughput test cases, *X-axis labels* are indeces of csit-vpp-perf-1704 jobs that created result output files used as data sources for the graph, *Y-axis labels* are measured Packets Per Second [pps] values, and the *graph legend* identifes the plotted test suites. @@ -16,11 +16,9 @@ legend* identifes the plotted test suites. .. note:: Data sources for reported test results: i) FD.io test executor jobs - `csit-vpp-perf-1701-all - <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and - `csit-vpp-perf-1701-long - <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_ - , ii) archived FD.io jobs test result `output files + `csit-vpp-perf-1704-all + <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_ , + ii) archived FD.io jobs test result `output files <../../_static/archive/>`_. NDR Throughput @@ -68,27 +66,6 @@ git repository by filtering with specified regex as follows: 10ge2p1x520-ethip6lispip4-ip6base-ndrdisc.robot:| tc07-78B-2t2c-ethip6lispip4-ip6base-ndrdisc 10ge2p1x520-ethip6lispip6-ip6base-ndrdisc.robot:| tc07-78B-2t2c-ethip6lispip6-ip6base-ndrdisc -VPP NDR Throughput - running in configuration of **four worker threads (4t) on -four physical cores (4c)** - is presented in the figure below. - -.. raw:: html - - <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-4t4c-ethip6-ndrdisc.html"></iframe> - -*Figure 3. VPP 4threads 4cores - NDR Throughput for Phy-to-Phy IPv6 Overlay -Tunnels.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. code-block:: bash - - $ csit/tests/perf - $ grep -E "78B-4t4c-ethip6[a-z0-9]+-[a-z0-9]*-ndrdisc" * - - 10ge2p1x520-ethip6lispip4-ip6base-ndrdisc.robot:| tc13-78B-4t4c-ethip6lispip4-ip6base-ndrdisc - 10ge2p1x520-ethip6lispip6-ip6base-ndrdisc.robot:| tc13-78B-4t4c-ethip6lispip6-ip6base-ndrdisc - PDR Throughput ~~~~~~~~~~~~~~ @@ -100,7 +77,7 @@ packet loss ratio. <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-1t1c-ethip6-pdrdisc.html"></iframe> -*Figure 4. VPP 1thread 1core - PDR Throughput for Phy-to-Phy IPv6 Overlay +*Figure 3. VPP 1thread 1core - PDR Throughput for Phy-to-Phy IPv6 Overlay Tunnels.* CSIT test cases used to generate results presented above can be found in CSIT @@ -121,7 +98,7 @@ two physical cores (2c)** - is presented in the figure below. <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-2t2c-ethip6-pdrdisc.html"></iframe> -*Figure 5. VPP 2thread 2core - PDR Throughput for Phy-to-Phy IPv6 Overlay +*Figure 4. VPP 2thread 2core - PDR Throughput for Phy-to-Phy IPv6 Overlay Tunnels.* CSIT test cases used to generate results presented above can be found in CSIT @@ -134,25 +111,3 @@ git repository by filtering with specified regex as follows: 10ge2p1x520-ethip6lispip4-ip6base-ndrdisc.robot:| tc08-78B-2t2c-ethip6lispip4-ip6base-pdrdisc 10ge2p1x520-ethip6lispip6-ip6base-ndrdisc.robot:| tc08-78B-2t2c-ethip6lispip6-ip6base-pdrdisc - -VPP PDR Throughput - running in configuration of **four worker threads (4t) on -four physical cores (4c)** - is presented in the figure below. - -.. raw:: html - - <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/78B-4t4c-ethip6-pdrdisc.html"></iframe> - -*Figure 6. VPP 4thread 4core - PDR Throughput for Phy-to-Phy IPv6 Overlay -Tunnels.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. code-block:: bash - - $ csit/tests/perf - $ grep -E "78B-4t4c-ethip6[a-z0-9]+-[a-z0-9]*-pdrdisc" * - - 10ge2p1x520-ethip6lispip4-ip6base-ndrdisc.robot:| tc14-78B-4t4c-ethip6lispip4-ip6base-pdrdisc - 10ge2p1x520-ethip6lispip6-ip6base-ndrdisc.robot:| tc14-78B-4t4c-ethip6lispip6-ip6base-pdrdisc - diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/l2.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/l2.rst index e28400ecce..429696bcae 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/l2.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/l2.rst @@ -8,7 +8,7 @@ configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and their physical CPU core(s) placement. *Title of each graph* is a regex (regular expression) matching all plotted -throughput test cases, *X-axis labels* are indeces of csit-vpp-perf-1701 jobs +throughput test cases, *X-axis labels* are indeces of csit-vpp-perf-1704 jobs that created result output files used as data sources for the graph, *Y-axis labels* are measured Packets Per Second [pps] values, and the *graph legend* identifes the plotted test suites. @@ -16,11 +16,9 @@ legend* identifes the plotted test suites. .. note:: Data sources for reported test results: i) FD.io test executor jobs - `csit-vpp-perf-1701-all - <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and - `csit-vpp-perf-1701-long - <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_ - , ii) archived FD.io jobs test result `output files + `csit-vpp-perf-1704-all + <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_ , + ii) archived FD.io jobs test result `output files <../../_static/archive/>`_. NDR Throughput @@ -82,34 +80,6 @@ git repository by filtering with specified regex as follows: 40ge2p1xl710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc07-64B-2t2c-eth-l2bdbasemaclrn-ndrdisc 40ge2p1xl710-eth-l2xcbase-ndrdisc.robot:| tc07-64B-2t2c-eth-l2xcbase-ndrdisc -VPP NDR Throughput - running in configuration of **four worker threads (4t) on -four physical cores (4c)** - is presented in the figure below. - -.. raw:: html - - <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-l2-ndrdisc.html"></iframe> - -*Figure 3. VPP 4threads 4cores - NDR Throughput for Phy-to-Phy L2 Ethernet -Switching.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. code-block:: bash - - $ csit/tests/perf - $ grep -E "64B-4t4c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc" * - - 10ge2p1vic1227-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc - 10ge2p1x520-dot1ad-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-dot1ad-l2xcbase-ndrdisc - 10ge2p1x520-dot1q-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-dot1q-l2xcbase-ndrdisc - 10ge2p1x520-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc - 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-ndrdisc - 10ge2p1x710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc - 40ge2p1vic1385-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc - 40ge2p1xl710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-ndrdisc - 40ge2p1xl710-eth-l2xcbase-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-ndrdisc - PDR Throughput ~~~~~~~~~~~~~~ @@ -121,7 +91,7 @@ loss ratio. <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-1t1c-l2-pdrdisc.html"></iframe> -*Figure 4. VPP 1thread 1core - PDR Throughput for Phy-to-Phy L2 Ethernet +*Figure 3. VPP 1thread 1core - PDR Throughput for Phy-to-Phy L2 Ethernet Switching.* CSIT test cases used to generate results presented above can be found in CSIT @@ -147,7 +117,7 @@ two physical cores (2c)** - is presented in the figure below. <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-2t2c-l2-pdrdisc.html"></iframe> -*Figure 5. VPP 2thread 2core - PDR Throughput for Phy-to-Phy L2 Ethernet +*Figure 4. VPP 2thread 2core - PDR Throughput for Phy-to-Phy L2 Ethernet Switching.* CSIT test cases used to generate results presented above can be found in CSIT @@ -166,28 +136,3 @@ git repository by filtering with specified regex as follows: 10ge2p1x710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc08-64B-2t2c-eth-l2bdbasemaclrn-pdrdisc 40ge2p1vic1385-eth-l2bdbasemaclrn-ndrdisc.robot:| tc08-64B-2t2c-eth-l2bdbasemaclrn-pdrdisc -VPP PDR Throughput - running in configuration of **four worker threads (4t) on -four physical cores (4c)** - is presented in the figure below. - -.. raw:: html - - <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-l2-pdrdisc.html"></iframe> - -*Figure 6. VPP 4thread 4core - PDR Throughput for Phy-to-Phy L2 Ethernet -Switching.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. code-block:: bash - - $ csit/tests/perf - $ grep -E "64B-4t4c-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-pdrdisc" * - - 10ge2p1vic1227-eth-l2bdbasemaclrn-ndrdisc.robot:| tc14-64B-4t4c-eth-l2bdbasemaclrn-pdrdisc - 10ge2p1x520-dot1ad-l2xcbase-ndrdisc.robot:| tc14-64B-4t4c-dot1ad-l2xcbase-pdrdisc - 10ge2p1x520-dot1q-l2xcbase-ndrdisc.robot:| tc14-64B-4t4c-dot1q-l2xcbase-pdrdisc - 10ge2p1x520-eth-l2bdbasemaclrn-ndrdisc.robot:| tc14-64B-4t4c-eth-l2bdbasemaclrn-pdrdisc - 10ge2p1x520-eth-l2xcbase-ndrdisc.robot:| tc14-64B-4t4c-eth-l2xcbase-pdrdisc - 10ge2p1x710-eth-l2bdbasemaclrn-ndrdisc.robot:| tc14-64B-4t4c-eth-l2bdbasemaclrn-pdrdisc - diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/vm_vhost.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/vm_vhost.rst index beadc052d4..9772912d9c 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/vm_vhost.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/vm_vhost.rst @@ -8,7 +8,7 @@ configurations of VPP worker thread(s), a.k.a. VPP data plane thread (s), and their physical CPU core(s) placement. *Title of each graph* is a regex (regular expression) matching all plotted -throughput test cases, *X-axis labels* are indeces of csit-vpp-perf-1701 jobs +throughput test cases, *X-axis labels* are indeces of csit-vpp-perf-1704 jobs that created result output files used as data sources for the graph, *Y-axis labels* are measured Packets Per Second [pps] values, and the *graph legend* identifes the plotted test suites. @@ -16,11 +16,9 @@ legend* identifes the plotted test suites. .. note:: Data sources for reported test results: i) FD.io test executor jobs - `csit-vpp-perf-1701-all - <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-all/>`_ and - `csit-vpp-perf-1701-long - <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1701-long/>`_ - , ii) archived FD.io jobs test result `output files + `csit-vpp-perf-1704-all + <https://jenkins.fd.io/view/csit/job/csit-vpp-perf-1704-all/>`_ , + ii) archived FD.io jobs test result `output files <../../_static/archive/>`_. NDR Throughput @@ -87,26 +85,6 @@ four physical cores (4c)** - is presented in the figure below. <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-vhost-ndrdisc.html"></iframe> -*Figure 3. VPP 4threads 4cores - NDR Throughput for Phy-to-VM-to-Phy VM -vhost-user.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. code-block:: bash - - $ csit/tests/perf - $ grep -E "64B-4t4c-.*vhost.*-ndrdisc" * - - 10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc - 10ge2p1x520-dot1q-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc - 10ge2p1x520-ethip4-ip4base-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-ethip4-ip4base-eth-2vhost-1vm-ndrdisc - 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc - 10ge2p1x520-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc - 10ge2p1x520-eth-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2xcbase-eth-2vhost-1vm-ndrdisc - 10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc - 40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc13-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc - PDR Throughput ~~~~~~~~~~~~~~ @@ -118,7 +96,7 @@ packet loss ratio. <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-1t1c-vhost-pdrdisc.html"></iframe> -*Figure 4. VPP 1thread 1core - PDR Throughput for Phy-to-VM-to-Phy VM +*Figure 3. VPP 1thread 1core - PDR Throughput for Phy-to-VM-to-Phy VM vhost-user.* CSIT test cases used to generate results presented above can be found in CSIT @@ -145,7 +123,7 @@ two physical cores (2c)** - is presented in the figure below. <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-2t2c-vhost-pdrdisc.html"></iframe> -*Figure 5. VPP 2thread 2core - PDR Throughput for Phy-to-VM-to-Phy VM +*Figure 4. VPP 2thread 2core - PDR Throughput for Phy-to-VM-to-Phy VM vhost-user.* CSIT test cases used to generate results presented above can be found in CSIT @@ -165,30 +143,3 @@ git repository by filtering with specified regex as follows: 10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc08-64B-2t2c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc 40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc08-64B-2t2c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc -VPP PDR Throughput - running in configuration of **four worker threads (4t) on -four physical cores (4c)** - is presented in the figure below. - -.. raw:: html - - <iframe width="700" height="700" frameborder="0" scrolling="no" src="../../_static/vpp/64B-4t4c-vhost-pdrdisc.html"></iframe> - -*Figure 6. VPP 4thread 4core - PDR Throughput for Phy-to-VM-to-Phy VM -vhost-user.* - -CSIT test cases used to generate results presented above can be found in CSIT -git repository by filtering with specified regex as follows: - -.. code-block:: bash - - $ csit/tests/perf - $ grep -E "64B-4t4c-.*vhost.*-pdrdisc" * - - 10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc14-64B-4t4c-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc - 10ge2p1x520-dot1q-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc14-64B-4t4c-eth-l2xcbase-eth-2vhost-1vm-pdrdisc - 10ge2p1x520-ethip4-ip4base-eth-2vhost-1vm-ndrdisc.robot:| tc14-64B-4t4c-ethip4-ip4base-eth-2vhost-1vm-pdrdisc - 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc14-64B-4t4c-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc - 10ge2p1x520-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc14-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc - 10ge2p1x520-eth-l2xcbase-eth-2vhost-1vm-ndrdisc.robot:| tc14-64B-4t4c-eth-l2xcbase-eth-2vhost-1vm-pdrdisc - 10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc14-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc - 40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot:| tc14-64B-4t4c-eth-l2bdbasemaclrn-eth-2vhost-1vm-pdrdisc - diff --git a/docs/report/vpp_performance_tests/test_environment.rst b/docs/report/vpp_performance_tests/test_environment.rst index 0fd22213b9..c8ea969728 100644 --- a/docs/report/vpp_performance_tests/test_environment.rst +++ b/docs/report/vpp_performance_tests/test_environment.rst @@ -6,20 +6,22 @@ consists of two SUTs and one TG. SUT Configuration - Host HW
---------------------------
-Hardware details (CPU, memory, NIC layout) are described in
-`CSIT/CSIT_LF_testbed <https://wiki.fd.io/view/CSIT/CSIT_LF_testbed>`_ in
-summary:
+Host hardware details (CPU, memory, NIC layout) and physical topology are
+described in detail in
+`LF FDio CSIT testbed wiki page <https://wiki.fd.io/view/CSIT/CSIT_LF_testbed>`_.
+
+**Host configuration**
- All hosts are Cisco UCS C240-M4 (2x Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz,
18c, 512GB RAM),
-- BIOS settings are default except for the following:
+- BIOS settings are default except the following:
- - Hyperthreading disabled,
- - SpeedStep disabled
- - TurboBoost disabled
- - Power Technology: Performance
+ - Hyperthreading disabled.
+ - SpeedStep disabled.
+ - TurboBoost disabled.
+ - Power Technology: Performance.
-- Hosts run Ubuntu 16.04.1, kernel 4.4.0-42-generic
+- Hosts run Ubuntu 16.04.1, kernel 4.4.0-42-generic.
- Linux kernel boot command line option "intel_pstate=disable" is applied to
both SUTs and TG. In addition, on SUTs, only cores 0 and 18 (the first core on
each socket) are available to the Linux operating system and generic tasks,
@@ -35,7 +37,7 @@ summary: - This allows for a total of five ring topologies, each using ports on
specific NIC model, enabling per NIC model benchmarking.
-**NIC types**
+**NIC models and placement**
- 0a:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+
Network Connection (rev 01) Subsystem: Intel Corporation Ethernet Server
@@ -67,9 +69,10 @@ summary: SUT Configuration - Host OS Linux
---------------------------------
-Software details (OS, configuration) is described in
-`CSIT/CSIT_LF_testbed <https://wiki.fd.io/view/CSIT/CSIT_LF_testbed>`_ in
-summary:
+Software details (OS, configuration) are described in
+`CSIT/CSIT_LF_testbed <https://wiki.fd.io/view/CSIT/CSIT_LF_testbed>`_.
+
+Below a subset of the configuration:
.. code-block:: xml
@@ -2107,10 +2110,10 @@ summary: <exec_command><![CDATA[lsblk -l]]></exec_command>
<exec_return_code>0</exec_return_code>
<exec_output><![CDATA[NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
- sda 8:0 0 1.8T 0 disk
+ sda 8:0 0 1.8T 0 disk
sda1 8:1 0 243M 0 part /boot
sda2 8:2 0 1.8T 0 part /
- sda3 8:3 0 1K 0 part
+ sda3 8:3 0 1K 0 part
sda5 8:5 0 976M 0 part [SWAP]
]]></exec_output>
</function>
@@ -2192,29 +2195,29 @@ summary: dev.cdrom.check_media = 0
dev.cdrom.debug = 0
dev.cdrom.info = CD-ROM information, Id: cdrom.c 3.20 2003/12/17
- dev.cdrom.info =
- dev.cdrom.info = drive name:
- dev.cdrom.info = drive speed:
+ dev.cdrom.info =
+ dev.cdrom.info = drive name:
+ dev.cdrom.info = drive speed:
dev.cdrom.info = drive # of slots:
- dev.cdrom.info = Can close tray:
- dev.cdrom.info = Can open tray:
- dev.cdrom.info = Can lock tray:
+ dev.cdrom.info = Can close tray:
+ dev.cdrom.info = Can open tray:
+ dev.cdrom.info = Can lock tray:
dev.cdrom.info = Can change speed:
dev.cdrom.info = Can select disk:
dev.cdrom.info = Can read multisession:
- dev.cdrom.info = Can read MCN:
+ dev.cdrom.info = Can read MCN:
dev.cdrom.info = Reports media changed:
- dev.cdrom.info = Can play audio:
- dev.cdrom.info = Can write CD-R:
+ dev.cdrom.info = Can play audio:
+ dev.cdrom.info = Can write CD-R:
dev.cdrom.info = Can write CD-RW:
- dev.cdrom.info = Can read DVD:
+ dev.cdrom.info = Can read DVD:
dev.cdrom.info = Can write DVD-R:
dev.cdrom.info = Can write DVD-RAM:
- dev.cdrom.info = Can read MRW:
- dev.cdrom.info = Can write MRW:
- dev.cdrom.info = Can write RAM:
- dev.cdrom.info =
- dev.cdrom.info =
+ dev.cdrom.info = Can read MRW:
+ dev.cdrom.info = Can write MRW:
+ dev.cdrom.info = Can write RAM:
+ dev.cdrom.info =
+ dev.cdrom.info =
dev.cdrom.lock = 0
dev.hpet.max-user-freq = 64
dev.mac_hid.mouse_button2_keycode = 97
@@ -2276,7 +2279,7 @@ summary: kernel.hardlockup_all_cpu_backtrace = 0
kernel.hardlockup_panic = 0
kernel.hostname = t1-sut1
- kernel.hotplug =
+ kernel.hotplug =
kernel.hung_task_check_count = 4194304
kernel.hung_task_panic = 0
kernel.hung_task_timeout_secs = 120
@@ -2577,7 +2580,7 @@ summary: net.ipv4.ip_forward = 0
net.ipv4.ip_forward_use_pmtu = 0
net.ipv4.ip_local_port_range = 32768 60999
- net.ipv4.ip_local_reserved_ports =
+ net.ipv4.ip_local_reserved_ports =
net.ipv4.ip_no_pmtu_disc = 0
net.ipv4.ip_nonlocal_bind = 0
net.ipv4.ipfrag_high_thresh = 4194304
|