diff options
Diffstat (limited to 'docs/report')
4 files changed, 38 insertions, 45 deletions
diff --git a/docs/report/introduction/methodology_kvm_vms_vhost_user.rst b/docs/report/introduction/methodology_kvm_vms_vhost_user.rst index 0a465cf0fb..299f708827 100644 --- a/docs/report/introduction/methodology_kvm_vms_vhost_user.rst +++ b/docs/report/introduction/methodology_kvm_vms_vhost_user.rst @@ -3,10 +3,7 @@ KVM VMs vhost-user QEMU is used for VPP-VM testing enviroment. By default, standard QEMU version preinstalled from OS repositories is used on VIRL/vpp_device functional testing -(qemu-2.11.x for Ubuntu 18.04, qemu-2.5.0 for Ubuntu 16.04). For perfomance -testing QEMU is downloaded from `project homepage <qemu.org>`_ and compiled -during testing. This allows framework to easily inject QEMU patches in case of -need. In QEMU version <2.8 we used it for increasing QEMU virtion queue size. +(qemu-2.11.x for Ubuntu 18.04, qemu-2.5.0 for Ubuntu 16.04). In CSIT setup DUTs have small VM image `/var/lib/vm/vhost-nested.img`. QEMU binary can be adjusted in global settings. VM image must have installed at least qemu-guest-agent, sshd, bridge-utils, VirtIO support and Testpmd/L3fwd diff --git a/docs/report/introduction/methodology_nfv_service_density.rst b/docs/report/introduction/methodology_nfv_service_density.rst index 51e56e294d..23bdac9d7c 100644 --- a/docs/report/introduction/methodology_nfv_service_density.rst +++ b/docs/report/introduction/methodology_nfv_service_density.rst @@ -22,14 +22,12 @@ different service density setups by varying two parameters: The initial implementation of NFV service density tests in |csit-release| is using two NF applications: -- VNF: DPDK L3fwd running in KVM VM, configured with /8 IPv4 prefix - routing. L3fwd got chosen as a lightweight fast IPv4 VNF application, - and follows CSIT approach of using DPDK sample applications in VMs for - performance testing. -- CNF: VPP running in Docker Container, configured with /24 IPv4 prefix - routing. VPP got chosen as a fast IPv4 NF application that supports - required memif interface (L3fwd does not). This is similar to all - other Container tests in CSIT that use VPP. +- VNF: VPP of the same version as vswitch running in KVM VM, configured with /8 + IPv4 prefix routing. +- CNF: VPP of the same version as vswitch running in Docker Container, + configured with /8 IPv4 prefix routing. VPP got chosen as a fast IPv4 NF + application that supports required memif interface (L3fwd does not). This is + similar to all other Container tests in CSIT that use VPP. Tests are designed such that in all tested cases VPP vswitch is the most stressed application, as for each flow vswitch is processing each packet @@ -84,22 +82,30 @@ physical core mapping ratios: - Data-plane on single core + - (main:core) = (1:1) => 1mt1c - 1 main thread on 1 core. - (data:core) = (1:1) => 2dt1c - 2 Data-plane Threads on 1 Core. - - (main:core) = (1:1) => 1mt1c - 1 Main Thread on 1 Core. - Data-plane on two cores - - (data:core) = (1:2) => 4dt2c - 4 Data-plane Threads on 2 Cores. - (main:core) = (1:1) => 1mt1c - 1 Main Thread on 1 Core. + - (data:core) = (1:2) => 4dt2c - 4 Data-plane Threads on 2 Cores. - VNF and CNF - Data-plane on single core + - (main:core) = (2:1) => 2mt1c - 2 Main Threads on 1 Core, 1 Thread + per NF, core shared between two NFs. - (data:core) = (1:1) => 2dt1c - 2 Data-plane Threads on 1 Core per NF. + + - Data-plane on single logical core (Two NFs per physical core) + - (main:core) = (2:1) => 2mt1c - 2 Main Threads on 1 Core, 1 Thread per NF, core shared between two NFs. + - (data:core) = (2:1) => 1dt1c - 1 Data-plane Threads on 1 Core per + NF. + Maximum tested service densities are limited by a number of physical cores per NUMA. |csit-release| allocates cores within NUMA0. Support for diff --git a/docs/report/introduction/methodology_vpp_startup_settings.rst b/docs/report/introduction/methodology_vpp_startup_settings.rst index 2b0f485cc0..81c5e4e73e 100644 --- a/docs/report/introduction/methodology_vpp_startup_settings.rst +++ b/docs/report/introduction/methodology_vpp_startup_settings.rst @@ -18,8 +18,12 @@ List of vpp startup.conf settings applied to all tests: #. no-tx-checksum-offload - disables UDP / TCP TX checksum offload in DPDK. Typically needed for use faster vector PMDs (together with no-multi-seg). -#. socket-mem <value>,<value> - memory per numa. (Not required anymore - due to VPP code changes, will be removed in CSIT-19.04.) +#. buffers-per-numa <value> - increases number of buffers allocated, needed + in scenarios with large number of interfaces and worker threads. + Value is per CPU socket. Default is 16384. CSIT is setting statically + 107520 buffers per CPU thread (215040 if HTT is enabled). This value is also + maximum possible amount limited by number of memory mappings in DPDK + libraries for 2MB Hugepages used in CSIT. Per Test Settings ~~~~~~~~~~~~~~~~~ @@ -31,11 +35,6 @@ List of vpp startup.conf settings applied dynamically per test: test configuration. #. num-rx-queues <value> - depends on a number of VPP threads and NIC interfaces. -#. num-rx-desc/num-tx-desc - number of rx/tx descriptors for specific - NICs, incl. xl710, x710, xxv710. -#. num-mbufs <value> - increases number of buffers allocated, needed - only in scenarios with large number of interfaces and worker threads. - Value is per CPU socket. Default is 16384. #. no-multi-seg - disables multi-segment buffers in DPDK, improves packet throughput, but disables Jumbo MTU support. Disabled for all tests apart from the ones that require Jumbo 9000B frame support. diff --git a/docs/report/vpp_performance_tests/csit_release_notes.rst b/docs/report/vpp_performance_tests/csit_release_notes.rst index 3d5465e7c6..70092ad098 100644 --- a/docs/report/vpp_performance_tests/csit_release_notes.rst +++ b/docs/report/vpp_performance_tests/csit_release_notes.rst @@ -6,13 +6,8 @@ Changes in |csit-release| #. VPP PERFORMANCE TESTS - - **Service density 2n-skx tests**: Network Function Virtualization (NFV) - service density tests focus on measuring total per server throughput at - varied NFV service *packing* densities with vswitch providing host - dataplane. The goal is to compare and contrast performance of a shared - vswitch for different network topologies and virtualization technologies, - and their impact on vswitch performance and efficiency in a range of NFV - service configurations. + - **Service density 2n-skx tests**: Added higher density tests with dtc=0.5 + (2 NF each with 1 DT per physical core) with VPP as a VNF payload. - **Experimental Soak Tests**: Added performamce soak tests framework code for extended time duration tests and throughput discovery @@ -22,16 +17,24 @@ Changes in |csit-release| #. TEST FRAMEWORK - - **Container code optimizations**: Optimized container library allows to - run containre_memif tests faster. + - **Qemu code refactor**: Complete code refactor of the key components + QemuUtil.py and QemuManager.py (L1 and L2 KW counterparts). Added + implementation of kernel-image-kvm based VM alongisde the previously used + NestedVM images. Added ability to run VPP as a payload in VNF. - **CSIT PAPI Support**: Continue converting existing VAT L1 keywords to PAPI L1 KWs in CSIT using VPP Python bindings. Required for migrating away - from VAT. + from VAT. Redesign of key components of PAPI Executor and PAPI history. - **General Code Housekeeping**: Ongoing RF keywords optimizations, removal of redundant RF keywords. + - **Test suite generator**: Extend the test suite generator for ability to + generate NIC permutation and search algorithm permutations from base + suites. + + - **TOX verification**: Added verifications for test suite generator. + #. PRESENTATION AND ANALYTICS LAYER - **Graphs Layout Improvements**: Improved performance graphs layout @@ -64,18 +67,6 @@ List of known issues in |csit-release| for VPP performance tests: | 1 | `CSIT-570 | Sporadic (1 in 200) NDR discovery test failures on x520. DPDK reporting rx-errors, indicating L1 issue. | | | <https://jira.fd.io/browse/CSIT-570>`_ | Suspected issue with HW combination of X710-X520 in LF testbeds. Not observed outside of LF testbeds. | +----+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+ -| 2 | `VPP-1563 | AVF L2patch tests are failing for all packet size and core combination. Reason: null-node blackholed packets in show error. | -| | <https://jira.fd.io/browse/VPP-1563>`_ | | -+----+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+ -| 3 | `CSIT-1234 | VPP IPSecHW/IPSecsW scale interface mode, low NDR and PDR 64B throughput in 3n-hsw testbeds, in CSIT-19.01 vs. CSIT-18.10. | -| | <https://jira.fd.io/browse/CSIT-1234>`_ | | -+----+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+ -| 4 | `CSIT-1431 | AVF 4 cores tests are sporadically failing. Under investigation. | -| | <https://jira.fd.io/browse/CSIT-1431>`_ | | -+----+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+ -| 5 | `CSIT-1465 | 4c VPP VM vhost tests failing on 3n-skx. Under investigation. | -| | <https://jira.fd.io/browse/CSIT-1465>`_ | | -+----+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+ -| 6 | `CSIT-1466 | IPSecHW scale tests failing due to VPP reset. Fixed in one of subsequent VPP patches. Confirmed by running tests with VPP | -| | <https://jira.fd.io/browse/CSIT-1466>`_ | build 19.01.1-8~g50a392f~b56. | +| 2 | `CSIT-???? | IPSecHW interface AES-GBC tests failing due to traffic not passing. | +| | <https://jira.fd.io/browse/CSIT-????>`_ | | +----+-----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+ |