diff options
| -rw-r--r-- | docs/report/vpp_performance_tests/packet_throughput_graphs/index.rst | 73 |
1 files changed, 68 insertions, 5 deletions
diff --git a/docs/report/vpp_performance_tests/packet_throughput_graphs/index.rst b/docs/report/vpp_performance_tests/packet_throughput_graphs/index.rst index d5bc33bf3e..aae566e9e4 100644 --- a/docs/report/vpp_performance_tests/packet_throughput_graphs/index.rst +++ b/docs/report/vpp_performance_tests/packet_throughput_graphs/index.rst @@ -8,11 +8,74 @@ Packet Throughput ================= -Throughput graphs are generated by multiple executions of the same -performance tests across physical testbeds hosted LF FD.io labs: 3n-hsw, -2n-skx, 2n-skx. Box-and-Whisker plots are used to display variations in -measured throughput values, without making any assumptions of the -underlying statistical distribution. +Throughput graphs are generated based on the results data obtained from +the |csit-release| test jobs. In order to verify benchmark results +repeatibility selected, CSIT performance tests are executed multiple +times (target: 10 times) on each physical testbed type. Box-and-Whisker +plots are used to display variations in measured throughput values. + +Lists of tests selected for multiple execution and graphing are captured +per testbed type in `test_select_list_{testbed_type}.md +<https://git.fd.io/csit/tree/docs/job_specs>`_ files. + +Graphs are split into sections as follows: + +#. **Level-1: VPP packet path and lookup types** + + - **L2 Ethernet Switching**: L2 bridge-doman, L2 cross-connect and L2 patch + - **IPv4 Routing**: IPv4 routing with /32 prefixes + - **IPv6 Routing**: IPv6 routing with /128 prefixes + - **SRv6 Routing**: SRv6 with IPv6 routing + - **IPv4 Tunnels**: IPv4 overlay tunnels + - **KVM VMs vhost-user**: KVM VMs connected over virtio and vhost-user interfaces + - **LXC/DRC Container Memif**: Linux containers and Docker containers connected over Memif interfaces + - **IPsec IPv4 Routing**: IPsec encryption/decryption with IPv4 routing + - **Virtual Topology System**: VXLAN configurations with L2 bridge-domains + +#. **Level-2: testbeds and NIC models** + + - section name format: {**testbed_type**}-{**nic_model**} + - **testbed_type**: + - 2n-skx: 2-node Xeon Skylake + - 3n-skx: 3-node Xeon Skylake + - 3n-hsw: 3-node Xeon Haswell + - 3n-tsh: 3-node Arm TaiShan + - 2n-dnv: 2-node Atom Denverton + - 3n-dnv: 3-node Atom Denverton + - **nic_model**: + - xxv710: xxv710 2p25GE Intel (Fortville) + - x710: x710 4p10GE Intel (Fortville) + - xl710: xl710 2p40GE Intel (Fortville) + - x520: x520 2p10GE Intel (Niantic) + - x553: x553 2p10GE Intel (Niantic) + +#. **Level-3: test group names** + + - section name format: {**frame_size**}-{**worker_thread_core_cfg**}-{**vpp_functionality**}-{**vpp_lookup_type**}-{**baseline_scale**}-{**nic_driver**} + - **frame_size**: + - 64b: 64 byte frames, smallest frame size for untagged IPv4 packets + - 78b: 78 byte frames, smallest frame size for untagged IPv6 packets + - 114b: VXLAN encapsulated L2 frames + - imix: a sequence of (7x64B, 4x570, 1x1518) byte frames + - **worker_thread_core_cfg**: + - 1t1c: 1 worker thread on 1 core, hyper-threading not used + - 2t1c: 2 worker threads on 1 core, hyper-threading used + - **vpp_functionality** (optional): + - features: including input-acl, output-acl, macip-iacl, nat44 + - srv6: srv6 encap/decap, proxy + - link-bonding: L2 link aggregation with 1 or 2 bonded links + - ipsec: IPsec encryption/decryption with different ciphers + - vts: Virtual Topology System specific tests + - **vpp_lookup_type**: + - l2switching, ip4routing, ip6routing, ip4tunnel, vhost, memif + - **baseline_scale**: + - base: baseline tests with less than 10 forwarding entries + - scale: scale tests with up to 2 million forwarding entries + - base-scale: both baseline and scale tests grouped together + - **nic_driver**: + - avf: VPP native avf driver for Intel Fortville NICs + - i40e: dpdk poll mode driver for Intel Fortville NICs + - ixgbe: dpdk poll mode driver for Intel Niantic NICs For each test case, Box-and-Whisker plots show the quartiles (Min, 1st quartile / 25th percentile, 2nd quartile / 50th percentile / mean, 3rd |