diff options
Diffstat (limited to 'docs/content')
85 files changed, 15054 insertions, 0 deletions
diff --git a/docs/content/_index.md b/docs/content/_index.md new file mode 100644 index 0000000000..f2736d5a57 --- /dev/null +++ b/docs/content/_index.md @@ -0,0 +1,49 @@ +--- +title: "FD.io CSIT" +type: "docs" +--- + +# Documentation Structure + +1. OVERVIEW: General introduction to CSIT Performance Dashboard and CSIT itself. + - **[CSIT-Dash]({{< relref "/overview/c_dash/design" >}})**: The design and + the structure of CSIT-Dash dashboard. + - **[CSIT]({{< relref "/overview/csit/" >}})**: The design of the + [FD.io](https://fd.io/) CSIT system, and the description of the test + scenarios, test naming and test tags. +2. METHODOLOGY + - **[Overview]({{< relref "/methodology/overview/" >}})**: Terminology, + per-thread resources, multi-core speedup, VPP forwarding modes and DUT + state considerations. + - **[Measurement]({{< relref "/methodology/measurements/" >}})**: Data plane + throughput, packet latency and the telemetry. + - **[Test]({{< relref "/methodology/test/" >}})**: Methodology of all tests + used in CSIT. + - **[Trending]({{< relref "/methodology/trending/" >}})**: A high-level + design of a system for continuous performance measuring, trending and + change detection for FD.io VPP SW data plane (and other performance tests + run within CSIT sub-project). + - **[Per-patch Testing]({{< relref "/methodology/per_patch_testing" >}})**: + A methodology similar to trending analysis is used for comparing + performance before a DUT code change is merged. +3. RELEASE NOTES: Performance tests executed in physical FD.io testbeds. + - **[{{< release_csit >}}]({{< relref "/release_notes/current/" >}})**: The + release notes of the current CSIT release. + - **[Previous]({{< relref "/release_notes/previous/" >}})**: Archived release + notes for the past releases. +4. INFRASTRUCTURE + - **[FD.io DC Vexxhost Inventory]({{< relref "/infrastructure/fdio_dc_vexxhost_inventory" >}})**: + Physical testbeds location. + - **[FD.io DC Testbed Specifications]({{< relref "/infrastructure/fdio_dc_testbed_specifications" >}})**: + Specification of the physical testbed infrastructure. + - **[FD.io DC Testbed Configuration]({{< relref "/infrastructure/testbed_configuration/" >}})**: + Configuration of the physical testbed infrastructure. + - **[FD.io CSIT Testbed Versioning]({{< relref "/infrastructure/fdio_csit_testbed_versioning" >}})**: + CSIT test environment versioning to track modifications of the test + environment. + - **[FD.io CSIT Logical Topologies]({{< relref "/infrastructure/fdio_csit_logical_topologies" >}})**: + CSIT performance tests are executed on physical testbeds. Based on the + packet path thru server SUTs, three distinct logical topology types are + used for VPP DUT data plane testing. + - **[VPP Startup Settings]({{< relref "/infrastructure/vpp_startup_settings" >}})**: + List of common settings applied to all tests and test dependent settings.
\ No newline at end of file diff --git a/docs/content/infrastructure/_index.md b/docs/content/infrastructure/_index.md new file mode 100644 index 0000000000..2402726c95 --- /dev/null +++ b/docs/content/infrastructure/_index.md @@ -0,0 +1,6 @@ +--- +bookCollapseSection: false +bookFlatSection: true +title: "Infrastructure" +weight: 4 +---
\ No newline at end of file diff --git a/docs/content/infrastructure/fdio_csit_logical_topologies.md b/docs/content/infrastructure/fdio_csit_logical_topologies.md new file mode 100644 index 0000000000..4e9c22b357 --- /dev/null +++ b/docs/content/infrastructure/fdio_csit_logical_topologies.md @@ -0,0 +1,138 @@ +--- +title: "FD.io CSIT Logical Topologies" +weight: 5 +--- + +# FD.io CSIT Logical Topologies + +CSIT VPP performance tests are executed on physical testbeds. Based on the +packet path thru server SUTs, three distinct logical topology types are used +for VPP DUT data plane testing: + +1. NIC-to-NIC switching topologies. +2. VM service switching topologies. +3. Container service switching topologies. + +## NIC-to-NIC Switching + +The simplest logical topology for software data plane application like +VPP is NIC-to-NIC switching. Tested topologies for 2-Node and 3-Node +testbeds are shown in figures below. + +{{< figure src="/cdocs/logical-2n-nic2nic.svg" >}} + +{{< figure src="/cdocs/logical-3n-nic2nic.svg" >}} + +Server Systems Under Test (SUT) run VPP application in Linux user-mode +as a Device Under Test (DUT). Server Traffic Generator (TG) runs T-Rex +application. Physical connectivity between SUTs and TG is provided using +different drivers and NIC models that need to be tested for performance +(packet/bandwidth throughput and latency). + +From SUT and DUT perspectives, all performance tests involve forwarding +packets between two (or more) physical Ethernet ports (10GE, 25GE, 40GE, +100GE). In most cases both physical ports on SUT are located on the same +NIC. The only exceptions are link bonding and 100GE tests. In the latter +case only one port per NIC can be driven at linerate due to PCIe Gen3 +x16 slot bandwidth limiations. 100GE NICs are not supported in PCIe Gen3 +x8 slots. + +Note that reported VPP DUT performance results are specific to the SUTs +tested. SUTs with other processors than the ones used in FD.io lab are +likely to yield different results. A good rule of thumb, that can be +applied to estimate VPP packet thoughput for NIC-to-NIC switching +topology, is to expect the forwarding performance to be proportional to +processor core frequency for the same processor architecture, assuming +processor is the only limiting factor and all other SUT parameters are +equivalent to FD.io CSIT environment. + +## VM Service Switching + +VM service switching topology test cases require VPP DUT to communicate +with Virtual Machines (VMs) over vhost-user virtual interfaces. + +Two types of VM service topologies are tested: + +1. "Parallel" topology with packets flowing within SUT from NIC(s) via + VPP DUT to VM, back to VPP DUT, then out thru NIC(s). +2. "Chained" topology (a.k.a. "Snake") with packets flowing within SUT + from NIC(s) via VPP DUT to VM, back to VPP DUT, then to the next VM, + back to VPP DUT and so on and so forth until the last VM in a chain, + then back to VPP DUT and out thru NIC(s). + +For each of the above topologies, VPP DUT is tested in a range of L2 +or IPv4/IPv6 configurations depending on the test suite. Sample VPP DUT +"Chained" VM service topologies for 2-Node and 3-Node testbeds with each +SUT running N of VM instances is shown in the figures below. + +{{< figure src="/cdocs/logical-2n-vm-vhost.svg" >}} + +{{< figure src="/cdocs/logical-3n-vm-vhost.svg" >}} + +In "Chained" VM topologies, packets are switched by VPP DUT multiple +times: twice for a single VM, three times for two VMs, N+1 times for N +VMs. Hence the external throughput rates measured by TG and listed in +this report must be multiplied by N+1 to represent the actual VPP DUT +aggregate packet forwarding rate. + +For "Parallel" service topology packets are always switched twice by VPP +DUT per service chain. + +Note that reported VPP DUT performance results are specific to the SUTs +tested. SUTs with other processor than the ones used in FD.io lab are +likely to yield different results. Similarly to NIC-to-NIC switching +topology, here one can also expect the forwarding performance to be +proportional to processor core frequency for the same processor +architecture, assuming processor is the only limiting factor. However +due to much higher dependency on intensive memory operations in VM +service chained topologies and sensitivity to Linux scheduler settings +and behaviour, this estimation may not always yield good enough +accuracy. + +## Container Service Switching + +Container service switching topology test cases require VPP DUT to +communicate with Containers (Ctrs) over memif virtual interfaces. + +Three types of VM service topologies are tested in |csit-release|: + +1. "Parallel" topology with packets flowing within SUT from NIC(s) via + VPP DUT to Container, back to VPP DUT, then out thru NIC(s). +2. "Chained" topology (a.k.a. "Snake") with packets flowing within SUT + from NIC(s) via VPP DUT to Container, back to VPP DUT, then to the + next Container, back to VPP DUT and so on and so forth until the + last Container in a chain, then back to VPP DUT and out thru NIC(s). +3. "Horizontal" topology with packets flowing within SUT from NIC(s) via + VPP DUT to Container, then via "horizontal" memif to the next + Container, and so on and so forth until the last Container, then + back to VPP DUT and out thru NIC(s). + +For each of the above topologies, VPP DUT is tested in a range of L2 +or IPv4/IPv6 configurations depending on the test suite. Sample VPP DUT +"Chained" Container service topologies for 2-Node and 3-Node testbeds +with each SUT running N of Container instances is shown in the figures +below. + +{{< figure src="/cdocs/logical-2n-container-memif.svg" >}} + +{{< figure src="/cdocs/logical-3n-container-memif.svg" >}} + +In "Chained" Container topologies, packets are switched by VPP DUT +multiple times: twice for a single Container, three times for two +Containers, N+1 times for N Containers. Hence the external throughput +rates measured by TG and listed in this report must be multiplied by N+1 +to represent the actual VPP DUT aggregate packet forwarding rate. + +For a "Parallel" and "Horizontal" service topologies packets are always +switched by VPP DUT twice per service chain. + +Note that reported VPP DUT performance results are specific to the SUTs +tested. SUTs with other processor than the ones used in FD.io lab are +likely to yield different results. Similarly to NIC-to-NIC switching +topology, here one can also expect the forwarding performance to be +proportional to processor core frequency for the same processor +architecture, assuming processor is the only limiting factor. However +due to much higher dependency on intensive memory operations in +Container service chained topologies and sensitivity to Linux scheduler +settings and behaviour, this estimation may not always yield good enough +accuracy. diff --git a/docs/content/infrastructure/fdio_csit_testbed_versioning.md b/docs/content/infrastructure/fdio_csit_testbed_versioning.md new file mode 100644 index 0000000000..7f6cdfc51c --- /dev/null +++ b/docs/content/infrastructure/fdio_csit_testbed_versioning.md @@ -0,0 +1,146 @@ +--- +bookToc: true +title: "FD.io CSIT Testbed Versioning" +weight: 4 +--- + +# FD.io CSIT Testbed Versioning + +CSIT test environment versioning has been introduced to track modifications of +the test environment. + +Any benchmark anomalies (progressions, regressions) between releases of a DUT +application (e.g. VPP, DPDK), are determined by testing it in the same test +environment, to avoid test environment changes clouding the picture. +To beter distinguish impact of test environment changes, we also execute tests +without any SUT (just with TRex TG sending packets over a link looping back to +TG). + +A mirror approach is introduced to determine benchmarking anomalies due to the +test environment change. This is achieved by testing the same DUT application +version between releases of CSIT test system. This works under the assumption +that the behaviour of the DUT is deterministic under the test conditions. + +CSIT test environment versioning scheme ensures integrity of all the test system +components, including their HW revisions, compiled SW code versions and SW +source code, within a specific CSIT version. Components included in the CSIT +environment versioning include: + +- **HW** Server hardware firmware and BIOS (motherboard, processsor, + NIC(s), accelerator card(s)), tracked in CSIT branch. +- **Linux** Server Linux OS version and configuration, tracked in CSIT + Reports. +- **TRex** TRex Traffic Generator version, drivers and configuration + tracked in TG Settings. +- **CSIT** CSIT framework code tracked in CSIT release branches. + +Following is the list of CSIT versions to date: + +- Ver. 14 associated with CSIT rls2402 branch ( + [HW](https://git.fd.io/csit/tree/docs/content/infrastructure/testbed_configuration?h=rls2402), + [CSIT](https://git.fd.io/csit/tree/?h=rls2402) + ). + - Intel NIC 700/800 series firmware upgrade based on DPDK compatibility + matrix. + - Mellanox 556A/CX6-DX/MCX713106AS-VEAT series firmware upgrade based on DPDK + compatibility matrix. +- Ver. 13 associated with CSIT rls2310 branch ( + [HW](https://git.fd.io/csit/tree/docs/content/infrastructure/testbed_configuration?h=rls2310), + [CSIT](https://git.fd.io/csit/tree/?h=rls2310) + ). + - Intel NIC 700/800 series firmware upgrade based on DPDK compatibility + matrix. + - Mellanox 556A/CX6-DX/MCX713106AS-VEAT series firmware upgrade based on DPDK + compatibility matrix. +- Ver. 12 associated with CSIT rls2306 branch ( + [HW](https://git.fd.io/csit/tree/docs/content/infrastructure/testbed_configuration?h=rls2306), + [CSIT](https://git.fd.io/csit/tree/?h=rls2306) + ). + - Intel NIC 700/800 series firmware upgrade based on DPDK compatibility + matrix. + - Mellanox 556A/CX6-DX/MCX713106AS-VEAT series firmware upgrade based on DPDK + compatibility matrix. + - TRex version upgrade: increase from 3.00 to 3.03. +- Ver. 11 associated with CSIT rls2210 branch ( + [HW](https://git.fd.io/csit/tree/docs/lab?h=rls2210), + [Linux](https://s3-docs.fd.io/csit/rls2210/report/vpp_performance_tests/test_environment.html#sut-settings-linux), + [TRex](https://s3-docs.fd.io/csit/rls2210/report/vpp_performance_tests/test_environment.html#tg-settings-trex), + [CSIT](https://git.fd.io/csit/tree/?h=rls2210) + ). + - Intel NIC 700/800 series firmware upgrade based on DPDK compatibility + matrix. + - Mellanox 556A series firmware upgrade based on DPDK compatibility + matrix. + - Ubuntu upgrade from 20.04.2 LTS to 22.04.1 LTS. + - TRex version upgrade: increase from 2.97 to 3.00. +- Ver. 10 associated with CSIT rls2206 branch ( + [HW](https://git.fd.io/csit/tree/docs/lab?h=rls2206), + [Linux](https://s3-docs.fd.io/csit/rls2206/report/vpp_performance_tests/test_environment.html#sut-settings-linux), + [TRex](https://s3-docs.fd.io/csit/rls2206/report/vpp_performance_tests/test_environment.html#tg-settings-trex), + [CSIT](https://git.fd.io/csit/tree/?h=rls2206) + ). + - Intel NIC 700/800 series firmware upgrade based on DPDK compatibility + matrix. + - Mellanox 556A series firmware upgrade based on DPDK compatibility + matrix. + - Intel IceLake all core turbo frequency turned off. Current base frequency + is 2.6GHz. + - TRex version upgrade: increase from 2.88 to 2.97. +- Ver. 9 associated with CSIT rls2202 branch ( + [HW](https://git.fd.io/csit/tree/docs/lab?h=rls2202), + [Linux](https://s3-docs.fd.io/csit/rls2202/report/vpp_performance_tests/test_environment.html#sut-settings-linux), + [TRex](https://s3-docs.fd.io/csit/rls2202/report/vpp_performance_tests/test_environment.html#tg-settings-trex), + [CSIT](https://git.fd.io/csit/tree/?h=rls2202) + ). + - Intel NIC 700/800 series firmware upgrade based on DPDK compatibility + matrix. +- Ver. 8 associated with CSIT rls2110 branch ( + [HW](https://git.fd.io/csit/tree/docs/lab?h=rls2110), + [Linux](https://s3-docs.fd.io/csit/rls2110/report/vpp_performance_tests/test_environment.html#sut-settings-linux), + [TRex](https://s3-docs.fd.io/csit/rls2110/report/vpp_performance_tests/test_environment.html#tg-settings-trex), + [CSIT](https://git.fd.io/csit/tree/?h=rls2110) + ). + - Intel NIC 700/800 series firmware upgrade based on DPDK compatibility + matrix. +- Ver. 7 associated with CSIT rls2106 branch ( + [HW](https://git.fd.io/csit/tree/docs/lab?h=rls2106), + [Linux](https://s3-docs.fd.io/csit/rls2106/report/vpp_performance_tests/test_environment.html#sut-settings-linux), + [TRex](https://s3-docs.fd.io/csit/rls2106/report/vpp_performance_tests/test_environment.html#tg-settings-trex), + [CSIT](https://git.fd.io/csit/tree/?h=rls2106) + ). + - TRex version upgrade: increase from 2.86 to 2.88. + - Ubuntu upgrade from 18.04 LTS to 20.04.2 LTS. +- Ver. 6 associated with CSIT rls2101 branch ( + [HW](https://git.fd.io/csit/tree/docs/lab?h=rls2101), + [Linux](https://docs.fd.io/csit/rls2101/report/vpp_performance_tests/test_environment.html#sut-settings-linux), + [TRex](https://docs.fd.io/csit/rls2101/report/vpp_performance_tests/test_environment.html#tg-settings-trex), + [CSIT](https://git.fd.io/csit/tree/?h=rls2101) + ). + - The main change is TRex version upgrade: increase from 2.82 to 2.86. +- Ver. 5 associated with CSIT rls2009 branch ( + [HW](https://git.fd.io/csit/tree/docs/lab?h=rls2009), + [Linux](https://docs.fd.io/csit/rls2009/report/vpp_performance_tests/test_environment.html#sut-settings-linux), + [TRex](https://docs.fd.io/csit/rls2009/report/vpp_performance_tests/test_environment.html#tg-settings-trex), + [CSIT](https://git.fd.io/csit/tree/?h=rls2009) + ). + - The main change is TRex data-plane core resource adjustments: + [increase from 7 to 8 cores and pinning cores to interfaces](https://gerrit.fd.io/r/c/csit/+/28184) + for better TRex performance with symmetric traffic profiles. +- Ver. 4 associated with CSIT rls2005 branch ( + [HW](https://git.fd.io/csit/tree/docs/lab?h=rls2005), + [Linux](https://docs.fd.io/csit/rls2005/report/vpp_performance_tests/test_environment.html#sut-settings-linux), + [TRex](https://docs.fd.io/csit/rls2005/report/vpp_performance_tests/test_environment.html#tg-settings-trex), + [CSIT](https://git.fd.io/csit/tree/?h=rls2005) + ). +- Ver. 2 associated with CSIT rls2001 branch ( + [HW](https://git.fd.io/csit/tree/docs/lab?h=rls2001), + [Linux](https://docs.fd.io/csit/rls2001/report/vpp_performance_tests/test_environment.html#sut-settings-linux), + [TRex](https://docs.fd.io/csit/rls2001/report/vpp_performance_tests/test_environment.html#tg-settings-trex), + [CSIT](https://git.fd.io/csit/tree/?h=rls2001) + ). +- Ver. 1 associated with CSIT rls1908 branch ( + [HW](https://git.fd.io/csit/tree/docs/lab?h=rls1908), + [Linux](https://docs.fd.io/csit/rls1908/report/vpp_performance_tests/test_environment.html#sut-settings-linux), + [TRex](https://docs.fd.io/csit/rls1908/report/vpp_performance_tests/test_environment.html#tg-settings-trex), + [CSIT](https://git.fd.io/csit/tree/?h=rls1908) + ). diff --git a/docs/content/infrastructure/fdio_dc_testbed_specifications.md b/docs/content/infrastructure/fdio_dc_testbed_specifications.md new file mode 100644 index 0000000000..da71124078 --- /dev/null +++ b/docs/content/infrastructure/fdio_dc_testbed_specifications.md @@ -0,0 +1,2053 @@ +--- +bookToc: true +title: "FD.io DC Testbed Specifications" +weight: 2 +--- + +# FD.io DC Testbed Specifications + +## Purpose + +This note includes specification of the physical testbed infrastructure +hosted by LFN FD.io CSIT project. + +## Server Management + +### Addressing + +Each server has a LOM (Lights-Out-Management e.g. SM IPMI) and a +Management port, which are connected to two different VLANs. + +#### LOM (IPMI) VLAN + - Subnet: 10.30.50.0/24 + - Gateway: 10.30.50.1 + - Broadcast: 10.30.50.255 + - DNS1: 199.204.44.24 + - DNS2: 199.204.47.54 + +#### Management VLAN + - Subnet: 10.30.51.0/24 + - Gateway: 10.30.51.1 + - Broadcast: 10.30.51.255 + - DNS1: 199.204.44.24 + - DNS2: 199.204.47.54 + +To access these hosts, VPN connection is required. + +## Testbeds Overview + +### Summary List + +``` + #. Type Purpose SUT TG #TB #SUT #TG #skx #ps1 #rng #tx2 #tsh #alt #clx #zn2 #icx #snr #spr #icxd + 1. 1-Node-Skylake nomad skx na 5 5 0 5 0 0 0 0 0 0 0 0 0 0 0 + 2. 1-Node-Cascadelake nomad clx na 4 4 0 0 0 0 0 0 0 4 0 0 0 0 0 + 3. 1-Node-AmpereAltra nomad alt na 4 4 0 0 0 0 0 0 4 0 0 0 0 0 0 + 4. 2-Node-IxiaPS1L47 tcp skx ps1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 + 5. 2-Node-Cascadelake perf clx clx 3 3 3 0 0 0 0 0 0 6 0 0 0 0 0 + 6. 2-Node-ThunderX2 perf tx2 skx 1 1 .5 .5 0 0 1 0 0 0 0 0 0 0 0 + 7. 2-Node-Icelake perf icx icx 4 4 4 0 0 0 0 0 0 0 0 8 0 0 0 + 8. 3-Node-Rangeley perf rng skx 1 3 1 0 0 2 0 0 0 0 0 0 0 0 0 + 9. 3-Node-Taishan perf tsh skx 1 2 .5 .5 0 0 0 2 0 0 0 0 0 0 0 +10. 3-Node-Altra perf alt icx 1 2 1 0 0 0 0 0 2 0 0 1 0 0 0 +11. 2-Node-Zen2 perf zn2 zn2 1 1 1 0 0 0 0 0 0 0 2 0 0 0 0 +12. 3-Node-Icelake perf icx icx 2 4 2 0 0 0 0 0 0 0 0 6 0 0 0 +13. 3-Node-SnowRidge perf snr icx 1 2 .5 0 0 0 0 0 0 0 0 .5 2 0 0 +14. 2-Node-SapphireRapids perf spr spr 4 4 4 0 0 0 0 0 0 0 0 0 0 8 0 +15. 1-Node-SapphireRapids nomad spr na 4 4 0 0 0 0 0 0 0 0 0 0 0 4 0 +16. 3-Node-IcelakeD perf icxd icx 4 6 1 0 0 0 0 0 0 0 0 1 0 0 4 + Totals: 39 48 19.5 7 1 2 1 2 6 10 2 16.5 2 12 4 +``` + +### 1-Node-Skylake Xeon Intel (1n-skx) + +Each 1-Node-Skylake testbed includes one SUT (Server-Type-B2) with NIC +ports connected back-to-back ([Server Types](#server-types)). +Used for FD.io VPP_Device functional driver tests. + +### 1-Node-Altra Arm Ampere (1n-alt) + +Each 1-Node-Altra testbed includes one SUT (Server-Type-E25) with NIC +ports connected back-to-back ([Server Types](#server-types)). +Used for FD.io VPP_Device functional driver tests. + +### 1-Node-Skylake Xeon Intel (1n-spr) + +Each 1-Node-SapphireRapids testbed includes one SUT (Server-Type-H7) with NIC +ports connected back-to-back ([Server Types](#server-types)). +Used for FD.io VPP_Device functional driver tests. + +### 2-Node-IxiaPS1L47 Ixia PSOne L47 (2n-ps1) + +Each 2-Node-IxiaPS1L47 testbed includes one SUT (Server-Type-B1) and one +TG (Ixia PSOne appliance) with 10GE interfaces connected in a 2-node +circular topology ([Server Types](#server-types)). +Used for FD.io TCP/IP and HTTP performance tests. + +### 2-Node-Cascadelake Xeon Intel (2n-clx) + +Each 2-Node-Cascadelake testbed includes one SUT (Server-Type-C2) and +one TG (Server-Type-C3) connected in a 2-node circular topology +([Server Types](#server-types)). +Used for FD.io performance tests. + +### 2-Node-Zen2 EPYC AMD (2n-zn2) + +Each 2-Node-Zen2 testbed includes one SUT (Server-Type-D1) and +one TG (Server-Type-D2) connected in a 2-node circular topology +([Server Types](#server-types)). +Used for FD.io performance tests. + +### 2-Node-ThunderX2 Arm Marvell (2x-tx2) + +Each 2-Node-ThunderX2 testbed includes one SUT (Server-Type-E22) and +one TG (Server-Type-E31) connected in a 2-node circular topology +([Server Types](#server-types)). +Used for FD.io performance tests. + +### 2-Node-Icelake Xeon Intel (2n-icx) + +Each 2-Node-Icelake testbed includes one SUT (Server-Type-F1) and +one TG (Server-Type-F2) connected in a 2-node circular topology +([Server Types](#server-types)). +Used for FD.io performance tests. + +### 3-Node-Rangeley Atom Testbeds + +Each 3-Node-Rangeley testbed includes two SUTs (Server-Type-B5) and one +TG (Server-Type-2) connected in a 3-node circular topology +([Server Types](#server-types)). +Used for FD.io performance tests. + +### 3-Node-TaiShan Arm Huawei (3n-tsh) + +Each 3-Node-TaiShan testbed includes two SUTs (Server-Type-E21) and one +TG (Server-Type-E31) connected in a 3-node circular topology +([Server Types](#server-types)). +Used for FD.io performance tests. + +### 3-Node-Altra Arm Ampere (3n-alt) + +Each 3-Node-Altra testbed includes two SUTs (Server-Type-E23) and one +TG (Server-Type-E32) connected in a 3-node circular topology +([Server Types](#server-types)). +Used for FD.io performance tests. + +### 3-Node-Icelake Xeon Intel (3n-icx) + +Each 3-Node-Icelake testbed includes two SUTs (Server-Type-F3) and one +TG (Server-Type-F3) connected in a 3-node circular topology +([Server Types](#server-types)). +Used for FD.io performance tests. + +### 3-Node-IcelakeD Xeon Intel (3n-icxd) + +Each 3-Node-IcelakeD testbed includes two SUTs (Server-Type-I1) and one numa of +TG (Server-Type-F5) connected in a 3-node circular topology +([Server Types](#server-types)). +Used for FD.io performance tests. + +### 3-Node-SnowRidge Atom Intel (3n-snr) + +Each 3-Node-SnowRidge testbed includes two SUTs (Server-Type-G1) and one +TG (Server-Type-F5) connected in a 3-node circular topology +([Server Types](#server-types)). +Used for FD.io performance tests. + +### 2-Node-SapphireRapids Xeon Intel (2n-spr) + +Each 2-Node-SapphireRapids testbed includes one SUT (Server-Type-H5) and +one TG (Server-Type-H6) connected in a 2-node or 3-node circular topology +([Server Types](#server-types)). +Used for FD.io performance tests. + + +## Testbed Naming Convention + +Following naming convention is used within this page to specify physical +connectivity and wiring across defined CSIT testbeds: + +- **testbedname**: testbedN. +- **hostname**: + - traffic-generator: tN-tgW. + - system-under-testX: tN-sutX. +- **portnames**: + - tN-tgW-cY/pZ. + - tN-sutX-cY/pZ. +- **where**: + - N - testbed number. + - tgW - server acts as traffic-generator with W index. + - sutX - server acts as system-under-test with X index. + - Y - PCIe slot number denoting a NIC card number within the host. + - Z - port number on the NIC card. + +## Server Types + +FD.io CSIT lab contains following server types: + +1. **Server-Type-B1**: Purpose - Skylake Xeon hosts for FD.io builds and data processing. + - Quantity: 2 + - Physical connectivity: + - IPMI and host management ports. + - Main HW configuration: + - Chassis: SuperMicro SYS-7049GP-TRT. + - Motherboard: SuperMicro X11DPG-QT. + - Processors: 2* Intel Platinum 8180 2.5 GHz. + - RAM Memory: 16* 16GB DDR4-2666MHz. + - Disks: 2* 1.6TB 6G SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe3.0 lanes) + - PCIe Slot2 18:00.xx: empty. + - PCIe Slot4 3b:00.xx: empty. + - PCIe Slot9 5e:00.xx: empty. + - Numa1: (x16, x16, x16 PCIe3.0 lanes) + - PCIe Slot6 86:00.xx: empty. + - PCIe Slot8 af:00.xx: empty. + - PCIe Slot10 d8:00.xx: empty. + +2. **Server-Type-B2**: Purpose - Skylake Xeon SUT for FD.io VPP_Device functional tests. + - Quantity: 2. + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 1-node topologies. + - Main HW configuration: + - Chassis: SuperMicro SYS-7049GP-TRT. + - Motherboard: SuperMicro X11DPG-QT. + - Processors: 2* Intel Platinum 8180 2.5 GHz. + - RAM Memory: 16* 16GB DDR4-2666MHz. + - Disks: 2* 1.6TB 6G SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe3.0 lanes) + - PCIe Slot2 18:00.xx: x710-4p10GE Intel. + - PCIe Slot4 3b:00.xx: x710-4p10GE Intel. + - PCIe Slot9 5e:00.xx: empty. + - Numa1: (x16, x16, x16 PCIe3.0 lanes) + - PCIe Slot6 86:00.xx: e810-2CQDA2-2p100GE Intel. + - PCIe Slot8 af:00.xx: e810-2CQDA2-2p100GE Intel. + - PCIe Slot10 d8:00.xx: empty. + +3. **Server-Type-B7**: Purpose - Ixia PerfectStorm One Appliance TG for FD.io TCP/IP performance tests. + - Quantity: 1. + - Physical connectivity: + - Host management interface: 10/100/1000-BaseT. + - 8-port 10GE SFP+ integrated NIC. + - Main HW configuration: + - Chassis: PS10GE4NG. + - Motherboard: SuperMicro X11DPG-QT. + - Processors: Quad-Core, Intel Processor. + - HW accelerators: FPGA offload. + - RAM Memory: 64GB. + - Disks: 1 * 1 TB, Enterprise Class, High MTBF. + - Physical Interfaces: 4 * 10GE SFP+. + - Operating System: Native IxOS. + - Interface configuration: + - Port-1: 10GE SFP+. + - Port-2: 10GE SFP+. + - Port-3: 10GE SFP+. + - Port-4: 10GE SFP+. + +4. **Server-Type-B8**: Purpose - Skylake Xeon SUT for TCP/IP host stack tests. + - Quantity: 1. + - Physical connectivity: + - IPMI and host management ports. + - NIC ports. + - Main HW configuration: + - Chassis: SuperMicro SYS-7049GP-TRT. + - Motherboard: SuperMicro X11DPG-QT. + - Processors: 2* Intel Platinum 8180 2.5 GHz. + - RAM Memory: 16* 16GB DDR4-2666MHz. + - Disks: 2* 1.6TB 6G SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe3.0 lanes) + - PCIe Slot2 18:00.xx: x710-4p10GE Intel. + - PCIe Slot4 3b:00.xx: empty. + - PCIe Slot9 5e:00.xx: empty. + - Numa1: (x16, x16, x16 PCIe3.0 lanes) + - PCIe Slot6 86:00.xx: empty. + - PCIe Slot8 af:00.xx: empty. + - PCIe Slot10 d8:00.xx: empty. + +5. **Server-Type-C2**: Purpose - Cascadelake Xeon SUT for FD.io performance testing. + - Quantity: 3 + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 2-node testbed topologies. + - Main HW configuration: + - Chassis: SuperMicro SYS-7049GP-TRT. + - Motherboard: SuperMicro X11DPG-QT. + - Processors: 2* Intel Gold 6252N 2.3 GHz. + - RAM Memory: 12* 16GB DDR4-2933. + - Disks: 2* 1.92TB SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe3.0 lanes) + - PCIe Slot2 18:00.xx: x710-4p10GE Intel. + - PCIe Slot4 3b:00.xx: xxv710-DA2-2p25GE Intel. + - PCIe Slot9 5e:00.xx: ConnectX5-2p100GE Mellanox. + - Numa1: (x16, x16, x16 PCIe3.0 lanes) + - PCIe Slot6 86:00.xx: e810-2p100GE Intel. + - PCIe Slot8 af:00.xx: empty. + - PCIe Slot10 d8:00.xx: empty. + +6. **Server-Type-C3**: Purpose - Cascadelake Xeon TG for FD.io performance testing. + - Quantity: 3. + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 2-node testbed topologies. + - Main HW configuration: + - Chassis: SuperMicro SYS-7049GP-TRT. + - Motherboard: SuperMicro X11DPG-QT. + - Processors: 2* Intel Platinum 8280 2.7 GHz. + - RAM Memory: 12* 16GB DDR4-2933. + - Disks: 2* 1.92TB SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe3.0 lanes) + - PCIe Slot2 18:00.xx: x710-4p10GE Intel. + - PCIe Slot4 3b:00.xx: xxv710-DA2 2p25GE Intel. + - PCIe Slot9 5e:00.xx: ConnectX5-2p100GE Mellanox. + - Numa1: (x16, x16, x16 PCIe3.0 lanes) + - PCIe Slot6 86:00.xx: ConnectX5-2p100GE Mellanox. + - PCIe Slot8 af:00.xx: ConnectX5-2p100GE Mellanox. + - PCIe Slot10 d8:00.xx: empty. + +7. **Server-Type-C4**: Purpose - Cascadelake Xeon Backend hosts for FD.io builds and data processing. + - Quantity: 4. + - Physical connectivity: + - IPMI and host management ports. + - no NIC ports, standalone setup. + - Main HW configuration: + - Chassis: SuperMicro 1029P-WTRT. + - Motherboard: SuperMicro X11DDW-NT. + - Processors: 2* Intel Platinum 8280 2.7 GHz. + - RAM Memory: 12* 16GB DDR4-2933. + - Disks: 4* 1.92TB SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe3.0 lanes) + - PCIe Slot2 18:00.xx: empty. + - PCIe Slot4 3b:00.xx: empty. + - PCIe Slot9 5e:00.xx: empty. + - Numa1: (x16, x16, x16 PCIe3.0 lanes) + - PCIe Slot6 86:00.xx: empty. + - PCIe Slot8 af:00.xx: empty. + - PCIe Slot10 d8:00.xx: empty. + +8. **Server-Type-D1**: Purpose - Zen2 EPYC SUT for FD.io performance testing. + - Quantity: 1. + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 2-node testbed topologies. + - Main HW configuration: + - Chassis: SuperMicro AS-1114S-WTRT + - Processors: 1* AMD EPYC 7532 2.4 GHz. + - RAM Memory: 8* 32GB DDR4-2933. + - Disks: 1* 1TB SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe3.0 lanes) + - PCIe Slot1 01:00.xx: x710-4p10GE Intel. + - PCIe Slot2 41:00.xx: xxv710-da2-2p25GE Intel. + - PCIe Slot3 81:00.xx: mcx556a-edat ConnectX5-2p100GE Mellanox. + +9. **Server-Type-D2**: Purpose - Zen2 EPYC TG for FD.io performance testing. + - Quantity: 1. + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 2-node testbed topologies. + - Main HW configuration: + - Chassis: SuperMicro AS-1114S-WTRT + - Processors: 1* AMD EPYC 7532 2.4 GHz. + - RAM Memory: 8* 32GB DDR4-2933. + - Disks: 1* 1TB SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe3.0 lanes) + - PCIe Slot1 01:00.xx: mcx556a-edat ConnectX5-2p100GE Mellanox. + - PCIe Slot2 41:00.xx: x710-4p10GE Intel. + - PCIe Slot3 81:00.xx: xxv710-da2 2p25GE Intel. + +10. **Server-Type-E21**: Purpose - TaiShan Arm Huawei SUT for FD.io performance testing. + - Quantity: 2. + - Physical connectivity: + - IPMI(?) and host management ports. + - NIC ports connected into 3-node topology. + - Main HW configuration: + - Chassis: Huawei TaiShan 2280. + - Processors: 2* hip07-d05 ~ 32* Arm Cortex-A72 + - RAM Memory: 8* 16GB DDR4-2400MT/s + - Disks: 1* 4TB SATA HDD + - NICs configuration: + - PCIe Slot4 e9:00.xx: connectx4-2p25GE Mellanox. + - PCIe Slot6 11:00.xx: x520-2p10GE Intel. + +11. **Server-Type-E22**: Purpose - ThunderX2 Arm Marvell SUT for FD.io performance testing. + - Quantity: 1. + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 2-node topologies. + - Main HW configuration: + - Chassis: Gigabyte R181-T90 1U + - Motherboard: MT91-FS1 + - Processors: 2* ThunderX2 ARMv8 CN9975 2.0 GHz + - RAM Memory: 4* 32GB RDIMM + - Disks: 1* 480GB SSD Micron, 1* 1000GB HDD Seagate_25 + - NICs configuration: + - Numa0: + - no cards + - Numa1: + - PCIe Slot18 91:00.xx: XL710-QDA2-2p40GE Intel. + +12. **Server-Type-E23**: Purpose - Altra Arm Ampere SUT for FD.io performance testing. + - Quantity: 2. + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 3-node topologies. + - Main HW configuration: + - Chassis: WIWYNN Mt.Jade Server System B81.030Z1.0007 2U + - Motherboard: Mt.Jade Motherboard + - Processors: 2* Ampere(R) Altra(R) Q80-30 Processor (Neoverse N1) + - Processor Signature: Implementor 0x41, Variant 0x3, Architecture 15, Part 0xd0c, Revision 1 + - RAM Memory: 16* 8GB DDR4-3200MT/s + - Disks: 2* 960GB SSD Samsung M.2 NVMe PM983 + - NICs configuration: + - Numa0: (x16, x16 PCIe4.0 lanes) + - PCIe Slot1 0004:04:00.x: xl710-QDA2-2p40GE Intel. + - PCIe Slot8 0001:00:00.x: ConnectX6-2p100GE Mellanox. + - Numa1: + - no cards. + +13. **Server-Type-E24**: Purpose - Altra Arm Ampere for FD.io build. + - Quantity: 2. + - Physical connectivity: + - IPMI and host management ports. + - Main HW configuration: + - Chassis: Gigabyte R152-P30-00 1U + - Motherboard: MP32-AR1-00 + - Processors: 1* Ampere(R) Altra(R) Q80-30 Processor (Neoverse N1) + - Processor Signature: Implementor 0x0a, Variant 0x1, Architecture 6, Part 0x000, Revision 1 + - RAM Memory: 12* 16GB DDR4-3200MT/s + - Disks: 1* 960GB SSD Samsung M.2 NVMe PM983 + +14. **Server-Type-E25**: Purpose - Altra Arm Ampere SUT for FD.io VPP_Device functional tests. + - Quantity: 2. + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 1-node topologies. + - Main HW configuration: + - Chassis: Gigabyte E252-P30-00 2U + - Motherboard: MP32-AR1-00 + - Processors: 1* Ampere(R) Altra(R) M128-30 Processor (Neoverse N1) + - Processor Signature: Implementor 0x41 (Arm), Variant 0x3, Architecture 0xf, Part 0xd0c (neoverse-n1), Revision 0x1 + - RAM Memory: 32* 32GB DDR4-3200MT/s + - Disks: 2* 960GB SSD Samsung M.2 NVMe PM9A3 + - NICs configuration: + - Numa0: + - PCIe Slot0 0000:01:00.xx: XL710-QDA2-2p40GE Intel. + - PCIe Slot1 0001:01:00.xx: ConnectX6-2p100GE Mellanox. + - PCIe Slot2 0002:03:00.xx: ConnectX5-2p10/25GE Mellanox. + - Numa1: + - PCIe Slot3 0003:02:00.xx: XL710-QDA2-2p40GE Intel. + - PCIe Slot5 0005:02:00.xx: ConnectX5-2p10/25GE Mellanox. + +15. **Server-Type-E31**: Purpose - Skylake Xeon Shared TG for FD.io performance testing. + - Quantity: 1. + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 2-node and 3-node topologies. + - Main HW configuration: + - Chassis: SuperMicro SYS-7049GP-TRT. + - Motherboard: SuperMicro X11DPG-QT. + - Processors: 2* Intel Platinum 8180 2.5 GHz. + - RAM Memory: 16* 16GB DDR4-2666MHz. + - Disks: 2* 1.6TB 6G SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe3.0 lanes) + - PCIe Slot2 18:00.xx: x710-4p10GE Intel. + - PCIe Slot4 3b:00.xx: xxv710-DA2 2p25GE Intel. + - PCIe Slot9 5e:00.xx: empty. + - Numa1: (x16, x16, x16 PCIe3.0 lanes) + - PCIe Slot6 86:00.xx: empty. + - PCIe Slot8 af:00.xx: xl710-QDA2-2p40GE Intel. + - PCIe Slot10 d8:00.xx: x710-4p10GE Intel. + +16. **Server-Type-E32**: Purpose - Icelake Xeon Shared TG for FD.io performance testing. + - Quantity: 1. + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 2-node and/or 3-node testbed topologies. + - Main HW configuration: + - Chassis: SuperMicro SYS-740GP-TNRT. + - Motherboard: Super X12DPG-QT6. + - Processors: 2* Intel Platinum 8358 2.6 GHz. + - RAM Memory: 16* 16GB DDR4-3200. + - Disks: 2* 960GB SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe4.0 lanes) + - PCIe Slot2 4b:00.xx: xxv710-DA2-2p25GE Intel. + - PCIe Slot4 31:00.xx: xl710-QDA2-2p40GE Intel. + - PCIe Slot9 ff:00.xx: empty. + - Numa1: (x16, x16, x16 PCIe4.0 lanes) + - PCIe Slot6 ca:00.xx: empty. + - PCIe Slot8 b1:00.xx: e810-2CQDA2-2p100GE Intel. + - PCIe Slot10 ff:00.xx: empty. + +17. **Server-Type-F1**: Purpose - Icelake Xeon SUT for FD.io performance testing. + - Quantity: 4. + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 2-node topology. + - Main HW configuration: + - Chassis: SuperMicro SYS-740GP-TNRT. + - Motherboard: Super X12DPG-QT6. + - Processors: 2* Intel Platinum 8358 2.6 GHz. + - RAM Memory: 16* 16GB DDR4-3200. + - Disks: 2* 960GB SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe4.0 lanes) + - PCIe Slot2 18:00.xx: xxv710-DA2-2p25GE Intel. + - PCIe Slot4 3b:00.xx: e810-XXVDA4-4p25GE Intel. + - PCIe Slot9 5e:00.xx: e810-2CQDA2-2p100GE Intel. + - Numa1: (x16, x16, x16 PCIe4.0 lanes) + - PCIe Slot6 86:00.xx: empty. + - PCIe Slot8 af:00.xx: ConnectX7-2p200GE Mellanox. + - PCIe Slot10 d8:00.xx: empty. + +18. **Server-Type-F2**: Purpose - Icelake Xeon TG for FD.io performance testing. + - Quantity: 4. + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 2-node testbed topologies. + - Main HW configuration: + - Chassis: SuperMicro SYS-740GP-TNRT. + - Motherboard: Super X12DPG-QT6. + - Processors: 2* Intel Platinum 8358 2.6 GHz. + - RAM Memory: 16* 16GB DDR4-3200. + - Disks: 2* 960GB SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe4.0 lanes) + - PCIe Slot2 18:00.xx: xxv710-DA2-2p25GE Intel. + - PCIe Slot4 3b:00.xx: e810-XXVDA4-4p25GE Intel. + - PCIe Slot9 5e:00.xx: e810-2CQDA2-2p100GE Intel. + - Numa1: (x16, x16, x16 PCIe4.0 lanes) + - PCIe Slot6 86:00.xx: e810-2CQDA2-2p100GE Intel. + - PCIe Slot8 af:00.xx: ConnectX7-2p200GE Mellanox. + - PCIe Slot10 d8:00.xx: empty. + +19. **Server-Type-F3**: Purpose - Icelake Xeon TG or SUT for FD.io performance testing. + - Quantity: 6. + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 3-node testbed topologies. + - Main HW configuration: + - Chassis: SuperMicro SYS-740GP-TNRT. + - Motherboard: Super X12DPG-QT6. + - Processors: 2* Intel Platinum 8358 2.6 GHz. + - RAM Memory: 16* 16GB DDR4-3200. + - Disks: 2* 960GB SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe4.0 lanes) + - PCIe Slot2 18:00.xx: xxv710-DA2-2p25GE Intel. + - PCIe Slot4 3b:00.xx: e810-XXVDA4-4p25GE Intel. + - PCIe Slot9 5e:00.xx: e810-2CQDA2-2p100GE Intel. + - Numa1: (x16, x16, x16 PCIe4.0 lanes) + - PCIe Slot6 86:00.xx: ConnectX6-2p100GE Mellanox. + - PCIe Slot8 af:00.xx: empty. + - PCIe Slot10 d8:00.xx: empty. + +20. **Server-Type-F4**: Purpose - Icelake Xeon TG for FD.io performance testing. + - Quantity: 3. + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 3-node testbed topologies. + - Main HW configuration: + - Chassis: SuperMicro SYS-740GP-TNRT. + - Motherboard: Super X12DPG-QT6. + - Processors: 2* Intel Platinum 8358 2.6 GHz. + - RAM Memory: 16* 16GB DDR4-3200. + - Disks: 2* 960GB SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe4.0 lanes) + - PCIe Slot2 18:00.xx: xxv710-DA2-2p25GE Intel. + - PCIe Slot4 3b:00.xx: e810-XXVDA4-4p25GE Intel. + - PCIe Slot9 5e:00.xx: e810-2CQDA2-2p100GE Intel. + - Numa1: (x16, x16, x16 PCIe4.0 lanes) + - PCIe Slot6 86:00.xx: ConnectX6-2p100GE Mellanox. + - PCIe Slot8 af:00.xx: empty. + - PCIe Slot10 d8:00.xx: empty. + +21. **Server-Type-F5**: Purpose - Icelake Xeon Shared TG for FD.io performance testing. + - Quantity: 2. + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 2-node and/or 3-node testbed topologies. + - Main HW configuration: + - Chassis: SuperMicro SYS-740GP-TNRT. + - Motherboard: Super X12DPG-QT6. + - Processors: 2* Intel Platinum 8358 2.6 GHz. + - RAM Memory: 16* 16GB DDR4-3200. + - Disks: 2* 960GB SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe4.0 lanes) + - PCIe Slot2 4b:00.xx: xxv710-DA2-2p25GE Intel. + - PCIe Slot4 31:00.xx: empty. + - PCIe Slot9 ff:00.xx: empty. + - Numa1: (x16, x16, x16 PCIe4.0 lanes) + - PCIe Slot6 ca:00.xx: e810-XXVDA4-4p25GE Intel. + - PCIe Slot8 b1:00.xx: e810-2CQDA2-2p100GE Intel. + - PCIe Slot10 ff:00.xx: empty. + +22. **Server-Type-G1**: Purpose - SnowRidge Atom SUT for FD.io performance testing. + - Quantity: 2 + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 3-node testbed topology. + - Main HW configuration: + - Chassis: Intel JACOBSVILLE SDP. + - Motherboard: Intel JACOBSVILLE E63448-400. + - Processors: 1* Intel Atom P5362B 2.2 GHz. + - RAM Memory: 2* 16GB DDR4-2933. + - Disks: ?* ? SATA SSD. + - NICs configuration: + - Numa0: (x16, PCIe3.0 lane) + - PCIe BuiltIn ec:00.xx: e810-XXVDA4-4p25GE Intel. + +23. **Server-Type-H1**: Purpose - SapphireRapids Xeon SUT for FD.io full system performance testing. + - Quantity: 1. + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 3-numa-node testbed topologies. + - Main HW configuration: + - Chassis: SuperMicro SYS-741GE-TNRT. + - Motherboard: Super X13DEG-QT-P. + - Processors: 2* Intel Platinum 8462Y+ 32 core 2.8 GHz 300W TDP. + - RAM Memory: 16* 32GB DDR5-4800. + - Disks: 2* 960GB SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe5.0 lanes) + - PCIe Slot2 18:00.xx: ConnectX7-2p200GE Nvidia. + - PCIe Slot4 3b:00.xx: ConnectX7-2p200GE Nvidia. + - PCIe Slot10 5e:00.xx: ConnectX7-2p200GE Nvidia. + - Numa1: (x16, x16, x16 PCIe5.0 lanes) + - PCIe Slot7 86:00.xx: ConnectX7-2p200GE Nvidia. + - PCIe Slot9 af:00.xx: ConnectX7-2p200GE Nvidia. + - PCIe Slot11 d8:00.xx: ConnectX7-2p200GE Nvidia. + +24. **Server-Type-H2**: Purpose - SapphireRapids Xeon TG for FD.io full system performance testing. + - Quantity: 1. + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 3-numa-node testbed topologies. + - Main HW configuration: + - Chassis: SuperMicro SYS-741GE-TNRT. + - Motherboard: Super X13DEG-QT-P. + - Processors: 2* Intel Platinum 8462Y+ 32 core 2.8 GHz 300W TDP. + - RAM Memory: 16* 32GB DDR5-4800. + - Disks: 2* 960GB SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe5.0 lanes) + - PCIe Slot2 18:00.xx: ConnectX7-2p200GE Nvidia. + - PCIe Slot4 3b:00.xx: ConnectX7-2p200GE Nvidia. + - PCIe Slot10 5e:00.xx: ConnectX7-2p200GE Nvidia. + - Numa1: (x16, x16, x16 PCIe5.0 lanes) + - PCIe Slot7 86:00.xx: ConnectX7-2p200GE Nvidia. + - PCIe Slot9 af:00.xx: ConnectX7-2p200GE Nvidia. + - PCIe Slot11 d8:00.xx: empty. + +25. **Server-Type-H3**: Purpose - SapphireRapids Xeon SUT for FD.io performance testing. + - Quantity: 1. + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 3-numa-node testbed topologies. + - Main HW configuration: + - Chassis: SuperMicro SYS-741GE-TNRT. + - Motherboard: Super X13DEG-QT-P. + - Processors: 2* Intel Platinum 8462Y+ 32 core 2.8 GHz 300W TDP. + - RAM Memory: 16* 32GB DDR5-4800. + - Disks: 2* 960GB SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe5.0 lanes) + - PCIe Slot2 18:00.xx: e810-2CQDA2-2p100GE Intel. + - PCIe Slot4 3b:00.xx: e810-XXVDA4-4p25GE Intel. + - PCIe Slot10 5e:00.xx: empty. + - Numa1: (x16, x16, x16 PCIe5.0 lanes) + - PCIe Slot7 86:00.xx: e810-2CQDA2-2p100GE Intel. + - PCIe Slot9 af:00.xx: e810-XXVDA4-4p25GE Intel. + - PCIe Slot11 d8:00.xx: empty. + +26. **Server-Type-H4**: Purpose - SapphireRapids Xeon TG for FD.io performance testing. + - Quantity: 1. + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 3-numa-node testbed topologies. + - Main HW configuration: + - Chassis: SuperMicro SYS-741GE-TNRT. + - Motherboard: Super X13DEG-QT-P. + - Processors: 2* Intel Platinum 8462Y+ 32 core 2.8 GHz 300W TDP. + - RAM Memory: 16* 32GB DDR5-4800. + - Disks: 2* 960GB SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe5.0 lanes) + - PCIe Slot2 18:00.xx: e810-2CQDA2-2p100GE Intel. + - PCIe Slot4 3b:00.xx: e810-XXVDA4-4p25GE Intel. + - PCIe Slot10 5e:00.xx: empty. + - Numa1: (x16, x16, x16 PCIe5.0 lanes) + - PCIe Slot7 86:00.xx: empty. + - PCIe Slot9 af:00.xx: empty. + - PCIe Slot11 d8:00.xx: empty. + +27. **Server-Type-H5**: Purpose - SapphireRapids Xeon SUT for FD.io performance testing. + - Quantity: 2. + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 2-node testbed topologies. + - Main HW configuration: + - Chassis: SuperMicro SYS-741GE-TNRT. + - Motherboard: Super X13DEG-QT-P. + - Processors: 2* Intel Platinum 8462Y+ 32 core 2.8 GHz 300W TDP. + - RAM Memory: 16* 32GB DDR5-4800. + - Disks: 2* 960GB SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe5.0 lanes) + - PCIe Slot2 3d:00.xx: MCX713106AS-VEAT ConnectX7-2p200GE Nvidia. + - PCIe Slot4 2c:00.xx: e810-2CQDA2-2p100GE Intel. + - PCIe Slot10 17:00.xx: e810-XXVDA4-4p25GE Intel. + - Numa1: (x16, x16, x16 PCIe5.0 lanes) + - PCIe Slot7 86:00.xx: empty. + - PCIe Slot9 af:00.xx: empty. + - PCIe Slot11 d8:00.xx: empty. + +28. **Server-Type-H6**: Purpose - SapphireRapids Xeon TG for FD.io performance testing. + - Quantity: 2. + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 2-node testbed topologies plus loopbacks in Numa1 for TG self-test. + - Main HW configuration: + - Chassis: SuperMicro SYS-741GE-TNRT. + - Motherboard: Super X13DEG-QT-P. + - Processors: 2* Intel Platinum 8462Y+ 32 core 2.8 GHz 300W TDP. + - RAM Memory: 16* 32GB DDR5-4800. + - Disks: 2* 960GB SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe5.0 lanes) + - PCIe Slot2 18:00.xx: MCX713106AS-VEAT ConnectX7-2p200GE Nvidia. + - PCIe Slot4 3b:00.xx: e810-2CQDA2-2p100GE Intel. + - PCIe Slot10 5e:00.xx: e810-XXVDA4-4p25GE Intel. + - Numa1: (x16, x16, x16 PCIe5.0 lanes) + - PCIe Slot7 86:00.xx: MCX713106AS-VEAT ConnectX7-2p200GE Nvidia. + - PCIe Slot9 af:00.xx: e810-2CQDA2-2p100GE Intel. + - PCIe Slot11 d8:00.xx: empty. + +29. **Server-Type-H7**: Purpose - SapphireRapids SUT for FD.io VPP_Device functional tests. + - Quantity: 2. + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 1-node topologies. + - Main HW configuration: + - Chassis: SuperMicro SYS-741GE-TNRT. + - Motherboard: Super X13DEG-QT-P. + - Processors: 2* Intel Platinum 8462Y+ 32 core 2.8 GHz 300W TDP. + - RAM Memory: 16* 32GB DDR5-4800. + - Disks: 2* 960GB SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe5.0 lanes) + - PCIe Slot2 18:00.xx: e810-2CQDA2-2p100GE Intel. + - PCIe Slot4 3b:00.xx: e810-2CQDA2-2p100GE Intel. + - PCIe Slot10 5e:00.xx: empty. + - Numa1: (x16, x16, x16 PCIe5.0 lanes) + - PCIe Slot7 86:00.xx: empty. + - PCIe Slot9 af:00.xx: empty. + - PCIe Slot11 d8:00.xx: empty. + +30. **Server-Type-I1**: Purpose - IcelakeD Xeon SUT for FD.io performance testing. + - Quantity: 4 + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 3-node testbed topology. + - Main HW configuration: + - Chassis: SuperMicro SYS-110D-20C-FRDN8TP. + - Motherboard: Super X12SDV-20C-SPT8F. + - Processors: 1* Intel Xeon D-2796NT. + - RAM Memory: 4* 32GB DDR4-3200 MEM-DR432MD-ER32. + - Disks: 1* 960GB SATA SSD HDS-25T0. + - NICs configuration: + - Numa0: (x16, PCIe4.0 lane) + - PCIe BuiltIn ??:00.xx: e810-XXVDA2-2p25GE Intel. + +## Testbeds Configuration + +### 1-Node-Skylake (1n-skx) + +``` +- SUT [Server-Type-B2]: + - testbedname: testbed11. + - hostname: s1-t11-sut1. + - IPMI IP: 10.30.50.47 + - Host IP: 10.30.51.50 + - portnames: + - s1-t11-sut1-c2/p1 - 10GE-port1 x710-4p10GE. + - s1-t11-sut1-c2/p2 - 10GE-port2 x710-4p10GE. + - s1-t11-sut1-c2/p3 - 10GE-port3 x710-4p10GE. + - s1-t11-sut1-c2/p4 - 10GE-port4 x710-4p10GE. + - s1-t11-sut1-c4/p1 - 10GE-port1 x710-4p10GE. + - s1-t11-sut1-c4/p2 - 10GE-port2 x710-4p10GE. + - s1-t11-sut1-c4/p3 - 10GE-port3 x710-4p10GE. + - s1-t11-sut1-c4/p4 - 10GE-port4 x710-4p10GE. + - s1-t11-sut1-c6/p1 - 100GE-port1 e810-2p100GE. + - s1-t11-sut1-c6/p2 - 100GE-port2 e810-2p100GE. + - s1-t11-sut1-c8/p1 - 100GE-port1 e810-2p100GE. + - s1-t11-sut1-c8/p2 - 100GE-port2 e810-2p100GE. +- SUT [Server-Type-B2]: + - testbedname: testbed12. + - hostname: s2-t12-sut1. + - IPMI IP: 10.30.50.48 + - Host IP: 10.30.51.51 + - portnames: + - s2-t12-sut1-c2/p1 - 10GE-port1 x710-4p10GE. + - s2-t12-sut1-c2/p2 - 10GE-port2 x710-4p10GE. + - s2-t12-sut1-c2/p3 - 10GE-port3 x710-4p10GE. + - s2-t12-sut1-c2/p4 - 10GE-port4 x710-4p10GE. + - s2-t12-sut1-c4/p1 - 10GE-port1 x710-4p10GE. + - s2-t12-sut1-c4/p2 - 10GE-port2 x710-4p10GE. + - s2-t12-sut1-c4/p3 - 10GE-port3 x710-4p10GE. + - s2-t12-sut1-c4/p4 - 10GE-port4 x710-4p10GE. + - s2-t12-sut1-c6/p1 - 100GE-port1 e810-2p100GE. + - s2-t12-sut1-c6/p2 - 100GE-port2 e810-2p100GE. + - s2-t12-sut1-c8/p1 - 100GE-port1 e810-2p100GE. + - s2-t12-sut1-c8/p2 - 100GE-port2 e810-2p100GE. +``` + +### 1-Node-Altra (1n-alt) + +``` +- SUT [Server-Type-E25]: + - testbedname: testbed13 + - hostname: s70-t13-sut1 + - IPMI IP: 10.30.50.70 + - Host IP: 10.30.51.70 + - portnames: + - s70-t13-sut1-c0/p1 - 40GE-port1 XL710-QDA2-2p40GE. + - s70-t13-sut1-c0/p2 - 40GE-port2 XL710-QDA2-2p40GE. + - s70-t13-sut1-c1/p1 - 100GE-port1 ConnectX6-2p100GE Mellanox. + - s70-t13-sut1-c1/p2 - 100GE-port2 ConnectX6-2p100GE Mellanox. + - s70-t13-sut1-c2/p1 - 25GE-port1 ConnectX5-2p10/25GE Mellanox. + - s70-t13-sut1-c2/p2 - 25GE-port2 ConnectX5-2p10/25GE Mellanox. + - s70-t13-sut1-c3/p1 - 40GE-port1 XL710-QDA2-2p40GE. + - s70-t13-sut1-c3/p2 - 40GE-port2 XL710-QDA2-2p40GE. + - s70-t13-sut1-c5/p1 - 25GE-port1 ConnectX5-2p10/25GE Mellanox. + - s70-t13-sut1-c5/p2 - 25GE-port2 ConnectX5-2p10/25GE Mellanox. +- SUT [Server-Type-E25]: + - testbedname: testbed14 + - hostname: s71-t14-sut1 + - IPMI IP: 10.30.50.71 + - Host IP: 10.30.51.71 + - portnames: + - s71-t14-sut1-c0/p1 - 40GE-port1 XL710-QDA2-2p40GE. + - s71-t14-sut1-c0/p2 - 40GE-port2 XL710-QDA2-2p40GE. + - s71-t14-sut1-c1/p1 - 100GE-port1 ConnectX6-2p100GE Mellanox. + - s71-t14-sut1-c1/p2 - 100GE-port2 ConnectX6-2p100GE Mellanox. + - s71-t14-sut1-c2/p1 - 25GE-port1 ConnectX5-2p10/25GE Mellanox. + - s71-t14-sut1-c2/p2 - 25GE-port2 ConnectX5-2p10/25GE Mellanox. + - s71-t14-sut1-c3/p1 - 40GE-port1 XL710-QDA2-2p40GE. + - s71-t14-sut1-c3/p2 - 40GE-port2 XL710-QDA2-2p40GE. + - s71-t14-sut1-c5/p1 - 25GE-port1 ConnectX5-2p10/25GE Mellanox. + - s71-t14-sut1-c5/p2 - 25GE-port2 ConnectX5-2p10/25GE Mellanox. +``` + +### 1-Node-SapphireRapids (1n-spr) + +``` +- SUT [Server-Type-H7]: + - testbedname: testbed15. + - hostname: s30-t15-sut1. + - IPMI IP: 10.30.50.30 + - Host IP: 10.30.51.31 + - portnames: + - s30-t15-sut1-c1/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s30-t15-sut1-c1/p2 - 100GE-port2 e810-2CQDA2-2p100GE. + - s30-t15-sut1-c2/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s30-t15-sut1-c2/p2 - 100GE-port1 e810-2CQDA2-2p100GE. +- SUT [Server-Type-H7]: + - testbedname: testbed16. + - hostname: s31-t16-sut1. + - IPMI IP: 10.30.50.31 + - Host IP: 10.30.51.31 + - portnames: + - s31-t16-sut1-c1/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s31-t16-sut1-c1/p2 - 100GE-port2 e810-2CQDA2-2p100GE. + - s31-t16-sut1-c2/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s31-t16-sut1-c2/p2 - 100GE-port1 e810-2CQDA2-2p100GE. +``` + +### 2-Node-IxiaPS1L47 (2n-ps1) + +``` +- SUT [Server-Type-B8]: + - testbedname: testbed25. + - hostname: s25-t25-sut1. + - IPMI IP: 10.30.50.58 + - Host IP: 10.30.51.61 + - portnames: + - s25-t25-sut1-c2/p1 - 10GE-port1 x710-4p10GE. + - s25-t25-sut1-c2/p2 - 10GE-port2 x710-4p10GE. + - s25-t25-sut1-c2/p3 - 10GE-port3 x710-4p10GE. + - s25-t25-sut1-c2/p4 - 10GE-port4 x710-4p10GE. +- TG [Server-Type-B7]: + - testbedname: testbed25. + - hostname: s26-t25-tg1. + - IPMI IP: 10.30.50.59 + - Host IP: 10.30.51.62 + - portnames: + - s26-t25-tg1-p1 - 10GE-port1. + - s26-t25-tg1-p2 - 10GE-port2. + - s26-t25-tg1-p3 - 10GE-port3. + - s26-t25-tg1-p4 - 10GE-port4. +``` + +### 2-Node-Cascadelake (2n-clx) + +{{< figure src="/cdocs/testbed-2n-clx.svg" >}} + +``` +- SUT [Server-Type-C2]: + - testbedname: testbed27. + - hostname: s33-t27-sut1. + - IPMI IP: 10.30.55.18 + - Host IP: 10.32.8.18 + - portnames: + - s33-t27-sut1-c2/p1 - 10GE-port1 x710-4p10GE. + - s33-t27-sut1-c2/p2 - 10GE-port2 x710-4p10GE. + - s33-t27-sut1-c2/p3 - 10GE-port3 x710-4p10GE. + - s33-t27-sut1-c2/p4 - 10GE-port4 x710-4p10GE. + - s33-t27-sut1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s33-t27-sut1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s33-t27-sut1-c6/p1 - 100GE-port1 e810-2p100GE. + - s33-t27-sut1-c6/p2 - 100GE-port2 e810-2p100GE. + - s33-t27-sut1-c9/p1 - 100GE-port1 ConnectX5-2p100GE. + - s33-t27-sut1-c9/p2 - 100GE-port2 ConnectX5-2p100GE. +- TG [Server-Type-C3]: + - testbedname: testbed27. + - hostname: s34-t27-tg1. + - IPMI IP: 10.30.55.19 + - Host IP: 10.32.8.19 + - portnames: + - s34-t27-tg1-c2/p1 - 10GE-port1 x710-4p10GE. + - s34-t27-tg1-c2/p2 - 10GE-port2 x710-4p10GE. + - s34-t27-tg1-c2/p3 - 10GE-port3 x710-4p10GE. + - s34-t27-tg1-c2/p4 - 10GE-port4 x710-4p10GE. + - s34-t27-tg1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s34-t27-tg1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s34-t27-tg1-c6/p1 - 100GE-port1 ConnectX5-2p100GE. + - s34-t27-tg1-c6/p2 - 100GE-port2 ConnectX5-2p100GE. + - s38-t27-tg1-c8/p1 - 100GE-port1 ConnectX5-2p100GE. + - s38-t27-tg1-c8/p2 - 100GE-port2 ConnectX5-2p100GE. + - s34-t27-tg1-c9/p1 - 100GE-port1 ConnectX5-2p100GE. + - s34-t27-tg1-c9/p2 - 100GE-port2 ConnectX5-2p100GE. +- SUT [Server-Type-C2]: + - testbedname: testbed28. + - hostname: s35-t28-sut1. + - IPMI IP: 10.30.55.20 + - Host IP: 10.32.8.20 + - portnames: + - s35-t28-sut1-c2/p1 - 10GE-port1 x710-4p10GE. + - s35-t28-sut1-c2/p2 - 10GE-port2 x710-4p10GE. + - s35-t28-sut1-c2/p3 - 10GE-port3 x710-4p10GE. + - s35-t28-sut1-c2/p4 - 10GE-port4 x710-4p10GE. + - s35-t28-sut1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s35-t28-sut1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s35-t28-sut1-c6/p1 - 100GE-port1 e810-2p100GE. + - s35-t28-sut1-c6/p2 - 100GE-port2 e810-2p100GE. + - s35-t28-sut1-c9/p1 - 100GE-port1 ConnectX5-2p100GE. + - s35-t28-sut1-c9/p2 - 100GE-port2 ConnectX5-2p100GE. +- TG [Server-Type-C3]: + - testbedname: testbed28. + - hostname: s36-t28-tg1. + - IPMI IP: 10.30.55.21 + - Host IP: 10.32.8.21 + - portnames: + - s36-t28-tg1-c2/p1 - 10GE-port1 x710-4p10GE. + - s36-t28-tg1-c2/p2 - 10GE-port2 x710-4p10GE. + - s36-t28-tg1-c2/p3 - 10GE-port3 x710-4p10GE. + - s36-t28-tg1-c2/p4 - 10GE-port4 x710-4p10GE. + - s36-t28-tg1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s36-t28-tg1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s36-t28-tg1-c6/p1 - 100GE-port1 ConnectX5-2p100GE. + - s36-t28-tg1-c6/p2 - 100GE-port2 ConnectX5-2p100GE. + - s38-t28-tg1-c8/p1 - 100GE-port1 ConnectX5-2p100GE. + - s38-t28-tg1-c8/p2 - 100GE-port2 ConnectX5-2p100GE. + - s36-t28-tg1-c9/p1 - 100GE-port1 ConnectX5-2p100GE. + - s36-t28-tg1-c9/p2 - 100GE-port2 ConnectX5-2p100GE. +- SUT [Server-Type-C2]: + - testbedname: testbed29. + - hostname: s37-t29-sut1. + - IPMI IP: 10.30.55.22 + - Host IP: 10.32.8.22 + - portnames: + - s37-t29-sut1-c2/p1 - 10GE-port1 x710-4p10GE. + - s37-t29-sut1-c2/p2 - 10GE-port2 x710-4p10GE. + - s37-t29-sut1-c2/p3 - 10GE-port3 x710-4p10GE. + - s37-t29-sut1-c2/p4 - 10GE-port4 x710-4p10GE. + - s37-t29-sut1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s37-t29-sut1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s37-t29-sut1-c6/p1 - 100GE-port1 e810-2p100GE. + - s37-t29-sut1-c6/p2 - 100GE-port2 e810-2p100GE. + - s37-t29-sut1-c9/p1 - 100GE-port1 ConnectX5-2p100GE. + - s37-t29-sut1-c9/p2 - 100GE-port2 ConnectX5-2p100GE. +- TG [Server-Type-C3]: + - testbedname: testbed29. + - hostname: s38-t29-tg1. + - IPMI IP: 10.30.55.23 + - Host IP: 10.32.8.23 + - portnames: + - s38-t29-tg1-c2/p1 - 10GE-port1 x710-4p10GE. + - s38-t29-tg1-c2/p2 - 10GE-port2 x710-4p10GE. + - s38-t29-tg1-c2/p3 - 10GE-port3 x710-4p10GE. + - s38-t29-tg1-c2/p4 - 10GE-port4 x710-4p10GE. + - s38-t29-tg1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s38-t29-tg1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s38-t29-tg1-c6/p1 - 100GE-port1 ConnectX5-2p100GE. + - s38-t29-tg1-c6/p2 - 100GE-port2 ConnectX5-2p100GE. + - s38-t29-tg1-c9/p1 - 100GE-port1 ConnectX5-2p100GE. + - s38-t29-tg1-c9/p2 - 100GE-port2 ConnectX5-2p100GE. +``` + +### 2-Node-Zen2 (2n-zn2) + +{{< figure src="/cdocs/testbed-2n-zn2.svg" >}} + +``` +- SUT [Server-Type-D1]: + - testbedname: testbed210. + - hostname: s60-t210-sut1. + - IPMI IP: 10.30.55.24 + - Host IP: 10.32.8.24 + - portnames: + - s60-t210-sut1-c1/p1 - 10GE-port1 x710-4p10GE. + - s60-t210-sut1-c1/p2 - 10GE-port2 x710-4p10GE. + - s60-t210-sut1-c1/p3 - 10GE-port3 x710-4p10GE. + - s60-t210-sut1-c1/p4 - 10GE-port4 x710-4p10GE. + - s60-t210-sut1-c2/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s60-t210-sut1-c2/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s60-t210-sut1-c3/p1 - 100GE-port1 ConnectX5-2p100GE. + - s60-t210-sut1-c3/p2 - 100GE-port2 ConnectX5-2p100GE. +- TG [Server-Type-D2]: + - testbedname: testbed210. + - hostname: s61-t210-tg1. + - IPMI IP: 10.30.55.25 + - Host IP: 10.32.8.25 + - portnames: + - s61-t210-tg1-c1/p1 - 100GE-port1 ConnectX5-2p100GE. + - s61-t210-tg1-c1/p2 - 100GE-port2 ConnectX5-2p100GE. + - s61-t210-tg1-c2/p1 - 10GE-port1 x710-4p10GE. + - s61-t210-tg1-c2/p2 - 10GE-port2 x710-4p10GE. + - s61-t210-tg1-c2/p3 - 10GE-port3 x710-4p10GE. + - s61-t210-tg1-c2/p4 - 10GE-port4 x710-4p10GE. + - s61-t210-tg1-c3/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s61-t210-tg1-c3/p2 - 25GE-port2 xxv710-DA2-2p25GE. +``` + +### 2-Node-ThunderX2 (2x-tx2) + +{{< figure src="/cdocs/testbed-2n-tx2.svg" >}} + +``` +- SUT [Server-Type-E22]: + - testbedname: testbed211. + - hostname: s27-t211-sut1. + - IPMI IP: 10.30.50.69 + - Host IP: 10.30.51.69 + - portnames: + - s27-t211-sut1-c18/p1 - 40GE-port1 xl710-QDA2-2p40GE. + - s27-t211-sut1-c18/p2 - 40GE-port2 xl710-QDA2-2p40GE. +- TG [Server-Type-E31]: + - testbedname: testbed33 and testbed211. + - hostname: s19-t33t211-tg1. + - IPMI IP: 10.30.50.46 + - Host IP: 10.30.51.49 + - portnames: + - s19-t33t211-tg1-c2/p1 - 10GE-port1 x710-4p10GE. + - s19-t33t211-tg1-c2/p2 - 10GE-port2 x710-4p10GE. + - s19-t33t211-tg1-c2/p3 - 10GE-port3 x710-4p10GE. + - s19-t33t211-tg1-c2/p4 - 10GE-port4 x710-4p10GE. + - s19-t33t211-tg1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s19-t33t211-tg1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s19-t33t211-tg1-c8/p1 - 40GE-port1 xl710-QDA2-2p40GE. + - s19-t33t211-tg1-c8/p2 - 40GE-port2 xl710-QDA2-2p40GE. + - s19-t33t211-tg1-c10/p1 - 10GE-port1 x710-4p10GE. + - s19-t33t211-tg1-c10/p2 - 10GE-port2 x710-4p10GE. + - s19-t33t211-tg1-c10/p3 - 10GE-port3 x710-4p10GE. + - s19-t33t211-tg1-c10/p4 - 10GE-port4 x710-4p10GE. +``` + +### 2-Node-Icelake (2n-icx) + +{{< figure src="/cdocs/testbed-2n-icx.svg" >}} + +``` +- SUT [Server-Type-F1]: + - testbedname: testbed212. + - hostname: s71-t212-sut1. + - IPMI IP: 10.30.50.81 + - Host IP: 10.30.51.81 + - portnames: + - s71-t212-sut1-c2/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s71-t212-sut1-c2/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s71-t212-sut1-c4/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s71-t212-sut1-c4/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s71-t212-sut1-c4/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s71-t212-sut1-c4/p4 - 25GE-port4 e810-XXVDA4-4p25GE. + - s71-t212-sut1-c9/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s71-t212-sut1-c9/p2 - 100GE-port2 e810-2CQDA2-2p100GE. + - s71-t212-sut1-c8/p1 - 200GE-port1 ConnectX7-2p200GE. + - s71-t212-sut1-c8/p2 - 200GE-port2 ConnectX7-2p200GE. +- TG [Server-Type-F2]: + - testbedname: testbed212. + - hostname: s72-t212-tg1. + - IPMI IP: 10.30.50.82 + - Host IP: 10.30.51.82 + - portnames: + - s72-t212-tg1-c2/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s72-t212-tg1-c2/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s72-t212-tg1-c4/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s72-t212-tg1-c4/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s72-t212-tg1-c4/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s72-t212-tg1-c4/p4 - 25GE-port4 e810-XXVDA4-4p25GE. + - s72-t212-tg1-c9/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s72-t212-tg1-c9/p2 - 100GE-port2 e810-2CQDA2-2p100GE. + - s72-t212-tg1-c6/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s72-t212-tg1-c6/p2 - 100GE-port2 e810-2CQDA2-2p100GE. + - s72-t212-tg1-c8/p1 - 200GE-port1 ConnectX7-2p200GE. + - s72-t212-tg1-c8/p2 - 200GE-port2 ConnectX7-2p200GE. +- SUT [Server-Type-F1]: + - testbedname: testbed213. + - hostname: s83-t213-sut1. + - IPMI IP: 10.30.50.83 + - Host IP: 10.30.51.83 + - portnames: + - s83-t213-sut1-c2/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s83-t213-sut1-c2/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s83-t213-sut1-c4/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s83-t213-sut1-c4/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s83-t213-sut1-c4/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s83-t213-sut1-c4/p4 - 25GE-port4 e810-XXVDA4-4p25GE. + - s83-t213-sut1-c9/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s83-t213-sut1-c9/p2 - 100GE-port2 e810-2CQDA2-2p100GE. + - s83-t213-sut1-c8/p1 - 200GE-port1 ConnectX7-2p200GE. + - s83-t213-sut1-c8/p2 - 200GE-port2 ConnectX7-2p200GE. +- TG [Server-Type-F2]: + - testbedname: testbed213. + - hostname: s84-t213-tg1. + - IPMI IP: 10.30.50.84 + - Host IP: 10.30.51.84 + - portnames: + - s84-t213-tg1-c2/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s84-t213-tg1-c2/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s84-t213-tg1-c4/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s84-t213-tg1-c4/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s84-t213-tg1-c4/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s84-t213-tg1-c4/p4 - 25GE-port4 e810-XXVDA4-4p25GE. + - s84-t213-tg1-c9/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s84-t213-tg1-c9/p2 - 100GE-port2 e810-2CQDA2-2p100GE. + - s84-t213-tg1-c6/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s84-t213-tg1-c6/p2 - 100GE-port2 e810-2CQDA2-2p100GE. + - s84-t213-tg1-c8/p1 - 200GE-port1 ConnectX7-2p200GE. + - s84-t213-tg1-c8/p2 - 200GE-port2 ConnectX7-2p200GE. +- SUT [Server-Type-F1]: + - testbedname: testbed214. + - hostname: s85-t214-sut1. + - IPMI IP: 10.30.50.85 + - Host IP: 10.30.51.85 + - portnames: + - s85-t214-sut1-c2/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s85-t214-sut1-c2/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s85-t214-sut1-c4/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s85-t214-sut1-c4/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s85-t214-sut1-c4/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s85-t214-sut1-c4/p4 - 25GE-port4 e810-XXVDA4-4p25GE. + - s85-t214-sut1-c9/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s85-t214-sut1-c9/p2 - 100GE-port2 e810-2CQDA2-2p100GE. + - s85-t214-sut1-c8/p1 - 200GE-port1 ConnectX7-2p200GE. + - s85-t214-sut1-c8/p2 - 200GE-port2 ConnectX7-2p200GE. +- TG [Server-Type-F2]: + - testbedname: testbed214. + - hostname: s86-t214-tg1. + - IPMI IP: 10.30.50.86 + - Host IP: 10.30.51.86 + - portnames: + - s86-t214-tg1-c2/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s86-t214-tg1-c2/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s86-t214-tg1-c4/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s86-t214-tg1-c4/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s86-t214-tg1-c4/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s86-t214-tg1-c4/p4 - 25GE-port4 e810-XXVDA4-4p25GE. + - s86-t214-tg1-c9/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s86-t214-tg1-c9/p2 - 100GE-port2 e810-2CQDA2-2p100GE. + - s86-t214-tg1-c6/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s86-t214-tg1-c6/p2 - 100GE-port2 e810-2CQDA2-2p100GE. + - s86-t214-tg1-c8/p1 - 200GE-port1 ConnectX7-2p200GE. + - s86-t214-tg1-c8/p2 - 200GE-port2 ConnectX7-2p200GE. +- SUT [Server-Type-F1]: + - testbedname: testbed215. + - hostname: s87-t215-sut1. + - IPMI IP: 10.30.50.87 + - Host IP: 10.30.51.87 + - portnames: + - s87-t215-sut1-c2/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s87-t215-sut1-c2/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s87-t215-sut1-c4/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s87-t215-sut1-c4/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s87-t215-sut1-c4/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s87-t215-sut1-c4/p4 - 25GE-port4 e810-XXVDA4-4p25GE. + - s87-t215-sut1-c9/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s87-t215-sut1-c9/p2 - 100GE-port2 e810-2CQDA2-2p100GE. + - s87-t215-sut1-c8/p1 - 200GE-port1 ConnectX7-2p200GE. + - s87-t215-sut1-c8/p2 - 200GE-port2 ConnectX7-2p200GE. +- TG [Server-Type-F2]: + - testbedname: testbed215. + - hostname: s88-t215-tg1. + - IPMI IP: 10.30.50.88 + - Host IP: 10.30.51.88 + - portnames: + - s88-t215-tg1-c2/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s88-t215-tg1-c2/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s88-t215-tg1-c4/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s88-t215-tg1-c4/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s88-t215-tg1-c4/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s88-t215-tg1-c4/p4 - 25GE-port4 e810-XXVDA4-4p25GE. + - s88-t215-tg1-c9/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s88-t215-tg1-c9/p2 - 100GE-port2 e810-2CQDA2-2p100GE. + - s88-t215-tg1-c6/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s88-t215-tg1-c6/p2 - 100GE-port2 e810-2CQDA2-2p100GE. + - s88-t215-tg1-c8/p1 - 200GE-port1 ConnectX7-2p200GE. + - s88-t215-tg1-c8/p2 - 200GE-port2 ConnectX7-2p200GE. +``` + +### 3-Node-Taishan (3n-tsh) + +{{< figure src="/cdocs/testbed-3n-tsh.svg" >}} + +``` +- SUT [Server-Type-E21]: + - testbedname: testbed33. + - hostname: s17-t33-sut1. + - IPMI IP: 10.30.50.36 + - Host IP: 10.30.51.36 + - portnames: + - s17-t33-sut1-c6/p1 - 10GE-port1 x520-2p10GE. + - s17-t33-sut1-c6/p2 - 10GE-port2 x520-2p10GE. + - s17-t33-sut1-c4/p1 - 25GE-port1 cx4-2p25GE. + - s17-t33-sut1-c4/p2 - 25GE-port2 cx4-2p25GE. +- SUT [Server-Type-E21]: + - testbedname: testbed33. + - hostname: s18-t33-sut2. + - IPMI IP: 10.30.50.37 + - Host IP: 10.30.51.37 + - portnames: + - s18-t33-sut2-c6/p1 - 10GE-port1 x520-2p10GE. + - s18-t33-sut2-c6/p2 - 10GE-port2 x520-2p10GE. + - s18-t33-sut2-c4/p1 - 25GE-port1 cx4-2p25GE. + - s18-t33-sut2-c4/p2 - 25GE-port2 cx4-2p25GE. +- TG [Server-Type-E31]: + - testbedname: testbed33 and testbed211. + - hostname: s19-t33t211-tg1. + - IPMI IP: 10.30.50.46 + - Host IP: 10.30.51.49 + - portnames: + - s19-t33t211-tg1-c2/p1 - 10GE-port1 x710-4p10GE. + - s19-t33t211-tg1-c2/p2 - 10GE-port2 x710-4p10GE. + - s19-t33t211-tg1-c2/p3 - 10GE-port3 x710-4p10GE. + - s19-t33t211-tg1-c2/p4 - 10GE-port4 x710-4p10GE. + - s19-t33t211-tg1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s19-t33t211-tg1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s19-t33t211-tg1-c8/p1 - 40GE-port1 xl710-QDA2-2p40GE. + - s19-t33t211-tg1-c8/p2 - 40GE-port2 xl710-QDA2-2p40GE. + - s19-t33t211-tg1-c10/p1 - 10GE-port1 x710-4p10GE. + - s19-t33t211-tg1-c10/p2 - 10GE-port2 x710-4p10GE. + - s19-t33t211-tg1-c10/p3 - 10GE-port3 x710-4p10GE. + - s19-t33t211-tg1-c10/p4 - 10GE-port4 x710-4p10GE. +``` + +### 3-Node-Altra (3n-alt) + +{{< figure src="/cdocs/testbed-3n-alt.svg" >}} + +``` +- SUT [Server-Type-E23]: + - testbedname: testbed34. + - hostname: s62-t34-sut1. + - IPMI IP: 10.30.50.72 + - Host IP: 10.30.51.72 + - portnames: + - s62-t34-sut1-c1/p1 - 40GE-port1 xl710-QDA2-2p40GE. + - s62-t34-sut1-c1/p2 - 40GE-port2 xl710-QDA2-2p40GE. + - s62-t34-sut1-c8/p1 - 100GE-port1 ConnectX6-2p100GE. + - s62-t34-sut1-c8/p2 - 100GE-port2 ConnectX6-2p100GE. +- SUT [Server-Type-E23]: + - testbedname: testbed34. + - hostname: s63-t34-sut2. + - IPMI IP: 10.30.50.73 + - Host IP: 10.30.51.73 + - portnames: + - s63-t34-sut2-c1/p1 - 40GE-port1 xl710-QDA2-2p40GE. + - s63-t34-sut2-c1/p2 - 40GE-port2 xl710-QDA2-2p40GE. + - s63-t34-sut2-c8/p1 - 100GE-port1 ConnectX6-2p100GE. + - s63-t34-sut2-c8/p2 - 100GE-port2 ConnectX6-2p100GE. +- TG [Server-Type-E32]: + - testbedname: testbed34. + - hostname: s64-t34-tg1. + - IPMI IP: 10.30.50.74 + - Host IP: 10.30.51.74 + - portnames: + - s64-t34-tg1-c2/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s64-t34-tg1-c2/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s64-t34-tg1-c4/p1 - 40GE-port1 xl710-QDA2-2p40GE. + - s64-t34-tg1-c4/p2 - 40GE-port2 xl710-QDA2-2p40GE. + - s64-t34-tg1-c8/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s64-t34-tg1-c8/p2 - 100GE-port2 e810-2CQDA2-2p100GE. +``` + +### 3-Node-Icelake (3n-icx) + +{{< figure src="/cdocs/testbed-3n-icx.svg" >}} + +``` +- SUT1 [Server-Type-F3]: + - testbedname: testbed37. + - hostname: s65-t37-sut1. + - IPMI IP: 10.30.50.75 + - Host IP: 10.30.51.75 + - portnames: + - s65-t37-sut1-c2/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s65-t37-sut1-c2/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s65-t37-sut1-c4/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s65-t37-sut1-c4/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s65-t37-sut1-c4/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s65-t37-sut1-c4/p4 - 25GE-port4 e810-XXVDA4-4p25GE. + - s65-t37-sut1-c9/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s65-t37-sut1-c9/p2 - 100GE-port2 e810-2CQDA2-2p100GE. + - s65-t37-sut1-c10/p1 - 100GE-port1 ConnectX6-2p100GE. + - s65-t37-sut1-c10/p2 - 100GE-port2 ConnectX6-2p100GE. +- SUT2 [Server-Type-F3]: + - testbedname: testbed37. + - hostname: s66-t37-sut2. + - IPMI IP: 10.30.50.76 + - Host IP: 10.30.51.76 + - portnames: + - s66-t37-sut2-c2/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s66-t37-sut2-c2/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s66-t37-sut2-c4/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s66-t37-sut2-c4/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s66-t37-sut2-c4/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s66-t37-sut2-c4/p4 - 25GE-port4 e810-XXVDA4-4p25GE. + - s66-t37-sut2-c9/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s66-t37-sut2-c9/p2 - 100GE-port2 e810-2CQDA2-2p100GE. + - s65-t37-sut1-c10/p1 - 100GE-port1 ConnectX6-2p100GE. + - s65-t37-sut1-c10/p2 - 100GE-port2 ConnectX6-2p100GE. +- TG [Server-Type-F3]: + - testbedname: testbed37. + - hostname: s67-t37-tg1. + - IPMI IP: 10.30.50.77 + - Host IP: 10.30.51.77 + - portnames: + - s67-t37-tg1-c2/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s67-t37-tg1-c2/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s67-t37-tg1-c4/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s67-t37-tg1-c4/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s67-t37-tg1-c4/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s67-t37-tg1-c4/p4 - 25GE-port4 e810-XXVDA4-4p25GE. + - s67-t37-tg1-c9/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s67-t37-tg1-c9/p2 - 100GE-port2 e810-2CQDA2-2p100GE. + - s67-t37-tg1-c10/p1 - 100GE-port1 ConnectX6-2p100GE. + - s67-t37-tg1-c10/p2 - 100GE-port2 ConnectX6-2p100GE. +- SUT1 [Server-Type-F3]: + - testbedname: testbed38. + - hostname: s78-t38-sut1. + - IPMI IP: 10.30.50.78 + - Host IP: 10.30.51.78 + - portnames: + - s78-t38-sut1-c2/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s78-t38-sut1-c2/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s78-t38-sut1-c4/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s78-t38-sut1-c4/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s78-t38-sut1-c4/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s78-t38-sut1-c4/p4 - 25GE-port4 e810-XXVDA4-4p25GE. + - s78-t38-sut1-c9/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s78-t38-sut1-c9/p2 - 100GE-port2 e810-2CQDA2-2p100GE. + - s78-t38-sut1-c10/p1 - 100GE-port1 ConnectX6-2p100GE. + - s78-t38-sut1-c10/p2 - 100GE-port2 ConnectX6-2p100GE. +- SUT2 [Server-Type-F3]: + - testbedname: testbed38. + - hostname: s79-t38-sut2. + - IPMI IP: 10.30.50.79 + - Host IP: 10.30.51.79 + - portnames: + - s79-t38-sut2-c2/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s79-t38-sut2-c2/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s79-t38-sut2-c4/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s79-t38-sut2-c4/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s79-t38-sut2-c4/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s79-t38-sut2-c4/p4 - 25GE-port4 e810-XXVDA4-4p25GE. + - s79-t38-sut2-c9/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s79-t38-sut2-c9/p2 - 100GE-port2 e810-2CQDA2-2p100GE. + - s79-t38-sut2-c10/p1 - 100GE-port1 ConnectX6-2p100GE. + - s79-t38-sut2-c10/p2 - 100GE-port2 ConnectX6-2p100GE. +- TG [Server-Type-F3]: + - testbedname: testbed38. + - hostname: s80-t38-tg1. + - IPMI IP: 10.30.50.80 + - Host IP: 10.30.51.80 + - portnames: + - s80-t38-tg1-c2/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s80-t38-tg1-c2/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s80-t38-tg1-c4/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s80-t38-tg1-c4/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s80-t38-tg1-c4/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s80-t38-tg1-c4/p4 - 25GE-port4 e810-XXVDA4-4p25GE. + - s80-t38-tg1-c9/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s80-t38-tg1-c9/p2 - 100GE-port2 e810-2CQDA2-2p100GE. + - s80-t38-tg1-c10/p1 - 100GE-port1 ConnectX6-2p100GE. + - s80-t38-tg1-c10/p2 - 100GE-port2 ConnectX6-2p100GE. +``` + +### 3-Node-SnowRidge (3n-snr) + +{{< figure src="/cdocs/testbed-3n-snr.svg" >}} + +``` +- ServerG1 [Server-Type-G1]: + - testbedname: testbed39. + - hostname: s93-t39-sut1. + - IPMI IP: 10.30.50.93 + - Host IP: 10.30.51.93 + - portnames: + - s93-t39-sut1-c1/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s93-t39-sut1-c1/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s93-t39-sut1-c1/p2 - 25GE-port3 e810-XXVDA4-4p25GE. + - s93-t39-sut1-c1/p2 - 25GE-port4 e810-XXVDA4-4p25GE. +- ServerG1 [Server-Type-G1]: + - testbedname: testbed39. + - hostname: s94-t39-sut2. + - IPMI IP: 10.30.50.94 + - Host IP: 10.30.51.94 + - portnames: + - s94-t39-sut2-c1/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s94-t39-sut2-c1/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s94-t39-sut2-c1/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s94-t39-sut2-c1/p4 - 25GE-port4 e810-XXVDA4-4p25GE. +- ServerF4 [Server-Type-F5]: + - testbedname: testbed39. + - hostname: s89-t39t310-tg1. + - IPMI IP: 10.30.50.89 + - Host IP: 10.30.51.89 + - portnames: + - s89-t39t310-tg1-c6/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s89-t39t310-tg1-c6/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s89-t39t310-tg1-c6/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s89-t39t310-tg1-c6/p4 - 25GE-port4 e810-XXVDA4-4p25GE. +``` + +### 2-Node-SapphireRapids (2n-spr) + +{{< figure src="/cdocs/testbed-2n-spr.svg" >}} + +``` +- SUT [Server-Type-H1]: + - testbedname: testbed21. + - hostname: s52-t21-sut1. + - IPMI IP: 10.30.50.52 + - Host IP: 10.30.51.52 + - portnames: + - s52-t21-sut1-c10/p1 - 200GE-port1 ConnectX7-2p200GE. + - s52-t21-sut1-c10/p2 - 200GE-port2 ConnectX7-2p200GE. + - s52-t21-sut1-c4/p1 - 200GE-port1 ConnectX7-2p200GE. + - s52-t21-sut1-c4/p2 - 200GE-port2 ConnectX7-2p200GE. + - s52-t21-sut1-c2/p1 - 200GE-port1 ConnectX7-2p200GE. + - s52-t21-sut1-c2/p2 - 200GE-port2 ConnectX7-2p200GE. + - s52-t21-sut1-c9/p1 - 200GE-port1 ConnectX7-2p200GE. + - s52-t21-sut1-c9/p2 - 200GE-port2 ConnectX7-2p200GE. + - s52-t21-sut1-c7/p1 - 200GE-port1 ConnectX7-2p200GE. + - s52-t21-sut1-c7/p2 - 200GE-port2 ConnectX7-2p200GE. +- TG [Server-Type-H2]: + - testbedname: testbed21. + - hostname: s53-t21-tg1. + - IPMI IP: 10.30.50.53 + - Host IP: 10.30.51.53 + - portnames: + - s53-t21-tg1-c10/p1 - 200GE-port1 ConnectX7-2p200GE. + - s53-t21-tg1-c10/p2 - 200GE-port2 ConnectX7-2p200GE. + - s53-t21-tg1-c4/p1 - 200GE-port1 ConnectX7-2p200GE. + - s53-t21-tg1-c4/p2 - 200GE-port2 ConnectX7-2p200GE. + - s53-t21-tg1-c2/p1 - 200GE-port1 ConnectX7-2p200GE. + - s53-t21-tg1-c2/p2 - 200GE-port2 ConnectX7-2p200GE. + - s53-t21-tg1-c9/p1 - 200GE-port1 ConnectX7-2p200GE. + - s53-t21-tg1-c9/p2 - 200GE-port2 ConnectX7-2p200GE. + - s53-t21-tg1-c7/p1 - 200GE-port1 ConnectX7-2p200GE. + - s53-t21-tg1-c7/p2 - 200GE-port2 ConnectX7-2p200GE. +- SUT [Server-Type-H3]: + - testbedname: testbed22. + - hostname: s54-t22-sut1. + - IPMI IP: 10.30.50.54 + - Host IP: 10.30.51.54 + - portnames: + - s54-t22-sut1-c2/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s54-t22-sut1-c2/p2 - 100GE-port2 e810-2CQDA2-2p100GE. + - s54-t22-sut1-c4/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s54-t22-sut1-c4/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s54-t22-sut1-c4/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s54-t22-sut1-c4/p4 - 25GE-port4 e810-XXVDA4-4p25GE. + - s54-t22-sut1-c7/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s54-t22-sut1-c7/p2 - 100GE-port2 e810-2CQDA2-2p100GE. + - s54-t22-sut1-c9/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s54-t22-sut1-c9/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s54-t22-sut1-c9/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s54-t22-sut1-c9/p4 - 25GE-port4 e810-XXVDA4-4p25GE. +- TG [Server-Type-H4]: + - testbedname: testbed22. + - hostname: s55-t22-tg1. + - IPMI IP: 10.30.50.55 + - Host IP: 10.30.51.55 + - portnames: + - s55-t22-tg1-c2/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s55-t22-tg1-c2/p2 - 100GE-port2 e810-2CQDA2-2p100GE. + - s55-t22-tg1-c4/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s55-t22-tg1-c4/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s55-t22-tg1-c4/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s55-t22-tg1-c4/p4 - 25GE-port4 e810-XXVDA4-4p25GE. +- SUT [Server-Type-H5]: + - testbedname: testbed23. + - hostname: s56-t23-sut1. + - IPMI IP: 10.30.50.56 + - Host IP: 10.30.51.56 + - portnames: + - s56-t23-sut1-c2/p1 - 200GE-port1 ConnectX7-2p200GE. + - s56-t23-sut1-c2/p2 - 200GE-port2 ConnectX7-2p200GE. + - s56-t23-sut1-c4/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s56-t23-sut1-c4/p2 - 100GE-port2 e810-2CQDA2-2p100GE. + - s56-t23-sut1-c10/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s56-t23-sut1-c10/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s56-t23-sut1-c10/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s56-t23-sut1-c10/p4 - 25GE-port4 e810-XXVDA4-4p25GE. +- TG [Server-Type-H6]: + - testbedname: testbed23. + - hostname: s57-t23-tg1. + - IPMI IP: 10.30.50.57 + - Host IP: 10.30.51.57 + - portnames: + - s57-t23-tg1-c2/p1 - 200GE-port1 ConnectX7-2p200GE. + - s57-t23-tg1-c2/p2 - 200GE-port2 ConnectX7-2p200GE. + - s57-t23-tg1-c4/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s57-t23-tg1-c4/p2 - 100GE-port2 e810-2CQDA2-2p100GE. + - s57-t23-tg1-c10/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s57-t23-tg1-c10/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s57-t23-tg1-c10/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s57-t23-tg1-c10/p4 - 25GE-port4 e810-XXVDA4-4p25GE. + - s57-t23-tg1-c7/p1 - 200GE-port1 ConnectX7-2p200GE. + - s57-t23-tg1-c7/p2 - 200GE-port2 ConnectX7-2p200GE. + - s57-t23-tg1-c9/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s57-t23-tg1-c9/p2 - 100GE-port2 e810-2CQDA2-2p100GE. +- SUT [Server-Type-H5]: + - testbedname: testbed24. + - hostname: s58-t24-sut1. + - IPMI IP: 10.30.50.58 + - Host IP: 10.30.51.58 + - portnames: + - s58-t24-sut1-c2/p1 - 200GE-port1 ConnectX7-2p200GE. + - s58-t24-sut1-c2/p2 - 200GE-port2 ConnectX7-2p200GE. + - s58-t24-sut1-c4/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s58-t24-sut1-c4/p2 - 100GE-port2 e810-2CQDA2-2p100GE. + - s58-t24-sut1-c10/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s58-t24-sut1-c10/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s58-t24-sut1-c10/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s58-t24-sut1-c10/p4 - 25GE-port4 e810-XXVDA4-4p25GE. +- TG [Server-Type-H6]: + - testbedname: testbed24. + - hostname: s59-t24-tg1. + - IPMI IP: 10.30.50.59 + - Host IP: 10.30.51.59 + - portnames: + - s59-t24-tg1-c2/p1 - 200GE-port1 ConnectX7-2p200GE. + - s59-t24-tg1-c2/p2 - 200GE-port2 ConnectX7-2p200GE. + - s59-t24-tg1-c4/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s59-t24-tg1-c4/p2 - 100GE-port2 e810-2CQDA2-2p100GE. + - s59-t24-tg1-c10/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s59-t24-tg1-c10/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s59-t24-tg1-c10/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s59-t24-tg1-c10/p4 - 25GE-port4 e810-XXVDA4-4p25GE. + - s59-t24-tg1-c7/p1 - 200GE-port1 ConnectX7-2p200GE. + - s59-t24-tg1-c7/p2 - 200GE-port2 ConnectX7-2p200GE. + - s59-t24-tg1-c9/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s59-t24-tg1-c9/p2 - 100GE-port2 e810-2CQDA2-2p100GE. +``` + +### 3-Node-IcelakeD (3n-icxd) + +{{< figure src="/cdocs/testbed-3n-icxd.svg" >}} + +``` +- ServerI1 [Server-Type-I1]: + - testbedname: testbed31. + - hostname: s32-t31-sut1. + - IPMI IP: 10.30.50.32 + - Host IP: 10.30.51.32 + - portnames: + - s32-t31-sut1-c1/p1 - 25GE-port1 e822cq-2p25GE. + - s32-t31-sut1-c1/p2 - 25GE-port2 e822cq-2p25GE. +- ServerI1 [Server-Type-I1]: + - testbedname: testbed31. + - hostname: s33-t31-sut2. + - IPMI IP: 10.30.50.33 + - Host IP: 10.30.51.33 + - portnames: + - s33-t31-sut2-c1/p1 - 25GE-port1 e822cq-2p25GE. + - s33-t31-sut2-c1/p2 - 25GE-port2 e822cq-2p25GE. +- ServerF3 [Server-Type-F5]: + - testbedname: testbed31. + - hostname: s90-t31t32-tg1. + - IPMI IP: 10.30.50.90 + - Host IP: 10.30.51.90 + - portnames: + - s90-t31t32-tg1-c4/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s90-t31t32-tg1-c4/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s90-t31t32-tg1-c4/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s90-t31t32-tg1-c4/p4 - 25GE-port4 e810-XXVDA4-4p25GE. +- ServerI1 [Server-Type-I1]: + - testbedname: testbed32. + - hostname: s34-t32-sut1. + - IPMI IP: 10.30.50.34 + - Host IP: 10.30.51.34 + - portnames: + - s34-t32-sut1-c1/p1 - 25GE-port1 e822cq-2p25GE. + - s34-t32-sut1-c1/p2 - 25GE-port2 e822cq-2p25GE. +- ServerI1 [Server-Type-I1]: + - testbedname: testbed32. + - hostname: s35-t32-sut2. + - IPMI IP: 10.30.50.35 + - Host IP: 10.30.51.35 + - portnames: + - s35-t32-sut2-c1/p1 - 25GE-port1 e822cq-2p25GE. + - s35-t32-sut2-c1/p2 - 25GE-port2 e822cq-2p25GE. +- ServerF3 [Server-Type-F5]: + - testbedname: testbed32. + - hostname: s90-t31t32-tg1. + - IPMI IP: 10.30.50.90 + - Host IP: 10.30.51.90 + - portnames: + - s90-t31t32-tg1-c6/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s90-t31t32-tg1-c6/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s90-t31t32-tg1-c6/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s90-t31t32-tg1-c6/p4 - 25GE-port4 e810-XXVDA4-4p25GE. +``` + +## Testbed Wiring + +### 1-Node-Skylake (1n-skx) + +``` +- testbed11: + - ring1 10GE-ports x710-4p10GE: + - s1-t11-sut1-c2/p1 to s1-t11-sut1-c4/p1. + - ring2 10GE-ports x710-4p10GE: + - s1-t11-sut1-c2/p2 to s1-t11-sut1-c4/p2. + - ring3 10GE-ports x710-4p10GE: + - s1-t11-sut1-c2/p3 to s1-t11-sut1-c4/p3. + - ring4 10GE-ports x710-4p10GE: + - s1-t11-sut1-c2/p4 to s1-t11-sut1-c4/p4. + - ring5 100GE-ports e810-2p100GE: + - s1-t11-sut1-c5/p1 to s1-t11-sut1-c6/p1. + - ring6 100GE-ports e810-2p100GE: + - s1-t11-sut1-c5/p2 to s1-t11-sut1-c6/p2. +- testbed12: + - ring1 10GE-ports x710-4p10GE: + - s2-t12-sut1-c2/p1 to s2-t12-sut1-c4/p1. + - ring2 10GE-ports x710-4p10GE: + - s2-t12-sut1-c2/p2 to s2-t12-sut1-c4/p2. + - ring3 10GE-ports x710-4p10GE: + - s2-t12-sut1-c2/p3 to s2-t12-sut1-c4/p3. + - ring4 10GE-ports x710-4p10GE: + - s2-t12-sut1-c2/p4 to s2-t12-sut1-c4/p4. + - ring5 100GE-ports e810-2p100GE: + - s2-t12-sut1-c5/p1 to s2-t12-sut1-c6/p1. + - ring6 100GE-ports e810-2p100GE: + - s2-t12-sut1-c5/p2 to s2-t12-sut1-c6/p2. +``` + +### 1-Node-Altra (1n-alt) + +``` +- testbed13: + - ring1 40GE-ports XL710-QDA2-2p40GE on SUTs: + - s70-t13-sut1-c0/p1 - s70-t13-sut1-c3/p1. + - ring2 40GE-ports XL710-QDA2-2p40GE on SUTs: + - s70-t13-sut1-c0/p2 - s70-t13-sut1-c3/p2. + - ring3 100GE-ports ConnectX6-2p100GE on SUTs: + - s70-t13-sut1-c1/p1 - s70-t13-sut1-c1/p2. + - ring4 10/25GE-ports ConnectX5-2p10/25GE on SUTs: + - s70-t13-sut1-c2/p1 - s70-t13-sut1-c5/p1. + - ring5 10/25GE-ports ConnectX5-2p10/25GE on SUTs: + - s70-t13-sut1-c2/p2 - s70-t13-sut1-c5/p2. + +- testbed14: + - ring1 40GE-ports XL710-QDA2-2p40GE on SUTs: + - s71-t14-sut1-c0/p1 - s71-t14-sut1-c3/p1. + - ring2 40GE-ports XL710-QDA2-2p40GE on SUTs: + - s71-t14-sut1-c0/p2 - s71-t14-sut1-c3/p2. + - ring3 100GE-ports ConnectX6-2p100GE on SUTs: + - s71-t14-sut1-c1/p1 - s71-t14-sut1-c1/p2. + - ring4 10/25GE-ports ConnectX5-2p10/25GE on SUTs: + - s71-t14-sut1-c2/p1 - s71-t14-sut1-c5/p1. + - ring5 10/25GE-ports ConnectX5-2p10/25GE on SUTs: + - s71-t14-sut1-c2/p2 - s71-t14-sut1-c5/p2. +``` + +### 1-Node-SapphireRapids (1n-spr) + +``` +- testbed15: + - ring1 100GE-ports e810-2p100GE: + - s30-t15-sut1-c1/p1 to s30-t15-sut1-c2/p1. + - ring2 100GE-ports e810-2p100GE: + - s30-t15-sut1-c1/p2 to s30-t15-sut1-c2/p2. +- testbed16: + - ring1 100GE-ports e810-2p100GE: + - s31-t16-sut1-c1/p1 to s31-t16-sut1-c2/p1. + - ring2 100GE-ports e810-2p100GE: + - s31-t16-sut1-c1/p2 to s31-t16-sut1-c2/p2. +``` + +### 2-Node-IxiaPS1L47 (2n-ps1) + +``` +- testbed25: + - link1 10GE-port x710-4p10GE on SUT: + - t25-tg1-p1 to t25-sut1-c2/p1. + - link2 10GE-port x710-4p10GE on SUT: + - t25-tg1-p2 to t25-sut1-c2/p2. + - link3 10GE-port x710-4p10GE on SUT: + - t25-tg1-p3 to t25-sut1-c2/p3. + - link4 10GE-port x710-4p10GE on SUT: + - t25-tg1-p4 to t25-sut1-c2/p4. +``` + +### 2-Node-Cascadelake (2n-clx) + +``` +- testbed27: + - ring1 10GE-ports x710-4p10GE on SUT: + - s34-t27-tg1-c2/p1 to s33-t27-sut1-c2/p1. + - s33-t27-sut1-c2/p2 to s34-t27-tg1-c2/p2. + - ring2 10GE-ports x710-4p10GE on SUT: + - s34-t27-tg1-c2/p3 to s33-t27-sut1-c2/p3. + - s33-t27-sut1-c2/p4 to s34-t27-tg1-c2/p4. + - ring3 25GE-ports xxv710-DA2-2p25GE on SUT + - s34-t27-tg1-c4/p1 to s33-t27-sut1-c4/p1. + - s33-t27-sut1-c4/p2 to s34-t27-tg1-c4/p2. + - ring4 100GE-ports ConnectX5-2p100GE on SUT: + - s34-t27-tg1-c9/p1 to s33-t27-sut1-c9/p1. + - s33-t27-sut1-c9/p2 to s34-t27-tg1-c9/p2. + - ring5 100GE-ports e810-2p100GE on SUT 100GE-ports ConnectX5-2p100GE on TG: + - s34-t27-tg1-c6/p1 to s33-t27-sut1-c6/p1. + - s33-t27-sut1-c6/p2 to s34-t27-tg1-c6/p2. + - ring6 100GE-ports e810-2p100GE on TG: + - s34-t27-tg1-c8/p1 to s34-t27-tg1-c8/p2. + - s34-t27-tg1-c8/p2 to s34-t27-tg1-c8/p1. +- testbed28: + - ring1 10GE-ports x710-4p10GE on SUT: + - s36-t28-tg1-c2/p1 to s35-t28-sut1-c2/p1. + - s35-t28-sut1-c2/p2 to s36-t28-tg1-c2/p2. + - ring2 10GE-ports x710-4p10GE on SUT: + - s36-t28-tg1-c2/p3 to s35-t28-sut1-c2/p3. + - s35-t28-sut1-c2/p4 to s36-t28-tg1-c2/p4. + - ring3 25GE-ports xxv710-DA2-2p25GE on SUT + - s36-t28-tg1-c4/p1 to s35-t28-sut1-c4/p1. + - s35-t28-sut1-c4/p2 to s36-t28-tg1-c4/p2. + - ring4 100GE-ports ConnectX5-2p100GE on SUT: + - s36-t28-tg1-c9/p1 to s35-t28-sut1-c9/p1. + - s35-t28-sut1-c9/p2 to s36-t28-tg1-c9/p2. + - ring5 100GE-ports e810-2p100GE on SUT 100GE-ports ConnectX5-2p100GE on TG: + - s36-t28-tg1-c6/p1 to s35-t28-sut1-c6/p1. + - s35-t28-sut1-c6/p2 to s36-t28-tg1-c6/p2. + - ring6 100GE-ports e810-2p100GE on TG: + - s36-t28-tg1-c8/p1 to s36-t28-tg1-c8/p2. + - s36-t28-tg1-c8/p2 to s36-t28-tg1-c8/p1. +- testbed29: + - ring1 10GE-ports x710-4p10GE on SUT: + - s38-t29-tg1-c2/p1 to s37-t29-sut1-c2/p1. + - s37-t29-sut1-c2/p2 to s38-t29-tg1-c2/p2. + - ring2 10GE-ports x710-4p10GE on SUT: + - s38-t29-tg1-c2/p3 to s37-t29-sut1-c2/p3. + - s37-t29-sut1-c2/p4 to s38-t29-tg1-c2/p4. + - ring3 25GE-ports xxv710-DA2-2p25GE on SUT + - s38-t29-tg1-c4/p1 to s37-t29-sut1-c4/p1. + - s37-t29-sut1-c4/p2 to s38-t29-tg1-c4/p2. + - ring4 100GE-ports ConnectX5-2p100GE on SUT: + - s38-t29-tg1-c9/p1 to s37-t29-sut1-c9/p1. + - s37-t29-sut1-c9/p2 to s38-t29-tg1-c9/p2. + - ring5 100GE-ports e810-2p100GE on SUT 100GE-ports ConnectX5-2p100GE on TG: + - s38-t29-tg1-c6/p1 to s37-t29-sut1-c6/p1. + - s37-t29-sut1-c6/p2 to s38-t29-tg1-c6/p2. +``` + +### 2-Node-Zen2 (2n-zn2) + +``` +- testbed210: + - ring1 10GE-ports x710-4p10GE on SUT: + - s61-t210-tg1-c2/p1 to s60-t210-sut1-c1/p1. + - s60-t210-sut1-c1/p2 to s61-t210-tg1-c2/p2. + - ring2 10GE-ports x710-4p10GE on SUT: + - s61-t210-tg1-c2/p3 to s60-t210-sut1-c1/p3. + - s60-t210-sut1-c1/p4 to s61-t210-tg1-c2/p4. + - ring3 25GE-ports xxv710-DA2-2p25GE on SUT + - s61-t210-tg1-c3/p1 to s60-t210-sut1-c2/p1. + - s60-t210-sut1-c2/p2 to s61-t210-tg1-c3/p2. + - ring4 100GE-ports ConnectX5-2p100GE on SUT: + - s61-t210-tg1-c1/p1 to s60-t210-sut1-c3/p1. + - s60-t210-sut1-c3/p2 to s61-t210-tg1-c1/p2. +``` + +### 2-Node-ThunderX2 (2n-tx2) + +``` +- testbed211: + - ring1 10GE-ports x520-2p10GE on SUTs: + - s27-t211-sut1-c18/p1 - s19-t33t211-tg1-c8/p1. + - s27-t211-sut1-c18/p2 - s19-t33t211-tg1-c8/p2. +``` + +### 2-Node-Icelake (2n-icx) + +``` +- testbed212: + - ring1 25GE-ports xxv710-DA2-2p25GE: + - s72-t212-tg1-c2/p1 to s71-t212-sut1-c2/p1. + - s71-t212-sut1-c2/p2 to s72-t212-tg1-c2/p2. + - ring2 25GE-ports e810-XXVDA4-2p25GE: + - s72-t212-tg1-c4/p1 to s71-t212-sut1-c4/p1. + - s71-t212-sut1-c4/p2 to s72-t212-tg1-c4/p2. + - s72-t212-tg1-c4/p3 to s71-t212-sut1-c4/p3. + - s71-t212-sut1-c4/p4 to s72-t212-tg1-c4/p4. + - ring3 100GE-ports e810-2CQDA2-2p100GE: + - s72-t212-tg1-c9/p1 to s71-t212-sut1-c9/p1. + - s71-t212-sut1-c9/p2 to s72-t212-tg1-c9/p2. + - ring4 100GE-ports e810-2CQDA2-2p100GE: + - s72-t212-tg1-c6/p1 to s72-t212-tg1-c6/p2. + - s72-t212-tg1-c6/p2 to s72-t212-tg1-c6/p1. + - ring5 200GE-ports ConnectX7-2p200GE: + - s72-t212-tg1-c8/p1 to s71-t212-sut1-c8/p1. + - s71-t212-sut1-c8/p2 to s72-t212-tg1-c8/p2. +- testbed213: + - ring1 25GE-ports xxv710-DA2-2p25GE: + - s84-t213-tg1-c2/p1 to s83-t213-sut1-c2/p1. + - s83-t213-sut1-c2/p2 to s84-t213-tg1-c2/p2. + - ring2 25GE-ports e810-XXVDA4-2p25GE: + - s84-t213-tg1-c4/p1 to s83-t213-sut1-c4/p1. + - s83-t213-sut1-c4/p2 to s84-t213-tg1-c4/p2. + - s84-t213-tg1-c4/p3 to s83-t213-sut1-c4/p3. + - s83-t213-sut1-c4/p4 to s84-t213-tg1-c4/p4. + - ring3 100GE-ports e810-2CQDA2-2p100GE: + - s84-t213-tg1-c9/p1 to s83-t213-sut1-c9/p1. + - s83-t213-sut1-c9/p2 to s84-t213-tg1-c9/p2. + - ring4 100GE-ports e810-2CQDA2-2p100GE: + - s84-t213-tg1-c6/p1 to s84-t213-tg1-c6/p2. + - s84-t213-tg1-c6/p2 to s84-t213-tg1-c6/p1. + - ring5 200GE-ports ConnectX7-2p200GE: + - s84-t213-tg1-c8/p1 to s83-t213-sut1-c8/p1. + - s83-t213-sut1-c8/p2 to s84-t213-tg1-c8/p2. +- testbed214: + - ring1 25GE-ports xxv710-DA2-2p25GE: + - s86-t214-tg1-c2/p1 to s85-t214-sut1-c2/p1. + - s85-t214-sut1-c2/p2 to s86-t214-tg1-c2/p2. + - ring2 25GE-ports e810-XXVDA4-2p25GE: + - s86-t214-tg1-c4/p1 to s85-t214-sut1-c4/p1. + - s85-t214-sut1-c4/p2 to s86-t214-tg1-c4/p2. + - s86-t214-tg1-c4/p3 to s85-t214-sut1-c4/p3. + - s85-t214-sut1-c4/p4 to s86-t214-tg1-c4/p4. + - ring3 100GE-ports e810-2CQDA2-2p100GE: + - s86-t214-tg1-c9/p1 to s85-t214-sut1-c9/p1. + - s85-t214-sut1-c9/p2 to s86-t214-tg1-c9/p2. + - ring4 100GE-ports e810-2CQDA2-2p100GE: + - s86-t214-tg1-c6/p1 to s86-t214-tg1-c6/p2. + - s86-t214-tg1-c6/p2 to s86-t214-tg1-c6/p1. + - ring5 200GE-ports ConnectX7-2p200GE: + - s86-t214-tg1-c8/p1 to s85-t214-sut1-c8/p1. + - s85-t214-sut1-c8/p2 to s86-t214-tg1-c8/p2. +- testbed215: + - ring1 25GE-ports xxv710-DA2-2p25GE: + - s88-t215-tg1-c2/p1 to s87-t215-sut1-c2/p1. + - s87-t215-sut1-c2/p2 to s88-t215-tg1-c2/p2. + - ring2 25GE-ports e810-XXVDA4-2p25GE: + - s88-t215-tg1-c4/p1 to s87-t215-sut1-c4/p1. + - s87-t215-sut1-c4/p2 to s88-t215-tg1-c4/p2. + - s88-t215-tg1-c4/p3 to s87-t215-sut1-c4/p3. + - s87-t215-sut1-c4/p4 to s88-t215-tg1-c4/p4. + - ring3 100GE-ports e810-2CQDA2-2p100GE: + - s88-t215-tg1-c9/p1 to s87-t215-sut1-c9/p1. + - s87-t215-sut1-c9/p2 to s88-t215-tg1-c9/p2. + - ring4 100GE-ports e810-2CQDA2-2p100GE: + - s88-t215-tg1-c6/p1 to s88-t215-tg1-c6/p2. + - s88-t215-tg1-c6/p2 to s88-t215-tg1-c6/p1. + - ring5 200GE-ports ConnectX7-2p200GE: + - s88-t215-tg1-c8/p1 to s87-t215-sut1-c8/p1. + - s87-t215-sut1-c8/p2 to s88-t215-tg1-c8/p2. +``` + +### 3-Node-Rangeley (3n-rng) + +``` +To be completed. +``` + +### 3-Node-Taishan (3n-tsh) + +``` +- testbed33: + - ring1 10GE-ports x520-2p10GE on SUTs: + - s19-t33t211-tg1-c2/p2 - s17-t33-sut1-c6/p2. + - s17-t33-sut1-c6/p1 - s18-t33-sut2-c6/p2. + - s18-t33-sut2-c6/p1 - s19-t33t211-tg1-c2/p1. + - ring2 25GE-ports cx4-2p25GE on SUTs: + - s19-t33t211-tg1-c4/p2 - s17-t33-sut1-c4/p2. + - s17-t33-sut1-c4/p1 - s18-t33-sut2-c4/p2. + - s18-t33-sut2-c4/p1 - s19-t33t211-tg1-c4/p1. +``` + +### 3-Node-Altra (3n-alt) + +``` +- testbed34: + - ring1 40GE-ports xl710-QDA2-2p40GE on SUTs: + - s64-t34-tg1-c4/p1 - s62-t34-sut1-c1/p2. + - s62-t34-sut1-c1/p1 - s63-t34-sut2-c1/p2. + - s63-t34-sut2-c1/p1 - s64-t34-tg1-c4/p2. + - ring2 100GE-ports ConnectX6-2p100GE Mellanox on SUTs: + - s64-t34-tg1-c8/p1 - s62-t34-sut2-c8/p1. + - s62-t34-sut1-c8/p1 - s63-t34-sut2-c8/p2. + - s63-t34-sut1-c8/p2 - s64-t34-tg1-c8/p2. +``` + +### 3-Node-Icelake (3n-icx) + +``` +- testbed37: + - ring1 25GE-ports xxv710-DA2-2p25GE: + - s67-t37-tg1-c2/p1 to s65-t37-sut1-c2/p1. + - s65-t37-sut1-c2/p2 to s66-t37-sut2-c2/p2. + - s66-t37-sut2-c2/p1 to s67-t37-tg1-c2/p2. + - ring2 25GE-ports e810-XXVDA4-4p25GE: + - s67-t37-tg1-c4/p1 to s65-t37-sut1-c4/p1. + - s65-t37-sut1-c4/p2 to s66-t37-sut2-c4/p2. + - s66-t37-sut2-c4/p1 to s67-t37-tg1-c4/p2. + - s67-t37-tg1-c4/p3 to s65-t37-sut1-c4/p3. + - s65-t37-sut1-c4/p4 to s66-t37-sut2-c4/p4. + - s66-t37-sut2-c4/p3 to s67-t37-tg1-c4/p4. + - ring3 100GE-ports e810-2CQDA2-2p100GE: + - s67-t37-tg1-c9/p1 to s65-t37-sut1-c9/p1. + - s65-t37-sut1-c9/p2 to s66-t37-sut2-c9/p2. + - s66-t37-sut2-c9/p1 to s67-t37-tg1-c9/p2. + - ring4 200GE-ports ConnectX6-2p200GE: + - s67-t37-tg1-c10/p1 - s65-t37-sut1-c10/p2. + - s65-t37-sut1-c10/p1 - s66-t37-sut2-c10/p2. + - s66-t37-sut2-c10/p1 - s67-t37-tg1-c10/p2. +- testbed38: + - ring1 25GE-ports xxv710-DA2-2p25GE: + - s80-t38-tg1-c2/p1 to s78-t38-sut1-c2/p1. + - s78-t38-sut1-c2/p2 to s79-t38-sut2-c2/p2. + - s79-t38-sut2-c2/p1 to s80-t38-tg1-c2/p2. + - ring2 25GE-ports e810-XXVDA4-4p25GE: + - s80-t38-tg1-c4/p1 to s78-t38-sut1-c4/p1. + - s78-t38-sut1-c4/p2 to s79-t38-sut2-c4/p2. + - s79-t38-sut2-c4/p1 to s80-t38-tg1-c4/p2. + - s80-t38-tg1-c4/p3 to s78-t38-sut1-c4/p3. + - s78-t38-sut1-c4/p4 to s79-t38-sut2-c4/p4. + - s79-t38-sut2-c4/p3 to s80-t38-tg1-c4/p4. + - ring3 100GE-ports e810-2CQDA2-2p100GE: + - s80-t38-tg1-c9/p1 to s78-t38-sut1-c9/p1. + - s78-t38-sut1-c9/p2 to s79-t38-sut2-c9/p2. + - s79-t38-sut2-c9/p1 to s80-t38-tg1-c9/p2. + - ring4 200GE-ports ConnectX6-2p200GE: + - s80-t38-tg1-c10/p1 to s78-t38-sut1-c10/p1. + - s78-t38-sut1-c10/p2 to s79-t38-sut2-c10/p2. + - s79-t38-sut2-c10/p1 to s80-t38-tg1-c10/p2. +``` + +### 3-Node-SnowRidge (3n-snr) + +``` +- testbed39: + - ring1 25GE-ports e810-XXVDA4-4p25GE: + - s89-t39t310-tg1-c6/p1 to s93-t39-sut1-c1/p1. + - s93-t39-sut1-c1/p2 to s94-t39-sut2-c1/p2. + - s94-t39-sut2-c1/p1 to s89-t39t310-tg1-c6/p2. + - s89-t39t310-tg1-c6/p3 to s93-t39-sut1-c1/p3. + - s93-t39-sut1-c1/p4 to s94-t39-sut2-c1/p4. + - s94-t39-sut2-c1/p3 to s89-t39t310-tg1-c6/p4. +``` + +### 2-Node-SapphireRapids (2n-spr) + +``` +- testbed21: + - ring1 200GE-ports ConnectX7-2p200GE: + - s53-t21-tg1-c2/p1 to s52-t21-sut1-c2/p1 + - s53-t21-tg1-c7/p1 to s52-t21-sut1-c7/p1 + - s52-t21-sut1-c4/p2 to s52-t21-sut1-c9/p2 + - ring2 200GE-ports ConnectX7-2p200GE: + - s53-t21-tg1-c2/p2 to s52-t21-sut1-c2/p2 + - s53-t21-tg1-c7/p2 to s52-t21-sut1-c7/p2 + - s52-t21-sut1-c10/p1 to s52-t21-sut1-c11/p1 + - ring3 200GE-ports ConnectX7-2p200GE: + - s53-t21-tg1-c4/p1 to s52-t21-sut1-c4/p1 + - s53-t21-tg1-c9/p1 to s52-t21-sut1-c9/p1 + - s52-t21-sut1-c10/p2 to s52-t21-sut1-c11/p2 +- testbed22: + - ring1 100GE-ports e810-2CQDA2-2p100GE: + - s55-t22-tg1-c4/p1 to s54-t22-sut1-c9/p2 + - s55-t22-tg1-c4/p2 to s54-t22-sut1-c4/p2 + - s54-t22-sut1-c9/p1 to s54-t22-sut1-c4/p1 + - ring2 25GE-ports e810-XXVDA4-4p25GE: + - s55-t22-tg1-c2/p1 to s54-t22-sut1-c2/p1 + - s55-t22-tg1-c2/p2 to s54-t22-sut1-c7/p1 + - s54-t22-sut1-c2/p2 to s54-t22-sut1-c7/p2 +- testbed23: + - ring1 200GE-ports ConnectX7-2p200GE: + - s56-t23-sut1-c2/p1 to s57-t23-tg1-c2/p1. + - s57-t23-tg1-c2/p2 to s56-t23-sut1-c2/p2. + - ring2 100GE-ports e810-2CQDA2-2p100GE: + - s56-t23-sut1-c4/p1 to s57-t23-tg1-c4/p1. + - s57-t23-tg1-c4/p2 to s56-t23-sut1-c4/p2. + - ring3 25GE-ports e810-XXVDA4-4p25GE: + - s56-t23-sut1-c10/p1 to s57-t23-tg1-c10/p1. + - s56-t23-sut1-c10/p2 to s57-t23-tg1-c10/p2. + - s56-t23-sut1-c10/p3 to s57-t23-tg1-c10/p3. + - s56-t23-sut1-c10/p4 to s57-t23-tg1-c10/p4. + - ring4 200GE-ports ConnectX7-2p200GE: + - s57-t23-tg1-c7/p1 to s57-t23-tg1-c7/p2. + - ring5 100GE-ports e810-2CQDA2-2p100GE: + - s57-t23-tg1-c9/p1 to s57-t23-tg1-c9/p2. +- testbed24: + - ring1 200GE-ports ConnectX7-2p200GE: + - s58-t24-sut1-c2/p1 to s59-t24-tg1-c2/p1. + - s59-t24-tg1-c2/p2 to s58-t24-sut1-c2/p2. + - ring2 100GE-ports e810-2CQDA2-2p100GE: + - s58-t24-sut1-c4/p1 to s59-t24-tg1-c4/p1. + - s59-t24-tg1-c4/p2 to s58-t24-sut1-c4/p2. + - ring3 25GE-ports e810-XXVDA4-4p25GE: + - s58-t24-sut1-c10/p1 to s59-t24-tg1-c10/p1. + - s58-t24-sut1-c10/p2 to s59-t24-tg1-c10/p2. + - s58-t24-sut1-c10/p3 to s59-t24-tg1-c10/p3. + - s58-t24-sut1-c10/p4 to s59-t24-tg1-c10/p4. + - ring4 200GE-ports ConnectX7-2p200GE: + - s59-t24-tg1-c7/p1 to s59-t24-tg1-c7/p2. + - ring5 100GE-ports e810-2CQDA2-2p100GE: + - s59-t24-tg1-c9/p1 to s59-t24-tg1-c9/p2. +``` + +### 3-Node-IcelakeD (3n-icxd) + +``` +- testbed31: + - ring1 25GE-ports e822cq-2p25GE: + - s90-t31t32-tg1-c4/p1 to s32-t31-sut1-c1/p1. + - s32-t31-sut1-c1/p2 to s33-t31-sut2-c1/p2. + - s33-t31-sut2-c1/p1 to s90-t31t32-tg1-c4/p2. +- testbed32: + - ring1 25GE-ports e822cq-2p25GE: + - s90-t31t32-tg1-c6/p1 to s34-t32-sut1-c1/p1. + - s34-t32-sut1-c1/p2 to s35-t32-sut2-c1/p2. + - s35-t32-sut2-c1/p1 to s90-t31t32-tg1-c6/p2. +```
\ No newline at end of file diff --git a/docs/content/infrastructure/fdio_dc_vexxhost_inventory.md b/docs/content/infrastructure/fdio_dc_vexxhost_inventory.md new file mode 100644 index 0000000000..b4741cf5d2 --- /dev/null +++ b/docs/content/infrastructure/fdio_dc_vexxhost_inventory.md @@ -0,0 +1,126 @@ +--- +title: "FD.io DC Vexxhost Inventory" +weight: 1 +--- + +# FD.io DC Vexxhost Inventory + +Captured inventory data: + - **name**: CSIT functional server name as tracked in + [CSIT testbed specification]({{< ref "fdio_dc_testbed_specifications#FD.io CSIT Testbed Specifications" >}}), + followed by "/" and the actual configured hostname, unless it is the same + as CSIT name. + - **role**: 2n/3n-xxx performance testbed, nomad-client, nomad-server. + - role exceptions: decommission, repurpose, spare. + - **model**: server model. + - **s/n**: serial number. + - **mgmt-ip4**: current management IPv4 address on management VLAN. + - **ipmi-ip4**: current IPMI IPv4 address on LOM VLAN. + - **rackid**: new location rack id. + - **rackunit**: new location rack unit id. + +## Missing Equipment Inventory + +1. Ixia PerfectStorm One Appliance + - [**Specification**]({{< ref "fdio_dc_testbed_specifications#2-node-ixiaps1l47-ixia-psone-l47-2n-ps1" >}}) + - [**Wiring**]({{< ref "fdio_dc_testbed_specifications#2-node-ixiaps1l47-2n-ps1" >}}) + - **mgmt-ip4**: 10.30.51.62 s26-t25-tg1 + - **ipmi-ip4**: 10.30.50.59 s26-t25-tg1 + +## YUL1 Inventory + +### Rack YUL1-8 (3016.8) + + **name** | **role** | **model** | **s/n** | **mgmt-ip4** | **ipmi-ip4** | **rackid** | **rackunit** +-----------------|---------------|---------------------|-----------------|--------------|--------------|------------|-------------- + mtl1-8-lb4m | uplink | ? | ? | ? | ? | 3016.8 | u47 + s65-t37-sut1 | 3n-icx | SYS-740GP-TNRT | C7470KK25P50098 | 10.30.51.75 | 10.30.50.75 | 3016.8 | u42-u45 + s66-t37-sut2 | 3n-icx | SYS-740GP-TNRT | C7470KK33P50247 | 10.30.51.76 | 10.30.50.76 | 3016.8 | u38-u41 + s67-t37-tg1 | 3n-icx | SYS-740GP-TNRT | C7470KK25P50076 | 10.30.51.77 | 10.30.50.77 | 3016.8 | u34-u37 + s71-t212-sut1 | 2n-icx | SYS-740GP-TNRT | C7470KK25P50173 | 10.30.51.81 | 10.30.50.81 | 3016.8 | u30-u33 + s72-t212-tg1 | 2n-icx | SYS-740GP-TNRT | C7470KK33P50220 | 10.30.51.82 | 10.30.50.82 | 3016.8 | u26-u29 + s83-t213-sut1 | 2n-icx | SYS-740GP-TNRT | C7470KL07P50300 | 10.30.51.83 | 10.30.50.83 | 3016.8 | u22-u25 + s84-t213-tg1 | 2n-icx | SYS-740GP-TNRT | C7470KL03P50187 | 10.30.51.84 | 10.30.50.84 | 3016.8 | u18-u21 + s85-t214-sut1 | 2n-icx | SYS-740GP-TNRT | C7470KK33P50219 | 10.30.51.85 | 10.30.50.85 | 3016.8 | u14-u17 + s86-t214-tg1 | 2n-icx | SYS-740GP-TNRT | C7470KL07P50312 | 10.30.51.86 | 10.30.50.86 | 3016.8 | u10-u13 + s87-t215-sut1 | 2n-icx | SYS-740GP-TNRT | C7470KL03P50171 | 10.30.51.87 | 10.30.50.87 | 3016.8 | u6-u9 + s88-t215-tg1 | 2n-icx | SYS-740GP-TNRT | C7470KL07P50301 | 10.30.51.88 | 10.30.50.88 | 3016.8 | u2-u5 + +### Rack YUL1-9 (3016.9) + + **name** | **role** | **model** | **s/n** | **mgmt-ip4** | **ipmi-ip4** | **rackid** | **rackunit** +-----------------|---------------|---------------------|-----------------|--------------|--------------|------------|-------------- + mtl1-5-lb4m | uplink | ? | ? | ? | ? | 3016.9 | u47 + s52-t21-sut1 | 2n-spr | SYS-741GE-TNRT | C7490FL36A40118 | 10.30.51.52 | 10.30.50.52 | 3016.9 | u42-u45 + s53-t21-tg1 | 2n-spr | SYS-741GE-TNRT | ??? | 10.30.51.53 | 10.30.50.53 | 3016.9 | u38-u41 + s54-t22-sut1 | 2n-spr | SYS-741GE-TNRT | ??? | 10.30.51.54 | 10.30.50.54 | 3016.9 | u34-u37 + s55-t22-tg1 | 2n-spr | SYS-741GE-TNRT | ??? | 10.30.51.55 | 10.30.50.55 | 3016.9 | u30-u33 + s56-t23-sut1 | 2n-spr | SYS-741GE-TNRT | ??? | 10.30.51.56 | 10.30.50.56 | 3016.9 | u26-u29 + s57-t23-tg1 | 2n-spr | SYS-741GE-TNRT | ??? | 10.30.51.57 | 10.30.50.57 | 3016.9 | u22-u25 + s25-t25-sut1 | 2n-p1 | SYS-7049GP-TRT | C7470KH06A20022 | 10.30.51.61 | 10.30.50.58 | 3016.9 | u18-u21 + s19-t33t211-tg1 | 3n-tsh/2n-tx2 | SYS-7049GP-TRT | C7470KH06A20056 | 10.30.51.49 | 10.30.50.46 | 3016.9 | u14-u17 + s27-t211-sut1 | 2n-tx2 | ThunderX2-9975 | K61186073100003 | 10.30.51.69 | 10.30.50.69 | 3016.9 | u13 + s18-t33-sut2 | 3n-tsh | HUAWEI-TAISHAN-2280 | N/A | 10.30.51.37 | 10.30.50.37 | 3016.9 | u11-u12 + s17-t33-sut1 | 3n-tsh | HUAWEI-TAISHAN-2280 | N/A | 10.30.51.36 | 10.30.50.36 | 3016.9 | u9-u10 + +### Rack YUL1-10 (3016.10) + + **name** | **role** | **model** | **s/n** | **mgmt-ip4** | **ipmi-ip4** | **rackid** | **rackunit** +-----------------|---------------|---------------------|-----------------|--------------|--------------|------------|-------------- + yul1-10-lb4m | uplink | ? | ? | ? | ? | 3016.10 | u47 + s2-t12-sut1 | 1n-skx | SYS-7049GP-TRT | C7470KH06A20119 | 10.30.51.51 | 10.30.50.48 | 3016.10 | u42-u45 + s1-t11-sut1 | 1n-skx | SYS-7049GP-TRT | C7470KH06A20154 | 10.30.51.50 | 10.30.50.47 | 3016.10 | u38-u41 + s58-t24-sut1 | 2n-spr | SYS-741GE-TNRT | ??? | 10.30.51.58 | 10.30.50.58 | 3016.10 | u34-u37 + s59-t24-tg1 | 2n-spr | SYS-741GE-TNRT | ??? | 10.30.51.59 | 10.30.50.59 | 3016.10 | u30-u33 + s32-t31-sut1 | 3n-icxd | SYS-110D-20C-FRDN8TP| C515MKK41A30950 | 10.30.51.32 | 10.30.50.32 | 3016.10 | u21 + s33-t31-sut2 | 3n-icxd | SYS-110D-20C-FRDN8TP| C515MKK41A30967 | 10.30.51.33 | 10.30.50.33 | 3016.10 | u20 + s34-t32-sut1 | 3n-icxd | SYS-110D-20C-FRDN8TP| C515MKK41A30959 | 10.30.51.34 | 10.30.50.34 | 3016.10 | u19 + s35-t32-sut2 | 3n-icxd | SYS-110D-20C-FRDN8TP| C515MKK41A30886 | 10.30.51.35 | 10.30.50.35 | 3016.10 | u18 + s90-t31t32-tg1 | 3n-icxd | SYS-740GP-TNRT | C7470KL03P50184 | 10.30.51.90 | 10.30.50.90 | 3016.10 | u14-u17 + s93-t39-sut1 | 3n-snr | ? | ? | 10.30.51.93 | 10.30.50.93 | 3016.10 | u10-u13 + s94-t39-sut2 | 3n-snr | ? | ? | 10.30.51.94 | 10.30.50.94 | 3016.10 | u6-u9 + s89-t39t310-tg1 | 3n-snr | ? | ? | 10.30.51.89 | 10.30.50.89 | 3016.10 | u2-u5 + + +### Rack YUL1-11 (3016.11) + + **name** | **role** | **model** | **s/n** | **mgmt-ip4** | **ipmi-ip4** | **rackid** | **rackunit** +-----------------------|--------------|----------------|-----------------|--------------|--------------|------------|-------------- + yul1-11-lb6m | arm-uplink | ? | ? | ? | ? | 3016.11 | u48 + yul1-11-lf-tor-switch | uplink | ? | ? | ? | ? | 3016.11 | u47 + mtl1-6-7050QX-32 | uplink | ? | ? | ? | ? | 3016.11 | u46 + fdio-marvell-dev | dev | ThunderX-88XX | N/A | 10.30.51.38 | 10.30.50.38 | 3016.11 | u45 + s21-nomad | nomad-client | SYS-741GE-TNRT | C7490FL47A50150 | 10.30.51.21 | 10.30.50.21 | 3016.11 | u39-u42 + s22-nomad | nomad-client | SYS-741GE-TNRT | C7490FL47A50155 | 10.30.51.22 | 10.30.50.22 | 3016.11 | u35-u38 + s78-t38-sut1 | 3n-icx | SYS-740GP-TNRT | C7470KL03P50450 | 10.30.51.78 | 10.30.50.78 | 3016.11 | u31-u34 + s79-t38-sut2 | 3n-icx | SYS-740GP-TNRT | C7470KL07P50297 | 10.30.51.79 | 10.30.50.79 | 3016.11 | u27-u30 + s80-t38-tg1 | 3n-icx | SYS-740GP-TNRT | C7470KL03P50454 | 10.30.51.80 | 10.30.50.80 | 3016.11 | u23-u26 + s30-t15-sut1 | nomad-client | SYS-741GE-TNRT | C7490FL47A50154 | 10.30.51.30 | 10.30.50.30 | 3016.11 | u19-u22 + s31-t16-sut1 | nomad-client | SYS-741GE-TNRT | C7490FL47A50149 | 10.30.51.31 | 10.30.50.31 | 3016.11 | u15-u18 + s70-t13-sut1 | 1n-alt | E252-P30-00 | GMG252012A0098 | 10.30.51.70 | 10.30.50.70 | 3016.11 | u13-u14 + s71-t14-sut1 | 1n-alt | E252-P30-00 | GMG252012A0089 | 10.30.51.71 | 10.30.50.71 | 3016.11 | u11-u12 + s62-t34-sut1 | 3n-alt | WIWYNN | 04000059N0SC | 10.30.51.72 | 10.30.50.72 | 3016.11 | u9-u10 + s63-t34-sut2 | 3n-alt | WIWYNN | 0390003EN0SC | 10.30.51.73 | 10.30.50.73 | 3016.11 | u7-u8 + s64-t34-tg1 | 3n-alt | SYS-740GP-TNRT | C7470KK40P50249 | 10.30.51.74 | 10.30.50.74 | 3016.11 | u3-u6 + +### Rack YUL1-12 (3016.12) + + **name** | **role** | **model** | **s/n** | **mgmt-ip4** | **ipmi-ip4** | **rackid** | **rackunit** +-----------------|---------------|---------------------|-----------------|--------------|--------------|------------|-------------- + yul1-12-lb4m | uplink | ? | ? | ? | ? | 3016.12 | u47 + s28-nomad | nomad-client | SYS-7049GP-TRT | C7470KH06A20196 | 10.30.51.28 | 10.30.50.28 | 3016.12 | u41-u44 + s27-nomad | nomad-client | SYS-7049GP-TRT | C7470KH06A20055 | 10.30.51.27 | 10.30.50.27 | 3016.12 | u37-u40 + s91-nomad | nomad-client | R152-P30-00 | GLG4P9912A0016 | 10.30.51.91 | 10.30.50.91 | 3016.12 | u36 + s92-nomad | nomad-client | R152-P30-00 | GLG4P9912A0004 | 10.30.51.92 | 10.30.50.92 | 3016.12 | u35 + s23-nomad | nomad-server | SYS-1029P-WTRT | C1160LI12NM0256 | 10.30.51.23 | 10.30.51.23 | 3016.12 | u34 + s24-nomad | nomad-server | SYS-1029P-WTRT | C1160LI12NM0241 | 10.30.51.24 | 10.30.51.24 | 3016.12 | u33 + s25-nomad | nomad-server | SYS-1029P-WTRT | C1160LI12NM0540 | 10.30.51.25 | 10.30.51.25 | 3016.12 | u32 + s61-t210-tg1 | 2n-zn2 | AS-1014S-WTRT | C8150LI50NS2689 | 10.32.8.25 | 10.30.55.25 | 3016.12 | u31 + s60-t210-sut1 | 2n-zn2 | AS-1114S-WTRT | N/A | 10.32.8.24 | 10.30.55.24 | 3016.12 | u30 + s26-nomad | nomad-server | SYS-7049GP-TRT | C7470KH37A30505 | 10.30.51.26 | 10.30.51.26 | 3016.12 | u26-u29 + s33-t27-sut1 | 2n-clx | SYS-7049GP-TRT | C7470KH37A30567 | 10.32.8.18 | 10.30.55.18 | 3016.12 | u22-u25 + s34-t27-tg1 | 2n-clx | SYS-7049GP-TRT | C7470KH37A30565 | 10.32.8.19 | 10.30.55.19 | 3016.12 | u18-u21 + s35-t28-sut1 | 2n-clx | SYS-7049GP-TRT | C7470KH37A30509 | 10.32.8.20 | 10.30.55.20 | 3016.12 | u14-u17 + s36-t28-tg1 | 2n-clx | SYS-7049GP-TRT | C7470KH37A30511 | 10.32.8.21 | 10.30.55.21 | 3016.12 | u10-u13 + s37-t29-sut1 | 2n-clx | SYS-7049GP-TRT | C7470KH37A30566 | 10.32.8.22 | 10.30.55.22 | 3016.12 | u6-u9 + s38-t29-tg1 | 2n-clx | SYS-7049GP-TRT | C7470KH37A30506 | 10.32.8.23 | 10.30.55.23 | 3016.12 | u2-u5
\ No newline at end of file diff --git a/docs/content/infrastructure/testbed_configuration/_index.md b/docs/content/infrastructure/testbed_configuration/_index.md new file mode 100644 index 0000000000..79d0250474 --- /dev/null +++ b/docs/content/infrastructure/testbed_configuration/_index.md @@ -0,0 +1,6 @@ +--- +bookCollapseSection: true +bookFlatSection: false +title: "FD.io DC Testbed Configuration" +weight: 3 +---
\ No newline at end of file diff --git a/docs/content/infrastructure/testbed_configuration/ami_alt_hw_bios_cfg.md b/docs/content/infrastructure/testbed_configuration/ami_alt_hw_bios_cfg.md new file mode 100644 index 0000000000..d36d075e12 --- /dev/null +++ b/docs/content/infrastructure/testbed_configuration/ami_alt_hw_bios_cfg.md @@ -0,0 +1,1264 @@ +--- +bookToc: true +title: "MegaRac Altra" +--- + +# MegaRac Altra + +## Linux lscpu + +``` +Architecture: aarch64 + CPU op-mode(s): 32-bit, 64-bit + Byte Order: Little Endian +CPU(s): 160 + On-line CPU(s) list: 0-159 +Vendor ID: ARM + Model name: Neoverse-N1 + Model: 1 + Thread(s) per core: 1 + Core(s) per socket: 80 + Socket(s): 2 + Stepping: r3p1 + Frequency boost: disabled + CPU max MHz: 3000.0000 + CPU min MHz: 1000.0000 + BogoMIPS: 50.00 + Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm lrcpc dcpop asimddp ssbs +Caches (sum of all): + L1d: 10 MiB (160 instances) + L1i: 10 MiB (160 instances) + L2: 160 MiB (160 instances) +NUMA: + NUMA node(s): 2 + NUMA node0 CPU(s): 0-79 + NUMA node1 CPU(s): 80-159 +Vulnerabilities: + Itlb multihit: Not affected + L1tf: Not affected + Mds: Not affected + Meltdown: Not affected + Mmio stale data: Not affected + Retbleed: Not affected + Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl + Spectre v1: Mitigation; __user pointer sanitization + Spectre v2: Mitigation; CSV2, BHB + Srbds: Not affected + Tsx async abort: Not affected +``` + +## Linux dmidecode + +``` +# dmidecode 3.3 +Getting SMBIOS data from sysfs. +SMBIOS 3.3.0 present. +Table at 0xB1E10000. + +Handle 0x0000, DMI type 0, 26 bytes +BIOS Information + Vendor: Ampere(R) + Version: 1.07.20210713 (SCP: 1.07.20210713) + Release Date: 2021/07/13 + ROM Size: 7680 kB + Characteristics: + PCI is supported + BIOS is upgradeable + Boot from CD is supported + Selectable boot is supported + ACPI is supported + UEFI is supported + BIOS Revision: 5.15 + Firmware Revision: 1.7 + +Handle 0x0001, DMI type 1, 27 bytes +System Information + Manufacturer: WIWYNN + Product Name: Mt.Jade Server System B81.030Z1.0007 + Version: DVT + Serial Number: B81030Z1000704000059N0SC + UUID: 57c97bbe-008e-368f-19d0-595df92c6de0 + Wake-up Type: Power Switch + SKU Number: NULL + Family: Altra + +Handle 0x0002, DMI type 2, 15 bytes +Base Board Information + Manufacturer: WIWYNN + Product Name: Mt.Jade Motherboard + Version: B81.03010.0033 + Serial Number: B8103010003303800033J0SA + Asset Tag: NULL + Features: + Board is a hosting board + Location In Chassis: Part Component + Chassis Handle: 0x0003 + Type: Motherboard + Contained Object Handles: 0 + +Handle 0x0003, DMI type 3, 22 bytes +Chassis Information + Manufacturer: WIWYNN + Type: Rack Mount Chassis + Lock: Present + Version: B60.03008.0001 + Serial Number: 04000059N0SC + Asset Tag: NULL + Boot-up State: Safe + Power Supply State: Safe + Thermal State: Safe + Security Status: None + OEM Information: 0x00000000 + Height: Unspecified + Number Of Power Cords: 1 + Contained Elements: 0 + SKU Number: Unknown +``` + +## Linux dmidecode memory + +``` +Handle 0x0020, DMI type 16, 23 bytes +Physical Memory Array + Location: System Board Or Motherboard + Use: System Memory + Error Correction Type: Multi-bit ECC + Maximum Capacity: 4 TB + Error Information Handle: No Error + Number Of Devices: 16 + +Handle 0x0026, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0020 + Error Information Handle: No Error + Total Width: 72 bits + Data Width: 64 bits + Size: 8 GB + Form Factor: DIMM + Set: None + Locator: DIMM 1 + Bank Locator: Bank 1 + Type: DDR4 + Type Detail: Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: 0197198A + Asset Tag: Array 1 Asset Tag 1 + Part Number: M393A1K43DB2-CWE + Rank: 1 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.14 V + Maximum Voltage: 1.26 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: None + Firmware Version: Not Specified + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 8 GB + Cache Size: None + Logical Size: None + +Handle 0x0028, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0020 + Error Information Handle: 0x0029 + Total Width: Unknown + Data Width: Unknown + Size: No Module Installed + Form Factor: DIMM + Set: None + Locator: DIMM 2 + Bank Locator: Bank 2 + Type: Unknown + Type Detail: Unknown + Speed: Unknown + Manufacturer: Array 1 Manufacturer 2 + Serial Number: Array 1 Serial Number 2 + Asset Tag: Array 1 Asset Tag 2 + Part Number: Array 1 Part Number 2 + Rank: Unknown + Configured Memory Speed: Unknown + Minimum Voltage: Unknown + Maximum Voltage: Unknown + Configured Voltage: Unknown + Memory Technology: Unknown + Memory Operating Mode Capability: Unknown + Firmware Version: Not Specified + Module Manufacturer ID: Unknown + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: None + Cache Size: None + Logical Size: None + +Handle 0x0030, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0020 + Error Information Handle: No Error + Total Width: 72 bits + Data Width: 64 bits + Size: 8 GB + Form Factor: DIMM + Set: None + Locator: DIMM 3 + Bank Locator: Bank 3 + Type: DDR4 + Type Detail: Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: 01971992 + Asset Tag: Array 1 Asset Tag 3 + Part Number: M393A1K43DB2-CWE + Rank: 1 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.14 V + Maximum Voltage: 1.26 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: None + Firmware Version: Not Specified + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 8 GB + Cache Size: None + Logical Size: None + +Handle 0x0032, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0020 + Error Information Handle: 0x0033 + Total Width: Unknown + Data Width: Unknown + Size: No Module Installed + Form Factor: DIMM + Set: None + Locator: DIMM 4 + Bank Locator: Bank 4 + Type: Unknown + Type Detail: Unknown + Speed: Unknown + Manufacturer: Array 1 Manufacturer 4 + Serial Number: Array 1 Serial Number 4 + Asset Tag: Array 1 Asset Tag 4 + Part Number: Array 1 Part Number 4 + Rank: Unknown + Configured Memory Speed: Unknown + Minimum Voltage: Unknown + Maximum Voltage: Unknown + Configured Voltage: Unknown + Memory Technology: Unknown + Memory Operating Mode Capability: Unknown + Firmware Version: Not Specified + Module Manufacturer ID: Unknown + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: None + Cache Size: None + Logical Size: None + +Handle 0x0034, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0020 + Error Information Handle: No Error + Total Width: 72 bits + Data Width: 64 bits + Size: 8 GB + Form Factor: DIMM + Set: None + Locator: DIMM 5 + Bank Locator: Bank 5 + Type: DDR4 + Type Detail: Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: 01971989 + Asset Tag: Array 1 Asset Tag 5 + Part Number: M393A1K43DB2-CWE + Rank: 1 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.14 V + Maximum Voltage: 1.26 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: None + Firmware Version: Not Specified + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 8 GB + Cache Size: None + Logical Size: None + +Handle 0x0036, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0020 + Error Information Handle: 0x0037 + Total Width: Unknown + Data Width: Unknown + Size: No Module Installed + Form Factor: DIMM + Set: None + Locator: DIMM 6 + Bank Locator: Bank 6 + Type: Unknown + Type Detail: Unknown + Speed: Unknown + Manufacturer: Array 1 Manufacturer 6 + Serial Number: Array 1 Serial Number 6 + Asset Tag: Array 1 Asset Tag 6 + Part Number: Array 1 Part Number 6 + Rank: Unknown + Configured Memory Speed: Unknown + Minimum Voltage: Unknown + Maximum Voltage: Unknown + Configured Voltage: Unknown + Memory Technology: Unknown + Memory Operating Mode Capability: Unknown + Firmware Version: Not Specified + Module Manufacturer ID: Unknown + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: None + Cache Size: None + Logical Size: None + +Handle 0x0038, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0020 + Error Information Handle: No Error + Total Width: 72 bits + Data Width: 64 bits + Size: 8 GB + Form Factor: DIMM + Set: None + Locator: DIMM 7 + Bank Locator: Bank 7 + Type: DDR4 + Type Detail: Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: 01971BA1 + Asset Tag: Array 1 Asset Tag 7 + Part Number: M393A1K43DB2-CWE + Rank: 1 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.14 V + Maximum Voltage: 1.26 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: None + Firmware Version: Not Specified + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 8 GB + Cache Size: None + Logical Size: None + +Handle 0x0040, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0020 + Error Information Handle: 0x0041 + Total Width: Unknown + Data Width: Unknown + Size: No Module Installed + Form Factor: DIMM + Set: None + Locator: DIMM 8 + Bank Locator: Bank 8 + Type: Unknown + Type Detail: Unknown + Speed: Unknown + Manufacturer: Array 1 Manufacturer 8 + Serial Number: Array 1 Serial Number 8 + Asset Tag: Array 1 Asset Tag 8 + Part Number: Array 1 Part Number 8 + Rank: Unknown + Configured Memory Speed: Unknown + Minimum Voltage: Unknown + Maximum Voltage: Unknown + Configured Voltage: Unknown + Memory Technology: Unknown + Memory Operating Mode Capability: Unknown + Firmware Version: Not Specified + Module Manufacturer ID: Unknown + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: None + Cache Size: None + Logical Size: None + +Handle 0x0042, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0020 + Error Information Handle: No Error + Total Width: 72 bits + Data Width: 64 bits + Size: 8 GB + Form Factor: DIMM + Set: None + Locator: DIMM 9 + Bank Locator: Bank 9 + Type: DDR4 + Type Detail: Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: 01971930 + Asset Tag: Array 1 Asset Tag 9 + Part Number: M393A1K43DB2-CWE + Rank: 1 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.14 V + Maximum Voltage: 1.26 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: None + Firmware Version: Not Specified + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 8 GB + Cache Size: None + Logical Size: None + +Handle 0x0044, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0020 + Error Information Handle: 0x0045 + Total Width: Unknown + Data Width: Unknown + Size: No Module Installed + Form Factor: DIMM + Set: None + Locator: DIMM 10 + Bank Locator: Bank 10 + Type: Unknown + Type Detail: Unknown + Speed: Unknown + Manufacturer: Array 1 Manufacturer 10 + Serial Number: Array 1 Serial Number 10 + Asset Tag: Array 1 Asset Tag 10 + Part Number: Array 1 Part Number 10 + Rank: Unknown + Configured Memory Speed: Unknown + Minimum Voltage: Unknown + Maximum Voltage: Unknown + Configured Voltage: Unknown + Memory Technology: Unknown + Memory Operating Mode Capability: Unknown + Firmware Version: Not Specified + Module Manufacturer ID: Unknown + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: None + Cache Size: None + Logical Size: None + +Handle 0x0046, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0020 + Error Information Handle: No Error + Total Width: 72 bits + Data Width: 64 bits + Size: 8 GB + Form Factor: DIMM + Set: None + Locator: DIMM 11 + Bank Locator: Bank 11 + Type: DDR4 + Type Detail: Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: 01971BA2 + Asset Tag: Array 1 Asset Tag 11 + Part Number: M393A1K43DB2-CWE + Rank: 1 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.14 V + Maximum Voltage: 1.26 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: None + Firmware Version: Not Specified + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 8 GB + Cache Size: None + Logical Size: None + +Handle 0x0048, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0020 + Error Information Handle: 0x0049 + Total Width: Unknown + Data Width: Unknown + Size: No Module Installed + Form Factor: DIMM + Set: None + Locator: DIMM 12 + Bank Locator: Bank 12 + Type: Unknown + Type Detail: Unknown + Speed: Unknown + Manufacturer: Array 1 Manufacturer 12 + Serial Number: Array 1 Serial Number 12 + Asset Tag: Array 1 Asset Tag 12 + Part Number: Array 1 Part Number 12 + Rank: Unknown + Configured Memory Speed: Unknown + Minimum Voltage: Unknown + Maximum Voltage: Unknown + Configured Voltage: Unknown + Memory Technology: Unknown + Memory Operating Mode Capability: Unknown + Firmware Version: Not Specified + Module Manufacturer ID: Unknown + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: None + Cache Size: None + Logical Size: None + +Handle 0x0050, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0020 + Error Information Handle: No Error + Total Width: 72 bits + Data Width: 64 bits + Size: 8 GB + Form Factor: DIMM + Set: None + Locator: DIMM 13 + Bank Locator: Bank 13 + Type: DDR4 + Type Detail: Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: 01971957 + Asset Tag: Array 1 Asset Tag 13 + Part Number: M393A1K43DB2-CWE + Rank: 1 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.14 V + Maximum Voltage: 1.26 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: None + Firmware Version: Not Specified + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 8 GB + Cache Size: None + Logical Size: None + +Handle 0x0052, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0020 + Error Information Handle: 0x0053 + Total Width: Unknown + Data Width: Unknown + Size: No Module Installed + Form Factor: DIMM + Set: None + Locator: DIMM 14 + Bank Locator: Bank 14 + Type: Unknown + Type Detail: Unknown + Speed: Unknown + Manufacturer: Array 1 Manufacturer 14 + Serial Number: Array 1 Serial Number 14 + Asset Tag: Array 1 Asset Tag 14 + Part Number: Array 1 Part Number 14 + Rank: Unknown + Configured Memory Speed: Unknown + Minimum Voltage: Unknown + Maximum Voltage: Unknown + Configured Voltage: Unknown + Memory Technology: Unknown + Memory Operating Mode Capability: Unknown + Firmware Version: Not Specified + Module Manufacturer ID: Unknown + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: None + Cache Size: None + Logical Size: None + +Handle 0x0054, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0020 + Error Information Handle: No Error + Total Width: 72 bits + Data Width: 64 bits + Size: 8 GB + Form Factor: DIMM + Set: None + Locator: DIMM 15 + Bank Locator: Bank 15 + Type: DDR4 + Type Detail: Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: 01971B9E + Asset Tag: Array 1 Asset Tag 15 + Part Number: M393A1K43DB2-CWE + Rank: 1 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.14 V + Maximum Voltage: 1.26 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: None + Firmware Version: Not Specified + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 8 GB + Cache Size: None + Logical Size: None + +Handle 0x0056, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0020 + Error Information Handle: 0x0057 + Total Width: Unknown + Data Width: Unknown + Size: No Module Installed + Form Factor: DIMM + Set: None + Locator: DIMM 16 + Bank Locator: Bank 16 + Type: Unknown + Type Detail: Unknown + Speed: Unknown + Manufacturer: Array 1 Manufacturer 16 + Serial Number: Array 1 Serial Number 16 + Asset Tag: Array 1 Asset Tag 16 + Part Number: Array 1 Part Number 16 + Rank: Unknown + Configured Memory Speed: Unknown + Minimum Voltage: Unknown + Maximum Voltage: Unknown + Configured Voltage: Unknown + Memory Technology: Unknown + Memory Operating Mode Capability: Unknown + Firmware Version: Not Specified + Module Manufacturer ID: Unknown + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: None + Cache Size: None + Logical Size: None + +Handle 0x0071, DMI type 16, 23 bytes +Physical Memory Array + Location: System Board Or Motherboard + Use: System Memory + Error Correction Type: Multi-bit ECC + Maximum Capacity: 4 TB + Error Information Handle: No Error + Number Of Devices: 16 + +Handle 0x0077, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0071 + Error Information Handle: No Error + Total Width: 72 bits + Data Width: 64 bits + Size: 8 GB + Form Factor: DIMM + Set: None + Locator: Socket 1 DIMM 1 + Bank Locator: Bank 1 + Type: DDR4 + Type Detail: Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: 01971918 + Asset Tag: Array 1 Asset Tag 1 + Part Number: M393A1K43DB2-CWE + Rank: 1 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.14 V + Maximum Voltage: 1.26 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: None + Firmware Version: Not Specified + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 8 GB + Cache Size: None + Logical Size: None + +Handle 0x007A, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0071 + Error Information Handle: 0x007C + Total Width: Unknown + Data Width: Unknown + Size: No Module Installed + Form Factor: DIMM + Set: None + Locator: Socket 1 DIMM 2 + Bank Locator: Bank 2 + Type: Unknown + Type Detail: Unknown + Speed: Unknown + Manufacturer: Array 1 Manufacturer 2 + Serial Number: Array 1 Serial Number 2 + Asset Tag: Array 1 Asset Tag 2 + Part Number: Array 1 Part Number 2 + Rank: Unknown + Configured Memory Speed: Unknown + Minimum Voltage: Unknown + Maximum Voltage: Unknown + Configured Voltage: Unknown + Memory Technology: Unknown + Memory Operating Mode Capability: Unknown + Firmware Version: Not Specified + Module Manufacturer ID: Unknown + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: None + Cache Size: None + Logical Size: None + +Handle 0x007D, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0071 + Error Information Handle: No Error + Total Width: 72 bits + Data Width: 64 bits + Size: 8 GB + Form Factor: DIMM + Set: None + Locator: Socket 1 DIMM 3 + Bank Locator: Bank 3 + Type: DDR4 + Type Detail: Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: 01971970 + Asset Tag: Array 1 Asset Tag 3 + Part Number: M393A1K43DB2-CWE + Rank: 1 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.14 V + Maximum Voltage: 1.26 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: None + Firmware Version: Not Specified + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 8 GB + Cache Size: None + Logical Size: None + +Handle 0x0080, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0071 + Error Information Handle: 0x0082 + Total Width: Unknown + Data Width: Unknown + Size: No Module Installed + Form Factor: DIMM + Set: None + Locator: Socket 1 DIMM 4 + Bank Locator: Bank 4 + Type: Unknown + Type Detail: Unknown + Speed: Unknown + Manufacturer: Array 1 Manufacturer 4 + Serial Number: Array 1 Serial Number 4 + Asset Tag: Array 1 Asset Tag 4 + Part Number: Array 1 Part Number 4 + Rank: Unknown + Configured Memory Speed: Unknown + Minimum Voltage: Unknown + Maximum Voltage: Unknown + Configured Voltage: Unknown + Memory Technology: Unknown + Memory Operating Mode Capability: Unknown + Firmware Version: Not Specified + Module Manufacturer ID: Unknown + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: None + Cache Size: None + Logical Size: None + +Handle 0x0083, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0071 + Error Information Handle: No Error + Total Width: 72 bits + Data Width: 64 bits + Size: 8 GB + Form Factor: DIMM + Set: None + Locator: Socket 1 DIMM 5 + Bank Locator: Bank 5 + Type: DDR4 + Type Detail: Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: 01971993 + Asset Tag: Array 1 Asset Tag 5 + Part Number: M393A1K43DB2-CWE + Rank: 1 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.14 V + Maximum Voltage: 1.26 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: None + Firmware Version: Not Specified + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 8 GB + Cache Size: None + Logical Size: None + +Handle 0x0086, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0071 + Error Information Handle: 0x0088 + Total Width: Unknown + Data Width: Unknown + Size: No Module Installed + Form Factor: DIMM + Set: None + Locator: Socket 1 DIMM 6 + Bank Locator: Bank 6 + Type: Unknown + Type Detail: Unknown + Speed: Unknown + Manufacturer: Array 1 Manufacturer 6 + Serial Number: Array 1 Serial Number 6 + Asset Tag: Array 1 Asset Tag 6 + Part Number: Array 1 Part Number 6 + Rank: Unknown + Configured Memory Speed: Unknown + Minimum Voltage: Unknown + Maximum Voltage: Unknown + Configured Voltage: Unknown + Memory Technology: Unknown + Memory Operating Mode Capability: Unknown + Firmware Version: Not Specified + Module Manufacturer ID: Unknown + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: None + Cache Size: None + Logical Size: None + +Handle 0x0089, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0071 + Error Information Handle: No Error + Total Width: 72 bits + Data Width: 64 bits + Size: 8 GB + Form Factor: DIMM + Set: None + Locator: Socket 1 DIMM 7 + Bank Locator: Bank 7 + Type: DDR4 + Type Detail: Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: 01971985 + Asset Tag: Array 1 Asset Tag 7 + Part Number: M393A1K43DB2-CWE + Rank: 1 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.14 V + Maximum Voltage: 1.26 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: None + Firmware Version: Not Specified + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 8 GB + Cache Size: None + Logical Size: None + +Handle 0x008C, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0071 + Error Information Handle: 0x008E + Total Width: Unknown + Data Width: Unknown + Size: No Module Installed + Form Factor: DIMM + Set: None + Locator: Socket 1 DIMM 8 + Bank Locator: Bank 8 + Type: Unknown + Type Detail: Unknown + Speed: Unknown + Manufacturer: Array 1 Manufacturer 8 + Serial Number: Array 1 Serial Number 8 + Asset Tag: Array 1 Asset Tag 8 + Part Number: Array 1 Part Number 8 + Rank: Unknown + Configured Memory Speed: Unknown + Minimum Voltage: Unknown + Maximum Voltage: Unknown + Configured Voltage: Unknown + Memory Technology: Unknown + Memory Operating Mode Capability: Unknown + Firmware Version: Not Specified + Module Manufacturer ID: Unknown + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: None + Cache Size: None + Logical Size: None + +Handle 0x008F, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0071 + Error Information Handle: No Error + Total Width: 72 bits + Data Width: 64 bits + Size: 8 GB + Form Factor: DIMM + Set: None + Locator: Socket 1 DIMM 9 + Bank Locator: Bank 9 + Type: DDR4 + Type Detail: Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: 01971917 + Asset Tag: Array 1 Asset Tag 9 + Part Number: M393A1K43DB2-CWE + Rank: 1 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.14 V + Maximum Voltage: 1.26 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: None + Firmware Version: Not Specified + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 8 GB + Cache Size: None + Logical Size: None + +Handle 0x0092, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0071 + Error Information Handle: 0x0094 + Total Width: Unknown + Data Width: Unknown + Size: No Module Installed + Form Factor: DIMM + Set: None + Locator: Socket 1 DIMM 10 + Bank Locator: Bank 10 + Type: Unknown + Type Detail: Unknown + Speed: Unknown + Manufacturer: Array 1 Manufacturer 10 + Serial Number: Array 1 Serial Number 10 + Asset Tag: Array 1 Asset Tag 10 + Part Number: Array 1 Part Number 10 + Rank: Unknown + Configured Memory Speed: Unknown + Minimum Voltage: Unknown + Maximum Voltage: Unknown + Configured Voltage: Unknown + Memory Technology: Unknown + Memory Operating Mode Capability: Unknown + Firmware Version: Not Specified + Module Manufacturer ID: Unknown + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: None + Cache Size: None + Logical Size: None + +Handle 0x0095, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0071 + Error Information Handle: No Error + Total Width: 72 bits + Data Width: 64 bits + Size: 8 GB + Form Factor: DIMM + Set: None + Locator: Socket 1 DIMM 11 + Bank Locator: Bank 11 + Type: DDR4 + Type Detail: Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: 01971984 + Asset Tag: Array 1 Asset Tag 11 + Part Number: M393A1K43DB2-CWE + Rank: 1 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.14 V + Maximum Voltage: 1.26 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: None + Firmware Version: Not Specified + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 8 GB + Cache Size: None + Logical Size: None + +Handle 0x0098, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0071 + Error Information Handle: 0x009A + Total Width: Unknown + Data Width: Unknown + Size: No Module Installed + Form Factor: DIMM + Set: None + Locator: Socket 1 DIMM 12 + Bank Locator: Bank 12 + Type: Unknown + Type Detail: Unknown + Speed: Unknown + Manufacturer: Array 1 Manufacturer 12 + Serial Number: Array 1 Serial Number 12 + Asset Tag: Array 1 Asset Tag 12 + Part Number: Array 1 Part Number 12 + Rank: Unknown + Configured Memory Speed: Unknown + Minimum Voltage: Unknown + Maximum Voltage: Unknown + Configured Voltage: Unknown + Memory Technology: Unknown + Memory Operating Mode Capability: Unknown + Firmware Version: Not Specified + Module Manufacturer ID: Unknown + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: None + Cache Size: None + Logical Size: None + +Handle 0x009B, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0071 + Error Information Handle: No Error + Total Width: 72 bits + Data Width: 64 bits + Size: 8 GB + Form Factor: DIMM + Set: None + Locator: Socket 1 DIMM 13 + Bank Locator: Bank 13 + Type: DDR4 + Type Detail: Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: 01971B37 + Asset Tag: Array 1 Asset Tag 13 + Part Number: M393A1K43DB2-CWE + Rank: 1 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.14 V + Maximum Voltage: 1.26 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: None + Firmware Version: Not Specified + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 8 GB + Cache Size: None + Logical Size: None + +Handle 0x009E, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0071 + Error Information Handle: 0x00A0 + Total Width: Unknown + Data Width: Unknown + Size: No Module Installed + Form Factor: DIMM + Set: None + Locator: Socket 1 DIMM 14 + Bank Locator: Bank 14 + Type: Unknown + Type Detail: Unknown + Speed: Unknown + Manufacturer: Array 1 Manufacturer 14 + Serial Number: Array 1 Serial Number 14 + Asset Tag: Array 1 Asset Tag 14 + Part Number: Array 1 Part Number 14 + Rank: Unknown + Configured Memory Speed: Unknown + Minimum Voltage: Unknown + Maximum Voltage: Unknown + Configured Voltage: Unknown + Memory Technology: Unknown + Memory Operating Mode Capability: Unknown + Firmware Version: Not Specified + Module Manufacturer ID: Unknown + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: None + Cache Size: None + Logical Size: None + +Handle 0x00A1, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0071 + Error Information Handle: No Error + Total Width: 72 bits + Data Width: 64 bits + Size: 8 GB + Form Factor: DIMM + Set: None + Locator: Socket 1 DIMM 15 + Bank Locator: Bank 15 + Type: DDR4 + Type Detail: Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: 01971C99 + Asset Tag: Array 1 Asset Tag 15 + Part Number: M393A1K43DB2-CWE + Rank: 1 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.14 V + Maximum Voltage: 1.26 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: None + Firmware Version: Not Specified + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 8 GB + Cache Size: None + Logical Size: None + +Handle 0x00A4, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0071 + Error Information Handle: 0x00A6 + Total Width: Unknown + Data Width: Unknown + Size: No Module Installed + Form Factor: DIMM + Set: None + Locator: Socket 1 DIMM 16 + Bank Locator: Bank 16 + Type: Unknown + Type Detail: Unknown + Speed: Unknown + Manufacturer: Array 1 Manufacturer 16 + Serial Number: Array 1 Serial Number 16 + Asset Tag: Array 1 Asset Tag 16 + Part Number: Array 1 Part Number 16 + Rank: Unknown + Configured Memory Speed: Unknown + Minimum Voltage: Unknown + Maximum Voltage: Unknown + Configured Voltage: Unknown + Memory Technology: Unknown + Memory Operating Mode Capability: Unknown + Firmware Version: Not Specified + Module Manufacturer ID: Unknown + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: None + Cache Size: None + Logical Size: None +``` + +## Linux cmdline + +``` +BOOT_IMAGE=/boot/vmlinuz-5.15.0-46-generic root=UUID=7d1d0e77-4df0-43df-9619-a99db29ffb83 ro audit=0 default_hugepagesz=2M hugepagesz=1G hugepages=32 hugepagesz=2M hugepages=32768 iommu.passthrough=1 isolcpus=1-10,29-38 nmi_watchdog=0 nohz_full=1-10,29-38 nosoftlockup processor.max_cstate=1 rcu_nocbs=1-10,29-38 console=ttyAMA0,115200n8 quiet +``` diff --git a/docs/content/infrastructure/testbed_configuration/gigabyte_tx2_hw_bios_cfg.md b/docs/content/infrastructure/testbed_configuration/gigabyte_tx2_hw_bios_cfg.md new file mode 100644 index 0000000000..5020cb70f9 --- /dev/null +++ b/docs/content/infrastructure/testbed_configuration/gigabyte_tx2_hw_bios_cfg.md @@ -0,0 +1,406 @@ +--- +bookToc: true +title: "GigaByte ThunderX2" +--- + +# GigaByte ThunderX2 + +## Linux lscpu + +``` +Architecture: aarch64 + CPU op-mode(s): 64-bit + Byte Order: Little Endian +CPU(s): 56 + On-line CPU(s) list: 0-55 +Vendor ID: Cavium + Model name: ThunderX2 99xx + Model: 1 + Thread(s) per core: 1 + Core(s) per socket: 28 + Socket(s): 2 + Stepping: 0x1 + Frequency boost: disabled + CPU max MHz: 2000.0000 + CPU min MHz: 1000.0000 + BogoMIPS: 400.00 + Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics cpuid asimdrdm +Caches (sum of all): + L1d: 1.8 MiB (56 instances) + L1i: 1.8 MiB (56 instances) + L2: 14 MiB (56 instances) + L3: 64 MiB (2 instances) +NUMA: + NUMA node(s): 2 + NUMA node0 CPU(s): 0-27 + NUMA node1 CPU(s): 28-55 +Vulnerabilities: + Itlb multihit: Not affected + L1tf: Not affected + Mds: Not affected + Meltdown: Not affected + Mmio stale data: Not affected + Retbleed: Not affected + Spec store bypass: Vulnerable + Spectre v1: Mitigation; __user pointer sanitization + Spectre v2: Vulnerable + Srbds: Not affected + Tsx async abort: Not affected +``` + +## Linux dmidecode + +``` +# dmidecode 3.3 +Getting SMBIOS data from sysfs. +SMBIOS 3.1.1 present. +Table at 0xFE340000. + +Handle 0x0000, DMI type 0, 26 bytes +BIOS Information + Vendor: GIGABYTE + Version: F28 + Release Date: 12/27/2019 + Address: 0xF0000 + Runtime Size: 64 kB + ROM Size: 32 MB + Characteristics: + PCI is supported + BIOS is upgradeable + BIOS shadowing is allowed + Boot from CD is supported + Selectable boot is supported + BIOS ROM is socketed + ACPI is supported + BIOS boot specification is supported + Targeted content distribution is supported + UEFI is supported + BIOS Revision: 7.3 + +Handle 0x0001, DMI type 1, 27 bytes +System Information + Manufacturer: GIGABYTE + Product Name: R181-T90-00 + Version: 0100 + Serial Number: GIG7P9512A0022 + UUID: 00000000-0000-0040-8000-e0d55eae7026 + Wake-up Type: Power Switch + SKU Number: SABER SKU + Family: Server + +Handle 0x0002, DMI type 2, 15 bytes +Base Board Information + Manufacturer: GIGABYTE + Product Name: MT91-FS1-00 + Version: 01000100 + Serial Number: IH6P8800035 + Asset Tag: 01234567890123456789AB + Features: + Board is a hosting board + Board is replaceable + Location In Chassis: Default string + Chassis Handle: 0x0003 + Type: Motherboard + Contained Object Handles: 0 + +Handle 0x0003, DMI type 3, 22 bytes +Chassis Information + Manufacturer: GIGABYTE + Type: Rack Mount Chassis + Lock: Not Present + Version: 1.0 + Serial Number: K61186073100003 + Asset Tag: 01234567890123456789AB + Boot-up State: Safe + Power Supply State: Safe + Thermal State: Safe + Security Status: None + OEM Information: 0x00000000 + Height: Unspecified + Number Of Power Cords: 1 + Contained Elements: 0 + SKU Number: Default string + +Handle 0x0004, DMI type 10, 6 bytes +On Board Device Information + Type: Unknown + Status: Enabled + Description: Device 1 + +Handle 0x0005, DMI type 12, 5 bytes +System Configuration Options + Option 1: Default string + +Handle 0x0006, DMI type 13, 22 bytes +BIOS Language Information + Language Description Format: Long + Installable Languages: 1 + en|US|iso8859-1 + Currently Installed Language: en|US|iso8859-1 + +Handle 0x0007, DMI type 31, 28 bytes +Boot Integrity Services Entry Point + Checksum: Invalid + 16-bit Entry Point Address: FFFF:FFFF + 32-bit Entry Point Address: 0xFFFFFFFF + +Handle 0x0008, DMI type 32, 11 bytes +System Boot Information + Status: No errors detected + +Handle 0x0009, DMI type 39, 22 bytes +System Power Supply + Power Unit Group: 1 + Location: CHINA + Name: FSP1200-20ERM + Manufacturer: FSP GROUP + Serial Number: WS8011100823 + Asset Tag: Default string + Model Part Number: FSP1200-20ERM + Revision: 10 + Max Power Capacity: 2648 W + Status: Present, OK + Type: Switching + Input Voltage Range Switching: Auto-switch + Plugged: Yes + Hot Replaceable: No + +Handle 0x0010, DMI type 39, 22 bytes +System Power Supply + Power Unit Group: 1 + Location: CHINA + Name: FSP1200-20ERM + Manufacturer: FSP GROUP + Serial Number: WS8011100830 + Asset Tag: Default string + Model Part Number: FSP1200-20ERM + Revision: 10 + Max Power Capacity: 2648 W + Status: Present, OK + Type: Switching + Input Voltage Range Switching: Auto-switch + Plugged: Yes + Hot Replaceable: No + +Handle 0x0011, DMI type 41, 11 bytes +Onboard Device + Reference Designation: Device 1 + Type: Unknown + Status: Enabled + Type Instance: 1 + Bus Address: 0000:00:00.0 + +Handle 0x0012, DMI type 41, 11 bytes +Onboard Device + Reference Designation: Device 2 + Type: Unknown + Status: Enabled + Type Instance: 1 + Bus Address: 0000:00:00.0 + +Handle 0x0013, DMI type 41, 11 bytes +Onboard Device + Reference Designation: Device 3 + Type: Unknown + Status: Enabled + Type Instance: 1 + Bus Address: 0000:00:00.0 + +Handle 0x0014, DMI type 41, 11 bytes +Onboard Device + Reference Designation: Device 4 + Type: Unknown + Status: Enabled + Type Instance: 1 + Bus Address: 0000:00:00.0 + +Handle 0x0015, DMI type 41, 11 bytes +Onboard Device + Reference Designation: Device 5 + Type: Unknown + Status: Enabled + Type Instance: 1 + Bus Address: 0000:00:00.0 + +Handle 0x0016, DMI type 38, 18 bytes +IPMI Device Information + Interface Type: SSIF (SMBus System Interface) + Specification Version: 2.0 + I2C Slave Address: 0x10 + NV Storage Device: Not Present + Base Address: 0x10 (SMBus) + +Handle 0x0017, DMI type 42, 12 bytes +Management Controller Host Interface + Interface Type: OEM + Vendor ID: 0xFF0102FF + +Handle 0x0029, DMI type 11, 5 bytes +OEM Strings + String 1: HWID=E38C + String 2: cavium.com + String 3: Saber + +Handle 0x002A, DMI type 13, 22 bytes +BIOS Language Information + Language Description Format: Abbreviated + Installable Languages: 1 + enUS + Currently Installed Language: enUS + +Handle 0x002B, DMI type 4, 48 bytes +Processor Information + Socket Designation: Socket 0 + Type: Central Processor + Family: ARM + Manufacturer: Cavium Inc. + ID: F1 0A 1F 43 00 00 00 00 + Signature: Implementor 0x43, Variant 0x1, Architecture 15, Part 0x0af, Revision 1 + Version: Cavium ThunderX2(R) CPU CN9975 v2.1 @ 2.0GHz + Voltage: 0.8 V + External Clock: 33 MHz + Max Speed: 2500 MHz + Current Speed: 2000 MHz + Status: Populated, Enabled + Upgrade: Other + L1 Cache Handle: 0x002C + L2 Cache Handle: 0x002E + L3 Cache Handle: 0x002F + Serial Number: 000081D4-4003326A + Asset Tag: Not Specified + Part Number: CN9975-2000BG4077-Y21-G + Core Count: 28 + Core Enabled: 28 + Thread Count: 28 + Characteristics: + 64-bit capable + Multi-Core + Hardware Thread + Execute Protection + Enhanced Virtualization + Power/Performance Control +``` + +## Linux dmidecode memory + +``` +Handle 0x003E, DMI type 16, 23 bytes +Physical Memory Array + Location: System Board Or Motherboard + Use: System Memory + Error Correction Type: Multi-bit ECC + Maximum Capacity: 2 TB + Error Information Handle: Not Provided + Number Of Devices: 12 + +Handle 0x003F, DMI type 17, 40 bytes +Memory Device + Array Handle: 0x003E + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: DIMM_P0_A0 + Bank Locator: N0 + Type: DDR4 + Type Detail: Registered (Buffered) + Speed: 2400 MT/s + Manufacturer: Micron Technology + Serial Number: 469570327 + Asset Tag: Not Specified + Part Number: 36ASF4G72PZ-2G3B1 + Rank: 2 + Configured Memory Speed: 2400 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + +Handle 0x0040, DMI type 17, 40 bytes +Memory Device + Array Handle: 0x003E + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: DIMM_P0_B0 + Bank Locator: N0 + Type: DDR4 + Type Detail: Registered (Buffered) + Speed: 2400 MT/s + Manufacturer: Micron Technology + Serial Number: 469570172 + Asset Tag: Not Specified + Part Number: 36ASF4G72PZ-2G3B1 + Rank: 2 + Configured Memory Speed: 2400 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + +Handle 0x0050, DMI type 16, 23 bytes +Physical Memory Array + Location: System Board Or Motherboard + Use: System Memory + Error Correction Type: Multi-bit ECC + Maximum Capacity: 2 TB + Error Information Handle: Not Provided + Number Of Devices: 12 + +Handle 0x0051, DMI type 17, 40 bytes +Memory Device + Array Handle: 0x0050 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: DIMM_P1_I0 + Bank Locator: N1 + Type: DDR4 + Type Detail: Registered (Buffered) + Speed: 2400 MT/s + Manufacturer: Micron Technology + Serial Number: 469567519 + Asset Tag: Not Specified + Part Number: 36ASF4G72PZ-2G3B1 + Rank: 2 + Configured Memory Speed: 2400 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + +Handle 0x0052, DMI type 17, 40 bytes +Memory Device + Array Handle: 0x0050 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: DIMM_P1_J0 + Bank Locator: N1 + Type: DDR4 + Type Detail: Registered (Buffered) + Speed: 2400 MT/s + Manufacturer: Micron Technology + Serial Number: 469567696 + Asset Tag: Not Specified + Part Number: 36ASF4G72PZ-2G3B1 + Rank: 2 + Configured Memory Speed: 2400 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V +``` + +## Linux cmdline + +``` +BOOT_IMAGE=/boot/vmlinuz-5.4.0-65-generic root=UUID=7d1d0e77-4df0-43df-9619-a99db29ffb83 ro audit=0 intel_iommu=on isolcpus=1-27,29-55 nmi_watchdog=0 nohz_full=1-27,29-55 nosoftlockup processor.max_cstate=1 rcu_nocbs=1-27,29-55 console=ttyAMA0,115200n8 quiet +``` diff --git a/docs/content/infrastructure/testbed_configuration/huawei_tsh_hw_bios_cfg.md b/docs/content/infrastructure/testbed_configuration/huawei_tsh_hw_bios_cfg.md new file mode 100644 index 0000000000..6803fd615b --- /dev/null +++ b/docs/content/infrastructure/testbed_configuration/huawei_tsh_hw_bios_cfg.md @@ -0,0 +1,534 @@ +--- +bookToc: true +title: "Huawei Taishan" +--- + +# Huawei Taishan + +## Linux lscpu + +``` +Architecture: aarch64 + CPU op-mode(s): 32-bit, 64-bit + Byte Order: Little Endian +CPU(s): 64 + On-line CPU(s) list: 0-63 +Vendor ID: ARM + Model name: Cortex-A72 + Model: 2 + Thread(s) per core: 1 + Core(s) per socket: 32 + Socket(s): 2 + Stepping: r0p2 + BogoMIPS: 100.00 + Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 cpuid +Caches (sum of all): + L1d: 2 MiB (64 instances) + L1i: 3 MiB (64 instances) + L2: 16 MiB (16 instances) + L3: 64 MiB (4 instances) +NUMA: + NUMA node(s): 4 + NUMA node0 CPU(s): 0-15 + NUMA node1 CPU(s): 16-31 + NUMA node2 CPU(s): 32-47 + NUMA node3 CPU(s): 48-63 +Vulnerabilities: + Itlb multihit: Not affected + L1tf: Not affected + Mds: Not affected + Meltdown: Not affected + Mmio stale data: Not affected + Retbleed: Not affected + Spec store bypass: Vulnerable + Spectre v1: Mitigation; __user pointer sanitization + Spectre v2: Vulnerable + Srbds: Not affected + Tsx async abort: Not affected +``` + +## Linux dmidecode + +``` +# dmidecode 3.3 +Getting SMBIOS data from sysfs. +SMBIOS 3.0.0 present. +Table at 0x39150000. + +Handle 0x0000, DMI type 0, 24 bytes +BIOS Information + Vendor: Huawei Corp. + Version: Estuary-5.1 D05 LTS + Release Date: 05/25/2018 + Address: 0xA4800 + Runtime Size: 366 kB + ROM Size: 3 MB + Characteristics: + PCI is supported + BIOS is upgradeable + BIOS shadowing is allowed + Boot from CD is supported + Selectable boot is supported + EDD is supported + Japanese floppy for NEC 9800 1.2 MB is supported (int 13h) + Japanese floppy for Toshiba 1.2 MB is supported (int 13h) + 5.25"/360 kB floppy services are supported (int 13h) + 5.25"/1.2 MB floppy services are supported (int 13h) + 3.5"/720 kB floppy services are supported (int 13h) + 3.5"/2.88 MB floppy services are supported (int 13h) + 8042 keyboard services are supported (int 9h) + CGA/mono video services are supported (int 10h) + ACPI is supported + USB legacy is supported + BIOS boot specification is supported + Targeted content distribution is supported + UEFI is supported + BIOS Revision: 0.0 + +Handle 0x0001, DMI type 1, 27 bytes +System Information + Manufacturer: Huawei + Product Name: D05 + Version: VER.A + Serial Number: 2102311TBJ10J1000089 + UUID: e11a0a38-f920-11e7-8c7d-a0a33bc11426 + Wake-up Type: Power Switch + SKU Number: To be filled by O.E.M. + Family: To be filled by O.E.M. + +Handle 0x0002, DMI type 3, 25 bytes +Chassis Information + Manufacturer: Huawei + Type: Main Server Chassis + Lock: Not Present + Version: To be filled by O.E.M. + Serial Number: To be filled by O.E.M. + Asset Tag: To be filled by O.E.M. + Boot-up State: Safe + Power Supply State: Safe + Thermal State: Safe + Security Status: None + OEM Information: 0x00000000 + Height: 2 U + Number Of Power Cords: 1 + Contained Elements: 0 + SKU Number: Not Specified + +Handle 0x0003, DMI type 2, 17 bytes +Base Board Information + Manufacturer: Huawei + Product Name: D05 + Version: Estuary + Serial Number: 024APL10H8000089 + Asset Tag: To be filled by O.E.M. + Features: + Board is a hosting board + Board is replaceable + Location In Chassis: To Be Filled By O.E.M. + Chassis Handle: 0x0002 + Type: Motherboard + Contained Object Handles: 0 +``` + +## Linux dmidecode memory + +``` +Handle 0x0007, DMI type 16, 23 bytes +Physical Memory Array + Location: System Board Or Motherboard + Use: System Memory + Error Correction Type: None + Maximum Capacity: 512 GB + Error Information Handle: Not Provided + Number Of Devices: 16 + +Handle 0x0009, DMI type 17, 40 bytes +Memory Device + Array Handle: 0x0007 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16 GB + Form Factor: DIMM + Set: None + Locator: DIMM000 J5 + Bank Locator: SOCKET 0 CHANNEL 0 DIMM 0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 2400 MT/s + Manufacturer: Samsung + Serial Number: 0x37663087 + Asset Tag: Unknown + Part Number: M393A2K43BB1-CRC + Rank: 2 + Configured Memory Speed: 2400 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 2.0 V + Configured Voltage: 1.2 V + +Handle 0x000A, DMI type 17, 40 bytes +Memory Device + Array Handle: 0x0007 + Error Information Handle: Not Provided + Total Width: Unknown + Data Width: Unknown + Size: No Module Installed + Form Factor: DIMM + Set: None + Locator: DIMM001 J6 + Bank Locator: SOCKET 0 CHANNEL 0 DIMM 1 + Type: Unknown + Type Detail: Unknown Synchronous + Speed: Unknown + Manufacturer: NO DIMM + Serial Number: NO DIMM + Asset Tag: NO DIMM + Part Number: NO DIMM + Rank: Unknown + Configured Memory Speed: Unknown + Minimum Voltage: Unknown + Maximum Voltage: Unknown + Configured Voltage: Unknown + +Handle 0x000B, DMI type 17, 40 bytes +Memory Device + Array Handle: 0x0007 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16 GB + Form Factor: DIMM + Set: None + Locator: DIMM010 J8 + Bank Locator: SOCKET 0 CHANNEL 1 DIMM 0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 2400 MT/s + Manufacturer: Samsung + Serial Number: 0x37663064 + Asset Tag: Unknown + Part Number: M393A2K43BB1-CRC + Rank: 2 + Configured Memory Speed: 2400 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 2.0 V + Configured Voltage: 1.2 V + +Handle 0x000C, DMI type 17, 40 bytes +Memory Device + Array Handle: 0x0007 + Error Information Handle: Not Provided + Total Width: Unknown + Data Width: Unknown + Size: No Module Installed + Form Factor: DIMM + Set: None + Locator: DIMM011 J9 + Bank Locator: SOCKET 0 CHANNEL 1 DIMM 1 + Type: Unknown + Type Detail: Unknown Synchronous + Speed: Unknown + Manufacturer: NO DIMM + Serial Number: NO DIMM + Asset Tag: NO DIMM + Part Number: NO DIMM + Rank: Unknown + Configured Memory Speed: Unknown + Minimum Voltage: Unknown + Maximum Voltage: Unknown + Configured Voltage: Unknown + +Handle 0x000D, DMI type 17, 40 bytes +Memory Device + Array Handle: 0x0007 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16 GB + Form Factor: DIMM + Set: None + Locator: DIMM020 J11 + Bank Locator: SOCKET 0 CHANNEL 2 DIMM 0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 2400 MT/s + Manufacturer: Samsung + Serial Number: 0x3766308B + Asset Tag: Unknown + Part Number: M393A2K43BB1-CRC + Rank: 2 + Configured Memory Speed: 2400 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 2.0 V + Configured Voltage: 1.2 V + +Handle 0x000E, DMI type 17, 40 bytes +Memory Device + Array Handle: 0x0007 + Error Information Handle: Not Provided + Total Width: Unknown + Data Width: Unknown + Size: No Module Installed + Form Factor: DIMM + Set: None + Locator: DIMM021 J12 + Bank Locator: SOCKET 0 CHANNEL 2 DIMM 1 + Type: Unknown + Type Detail: Unknown Synchronous + Speed: Unknown + Manufacturer: NO DIMM + Serial Number: NO DIMM + Asset Tag: NO DIMM + Part Number: NO DIMM + Rank: Unknown + Configured Memory Speed: Unknown + Minimum Voltage: Unknown + Maximum Voltage: Unknown + Configured Voltage: Unknown + +Handle 0x000F, DMI type 17, 40 bytes +Memory Device + Array Handle: 0x0007 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16 GB + Form Factor: DIMM + Set: None + Locator: DIMM030 J14 + Bank Locator: SOCKET 0 CHANNEL 3 DIMM 0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 2400 MT/s + Manufacturer: Samsung + Serial Number: 0x376630DA + Asset Tag: Unknown + Part Number: M393A2K43BB1-CRC + Rank: 2 + Configured Memory Speed: 2400 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 2.0 V + Configured Voltage: 1.2 V + +Handle 0x0010, DMI type 17, 40 bytes +Memory Device + Array Handle: 0x0007 + Error Information Handle: Not Provided + Total Width: Unknown + Data Width: Unknown + Size: No Module Installed + Form Factor: DIMM + Set: None + Locator: DIMM031 J15 + Bank Locator: SOCKET 0 CHANNEL 3 DIMM 1 + Type: Unknown + Type Detail: Unknown Synchronous + Speed: Unknown + Manufacturer: NO DIMM + Serial Number: NO DIMM + Asset Tag: NO DIMM + Part Number: NO DIMM + Rank: Unknown + Configured Memory Speed: Unknown + Minimum Voltage: Unknown + Maximum Voltage: Unknown + Configured Voltage: Unknown + +Handle 0x0011, DMI type 17, 40 bytes +Memory Device + Array Handle: 0x0007 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16 GB + Form Factor: DIMM + Set: None + Locator: DIMM100 J17 + Bank Locator: SOCKET 1 CHANNEL 0 DIMM 0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 2400 MT/s + Manufacturer: Samsung + Serial Number: 0x379A2774 + Asset Tag: Unknown + Part Number: M393A2K43BB1-CRC + Rank: 2 + Configured Memory Speed: 2400 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 2.0 V + Configured Voltage: 1.2 V + +Handle 0x0012, DMI type 17, 40 bytes +Memory Device + Array Handle: 0x0007 + Error Information Handle: Not Provided + Total Width: Unknown + Data Width: Unknown + Size: No Module Installed + Form Factor: DIMM + Set: None + Locator: DIMM101 J18 + Bank Locator: SOCKET 1 CHANNEL 0 DIMM 1 + Type: Unknown + Type Detail: Unknown Synchronous + Speed: Unknown + Manufacturer: NO DIMM + Serial Number: NO DIMM + Asset Tag: NO DIMM + Part Number: NO DIMM + Rank: Unknown + Configured Memory Speed: Unknown + Minimum Voltage: Unknown + Maximum Voltage: Unknown + Configured Voltage: Unknown + +Handle 0x0013, DMI type 17, 40 bytes +Memory Device + Array Handle: 0x0007 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16 GB + Form Factor: DIMM + Set: None + Locator: DIMM110 J20 + Bank Locator: SOCKET 1 CHANNEL 1 DIMM 0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 2400 MT/s + Manufacturer: Samsung + Serial Number: 0x3766308A + Asset Tag: Unknown + Part Number: M393A2K43BB1-CRC + Rank: 2 + Configured Memory Speed: 2400 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 2.0 V + Configured Voltage: 1.2 V + +Handle 0x0014, DMI type 17, 40 bytes +Memory Device + Array Handle: 0x0007 + Error Information Handle: Not Provided + Total Width: Unknown + Data Width: Unknown + Size: No Module Installed + Form Factor: DIMM + Set: None + Locator: DIMM111 J21 + Bank Locator: SOCKET 1 CHANNEL 1 DIMM 1 + Type: Unknown + Type Detail: Unknown Synchronous + Speed: Unknown + Manufacturer: NO DIMM + Serial Number: NO DIMM + Asset Tag: NO DIMM + Part Number: NO DIMM + Rank: Unknown + Configured Memory Speed: Unknown + Minimum Voltage: Unknown + Maximum Voltage: Unknown + Configured Voltage: Unknown + +Handle 0x0015, DMI type 17, 40 bytes +Memory Device + Array Handle: 0x0007 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16 GB + Form Factor: DIMM + Set: None + Locator: DIMM120 J23 + Bank Locator: SOCKET 1 CHANNEL 2 DIMM 0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 2400 MT/s + Manufacturer: Samsung + Serial Number: 0x376630B0 + Asset Tag: Unknown + Part Number: M393A2K43BB1-CRC + Rank: 2 + Configured Memory Speed: 2400 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 2.0 V + Configured Voltage: 1.2 V + +Handle 0x0016, DMI type 17, 40 bytes +Memory Device + Array Handle: 0x0007 + Error Information Handle: Not Provided + Total Width: Unknown + Data Width: Unknown + Size: No Module Installed + Form Factor: DIMM + Set: None + Locator: DIMM121 J24 + Bank Locator: SOCKET 1 CHANNEL 2 DIMM 1 + Type: Unknown + Type Detail: Unknown Synchronous + Speed: Unknown + Manufacturer: NO DIMM + Serial Number: NO DIMM + Asset Tag: NO DIMM + Part Number: NO DIMM + Rank: Unknown + Configured Memory Speed: Unknown + Minimum Voltage: Unknown + Maximum Voltage: Unknown + Configured Voltage: Unknown + +Handle 0x0017, DMI type 17, 40 bytes +Memory Device + Array Handle: 0x0007 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16 GB + Form Factor: DIMM + Set: None + Locator: DIMM130 J26 + Bank Locator: SOCKET 1 CHANNEL 3 DIMM 0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 2400 MT/s + Manufacturer: Samsung + Serial Number: 0x376630A0 + Asset Tag: Unknown + Part Number: M393A2K43BB1-CRC + Rank: 2 + Configured Memory Speed: 2400 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 2.0 V + Configured Voltage: 1.2 V + +Handle 0x0018, DMI type 17, 40 bytes +Memory Device + Array Handle: 0x0007 + Error Information Handle: Not Provided + Total Width: Unknown + Data Width: Unknown + Size: No Module Installed + Form Factor: DIMM + Set: None + Locator: DIMM131 J27 + Bank Locator: SOCKET 1 CHANNEL 3 DIMM 1 + Type: Unknown + Type Detail: Unknown Synchronous + Speed: Unknown + Manufacturer: NO DIMM + Serial Number: NO DIMM + Asset Tag: NO DIMM + Part Number: NO DIMM + Rank: Unknown + Configured Memory Speed: Unknown + Minimum Voltage: Unknown + Maximum Voltage: Unknown + Configured Voltage: Unknown +``` + +## Linux cmdline + +``` +BOOT_IMAGE=/boot/vmlinuz-5.4.0-65-generic root=UUID=7d1d0e77-4df0-43df-9619-a99db29ffb83 ro audit=0 intel_iommu=on isolcpus=1-27,29-55 nmi_watchdog=0 nohz_full=1-27,29-55 nosoftlockup processor.max_cstate=1 rcu_nocbs=1-27,29-55 console=ttyAMA0,115200n8 quiet +``` diff --git a/docs/content/infrastructure/testbed_configuration/sm_clx_hw_bios_cfg.md b/docs/content/infrastructure/testbed_configuration/sm_clx_hw_bios_cfg.md new file mode 100644 index 0000000000..c955b424fe --- /dev/null +++ b/docs/content/infrastructure/testbed_configuration/sm_clx_hw_bios_cfg.md @@ -0,0 +1,1425 @@ +--- +bookToc: true +title: "SuperMicro Cascadelake" +--- + +# SuperMicro Cascadelake + +## Linux lscpu + +``` +Architecture: x86_64 +CPU op-mode(s): 32-bit, 64-bit +Byte Order: Little Endian +CPU(s): 112 +On-line CPU(s) list: 0-111 +Thread(s) per core: 2 +Core(s) per socket: 28 +Socket(s): 2 +NUMA node(s): 2 +Vendor ID: GenuineIntel +CPU family: 6 +Model: 85 +Model name: Intel(R) Xeon(R) Platinum 8280 CPU @ 2.70GHz +Stepping: 7 +CPU MHz: 3299.609 +BogoMIPS: 5400.00 +Virtualization: VT-x +L1d cache: 32K +L1i cache: 32K +L2 cache: 1024K +L3 cache: 39424K +NUMA node0 CPU(s): 0-27,56-83 +NUMA node1 CPU(s): 28-55,84-111 +Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca +cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx +pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology +nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est +tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt +tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch +cpuid_fault epb cat_l3 cdp_l3 invpcid_single ssbd mba ibrs ibpb stibp +ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 +hle avx2 smep bmi2 erms invpcid rtm cqm mpx rdt_a avx512f avx512dq rdseed adx +smap clflushopt clwb intel_pt avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 +xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts +pku ospke avx512_vnni md_clear flush_l1d arch_capabilities +``` + +``` +Architecture: x86_64 +CPU op-mode(s): 32-bit, 64-bit +Byte Order: Little Endian +CPU(s): 96 +On-line CPU(s) list: 0-95 +Thread(s) per core: 2 +Core(s) per socket: 24 +Socket(s): 2 +NUMA node(s): 2 +Vendor ID: GenuineIntel +CPU family: 6 +Model: 85 +Model name: Intel(R) Xeon(R) Gold 6252N CPU @ 2.30GHz +Stepping: 7 +CPU MHz: 3000.989 +BogoMIPS: 4600.00 +Virtualization: VT-x +L1d cache: 32K +L1i cache: 32K +L2 cache: 1024K +L3 cache: 36608K +NUMA node0 CPU(s): 0-23,48-71 +NUMA node1 CPU(s): 24-47,72-95 +Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca +cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx +pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology +nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 +ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt +tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch +cpuid_fault epb cat_l3 cdp_l3 invpcid_single ssbd mba ibrs ibpb stibp +ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle +avx2 smep bmi2 erms invpcid rtm cqm mpx rdt_a avx512f avx512dq rdseed adx smap +clflushopt clwb intel_pt avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 +xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts +pku ospke avx512_vnni md_clear flush_l1d arch_capabilities +``` + +## Linux dmidecode + +``` + # dmidecode 3.1 + Getting SMBIOS data from sysfs. + SMBIOS 3.1.2 present. + Table at 0x6EB92000. + + Handle 0x0000, DMI type 0, 26 bytes + BIOS Information + Vendor: American Megatrends Inc. + Version: 3.0c + Release Date: 03/27/2019 + Address: 0xF0000 + Runtime Size: 64 kB + ROM Size: 32 MB + Characteristics: + PCI is supported + BIOS is upgradeable + BIOS shadowing is allowed + Boot from CD is supported + Selectable boot is supported + BIOS ROM is socketed + EDD is supported + 5.25"/1.2 MB floppy services are supported (int 13h) + 3.5"/720 kB floppy services are supported (int 13h) + 3.5"/2.88 MB floppy services are supported (int 13h) + Print screen service is supported (int 5h) + Serial services are supported (int 14h) + Printer services are supported (int 17h) + ACPI is supported + USB legacy is supported + BIOS boot specification is supported + Targeted content distribution is supported + UEFI is supported + BIOS Revision: 5.14 + + Handle 0x0001, DMI type 1, 27 bytes + System Information + Manufacturer: Supermicro + Product Name: SYS-7049GP-TRT + Version: 0123456789 + Serial Number: S291427X9525476 + UUID: 00000000-0000-0000-0000-AC1F6BACD7BA + Wake-up Type: Power Switch + SKU Number: To be filled by O.E.M. + Family: To be filled by O.E.M. + + Handle 0x0002, DMI type 2, 15 bytes + Base Board Information + Manufacturer: Supermicro + Product Name: X11DPG-QT + Version: 1.10A + Serial Number: VM189S007860 + Asset Tag: To be filled by O.E.M. + Features: + Board is a hosting board + Board is replaceable + Location In Chassis: To be filled by O.E.M. + Chassis Handle: 0x0003 + Type: Motherboard + Contained Object Handles: 0 + + Handle 0x0003, DMI type 3, 22 bytes + Chassis Information + Manufacturer: Supermicro + Type: Other + Lock: Not Present + Version: 0123456789 + Serial Number: C7470KH37A30566 + Asset Tag: To be filled by O.E.M. + Boot-up State: Safe + Power Supply State: Safe + Thermal State: Safe + Security Status: None + OEM Information: 0x00000000 + Height: Unspecified + Number Of Power Cords: 1 + Contained Elements: 0 + SKU Number: To be filled by O.E.M. + + Handle 0x0055, DMI type 4, 48 bytes + Processor Information + Socket Designation: CPU1 + Type: Central Processor + Family: Xeon + Manufacturer: Intel(R) Corporation + ID: 57 06 05 00 FF FB EB BF + Signature: Type 0, Family 6, Model 85, Stepping 7 + Flags: + FPU (Floating-point unit on-chip) + VME (Virtual mode extension) + DE (Debugging extension) + PSE (Page size extension) + TSC (Time stamp counter) + MSR (Model specific registers) + PAE (Physical address extension) + MCE (Machine check exception) + CX8 (CMPXCHG8 instruction supported) + APIC (On-chip APIC hardware supported) + SEP (Fast system call) + MTRR (Memory type range registers) + PGE (Page global enable) + MCA (Machine check architecture) + CMOV (Conditional move instruction supported) + PAT (Page attribute table) + PSE-36 (36-bit page size extension) + CLFSH (CLFLUSH instruction supported) + DS (Debug store) + ACPI (ACPI supported) + MMX (MMX technology supported) + FXSR (FXSAVE and FXSTOR instructions supported) + SSE (Streaming SIMD extensions) + SSE2 (Streaming SIMD extensions 2) + SS (Self-snoop) + HTT (Multi-threading) + TM (Thermal monitor supported) + PBE (Pending break enabled) + Version: Intel(R) Xeon(R) Gold 6252N CPU @ 2.30GHz + Voltage: 1.6 V + External Clock: 100 MHz + Max Speed: 4500 MHz + Current Speed: 2300 MHz + Status: Populated, Enabled + Upgrade: Socket LGA3647-1 + L1 Cache Handle: 0x0052 + L2 Cache Handle: 0x0053 + L3 Cache Handle: 0x0054 + Serial Number: Not Specified + Asset Tag: UNKNOWN + Part Number: Not Specified + Core Count: 24 + Core Enabled: 24 + Thread Count: 48 + Characteristics: + 64-bit capable + Multi-Core + Hardware Thread + Execute Protection + Enhanced Virtualization + Power/Performance Control + + Handle 0x0059, DMI type 4, 48 bytes + Processor Information + Socket Designation: CPU2 + Type: Central Processor + Family: Xeon + Manufacturer: Intel(R) Corporation + ID: 57 06 05 00 FF FB EB BF + Signature: Type 0, Family 6, Model 85, Stepping 7 + Flags: + FPU (Floating-point unit on-chip) + VME (Virtual mode extension) + DE (Debugging extension) + PSE (Page size extension) + TSC (Time stamp counter) + MSR (Model specific registers) + PAE (Physical address extension) + MCE (Machine check exception) + CX8 (CMPXCHG8 instruction supported) + APIC (On-chip APIC hardware supported) + SEP (Fast system call) + MTRR (Memory type range registers) + PGE (Page global enable) + MCA (Machine check architecture) + CMOV (Conditional move instruction supported) + PAT (Page attribute table) + PSE-36 (36-bit page size extension) + CLFSH (CLFLUSH instruction supported) + DS (Debug store) + ACPI (ACPI supported) + MMX (MMX technology supported) + FXSR (FXSAVE and FXSTOR instructions supported) + SSE (Streaming SIMD extensions) + SSE2 (Streaming SIMD extensions 2) + SS (Self-snoop) + HTT (Multi-threading) + TM (Thermal monitor supported) + PBE (Pending break enabled) + Version: Intel(R) Xeon(R) Gold 6252N CPU @ 2.30GHz + Voltage: 1.6 V + External Clock: 100 MHz + Max Speed: 4500 MHz + Current Speed: 2300 MHz + Status: Populated, Enabled + Upgrade: Socket LGA3647-1 + L1 Cache Handle: 0x0056 + L2 Cache Handle: 0x0057 + L3 Cache Handle: 0x0058 + Serial Number: Not Specified + Asset Tag: UNKNOWN + Part Number: Not Specified + Core Count: 24 + Core Enabled: 24 + Thread Count: 48 + Characteristics: + 64-bit capable + Multi-Core + Hardware Thread + Execute Protection + Enhanced Virtualization + Power/Performance Control +``` + +## Linux dmidecode pci + +``` + Handle 0x000B, DMI type 9, 17 bytes + System Slot Information + Designation: CPU1 SLOT2 PCI-E 3.0 X16 + Type: x16 PCI Express 3 x16 + Current Usage: In Use + Length: Long + ID: 2 + Characteristics: + 3.3 V is provided + Opening is shared + PME signal is supported + Bus Address: 0000:18:00.0 + + Handle 0x000C, DMI type 9, 17 bytes + System Slot Information + Designation: CPU1 SLOT4 PCI-E 3.0 X16 + Type: x16 PCI Express 3 x16 + Current Usage: In Use + Length: Short + ID: 4 + Characteristics: + 3.3 V is provided + Opening is shared + PME signal is supported + Bus Address: 0000:3b:00.0 + + Handle 0x000D, DMI type 9, 17 bytes + System Slot Information + Designation: CPU2 SLOT6 PCI-E 3.0 X16 + Type: x16 PCI Express 3 x16 + Current Usage: Available + Length: Short + ID: 6 + Characteristics: + 3.3 V is provided + Opening is shared + PME signal is supported + Bus Address: 0000:ff:00.0 + + Handle 0x000E, DMI type 9, 17 bytes + System Slot Information + Designation: CPU2 SLOT8 PCI-E 3.0 X16 + Type: x16 PCI Express 3 x16 + Current Usage: Available + Length: Short + ID: 8 + Characteristics: + 3.3 V is provided + Opening is shared + PME signal is supported + Bus Address: 0000:ff:00.0 + + Handle 0x000F, DMI type 9, 17 bytes + System Slot Information + Designation: CPU1 SLOT9 PCI-E 3.0 X16 + Type: x16 PCI Express 3 x16 + Current Usage: Available + Length: Short + ID: 9 + Characteristics: + 3.3 V is provided + Opening is shared + PME signal is supported + Bus Address: 0000:ff:00.0 + + Handle 0x0010, DMI type 9, 17 bytes + System Slot Information + Designation: CPU2 SLOT10 PCI-E 3.0 X16 + Type: x16 PCI Express 3 x16 + Current Usage: Available + Length: Short + ID: 10 + Characteristics: + 3.3 V is provided + Opening is shared + PME signal is supported + Bus Address: 0000:ff:00.0 + + Handle 0x0011, DMI type 9, 17 bytes + System Slot Information + Designation: CPU2 SLOT11 PCI-E 3.0 X4(IN X8) + Type: x4 PCI Express 3 x8 + Current Usage: Available + Length: Short + ID: 11 + Characteristics: + 3.3 V is provided + Opening is shared + PME signal is supported + Bus Address: 0000:ff:00.0 + + Handle 0x0012, DMI type 9, 17 bytes + System Slot Information + Designation: M.2 CONNECTOR + Type: x4 M.2 Socket 2 + Current Usage: Available + Length: Short + Characteristics: + 3.3 V is provided + Opening is shared + PME signal is supported + Bus Address: 0000:ff:00.0 +``` + +## Linux dmidecode memory + +``` + Handle 0x0023, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0021 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P1-DIMMA1 + Bank Locator: P0_Node0_Channel0_Dimm0 + Type: DDR4 + Type Detail: Synchronous + Speed: 2933 MT/s + Manufacturer: SK Hynix + Serial Number: 93275F0E + Asset Tag: P1-DIMMA1_AssetTag (date:19/22) + Part Number: HMA82GR7CJR8N-WM + Rank: 2 + Configured Clock Speed: 2934 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x0024, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0021 + Error Information Handle: Not Provided + Total Width: Unknown + Data Width: Unknown + Size: No Module Installed + Form Factor: DIMM + Set: None + Locator: P1-DIMMA2 + Bank Locator: P0_Node0_Channel0_Dimm1 + Type: Unknown + Type Detail: Unknown + Speed: Unknown + Manufacturer: NO DIMM + Serial Number: NO DIMM + Asset Tag: NO DIMM + Part Number: NO DIMM + Rank: Unknown + Configured Clock Speed: Unknown + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x0025, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0021 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P1-DIMMB1 + Bank Locator: P0_Node0_Channel1_Dimm0 + Type: DDR4 + Type Detail: Synchronous + Speed: 2933 MT/s + Manufacturer: SK Hynix + Serial Number: 93275F1F + Asset Tag: P1-DIMMB1_AssetTag (date:19/22) + Part Number: HMA82GR7CJR8N-WM + Rank: 2 + Configured Clock Speed: 2934 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x0027, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0021 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P1-DIMMC1 + Bank Locator: P0_Node0_Channel2_Dimm0 + Type: DDR4 + Type Detail: Synchronous + Speed: 2933 MT/s + Manufacturer: SK Hynix + Serial Number: 93275F07 + Asset Tag: P1-DIMMC1_AssetTag (date:19/22) + Part Number: HMA82GR7CJR8N-WM + Rank: 2 + Configured Clock Speed: 2934 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x002B, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0029 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P1-DIMMD1 + Bank Locator: P0_Node1_Channel0_Dimm0 + Type: DDR4 + Type Detail: Synchronous + Speed: 2933 MT/s + Manufacturer: SK Hynix + Serial Number: 93275F02 + Asset Tag: P1-DIMMD1_AssetTag (date:19/22) + Part Number: HMA82GR7CJR8N-WM + Rank: 2 + Configured Clock Speed: 2934 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x002C, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0029 + Error Information Handle: Not Provided + Total Width: Unknown + Data Width: Unknown + Size: No Module Installed + Form Factor: DIMM + Set: None + Locator: P1-DIMMD2 + Bank Locator: P0_Node1_Channel0_Dimm1 + Type: Unknown + Type Detail: Unknown + Speed: Unknown + Manufacturer: NO DIMM + Serial Number: NO DIMM + Asset Tag: NO DIMM + Part Number: NO DIMM + Rank: Unknown + Configured Clock Speed: Unknown + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x002D, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0029 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P1-DIMME1 + Bank Locator: P0_Node1_Channel1_Dimm0 + Type: DDR4 + Type Detail: Synchronous + Speed: 2933 MT/s + Manufacturer: SK Hynix + Serial Number: 93275F19 + Asset Tag: P1-DIMME1_AssetTag (date:19/22) + Part Number: HMA82GR7CJR8N-WM + Rank: 2 + Configured Clock Speed: 2934 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x002F, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0029 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P1-DIMMF1 + Bank Locator: P0_Node1_Channel2_Dimm0 + Type: DDR4 + Type Detail: Synchronous + Speed: 2933 MT/s + Manufacturer: SK Hynix + Serial Number: 93275FD3 + Asset Tag: P1-DIMMF1_AssetTag (date:19/22) + Part Number: HMA82GR7CJR8N-WM + Rank: 2 + Configured Clock Speed: 2934 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x0031, DMI type 16, 23 bytes + Physical Memory Array + Location: System Board Or Motherboard + Use: System Memory + Error Correction Type: Single-bit ECC + Maximum Capacity: 2304 GB + Error Information Handle: Not Provided + Number Of Devices: 4 + + Handle 0x0033, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0031 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P2-DIMMA1 + Bank Locator: P1_Node0_Channel0_Dimm0 + Type: DDR4 + Type Detail: Synchronous + Speed: 2933 MT/s + Manufacturer: SK Hynix + Serial Number: 93275FE2 + Asset Tag: P2-DIMMA1_AssetTag (date:19/22) + Part Number: HMA82GR7CJR8N-WM + Rank: 2 + Configured Clock Speed: 2934 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x0034, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0031 + Error Information Handle: Not Provided + Total Width: Unknown + Data Width: Unknown + Size: No Module Installed + Form Factor: DIMM + Set: None + Locator: P2-DIMMA2 + Bank Locator: P1_Node0_Channel0_Dimm1 + Type: Unknown + Type Detail: Unknown + Speed: Unknown + Manufacturer: NO DIMM + Serial Number: NO DIMM + Asset Tag: NO DIMM + Part Number: NO DIMM + Rank: Unknown + Configured Clock Speed: Unknown + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x0035, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0031 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P2-DIMMB1 + Bank Locator: P1_Node0_Channel1_Dimm0 + Type: DDR4 + Type Detail: Synchronous + Speed: 2933 MT/s + Manufacturer: SK Hynix + Serial Number: 93276001 + Asset Tag: P2-DIMMB1_AssetTag (date:19/22) + Part Number: HMA82GR7CJR8N-WM + Rank: 2 + Configured Clock Speed: 2934 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x0037, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0031 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P2-DIMMC1 + Bank Locator: P1_Node0_Channel2_Dimm0 + Type: DDR4 + Type Detail: Synchronous + Speed: 2933 MT/s + Manufacturer: SK Hynix + Serial Number: 93276005 + Asset Tag: P2-DIMMC1_AssetTag (date:19/22) + Part Number: HMA82GR7CJR8N-WM + Rank: 2 + Configured Clock Speed: 2934 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x0039, DMI type 16, 23 bytes + Physical Memory Array + Location: System Board Or Motherboard + Use: System Memory + Error Correction Type: Single-bit ECC + Maximum Capacity: 2304 GB + Error Information Handle: Not Provided + Number Of Devices: 4 + + Handle 0x003B, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0039 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P2-DIMMD1 + Bank Locator: P1_Node1_Channel0_Dimm0 + Type: DDR4 + Type Detail: Synchronous + Speed: 2933 MT/s + Manufacturer: SK Hynix + Serial Number: 93275F44 + Asset Tag: P2-DIMMD1_AssetTag (date:19/22) + Part Number: HMA82GR7CJR8N-WM + Rank: 2 + Configured Clock Speed: 2934 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x003C, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0039 + Error Information Handle: Not Provided + Total Width: Unknown + Data Width: Unknown + Size: No Module Installed + Form Factor: DIMM + Set: None + Locator: P2-DIMMD2 + Bank Locator: P1_Node1_Channel0_Dimm1 + Type: Unknown + Type Detail: Unknown + Speed: Unknown + Manufacturer: NO DIMM + Serial Number: NO DIMM + Asset Tag: NO DIMM + Part Number: NO DIMM + Rank: Unknown + Configured Clock Speed: Unknown + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x003D, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0039 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P2-DIMME1 + Bank Locator: P1_Node1_Channel1_Dimm0 + Type: DDR4 + Type Detail: Synchronous + Speed: 2933 MT/s + Manufacturer: SK Hynix + Serial Number: 93275FDF + Asset Tag: P2-DIMME1_AssetTag (date:19/22) + Part Number: HMA82GR7CJR8N-WM + Rank: 2 + Configured Clock Speed: 2934 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x003F, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0039 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P2-DIMMF1 + Bank Locator: P1_Node1_Channel2_Dimm0 + Type: DDR4 + Type Detail: Synchronous + Speed: 2933 MT/s + Manufacturer: SK Hynix + Serial Number: 93275FDD + Asset Tag: P2-DIMMF1_AssetTag (date:19/22) + Part Number: HMA82GR7CJR8N-WM + Rank: 2 + Configured Clock Speed: 2934 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V +``` + +## Xeon CLX Server BIOS Configuration - TG + +### Boot Feature + +``` + | Quiet Boot [Enabled] |Boot option | + | | | + | Option ROM Messages [Force BIOS] | | + | Bootup NumLock State [On] | | + | Wait For "F1" If Error [Enabled] | | + | INT19 Trap Response [Immediate] | | + | Re-try Boot [Disabled] | | + | Install Windows 7 USB support [Disabled] | | + | Port 61h Bit-4 Emulation [Disabled] | | + | | | + | Power Configuration | | + | Watch Dog Function [Disabled] | | + | Restore on AC Power Loss [Last State] | | + | Power Button Function [Instant Off] | | +``` + +### CPU Configuration + +``` + | Processor Configuration ^|Enables Hyper Threading | + | -------------------------------------------------- *|(Software Method to | + | Processor BSP Revision 50657 - CLX B1 *|Enable/Disable Logical | + | Processor Socket CPU1 | CPU2 *|Processor threads. | + | Processor ID 00050657* | 00050657 *| | + | Processor Frequency 2.700GHz | 2.700GHz *| | + | Processor Max Ratio 1BH | 1BH *| | + | Processor Min Ratio 0AH | 0AH *| | + | Microcode Revision 0500002C | 0500002C *| | + | L1 Cache RAM 64KB | 64KB *| | + | L2 Cache RAM 1024KB | 1024KB *| | + | L3 Cache RAM 39424KB | 39424KB *| | + | Processor 0 Version *| | + | Intel(R) Xeon(R) Platinum 8280 CPU @ 2.70GHz *| | + | Processor 1 Version *| | + | Intel(R) Xeon(R) Platinum 8280 CPU @ 2.70GHz *| | + | *|-----------------------------| + | Hyper-Threading [ALL] [Enable] *|><: Select Screen | + | Cores Enabled 0 *|^v: Select Item | + | Monitor/Mwait [Auto] *|Enter: Select | + | Execute Disable Bit [Enable] +|+/-: Change Opt. | + | Intel Virtualization Technology [Enable] +|F1: General Help | + | PPIN Control [Unlock/Enable] +|F2: Previous Values | + | Hardware Prefetcher [Enable] +|F3: Optimized Defaults | + | Adjacent Cache Prefetch [Enable] v|F4: Save & Exit | + | DCU Streamer Prefetcher [Enable] | | + | DCU IP Prefetcher [Enable] | | + | LLC Prefetch [Disable] | | + | Extended APIC [Disable] | | + | AES-NI [Enable] | | + |> Advanced Power Management Configuration | | +``` + +#### Advanced Power Management Configuration + +``` + | Advanced Power Management Configuration |Switch CPU Power Management | + | -------------------------------------------------- |profile | + | Power Technology [Custom] | | + | Power Performance Tuning [BIOS Controls EPB] | | + | ENERGY_PERF_BIAS_CFG mode [Maximum Performance] | | + |> CPU P State Control | | + |> Hardware PM State Control | | + |> CPU C State Control | | + |> Package C State Control | | + |> CPU T State Control | | +``` + +##### CPU P State Control + +``` + | CPU P State Control |EIST allows the processor | + | |to dynamically adjust | + | SpeedStep (P-States) [Disable] |frequency and voltage based | + | EIST PSD Function [HW_ALL] |on power versus performance | + | |needs. | + | | | +``` + +##### Hardware PM State Control + +``` + | Hardware PM State Control |If set to Disable, hardware ^| + | |will choose a P-state *| + | Hardware P-States [Disable] |setting for the system *| + | |based on an OS request. *| + | |If set to Native Mode, *| + | |hardware will choose a *| + | |P-state setting based on OS *| + | |guidance. *| + | |If set to Native Mode with *| + | |No Legacy Support, hardware *| + | |will choose a P-state *| + | |setting independently *| + | |without OS guidance. +| + | |If set to Out of Band Mode, +| + | |hardware autonomously v| +``` + +##### CPU C State Control + +``` + | CPU C State Control |Select Enable to support | + | |Autonomous Core C-State | + | Autonomous Core C-State [Disable] |control which will allow | + | CPU C6 report [Disable] |the processor core to | + | Enhanced Halt State (C1E) [Disable] |control its C-State setting | + | |automatically and | + | |independently. | +``` + +##### Package C State Control + +``` + | Package C State Control |Limit the lowest package | + | |level C-State to | + | Package C State [C0/C1 state] |processors. Lower package | + | |C-State lower processor | + | |power consumption upon idle. | +``` + +##### CPU T State Control + +``` + | CPU T State Control |Enable/Disable CPU | + | |throttling by OS. | + | Software Controlled T-States [Disable] |Throttling reduces power | + | |consumption | +``` + +#### Chipset Configuration + +``` + | WARNING: Setting wrong values in below sections may cause |North Bridge Parameters | + | system to malfunction. | | + |> North Bridge | | + |> South Bridge | | +``` + +##### North Bridge + +``` + |> UPI Configuration |Displays and provides | + |> Memory Configuration |option to change the UPI | + |> IIO Configuration |Settings | +``` + +##### UPI Configuration + +``` + | UPI Configuration |Use this feature to select | + | -------------------------------------------------- |the degrading precedence | + | Number of CPU 2 |option for Ultra Path | + | Number of Active UPI Link 3 |Interconnect connections. | + | Current UPI Link Speed Fast |Select Topology Precedent | + | Current UPI Link Frequency 10.4 GT/s |to degrade UPI features if | + | UPI Global MMIO Low Base / Limit 90000000 / FBFFFFFF |system options are in | + | UPI Global MMIO High Base / Limit 0000000000000000 / |conflict. Select Feature | + | 00000000FFFFFFFF |Precedent to degrade UPI | + | UPI Pci-e Configuration Base / Size 80000000 / 10000000 |topology if system options | + | Degrade Precedence [Topology Precedence] |are in conflict. | + | Link L0p Enable [Disable] | | + | Link L1 Enable [Disable] | | + | IO Directory Cache (IODC) [Auto] | | + | SNC [Disable] | | + | XPT Prefetch [Disable] | | + | KTI Prefetch [Enable] |-----------------------------| + | Local/Remote Threshold [Auto] |><: Select Screen | + | Stale AtoS [Auto] |^v: Select Item | + | LLC Dead Line Alloc [Enable] |Enter: Select | + | Isoc Mode [Auto] |+/-: Change Opt. | +``` + +##### Memory Configuration + +``` + | |Select POR to enforce POR | + | -------------------------------------------------- |restrictions for DDR4 | + | Integrated Memory Controller (iMC) |frequency and voltage | + | -------------------------------------------------- |programming | + | | | + | Enforce POR [POR] | | + | PPR Type [Hard PPR] | | + | Enhanced PPR [Disable] | | + | Operation Mode [Test and Repair] | | + | Memory Frequency [2933] | | + | Data Scrambling for DDR4 [Auto] | | + | tCCD_L Relaxation [Auto] | | + | tRWSR Relaxation [Disable] | | + | tRFC Optimization for 16Gb Based DIMM [Force 550ns] | | + | 2x Refresh [Auto] | | + | Page Policy [Auto] | | + | IMC Interleaving [2-way Interleave] |-----------------------------| + |> Memory Topology |><: Select Screen | + |> Memory RAS Configuration |^v: Select Item | +``` + +##### IIO Configuration + +``` + | IIO Configuration |Expose IIO DFX devices and | + | -------------------------------------------------- |other CPU devices like PMON | + | | | + | EV DFX Features [Disable] | | + |> CPU1 Configuration | | + |> CPU2 Configuration | | + |> IOAT Configuration | | + |> Intel. VT for Directed I/O (VT-d) | | + |> Intel. VMD technology | | + | | | + | IIO-PCIE Express Global Options | | + | ======================================== | | + | PCI-E Completion Timeout Disable [No] | | +``` + +##### CPU1 Configuration + +``` + | IOU0 (IIO PCIe Br1) [Auto] |Selects PCIe port | + | IOU1 (IIO PCIe Br2) [Auto] |Bifurcation for selected | + | IOU2 (IIO PCIe Br3) [Auto] |slot(s) | + |> CPU1 SLOT2 PCI-E 3.0 X16 | | + |> CPU1 SLOT4 PCI-E 3.0 X16 | | + |> CPU1 SLOT9 PCI-E 3.0 X16 | | +``` + +##### CPU2 Configuration + +``` + | IOU0 (IIO PCIe Br1) [Auto] |Selects PCIe port | + | IOU1 (IIO PCIe Br2) [Auto] |Bifurcation for selected | + | IOU2 (IIO PCIe Br3) [Auto] |slot(s) | + |> CPU2 SLOT6 PCI-E 3.0 X16 | | + |> CPU2 SLOT8 PCI-E 3.0 X16 | | + |> CPU2 SLOT10 PCI-E 3.0 X16 | | +``` + +#### South Bridge + +``` + | |Enables Legacy USB support. | + | USB Module Version 21 |AUTO option disables legacy | + | |support if no USB devices | + | USB Devices: |are connected. DISABLE | + | 1 Keyboard, 1 Mouse, 1 Hub |option will keep USB | + | |devices available only for | + | Legacy USB Support [Enabled] |EFI applications. | + | XHCI Hand-off [Enabled] | | + | Port 60/64 Emulation [Enabled] | | + | PCIe PLL SSC [Disable] | | + | Real USB Wake Up [Enabled] | | + | Front USB Wake Up [Enabled] | | + | | | + | Azalia [Auto] | | + | Azalia PME Enable [Disabled] | | +``` + +### PCIe/PCI/PnP Configuration + +``` + | PCI Bus Driver Version A5.01.18 ^|Enables or Disables 64bit | + | *|capable Devices to be | + | PCI Devices Common Settings: *|Decoded in Above 4G Address | + | Above 4G Decoding [Enabled] *|Space (Only if System | + | SR-IOV Support [Enabled] *|Supports 64 bit PCI | + | ARI Support [Enabled] *|Decoding). | + | MMIO High Base [56T] *| | + | MMIO High Granularity Size [256G] *| | + | Maximum Read Request [Auto] *| | + | MMCFG Base [2G] *| | + | NVMe Firmware Source [Vendor Defined *| | + | Firmware] *| | + | VGA Priority [Onboard] *| | + | CPU1 SLOT2 PCI-E 3.0 X16 OPROM [Legacy] *| | + | CPU1 SLOT4 PCI-E 3.0 X16 OPROM [Legacy] *| | + | CPU2 SLOT6 PCI-E 3.0 X16 OPROM [Legacy] *| | + | CPU2 SLOT8 PCI-E 3.0 X16 OPROM [Legacy] *|-----------------------------| + | CPU1 SLOT9 PCI-E 3.0 X16 OPROM [Legacy] *|><: Select Screen | + | CPU2 SLOT10 PCI-E 3.0 X16 OPROM [Legacy] *|^v: Select Item | + | CPU2 SLOT11 PCI-E 3.0 X4(IN X8) OPROM [Legacy] *|Enter: Select | + | M.2 CONNECTOR OPROM [Legacy] *|+/-: Change Opt. | + | Bus Master Enable [Enabled] +|F1: General Help | + | Onboard LAN1 Option ROM [Legacy] +|F2: Previous Values | + | Onboard LAN2 Option ROM [Disabled] +|F3: Optimized Defaults | + | Onboard Video Option ROM [Legacy] v|F4: Save & Exit | + |> Network Stack Configuration | | +``` + +### ACPI Settings + +``` + | ACPI Settings |Enable or Disable Non | + | |uniform Memory Access | + | NUMA [Enabled] |(NUMA). | + | WHEA Support [Enabled] | | + | High Precision Event Timer [Enabled] | | +``` + +## Xeon CLX Server BIOS Configuration - DUT + +### Boot Feature + +``` + | Quiet Boot [Enabled] |Boot option | + | | | + | Option ROM Messages [Force BIOS] | | + | Bootup NumLock State [On] | | + | Wait For "F1" If Error [Enabled] | | + | INT19 Trap Response [Immediate] | | + | Re-try Boot [Disabled] | | + | Install Windows 7 USB support [Disabled] | | + | Port 61h Bit-4 Emulation [Disabled] | | + | | | + | Power Configuration | | + | Watch Dog Function [Disabled] | | + | Restore on AC Power Loss [Last State] | | + | Power Button Function [Instant Off] | | +``` + +### CPU Configuration + +``` + |--------------------------------------------------------------------+-----------------------------\ + | Processor Configuration ^|Enables Hyper Threading | + | -------------------------------------------------- *|(Software Method to | + | Processor BSP Revision 50657 - CLX B1 *|Enable/Disable Logical | + | Processor Socket CPU1 | CPU2 *|Processor threads. | + | Processor ID 00050657* | 00050657 *| | + | Processor Frequency 2.300GHz | 2.300GHz *| | + | Processor Max Ratio 17H | 17H *| | + | Processor Min Ratio 0AH | 0AH *| | + | Microcode Revision 0500002C | 0500002C *| | + | L1 Cache RAM 64KB | 64KB *| | + | L2 Cache RAM 1024KB | 1024KB *| | + | L3 Cache RAM 36608KB | 36608KB *| | + | Processor 0 Version *| | + | Intel(R) Xeon(R) Gold 6252N CPU @ 2.30GHz *| | + | Processor 1 Version *| | + | Intel(R) Xeon(R) Gold 6252N CPU @ 2.30GHz *| | + | *|-----------------------------| + | Hyper-Threading [ALL] [Enable] *|><: Select Screen | + | Cores Enabled 0 *|^v: Select Item | + | Monitor/Mwait [Auto] *|Enter: Select | + | Execute Disable Bit [Enable] +|+/-: Change Opt. | + | Intel Virtualization Technology [Enable] +|F1: General Help | + | PPIN Control [Unlock/Enable] +|F2: Previous Values | + | Hardware Prefetcher [Enable] +|F3: Optimized Defaults | + | Adjacent Cache Prefetch [Enable] v|F4: Save & Exit | + | DCU Streamer Prefetcher [Enable] | | + | DCU IP Prefetcher [Enable] | | + | LLC Prefetch [Disable] | | + | Extended APIC [Disable] | | + | AES-NI [Enable] | | + |> Advanced Power Management Configuration | | +``` + +#### Advanced Power Management Configuration + +``` + | Advanced Power Management Configuration |Switch CPU Power Management | + | -------------------------------------------------- |profile | + | Power Technology [Custom] | | + | Power Performance Tuning [BIOS Controls EPB] | | + | ENERGY_PERF_BIAS_CFG mode [Maximum Performance] | | + |> CPU P State Control | | + |> Hardware PM State Control | | + |> CPU C State Control | | + |> Package C State Control | | + |> CPU T State Control | | +``` + +##### CPU P State Control + +``` + | CPU P State Control |EIST allows the processor | + | |to dynamically adjust | + | SpeedStep (P-States) [Disable] |frequency and voltage based | + | Activate PBF [Disable] |on power versus performance | + | Configure PBF [Enable] |needs. | + | EIST PSD Function [HW_ALL] | | +``` + +##### Hardware PM State Control + +``` + | Hardware PM State Control |If set to Disable, hardware ^| + | |will choose a P-state *| + | Hardware P-States [Disable] |setting for the system *| + | |based on an OS request. *| + | |If set to Native Mode, *| + | |hardware will choose a *| + | |P-state setting based on OS *| + | |guidance. *| + | |If set to Native Mode with *| + | |No Legacy Support, hardware *| + | |will choose a P-state *| + | |setting independently *| + | |without OS guidance. +| + | |If set to Out of Band Mode, +| + | |hardware autonomously v| +``` + +##### CPU C State Control + +``` + | CPU C State Control |Select Enable to support | + | |Autonomous Core C-State | + | Autonomous Core C-State [Disable] |control which will allow | + | CPU C6 report [Disable] |the processor core to | + | Enhanced Halt State (C1E) [Disable] |control its C-State setting | + | |automatically and | + | |independently. | +``` + +##### Package C State Control + +``` + | Package C State Control |Limit the lowest package | + | |level C-State to | + | Package C State [C0/C1 state] |processors. Lower package | + | |C-State lower processor | + | |power consumption upon idle. | +``` + +##### CPU T State Control + +``` + | CPU T State Control |Enable/Disable CPU | + | |throttling by OS. | + | Software Controlled T-States [Disable] |Throttling reduces power | + | |consumption | +``` + +#### Chipset Configuration + +``` + | WARNING: Setting wrong values in below sections may cause |North Bridge Parameters | + | system to malfunction. | | + |> North Bridge | | + |> South Bridge | | +``` + +##### North Bridge + +``` + |> UPI Configuration |Displays and provides | + |> Memory Configuration |option to change the UPI | + |> IIO Configuration |Settings | +``` + +##### UPI Configuration + +``` + | UPI Configuration |Use this feature to select | + | -------------------------------------------------- |the degrading precedence | + | Number of CPU 2 |option for Ultra Path | + | Number of Active UPI Link 3 |Interconnect connections. | + | Current UPI Link Speed Fast |Select Topology Precedent | + | Current UPI Link Frequency 10.4 GT/s |to degrade UPI features if | + | UPI Global MMIO Low Base / Limit 90000000 / FBFFFFFF |system options are in | + | UPI Global MMIO High Base / Limit 0000000000000000 / |conflict. Select Feature | + | 00000000FFFFFFFF |Precedent to degrade UPI | + | UPI Pci-e Configuration Base / Size 80000000 / 10000000 |topology if system options | + | Degrade Precedence [Topology Precedence] |are in conflict. | + | Link L0p Enable [Disable] | | + | Link L1 Enable [Disable] | | + | IO Directory Cache (IODC) [Auto] | | + | SNC [Disable] | | + | XPT Prefetch [Disable] | | + | KTI Prefetch [Enable] |-----------------------------| + | Local/Remote Threshold [Auto] |><: Select Screen | + | Stale AtoS [Auto] |^v: Select Item | + | LLC Dead Line Alloc [Enable] |Enter: Select | + | Isoc Mode [Auto] |+/-: Change Opt. | +``` + +##### Memory Configuration + +``` + | |Select POR to enforce POR | + | -------------------------------------------------- |restrictions for DDR4 | + | Integrated Memory Controller (iMC) |frequency and voltage | + | -------------------------------------------------- |programming | + | | | + | Enforce POR [POR] | | + | PPR Type [Hard PPR] | | + | Enhanced PPR [Disable] | | + | Operation Mode [Test and Repair] | | + | Memory Frequency [2933] | | + | Data Scrambling for DDR4 [Auto] | | + | tCCD_L Relaxation [Auto] | | + | tRWSR Relaxation [Disable] | | + | tRFC Optimization for 16Gb Based DIMM [Force 550ns] | | + | 2x Refresh [Auto] | | + | Page Policy [Auto] | | + | IMC Interleaving [2-way Interleave] |-----------------------------| + |> Memory Topology |><: Select Screen | + |> Memory RAS Configuration |^v: Select Item | +``` + +##### IIO Configuration + +``` + | IIO Configuration |Expose IIO DFX devices and | + | -------------------------------------------------- |other CPU devices like PMON | + | | | + | EV DFX Features [Disable] | | + |> CPU1 Configuration | | + |> CPU2 Configuration | | + |> IOAT Configuration | | + |> Intel. VT for Directed I/O (VT-d) | | + |> Intel. VMD technology | | + | | | + | IIO-PCIE Express Global Options | | + | ======================================== | | + | PCI-E Completion Timeout Disable [No] | | +``` + +##### CPU1 Configuration + +``` + | IOU0 (IIO PCIe Br1) [Auto] |Selects PCIe port | + | IOU1 (IIO PCIe Br2) [Auto] |Bifurcation for selected | + | IOU2 (IIO PCIe Br3) [Auto] |slot(s) | + |> CPU1 SLOT2 PCI-E 3.0 X16 | | + |> CPU1 SLOT4 PCI-E 3.0 X16 | | + |> CPU1 SLOT9 PCI-E 3.0 X16 | | +``` + +##### CPU2 Configuration + +``` + | IOU0 (IIO PCIe Br1) [Auto] |Selects PCIe port | + | IOU1 (IIO PCIe Br2) [Auto] |Bifurcation for selected | + | IOU2 (IIO PCIe Br3) [Auto] |slot(s) | + |> CPU2 SLOT6 PCI-E 3.0 X16 | | + |> CPU2 SLOT8 PCI-E 3.0 X16 | | + |> CPU2 SLOT10 PCI-E 3.0 X16 | | +``` + +#### South Bridge + +``` + | |Enables Legacy USB support. | + | USB Module Version 21 |AUTO option disables legacy | + | |support if no USB devices | + | USB Devices: |are connected. DISABLE | + | 1 Keyboard, 1 Mouse, 1 Hub |option will keep USB | + | |devices available only for | + | Legacy USB Support [Enabled] |EFI applications. | + | XHCI Hand-off [Enabled] | | + | Port 60/64 Emulation [Enabled] | | + | PCIe PLL SSC [Disable] | | + | Real USB Wake Up [Enabled] | | + | Front USB Wake Up [Enabled] | | + | | | + | Azalia [Auto] | | + | Azalia PME Enable [Disabled] | | +``` + +### PCIe/PCI/PnP Configuration + +``` + | PCI Bus Driver Version A5.01.18 ^|Enables or Disables 64bit | + | *|capable Devices to be | + | PCI Devices Common Settings: *|Decoded in Above 4G Address | + | Above 4G Decoding [Enabled] *|Space (Only if System | + | SR-IOV Support [Enabled] *|Supports 64 bit PCI | + | ARI Support [Enabled] *|Decoding). | + | MMIO High Base [56T] *| | + | MMIO High Granularity Size [256G] *| | + | Maximum Read Request [Auto] *| | + | MMCFG Base [2G] *| | + | NVMe Firmware Source [Vendor Defined *| | + | Firmware] *| | + | VGA Priority [Onboard] *| | + | CPU1 SLOT2 PCI-E 3.0 X16 OPROM [Legacy] *| | + | CPU1 SLOT4 PCI-E 3.0 X16 OPROM [Legacy] *| | + | CPU2 SLOT6 PCI-E 3.0 X16 OPROM [Legacy] *| | + | CPU2 SLOT8 PCI-E 3.0 X16 OPROM [Legacy] *|-----------------------------| + | CPU1 SLOT9 PCI-E 3.0 X16 OPROM [Legacy] *|><: Select Screen | + | CPU2 SLOT10 PCI-E 3.0 X16 OPROM [Legacy] *|^v: Select Item | + | CPU2 SLOT11 PCI-E 3.0 X4(IN X8) OPROM [Legacy] *|Enter: Select | + | M.2 CONNECTOR OPROM [Legacy] *|+/-: Change Opt. | + | Bus Master Enable [Enabled] +|F1: General Help | + | Onboard LAN1 Option ROM [Legacy] +|F2: Previous Values | + | Onboard LAN2 Option ROM [Disabled] +|F3: Optimized Defaults | + | Onboard Video Option ROM [Legacy] v|F4: Save & Exit | + |> Network Stack Configuration | | +``` + +### ACPI Settings + +``` + | ACPI Settings |Enable or Disable Non | + | |uniform Memory Access | + | NUMA [Enabled] |(NUMA). | + | WHEA Support [Enabled] | | + | High Precision Event Timer [Enabled] | | +``` + +## Linux cmdline + +``` +$ cat /proc/cmdline +BOOT_IMAGE=/boot/vmlinuz-5.15.0-46-generic root=UUID=2d6f4d44-76b1-4343-bc73-c066a3e95b32 ro audit=0 default_hugepagesz=2M hugepagesz=1G hugepages=32 hugepagesz=2M hugepages=32768 hpet=disable intel_idle.max_cstate=1 intel_iommu=on intel_pstate=disable iommu=pt isolcpus=1-23,25-47,49-71,73-95 mce=off nmi_watchdog=0 nohz_full=1-23,25-47,49-71,73-95 nosoftlockup numa_balancing=disable processor.max_cstate=1 rcu_nocbs=1-23,25-47,49-71,73-95 tsc=reliable console=ttyS0,115200n8 quiet +``` + +## Xeon Clx Server Firmware Inventory + +``` +Host. IPMI IP. BMC. BIOS. CPLD. CPU Microcode. PCI Bus. X710 Firmware. XXV710 Firmware. i40e. CX-5 Firmware. mlx5_core E810 Firmware. ice. +s33-t27-sut1. 10.30.55.18. 1.67. 3.2. 03.B1.05. 0500002C. A5.01.18. 9.20. 9.20. 2.22.20. 16.35.2000. 23.07-0.5.0. 4.30. 1.12.6. +s34-t27-tg1. 10.30.55.19. 1.67. 3.2. 03.B1.05. 0500002C. A5.01.18. 8.30. 8.30. 2.19.3. 16.32.1010. 5.9-0.5.5. N/A. N/A. +s35-t28-sut1. 10.30.55.20. 1.67. 3.2. 03.B1.05. 0500002C. A5.01.18. 9.20. 9.20. 2.22.20. 16.35.2000. 23.07-0.5.0. 4.30. 1.12.6. +s36-t28-tg1. 10.30.55.21. 1.67. 3.2. 03.B1.05. 0500002C. A5.01.18. 8.30. 8.30. 2.19.3. 16.32.1010. 5.9-0.5.5. N/A. N/A. +s37-t29-sut1. 10.30.55.22. 1.67. 3.2. 03.B1.05. 0500002C. A5.01.18. 9.20. 9.20. 2.22.20. 16.35.2000. 23.07-0.5.0. 4.30. 1.12.6. +s38-t29-tg1. 10.30.55.23. 1.67. 3.2. 03.B1.05. 0500002C. A5.01.18. 8.30. 8.30. 2.19.3. 16.32.1010. 5.9-0.5.5. N/A. N/A. +``` diff --git a/docs/content/infrastructure/testbed_configuration/sm_icx_hw_bios_cfg.md b/docs/content/infrastructure/testbed_configuration/sm_icx_hw_bios_cfg.md new file mode 100644 index 0000000000..9a5fe51b3c --- /dev/null +++ b/docs/content/infrastructure/testbed_configuration/sm_icx_hw_bios_cfg.md @@ -0,0 +1,1121 @@ +--- +bookToc: true +title: "SuperMicro Icelake" +--- + +# SuperMicro Icelake + +## Linux lscpu + +``` +$ lscpu +Architecture: x86_64 +CPU op-mode(s): 32-bit, 64-bit +Byte Order: Little Endian +Address sizes: 46 bits physical, 57 bits virtual +CPU(s): 128 +On-line CPU(s) list: 0-127 +Thread(s) per core: 2 +Core(s) per socket: 32 +Socket(s): 2 +NUMA node(s): 2 +Vendor ID: GenuineIntel +CPU family: 6 +Model: 106 +Model name: Intel(R) Xeon(R) Platinum 8358 CPU @ 2.60GHz +Stepping: 6 +CPU MHz: 3283.980 +BogoMIPS: 5200.00 +Virtualization: VT-x +L1d cache: 3 MiB +L1i cache: 2 MiB +L2 cache: 80 MiB +L3 cache: 96 MiB +NUMA node0 CPU(s): 0-31,64-95 +NUMA node1 CPU(s): 32-63,96-127 +Vulnerability Itlb multihit: Not affected +Vulnerability L1tf: Not affected +Vulnerability Mds: Not affected +Vulnerability Meltdown: Not affected +Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp +Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization +Vulnerability Spectre v2: Mitigation; Enhanced IBRS, IBPB conditional, RSB filling +Vulnerability Srbds: Not affected +Vulnerability Tsx async abort: Not affected +Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe sysca + ll nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmu + lqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadl + ine_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 invpcid_single ssbd mba ibrs ibpb stibp + ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a avx + 512f avx512dq rdseed adx smap avx512ifma clflushopt clwb intel_pt avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 x + saves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local wbnoinvd dtherm ida arat pln pts avx512vbmi umip pku ospke avx512_vbm + i2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg tme avx512_vpopcntdq rdpid md_clear pconfig flush_l1d arch_capabilities +``` + +## Linux dmidecode + +``` +# dmidecode 3.2 +Getting SMBIOS data from sysfs. +SMBIOS 3.3.0 present. +# SMBIOS implementations newer than version 3.2.0 are not +# fully supported by this version of dmidecode. +Table at 0x6BAEE000. + +Handle 0x0000, DMI type 0, 26 bytes +BIOS Information + Vendor: American Megatrends International, LLC. + Version: 1.1 + Release Date: 04/09/2021 + Address: 0xF0000 + Runtime Size: 64 kB + ROM Size: 32 MB + Characteristics: + PCI is supported + BIOS is upgradeable + BIOS shadowing is allowed + Boot from CD is supported + Selectable boot is supported + BIOS ROM is socketed + EDD is supported + Japanese floppy for NEC 9800 1.2 MB is supported (int 13h) + Japanese floppy for Toshiba 1.2 MB is supported (int 13h) + 5.25"/360 kB floppy services are supported (int 13h) + 5.25"/1.2 MB floppy services are supported (int 13h) + 3.5"/720 kB floppy services are supported (int 13h) + 3.5"/2.88 MB floppy services are supported (int 13h) + Print screen service is supported (int 5h) + Serial services are supported (int 14h) + Printer services are supported (int 17h) + CGA/mono video services are supported (int 10h) + USB legacy is supported + BIOS boot specification is supported + Targeted content distribution is supported + UEFI is supported + BIOS Revision: 5.22 + +Handle 0x0001, DMI type 1, 27 bytes +System Information + Manufacturer: Supermicro + Product Name: SYS-740GP-TNRT + Version: 0123456789 + Serial Number: S424016X1B00510 + UUID: 0698ae00-2383-11ec-8000-3cecefb9a6ba + Wake-up Type: Power Switch + SKU Number: To be filled by O.E.M. + Family: Family + +Handle 0x0002, DMI type 2, 15 bytes +Base Board Information + Manufacturer: Supermicro + Product Name: X12DPG-QT6 + Version: 1.00 + Serial Number: UM219S003392 + Asset Tag: Base Board Asset Tag + Features: + Board is a hosting board + Board is replaceable + Location In Chassis: Part Component + Chassis Handle: 0x0003 + Type: Motherboard + Contained Object Handles: 0 + +Handle 0x0003, DMI type 3, 22 bytes +Chassis Information + Manufacturer: Supermicro + Type: Other + Lock: Not Present + Version: 0123456789 + Serial Number: C7470KK25P50098 + Asset Tag: Chassis Asset Tag + Boot-up State: Safe + Power Supply State: Safe + Thermal State: Safe + Security Status: None + OEM Information: 0x00000000 + Height: Unspecified + Number Of Power Cords: 1 + Contained Elements: 0 + SKU Number: To be filled by O.E.M. + +Handle 0x001B, DMI type 38, 18 bytes +IPMI Device Information + Interface Type: KCS (Keyboard Control Style) + Specification Version: 2.0 + I2C Slave Address: 0x10 + NV Storage Device: Not Present + Base Address: 0x0000000000000CA2 (I/O) + Register Spacing: Successive Byte Boundaries + +Handle 0x002A, DMI type 4, 48 bytes +Processor Information + Socket Designation: CPU1 + Type: Central Processor + Family: Xeon + Manufacturer: Intel(R) Corporation + ID: A6 06 06 00 FF FB EB BF + Signature: Type 0, Family 6, Model 106, Stepping 6 + Flags: + FPU (Floating-point unit on-chip) + VME (Virtual mode extension) + DE (Debugging extension) + PSE (Page size extension) + TSC (Time stamp counter) + MSR (Model specific registers) + PAE (Physical address extension) + MCE (Machine check exception) + CX8 (CMPXCHG8 instruction supported) + APIC (On-chip APIC hardware supported) + SEP (Fast system call) + MTRR (Memory type range registers) + PGE (Page global enable) + MCA (Machine check architecture) + CMOV (Conditional move instruction supported) + PAT (Page attribute table) + PSE-36 (36-bit page size extension) + CLFSH (CLFLUSH instruction supported) + DS (Debug store) + ACPI (ACPI supported) + MMX (MMX technology supported) + FXSR (FXSAVE and FXSTOR instructions supported) + SSE (Streaming SIMD extensions) + SSE2 (Streaming SIMD extensions 2) + SS (Self-snoop) + HTT (Multi-threading) + TM (Thermal monitor supported) + PBE (Pending break enabled) + Version: Intel(R) Xeon(R) Platinum 8358 CPU @ 2.60GHz + Voltage: 1.6 V + External Clock: 100 MHz + Max Speed: 4500 MHz + Current Speed: 2600 MHz + Status: Populated, Enabled + Upgrade: <OUT OF SPEC> + L1 Cache Handle: 0x0027 + L2 Cache Handle: 0x0028 + L3 Cache Handle: 0x0029 + Serial Number: Not Specified + Asset Tag: UNKNOWN + Part Number: Not Specified + Core Count: 32 + Core Enabled: 32 + Thread Count: 64 + Characteristics: + 64-bit capable + Multi-Core + Hardware Thread + Execute Protection + Enhanced Virtualization + Power/Performance Control + +Handle 0x002E, DMI type 4, 48 bytes +Processor Information + Socket Designation: CPU2 + Type: Central Processor + Family: Xeon + Manufacturer: Intel(R) Corporation + ID: A6 06 06 00 FF FB EB BF + Signature: Type 0, Family 6, Model 106, Stepping 6 + Flags: + FPU (Floating-point unit on-chip) + VME (Virtual mode extension) + DE (Debugging extension) + PSE (Page size extension) + TSC (Time stamp counter) + MSR (Model specific registers) + PAE (Physical address extension) + MCE (Machine check exception) + CX8 (CMPXCHG8 instruction supported) + APIC (On-chip APIC hardware supported) + SEP (Fast system call) + MTRR (Memory type range registers) + PGE (Page global enable) + MCA (Machine check architecture) + CMOV (Conditional move instruction supported) + PAT (Page attribute table) + PSE-36 (36-bit page size extension) + CLFSH (CLFLUSH instruction supported) + DS (Debug store) + ACPI (ACPI supported) + MMX (MMX technology supported) + FXSR (FXSAVE and FXSTOR instructions supported) + SSE (Streaming SIMD extensions) + SSE2 (Streaming SIMD extensions 2) + SS (Self-snoop) + HTT (Multi-threading) + TM (Thermal monitor supported) + PBE (Pending break enabled) + Version: Intel(R) Xeon(R) Platinum 8358 CPU @ 2.60GHz + Voltage: 1.6 V + External Clock: 100 MHz + Max Speed: 4500 MHz + Current Speed: 2600 MHz + Status: Populated, Enabled + Upgrade: <OUT OF SPEC> + L1 Cache Handle: 0x002B + L2 Cache Handle: 0x002C + L3 Cache Handle: 0x002D + Serial Number: Not Specified + Asset Tag: UNKNOWN + Part Number: Not Specified + Core Count: 32 + Core Enabled: 32 + Thread Count: 64 + Characteristics: + 64-bit capable + Multi-Core + Hardware Thread + Execute Protection + Enhanced Virtualization + Power/Performance Control +``` + +## Linux dmidecode memory + +``` +Handle 0x0034, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0033 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P1-DIMMA1 + Bank Locator: P0_Node0_Channel0_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: H0MK0001304705C2E3 + Asset Tag: P1-DIMMA1_AssetTag (date:21/30) + Part Number: M393A2K43DB3-CWE + Rank: 2 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 16 GB + Cache Size: None + Logical Size: None + +Handle 0x0036, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0033 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P1-DIMMB1 + Bank Locator: P0_Node0_Channel1_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: H0MK0001304705CE60 + Asset Tag: P1-DIMMB1_AssetTag (date:21/30) + Part Number: M393A2K43DB3-CWE + Rank: 2 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 16 GB + Cache Size: None + Logical Size: None + +Handle 0x0038, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0033 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P1-DIMMC1 + Bank Locator: P0_Node0_Channel2_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: H0MK0001304705C59E + Asset Tag: P1-DIMMC1_AssetTag (date:21/30) + Part Number: M393A2K43DB3-CWE + Rank: 2 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 16 GB + Cache Size: None + Logical Size: None + +Handle 0x003A, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0033 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P1-DIMMD1 + Bank Locator: P0_Node0_Channel3_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: H0MK0001304705D12D + Asset Tag: P1-DIMMD1_AssetTag (date:21/30) + Part Number: M393A2K43DB3-CWE + Rank: 2 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 16 GB + Cache Size: None + Logical Size: None + +Handle 0x003C, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0033 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P1-DIMME1 + Bank Locator: P0_Node1_Channel0_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: H0MK0001304705C69C + Asset Tag: P1-DIMME1_AssetTag (date:21/30) + Part Number: M393A2K43DB3-CWE + Rank: 2 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 16 GB + Cache Size: None + Logical Size: None + +Handle 0x003E, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0033 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P1-DIMMF1 + Bank Locator: P0_Node1_Channel1_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: H0MK0001304705C33A + Asset Tag: P1-DIMMF1_AssetTag (date:21/30) + Part Number: M393A2K43DB3-CWE + Rank: 2 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 16 GB + Cache Size: None + Logical Size: None + +Handle 0x0040, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0033 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P1-DIMMG1 + Bank Locator: P0_Node1_Channel2_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: H0MK0001304705C59F + Asset Tag: P1-DIMMG1_AssetTag (date:21/30) + Part Number: M393A2K43DB3-CWE + Rank: 2 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 16 GB + Cache Size: None + Logical Size: None + +Handle 0x0042, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0033 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P1-DIMMH1 + Bank Locator: P0_Node1_Channel3_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: H0MK0001304705CA16 + Asset Tag: P1-DIMMH1_AssetTag (date:21/30) + Part Number: M393A2K43DB3-CWE + Rank: 2 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 16 GB + Cache Size: None + Logical Size: None + +Handle 0x0044, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0033 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P2-DIMMA1 + Bank Locator: P1_Node0_Channel0_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: H0MK0001304705CBFE + Asset Tag: P2-DIMMA1_AssetTag (date:21/30) + Part Number: M393A2K43DB3-CWE + Rank: 2 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 16 GB + Cache Size: None + Logical Size: None + +Handle 0x0046, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0033 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P2-DIMMB1 + Bank Locator: P1_Node0_Channel1_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: H0MK0001304705CFC8 + Asset Tag: P2-DIMMB1_AssetTag (date:21/30) + Part Number: M393A2K43DB3-CWE + Rank: 2 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 16 GB + Cache Size: None + Logical Size: None + +Handle 0x0048, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0033 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P2-DIMMC1 + Bank Locator: P1_Node0_Channel2_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: H0MK0001304705CC02 + Asset Tag: P2-DIMMC1_AssetTag (date:21/30) + Part Number: M393A2K43DB3-CWE + Rank: 2 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 16 GB + Cache Size: None + Logical Size: None + +Handle 0x004A, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0033 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P2-DIMMD1 + Bank Locator: P1_Node0_Channel3_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: H0MK0001304705CB5A + Asset Tag: P2-DIMMD1_AssetTag (date:21/30) + Part Number: M393A2K43DB3-CWE + Rank: 2 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 16 GB + Cache Size: None + Logical Size: None + +Handle 0x004C, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0033 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P2-DIMME1 + Bank Locator: P1_Node1_Channel0_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: H0MK0001304705CB30 + Asset Tag: P2-DIMME1_AssetTag (date:21/30) + Part Number: M393A2K43DB3-CWE + Rank: 2 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 16 GB + Cache Size: None + Logical Size: None + +Handle 0x004E, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0033 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P2-DIMMF1 + Bank Locator: P1_Node1_Channel1_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: H0MK0001304705CB87 + Asset Tag: P2-DIMMF1_AssetTag (date:21/30) + Part Number: M393A2K43DB3-CWE + Rank: 2 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 16 GB + Cache Size: None + Logical Size: None + +Handle 0x0050, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0033 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P2-DIMMG1 + Bank Locator: P1_Node1_Channel2_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: H0MK0001304705CB08 + Asset Tag: P2-DIMMG1_AssetTag (date:21/30) + Part Number: M393A2K43DB3-CWE + Rank: 2 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 16 GB + Cache Size: None + Logical Size: None + +Handle 0x0052, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0033 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P2-DIMMH1 + Bank Locator: P1_Node1_Channel3_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: H0MK0001304705CC01 + Asset Tag: P2-DIMMH1_AssetTag (date:21/30) + Part Number: M393A2K43DB3-CWE + Rank: 2 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 16 GB + Cache Size: None + Logical Size: None +``` + +## Xeon ICX Server BIOS Configuration + +### Boot Feature + +``` + | |Enables or disables Quiet | + | Quiet Boot [Enabled] |Boot option | + | | | + | Option ROM Messages [Force BIOS] | | + | Bootup NumLock State [On] | | + | Wait For "F1" If Error [Disabled] | | + | INT19 Trap Response [Immediate] | | + | Re-try Boot [Disabled] | | + | | | + | Power Configuration | | + | | | + | Watch Dog Function [Disabled] | | + | Restore on AC Power Loss [Last State] | | + | Power Button Function [Instant Off] | | + | Deep Sleep Mode [Disabled] | | +``` + +### CPU Configuration + +``` + | Processor Configuration ^|Enables Hyper Threading | + | -------------------------------------------------- *|(Software Method to | + | Processor BSP Revision 606A6 - ICX D2 *|Enable/Disable Logical | + | Processor Socket CPU1 CPU2 *|Processor threads. | + | Processor ID 000606A6* | 000606A6 *| | + | Processor Frequency 2.600GHz | 2.600GHz *| | + | Processor Max Ratio 1AH | 1AH *| | + | Processor Min Ratio 08H | 08H *| | + | Microcode Revision 0D000280 | 0D000280 *| | + | L1 Cache RAM(Per Core) 80KB | 80KB *| | + | L2 Cache RAM(Per Core) 1280KB | 1280KB *| | + | L3 Cache RAM(Per Package) 49152KB | 49152KB *| | + | Processor 0 Version Intel(R) Xeon(R) *| | + | Platinum 8358 CPU @ *| | + | 2.60GHz *| | + | Processor 1 Version Intel(R) Xeon(R) *| | + | Platinum 8358 CPU @ *| | + | 2.60GHz *| | + | +| | + |> CPU1 Core Disable Bitmap +| | + |> CPU2 Core Disable Bitmap +|-----------------------------| + | Hyper-Threading [ALL] [Enable] +|><: Select Screen | + | Hardware Prefetcher [Enable] +|^v: Select Item | + | Adjacent Cache Prefetch [Enable] +|Enter: Select | + | DCU Streamer Prefetcher [Enable] +|+/-: Change Opt. | + | DCU IP Prefetcher [Enable] +|F1: General Help | + | LLC Prefetch [Enable] +|F2: Previous Values | + | Extended APIC [Disable] +|F3: Optimized Defaults | + | VMX [Enable] v|F4: Save & Exit | + | Enable SMX [Disable] +| | + | PPIN Control [Unlock/Enable] *| | + | AES-NI [Enable] *| | + | -------------------------------------------------- *| | + | TME, TME-MT, TDX *| | + | -------------------------------------------------- *| | + | Total Memory Encryption (TME) [Disabled] *| | + | -------------------------------------------------- *|-----------------------------| + | Software Guard Extension (SGX) *|><: Select Screen | + | -------------------------------------------------- *|^v: Select Item | + | SGX Factory Reset [Disabled] *|Enter: Select | + | SW Guard Extensions (SGX) [Disabled] *|+/-: Change Opt. | + | SGX Package Info In-Band Access [Disabled] *|F1: General Help | + | -------------------------------------------------- *|F2: Previous Values | + | Limit CPU PA to 46 Bits [Enable] *|F3: Optimized Defaults | + |> Advanced Power Management Configuration v|F4: Save & Exit | +``` + +#### Advanced Power Management Configuration + +``` + | Advanced Power Management Configuration |Enable processor power | + | -------------------------------------------------- |management features. | + | Power Technology [Custom] | | + | Power Performance Tuning [BIOS Controls EPB] | | + | ENERGY_PERF_BIAS_CFG Mode [Maximum Performance] | | +``` + +##### CPU P State Control + +``` + | CPU P State Control |EIST allows the processor | + | |to dynamically adjust | + | SpeedStep (P-States) [Disable] |frequency and voltage based | + | Activate SST-BF [Disable] |on power versus performance | + | Configure SST-BF [Enable] |needs. | + | EIST PSD Function [HW_ALL] | | +``` + +##### Hardware PM State Control + +``` + | Hardware PM State Control |If set to Disable, hardware ^| + | |will choose a P-state *| + | Hardware P-States [Disable] |setting for the system *| + | |based on an OS request. *| + + | Frequency Prioritization |This knob controls whether | + | |RAPL balancer is enabled. | + | RAPL Prioritization [Disable] |When enabled it activates | +``` + +##### CPU C State Control + +``` + | CPU C State Control |Allows Monitor and MWAIT | + | |instructions. | + | Enable Monitor MWAIT [Enable] | | + | CPU C6 Report [Disable] | | + | Enhanced Halt State (C1E) [Disable] | | +``` + +##### Package C State Control + +``` + | Package C State Control |Limit the lowest package | + | |level C-State to | + | Package C State [C0/C1 state] |processors. Lower package | +``` + +##### CPU T State Control + +``` + | CPU T State Control |Enable/Disable CPU | + | |throttling by OS. | + | Software Controlled T-States [Disable] |Throttling reduces power | +``` + +##### UPI Configuration + +``` + | Uncore Configuration |Choose Topology Precedence | + | -------------------------------------------------- |to degrade features if | + | Number of CPU 2 |system options are in | + | Number of IIO 2 |conflict or choose Feature | + | Current UPI Link Speed Fast |Precedence to degrade | + | Current UPI Link Frequency 11.2 GT/s |topology if system options | + | Global MMIO Low Base / Limit 90000000 / FBFFFFFF |are in conflict. | + | Global MMIO High Base / Limit 0000200000000000 / | | + | 0000204FFFFFFFFF | | + | Pci-e Configuration Base / Size 80000000 / 10000000 | | + | Degrade Precedence [Topology Precedence] | | + | Link L0p Enable [Disable] | | + | Link L1 Enable [Disable] | | + | XPT Remote Prefetch [Auto] | | + | KTI Prefetch [Auto] |-----------------------------| + | Local/Remote Threshold [Auto] |><: Select Screen | + | IO Directory Cache (IODC) [Auto] |^v: Select Item | + | SNC (Sub NUMA) [Disable] |Enter: Select | + | XPT Prefetch [Auto] |+/-: Change Opt. | + | Snoop Throttle Configuration [Auto] |F1: General Help | + | PCIe Remote P2P Relaxed Ordering [Disable] |F2: Previous Values | + | Stale AtoS [Auto] |F3: Optimized Defaults | + | LLC Dead Line Alloc [Enable] |F4: Save & Exit | +``` + +##### Memory Configuration + +``` + | |Set Enable or Disable | + | -------------------------------------------------- |STEP(Samsung TestBIOS & | + | Integrated Memory Controller (iMC) |Enhanced PPR)function | + | -------------------------------------------------- | | + | | | + | STEP DRAM Test [Disable] | | + | Operation Mode [Test and Repair] | | + | Enforce POR [POR] | | + | PPR Type [Hard PPR] | | + | Memory Frequency [Auto] | | + | Data Scrambling for DDR4 [Enable] | | + | 2x Refresh Enable [Auto] | | +``` + +##### IIO Configuration + +``` + | IIO Configuration |Press <Enter> to bring up | + | -------------------------------------------------- |the Intel. Virtualization | + | |for Directed I/O (VT-d) | + |> CPU1 Configuration |Configuration menu. | + |> CPU2 Configuration | | + |> IOAT Configuration | | + |> Intel. VT for Directed I/O (VT-d) | | + |> Intel. VMD Technology | | + | PCI-E ASPM Support (Global) [Disable] | | + | IIO eDPC Support [Disable] | | +``` + +##### CPU1 Configuration + +``` + | IOU0 (IIO PCIe Port 1) [Auto] |Selects PCIe port | + | IOU1 (IIO PCIe Port 2) [Auto] |Bifurcation for selected | + | IOU3 (IIO PCIe Port 4) [Auto] |slot(s) | + | IOU4 (IIO PCIe Port 5) [Auto] | | +``` + +##### CPU2 Configuration + +``` + | IOU0 (IIO PCIe Port 1) [Auto] |Selects PCIe port | + | IOU1 (IIO PCIe Port 2) [Auto] |Bifurcation for selected | + | IOU3 (IIO PCIe Port 4) [Auto] |slot(s) | + | IOU4 (IIO PCIe Port 5) [Auto] | | +``` + +#### South Bridge + +``` + | USB Module Version 26 |AUTO option disables legacy | + | |support if no USB devices | + | USB Devices: |are connected. DISABLE | + | 1 Drive, 2 Keyboards, 2 Mice, 1 Hub |option will keep USB | + | |devices available only for | + | Legacy USB Support [Enabled] |EFI applications. | + | XHCI Hand-off [Enabled] | | + | Port 60/64 Emulation [Disabled] | | + | PCIe PLL SSC [Disabled] | | + | Port 61h Bit-4 Emulation [Disabled] | | +``` + +### PCIe/PCI/PnP Configuration + +``` + | PCI Bus Driver Version A5.01.24 ^|Enables or Disables 64bit | + | *|capable Devices to be | + | PCI Devices Common Settings: *|Decoded in Above 4G Address | + | Above 4G Decoding [Enabled] *|Space (Only if System | + | SR-IOV Support [Enabled] *|Supports 64 bit PCI | + | ARI Support [Enabled] *|Decoding). | + | Bus Master Enable [Enabled] *| | + | Consistent Device Name Support [Disabled] *| | + | MMIO High Base [32T] *| | + | MMIO High Granularity Size [64G] *| | + | Maximum Read Request [Auto] *| | + | MMCFG Base [Auto] *| | + | NVMe Firmware Source [Vendor Defined *| | + | Firmware] *| | + | VGA Priority [Onboard] *| | + | CPU1 SLOT2 PCI-E 3.0 X16 OPROM [EFI] *| | + | CPU1 SLOT4 PCI-E 3.0 X16 OPROM [EFI] *| | + | CPU2 SLOT6 PCI-E 3.0 X16 OPROM [EFI] *| | + | CPU2 SLOT8 PCI-E 3.0 X16 OPROM [EFI] *|-----------------------------| + | CPU1 SLOT9 PCI-E 3.0 X16 OPROM [EFI] *|><: Select Screen | + | CPU2 SLOT10 PCI-E 3.0 X16 OPROM [EFI] *|^v: Select Item | + | CPU2 SLOT11 PCI-E 3.0 X4(IN X8) OPROM [EFI] *|Enter: Select | + | M.2 CONNECTOR OPROM [EFI] *|+/-: Change Opt. | + | Bus Master Enable [Enabled] +|F1: General Help | + | Onboard LAN1 Option ROM [EFI] +|F2: Previous Values | + | Onboard LAN2 Option ROM [Disabled] +|F3: Optimized Defaults | + | Onboard Video Option ROM [EFI] v|F4: Save & Exit | + |> Network Stack Configuration | | +``` + +## Linux cmdline + +``` +$ cat /proc/cmdline +BOOT_IMAGE=/boot/vmlinuz-5.15.0-46-generic root=UUID=6ff26c8a-8c65-4025-a6e7-d97dee6025d0 ro audit=0 default_hugepagesz=2M hugepagesz=1G hugepages=32 hugepagesz=2M hugepages=32768 hpet=disable intel_idle.max_cstate=1 intel_iommu=on intel_pstate=disable iommu=pt isolcpus=1-31,33-63,65-95,97-127 mce=off nmi_watchdog=0 nohz_full=1-31,33-63,65-95,97-127 nosoftlockup numa_balancing=disable processor.max_cstate=1 rcu_nocbs=1-31,33-63,65-95,97-127 tsc=reliable console=ttyS0,115200n8 quiet +``` + +## Xeon ICX Server Firmware Inventory + +``` +Host. IPMI IP. BMC. BIOS. CPLD. CPU Microcode. PCI Bus. X710 Firmware. i40e. E810 Firmware. ice. CX-6 Firmware. mlx5_core +s65-t37-sut1. 10.30.50.75. 1.00.21. 1.1. F1.00.07. 0D000280. A5.01.24. 9.20. 2.22.20. 4.30. 1.12.6. 22.36.1010. 23.07-0.5.0. +s66-t37-sut2. 10.30.50.76. 1.00.21. 1.1. F1.00.07. 0D000280. A5.01.24. 9.20. 2.22.20. 4.30. 1.12.6. 22.36.1010. 23.07-0.5.0. +s67-t37-tg1. 10.30.50.77. 1.00.21. 1.1. F1.00.07. 0D000280. A5.01.24. 8.30. 2.19.3. 3.20. 1.9.7. 22.36.1010. 23.07-0.5.0. +s78-t38-sut1. 10.30.50.78. 1.00.21. 1.1. F1.00.07. 0D000280. A5.01.24. 9.20. 2.22.20. 4.30. 1.12.6. 22.36.1010. 23.07-0.5.0. +s79-t38-sut2. 10.30.50.79. 1.00.21. 1.1. F1.00.07. 0D000280. A5.01.24. 9.20. 2.22.20. 4.30. 1.12.6. 22.36.1010. 23.07-0.5.0. +s80-t38-tg1. 10.30.50.80. 1.00.21. 1.1. F1.00.07. 0D000280. A5.01.24. 8.30. 2.19.3. 3.20. 1.9.7. 22.36.1010. 23.07-0.5.0. +s71-t212-sut1. 10.30.50.81. 1.00.21. 1.1. F1.00.07. 0D000280. A5.01.24. 9.20. 2.22.20. 4.30. 1.12.6. 22.36.1010. 23.07-0.5.0. +s72-t212-tg1. 10.30.50.82. 1.00.21. 1.1. F1.00.07. 0D000280. A5.01.24. 8.30. 2.19.3. 3.20. 1.9.7. 22.36.1010. 23.07-0.5.0. +s83-t213-sut1. 10.30.50.83. 1.00.21. 1.1. F1.00.07. 0D000280. A5.01.24. 9.20. 2.22.20. 4.30. 1.12.6. 22.36.1010. 23.07-0.5.0. +s84-t213-tg1. 10.30.50.84. 1.00.21. 1.1. F1.00.07. 0D000280. A5.01.24. 8.30. 2.19.3. 3.20. 1.9.7. 22.36.1010. 23.07-0.5.0. +s85-t214-sut1. 10.30.50.85. 1.00.21. 1.1. F1.00.07. 0D000280. A5.01.24. 9.20. 2.22.20. 4.30. 1.12.6. 22.36.1010. 23.07-0.5.0. +s86-t214-tg1. 10.30.50.86. 1.00.21. 1.1. F1.00.07. 0D000280. A5.01.24. 8.30. 2.19.3. 3.20. 1.9.7. 22.36.1010. 23.07-0.5.0. +s87-t215-sut1. 10.30.50.87. 1.00.21. 1.1. F1.00.07. 0D000280. A5.01.24. 9.20. 2.22.20. 4.30. 1.12.6. 22.36.1010. 23.07-0.5.0. +s88-t215-tg1. 10.30.50.88. 1.00.21. 1.1. F1.00.07. 0D000280. A5.01.24. 8.30. 2.19.3. 3.20. 1.9.7. 22.36.1010. 23.07-0.5.0. +s90-t31t32-tg1.10.30.50.90. 1.00.21. 1.1. F1.00.07. 0D000280. A5.01.24. -. -. 4.30. 1.12.6. -. -. +``` diff --git a/docs/content/infrastructure/testbed_configuration/sm_spr_hw_bios_cfg.md b/docs/content/infrastructure/testbed_configuration/sm_spr_hw_bios_cfg.md new file mode 100644 index 0000000000..a55835c07d --- /dev/null +++ b/docs/content/infrastructure/testbed_configuration/sm_spr_hw_bios_cfg.md @@ -0,0 +1,851 @@ +--- +bookToc: true +title: "SuperMicro SapphireRapids" +--- + +# SuperMicro SapphireRapids + +## Linux lscpu + +``` +Architecture: x86_64 + CPU op-mode(s): 32-bit, 64-bit + Address sizes: 46 bits physical, 57 bits virtual + Byte Order: Little Endian +CPU(s): 128 + On-line CPU(s) list: 0-127 +Vendor ID: GenuineIntel + Model name: Intel(R) Xeon(R) Platinum 8462Y+ + CPU family: 6 + Model: 143 + Thread(s) per core: 2 + Core(s) per socket: 32 + Socket(s): 2 + Stepping: 8 + BogoMIPS: 5600.00 + Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1 + gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dt + es64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsav + e avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cat_l2 cdp_l3 invpcid_single cdp_l2 ssbd mba ibrs ibpb stibp ibrs_enh + anced tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a avx512f avx512dq rdsee + d adx smap avx512ifma clflushopt clwb intel_pt avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cq + m_mbm_total cqm_mbm_local split_lock_detect avx_vnni avx512_bf16 wbnoinvd dtherm arat pln pts hwp hwp_act_window hwp_epp hwp_pkg_req avx + 512vbmi umip pku ospke waitpkg avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg tme avx512_vpopcntdq la57 rdpid bus_lock_dete + ct cldemote movdiri movdir64b enqcmd fsrm md_clear serialize tsxldtrk pconfig arch_lbr amx_bf16 avx512_fp16 amx_tile amx_int8 flush_l1d + arch_capabilities +Virtualization features: + Virtualization: VT-x +Caches (sum of all): + L1d: 3 MiB (64 instances) + L1i: 2 MiB (64 instances) + L2: 128 MiB (64 instances) + L3: 120 MiB (2 instances) +NUMA: + NUMA node(s): 2 + NUMA node0 CPU(s): 0-31,64-95 + NUMA node1 CPU(s): 32-63,96-127 +Vulnerabilities: + Itlb multihit: Not affected + L1tf: Not affected + Mds: Not affected + Meltdown: Not affected + Mmio stale data: Not affected + Retbleed: Not affected + Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp + Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization + Spectre v2: Mitigation; Enhanced IBRS, IBPB conditional, RSB filling + Srbds: Not affected + Tsx async abort: Not affected +``` + +## Linux dmidecode + +``` +# dmidecode 3.3 +Getting SMBIOS data from sysfs. +SMBIOS 3.5.0 present. +Table at 0x000E6E00. + +Handle 0x0000, DMI type 0, 26 bytes +BIOS Information + Vendor: American Megatrends International, LLC. + Version: 1.0 + Release Date: 11/16/2022 + Address: 0xF0000 + Runtime Size: 64 kB + ROM Size: 32 MB + Characteristics: + PCI is supported + BIOS is upgradeable + BIOS shadowing is allowed + Boot from CD is supported + Selectable boot is supported + BIOS ROM is socketed + EDD is supported + Japanese floppy for NEC 9800 1.2 MB is supported (int 13h) + Japanese floppy for Toshiba 1.2 MB is supported (int 13h) + 5.25"/360 kB floppy services are supported (int 13h) + 5.25"/1.2 MB floppy services are supported (int 13h) + 3.5"/720 kB floppy services are supported (int 13h) + 3.5"/2.88 MB floppy services are supported (int 13h) + Print screen service is supported (int 5h) + Serial services are supported (int 14h) + Printer services are supported (int 17h) + CGA/mono video services are supported (int 10h) + ACPI is supported + USB legacy is supported + BIOS boot specification is supported + Targeted content distribution is supported + UEFI is supported + BIOS Revision: 5.29 + +Handle 0x0001, DMI type 1, 27 bytes +System Information + Manufacturer: Supermicro + Product Name: SYS-741GE-TNRT + Version: 0123456789 + Serial Number: S512539X3109946 + UUID: 00000000-0000-0000-0000-7cc255275836 + Wake-up Type: Power Switch + SKU Number: To be filled by O.E.M. + Family: Family + +Handle 0x0002, DMI type 2, 15 bytes +Base Board Information + Manufacturer: Supermicro + Product Name: X13DEG-QT + Version: 1.10 + Serial Number: OM22CS039806 + Asset Tag: Base Board Asset Tag + Features: + Board is a hosting board + Board is replaceable + Location In Chassis: Part Component + Chassis Handle: 0x0003 + Type: Motherboard + Contained Object Handles: 0 + +Handle 0x0003, DMI type 3, 22 bytes +Chassis Information + Manufacturer: Supermicro + Type: Other + Lock: Not Present + Version: 0123456789 + Serial Number: C7490FL36A40118 + Asset Tag: Chassis Asset Tag + Boot-up State: Safe + Power Supply State: Safe + Thermal State: Safe + Security Status: None + OEM Information: 0x00000000 + Height: Unspecified + Number Of Power Cords: 1 + Contained Elements: 0 + SKU Number: 0123456789 + +Handle 0x0032, DMI type 4, 50 bytes +Processor Information + Socket Designation: CPU1 + Type: Central Processor + Family: Xeon + Manufacturer: Intel(R) Corporation + ID: F8 06 08 00 FF FB EB BF + Signature: Type 0, Family 6, Model 143, Stepping 8 + Flags: + FPU (Floating-point unit on-chip) + VME (Virtual mode extension) + DE (Debugging extension) + PSE (Page size extension) + TSC (Time stamp counter) + MSR (Model specific registers) + PAE (Physical address extension) + MCE (Machine check exception) + CX8 (CMPXCHG8 instruction supported) + APIC (On-chip APIC hardware supported) + SEP (Fast system call) + MTRR (Memory type range registers) + PGE (Page global enable) + MCA (Machine check architecture) + CMOV (Conditional move instruction supported) + PAT (Page attribute table) + PSE-36 (36-bit page size extension) + CLFSH (CLFLUSH instruction supported) + DS (Debug store) + ACPI (ACPI supported) + MMX (MMX technology supported) + FXSR (FXSAVE and FXSTOR instructions supported) + SSE (Streaming SIMD extensions) + SSE2 (Streaming SIMD extensions 2) + SS (Self-snoop) + HTT (Multi-threading) + TM (Thermal monitor supported) + PBE (Pending break enabled) + Version: Intel(R) Xeon(R) Platinum 8462Y+ + Voltage: 1.6 V + External Clock: 100 MHz + Max Speed: 4000 MHz + Current Speed: 2800 MHz + Status: Populated, Enabled + Upgrade: Socket LGA4677 + L1 Cache Handle: 0x002F + L2 Cache Handle: 0x0030 + L3 Cache Handle: 0x0031 + Serial Number: Not Specified + Asset Tag: UNKNOWN + Part Number: Not Specified + Core Count: 32 + Core Enabled: 32 + Thread Count: 64 + Characteristics: + 64-bit capable + Multi-Core + Hardware Thread + Execute Protection + Enhanced Virtualization + Power/Performance Control + +Handle 0x0036, DMI type 4, 50 bytes +Processor Information + Socket Designation: CPU2 + Type: Central Processor + Family: Xeon + Manufacturer: Intel(R) Corporation + ID: F8 06 08 00 FF FB EB BF + Signature: Type 0, Family 6, Model 143, Stepping 8 + Flags: + FPU (Floating-point unit on-chip) + VME (Virtual mode extension) + DE (Debugging extension) + PSE (Page size extension) + TSC (Time stamp counter) + MSR (Model specific registers) + PAE (Physical address extension) + MCE (Machine check exception) + CX8 (CMPXCHG8 instruction supported) + APIC (On-chip APIC hardware supported) + SEP (Fast system call) + MTRR (Memory type range registers) + PGE (Page global enable) + MCA (Machine check architecture) + CMOV (Conditional move instruction supported) + PAT (Page attribute table) + PSE-36 (36-bit page size extension) + CLFSH (CLFLUSH instruction supported) + DS (Debug store) + ACPI (ACPI supported) + MMX (MMX technology supported) + FXSR (FXSAVE and FXSTOR instructions supported) + SSE (Streaming SIMD extensions) + SSE2 (Streaming SIMD extensions 2) + SS (Self-snoop) + HTT (Multi-threading) + TM (Thermal monitor supported) + PBE (Pending break enabled) + Version: Intel(R) Xeon(R) Platinum 8462Y+ + Voltage: 1.6 V + External Clock: 100 MHz + Max Speed: 4000 MHz + Current Speed: 2800 MHz + Status: Populated, Enabled + Upgrade: Socket LGA4677 + L1 Cache Handle: 0x0033 + L2 Cache Handle: 0x0034 + L3 Cache Handle: 0x0035 + Serial Number: Not Specified + Asset Tag: UNKNOWN + Part Number: Not Specified + Core Count: 32 + Core Enabled: 32 + Thread Count: 64 + Characteristics: + 64-bit capable + Multi-Core + Hardware Thread + Execute Protection + Enhanced Virtualization + Power/Performance Control +``` + +## Linux dmidecode memory + +``` +Handle 0x003D, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x003C + Error Information Handle: Not Provided + Total Width: 80 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: P1-DIMMA1 + Bank Locator: P0_Node0_Channel0_Dimm0 + Type: DDR5 + Type Detail: Synchronous Registered (Buffered) + Speed: 4800 MT/s + Manufacturer: SK Hynix + Serial Number: 80AD0122149612A27F + Asset Tag: P1-DIMMA1_AssetTag (date:22/14) + Part Number: HMCG88MEBRA107N + Rank: 2 + Configured Memory Speed: 4800 MT/s + Minimum Voltage: 1.1 V + Maximum Voltage: 1.1 V + Configured Voltage: 1.1 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xAD + Module Product ID: 0xAD00 + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 32 GB + Cache Size: None + Logical Size: None + +Handle 0x003E, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x003C + Error Information Handle: Not Provided + Total Width: 80 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: P1-DIMMB1 + Bank Locator: P0_Node0_Channel1_Dimm0 + Type: DDR5 + Type Detail: Synchronous Registered (Buffered) + Speed: 4800 MT/s + Manufacturer: SK Hynix + Serial Number: 80AD0122149612A1CC + Asset Tag: P1-DIMMB1_AssetTag (date:22/14) + Part Number: HMCG88MEBRA107N + Rank: 2 + Configured Memory Speed: 4800 MT/s + Minimum Voltage: 1.1 V + Maximum Voltage: 1.1 V + Configured Voltage: 1.1 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xAD + Module Product ID: 0xAD00 + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 32 GB + Cache Size: None + Logical Size: None + +Handle 0x003F, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x003C + Error Information Handle: Not Provided + Total Width: 80 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: P1-DIMMC1 + Bank Locator: P0_Node0_Channel2_Dimm0 + Type: DDR5 + Type Detail: Synchronous Registered (Buffered) + Speed: 4800 MT/s + Manufacturer: SK Hynix + Serial Number: 80AD0122149612A1B7 + Asset Tag: P1-DIMMC1_AssetTag (date:22/14) + Part Number: HMCG88MEBRA107N + Rank: 2 + Configured Memory Speed: 4800 MT/s + Minimum Voltage: 1.1 V + Maximum Voltage: 1.1 V + Configured Voltage: 1.1 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xAD + Module Product ID: 0xAD00 + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 32 GB + Cache Size: None + Logical Size: None + +Handle 0x0040, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x003C + Error Information Handle: Not Provided + Total Width: 80 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: P1-DIMMD1 + Bank Locator: P0_Node0_Channel3_Dimm0 + Type: DDR5 + Type Detail: Synchronous Registered (Buffered) + Speed: 4800 MT/s + Manufacturer: SK Hynix + Serial Number: 80AD0122149612927A + Asset Tag: P1-DIMMD1_AssetTag (date:22/14) + Part Number: HMCG88MEBRA107N + Rank: 2 + Configured Memory Speed: 4800 MT/s + Minimum Voltage: 1.1 V + Maximum Voltage: 1.1 V + Configured Voltage: 1.1 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xAD + Module Product ID: 0xAD00 + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 32 GB + Cache Size: None + Logical Size: None + +Handle 0x0041, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x003C + Error Information Handle: Not Provided + Total Width: 80 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: P1-DIMME1 + Bank Locator: P0_Node1_Channel0_Dimm0 + Type: DDR5 + Type Detail: Synchronous Registered (Buffered) + Speed: 4800 MT/s + Manufacturer: SK Hynix + Serial Number: 80AD0122149612A2B2 + Asset Tag: P1-DIMME1_AssetTag (date:22/14) + Part Number: HMCG88MEBRA107N + Rank: 2 + Configured Memory Speed: 4800 MT/s + Minimum Voltage: 1.1 V + Maximum Voltage: 1.1 V + Configured Voltage: 1.1 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xAD + Module Product ID: 0xAD00 + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 32 GB + Cache Size: None + Logical Size: None + +Handle 0x0042, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x003C + Error Information Handle: Not Provided + Total Width: 80 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: P1-DIMMF1 + Bank Locator: P0_Node1_Channel1_Dimm0 + Type: DDR5 + Type Detail: Synchronous Registered (Buffered) + Speed: 4800 MT/s + Manufacturer: SK Hynix + Serial Number: 80AD0122149612A7F0 + Asset Tag: P1-DIMMF1_AssetTag (date:22/14) + Part Number: HMCG88MEBRA107N + Rank: 2 + Configured Memory Speed: 4800 MT/s + Minimum Voltage: 1.1 V + Maximum Voltage: 1.1 V + Configured Voltage: 1.1 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xAD + Module Product ID: 0xAD00 + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 32 GB + Cache Size: None + Logical Size: None + +Handle 0x0043, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x003C + Error Information Handle: Not Provided + Total Width: 80 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: P1-DIMMG1 + Bank Locator: P0_Node1_Channel2_Dimm0 + Type: DDR5 + Type Detail: Synchronous Registered (Buffered) + Speed: 4800 MT/s + Manufacturer: SK Hynix + Serial Number: 80AD0122149612A1B0 + Asset Tag: P1-DIMMG1_AssetTag (date:22/14) + Part Number: HMCG88MEBRA107N + Rank: 2 + Configured Memory Speed: 4800 MT/s + Minimum Voltage: 1.1 V + Maximum Voltage: 1.1 V + Configured Voltage: 1.1 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xAD + Module Product ID: 0xAD00 + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 32 GB + Cache Size: None + Logical Size: None + +Handle 0x0044, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x003C + Error Information Handle: Not Provided + Total Width: 80 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: P1-DIMMH1 + Bank Locator: P0_Node1_Channel3_Dimm0 + Type: DDR5 + Type Detail: Synchronous Registered (Buffered) + Speed: 4800 MT/s + Manufacturer: SK Hynix + Serial Number: 80AD012214961292F4 + Asset Tag: P1-DIMMH1_AssetTag (date:22/14) + Part Number: HMCG88MEBRA107N + Rank: 2 + Configured Memory Speed: 4800 MT/s + Minimum Voltage: 1.1 V + Maximum Voltage: 1.1 V + Configured Voltage: 1.1 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xAD + Module Product ID: 0xAD00 + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 32 GB + Cache Size: None + Logical Size: None + +Handle 0x0045, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x003C + Error Information Handle: Not Provided + Total Width: 80 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: P2-DIMMA1 + Bank Locator: P1_Node0_Channel0_Dimm0 + Type: DDR5 + Type Detail: Synchronous Registered (Buffered) + Speed: 4800 MT/s + Manufacturer: SK Hynix + Serial Number: 80AD01221496129322 + Asset Tag: P2-DIMMA1_AssetTag (date:22/14) + Part Number: HMCG88MEBRA107N + Rank: 2 + Configured Memory Speed: 4800 MT/s + Minimum Voltage: 1.1 V + Maximum Voltage: 1.1 V + Configured Voltage: 1.1 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xAD + Module Product ID: 0xAD00 + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 32 GB + Cache Size: None + Logical Size: None + +Handle 0x0046, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x003C + Error Information Handle: Not Provided + Total Width: 80 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: P2-DIMMB1 + Bank Locator: P1_Node0_Channel1_Dimm0 + Type: DDR5 + Type Detail: Synchronous Registered (Buffered) + Speed: 4800 MT/s + Manufacturer: SK Hynix + Serial Number: 80AD0122149612A282 + Asset Tag: P2-DIMMB1_AssetTag (date:22/14) + Part Number: HMCG88MEBRA107N + Rank: 2 + Configured Memory Speed: 4800 MT/s + Minimum Voltage: 1.1 V + Maximum Voltage: 1.1 V + Configured Voltage: 1.1 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xAD + Module Product ID: 0xAD00 + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 32 GB + Cache Size: None + Logical Size: None + +Handle 0x0047, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x003C + Error Information Handle: Not Provided + Total Width: 80 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: P2-DIMMC1 + Bank Locator: P1_Node0_Channel2_Dimm0 + Type: DDR5 + Type Detail: Synchronous Registered (Buffered) + Speed: 4800 MT/s + Manufacturer: SK Hynix + Serial Number: 80AD0122149612936B + Asset Tag: P2-DIMMC1_AssetTag (date:22/14) + Part Number: HMCG88MEBRA107N + Rank: 2 + Configured Memory Speed: 4800 MT/s + Minimum Voltage: 1.1 V + Maximum Voltage: 1.1 V + Configured Voltage: 1.1 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xAD + Module Product ID: 0xAD00 + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 32 GB + Cache Size: None + Logical Size: None + +Handle 0x0048, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x003C + Error Information Handle: Not Provided + Total Width: 80 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: P2-DIMMD1 + Bank Locator: P1_Node0_Channel3_Dimm0 + Type: DDR5 + Type Detail: Synchronous Registered (Buffered) + Speed: 4800 MT/s + Manufacturer: SK Hynix + Serial Number: 80AD012214961292FA + Asset Tag: P2-DIMMD1_AssetTag (date:22/14) + Part Number: HMCG88MEBRA107N + Rank: 2 + Configured Memory Speed: 4800 MT/s + Minimum Voltage: 1.1 V + Maximum Voltage: 1.1 V + Configured Voltage: 1.1 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xAD + Module Product ID: 0xAD00 + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 32 GB + Cache Size: None + Logical Size: None + +Handle 0x0049, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x003C + Error Information Handle: Not Provided + Total Width: 80 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: P2-DIMME1 + Bank Locator: P1_Node1_Channel0_Dimm0 + Type: DDR5 + Type Detail: Synchronous Registered (Buffered) + Speed: 4800 MT/s + Manufacturer: SK Hynix + Serial Number: 80AD012214961292ED + Asset Tag: P2-DIMME1_AssetTag (date:22/14) + Part Number: HMCG88MEBRA107N + Rank: 2 + Configured Memory Speed: 4800 MT/s + Minimum Voltage: 1.1 V + Maximum Voltage: 1.1 V + Configured Voltage: 1.1 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xAD + Module Product ID: 0xAD00 + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 32 GB + Cache Size: None + Logical Size: None + +Handle 0x004A, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x003C + Error Information Handle: Not Provided + Total Width: 80 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: P2-DIMMF1 + Bank Locator: P1_Node1_Channel1_Dimm0 + Type: DDR5 + Type Detail: Synchronous Registered (Buffered) + Speed: 4800 MT/s + Manufacturer: SK Hynix + Serial Number: 80AD0122149612A888 + Asset Tag: P2-DIMMF1_AssetTag (date:22/14) + Part Number: HMCG88MEBRA107N + Rank: 2 + Configured Memory Speed: 4800 MT/s + Minimum Voltage: 1.1 V + Maximum Voltage: 1.1 V + Configured Voltage: 1.1 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xAD + Module Product ID: 0xAD00 + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 32 GB + Cache Size: None + Logical Size: None + +Handle 0x004B, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x003C + Error Information Handle: Not Provided + Total Width: 80 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: P2-DIMMG1 + Bank Locator: P1_Node1_Channel2_Dimm0 + Type: DDR5 + Type Detail: Synchronous Registered (Buffered) + Speed: 4800 MT/s + Manufacturer: SK Hynix + Serial Number: 80AD0122149612A299 + Asset Tag: P2-DIMMG1_AssetTag (date:22/14) + Part Number: HMCG88MEBRA107N + Rank: 2 + Configured Memory Speed: 4800 MT/s + Minimum Voltage: 1.1 V + Maximum Voltage: 1.1 V + Configured Voltage: 1.1 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xAD + Module Product ID: 0xAD00 + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 32 GB + Cache Size: None + Logical Size: None + +Handle 0x004C, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x003C + Error Information Handle: Not Provided + Total Width: 80 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: P2-DIMMH1 + Bank Locator: P1_Node1_Channel3_Dimm0 + Type: DDR5 + Type Detail: Synchronous Registered (Buffered) + Speed: 4800 MT/s + Manufacturer: SK Hynix + Serial Number: 80AD0122149612A195 + Asset Tag: P2-DIMMH1_AssetTag (date:22/14) + Part Number: HMCG88MEBRA107N + Rank: 2 + Configured Memory Speed: 4800 MT/s + Minimum Voltage: 1.1 V + Maximum Voltage: 1.1 V + Configured Voltage: 1.1 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xAD + Module Product ID: 0xAD00 + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 32 GB + Cache Size: None + Logical Size: None +``` + +## Linux cmdline + +``` +BOOT_IMAGE=/boot/vmlinuz-5.15.0-46-generic root=UUID=b99a7749-d0ee-4afe-88a0-0be6c5873645 ro audit=0 default_hugepagesz=2M hugepagesz=1G hugepages=32 hugepagesz=2M hugepages=32768 hpet=disable intel_idle.max_cstate=1 intel_iommu=on intel_pstate=disable iommu=pt isolcpus=1-31,33-63,65-95,97-127 mce=off nmi_watchdog=0 nohz_full=1-31,33-63,65-95,97-127 nosoftlockup numa_balancing=disable processor.max_cstate=1 rcu_nocbs=1-31,33-63,65-95,97-127 tsc=reliable +``` + +## Xeon ICX Server Firmware Inventory + +``` +Host. IPMI IP. BMC. BIOS. CPLD. CPU Microcode. Cx-7 Firmware. mlx5. E810 Firmware. ice. +s52-t21-sut1. 10.30.50.52. 1.00.2. 1.4. F2.43.09. 0x2b0000c0. 28.38.1002. 23.07-0.5.0. -. -. +s53-t21-tg1. 10.30.50.53. 1.00.2. 1.4. F2.43.09. 0x2b0000c0. 28.38.1002. 23.07-0.5.0. -. -. +s54-t22-sut1. 10.30.50.54. 1.00.2. 1.4. F2.43.09. 0x2b0000c0. -. -. 4.30. 1.12.6. +s55-t22-tg1. 10.30.50.55. 1.00.2. 1.4. F2.43.09. 0x2b0000c0. -. -. 3.20. 1.9.7. +s56-t23-sut1. 10.30.50.56. 1.00.2. 1.4. F2.43.09. 0x2b0000c0. 28.38.1002. 23.07-0.5.0. 4.30. 1.12.6. +s57-t23-tg1. 10.30.50.57. 1.00.2. 1.4. F2.43.09. 0x2b0000c0. 28.38.1002. 23.07-0.5.0. 3.20. 1.9.7. +s58-t24-sut1. 10.30.50.58. 1.00.2. 1.4. F2.43.09. 0x2b0000c0. 28.38.1002. 23.07-0.5.0. 4.30. 1.12.6. +s59-t24-tg1. 10.30.50.59. 1.00.2. 1.4. F2.43.09. 0x2b0000c0. 28.38.1002. 23.07-0.5.0. 3.20. 1.9.7. +``` diff --git a/docs/content/infrastructure/testbed_configuration/sm_zn2_hw_bios_cfg.md b/docs/content/infrastructure/testbed_configuration/sm_zn2_hw_bios_cfg.md new file mode 100644 index 0000000000..89898f8f1d --- /dev/null +++ b/docs/content/infrastructure/testbed_configuration/sm_zn2_hw_bios_cfg.md @@ -0,0 +1,620 @@ +--- +bookToc: true +title: "SuperMicro EPYC Zen2" +--- + +# SuperMicro EPYC Zen2 + +## Linux lscpu + +``` +$ lscpu +Architecture: x86_64 +CPU op-mode(s): 32-bit, 64-bit +Byte Order: Little Endian +CPU(s): 64 +On-line CPU(s) list: 0-63 +Thread(s) per core: 2 +Core(s) per socket: 32 +Socket(s): 1 +NUMA node(s): 2 +Vendor ID: AuthenticAMD +CPU family: 23 +Model: 49 +Model name: AMD EPYC 7532 32-Core Processor +Stepping: 0 +CPU MHz: 1981.470 +CPU max MHz: 2400.0000 +CPU min MHz: 1500.0000 +BogoMIPS: 4800.05 +Virtualization: AMD-V +L1d cache: 32K +L1i cache: 32K +L2 cache: 512K +L3 cache: 16384K +NUMA node0 CPU(s): 0-15,32-47 +NUMA node1 CPU(s): 16-31,48-63 +Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate sme ssbd ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif umip rdpid overflow_recov succor smca +``` + +``` +$ lscpu +Architecture: x86_64 +CPU op-mode(s): 32-bit, 64-bit +Byte Order: Little Endian +CPU(s): 64 +On-line CPU(s) list: 0-63 +Thread(s) per core: 2 +Core(s) per socket: 32 +Socket(s): 1 +NUMA node(s): 2 +Vendor ID: AuthenticAMD +CPU family: 23 +Model: 49 +Model name: AMD EPYC 7532 32-Core Processor +Stepping: 0 +CPU MHz: 1981.470 +CPU max MHz: 2400.0000 +CPU min MHz: 1500.0000 +BogoMIPS: 4800.05 +Virtualization: AMD-V +L1d cache: 32K +L1i cache: 32K +L2 cache: 512K +L3 cache: 16384K +NUMA node0 CPU(s): 0-15,32-47 +NUMA node1 CPU(s): 16-31,48-63 +Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate sme ssbd ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif umip rdpid overflow_recov succor smca +``` + +## Linux dmidecode + +``` + $ dmidecode -t slot + Handle 0x0026, DMI type 7, 27 bytes + Cache Information + Socket Designation: L1 Cache + Configuration: Enabled, Not Socketed, Level 1 + Operational Mode: Write Back + Location: Internal + Installed Size: 2048 kB + Maximum Size: 2048 kB + Supported SRAM Types: + Pipeline Burst + Installed SRAM Type: Pipeline Burst + Speed: 1 ns + Error Correction Type: Multi-bit ECC + System Type: Unified + Associativity: 8-way Set-associative + + Handle 0x0027, DMI type 7, 27 bytes + Cache Information + Socket Designation: L2 Cache + Configuration: Enabled, Not Socketed, Level 2 + Operational Mode: Write Back + Location: Internal + Installed Size: 16384 kB + Maximum Size: 16384 kB + Supported SRAM Types: + Pipeline Burst + Installed SRAM Type: Pipeline Burst + Speed: 1 ns + Error Correction Type: Multi-bit ECC + System Type: Unified + Associativity: 8-way Set-associative + + Handle 0x0028, DMI type 7, 27 bytes + Cache Information + Socket Designation: L3 Cache + Configuration: Enabled, Not Socketed, Level 3 + Operational Mode: Write Back + Location: Internal + Installed Size: 262144 kB + Maximum Size: 262144 kB + Supported SRAM Types: + Pipeline Burst + Installed SRAM Type: Pipeline Burst + Speed: 1 ns + Error Correction Type: Multi-bit ECC + System Type: Unified + Associativity: 16-way Set-associative + + Handle 0x0029, DMI type 4, 48 bytes + Processor Information + Socket Designation: CPU + Type: Central Processor + Family: Zen + Manufacturer: Advanced Micro Devices, Inc. + ID: 10 0F 83 00 FF FB 8B 17 + Signature: Family 23, Model 49, Stepping 0 + Flags: + FPU (Floating-point unit on-chip) + VME (Virtual mode extension) + DE (Debugging extension) + PSE (Page size extension) + TSC (Time stamp counter) + MSR (Model specific registers) + PAE (Physical address extension) + MCE (Machine check exception) + CX8 (CMPXCHG8 instruction supported) + APIC (On-chip APIC hardware supported) + SEP (Fast system call) + MTRR (Memory type range registers) + PGE (Page global enable) + MCA (Machine check architecture) + CMOV (Conditional move instruction supported) + PAT (Page attribute table) + PSE-36 (36-bit page size extension) + CLFSH (CLFLUSH instruction supported) + MMX (MMX technology supported) + FXSR (FXSAVE and FXSTOR instructions supported) + SSE (Streaming SIMD extensions) + SSE2 (Streaming SIMD extensions 2) + HTT (Multi-threading) + Version: AMD EPYC 7532 32-Core Processor + Voltage: 1.1 V + External Clock: 100 MHz + Max Speed: 3300 MHz + Current Speed: 2400 MHz + Status: Populated, Enabled + Upgrade: Socket SP3 + L1 Cache Handle: 0x0026 + L2 Cache Handle: 0x0027 + L3 Cache Handle: 0x0028 + Serial Number: Unknown + Asset Tag: Unknown + Part Number: Unknown + Core Count: 32 + Core Enabled: 32 + Thread Count: 64 + Characteristics: + 64-bit capable + Multi-Core + Hardware Thread + Execute Protection + Enhanced Virtualization + Power/Performance Control +``` + +## Linux dmidecode pci + +``` + $ dmidecode -t slot + Getting SMBIOS data from sysfs. + SMBIOS 3.2.0 present. + # SMBIOS implementations newer than version 3.1.1 are not + # fully supported by this version of dmidecode. + + Handle 0x000A, DMI type 9, 17 bytes + System Slot Information + Designation: M.2-HC1 CPU PCI-E 4.0 X4/X2 + Type: x4 PCI Express 3 x4 + Current Usage: Available + Length: Short + ID: 1 + Characteristics: + 3.3 V is provided + Opening is shared + PME signal is supported + Bus Address: 0000:ff:00.0 + + Handle 0x000B, DMI type 9, 17 bytes + System Slot Information + Designation: M.2-HC2 CPU PCI-E 4.0 X2 + Type: x2 PCI Express 3 x2 + Current Usage: Available + Length: Short + ID: 2 + Characteristics: + 3.3 V is provided + Opening is shared + PME signal is supported + Bus Address: 0000:ff:00.0 + + Handle 0x0042, DMI type 9, 17 bytes + System Slot Information + Designation: RSC-W-66G4 SLOT1 PCI-E 4.0 X16 + Type: x16 PCI Express 3 x16 + Current Usage: In Use + Length: Long + ID: 1 + Characteristics: + 3.3 V is provided + PME signal is supported + Bus Address: 0000:41:00.0 + + Handle 0x0043, DMI type 9, 17 bytes + System Slot Information + Designation: RSC-W-66G4 SLOT2 PCI-E 4.0 X16 + Type: x16 PCI Express 3 x16 + Current Usage: In Use + Length: Long + ID: 2 + Characteristics: + 3.3 V is provided + PME signal is supported + Bus Address: 0000:81:00.0 + + Handle 0x0045, DMI type 9, 17 bytes + System Slot Information + Designation: RSC-WR-6 SLOT1 PCI-E 4.0 X16 + Type: x16 PCI Express 3 x16 + Current Usage: In Use + Length: Long + ID: 1 + Characteristics: + 3.3 V is provided + PME signal is supported + Bus Address: 0000:01:00.0 + +``` + +## Linux dmidecode memory + +``` + $ dmidecode -t memory + # dmidecode 3.1 + Getting SMBIOS data from sysfs. + SMBIOS 3.2.0 present. + # SMBIOS implementations newer than version 3.1.1 are not + # fully supported by this version of dmidecode. + + Handle 0x0023, DMI type 16, 23 bytes + Physical Memory Array + Location: System Board Or Motherboard + Use: System Memory + Error Correction Type: Multi-bit ECC + Maximum Capacity: 2 TB + Error Information Handle: 0x0022 + Number Of Devices: 8 + + Handle 0x002B, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0023 + Error Information Handle: 0x002A + Total Width: 72 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: DIMMA1 + Bank Locator: P0_Node0_Channel0_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: SK Hynix + Serial Number: 431E9216 + Asset Tag: P1-DIMMA1_AssetTag (date:19/00) + Part Number: HMA84GR7CJR4N-XN + Rank: 2 + Configured Clock Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x002E, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0023 + Error Information Handle: 0x002D + Total Width: 72 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: DIMMB1 + Bank Locator: P0_Node0_Channel1_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: SK Hynix + Serial Number: 431E91D2 + Asset Tag: P1-DIMMB1_AssetTag (date:19/00) + Part Number: HMA84GR7CJR4N-XN + Rank: 2 + Configured Clock Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x0031, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0023 + Error Information Handle: 0x0030 + Total Width: 72 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: DIMMC1 + Bank Locator: P0_Node0_Channel2_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: SK Hynix + Serial Number: 431E918A + Asset Tag: P1-DIMMC1_AssetTag (date:19/00) + Part Number: HMA84GR7CJR4N-XN + Rank: 2 + Configured Clock Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x0034, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0023 + Error Information Handle: 0x0033 + Total Width: 72 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: DIMMD1 + Bank Locator: P0_Node0_Channel3_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: SK Hynix + Serial Number: 431E9187 + Asset Tag: P1-DIMMD1_AssetTag (date:19/00) + Part Number: HMA84GR7CJR4N-XN + Rank: 2 + Configured Clock Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x0037, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0023 + Error Information Handle: 0x0036 + Total Width: 72 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: DIMME1 + Bank Locator: P0_Node0_Channel4_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: SK Hynix + Serial Number: 431E9178 + Asset Tag: P1-DIMME1_AssetTag (date:19/00) + Part Number: HMA84GR7CJR4N-XN + Rank: 2 + Configured Clock Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x003A, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0023 + Error Information Handle: 0x0039 + Total Width: 72 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: DIMMF1 + Bank Locator: P0_Node0_Channel5_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: SK Hynix + Serial Number: 431E9206 + Asset Tag: P1-DIMMF1_AssetTag (date:19/00) + Part Number: HMA84GR7CJR4N-XN + Rank: 2 + Configured Clock Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x003D, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0023 + Error Information Handle: 0x003C + Total Width: 72 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: DIMMG1 + Bank Locator: P0_Node0_Channel6_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: SK Hynix + Serial Number: 431E9207 + Asset Tag: P1-DIMMG1_AssetTag (date:19/00) + Part Number: HMA84GR7CJR4N-XN + Rank: 2 + Configured Clock Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x0040, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0023 + Error Information Handle: 0x003F + Total Width: 72 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: DIMMH1 + Bank Locator: P0_Node0_Channel7_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: SK Hynix + Serial Number: 431E9209 + Asset Tag: P1-DIMMH1_AssetTag (date:19/00) + Part Number: HMA84GR7CJR4N-XN + Rank: 2 + Configured Clock Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V +``` + +## EPYC zn2 Server BIOS Configuration - TG + +### Boot Feature + +``` + | Quiet Boot [Enabled] |Boot option | + | | | + | Option ROM Messages [Force BIOS] | | + | Bootup NumLock State [On] | | + | Wait For "F1" If Error [Enabled] | | + | INT19 Trap Response [Immediate] | | + | Re-try Boot [Disabled] | | + | | | + | Power Configuration | | + | Watch Dog Function [Disabled] | | + | Restore on AC Power Loss [Last State] | | + | Power Button Function [Instant Off] | | +``` + +### CPU Configuration + +``` + | ACPI Settings ^| | + | -------------------------------------------------- *| | + | PCI AER Support [Disabled] *| | + | High Precision Event Timer [Disabled] *| | + | NUMA Nodes Per Socket [NPS2] *| | + | ACPI SRAT L3 Cache As NUMA Domain [Auto] *| | + | *| | + | CPU Configuration ^| | + | -------------------------------------------------- *| | + | SMT Control [Auto] *| | + | Core Performance Boost [Auto] *| | + | Global C-state control [Disabled] *| | + | Local APIC Mode [Auto] *| | + | CCD Control [Auto] *| | + | Core Control [Auto] *| | + | Core Control [Auto] *| | + | L1 Stream HW Prefetcher [Enabled] *| | + | L2 Stream HW Prefetcher [Enabled] *| | + | SVM Mode [Enabled] *| | + | SMEE [Disabled] *| | + | *| | + |> CPU1 Information *| | + | *| | + | NB Configuration ^| | + | -------------------------------------------------- *| | + | Determinism Control [Manual] *| | + | Determinism Slider [Performance] *| | + | cTDP Control [Disabled] *| | + | IOMMU [Disabled] *| | + | ACS Enable [Auto] *| | + | Package Power Limit Control [Auto] *| | + | APBDIS [1] *| | + | Fixed SOC Pstate [P0] *| | + | DF Cstates [Enabled] *| | + | Preferred IO [Manual] *| | + | Preferred IO Bus [##] *| | + | *| | + | *|-----------------------------| + | *|><: Select Screen | + | *|^v: Select Item | + | *|Enter: Select | + | +|+/-: Change Opt. | + | +|F1: General Help | + | +|F2: Previous Values | + | +|F3: Optimized Defaults | + | v|F4: Save & Exit | + | | | +``` + + +## EPYC zn2 Server BIOS Configuration - DUT + +### Boot Feature + +``` + | Quiet Boot [Enabled] |Boot option | + | | | + | Option ROM Messages [Force BIOS] | | + | Bootup NumLock State [On] | | + | Wait For "F1" If Error [Enabled] | | + | INT19 Trap Response [Immediate] | | + | Re-try Boot [Disabled] | | + | | | + | Power Configuration | | + | Watch Dog Function [Disabled] | | + | Restore on AC Power Loss [Last State] | | + | Power Button Function [Instant Off] | | +``` + +### CPU Configuration + +``` + | ACPI Settings ^| | + | -------------------------------------------------- *| | + | PCI AER Support [Disabled] *| | + | High Precision Event Timer [Disabled] *| | + | NUMA Nodes Per Socket [NPS2] *| | + | ACPI SRAT L3 Cache As NUMA Domain [Auto] *| | + | *| | + | CPU Configuration ^| | + | -------------------------------------------------- *| | + | SMT Control [Auto] *| | + | Core Performance Boost [Auto] *| | + | Global C-state control [Disabled] *| | + | Local APIC Mode [Auto] *| | + | CCD Control [Auto] *| | + | Core Control [Auto] *| | + | Core Control [Auto] *| | + | L1 Stream HW Prefetcher [Enabled] *| | + | L2 Stream HW Prefetcher [Enabled] *| | + | SVM Mode [Enabled] *| | + | SMEE [Disabled] *| | + | *| | + |> CPU1 Information *| | + | *| | + | NB Configuration ^| | + | -------------------------------------------------- *| | + | Determinism Control [Manual] *| | + | Determinism Slider [Performance] *| | + | cTDP Control [Disabled] *| | + | IOMMU [Disabled] *| | + | ACS Enable [Auto] *| | + | Package Power Limit Control [Auto] *| | + | APBDIS [1] *| | + | Fixed SOC Pstate [P0] *| | + | DF Cstates [Enabled] *| | + | Preferred IO [Manual] *| | + | Preferred IO Bus [##] *| | + | *| | + | *|-----------------------------| + | *|><: Select Screen | + | *|^v: Select Item | + | *|Enter: Select | + | +|+/-: Change Opt. | + | +|F1: General Help | + | +|F2: Previous Values | + | +|F3: Optimized Defaults | + | v|F4: Save & Exit | + | | | +``` + +## Linux cmdline + +``` +$ cat /proc/cmdline +BOOT_IMAGE=/boot/vmlinuz-5.15.0-46-generic root=UUID=cac1254f-9426-4ea6-a8db-2554f075db99 ro amd_iommu=on audit=0 default_hugepagesz=2M hugepagesz=1G hugepages=32 hugepagesz=2M hugepages=32768 hpet=disable iommu=pt isolcpus=1-15,17-31,33-47,49-63 nmi_watchdog=0 nohz_full=off nosoftlockup numa_balancing=disable processor.max_cstate=0 rcu_nocbs=1-15,17-31,33-47,49-63 tsc=reliable console=ttyS0,115200n8 quiet +``` + +## EPYC zn2 Server Firmware Inventory + +``` +Host. IPMI IP. BMC. BIOS. CPLD. CPU Microcode. PCI Bus. X710 Firmware. XXV710 Firmware. i40e. CX-5 Firmware. mlx5_core +s60-t210-sut1. 10.30.55.24. 03.10.04. 1.1a. 02.c2.00. 0x8301038. ?. 9.20. 9.20. 2.22.20. 16.35.2000. 23.07-0.5.0. +s61-t210-tg1. 10.30.55.25. 03.10.04. 1.1a. 02.c2.00. 0x8301038. ?. 8.30. 8.30. 2.19.3. 16.32.1010. 5.9-0.5.5. +``` diff --git a/docs/content/infrastructure/vpp_startup_settings.md b/docs/content/infrastructure/vpp_startup_settings.md new file mode 100644 index 0000000000..7361d4b21f --- /dev/null +++ b/docs/content/infrastructure/vpp_startup_settings.md @@ -0,0 +1,44 @@ +--- +title: "VPP Startup Settings" +weight: 6 +--- + +# VPP Startup Settings + +CSIT code manipulates a number of VPP settings in startup.conf for +optimized performance. List of common settings applied to all tests and +test dependent settings follows. + +## Common Settings + +List of VPP startup.conf settings applied to all tests: + +1. heap-size <value> - set separately for ip4, ip6, stats, main + depending on scale tested. +2. no-tx-checksum-offload - disables UDP / TCP TX checksum offload in + DPDK. Typically needed for use faster vector PMDs (together with + no-multi-seg). +3. buffers-per-numa <value> - sets a number of memory buffers allocated + to VPP per CPU socket. VPP default is 16384. Needs to be increased for + scenarios with large number of interfaces and worker threads. To + accommodate for scale tests, CSIT is setting it to the maximum possible + value corresponding to the limit of DPDK memory mappings (currently + 256). For Xeon Skylake platforms configured with 2MB hugepages and VPP + data-size and buffer-size defaults (2048B and 2496B respectively), this + results in value of 215040 (256 * 840 = 215040, 840 * 2496B buffers fit + in 2MB hugepage). + +## Per Test Settings + +List of vpp startup.conf settings applied dynamically per test: + +1. corelist-workers <list_of_cores> - list of logical cores to run VPP + worker data plane threads. Depends on HyperThreading and core per + test configuration. +2. num-rx-queues <value> - depends on a number of VPP threads and NIC + interfaces. +3. no-multi-seg - disables multi-segment buffers in DPDK, improves + packet throughput, but disables Jumbo MTU support. Disabled for all + tests apart from the ones that require Jumbo 9000B frame support. +4. UIO driver - depends on topology file definition. +5. QAT VFs - depends on NRThreads, each thread = 1QAT VFs. diff --git a/docs/content/methodology/_index.md b/docs/content/methodology/_index.md new file mode 100644 index 0000000000..dbef64db94 --- /dev/null +++ b/docs/content/methodology/_index.md @@ -0,0 +1,6 @@ +--- +bookCollapseSection: false +bookFlatSection: true +title: "Methodology" +weight: 2 +--- diff --git a/docs/content/methodology/bisecting.md b/docs/content/methodology/bisecting.md new file mode 100644 index 0000000000..25be7f94af --- /dev/null +++ b/docs/content/methodology/bisecting.md @@ -0,0 +1,114 @@ +--- +title: "Bisecting" +weight: 5 +--- + +# Bisecting + +Updated for CSIT git commit hash: 153c9e1215f27ad166df0ce4bd2541d9f37a7afa. + +When trending (or report release comparison) detects a performance anomaly, +it is possible to narrow down its cause in VPP repository. +This document explains how. + +## Naming + +Bisect is a binary search, it relies on "git bisect" command. +At the start, two commits need to be marked. One as "old", the other as "new". +Upon second mark, "git bisect" checks out a commit in the middle, +and waits for the user to mark it either old or new. +This effectively replaces the previous mark of the same type, +so "new middle" is checked out, halving the search interval. + +But, "old" and "new" frequently refers to the time order bisect chooses commits, +so in this document we use different adjectives: +Early, mid, late. Early commit and late commit are the current +boundaries of the search interval, mid commit is the next one +to test and classify. +The initial boundaries, as input parameters to the whole search process +are called the earliest commit and the latest commit. + +## Bisect jobs + +VPP is the only project currently using such jobs. +They are not started automatically, they must be triggered on demand. +They allow full tag expressions, but only some result types are supported. +Currently it is all perf types in UTI model: +"mrr", "ndrpdr", "soak", "reconf" and "hoststack". +Device tests (pass/fail) are not supported yet. +If a test fails, a low fake value is used instead of results, +so the bisect procedure can also find breakages (and fixes). + +The trigger word contains the intended testbed type, +e.g. "bisecttest-2n-spr". + +The next word needs to be a commit hash of the intended earliest VPP build. +The latest VPP build is the change the comment is added to. + +If additional arguments are added to the Gerrit trigger, they are treated +as Robot tag expressions to select tests to run. + +## Basic operation + +The job builds VPP .deb packages for both the earliest and latest VPP commit, +then runs the selected tests on both (using CSIT code at HEAD +of the newest CSIT oper branch, or CSIT_REF if started manually on Jenkins). +In archived logs, the results of earliest VPP build are in "earliest" directory, +and results of latest VPP build are in "latest" directory. + +Then the job follows VPP mid commits selected by "git bisect". +They are built and tested, results appear in "middle" directory, +numbered in order "git bisect" has chosen them. + +When classifying the newly measured performance of the current mid commit, +the three sets of current results (early, mid, late) are grouped +in three ways. The mid is either added to early group, or to late group, +or kept as a separate group. +The same Minimal Description Length algorithm as in trend analysis +is used to select the grouping with smallest information content. +If the grouping with the mid results added to the early group +is the smallest, the mid commit becomes the new early. +If the grouping with the mid results added to the late group +is the smallest, the mid commit becomes the new late. +If the grouping with the mid results separate is the smallest, +the mid commit becomes that boundary which keeps larger difference +of average performances (relative to the larger value, pairwise). + +## Temporary specifics + +The Minimal Description Length analysis is performed by +jumpavg-0.4.1 (available on PyPI). + +In contrast to trending, MRR trial duration is kept at 1 second, +but trial multiplicity is set to 60 samples. +Both parameters are set in ci-management, +overridable when triggerring manually on Jenkins. + +The 60x1s setting increases probability of false anomalies, +but bisect always converges to a commit; +it is up to humans to decide if that is a real anomaly. +On upside, stdev is estimated better, making the bisection less sensitive +to randomness. Systematic errors are still possible, +but overall this choice leads to more human-like search decisions. + +As test failures are tolerated, the bisect job usually succeeds +(unless there is a fatal infrastructure issue). +Human investigation is needed to confirm the identified commit is the real cause. +For example, if the cause is in CSIT and all builds lead to failed tests, +the bisect will converge to the earliest commit, which is probably innocent. + +## Console output + +After each mid build is tested, the tree sets of relevant results +are visible in the console output, the prefixes are (without quotes) +"Read csit_early: " +"Read csit_late: " +"Read csit_mid: ". +Each prefix is followed by a list of float values extracted from the tests, +the meaning and units depend on tests chosen +(but do not matter as the same set of tests is executed for each build). +There are also lines starting with "Stats: AvgStdevStats" +which give and overview for average and standard deviation. +Then, the information content in bits for the three possible groupings is listed, +followed by the decision, bits saving and new performance difference. +After the last iteration, the commit message of the offending commit is listed. diff --git a/docs/content/methodology/measurements/_index.md b/docs/content/methodology/measurements/_index.md new file mode 100644 index 0000000000..21176fef80 --- /dev/null +++ b/docs/content/methodology/measurements/_index.md @@ -0,0 +1,12 @@ +--- +bookCollapseSection: true +bookFlatSection: false +title: "Measurements" +weight: 2 +--- + +# Measurement + +- [Data Plane Throughput]({{< relref "/methodology/measurements/data_plane_throughput" >}}) +- [Packet Latency]({{< relref "/methodology/measurements/packet_latency" >}}) +- [Telemetry]({{< relref "/methodology/measurements/telemetry" >}}) diff --git a/docs/content/methodology/measurements/data_plane_throughput/_index.md b/docs/content/methodology/measurements/data_plane_throughput/_index.md new file mode 100644 index 0000000000..30b55f149d --- /dev/null +++ b/docs/content/methodology/measurements/data_plane_throughput/_index.md @@ -0,0 +1,13 @@ +--- +bookCollapseSection: true +bookFlatSection: false +title: "Data Plane Throughput" +weight: 1 +--- + +# Data Plane Throughput + +- [Overview]({{< relref "/methodology/measurements/data_plane_throughput/data_plane_throughput" >}}) +- [MLR Search]({{< relref "/methodology/measurements/data_plane_throughput/mlr_search" >}}) +- [PLR Search]({{< relref "/methodology/measurements/data_plane_throughput/plr_search" >}}) +- [MRR]({{< relref "/methodology/measurements/data_plane_throughput/mrr" >}})
\ No newline at end of file diff --git a/docs/content/methodology/measurements/data_plane_throughput/data_plane_throughput.md b/docs/content/methodology/measurements/data_plane_throughput/data_plane_throughput.md new file mode 100644 index 0000000000..c7dce24c1a --- /dev/null +++ b/docs/content/methodology/measurements/data_plane_throughput/data_plane_throughput.md @@ -0,0 +1,139 @@ +--- +title: "Overview" +weight: 1 +--- + +# Data Plane Throughput + +Network data plane throughput is measured using multiple test methods in +order to obtain representative and repeatable results across the large +set of performance test cases implemented and executed within CSIT. + +Following throughput test methods are used: + +- MLRsearch - Multiple Loss Ratio search, used in NDRPDR tests. +- PLRsearch - Probabilistic Loss Ratio search, used in SOAK tests. +- MRR - Maximum Receive Rate tests, the method based on FRMOL from RFC 2285. + +Description of each test method is followed by generic test properties +shared by all methods. + +## NDRPDR Tests + +These tests employ MLRsearch to find two conditional throughput values. +NDR for zero loss ratio goal and PDR for 0.5% loss ratio goal. + +### Algorithm Details + +See [MLRSearch]({{< ref "mlr_search/#MLRsearch" >}}) section for more detail. +MLRsearch is being standardized in IETF in +[draft-ietf-bmwg-mlrsearch](https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-mlrsearch-06). + +### Description + +Multiple Loss Ratio search (MLRsearch) algorithm can discover multiple +conditional throughputs in a single search, +reducing the overall test execution time compared to a binary search. +In FD.io CSIT, conditional throughputs are discovered for two search goals: +Non-Drop Rate (NDR, zero loss ratio goal) +and Partial Drop Rate (PDR, 0.5% loss ratio goal). +Other inputs are common for both goals: +Goal width is 0.5%, trial duration is 1 second, duration sum goal is 21 seconds +and exceed ratio is 50%. + +The main algorithm expresses the conditional throughput based on one-port load. +The results presented in CSIT show aggregate load, +(the value from the search is doubled if the tests uses bidirectional traffic). + +### Usage + +MLRsearch tests are run to discover NDR and PDR rates for each VPP and +DPDK release covered by CSIT report. Results for small frame sizes +(64B/78B, IMIX) are presented in packet throughput graphs +(Box-and-Whisker Plots) with NDR and PDR rates plotted against the test +cases covering popular VPP packet paths. + +Each test is executed at least 10 times to verify measurements +repeatability and results are compared between releases and test +environments. NDR and PDR packet and bandwidth throughput results for +all frame sizes and for all tests are presented in detailed results +tables. + +## SOAK Tests + +These tests employ PLRsearch to find a critical load value. + +### Algorithm Details + +See [PLRSearch]({{< ref "plr_search/#PLRsearch" >}}) methodology section for +more detail. PLRsearch is being standardized in IETF in +[draft-vpolak-bmwg-plrsearch](https://tools.ietf.org/html/draft-vpolak-bmwg-plrsearch). + +### Description + +Probabilistic Loss Ratio search (PLRsearch) tests discovers a packet +throughput rate associated with configured Packet Loss Ratio (PLR) +target for tests run over an extended period of time a.k.a. soak +testing. PLRsearch assumes that system under test is probabilistic in +nature, and not deterministic. + +### Usage + +PLRsearch are run to discover a critical load for PLR=10^-7^ +(close to NDR) for VPP release covered by CSIT report. Results for small +frame sizes (64B/78B) are presented in packet throughput graphs (Box +Plots) for a small subset of baseline tests. + +Each soak test lasts 30 minutes and is executed at least twice. + +## MRR Tests + +### Algorithm Details + +See [MRR Throughput]({{< ref "mrr/#MRR" >}}) +section for more detail about MRR tests configuration. + +FD.io CSIT performance dashboard includes complete description of +[daily performance trending tests]({{< ref "../../trending/analysis" >}}) +and [VPP per patch tests]({{< ref "../../per_patch_testing.md" >}}). + +### Description + +Maximum Receive Rate (MRR) tests are complementary to MLRsearch tests, +as they provide a maximum “raw” throughput benchmark for development and +testing community. + +MRR tests measure the packet forwarding rate under the maximum load +offered by traffic generator (dependent on link type and NIC model) over +a set trial duration, regardless of packet loss. Maximum load for +specified Ethernet frame size is set to the bi-directional link rate. + +### Usage + +MRR tests are much faster than MLRsearch as they rely on +a small set of trials with very short duration. It is this property +that makes them suitable for continuous execution in daily performance +trending jobs enabling detection of performance anomalies (regressions, +progressions) resulting from data plane code changes. + +MRR tests are also used for VPP per patch performance jobs verifying +patch performance vs parent. CSIT reports include MRR throughput +comparisons between releases and test environments. Small frame sizes +only (64B/78B, IMIX). + +## Generic Test Properties + +All data plane throughput test methodologies share following generic +properties: + +- Tested L2 frame sizes (untagged Ethernet): + + - IPv4 payload: 64B, IMIX (28x64B, 16x570B, 4x1518B), 1518B, 9000B. + - IPv6 payload: 78B, IMIX (28x78B, 16x570B, 4x1518B), 1518B, 9000B. + - All quoted sizes include frame CRC, but exclude per frame + transmission overhead of 20B (preamble, inter frame gap). + +- Offered packet load is always bi-directional and symmetric. +- All measured and reported packet and bandwidth rates are aggregate + bi-directional rates reported from external Traffic Generator + perspective. diff --git a/docs/content/methodology/measurements/data_plane_throughput/mlr_search.md b/docs/content/methodology/measurements/data_plane_throughput/mlr_search.md new file mode 100644 index 0000000000..71e4471905 --- /dev/null +++ b/docs/content/methodology/measurements/data_plane_throughput/mlr_search.md @@ -0,0 +1,112 @@ +--- +title: "MLR Search" +weight: 2 +--- + +# MLR Search + +## Overview + +Multiple Loss Ratio search (MLRsearch) tests use an optimized search algorithm +implemented in FD.io CSIT project. MLRsearch discovers conditional throughput +corresponding to any number of loss ratio goals, within a single search. + +Two loss ratio goals are of interest in FD.io CSIT, leading to Non-Drop Rate +(NDR, loss ratio goal is exact zero) and Partial Drop Rate +(PDR, 0.5% loss ratio goal). +Instead of a single long trial, a sequence of short (1s) trials is done. +Thus, instead of final trial duration, a duration sum (21s) is prescribed. +This allows the algorithm to make a decision sooner, +when the results are quite one-sided. +Also, only one half of the trial results is required to meet +the loss ratio requirement, making the conditional throughput more stable. +The conditional throughput in this case is in principle the median forwarding rate +among all trials at the relevant lower bound intended load. +In practice, the search stops when missing trial results cannot +disprove the load as a lower bound, so conditional throughput +is the worst forwarding rate among the measured good trials. + +MLRsearch discovers all the loads in single search, reducing required time +duration compared to separate `binary search`es[^1] for each rate. Overall +search time is reduced even further by relying on shorter trial +duration sums for intermediate targets, with only measurements for +final targets require the full duration sum. This results in the +shorter overall execution time when compared to standard NDR/PDR binary +search, while guaranteeing similar results. + + Note: The conditional throughput is *always* reported by Robot code + as a bi-directional aggregate of two (usually symmetric) + uni-directional packet rates received and reported by an + external traffic generator (TRex), unless the test specifically requires + unidirectional traffic. The underlying Python library uses + unidirectional values instead, as min and max load are given for those. + +## Search Implementation + +Detailed description of the MLRsearch algorithm is included in the IETF +draft +[draft-ietf-bmwg-mlrsearch](https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-mlrsearch) +that is in the process of being standardized in the IETF Benchmarking +Methodology Working Group (BMWG). + +MLRsearch is also available as a +[PyPI (Python Package Index) library](https://pypi.org/project/MLRsearch/). + +## Algorithm highlights + +MRR and receive rate at MRR load are used as initial guesses for the search. + +All previously measured trials (except the very first one which acts +as a warm-up) are taken into consideration. + +For every loss ratio goal, the relevant upper and lower bound +(intended loads, among loads of large enough duration sum) form an interval. +Exit condition is given by that interval reaching low enough relative width. +Small enough width is achieved by bisecting the current interval. +The bisection can be uneven, to save measurements based on information theory. +The width value is 0.5%, the same as PDR goal loss ratio, +as smaller values may report PDR conditional throughput smaller than NDR. + +Switching to higher trial duration sum generally requires additional trials +at a load from previous duration sum target. +When this refinement does not confirm previous bound classification +(e.g. a lower bound for preceding target +becomes an upper bound of the new target due to new trail results), +external search is used to find close enough bound of the lost type. +External search is a generalization of the first stage of +`exponential search`[^2]. + +A preceding target uses double of the next width goal, +because one bisection is always safe before risking external search. + +As different search targets are interested at different loads, +lower intended load are measured first, +as that approach saves more time when trial results are not very consistent. +Other heuristics are there, aimed to prevent unneccessarily narrow intervals, +and to handle corner cases around min and max load. + +## Deviations from RFC 2544 + +RFC 2544 implies long final trial duration (just one long trial is needed +for classification to lower or uper bound, so exceed ratio does not matter). +With 1s trials and 0.5 exceed ratio, NDR values reported by CSIT +are likely higher than RFC 2544 throughput (especially for less stable tests). + +CSIT does not have any explicit wait times before and after trial traffic. +(But the TRex-based measurer takes almost half a second between targets.) + +Small difference between intended load and offered load is tolerated, +mainly due to various time overheads preventing precise measurement +of the traffic duration (and TRex can sometimes suffer from duration +stretching). Large difference is reported as unsent packets +(measurement is forcibly stopped after given time), counted as +a packet loss, so search focuses on loads actually achievable by TRex. + +In some tests, negative loss count is observed (TRex sees more packets +coming back to it than TRex sent this trial). CSIT code treats that +as a packet loss (as if VPP duplicated the packets), +but TRex does not check other packets for duplication +(as many traffic profiles generate non-unique packets). + +[^1]: [binary search](https://en.wikipedia.org/wiki/Binary_search) +[^2]: [exponential search](https://en.wikipedia.org/wiki/Exponential_search) diff --git a/docs/content/methodology/measurements/data_plane_throughput/mrr.md b/docs/content/methodology/measurements/data_plane_throughput/mrr.md new file mode 100644 index 0000000000..e8c3e62eb6 --- /dev/null +++ b/docs/content/methodology/measurements/data_plane_throughput/mrr.md @@ -0,0 +1,56 @@ +--- +title: "MRR" +weight: 4 +--- + +# MRR + +Maximum Receive Rate (MRR) tests are complementary to MLRsearch tests, +as they provide a maximum "raw" throughput benchmark for development and +testing community. MRR tests measure the packet forwarding rate under +the maximum load offered by traffic generator over a set trial duration, +regardless of packet loss. + +MRR tests are currently used for following test jobs: + +- Report performance comparison: 64B, IMIX for vhost, memif. +- Daily performance trending: 64B, IMIX for vhost, memif. +- Per-patch performance verification: 64B. +- Initial iterations of MLRsearch and PLRsearch: 64B. + +Maximum offered load for specific L2 Ethernet frame size is set to +either the maximum bi-directional link rate or tested NIC model +capacity, as follows: + +- For 10GE NICs the maximum packet rate load is 2x14.88 Mpps for 64B, a + 10GE bi-directional link rate. +- For 25GE NICs the maximum packet rate load is 2x18.75 Mpps for 64B, a + 25GE bi-directional link sub-rate limited by 25GE NIC used on TRex TG, + XXV710. +- For 40GE NICs the maximum packet rate load is 2x18.75 Mpps for 64B, a + 40GE bi-directional link sub-rate limited by 40GE NIC used on TRex + TG, XL710. Packet rate for other tested frame sizes is limited by + PCIeGen3 x8 bandwidth limitation of ~50Gbps. + +MRR test code implements multiple bursts of offered packet load and has +two configurable burst parameters: individual trial duration and number +of trials in a single burst. This enables more precise performance +trending by providing more results data for analysis. + +Burst parameter settings vary between different tests using MRR: + +- MRR individual trial duration: + + - Report performance comparison: 1 sec. + - Daily performance trending: 1 sec. + - Per-patch performance verification: 10 sec. + - Initial iteration for MLRsearch: 1 sec. + - Initial iteration for PLRsearch: 5.2 sec. + +- Number of MRR trials per burst: + + - Report performance comparison: 10. + - Daily performance trending: 10. + - Per-patch performance verification: 5. + - Initial iteration for MLRsearch: 1. + - Initial iteration for PLRsearch: 1. diff --git a/docs/content/methodology/measurements/data_plane_throughput/plr_search.md b/docs/content/methodology/measurements/data_plane_throughput/plr_search.md new file mode 100644 index 0000000000..6f208c1ece --- /dev/null +++ b/docs/content/methodology/measurements/data_plane_throughput/plr_search.md @@ -0,0 +1,386 @@ +--- +title: "PLR Search" +weight: 3 +--- + +# PLR Search + +## Motivation for PLRsearch + +Network providers are interested in throughput a system can sustain. + +`RFC 2544`[^1] assumes loss ratio is given by a deterministic function of +offered load. But NFV software systems are not deterministic enough. +This makes deterministic algorithms (such as `binary search`[^2] per RFC 2544 +and MLRsearch with single trial) to return results, +which when repeated show relatively high standard deviation, +thus making it harder to tell what "the throughput" actually is. + +We need another algorithm, which takes this indeterminism into account. + +## Generic Algorithm + +Detailed description of the PLRsearch algorithm is included in the IETF +draft `Probabilistic Loss Ratio Search for Packet Throughput`[^3] that is in the +process of being standardized in the IETF Benchmarking Methodology Working Group +(BMWG). + +### Terms + +The rest of this page assumes the reader is familiar with the following terms +defined in the IETF draft: + ++ Trial Order Independent System ++ Duration Independent System ++ Target Loss Ratio ++ Critical Load ++ Offered Load regions + + + Zero Loss Region + + Non-Deterministic Region + + Guaranteed Loss Region + ++ Fitting Function + + + Stretch Function + + Erf Function + ++ Bayesian Inference + + + Prior distribution + + Posterior Distribution + ++ Numeric Integration + + + Monte Carlo + + Importance Sampling + +## FD.io CSIT Implementation Specifics + +The search receives min_rate and max_rate values, to avoid measurements +at offered loads not supporeted by the traffic generator. + +The implemented tests cases use bidirectional traffic. +The algorithm stores each rate as bidirectional rate (internally, +the algorithm is agnostic to flows and directions, +it only cares about aggregate counts of packets sent and packets lost), +but debug output from traffic generator lists unidirectional values. + +In CSIT, tests that employ PLRsearch are identified as SOAK tests, +the search time is set to 30 minuts. + +### Measurement Delay + +In a sample implemenation in FD.io CSIT project, there is roughly 0.5 +second delay between trials due to restrictons imposed by packet traffic +generator in use (T-Rex). + +As measurements results come in, posterior distribution computation takes +more time (per sample), although there is a considerable constant part +(mostly for inverting the fitting functions). + +Also, the integrator needs a fair amount of samples to reach the region +the posterior distribution is concentrated at. + +And of course, the speed of the integrator depends on computing power +of the CPU the algorithm is able to use. + +All those timing related effects are addressed by arithmetically increasing +trial durations with configurable coefficients +(currently 5.1 seconds for the first trial, +each subsequent trial being 0.1 second longer). + +### Rounding Errors and Underflows + +In order to avoid them, the current implementation tracks natural logarithm +(instead of the original quantity) for any quantity which is never negative. +Logarithm of zero is minus infinity (not supported by Python), +so special value "None" is used instead. +Specific functions for frequent operations (such as "logarithm +of sum of exponentials") are defined to handle None correctly. + +### Fitting Functions + +Current implementation uses two fitting functions, called "stretch" and "erf". +In general, their estimates for critical rate differ, +which adds a simple source of systematic error, +on top of randomness error reported by integrator. +Otherwise the reported stdev of critical rate estimate +is unrealistically low. + +Both functions are not only increasing, but also convex +(meaning the rate of increase is also increasing). + +Both fitting functions have several mathematically equivalent formulas, +each can lead to an arithmetic overflow or underflow in different sub-terms. +Overflows can be eliminated by using different exact formulas +for different argument ranges. +Underflows can be avoided by using approximate formulas +in affected argument ranges, such ranges have their own formulas to compute. +At the end, both fitting function implementations +contain multiple "if" branches, discontinuities are a possibility +at range boundaries. + +### Prior Distributions + +The numeric integrator expects all the parameters to be distributed +(independently and) uniformly on an interval (-1, 1). + +As both "mrr" and "spread" parameters are positive and not dimensionless, +a transformation is needed. Dimentionality is inherited from max_rate value. + +The "mrr" parameter follows a `Lomax distribution`[^4] +with alpha equal to one, but shifted so that mrr is always greater than 1 +packet per second. + +The "stretch" parameter is generated simply as the "mrr" value +raised to a random power between zero and one; +thus it follows a `reciprocal distribution`[^5]. + +### Integrator + +After few measurements, the posterior distribution of fitting function +arguments gets quite concentrated into a small area. +The integrator is using `Monte Carlo`[^6] with `importance sampling`[^7] +where the biased distribution is `bivariate Gaussian`[^8] distribution, +with deliberately larger variance. +If the generated sample falls outside (-1, 1) interval, +another sample is generated. + +The center and the covariance matrix for the biased distribution +is based on the first and second moments of samples seen so far +(within the computation). The center is used directly, +covariance matrix is scaled up by a heurictic constant (8.0 by default). +The following additional features are applied +designed to avoid hyper-focused distributions. + +Each computation starts with the biased distribution inherited +from the previous computation (zero point and unit covariance matrix +is used in the first computation), but the overal weight of the data +is set to the weight of the first sample of the computation. +Also, the center is set to the first sample point. +When additional samples come, their weight (including the importance correction) +is compared to sum of the weights of data seen so far (within the iteration). +If the new sample is more than one e-fold more impactful, both weight values +(for data so far and for the new sample) are set to (geometric) average +of the two weights. + +This combination showed the best behavior, as the integrator usually follows +two phases. First phase (where inherited biased distribution +or single big sample are dominating) is mainly important +for locating the new area the posterior distribution is concentrated at. +The second phase (dominated by whole sample population) +is actually relevant for the critical rate estimation. + +### Offered Load Selection + +First two measurements are hardcoded to happen at the middle of rate interval +and at max_rate. Next two measurements follow MRR-like logic, +offered load is decreased so that it would reach target loss ratio +if offered load decrease lead to equal decrease of loss rate. + +The rest of measurements start directly in between +erf and stretch estimate average. +There is one workaround implemented, aimed at reducing the number of consequent +zero loss measurements (per fitting function). The workaround first stores +every measurement result which loss ratio was the targed loss ratio or higher. +Sorted list (called lossy loads) of such results is maintained. + +When a sequence of one or more zero loss measurement results is encountered, +a smallest of lossy loads is drained from the list. +If the estimate average is smaller than the drained value, +a weighted average of this estimate and the drained value is used +as the next offered load. The weight of the estimate decreases exponentially +with the length of consecutive zero loss results. + +This behavior helps the algorithm with convergence speed, +as it does not need so many zero loss result to get near critical region. +Using the smallest (not drained yet) of lossy loads makes it sure +the new offered load is unlikely to result in big loss region. +Draining even if the estimate is large enough helps to discard +early measurements when loss hapened at too low offered load. +Current implementation adds 4 copies of lossy loads and drains 3 of them, +which leads to fairly stable behavior even for somewhat inconsistent SUTs. + +### Caveats + +As high loss count measurements add many bits of information, +they need a large amount of small loss count measurements to balance them, +making the algorithm converge quite slowly. Typically, this happens +when few initial measurements suggest spread way bigger then later measurements. +The workaround in offered load selection helps, +but more intelligent workarounds could get faster convergence still. + +Some systems evidently do not follow the assumption of repeated measurements +having the same average loss rate (when the offered load is the same). +The idea of estimating the trend is not implemented at all, +as the observed trends have varied characteristics. + +Probably, using a more realistic fitting functions +will give better estimates than trend analysis. + +## Bottom Line + +The notion of Throughput is easy to grasp, but it is harder to measure +with any accuracy for non-deterministic systems. + +Even though the notion of critical rate is harder to grasp than the notion +of throughput, it is easier to measure using probabilistic methods. + +In testing, the difference between througput measurements and critical +rate measurements is usually small. + +In pactice, rules of thumb such as "send at max 95% of purported throughput" +are common. The correct benchmarking analysis should ask "Which notion is +95% of throughput an approximation to?" before attempting to answer +"Is 95% of critical rate safe enough?". + +## Algorithmic Analysis + +### Motivation + +While the estimation computation is based on hard probability science; +the offered load selection part of PLRsearch logic is pure heuristics, +motivated by what would a human do based on measurement and computation results. + +The quality of any heuristic is not affected by soundness of its motivation, +just by its ability to achieve the intended goals. +In case of offered load selection, the goal is to help the search to converge +to the long duration estimates sooner. + +But even those long duration estimates could still be of poor quality. +Even though the estimate computation is Bayesian (so it is the best it could be +within the applied assumptions), it can still of poor quality when compared +to what a human would estimate. + +One possible source of poor quality is the randomnes inherently present +in Monte Carlo numeric integration, but that can be supressed +by tweaking the time related input parameters. + +The most likely source of poor quality then are the assumptions. +Most importantly, the number and the shape of fitting functions; +but also others, such as trial order independence and duration independence. + +The result can have poor quality in basically two ways. +One way is related to location. Both upper and lower bounds +can be overestimates or underestimates, meaning the entire estimated interval +between lower bound and upper bound lays above or below (respectively) +of human-estimated interval. +The other way is related to the estimation interval width. +The interval can be too wide or too narrow, compared to human estimation. + +An estimate from a particular fitting function can be classified +as an overestimate (or underestimate) just by looking at time evolution +(without human examining measurement results). Overestimates +decrease by time, underestimates increase by time (assuming +the system performance stays constant). + +Quality of the width of the estimation interval needs human evaluation, +and is unrelated to both rate of narrowing (both good and bad estimate intervals +get narrower at approximately the same relative rate) and relatative width +(depends heavily on the system being tested). + +### Graphical Examples + +The following pictures show the upper (red) and lower (blue) bound, +as well as average of Stretch (pink) and Erf (light green) estimate, +and offered load chosen (grey), as computed by PLRsearch, +after each trial measurement within the 30 minute duration of a test run. + +Both graphs are focusing on later estimates. Estimates computed from +few initial measurements are wildly off the y-axis range shown. + +The following analysis will rely on frequency of zero loss measurements +and magnitude of loss ratio if nonzero. + +The offered load selection strategy used implies zero loss measurements +can be gleaned from the graph by looking at offered load points. +When the points move up farther from lower estimate, it means +the previous measurement had zero loss. After non-zero loss, +the offered load starts again right between (the previous values of) +the estimate curves. + +The very big loss ratio results are visible as noticeable jumps +of both estimates downwards. Medium and small loss ratios are much harder +to distinguish just by looking at the estimate curves, +the analysis is based on raw loss ratio measurement results. + +The following descriptions should explain why the graphs seem to signal +low quality estimate at first sight, but a more detailed look +reveals the quality is good (considering the measurement results). + +#### L2 patch + +Both fitting functions give similar estimates, the graph shows +"stochasticity" of measurements (estimates increase and decrease +within small time regions), and an overall trend of decreasing estimates. + +On the first look, the final interval looks fairly narrow, +especially compared to the region the estimates have travelled +during the search. But the look at the frequency of zero loss results shows +this is not a case of overestimation. Measurements at around the same +offered load have higher probability of zero loss earlier +(when performed farther from upper bound), but smaller probability later +(when performed closer to upper bound). That means it is the performance +of the system under test that decreases (slightly) over time. + +With that in mind, the apparent narrowness of the interval +is not a sign of low quality, just a consequence of PLRsearch assuming +the performance stays constant. + +{{< figure src="/cdocs/PLR_patch.svg" >}} + +#### Vhost + +This test case shows what looks like a quite broad estimation interval, +compared to other test cases with similarly looking zero loss frequencies. +Notable features are infrequent high-loss measurement results +causing big drops of estimates, and lack of long-term convergence. + +Any convergence in medium-sized intervals (during zero loss results) +is reverted by the big loss results, as they happen quite far +from the critical load estimates, and the two fitting functions +extrapolate differently. + +In other words, human only seeing estimates from one fitting function +would expect narrower end interval, but human seeing the measured loss ratios +agrees that the interval should be wider than that. + +{{< figure src="/cdocs/PLR_vhost.svg" >}} + +#### Summary + +The two graphs show the behavior of PLRsearch algorithm applied to soak test +when some of PLRsearch assumptions do not hold: + ++ L2 patch measurement results violate the assumption + of performance not changing over time. ++ Vhost measurement results violate the assumption + of Poisson distribution matching the loss counts. + +The reported upper and lower bounds can have distance larger or smaller +than a first look by a human would expect, but a more closer look reveals +the quality is good, considering the circumstances. + +The usefullness of the critical load estimate is of questionable value +when the assumptions are violated. + +Some improvements can be made via more specific workarounds, +for example long term limit of L2 patch performance could be estmated +by some heuristic. + +Other improvements can be achieved only by asking users +whether loss patterns matter. Is it better to have single digit losses +distributed fairly evenly over time (as Poisson distribution would suggest), +or is it better to have short periods of medium losses +mixed with long periods of zero losses (as happens in Vhost test) +with the same overall loss ratio? + +[^1]: [RFC 2544: Benchmarking Methodology for Network Interconnect Devices](https://tools.ietf.org/html/rfc2544) +[^2]: [Binary search](https://en.wikipedia.org/wiki/Binary_search_algorithm) +[^3]: [Probabilistic Loss Ratio Search for Packet Throughput](https://tools.ietf.org/html/draft-vpolak-bmwg-plrsearch-02) +[^4]: [Lomax distribution](https://en.wikipedia.org/wiki/Lomax_distribution) +[^5]: [Reciprocal distribution](https://en.wikipedia.org/wiki/Reciprocal_distribution) +[^6]: [Monte Carlo](https://en.wikipedia.org/wiki/Monte_Carlo_integration) +[^7]: [Importance sampling](https://en.wikipedia.org/wiki/Importance_sampling) +[^8]: [Bivariate Gaussian](https://en.wikipedia.org/wiki/Multivariate_normal_distribution) diff --git a/docs/content/methodology/measurements/packet_latency.md b/docs/content/methodology/measurements/packet_latency.md new file mode 100644 index 0000000000..f3606b5ffb --- /dev/null +++ b/docs/content/methodology/measurements/packet_latency.md @@ -0,0 +1,52 @@ +--- +title: "Packet Latency" +weight: 2 +--- + +# Packet Latency + +TRex Traffic Generator (TG) is used for measuring one-way latency in +2-Node and 3-Node physical testbed topologies. TRex integrates +[High Dynamic Range Histogram (HDRH)](http://hdrhistogram.org/) +functionality and reports per packet latency distribution for latency +streams sent in parallel to the main load packet streams. + +Following methodology is used: + +- Only NDRPDR test type measures latency and only after NDR and PDR + values are determined. Other test types do not involve latency + streams. + +- Latency is measured at different background load packet rates: + + - No-Load: latency streams only. + - Low-Load: at 10% PDR. + - Mid-Load: at 50% PDR. + - High-Load: at 90% PDR. + +- Latency is measured for all tested packet sizes except IMIX due to + TRex TG restriction. + +- TG sends dedicated latency streams, one per direction, each at the + rate of 9 kpps at the prescribed packet size; these are sent in + addition to the main load streams. + +- TG reports Min/Avg/Max and HDRH latency values distribution per stream + direction, hence two sets of latency values are reported per test case + (marked as E-W and W-E). + +- +/- 1 usec is the measurement accuracy of TRex TG and the data in HDRH + latency values distribution is rounded to microseconds. + +- TRex TG introduces a (background) always-on Tx + Rx latency bias of 4 + usec on average per direction resulting from TRex software writing and + reading packet timestamps on CPU cores. Quoted values are based on TG + back-to-back latency measurements. + +- Latency graphs are not smoothed, each latency value has its own + horizontal line across corresponding packet percentiles. + +- Percentiles are shown on X-axis using a logarithmic scale, so the + maximal latency value (ending at 100% percentile) would be in + infinity. The graphs are cut at 99.9999% (hover information still + lists 100%). diff --git a/docs/content/methodology/measurements/telemetry.md b/docs/content/methodology/measurements/telemetry.md new file mode 100644 index 0000000000..aed32d9e17 --- /dev/null +++ b/docs/content/methodology/measurements/telemetry.md @@ -0,0 +1,158 @@ +--- +title: "Telemetry" +weight: 3 +--- + +# Telemetry + +OpenMetrics specifies the de-facto standard for transmitting cloud-native +metrics at scale, with support for both text representation and Protocol +Buffers. + +## RFC + +- RFC2119 +- RFC5234 +- RFC8174 +- draft-richih-opsawg-openmetrics-00 + +## Reference + +[OpenMetrics](https://github.com/OpenObservability/OpenMetrics/blob/master/specification/OpenMetrics.md) + +## Metric Types + +- Gauge +- Counter +- StateSet +- Info +- Histogram +- GaugeHistogram +- Summary +- Unknown + +Telemetry module in CSIT currently support only Gauge, Counter and Info. + +## Anatomy of CSIT telemetry implementation + +Existing implementation consists of several measurment building blocks: +the main measuring block running search algorithms (MLR, PLR, SOAK, MRR, ...), +the latency measuring block and the several telemetry blocks with or without +traffic running on a background. + +The main measuring block must not be interrupted by any read operation that can +impact data plane traffic processing during throughput search algorithm. Thus +operational reads are done before (pre-stat) and after (post-stat) that block. + +Some operational reads must be done while traffic is running and usually +consists of two reads (pre-run-stat, post-run-stat) with defined delay between +them. + +## MRR measurement + + traffic_start(r=mrr) traffic_stop |< measure >| + | | | (r=mrr) | + | pre_run_stat post_run_stat | pre_stat | | post_stat + | | | | | | | | + o--------o---------------o-------o------o------+---------------+------o------> + t + Legend: + - pre_run_stat + - vpp-clear-runtime + - post_run_stat + - vpp-show-runtime + - bash-perf-stat // if extended_debug == True + - pre_stat + - vpp-clear-stats + - vpp-enable-packettrace // if extended_debug == True + - vpp-enable-elog + - post_stat + - vpp-show-stats + - vpp-show-packettrace // if extended_debug == True + - vpp-show-elog + + |< measure >| + | (r=mrr) | + | | + |< traffic_trial0 >|< traffic_trial1 >|< traffic_trialN >| + | (i=0,t=duration) | (i=1,t=duration) | (i=N,t=duration) | + | | | | + o-----------------------o------------------------o------------------------o---> + t + + +## MLR measurement + + |< measure >| traffic_start(r=pdr) traffic_stop traffic_start(r=ndr) traffic_stop |< [ latency ] >| + | (r=mlr) | | | | | | .9/.5/.1/.0 | + | | | pre_run_stat post_run_stat | | pre_run_stat post_run_stat | | | + | | | | | | | | | | | | + +-------------+---o-------o---------------o--------o-------------o-------o---------------o--------o------------[-------------------]---> + t + Legend: + - pre_run_stat + - vpp-clear-runtime + - post_run_stat + - vpp-show-runtime + - bash-perf-stat // if extended_debug == True + - pre_stat + - vpp-clear-stats + - vpp-enable-packettrace // if extended_debug == True + - vpp-enable-elog + - post_stat + - vpp-show-stats + - vpp-show-packettrace // if extended_debug == True + - vpp-show-elog + +## MRR measurement + + traffic_start(r=mrr) traffic_stop |< measure >| + | | | (r=mrr) | + | |< stat_runtime >| | stat_pre_trial | | stat_post_trial + | | | | | | | | + o---+------------------+---o------o------------+-------------+----o------------> + t + Legend: + - stat_runtime + - vpp-runtime + - stat_pre_trial + - vpp-clear-stats + - vpp-enable-packettrace // if extended_debug == True + - stat_post_trial + - vpp-show-stats + - vpp-show-packettrace // if extended_debug == True + + |< measure >| + | (r=mrr) | + | | + |< traffic_trial0 >|< traffic_trial1 >|< traffic_trialN >| + | (i=0,t=duration) | (i=1,t=duration) | (i=N,t=duration) | + | | | | + o------------------------o------------------------o------------------------o---> + t + + |< stat_runtime >| + | | + |< program0 >|< program1 >|< programN >| + | (@=params) | (@=params) | (@=params) | + | | | | + o------------------------o------------------------o------------------------o---> + t + +## MLR measurement + + |< measure >| traffic_start(r=pdr) traffic_stop traffic_start(r=ndr) traffic_stop |< [ latency ] >| + | (r=mlr) | | | | | | .9/.5/.1/.0 | + | | | |< stat_runtime >| | | |< stat_runtime >| | | | + | | | | | | | | | | | | + +-------------+---o---+------------------+---o--------------o---+------------------+---o-----------[-----------------]---> + t + Legend: + - stat_runtime + - vpp-runtime + - stat_pre_trial + - vpp-clear-stats + - vpp-enable-packettrace // if extended_debug == True + - stat_post_trial + - vpp-show-stats + - vpp-show-packettrace // if extended_debug == True diff --git a/docs/content/methodology/overview/_index.md b/docs/content/methodology/overview/_index.md new file mode 100644 index 0000000000..d7efd15d02 --- /dev/null +++ b/docs/content/methodology/overview/_index.md @@ -0,0 +1,15 @@ +--- +bookCollapseSection: true +bookFlatSection: false +title: "Overview" +weight: 1 +--- + +# Methodology + +- [Terminology]({{< relref "/methodology/overview/terminology" >}}) +- [Per Thread Resources]({{< relref "/methodology/overview/per_thread_resources" >}}) +- [Multi-Core Speedup]({{< relref "/methodology/overview/multi_core_speedup" >}}) +- [VPP Forwarding Modes]({{< relref "/methodology/overview/vpp_forwarding_modes" >}}) +- [DUT State Considerations]({{< relref "/methodology/overview/dut_state_considerations" >}}) +- [TRex Traffic Generator]({{< relref "/methodology/overview/trex_traffic_generator" >}}) diff --git a/docs/content/methodology/overview/dut_state_considerations.md b/docs/content/methodology/overview/dut_state_considerations.md new file mode 100644 index 0000000000..eca10a22cd --- /dev/null +++ b/docs/content/methodology/overview/dut_state_considerations.md @@ -0,0 +1,148 @@ +--- +title: "DUT State Considerations" +weight: 5 +--- + +# DUT State Considerations + +This page discusses considerations for Device Under Test (DUT) state. +DUTs such as VPP require configuration, to be provided before the aplication +starts (via config files) or just after it starts (via API or CLI access). + +During operation DUTs gather various telemetry data, depending on configuration. +This internal state handling is part of normal operation, +so any performance impact is included in the test results. +Accessing telemetry data is additional load on DUT, +so we are not doing that in main trial measurements that affect results, +but we include separate trials specifically for gathering runtime telemetry. + +But there is one kind of state that needs specific handling. +This kind of DUT state is dynamically created based on incoming traffic, +it affects how DUT handles the traffic, and (unlike telemetry counters) +it has uneven impact on CPU load. +Typical example is NAT, where detecting new sessions takes more CPU than +forwarding packet on existing (open or recently closed) sessions. +We call DUT configurations with this kind of state "stateful", +and configurations without them "stateless". +(Even though stateless configurations contain state described in previous +paragraphs, and some configuration items may have "stateful" in their name, +such as stateful ACLs.) + +# Stateful DUT configurations + +Typically, the level of CPU impact of traffic depends on DUT state. +The first packets causing DUT state to change have higher impact, +subsequent packets matching that state have lower impact. + +From performance point of view, this is similar to traffic phases +for stateful protocols, see +[NGFW draft](https://tools.ietf.org/html/draft-ietf-bmwg-ngfw-performance-05#section-4.3.4). +In CSIT we borrow the terminology (even if it does not fit perfectly, +see discussion below). Ramp-up traffic causes the state change, +sustain traffic does not change the state. + +As the performance is different, each test has to choose which traffic +it wants to test, and manipulate the DUT state to achieve the intended impact. + +## Ramp-up trial + +Tests aiming at sustain performance need to make sure DUT state is created. +We achieve this via a ramp-up trial, specific purpose of which +is to create the state. + +Subsequent trials need no specific handling, as long as the state +remains the same. But some state can time-out, so additional ramp-up +trials are inserted whenever the code detects the state can time-out. +Note that a trial with zero loss refreshes the state, +so only the time since the last non-zero loss trial is tracked. + +For the state to be set completely, it is important both DUT and TG +do not lose any packets. We achieve this by setting the profile multiplier +(TPS from now on) to low enough value. + +It is also important each state-affecting packet is sent. +For size-limited traffic profile it is guaranteed by the size limit. +For continuous traffic, we set a long enough duration (based on TPS). + +At the end of the ramp-up trial, we check DUT state to confirm +it has been created as expected. +Test fails if the state is not (completely) created. + +## State Reset + +Tests aiming at ramp-up performance do not use ramp-up trial, +and they need to reset the DUT state before each trial measurement. +The way of resetting the state depends on test, +usually an API call is used to partially de-configure +the part that holds the state, and then re-configure it back. + +In CSIT we control the DUT state behavior via a test variable "resetter". +If it is not set, DUT state is not reset. +If it is set, each search algorithm (including MRR) will invoke it +before all trial measurements (both main and telemetry ones). +Any configuration keyword enabling a feature with DUT state +will check whether a test variable for ramp-up rate is present. +If it is present, resetter is not set. +If it is not present, the keyword sets the apropriate resetter value. +This logic makes sure either ramp-up or state reset are used. + +Notes: If both ramp-up and state reset were used, the DUT behavior +would be identical to just reset, while test would take longer to execute. +If neither were used, DUT will show different performance in subsequent trials, +violating assumptions of search algorithms. + +## DUT versus protocol ramp-up + +There are at least three different causes for bandwidth possibly increasing +within a single measurement trial. + +The first is DUT switching from state modification phase to constant phase, +it is the primary focus of this document. +Using ramp-up traffic before main trials eliminates this cause +for tests wishing to measure the performance of the next phase. +Using size-limited profiles eliminates the next phase +for tests wishing to measure performance of this phase. + +The second is protocol such as TCP ramping up their throughput to utilize +the bandwidth available. This is the original meaning of "ramp up" +in the NGFW draft (see above). +In existing tests we are not using this meaning of TCP ramp-up. +Instead we use only small transactions, and large enough initial window +so TCP acts as ramped-up already. + +The third is TCP increasing offered load due to retransmissions triggered by +packet loss. In CSIT we again try to avoid this behavior +by using small enough data to transfer, so overlap of multiple transactions +(primary cause of packet loss) is unlikely. +But in MRR tests, packet loss and non-constant offered load are still expected. + +# Stateless DUT configuratons + +These are simple configurations, which do not set any resetter value +(even if ramp-up duration is not configured). +Majority of existing tests are of this type, using continuous traffic profiles. + +In order to identify limits of Trex performance, +we have added suites with stateless DUT configuration (VPP ip4base) +subjected to size-limited ASTF traffic. +The discovered rates serve as a basis of comparison +for evaluating the results for stateful DUT configurations (VPP NAT44ed) +subjected to the same traffic profiles. + +# DUT versus TG state + +Traffic Generator profiles can be stateful (ASTF) or stateless (STL). +DUT configuration can be stateful or stateless (with respect to packet traffic). + +In CSIT we currently use all four possible configurations: + +- Regular stateless VPP tests use stateless traffic profiles. + +- Stateless VPP configuration with stateful profile is used as a base for + comparison. + +- Some stateful DUT configurations (NAT44DET, NAT44ED unidirectional) + are tested using stateless traffic profiles and continuous traffic. + +- The rest of stateful DUT configurations (NAT44ED bidirectional) + are tested using stateful traffic profiles and size limited traffic. diff --git a/docs/content/methodology/overview/multi_core_speedup.md b/docs/content/methodology/overview/multi_core_speedup.md new file mode 100644 index 0000000000..f438e8e996 --- /dev/null +++ b/docs/content/methodology/overview/multi_core_speedup.md @@ -0,0 +1,51 @@ +--- +title: "Multi-Core Speedup" +weight: 3 +--- + +# Multi-Core Speedup + +All performance tests are executed with single physical core and with +multiple cores scenarios. + +## Intel Hyper-Threading (HT) + +Intel Xeon processors used in FD.io CSIT can operate either in HT +Disabled mode (single logical core per each physical core) or in HT +Enabled mode (two logical cores per each physical core). HT setting is +applied in BIOS and requires server SUT reload for it to take effect, +making it impractical for continuous changes of HT mode of operation. + +Performance tests are executed with server SUTs' Intel XEON processors +configured with Intel Hyper-Threading Enabled for all Xeon +Cascadelake and Xeon Icelake testbeds. + +## Multi-core Tests + +Multi-core tests are executed in the following VPP worker thread and physical +core configurations: + +1. Intel Xeon Icelake and Cascadelake testbeds (2n-icx, 3n-icx, 2n-clx) + with Intel HT enabled (2 logical CPU cores per each physical core): + + 1. 2t1c - 2 VPP worker threads on 1 physical core. + 2. 4t2c - 4 VPP worker threads on 2 physical cores. + 3. 8t4c - 8 VPP worker threads on 4 physical cores. + +VPP worker threads are the data plane threads running on isolated +logical cores. With Intel HT enabled VPP workers are placed as sibling +threads on each used physical core. VPP control threads (main, stats) +are running on a separate non-isolated core together with other Linux +processes. + +In all CSIT tests care is taken to ensure that each VPP worker handles +the same amount of received packet load and does the same amount of +packet processing work. This is achieved by evenly distributing per +interface type (e.g. physical, virtual) receive queues over VPP workers +using default VPP round-robin mapping and by loading these queues with +the same amount of packet flows. + +If number of VPP workers is higher than number of physical or virtual +interfaces, multiple receive queues are configured on each interface. +NIC Receive Side Scaling (RSS) for physical interfaces and multi-queue +for virtual interfaces are used for this purpose. diff --git a/docs/content/methodology/overview/per_thread_resources.md b/docs/content/methodology/overview/per_thread_resources.md new file mode 100644 index 0000000000..c23efb50bd --- /dev/null +++ b/docs/content/methodology/overview/per_thread_resources.md @@ -0,0 +1,101 @@ +--- +title: "Per Thread Resources" +weight: 2 +--- + +# Per Thread Resources + +CSIT test framework is managing mapping of the following resources per thread: + +1. Cores, physical cores (pcores) allocated as pairs of sibling logical cores + (lcores) if server in HyperThreading/SMT mode, or as single lcores + if server not in HyperThreading/SMT mode. Note that if server's + processors are running in HyperThreading/SMT mode sibling lcores are + always used. +2. Receive Queues (RxQ), packet receive queues allocated on each + physical and logical interface tested. +3. Transmit Queues(TxQ), packet transmit queues allocated on each + physical and logical interface tested. + +Approach to mapping per thread resources depends on the application/DUT +tested (VPP or DPDK apps) and associated thread types, as follows: + +1. Data-plane workers, used for data-plane packet processing, when no + feature workers present. + + - Cores: data-plane workers are typically tested in 1, 2 and 4 pcore + configurations, running on single lcore per pcore or on sibling + lcores per pcore. Result is a set of {T}t{C}c thread-core + configurations, where{T} stands for a total number of threads + (lcores), and {C} for a total number of pcores. Tested + configurations are encoded in CSIT test case names, + e.g. "1c", "2c", "4c", and test tags "2T1C" (or "1T1C"), "4T2C" + (or "2T2C"), "8T4C" (or "4T4C"). + - Interface Receive Queues (RxQ): as of CSIT-2106 release, number of + RxQs used on each physical or virtual interface is equal to the + number of data-plane workers. In other words each worker has a + dedicated RxQ on each interface tested. This ensures packet + processing load to be equal for each worker, subject to RSS flow + load balancing efficacy. Note: Before CSIT-2106 total number of + RxQs across all interfaces of specific type was equal to the + number of data-plane workers. + - Interface Transmit Queues (TxQ): number of TxQs used on each + physical or virtual interface is equal to the number of data-plane + workers. In other words each worker has a dedicated TxQ on each + interface tested. + - Applies to VPP and DPDK Testpmd and L3Fwd. + +2. Data-plane and feature workers (e.g. IPsec async crypto workers), the + latter dedicated to specific feature processing. + + - Cores: data-plane and feature workers are tested in 2, 3 and 4 + pcore configurations, running on single lcore per pcore or on + sibling lcores per pcore. This results in a two sets of + thread-core combinations separated by "-", {T}t{C}c-{T}t{C}c, with + the leading set denoting total number of threads (lcores) and + pcores used for data-plane workers, and the trailing set denoting + total number of lcores and pcores used for feature workers. + Accordingly, tested configurations are encoded in CSIT test case + names, e.g. "1c-1c", "1c-2c", "1c-3c", and test tags "2T1C_2T1C" + (or "1T1C_1T1C"), "2T1C_4T2C" (or "1T1C_2T2C"), "2T1C_6T3C" + (or "1T1C_3T3C"). + - RxQ and TxQ: no RxQs and no TxQs are used by feature workers. + - Applies to VPP only. + +3. Management/main worker, control plane and management. + + - Cores: single lcore. + - RxQ: not used (VPP default behaviour). + - TxQ: single TxQ per interface, allocated but not used (VPP default + behaviour). + - Applies to VPP only. + +## VPP Thread Configuration + +Mapping of cores and RxQs to VPP data-plane worker threads is done in +the VPP startup.conf during test suite setup: + +1. `corelist-workers <list_of_cores>`: List of logical cores to run VPP + data-plane workers and feature workers. The actual lcores' + allocations depends on HyperThreading/SMT server configuration and + per test core configuration. + + - For tests without feature workers, by default, all CPU cores + configured in startup.conf are used for data-plane workers. + - For tests with feature workers, CSIT code distributes lcores across + data-plane and feature workers. + +2. `num-rx-queues <value>`: Number of Rx queues used per interface. + +Mapping of TxQs to VPP data-plane worker threads uses the default VPP +setting of one TxQ per interface per data-plane worker. + +## DPDK Thread Configuration + +Mapping of cores and RxQs to DPDK Testpmd/L3Fwd data-plane worker +threads is done in the startup CLI: + +1. `-l <list_of_cores>` - List of logical cores to run DPDK + application. +2. `nb-cores=<N>` - Number of forwarding cores. +3. `rxq=<N>` - Number of Rx queues used per interface. diff --git a/docs/content/methodology/overview/terminology.md b/docs/content/methodology/overview/terminology.md new file mode 100644 index 0000000000..c9115e9291 --- /dev/null +++ b/docs/content/methodology/overview/terminology.md @@ -0,0 +1,97 @@ +--- +title: "Terminology" +weight: 1 +--- + +# Terminology + +- **Frame size**: size of an Ethernet Layer-2 frame on the wire, including + any VLAN tags (dot1q, dot1ad) and Ethernet FCS, but excluding Ethernet + preamble and inter-frame gap. Measured in Bytes. + +- **Packet size**: same as frame size, both terms used interchangeably. + +- **Inner L2 size**: for tunneled L2 frames only, size of an encapsulated + Ethernet Layer-2 frame, preceded with tunnel header, and followed by + tunnel trailer. Measured in Bytes. + +- **Inner IP size**: for tunneled IP packets only, size of an encapsulated + IPv4 or IPv6 packet, preceded with tunnel header, and followed by + tunnel trailer. Measured in Bytes. + +- **Device Under Test (DUT)**: In software networking, "device" denotes a + specific piece of software tasked with packet processing. Such device + is surrounded with other software components (such as operating system + kernel). It is not possible to run devices without also running the + other components, and hardware resources are shared between both. For + purposes of testing, the whole set of hardware and software components + is called "System Under Test" (SUT). As SUT is the part of the whole + test setup performance of which can be measured with RFC2544, using + SUT instead of RFC2544 DUT. Device under test + (DUT) can be re-introduced when analyzing test results using whitebox + techniques, but this document sticks to blackbox testing. + +- **System Under Test (SUT)**: System under test (SUT) is a part of the + whole test setup whose performance is to be benchmarked. The complete + methodology contains other parts, whose performance is either already + established, or not affecting the benchmarking result. + +- **Bi-directional throughput tests**: involve packets/frames flowing in + both east-west and west-east directions over every tested interface of + SUT/DUT. Packet flow metrics are measured per direction, and can be + reported as aggregate for both directions (i.e. throughput) and/or + separately for each measured direction (i.e. latency). In most cases + bi-directional tests use the same (symmetric) load in both directions. + +- **Uni-directional throughput tests**: involve packets/frames flowing in + only one direction, i.e. either east-west or west-east direction, over + every tested interface of SUT/DUT. Packet flow metrics are measured + and are reported for measured direction. + +- **Packet Loss Ratio (PLR)**: ratio of packets received relative to packets + transmitted over the test trial duration, calculated using formula: + PLR = ( pkts_transmitted - pkts_received ) / pkts_transmitted. + For bi-directional throughput tests aggregate PLR is calculated based + on the aggregate number of packets transmitted and received. + +- **Packet Throughput Rate**: maximum packet offered load DUT/SUT forwards + within the specified Packet Loss Ratio (PLR). In many cases the rate + depends on the frame size processed by DUT/SUT. Hence packet + throughput rate MUST be quoted with specific frame size as received by + DUT/SUT during the measurement. For bi-directional tests, packet + throughput rate should be reported as aggregate for both directions. + Measured in packets-per-second (pps) or frames-per-second (fps), + equivalent metrics. + +- **Bandwidth Throughput Rate**: a secondary metric calculated from packet + throughput rate using formula: bw_rate = pkt_rate * (frame_size + + L1_overhead) * 8, where L1_overhead for Ethernet includes preamble (8 + Bytes) and inter-frame gap (12 Bytes). For bi-directional tests, + bandwidth throughput rate should be reported as aggregate for both + directions. Expressed in bits-per-second (bps). + +- **Non Drop Rate (NDR)**: maximum packet/bandwith throughput rate sustained + by DUT/SUT at PLR equal zero (zero packet loss) specific to tested + frame size(s). MUST be quoted with specific packet size as received by + DUT/SUT during the measurement. Packet NDR measured in + packets-per-second (or fps), bandwidth NDR expressed in + bits-per-second (bps). + +- **Partial Drop Rate (PDR)**: maximum packet/bandwith throughput rate + sustained by DUT/SUT at PLR greater than zero (non-zero packet loss) + specific to tested frame size(s). MUST be quoted with specific packet + size as received by DUT/SUT during the measurement. Packet PDR + measured in packets-per-second (or fps), bandwidth PDR expressed in + bits-per-second (bps). + +- **Maximum Receive Rate (MRR)**: packet/bandwidth rate regardless of PLR + sustained by DUT/SUT under specified Maximum Transmit Rate (MTR) + packet load offered by traffic generator. MUST be quoted with both + specific packet size and MTR as received by DUT/SUT during the + measurement. Packet MRR measured in packets-per-second (or fps), + bandwidth MRR expressed in bits-per-second (bps). + +- **Trial**: a single measurement step. + +- **Trial duration**: amount of time over which packets are transmitted and + received in a single measurement step. diff --git a/docs/content/methodology/overview/trex_traffic_generator.md b/docs/content/methodology/overview/trex_traffic_generator.md new file mode 100644 index 0000000000..8771bf9780 --- /dev/null +++ b/docs/content/methodology/overview/trex_traffic_generator.md @@ -0,0 +1,195 @@ +--- +title: "TRex Traffic Generator" +weight: 6 +--- + +# TRex Traffic Generator + +## Usage + +[TRex traffic generator](https://trex-tgn.cisco.com) is used for majority of +CSIT performance tests. TRex is used in multiple types of performance tests, +see [Data Plane Throughtput]({{< ref "../measurements/data_plane_throughput/data_plane_throughput/#Data Plane Throughtput" >}}) +for more details. + +## Traffic modes + +TRex is primarily used in two (mutually incompatible) modes. + +### Stateless mode + +Sometimes abbreviated as STL. +A mode with high performance, which is unable to react to incoming traffic. +We use this mode whenever it is possible. +Typical test where this mode is not applicable is NAT44ED, +as DUT does not assign deterministic outside address+port combinations, +so we are unable to create traffic that does not lose packets +in out2in direction. + +Measurement results are based on simple L2 counters +(opackets, ipackets) for each traffic direction. + +### Stateful mode + +A mode capable of reacting to incoming traffic. +Contrary to the stateless mode, only UDP and TCP is supported +(carried over IPv4 or IPv6 packets). +Performance is limited, as TRex needs to do more CPU processing. +TRex suports two subtypes of stateful traffic, +CSIT uses ASTF (Advanced STateFul mode). + +This mode is suitable for NAT44ED tests, as clients send packets from inside, +and servers react to it, so they see the outside address and port to respond to. +Also, they do not send traffic before NAT44ED has created the corresponding +translation entry. + +When possible, L2 counters (opackets, ipackets) are used. +Some tests need L7 counters, which track protocol state (e.g. TCP), +but those values are less than reliable on high loads. + +## Traffic Continuity + +Generated traffic is either continuous, or limited (by number of transactions). +Both modes support both continuities in principle. + +### Continuous traffic + +Traffic is started without any data size goal. +Traffic is ended based on time duration, as hinted by search algorithm. +This is useful when DUT behavior does not depend on the traffic duration. +The default for stateless mode. + +### Limited traffic + +Traffic has defined data size goal (given as number of transactions), +duration is computed based on this goal. +Traffic is ended when the size goal is reached, +or when the computed duration is reached. +This is useful when DUT behavior depends on traffic size, +e.g. target number of NAT translation entries, each to be hit exactly once +per direction. +This is used mainly for stateful mode. + +## Traffic synchronicity + +Traffic can be generated synchronously (test waits for duration) +or asynchronously (test operates during traffic and stops traffic explicitly). + +### Synchronous traffic + +Trial measurement is driven by given (or precomputed) duration, +no activity from test driver during the traffic. +Used for most trials. + +### Asynchronous traffic + +Traffic is started, but then the test driver is free to perform +other actions, before stopping the traffic explicitly. +This is used mainly by reconf tests, but also by some trials +used for runtime telemetry. + +## Trafic profiles + +TRex supports several ways to define the traffic. +CSIT uses small Python modules based on Scapy as definitions. +Details of traffic profiles depend on modes (STL or ASTF), +but some are common for both modes. + +Search algorithms are intentionally unaware of the traffic mode used, +so CSIT defines some terms to use instead of mode-specific TRex terms. + +### Transactions + +TRex traffic profile defines a small number of behaviors, +in CSIT called transaction templates. Traffic profiles also instruct +TRex how to create a large number of transactions based on the templates. + +Continuous traffic loops over the generated transactions. +Limited traffic usually executes each transaction once +(typically as constant number of loops over source addresses, +each loop with different source ports). + +Currently, ASTF profiles define one transaction template each. +Number of packets expected per one transaction varies based on profile details, +as does the criterion for when a transaction is considered successful. + +Stateless transactions are just one packet (sent from one TG port, +successful if received on the other TG port). +Thus unidirectional stateless profiles define one transaction template, +bidirectional stateless profiles define two transaction templates. + +### TPS multiplier + +TRex aims to open transaction specified by the profile at a steady rate. +While TRex allows the transaction template to define its intended "cps" value, +CSIT does not specify it, so the default value of 1 is applied, +meaning TRex will open one transaction per second (and transaction template) +by default. But CSIT invocation uses "multiplier" (mult) argument +when starting the traffic, that multiplies the cps value, +meaning it acts as TPS (transactions per second) input. + +With a slight abuse of nomenclature, bidirectional stateless tests +set "packets per transaction" value to 2, just to keep the TPS semantics +as a unidirectional input value. + +### Duration stretching + +TRex can be IO-bound, CPU-bound, or have any other reason +why it is not able to generate the traffic at the requested TPS. +Some conditions are detected, leading to TRex failure, +for example when the bandwidth does not fit into the line capacity. +But many reasons are not detected. + +Unfortunately, TRex frequently reacts by not honoring the duration +in synchronous mode, taking longer to send the traffic, +leading to lower then requested load offered to DUT. +This usualy breaks assumptions used in search algorithms, +so it has to be avoided. + +For stateless traffic, the behavior is quite deterministic, +so the workaround is to apply a fictional TPS limit (max_rate) +to search algorithms, usually depending only on the NIC used. + +For stateful traffic the behavior is not deterministic enough, +for example the limit for TCP traffic depends on DUT packet loss. +In CSIT we decided to use logic similar to asynchronous traffic. +The traffic driver sleeps for a time, then stops the traffic explicitly. +The library that parses counters into measurement results +than usually treats unsent packets/transactions as lost/failed. + +We have added a IP4base tests for every NAT44ED test, +so that users can compare results. +If the results are very similar, it is probable TRex was the bottleneck. + +### Startup delay + +By investigating TRex behavior, it was found that TRex does not start +the traffic in ASTF mode immediately. There is a delay of zero traffic, +after which the traffic rate ramps up to the defined TPS value. + +It is possible to poll for counters during the traffic +(fist nonzero means traffic has started), +but that was found to influence the NDR results. + +Thus "sleep and stop" stategy is used, which needs a correction +to the computed duration so traffic is stopped after the intended +duration of real traffic. Luckily, it turns out this correction +is not dependend on traffic profile nor CPU used by TRex, +so a fixed constant (0.112 seconds) works well. +Unfortunately, the constant may depend on TRex version, +or execution environment (e.g. TRex in AWS). + +The result computations need a precise enough duration of the real traffic, +luckily server side of TRex has precise enough counter for that. + +It is unknown whether stateless traffic profiles also exhibit a startup delay. +Unfortunately, stateless mode does not have similarly precise duration counter, +so some results (mostly MRR) are affected by less precise duration measurement +in Python part of CSIT code. + +## Measuring Latency + +If measurement of latency is requested, two more packet streams are +created (one for each direction) with TRex flow_stats parameter set to +STLFlowLatencyStats. In that case, returned statistics will also include +min/avg/max latency values and encoded HDRHistogram data. diff --git a/docs/content/methodology/overview/vpp_forwarding_modes.md b/docs/content/methodology/overview/vpp_forwarding_modes.md new file mode 100644 index 0000000000..b3c3bba984 --- /dev/null +++ b/docs/content/methodology/overview/vpp_forwarding_modes.md @@ -0,0 +1,104 @@ +--- +title: "VPP Forwarding Modes" +weight: 4 +--- + +# VPP Forwarding Modes + +VPP is tested in a number of L2, IPv4 and IPv6 packet lookup and forwarding +modes. Within each mode baseline and scale tests are executed, the latter with +varying number of FIB entries. + +## L2 Ethernet Switching + +VPP is tested in three L2 forwarding modes: + +- *l2patch*: L2 patch, the fastest point-to-point L2 path that loops + packets between two interfaces without any Ethernet frame checks or + lookups. +- *l2xc*: L2 cross-connect, point-to-point L2 path with all Ethernet + frame checks, but no MAC learning and no MAC lookup. +- *l2bd*: L2 bridge-domain, multipoint-to-multipoint L2 path with all + Ethernet frame checks, with MAC learning (unless static MACs are used) + and MAC lookup. + +l2bd tests are executed in baseline and scale configurations: + +- *l2bdbase*: Two MAC FIB entries are learned by VPP to enable packet + switching between two interfaces in two directions. VPP L2 switching + is tested with 254 IPv4 unique flows per direction, varying IPv4 + source address per flow in order to invoke RSS based packet + distribution across VPP workers. The same source and destination MAC + address is used for all flows per direction. IPv4 source address is + incremented for every packet. + +- *l2bdscale*: A high number of MAC FIB entries are learned by VPP to + enable packet switching between two interfaces in two directions. + Tested MAC FIB sizes include: i) 10k with 5k unique flows per + direction, ii) 100k with 2 x 50k flows and iii) 1M with 2 x 500k + flows. Unique flows are created by using distinct source and + destination MAC addresses that are changed for every packet using + incremental ordering, making VPP learn (or refresh) distinct src MAC + entries and look up distinct dst MAC entries for every packet. For + details, see + [Packet Flow Ordering]({{< ref "packet_flow_ordering#Packet Flow Ordering" >}}). + +Ethernet wire encapsulations tested include: untagged, dot1q, dot1ad. + +## IPv4 Routing + +IPv4 routing tests are executed in baseline and scale configurations: + +- *ip4base*: Two /32 IPv4 FIB entries are configured in VPP to enable + packet routing between two interfaces in two directions. VPP routing + is tested with 253 IPv4 unique flows per direction, varying IPv4 + source address per flow in order to invoke RSS based packet + distribution across VPP workers. IPv4 source address is incremented + for every packet. + +- *ip4scale*: A high number of /32 IPv4 FIB entries are configured in + VPP. Tested IPv4 FIB sizes include: i) 20k with 10k unique flows per + direction, ii) 200k with 2 * 100k flows and iii) 2M with 2 * 1M + flows. Unique flows are created by using distinct IPv4 destination + addresses that are changed for every packet, using incremental or + random ordering. For details, see + [Packet Flow Ordering]({{< ref "packet_flow_ordering#Packet Flow Ordering" >}}). + +## IPv6 Routing + +Similarly to IPv4, IPv6 routing tests are executed in baseline and scale +configurations: + +- *ip6base*: Two /128 IPv4 FIB entries are configured in VPP to enable + packet routing between two interfaces in two directions. VPP routing + is tested with 253 IPv6 unique flows per direction, varying IPv6 + source address per flow in order to invoke RSS based packet + distribution across VPP workers. IPv6 source address is incremented + for every packet. + +- *ip4scale*: A high number of /128 IPv6 FIB entries are configured in + VPP. Tested IPv6 FIB sizes include: i) 20k with 10k unique flows per + direction, ii) 200k with 2 * 100k flows and iii) 2M with 2 * 1M + flows. Unique flows are created by using distinct IPv6 destination + addresses that are changed for every packet, using incremental or + random ordering. For details, see + [Packet Flow Ordering]({{< ref "packet_flow_ordering#Packet Flow Ordering" >}}). + +## SRv6 Routing + +SRv6 routing tests are executed in a number of baseline configurations, +in each case SR policy and steering policy are configured for one +direction and one (or two) SR behaviours (functions) in the other +directions: + +- *srv6enc1sid*: One SID (no SRH present), one SR function - End. +- *srv6enc2sids*: Two SIDs (SRH present), two SR functions - End and + End.DX6. +- *srv6enc2sids-nodecaps*: Two SIDs (SRH present) without decapsulation, + one SR function - End. +- *srv6proxy-dyn*: Dynamic SRv6 proxy, one SR function - End.AD. +- *srv6proxy-masq*: Masquerading SRv6 proxy, one SR function - End.AM. +- *srv6proxy-stat*: Static SRv6 proxy, one SR function - End.AS. + +In all listed cases low number of IPv6 flows (253 per direction) is +routed by VPP. diff --git a/docs/content/methodology/per_patch_testing.md b/docs/content/methodology/per_patch_testing.md new file mode 100644 index 0000000000..6ae40a13dc --- /dev/null +++ b/docs/content/methodology/per_patch_testing.md @@ -0,0 +1,229 @@ +--- +title: "Per-patch Testing" +weight: 5 +--- + +# Per-patch Testing + +Updated for CSIT git commit id: d8ec3f8673346c0dc93e567159771f24c1bf74fc. + +A methodology similar to trending analysis is used for comparing performance +before a DUT code change is merged. This can act as a verify job to disallow +changes which would decrease performance without a good reason. + +## Existing jobs + +They are not started automatically, must be triggered on demand. +They allow full tag expressions, all types of perf tests are supported. + +There are jobs available for multiple types of testbeds, +based on various processors. +Their Gerrit triggers words are of the form "perftest-{node_arch}" +where the node_arch combinations currently supported are: +2n-icx, 2n-clx, 2n-spr, 2n-zn2, 3n-icx, 3n-tsh, 3n-alt, 2n-tx2, 3n-snr, +3na-spr, 3nb-spr. + +## Test selection + +Gerrit trigger line without any additional arguments selects +a small set of test cases to run. +If additional arguments are added to the Gerrit trigger, they are treated +as Robot tag expressions to select tests to run. +While very flexible, this method of test selection also allows the user +to accidentally select too high number of tests, blocking the testbed for days. + +What follows is a list of explanations and recommendations +to help users to select the minimal set of tests cases. + +### Verify cycles + +When Gerrit schedules multiple jobs to run for the same patch set, +it waits until all runs are complete. +While it is waiting, it is possible to trigger more jobs +(adding runs to the set Gerrit is waiting for), but it is not possible +to trigger more runs for the same job, until Gerrit is done waiting. +After Gerrit is done waiting, it becames possible to trigger +the same job again. + +Example. User triggers one set of tests on 2n-icx and immediately +also triggers other set of tests on 3n-icx. Then the user notices +2n-icx run ended early because of a typo in tag expression. +When the user tries to re-trigger 2n-icx (with a fixed tag expression), +that comment is ignored by Jenkins. +Only when 3n-icx job finishes, the user can trigger 2n-icx again. + +### One comment many jobs + +In the past, the CSIT code which parses for perftest trigger comments +was buggy, which lead to bad behavior (as in selection all performance test, +because "perftest" is also a robot tag) when user included multiple +perftest trigger words in the same comment. + +The worst bugs were fixed since then, but it is still recommended +to use just one trigger word per Gerrit comment, just to be safe. + +### Multiple test cases in run + +While Robot supports OR operator, it does not support parentheses, +so the OR operator is not very useful. +It is recommended to use space instead of OR operator. + +Example template: +perftest-2n-icx {tag_expression_1} {tag_expression_2} + +See below for more concrete examples. + +### Suite tags + +Traditionally, CSIT maintains broad Robot tags that can be used to select tests. + +But it is not recommended to use them for test selection, +as it is not that easy to determine how many test cases are selected. + +The recommended way is to look into CSIT repository first, +and locate a specific suite the user is interested in, +and use its suite tag. For example, "ethip4-ip4base" is a suite tag +selecting just one suite in CSIT git repository, +avoiding all scale, container, and other simialr variants. + +Note that CSIT uses "autogen" code generator, +so the robot running in Jenkins has access to more suites +than visible just by looking into CSIT git repository. +Thus, suite tag is not enough to select precisely the intended suite, +and user is encouraged to narrow down to a single test case within a suite. + +### Fully specified tag expressions + +Here is one template to select a single test case: +{test_type}AND{nic_model}AND{nic_driver}AND{cores}AND{frame_size}AND{suite_tag} +where the variables are all lower case (so AND operator stands out). + +The fastest and the most widely used type of performance test is "mrr". +As an alternative, "ndrpdr" focuses on small losses (ax opposed to max load), +but takes longer to finish. +The nic_driver options depend on nic_model. For Intel cards "drv_avf" +(AVF plugin) and "drv_vfio_pci" (DPDK plugin) are popular, for Mellanox +"drv_mlx5_core". Currently, the performance using "drv_af_xdp" is not reliable +enough, so do not use it unless you are specifically testing for AF_XDP. + +The most popular nic_model is "nic_intel-e810cq", but that is not available +on all testbed types. +It is safe to use "1c" for cores (unless you are suspecting multi-core +performance is affected differently) and "64b" for frame size ("78b" for ip6 +and more for dot1q and other encapsulated traffic; +"1518b" is popular for ipsec and other CPU-bound tests). + +As there are more test cases than CSIT can periodically test, +it is possible to encounter an old test case that currently fails. +To avoid that, you can look at "job spec" files we use for periodic testing, +for example +[this one](https://raw.githubusercontent.com/FDio/csit/master/resources/job_specs/report_iterative/2n-spr/vpp-mrr-00.md). + +### Shortening triggers + +Advanced users may use the following tricks to avoid writing long trigger +comments. + +Robot supports glob matching, which can be used to select multiple suite tags at +once. + +Not specifying one of 6 parts of the recommended expression pattern +will select all available options. For example not specifying nic_driver +for nic_intel-e810cq will select all 3 applicable drivers. +You can use NOT operator to reject some options (e.g. NOTdrv_af_xdp). +Beware, with NOT the order matters: +tag1ANDtag2NOTtag3 is not the same as tag1NOTtag3ANDtag2, +the latter is evaluated as tag1AND(NOT(tag3ANDtag2)). + +Beware when not specifying nic_model. As a precaution, +CSIT code will insert the defailt NIC model for the tetsbed used. +Example: Specifying drv_rdma_core without specifying nic_model +will fail, as the default nic_model is nic_intel-e810cq +which does not support RDMA core driver. + +### Complete example + +A user wants to test a VPP change which may affect load balance whith bonding. +Searching tag documentation for "bonding" finds LBOND tag and its variants. +Searching CSIT git repository (directory tests/) finds 8 suite files, +all suited only for 3-node testbeds. +All suites are using vhost, but differ by the forwarding app inside VM +(DPDK or VPP), by the forwarding mode of VPP acting as host level vswitch +(MAC learning or cross connect), and by the number of DUT1-DUT2 links +available (1 or 2). + +As not all NICs and testbeds offer enogh ports for 2 parallel DUT-DUT links, +the user looks at +[testbed specifications](https://github.com/FDio/csit/tree/master/topologies/available) +and finds that only e810xxv NIC on 3n-icx testbed matches the requirements. +Quick look into the suites confirm the smallest frame size is 64 bytes +(despite DOT1Q robot tag, as the encapsulation does not happen on TG-DUT links). +It is ok to use just 1 physical core, as 3n-icx has hyperthreading enabled, +so VPP vswitch will use 2 worker threads. + +The user decides the vswitch forwarding mode is not important +(so choses cross connect as that has less CPU overhead), +but wants to test both NIC drivers (not AF_XDP), both apps in VM, +and both 1 and 2 parallel links. + +After shortening, this is the trigger comment fianlly used: +perftest-3n-icx mrrANDnic_intel-e810cqAND1cAND64bAND?lbvpplacp-dot1q-l2xcbase-eth-2vhostvr1024-1vm\*NOTdrv_af_xdp + +## Basic operation + +The job builds VPP .deb packages for both the patch under test +(called "current") and its parent patch (called "parent"). + +For each test (from the set defined by tag expressions), +both builds are subjected to several trial measurements (in case of MRR). +Measured samples are grouped to "parent" sequence, +followed by "current" sequence. The same Minimal Description Length +algorithm as in trending is used to decide whether it is one big group, +or two smaller gropus. If it is one group, a "normal" result +is declared for the test. If it is two groups, and current average +is less then parent average, the test is declared a regression. +If it is two groups and current average is larger or equal, +the test is declared a progression. + +The whole job fails (giving -1) if any test was declared a regression. +If a test fails, a fake result values are used, +so it is possible to use the job fo verify current fixes a test failing in parent +(if a test is not fixed, it is treated as a regression). + +## Temporary specifics + +The Minimal Description Length analysis is performed by +CSIT code equivalent to jumpavg-0.4.1 library available on PyPI. + +In hopes of strengthening of signal (code performance) compared to noise +(all other factors influencing the measured values), several workarounds +are applied. + +In contrast to trending, MRR trial duration is set to 10 seconds, +and only 5 samples are measured for each build. +Both parameters are set in ci-management. + +This decreases sensitivity to regressions, but also decreases +probability of false positives. + +## Console output + +The following information as visible towards the end of Jenkins console output, +repeated for each analyzed test. + +The original 5 values (or 1 for non-mrr) are visible in order they were measured. +The values after processing are also visible in output, +this time sorted by value (so people can see minimum and maximum). + +The next output is difference of averages. It is the current average +minus the parent average, expressed as percentage of the parent average. + +The next three outputs contain the jumpavg representation +of the two groups and a combined group. +Here, "bits" is the description length; for "current" sequence +it includes effect from "parent" average value +(jumpavg-0.4.1 penalizes sequences with too close averages). + +Next, a sentence describing which grouping description is shorter, +and by how much bits. +Finally, the test result classification is visible. diff --git a/docs/content/methodology/test/_index.md b/docs/content/methodology/test/_index.md new file mode 100644 index 0000000000..e9864ac28d --- /dev/null +++ b/docs/content/methodology/test/_index.md @@ -0,0 +1,19 @@ +--- +bookCollapseSection: true +bookFlatSection: false +title: "Tests" +weight: 3 +--- + +# Tests + +- [Network Address Translation]({{< relref "/methodology/test/network_address_translation" >}}) +- [Packet Flow Ordering]({{< relref "/methodology/test/packet_flow_ordering" >}}) +- [Tunnel Encapsulations]({{< relref "/methodology/test/tunnel_encapsulations" >}}) +- [Internet Protocol Security]({{< relref "/methodology/test/internet_protocol_security" >}}) +- [Access Control Lists]({{< relref "/methodology/test/access_control_lists" >}}) +- [Hoststack]({{< relref "/methodology/test/hoststack/" >}}) +- [Generic Segmentation Offload]({{< relref "/methodology/test/generic_segmentation_offload" >}}) +- [Reconfiguration]({{< relref "/methodology/test/reconfiguration" >}}) +- [VPP Device]({{< relref "/methodology/test/vpp_device" >}}) +- [Reassembly]({{< relref "/methodology/test/reassembly" >}}) diff --git a/docs/content/methodology/test/access_control_lists.md b/docs/content/methodology/test/access_control_lists.md new file mode 100644 index 0000000000..354e6b72bb --- /dev/null +++ b/docs/content/methodology/test/access_control_lists.md @@ -0,0 +1,66 @@ +--- +title: "Access Control Lists" +weight: 5 +--- + +# Access Control Lists + +VPP is tested in a number of data plane feature configurations across +different forwarding modes. Following sections list features tested. + +## ACL Security-Groups + +Both stateless and stateful access control lists (ACL), also known as +security-groups, are supported by VPP. + +Following ACL configurations are tested for MAC switching with L2 +bridge-domains: + +- *l2bdbasemaclrn-iacl{E}sl-{F}flows*: Input stateless ACL, with {E} + entries and {F} flows. +- *l2bdbasemaclrn-oacl{E}sl-{F}flows*: Output stateless ACL, with {E} + entries and {F} flows. +- *l2bdbasemaclrn-iacl{E}sf-{F}flows*: Input stateful ACL, with {E} + entries and {F} flows. +- *l2bdbasemaclrn-oacl{E}sf-{F}flows*: Output stateful ACL, with {E} + entries and {F} flows. + +Following ACL configurations are tested with IPv4 routing: + +- *ip4base-iacl{E}sl-{F}flows*: Input stateless ACL, with {E} entries + and {F} flows. +- *ip4base-oacl{E}sl-{F}flows*: Output stateless ACL, with {E} entries + and {F} flows. +- *ip4base-iacl{E}sf-{F}flows*: Input stateful ACL, with {E} entries and + {F} flows. +- *ip4base-oacl{E}sf-{F}flows*: Output stateful ACL, with {E} entries + and {F} flows. + +ACL tests are executed with the following combinations of ACL entries +and number of flows: + +- ACL entry definitions + - flow non-matching deny entry: (src-ip4, dst-ip4, src-port, dst-port). + - flow matching permit ACL entry: (src-ip4, dst-ip4). +- {E} - number of non-matching deny ACL entries, {E} = [1, 10, 50]. +- {F} - number of UDP flows with different tuple (src-ip4, dst-ip4, + src-port, dst-port), {F} = [100, 10k, 100k]. +- All {E}x{F} combinations are tested per ACL type, total of 9. + +## ACL MAC-IP + +MAC-IP binding ACLs are tested for MAC switching with L2 bridge-domains: + +- *l2bdbasemaclrn-macip-iacl{E}sl-{F}flows*: Input stateless ACL, with + {E} entries and {F} flows. + +MAC-IP ACL tests are executed with the following combinations of ACL +entries and number of flows: + +- ACL entry definitions + - flow non-matching deny entry: (dst-ip4, dst-mac, bit-mask) + - flow matching permit ACL entry: (dst-ip4, dst-mac, bit-mask) +- {E} - number of non-matching deny ACL entries, {E} = [1, 10, 50] +- {F} - number of UDP flows with different tuple (dst-ip4, dst-mac), + {F} = [100, 10k, 100k] +- All {E}x{F} combinations are tested per ACL type, total of 9. diff --git a/docs/content/methodology/test/generic_segmentation_offload.md b/docs/content/methodology/test/generic_segmentation_offload.md new file mode 100644 index 0000000000..0032d203de --- /dev/null +++ b/docs/content/methodology/test/generic_segmentation_offload.md @@ -0,0 +1,117 @@ +--- +title: "Generic Segmentation Offload" +weight: 7 +--- + +# Generic Segmentation Offload + +## Overview + +Generic Segmentation Offload (GSO) reduces per-packet processing +overhead by enabling applications to pass a multi-packet buffer to +(v)NIC and process a smaller number of large packets (e.g. frame size of +64 KB), instead of processing higher numbers of small packets (e.g. +frame size of 1500 B), thus reducing per-packet overhead. + +GSO tests for VPP vhostuser and tapv2 interfaces. All tests cases use iPerf3 +client and server applications running TCP/IP as a traffic generator. For +performance comparison the same tests are run without GSO enabled. + +## GSO Test Topologies + +Two VPP GSO test topologies are implemented: + +1. iPerfC_GSOvirtio_LinuxVM --- GSOvhost_VPP_GSOvhost --- iPerfS_GSOvirtio_LinuxVM + - Tests VPP GSO on vhostuser interfaces and interaction with Linux + virtio with GSO enabled. +2. iPerfC_GSOtap_LinuxNspace --- GSOtapv2_VPP_GSOtapv2 --- iPerfS_GSOtap_LinuxNspace + - Tests VPP GSO on tapv2 interfaces and interaction with Linux tap + with GSO enabled. + +Common configuration: + +- iPerfC (client) and iPerfS (server) run in TCP/IP mode without upper + bandwidth limit. +- Trial duration is set to 30 sec. +- iPerfC, iPerfS and VPP run in the single SUT node. + + +## VPP GSOtap Topology + +### VPP Configuration + +VPP GSOtap tests are executed without using hyperthreading. VPP worker runs on +a single core. Multi-core tests are not executed. Each interface belongs to +separate namespace. Following core pinning scheme is used: + +- 1t1c (rxq=1, rx_qsz=4096, tx_qsz=4096) + - system isolated: 0,28,56,84 + - vpp mt: 1 + - vpp wt: 2 + - vhost: 3-5 + - iperf-s: 6 + - iperf-c: 7 + +### iPerf3 Server Configuration + +iPerf3 version used 3.7 + + $ sudo -E -S ip netns exec tap1_namespace iperf3 \ + --server --daemon --pidfile /tmp/iperf3_server.pid \ + --logfile /tmp/iperf3.log --port 5201 --affinity <X> + +For the full iPerf3 reference please see +[iPerf3 docs](https://github.com/esnet/iperf/blob/master/docs/invoking.rst). + + +### iPerf3 Client Configuration + +iPerf3 version used 3.7 + + $ sudo -E -S ip netns exec tap1_namespace iperf3 \ + --client 2.2.2.2 --bind 1.1.1.1 --port 5201 --parallel <Y> \ + --time 30.0 --affinity <X> --zerocopy + +For the full iPerf3 reference please see +[iPerf3 docs](https://github.com/esnet/iperf/blob/master/docs/invoking.rst). + + +## VPP GSOvhost Topology + +### VPP Configuration + +VPP GSOvhost tests are executed without using hyperthreading. VPP worker runs +on a single core. Multi-core tests are not executed. Following core pinning +scheme is used: + +- 1t1c (rxq=1, rx_qsz=1024, tx_qsz=1024) + - system isolated: 0,28,56,84 + - vpp mt: 1 + - vpp wt: 2 + - vm-iperf-s: 3,4,5,6,7 + - vm-iperf-c: 8,9,10,11,12 + - iperf-s: 1 + - iperf-c: 1 + +### iPerf3 Server Configuration + +iPerf3 version used 3.7 + + $ sudo iperf3 \ + --server --daemon --pidfile /tmp/iperf3_server.pid \ + --logfile /tmp/iperf3.log --port 5201 --affinity X + +For the full iPerf3 reference please see +[iPerf3 docs](https://github.com/esnet/iperf/blob/master/docs/invoking.rst). + + +### iPerf3 Client Configuration + +iPerf3 version used 3.7 + + $ sudo iperf3 \ + --client 2.2.2.2 --bind 1.1.1.1 --port 5201 --parallel <Y> \ + --time 30.0 --affinity X --zerocopy + +For the full iPerf3 reference please see +[iPerf3 docs](https://github.com/esnet/iperf/blob/master/docs/invoking.rst). diff --git a/docs/content/methodology/test/hoststack/_index.md b/docs/content/methodology/test/hoststack/_index.md new file mode 100644 index 0000000000..6c0cb292fb --- /dev/null +++ b/docs/content/methodology/test/hoststack/_index.md @@ -0,0 +1,13 @@ +--- +bookCollapseSection: true +bookFlatSection: false +title: "Hoststack" +weight: 6 +--- + +# Hoststack + +- [QUIC/UDP/IP with vpp_echo]({{< relref "/methodology/test/hoststack/quicudpip_with_vppecho" >}}) +- [TCP/IP with iperf3]({{< relref "/methodology/test/hoststack/tcpip_with_iperf3" >}}) +- [UDP/IP with iperf3]({{< relref "/methodology/test/hoststack/udpip_with_iperf3" >}}) +- [VSAP ab with nginx]({{< relref "/methodology/test/hoststack/vsap_ab_with_nginx" >}}) diff --git a/docs/content/methodology/test/hoststack/quicudpip_with_vppecho.md b/docs/content/methodology/test/hoststack/quicudpip_with_vppecho.md new file mode 100644 index 0000000000..c7d57a51b3 --- /dev/null +++ b/docs/content/methodology/test/hoststack/quicudpip_with_vppecho.md @@ -0,0 +1,48 @@ +--- +title: "QUIC/UDP/IP with vpp_echo" +weight: 1 +--- + +# QUIC/UDP/IP with vpp_echo + +[vpp_echo performance testing tool](https://wiki.fd.io/view/VPP/HostStack#External_Echo_Server.2FClient_.28vpp_echo.29) +is a bespoke performance test application which utilizes the 'native +HostStack APIs' to verify performance and correct handling of +connection/stream events with uni-directional and bi-directional +streams of data. + +Because iperf3 does not support the QUIC transport protocol, vpp_echo +is used for measuring the maximum attainable goodput of the VPP Host +Stack connection utilizing the QUIC transport protocol across two +instances of VPP running on separate DUT nodes. The QUIC transport +protocol supports multiple streams per connection and test cases +utilize different combinations of QUIC connections and number of +streams per connection. + +The test configuration is as follows: + + DUT1 Network DUT2 + [ vpp_echo-client -> VPP1 ]=======[ VPP2 -> vpp_echo-server] + N-streams/connection + +where, + +1. vpp_echo server attaches to VPP2 and LISTENs on VPP2:TCP port 1234. +2. vpp_echo client creates one or more connections to VPP1 and opens + one or more stream per connection to VPP2:TCP port 1234. +3. vpp_echo client transmits a uni-directional stream as fast as the + VPP Host Stack allows to the vpp_echo server for the test duration. +4. At the end of the test the vpp_echo client emits the goodput + measurements for all streams and the sum of all streams. + +Test cases include + +1. 1 QUIC Connection with 1 Stream +2. 1 QUIC connection with 10 Streams +3. 10 QUIC connetions with 1 Stream +4. 10 QUIC connections with 10 Streams + +with stream sizes to provide reasonable test durations. The VPP Host +Stack QUIC transport is configured to utilize the picotls encryption +library. In the future, tests utilizing addtional encryption +algorithms will be added. diff --git a/docs/content/methodology/test/hoststack/tcpip_with_iperf3.md b/docs/content/methodology/test/hoststack/tcpip_with_iperf3.md new file mode 100644 index 0000000000..7baa88ab50 --- /dev/null +++ b/docs/content/methodology/test/hoststack/tcpip_with_iperf3.md @@ -0,0 +1,52 @@ +--- +title: "TCP/IP with iperf3" +weight: 2 +--- + +# TCP/IP with iperf3 + +[iperf3 goodput measurement tool](https://github.com/esnet/iperf) +is used for measuring the maximum attainable goodput of the VPP Host +Stack connection across two instances of VPP running on separate DUT +nodes. iperf3 is a popular open source tool for active measurements +of the maximum achievable goodput on IP networks. + +Because iperf3 utilizes the POSIX socket interface APIs, the current +test configuration utilizes the LD_PRELOAD mechanism in the linux +kernel to connect iperf3 to the VPP Host Stack using the VPP +Communications Library (VCL) LD_PRELOAD library (libvcl_ldpreload.so). + +In the future, a forked version of iperf3 which has been modified to +directly use the VCL application APIs may be added to determine the +difference in performance of 'VCL Native' applications versus utilizing +LD_PRELOAD which inherently has more overhead and other limitations. + +The test configuration is as follows: + + DUT1 Network DUT2 + [ iperf3-client -> VPP1 ]=======[ VPP2 -> iperf3-server] + +where, + +1. iperf3 server attaches to VPP2 and LISTENs on VPP2:TCP port 5201. +2. iperf3 client attaches to VPP1 and opens one or more stream + connections to VPP2:TCP port 5201. +3. iperf3 client transmits a uni-directional stream as fast as the + VPP Host Stack allows to the iperf3 server for the test duration. +4. At the end of the test the iperf3 client emits the goodput + measurements for all streams and the sum of all streams. + +Test cases include 1 and 10 Streams with a 20 second test duration +with the VPP Host Stack configured to utilize the Cubic TCP +congestion algorithm. + +Note: iperf3 is single threaded, so it is expected that the 10 stream +test shows little or no performance improvement due to +multi-thread/multi-core execution. + +There are also variations of these test cases which use the VPP Network +Simulator (NSIM) plugin to test the VPP Hoststack goodput with 1 percent +of the traffic being dropped at the output interface of VPP1 thereby +simulating a lossy network. The NSIM tests are experimental and the +test results are not currently representative of typical results in a +lossy network. diff --git a/docs/content/methodology/test/hoststack/udpip_with_iperf3.md b/docs/content/methodology/test/hoststack/udpip_with_iperf3.md new file mode 100644 index 0000000000..01ddf61269 --- /dev/null +++ b/docs/content/methodology/test/hoststack/udpip_with_iperf3.md @@ -0,0 +1,44 @@ +--- +title: "UDP/IP with iperf3" +weight: 3 +--- + +# UDP/IP with iperf3 + +[iperf3 goodput measurement tool](https://github.com/esnet/iperf) +is used for measuring the maximum attainable goodput of the VPP Host +Stack connection across two instances of VPP running on separate DUT +nodes. iperf3 is a popular open source tool for active measurements +of the maximum achievable goodput on IP networks. + +Because iperf3 utilizes the POSIX socket interface APIs, the current +test configuration utilizes the LD_PRELOAD mechanism in the linux +kernel to connect iperf3 to the VPP Host Stack using the VPP +Communications Library (VCL) LD_PRELOAD library (libvcl_ldpreload.so). + +In the future, a forked version of iperf3 which has been modified to +directly use the VCL application APIs may be added to determine the +difference in performance of 'VCL Native' applications versus utilizing +LD_PRELOAD which inherently has more overhead and other limitations. + +The test configuration is as follows: + + DUT1 Network DUT2 + [ iperf3-client -> VPP1 ]=======[ VPP2 -> iperf3-server] + +where, + +1. iperf3 server attaches to VPP2 and LISTENs on VPP2:UDP port 5201. +2. iperf3 client attaches to VPP1 and transmits one or more streams + of packets to VPP2:UDP port 5201. +3. iperf3 client transmits a uni-directional stream as fast as the + VPP Host Stack allows to the iperf3 server for the test duration. +4. At the end of the test the iperf3 client emits the goodput + measurements for all streams and the sum of all streams. + +Test cases include 1 and 10 Streams with a 20 second test duration +with the VPP Host Stack using the UDP transport layer.. + +Note: iperf3 is single threaded, so it is expected that the 10 stream +test shows little or no performance improvement due to +multi-thread/multi-core execution. diff --git a/docs/content/methodology/test/hoststack/vsap_ab_with_nginx.md b/docs/content/methodology/test/hoststack/vsap_ab_with_nginx.md new file mode 100644 index 0000000000..2dc4d2b7f9 --- /dev/null +++ b/docs/content/methodology/test/hoststack/vsap_ab_with_nginx.md @@ -0,0 +1,39 @@ +--- +title: "VSAP ab with nginx" +weight: 4 +--- + +# VSAP ab with nginx + +[VSAP (VPP Stack Acceleration Project)](https://wiki.fd.io/view/VSAP) +aims to establish an industry user space application ecosystem based on +the VPP hoststack. As a pre-requisite to adapting open source applications +using VPP Communications Library to accelerate performance, the VSAP team +has introduced baseline tests utilizing the LD_PRELOAD mechanism to capture +baseline performance data. + +[AB (Apache HTTP server benchmarking tool)](https://httpd.apache.org/docs/2.4/programs/ab.html) +is used for measuring the maximum connections-per-second and requests-per-second. + +[NGINX](https://www.nginx.com) is a popular open source HTTP server +application. Because NGINX utilizes the POSIX socket interface APIs, the test +configuration uses the LD_PRELOAD mechanism to connect NGINX to the VPP +Hoststack using the VPP Communications Library (VCL) LD_PRELOAD library +(libvcl_ldpreload.so). + +In the future, a version of NGINX which has been modified to +directly use the VCL application APIs will be added to determine the +difference in performance of 'VCL Native' applications versus utilizing +LD_PRELOAD which inherently has more overhead and other limitations. + +The test configuration is as follows: + + TG Network DUT + [ AB ]=============[ VPP -> nginx ] + +where, + +1. nginx attaches to VPP and listens on TCP port 80 +2. ab runs CPS and RPS tests with packets flowing from the Test Generator node, + across 100G NICs, through VPP hoststack to NGINX. +3. At the end of the tests, the results are reported by AB. diff --git a/docs/content/methodology/test/internet_protocol_security.md b/docs/content/methodology/test/internet_protocol_security.md new file mode 100644 index 0000000000..1a02c43a0a --- /dev/null +++ b/docs/content/methodology/test/internet_protocol_security.md @@ -0,0 +1,73 @@ +--- +title: "Internet Protocol Security" +weight: 4 +--- + +# Internet Protocol Security + +VPP Internet Protocol Security (IPsec) performance tests are executed for the +following crypto plugins: + +- `crypto_native`, used for software based crypto leveraging CPU + platform optimizations e.g. Intel's AES-NI instruction set. +- `crypto_ipsecmb`, used for hardware based crypto with Intel QAT PCIe cards. + +## IPsec with VPP Native SW Crypto + +CSIT implements following IPsec test cases relying on VPP native crypto +(`crypto_native` plugin): + + **VPP Crypto Engine** | **ESP Encryption** | **ESP Integrity** | **Scale Tested** +----------------------:|-------------------:|------------------:|-----------------: + crypto_native | AES[128\|256]-GCM | GCM | 1 to 60k tunnels + crypto_native | AES128-CBC | SHA[256\|512] | 1 to 60k tunnels + +VPP IPsec with SW crypto are executed in both tunnel and policy modes, +with tests running on 3-node testbeds: 3n-icx, 3n-tsh. + +## IPsec with Intel QAT HW + +CSIT implements following IPsec test cases relying on ipsecmb library +(`crypto_ipsecmb` plugin) and Intel QAT 8950 (50G HW crypto card): + +dpdk_cryptodev + + **VPP Crypto Engine** | **VPP Crypto Workers** | **ESP Encryption** | **ESP Integrity** | **Scale Tested** +----------------------:|-----------------------:|-------------------:|------------------:|-----------------: + crypto_ipsecmb | sync/all workers | AES[128\|256]-GCM | GCM | 1, 1k tunnels + crypto_ipsecmb | sync/all workers | AES[128]-CBC | SHA[256\|512] | 1, 1k tunnels + crypto_ipsecmb | async/crypto worker | AES[128\|256]-GCM | GCM | 1, 4, 1k tunnels + crypto_ipsecmb | async/crypto worker | AES[128]-CBC | SHA[256\|512] | 1, 4, 1k tunnels + +## IPsec with Async Crypto Feature Workers + +*TODO Description to be added* + +## IPsec Uni-Directional Tests with VPP Native SW Crypto + +CSIT implements following IPsec uni-directional test cases relying on VPP native +crypto (`crypto_native` plugin) in tunnel mode: + + **VPP Crypto Engine** | **ESP Encryption** | **ESP Integrity** | **Scale Tested** +----------------------:|-------------------:|------------------:|-------------------: + crypto_native | AES[128\|256]-GCM | GCM | 4, 1k, 10k tunnels + crypto_native | AES128-CBC | SHA[512] | 4, 1k, 10k tunnels + +In policy mode: + + **VPP Crypto Engine** | **ESP Encryption** | **ESP Integrity** | **Scale Tested** +----------------------:|-------------------:|------------------:|------------------: + crypto_native | AES[256]-GCM | GCM | 1, 40, 1k tunnels + +The tests are running on 2-node testbeds: 2n-tx2. The uni-directional tests +are partially addressing a weakness in 2-node testbed setups with T-Rex as +the traffic generator. With just one DUT node, we can either encrypt or decrypt +traffic in each direction. + +The testcases are only doing encryption - packets are encrypted on the DUT and +then arrive at TG where no additional packet processing is needed (just +counting packets). + +Decryption would require that the traffic generator generated encrypted packets +which the DUT then would decrypt. However, T-Rex does not have the capability +to encrypt packets. diff --git a/docs/content/methodology/test/network_address_translation.md b/docs/content/methodology/test/network_address_translation.md new file mode 100644 index 0000000000..f443eabc5f --- /dev/null +++ b/docs/content/methodology/test/network_address_translation.md @@ -0,0 +1,445 @@ +--- +title: "Network Address Translation" +weight: 1 +--- + +# Network Address Translation + +## NAT44 Prefix Bindings + +NAT44 prefix bindings should be representative to target applications, +where a number of private IPv4 addresses from the range defined by +RFC1918 is mapped to a smaller set of public IPv4 addresses from the +public range. + +Following quantities are used to describe inside to outside IP address +and port bindings scenarios: + +- Inside-addresses, number of inside source addresses + (representing inside hosts). +- Ports-per-inside-address, number of TCP/UDP source + ports per inside source address. +- Outside-addresses, number of outside (public) source addresses + allocated to NAT44. +- Ports-per-outside-address, number of TCP/UDP source + ports per outside source address. The maximal number of + ports-per-outside-address usable for NAT is 64 512 + (in non-reserved port range 1024-65535, RFC4787). +- Sharing-ratio, equal to inside-addresses divided by outside-addresses. + +CSIT NAT44 tests are designed to take into account the maximum number of +ports (sessions) required per inside host (inside-address) and at the +same time to maximize the use of outside-address range by using all +available outside ports. With this in mind, the following scheme of +NAT44 sharing ratios has been devised for use in CSIT: + + **ports-per-inside-address** | **sharing-ratio** +-----------------------------:|------------------: + 63 | 1024 + 126 | 512 + 252 | 256 + 504 | 128 + +Initial CSIT NAT44 tests, including associated TG/TRex traffic profiles, +are based on ports-per-inside-address set to 63 and the sharing ratio of +1024. This approach is currently used for all NAT44 tests including +NAT44det (NAT44 deterministic used for Carrier Grade NAT applications) +and NAT44ed (Endpoint Dependent). + +Private address ranges to be used in tests: + +- 192.168.0.0 - 192.168.255.255 (192.168/16 prefix) + + - Total of 2^16 (65 536) of usable IPv4 addresses. + - Used in tests for up to 65 536 inside addresses (inside hosts). + +- 172.16.0.0 - 172.31.255.255 (172.16/12 prefix) + + - Total of 2^20 (1 048 576) of usable IPv4 addresses. + - Used in tests for up to 1 048 576 inside addresses (inside hosts). + +### NAT44 Session Scale + +NAT44 session scale tested is govern by the following logic: + +- Number of inside-addresses(hosts) H[i] = (H[i-1] x 2^2) with H(0)=1 024, + i = 1,2,3, ... + + - H[i] = 1 024, 4 096, 16 384, 65 536, 262 144, ... + +- Number of sessions S[i] = H[i] * ports-per-inside-address + + - ports-per-inside-address = 63 + + **i** | **hosts** | **sessions** +------:|----------:|-------------: + 0 | 1 024 | 64 512 + 1 | 4 096 | 258 048 + 2 | 16 384 | 1 032 192 + 3 | 65 536 | 4 128 768 + 4 | 262 144 | 16 515 072 + +### NAT44 Deterministic + +NAT44det performance tests are using TRex STL (Stateless) API and traffic +profiles, similar to all other stateless packet forwarding tests like +ip4, ip6 and l2, sending UDP packets in both directions +inside-to-outside and outside-to-inside. + +The inside-to-outside traffic uses single destination address (20.0.0.0) +and port (1024). +The inside-to-outside traffic covers whole inside address and port range, +the outside-to-inside traffic covers whole outside address and port range. + +NAT44det translation entries are created during the ramp-up phase, +followed by verification that all entries are present, +before proceeding to the main measurements of the test. +This ensures session setup does not impact the forwarding performance test. + +Associated CSIT test cases use the following naming scheme to indicate +NAT44det scenario tested: + +- ethip4udp-nat44det-h{H}-p{P}-s{S}-[mrr|ndrpdr|soak] + + - {H}, number of inside hosts, H = 1024, 4096, 16384, 65536, 262144. + - {P}, number of ports per inside host, P = 63. + - {S}, number of sessions, S = 64512, 258048, 1032192, 4128768, + 16515072. + - [mrr|ndrpdr|soak], MRR, NDRPDR or SOAK test. + +### NAT44 Endpoint-Dependent + +In order to excercise NAT44ed ability to translate based on both +source and destination address and port, the inside-to-outside traffic +varies also destination address and port. Destination port is the same +as source port, destination address has the same offset as the source address, +but applied to different subnet (starting with 20.0.0.0). + +As the mapping is not deterministic (for security reasons), +we cannot easily use stateless bidirectional traffic profiles. +Inside address and port range is fully covered, +but we do not know which outside-to-inside source address and port to use +to hit an open session. + +Therefore, NAT44ed is benchmarked using following methodologies: + +- Unidirectional throughput using *stateless* traffic profile. +- Connections-per-second (CPS) using *stateful* traffic profile. +- Bidirectional throughput (TPUT, see below) using *stateful* traffic profile. + +Unidirectional NAT44ed throughput tests are using TRex STL (Stateless) +APIs and traffic profiles, but with packets sent only in +inside-to-outside direction. +Similarly to NAT44det, NAT44ed unidirectional throughput tests include +a ramp-up phase to establish and verify the presence of required NAT44ed +binding entries. As the sessions have finite duration, the test code +keeps inserting ramp-up trials during the search, if it detects a risk +of sessions timing out. Any zero loss trial visits all sessions, +so it acts also as a ramp-up. + +Stateful NAT44ed tests are using TRex ASTF (Advanced Stateful) APIs and +traffic profiles, with packets sent in both directions. Tests are run +with both UDP and TCP sessions. +As NAT44ed CPS (connections-per-second) stateful tests +measure (also) session opening performance, +they use state reset instead of ramp-up trial. +NAT44ed TPUT (bidirectional throughput) tests prepend ramp-up trials +as in the unidirectional tests, +so the test results describe performance without translation entry +creation overhead. + +Associated CSIT test cases use the following naming scheme to indicate +NAT44det case tested: + +- Stateless: ethip4udp-nat44ed-h{H}-p{P}-s{S}-udir-[mrr|ndrpdr|soak] + + - {H}, number of inside hosts, H = 1024, 4096, 16384, 65536, 262144. + - {P}, number of ports per inside host, P = 63. + - {S}, number of sessions, S = 64512, 258048, 1032192, 4128768, + 16515072. + - udir-[mrr|ndrpdr|soak], unidirectional stateless tests MRR, NDRPDR + or SOAK. + +- Stateful: ethip4[udp|tcp]-nat44ed-h{H}-p{P}-s{S}-[cps|tput]-[mrr|ndrpdr|soak] + + - [udp|tcp], UDP or TCP sessions + - {H}, number of inside hosts, H = 1024, 4096, 16384, 65536, 262144. + - {P}, number of ports per inside host, P = 63. + - {S}, number of sessions, S = 64512, 258048, 1032192, 4128768, + 16515072. + - [cps|tput], connections-per-second session establishment rate or + packets-per-second average rate, or packets-per-second rate + without session establishment. + - [mrr|ndrpdr|soak], bidirectional stateful tests MRR, NDRPDR, or SOAK. + +## Stateful traffic profiles + +There are several important details which distinguish ASTF profiles +from stateless profiles. + +### General considerations + +#### Protocols + +ASTF profiles are limited to either UDP or TCP protocol. + +#### Programs + +Each template in the profile defines two "programs", one for the client side +and one for the server side. + +Each program specifies when that side has to wait until enough data is received +(counted in packets for UDP and in bytes for TCP) +and when to send additional data. Together, the two programs +define a single transaction. Due to packet loss, transaction may take longer, +use more packets (retransmission) or never finish in its entirety. + +#### Instances + +A client instance is created according to TPS parameter for the trial, +and sends the first packet of the transaction (in some cases more packets). +Each client instance uses a different source address (see sequencing below) +and some source port. The destination address also comes from a range, +but destination port has to be constant for a given program. + +TRex uses an opaque way to chose source ports, but as session counting shows, +next client with the same source address uses a different source port. + +Server instance is created when the first packet arrives to the server side. +Source address and port of the first packet are used as destination address +and port for the server responses. This is the ability we need +when outside surface is not predictable. + +When a program reaches its end, the instance is deleted. +This creates possible issues with server instances. If the server instance +does not read all the data client has sent, late data packets +can cause a second copy of server instance to be created, +which breaks assumptions on how many packet a transaction should have. + +The need for server instances to read all the data reduces the overall +bandwidth TRex is able to create in ASTF mode. + +Note that client instances are not created on packets, +so it is safe to end client program without reading all server data +(unless the definition of transaction success requires that). + +#### Sequencing + +ASTF profiles offer two modes for choosing source and destination IP addresses +for client programs: seqential and pseudorandom. +In current tests we are using sequential addressing only (if destination +address varies at all). + +For client destination UDP/TCP port, we use a single constant value. +(TRex can support multiple program pairs in the same traffic profile, +distinguished by the port number.) + +#### Transaction overlap + +If a transaction takes longer to finish, compared to period implied by TPS, +TRex will have multiple client or server instances active at a time. + +During calibration testing we have found this increases CPU utilization, +and for high TPS it can lead to TRex's Rx or Tx buffers becoming full. +This generally leads to duration stretching, and/or packet loss on TRex. + +Currently used transactions were chosen to be short, so risk of bad behavior +is decreased. But in MRR tests, where load is computed based on NIC ability, +not TRex ability, anomalous behavior is still possible +(e.g. MRR values being way lower than NDR). + +#### Delays + +TRex supports adding constant delays to ASTF programs. +This can be useful, for example if we want to separate connection establishment +from data transfer. + +But as TRex tracks delayed instances as active, this still results +in higher CPU utilization and reduced performance issues +(as other overlaping transactions). So the current tests do not use any delays. + +#### Keepalives + +Both UDP and TCP protocol implementations in TRex programs support keepalive +duration. That means there is a configurable period of keepalive time, +and TRex sends keepalive packets automatically (outside the program) +for the time the program is active (started, not ended yet) +but not sending any packets. + +For TCP this is generally not a big deal, as the other side usually +retransmits faster. But for UDP it means a packet loss may leave +the receiving program running. + +In order to avoid keepalive packets, keepalive value is set to a high number. +Here, "high number" means that even at maximum scale and minimum TPS, +there are still no keepalive packets sent within the corresponding +(computed) trial duration. This number is kept the same also for +smaller scale traffic profiles, to simplify maintenance. + +#### Transaction success + +The transaction is considered successful at Layer-7 (L7) level +when both program instances close. At this point, various L7 counters +(unofficial name) are updated on TRex. + +We found that proper close and L7 counter update can be CPU intensive, +whereas lower-level counters (ipackets, opackets) called L2 counters +can keep up with higher loads. + +For some tests, we do not need to confirm the whole transaction was successful. +CPS (connections per second) tests are a typical example. +We care only for NAT44ed creating a session (needs one packet +in inside-to-outside direction per session) and being able to use it +(needs one packet in outside-to-inside direction). + +Similarly in TPUT tests (packet throuput, counting both control +and data packets), we care about NAT44ed ability to forward packets, +we do not care whether aplications (TRex) can fully process them at that rate. + +Therefore each type of tests has its own formula (usually just one counter +already provided by TRex) to count "successful enough" transactions +and attempted transactions. Currently, all tests relying on L7 counters +use size-limited profiles, so they know what the count of attempted +transactions should be, but due to duration stretching +TRex might have been unable to send that many packets. +For search purposes, unattempted transactions are treated the same +as attempted but failed transactions. + +Sometimes even the number of transactions as tracked by search algorithm +does not match the transactions as defined by ASTF programs. +See TCP TPUT profile below. + +### UDP CPS + +This profile uses a minimalistic transaction to verify NAT44ed session has been +created and it allows outside-to-inside traffic. + +Client instance sends one packet and ends. +Server instance sends one packet upon creation and ends. + +In principle, packet size is configurable, +but currently used tests apply only one value (100 bytes frame). + +Transaction counts as attempted when opackets counter increases on client side. +Transaction counts as successful when ipackets counter increases on client side. + +### TCP CPS + +This profile uses a minimalistic transaction to verify NAT44ed session has been +created and it allows outside-to-inside traffic. + +Client initiates TCP connection. Client waits until connection is confirmed +(by reading zero data bytes). Client ends. +Server accepts the connection. Server waits for indirect confirmation +from client (by waiting for client to initiate close). Server ends. + +Without packet loss, the whole transaction takes 7 packets to finish +(4 and 3 per direction). +From NAT44ed point of view, only the first two are needed to verify +the session got created. + +Packet size is not configurable, but currently used tests report +frame size as 64 bytes. + +Transaction counts as attempted when tcps_connattempt counter increases +on client side. +Transaction counts as successful when tcps_connects counter increases +on client side. + +### UDP TPUT + +This profile uses a small transaction of "request-response" type, +with several packets simulating data payload. + +Client sends 5 packets and closes immediately. +Server reads all 5 packets (needed to avoid late packets creating new +server instances), then sends 5 packets and closes. +The value 5 was chosen to mirror what TCP TPUT (see below) choses. + +Packet size is configurable, currently we have tests for 100, +1518 and 9000 bytes frame (to match size of TCP TPUT data frames, see below). + +As this is a packet oriented test, we do not track the whole +10 packet transaction. Similarly to stateless tests, we treat each packet +as a "transaction" for search algorthm packet loss ratio purposes. +Therefore a "transaction" is attempted when opacket counter on client +or server side is increased. Transaction is successful if ipacket counter +on client or server side is increased. + +If one of 5 client packets is lost, server instance will get stuck +in the reading phase. This probably decreases TRex performance, +but it leads to more stable results then alternatives. + +### TCP TPUT + +This profile uses a small transaction of "request-response" type, +with some data amount to be transferred both ways. + +In CSIT release 22.06, TRex behavior changed, so we needed to edit +the traffic profile. Let us describe the pre-22.06 profile first. + +Client connects, sends 5 data packets worth of data, +receives 5 data packets worth of data and closes its side of the connection. +Server accepts connection, reads 5 data packets worth of data, +sends 5 data packets worth of data and closes its side of the connection. +As usual in TCP, sending side waits for ACK from the receiving side +before proceeding with next step of its program. + +Server read is needed to avoid premature close and second server instance. +Client read is not stricly needed, but ACKs allow TRex to close +the server instance quickly, thus saving CPU and improving performance. + +The number 5 of data packets was chosen so TRex is able to send them +in a single burst, even with 9000 byte frame size (TRex has a hard limit +on initial window size). +That leads to 16 packets (9 of them in c2s direction) to be exchanged +if no loss occurs. +The size of data packets is controlled by the traffic profile setting +the appropriate maximum segment size. Due to TRex restrictions, +the minimal size for IPv4 data frame achievable by this method is 70 bytes, +which is more than our usual minimum of 64 bytes. +For that reason, the data frame sizes available for testing are 100 bytes +(that allows room for eventually adding IPv6 ASTF tests), +1518 bytes and 9000 bytes. There is no control over control packet sizes. + +Exactly as in UDP TPUT, ipackets and opackets counters are used for counting +"transactions" (in fact packets). + +If packet loss occurs, there can be large transaction overlap, even if most +ASTF programs finish eventually. This can lead to big duration stretching +and somehow uneven rate of packets sent. This makes it hard to interpret +MRR results (frequently MRR is below NDR for this reason), +but NDR and PDR results tend to be stable enough. + +In 22.06, the "ACK from the receiving side" behavior changed, +the receiving side started sending ACK sometimes +also before receiving the full set of 5 data packets. +If the previous profile is understood as a "single challenge, single response" +where challenge (and also response) is sent as a burst of 5 data packets, +the new profile uses "bursts" of 1 packet instead, but issues +the challenge-response part 5 times sequentially +(waiting for receiving the response before sending next challenge). +This new profile happens to have the same overall packet count +(when no re-transmissions are needed). +Although it is possibly more taxing for TRex CPU, +the results are comparable to the old traffic profile. + +## Ip4base tests + +Contrary to stateless traffic profiles, we do not have a simple limit +that would guarantee TRex is able to send traffic at specified load. +For that reason, we have added tests where "nat44ed" is replaced by "ip4base". +Instead of NAT44ed processing, the tests set minimalistic IPv4 routes, +so that packets are forwarded in both inside-to-outside and outside-to-inside +directions. + +The packets arrive to server end of TRex with different source address&port +than in NAT44ed tests (no translation to outside values is done with ip4base), +but those are not specified in the stateful traffic profiles. +The server end (as always) uses the received address&port as destination +for outside-to-inside traffic. Therefore the same stateful traffic profile +works for both NAT44ed and ip4base test (of the same scale). + +The NAT44ed results are displayed together with corresponding ip4base results. +If they are similar, TRex is probably the bottleneck. +If NAT44ed result is visibly smaller, it describes the real VPP performance. diff --git a/docs/content/methodology/test/packet_flow_ordering.md b/docs/content/methodology/test/packet_flow_ordering.md new file mode 100644 index 0000000000..c2c87038d4 --- /dev/null +++ b/docs/content/methodology/test/packet_flow_ordering.md @@ -0,0 +1,42 @@ +--- +title: "Packet Flow Ordering" +weight: 2 +--- + +# Packet Flow Ordering + +TRex Traffic Generator (TG) supports two main ways how to cover +address space (on allowed ranges) in scale tests. + +In most cases only one field value (e.g. IPv4 destination address) is +altered, in some cases two fields (e.g. IPv4 destination address and UDP +destination port) are altered. + +## Incremental Ordering + +This case is simpler to implement and offers greater control. + +When changing two fields, they can be incremented synchronously, or one +after another. In the latter case we can specify which one is +incremented each iteration and which is incremented by "carrying over" +only when the other "wraps around". This way also visits all +combinations once before the "carry" field also wraps around. + +It is possible to use increments other than 1. + +## Randomized Ordering + +This case chooses each field value at random (from the allowed range). +In case of two fields, they are treated independently. +TRex allows to set random seed to get deterministic numbers. +We use a different seed for each field and traffic direction. +The seed has to be a non-zero number, we use 1, 2, 3, and so on. + +The seeded random mode in TRex requires a "limit" value, +which acts as a cycle length limit (after this many iterations, +the seed resets to its initial value). +We use the maximal allowed limit value (computed as 2^24 - 1). + +Randomized profiles do not avoid duplicated values, +and do not guarantee each possible value is visited, +so it is not very useful for stateful tests. diff --git a/docs/content/methodology/test/reassembly.md b/docs/content/methodology/test/reassembly.md new file mode 100644 index 0000000000..61064d1bbc --- /dev/null +++ b/docs/content/methodology/test/reassembly.md @@ -0,0 +1,48 @@ +--- +title: "Reassembly" +weight: 10 +--- + +# Packet reassembly performance + +IP protocols (mainly IPv4) specify conditions for packet fragmentation +and packet reassembly. For VPP, the reassembly operation is more CPU intensive. +By default, VPP avoids unnecessary work, so there are only few scenarios +where VPP fragments IP packets, and even less scenarios where it reassemblies +the fragmented packets. + +The typical situation when fragmentation is performed occurs with +tunnel encapsulation protocols, when the packet after encapsulation +would not fit into interface MTU (maximum transmission unit). +Some, but not all, encapsulation protocols also require +packet reassembly for decapsulation. + +As the search algorithms used in CSIT work best when the number of packets +coming from TG (traffic generator) is the same +as the number of packets expected to come back to TG, +the easiest way to test reassembly performance of VPP is using +a 3-node testbed and a tunneling test suite adapted to cause fragmentation. + +## MTU + +By default, testbeds in CSIT are configured with MTU high enough +for encapsulated packets to fit in. +Not all devices and drivers used by VPP do support lowering MTU value. +For reassembly tests, only the physical interfaces on the DUT1-DUT2 link +have lowered MTU, and that currently works only with dpdk plugin. + +## Impacts + +Reassembly suites with small number of flows and tunnels +usually place encapsulation+fragmentation and reassembly+decapsulation +on different workers, so the bottleneck seen in performance results +is not affected by fragmentation performance. + +Reassembly suites with high number of flows and tunnels +achieve balanced load on all workers, so their overall performance +is affected by both fragmentation and reassembly performance. + +Some protocols (e.g. IPsec) are CPU intensive not only +on fragmentation and reassembly, but also on encapsulation and decapsulation. +Reassembly (and depending on scale also fragmentation) impact +on those tests can still be visible, at least for big regressions. diff --git a/docs/content/methodology/test/reconfiguration.md b/docs/content/methodology/test/reconfiguration.md new file mode 100644 index 0000000000..a0678103d9 --- /dev/null +++ b/docs/content/methodology/test/reconfiguration.md @@ -0,0 +1,70 @@ +--- +title: "Reconfiguration" +weight: 8 +--- + +# Reconfiguration + +## Overview + +Reconf tests are designed to measure the impact of VPP re-configuration +on data plane traffic. +While VPP takes some measures against the traffic being +entirely stopped for a prolonged time, +the immediate forwarding rate varies during the re-configuration, +as some configurations steps need the active dataplane worker threads +to be stopped temporarily. + +As the usual methods of measuring throughput need multiple trial measurements +with somewhat long durations, and the re-configuration process can also be long, +finding an offered load which would result in zero loss +during the re-configuration process would be time-consuming. + +Instead, reconf tests first find a througput value (lower bound for NDR) +without re-configuration, and then maintain that ofered load +during re-configuration. The measured loss count is then assumed to be caused +by the re-configuration process. The result published by reconf tests +is the effective blocked time, that is +the loss count divided by the offered load. + +## Current Implementation + +Each reconf suite is based on a similar MLRsearch performance suite. + +MLRsearch parameters are changed to speed up the throughput discovery. +For example, PDR is not searched for, and the goal duration sum is shorter. +Contrary to usual NDR, exceed ratio is set to zero in reconf tests +as load stability is less important than attribution of losses. + +The MLRsearch suite has to contain a configuration parameter +that can be scaled up, e.g. number of tunnels or number of service chains. +Currently, only increasing the scale is supported +as the re-configuration operation. In future, scale decrease +or other operations can be implemented. + +The traffic profile is not changed, so the traffic present is processed +only by the smaller scale configuration. The added tunnels / chains +are not targetted by the traffic. + +For the re-configuration, the same Robot Framework and Python libraries +are used, as were used in the initial configuration, with the exception +of the final calls that do not interact with VPP (e.g. starting +virtual machines) being skipped to reduce the test overall duration. + +## Discussion + +Robot Framework introduces a certain overhead, which may affect timing +of individual VPP API calls, which in turn may affect +the number of packets lost. + +The exact calls executed may contain unnecessary info dumps, repeated commands, +or commands which change a value that do not need to be changed (e.g. MTU). +Thus, implementation details are affecting the results, even if their effect +on the corresponding MLRsearch suite is negligible. + +The lower bound for NDR is the only value safe to be used when zero packets lost +are expected without re-configuration. But different suites show different +"jitter" in that value. For some suites, the lower bound is not tight, +allowing full NIC buffers to drain quickly between worker pauses. +For other suites, lower bound for NDR still has quite a large probability +of non-zero packet loss even without re-configuration. diff --git a/docs/content/methodology/test/tunnel_encapsulations.md b/docs/content/methodology/test/tunnel_encapsulations.md new file mode 100644 index 0000000000..c047c43dfa --- /dev/null +++ b/docs/content/methodology/test/tunnel_encapsulations.md @@ -0,0 +1,87 @@ +--- +title: "Tunnel Encapsulations" +weight: 3 +--- + +# Tunnel Encapsulations + +Tunnel encapsulations testing is grouped based on the type of outer +header: IPv4 or IPv6. + +## IPv4 Tunnels + +VPP is tested in the following IPv4 tunnel baseline configurations: + +- *ip4vxlan-l2bdbase*: VXLAN over IPv4 tunnels with L2 bridge-domain MAC + switching. +- *ip4vxlan-l2xcbase*: VXLAN over IPv4 tunnels with L2 cross-connect. +- *ip4lispip4-ip4base*: LISP over IPv4 tunnels with IPv4 routing. +- *ip4lispip6-ip6base*: LISP over IPv4 tunnels with IPv6 routing. +- *ip4gtpusw-ip4base*: GTPU over IPv4 tunnels with IPv4 routing. + +In all cases listed above low number of MAC, IPv4, IPv6 flows (253 or 254 per +direction) is switched or routed by VPP. + +In addition selected IPv4 tunnels are tested at scale: + +- *dot1q--ip4vxlanscale-l2bd*: VXLAN over IPv4 tunnels with L2 bridge- + domain MAC switching, with scaled up dot1q VLANs (10, 100, 1k), + mapped to scaled up L2 bridge-domains (10, 100, 1k), that are in turn + mapped to (10, 100, 1k) VXLAN tunnels. 64.5k flows are transmitted per + direction. + +## IPv6 Tunnels + +VPP is tested in the following IPv6 tunnel baseline configurations: + +- *ip6lispip4-ip4base*: LISP over IPv4 tunnels with IPv4 routing. +- *ip6lispip6-ip6base*: LISP over IPv4 tunnels with IPv6 routing. + +In all cases listed above low number of IPv4, IPv6 flows (253 or 254 per +direction) is routed by VPP. + +## GENEVE + +### GENEVE Prefix Bindings + +GENEVE prefix bindings should be representative to target applications, where +a packet flows of particular set of IPv4 addresses (L3 underlay network) is +routed via dedicated GENEVE interface by building an L2 overlay. + +Private address ranges to be used in tests: + +- East hosts ip address range: 10.0.1.0 - 10.127.255.255 (10.0/9 prefix) + - Total of 2^23 - 256 (8 388 352) of usable IPv4 addresses + - Usable in tests for up to 32 767 GENEVE tunnels (IPv4 underlay networks) +- West hosts ip address range: 10.128.1.0 - 10.255.255.255 (10.128/9 prefix) + - Total of 2^23 - 256 (8 388 352) of usable IPv4 addresses + - Usable in tests for up to 32 767 GENEVE tunnels (IPv4 underlay networks) + +### GENEVE Tunnel Scale + +If N is a number of GENEVE tunnels (and IPv4 underlay networks) then TG sends +256 packet flows in every of N different sets: + +- i = 1,2,3, ... N - GENEVE tunnel index +- East-West direction: GENEVE encapsulated packets + - Outer IP header: + - src ip: 1.1.1.1 + - dst ip: 1.1.1.2 + - GENEVE header: + - vni: i + - Inner IP header: + - src_ip_range(i) = 10.(0 + rounddown(i/255)).(modulo(i/255)).(0-to-255) + - dst_ip_range(i) = 10.(128 + rounddown(i/255)).(modulo(i/255)).(0-to-255) +- West-East direction: non-encapsulated packets + - IP header: + - src_ip_range(i) = 10.(128 + rounddown(i/255)).(modulo(i/255)).(0-to-255) + - dst_ip_range(i) = 10.(0 + rounddown(i/255)).(modulo(i/255)).(0-to-255) + + **geneve-tunnels** | **total-flows** +-------------------:|----------------: + 1 | 256 + 4 | 1 024 + 16 | 4 096 + 64 | 16 384 + 256 | 65 536 + 1 024 | 262 144 diff --git a/docs/content/methodology/test/vpp_device.md b/docs/content/methodology/test/vpp_device.md new file mode 100644 index 0000000000..0a5ee90308 --- /dev/null +++ b/docs/content/methodology/test/vpp_device.md @@ -0,0 +1,15 @@ +--- +title: "VPP Device" +weight: 9 +--- + +# VPP Device + +Includes VPP_Device test environment for functional VPP +device tests integrated into LFN CI/CD infrastructure. VPP_Device tests +run on 1-Node testbeds (1n-skx, 1n-arm) and rely on Linux SRIOV Virtual +Function (VF), dot1q VLAN tagging and external loopback cables to +facilitate packet passing over external physical links. Initial focus is +on few baseline tests. New device tests can be added by small edits +to existing CSIT Performance (2-node) test. RF test definition code +stays unchanged with the exception of traffic generator related L2 KWs. diff --git a/docs/content/methodology/trending/_index.md b/docs/content/methodology/trending/_index.md new file mode 100644 index 0000000000..94fe7c8288 --- /dev/null +++ b/docs/content/methodology/trending/_index.md @@ -0,0 +1,16 @@ +--- +bookCollapseSection: true +bookFlatSection: false +title: "Trending" +weight: 4 +--- + +# Trending + +This document describes a high-level design of a system for continuous +performance measuring, trending and change detection for FD.io VPP SW +data plane (and other performance tests run within CSIT sub-project). + +- [Analysis]({{< relref "/methodology/trending/analysis" >}}) +- [Presentation]({{< relref "/methodology/trending/presentation" >}}) +- [Previous]({{< relref "/methodology/trending/previous" >}}) diff --git a/docs/content/methodology/trending/analysis.md b/docs/content/methodology/trending/analysis.md new file mode 100644 index 0000000000..eb1c8a741b --- /dev/null +++ b/docs/content/methodology/trending/analysis.md @@ -0,0 +1,231 @@ +--- +title: "Analysis" +weight: 1 +--- + +# Trend Analysis + +All measured performance trend data is treated as time-series data +that is modeled as a concatenation of groups, +within each group the samples come (independently) from +the same normal distribution (with some center and standard deviation). + +Center of the normal distribution for the group (equal to population average) +is called a trend for the group. +All the analysis is based on finding the right partition into groups +and comparing their trends. + +## Anomalies in graphs + +In graphs, the start of the following group is marked as a regression (red +circle) or progression (green circle), if the new trend is lower (or higher +respectively) then the previous group's. + +## Implementation details + +### Partitioning into groups + +While sometimes the samples within a group are far from being distributed +normally, currently we do not have a better tractable model. + +Here, "sample" should be the result of single trial measurement, with group +boundaries set only at test run granularity. But in order to avoid detecting +causes unrelated to VPP performance, the current presentation takes average of +all trials within the MRR run as the sample. Effectively, this acts as a single +trial with aggregate duration. +(Trending of NDR or PDR results take just one sample, the conditional throughput). + +Performance graphs show the run average as a dot (not all individual trial +results). + +The group boundaries are selected based on `Minimum Description Length`[^1]. + +### Minimum Description Length + +`Minimum Description Length`[^1] (MDL) is a particular formalization +of `Occam's razor`[^2] principle. + +The general formulation mandates to evaluate a large set of models, +but for anomaly detection purposes, it is useful to consider +a smaller set of models, so that scoring and comparing them is easier. + +For each candidate model, the data should be compressed losslessly, +which includes model definitions, encoded model parameters, +and the raw data encoded based on probabilities computed by the model. +The model resulting in shortest compressed message is the "the" correct model. + +For our model set (groups of normally distributed samples), +we need to encode group length (which penalizes too many groups), +group average (more on that later), group stdev and then all the samples. + +Luckily, the "all the samples" part turns out to be quite easy to compute. +If sample values are considered as coordinates in (multi-dimensional) +Euclidean space, fixing average restrict possible values to a (hyper-)plane. +Then, fixing stdev means the point with allowed coordinates +lays on a sphere (centered the "all samples equal to average" point) +within that hyper-plane. +And the Gaussian probability density on the resulting sphere is constant. +So the only contribution is the "area" of the sphere, which only depends +on the number of samples and stdev. + +Still, to get the information content in bits, we need to know what "size" +one "pixel" of that area is. +Our implementation assumes that measurement precision is such that +the max sample value is 4096 (2^12) pixels (inspired by 0.5% precision +of NDRPDR tests, roughly two pixels around max value). + +A somehow ambiguous part is in choosing which encoding +is used for group size, average and stdev. +Different encodings cause different biases to large or small values. +In our implementation we have chosen probability density +corresponding to uniform distribution (from zero to maximal sample value) +for stdev and average of the first group, +but for averages of subsequent groups we have chosen a distribution +which discourages delimiting groups with averages close together. + +The group selection algorithm currently has no parameters, +all the aforementioned encodings and handling of precision is hard-coded. +(Although the underlying library "jumpavg" allows users to change the precision, +either in absolute units or in bits per max sample.) + +In principle, every group selection is examined, and the one encodable +with least amount of bits is selected. +As the bit amount for a selection is just sum of bits for every group, +finding the best selection takes number of comparisons +quadratically increasing with the size of data, +the overall time complexity being probably cubic. + +The resulting group distribution looks good +if samples are distributed normally enough within a group. +But for obviously different distributions (for example +`bimodal distribution`[^3]) the groups tend to focus on less relevant factors +(such as "outlier" density). + +## Common Patterns + +When an anomaly is detected, it frequently falls into few known patterns, +each having its typical behavior over time. + +We are going to describe the behaviors, +as they motivate our choice of trend compliance metrics. + +### Sample time and analysis time + +But first we need to distinguish two roles time plays in analysis, +so it is more clear which role we are referring to. + +Sample time is the more obvious one. +It is the time the sample is generated. +It is the start time or the end time of the Jenkins job run, +does not really matter which (parallel runs are disabled, +and length of gap between samples does not affect metrics). + +Analysis time is the time the current analysis is computed. +Again, the exact time does not usually matter, +what matters is how many later (and how fewer earlier) samples +were considered in the computation. + +For some patterns, it is usual for a previously reported +anomaly to "vanish", or previously unseen anomaly to "appear late", +as later samples change which partition into groups is more probable. + +Dashboard and graphs are always showing the latest analysis time, +the compliance metrics are using earlier sample time +with the same latest analysis time. + +Alerting e-mails use the latest analysis time at the time of sending, +so the values reported there are likely to be different +from the later analysis time results shown in dashboard and graphs. + +### Ordinary regression + +The real performance changes from previously stable value +into a new stable value. + +For medium to high magnitude of the change, one run +is enough for anomaly detection to mark this regression. + +Ordinary progressions are detected in the same way. + +### Small regression + +The real performance changes from previously stable value +into a new stable value, but the difference is small. + +For the anomaly detection algorithm, this change is harder to detect, +depending on the standard deviation of the previous group. + +If the new performance value stays stable, eventually +the detection algorithm is able to detect this anomaly +when there are enough samples around the new value. + +If the difference is too small, it may remain undetected +(as new performance change happens, or full history of samples +is still not enough for the detection). + +Small progressions have the same behavior. + +### Reverted regression + +This pattern can have two different causes. +We would like to distinguish them, but that is usually +not possible to do just by looking at the measured values (and not telemetry). + +In one cause, the real DUT performance has changed, +but got restored immediately. +In the other cause, no real performance change happened, +just some temporary infrastructure issue +has caused a wrong low value to be measured. + +For small measured changes, this pattern may remain undetected. +For medium and big measured changes, this is detected when the regression +happens on just the last sample. + +For big changes, the revert is also immediately detected +as a subsequent progression. The trend is usually different +from the previously stable trend (as the two population averages +are not likely to be exactly equal), but the difference +between the two trends is relatively small. + +For medium changes, the detection algorithm may need several new samples +to detect a progression (as it dislikes single sample groups), +in the meantime reporting regressions (difference decreasing +with analysis time), until it stabilizes the same way as for big changes +(regression followed by progression, small difference +between the old stable trend and last trend). + +As it is very hard for a fault code or an infrastructure issue +to increase performance, the opposite (temporary progression) +almost never happens. + +### Summary + +There is a trade-off between detecting small regressions +and not reporting the same old regressions for a long time. + +For people reading e-mails, a sudden regression with a big number of samples +in the last group means this regression was hard for the algorithm to detect. + +If there is a big regression with just one run in the last group, +we are not sure if it is real, or just a temporary issue. +It is useful to wait some time before starting an investigation. + +With decreasing (absolute value of) difference, the number of expected runs +increases. If there is not enough runs, we still cannot distinguish +real regression from temporary regression just from the current metrics +(although humans frequently can tell by looking at the graph). + +When there is a regression or progression with just a small difference, +it is probably an artifact of a temporary regression. +Not worth examining, unless temporary regressions happen somewhat frequently. + +It is not easy for the metrics to locate the previous stable value, +especially if multiple anomalies happened in the last few weeks. +It is good to compare last trend with long term trend maximum, +as it highlights the difference between "now" and "what could be". +It is good to exclude last week from the trend maximum, +as including the last week would hide all real progressions. + +[^1]: [Minimum Description Length](https://en.wikipedia.org/wiki/Minimum_description_length) +[^2]: [Occam's Razor](https://en.wikipedia.org/wiki/Occam%27s_razor) +[^3]: [Bimodal Distribution](https://en.wikipedia.org/wiki/Bimodal_distribution) diff --git a/docs/content/methodology/trending/presentation.md b/docs/content/methodology/trending/presentation.md new file mode 100644 index 0000000000..91bbef8db9 --- /dev/null +++ b/docs/content/methodology/trending/presentation.md @@ -0,0 +1,36 @@ +--- +title: "Presentation" +weight: 2 +--- + +# Trend Presentation + +## Failed tests + +The [Failed tests tables](https://csit.fd.io/news/) list the tests which failed +during the last test run. Separate tables are generated for each testbed. + +## Regressions and progressions + +[These tables](https://csit.fd.io/news/) list tests which encountered +a regression or progression during the specified time period, which is currently +set to the last 1, 7, and 130 days. + +## Trendline Graphs + +[Trendline graphs](https://csit.fd.io/trending/) show measured per run averages +of MRR values, NDR or PDR values, user-selected telemetry metrics, group average +values, and detected anomalies. The graphs are constructed as follows: + +- X-axis represents the date in the format MMDD. +- Y-axis represents run-average MRR value, NDR or PDR values in Mpps or selected + metrics. For PDR tests also a graph with average latency at 50% PDR [us] is + generated. +- Markers to indicate anomaly classification: + - Regression - red circle. + - Progression - green circle. +- The line shows average value of each group. + +In addition the graphs show dynamic labels while hovering over graph data +points, presenting the CSIT build date, measured value, VPP reference, trend job +build ID and the LF testbed ID. diff --git a/docs/content/methodology/trending/previous.md b/docs/content/methodology/trending/previous.md new file mode 100644 index 0000000000..5f402c42c4 --- /dev/null +++ b/docs/content/methodology/trending/previous.md @@ -0,0 +1,11 @@ +--- +title: "Previous" +weight: 3 +--- + +# Previous + +The previous version of +[Continuous Performance Trending](https://s3-docs.fd.io/csit/master/trending/) +is no more maintained nor updated. It displays valid but old data until the 18th +of August 2022 when it was replaced by [CSIT-Dash](https://csit.fd.io/). diff --git a/docs/content/overview/_index.md b/docs/content/overview/_index.md new file mode 100644 index 0000000000..6c4d4210fd --- /dev/null +++ b/docs/content/overview/_index.md @@ -0,0 +1,6 @@ +--- +bookCollapseSection: false +bookFlatSection: true +title: "Overview" +weight: 1 +---
\ No newline at end of file diff --git a/docs/content/overview/c_dash/_index.md b/docs/content/overview/c_dash/_index.md new file mode 100644 index 0000000000..fdf583f377 --- /dev/null +++ b/docs/content/overview/c_dash/_index.md @@ -0,0 +1,6 @@ +--- +bookCollapseSection: true +bookFlatSection: false +title: "CSIT-Dash" +weight: 1 +---
\ No newline at end of file diff --git a/docs/content/overview/c_dash/design.md b/docs/content/overview/c_dash/design.md new file mode 100644 index 0000000000..ca985f993e --- /dev/null +++ b/docs/content/overview/c_dash/design.md @@ -0,0 +1,16 @@ +--- +title: "Design" +weight: 1 +--- + +# Design + +From a test to the graph or table. + +## Tests + +## ETL Pipeline + +{{< figure src="/cdocs/csit_etl_for_uti_data_flow_simplified.svg" title="CSIT ETL pipeline for UTI data" >}} + +## Presentation diff --git a/docs/content/overview/c_dash/structure.md b/docs/content/overview/c_dash/structure.md new file mode 100644 index 0000000000..982655e984 --- /dev/null +++ b/docs/content/overview/c_dash/structure.md @@ -0,0 +1,111 @@ +--- +title: "Structure" +weight: 2 +--- + +# Structure + +CSIT-Dash provides customizable views on performance data. We can split it into +two groups. The first one is performance trending, which displays data collected +on daily (MRR) or weekly (NDRPDR) basis. +The other one presents data coming from release testing. In addition, we publish +also information and statistics about our test jobs, failures and anomalies and +the CSIT documentation. +The screen of CSIT-Dash is divided in two parts. On the left side, there is the +control panel which makes possible to select required information. The right +side then displays the user-selected data in graphs or tables. + +The structure of CSIT-Dash consist of: + +- Performance Trending +- Per Release Performance +- Per Release Performance Comparisons +- Per Release Coverage Data +- Test Job Statistics +- Failures and Anomalies +- Documentation + +## Performance Trending + +Performance trending shows measured per run averages of MRR values, NDR or PDR +values, user-selected telemetry metrics, group average values, and detected +anomalies. + +In addition, the graphs show dynamic labels while hovering over graph data +points. By clicking on data samples, the user gets detailed information and for +latency graphs also high dynamic range histogram of measured latency. +Latency by percentile distribution plots are used to show packet latency +percentiles at different packet rate load levels: +- No-Load, latency streams only, +- Low-Load at 10% PDR, +- Mid-Load at 50% PDR and +- High-Load at 90% PDR. + +## Per Release Performance + +Per release performance section presents the graphs based on the results data +obtained from the release test jobs. In order to verify benchmark results +repeatability, CSIT performance tests are executed multiple times (target: 10 +times) on each physical testbed type. Box-and-Whisker plots are used to display +variations in measured throughput and latency (PDR tests only) values. + +In addition, the graphs show dynamic labels while hovering over graph data +points. By clicking on data samples or the box, the user gets detailed +information and for latency graphs also high dynamic range histogram of measured +latency. +Latency by percentile distribution plots are used to show packet latency +percentiles at different packet rate load levels: +- No-Load, latency streams only, +- Low-Load at 10% PDR, +- Mid-Load at 50% PDR and +- High-Load at 90% PDR. + +## Per Release Performance Comparisons + +Relative comparison of packet throughput (NDR, PDR and MRR) and latency (PDR) +between user-selected releases, test beds, NICs, ... is calculated from results +of tests running on physical test beds, in 1-core, 2-core and 4-core +configurations. + +Listed mean and standard deviation values are computed based on a series of the +same tests executed against respective VPP releases to verify test results +repeatability, with percentage change calculated for mean values. Note that the +standard deviation is quite high for a small number of packet throughput tests, +what indicates poor test results repeatability and makes the relative change of +mean throughput value not fully representative for these tests. The root causes +behind poor results repeatability vary between the test cases. + +## Per Release Coverage Data + +Detailed result tables generated from CSIT test job executions. The coverage +tests include also tests which are not run in iterative performance builds. +The tables present NDR and PDR packet throughput (packets per second and bits +per second) and latency percentiles (microseconds) at different packet rate load +levels: +- No-Load, latency streams only, +- Low-Load at 10% PDR, +- Mid-Load at 50% PDR and +- High-Load at 90% PDR. + +## Test Job Statistics + +The elementary statistical data (number of passed and failed tests and the +duration) of all daily and weekly trending performance jobs. +In addition, the graphs show dynamic labels while hovering over graph data +points with detailed information. By clicking on the graph, user gets the job +summary with the list of failed tests. + +## Failures and Anomalies + +The presented tables list: +- last build summary, +- failed tests, +- progressions and +- regressions + +for all daily and weekly trending performance jobs. + +## Documentation + +This documentation describing the methodology, infrastructure and release notes +for each CSIT release. diff --git a/docs/content/overview/csit/_index.md b/docs/content/overview/csit/_index.md new file mode 100644 index 0000000000..167c872c0b --- /dev/null +++ b/docs/content/overview/csit/_index.md @@ -0,0 +1,45 @@ +--- +bookCollapseSection: true +bookFlatSection: false +title: "CSIT" +weight: 2 +--- + +# Continuous System Integration and Testing + +## CSIT Description + +1. Development of software code for fully automated VPP code testing, + functionality, performance, regression and new functions. +2. Execution of CSIT test suites on VPP code running on LF FD.io virtual and + physical compute environments. +3. Integration with FD.io continuous integration systems (Gerrit, Jenkins and + such). +4. Identified existing FD.io project dependencies and interactions: + - vpp - Vector Packet Processing. + - ci-management - Management repo for Jenkins Job Builder, script and + management related to the Jenkins CI configuration. + +## Project Scope + +1. Automated regression testing of VPP code changes + - Functionality of VPP data plane, network control plane, management plane + against functional specifications. + - Performance of VPP data plane including non-drop-rate packet throughput + and delay, against established reference benchmarks. + - Performance of network control plane against established reference + benchmarks. + - Performance of management plane against established reference benchmarks. +2. Test case definitions driven by supported and planned VPP functionality, + interfaces and performance: + - Uni-dimensional tests: Data plane, (Network) Control plane, Management + plane. + - Multi-dimensional tests: Use case driven. +3. Integration with FD.io Continuous Integration system including FD.io Gerrit + and Jenkins + - Automated test execution triggered by VPP-VERIFY jobs other VPP and CSIT + project jobs. +4. Integration with LF VPP test execution environment + - Functional tests execution on LF hosted VM environment. + - Performance and functional tests execution on LF hosted physical compute + environment. diff --git a/docs/content/overview/csit/branching_strategy.md b/docs/content/overview/csit/branching_strategy.md new file mode 100644 index 0000000000..16d8e0f471 --- /dev/null +++ b/docs/content/overview/csit/branching_strategy.md @@ -0,0 +1,109 @@ +--- +title: "Branching Strategy" +weight: 6 +--- + +# Branching Strategy + +## Definitions + +**CSIT development branch:** A CSIT branch used for test development which has a +1:1 association with a VPP branch of the same name. CSIT development branches +are never used for operational testing of VPP patches or images. + +**CSIT operational branch:** A CSIT branch pulled from a CSIT development or +release branch which is used for operational testing of the VPP branch +associated from its' parent branch. CSIT operational branches are named +`oper-<YYMMDD>` for master and `oper-<release>-<YYMMDD>` for release branches. +CSIT operational branches are the only branches which should be used to run +verify jobs against VPP patches or images. + +**CSIT release branch:** A CSIT branch which is pulled from a development branch +and is associated with a VPP release branch. CSIT release branches are never +merged back into their parent branch and are never used for operational testing +of VPP patches or images. + +## VPP Selection of CSIT Operational Branches + +Each VPP and release branch will have a script which specifies which CSIT +operational branch is used when executing the per patch verify jobs. This is +maintained in the VPP branch in the file +`.../vpp/build-root/scripts/csit-test-branch`. + +## Branches + +### Main development branch: 'master' + +The CSIT development branch 'master' will be the main development for new VPP +feature tests that have not been included in a release. Weekly CSIT operational +branches will be pulled from 'master'. After validation of all CSIT verify jobs, +the VPP script 'csit-test-branch' will be updated with the latest CSIT +operational branch name. Older CSIT operational branches will be available for +manual triggered vpp-csit-verify-* jobs. + +### Release branch: 'rls1606', 'rls1609', ... + +CSIT release branches shall be pulled from 'master' with the the convention +`rls<release>` (e.g. rls1606, rls1609). New tests that are developed for +existing VPP features will be committed into the 'master' branch, then +cherry-picked|double committed into the latest CSIT release branch. +Periodically CSIT operational branches will be pulled from the CSIT release +branch when necessary and the VPP release branch updated to use the new CSIT +operational branch. + +**VPP branch diagram:** + + -- master ---------------------------------------------------------------> + \ \ + \--- stable/1606 ---[end] \--- stable/1609---[end] + + +**CSIT branch diagram:** + + /--- oper-rls1606-160623 + / /--- oper-rls1606-$(DATE) + / / . . . + / / /--- oper-rls1609-$(DATE) + / / / . . . + /--- rls1606 ---[end] /--- rls1609 ---[end] + / / / / + / (cherry-picking) / (cherry-picking) + / / / / + -- master ---------------------------------------------------------------> + \ \ . . . + \ \--- oper-$(DATE) + \--- oper-160710 + +## Creating a CSIT Operational Branch + +### Run verify weekly job + +`csit-vpp-device-master-<OS>-<arch>-<testbed>-weekly` is run on the CSIT +development or release branch (e.g. 'master' or 'stable/1606') using the latest +VPP package set on nexus.fd.io for the associated VPP branch. Any anomalies will +have the root cause identified and be resolved in the CSIT development branch +prior to pulling the CSIT operational branch. + +### Pull CSIT operational branch from parent + +The CSIT operational branch is pulled from the parent CSIT development or +release branch. + +### Run verify semiweekly job + +`csit-vpp-device-master-<OS>-<arch>-<testbed>-semiweekly` is run on the CSIT +operational branch with the latest image of the associated VPP development or +release branch. This job is run to validate the next reference VPP build for +validating the results of all of the csit-vpp-verify* jobs. + +### Update VPP branch to use the new CSIT operational branch + +Push a patch updating the VPP branch to use the new CSIT operational branch. The +VPP verify jobs will then be run and any anomalies will have the root cause +identified and fixed in the CSIT operational branch prior to 'csit-test-branch' +being merged. + +### Periodically lock/deprecate old CSIT Operational Branches + +Periodically old CSIT operational branches will be locked and/or deprecated to +prevent changes being made to the operational branch. diff --git a/docs/content/overview/csit/design.md b/docs/content/overview/csit/design.md new file mode 100644 index 0000000000..f43d91a28e --- /dev/null +++ b/docs/content/overview/csit/design.md @@ -0,0 +1,148 @@ +--- +title: "Design" +weight: 1 +--- + +# Design + +FD.io CSIT system design needs to meet continuously expanding requirements of +FD.io projects including VPP, related sub-systems (e.g. plugin applications, +DPDK drivers) and FD.io applications (e.g. DPDK applications), as well as +growing number of compute platforms running those applications. With CSIT +project scope and charter including both FD.io continuous testing AND +performance trending/comparisons, those evolving requirements further amplify +the need for CSIT framework modularity, flexibility and usability. + +## Design Hierarchy + +CSIT follows a hierarchical system design with SUTs and DUTs at the bottom level +of the hierarchy, presentation level at the top level and a number of functional +layers in-between. The current CSIT system design including CSIT framework is +depicted in the figure below. + +{{< figure src="/cdocs/csit_design_picture.svg" title="CSIT Design" >}} + +A brief bottom-up description is provided here: + +1. SUTs, DUTs, TGs + - SUTs - Systems Under Test; + - DUTs - Devices Under Test; + - TGs - Traffic Generators; +2. Level-1 libraries - Robot and Python + - Lowest level CSIT libraries abstracting underlying test environment, SUT, + DUT and TG specifics; + - Used commonly across multiple L2 KWs; + - Performance and functional tests: + - L1 KWs (KeyWords) are implemented as RF libraries and Python + libraries; + - Performance TG L1 KWs: + - All L1 KWs are implemented as Python libraries: + - Support for TRex only today; + - CSIT IXIA drivers in progress; + - Performance data plane traffic profiles: + - TG-specific stream profiles provide full control of: + - Packet definition - layers, MACs, IPs, ports, combinations thereof + e.g. IPs and UDP ports; + - Stream definitions - different streams can run together, delayed, + one after each other; + - Stream profiles are independent of CSIT framework and can be used + in any T-rex setup, can be sent anywhere to repeat tests with + exactly the same setup; + - Easily extensible - one can create a new stream profile that meets + tests requirements; + - Same stream profile can be used for different tests with the same + traffic needs; + - Functional data plane traffic scripts: + - Scapy specific traffic scripts; +3. Level-2 libraries - Robot resource files: + - Higher level CSIT libraries abstracting required functions for executing + tests; + - L2 KWs are classified into the following functional categories: + - Configuration, test, verification, state report; + - Suite setup, suite teardown; + - Test setup, test teardown; +4. Tests - Robot: + - Test suites with test cases; + - Performance tests using physical testbed environment: + - VPP; + - DPDK-Testpmd; + - DPDK-L3Fwd; + - TRex + - Tools: + - CSIT-Dash + - Testbed environment setup ansible playbooks; + - Operational debugging scripts; + +5. Test Lifecycle Abstraction + +A well coded test must follow a disciplined abstraction of the test +lifecycles that includes setup, configuration, test and verification. In +addition to improve test execution efficiency, the commmon aspects of +test setup and configuration shared across multiple test cases should be +done only once. Translating these high-level guidelines into the Robot +Framework one arrives to definition of a well coded RF tests for FD.io +CSIT. Anatomy of Good Tests for CSIT: + +1. Suite Setup - Suite startup Configuration common to all Test Cases in suite: + uses Configuration KWs, Verification KWs, StateReport KWs; +2. Test Setup - Test startup Configuration common to multiple Test Cases: uses + Configuration KWs, StateReport KWs; +3. Test Case - uses L2 KWs with RF Gherkin style: + - prefixed with {Given} - Verification of Test setup, reading state: uses + Configuration KWs, Verification KWs, StateReport KWs; + - prefixed with {When} - Test execution: Configuration KWs, Test KWs; + - prefixed with {Then} - Verification of Test execution, reading state: uses + Verification KWs, StateReport KWs; +4. Test Teardown - post Test teardown with Configuration cleanup and + Verification common to multiple Test Cases - uses: Configuration KWs, + Verification KWs, StateReport KWs; +5. Suite Teardown - Suite post-test Configuration cleanup: uses Configuration + KWs, Verification KWs, StateReport KWs; + +## RF Keywords Functional Classification + +CSIT RF KWs are classified into the functional categories matching the test +lifecycle events described earlier. All CSIT RF L2 and L1 KWs have been grouped +into the following functional categories: + +1. Configuration; +2. Test; +3. Verification; +4. StateReport; +5. SuiteSetup; +6. TestSetup; +7. SuiteTeardown; +8. TestTeardown; + +## RF Keywords Naming Guidelines + +Readability counts: "..code is read much more often than it is written." +Hence following a good and consistent grammar practice is important when +writing Robot Framework KeyWords and Tests. All CSIT test cases +are coded using Gherkin style and include only L2 KWs references. L2 KWs are +coded using simple style and include L2 KWs, L1 KWs, and L1 python references. +To improve readability, the proposal is to use the same grammar for both +Robot Framework KW styles, and to formalize the grammar of English +sentences used for naming the Robot Framework KWs. Robot +Framework KWs names are short sentences expressing functional description of +the command. They must follow English sentence grammar in one of the following +forms: + +1. **Imperative** - verb-object(s): *"Do something"*, verb in base form. +2. **Declarative** - subject-verb-object(s): *"Subject does something"*, verb in + a third-person singular present tense form. +3. **Affirmative** - modal_verb-verb-object(s): *"Subject should be something"*, + *"Object should exist"*, verb in base form. +4. **Negative** - modal_verb-Not-verb-object(s): *"Subject should not be + something"*, *"Object should not exist"*, verb in base form. + +Passive form MUST NOT be used. However a usage of past participle as an +adjective is okay. See usage examples provided in the Coding guidelines +section below. Following sections list applicability of the above +grammar forms to different Robot Framework KW categories. Usage +examples are provided, both good and bad. + +## Coding Guidelines + +Coding guidelines can be found on +[Design optimizations wiki page](https://wiki.fd.io/view/CSIT/Design_Optimizations). diff --git a/docs/content/overview/csit/suite_generation.md b/docs/content/overview/csit/suite_generation.md new file mode 100644 index 0000000000..84a19b8ab9 --- /dev/null +++ b/docs/content/overview/csit/suite_generation.md @@ -0,0 +1,123 @@ +--- +title: "Suite Generation" +weight: 5 +--- + +# Suite Generation + +CSIT uses robot suite files to define tests. However, not all suite files +available for Jenkins jobs (or manually started bootstrap scripts) are present +in CSIT git repository. They are generated only when needed. + +## Autogen Library + +There is a code generation layer implemented as Python library called "autogen", +called by various bash scripts. + +It generates the full extent of CSIT suites, using the ones in git as templates. + +## Sources + +The generated suites (and their contents) are affected by multiple information +sources, listed below. + +### Git Suites + +The suites present in git repository act as templates for generating suites. +One of autogen design principles is that any template suite should also act +as a full suite (no placeholders). + +In practice, autogen always re-creates the template suite with exactly +the same content, it is one of checks that autogen works correctly. + +### Regenerate Script + +Not all suites present in CSIT git repository act as template for autogen. +The distinction is on per-directory level. Directories with +`regenerate_testcases.py` script usually consider all suites as templates +(unless possibly not included by the glob patten in the script). + +The script also specifies minimal frame size, indirectly, by specifying protocol +(protocol "ip4" is the default, leading to 64B frame size). + +### Constants + +Values in `Constants.py` are taken into consideration when generating suites. +The values are mostly related to different NIC models and NIC drivers. + +### Python Code + +Python code in `resources/libraries/python/autogen` contains several other +information sources. + +#### Testcase Templates + +The test case part of template suite is ignored, test case lines +are created according to text templates in `Testcase.py` file. + +#### Testcase Argument Lists + +Each testcase template has different number of "arguments", e.g. values +to put into various placeholders. Different test types need different +lists of the argument values, the lists are in `regenerate_glob` method +in `Regenerator.py` file. + +#### Iteration Over Values + +Python code detects the test type (usually by substrings of suite file name), +then iterates over different quantities based on type. +For example, only ndrpdr suite templates generate other types (mrr and soak). + +#### Hardcoded Exclusions + +Some combinations of values are known not to work, so they are excluded. +Examples: Density tests for too much CPUs; IMIX for ASTF. + +## Non-Sources + +Some information sources are available in CSIT repository, +but do not affect the suites generated by autogen. + +### Testbeds + +Overall, no information visible in topology yaml files is taken into account +by autogen. + +#### Testbed Architecture + +Historically, suite files are agnostic to testbed architecture, e.g. ICX or ALT. + +#### Testbed Size + +Historically, 2-node and 3-node suites have diferent names, and while +most of the code is common, the differences are not always simple enough. +Autogen treat 2-node and 3-node suites as independent templates. + +TRex suites are intended for a 1-node circuit of otherwise 2-node or 3-node +testbeds, so they support all 3 robot tags. +They are also detected and treated differently by autogen, +mainly because they need different testcase arguments (no CPU count). +Autogen does nothing specifically related to the fact they should run +only in testbeds/NICs with TG-TG line available. + +#### Other Topology Info + +Some bonding tests need two (parallel) links between DUTs. Autogen does not +care, as suites are agnostic. Robot tag marks the difference, but the link +presence is not explicitly checked. + +### Job specs + +Information in job spec files depend on generated suites (not the other way). +Autogen should generate more suites, as job spec is limited by time budget. +More suites should be available for manually triggered verify jobs, +so autogen covers that. + +### Bootstrap Scripts + +Historically, bootstrap scripts perform some logic, +perhaps adding exclusion options to Robot invocation +(e.g. skipping testbed+NIC combinations for tests that need parallel links). + +Once again, the logic here relies on what autogen generates, +autogen does not look into bootstrap scripts. diff --git a/docs/content/overview/csit/test_naming.md b/docs/content/overview/csit/test_naming.md new file mode 100644 index 0000000000..d7a32518e5 --- /dev/null +++ b/docs/content/overview/csit/test_naming.md @@ -0,0 +1,112 @@ +--- +title: "Test Naming" +weight: 3 +--- + +# Test Naming + +## Background + +{{< release_csit >}} follows a common structured naming convention for all +performance and system functional tests, introduced in CSIT 17.01. + +The naming should be intuitive for majority of the tests. Complete +description of CSIT test naming convention is provided on +[CSIT test naming wiki page](https://wiki.fd.io/view/CSIT/csit-test-naming). +Below few illustrative examples of the naming usage for test suites across CSIT +performance, functional and Honeycomb management test areas. + +## Naming Convention + +The CSIT approach is to use tree naming convention and to encode following +testing information into test suite and test case names: + +1. packet network port configuration + * port type, physical or virtual; + * number of ports; + * NIC model, if applicable; + * port-NIC locality, if applicable; +2. packet encapsulations; +3. VPP packet processing + * packet forwarding mode; + * packet processing function(s); +4. packet forwarding path + * if present, network functions (processes, containers, VMs) and their + topology within the computer; +5. main measured variable, type of test. + +Proposed convention is to encode ports and NICs on the left (underlay), +followed by outer-most frame header, then other stacked headers up to the +header processed by vSwitch-VPP, then VPP forwarding function, then encap on +vhost interface, number of vhost interfaces, number of VMs. If chained VMs +present, they get added on the right. Test topology is expected to be +symmetric, in other words packets enter and leave SUT through ports specified +on the left of the test name. Here some examples to illustrate the convention +followed by the complete legend, and tables mapping the new test filenames to +old ones. + +## Naming Examples + +CSIT test suite naming examples (filename.robot) for common tested VPP +topologies: + +1. **Physical port to physical port - a.k.a. NIC-to-NIC, Phy-to-Phy, P2P** + * *PortNICConfig-WireEncapsulation-PacketForwardingFunction- + PacketProcessingFunction1-...-PacketProcessingFunctionN-TestType* + * *10ge2p1x520-dot1q-l2bdbasemaclrn-ndrdisc.robot* => 2 ports of 10GE on + Intel x520 NIC, dot1q tagged Ethernet, L2 bridge-domain baseline switching + with MAC learning, NDR throughput discovery. + * *10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrchk.robot* => 2 ports of 10GE on + Intel x520 NIC, IPv4 VXLAN Ethernet, L2 bridge-domain baseline switching + with MAC learning, NDR throughput discovery. + * *10ge2p1x520-ethip4-ip4base-ndrdisc.robot* => 2 ports of 10GE on Intel x520 + NIC, IPv4 baseline routed forwarding, NDR throughput discovery. + * *10ge2p1x520-ethip6-ip6scale200k-ndrdisc.robot* => 2 ports of 10GE on Intel + x520 NIC, IPv6 scaled up routed forwarding, NDR throughput discovery. + * *10ge2p1x520-ethip4-ip4base-iacldstbase-ndrdisc.robot* => 2 ports of 10GE + on Intel x520 NIC, IPv4 baseline routed forwarding, ingress Access Control + Lists baseline matching on destination, NDR throughput discovery. + * *40ge2p1vic1385-ethip4-ip4base-ndrdisc.robot* => 2 ports of 40GE on Cisco + vic1385 NIC, IPv4 baseline routed forwarding, NDR throughput discovery. + * *eth2p-ethip4-ip4base-func.robot* => 2 ports of Ethernet, IPv4 baseline + routed forwarding, functional tests. + +2. **Physical port to VM (or VM chain) to physical port - a.k.a. NIC2VM2NIC, + P2V2P, NIC2VMchain2NIC, P2V2V2P** + * *PortNICConfig-WireEncapsulation-PacketForwardingFunction- + PacketProcessingFunction1-...-PacketProcessingFunctionN-VirtEncapsulation- + VirtPortConfig-VMconfig-TestType* + * *10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot* => 2 ports + of 10GE on Intel x520 NIC, dot1q tagged Ethernet, L2 bridge-domain + switching to/from two vhost interfaces and one VM, NDR throughput + discovery. + * *10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot* => 2 + ports of 10GE on Intel x520 NIC, IPv4 VXLAN Ethernet, L2 bridge-domain + switching to/from two vhost interfaces and one VM, NDR throughput + discovery. + * *10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-4vhost-2vm-ndrdisc.robot* => 2 + ports of 10GE on Intel x520 NIC, IPv4 VXLAN Ethernet, L2 bridge-domain + switching to/from four vhost interfaces and two VMs, NDR throughput + discovery. + * *eth2p-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-func.robot* => 2 ports of + Ethernet, IPv4 VXLAN Ethernet, L2 bridge-domain switching to/from two vhost + interfaces and one VM, functional tests. + +3. **API CRUD tests - Create (Write), Read (Retrieve), Update (Modify), Delete + (Destroy) operations for configuration and operational data** + * *ManagementTestKeyword-ManagementOperation-ManagedFunction1-...- + ManagedFunctionN-ManagementAPI1-ManagementAPIN-TestType* + * *mgmt-cfg-lisp-apivat-func* => configuration of LISP with VAT API calls, + functional tests. + * *mgmt-cfg-l2bd-apihc-apivat-func* => configuration of L2 Bridge-Domain with + Honeycomb API and VAT API calls, functional tests. + * *mgmt-oper-int-apihcnc-func* => reading status and operational data of + interface with Honeycomb NetConf API calls, functional tests. + * *mgmt-cfg-int-tap-apihcnc-func* => configuration of tap interfaces with + Honeycomb NetConf API calls, functional tests. + * *mgmt-notif-int-subint-apihcnc-func* => notifications of interface and + sub-interface events with Honeycomb NetConf Notifications, functional + tests. + +For complete description of CSIT test naming convention please refer to +[CSIT test naming wiki page](https://wiki.fd.io/view/CSIT/csit-test-naming). diff --git a/docs/content/overview/csit/test_scenarios.md b/docs/content/overview/csit/test_scenarios.md new file mode 100644 index 0000000000..1f06765eae --- /dev/null +++ b/docs/content/overview/csit/test_scenarios.md @@ -0,0 +1,66 @@ +--- +title: "Test Scenarios" +weight: 2 +--- + +# Test Scenarios + +FD.io CSIT Dashboard includes multiple test scenarios of VPP +centric applications, topologies and use cases. In addition it also +covers baseline tests of DPDK sample applications. Tests are executed in +physical (performance tests) and virtual environments (functional +tests). + +Brief overview of test scenarios covered in this documentation: + +1. **VPP Performance**: VPP performance tests are executed in physical + FD.io testbeds, focusing on VPP network data plane performance in + NIC-to-NIC switching topologies. VPP application runs in + bare-metal host user-mode handling NICs. TRex is used as a traffic generator. + +2. **VPP Vhostuser Performance with KVM VMs**: VPP VM service switching + performance tests using vhostuser virtual interface for + interconnecting multiple NF-in-VM instances. VPP vswitch + instance runs in bare-metal user-mode handling NICs and connecting + over vhost-user interfaces to VM instances each running VPP with virtio + virtual interfaces. Similarly to VPP Performance, tests are run across a + range of configurations. TRex is used as a traffic generator. + +3. **VPP Memif Performance with LXC and Docker Containers**: VPP + Container service switching performance tests using memif virtual + interface for interconnecting multiple VPP-in-container instances. + VPP vswitch instance runs in bare-metal user-mode handling NICs and + connecting over memif (Slave side) interfaces to more instances of + VPP running in LXC or in Docker Containers, both with memif + interfaces (Master side). Similarly to VPP Performance, tests are + run across a range of configurations. TRex is used as a traffic + generator. + +4. **DPDK Performance**: VPP uses DPDK to drive the NICs and physical + interfaces. DPDK performance tests are used as a baseline to + profile performance of the DPDK sub-system. Two DPDK applications + are tested: Testpmd and L3fwd. DPDK tests are executed in the same + testing environment as VPP tests. DPDK Testpmd and L3fwd + applications run in host user-mode. TRex is used as a traffic + generator. + +5. **T-Rex Performance**: T-Rex perfomance tests are executed in physical + FD.io testbeds, focusing on T-Rex data plane performance in NIC-to-NIC + loopback topologies. + +6. **VPP Functional**: VPP functional tests are executed in virtual + FD.io testbeds, focusing on VPP packet processing functionality, + including both network data plane and in-line control plane. Tests + cover vNIC-to-vNIC vNIC-to-nestedVM-to-vNIC forwarding topologies. + Scapy is used as a traffic generator. + +All CSIT test data included in this report is auto-generated from Robot +Framework json output files produced by Linux Foundation FD.io Jenkins jobs +executed against {{< release_vpp >}} artifacts. + +FD.io CSIT system is developed using two main coding platforms: Robot +Framework and Python. {{< release_csit >}} source code for the executed test +suites is available in corresponding CSIT branch in the directory +`./tests/<name_of_the_test_suite>`. A local copy of CSIT source code +can be obtained by cloning CSIT git repository - `git clone +https://gerrit.fd.io/r/csit`. diff --git a/docs/content/overview/csit/test_tags.md b/docs/content/overview/csit/test_tags.md new file mode 100644 index 0000000000..de38945c17 --- /dev/null +++ b/docs/content/overview/csit/test_tags.md @@ -0,0 +1,876 @@ +--- +title: "Test Tags" +weight: 4 +--- + +# Test Tags + +All CSIT test cases are labelled with Robot Framework tags used to allow for +easy test case type identification, test case grouping and selection for +execution. Following sections list currently used CSIT tags and their +descriptions. + +## Testbed Topology Tags + +**2_NODE_DOUBLE_LINK_TOPO** + + 2 nodes connected in a circular topology with two links interconnecting + the devices. + +**2_NODE_SINGLE_LINK_TOPO** + + 2 nodes connected in a circular topology with at least one link + interconnecting devices. + +**3_NODE_DOUBLE_LINK_TOPO** + + 3 nodes connected in a circular topology with two links interconnecting + the devices. + +**3_NODE_SINGLE_LINK_TOPO** + + 3 nodes connected in a circular topology with at least one link + interconnecting devices. + +## Objective Tags + +**SKIP_PATCH** + + Test case(s) marked to not run in case of vpp-csit-verify (i.e. VPP + patch) and csit-vpp-verify jobs (i.e. CSIT patch). + +**SKIP_VPP_PATCH** + + Test case(s) marked to not run in case of vpp-csit-verify (i.e. VPP + patch). + +## Environment Tags + +**HW_ENV** + + DUTs and TGs are running on bare metal. + +**VM_ENV** + + DUTs and TGs are running in virtual environment. + +**VPP_VM_ENV** + + DUTs with VPP and capable of running Virtual Machine. + +## NIC Model Tags + +**NIC_Intel-X520-DA2** + + Intel X520-DA2 NIC. + +**NIC_Intel-XL710** + + Intel XL710 NIC. + +**NIC_Intel-X710** + + Intel X710 NIC. + +**NIC_Intel-XXV710** + + Intel XXV710 NIC. + +**NIC_Cisco-VIC-1227** + + VIC-1227 by Cisco. + +**NIC_Cisco-VIC-1385** + + VIC-1385 by Cisco. + +**NIC_Amazon-Nitro-50G** + + Amazon EC2 ENA NIC. + +## Scaling Tags + +**FIB_20K** + + 2x10,000 entries in single fib table + +**FIB_200K** + + 2x100,000 entries in single fib table + +**FIB_1M** + + 2x500,000 entries in single fib table + +**FIB_2M** + + 2x1,000,000 entries in single fib table + +**L2BD_1** + + Test with 1 L2 bridge domain. + +**L2BD_10** + + Test with 10 L2 bridge domains. + +**L2BD_100** + + Test with 100 L2 bridge domains. + +**L2BD_1K** + + Test with 1000 L2 bridge domains. + +**VLAN_1** + + Test with 1 VLAN sub-interface. + +**VLAN_10** + + Test with 10 VLAN sub-interfaces. + +**VLAN_100** + + Test with 100 VLAN sub-interfaces. + +**VLAN_1K** + + Test with 1000 VLAN sub-interfaces. + +**VXLAN_1** + + Test with 1 VXLAN tunnel. + +**VXLAN_10** + + Test with 10 VXLAN tunnels. + +**VXLAN_100** + + Test with 100 VXLAN tunnels. + +**VXLAN_1K** + + Test with 1000 VXLAN tunnels. + +**TNL_{t}** + + IPSec in tunnel mode - {t} tunnels. + +**SRC_USER_{u}** + + Traffic flow with {u} unique IPs (users) in one direction. + {u}=(1,10,100,1000,2000,4000). + +**100_FLOWS** + + Traffic stream with 100 unique flows (10 IPs/users x 10 UDP ports) in + one direction. + +**10k_FLOWS** + + Traffic stream with 10 000 unique flows (10 IPs/users x 1000 UDP ports) + in one direction. + +**100k_FLOWS** + + Traffic stream with 100 000 unique flows (100 IPs/users x 1000 UDP + ports) in one direction. + +**HOSTS_{h}** + + Stateless or stateful traffic stream with {h} client source IP4 + addresses, usually with 63 flow differing in source port number. + Could be UDP or TCP. If NAT is used, the clients are inside. + Outside IP range can differ. + {h}=(1024,4096,16384,65536,262144). + +**GENEVE4_{t}TUN** + + Test with {t} GENEVE IPv4 tunnel. + {t}=(1,4,16,64,256,1024) + +## Test Category Tags + +**DEVICETEST** + + All vpp_device functional test cases. + +**PERFTEST** + + All performance test cases. + +## VPP Device Type Tags + +**SCAPY** + + All test cases that uses Scapy for packet generation and validation. + +## Performance Type Tags + +**NDRPDR** + + Single test finding both No Drop Rate and Partial Drop Rate + simultaneously. The search is done by optimized algorithm which + performs multiple trial runs at different durations and transmit + rates. The results come from the final trials, which have duration + of 30 seconds. + +**MRR** + + Performance tests where TG sends the traffic at maximum rate (line rate) + and reports total sent/received packets over trial duration. + The result is an average of 10 trials of 1 second duration. + +**SOAK** + + Performance tests using PLRsearch to find the critical load. + +**RECONF** + + Performance tests aimed to measure lost packets (time) when performing + reconfiguration while full throughput offered load is applied. + +## Ethernet Frame Size Tags + +These are describing the traffic offered by Traffic Generator, +"primary" traffic in case of asymmetric load. +For traffic between DUTs, or for "secondary" traffic, see ${overhead} value. + +**{b}B** + + {b} Bytes frames used for test. + +**IMIX** + + IMIX frame sequence (28x 64B, 16x 570B, 4x 1518B) used for test. + +## Test Type Tags + +**BASE** + + Baseline test cases, no encapsulation, no feature(s) configured in tests. + No scaling whatsoever, beyond minimum needed for RSS. + +**IP4BASE** + + IPv4 baseline test cases, no encapsulation, no feature(s) configured in + tests. Minimal number of routes. Other quantities may be scaled. + +**IP6BASE** + + IPv6 baseline test cases, no encapsulation, no feature(s) configured in + tests. + +**L2XCBASE** + + L2XC baseline test cases, no encapsulation, no feature(s) configured in + tests. + +**L2BDBASE** + + L2BD baseline test cases, no encapsulation, no feature(s) configured in + tests. + +**L2PATCH** + + L2PATCH baseline test cases, no encapsulation, no feature(s) configured + in tests. + +**SCALE** + + Scale test cases. Other tags specify which quantities are scaled. + Also applies if scaling is set on TG only (e.g. DUT works as IP4BASE). + +**ENCAP** + + Test cases where encapsulation is used. Use also encapsulation tag(s). + +**FEATURE** + + At least one feature is configured in test cases. Use also feature + tag(s). + +**UDP** + + Tests which use any kind of UDP traffic (STL or ASTF profile). + +**TCP** + + Tests which use any kind of TCP traffic (STL or ASTF profile). + +**TREX** + + Tests which test trex traffic without any software DUTs in the + traffic path. + +**UDP_UDIR** + + Tests which use unidirectional UDP traffic (STL profile only). + +**UDP_BIDIR** + + Tests which use bidirectional UDP traffic (STL profile only). + +**UDP_CPS** + + Tests which measure connections per second on minimal UDP + pseudoconnections. This implies ASTF traffic profile is used. + +**TCP_CPS** + + Tests which measure connections per second on empty TCP connections. + This implies ASTF traffic profile is used. + +**TCP_RPS** + + Tests which measure requests per second on empty TCP connections. + This implies ASTF traffic profile is used. + +**UDP_PPS** + + Tests which measure packets per second on lightweight UDP transactions. + This implies ASTF traffic profile is used. + +**TCP_PPS** + + Tests which measure packets per second on lightweight TCP transactions. + This implies ASTF traffic profile is used. + +**HTTP** + + Tests which use traffic formed of valid HTTP requests (and responses). + +**LDP_NGINX** + + LDP NGINX is un-modified NGINX with VPP via LD_PRELOAD. + +**NF_DENSITY** + + Performance tests that measure throughput of multiple VNF and CNF + service topologies at different service densities. + +## NF Service Density Tags + +**CHAIN** + + NF service density tests with VNF or CNF service chain topology(ies). + +**PIPE** + + NF service density tests with CNF service pipeline topology(ies). + +**NF_L3FWDIP4** + + NF service density tests with DPDK l3fwd IPv4 routing as NF workload. + +**NF_VPPIP4** + + NF service density tests with VPP IPv4 routing as NF workload. + +**{r}R{c}C** + + Service density matrix locator {r}R{c}C, {r}Row denoting number of + service instances, {c}Column denoting number of NFs per service + instance. + {r}=(1,2,4,6,8,10), {c}=(1,2,4,6,8,10). + +**{n}VM{t}T** + + Service density {n}VM{t}T, {n}Number of NF Qemu VMs, {t}Number of + threads per NF. + +**{n}DCR{t}T** + + Service density {n}DCR{t}T, {n}Number of NF Docker containers, + {t}Number of threads per NF. + +**{n}_ADDED_CHAINS** + + {n}Number of chains (or pipelines) added (and/or removed) + during RECONF test. + +## Forwarding Mode Tags + +**L2BDMACSTAT** + + VPP L2 bridge-domain, L2 MAC static. + +**L2BDMACLRN** + + VPP L2 bridge-domain, L2 MAC learning. + +**L2XCFWD** + + VPP L2 point-to-point cross-connect. + +**IP4FWD** + + VPP IPv4 routed forwarding. + +**IP6FWD** + + VPP IPv6 routed forwarding. + +**LOADBALANCER_MAGLEV** + + VPP Load balancer maglev mode. + +**LOADBALANCER_L3DSR** + + VPP Load balancer l3dsr mode. + +**LOADBALANCER_NAT4** + + VPP Load balancer nat4 mode. + +**N2N** + + Mode, where NICs from the same physical server are directly + connected with a cable. + +## Underlay Tags + +**IP4UNRLAY** + + IPv4 underlay. + +**IP6UNRLAY** + + IPv6 underlay. + +**MPLSUNRLAY** + + MPLS underlay. + +## Overlay Tags + +**L2OVRLAY** + + L2 overlay. + +**IP4OVRLAY** + + IPv4 overlay (IPv4 payload). + +**IP6OVRLAY** + + IPv6 overlay (IPv6 payload). + +## Tagging Tags + +**DOT1Q** + + All test cases with dot1q. + +**DOT1AD** + + All test cases with dot1ad. + +## Encapsulation Tags + +**ETH** + + All test cases with base Ethernet (no encapsulation). + +**LISP** + + All test cases with LISP. + +**LISPGPE** + + All test cases with LISP-GPE. + +**LISP_IP4o4** + + All test cases with LISP_IP4o4. + +**LISPGPE_IP4o4** + + All test cases with LISPGPE_IP4o4. + +**LISPGPE_IP6o4** + + All test cases with LISPGPE_IP6o4. + +**LISPGPE_IP4o6** + + All test cases with LISPGPE_IP4o6. + +**LISPGPE_IP6o6** + + All test cases with LISPGPE_IP6o6. + +**VXLAN** + + All test cases with Vxlan. + +**VXLANGPE** + + All test cases with VXLAN-GPE. + +**GRE** + + All test cases with GRE. + +**GTPU** + + All test cases with GTPU. + +**GTPU_HWACCEL** + + All test cases with GTPU_HWACCEL. + +**IPSEC** + + All test cases with IPSEC. + +**WIREGUARD** + + All test cases with WIREGUARD. + +**SRv6** + + All test cases with Segment routing over IPv6 dataplane. + +**SRv6_1SID** + + All SRv6 test cases with single SID. + +**SRv6_2SID_DECAP** + + All SRv6 test cases with two SIDs and with decapsulation. + +**SRv6_2SID_NODECAP** + + All SRv6 test cases with two SIDs and without decapsulation. + +**GENEVE** + + All test cases with GENEVE. + +**GENEVE_L3MODE** + + All test cases with GENEVE tunnel in L3 mode. + +**FLOW** + + All test cases with FLOW. + +**FLOW_DIR** + + All test cases with FLOW_DIR. + +**FLOW_RSS** + + All test cases with FLOW_RSS. + +**NTUPLE** + + All test cases with NTUPLE. + +**L2TPV3** + + All test cases with L2TPV3. + +**REASSEMBLY** + + All encap/decap tests where MTU induces IP fragmentation and reassembly. + +## Interface Tags + +**PHY** + + All test cases which use physical interface(s). + +**GSO** + + All test cases which uses Generic Segmentation Offload. + +**VHOST** + + All test cases which uses VHOST. + +**VHOST_1024** + + All test cases which uses VHOST DPDK driver with qemu queue size set + to 1024. + +**VIRTIO** + + All test cases which uses VIRTIO native VPP driver. + +**VIRTIO_1024** + + All test cases which uses VIRTIO native VPP driver with qemu queue + size set to 1024. + +**CFS_OPT** + + All test cases which uses VM with optimised scheduler policy. + +**TUNTAP** + + All test cases which uses TUN and TAP. + +**AFPKT** + + All test cases which uses AFPKT. + +**NETMAP** + + All test cases which uses Netmap. + +**MEMIF** + + All test cases which uses Memif. + +**SINGLE_MEMIF** + + All test cases which uses only single Memif connection per DUT. One DUT + instance is running in container having one physical interface exposed + to container. + +**LBOND** + + All test cases which uses link bonding (BondEthernet interface). + +**LBOND_DPDK** + + All test cases which uses DPDK link bonding. + +**LBOND_VPP** + + All test cases which uses VPP link bonding. + +**LBOND_MODE_XOR** + + All test cases which uses link bonding with mode XOR. + +**LBOND_MODE_LACP** + + All test cases which uses link bonding with mode LACP. + +**LBOND_LB_L34** + + All test cases which uses link bonding with load-balance mode l34. + +**LBOND_{n}L** + + All test cases which use {n} link(s) for link bonding. + +**DRV_{d}** + + All test cases which NIC Driver for DUT is set to {d}. + Default is VFIO_PCI. + {d}=(AVF, RDMA_CORE, VFIO_PCI, AF_XDP). + +**TG_DRV_{d}** + + All test cases which NIC Driver for TG is set to {d}. + Default is IGB_UIO. + {d}=(RDMA_CORE, IGB_UIO). + +**RXQ_SIZE_{n}** + + All test cases which RXQ size (RX descriptors) are set to {n}. + Default is 0, which means VPP (API) default. + +**TXQ_SIZE_{n}** + + All test cases which TXQ size (TX descriptors) are set to {n}. + Default is 0, which means VPP (API) default. + +## Feature Tags + +**IACLDST** + + iACL destination. + +**ADLALWLIST** + + ADL allowlist. + +**NAT44** + + NAT44 configured and tested. + +**NAT64** + + NAT44 configured and tested. + +**ACL** + + ACL plugin configured and tested. + +**IACL** + + ACL plugin configured and tested on input path. + +**OACL** + + ACL plugin configured and tested on output path. + +**ACL_STATELESS** + + ACL plugin configured and tested in stateless mode + (permit action). + +**ACL_STATEFUL** + + ACL plugin configured and tested in stateful mode + (permit+reflect action). + +**ACL1** + + ACL plugin configured and tested with 1 not-hitting ACE. + +**ACL10** + + ACL plugin configured and tested with 10 not-hitting ACEs. + +**ACL50** + + ACL plugin configured and tested with 50 not-hitting ACEs. + +**SRv6_PROXY** + + SRv6 endpoint to SR-unaware appliance via proxy. + +**SRv6_PROXY_STAT** + + SRv6 endpoint to SR-unaware appliance via static proxy. + +**SRv6_PROXY_DYN** + + SRv6 endpoint to SR-unaware appliance via dynamic proxy. + +**SRv6_PROXY_MASQ** + + SRv6 endpoint to SR-unaware appliance via masquerading proxy. + +## Encryption Tags + +**IPSECSW** + + Crypto in software. + +**IPSECHW** + + Crypto in hardware. + +**IPSECTRAN** + + IPSec in transport mode. + +**IPSECTUN** + + IPSec in tunnel mode. + +**IPSECINT** + + IPSec in interface mode. + +**AES** + + IPSec using AES algorithms. + +**AES_128_CBC** + + IPSec using AES 128 CBC algorithms. + +**AES_128_GCM** + + IPSec using AES 128 GCM algorithms. + +**AES_256_GCM** + + IPSec using AES 256 GCM algorithms. + +**HMAC** + + IPSec using HMAC integrity algorithms. + +**HMAC_SHA_256** + + IPSec using HMAC SHA 256 integrity algorithms. + +**HMAC_SHA_512** + + IPSec using HMAC SHA 512 integrity algorithms. + +**SCHEDULER** + + IPSec using crypto sw scheduler engine. + +**FASTPATH** + + IPSec policy mode with spd fast path enabled. + +## Client-Workload Tags + +**VM** + + All test cases which use at least one virtual machine. + +**LXC** + + All test cases which use Linux container and LXC utils. + +**DRC** + + All test cases which use at least one Docker container. + +**DOCKER** + + All test cases which use Docker as container manager. + +**APP** + + All test cases with specific APP use. + +## Container Orchestration Tags + +**{n}VSWITCH** + + {n} VPP running in {n} Docker container(s) acting as a VSWITCH. + {n}=(1). + +**{n}VNF** + + {n} VPP running in {n} Docker container(s) acting as a VNF work load. + {n}=(1). + +## Multi-Threading Tags + +**STHREAD** + + Dynamic tag. + All test cases using single poll mode thread. + +**MTHREAD** + + Dynamic tag. + All test cases using more then one poll mode driver thread. + +**{n}NUMA** + + All test cases with packet processing on {n} socket(s). {n}=(1,2). + +**{c}C** + + {c} worker thread pinned to {c} dedicated physical core; or if + HyperThreading is enabled, {c}*2 worker threads each pinned to + a separate logical core within 1 dedicated physical core. Main + thread pinned to core 1. + {t}=(1,2,4). + +**{t}T{c}C** + + *Dynamic tag*. + {t} worker threads pinned to {c} dedicated physical cores. Main thread + pinned to core 1. By default CSIT is configuring same amount of receive + queues per interface as worker threads. + {t}=(1,2,4,8), + {c}=(1,2,4). diff --git a/docs/content/release_notes/_index.md b/docs/content/release_notes/_index.md new file mode 100644 index 0000000000..3a8318d09f --- /dev/null +++ b/docs/content/release_notes/_index.md @@ -0,0 +1,6 @@ +--- +bookCollapseSection: false +bookFlatSection: true +title: "Release Notes" +weight: 3 +--- diff --git a/docs/content/release_notes/current/_index.md b/docs/content/release_notes/current/_index.md new file mode 100644 index 0000000000..6e1512d0ee --- /dev/null +++ b/docs/content/release_notes/current/_index.md @@ -0,0 +1,121 @@ +--- +bookCollapseSection: true +bookFlatSection: false +title: "CSIT rls2402" +weight: 1 +--- + +# CSIT-2402 Release Report + +This section will include release notes for FD.io CSIT-2402. The CSIT report +will be published on **Mar-13 2024**. The release plan is published on +[CSIT wiki](https://wiki.fd.io/view/CSIT/csit2402_plan) pages. + +The release notes of the previous CSIT release can be found +[here]({{< relref "../previous/csit_rls2310" >}}). + +## CSIT-2402 Release Notes + +- [VPP Performance]({{< relref "vpp_performance" >}}) +- [DPDK Performance]({{< relref "dpdk_performance" >}}) +- [TRex Performance]({{< relref "trex_performance" >}}) +- [VPP Device]({{< relref "vpp_device" >}}) + +## CSIT-2402 Release Data + +To access CSIT-2402 Release data please use following web resources: + +- [CSIT Per Release Performance](https://csit.fd.io/report/) + - `CSIT Release` > `rls2402` + - `DUT` > `vpp` + - `DUT Version` > `24.02-release` + - `Infra` > `testbed-nic-driver of choice` + - `Area` > `IPv4 Routing` `IPv4 Tunnels` `IPv6 Routing` `Hoststack` ... + - `Test` > `test of chioce` + - `Frame Size` > `64B` `78B` + - `Number of Cores` > `1C` `2C` `4C` + - `Test Type` > `MRR` `NDR` `PDR` +- [CSIT Per Release Comparisons](https://csit.fd.io/comparisons/) for VPP + v24.02 vs v23.10 + - `REFERENCE VALUE` + - `DUT` > `vpp` + - `CSIT and DUT version` > `rls2402-23.10-release` + - `Infra` > `testbed-nic-driver of choice` + - `Frame Size` > `64B` `78B` + - `Number of Cores` > `1C` `2C` `4C` + - `Measurement` > `Latency` `MRR` `NDR` `PDR` + - `COMPARED VALUE` + - `Parameter` > `Release and Version` + - `Value` > `rls2402-24.02-release` +- [CSIT Per Release Coverage Data](https://csit.fd.io/coverage/) + - `CSIT Release` > `rls2402` +- [CSIT Search Tests](https://csit.fd.io/search/) + - `Data Type` > `iterative` + - `DUT` > `vpp` + - `Release` > `rls2402` + - `Type a Regular Expression` > `2n-zn2 -1c ethip4-ip4base-[mrr|ndrpdr]` + ".*" can be replaced by " " (white space). + - `Choose a cell in the table` > A corresponding graph(s) is displayed. + - `Click a datapoint in the graph` > Detailed information is displayed. + +## CSIT-2402 Selected Performance Tests + +CSIT-2310 VPP v24.02 Performance Tests: + +- ip4 + - [2n-icx 100ge e810cq avf ip4scale20k-rnd](https://csit.fd.io/report/#eNrtVstOwzAQ_JpwQYvsrUO4cGjJfyDjbGlEmpq1G1G-HreqtIkAqUgtvfjgl2asHe9oJIe4YXoO1D0W5aKoFgVWbZOmYja_TQt3AY1CGLwHNHdpx9SRDQTYQ-s-QCv1Sug1PWjl3sEOS2i9gXvzAtoBxdX-lEZwtiNUb8B9A33D-xL4dCzxrZ6gzTYKmlRMkIFYwIk8ofnVbsT5TbTwLZOVC0m5QJHCSMzPbxP2ku2aQvtJciV1RXCXGi-QdtM6cedH6LFhVX1g_JNLPrv0N5f85VzCnKXTXcJrZQlzls7l0gWzZHKWTnfJXCtLJmfpXC5Jlsr6pt_w-vDXK-sv2WmuRg) + - [2n-spr 100ge e810cq avf ip4scale20k-rnd](https://csit.fd.io/report/#eNrtVstqwzAQ_BrnUrZIWznupYem_o-iypvG1HHUlWpIv75KCKxNUkghaS466MWM2NEOAwpxw_QaqHsqykVRLQqs2iZNxcPzXVq4C2gUwuA9oLlPO6aObCDAHoJn0Eq9E3pNj1q5T7DDElpvYG7eQDuguNqd0gjOdoTqA7hvoG94VwJfDiWO6gnafEVBk4oJMhALOJEnNL_ajji_iRa-ZbJyISkXKFIYiTn9NmEv2a4ptN8kV1JXBHep8QJpN60Tt36EHhpW1XvGP7nks0t_c8lfzyXMWTrfJbxVljBn6VIuXTFLJmfpfJfMrbJkcpYu5ZJkqaxn_YbX-79eWf8ADdavEg) + - [2n-spr 100ge e810cq dpdk ip4scale20k-rnd](https://csit.fd.io/report/#eNrtVkFOwzAQfE24oEX24hAuHFryD2TshUZN08U2lcrrcatKmwg4RGrpxYc4tmZXM97RSI5pG-glUv9U1cuqWVbYdD4v1f3iNv9CH9EohB0zoLnLu0A92UiAA0QOoJV6J2RNj1q5D_Ds19CxgQfzCtoBpdXhlL_obE-o1hAGD4MPBw58PnH8IBTUfyZBs4wJsqMg4ESflPFqP6r5U7U02EBWOrJ0gRLFkZrfLyfVb8FuKHZfJC15LIK7PHqBtJvypD2P0NPEmvZY8V8-cfFppk98QZ-w5GmGT3i1PGHJ09l8umSeTMnTDJ_M1fJkSp7O5pPkqW5vhm3YHN99dfsNi3az2g) + - [2n-spr 200ge cx7 mlx5 ip4scale20k-rnd](https://csit.fd.io/report/#eNrtVstqwzAQ_Br3UjbYG7k-9dDU_xEUeZuYyo5YqSbp11cJgbVpezDkcdHBssTsMqMdBuTDnmntyb5m5SqrVhlWbROXbPn2HH9sPaocYXAOUC3ijsmS9gTYg3cMmOdbQleYQzWQDtDZQwmtU_CiNlAYoLA7neLnjbaE-Sdw30Df8IkE3y8kvxgFbb6CoFHHBBmIBZwIlDK3O45q_pctHZpJS0vULlAgP5Lz9-2k-oN1R779JmmJcxHcxOELVJgpTzi6EXoZWVWfK-7mlEtOzXXK3dIpTJma4xQ-LlOYMnU9p26aKZUyNccp9bhMqZSp6zklmSrrp37P3fkNWNY_glq9yg) + - [2n-c6in 200ge c6in.4xl ena dpdk ip4scale20k-rnd](https://csit.fd.io/report/#eNrtl91qwzAMhZ8muxkasZo0u9nFurzH8GxtDU1dYaeF7unnhIISuo0V2rUXvsgf5wQp-jiIhG7j6TVQ-5SVi6xaZFg1Np6y2fN9vPg2YJEj7JgBi4d456klHQjQgZk3DjDPP0ixIqfBsl1BwwWoUj2-gTJA3bJ_jkcwuiXMV-CdBWd9XwNfDjWOCopqt52osY2JsiMv4qQ_sfFyP_L80LXYtSct_ti4SB2FUS_ff5q4371eU2g-SV4ZxiIOE0c_Es20UrfnkXqYWFUPjv_ixInTiZz4gpww5ekETni1PGHK09k4XSZP8yJtpyNKcSi3tJt-Z8SJ0Q3spZ5R2kp_Z4RXylHaSOdiJDkq6zu38evhv6msvwDzBpLJ) + - [2n-c7gn 100ge c7gn ena dpdk ip4scale20k-rnd](https://csit.fd.io/report/#eNrtl91qwzAMhZ8muxkasZosu9nFurzH8GytDU1dYaeF7unnhIISuo0V2rUXvsgf5wQp-jiIhG7j6S1Q-5yV86yaZ1g1Np6y2ct9vPg2YJEj7JgBi4d456klHQjQgakWDlSeL0ixIqfBsl1BwwWoUj29gzJA3bJ_jkcwuiXMV-CdBWd9XwNfDzWOCopqt52osY2JsiMv4qQ_sfFyP_L80LXYtSct_ti4SB2FUS_ff5q4P7xeU2g-SV4ZxiIOE0c_Es20UrfnkXqYWFUPjv_ixInTiZz4gpww5ekETni1PGHK09k4XSZPj0XaTkeU4lBuaTf9zogToxvYSz2jtJX-zgivlKO0kc7FSHJU1ndu49fDf1NZfwEWdpKp) +- ip6 + - [2n-icx 100ge e810cq avf ip6scale20k-rnd](https://csit.fd.io/report/#eNrtVstOwzAQ_JpwQYvsrUN64UCb_0DG2dKINDVrE1G-HreqtIkAqUgtvfjgl2asHe9oJIe4ZXoK1D0U5aKoFgVWbZOmYvZ4mxbuAhqFMHgPaO7SjqkjGwiwh9Z9gFbqhdBrmmvl3sAOK2j9PVTzZ9AOKK73pzSCsx2hegXuG-gb3pfA5bHEt3qCNu9R0KRiggzEAk7kCc2vdyPOb6KFb5msXEjKBYoURmJ-fpuwV2w3FNpPkiupK4K71HiBtJvWiTs_Qo8Nq-oD459c8tmlv7nkL-cS5iyd7hJeK0uYs3Quly6YJZOzdLpL5lpZMjlL53JJslTWN_2WN4e_Xll_AXFKrso) + - [2n-spr 100ge e810cq avf ip6scale20k-rnd](https://csit.fd.io/report/#eNrtVstqwzAQ_Br3UrZIG7nOpYek_o-iypvG1HHUlWJIv75KCKxNW0ghaS466MWM2NEOAwpxy_QSqHsqymVRLQus2iZNxWxxnxbuAhqFMHgPaB7SjqkjGwiwh-AZtFJvhF7TXCv3AXZYQesfoZq_gnZAcX04pRGc7QjVO3DfQN_woQQ-n0p8qydos4uCJhUTZCAWcCJPaH69H3F-Ey18y2TlQlIuUKQwEvPz24S9Yruh0H6SXEldEdylxguk3bRO3PsRempYVR8Z_-SSzy79zSV_PZcwZ-l8l_BWWcKcpUu5dMUsmZyl810yt8qSyVm6lEuSpbK-67e8Of71yvoLpaivlg) + - [2n-spr 100ge e810cq dpdk ip6scale20k-rnd](https://csit.fd.io/report/#eNrtVkFOwzAQfE24oEX24pBeOFDyD2TshUZN08U2lcrrcatKmwg4RGrpxYc4tmZXM97RSI5pG-glUv9Y1cuqWVbYdD4v1f3Tbf6FPqJRCDtmQHOXd4F6spEAB4gcQCv1TsiaFlq5D_Ds19DxAzSLV9AOKK0Op_xFZ3tCtYYweBh8OHDg84njB6Gg_jMJmmVMkB0FASf6pIxX-1HNn6qlwQay0pGlC5QojtT8fjmpfgt2Q7H7ImnJYxHc5dELpN2UJ-15hJ4m1rTHiv_yiYtPM33iC_qEJU8zfMKr5QlLns7m0yXzZEqeZvhkrpYnU_J0Np8kT3V7M2zD5vjuq9tvJi20Xg) + - [2n-spr 200ge cx7 mlx5 ip6scale20k-rnd](https://csit.fd.io/report/#eNrtVkFOwzAQfE24oEXJ4mAuHFryD2SchUY4qbU2UcvrcatKmwg4RGrpxYc4tmZXM97RSA5xy_QSyD0V9brQ6wJ116aluF_dph-7gKpEGL0HVHdpx-TIBAIcIHgGLMt3Ql_ZnR7JROjdrobOP4B-fIXKAsXN4ZS-YI0jLD-AhxaGlg8k-Hwi-cEoaPsZBU06ZshILOBMoJT5zX5S87ds6TBMRlqSdoEihYmc328n1W9segrdF0lLmovgNg1foMrOeeLeT9DTyHRzrPg3p3x2aqlT_pJOYc7UEqfwepnCnKnzOXXRTKmcqSVOqetlSuVMnc8pyVTd3Axb7o9vwLr5Bh_nvk4) + - [2n-c6in 200ge c6in.4xl ena dpdk ip6scale20k-rnd](https://csit.fd.io/report/#eNrtV8tqwzAQ_Br3UrZY2zjqpYem_o-iSNvGxFGEpAbSr49iAmuTBBqwmx508IsZs-MdhsEhbj19BGpfi2pRyEWBsjHpVDy_PaaLbwPOSoSdc4Czp3TnqSUVCNCCnjcWsCy_SDhBVoFxZg2Nm4OoxMsShAaKq-NzOoJWLWG5Bm8NWOOPM_D9NONsIKPmOzKaZAyQHXkGB_qY5lb7HueKaqYrT4r5SThDkUJPy-VPY_anVxsKzQ_xK91amKHT6nugHk6Ke9dDTxuTdcf4K59c9ulGn9yEPmHO0w0-4d3yhDlPo_k0TZ5kbqdzl-T_6iaZm2lEj6bLUW6l33uEd8pRbqSxPOIcVfWD3fpN999U1Qf7j5NR) + - [2n-c7gn 200ge c7gn ena dpdk ip6scale20k-rnd](https://csit.fd.io/report/#eNrtV8tqwzAQ_BrnUjZY27jqpYem_o-gStvExFGEpAaSr69iAmuTFhqwmx508IsZs-MdhsEh7j2tArUvRbUs5LJA2Zh0Kh5fH9LFtwEXJcLBOcDFPN15akkFArSg5dqCKMs1CSfIKjDObKFxTyAq8fwOQgPFzfk5HUGrlrDcgrcGrPHnGfh2mXE1kFHzGRlNMgbIgTyDA31Mc5tjj_ODaqYrT4r5SThDkUJPy_efxuwPr3YUmhPxK91amKHT6nugHk6KR9dDLxuTdcf4K59c9ulGn9yEPmHO0w0-4d3yhDlPo_k0TZ5kbqdrl-T_6iaZm2lEj6bLUW6l33uEd8pRbqSxPOIcVfXM7v2u-2-q6i8e_5Mx) +- ipsec + - [3n-icx 100ge e810cq avf 40tnlsw](https://csit.fd.io/report/#eNrtmM1OxCAQgJ-mXsyYwlK7Fw-ufQ_D0tldkv4gYN369NK6CW2MiZql9cClP5kBBr58yQRjW43PBquHJNsl-S6huSzdI9k83rqXrgxlKYVOKaDszn1prJAbhE0DUpyBpOkRqSK4Jal4Ad4dQOhe2RZIRrZ7IALQnqRiUhkULLVNZd7A_e-HSWRjgaOh2f1R1NCUeliZPl1W_lKGj5av1kddcbNIh9oHZ1X7NHXqfc63e_H5XCP3Az636KMWzaSen27Yjz9oXqOR7-gnGY_PZwiHaRIU87VtrybRyznmxZixLlMVmV6FqQrMlEZPAzClq3pKo6eLMw3tKYueBmDKVvWURU8XZxrOU1nLc2x7_4p0OL3_1vX-mmiU9CpEAzsaW94AROmajsaGd3GigR2N7W4AomxNR2OzuzhR72hW3DStrse73qz4ALThK64) + - [3n-icx 100ge cx6 dpdk 40tnlsw](https://csit.fd.io/report/#eNrtmEtOwzAQQE8TNmhQ7DqkGxaU3AOlztBa5GNsUxpOjxMqTSqEBKhOWHiTj2Zsj_30pJGt6ww-WqzvkmyT5JuE56ryj2R1f-1fprZcpBwOWgMXN_7LYI2lRVi1oOQRWJrukGuGa5bKF6h09QzS9Np1wDK23gKTgG6vtFDaohSpa2v7Bv5_O8yiWgclWp7d7mQDbWWGpfnDaekvdVC0enUU9dWdRQ5oKHhWNqXpfU8532-GBpQGSxrxuUeKOrSTgn66Yxr_ZMoGrXpHmmQ8P8qQHtQkKM_Xdr2eRE8HmRdjxsJUdaR6Gao6NFUeXQ1BlS_rKo-uzk81uKsiuhqCqljWVRFdnZ9qQFdVo46xBf4z1OH4_l0H_GumUdTLMA3taWx_QzDli3oam9_5mYb2NLa-IZiKRT2Nje_8TMnTrLhqO9OMd8BZ8QHg7TU-) + - [3n-spr 100ge e810cq avf 40tnlsw](https://csit.fd.io/report/#eNrtmM1OxCAQgJ-mXsyYwlK7Fw-79j0MpbO7JP1BwGp9emndhG2MiZql9cClP5kBBr58yQRjO41PBuuHJNsn-T6huazcI9nsbt1L14aylEKvFFB257401sgNwqYtwSgNJE2PSBXBLUnFM_D-AEIPynZAMrItgQhAe5KKSWVQsNS2tXkF91-Os8jWAkdDs_ujaKCt9Lg0fTwv_aUOH61erI-66maRHrUPzsr2aeo0-JzvN-MHcI3cj_jco49aNBcF_XTHfvxB8waNfEc_yXR-PkM4UBdBMV_bDuoiej7IvJgyVqaqItXrUFWhqdLoagiqdF1XaXR1earBXWXR1RBU2bqusujq8lQDuiob-RZb4D9DHY_v33XAv2YaRb0O09CexvY3BFO6qqex-V2eaWhPY-sbgilb1dPY-C7P1HuaFTdtp5vpDjgrPgDqXTZ2) + - [3n-spr 200ge cx7 mlx5 40tnlsw](https://csit.fd.io/report/#eNrtmM1OxCAQgJ-mXsyYlgXrxYNr38OwdNwl6Q8BrK1PL62b0MbEaFzEA5f-ZAYY-PIlE4ztNT4ZbO4zts_KfUZKWbtHtnu4di_dGEJzAoNSQOiN-9LYIDcIu46DURpInh-RqEKM5YDcQtuMDISelO2hYMXdAQoBaE9SUakMCprbrjGv4P4P8zyys8DREHZ7FC10tZ4XJ4_nxT9V4qP1i_VRV98mMqD2wU3hPk2dJp_z1Xb8EK6R-zEfu_RRi2ZV0nf37Mc_a96ikW_oJ1lO0GcIB2sVFNu17aRW0fNRltWSEZ2sSmQvRVaFJ0uSs2HIktjOkuRsDLJ_4CxNzoYhS2M7S5OzMcgGdVa2ckyt8S_Azgf4DzvjH3NNwl6Ka3hfU1schiuJ7GtqimNwDe9raonDcKWRfU0NcQyu3ldWXXW9bpc7Y1a9A6ljU84) +- hoststack quic + - [3n-icx 100ge e810cq dpdk ip4udpquic](https://csit.fd.io/report/#eNrlVctuwyAQ_Br3Um0FxJZz6aGJ_6PCsK1RSExYHDX9-hIr6tpqc80hvgDSzOxrtIJSH_Gd0L8W1aaoN4Wqnc1HsXp7zlf0pEqh4BQCqPIlvyJ61ISwOoAzXyCF-EQVJK6lMEewwe6g6ylR0mYHUq1FC9IApg5cKAcbjoMz7SVADomm66ENdMmnttd8f5IzaofEaNbPkBNGBme1Mi10Z-bc7oAFOqJmxW9jTEhIk5put8mKj6j3SO4bWTaOiRkmWzIBzTxbOocJep1e3YyMO_lHRnvMfC8FLcDH_9p9UD-XZecju7ms5bzzblbN06GP-_HPrJof9LcJqA) + - [3n-icx 100ge cx6 mlx5 ip4udpquic](https://csit.fd.io/report/#eNrlVUFuwyAQfI17qbYCYte99NDU_6gw3taoOKYsjpy8PsSKurbaXHNwLoA0M8zujhAU-4AfhO41K7ZZuc1UaZu0ZJu3x7QFRyoXCvbeg8qf0imgQ00Imx1YM4IU4guVl2Z8bkbo3FhA21OkqM03SPUiapAGMLZgfT40_mewpj7r041o2h5qT2c79X6x--PNaDNERpN-gewxMLgolWm-PTDnagPM1wE1C377YkJEmpV0vUtWfAbdIdkjsmyaEjNMCmQGmqVbPPgZehleWU2M26RHRjuUwjgpaP0p_tftOtO8qzBXnOVdPcwbv8uietj1oZv-yqI6ATY_CZg) + - [3n-spr 200ge cx7 mlx5 ip4udpquic](https://csit.fd.io/report/#eNrlVctugzAQ_Bp6qbayHRC99NCE_4iM2RRUExyvQUm-Pg6KsqA-jjmEi21pZvY1WplC53FLaD-SbJ3k60TlTRWPZPX5Gi9vSaVCweAcqPQtvjxa1ISw2msg50EJ8YXKSXPMB9QBWnvMoO4oUNDmG6R6FyVIAxhqaFzaV-7QN6a8hohB0dQdlI6uGdXmlvFHekarPjAa9TNkQM_grFqmufrEnP96YIn2qFlzb40JAWlS1d-NsmLndYvUnJFl46CYYaItE9DMs4WTm6C3-eXFyHiYh2S0RSmMlYIW4eVvDT-tp0uz9LkdXdqSPnhHs-Jl3_l2_EOz4gJZsRPY) +- hoststack tcp udp + - [3n-icx 100ge e810cq dpdk ip4tcp ipudp](https://csit.fd.io/report/#eNrlVUFuwyAQfI17qbYCbNe99JDU_4gwbGorJKYsiZK-vsSKuraqNKf2EF8AMTPsDiMExT7gitC9ZuUyq5aZqjqbhixfPKYpOFKFUHDwHlTxlFYBHWpCyHfQmSNIId5ReYkvUpgPsN5uoO0pUtRmA7J4Fg1IAxhb6HwRjW_OYmd9OqfXNm1iWOfQeDoXVW-Xoj86YNTuI6OprwlywMDgpGGm-fbEnOs2WKADalZ8u2NCRBr1dMMry9ZBb5G6T2TtcGHMMCmcEWimJePJj9DLFVb1wPiHJMloh9I4KWgugf5m-T5y3dv5vNBrXu8mybm90BuW_z7Xsn7Y9WE7_KVl_QUYjRDu) + - [3n-icx 100ge cx6 mlx5 ip4tcp ipudp](https://csit.fd.io/report/#eNrlVcFuwyAM_ZrsMnkCkjSnHdblPyYC7hKNNAjTKt3Xj0bVnGjqeuqluQDiPWM_P1lQHAJ-ELrXrNxm1TZTVWfTkuVvz2kLjlQhFBy9B1W8pFNAh5oQ8j10ZgQpxCcqL824sSP0biyhHShS1OYLZLERDUgDGFvofBGNb86xzvr0zKBtusSwy6HxdM6p3i85_xTAqD1ERlNZC-SIgcFFvUzz7Yk5V1UwXwfUHPArjgkRaVbSDakctgu6R-q-kWOnfjHDJGtmoFmmjCc_Qy8drOqJcX8fyWiH0jgpaCV2_qf4IVw92NVM5zWpj-LjyqbzhuL7u1rWT_sh9NMfWtY_O78Q3g) + - [3n-spr 100ge e810cq dpdk ip4tcp ipudp](https://csit.fd.io/report/#eNrlVUFOwzAQfE24oEW2kxAuHCj5B3LsLYmaNovXrVRejxtVbCJEe4JDc7Etz4x3xyPLHIeAb4z9c1ausmqVmarzacjyl_s0hZ5NoQwciMAUD2kVsEfLCPmuAaYAWql3NKTxSSv3AZ78BtqBI0frNqCLR9WAdoCxhY6K6Kg5qXtP6aDB-rSJYZ1DQ3yqal7PVX-0IKjfR0FTYzPkgEHAWcdCo_YonAs-RGEDWpF82xNCRJ40dcWsyNbBbpG7TxTteGPCcCmeCejmJeORJuj5Dqt6ZPxHluxsj9r1WvFiIr3k-UaS3fsFvdLfzN5Olot7pVc8_32yZX23G8J2_FPL-gse3xSG) + - [3n-spr 200ge cx7 mlx5 ip4tcp ipudp](https://csit.fd.io/report/#eNrlVUFuwyAQfI17qbYCbNenHpr4HxWGTW0Vx4glVtLXl1hR11bV5NQe4gsgZobdYYSgOAR8I3QvWbnJqk2mqs6mIctfH9MUHKlCKBi9B1U8pVVAh5oQ8r0G8gGUEO-ovDTHakQdoXfHEtqBIkVtPkAWz6IBaQBjC50vovHNWe6sTycN2qZNDLscGk_nsmp7KfujB0btITKaOlsgIwYGFy0zzbcn5lwzwhIdULPm2x8TItKsqxtuWbYLukfqPpG105Uxw6SAZqBZlownP0Mvl1jVE-Nf0iSjHUrjpKD1hHrN9L1ke7Breqm_ub2jNNf3Um-Y_vtsy_phP4R--lvL-gsy1Rse) +- nat44 + - [2n-icx 100ge e810cq avf ethip4tcp tput](https://csit.fd.io/report/#eNrtVctqwzAQ_Br3UrZIsiT70kNS_0dQ5E1tcJytpJikX185DcimGAotLYRc9JpZ7Y6GRT4cHG48ds-ZWmfFOhNFW8chy1ePcXKdF5IJGIhAyKe4ctih8Qiih9aegDP2ioI4lpzZNzDDDlqS4_EWuAUMTdwGS-PpdoxrOBMSSOfgtVRcQKBjgL52Y1Lxck36pYKE1seQ0FjXDBnQJXBWcKJRc55wlmQkvnFoUkDUkaCAflLMN8Wm8J0ze_TtO6Y7xodLBBvNmWB2njmcaYJen7CoLox_c5LuTv7USfpzJ7VSuf5UJ7koC13edlsu6L2Zzlzyk-5-_mp_quqhP7j95e9U1QekIvTr) + - [2n-spr 100ge e810cq avf ethip4tcp-nat44ed tput](https://csit.fd.io/report/#eNrtVctqwzAQ_Br3UrZYa0n2pYek_o-iypva4DhbSTGkX185DcimGAotLYRc9JqRdmeHRT4cHD176h8ztc3KbYZl18QhKzb3cXK9R5kjjMyA8iGuHPVkPAEO4NmByPNXQhZUidy-gRl30LGcjl9AWKDQxm2wDIMJUlIDrchRAusCvJZKIAQ-BhgaNwXFp0vQLxkktDmGhMa8FshILoGLhBON29OMsyYj8Y0jky5EOQkK5GfJfFNsur5zZk--e6f0xlS4RLDRnBlml5HDiWfopYRlfWb8m5N8c_KnTvKfO6mVKvSnOimwKnV13W25ovdqOnPNT775-av9qeq74eD2579T1R-52PPT) + - [2n-spr 100ge e810cq avf ethip4tcp-nat44ed cps](https://csit.fd.io/report/#eNrtVdtqwzAM_ZrsZWjYju3kZQ_r8h_Fc9QlkKaa7QXar5_XFZSwFQYbG5S--MKRfHR0EI5pF3AdcbgvzKqoVoWq-jYvRflwm7cwRKWFgokIlL7Lp4ADuoigRogUQArxjIok1lL4F3DTBnrSYPUTSA-YunxLnmB0SWtsoZNCaSBbQrTaSAWeIoxteKdUjyfKT_yMtq-J0VzVApkwMLgol8Oo289izongeBfQcUJWw1DCOCvme1o5exPcFmN_QH4id41xn41hSPolb9rTDD01sGqOEf_kIl1d_JmL9NcuWmNK-yFNS1VXtr7ocfxa7oVM5Bkv6erlL86laW7GXdge_0rTvAFXze5j) + - [2n-spr 200ge cx7 mlx5 ethip4tcp-nat44ed tput](https://csit.fd.io/report/#eNrtVdtqwzAM_ZrsZWjYip30ZQ_t8h_Dc7QlkKTCdkPbr6_bFZwwAoMNCqUvvnAk6xwdhH3YOnr31L1mepOVmwzLto5Llq-f4-Y6j0ogjMyA6iWeHHVkPAEO4NkBCvFFyNLuy5FMgL7ba2hZgRTiA6QFCk28BsswmKAU1dBIgQq4yMEXSkuEwLsAQ-3OZfHtWvYHh4TWu5DQyGyGjOQSOKOcwrg5TGKWhaQM48iklCgoQYH8hM4v5ab0T2d68u2R0hvn1qUAGw2aYHZeORx4gl6bWFaXiBu6yQ83_-4m38DNQuu8-NanJK7KYnXv47mg-I4mdMlTfnj6z3Oqq6dh6_rLX6qrEyEm_aM) +- tunnels (gnv, vxlan, gtpu) + - [2n-icx 100ge e810cq avf ethip4udpgeneve](https://csit.fd.io/report/#eNrtVsGKwjAQ_ZruZZmlia3dyx7U_ofEdNRCjWOSFvXrN5XCtCvCHgSh9ZKEvDeZmTweifNHi2uH1U-ULqNsGcmsLMIQzRafYbKVk0ksoSECmXyFlcUKlUOQBkp9BhHHO5Qk8FvE-gSq2UJJydrXxmDlYJ5sQGhAvw-70M11QTs02CCIeSC2AZv2SFPYNrNcdZnvymC0qD2jobgB0qBlcFA102h_6XEe9cJ8ZVFxQK9Fpnh0vaL-1TIHb606oCuvyCeEu2NcB5kYEnqY1l-oh3b3mOU3xms1pbemT9GUXqvp9Gw6dpdOz6Qj96hMp_eYDnseo0__qEpvVZ_t1TT_MEd7uP1_0_wX2pkbKQ) + - [3n-icx 100ge cx6 mlx5 gtpu sw](https://csit.fd.io/report/#eNrtlkFuwjAQRU-TbqqpksEhqy4KuQcKzhQiOcayHQicvg5CmkQtCxCFjTdx5P_tGfvpS3Z-Z2nlSH0m-SIpFgkWTR0-yezrPQxWORQpwt4YQPER_iwpqhzBTEMje8jSdENoMtnP6x5a1efQGLHyndakHMzFGjIJ5LdhduNN5w6Dvh520LUdCuHyUuhXVVbrzrMaepkoe7IsTppkm9ke2XO1dfZXlipeMDoRWzy5UVN_nZC937ZqyTUn4gXhZliXAQJLmZxW8UczUi_XVpRnx1OJmUjsHmLm_4lhzNhtxPDVGcOYsUcSe0LGRMzYbcTEqzMmYsYeSYwzlpdvemfb85sxL38A-znb1g) + - [3n-spr 200ge cx7 mlx5 vxlan](https://csit.fd.io/report/#eNrtVcFuwjAM_ZpymTy1oV1PHAb9D5SmHlRKg-WEquzrF1glt0Jw4YCEuCRRnl_s5ycrPhwYtx7tKinWSblOVNk2cUmW3x9xY-tVniroiUDln_HEaFF7hKXT4IlBpekOFWVmKHvUATo7FNBSvg1H59B6-MpryAxg2MfbfrDagVV1U8dHOm0sO3ANnzOqzZjxKr2gzTEIGouaIT2ygLNqJYz2J4m5p0EomlELZyJNQgL6SV23pQrjh3WHvv1FocVGCW6iLQJlZp4rnGiCjv0rq0vEczykt4cPeUjP8XAwZ2EvPoH_Il9q9kbf6O3bw_NWVAt34O7y_xXVH06v_as) + - [3n-spr 200ge cx7 mlx5 gtpu sw](https://csit.fd.io/report/#eNrtlsFugzAMhp-GXSZP4IZx2mEt71Gl4LVIgVpJYO2efqGqZNCmSZ0q1EMuBPH_xk4-_VKcP1raOjJvSb5OinWCRVOHR7J6fw6LNQ5VijAwA6qX8GbJkHYEq06DYwuYpntCzqpTMZD20JpTDg2rre-7joyDV7WDrALyh_B177l3n6O-G3_S1XbshZtrrx-NRa17L2oYZ6YMZEWczSk2PpzF89f0UqItaamZbEosntxkrt82Kd4Pq1tyzRdJQTgc0auAQqSsmnfxZ56o15MryotjaW4cuf2TGy_CDWPebuaGD5A3jHm7M7dl8qZi3m7mph4gbyrm7c7cJG95-dQdbXu5V-blN__K6zY) + - [3n-spr 200ge cx7 mlx5 wireguard](https://csit.fd.io/report/#eNrtVl1rwyAU_TXZy3CoTUhe-rA2_6PYeNcGjJWrSdv9-plQuAmD7WWsUPui4jnX-3E4oA8nhJ0Hs86KTVZuMlm2Oi7Z6v01bmi8zLlkg3NM5m_xhGBAeWArq5h3yCTnB5BONJdyABVYZy4Fa12-C721YDwThaj2TDQMwjHe99qdW4RDr1ALzoM1_jzy9-OjVuOYW25vub8VQqjuA6GxvAUyABK4qJto7nglzk_dUIhCUBQza5IoAfysrt9bpsgPVB349hMofBodMZoo1QxsllnD1c3Q2yTLemLcW1f31PWPdHV31TU5tybg1eSc-vg-rVLzafX4Pq1S82n1zz4t6hd7wm76Bxf1F_3qK2U) +- reassembly + - [3n-icx 100ge e810cq dpdk reassembly](https://csit.fd.io/report/#eNrtVkFOwzAQfE24oEWx65BeOFDyD5Q4S2uROMZrCuH1uKHSJkJckNLmkIttecar2R2NZAqdx2fC5iHJdkm-S2Ru6rgkm8fbuPmGpEolHJ0Dqe7iyWODJSFsLBj9CSJN9yidwK1I9RvUrn4F7XsXOhCZ2FYgNGA4GKeMI9SRngbb0AfEm-pUx9gAJZLM7ve6jdVLImyrpgdb-5MO-XTW8UsUo_V7YDRKnSBH9AxOemCaO_TM-bszflBGofzip2FGA9JI0L_a52IvvmyRzBdyxWGyzNDRwhGop0JC70boeap5MTCW5Ldb_Z7Bbze333LN92X9ltfNt1zzvSS_Z8-3WvN9Wb_VdfOt1nwvyW_Od1bc2M63wz89K74B21hUdQ) + +## CSIT-2402 Selected Performance Comparisons + +Comparisons 24.02 vs 23.10 +- [2n-icx 100ge e810cq avf 1c 64B PDR](https://csit.fd.io/comparisons/#eNqNkE0OwiAQhU9TN4YGsLVuXFh7AGO8AMGpadJSHGijnl7oj9jEhQkBZt43zPAM1CAtXPdRlkc8QygBQUlw92hzWI9ZA3aKL9g56bgekr6yatUCvXYz6o5eax-MvFN6wCBibXhCOeGbmFGC7jVhIOCVKlEEmitSyQdhlN6AawY7RuWdiL4MFbLFz9ipn4UdfZgWk15iY6rXEtkm-YKx9qmXxKk4T0Q2rLlXowX-4ZKjRAP2--PBiBHpRd3BD1-S2O1fvgz9V6rFZu9HS4s34Cl0zQ) + +## CSIT-2402 Selected Performance Coverage Data + +CSIT-2402 VPP v24.02 coverage data +- [2n-icx 200ge cx7 mlx5 ip4](https://csit.fd.io/coverage/#eNpVjsEOwiAQRL8GLwaDK4RTD9r-h9ngxpIgJYBI_94SD9TLJjNvZjOJHJlsFz8wfWOgo0vbZZfr8SdACmgGjM14vHOnJYQ_Uih2CPIkgMftOybqsTCvu4zn1lQOQjwJwtlUXQgzf7mqegMjYa_YIJvQ0yHNy-fuMJM368BU2763WkpNXye-PaE) + +## Further Information + +For further information including instructions how to access the needed +information with user selectable options, please refer to +[csit.fd.io documentation]({{< relref "/" >}}). diff --git a/docs/content/release_notes/current/dpdk_performance.md b/docs/content/release_notes/current/dpdk_performance.md new file mode 100644 index 0000000000..97e757fe50 --- /dev/null +++ b/docs/content/release_notes/current/dpdk_performance.md @@ -0,0 +1,38 @@ +--- +title: "DPDK Performance" +weight: 2 +--- + +# CSIT 24.02 - DPDK Performance + +1. TEST FRAMEWORK + - **CSIT test environment** version has been updated to ver. 14, see + [Environment Versioning]({{< ref "../../../infrastructure/fdio_csit_testbed_versioning" >}}). +2. DPDK PERFORMANCE TESTS + - No updates +3. DPDK RELEASE VERSION CHANGE + - Version 23.11 is now tested. + +# Known Issues + +List of known issues in CSIT 24.02 for DPDK performance tests: + +## New + +List of new issues in CSIT 24.02 for DPDK performance tests: + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | | + +## Previous + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | | + +## Fixed + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | | diff --git a/docs/content/release_notes/current/trex_performance.md b/docs/content/release_notes/current/trex_performance.md new file mode 100644 index 0000000000..d0f82fd2e7 --- /dev/null +++ b/docs/content/release_notes/current/trex_performance.md @@ -0,0 +1,40 @@ +--- +title: "TRex Performance" +weight: 3 +--- + +# CSIT 24.02 - TRex Performance + +1. TEST FRAMEWORK + - **CSIT test environment** version has been updated to ver. 14, see + [Environment Versioning]({{< ref "../../../infrastructure/fdio_csit_testbed_versioning" >}}). +2. TREX TESTS + - No updates +3. TREX VERSION + - Currently using v3.03 of TRex. + +# Known Issues + +## New + +List of new issues in CSIT 24.02 for TRex performance tests: + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | | + +## Previous + +List of known issues in CSIT 24.02 for TRex performance tests + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | | + +## Fixed + +List of known issues in CSIT 24.02 for TRex performance tests + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | | diff --git a/docs/content/release_notes/current/vpp_device.md b/docs/content/release_notes/current/vpp_device.md new file mode 100644 index 0000000000..3ade1c6b68 --- /dev/null +++ b/docs/content/release_notes/current/vpp_device.md @@ -0,0 +1,27 @@ +--- +title: "VPP Device" +weight: 4 +--- + +# CSIT 24.02 - VPP Device + +1. TEST FRAMEWORK + - **CSIT test environment** version has been updated to ver. 14, see + [Environment Versioning]({{< ref "../../../infrastructure/fdio_csit_testbed_versioning" >}}). + +# Known Issues + +List of known issues in CSIT 24.02 for VPP functional tests in VPP Device: + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | [CSIT-1931](https://jira.fd.io/browse/CSIT-1931) | Vhost test not running in device jobs + 2 | [CSIT-1932](https://jira.fd.io/browse/CSIT-1932) | 1n-spr: Occasional packet loss in L2 tests + +## New + +List of new issues in CSIT 24.02 for VPP functional tests in VPP Device: + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | | diff --git a/docs/content/release_notes/current/vpp_performance.md b/docs/content/release_notes/current/vpp_performance.md new file mode 100644 index 0000000000..3b3711039e --- /dev/null +++ b/docs/content/release_notes/current/vpp_performance.md @@ -0,0 +1,108 @@ +--- +title: "VPP Performance" +weight: 1 +--- + +# CSIT 24.02 - VPP Performance + +1. TEST FRAMEWORK + - **CSIT test environment** version has been updated to ver. 14, see + [Environment Versioning]({{< ref "../../../infrastructure/fdio_csit_testbed_versioning" >}}). + - **General Code Housekeeping**: Ongoing code optimizations and bug fixes. + - **Trending and release testing**: Ndrpdr tests use newer code + (MLRsearch 1.2.1) and configuration, gaining more stability and speed. +1. VPP PERFORMANCE TESTS + - Added 2n-c7gn and 3n-icxd testbeds. +2. PRESENTATION AND ANALYTICS LAYER + - [Performance dashboard](https://csit.fd.io/) got updated with the + possibility to [search in tests](https://csit.fd.io/search/). + - [Per Release Performance Comparisons](https://csit.fd.io/comparisons/) got + updated with the function removing extreme outliers from data presented in + the comparison table. + +# Known Issues + +These are issues that cause test failures or otherwise limit usefulness of CSIT +testing. + +## New + +Any issue listed here may have been present also in a previous release, +but was not detected/recognized/reported enough back then. +Also, issues previously thought fixed but now reopened are listed here. + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | [CSIT-1845](https://jira.fd.io/browse/CSIT-1845) | AVF 9000B any ndrpdr test may start failing due to packets not arriving in one or both directions. + 2 | [CSIT-1946](https://jira.fd.io/browse/CSIT-1946) | Ipsec hwasync fails with large scale and multiple queues. + 3 | [CSIT-1947](https://jira.fd.io/browse/CSIT-1947) | VPP crash in udp nat avf 4c tests. + 4 | [CSIT-1948](https://jira.fd.io/browse/CSIT-1948) | NICs do not consistently distribute tunnels over RXQs depending on model or plugin. + 5 | [CSIT-1950](https://jira.fd.io/browse/CSIT-1950) | 9000B tests with high encap overhead see fragmented packets. + 6 | [CSIT-1951](https://jira.fd.io/browse/CSIT-1951) | Combination of AVF and vhost drops all 9000B packets. + 7 | [CSIT-1954](https://jira.fd.io/browse/CSIT-1954) | 3n-icx: 9000B AVF ip6 tests show zero traffic in one direction due to no free tx slots. + +## Previous + +Issues reported in previous releases which still affect the current results. + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | [CSIT-1782](https://jira.fd.io/browse/CSIT-1782) | Multicore AVF tests are failing when trying to create interface. Frequency is reduced by CSIT workaround, but occasional failures do still happen. + 2 | [CSIT-1785](https://jira.fd.io/browse/CSIT-1785) | NAT44ED tests failing to establish all TCP sessions. At least for max scale, in allotted time (limited by session 500s timeout) due to worse slow path performance than previously measured and calibrated for. CSIT removed the max scale NAT tests to avoid this issue. + 3 | [CSIT-1795](https://jira.fd.io/browse/CSIT-1795) | Ocassionally not all DET44 sessions have been established: 4128767 != 4128768 + 4 | [CSIT-1802](https://jira.fd.io/browse/CSIT-1802) | All testbeds: AF-XDP - NDR tests failing from time to time on small loss. + 5 | [CSIT-1804](https://jira.fd.io/browse/CSIT-1804) | 3n-tsh: NDR fails on ierrors. + 6 | [CSIT-1849](https://jira.fd.io/browse/CSIT-1849) | 2n-clx, 2n-icx: UDP 16m TPUT tests fail to create all sessions. + 7 | [CSIT-1881](https://jira.fd.io/browse/CSIT-1881) | 2n-icx: NFV density tests ocassionally breaks VPP which fails to start. + 8 | [CSIT-1886](https://jira.fd.io/browse/CSIT-1886) | 3n-icx: Wireguard tests with 100 and more tunnels are failing PDR criteria. + 9 | [CSIT-1892](https://jira.fd.io/browse/CSIT-1892) | 3n-alt: Unexpected two-band structure of ipsec and vxlan. + 10 | [CSIT-1896](https://jira.fd.io/browse/CSIT-1896) | Depending on topology, l3fwd avoids dut-dut link. + 11 | [CSIT-1901](https://jira.fd.io/browse/CSIT-1901) | 3n-icx: Negative ipackets on TB38 AVF 4c l2patch. + 12 | [CSIT-1904](https://jira.fd.io/browse/CSIT-1904) | DPDK 23.03 testpmd startup fails on some testbeds. + 13 | [CSIT-1906](https://jira.fd.io/browse/CSIT-1906) | Zero traffic with cx7 rdma. Testing migrated to mlx5-core on all Mellanox NICs. + 14 | [VPP-2077](https://jira.fd.io/browse/VPP-2077) | IP fragmentation: running_fragment_id is not thread safe. Causes reduced performance and failures in gtpu reassembly tests. + 15 | [CSIT-1914](https://jira.fd.io/browse/CSIT-1914) | TRex does not produce latency data on ICE NICs. + 16 | [CSIT-1915](https://jira.fd.io/browse/CSIT-1915) | The 2n-icx testbeds to not have the same performance. + 17 | [CSIT-1916](https://jira.fd.io/browse/CSIT-1916) | Poor CPU scaling on 2n-zn2 RDMA. + 18 | [CSIT-1917](https://jira.fd.io/browse/CSIT-1917) | TRex STL performance is unstable at high pps due to unsent packets. + 19 | [CSIT-1921](https://jira.fd.io/browse/CSIT-1921) | Two-band structure in SRv6, causes PDR failure in rare cases. + 20 | [CSIT-1922](https://jira.fd.io/browse/CSIT-1922) | 2n-tx2: AF_XDP MRR failures. On other testbeds MRR regressions and PDR failures. + 21 | [CSIT-1924](https://jira.fd.io/browse/CSIT-1924) | An l3fwd error in 200Ge2P1Cx7Veat-Mlx5 test with 9000B. + 22 | [CSIT-1935](https://jira.fd.io/browse/CSIT-1935) | Zero traffic reported in udpquic tests due to session close errors. + 23 | [CSIT-1936](https://jira.fd.io/browse/CSIT-1936) | TRex occasionally sees link down in L2 perf tests. + 24 | [CSIT-1937](https://jira.fd.io/browse/CSIT-1937) | Small but frequent loss in ASTF UDP on cx7 mlx5. + 25 | [CSIT-1938](https://jira.fd.io/browse/CSIT-1938) | 3n-alt: High scale ipsec policy tests may crash VPP. + 26 | [CSIT-1939](https://jira.fd.io/browse/CSIT-1939) | 3na-spr, 2n-zn2: VPP fails to start in first test cases. + 27 | [CSIT-1941](https://jira.fd.io/browse/CSIT-1941) | TRex may wrongly detect link bandwidth. + 28 | [CSIT-1942](https://jira.fd.io/browse/CSIT-1942) | 3nb-spr hoststack: Interface not up after first test. + 29 | [CSIT-1943](https://jira.fd.io/browse/CSIT-1943) | IMIX 4c tests may fail PDR due to ~10% loss. + 30 | [CSIT-1944](https://jira.fd.io/browse/CSIT-1944) | Memif LXC: unrecognized option '--no-validate'. + 31 | [VPP-2090](https://jira.fd.io/browse/VPP-2090) | MRR < PDR: DPDK plugin with MLX5 driver does not read full queue. + 32 | [VPP-2091](https://jira.fd.io/browse/VPP-2091) | Memif crashes with jumbo frames. + +## Fixed + +Issues reported in previous releases which were fixed in this release: + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | [CSIT-1883](https://jira.fd.io/browse/CSIT-1883) | 3n-snr: All hwasync wireguard tests failing when trying to verify device. + 2 | [CSIT-1940](https://jira.fd.io/browse/CSIT-1940) | Hardware acceleration does not work yet. + 3 | [VPP-2087](https://jira.fd.io/browse/VPP-2087) | VPP crash and other symptoms in tests with AVF, jumbo packets. + 4 | [VPP-2088](https://jira.fd.io/browse/VPP-2088) | virtio: Bad CLI argument parsing introduced with tx-queue-size. + +# Root Cause Analysis for Regressions + +List of RCAs in CSIT 24.02 for VPP performance regressions. +Not listing differences caused by known issues (uneven worker load +due to randomized RSS or other per-worker issues). +Also not listing tests which historically show large performance variance. + +Contrary to issues, these genuine regressions do not limit usefulness +of CSIT testing. So even if they are not fixed +(e.g. when the regression is an expected consequence of added functionality), +they will not be re-listed in the next release report. + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | [VPP-2099](https://jira.fd.io/browse/VPP-2099) | Bump of rdma-core to 49.0 decreased performance. diff --git a/docs/content/release_notes/previous/_index.md b/docs/content/release_notes/previous/_index.md new file mode 100644 index 0000000000..a1b468f16c --- /dev/null +++ b/docs/content/release_notes/previous/_index.md @@ -0,0 +1,31 @@ +--- +bookCollapseSection: true +bookFlatSection: false +title: "Previous" +weight: 2 +--- + +# Previous Releases + +- [CSIT rls2306]({{< relref "csit_rls2306" >}}) +- [CSIT rls2302]({{< relref "csit_rls2302" >}}) +- [CSIT rls2210](https://s3-docs.fd.io/csit/rls2210/report/) +- [CSIT rls2206](https://s3-docs.fd.io/csit/rls2206/report/) +- [CSIT rls2202](https://s3-docs.fd.io/csit/rls2202/report/) +- [CSIT rls2110](https://s3-docs.fd.io/csit/rls2110/report/) +- [CSIT rls2106](https://s3-docs.fd.io/csit/rls2106/report/) +- [CSIT rls2101](https://s3-docs.fd.io/csit/rls2101/report/) +- [CSIT rls2009](https://s3-docs.fd.io/csit/rls2009/report/) +- [CSIT rls2005](https://s3-docs.fd.io/csit/rls2005/report/) +- [CSIT rls2001](https://s3-docs.fd.io/csit/rls2001/report/) +- [CSIT rls1908](https://s3-docs.fd.io/csit/rls1908/report/) +- [CSIT rls1904](https://s3-docs.fd.io/csit/rls1904/report/) +- [CSIT rls1901](https://s3-docs.fd.io/csit/rls1901/report/) +- [CSIT rls1810](https://s3-docs.fd.io/csit/rls1810/report/) +- [CSIT rls1804](https://s3-docs.fd.io/csit/rls1804/report/) +- [CSIT rls1801](https://s3-docs.fd.io/csit/rls1801/report/) +- [CSIT rls1710](https://s3-docs.fd.io/csit/rls1710/report/) +- [CSIT rls1704](https://s3-docs.fd.io/csit/rls1704/report/) +- [CSIT rls1701](https://s3-docs.fd.io/csit/rls1701/report/) +- [CSIT rls1609](https://wiki.fd.io/view/CSIT/VPP-16.09_Test_Report) +- [CSIT rls1606](https://wiki.fd.io/view/CSIT/VPP-16.06_Test_Report) diff --git a/docs/content/release_notes/previous/csit_rls2302/_index.md b/docs/content/release_notes/previous/csit_rls2302/_index.md new file mode 100644 index 0000000000..34425fbc69 --- /dev/null +++ b/docs/content/release_notes/previous/csit_rls2302/_index.md @@ -0,0 +1,13 @@ +--- +bookCollapseSection: true +bookFlatSection: false +title: "CSIT rls2302" +weight: 3 +--- + +# CSIT rls2302 + +- [VPP Performance]({{< relref "vpp_performance" >}}) +- [DPDK Performance]({{< relref "dpdk_performance" >}}) +- [TRex Performance]({{< relref "trex_performance" >}}) +- [VPP Device]({{< relref "vpp_device" >}}) diff --git a/docs/content/release_notes/previous/csit_rls2302/dpdk_performance.md b/docs/content/release_notes/previous/csit_rls2302/dpdk_performance.md new file mode 100644 index 0000000000..320dccf746 --- /dev/null +++ b/docs/content/release_notes/previous/csit_rls2302/dpdk_performance.md @@ -0,0 +1,31 @@ +--- +title: "DPDK Performance" +weight: 2 +--- + +# CSIT 23.02 - DPDK Performance + +1. TEST FRAMEWORK + - **CSIT test environment** version has been updated to ver. 11, see + [Environment Versioning]({{< ref "../../../infrastructure/fdio_csit_testbed_versioning" >}}). +2. DPDK PERFORMANCE TESTS + - No updates +3. DPDK RELEASE VERSION CHANGE + - CSIT 23.02 tested DPDK 22.07, as used by VPP 23.02. + +# Known Issues + +List of known issues in CSIT 23.02 for DPDK performance tests: + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | [CSIT-1848](https://jira.fd.io/browse/CSIT-1848) | 2n-clx, 3n-alt: sporadic testpmd/l3fwd tests fail with no or low traffic. + + +## New + +List of new issues in {{< release_csit >}} for DPDK performance tests: + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | | diff --git a/docs/content/release_notes/previous/csit_rls2302/trex_performance.md b/docs/content/release_notes/previous/csit_rls2302/trex_performance.md new file mode 100644 index 0000000000..67f2947891 --- /dev/null +++ b/docs/content/release_notes/previous/csit_rls2302/trex_performance.md @@ -0,0 +1,26 @@ +--- +title: "TRex Performance" +weight: 3 +--- + +# CSIT 23.02 - TRex Performance + +1. TEST FRAMEWORK + - **CSIT test environment** version has been updated to ver. 11, see + [Environment Versioning]({{< ref "../../../infrastructure/fdio_csit_testbed_versioning" >}}). + +# Known Issues + +List of known issues in CSIT 23.02 for TRex performance tests + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | [CSIT-1876](https://jira.fd.io/browse/CSIT-1876) | 1n-aws: TRex NDR PDR ALL IP4 scale and L2 scale tests failing with 50% packet loss. CSIT removed ip4scale and l2scale except ip4scale2m where it's still failing. + +## New + +List of new issues in CSIT 23.02 for TRex performance tests: + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | | diff --git a/docs/content/release_notes/previous/csit_rls2302/vpp_device.md b/docs/content/release_notes/previous/csit_rls2302/vpp_device.md new file mode 100644 index 0000000000..44ba9f5ce5 --- /dev/null +++ b/docs/content/release_notes/previous/csit_rls2302/vpp_device.md @@ -0,0 +1,26 @@ +--- +title: "VPP Device" +weight: 4 +--- + +# CSIT 23.02 - VPP Device + +1. TEST FRAMEWORK + - **CSIT test environment** version has been updated to ver. 11, see + [Environment Versioning]({{< ref "../../../infrastructure/fdio_csit_testbed_versioning" >}}). + +# Known Issues + +List of known issues in CSIT 23.02 for VPP functional tests in VPP Device: + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | | + +## New + +List of new issues in CSIT 23.02 for VPP functional tests in VPP Device: + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | | diff --git a/docs/content/release_notes/previous/csit_rls2302/vpp_performance.md b/docs/content/release_notes/previous/csit_rls2302/vpp_performance.md new file mode 100644 index 0000000000..072c55f14e --- /dev/null +++ b/docs/content/release_notes/previous/csit_rls2302/vpp_performance.md @@ -0,0 +1,93 @@ +--- +title: "VPP Performance" +weight: 1 +--- + +# CSIT 23.02 - VPP Performance + +1. VPP PERFORMANCE TESTS + - **Enhanced and added VPP hoststack tests** to daily and weekly + trending including: Quic VPP Echo, UDP+TCP LD_PRELOAD iPerf3, + LD_PRELOAD NGINX. + - **Added Nvidia/Mellanox DPDK tests** to daily and weekly trending + and report, in addition to RDMA_CORE ones that were already + there. + - **Jumbo frames tests** got fixed and re-added number of to report + coverage tests. + - **Intel Xeon SKX performance testbeds** got decommissioned and + removed from FD.io performance lab. +2. TEST FRAMEWORK + - **CSIT test environment** version has not changed from ver. 11 used + in previous release, see + [Environment Versioning]({{< ref "../../../infrastructure/fdio_csit_testbed_versioning" >}}). + - **CSIT PAPI optimizations for scale** got applied improving PAPI + programming speed especially for large scale tests. VAT has been + now completely deprecated from CSIT. + - **General Code Housekeeping**: Ongoing code optimizations and bug + fixes. +3. PRESENTATION AND ANALYTICS LAYER + - [Performance dashboard](https://csit.fd.io/) got updated with + addition of VPP telemetry trending across all VPP tests. A number + of code and AWS resource usage optimizations got applied to the + data processing pipeline and UI frontend and backend. + - Examples of release iterative data visualisation: + - [Packet throughput 2n-icx-e810cq-ip4-base-scale-pdr](https://csit.fd.io/report/#eNrdVcluwjAQ_Zr0ggbZDml64QDkP5BxhhJlwYxNVPr1OAhpYiGO7cEHb3pv1qeRnT8T7h1266zYZuU2U2VThy3LN4twUOdULhSM1oLKl-FG2KF2CGqAxvyAFOIblZX4JYW5gB6P0NgVfK4OIA2gP02vsA6Tja1pcq12T9cvcRitr57RED1CRiQGo7SYZk-3GeddsszXhJoNQsYMeXSzZOKamHUk3aNrfpGpoQuMm9BohqSJ_fubnaHPRpXVg_F3qjijO1RCtEBDnZo8UXFJ6NQmKlGbgjp9ujPU_8cEFdXHcKb-8Q8V1R2PI8PX) + - [Speedup Multi-Core throughput graph for 2n-icx-e810cq-ip4-base-pdr](https://csit.fd.io/report/#eNrtlM8OgjAMxp8GL6aGFRAvHlTew8xRhAR1bpOoT-8wJIUYEg8mXjjsX35fu65fMusuhvaW6nWQbIN0G2Ba5X4Kos3cL6a2GIUIjdaA0cLvDNUkLQGeoVJ3EGF4JNSCViJUV5BNAZWOYRkfQCggV7YnPw5tjM5Nmxp3XeqPe5jmN8fU3z4gDRmGg7JYpstHTzNWLOulIckBvmJGjmyvmOGbWFUYeSJbPYmlvgvMlW80I6GG-d1D92jXqDR7K37qCk6ujLuC_3IlnlwZdyX-0pUkm50v5vT-yZLsBXP6Swk>) + - [MRR, NDR and PDR comparison for 2n-icx-e810cq-ip4-base](https://csit.fd.io/report/#eNrtVMsOgjAQ_Bq8mDW0gHjxoPIfppZVSQDrthLx6y2GuBBj4kVPHvrKzG6nM0mtOxFuLZbLIFkH6TqQaZH7KYhWU79QaWUUSmiMARnN_I6wRGURZA2FvoIIwwNKI3AhQn0G1eyhMDHM4x0IDeiO3cmPXVdTEXWt5aZv_XIPo_nFMepvHyENEoMjWUwzx3bAeSeW-YpQcYFXzJBDOxAzfhOz9qQqtMUNmepdYFx7oxkSetzftWaA9kal2YPx5VTq_J_KR6n0Rv0mFfNP5bNUzDOVJJvUJ6oeP1mS3QG2H0sT>) + - [Normalized throughput architecture comparison for 2n-[icx|clx]-e810cq-ip4-base-pdr](https://csit.fd.io/report/#eNrVk00OgjAQhU-DGzOGFhA3LlTuYUoZhKRibSsRT28hJANRF-500b98rzOvM6l1F4NHi2obJPsg3Qc8rQs_BdFu6RejLI9CDq3WwKOV3xlUKCwCb0CqO7AwPCHXDDcslFcQbQm1jmEd58AkoKv6kx95f0cXpg_ND2PolzxEi5sj6rPPSIuG4MwWyXTVTTSfzJJeGBR0wTsm5NBOzMzfRKrSiDPa-oEk9VUgLn2hCTE5j-86PaFjodJsUHzXlVr-UVfem_35riTZormY8_BneNpvhRpzJNkT6FzkMw>) + - [NICs comparison for 2n-icx-ip4-base-pdr](https://csit.fd.io/report/#eNrll99ugyAUh5_G3SxnESx1N7to53s0FI6rmbYMnKF7-qFrcmRmV7vReuG__A74wSckuvZi8eCwfknEPsn3Cc8rHU5JtnsMF1s7nqUcOmOAZ0_hzmKN0iHwM6jaA0vTN-SGKS_EVkJTewGV2cB2cwSmANtT_xSOY9_IaNv3zV9vfU9eRKn-bCkNr4-SDi2FEReVmdN1VPMnLTWQFiW1CMgUtehGNPGgqKq0skFXfSGVhmmgXIWppoipuP_2akbpbabyYqj4txerG7kcLz3tnXvBZ5aqD5BduQAtBLsOK9ro9-Vo6Wnv1sswUJ-zdPZLJSJdgY_ZL5IY9U6NcPEzTN8NX14JXpsZW_mNewi46zAz691rwroKJzPfwaaws7ciiofzxTbDv6QovgETwNPp>) + +# Known Issues + +Editing Note: below listed known issues need to be updated to reflect the +current state as tracked on +[CSIT TestFailuresTracking wiki](https://wiki.fd.io/view/CSIT/TestFailuresTracking). + +## New + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | [CSIT-1890](https://jira.fd.io/browse/CSIT-1890) | 3n-alt: Tests failing until 40Ge Interface comes up. + +## Previous + +Issues reported in previous releases which still affect the current results. + +**#** | **JiraID** | **Issue Description** +------|-------------------------------------------------------------------------------------------------|--------------- + 1 | [CSIT-1782](https://jira.fd.io/browse/CSIT-1782) | Multicore AVF tests are failing when trying to create interface. Frequency is reduced by CSIT workaround, but occasional failures do still happen. + 2 | [CSIT-1785](https://jira.fd.io/browse/CSIT-1785) [VPP-1972](https://jira.fd.io/browse/VPP-1972) | NAT44ED tests failing to establish all TCP sessions. At least for max scale, in allotted time (limited by session 500s timeout) due to worse slow path performance than previously measured and calibrated for. CSIT removed the max scale NAT tests to avoid this issue. + 3 | [CSIT-1799](https://jira.fd.io/browse/CSIT-1799) | All NAT44-ED 16M sessions CPS scale tests fail while setting NAT44 address range. + 4 | [CSIT-1800](https://jira.fd.io/browse/CSIT-1800) | All Geneve L3 mode scale tests (1024 tunnels) are failing. + 5 | [CSIT-1801](https://jira.fd.io/browse/CSIT-1801) | 9000B payload frames not forwarded over tunnels due to violating supported Max Frame Size (VxLAN, LISP, + 6 | [CSIT-1802](https://jira.fd.io/browse/CSIT-1802) | all testbeds: AF-XDP - NDR tests failing from time to time. + 7 | [CSIT-1804](https://jira.fd.io/browse/CSIT-1804) | All testbeds: NDR tests failing from time to time. + 8 | [CSIT-1808](https://jira.fd.io/browse/CSIT-1808) | All tests with 9000B payload frames not forwarded over memif interfaces. + 9 | [CSIT-1827](https://jira.fd.io/browse/CSIT-1827) | 3n-icx, 3n-skx: all AVF crypto tests sporadically fail. 1518B with no traffic, IMIX with excessive + 10 | [CSIT-1835](https://jira.fd.io/browse/CSIT-1835) | 3n-icx: QUIC vppecho BPS tests failing on timeout when checking hoststack finished. + 11 | [CSIT-1849](https://jira.fd.io/browse/CSIT-1849) | 2n-skx, 2n-clx, 2n-icx: UDP 16m TPUT tests fail to create all sessions. + 12 | [CSIT-1864](https://jira.fd.io/browse/CSIT-1864) | 2n-clx: half of the packets lost on PDR tests. + 13 | [CSIT-1877](https://jira.fd.io/browse/CSIT-1877) | 3n-tsh: all VM tests failing to boot VM. + 14 | [CSIT-1883](https://jira.fd.io/browse/CSIT-1883) | 3n-snr: All hwasync wireguard tests failing when trying to verify device. + 15 | [CSIT-1884](https://jira.fd.io/browse/CSIT-1884) | 2n-clx, 2n-icx: All NAT44DET NDR PDR IMIX over 1M sessions BIDIR tests failing to create enough sessions. + 16 | [CSIT-1885](https://jira.fd.io/browse/CSIT-1885) | 3n-icx: 9000b ip4 ip6 l2 NDRPDR AVF tests are failing to forward traffic. + 17 | [CSIT-1886](https://jira.fd.io/browse/CSIT-1886) | 3n-icx: Wireguard tests with 100 and more tunnels are failing PDR criteria. + +## Fixed + +Issues reported in previous releases which were fixed in this release: + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | [CSIT-1868](https://jira.fd.io/browse/CSIT-1868) | 2n-clx: ALL ldpreload-nginx tests fails when trying to start nginx. + 2 | [CSIT-1871](https://jira.fd.io/browse/CSIT-1871) | 3n-snr: 25GE interface between SUT and TG/TRex goes down randomly. + +# Root Cause Analysis for Performance Changes + +List of RCAs in CSIT 23.02 for VPP performance changes: + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | [CSIT-1887](https://jira.fd.io/browse/CSIT-1887) | rls2210 RCA: ASTF tests TRex upgrade decreased TRex performance. NAT results not affected, except on Denverton due to interference from VPP-2010. + 2 | [CSIT-1888](https://jira.fd.io/browse/CSIT-1888) | rls2210 RCA: testbed differences, especially for ipsec. Not caused by VPP code nor CSIT code. Most probable cause is clang-14 behavior. + 3 | [CSIT-1889](https://jira.fd.io/browse/CSIT-1889) | rls2210 RCA: policy-outbound-nocrypto. When VPP added spd fast path matching (Gerrit 36097), it decreased MRR of the corresponding tests, at least on 3-alt. diff --git a/docs/content/release_notes/previous/csit_rls2306/_index.md b/docs/content/release_notes/previous/csit_rls2306/_index.md new file mode 100644 index 0000000000..6578ef91f7 --- /dev/null +++ b/docs/content/release_notes/previous/csit_rls2306/_index.md @@ -0,0 +1,111 @@ +--- +bookCollapseSection: true +bookFlatSection: false +title: "CSIT rls2306" +weight: 2 +--- + +# CSIT-2306 Release Report + +The format of FD.io CSIT reports has now changed. It is no longer available +in the static html format. + +Instead current CSIT release information can be found in csit.fd.io. +Previous CSIT releases are also linked from there. + +## CSIT-2306 Release Notes + +- [VPP Performance]({{< relref "vpp_performance" >}}) +- [DPDK Performance]({{< relref "dpdk_performance" >}}) +- [TRex Performance]({{< relref "trex_performance" >}}) +- [VPP Device]({{< relref "vpp_device" >}}) + +## CSIT-2306 Release Data + +To access CSIT-2306 Release data please use following web resources: + +- [CSIT Per Release Performance](https://csit.fd.io/report/) + - `CSIT Release` > `rls2306` + - `DUT` > `vpp` + - `DUT Version` > `23.06-release` + - `Infra` > `testbed-nic-driver of choice` + - `Area` > `IPv4 Routing` `IPv4 Tunnels` `IPv6 Routing` `Hoststack` ... + - `Test` > `test of chioce` + - `Frame Size` > `64B` `78B` + - `Number of Cores` > `1C` `2C` `4C` + - `Test Type` > `MRR` `NDR` `PDR` +- [CSIT Per Release Comparisons](https://csit.fd.io/comparisons/) for VPP + v23.06 vs v23.02 + - `REFERENCE VALUE` + - `DUT` > `vpp` + - `CSIT and DUT version` > `rls2306-23.02-release` + - `Infra` > `testbed-nic-driver of choice` + - `Frame Size` > `64B` `78B` + - `Number of Cores` > `1C` `2C` `4C` + - `Measurement` > `Latency` `MRR` `NDR` `PDR` + - `COMPARED VALUE` + - `Parameter` > `Release and Version` + - `Value` > `rls2306-23.06-release` +- [CSIT Per Release Coverage Data](https://csit.fd.io/coverage/) + - `CSIT Release` > `rls2306` + +## CSIT-2306 Selected Performance Tests + +CSIT-2306 VPP v23.06 Performance Tests: + +- ip4 + - [2n-icx 100ge e810cq avf ip4scale20k](https://csit.fd.io/report/#eNrtVstuwjAQ_JpwqRbZG4f00gM0_4FcZylRQ3BtEwFfX4OQNlFbiUpQLj74pRlrxzsayT5sHS09tS9ZscjKRYZlU8cpy-dPcXGtx1zMoLcWMJ_GnaOWtCfADhqzBynEO6GV9CyF-QTdr6CxCmbqDaQBCuvTKQ5vdEsoPsB1NXS1O5XA10uJb_UYrXeB0ahihPTkGBzJY5pdHwac30QzXzvSfCEqZyiQH4j5-W3MXjm9Id8cia_ErjBuYuMZkmZcJxzsAL00rKzOjH9yySaX_uaSvZ9LmLJ0vUv4qCxhytKtXLpjllTK0vUuqUdlSaUs3colzlJRTbqt25z_ekX1BWCVro4) + - [2n-spr 100ge e810cq avf ip4scale20k](https://csit.fd.io/report/#eNrtVstOwzAQ_JpwQYvsjdNw4UDJfyDjbGlEmpq1iVS-HreqtIkAqUgtvfjgl2asHe9oJIe4ZXoO1D8U1bKolwXWXZumony8TQv3AUu1gNF7wPIu7Zh6soEABwieQSv1Sug13Wvl3sGOK-i8gYV5Ae2A4np_SiM42xOqN-ChhaHlfQl8Opb4Vk_Q9iMKmlTMkJFYwJk8ofn1bsL5TbTwLZOVC0m5QJHCRMzPbxP2iu2GQvdJciV1RXCXGi-QdvM6cecn6LFhdXNg_JNLPrv0N5f85VzCnKXTXcJrZQlzls7l0gWzZHKWTnfJXCtLJmfpXC5JlqrmZtjy5vDXq5ovlPOvWg) + - [2n-spr 100ge e810cq dpdk ip4scale20k](https://csit.fd.io/report/#eNrtVkFOwzAQfE24oEXOxmm4cGjJP5CxFxo1Tc3aVCqvx60qbSLgEKmlFx_i2JpdzXhHIznEHdNLoP6pqFdFsyqw6Vxaimp5n37cB6zUAvbeA1YPacfUkwkEOEDwDKVS74S-pMdS2Q9w3m2g8xoW-hVKCxTXx1P6gjU9odoADw4Gx0cOfD5z_CAU1H1GQZOMCbInFnCiT8r8-jCq-VO1NBgmIx1JukCRwkjN75eT6jc2WwrdF0lLGovgNo1eoNJOeeLBj9DzxJr2VPFfPvns00yf_BV9wpynGT7hzfKEOU8X8-maedI5TzN80jfLk855uphPkqe6vRt2vD29--r2GxR2tCI) + - [2n-spr 200ge cx7 mlx5 ip4scale20k](https://csit.fd.io/report/#eNrtVkFqwzAQfI17KVvstRyfemjqfwRV3jamsiNWqkny-iohsDZtD4akuehgWWJ2mdEOA_Jhx7TxZJ-zap3V6wzrro1LVr48xh9bj2W-gtE5wPIp7pgsaU-AA3jHgHn-QegKs69H0gF6u6-gcwpW6g0KAxS2p1P8vNGWMP8EHloYWj6R4OuF5AejoO1XEDTqmCEjsYAzgVLmtodJzd-ypUMzaWmJ2gUK5Cdyfr-dVL-z7sl3R5KWOBfBTRy-QIWZ84SDm6CXkdXNueLfnHLJqaVOuVs6hSlTS5zC-2UKU6au59RNM6VSppY4pe6XKZUydT2nJFNV8zDsuD-_AavmGw0uvhI) + - [2n-aws 50ge c5n.4xl ena dpdk ip4scale20k](https://csit.fd.io/report/#eNrtl01OwzAQhU8TNmhQ7NQNGxYtuUdl4oFGTd2RbVqV0-NElSaBChWpJSy8yJ_ei2Yyn55G8WHncOWxfcrUMiuXmSwbE09ZsbiPF9d6WeRz2BOBLB7incMWtUeQFvTBg8rfUJBAq8GQ2UBDMxBKPL6AqAHDunuOh691izLfgLMGrHFdBfl8qvCtHKvmPbAamxgpe3QsjrpjG62PA8_ZntmsHWp2x7ZZCugHnZz_MHa_Or1F33wgv9IPhR11HPtArMeVwpEG6mleZdU7_oYRJUa_YkQ3YyRTji5mJCfKkUw5uhKjW-RoPkvb6AuhOJL_s4t-5kOJz8R7qOOTttClfOQk-Ukb6Dp8OD-qurM7t-3_i1T1CXgFjgk) +- ip6 + - [2n-icx 100ge e810cq avf ip6scale20k](https://csit.fd.io/report/#eNrtVstqwzAQ_Br3UrZIazvOpYcm_o-iypvG1HHUlWKafn2VEFibtpBC0lx00IsZsaMdBuTDlunZU_eYlYusWmRYtU2csvzpPi7ceczVDAbnAPOHuGPqyHgC7KG1H6CVeiV0muZa2XcwwwpaN4Nq_gLaAoX14RSHt6YjVG_AfQN9w4cSuDyV-FZP0GYXBI0qJshALOBEntDcej_i_CZa-IbJyIWoXKBAfiTm57cJe8VmQ779JLkSuyK4jY0XSNtpnbB3I_TUsKo-Mv7JJZdc-ptL7nouYcrS-S7hrbKEKUuXcumKWSpSls53qbhVloqUpUu5JFkq67t-y5vjX6-svwD4Z68S) + - [2n-spr 100ge e810cq avf ip6scale20k](https://csit.fd.io/report/#eNrtVstqwzAQ_Br3UrZIazvOpYek_o-gypvG1HHUlWJIv75KCKxNW0ghaS466MWM2NEOA_Jhx7Ty1D1n5TKrlhlWbROnLF88xoU7j7maweAcYP4Ud0wdGU-APXjHoJV6I3Sa5lrZDzDDGlo3g2r-CtoChc3xFIe3piNU78B9A33DxxL4ci7xrZ6gzT4IGlVMkIFYwIk8obnNYcT5TbTwDZORC1G5QIH8SMzPbxP2ms2WfPtJciV2RXAbGy-QttM64eBG6LlhVX1i_JNLLrn0N5fc7VzClKXLXcJ7ZQlTlq7l0g2zVKQsXe5Sca8sFSlL13JJslTWD_2Ot6e_Xll_ASzUr94) + - [2n-spr 100ge e810cq dpdk ip6scale20k](https://csit.fd.io/report/#eNrtVkFOwzAQfE24oEX2Jml64UDJP5CxFxo1TRfbVCqvx60qbSLgEKmlFx_i2JpdzXhHIznEnaeXQP1jUa-KZlVg07m0FOXTffr5PmCpFrBnBiwf0s5TTyYQ4ACBPWil3glZ01Ir-wGO3QY6XkCzfAVtgeL6eEpfsKYnVBvwg4PB-SMHPp85fhAK6j6joEnGBNmTF3CiT8p4fRjV_KlaGownIx1JukCRwkjN75eT6jdvthS6L5KWNBbBbRq9QNpOeeKBR-h5Yk17qvgvnzj7NNMnvqJPmPM0wye8WZ4w5-liPl0zT1XO0wyfqpvlqcp5uphPkqe6vRt2fnt699XtN68etKY) + - [2n-spr 200ge cx7 mlx5 ip6scale20k](https://csit.fd.io/report/#eNrtVkFOwzAQfE24oEXJJqm5cGjJP5BxFhrhpNbaRC2vx60qbSLgEKmlFx_i2JpdzXhHI9mHHdOLJ_uU1ZtMbTJUXRuXrFzfxx9bj2W-gtE5wPIh7pgsaU-AA3jHgHn-TugKs1cj6QC93dfQuRWox1coDFDYHk_x80ZbwvwDeGhhaPlIgs9nkh-MgrafQdCoY4aMxALOBEqZ2x4mNX_Llg7NpKUlahcokJ_I-f12Uv3GuifffZG0xLkIbuLwBSrMnCcc3AQ9j0w1p4p_c8olp5Y65a7pFKZMLXEKb5cpTJm6nFNXzVSVMrXEqep2mapSpi7nlGSqbu6GHfenN2DdfAOqrL6W) + - [2n-aws 50ge c5n.4xl ena dpdk ip6scale20k](https://csit.fd.io/report/#eNrtV8tOwzAQ_JpwQYvijVxz4UDJfyATLzRq6lq2KWq_vm5UaRPooUgJ5eBDXpqJdrKj0Sghbj29BuqeCrks1LJA1Zp0Kqrn-3TxXcCqXMDOOcDqId156kgHArSgvwLI8oOEE2Q1GGfW0LoFCCke30A0QHF1ek5HaHRHWK7BWwPW-NMEfDlP-DGOUfMZGU0iRsiOPIMjdUxzq_2Ac1Ezk7Unzewkm6FIYaDk8ocx-93rDYX2QPxKvxRmNGntA7AZT4p7N0DP-1J1z_gbj1z26Fceudk8wpyjqz3CG-UIc44m8miOHKncRt8dUv-pi1Ruosn8mSs_uYWu9Qdvkp_cQNP4w_mR9Z3d-k3_XyTrI3nWjpE``) +- ipsec + - [3n-icx 100ge e810cq avf 40tnlsw](https://csit.fd.io/report/#eNrtmEtuwyAQQE_jbqqpbIzjbLpI6ntUBE8SJH8oUDfu6YvdSNiqKrVVsLtg449mgIGnJ43QplX4rLF6jLJ9lO8jkovSPqJ0d29fqtIkjTfQSQkkfbBfCitkGiFtQPALJHF8QiIT3CYxfwHWHYGrXpoWkizZHiDhgOYsJBVSI6exaSr9Bvb_MEwiGgMMNck2J15DU6phZfJ0XflLGS5avhoXtcXNIh0qF5xV7dLkuXc53-7F5TOFzA343KKLGtSTen66YTf-qFiNWryjm2Q8PpfBLaZJkM_XNr2cRK_nmBdjxrpMZWB6E6bSM1MSPPXAlKzqKQmeLs7Ut6c0eOqBKV3VUxo8XZypP09FLS6h7f0r0uH0_lvX-2uiQdKbEPXsaGh5PRAlazoaGt7FiXp2NLS7HojSNR0Nze7iRJ2jWXHXtKoe73qz4gMoCiw-) + - [3n-icx 100ge cx6 mlx5 40tnlsw](https://csit.fd.io/report/#eNrtmM1OhDAQgJ8GL2YMlMJ68eDKexi2jLtN-GnauoJPb8FNBmJM1GzBQy_8ZKbttF--ZFJjO43PBuuHKNtHu33EdrJyjyh9vHUvXRuWxjmclQKW3rkvjTWWBiFtQYoekjg-IlOJ6POqh6buMxB6ULaDJEvuD5AIQHuSiktlUPDYtrV5A_d_GCeRrYUSDcvyo2igrfS4Mnu6rPylDIpWr5airrhF5IyagouqKU2dBsr5di-UX2osacDnFilq0czq-emGafyLLhs08h1pkun4KEM4TLOgWK5tBzWLXs5xV0wZ2zJVgelVmCrPTFnw1ANTtqmnLHi6OlPfnvLgqQemfFNPefB0dab-PJWN7EPb-1ek4-n9t67310SDpFch6tnR0PJ6IMq2dDQ0vKsT9exoaHc9EOVbOhqa3dWJkqNZcdN2upnuerPiA78JNZ4) + - [3n-spr 100ge e810cq avf 40tnlsw](https://csit.fd.io/report/#eNrtmM1OhDAQgJ8GL2YMLWXZiwdX3sOUMrvbhJ_aVhSfXsBNCjEmarbgoRd-MtN22i9fMqmxrcYng9V9lB6i7BDRTJbDI0oeboeXrgxN4h10SgFN7oYvjRVyg5A0BRilgcTxCakiuCexeAbeHUHoXtkWSEr2BRABaM9SMakMChbbpjKvMPwX4yyyscDR0HR3EjU0pR6Xpo-Xpb_U4aLli3XRobpFpEPtgouyXZo69y7n-824AVwjdyM-9-iiFs2soJ_u2I0_al6jke_oJpnOz2WIAdQsKJZr217NopeDzPIpY2OqKlC9DlXlmyoNrvqgSrd1lQZX16fq3VUWXPVBlW3rKguurk_Vo6uylm-hBf4z1PH4_l0H_GumQdTrMPXtaWh_fTClm3oamt_1mfr2NLS-PpiyTT0Nje_6TJ2naX7TtLqe7oDT_ANkjjcG) + - [3n-spr 200ge cx7 mlx5 40tnlsw](https://csit.fd.io/report/#eNrtmM1OxCAQgJ-mXsyYlsLWi4dd-x6GpeMuSX8IYG19etu6CW1MjMZFPHDpT2aAgS9fMsHYTuOTwfohYYekOCSkkNX0SPL97fTStSF5uoNeKSD53fSlsUZuEPKWg1EaSJqekKhMDEWP3EJTDwyEHpXtIGPZ_REyAWjPUlGpDAqa2rY2rzD9H-d5ZGuBoyFsdxINtJWeFyePl8U_VeKi1Yt10am-TaRH7YKbwl2aOo8u56vtuCFcI3djPnbpohbNqqTv7tmNf9a8QSPf0E2ynKDLEBOsVVBs17ajWkUvR1mUS0ZwsiqSvRZZ5Z8sic76IUtCO0uisyHI_oGzNDrrhywN7SyNzoYg69VZ2cghtsa_ADsf4D_sjH_MNQp7La7-fY1tsR-uJLCvsSkOwdW_r7El9sOVBvY1NsQhuDpfWXnTdrpZ7oxZ-Q4xpFRe) +- hoststack quic + - [3n-icx 100ge e810cq dpdk ip4udpquic](https://csit.fd.io/report/#eNrlVctOwzAQ_JpwQYtsh5BeOLTkP5BjL8Sq27hep6J8PW5UsYmg1x6ai21pZvY1WplSH_Gd0L8W1aaoN4Wqnc1HUa4f8xU9qVK8wDEEUOVTfkX0qAmh3IMzXyCF-EQVJK6kMAewwW6h6ylR0mYLUq1EC9IApg5ceB5sOAzOtOcAOSSaroc20Dmfervk-5OcUTskRrN-hhwxMjirlWmhOzHnegcs0BE1K34bY0JCmtR0vU1WfES9Q3LfyLJxTMww2ZIJaObZ0ilM0Mv06mZk3Mg_Mtpj5nspaAE-_tfunfq5LDvv2c1lLeeNd7NqHvZ93I1_ZtX8ALrmCdg) + - [3n-icx 100ge cx6 mlx5 ip4udpquic](https://csit.fd.io/report/#eNrlVUFuwyAQfI17qbYCXMe99NDE_4gw3taoOKYsjpy-vsSKurbaXHNwLoA0M8zujhAU-4B7QveaFdus3GaqtE1asvztMW3BkcrFBo7eg8qf0imgQ00I-QGsGUEK8YHKSzNumhE6NxbQ9hQpavMJUr2IGqQBjC1Y_zw0_muwpj7r041o2h5qT2c7tbvY_fFmtBkio0m_QI4YGFyUyjTfnphztQHm64CaBb99MSEizUq63iUr3oPukOw3smyaEjNMCmQGmqVbPPkZehleWU2M26RHRjuUwjgpaP0p_tftOtO8qzBXnOVdPcwbv8uiejj0oZv-yqL6AfuHCcg) + - [3n-spr 100ge e810cq dpdk ip4udpquic](https://csit.fd.io/report/#eNrlVctugzAQ_Bp6qbayTSm59NCU_4iMvS0oJGy8JlLy9XVQ1AX1ccwhXGxLM7Ov0coc-4Abxu41K9ZZuc5M2fp0ZPnbY7pCxyZXL3AkApM_pVfADi0j5PsamAJopT7RkMaVVu4AnvwWmp4jR-u2oM1K1aAdYGygpefB02FoXX2JkGKia3qoiS8Jzfs14Y_sgvohCpr0M-SIQcBZsUKj5iScf1oQhQ1oRfLdmRAi8qSov_sUxUewO-T2jCIb5yQMl0yZgG6eLZ5ogl7HV1Yj41YOsrMdJn6nFS_Byd_6vVdHF2boXfu5sAW98X4W1cO-D7vx7yyqL3lpDXA) + - [3n-spr 200ge cx7 mlx5 ip4udpquic](https://csit.fd.io/report/#eNrlVctugzAQ_Bp6qbayTQi99JCU_4iM2RZUE1yvQUm-Pg6KsqA-jjmEi21pZvY1WplC53FHaN-SbJvk20TlTRWPJN08x8tbUqlYw-AcqPQlvjxa1ISQ7jWQ86CE-ETlpDnkA-oArT1kUHcUKGjzBVK9ihKkAQw1NG7VV-67b0x5CRGDoqk7KB1dMqr3a8Yf6Rmt-sBo1M-QAT2Ds2qZ5uojc_7rgSXao2bNrTUmBKRJVX83yooPr1uk5oQsGwfFDBNtmYBmni0c3QS9zi8vRsbdPCSjLUphrBS0CC9_a_hhPV2apY_t6NKW9M47mhVP-8634x-aFWchkBQI) +- hoststack tcp udp + - [3n-icx 100ge e810cq dpdk ip4tcp ipudp](https://csit.fd.io/report/#eNrlVctOwzAQ_JpwQYv8aBouHCj5D-TYWxLVbYzXrShfjxtVbCJUeoJDc7Etz4x3xyPLlPqIr4T-qShXRbUqVNW5PBT6-T5P0ZPSYgmHEEDph7yK6NEQgt5BZz9ACvGGKkh8lMK-gwtuA21PiZKxG5CLpWhAWsDUQhcWyYbmJPYu5HN64_ImxrWGJtCpqHo5F_3RAaNunxjNfU2QA0YGJw0zLbRH5ly2wQIT0bDi2x0TEtKopyteWbaOZovUfSJrhwtjhs3hjEA7LZmOYYSer7CqB8Y_JEnWeJTWS0FzCfQ3y7eR697N54Ve8nozSc7thV6x_Pe5lvXdro_b4S8t6y_g7REe) + - [3n-icx 100ge cx6 mlx5 ip4tcp ipudp](https://csit.fd.io/report/#eNrlVcFuwyAM_ZrsMnkC0iSnHdblPyYC7hKNNAjTKt3Xj0bVnGjqeuqluQDiPWM_P1lQHAJ-ELrXrNhm1TZTVWfTkuVvz2kLjlQuSjh6Dyp_SaeADjUh5HvozAhSiE9UXpqxtCP0biygHShS1OYL5KYUDUgDGFvo_CYa35xjnfXpmUHbdIlhl0Pj6ZxTvV9y_imAUXuIjKayFsgRA4OLepnm2xNzrqpgvg6oOeBXHBMi0qykG1I5bBd0j9R9I8dO_WKGSdbMQLNMGU9-hl46WNUT4_4-ktEOpXFS0Ers_E_xQ7h6sKuZzmtSH8XHlU3nDcX3d7Won_ZD6Kc_tKh_AANWEQ4) + - [3n-spr 100ge e810cq dpdk ip4tcp ipudp](https://csit.fd.io/report/#eNrlVUFuwyAQfI17qbYCO4576SGp_1Fh2NRWSLxlSaT09SVW1LVVNTm1h_gCiJlhdxghOPYB3xj9S1aus2qd5VXn0pAVq8c0Bc95oZZwJIK8eEqrgB4NIxT7BpgCaKXeMSeNz1rZD3DkttD2HDkauwW9WKoGtAWMLXS0iJaas9o7Sgf1xqVNDJsCGuJz1fz1UvVHC4K6QxQ0NTZBjhgEnHQsNGpPwrniQxQmoBHJtz0hRORRUzfMimwTzA65-0TRDjcmDJviGYF2WjKeaIRe7rCqB8Z_ZMnWeNTWa8WzifSa5ztJ9uBm9Ep_M3s_Wc7uld7w_PfJlvXDvg-74U8t6y_oFxS2) + - [3n-spr 200ge cx7 mlx5 ip4tcp ipudp](https://csit.fd.io/report/#eNrlVUFuwyAQfI17qbbCEMenHpr6HxWGTW0Vx4glVpLXh1hR11bV5NQe4gsgZobdYYSg2Af8IHSvWbHJyk0my9amIVNvz2kKjqQSaxi8B6le0iqgQ00IaqeBfAApxCdKn5tDOaCO0LlDAU1PkaI2X5Cv1qKG3ADGBlq_isbXF7mzPp3Ua5s2MWwV1J4uZeX7teyPHhi1-8ho6myGDBgYnLXMNN8cmXPLCEt0QM2ab39MiEiTru64Zdk26A6pPSFrxytjhkkBTUAzLxmPfoJeL7GsRsa_pElGO8yNywUtJ9Rbph8l271d0kv9ze0Dpbm8l3rH9N9nW1RPuz50499aVGf85RtO) +- nat44 + - [2n-icx 100ge e810cq avf ethip4tcp tput](https://csit.fd.io/report/#eNrtVl1PhDAQ_DX4Ytb0gxZ88MGT_2G4sickd9za9ojnr7ecJIUoiYlGI-GllM4su9PJJDh_tPjocH-XqE2SbRKRNVVYEnl_HR5274RkGjoiEPIm7CzusXQIooXGvABn7AkFccw5M89QdjtoKO2Pt8ANoK_DqzfUn277upozkQJpCU6nigvwdPLQVrZvKh6Gph8miGh18hENc02QDm0EJwNHGtXnEWdORuSXFstYEHREyKMbDfNFsbF8Z8sDuuYV4zf6i4sEE8wZYWba2Z9phA5XmBUXxp85SauT33WSft9JLfNBHWdS8Nulx_JzvctJ5oyftPr5L_OplZL6XV_KRZ7pfNn5nNG7mHzO-Umrnz-aT1VctUd7uPzbquINdw_s1g) + - [2n-spr 100ge e810cq avf ethip4tcp-nat44ed tput](https://csit.fd.io/report/#eNrtVl1PhDAQ_DX4Ytb0gxZ88MGT_2Eq7AkJx61tj-T89ZaTpBAlMdFoJLwAZabdnZ1MUuePFh8dtneJ2iXZLhFZU4VHIu-vw8u2TkimoScCIW_Cl8UWjUMQHTiywBl7RkEcc87KFzD9HhpKh99PwEtAX4elLwk649MUK6g5EymQluB0qrgATycPXWWHouJhLPqhg4hWJx_R0NcM6dFGcNZwpFF9nnCWZES-sWjihiAnQh7dpJkvio3b99Yc0DWvGM8YBhcJZTBngpXzyv5ME3QcYVZcGH_mJG1OftdJ-n0ntcxHdZxJwW_XHsvP9a4nmQt-0ubnv8ynVkrqd30pF3mm83Xnc0HvavK55Cdtfv5oPlVx1R3t4XK3VcUbnCbrMg) + - [2n-spr 100ge e810cq avf ethip4tcp-nat44ed cps](https://csit.fd.io/report/#eNrtlt1KxDAQhZ-m3shI_lsv9sK17yGxnbWFbndMYmF9euNaSItdEBTFsjdNy5l05uTjQHw4OHzw2G0yvc3ybSbyto6PTN5dx8V1XkhmYCACIW_im8MOrUcQPXhywBl7QkEcC86qZ7DDDlpSYNQj8AowNPErVAS9DUphDQ1nQgEZCd4ozQVU5KGv3XtLcT-2_NQ_qfVLSGqcaqYM6JI4GzeVUXOc1JwzkeqtQ5s2RDdJCugnw3zNa9q9c3aPvn3F9It4akmvIpgk8WreNxxpoo4HmJenij-iSBeK36NIv07RyGK0xpkU_HblcVy0u5ZELrOkC8v_l0ujtTQf5hQXRW6KVedy2e5KcnmGJV1Y_mAudXnVH9z-dIfV5RtGI-MK) + - [2n-spr 200ge cx7 mlx5 ethip4tcp-nat44ed tput](https://csit.fd.io/report/#eNrtlt1qhDAQhZ_G3pQp-THRXvSiW9-jpDqtgrpDkpXdPn2zWyFKKxRaWCre-MOZcebk44DO7y0-O2wfErVLsl0isqYKl0Q-3oabbZ2QTMNABELehSeLLRqHIHpwZEEw9oaCeHnMBjQeuvaooKEUOGMvwEtAX4dXXxL0xqcpVlBzJlIgLcHpVHEBng4e-sqex4qnceyXHaJaHXxUw2YzZUAbxdnKsYzq06Rm2UjsMBZNbAmGouTRTdb5od3Y_mpNh655x_iN89HFgjIAmmjlfLI_0UQdDzErLhVXpEkbzd_TpGvQ1DIf_XEmBb9ffzy_d7ymhC4wpY3pv82pVkrqT4cpF3mm87XndMHxinK6xJQ2pn-cU1Xc9HvbXf55VfEBRW_56g) +- tunnels (gnv, vxlan, gtpu) + - [2n-icx 100ge e810cq avf ethip4udpgeneve](https://csit.fd.io/report/#eNrtmFFrgzAQxz-Nexk3TIxxL3to5_coVq-tYNMsidL20y8W4XRjsIexQtIXI_7vcnf580OIdSeDG4vdW5Kvk2Kd8KJt_CPJVs9-MZ3lWSph0Bp49uLfDHZYWQSuoK3PwNJ0j1wzfGVp_QHVsINWi43rlcLOghRbYDWgO_ivMK19o_eocEBg0geOCdtxS9WYsTJ_nyp_a4PUpnek-uYWyoCGxEXXFKYPl1nMT7NQfGWwooTZiBTi0M6a-tXIlLwz1RFte0XawZ8d6bW3iSRWL8u6i56p0zkW5S3ivp7qh6d_4qm-r6fxYRo6pfFBGjijPI_vZ7qcOUROv7iqH66GwKqIjlQROKciOkpF4IzK-CCVoVMq48NU_ienefmkTuZ4u0vKy0_aCie_) + - [3n-icx 100ge cx6 mlx5 gtpu sw](https://csit.fd.io/report/#eNrtlkFugzAQRU9DN9VUYAysumjCPSJipgmScUa2SUlPXxNFGlDbRao02XiDkf-3Z-ynL9n5g8WNQ_2aFKukWiWi6trwSfK35zBY7USelnAkApG_hD-LGhuHkBvo1AhZmu5QUKbGsh2h12MBHcmNH4xB7aCUW8gUoN-H2Z2nwX1M-nbawbR2KiTWl0LfqrLaDp7V0MtCOaJlcdEk22h_Ys-vrbO_sdjwgtmJ2OLRzZr66YTsfbdNj677RF4QboZ1FSCwlKllFX-imXq5tqo-O-5KjCKxvxCj_ycmYsauIyYenTERM3ZLYnfImIwZu46YfHTGZMzYLYlxxor6yRxsf34zFvUXjV3cHg) + - [3n-spr 200ge cx7 mlx5 vxlan](https://csit.fd.io/report/#eNrtVcFuwjAM_Zrugjy1KaUnDrD-B0pTDyqlwXJCVfb1BKjkVtN24YCEuCRRnl_s5ycrPhwZdx7tOim2SblNVNk2cUnyzSJubL3K0xX0RKDyz3hitKg9Qu40eGJQabpHRZkZyh51gM4OBbS03IWTc2g9rJY1ZAYwHOJtP1jtwKq6qeMjnTaWHbiGrxnV15jxV3pBm1MQNBY1Q3pkAWfVShgdzhLznwahaEYtnIk0CQnoJ3X9LVUY36w79O0PCi02SnATbREoM_Nc4UwTdOxfWd0inuMhvT18yEN6joeDuQp78Qm8i3yp2Rt9o7dvD89bUX24I3e3_6-oLg1S_ds) + - [3n-spr 200ge cx7 mlx5 gtpu sw](https://csit.fd.io/report/#eNrtlsGKgzAQhp_GvSyzaIz1tIft-h4l1dlWiOmQRLft0zeWwii7LHQp0kMuRvz_cSb5-CHOHyxuHOr3pFgn5ToRZduER5J_vIbFaifydAUDEYj8LbxZ1KgcQm4UOLIg0nSHgrL6WA6oPHT6WEBLcuN7Y1A7WMktZDWg34evO0-9-x717fgT09ixl_i89frRmNWm96yGcWbKgJbF2Zxso_2JPX9NzyXKouKayabY4tFN5vptk-z9sqpD156RC8LhsF4HFCxl9byLP9FEvZ1cWV0dS3OjyO2f3GgRbiLm7W5u4gnyJmLeHsxtmbzJmLe7ucknyJuMeXswN85bUb2Yg-2u98qiugCXaut-) + - [3n-spr 200ge cx7 mlx5 wireguard](https://csit.fd.io/report/#eNrtl0FugzAQRU9DN5Ur24TCpouk3CNy8DRBMo41NiTp6WNQpAFVqrqoGinuBhD_DzPjr7fAhyPC1oN5y4pNVm4yWbY6XrJ8_RxvaLzM-SsbnGMyf4lPCAaUB5ZbxbxDJjnfg3SiOZcDqMA6cy5Y61bb0FsLxjNRiGrHRMMgHOL7XrtTi7DvFWrBOQ_W-NNYsBu_ajWOzeX7rfmXSUjVfSA1zrdQBkASF4OTzR0u5PluHSpRCIpqZluSJYCfzfWDnan0A1UHvv0Eqp8OjxxNDGsmNsu24eJm6u0oy3py3D1Z95_sbyXr7pxsgsgmQWyCwCbAa3K0JsBqcqQ-PqdVapxWj89plRqn1R9zWtRP9ojd9O9a1FdcSTlU) +- reassembly + - [3n-icx 100ge e810cq dpdk reassembly](https://csit.fd.io/report/#eNrtVkFugzAQfA29VFthAyGXHpLyjwrMNrEKxvW6aenr69BIC6p6qUTCgYttecar2R2NZPKdw2fC5jHK9lG-j2Su67BEye4-bK4hmcQbOFkLMnkIJ4cNloSQGNDqE0QcH1BagVsRqzeobf0KyvXWdyAysa1AKEB_1DbVllAFeuxNQx8QbqpzHW08lEgy2xxUG6qXRNhWTQ-mdmcd8umi45coRut3z2iQOkFO6Bic9MA0e-yZ83dn_KAMQvnFT8OMeqSRoH-1z8VeXNki6S_kisNkmaGChSNQTYX43o7Qy1TzYmAsyW-7-j2D33Zuv-Wa7-v6LW-bb7nme0l-z57vdM33df1Ob5vvdM33kvzmfGfFnelcO_zTs-IbpihUvQ) + +## CSIT-2306 Selected Performance Comparisons + +Comparisons 23.06 vs 23.02 +- [2n-icx 100ge e810cq avf 1c 64B PDR](https://csit.fd.io/comparisons/#eNqNkEsOgjAQhk-DG1JTioAbFyoHMMYLNGUwJFDqtBD19LY8LCQuTJq0M_83j_4aahAGikOQnQKWIZSAIAXYdxAfwzGrwUzxDTsrncMh6SqrVq7QoptRe_VKuWDkrdIDehFrzWKaEhZvKSNou3ENHq9kidzTTJJKPElE6R2YimAfUfEgvC99hWjxu3bidonOLkzySS-x0dV7jaS704ox5qXWxCW_TkQ2nHlWozj-4ZKleANm-XFvxIj0vO7gty_p0pdh_ka22Bzcakn-AeXRdNc) + +## CSIT-2306 Selected Performance Coverage Data + +CSIT-2306 VPP v23.06 coverage data +- [2n-icx 200ge cx7 mlx5 ip4](https://csit.fd.io/coverage/#eNpVjsEOwiAQRL8GLwaDi5RTD9b-hyG4sSRICSDSv29JD9TLJjNvZjMRLepkZtcTORCQwcbtEn4_7wI466oBj2q8vqnR7P0fyRgaBH5hHQ3bdxWxxfy0HDKOGl0oMPZG8FddZEaV6McW0RoqoGoV429VyPEUp_n3tCqh00tPRN1-tGpKjCsrcD2n) + +## Further Information + +For further information including instructions how to access the needed +information with user selectable options, please refer to +[csit.fd.io documentation]({{< relref "/" >}}). diff --git a/docs/content/release_notes/previous/csit_rls2306/dpdk_performance.md b/docs/content/release_notes/previous/csit_rls2306/dpdk_performance.md new file mode 100644 index 0000000000..7d15a50dd8 --- /dev/null +++ b/docs/content/release_notes/previous/csit_rls2306/dpdk_performance.md @@ -0,0 +1,38 @@ +--- +title: "DPDK Performance" +weight: 2 +--- + +# CSIT 23.06 - DPDK Performance + +1. TEST FRAMEWORK + - **CSIT test environment** version has been updated to ver. 12, see + [Environment Versioning]({{< ref "../../../infrastructure/fdio_csit_testbed_versioning" >}}). +2. DPDK PERFORMANCE TESTS + - Added support for new NICs. +3. DPDK RELEASE VERSION CHANGE + - Version 23.03 is now tested. + +# Known Issues + +List of known issues in CSIT 23.06 for DPDK performance tests: + +## New + +List of new issues in CSIT 23.06 for DPDK performance tests: + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | [CSIT-1904](https://jira.fd.io/browse/CSIT-1904) | DPDK 23.03 testpmd starup fails on some testbeds. Different cause but the same consequences as CSIT-1848. + +## Previous + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | | + +## Fixed + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | [CSIT-1848](https://jira.fd.io/browse/CSIT-1848) | 3n-alt: testpmd tests fail due DUT-DUT link taking long to come up. Fixed for 3n-alt on infra level, but reapeared after DPDK bump as CSIT-1904 on multiple more platforms. diff --git a/docs/content/release_notes/previous/csit_rls2306/trex_performance.md b/docs/content/release_notes/previous/csit_rls2306/trex_performance.md new file mode 100644 index 0000000000..05f8a5615f --- /dev/null +++ b/docs/content/release_notes/previous/csit_rls2306/trex_performance.md @@ -0,0 +1,40 @@ +--- +title: "TRex Performance" +weight: 3 +--- + +# CSIT 23.06 - TRex Performance + +1. TEST FRAMEWORK + - **CSIT test environment** version has been updated to ver. 12, see + [Environment Versioning]({{< ref "../../../infrastructure/fdio_csit_testbed_versioning" >}}). +2. TREX TESTS + - No longer testing scale2m, testing scale20k instead (for AWS reasons). +3. TREX VERSION + - Currently using v3.03 of TRex. + +# Known Issues + +## New + +List of new issues in CSIT 23.06 for TRex performance tests: + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | | + +## Previous + +List of known issues in CSIT 23.06 for TRex performance tests + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | | + +## Fixed + +List of known issues in CSIT 23.02 for TRex performance tests + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | [CSIT-1876](https://jira.fd.io/browse/CSIT-1876) | 1n-aws: TRex NDR PDR ALL IP4 scale and L2 scale tests failing with 50% packet loss. Fixed for most scales. Only ip4scale2m still fails, but we removed that from jobspecs. diff --git a/docs/content/release_notes/previous/csit_rls2306/vpp_device.md b/docs/content/release_notes/previous/csit_rls2306/vpp_device.md new file mode 100644 index 0000000000..76d47c5c4b --- /dev/null +++ b/docs/content/release_notes/previous/csit_rls2306/vpp_device.md @@ -0,0 +1,26 @@ +--- +title: "VPP Device" +weight: 4 +--- + +# CSIT 23.06 - VPP Device + +1. TEST FRAMEWORK + - **CSIT test environment** version has been updated to ver. 12, see + [Environment Versioning]({{< ref "../../../infrastructure/fdio_csit_testbed_versioning" >}}). + +# Known Issues + +List of known issues in CSIT 23.06 for VPP functional tests in VPP Device: + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | | + +## New + +List of new issues in CSIT 23.06 for VPP functional tests in VPP Device: + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | | diff --git a/docs/content/release_notes/previous/csit_rls2306/vpp_performance.md b/docs/content/release_notes/previous/csit_rls2306/vpp_performance.md new file mode 100644 index 0000000000..88bd316e1f --- /dev/null +++ b/docs/content/release_notes/previous/csit_rls2306/vpp_performance.md @@ -0,0 +1,92 @@ +--- +title: "VPP Performance" +weight: 1 +--- + +# CSIT 23.06 - VPP Performance + +1. TEST FRAMEWORK + - **CSIT test environment** version has been updated to ver. 12, see + [Environment Versioning]({{< ref "../../../infrastructure/fdio_csit_testbed_versioning" >}}). +2. VPP PERFORMANCE TESTS + - Added tests for IP packet reassembly (using IPsec or GTPU). + - Existing IPsec fastpath tests converted to apply optimizations also + on inbound (previously only on outbound). + +# Known Issues + +These are issues that cause test failures +or otherwise limit usefulness of CSIT testing. + +## New + +Any issue listed here may have been present also in a previous release, +but was not detected/recognized/reported enough back then. + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | [CSIT-1892](https://jira.fd.io/browse/CSIT-1892) | 3n-alt: two-band structure of ipsec and vxlan. + 2 | [CSIT-1896](https://jira.fd.io/browse/CSIT-1896) | Depending on topology, l3fwd avoids dut-dut link. + 3 | [CSIT-1901](https://jira.fd.io/browse/CSIT-1901) | 3n-icx: negative ipackets on TB38 AVF 4c l2patch. + 4 | [CSIT-1904](https://jira.fd.io/browse/CSIT-1904) | DPDK 23.03 testpmd startup fails on some testbeds. + 5 | [CSIT-1906](https://jira.fd.io/browse/CSIT-1906) | Zero traffic with cx7 rdma. Cause not know yet, trending uses mlx5-core for cx7 and cx6. + 6 | [VPP-2077](https://jira.fd.io/browse/VPP-2077) | IP fragmentation: running_fragment_id is not thread safe. Causes reduced performance and failures in gtpu reassembly tests. + 7 | [CSIT-1914](https://jira.fd.io/browse/CSIT-1914) | TRex does not produce latency data on ICE NICs. + 8 | [CSIT-1915](https://jira.fd.io/browse/CSIT-1915) | 2n-icx testbeds to not have the same performance + 9 | [CSIT-1916](https://jira.fd.io/browse/CSIT-1916) | Poor CPU scaling on 2n-zn2 RDMA. + 10 | [CSIT-1917](https://jira.fd.io/browse/CSIT-1917) | TRex STL performance is unstable at high pps due to unsent packets. + 11 | [CSIT-1922](https://jira.fd.io/browse/CSIT-1922) | 2n-tx2: af_xdp mrr failures. + 12 | [CSIT-1923](https://jira.fd.io/browse/CSIT-1923) | 3n-icx, 3n-snr: first few swasync scheduler tests timing out in runtime stat. + 13 | [CSIT-1924](https://jira.fd.io/browse/CSIT-1924) | l3fwd error in 200Ge2P1Cx7Veat-Mlx5 test with 9000B. + +## Previous + +Issues reported in previous releases which still affect the current results. + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | [CSIT-1782](https://jira.fd.io/browse/CSIT-1782) | Multicore AVF tests are failing when trying to create interface. Frequency is reduced by CSIT workaround, but occasional failures do still happen. + 2 | [CSIT-1785](https://jira.fd.io/browse/CSIT-1785) | NAT44ED tests failing to establish all TCP sessions. At least for max scale, in allotted time (limited by session 500s timeout) due to worse slow path performance than previously measured and calibrated for. CSIT removed the max scale NAT tests to avoid this issue. + 3 | [CSIT-1795](https://jira.fd.io/browse/CSIT-1795) | Ocassionally not all DET44 sessions have been established: 4128767 != 4128768 + 4 | [CSIT-1800](https://jira.fd.io/browse/CSIT-1800) | All Geneve L3 mode scale tests (1024 tunnels) are failing. + 5 | [CSIT-1801](https://jira.fd.io/browse/CSIT-1801) | 9000B payload frames not forwarded over tunnels due to violating supported Max Frame Size (VxLAN, LISP, SRv6) + 6 | [CSIT-1802](https://jira.fd.io/browse/CSIT-1802) | all testbeds: AF-XDP - NDR tests failing from time to time. + 7 | [CSIT-1804](https://jira.fd.io/browse/CSIT-1804) | All testbeds: NDR tests failing from time to time. + 8 | [CSIT-1808](https://jira.fd.io/browse/CSIT-1808) | All tests with 9000B payload frames not forwarded over memif interfaces. + 9 | [CSIT-1809](https://jira.fd.io/browse/CSIT-1809) | All tests with 9000B payload frames not forwarded over vhost-user interfaces. + 10 | [CSIT-1827](https://jira.fd.io/browse/CSIT-1827) | 3n-icx, 3n-skx: all AVF crypto tests sporadically fail. 1518B with no traffic, IMIX with excessive packet loss + 11 | [CSIT-1849](https://jira.fd.io/browse/CSIT-1849) | 2n-skx, 2n-clx, 2n-icx: UDP 16m TPUT tests fail to create all sessions. + 12 | [CSIT-1864](https://jira.fd.io/browse/CSIT-1864) | 2n-clx: half of the packets lost on PDR tests. + 13 | [CSIT-1881](https://jira.fd.io/browse/CSIT-1881) | 2n-icx: NFV density tests ocassionally breaks VPP which fails to start. + 14 | [CSIT-1883](https://jira.fd.io/browse/CSIT-1883) | 3n-snr: All hwasync wireguard tests failing when trying to verify device. + 15 | [CSIT-1884](https://jira.fd.io/browse/CSIT-1884) | 2n-clx, 2n-icx: All NAT44DET NDR PDR IMIX over 1M sessions BIDIR tests failing to create enough sessions. + 16 | [CSIT-1885](https://jira.fd.io/browse/CSIT-1885) | 3n-icx: 9000b ip4 ip6 l2 NDRPDR AVF tests are failing to forward traffic. + 17 | [CSIT-1886](https://jira.fd.io/browse/CSIT-1886) | 3n-icx: Wireguard tests with 100 and more tunnels are failing PDR criteria. + +## Fixed + +Issues reported in previous releases which were fixed in this release: + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | [CSIT-1799](https://jira.fd.io/browse/CSIT-1799) | All NAT44-ED 16M sessions CPS scale tests fail while setting NAT44 address range. + 2 | [CSIT-1890](https://jira.fd.io/browse/CSIT-1890) | 3n-alt: Tests failing until 40Ge Interface comes up. The fix for CSIT-1848 was enough to prevent this from happening. + 3 | [CSIT-1835](https://jira.fd.io/browse/CSIT-1835) | 3n-icx: QUIC vppecho BPS tests failing on timeout when checking hoststack finished. + 4 | [CSIT-1877](https://jira.fd.io/browse/CSIT-1877) | 3n-tsh: all VM tests failing to boot VM. Fixed by increasing the timeout. + +# Root Cause Analysis for Regressions + +List of RCAs in CSIT 23.06 for VPP performance regressions. + +Contrary to issues, these regressions do not limit usefulness of CSIT testing. +So even if they are not fixed (e.g. when the regression is an expected +consequence of added functionality), they will not be re-listed in the next +release report. + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | [CSIT-1912](https://jira.fd.io/browse/CSIT-1912) | trending regression: l2scale around 2023-03-18. + 2 | [CSIT-1918](https://jira.fd.io/browse/CSIT-1918) | summarize performance consequences of ipsec changes. + 3 | [CSIT-1919](https://jira.fd.io/browse/CSIT-1919) | rls2306: find cause of wireguard regression. + 4 | [CSIT-1920](https://jira.fd.io/browse/CSIT-1920) | find cause of zn2 mlx5 memif regression near 2023-04-11. + 5 | [CSIT-1921](https://jira.fd.io/browse/CSIT-1921) | investigate two-band structure in SRv6. diff --git a/docs/content/release_notes/previous/csit_rls2310/_index.md b/docs/content/release_notes/previous/csit_rls2310/_index.md new file mode 100644 index 0000000000..6be4f8811e --- /dev/null +++ b/docs/content/release_notes/previous/csit_rls2310/_index.md @@ -0,0 +1,109 @@ +--- +bookCollapseSection: true +bookFlatSection: false +title: "CSIT rls2310" +weight: 1 +--- + +# CSIT-2310 Release Report + +This section includes release notes for FD.io CSIT-2310. The CSIT report +was published on **Nov-08 2023**. The release plan is published on +[CSIT wiki](https://wiki.fd.io/view/CSIT/csit2310_plan) pages. + +## CSIT-2310 Release Notes + +- [VPP Performance]({{< relref "vpp_performance" >}}) +- [DPDK Performance]({{< relref "dpdk_performance" >}}) +- [TRex Performance]({{< relref "trex_performance" >}}) +- [VPP Device]({{< relref "vpp_device" >}}) + +## CSIT-2310 Release Data + +To access CSIT-2310 Release data please use following web resources: + +- [CSIT Per Release Performance](https://csit.fd.io/report/) + - `CSIT Release` > `rls2310` + - `DUT` > `vpp` + - `DUT Version` > `23.10-release` + - `Infra` > `testbed-nic-driver of choice` + - `Area` > `IPv4 Routing` `IPv4 Tunnels` `IPv6 Routing` `Hoststack` ... + - `Test` > `test of chioce` + - `Frame Size` > `64B` `78B` + - `Number of Cores` > `1C` `2C` `4C` + - `Test Type` > `MRR` `NDR` `PDR` +- [CSIT Per Release Comparisons](https://csit.fd.io/comparisons/) for VPP + v23.10 vs v23.06 + - `REFERENCE VALUE` + - `DUT` > `vpp` + - `CSIT and DUT version` > `rls2310-23.06-release` + - `Infra` > `testbed-nic-driver of choice` + - `Frame Size` > `64B` `78B` + - `Number of Cores` > `1C` `2C` `4C` + - `Measurement` > `Latency` `MRR` `NDR` `PDR` + - `COMPARED VALUE` + - `Parameter` > `Release and Version` + - `Value` > `rls2310-23.10-release` +- [CSIT Per Release Coverage Data](https://csit.fd.io/coverage/) + - `CSIT Release` > `rls2310` + +## CSIT-2310 Selected Performance Tests + +CSIT-2310 VPP v23.10 Performance Tests: + +- ip4 + - [2n-icx 100ge e810cq avf ip4scale20k-rnd](https://csit.fd.io/report/#eNrtVstOwzAQ_JpwQYvsjUO4cKDkP5BxtjQiTc3aRJSvx60qbSJAKlJLLz74pRlrxzsaySFumJ4C9fdFtSjqRYF116apKB-u08J9wFIrGL0HLG_SjqknGwhwgM59gFbqhdBrutPKvYEdl9B5A7fmGbQDiqvdKY3gbE-oXoGHFoaWdyXw8VDiWz1B2_coaFIxQ0ZiAWfyhOZX2wnnN9HCt0xWLiTlAkUKEzE_v03YS7ZrCt0nyZXUFcFdarxA2s3rxK2foIeG1c2e8U8u-ezS31zy53MJc5aOdwkvlSXMWTqVS2fMkslZOt4lc6ksmZylU7kkWaqaq2HD6_1fr2q-ANRarhY) + - [2n-spr 100ge e810cq avf ip4scale20k-rnd](https://csit.fd.io/report/#eNrtVstOwzAQ_JpwQYvsjUO4cGjJfyDjbGlEmpq1iVS-HreqtIkAqUgtvfjgl2asHe9oJIe4ZXoO1D8W1bKolwXWXZumolzcpoX7gKVWMHoPWN6lHVNPNhDgAMEzaKVeCb2mB63cO9hxBZ03cG9eQDuguN6f0gjO9oTqDXhoYWh5XwKfjiW-1RO0_YiCJhUzZCQWcCZPaH69m3B-Ey18y2TlQlIuUKQwEfPz24S9Yruh0H2SXEldEdylxguk3bxO3PkJemxY3RwY_-SSzy79zSV_OZcwZ-l0l_BaWcKcpXO5dMEsmZyl010y18qSyVk6l0uSpaq5Gba8Ofz1quYLCMeu4g) + - [2n-spr 100ge e810cq dpdk ip4scale20k-rnd](https://csit.fd.io/report/#eNrtVkFOwzAQfE24oEX2xiFcONDmH8jYC42apmbtViqvx60qbSLgEKmllxzi2JpdzXhHIzmmLdNrpO65qBZFvSiwbn1eivLlPv-4i1hqBfsQAMuHvGPqyEYC7CEGBq3UB2HQ9KSV-wQf_BraYODRvIF2QGl1POUvOtsRqjVw76H3fOTA5ZnjB6GgfpcEzTJGyJ5YwJE-KQurw6DmT9XSYJmsdGTpAiWKAzW_X06q39luKLZfJC15LIK7PHqBtBvzpEMYoOeJ1c2p4r98CrNPE30KV_QJ5zxN8Alvliec83Qxn66ZJzPnaYJP5mZ5MnOeLuaT5Klq7votb07vvqr5BoUvs6o) + - [2n-spr 200ge cx7 mlx5 ip4scale20k-rnd](https://csit.fd.io/report/#eNrtVkFqwzAQfI17KVvstVyfemjifxRV3jamsiNWqkny-ighsDZtD4akuehgWWJ2mdEOA_Jhy_Tmyb5k1SqrVxnWXRuXrHx9jD-2Hssih9E5wPIp7pgsaU-AA3jHgHn-SegKs6tH0gF6u6ugcwqe1TsUBihsTqf4eaMtYf4FPLQwtHwiwfWF5AejoO13EDTqmCEjsYAzgVLmNvtJzd-ypUMzaWmJ2gUK5Cdyfr-dVH-w7sl3B5KWOBfBTRy-QIWZ84S9m6CXkdXNueLfnHLJqaVOuVs6hSlTS5zC-2UKU6au59RNM6VSppY4pe6XKZUydT2nJFNV8zBsuT-_AavmCHrbvZo) + - [2n-c6in 200ge c6in.4xl ena dpdk ip4scale20k-rnd](https://csit.fd.io/report/#eNrtl91qwzAMhZ8muxkasdI0u9lFu7zH8GxtDU1dYWeF7unnhIISuo0V2rUXvsgf5wQp-jiIhG7r6SVQ-5SVy6xaZlg1Np6yYnEfL74NWKgcdsyAxUO889SSDgTowMwbB5jn76RYkdNg2a6h4RmoUj2-gjJA3ap_jkcwuiXM1-CdBWd9XwOfDzWOCopqPzpRYxsTZUdexEl_YuPVfuT5oWuxa09a_LFxkToKo16-_zRxv3m9odB8krwyjEUcJo5-JJpppW7PI_UwsaoeHP_FiROnEznxBTlhytMJnPBqecKUp7Nxukye5rO0nY4oxaHc0m76nREnRjewl3pGaSv9nRFeKUdpI52LkeSorO_c1m-G_6ay_gIud5KJ) +- ip6 + - [2n-icx 100ge e810cq avf ip6scale20k-rnd](https://csit.fd.io/report/#eNrtVstqwzAQ_Br3UrZIa7vOpYek_o-gypvG1HHUlWKafn2VEFibtpBC0lx00IsZsaMdBuTDlmnpqXvKykVWLTKs2iZOWT6_jwt3HnOtYHAOMH-IO6aOjCfAHlr7AVqpV0KnaaaVfQczrKB1j1DNXkBboLA-nOLw1nSE6g24b6Bv-FACn08lvtUTtNkFQaOKCTIQCziRJzS33o84v4kWvmEyciEqFyiQH4n5-W3CXrHZkG8_Sa7ErghuY-MF0nZaJ-zdCD01rKqPjH9yySWX_uaSu55LmLJ0vkt4qyxhytKlXLpiloqUpfNdKm6VpSJl6VIuSZbK-q7f8ub41yvrL2w7rpo) + - [2n-spr 100ge e810cq avf ip6scale20k-rnd](https://csit.fd.io/report/#eNrtVstqwzAQ_Br3ErZIa7vOJYem_o-iypvG1HHUlWpIv75KCKxNE0ghaS466MWM2NEOA_Jhy_TqqVtk5TKrlhlWbROnLH-exYU7j7lWMDgHmD_GHVNHxhNgD94xaKXeCZ2muVb2E8ywgtY9QTV_A22Bwnp_isNb0xGqD-C-gb7hfQl8OZb4VU_Q5isIGlVMkIFYwIk8obn1bsQ5J1r4hsnIhahcoEB-JOb024S9YrMh336TXIldEdzGxguk7bRO2LkRemxYVR8Y_-SSSy79zSV3O5cwZelyl_BeWcKUpWu5dMMsFSlLl7tU3CtLRcrStVySLJX1Q7_lzeGvV9Y_oJmvZg) + - [2n-spr 100ge e810cq dpdk ip6scale20k-rnd](https://csit.fd.io/report/#eNrtVkFOwzAQfE24oEX2JiG9cGjJP5CxFxo1Tc3aVCqvx60qbSLgEKmlFx_i2JpdzXhHIznEHdNLoP6pqFdFsyqw6VxainJ5n37cByy1gr33gOVD2jH1ZAIBDhA8g1bqndBrWmhlP8B5t4HOP0KzeAVtgeL6eEpfsKYnVBvgwcHg-MiBz2eOH4SCus8oaJIxQfbEAk70SZlfH0Y1f6qWBsNkpCNJFyhSGKn5_XJS_cZmS6H7ImlJYxHcptELpO2UJx78CD1PrGlPFf_lk88-zfTJX9EnzHma4RPeLE-Y83Qxn66ZpyrnaYZP1c3yVOU8XcwnyVPd3g073p7efXX7DR_mtC4) + - [2n-spr 200ge cx7 mlx5 ip6scale20k-rnd](https://csit.fd.io/report/#eNrtVkFOwzAQfE24oEXJJsFcOFDyj8o4C41wUmvtRi2vx60qbSLgEKmlFx_i2JpdzXhHI9mHLdPak33O6lWmVhmqro1LVr7cxx9bj2WRw-gcYPkQd0yWtCfAAbxjwDz_IHSF2auRdIDe7mvo3COopzcoDFDYHE_x80ZbwvwTeGhhaPlIgq9nkh-Mgra7IGjUMUNGYgFnAqXMbQ6Tmr9lS4dm0tIStQsUyE_k_H47qX5n3ZPvvkha4lwEN3H4AhVmzhMOboKeR6aaU8W_OeWSU0udctd0ClOmljiFt8sUpkxdzqmrZqpKmVriVHW7TFUpU5dzSjJVN3fDlvvTG7BuvgEYaL4e) + - [2n-c6in 200ge c6in.4xl ena dpdk ip6scale20k-rnd](https://csit.fd.io/report/#eNrtV8tqwzAQ_Br3UrZYaxz10kMT_0dRpW1j4ihCUgPJ10cxgbVJCw3YTQ86-MWM2fEOw-AQd57eAnUvRb0s5LJA2Zp0KqrXx3TxXcBKlLB3DrB6SneeOlKBAC3oRWsBy_KThBNkFRhnNtC6BYhaPL-D0EBxfX5OR9CqIyw34K0Ba_x5Bq4uM64GMmq-IqNJxgjZk2dwpI9pbn0YcH5QzXTlSTE_CWcoUhho-f7TmP3h1ZZCeyR-pV8LM3Ra_QDU40nx4AboZWOy6Rl_5ZPLPt3ok5vRJ8x5usEnvFueMOdpMp_myZPM7XTtkvxf3SRzM03o0Xw5yq30e4_wTjnKjTSVR5yjunmwO7_t_5vq5gQ3AJMR) +- ipsec + - [3n-icx 100ge e810cq avf 40tnlsw](https://csit.fd.io/report/#eNrtmM1OxCAQgJ-mXsyYQqndiwfXvodh6ewuSX8QsG59emndhG2MiZql9cClP5kBBr58yQRjO43PBuuHJN8mxTahhazcI8keb91L14ZmJIVeKaDZnfvSWCM3CFkLUpyApOkBqSK4Ial4Ad7vQehB2Q5ITjY7IALQHqViUhkULLVtbd7A_e_GSWRrgaOh-f1BNNBWelyZPp1X_lKGj1av1kddcbNIj9oHZ1X7NHUcfM63e_H5XCP3Az636KMWzUU9P92wH7_XvEEj39FPMh2fzxAO00VQzNe2g7qIns-xKKeMdZmqyPQqTFVgpjR6GoApXdVTGj1dnGloT1n0NABTtqqnLHq6ONNwnspGnmLb-1ek4-n9t67310SjpFchGtjR2PIGIErXdDQ2vIsTDexobHcDEGVrOhqb3cWJekfz8qbtdDPd9eblB2eNK04) + - [3n-icx 100ge cx6 dpdk 40tnlsw](https://csit.fd.io/report/#eNrtmM1OhDAQgJ8GL2YMLSB78eDKexi2jLuN_NS2rotPb8FNBmJM1GzBQy_8ZKbttF--ZFJjO42PBuu7KNtG-TbiuazcI0rur91L14YnLIajUsCTG_elscbSICQtSHECFsd75IrhhsXiBSpVPYPQvbIdsIxtdsAEoD1IlUplUKSxbWvzBu5_N8wiWwslGp7d7kUDbaWHpfnDeekvdVC0erUUddXNIkfUFJyVTWnq0FPO95uhAaXGkkZ87pGiFs2koJ_umMY_6bJBI9-RJhnPjzKEAzUJivnatleT6Pkg82LMWJmqClQvQ1X5psqDqz6o8nVd5cHV5al6dzUNrvqgmq7rahpcXZ6qR1dlI0-hBf4z1OH4_l0H_GumQdTLMPXtaWh_fTDlq3oamt_lmfr2NLS-Ppimq3oaGt_lmZKnWXHVdroZ74Cz4gOO6TTe) + - [3n-spr 100ge e810cq avf 40tnlsw](https://csit.fd.io/report/#eNrtmM1OhDAQgJ8GL2YMLUX24mFX3sOUMrvbhJ_aVhSfXsBNCjEmarbgoRd-MtN22i9fMqmxrcYng9VDlB6i7BDRTJbDI0r2t8NLV4YmJIZOKaDJ3fClsUJuEJKmAKM0kDg-IVUEdyQWz8C7IwjdK9sCScmuACIA7VkqJpVBwWLbVOYVhv9inEU2Fjgamt6fRA1Nqcel6eNl6S91uGj5Yl10qG4R6VC74KJsl6bOvcv5fjNuANfI3YjPPbqoRTMr6Kc7duOPmtdo5Du6SabzcxliADULiuXatlez6OUgs3zK2JiqClSvQ1X5pkqDqz6o0m1dpcHV9al6d5UFV31QZdu6yoKr61P16Kqs5Vtogf8MdTy-f9cB_5ppEPU6TH17GtpfH0zppp6G5nd9pr49Da2vD6ZsU09D47s-U-dpmt80ra6nO-A0_wCYWTYW) + - [3n-spr 200ge cx7 mlx5 40tnlsw](https://csit.fd.io/report/#eNrtmM1OxCAQgJ-mXsyYloL14sG172FYdtwl6Q8BrK1PL62b0MbEaFzEA5f-ZAYY-PIlE4ztNT4ZbO4ztsuqXUYqeXCPrHy4di_dGFIWOQxKASlv3JfGBrlBKDsORmkgeX5EogoxVgNyC20zMhB6UraHghV3eygEoD1JRaUyKGhuu8a8gvvfz_PIzgJHQ9jtUbTQHfS8OHk8L_6pEh89vFgfdfVtIgNqH9wU7tPUafI5X23HD-EauR_zsUsftWhWJX13z378s-YtGvmGfpLlBH2GcLBWQbFd205qFT0fZVUvGdHJqkT2UmRVeLIkORuGLIntLEnOxiD7B87S5GwYsjS2szQ5G4NsUGdlK8fUGv8C7HyA_7Az_jHXJOyluIb3NbXFYbiSyL6mpjgG1_C-ppY4DFca2dfUEMfg6n1l9VXX63a5M2b1O03_U24) +- hoststack quic + - [3n-icx 100ge e810cq dpdk ip4udpquic](https://csit.fd.io/report/#eNrlVctuwyAQ_Br3Um0FuJZz6aGp_6PCsK1RSExYHCX5-hIr6tpqc80hvgDSzOxrtIJSH_GT0L8V1bqo14Wqnc1HUb4_5yt6UqUUcAgBVPmSXxE9akIod-DMEaQQ36iCxJUUZg822A10PSVK2mxAqpVoQRrA1IELr4MN-8GZ9hIgh0TT9dAGuuRTH9d8f5IzaofEaNbPkANGBme1Mi10J-bc7oAFOqJmxW9jTEhIk5put8mKr6i3SO6MLBvHxAyTLZmAZp4tncIEvU6vbkbGnfwjoz1mvpeCFuDjf-0-qJ_LsvOR3VzWct55N6vmadfH7fhnVs0PcG8JiA) + - [3n-icx 100ge cx6 mlx5 ip4udpquic](https://csit.fd.io/report/#eNrlVdFKxDAQ_Jr6IitJaq0vPtzZ_5A0XW0xvcZsWnp-vbly3LZoQUFE7l6SwMxkd3YIodB5fCK0D0m2TfJtovKmikuSbq7j5i2pVAoYnAOV3sSTR4uaENIdNGYEKcQLKifNeFeN0Noxg7qjQEGbV5DqXpQgDWCooXG3feXe-saUB328EU3dQenoUE49Hst9qs1o1QdGo36BDOgZXLTKNFfvmbNqgPnao2bByRcTAtKspXWXrHj2ukVq3pFl05SYYWIgM9Asq4W9m6HH4eXFxPhRehrIeVAn9_mAOnwzQDLaohTGSkH_Ich1L7-U5VeGzzbTS4v0vBO9tEf6x280K652nW-n3zMrPgCceREo) + - [3n-spr 100ge e810cq dpdk ip4udpquic](https://csit.fd.io/report/#eNrlVdFOwyAU_Zr6Yq4BatO9-ODW_zAUrrYZWxmXLs6vlzaLt40uRmOWZb4AyTmHew-HAMUu4BOhe8iKZVYuM1W2Ng1Z_nibpuBI5VLA3ntQ-V1aBXSoCSHfQmteQQrxgspLXEhhdmC9XUPTUaSozRqkWogapAGMDbT-vrd-17eGjHaY-E4KGrZG03RQexrqqtWx7qcmGLV9ZDTpZ8geA4OznpnmmwNzTjthgQ6oWfFhkAkRadLT93ZZ-Rz0Bql9Q5aPx8YMkyKagGZeNR78BD2eYlmNjB_lWQP58LtA62GHCwjypIU_SnLq87oS5DtK_yHIL-xeY54X88KeJ88zv7BFdbPtwmb8OYvqHYwFDDo) + - [3n-spr 200ge cx7 mlx5 ip4udpquic](https://csit.fd.io/report/#eNrlVctugzAQ_Bp6qbayTRG99NCU_6jMsi2oJrheg5J-fR0UdUF9HHMIF9vSzOxrtDLHIdALk3vMil1W7jJTdk06svzpNl3Bscm1gsl7MPldegVyZJkg31tgH8Ao9UbGazyUE9kIvTsU0A4cOVp8B20eVA0agWILnb8fG_8xdlifQqSghO0AtedTRvN8zvgjvaDNGAVN-hUyURBwVa3QfHsUzn89iMQGsqL5bk0IkXhR1d-NiuI12J64-ySRzYMSBiZbFiCus8WjX6Dn-ZXVzLiYh4zWkVbotOJNePlbw1fr6dYsvW5Ht7akF97RorrZD6Gf_9Ci-gLUOhO4) +- hoststack tcp udp + - [3n-icx 100ge e810cq dpdk ip4tcp ipudp](https://csit.fd.io/report/#eNrlVctOwzAQ_JpwQYv8aAgXDi35D-TYWxLVbYzXrShfjxtVbCJUeoJDc7Etz4x3xyPLlPqIr4T-uShXRbUqVNW5PBR6eZ-n6ElpKeAQAij9kFcRPRpC0Dvo7AdIId5QBYlPUth3cMFtoO0pUTJ2A3LxKBqQFjC10IVFsqE5ib0L-ZzeuLyJca2hCXQqql7ORX90wKjbJ0ZzXxPkgJHBScNMC-2ROZdtsMBENKz4dseEhDTq6YpXlq2j2SJ1n8ja4cKYYXM4I9BOS6ZjGKHnK6zqgfEPSZI1HqX1UtBcAv3N8m3kunfzeaGXvN5MknN7oVcs_32uZX236-N2-EvL-guSthDO) + - [3n-icx 100ge cx6 mlx5 ip4tcp ipudp](https://csit.fd.io/report/#eNrlVcFuwyAM_ZrsMnkK0DSnHdblPyYC7hKNNAjTKu3Xl0bVnGjqeuqluQDiPWM_P1lQ7AN-Ebr3rNhk5SaTZWvTkqmP17QFR1KJHA7eg1Rv6RTQoSYEtYPWDCDy_BulF2ZY2wE6NxTQ9BQpavMDYrXOaxAGMDbQ-lU0vr7EOuvTM7226RLDVkHt6ZJTfl5z_imAUbuPjKayZsgBA4OzepnmmyNzbqpgvg6oOeBXHBMi0qSkO1I5bBt0h9SekGPHfjHDJGsmoJmnjEc_Qa8dLKuR8XgfyWiHwjiR00Ls_E_xU7i6t4uZzltSn8XHhU3nHcWPd7WoXnZ96MY_tKjOtngQvg) + - [3n-spr 100ge e810cq dpdk ip4tcp ipudp](https://csit.fd.io/report/#eNrlVUFuwyAQfI17qbYCO6576aGJ_xFh2NRWSLxlSaT09SVW1LVVNTm1h_gCiJlhdxghOPYB14z-NSuXWbXM8qpzaciKt8c0Bc95oRUciSAvntIqoEfDCMW-AaYAWql3zEnji1b2Axy5LbQ9R47GbkEvnlUD2gLGFjpaREvNWe0dpYN649Imhk0BDfG5ar66VP3RgqDuEAVNjU2QIwYBJx0LjdqTcK74EIUJaETybU8IEXnU1A2zItsEs0PuPlG0w40Jw6Z4RqCdlownGqGXO6zqgfEfWbI1HrX1WvFsIr3m-U6SPbgZvdLfzN5PlrN7pTc8_32yZf2w78Nu-FPL-guYeBRm) + - [3n-spr 200ge cx7 mlx5 ip4tcp ipudp](https://csit.fd.io/report/#eNrlVUFuwyAQfI17qTYCHNenHpr4HxWGTW0Vx4glVtLXh1hR11bV5NQe4gsgZobdYYSg2Ad8J3SvWbHJyk2mytamIcvfntMUHKlcChi8B5Wv0iqgQ00I-V4D-QBKiA9UXppjOaCO0LljAU1PkaI2nyDXL6IGaQBjA61fR-Pri9xZn07qtU2bGHY51J4uZdX2WvZHD4zaQ2Q0dTZDBgwMzlpmmm9OzLllhCU6oGbNtz8mRKRJV3fcsmwXdIfUfiFrxytjhkkBTUAzLxlPfoJeL7GsRsa_pElGO5TGSUHLCfWW6UfJ9mCX9FJ_c_tAaS7vpd4x_ffZFtXTvg_d-LcW1Rmr3hr-) +- nat44 + - [2n-icx 100ge e810cq avf ethip4tcp tput](https://csit.fd.io/report/#eNrtVctqwzAQ_Br3UrboYcm-9JDU_xEUeVMbHGcrKSbp11dOA7IphkJLCyEXvWZWu6NhkQ8HhxuP3XOm1lmxzkTR1nHI5OoxTq7zQnIGAxEI-RRXDjs0HkH00NoTcMZeURDHkjP7BmbYQUv5eLwFbgFDE7fB0ni6HeMazkQOpCV4nSsuINAxQF-7Mal4uSb9UkFC62NIaKxrhgzoEjgrONGoOU84SzIS3zg0KSDqSFBAPynmm2JT-M6ZPfr2HdMd48Mlgo3mTDA7zxzONEGvT1hUF8a_OUl3J3_qJP25k1opqT_V5VyUhS5vuy0X9N5MZy75SXc_f7U_VfXQH9z-8neq6gMfivTL) + - [2n-spr 100ge e810cq avf ethip4tcp-nat44ed tput](https://csit.fd.io/report/#eNrtVctqwzAQ_Br3UrbobV96SOr_KKq8qQ2Os5UUQ_r1ldOAbIqh0NJCyEWvGWl3dlgU4sHjc8D-sdDbotwWouyaNBRyc58m3wchOYORCIR8SCuPPdqAIAYI5IEz9oqCOFacuTew4w46UtPxC3AHGNu0jY5gsFEpbKDlTCggIyEYpbmASMcIQ-OnoOLpEvRLBhltjjGjKa8FMqLP4CLhTKP2NOOsych869HmC0lOhiKGWTLfFJuv77zdY-jeMb8xFS4TXDJnhrll5HiiGXopYVmfGf_mJN2c_KmT9OdOGq2l-VSnuKhKU113W67ovZrOXPOTbn7-an_q-m44-P3579T1BzVA87M) + - [2n-spr 100ge e810cq avf ethip4tcp-nat44ed cps](https://csit.fd.io/report/#eNrtVdtKw0AQ_Zr4IiPZa_LigzX_UdbN1ATSdNxdA-3Xu9bCJGhBUBRKX_bCmdkzZw7DxrQLuI443BdmVVSrQlZ9m5dCPdzmLQxRKlHCRARS3eVTwAFdRJAjRAogyvIZJQmsRelfwE0b6EmD1U8gPGDq8i15gtElrbGFTpRSA1kF0WojJHiKMLbhnVI-nig_8TPaviZGc1ULZMLA4KJcDqNuP4s5J4LjXUDHCVkNQwnjrJjvaeXsTXBbjP0B-YncNcZ9NoYh4Ze8aU8z9NTAqjlG_JOLdHXxZy7SX7tojVH2Q5oWsq5sfdHj-LXcC5nIM17S1ctfnEvT3Iy7sD3-laZ5A9RG7kM) + - [2n-spr 200ge cx7 mlx5 ethip4tcp-nat44ed tput](https://csit.fd.io/report/#eNrtVctqwzAQ_Br3Urbo7Vx6SOr_KKq8rQ22s0iKSfL1UdKAbIqh0EIg5KIHs6ud2WFRiFuP7wG710JvinJTiLKt01LI9XPafBeE5AxGIhDyJZ08dmgDghggkAfB2BcK4m5fjmgj9N1eQ0sKOGMfwB1gbNI1OoLBRqWwhoYzoYCMhGCU5gIi7SIMtT-XFW_Xsj84ZLTexYwmZjNkRJ_BGeUcRs1hErMsJGdYjzanJEEZihgmdH4pN6d_ettjaI-Y3zi3Lge4ZNAEc_PK8UAT9NrEsrpE3NBNerj5dzfpBm4araX51qe4WJVmde_juaD4jiZ0yVN6ePrPc6qrp2Hr-8tfqqsTm1_9gw) +- tunnels (gnv, vxlan, gtpu) + - [2n-icx 100ge e810cq avf ethip4udpgeneve](https://csit.fd.io/report/#eNrtVsGKwjAQ_Zp6WUaa1Nq97EG3_yExHbVQ42ySFvXrTaUwrcvCHgSh9ZKEvDeZmTweifMnixuH1VeUrqNsHcmsLMIQJauPMNnKyUTE0BCBTOZhZbFC5RCkgVKfQcTxHiUJ_BSx_gHV7KCkxcbXxmDlYLnYgtCA_hB2oZvrgvZosEEQy0BsA7btkaawbWb53WX-VQajRe0ZDcUNkAYtg4OqmUaHS4_zVy_MVxYVB_RaZIpH1yvqXy1z8M6qI7ryinxCuDvGdZCJIaGHaf2Femh3j1l-Z7xWU3pr-hRN6bWaTs-mY3fp9Ew6co_KdHqP6bDnMfr0QVV6q_psr6b5zJzs8f7_TfMbuUIa-Q) + - [3n-icx 100ge cx6 mlx5 gtpu sw](https://csit.fd.io/report/#eNrtls2OwiAUhZ-ms5lcUyi1KxejfQ9T6R1tQpEAdeo8vdSY3DYzLjT-bNiUhnPgXvhyEpzfW1w7VIskXybFMuFFU4dPkn19hsEqxzOWwsEY4Nks_FlUWDmETEMje2BpukVumOzndQ-t6nNojFj7TmtUDuZiA0wC-l2Y3XrTuZ9B3ww76NoOhfjqUuhPVVLrzpMaepkoB7QkTpokm9kdyXO1dfJXFitaMDoRWTy6UVP_nZC837Zq0TW_SAvCzZAuAwSSmJxW8UczUi_XVpRnx0uJmUjsHmLm-cR4zNhtxPi7M8Zjxh5J7AUZEzFjtxET786YiBl7JDHKWF5-6L1tz2_GvDwB7trbpg) + - [3n-spr 200ge cx7 mlx5 vxlan](https://csit.fd.io/report/#eNrtVcFuwjAM_ZpymTy1KV1PHAb9D5SmHlRKg-WEquzrF1glt0Jw4YCEuCRRnl_s5ycrPhwYtx7tKinWSblOVNk2cUny74-4sfUqz1LoiUDln_HEaFF7hNxp8MSg0nSHijIzlD3qAJ0dCmhpuQ1H59B6-FrWkBnAsI-3_WC1A6vqpo6PdNpYduAaPmdUmzHjVXpBm2MQNBY1Q3pkAWfVShjtTxJzT4NQNKMWzkSahAT0k7puSxXGD-sOffuLQouNEtxEWwTKzDxXONEEHftXVpeI53hIbw8f8pCe4-FgzsJefAL_Rb7U7I2-0du3h-etqBbuwN3l_yuqP89g_Ys) + - [3n-spr 200ge cx7 mlx5 gtpu sw](https://csit.fd.io/report/#eNrtlsGKgzAQhp_GvSxTNMb1tIdtfY-S6mwrxHRIotvu0zeWwii7LHQp0kMuRvz_cSb5-CHOHy1uHer3pFgn5ToRZduER5J_vIbFaifyLIWBCES-Cm8WNSqHkBsFjiyINN2joKw-lQMqD50-FdCS3PreGNQO3uQOshrQH8LXvafefY36bvyJaezYS2xuvX40ZrXpPathnJkyoGVxNifb6HBmz1_Tc4myqLhmsim2eHSTuX7bJHs_rerQtd_IBeFwWK8DCpayet7Fn2mi3k6urK6OpblR5PZPbrQINxHzdjc38QR5EzFvD-a2TN5kzNvd3OQT5E3GvD2YG-etqF7M0XbXe2VRXQDvw-sG) + - [3n-spr 200ge cx7 mlx5 wireguard](https://csit.fd.io/report/#eNrtVl1rwyAU_TXZy3BEM4kve1iX_1FsvGsDxsrVpO1-_Uwo3ITB9jJWqHtR8Zzr_TgcMMQjwjaAfSnkpqg3hag7k5aien1MG9ogKl6y0Xsmqqd0QrCgA7DKaRY8MlGWexCet-d6BB1Zb8-Sdf55GwfnwAbGJVc7xlsG8ZDuB-NPHcJ-0Gh4WUZnw2ni76ZHncEpt3i75v5SCKFmiISm8lbICEjgqm6i-cOFON91QyEaQVPMokmiRAiLun5umSLfUfcQug-g8Hl0xGiTVAuwXWeNF79Ar5Osm5lxa139v66_pKu_qa7ZuTUDr2bn1Pv3qcrNp-r-fapy86n6Y5_K5sEdsZ__wbL5BNyDKzU) +- reassembly + - [3n-icx 100ge e810cq dpdk reassembly](https://csit.fd.io/report/#eNrtVstOwzAQ_JpwQYtiJyG9cKDkP1DiLK1FHovXFMLX44ZKmwhxQUrbQy625RmvZnc0ktn3Dp8Zm4co20b5NtK5rcMSJY-3YXMN60TFcCACndyFk8MGS0ZIOrDmE1Qc71CTwo2KzRvUVL-CcQP5HlSmNhUoA-j3llJLjCbQY981_AHhpjrWsZ2HElln9zvThuolM7ZVM0BXu6MO_XTS8UuUoPW7FzRInSEHdALOehAa7Qfh_N2ZPCiDUHnx07CgHnki6F_tS7EXV7bI9gul4jhZYZhg4QQ0cyF-oAl6mmpejIxr8ptWvxfwm5b2W6_5Pq_f-rL51mu-r8nvxfOdrvk-r9_pZfOdrvm-Jr8l31lx0_WuHf_pWfENqTFURQ) + +## CSIT-2310 Selected Performance Comparisons + +Comparisons 23.10 vs 23.06 +- [2n-icx 100ge e810cq avf 1c 64B PDR](https://csit.fd.io/comparisons/#eNqNkNEOgiAUhp_GbhoNMLWbLjIfoLVegOGxuSnSAV319IFa5NZFGwPO-b_DOfwGGpAWyn2U5RHPECpAUBLcPYoP6ylrwM7xBXsnHddj0lfWnVqgZf9G3TFo7YOJd8oAGERsDI8ZJTze0JSge00YCHitKhSB5orU8k4YpVfgmsGOUXkjYqhChezwM3biZ2FHHybFrFfYmvq5RNJtvmCsfeglcSrOM5GN692r1QL_cMlRogX7_fFgxIQMounhpy9u__Jl7L9SHbZ7P1pSvADjNnTM) + +## CSIT-2310 Selected Performance Coverage Data + +CSIT-2310 VPP v23.10 coverage data +- [2n-icx 200ge cx7 mlx5 ip4](https://csit.fd.io/coverage/#eNpVjsEOwiAQRL8GLwYDi4RTD2r_wxDcWBKkBBDp37ekB-plk5k3s5mEDk22sx-IuhNQ0aXtEnE77wIEZ82ARzNe39xpCeGPFIwdgrhwRuP2XSfssTAth4yn1lQKjL0RAjdVFdSZflyVvaEj6l6x4dqEGk9pmn9PpzN6swxEtu1Hq6XkuAIlQT2d) + +## Further Information + +For further information including instructions how to access the needed +information with user selectable options, please refer to +[csit.fd.io documentation]({{< relref "/" >}}). diff --git a/docs/content/release_notes/previous/csit_rls2310/dpdk_performance.md b/docs/content/release_notes/previous/csit_rls2310/dpdk_performance.md new file mode 100644 index 0000000000..cc18385376 --- /dev/null +++ b/docs/content/release_notes/previous/csit_rls2310/dpdk_performance.md @@ -0,0 +1,38 @@ +--- +title: "DPDK Performance" +weight: 2 +--- + +# CSIT 23.10 - DPDK Performance + +1. TEST FRAMEWORK + - **CSIT test environment** version has been updated to ver. 13, see + [Environment Versioning]({{< ref "../../../infrastructure/fdio_csit_testbed_versioning" >}}). +2. DPDK PERFORMANCE TESTS + - No updates +3. DPDK RELEASE VERSION CHANGE + - Version 23.07 is now tested. + +# Known Issues + +List of known issues in CSIT 23.10 for DPDK performance tests: + +## New + +List of new issues in CSIT 23.10 for DPDK performance tests: + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | | + +## Previous + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | [CSIT-1904](https://jira.fd.io/browse/CSIT-1904) | DPDK 23.03 testpmd starup fails on some testbeds. Different cause but the same consequences as CSIT-1848. + +## Fixed + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | | diff --git a/docs/content/release_notes/previous/csit_rls2310/trex_performance.md b/docs/content/release_notes/previous/csit_rls2310/trex_performance.md new file mode 100644 index 0000000000..3d91f8e164 --- /dev/null +++ b/docs/content/release_notes/previous/csit_rls2310/trex_performance.md @@ -0,0 +1,40 @@ +--- +title: "TRex Performance" +weight: 3 +--- + +# CSIT 23.10 - TRex Performance + +1. TEST FRAMEWORK + - **CSIT test environment** version has been updated to ver. 13, see + [Environment Versioning]({{< ref "../../../infrastructure/fdio_csit_testbed_versioning" >}}). +2. TREX TESTS + - No updates +3. TREX VERSION + - Currently using v3.03 of TRex. + +# Known Issues + +## New + +List of new issues in CSIT 23.10 for TRex performance tests: + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | | + +## Previous + +List of known issues in CSIT 23.10 for TRex performance tests + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | | + +## Fixed + +List of known issues in CSIT 23.10 for TRex performance tests + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | | diff --git a/docs/content/release_notes/previous/csit_rls2310/vpp_device.md b/docs/content/release_notes/previous/csit_rls2310/vpp_device.md new file mode 100644 index 0000000000..6e00090798 --- /dev/null +++ b/docs/content/release_notes/previous/csit_rls2310/vpp_device.md @@ -0,0 +1,27 @@ +--- +title: "VPP Device" +weight: 4 +--- + +# CSIT 23.10 - VPP Device + +1. TEST FRAMEWORK + - **CSIT test environment** version has been updated to ver. 13, see + [Environment Versioning]({{< ref "../../../infrastructure/fdio_csit_testbed_versioning" >}}). + +# Known Issues + +List of known issues in CSIT 23.10 for VPP functional tests in VPP Device: + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | | + +## New + +List of new issues in CSIT 23.10 for VPP functional tests in VPP Device: + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | [CSIT-1931](https://jira.fd.io/browse/CSIT-1931) | Vhost test not running in device jobs + 2 | [CSIT-1932](https://jira.fd.io/browse/CSIT-1932) | 1n-spr: Occasional packet loss in L2 tests diff --git a/docs/content/release_notes/previous/csit_rls2310/vpp_performance.md b/docs/content/release_notes/previous/csit_rls2310/vpp_performance.md new file mode 100644 index 0000000000..34cf277b3d --- /dev/null +++ b/docs/content/release_notes/previous/csit_rls2310/vpp_performance.md @@ -0,0 +1,106 @@ +--- +title: "VPP Performance" +weight: 1 +--- + +# CSIT 23.10 - VPP Performance + +1. TEST FRAMEWORK + - **CSIT test environment** version has been updated to ver. 13, see + [Environment Versioning]({{< ref "../../../infrastructure/fdio_csit_testbed_versioning" >}}). + - **General Code Housekeeping**: Ongoing code optimizations and bug fixes. + - **Trending**: Ndrpdr tests use newer code (MLRsearch 1.2.1) and configuration, + gaining more stability and speed. Release results still use the old code + to keep comparability with RC1 and RC2 results. +2. VPP PERFORMANCE TESTS + - Added 2n-c6in testbed. +3. PRESENTATION AND ANALYTICS LAYER + - [Performance dashboard](https://csit.fd.io/) got updated with graphs + presenting bandwidth in bits per second for MRR and NDRPDR tests. + +# Known Issues + +These are issues that cause test failures or otherwise limit usefulness of CSIT +testing. + +## New + +Any issue listed here may have been present also in a previous release, +but was not detected/recognized/reported enough back then. + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | [CSIT-1935](https://jira.fd.io/browse/CSIT-1935) | Zero traffic reported in udpquic tests due to session close errors. + 2 | [CSIT-1936](https://jira.fd.io/browse/CSIT-1936) | TRex occasionally sees link down in L2 perf tests. + 3 | [CSIT-1937](https://jira.fd.io/browse/CSIT-1937) | Small but frequent loss in ASTF UDP on cx7 mlx5. + 4 | [CSIT-1938](https://jira.fd.io/browse/CSIT-1938) | 3n-alt: High scale ipsec policy tests may crash VPP. + 5 | [CSIT-1939](https://jira.fd.io/browse/CSIT-1939) | 3na-spr, 2n-zn2: VPP fails to start in first test cases. + 6 | [CSIT-1940](https://jira.fd.io/browse/CSIT-1940) | Hardware acceleration does not work yet. + 7 | [CSIT-1941](https://jira.fd.io/browse/CSIT-1941) | TRex may wrongly detect link bandwidth. + 8 | [CSIT-1942](https://jira.fd.io/browse/CSIT-1942) | 3nb-spr hoststack: Interface not up after first test. + 9 | [CSIT-1943](https://jira.fd.io/browse/CSIT-1943) | IMIX 4c tests may fail PDR due to ~10% loss. + 10 | [VPP-2087](https://jira.fd.io/browse/VPP-2087) | VPP crash and other symptoms in tests with AVF, jumbo packets. + 11 | [VPP-2088](https://jira.fd.io/browse/VPP-2088) | virtio: Bad CLI argument parsing introduced with tx-queue-size. + 12 | [CSIT-1944](https://jira.fd.io/browse/CSIT-1944) | Memif LXC: unrecognized option '--no-validate'. + 13 | [CSIT-1945](https://jira.fd.io/browse/CSIT-1945) | Some srv6 9000B tests crash VPP. + 14 | [VPP-2090](https://jira.fd.io/browse/VPP-2090) | MRR < PDR: DPDK plugin with MLX5 driver does not read full queue. + 15 | [VPP-2091](https://jira.fd.io/browse/VPP-2091) | Memif crashes with jumbo frames. + +## Previous + +Issues reported in previous releases which still affect the current results. + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | [CSIT-1782](https://jira.fd.io/browse/CSIT-1782) | Multicore AVF tests are failing when trying to create interface. Frequency is reduced by CSIT workaround, but occasional failures do still happen. + 2 | [CSIT-1785](https://jira.fd.io/browse/CSIT-1785) | NAT44ED tests failing to establish all TCP sessions. At least for max scale, in allotted time (limited by session 500s timeout) due to worse slow path performance than previously measured and calibrated for. CSIT removed the max scale NAT tests to avoid this issue. + 3 | [CSIT-1795](https://jira.fd.io/browse/CSIT-1795) | Ocassionally not all DET44 sessions have been established: 4128767 != 4128768 + 4 | [CSIT-1802](https://jira.fd.io/browse/CSIT-1802) | All testbeds: AF-XDP - NDR tests failing from time to time on small loss. + 5 | [CSIT-1804](https://jira.fd.io/browse/CSIT-1804) | 3n-tsh: NDR fails on ierrors. + 6 | [CSIT-1849](https://jira.fd.io/browse/CSIT-1849) | 2n-clx, 2n-icx: UDP 16m TPUT tests fail to create all sessions. + 7 | [CSIT-1881](https://jira.fd.io/browse/CSIT-1881) | 2n-icx: NFV density tests ocassionally breaks VPP which fails to start. + 8 | [CSIT-1883](https://jira.fd.io/browse/CSIT-1883) | 3n-snr: All hwasync wireguard tests failing when trying to verify device. + 9 | [CSIT-1886](https://jira.fd.io/browse/CSIT-1886) | 3n-icx: Wireguard tests with 100 and more tunnels are failing PDR criteria. + 10 | [CSIT-1892](https://jira.fd.io/browse/CSIT-1892) | 3n-alt: Unexpected two-band structure of ipsec and vxlan. + 11 | [CSIT-1896](https://jira.fd.io/browse/CSIT-1896) | Depending on topology, l3fwd avoids dut-dut link. + 12 | [CSIT-1901](https://jira.fd.io/browse/CSIT-1901) | 3n-icx: Negative ipackets on TB38 AVF 4c l2patch. + 13 | [CSIT-1904](https://jira.fd.io/browse/CSIT-1904) | DPDK 23.03 testpmd startup fails on some testbeds. + 14 | [CSIT-1906](https://jira.fd.io/browse/CSIT-1906) | Zero traffic with cx7 rdma. Testing migrated to mlx5-core on all Mellanox NICs. + 15 | [VPP-2077](https://jira.fd.io/browse/VPP-2077) | IP fragmentation: running_fragment_id is not thread safe. Causes reduced performance and failures in gtpu reassembly tests. + 16 | [CSIT-1914](https://jira.fd.io/browse/CSIT-1914) | TRex does not produce latency data on ICE NICs. + 17 | [CSIT-1915](https://jira.fd.io/browse/CSIT-1915) | The 2n-icx testbeds to not have the same performance. + 18 | [CSIT-1916](https://jira.fd.io/browse/CSIT-1916) | Poor CPU scaling on 2n-zn2 RDMA. + 19 | [CSIT-1917](https://jira.fd.io/browse/CSIT-1917) | TRex STL performance is unstable at high pps due to unsent packets. + 20 | [CSIT-1921](https://jira.fd.io/browse/CSIT-1921) | Two-band structure in SRv6, causes PDR failure in rare cases. + 21 | [CSIT-1922](https://jira.fd.io/browse/CSIT-1922) | 2n-tx2: AF_XDP MRR failures. On other testbeds MRR regressions and PDR failures. + 22 | [CSIT-1924](https://jira.fd.io/browse/CSIT-1924) | An l3fwd error in 200Ge2P1Cx7Veat-Mlx5 test with 9000B. + +## Fixed + +Issues reported in previous releases which were fixed in this release: + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | [CSIT-1800](https://jira.fd.io/browse/CSIT-1800) | All Geneve L3 mode scale tests (1024 tunnels) are failing. + 2 | [CSIT-1801](https://jira.fd.io/browse/CSIT-1801) | 9000B payload frames not forwarded over tunnels due to violating supported Max Frame Size (VxLAN, LISP, SRv6). + 3 | [CSIT-1809](https://jira.fd.io/browse/CSIT-1809) | All tests with 9000B payload frames not forwarded over vhost-user interfaces. + 4 | [CSIT-1864](https://jira.fd.io/browse/CSIT-1864) | 2n-clx: Half of the packets lost on PDR tests. + 5 | [CSIT-1884](https://jira.fd.io/browse/CSIT-1884) | 2n-clx, 2n-icx: All NAT44DET NDR PDR IMIX over 1M sessions BIDIR tests failing to create enough sessions. + 6 | [CSIT-1923](https://jira.fd.io/browse/CSIT-1923) | 3n-icx, 3n-snr: First few swasync scheduler tests timing out in runtime stat. + +# Root Cause Analysis for Regressions + +List of RCAs in CSIT 23.10 for VPP performance regressions. +Not listing differences caused by known issues (uneven worker load +due to randomized RSS or other per-worker issues). +Also not listing tests which historically show large performance variance. + +Contrary to issues, these genuine regressions do not limit usefulness +of CSIT testing. So even if they are not fixed +(e.g. when the regression is an expected consequence of added functionality), +they will not be re-listed in the next release report. + +**#** | **JiraID** | **Issue Description** +------|--------------------------------------------------|-------------------------------------------------------------- + 1 | [CSIT-1933](https://jira.fd.io/browse/CSIT-1933) | Regression in nat44ed tests around 2023-09-07. + 2 | [CSIT-1934](https://jira.fd.io/browse/CSIT-1934) | Regression in nginx rps around 2023-10-09. |