diff options
| author |   pmikus <peter.mikus@protonmail.ch> | 2023-03-09 13:32:33 +0000 |
|---|---|---|
| committer | Peter Mikus <peter.mikus@protonmail.ch> | 2023-03-15 10:06:55 +0000 |
| commit | b928e56347c682fdb9762ccbe2f368329d4037e4 (patch) | |
| tree | 9c0f6a02efc1704821229b8e18a23468fdf8674a /docs/content | |
| parent | ddcdf45806d0efa9e89dd4446b4c7da39cfb27a8 (diff) | |
feat(docs): Hugo
Signed-off-by: pmikus <peter.mikus@protonmail.ch>
Change-Id: Id8b43ef1f31f39b19a0629c52581514fda278f3b
Diffstat (limited to 'docs/content')
28 files changed, 9005 insertions, 0 deletions
diff --git a/docs/content/_index.md b/docs/content/_index.md new file mode 100644 index 0000000000..6f3e0b53e0 --- /dev/null +++ b/docs/content/_index.md @@ -0,0 +1,40 @@ +--- +title: "FD.io CSIT" +type: "docs" +--- + +# Report Structure + +FD.io CSIT Dashboard Documentation contains system performance and functional +testing data. + +Documentation is structured as follows: + +1. INTRODUCTION: General introduction to CSIT Performance Dashboard. + - **Dashboard History**: Version changes. + - **Test Scenarios Overview**: A brief overview of test scenarios + covered in this report. + - **Design**: Framework modular design hierarchy. + - **Test Naming**: Test naming convention. + - **Test Tags Descriptions**: Robot Framework Tags used for test suite and + test case grouping and selection. +2. METHODOLOGY: + - **Overview**: Tested logical topologies, test coverage and naming + specifics. +3. RELEASE NOTES: Performance tests executed in physical FD.io + testbeds. + - **VPP Performance**: Changes, added tests, environment or methodology + changes, known issues. + - **DPDK Performance**: Changes, added tests, environment or methodology + changes, known issues. + - **TRex Performance**: Changes, added tests, environment or methodology + changes, known issues. + - **VPP Device**: Changes, added tests, environment or methodology + changes, known issues. +4. INFRASTRUCTURE: + - **FD.io DC Vexxhost Inventory**: Physical testbeds location. + - **FD.io CSIT Testbed Specifications**: Specification of the physical + testbed infrastructure. + - **FD.io CSIT Testbed Configuration**: Configuration of the physical + testbed infrastructure. + - **Test Environment**: Performance test environment configuration. diff --git a/docs/content/infrastructure/_index.md b/docs/content/infrastructure/_index.md new file mode 100644 index 0000000000..3ccc042a8b --- /dev/null +++ b/docs/content/infrastructure/_index.md @@ -0,0 +1,5 @@ +--- +bookFlatSection: true +title: "Infrastructure" +weight: 4 +---
\ No newline at end of file diff --git a/docs/content/infrastructure/fdio_csit_testbed_specifications.md b/docs/content/infrastructure/fdio_csit_testbed_specifications.md new file mode 100644 index 0000000000..20bec11f9f --- /dev/null +++ b/docs/content/infrastructure/fdio_csit_testbed_specifications.md @@ -0,0 +1,1830 @@ +--- +bookToc: true +title: "FD.io CSIT Testbed Specifications" +weight: 2 +--- + +# FD.io CSIT Testbed Specifications + +## Purpose + +This note includes specification of the physical testbed infrastructure +hosted by LFN FD.io CSIT project. + +## Testbeds Overview + +### Summary List + +``` + #. Type Purpose SUT TG #TB #SUT #TG #skx #ps1 #rng #tx2 #tsh #alt #clx #zn2 #icx #snr #spr + 1. 1-Node-Skylake nomad skx na 5 5 0 5 0 0 0 0 0 0 0 0 0 0 + 2. 1-Node-Cascadelake nomad clx na 1 1 0 0 0 0 0 0 0 1 0 0 0 0 + 3. 1-Node-AmpereAltra nomad alt na 2 2 0 0 0 0 0 0 2 0 0 0 0 0 + 4. 2-Node-IxiaPS1L47 tcp skx ps1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 + 5. 2-Node-Cascadelake perf clx clx 3 3 3 0 0 0 0 0 0 6 0 0 0 0 + 6. 2-Node-ThunderX2 perf tx2 skx 1 1 .5 .5 0 0 1 0 0 0 0 0 0 0 + 7. 2-Node-Icelake perf icx icx 4 4 4 0 0 0 0 0 0 0 0 8 0 0 + 8. 3-Node-Rangeley perf rng skx 1 3 1 0 0 2 0 0 0 0 0 0 0 0 + 9. 3-Node-Taishan perf tsh skx 1 2 .5 .5 0 0 0 2 0 0 0 0 0 0 +10. 3-Node-Altra perf alt icx 1 2 1 0 0 0 0 0 2 0 0 1 0 0 +11. 2-Node-Zen2 perf zn2 zn2 1 1 1 0 0 0 0 0 0 0 2 0 0 0 +12. 3-Node-Icelake perf icx icx 2 4 2 0 0 0 0 0 0 0 0 6 0 0 +13. 3-Node-SnowRidge perf snr icx 1 2 .5 0 0 0 0 0 0 0 0 .5 2 0 +13. 2-Node-SapphireRapids perf spr spr 4 4 4 0 0 0 0 0 0 0 0 0 0 8 + Totals: 28 35 18.5 7 1 2 1 2 4 7 2 15.5 2 8 +``` + +### 1-Node-Skylake Xeon Intel (1n-skx) + +Each 1-Node-Skylake testbed includes one SUT (Server-Type-B6) with NIC +ports connected back-to-back ([Server Types](#server-types)). +Used for FD.io VPP_Device functional driver tests. + +### 1-Node-ThunderX2 Arm Marvell (1n-tx2) + +Each 1-Node-ThunderX2 testbed includes one SUT (Server-Type-E11) with NIC +ports connected back-to-back ([Server Types](#server-types)). +Used for FD.io VPP_Device functional driver tests. + +### 1-Node-Cascadelake Xeon Intel (1n-clx) + +Each 1-Node-Cascadelake testbed includes one SUT (Server-Type-C1) with +NIC ports connected back-to-back ([Server Types](#server-types)). + +Used for FD.io VPP_Device functional driver tests. + +### 2-Node-IxiaPS1L47 Ixia PSOne L47 (2n-ps1) + +Each 2-Node-IxiaPS1L47 testbed includes one SUT (Server-Type-B1) and one +TG (Ixia PSOne appliance) with 10GE interfaces connected in a 2-node +circular topology ([Server Types](#server-types)). +Used for FD.io TCP/IP and HTTP performance tests. + +### 2-Node-Cascadelake Xeon Intel (2n-clx) + +Each 2-Node-Cascadelake testbed includes one SUT (Server-Type-C2) and +one TG (Server-Type-C3) connected in a 2-node circular topology +([Server Types](#server-types)). +Used for FD.io performance tests. + +### 2-Node-Zen2 EPYC AMD (2n-zn2) + +Each 2-Node-Zen2 testbed includes one SUT (Server-Type-D1) and +one TG (Server-Type-D2) connected in a 2-node circular topology +([Server Types](#server-types)). +Used for FD.io performance tests. + +### 2-Node-ThunderX2 Arm Marvell (2x-tx2) + +Each 2-Node-ThunderX2 testbed includes one SUT (Server-Type-E22) and +one TG (Server-Type-E31) connected in a 2-node circular topology +([Server Types](#server-types)). +Used for FD.io performance tests. + +### 2-Node-Icelake Xeon Intel (2n-icx) + +Each 2-Node-Icelake testbed includes one SUT (Server-Type-F1) and +one TG (Server-Type-F2) connected in a 2-node circular topology +([Server Types](#server-types)). +Used for FD.io performance tests. + +### 3-Node-Rangeley Atom Testbeds + +Each 3-Node-Rangeley testbed includes two SUTs (Server-Type-B5) and one +TG (Server-Type-2) connected in a 3-node circular topology +([Server Types](#server-types)). +Used for FD.io performance tests. + +### 3-Node-TaiShan Arm Huawei (3n-tsh) + +Each 3-Node-TaiShan testbed includes two SUTs (Server-Type-E21) and one +TG (Server-Type-E31) connected in a 3-node circular topology +([Server Types](#server-types)). +Used for FD.io performance tests. + +### 3-Node-Altra Arm Ampere (3n-alt) + +Each 3-Node-Altra testbed includes two SUTs (Server-Type-E23) and one +TG (Server-Type-F4) connected in a 3-node circular topology +([Server Types](#server-types)). +Used for FD.io performance tests. + +### 3-Node-Icelake Xeon Intel (3n-icx) + +Each 3-Node-Icelake testbed includes two SUTs (Server-Type-F1) and one +TG (Server-Type-F3) connected in a 3-node circular topology +([Server Types](#server-types)). +Used for FD.io performance tests. + +### 3-Node-SnowRidge Atom Intel (3n-snr) + +Each 3-Node-SnowRidge testbed includes two SUTs (Server-Type-G1) and one +TG (Server-Type-F4) connected in a 3-node circular topology +([Server Types](#server-types)). +Used for FD.io performance tests. + +### 2-Node-Full-SapphireRapids Xeon Intel (2nf-spr) + +One 2-Node-Full-SapphireRapids testbed includes one SUT (Server-Type-H1) and +one TG (Server-Type-H2) connected in a 2-node physical topology +with NUMA (socket) daisy chaining. For more detail see +[Server Types](#server-types) and [Testbed Topology-TODO](#TODO). +Used for FD.io performance tests in a full system SUT setup with all PCIe +Gen5 x16 lane slots populated with 2p200GbE NICs. + +### 2-Node-SapphireRapids Xeon Intel (2n-spr) + +Each 2-Node-SapphireRapids testbed includes one SUT (Server-Type-H5) and +one TG (Server-Type-H6) connected in a 2-node circular topology. For more +detail see [Server Types](#server-types) and [Testbed Topology-TODO](#TODO). +Used for FD.io performance tests. + +## Server Management + +### Addressing + +Each server has a LOM (Lights-Out-Management e.g. SM IPMI) and a +Management port, which are connected to two different VLANs. + +``` +1. LOM (IPMI) VLAN: + - Subnet: 10.30.50.0/24 + - Gateway: 10.30.50.1 + - Broadcast: 10.30.50.255 + - DNS1: 199.204.44.24 + - DNS2: 199.204.47.54 +2. Management Vlan: + - Subnet: 10.30.51.0/24 + - Gateway: 10.30.51.1 + - Broadcast: 10.30.51.255 + - DNS1: 199.204.44.24 + - DNS2: 199.204.47.54 +``` + +To access these hosts, VPN connection is required. + +## Server Specifications + +### Server Types + +FD.io CSIT lab contains following server types: +``` +1. Server-Type-B2: Purpose - Skylake Xeon hosts for FD.io builds and data processing. + - Quantity: 2 + - Physical connectivity: + - IPMI and host management ports. + - Main HW configuration: + - Chassis: SuperMicro SYS-7049GP-TRT. + - Motherboard: SuperMicro X11DPG-QT. + - Processors: 2* Intel Platinum 8180 2.5 GHz. + - RAM Memory: 16* 16GB DDR4-2666MHz. + - Disks: 2* 1.6TB 6G SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe3.0 lanes) + - PCIe Slot2 18:00.xx: empty. + - PCIe Slot4 3b:00.xx: empty. + - PCIe Slot9 5e:00.xx: empty. + - Numa1: (x16, x16, x16 PCIe3.0 lanes) + - PCIe Slot6 86:00.xx: empty. + - PCIe Slot8 af:00.xx: empty. + - PCIe Slot10 d8:00.xx: empty. +2. Server-Type-B6: Purpose - Skylake Xeon SUT for FD.io VPP_Device functional tests. + - Quantity: 2. + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 1-node topologies. + - Main HW configuration: + - Chassis: SuperMicro SYS-7049GP-TRT. + - Motherboard: SuperMicro X11DPG-QT. + - Processors: 2* Intel Platinum 8180 2.5 GHz. + - RAM Memory: 16* 16GB DDR4-2666MHz. + - Disks: 2* 1.6TB 6G SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe3.0 lanes) + - PCIe Slot2 18:00.xx: x710-4p10GE Intel. + - PCIe Slot4 3b:00.xx: x710-4p10GE Intel. + - PCIe Slot9 5e:00.xx: empty. + - Numa1: (x16, x16, x16 PCIe3.0 lanes) + - PCIe Slot6 86:00.xx: empty. + - PCIe Slot8 af:00.xx: empty. + - PCIe Slot10 d8:00.xx: empty. +3. Server-Type-B7: Purpose - Ixia PerfectStorm One Appliance TG for FD.io TCP/IP performance tests. + - Quantity: 1. + - Physical connectivity: + - Host management interface: 10/100/1000-BaseT. + - 8-port 10GE SFP+ integrated NIC. + - Main HW configuration: + - Chassis: PS10GE4NG. + - Motherboard: SuperMicro X11DPG-QT. + - Processors: Quad-Core, Intel Processor. + - HW accelerators: FPGA offload. + - RAM Memory: 64GB. + - Disks: 1 * 1 TB, Enterprise Class, High MTBF. + - Physical Interfaces: 4 * 10GE SFP+. + - Operating System: Native IxOS. + - Interface configuration: + - Port-1: 10GE SFP+. + - Port-2: 10GE SFP+. + - Port-3: 10GE SFP+. + - Port-4: 10GE SFP+. +4. Server-Type-B8: Purpose - Skylake Xeon SUT for TCP/IP host stack tests. + - Quantity: 1. + - Physical connectivity: + - IPMI and host management ports. + - NIC ports. + - Main HW configuration: + - Chassis: SuperMicro SYS-7049GP-TRT. + - Motherboard: SuperMicro X11DPG-QT. + - Processors: 2* Intel Platinum 8180 2.5 GHz. + - RAM Memory: 16* 16GB DDR4-2666MHz. + - Disks: 2* 1.6TB 6G SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe3.0 lanes) + - PCIe Slot2 18:00.xx: x710-4p10GE Intel. + - PCIe Slot4 3b:00.xx: empty. + - PCIe Slot9 5e:00.xx: empty. + - Numa1: (x16, x16, x16 PCIe3.0 lanes) + - PCIe Slot6 86:00.xx: empty. + - PCIe Slot8 af:00.xx: empty. + - PCIe Slot10 d8:00.xx: empty. +5. Server-Type-C1: Purpose - Cascadelake Xeon SUT for FD.io VPP_Device functional tests. + - Quantity: 1. + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 1-node testbed topologies. + - Main HW configuration: + - Chassis: SuperMicro SYS-7049GP-TRT. + - Motherboard: SuperMicro X11DPG-QT. + - Processors: 2* Intel Platinum 8280 2.7 GHz. + - RAM Memory: 12* 16GB DDR4-2933. + - Disks: 2* 1.92TB SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe3.0 lanes) + - PCIe Slot2 18:00.xx: x710-4p10GE Intel. + - PCIe Slot4 3b:00.xx: x710-4p10GE Intel. + - PCIe Slot9 5e:00.xx: empty. + - Numa1: (x16, x16, x16 PCIe3.0 lanes) + - PCIe Slot6 86:00.xx: empty. + - PCIe Slot8 af:00.xx: empty. + - PCIe Slot10 d8:00.xx: empty. +6. Server-Type-C2: Purpose - Cascadelake Xeon SUT for FD.io performance testing. + - Quantity: 3 + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 2-node testbed topologies. + - Main HW configuration: + - Chassis: SuperMicro SYS-7049GP-TRT. + - Motherboard: SuperMicro X11DPG-QT. + - Processors: 2* Intel Gold 6252N 2.3 GHz. + - RAM Memory: 12* 16GB DDR4-2933. + - Disks: 2* 1.92TB SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe3.0 lanes) + - PCIe Slot2 18:00.xx: x710-4p10GE Intel. + - PCIe Slot4 3b:00.xx: xxv710-DA2-2p25GE Intel. + - PCIe Slot9 5e:00.xx: ConnectX5-2p100GE Mellanox. + - Numa1: (x16, x16, x16 PCIe3.0 lanes) + - PCIe Slot6 86:00.xx: e810-2p100GE Intel. + - PCIe Slot8 af:00.xx: empty. + - PCIe Slot10 d8:00.xx: empty. +7. Server-Type-C3: Purpose - Cascadelake Xeon TG for FD.io performance testing. + - Quantity: 3. + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 2-node testbed topologies. + - Main HW configuration: + - Chassis: SuperMicro SYS-7049GP-TRT. + - Motherboard: SuperMicro X11DPG-QT. + - Processors: 2* Intel Platinum 8280 2.7 GHz. + - RAM Memory: 12* 16GB DDR4-2933. + - Disks: 2* 1.92TB SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe3.0 lanes) + - PCIe Slot2 18:00.xx: x710-4p10GE Intel. + - PCIe Slot4 3b:00.xx: xxv710-DA2 2p25GE Intel. + - PCIe Slot9 5e:00.xx: ConnectX5-2p100GE Mellanox. + - Numa1: (x16, x16, x16 PCIe3.0 lanes) + - PCIe Slot6 86:00.xx: ConnectX5-2p100GE Mellanox. + - PCIe Slot8 af:00.xx: ConnectX5-2p100GE Mellanox. + - PCIe Slot10 d8:00.xx: empty. +8. Server-Type-C4: Purpose - Cascadelake Xeon Backend hosts for FD.io builds and data processing. + - Quantity: 3. + - Physical connectivity: + - IPMI and host management ports. + - no NIC ports, standalone setup. + - Main HW configuration: + - Chassis: SuperMicro 1029P-WTRT. + - Motherboard: SuperMicro X11DDW-NT. + - Processors: 2* Intel Platinum 8280 2.7 GHz. + - RAM Memory: 12* 16GB DDR4-2933. + - Disks: 4* 1.92TB SATA SSD. + - NICs configuration: + - Numa0: + - no cards. + - Numa1: + - no cards. +9. Server-Type-D1: Purpose - Zen2 EPYC SUT for FD.io performance testing. + - Quantity: 1. + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 2-node testbed topologies. + - Main HW configuration: + - Chassis: SuperMicro AS-1114S-WTRT + - Processors: 1* AMD EPYC 7532 2.4 GHz. + - RAM Memory: 8* 32GB DDR4-2933. + - Disks: 1* 1TB SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe3.0 lanes) + - PCIe Slot1 01:00.xx: x710-4p10GE Intel. + - PCIe Slot2 41:00.xx: xxv710-DA2-2p25GE Intel. + - PCIe Slot3 81:00.xx: mcx556a-edat ConnectX5-2p100GE Mellanox. +10. Server-Type-D2: Purpose - Zen2 EPYC TG for FD.io performance testing. + - Quantity: 1. + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 2-node testbed topologies. + - Main HW configuration: + - Chassis: SuperMicro AS-1114S-WTRT + - Processors: 1* AMD EPYC 7532 2.4 GHz. + - RAM Memory: 8* 32GB DDR4-2933. + - Disks: 1* 1TB SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe3.0 lanes) + - PCIe Slot1 01:00.xx: mcx556a-edat ConnectX5-2p100GE Mellanox. + - PCIe Slot2 41:00.xx: x710-4p10GE Intel. + - PCIe Slot3 81:00.xx: xxv710-DA2 2p25GE Intel. +11. Server-Type-E11: Purpose - ThunderX2 Arm Marvell SUT for FD.io VPP_Device functional tests. + - Quantity: 2 + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 1-node topologies. + - Main HW configuration: + - Chassis: GIGABYTE Rack Mount + - Motherboard: MT91-FS4-00 + - Processors: 2 * ThunderX2 ARMv8 CN9980 2.20 GHz + - RAM Memory: 16 * 16GB DIMM + - Disks: 2 * 480GB 6G SATA SSD SAMSUNG MZ7LH480 + - NICs configuration: + - Numa0: + - PCIe Slot4 05:00.xx: XL710-QDA2-2p40GE Intel. + - PCIe Slot8 0b:00.xx: ConnectX5-2p10/25GE Mellanox. + - Numa1: + - PCIe Slot14 91:00.xx: XL710-QDA2-2p40GE Intel. + - PCIe Slot26 9a:00.xx: ConnectX5-2p10/25GE Mellanox. +12. Server-Type-E21: Purpose - TaiShan Arm Huawei SUT for FD.io performance testing. + - Quantity: 2 + - Physical connectivity: + - IPMI(?) and host management ports. + - NIC ports connected into 3-node topology. + - Main HW configuration: + - Chassis: Huawei TaiShan 2280. + - Processors: 2* hip07-d05 ~ 32* Arm Cortex-A72 + - RAM Memory: 8* 16GB DDR4-2400MT/s + - Disks: 1* 4TB SATA HDD + - NICs configuration: + - PCIe Slot4 e9:00.xx: connectx4-2p25GE Mellanox. + - PCIe Slot6 11:00.xx: x520-2p10GE Intel. +13. Server-Type-E22: Purpose - ThunderX2 Arm Marvell SUT for FD.io performance testing. + - Quantity: 1 + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 2-node topologies. + - Main HW configuration: + - Chassis: Gigabyte R181-T90 1U + - Motherboard: MT91-FS1 + - Processors: 2* ThunderX2 ARMv8 CN9975 2.0 GHz + - RAM Memory: 4* 32GB RDIMM + - Disks: 1* 480GB SSD Micron, 1* 1000GB HDD Seagate_25 + - NICs configuration: + - Numa0: + - no cards + - Numa1: + - PCIe Slot18 91:00.xx: XL710-QDA2-2p40GE Intel. +14. Server-Type-E23: Purpose - Altra Arm Ampere SUT for FD.io performance testing. + - Quantity: 2 + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 3-node topologies. + - Main HW configuration: + - Chassis: WIWYNN Mt.Jade Server System B81.030Z1.0007 2U + - Motherboard: Mt.Jade Motherboard + - Processors: 2* Ampere(R) Altra(R) Q80-30 Processor (Neoverse N1) + - Processor Signature: Implementor 0x41, Variant 0x3, Architecture 15, Part 0xd0c, Revision 1 + - RAM Memory: 16* 8GB DDR4-3200MT/s + - Disks: 2* 960GB SSD Samsung M.2 NVMe PM983 + - NICs configuration: + - Numa0: + - PCIe Slot1 0004:04:00.x: xl710-QDA2-2p40GE Intel. + - Numa1: + - no cards. +15. Server-Type-E24 : Purpose - Altra Arm Ampere for FD.io build. + - Quantity: 2. + - Physical connectivity: + - IPMI and host management ports. + - Main HW configuration: + - Chassis: Gigabyte R152-P30-00 1U + - Motherboard: MP32-AR1-00 + - Processors: 1* Ampere(R) Altra(R) Q80-30 Processor (Neoverse N1) + - Processor Signature: Implementor 0x0a, Variant 0x1, Architecture 6, Part 0x000, Revision 1 + - RAM Memory: 12* 16GB DDR4-3200MT/s + - Disks: 1* 960GB SSD Samsung M.2 NVMe PM983 +16. Server-Type-E31: Purpose - Skylake Xeon TG for FD.io performance testing. + - Quantity: 1 + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 2-node and 3-node topologies. + - Main HW configuration: + - Chassis: SuperMicro SYS-7049GP-TRT. + - Motherboard: SuperMicro X11DPG-QT. + - Processors: 2* Intel Platinum 8180 2.5 GHz. + - RAM Memory: 16* 16GB DDR4-2666MHz. + - Disks: 2* 1.6TB 6G SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe3.0 lanes) + - PCIe Slot2 18:00.xx: x710-4p10GE Intel. + - PCIe Slot4 3b:00.xx: xxv710-DA2 2p25GE Intel. + - PCIe Slot9 5e:00.xx: empty. + - Numa1: (x16, x16, x16 PCIe3.0 lanes) + - PCIe Slot6 86:00.xx: empty. + - PCIe Slot8 af:00.xx: XL710-QDA2-2p40GE Intel. + - PCIe Slot10 d8:00.xx: x710-4p10GE Intel. +17. Server-Type-F1: Purpose - Icelake Xeon SUT for FD.io performance testing. + - Quantity: 8. + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 2-node or 3-node testbed topologies. + - Main HW configuration: + - Chassis: SuperMicro SYS-740GP-TNRT. + - Motherboard: Super X12DPG-QT6. + - Processors: 2* Intel Platinum 8358 2.6 GHz. + - RAM Memory: 16* 16GB DDR4-3200. + - Disks: 2* 960GB SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe4.0 lanes) + - PCIe Slot2 18:00.xx: xxv710-DA2-2p25GE Intel. + - PCIe Slot4 3b:00.xx: e810-XXVDA4-4p25GE Intel. + - PCIe Slot9 5e:00.xx: e810-2CQDA2-2p100GE Intel. + - Numa1: (x16, x16, x16 PCIe4.0 lanes) + - PCIe Slot6 86:00.xx: empty. + - PCIe Slot8 af:00.xx: empty. + - PCIe Slot10 d8:00.xx: empty. +18. Server-Type-F2: Purpose - Icelake Xeon TG for FD.io performance testing. + - Quantity: 3. + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 2-node testbed topologies. + - Main HW configuration: + - Chassis: SuperMicro SYS-740GP-TNRT. + - Motherboard: Super X12DPG-QT6. + - Processors: 2* Intel Platinum 8358 2.6 GHz. + - RAM Memory: 16* 16GB DDR4-3200. + - Disks: 2* 960GB SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe4.0 lanes) + - PCIe Slot2 18:00.xx: xxv710-DA2-2p25GE Intel. + - PCIe Slot4 3b:00.xx: e810-XXVDA4-4p25GE Intel. + - PCIe Slot9 5e:00.xx: e810-2CQDA2-2p100GE Intel. + - Numa1: (x16, x16, x16 PCIe4.0 lanes) + - PCIe Slot6 86:00.xx: e810-2CQDA2-2p100GE Intel. + - PCIe Slot8 af:00.xx: empty. + - PCIe Slot10 d8:00.xx: empty. +19. Server-Type-F3: Purpose - Icelake Xeon TG for FD.io performance testing. + - Quantity: 3. + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 3-node testbed topologies. + - Main HW configuration: + - Chassis: SuperMicro SYS-740GP-TNRT. + - Motherboard: Super X12DPG-QT6. + - Processors: 2* Intel Platinum 8358 2.6 GHz. + - RAM Memory: 16* 16GB DDR4-3200. + - Disks: 2* 960GB SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe4.0 lanes) + - PCIe Slot2 18:00.xx: xxv710-DA2-2p25GE Intel. + - PCIe Slot4 3b:00.xx: e810-XXVDA4-4p25GE Intel. + - PCIe Slot9 5e:00.xx: e810-2CQDA2-2p100GE Intel. + - Numa1: (x16, x16, x16 PCIe4.0 lanes) + - PCIe Slot6 86:00.xx: empty. + - PCIe Slot8 af:00.xx: empty. + - PCIe Slot10 d8:00.xx: empty. +20. Server-Type-F4: Purpose - Icelake Xeon Shared TG for FD.io performance testing. + - Quantity: 3. + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 2-node and/or 3-node testbed topologies. + - Main HW configuration: + - Chassis: SuperMicro SYS-740GP-TNRT. + - Motherboard: Super X12DPG-QT6. + - Processors: 2* Intel Platinum 8358 2.6 GHz. + - RAM Memory: 16* 16GB DDR4-3200. + - Disks: 2* 960GB SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe4.0 lanes) + - PCIe Slot2 18:00.xx: xxv710-DA2-2p25GE Intel. + - PCIe Slot4 3b:00.xx: empty. + - PCIe Slot9 5e:00.xx: empty. + - Numa1: (x16, x16, x16 PCIe4.0 lanes) + - PCIe Slot6 86:00.xx: e810-XXVDA4-4p25GE Intel. + - PCIe Slot8 af:00.xx: e810-2CQDA2-2p100GE Intel. + - PCIe Slot10 d8:00.xx: empty. +21. Server-Type-G1: Purpose - SnowRidge Atom SUT for FD.io performance testing. + - Quantity: 2 + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 3-node testbed topology. + - Main HW configuration: + - Chassis: Intel JACOBSVILLE SDP. + - Motherboard: Intel JACOBSVILLE E63448-400. + - Processors: 1* Intel Atom P5362B 2.2 GHz. + - RAM Memory: 2* 16GB DDR4-2933. + - Disks: ?* ? SATA SSD. + - NICs configuration: + - Numa0: (x16, PCIe3.0 lane) + - PCIe BuiltIn ec:00.xx: e810-XXVDA4-4p25GE Intel. +22. Server-Type-H1: Purpose - SapphireRapids Xeon SUT for FD.io full system performance testing. + - Quantity: 1. + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 3-numa-node testbed topologies. + - Main HW configuration: + - Chassis: SuperMicro SYS-741GE-TNRT. + - Motherboard: Super X13DEG-QT-P. + - Processors: 2* Intel Platinum 8462Y+ 32 core 2.8 GHz 300W TDP. + - RAM Memory: 16* 32GB DDR5-4800. + - Disks: 2* 960GB SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe5.0 lanes) + - PCIe Slot2 18:00.xx: MCX713106AS-VEAT ConnectX7-2p200GE Nvidia. + - PCIe Slot4 3b:00.xx: MCX713106AS-VEAT ConnectX7-2p200GE Nvidia. + - PCIe Slot10 5e:00.xx: MCX713106AS-VEAT ConnectX7-2p200GE Nvidia. + - Numa1: (x16, x16, x16 PCIe5.0 lanes) + - PCIe Slot7 86:00.xx: MCX713106AS-VEAT ConnectX7-2p200GE Nvidia. + - PCIe Slot9 af:00.xx: MCX713106AS-VEAT ConnectX7-2p200GE Nvidia. + - PCIe Slot11 d8:00.xx: MCX713106AS-VEAT ConnectX7-2p200GE Nvidia. +23. Server-Type-H2: Purpose - SapphireRapids Xeon TG for FD.io full system performance testing. + - Quantity: 1. + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 3-numa-node testbed topologies. + - Main HW configuration: + - Chassis: SuperMicro SYS-741GE-TNRT. + - Motherboard: Super X13DEG-QT-P. + - Processors: 2* Intel Platinum 8462Y+ 32 core 2.8 GHz 300W TDP. + - RAM Memory: 16* 32GB DDR5-4800. + - Disks: 2* 960GB SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe5.0 lanes) + - PCIe Slot2 18:00.xx: MCX713106AS-VEAT ConnectX7-2p200GE Nvidia. + - PCIe Slot4 3b:00.xx: MCX713106AS-VEAT ConnectX7-2p200GE Nvidia. + - PCIe Slot10 5e:00.xx: MCX713106AS-VEAT ConnectX7-2p200GE Nvidia. + - Numa1: (x16, x16, x16 PCIe5.0 lanes) + - PCIe Slot7 86:00.xx: MCX713106AS-VEAT ConnectX7-2p200GE Nvidia. + - PCIe Slot9 af:00.xx: MCX713106AS-VEAT ConnectX7-2p200GE Nvidia. + - PCIe Slot11 d8:00.xx: empty. +24. Server-Type-H3: Purpose - SapphireRapids Xeon SUT for FD.io performance testing. + - Quantity: 1. + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 3-numa-node testbed topologies. + - Main HW configuration: + - Chassis: SuperMicro SYS-741GE-TNRT. + - Motherboard: Super X13DEG-QT-P. + - Processors: 2* Intel Platinum 8462Y+ 32 core 2.8 GHz 300W TDP. + - RAM Memory: 16* 32GB DDR5-4800. + - Disks: 2* 960GB SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe5.0 lanes) + - PCIe Slot2 18:00.xx: e810-2CQDA2-2p100GE Intel. + - PCIe Slot4 3b:00.xx: e810-XXVDA4-4p25GE Intel. + - PCIe Slot10 5e:00.xx: empty. + - Numa1: (x16, x16, x16 PCIe5.0 lanes) + - PCIe Slot7 86:00.xx: e810-2CQDA2-2p100GE Intel. + - PCIe Slot9 af:00.xx: e810-XXVDA4-4p25GE Intel. + - PCIe Slot11 d8:00.xx: empty. +25. Server-Type-H4: Purpose - SapphireRapids Xeon TG for FD.io performance testing. + - Quantity: 1. + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 3-numa-node testbed topologies. + - Main HW configuration: + - Chassis: SuperMicro SYS-741GE-TNRT. + - Motherboard: Super X13DEG-QT-P. + - Processors: 2* Intel Platinum 8462Y+ 32 core 2.8 GHz 300W TDP. + - RAM Memory: 16* 32GB DDR5-4800. + - Disks: 2* 960GB SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe5.0 lanes) + - PCIe Slot2 18:00.xx: e810-2CQDA2-2p100GE Intel. + - PCIe Slot4 3b:00.xx: e810-XXVDA4-4p25GE Intel. + - PCIe Slot10 5e:00.xx: empty. + - Numa1: (x16, x16, x16 PCIe5.0 lanes) + - PCIe Slot7 86:00.xx: empty. + - PCIe Slot9 af:00.xx: empty. + - PCIe Slot11 d8:00.xx: empty. +26. Server-Type-H5: Purpose - SapphireRapids Xeon SUT for FD.io performance testing. + - Quantity: 2. + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 2-node testbed topologies. + - Main HW configuration: + - Chassis: SuperMicro SYS-741GE-TNRT. + - Motherboard: Super X13DEG-QT-P. + - Processors: 2* Intel Platinum 8462Y+ 32 core 2.8 GHz 300W TDP. + - RAM Memory: 16* 32GB DDR5-4800. + - Disks: 2* 960GB SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe5.0 lanes) + - PCIe Slot2 18:00.xx: MCX713106AS-VEAT ConnectX7-2p200GE Nvidia. + - PCIe Slot4 3b:00.xx: e810-2CQDA2-2p100GE Intel. + - PCIe Slot10 5e:00.xx: e810-XXVDA4-4p25GE Intel. + - Numa1: (x16, x16, x16 PCIe5.0 lanes) + - PCIe Slot7 86:00.xx: empty. + - PCIe Slot9 af:00.xx: empty. + - PCIe Slot11 d8:00.xx: empty. +27. Server-Type-H6: Purpose - SapphireRapids Xeon TG for FD.io performance testing. + - Quantity: 2. + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 2-node testbed topologies plus loopbacks in Numa1 for TG self-test. + - Main HW configuration: + - Chassis: SuperMicro SYS-741GE-TNRT. + - Motherboard: Super X13DEG-QT-P. + - Processors: 2* Intel Platinum 8462Y+ 32 core 2.8 GHz 300W TDP. + - RAM Memory: 16* 32GB DDR5-4800. + - Disks: 2* 960GB SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe5.0 lanes) + - PCIe Slot2 18:00.xx: MCX713106AS-VEAT ConnectX7-2p200GE Nvidia. + - PCIe Slot4 3b:00.xx: e810-2CQDA2-2p100GE Intel. + - PCIe Slot10 5e:00.xx: e810-XXVDA4-4p25GE Intel. + - Numa1: (x16, x16, x16 PCIe5.0 lanes) + - PCIe Slot7 86:00.xx: MCX713106AS-VEAT ConnectX7-2p200GE Nvidia. + - PCIe Slot9 af:00.xx: e810-2CQDA2-2p100GE Intel. + - PCIe Slot11 d8:00.xx: empty. +``` + +### Naming Convention + +Following naming convention is used within this page to specify physical +connectivity and wiring across defined CSIT testbeds: + +``` +- testbedname: testbedN. +- hostname: + - traffic-generator: tN-tgW. + - system-under-testX: tN-sutX. +- portnames: + - tN-tgW-cY/pZ. + - tN-sutX-cY/pZ. +- where: + - N - testbed number. + - tgW - server acts as traffic-generator with W index. + - sutX - server acts as system-under-test with X index. + - Y - PCIe slot number denoting a NIC card number within the host. + - Z - port number on the NIC card. +``` + +## Testbeds Configuration + +### Per Testbed Server Allocation and Naming + +#### 1-Node-Skylake Servers (1n-skx) PROD + +``` +- SUT [Server-Type-B6]: + - testbedname: testbed11. + - hostname: s1-t11-sut1. + - IPMI IP: 10.30.50.47 + - Host IP: 10.30.51.50 + - portnames: + - s1-t11-sut1-c2/p1 - 10GE-port1 x710-4p10GE. + - s1-t11-sut1-c2/p2 - 10GE-port2 x710-4p10GE. + - s1-t11-sut1-c2/p3 - 10GE-port3 x710-4p10GE. + - s1-t11-sut1-c2/p4 - 10GE-port4 x710-4p10GE. + - s1-t11-sut1-c4/p1 - 10GE-port1 x710-4p10GE. + - s1-t11-sut1-c4/p2 - 10GE-port2 x710-4p10GE. + - s1-t11-sut1-c4/p3 - 10GE-port3 x710-4p10GE. + - s1-t11-sut1-c4/p4 - 10GE-port4 x710-4p10GE. +- SUT [Server-Type-B6]: + - testbedname: testbed12. + - hostname: s2-t12-sut1. + - IPMI IP: 10.30.50.48 + - Host IP: 10.30.51.51 + - portnames: + - s2-t12-sut1-c2/p1 - 10GE-port1 x710-4p10GE. + - s2-t12-sut1-c2/p2 - 10GE-port2 x710-4p10GE. + - s2-t12-sut1-c2/p3 - 10GE-port3 x710-4p10GE. + - s2-t12-sut1-c2/p4 - 10GE-port4 x710-4p10GE. + - s2-t12-sut1-c4/p1 - 10GE-port1 x710-4p10GE. + - s2-t12-sut1-c4/p2 - 10GE-port2 x710-4p10GE. + - s2-t12-sut1-c4/p3 - 10GE-port3 x710-4p10GE. + - s2-t12-sut1-c4/p4 - 10GE-port4 x710-4p10GE. +``` + +#### 1-Node-ThunderX2 Servers (1n-tx2) PROD + +``` +- SUT [Server-Type-E11]: + - testbedname: testbed13 + - hostname: s55-t13-sut1 + - IPMI IP: 10.30.50.70 + - Host IP: 10.30.51.70 + - portnames: + - s55-t13-sut1-c4/p1 - 40GE-port1 XL710-QDA2-2p40GE. + - s55-t13-sut1-c4/p2 - 40GE-port2 XL710-QDA2-2p40GE. + - s55-t13-sut1-c8/p1 - 40GE-port1 ConnectX5-2p10/25GE Mellanox. + - s55-t13-sut1-c8/p2 - 40GE-port2 ConnectX5-2p10/25GE Mellanox. + - s55-t13-sut1-c14/p1 - 40GE-port1 XL710-QDA2-2p40GE. + - s55-t13-sut1-c14/p2 - 40GE-port2 XL710-QDA2-2p40GE. + - s55-t13-sut1-c26/p1 - 40GE-port1 ConnectX5-2p10/25GE Mellanox. + - s55-t13-sut1-c26/p2 - 40GE-port2 ConnectX5-2p10/25GE Mellanox. +- SUT [Server-Type-E11]: + - testbedname: testbed14 + - hostname: s56-t14-sut1 + - IPMI IP: 10.30.50.71 + - Host IP: 10.30.51.71 + - portnames: + - s56-t14-sut1-c4/p1 - 40GE-port1 XL710-QDA2-2p40GE. + - s56-t14-sut1-c4/p2 - 40GE-port2 XL710-QDA2-2p40GE. + - s56-t14-sut1-c8/p1 - 40GE-port1 ConnectX5-2p10/25GE Mellanox. + - s56-t14-sut1-c8/p2 - 40GE-port2 ConnectX5-2p10/25GE Mellanox. + - s56-t14-sut1-c14/p1 - 40GE-port1 XL710-QDA2-2p40GE. + - s56-t14-sut1-c14/p2 - 40GE-port2 XL710-QDA2-2p40GE. + - s56-t14-sut1-c26/p1 - 40GE-port1 ConnectX5-2p10/25GE Mellanox. + - s56-t14-sut1-c26/p2 - 40GE-port2 ConnectX5-2p10/25GE Mellanox. +``` + +#### 1-Node-Cascadelake Servers (1n-clx) PROD + +``` +- SUT [Server-Type-C1]: + - testbedname: testbed11. + - hostname: s32-t14-sut1. + - IPMI IP: 10.30.55.17 + - Host IP: 10.32.8.17 + - portnames: + - s32-t14-sut1-c2/p1 - 10GE-port1 x710-4p10GE. + - s32-t14-sut1-c2/p2 - 10GE-port2 x710-4p10GE. + - s32-t14-sut1-c2/p3 - 10GE-port3 x710-4p10GE. + - s32-t14-sut1-c2/p4 - 10GE-port4 x710-4p10GE. + - s32-t14-sut1-c4/p1 - 10GE-port1 x710-4p10GE. + - s32-t14-sut1-c4/p2 - 10GE-port2 x710-4p10GE. + - s32-t14-sut1-c4/p3 - 10GE-port3 x710-4p10GE. + - s32-t14-sut1-c4/p4 - 10GE-port4 x710-4p10GE. +``` + +#### 2-Node-IxiaPS1L47 Servers (2n-ps1) VERIFY + +``` +- SUT [Server-Type-B8]: + - testbedname: testbed25. + - hostname: s25-t25-sut1. + - IPMI IP: 10.30.50.58 + - Host IP: 10.30.51.61 + - portnames: + - s25-t25-sut1-c2/p1 - 10GE-port1 x710-4p10GE. + - s25-t25-sut1-c2/p2 - 10GE-port2 x710-4p10GE. + - s25-t25-sut1-c2/p3 - 10GE-port3 x710-4p10GE. + - s25-t25-sut1-c2/p4 - 10GE-port4 x710-4p10GE. +- TG [Server-Type-B7]: + - testbedname: testbed25. + - hostname: s26-t25-tg1. + - IPMI IP: 10.30.50.59 + - Host IP: 10.30.51.62 + - portnames: + - s26-t25-tg1-p1 - 10GE-port1. + - s26-t25-tg1-p2 - 10GE-port2. + - s26-t25-tg1-p3 - 10GE-port3. + - s26-t25-tg1-p4 - 10GE-port4. +``` + +#### 2-Node-Cascadelake Servers (2n-clx) PROD + +``` +- SUT [Server-Type-C2]: + - testbedname: testbed27. + - hostname: s33-t27-sut1. + - IPMI IP: 10.30.55.18 + - Host IP: 10.32.8.18 + - portnames: + - s33-t27-sut1-c2/p1 - 10GE-port1 x710-4p10GE. + - s33-t27-sut1-c2/p2 - 10GE-port2 x710-4p10GE. + - s33-t27-sut1-c2/p3 - 10GE-port3 x710-4p10GE. + - s33-t27-sut1-c2/p4 - 10GE-port4 x710-4p10GE. + - s33-t27-sut1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s33-t27-sut1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s33-t27-sut1-c6/p1 - 100GE-port1 e810-2p100GE. + - s33-t27-sut1-c6/p2 - 100GE-port2 e810-2p100GE. + - s33-t27-sut1-c9/p1 - 100GE-port1 ConnectX5-2p100GE. + - s33-t27-sut1-c9/p2 - 100GE-port2 ConnectX5-2p100GE. +- TG [Server-Type-C3]: + - testbedname: testbed27. + - hostname: s34-t27-tg1. + - IPMI IP: 10.30.55.19 + - Host IP: 10.32.8.19 + - portnames: + - s34-t27-tg1-c2/p1 - 10GE-port1 x710-4p10GE. + - s34-t27-tg1-c2/p2 - 10GE-port2 x710-4p10GE. + - s34-t27-tg1-c2/p3 - 10GE-port3 x710-4p10GE. + - s34-t27-tg1-c2/p4 - 10GE-port4 x710-4p10GE. + - s34-t27-tg1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s34-t27-tg1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s34-t27-tg1-c6/p1 - 100GE-port1 ConnectX5-2p100GE. + - s34-t27-tg1-c6/p2 - 100GE-port2 ConnectX5-2p100GE. + - s38-t27-tg1-c8/p1 - 100GE-port1 ConnectX5-2p100GE. + - s38-t27-tg1-c8/p2 - 100GE-port2 ConnectX5-2p100GE. + - s34-t27-tg1-c9/p1 - 100GE-port1 ConnectX5-2p100GE. + - s34-t27-tg1-c9/p2 - 100GE-port2 ConnectX5-2p100GE. +- SUT [Server-Type-C2]: + - testbedname: testbed28. + - hostname: s35-t28-sut1. + - IPMI IP: 10.30.55.20 + - Host IP: 10.32.8.20 + - portnames: + - s35-t28-sut1-c2/p1 - 10GE-port1 x710-4p10GE. + - s35-t28-sut1-c2/p2 - 10GE-port2 x710-4p10GE. + - s35-t28-sut1-c2/p3 - 10GE-port3 x710-4p10GE. + - s35-t28-sut1-c2/p4 - 10GE-port4 x710-4p10GE. + - s35-t28-sut1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s35-t28-sut1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s35-t28-sut1-c6/p1 - 100GE-port1 e810-2p100GE. + - s35-t28-sut1-c6/p2 - 100GE-port2 e810-2p100GE. + - s35-t28-sut1-c9/p1 - 100GE-port1 ConnectX5-2p100GE. + - s35-t28-sut1-c9/p2 - 100GE-port2 ConnectX5-2p100GE. +- TG [Server-Type-C3]: + - testbedname: testbed28. + - hostname: s36-t28-tg1. + - IPMI IP: 10.30.55.21 + - Host IP: 10.32.8.21 + - portnames: + - s36-t28-tg1-c2/p1 - 10GE-port1 x710-4p10GE. + - s36-t28-tg1-c2/p2 - 10GE-port2 x710-4p10GE. + - s36-t28-tg1-c2/p3 - 10GE-port3 x710-4p10GE. + - s36-t28-tg1-c2/p4 - 10GE-port4 x710-4p10GE. + - s36-t28-tg1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s36-t28-tg1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s36-t28-tg1-c6/p1 - 100GE-port1 ConnectX5-2p100GE. + - s36-t28-tg1-c6/p2 - 100GE-port2 ConnectX5-2p100GE. + - s38-t28-tg1-c8/p1 - 100GE-port1 ConnectX5-2p100GE. + - s38-t28-tg1-c8/p2 - 100GE-port2 ConnectX5-2p100GE. + - s36-t28-tg1-c9/p1 - 100GE-port1 ConnectX5-2p100GE. + - s36-t28-tg1-c9/p2 - 100GE-port2 ConnectX5-2p100GE. +- SUT [Server-Type-C2]: + - testbedname: testbed29. + - hostname: s37-t29-sut1. + - IPMI IP: 10.30.55.22 + - Host IP: 10.32.8.22 + - portnames: + - s37-t29-sut1-c2/p1 - 10GE-port1 x710-4p10GE. + - s37-t29-sut1-c2/p2 - 10GE-port2 x710-4p10GE. + - s37-t29-sut1-c2/p3 - 10GE-port3 x710-4p10GE. + - s37-t29-sut1-c2/p4 - 10GE-port4 x710-4p10GE. + - s37-t29-sut1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s37-t29-sut1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s37-t29-sut1-c6/p1 - 100GE-port1 e810-2p100GE. + - s37-t29-sut1-c6/p2 - 100GE-port2 e810-2p100GE. + - s37-t29-sut1-c9/p1 - 100GE-port1 ConnectX5-2p100GE. + - s37-t29-sut1-c9/p2 - 100GE-port2 ConnectX5-2p100GE. +- TG [Server-Type-C3]: + - testbedname: testbed29. + - hostname: s38-t29-tg1. + - IPMI IP: 10.30.55.23 + - Host IP: 10.32.8.23 + - portnames: + - s38-t29-tg1-c2/p1 - 10GE-port1 x710-4p10GE. + - s38-t29-tg1-c2/p2 - 10GE-port2 x710-4p10GE. + - s38-t29-tg1-c2/p3 - 10GE-port3 x710-4p10GE. + - s38-t29-tg1-c2/p4 - 10GE-port4 x710-4p10GE. + - s38-t29-tg1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s38-t29-tg1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s38-t29-tg1-c6/p1 - 100GE-port1 ConnectX5-2p100GE. + - s38-t29-tg1-c6/p2 - 100GE-port2 ConnectX5-2p100GE. + - s38-t29-tg1-c9/p1 - 100GE-port1 ConnectX5-2p100GE. + - s38-t29-tg1-c9/p2 - 100GE-port2 ConnectX5-2p100GE. +``` + +#### 2-Node-Zen2 Servers (2n-zn2) PROD + +``` +- SUT [Server-Type-D1]: + - testbedname: testbed210. + - hostname: s60-t210-sut1. + - IPMI IP: 10.30.55.24 + - Host IP: 10.32.8.24 + - portnames: + - s60-t210-sut1-c1/p1 - 10GE-port1 x710-4p10GE. + - s60-t210-sut1-c1/p2 - 10GE-port2 x710-4p10GE. + - s60-t210-sut1-c1/p3 - 10GE-port3 x710-4p10GE. + - s60-t210-sut1-c1/p4 - 10GE-port4 x710-4p10GE. + - s60-t210-sut1-c2/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s60-t210-sut1-c2/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s60-t210-sut1-c3/p1 - 100GE-port1 ConnectX5-2p100GE. + - s60-t210-sut1-c3/p2 - 100GE-port2 ConnectX5-2p100GE. +- TG [Server-Type-D2]: + - testbedname: testbed210. + - hostname: s61-t210-tg1. + - IPMI IP: 10.30.55.25 + - Host IP: 10.32.8.25 + - portnames: + - s61-t210-tg1-c1/p1 - 100GE-port1 ConnectX5-2p100GE. + - s61-t210-tg1-c1/p2 - 100GE-port2 ConnectX5-2p100GE. + - s61-t210-tg1-c2/p1 - 10GE-port1 x710-4p10GE. + - s61-t210-tg1-c2/p2 - 10GE-port2 x710-4p10GE. + - s61-t210-tg1-c2/p3 - 10GE-port3 x710-4p10GE. + - s61-t210-tg1-c2/p4 - 10GE-port4 x710-4p10GE. + - s61-t210-tg1-c3/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s61-t210-tg1-c3/p2 - 25GE-port2 xxv710-DA2-2p25GE. +``` + +#### 2-Node-ThunderX2 Servers (2x-tx2) PROD + +Note: Server19 (TG) is shared between testbed33 & testbed211 + +``` +- SUT [Server-Type-E22]: + - testbedname: testbed211. + - hostname: s27-t211-sut1. + - IPMI IP: 10.30.50.69 + - Host IP: 10.30.51.69 + - portnames: + - s27-t211-sut1-c18/p1 - 40GE-port1 XL710-QDA2-2p40GE. + - s27-t211-sut1-c18/p2 - 40GE-port2 XL710-QDA2-2p40GE. +- TG [Server-Type-E31]: + - testbedname: testbed33 and testbed211. + - hostname: s19-t33t211-tg1. + - IPMI IP: 10.30.50.46 + - Host IP: 10.30.51.49 + - portnames: + - s19-t33t211-tg1-c2/p1 - 10GE-port1 x710-4p10GE. + - s19-t33t211-tg1-c2/p2 - 10GE-port2 x710-4p10GE. + - s19-t33t211-tg1-c2/p3 - 10GE-port3 x710-4p10GE. + - s19-t33t211-tg1-c2/p4 - 10GE-port4 x710-4p10GE. + - s19-t33t211-tg1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s19-t33t211-tg1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s19-t33t211-tg1-c8/p1 - 40GE-port1 xl710-QDA2-2p40GE. + - s19-t33t211-tg1-c8/p2 - 40GE-port2 xl710-QDA2-2p40GE. + - s19-t33t211-tg1-c10/p1 - 10GE-port1 x710-4p10GE. + - s19-t33t211-tg1-c10/p2 - 10GE-port2 x710-4p10GE. + - s19-t33t211-tg1-c10/p3 - 10GE-port3 x710-4p10GE. + - s19-t33t211-tg1-c10/p4 - 10GE-port4 x710-4p10GE. +``` + +#### 2-Node-Icelake Servers (2n-icx) PROD + +``` +- SUT [Server-Type-F1]: + - testbedname: testbed212. + - hostname: s71-t212-sut1. + - IPMI IP: 10.30.50.81 + - Host IP: 10.30.51.81 + - portnames: + - s71-t212-sut1-c2/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s71-t212-sut1-c2/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s71-t212-sut1-c4/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s71-t212-sut1-c4/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s71-t212-sut1-c4/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s71-t212-sut1-c4/p4 - 25GE-port4 e810-XXVDA4-4p25GE. + - s71-t212-sut1-c9/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s71-t212-sut1-c9/p2 - 100GE-port2 e810-2CQDA2-2p100GE. +- TG [Server-Type-F2]: + - testbedname: testbed212. + - hostname: s72-t212-tg1. + - IPMI IP: 10.30.50.82 + - Host IP: 10.30.51.82 + - portnames: + - s72-t212-tg1-c2/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s72-t212-tg1-c2/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s72-t212-tg1-c4/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s72-t212-tg1-c4/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s72-t212-tg1-c4/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s72-t212-tg1-c4/p4 - 25GE-port4 e810-XXVDA4-4p25GE. + - s72-t212-tg1-c9/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s72-t212-tg1-c9/p2 - 100GE-port2 e810-2CQDA2-2p100GE. + - s72-t212-tg1-c6/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s72-t212-tg1-c6/p2 - 100GE-port2 e810-2CQDA2-2p100GE. +- SUT [Server-Type-F1]: + - testbedname: testbed213. + - hostname: s83-t213-sut1. + - IPMI IP: 10.30.50.83 + - Host IP: 10.30.51.83 + - portnames: + - s83-t213-sut1-c2/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s83-t213-sut1-c2/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s83-t213-sut1-c4/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s83-t213-sut1-c4/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s83-t213-sut1-c4/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s83-t213-sut1-c4/p4 - 25GE-port4 e810-XXVDA4-4p25GE. + - s83-t213-sut1-c9/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s83-t213-sut1-c9/p2 - 100GE-port2 e810-2CQDA2-2p100GE. +- TG [Server-Type-F2]: + - testbedname: testbed213. + - hostname: s84-t213-tg1. + - IPMI IP: 10.30.50.84 + - Host IP: 10.30.51.84 + - portnames: + - s84-t213-tg1-c2/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s84-t213-tg1-c2/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s84-t213-tg1-c4/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s84-t213-tg1-c4/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s84-t213-tg1-c4/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s84-t213-tg1-c4/p4 - 25GE-port4 e810-XXVDA4-4p25GE. + - s84-t213-tg1-c9/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s84-t213-tg1-c9/p2 - 100GE-port2 e810-2CQDA2-2p100GE. + - s84-t213-tg1-c6/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s84-t213-tg1-c6/p2 - 100GE-port2 e810-2CQDA2-2p100GE. +- SUT [Server-Type-F1]: + - testbedname: testbed214. + - hostname: s85-t214-sut1. + - IPMI IP: 10.30.50.85 + - Host IP: 10.30.51.85 + - portnames: + - s85-t214-sut1-c2/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s85-t214-sut1-c2/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s85-t214-sut1-c4/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s85-t214-sut1-c4/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s85-t214-sut1-c4/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s85-t214-sut1-c4/p4 - 25GE-port4 e810-XXVDA4-4p25GE. + - s85-t214-sut1-c9/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s85-t214-sut1-c9/p2 - 100GE-port2 e810-2CQDA2-2p100GE. +- TG [Server-Type-F2]: + - testbedname: testbed214. + - hostname: s86-t214-tg1. + - IPMI IP: 10.30.50.86 + - Host IP: 10.30.51.86 + - portnames: + - s86-t214-tg1-c2/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s86-t214-tg1-c2/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s86-t214-tg1-c4/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s86-t214-tg1-c4/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s86-t214-tg1-c4/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s86-t214-tg1-c4/p4 - 25GE-port4 e810-XXVDA4-4p25GE. + - s86-t214-tg1-c9/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s86-t214-tg1-c9/p2 - 100GE-port2 e810-2CQDA2-2p100GE. + - s86-t214-tg1-c6/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s86-t214-tg1-c6/p2 - 100GE-port2 e810-2CQDA2-2p100GE. +- SUT [Server-Type-F1]: + - testbedname: testbed215. + - hostname: s87-t215-sut1. + - IPMI IP: 10.30.50.87 + - Host IP: 10.30.51.87 + - portnames: + - s87-t215-sut1-c2/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s87-t215-sut1-c2/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s87-t215-sut1-c4/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s87-t215-sut1-c4/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s87-t215-sut1-c4/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s87-t215-sut1-c4/p4 - 25GE-port4 e810-XXVDA4-4p25GE. + - s87-t215-sut1-c9/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s87-t215-sut1-c9/p2 - 100GE-port2 e810-2CQDA2-2p100GE. +- TG [Server-Type-F2]: + - testbedname: testbed215. + - hostname: s88-t215-tg1. + - IPMI IP: 10.30.50.88 + - Host IP: 10.30.51.88 + - portnames: + - s88-t215-tg1-c2/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s88-t215-tg1-c2/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s88-t215-tg1-c4/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s88-t215-tg1-c4/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s88-t215-tg1-c4/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s88-t215-tg1-c4/p4 - 25GE-port4 e810-XXVDA4-4p25GE. + - s88-t215-tg1-c9/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s88-t215-tg1-c9/p2 - 100GE-port2 e810-2CQDA2-2p100GE. + - s88-t215-tg1-c6/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s88-t215-tg1-c6/p2 - 100GE-port2 e810-2CQDA2-2p100GE. +``` + +#### 3-Node-Rangeley Servers (3n-rng) VERIFY + +Note: There is no IPMI. Serial console is accessible via VIRL2 and VIRL3 USB. + +``` +- ServerB22 [Server-Type-B5]: + - testbedname: testbed35. + - hostname: s22-t35-sut1 (vex-yul-rot-netgate-1). + - IPMI IP: 10.30.51.29 - screen -r /dev/ttyUSB0 + - Host IP: 10.30.51.9 + - portnames: + - s22-t35-sut1-p1 - 10GE-port1 ix0 82599. + - s22-t35-sut1-p2 - 10GE-port2 ix1 82599. + - 1GB ports (tbd) +- ServerB23 [Server-Type-B5]: + - testbedname: testbed35. + - hostname: s23-t35-sut2 (vex-yul-rot-netgate-2). + - IPMI IP: 10.30.51.30 - screen -r /dev/ttyUSB1 + - Host IP: 10.30.51.10 + - portnames: + - s23-t35-sut1-p1 - 10GE-port1 ix0 82599. + - s23-t35-sut1-p2 - 10GE-port2 ix1 82599. + - 1GB ports (tbd) +- ServerB24 [Server-Type-B5]: + - testbedname: testbed35. + - hostname: s24-t35-sut3 (vex-yul-rot-netgate-3). + - IPMI IP: 10.30.51.30 - screen -r /dev/ttyUSB2 + - Host IP: 10.30.51.11 + - portnames: + - s24-t35-sut1-p1 - 10GE-port1 ix0 82599. + - s24-t35-sut1-p2 - 10GE-port2 ix1 82599. + - 1GB ports (tbd) +``` + +#### 3-Node-Taishan Servers (3n-tsh) PROD + +Note: Server19 (TG) is shared between testbed33 & testbed211 + +``` +- SUT [Server-Type-E21]: + - testbedname: testbed33. + - hostname: s17-t33-sut1. + - IPMI IP: 10.30.50.36 + - Host IP: 10.30.51.36 + - portnames: + - s17-t33-sut1-c6/p1 - 10GE-port1 x520-2p10GE. + - s17-t33-sut1-c6/p2 - 10GE-port2 x520-2p10GE. + - s17-t33-sut1-c4/p1 - 25GE-port1 cx4-2p25GE. + - s17-t33-sut1-c4/p2 - 25GE-port2 cx4-2p25GE. +- SUT [Server-Type-E21]: + - testbedname: testbed33. + - hostname: s18-t33-sut2. + - IPMI IP: 10.30.50.37 + - Host IP: 10.30.51.37 + - portnames: + - s18-t33-sut2-c6/p1 - 10GE-port1 x520-2p10GE. + - s18-t33-sut2-c6/p2 - 10GE-port2 x520-2p10GE. + - s18-t33-sut2-c4/p1 - 25GE-port1 cx4-2p25GE. + - s18-t33-sut2-c4/p2 - 25GE-port2 cx4-2p25GE. +- TG [Server-Type-E31]: + - testbedname: testbed33 and testbed211. + - hostname: s19-t33t211-tg1. + - IPMI IP: 10.30.50.46 + - Host IP: 10.30.51.49 + - portnames: + - s19-t33t211-tg1-c2/p1 - 10GE-port1 x710-4p10GE. + - s19-t33t211-tg1-c2/p2 - 10GE-port2 x710-4p10GE. + - s19-t33t211-tg1-c2/p3 - 10GE-port3 x710-4p10GE. + - s19-t33t211-tg1-c2/p4 - 10GE-port4 x710-4p10GE. + - s19-t33t211-tg1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s19-t33t211-tg1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s19-t33t211-tg1-c8/p1 - 40GE-port1 xl710-QDA2-2p40GE. + - s19-t33t211-tg1-c8/p2 - 40GE-port2 xl710-QDA2-2p40GE. + - s19-t33t211-tg1-c10/p1 - 10GE-port1 x710-4p10GE. + - s19-t33t211-tg1-c10/p2 - 10GE-port2 x710-4p10GE. + - s19-t33t211-tg1-c10/p3 - 10GE-port3 x710-4p10GE. + - s19-t33t211-tg1-c10/p4 - 10GE-port4 x710-4p10GE. +``` + +#### 3-Node-Altra Servers (3n-alt) PROD + +``` +- SUT [Server-Type-E23]: + - testbedname: testbed34. + - hostname: s62-t34-sut1. + - IPMI IP: 10.30.50.72 + - Host IP: 10.30.51.72 + - portnames: + - s62-t34-sut1-c1/p1 - 40GE-port1 xl710-QDA2-2p40GE. + - s62-t34-sut1-c1/p2 - 40GE-port2 xl710-QDA2-2p40GE. +- SUT [Server-Type-E23]: + - testbedname: testbed34. + - hostname: s63-t34-sut2. + - IPMI IP: 10.30.50.73 + - Host IP: 10.30.51.73 + - portnames: + - s63-t34-sut2-c1/p1 - 40GE-port1 xl710-QDA2-2p40GE. + - s63-t34-sut2-c1/p2 - 40GE-port2 xl710-QDA2-2p40GE. +- TG [Server-Type-F4]: + - testbedname: testbed34. + - hostname: s64-t34-tg1. + - IPMI IP: 10.30.50.74 + - Host IP: 10.30.51.74 + - portnames: + - s64-t34-tg1-c2/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s64-t34-tg1-c2/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s64-t34-tg1-c4/p1 - 40GE-port1 xl710-QDA2-2p40GE. + - s64-t34-tg1-c4/p2 - 40GE-port2 xl710-QDA2-2p40GE. + - s64-t34-tg1-c6/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s64-t34-tg1-c6/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s64-t34-tg1-c6/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s64-t34-tg1-c6/p4 - 25GE-port4 e810-XXVDA4-4p25GE. + - s64-t34-tg1-c8/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s64-t34-tg1-c8/p2 - 100GE-port2 e810-2CQDA2-2p100GE. +``` + +#### 3-Node-Icelake Servers (3n-icx) PROD + +``` +- ServerF1 [Server-Type-F1]: + - testbedname: testbed37. + - hostname: s65-t37-sut1. + - IPMI IP: 10.30.50.75 + - Host IP: 10.30.51.75 + - portnames: + - s65-t37-sut1-c2/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s65-t37-sut1-c2/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s65-t37-sut1-c4/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s65-t37-sut1-c4/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s65-t37-sut1-c4/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s65-t37-sut1-c4/p4 - 25GE-port4 e810-XXVDA4-4p25GE. + - s65-t37-sut1-c9/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s65-t37-sut1-c9/p2 - 100GE-port2 e810-2CQDA2-2p100GE. +- ServerF1 [Server-Type-F1]: + - testbedname: testbed37. + - hostname: s66-t37-sut2. + - IPMI IP: 10.30.50.76 + - Host IP: 10.30.51.76 + - portnames: + - s66-t37-sut2-c2/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s66-t37-sut2-c2/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s66-t37-sut2-c4/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s66-t37-sut2-c4/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s66-t37-sut2-c4/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s66-t37-sut2-c4/p4 - 25GE-port4 e810-XXVDA4-4p25GE. + - s66-t37-sut2-c9/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s66-t37-sut2-c9/p2 - 100GE-port2 e810-2CQDA2-2p100GE. +- ServerF3 [Server-Type-F3]: + - testbedname: testbed37. + - hostname: s67-t37-tg1. + - IPMI IP: 10.30.50.77 + - Host IP: 10.30.51.77 + - portnames: + - s67-t37-tg1-c2/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s67-t37-tg1-c2/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s67-t37-tg1-c4/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s67-t37-tg1-c4/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s67-t37-tg1-c4/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s67-t37-tg1-c4/p4 - 25GE-port4 e810-XXVDA4-4p25GE. + - s67-t37-tg1-c9/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s67-t37-tg1-c9/p2 - 100GE-port2 e810-2CQDA2-2p100GE. +- ServerF1 [Server-Type-F1]: + - testbedname: testbed38. + - hostname: s78-t38-sut1. + - IPMI IP: 10.30.50.78 + - Host IP: 10.30.51.78 + - portnames: + - s78-t38-sut1-c2/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s78-t38-sut1-c2/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s78-t38-sut1-c4/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s78-t38-sut1-c4/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s78-t38-sut1-c4/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s78-t38-sut1-c4/p4 - 25GE-port4 e810-XXVDA4-4p25GE. + - s78-t38-sut1-c9/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s78-t38-sut1-c9/p2 - 100GE-port2 e810-2CQDA2-2p100GE. +- ServerF1 [Server-Type-F1]: + - testbedname: testbed38. + - hostname: s79-t38-sut2. + - IPMI IP: 10.30.50.79 + - Host IP: 10.30.51.79 + - portnames: + - s79-t38-sut2-c2/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s79-t38-sut2-c2/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s79-t38-sut2-c4/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s79-t38-sut2-c4/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s79-t38-sut2-c4/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s79-t38-sut2-c4/p4 - 25GE-port4 e810-XXVDA4-4p25GE. + - s79-t38-sut2-c9/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s79-t38-sut2-c9/p2 - 100GE-port2 e810-2CQDA2-2p100GE. +- ServerF3 [Server-Type-F3]: + - testbedname: testbed38. + - hostname: s80-t38-tg1. + - IPMI IP: 10.30.50.80 + - Host IP: 10.30.51.80 + - portnames: + - s80-t38-tg1-c2/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s80-t38-tg1-c2/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s80-t38-tg1-c4/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s80-t38-tg1-c4/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s80-t38-tg1-c4/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s80-t38-tg1-c4/p4 - 25GE-port4 e810-XXVDA4-4p25GE. + - s80-t38-tg1-c9/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s80-t38-tg1-c9/p2 - 100GE-port2 e810-2CQDA2-2p100GE. +``` + +#### 3-Node-SnowRidge Servers (3n-snr) PROD + +``` +- ServerG1 [Server-Type-G1]: + - testbedname: testbed39. + - hostname: s93-t39-sut1. + - IPMI IP: 10.30.50.93 + - Host IP: 10.30.51.93 + - portnames: + - s93-t39-sut1-c1/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s93-t39-sut1-c1/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s93-t39-sut1-c1/p2 - 25GE-port3 e810-XXVDA4-4p25GE. + - s93-t39-sut1-c1/p2 - 25GE-port4 e810-XXVDA4-4p25GE. +- ServerG1 [Server-Type-G1]: + - testbedname: testbed39. + - hostname: s94-t39-sut2. + - IPMI IP: 10.30.50.94 + - Host IP: 10.30.51.94 + - portnames: + - s94-t39-sut2-c1/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s94-t39-sut2-c1/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s94-t39-sut2-c1/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s94-t39-sut2-c1/p4 - 25GE-port4 e810-XXVDA4-4p25GE. +- ServerF4 [Server-Type-F4]: + - testbedname: testbed39. + - hostname: s89-t39t310-tg1. + - IPMI IP: 10.30.50.89 + - Host IP: 10.30.51.89 + - portnames: + - s89-t39t310-tg1-c6/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s89-t39t310-tg1-c6/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s89-t39t310-tg1-c6/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s89-t39t310-tg1-c6/p4 - 25GE-port4 e810-XXVDA4-4p25GE. +``` + +#### 2-Node-SapphireRapids Servers (2n-spr) PROD + +``` +- SUT [Server-Type-H1]: + - testbedname: testbed21. + - hostname: s52-t21-sut1. + - IPMI IP: 10.30.50.52 + - Host IP: 10.30.51.52 + - portnames: + - s52-t21-sut1-c2/p1 - 200GE-port1 ConnectX7-2p200GE. + - s52-t21-sut1-c2/p2 - 200GE-port2 ConnectX7-2p200GE. + - s52-t21-sut1-c4/p1 - 200GE-port1 ConnectX7-2p200GE. + - s52-t21-sut1-c4/p2 - 200GE-port2 ConnectX7-2p200GE. + - s52-t21-sut1-c10/p1 - 200GE-port1 ConnectX7-2p200GE. + - s52-t21-sut1-c10/p2 - 200GE-port2 ConnectX7-2p200GE. + - s52-t21-sut1-c7/p1 - 200GE-port1 ConnectX7-2p200GE. + - s52-t21-sut1-c7/p2 - 200GE-port2 ConnectX7-2p200GE. + - s52-t21-sut1-c9/p1 - 200GE-port1 ConnectX7-2p200GE. + - s52-t21-sut1-c9/p2 - 200GE-port2 ConnectX7-2p200GE. + - s52-t21-sut1-c11/p1 - 200GE-port1 ConnectX7-2p200GE. + - s52-t21-sut1-c11/p2 - 200GE-port2 ConnectX7-2p200GE. +- TG [Server-Type-H2]: + - testbedname: testbed21. + - hostname: s53-t21-tg1. + - IPMI IP: 10.30.50.53 + - Host IP: 10.30.51.53 + - portnames: + - s53-t21-tg1-c2/p1 - 200GE-port1 ConnectX7-2p200GE. + - s53-t21-tg1-c2/p2 - 200GE-port2 ConnectX7-2p200GE. + - s53-t21-tg1-c4/p1 - 200GE-port1 ConnectX7-2p200GE. + - s53-t21-tg1-c4/p2 - 200GE-port2 ConnectX7-2p200GE. + - s53-t21-tg1-c10/p1 - 200GE-port1 ConnectX7-2p200GE. + - s53-t21-tg1-c10/p2 - 200GE-port2 ConnectX7-2p200GE. + - s53-t21-tg1-c7/p1 - 200GE-port1 ConnectX7-2p200GE. + - s53-t21-tg1-c7/p2 - 200GE-port2 ConnectX7-2p200GE. + - s53-t21-tg1-c9/p1 - 200GE-port1 ConnectX7-2p200GE. + - s53-t21-tg1-c9/p2 - 200GE-port2 ConnectX7-2p200GE. +- SUT [Server-Type-H3]: + - testbedname: testbed22. + - hostname: s54-t22-sut1. + - IPMI IP: 10.30.50.54 + - Host IP: 10.30.51.54 + - portnames: + - s54-t22-sut1-c2/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s54-t22-sut1-c2/p2 - 100GE-port2 e810-2CQDA2-2p100GE. + - s54-t22-sut1-c4/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s54-t22-sut1-c4/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s54-t22-sut1-c4/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s54-t22-sut1-c4/p4 - 25GE-port4 e810-XXVDA4-4p25GE. + - s54-t22-sut1-c7/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s54-t22-sut1-c7/p2 - 100GE-port2 e810-2CQDA2-2p100GE. + - s54-t22-sut1-c9/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s54-t22-sut1-c9/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s54-t22-sut1-c9/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s54-t22-sut1-c9/p4 - 25GE-port4 e810-XXVDA4-4p25GE. +- TG [Server-Type-H4]: + - testbedname: testbed22. + - hostname: s55-t22-tg1. + - IPMI IP: 10.30.50.55 + - Host IP: 10.30.51.55 + - portnames: + - s55-t22-tg1-c2/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s55-t22-tg1-c2/p2 - 100GE-port2 e810-2CQDA2-2p100GE. + - s55-t22-tg1-c4/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s55-t22-tg1-c4/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s55-t22-tg1-c4/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s55-t22-tg1-c4/p4 - 25GE-port4 e810-XXVDA4-4p25GE. +- SUT [Server-Type-H5]: + - testbedname: testbed23. + - hostname: s56-t23-sut1. + - IPMI IP: 10.30.50.56 + - Host IP: 10.30.51.56 + - portnames: + - s56-t23-sut1-c2/p1 - 200GE-port1 ConnectX7-2p200GE. + - s56-t23-sut1-c2/p2 - 200GE-port2 ConnectX7-2p200GE. + - s56-t23-sut1-c4/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s56-t23-sut1-c4/p2 - 100GE-port2 e810-2CQDA2-2p100GE. + - s56-t23-sut1-c10/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s56-t23-sut1-c10/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s56-t23-sut1-c10/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s56-t23-sut1-c10/p4 - 25GE-port4 e810-XXVDA4-4p25GE. +- TG [Server-Type-H6]: + - testbedname: testbed23. + - hostname: s57-t23-tg1. + - IPMI IP: 10.30.50.57 + - Host IP: 10.30.51.57 + - portnames: + - s57-t23-tg1-c2/p1 - 200GE-port1 ConnectX7-2p200GE. + - s57-t23-tg1-c2/p2 - 200GE-port2 ConnectX7-2p200GE. + - s57-t23-tg1-c4/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s57-t23-tg1-c4/p2 - 100GE-port2 e810-2CQDA2-2p100GE. + - s57-t23-tg1-c10/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s57-t23-tg1-c10/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s57-t23-tg1-c10/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s57-t23-tg1-c10/p4 - 25GE-port4 e810-XXVDA4-4p25GE. + - s57-t23-tg1-c7/p1 - 200GE-port1 ConnectX7-2p200GE. + - s57-t23-tg1-c7/p2 - 200GE-port2 ConnectX7-2p200GE. + - s57-t23-tg1-c9/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s57-t23-tg1-c9/p2 - 100GE-port2 e810-2CQDA2-2p100GE. +- SUT [Server-Type-H5]: + - testbedname: testbed24. + - hostname: s58-t24-sut1. + - IPMI IP: 10.30.50.58 + - Host IP: 10.30.51.58 + - portnames: + - s58-t24-sut1-c2/p1 - 200GE-port1 ConnectX7-2p200GE. + - s58-t24-sut1-c2/p2 - 200GE-port2 ConnectX7-2p200GE. + - s58-t24-sut1-c4/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s58-t24-sut1-c4/p2 - 100GE-port2 e810-2CQDA2-2p100GE. + - s58-t24-sut1-c10/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s58-t24-sut1-c10/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s58-t24-sut1-c10/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s58-t24-sut1-c10/p4 - 25GE-port4 e810-XXVDA4-4p25GE. +- TG [Server-Type-H6]: + - testbedname: testbed24. + - hostname: s59-t24-tg1. + - IPMI IP: 10.30.50.59 + - Host IP: 10.30.51.59 + - portnames: + - s59-t24-tg1-c2/p1 - 200GE-port1 ConnectX7-2p200GE. + - s59-t24-tg1-c2/p2 - 200GE-port2 ConnectX7-2p200GE. + - s59-t24-tg1-c4/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s59-t24-tg1-c4/p2 - 100GE-port2 e810-2CQDA2-2p100GE. + - s59-t24-tg1-c10/p1 - 25GE-port1 e810-XXVDA4-4p25GE. + - s59-t24-tg1-c10/p2 - 25GE-port2 e810-XXVDA4-4p25GE. + - s59-t24-tg1-c10/p3 - 25GE-port3 e810-XXVDA4-4p25GE. + - s59-t24-tg1-c10/p4 - 25GE-port4 e810-XXVDA4-4p25GE. + - s59-t24-tg1-c7/p1 - 200GE-port1 ConnectX7-2p200GE. + - s59-t24-tg1-c7/p2 - 200GE-port2 ConnectX7-2p200GE. + - s59-t24-tg1-c9/p1 - 100GE-port1 e810-2CQDA2-2p100GE. + - s59-t24-tg1-c9/p2 - 100GE-port2 e810-2CQDA2-2p100GE. +``` + +### Per Testbed Wiring + +#### 1-Node-Skylake Wiring (1n-skx) PROD + +``` +- testbed11: + - ring1 10GE-ports x710-4p10GE: + - s1-t11-sut1-c2/p1 to s1-t11-sut1-c4/p1. + - ring2 10GE-ports x710-4p10GE: + - s1-t11-sut1-c2/p2 to s1-t11-sut1-c4/p2. + - ring3 10GE-ports x710-4p10GE: + - s1-t11-sut1-c2/p3 to s1-t11-sut1-c4/p3. + - ring4 10GE-ports x710-4p10GE: + - s1-t11-sut1-c2/p4 to s1-t11-sut1-c4/p4. + - ring5 100GE-ports e810-2p100GE: + - s1-t11-sut1-c5/p1 to s1-t11-sut1-c6/p1. + - ring6 100GE-ports e810-2p100GE: + - s1-t11-sut1-c5/p2 to s1-t11-sut1-c6/p2. +- testbed12: + - ring1 10GE-ports x710-4p10GE: + - s2-t12-sut1-c2/p1 to s2-t12-sut1-c4/p1. + - ring2 10GE-ports x710-4p10GE: + - s2-t12-sut1-c2/p2 to s2-t12-sut1-c4/p2. + - ring3 10GE-ports x710-4p10GE: + - s2-t12-sut1-c2/p3 to s2-t12-sut1-c4/p3. + - ring4 10GE-ports x710-4p10GE: + - s2-t12-sut1-c2/p4 to s2-t12-sut1-c4/p4. + - ring5 100GE-ports e810-2p100GE: + - s2-t12-sut1-c5/p1 to s2-t12-sut1-c6/p1. + - ring6 100GE-ports e810-2p100GE: + - s2-t12-sut1-c5/p2 to s2-t12-sut1-c6/p2. +``` + +#### 1-Node-ThunderX2 Wiring (1n-tx2) PROD + +``` +- testbed13: + - ring1 40GE-ports XL710-QDA2-2p40GE on SUTs: + - s55-t13-sut1-c4/p1 - s55-t13-sut1-c14/p1. + - ring2 40GE-ports XL710-QDA2-2p40GE on SUTs: + - s55-t13-sut1-c4/p2 - s55-t13-sut1-c14/p2. + - ring3 10/25GE-ports ConnectX5-2p10/25GE on SUTs: + - s55-t13-sut1-c8/p1 - s55-t13-sut1-c26/p1. + - ring4 10/25GE-ports ConnectX5-2p10/25GE on SUTs: + - s55-t13-sut1-c8/p2 - s55-t13-sut1-c26/p2. + +- testbed14: + - ring1 40GE-ports XL710-QDA2-2p40GE on SUTs: + - s56-t14-sut1-c4/p1 - s56-t14-sut1-c14/p1. + - ring2 40GE-ports XL710-QDA2-2p40GE on SUTs: + - s56-t14-sut1-c4/p2 - s56-t14-sut1-c14/p2. + - ring3 10/25GE-ports ConnectX5-2p10/25GE on SUTs: + - s56-t14-sut1-c8/p1 - s56-t14-sut1-c26/p1. + - ring4 10/25GE-ports ConnectX5-2p10/25GE on SUTs: + - s56-t14-sut1-c8/p2 - s56-t14-sut1-c26/p2. +``` + +#### 2-Node-IxiaPS1L47 Wiring (2n-ps1) VERIFY + +``` +- testbed25: + - link1 10GE-port x710-4p10GE on SUT: + - t25-tg1-p1 to t25-sut1-c2/p1. + - link2 10GE-port x710-4p10GE on SUT: + - t25-tg1-p2 to t25-sut1-c2/p2. + - link3 10GE-port x710-4p10GE on SUT: + - t25-tg1-p3 to t25-sut1-c2/p3. + - link4 10GE-port x710-4p10GE on SUT: + - t25-tg1-p4 to t25-sut1-c2/p4. +``` + +#### 2-Node-Cascadelake Wiring (2n-clx) PROD + +``` +- testbed27: + - ring1 10GE-ports x710-4p10GE on SUT: + - s34-t27-tg1-c2/p1 to s33-t27-sut1-c2/p1. + - s33-t27-sut1-c2/p2 to s34-t27-tg1-c2/p2. + - ring2 10GE-ports x710-4p10GE on SUT: + - s34-t27-tg1-c2/p3 to s33-t27-sut1-c2/p3. + - s33-t27-sut1-c2/p4 to s34-t27-tg1-c2/p4. + - ring3 25GE-ports xxv710-DA2-2p25GE on SUT + - s34-t27-tg1-c4/p1 to s33-t27-sut1-c4/p1. + - s33-t27-sut1-c4/p2 to s34-t27-tg1-c4/p2. + - ring4 100GE-ports ConnectX5-2p100GE on SUT: + - s34-t27-tg1-c9/p1 to s33-t27-sut1-c9/p1. + - s33-t27-sut1-c9/p2 to s34-t27-tg1-c9/p2. + - ring5 100GE-ports e810-2p100GE on SUT 100GE-ports ConnectX5-2p100GE on TG: + - s34-t27-tg1-c6/p1 to s33-t27-sut1-c6/p1. + - s33-t27-sut1-c6/p2 to s34-t27-tg1-c6/p2. + - ring6 100GE-ports e810-2p100GE on TG: + - s34-t27-tg1-c8/p1 to s34-t27-tg1-c8/p2. + - s34-t27-tg1-c8/p2 to s34-t27-tg1-c8/p1. +- testbed28: + - ring1 10GE-ports x710-4p10GE on SUT: + - s36-t28-tg1-c2/p1 to s35-t28-sut1-c2/p1. + - s35-t28-sut1-c2/p2 to s36-t28-tg1-c2/p2. + - ring2 10GE-ports x710-4p10GE on SUT: + - s36-t28-tg1-c2/p3 to s35-t28-sut1-c2/p3. + - s35-t28-sut1-c2/p4 to s36-t28-tg1-c2/p4. + - ring3 25GE-ports xxv710-DA2-2p25GE on SUT + - s36-t28-tg1-c4/p1 to s35-t28-sut1-c4/p1. + - s35-t28-sut1-c4/p2 to s36-t28-tg1-c4/p2. + - ring4 100GE-ports ConnectX5-2p100GE on SUT: + - s36-t28-tg1-c9/p1 to s35-t28-sut1-c9/p1. + - s35-t28-sut1-c9/p2 to s36-t28-tg1-c9/p2. + - ring5 100GE-ports e810-2p100GE on SUT 100GE-ports ConnectX5-2p100GE on TG: + - s36-t28-tg1-c6/p1 to s35-t28-sut1-c6/p1. + - s35-t28-sut1-c6/p2 to s36-t28-tg1-c6/p2. + - ring6 100GE-ports e810-2p100GE on TG: + - s36-t28-tg1-c8/p1 to s36-t28-tg1-c8/p2. + - s36-t28-tg1-c8/p2 to s36-t28-tg1-c8/p1. +- testbed29: + - ring1 10GE-ports x710-4p10GE on SUT: + - s38-t29-tg1-c2/p1 to s37-t29-sut1-c2/p1. + - s37-t29-sut1-c2/p2 to s38-t29-tg1-c2/p2. + - ring2 10GE-ports x710-4p10GE on SUT: + - s38-t29-tg1-c2/p3 to s37-t29-sut1-c2/p3. + - s37-t29-sut1-c2/p4 to s38-t29-tg1-c2/p4. + - ring3 25GE-ports xxv710-DA2-2p25GE on SUT + - s38-t29-tg1-c4/p1 to s37-t29-sut1-c4/p1. + - s37-t29-sut1-c4/p2 to s38-t29-tg1-c4/p2. + - ring4 100GE-ports ConnectX5-2p100GE on SUT: + - s38-t29-tg1-c9/p1 to s37-t29-sut1-c9/p1. + - s37-t29-sut1-c9/p2 to s38-t29-tg1-c9/p2. + - ring5 100GE-ports e810-2p100GE on SUT 100GE-ports ConnectX5-2p100GE on TG: + - s38-t29-tg1-c6/p1 to s37-t29-sut1-c6/p1. + - s37-t29-sut1-c6/p2 to s38-t29-tg1-c6/p2. +``` + +#### 2-Node-Zen2 Wiring (2n-zn2) PROD + +``` +- testbed210: + - ring1 10GE-ports x710-4p10GE on SUT: + - s61-t210-tg1-c2/p1 to s60-t210-sut1-c1/p1. + - s60-t210-sut1-c1/p2 to s61-t210-tg1-c2/p2. + - ring2 10GE-ports x710-4p10GE on SUT: + - s61-t210-tg1-c2/p3 to s60-t210-sut1-c1/p3. + - s60-t210-sut1-c1/p4 to s61-t210-tg1-c2/p4. + - ring3 25GE-ports xxv710-DA2-2p25GE on SUT + - s61-t210-tg1-c3/p1 to s60-t210-sut1-c2/p1. + - s60-t210-sut1-c2/p2 to s61-t210-tg1-c3/p2. + - ring4 100GE-ports ConnectX5-2p100GE on SUT: + - s61-t210-tg1-c1/p1 to s60-t210-sut1-c3/p1. + - s60-t210-sut1-c3/p2 to s61-t210-tg1-c1/p2. +``` + +#### 2-Node-ThunderX2 Wiring (2n-tx2) PROD + +``` +- testbed211: + - ring1 10GE-ports x520-2p10GE on SUTs: + - s27-t211-sut1-c18/p1 - s19-t33t211-tg1-c8/p1. + - s27-t211-sut1-c18/p2 - s19-t33t211-tg1-c8/p2. +``` + +#### 2-Node-Icelake Wiring (2n-icx) PROD + +``` +- testbed212: + - ring1 25GE-ports xxv710-DA2-2p25GE on SUT + - s72-t212-tg1-c2/p1 to s71-t212-sut1-c2/p1. + - s71-t212-sut1-c2/p2 to s72-t212-tg1-c2/p2. + - ring2 25GE-ports e810-XXVDA4-2p25GE on SUT: + - s72-t212-tg1-c4/p1 to s71-t212-sut1-c4/p1. + - s71-t212-sut1-c4/p2 to s72-t212-tg1-c4/p2. + - s72-t212-tg1-c4/p3 to s71-t212-sut1-c4/p3. + - s71-t212-sut1-c4/p4 to s72-t212-tg1-c4/p4. + - ring3 100GE-ports e810-2CQDA2-2p100GE on SUT: + - s72-t212-tg1-c9/p1 to s71-t212-sut1-c9/p1. + - s71-t212-sut1-c9/p2 to s72-t212-tg1-c9/p2. + - ring4 100GE-ports e810-2CQDA2-2p100GE on SUT: + - s72-t212-tg1-c6/p1 to s72-t212-tg1-c6/p2. + - s72-t212-tg1-c6/p2 to s72-t212-tg1-c6/p1. +- testbed213: + - ring1 25GE-ports xxv710-DA2-2p25GE on SUT + - s84-t213-tg1-c2/p1 to s83-t213-sut1-c2/p1. + - s83-t213-sut1-c2/p2 to s84-t213-tg1-c2/p2. + - ring2 25GE-ports e810-XXVDA4-2p25GE on SUT: + - s84-t213-tg1-c4/p1 to s83-t213-sut1-c4/p1. + - s83-t213-sut1-c4/p2 to s84-t213-tg1-c4/p2. + - s84-t213-tg1-c4/p3 to s83-t213-sut1-c4/p3. + - s83-t213-sut1-c4/p4 to s84-t213-tg1-c4/p4. + - ring3 100GE-ports e810-2CQDA2-2p100GE on SUT: + - s84-t213-tg1-c9/p1 to s83-t213-sut1-c9/p1. + - s83-t213-sut1-c9/p2 to s84-t213-tg1-c9/p2. + - ring4 100GE-ports e810-2CQDA2-2p100GE on SUT: + - s84-t213-tg1-c6/p1 to s84-t213-tg1-c6/p2. + - s84-t213-tg1-c6/p2 to s84-t213-tg1-c6/p1. +- testbed214: + - ring1 25GE-ports xxv710-DA2-2p25GE on SUT + - s86-t214-tg1-c2/p1 to s85-t214-sut1-c2/p1. + - s85-t214-sut1-c2/p2 to s86-t214-tg1-c2/p2. + - ring2 25GE-ports e810-XXVDA4-2p25GE on SUT: + - s86-t214-tg1-c4/p1 to s85-t214-sut1-c4/p1. + - s85-t214-sut1-c4/p2 to s86-t214-tg1-c4/p2. + - s86-t214-tg1-c4/p3 to s85-t214-sut1-c4/p3. + - s85-t214-sut1-c4/p4 to s86-t214-tg1-c4/p4. + - ring3 100GE-ports e810-2CQDA2-2p100GE on SUT: + - s86-t214-tg1-c9/p1 to s85-t214-sut1-c9/p1. + - s85-t214-sut1-c9/p2 to s86-t214-tg1-c9/p2. + - ring4 100GE-ports e810-2CQDA2-2p100GE on SUT: + - s86-t214-tg1-c6/p1 to s86-t214-tg1-c6/p2. + - s86-t214-tg1-c6/p2 to s86-t214-tg1-c6/p1. +- testbed215: + - ring1 25GE-ports xxv710-DA2-2p25GE on SUT + - s88-t215-tg1-c2/p1 to s87-t215-sut1-c2/p1. + - s87-t215-sut1-c2/p2 to s88-t215-tg1-c2/p2. + - ring2 25GE-ports e810-XXVDA4-2p25GE on SUT: + - s88-t215-tg1-c4/p1 to s87-t215-sut1-c4/p1. + - s87-t215-sut1-c4/p2 to s88-t215-tg1-c4/p2. + - s88-t215-tg1-c4/p3 to s87-t215-sut1-c4/p3. + - s87-t215-sut1-c4/p4 to s88-t215-tg1-c4/p4. + - ring3 100GE-ports e810-2CQDA2-2p100GE on SUT: + - s88-t215-tg1-c9/p1 to s87-t215-sut1-c9/p1. + - s87-t215-sut1-c9/p2 to s88-t215-tg1-c9/p2. + - ring4 100GE-ports e810-2CQDA2-2p100GE on SUT: + - s88-t215-tg1-c6/p1 to s88-t215-tg1-c6/p2. + - s88-t215-tg1-c6/p2 to s88-t215-tg1-c6/p1. +``` + +#### 3-Node-Rangeley Wiring (3n-rng) VERIFY + +``` +To be completed. +``` + +#### 3-Node-Taishan Wiring (3n-tsh) PROD + +``` +- testbed33: + - ring1 10GE-ports x520-2p10GE on SUTs: + - s19-t33t211-tg1-c2/p2 - s17-t33-sut1-c6/p2. + - s17-t33-sut1-c6/p1 - s18-t33-sut2-c6/p2. + - s18-t33-sut2-c6/p1 - s19-t33t211-tg1-c2/p1. + - ring2 25GE-ports cx4-2p25GE on SUTs: + - s19-t33t211-tg1-c4/p2 - s17-t33-sut1-c4/p2. + - s17-t33-sut1-c4/p1 - s18-t33-sut2-c4/p2. + - s18-t33-sut2-c4/p1 - s19-t33t211-tg1-c4/p1. +``` + +#### 3-Node-Altra Wiring (3n-alt) PROD + +``` +- testbed34: + - ring1 40GE-ports xl710-QDA2-2p40GE on SUTs: + - s64-t34-tg1-c4/p1 - s62-t34-sut1-c1/p2. + - s62-t34-sut1-c1/p1 - s63-t34-sut2-c1/p2. + - s63-t34-sut2-c1/p1 - s64-t34-tg1-c4/p2. +``` + +#### 3-Node-Icelake Wiring (3n-icx) PROD + +``` +- testbed37: + - ring1 25GE-ports xxv710-DA2-2p25GE on SUTs: + - s67-t37-tg1-c2/p1 to s65-t37-sut1-c2/p1. + - s65-t37-sut1-c2/p2 to s66-t37-sut2-c2/p2. + - s66-t37-sut2-c2/p1 to s67-t37-tg1-c2/p2. + - ring2 25GE-ports e810-XXVDA4-4p25GE on SUT: + - s67-t37-tg1-c4/p1 to s65-t37-sut1-c4/p1. + - s65-t37-sut1-c4/p2 to s66-t37-sut2-c4/p2. + - s66-t37-sut2-c4/p1 to s67-t37-tg1-c4/p2. + - s67-t37-tg1-c4/p3 to s65-t37-sut1-c4/p3. + - s65-t37-sut1-c4/p4 to s66-t37-sut2-c4/p4. + - s66-t37-sut2-c4/p3 to s67-t37-tg1-c4/p4. + - ring3 100GE-ports e810-2CQDA2-2p100GE on SUT + - s67-t37-tg1-c9/p1 to s65-t37-sut1-c9/p1. + - s65-t37-sut1-c9/p2 to s66-t37-sut2-c9/p2. + - s66-t37-sut2-c9/p1 to s67-t37-tg1-c9/p2. +- testbed38: + - ring1 25GE-ports xxv710-DA2-2p25GE on SUTs: + - s80-t38-tg1-c2/p1 to s78-t38-sut1-c2/p1. + - s78-t38-sut1-c2/p2 to s79-t38-sut2-c2/p2. + - s79-t38-sut2-c2/p1 to s80-t38-tg1-c2/p2. + - ring2 25GE-ports e810-XXVDA4-4p25GE on SUT: + - s80-t38-tg1-c4/p1 to s78-t38-sut1-c4/p1. + - s78-t38-sut1-c4/p2 to s79-t38-sut2-c4/p2. + - s79-t38-sut2-c4/p1 to s80-t38-tg1-c4/p2. + - s80-t38-tg1-c4/p3 to s78-t38-sut1-c4/p3. + - s78-t38-sut1-c4/p4 to s79-t38-sut2-c4/p4. + - s79-t38-sut2-c4/p3 to s80-t38-tg1-c4/p4. + - ring3 100GE-ports e810-2CQDA2-2p100GE on SUT + - s80-t38-tg1-c9/p1 to s78-t38-sut1-c9/p1. + - s78-t38-sut1-c9/p2 to s79-t38-sut2-c9/p2. + - s79-t38-sut2-c9/p1 to s80-t38-tg1-c9/p2. +``` + +#### 3-Node-SnowRidge Wiring (3n-snr) PROD + +``` +- testbed39: + - ring1 25GE-ports e810-XXVDA4-4p25GE: + - s89-t39t310-tg1-c6/p1 to s93-t39-sut1-c1/p1. + - s93-t39-sut1-c1/p2 to s94-t39-sut2-c1/p2. + - s94-t39-sut2-c1/p1 to s89-t39t310-tg1-c6/p2. + - s89-t39t310-tg1-c6/p3 to s93-t39-sut1-c1/p3. + - s93-t39-sut1-c1/p4 to s94-t39-sut2-c1/p4. + - s94-t39-sut2-c1/p3 to s89-t39t310-tg1-c6/p4. +``` + +#### 2-Node-SapphireRapids Wiring (2n-spr) PROD + +``` +- testbed21: + - ring1 200GE-ports ConnectX7-2p200GE: + - s52-t21-tg1-c2/p1 to s53-t21-sut1-c2/p1 + - s52-t21-tg1-c7/p1 to s53-t21-sut1-c7/p1 + - s53-t21-sut1-c4/p2 to s53-t21-sut1-c9/p2 + - ring2 200GE-ports ConnectX7-2p200GE: + - s52-t21-tg1-c2/p2 to s53-t21-sut1-c2/p2 + - s52-t21-tg1-c7/p2 to s53-t21-sut1-c7/p2 + - s53-t21-sut1-c10/p1 to s53-t21-sut1-c11/p1 + - ring3 200GE-ports ConnectX7-2p200GE: + - s52-t21-tg1-c4/p1 to s53-t21-sut1-c4/p1 + - s52-t21-tg1-c9/p1 to s53-t21-sut1-c9/p1 + - s53-t21-sut1-c10/p2 to s53-t21-sut1-c11/p2 +- testbed22: + - ring1 100GE-ports e810-2CQDA2-2p100GE: + - s54-t22-tg1-c2/p1 to s55-t22-sut1-c2/p1 + - s54-t22-tg1-c2/p2 to s55-t22-sut1-c7/p1 + - s54-t22-sut1-c2/p2 to s55-t22-sut1-c7/p2 + - ring2 25GE-ports e810-XXVDA4-4p25GE: + - s54-t22-tg1-c4/p1 to s55-t22-sut1-c4/p1 + - s54-t22-tg1-c4/p2 to s55-t22-sut1-c9/p1 + - s54-t22-sut1-c4/p2 to s55-t22-sut1-c9/p2 +- testbed23: + - ring1 200GE-ports ConnectX7-2p200GE: + - s56-t23-sut1-c2/p1 to s57-t23-tg1-c2/p1. + - s57-t23-tg1-c2/p2 to s56-t23-sut1-c2/p2. + - ring2 100GE-ports e810-2CQDA2-2p100GE: + - s56-t23-sut1-c4/p1 to s57-t23-tg1-c4/p1. + - s57-t23-tg1-c4/p2 to s56-t23-sut1-c4/p2. + - ring3 25GE-ports e810-XXVDA4-2p25GE: + - s56-t23-sut1-c10/p1 to s57-t23-tg1-c10/p1. + - s56-t23-sut1-c10/p2 to s57-t23-tg1-c10/p2. + - s56-t23-sut1-c10/p3 to s57-t23-tg1-c10/p3. + - s56-t23-sut1-c10/p4 to s57-t23-tg1-c10/p4. + - ring4 200GE-ports ConnectX7-2p200GE: + - s57-t23-tg1-c7/p1 to s57-t23-tg1-c7/p2. + - ring5 100GE-ports e810-2CQDA2-2p100GE: + - s57-t23-tg1-c9/p1 to s57-t23-tg1-c9/p2. +- testbed24: + - ring1 200GE-ports ConnectX7-2p200GE: + - s58-t24-sut1-c2/p1 to s59-t24-tg1-c2/p1. + - s59-t24-tg1-c2/p2 to s58-t24-sut1-c2/p2. + - ring2 100GE-ports e810-2CQDA2-2p100GE: + - s58-t24-sut1-c4/p1 to s59-t24-tg1-c4/p1. + - s59-t24-tg1-c4/p2 to s58-t24-sut1-c4/p2. + - ring3 25GE-ports e810-XXVDA4-2p25GE: + - s58-t24-sut1-c10/p1 to s59-t24-tg1-c10/p1. + - s58-t24-sut1-c10/p2 to s59-t24-tg1-c10/p2. + - s58-t24-sut1-c10/p3 to s59-t24-tg1-c10/p3. + - s58-t24-sut1-c10/p4 to s59-t24-tg1-c10/p4. + - ring4 200GE-ports ConnectX7-2p200GE: + - s59-t24-tg1-c7/p1 to s59-t24-tg1-c7/p2. + - ring5 100GE-ports e810-2CQDA2-2p100GE: + - s59-t24-tg1-c9/p1 to s59-t24-tg1-c9/p2. +```
\ No newline at end of file diff --git a/docs/content/infrastructure/fdio_dc_vexxhost_inventory.md b/docs/content/infrastructure/fdio_dc_vexxhost_inventory.md new file mode 100644 index 0000000000..a0c0df52f0 --- /dev/null +++ b/docs/content/infrastructure/fdio_dc_vexxhost_inventory.md @@ -0,0 +1,118 @@ +--- +title: "FD.io DC Vexxhost Inventory" +weight: 1 +--- + +## FD.io DC Vexxhost Inventory + +- for each DC location, per rack .csv table with server inventory +- captured inventory data: name,oper-status,testbed-id,role,model,s/n,rackid,rackunit,mgmt-ip4,ipmi-ip4,new-rackid,new-rackunit,new-mgmt-ip4,new-ipmi-ip4 + - name: CSIT functional server name as tracked in [CSIT testbed specification](https://git.fd.io/csit/tree/docs/lab/testbed_specifications.md), followed by "/" and the actual configured hostname, unless it is the same as CSIT name. + - oper-status: operational status (up|down|ipmi). + - testbed-id: CSIT testbed identifier. + - role: 2n/3n-xxx performance testbed, nomad-client, nomad-server. + - role exceptions: decommission, repurpose, spare. + - model: server model. + - s/n: serial number. + - mgmt-ip4: current management IPv4 address on management VLAN. + - ipmi-ip4: current IPMI IPv4 address on LOM VLAN. + - rackid: new location rack id. + - rackunit: new location rack unit id. + +### Missing Equipment Inventory + +1. Ixia PerfectStorm One Appliance + - [Specification: Ixia PerfectStorm One Appliance TG for FD.io TCP/IP performance tests](https://git.fd.io/csit/tree/docs/lab/testbed_specifications.md#n554) + - [Wiring: 2-Node-IxiaPS1L47 Servers (2n-ps1)](https://git.fd.io/csit/tree/docs/lab/testbed_specifications.md#n1017) + - [mgmt-ip4 10.30.51.62 s26-t25-tg1](https://git.fd.io/csit/tree/docs/lab/testbed_specifications.md#n374) + - [ipmi-ip4 10.30.50.59 s26-t25-tg1](https://git.fd.io/csit/tree/docs/lab/testbed_specifications.md#n281) + +### YUL1 Inventory + +#### Rack YUL1-8 (3016.8) + + **name** | **oper-status** | **testbed-id** | **role** | **model** | **s/n** | **mgmt-ip4** | **ipmi-ip4** | **rackid** | **rackunit** +---------------|-----------------|----------------|----------|----------------|-----------------|--------------|--------------|------------|-------------- + mtl1-8-lb4m | up | switch | uplink | ? | ? | ? | ? | 3016.8 | u47 + s65-t37-sut1 | up | t37 | 3n-icx | SYS-740GP-TNRT | C7470KK25P50098 | 10.30.51.75 | 10.30.50.75 | 3016.8 | u42-u45 + s66-t37-sut2 | up | t37 | 3n-icx | SYS-740GP-TNRT | C7470KK33P50247 | 10.30.51.76 | 10.30.50.76 | 3016.8 | u38-u41 + s67-t37-tg1 | up | t37 | 3n-icx | SYS-740GP-TNRT | C7470KK25P50076 | 10.30.51.77 | 10.30.50.77 | 3016.8 | u34-u37 + s71-t212-sut1 | up | t212 | 2n-icx | SYS-740GP-TNRT | C7470KK25P50173 | 10.30.51.81 | 10.30.50.81 | 3016.8 | u30-u33 + s72-t212-tg1 | up | t212 | 2n-icx | SYS-740GP-TNRT | C7470KK33P50220 | 10.30.51.82 | 10.30.50.82 | 3016.8 | u26-u29 + s83-t213-sut1 | up | t213 | 2n-icx | SYS-740GP-TNRT | C7470KL07P50300 | 10.30.51.83 | 10.30.50.83 | 3016.8 | u22-u25 + s84-t213-tg1 | up | t213 | 2n-icx | SYS-740GP-TNRT | C7470KL03P50187 | 10.30.51.84 | 10.30.50.84 | 3016.8 | u18-u21 + s85-t214-sut1 | up | t214 | 2n-icx | SYS-740GP-TNRT | C7470KK33P50219 | 10.30.51.85 | 10.30.50.85 | 3016.8 | u14-u17 + s86-t214-tg1 | up | t214 | 2n-icx | SYS-740GP-TNRT | C7470KL07P50312 | 10.30.51.86 | 10.30.50.86 | 3016.8 | u10-u13 + s87-t215-sut1 | up | t215 | 2n-icx | SYS-740GP-TNRT | C7470KL03P50171 | 10.30.51.87 | 10.30.50.87 | 3016.8 | u6-u9 + s88-t215-tg1 | up | t215 | 2n-icx | SYS-740GP-TNRT | C7470KL07P50301 | 10.30.51.88 | 10.30.50.88 | 3016.8 | u2-u5 + +#### Rack YUL1-9 (3016.9) + + **name** | **oper-status** | **testbed-id** | **role** | **model** | **s/n** | **mgmt-ip4** | **ipmi-ip4** | **rackid** | **rackunit** +-----------------|-----------------|----------------|---------------|---------------------|-----------------|--------------|--------------|------------|-------------- + mtl1-5-lb4m | up | switch | uplink | ? | ? | ? | ? | 3016.9 | u47 + s52-t21-sut1 | up | t21 | 2n-spr | SYS-741GE-TNRT | C7490FL36A40118 | 10.30.51.52 | 10.30.50.52 | 3016.9 | u42-u45 + s53-t21-tg1 | up | t21 | 2n-spr | SYS-741GE-TNRT | ??? | 10.30.51.53 | 10.30.50.53 | 3016.9 | u38-u41 + s54-t22-sut1 | up | t22 | 2n-spr | SYS-741GE-TNRT | ??? | 10.30.51.54 | 10.30.50.54 | 3016.9 | u34-u37 + s55-t22-tg1 | up | t22 | 2n-spr | SYS-741GE-TNRT | ??? | 10.30.51.55 | 10.30.50.55 | 3016.9 | u30-u33 + s56-t23-sut1 | up | t23 | 2n-spr | SYS-741GE-TNRT | ??? | 10.30.51.56 | 10.30.50.56 | 3016.9 | u26-u29 + s57-t23-tg1 | up | t23 | 2n-spr | SYS-741GE-TNRT | ??? | 10.30.51.57 | 10.30.50.57 | 3016.9 | u22-u25 + s25-t25-sut1 | down | t25 | 2n-p1 | SYS-7049GP-TRT | C7470KH06A20022 | 10.30.51.61 | 10.30.50.58 | 3016.9 | u18-u21 + s19-t33t211-tg1 | up | t33t211 | 3n-tsh/2n-tx2 | SYS-7049GP-TRT | C7470KH06A20056 | 10.30.51.49 | 10.30.50.46 | 3016.9 | u14-u17 + s27-t211-sut1 | up | t211 | 2n-tx2 | ThunderX2-9975 | K61186073100003 | 10.30.51.69 | 10.30.50.69 | 3016.9 | u13 + s18-t33-sut2 | up | t33 | 3n-tsh | HUAWEI-TAISHAN-2280 | N/A | 10.30.51.37 | 10.30.50.37 | 3016.9 | u11-u12 + s17-t33-sut1 | up | t33 | 3n-tsh | HUAWEI-TAISHAN-2280 | N/A | 10.30.51.36 | 10.30.50.36 | 3016.9 | u9-u10 + s79-t39t310-tg1 | in-transit | t39t310 | tbd | SYS-740GP-TNRT | ??? | 10.30.51.89 | 10.30.50.89 | 3016.9 | u5-u8 + +#### Rack YUL1-10 (3016.10) + + **name** | **oper-status** | **testbed-id** | **role** | **model** | **s/n** | **mgmt-ip4** | **ipmi-ip4** | **rackid** | **rackunit** +-----------------|-----------------|----------------|----------|----------------|-----------------|--------------|--------------|------------|-------------- + yul1-10-lb4m | up | switch | uplink | ? | ? | ? | ? | 3016.10 | u47 + s2-t12-sut1 | up | t12 | 1n-skx | SYS-7049GP-TRT | C7470KH06A20119 | 10.30.51.51 | 10.30.50.48 | 3016.10 | u42-u45 + s1-t11-sut1 | up | t11 | 1n-skx | SYS-7049GP-TRT | C7470KH06A20154 | 10.30.51.50 | 10.30.50.47 | 3016.10 | u38-u41 + s58-t24-sut1 | up | t24 | 2n-spr | SYS-741GE-TNRT | ??? | 10.30.51.58 | 10.30.50.58 | 3016.10 | u34-u37 + s59-t24-tg1 | up | t24 | 2n-spr | SYS-741GE-TNRT | ??? | 10.30.51.59 | 10.30.50.59 | 3016.10 | u30-u33 + s93-t39-sut1 | up | t39 | 3n-snr | ? | ? | 10.30.51.93 | 10.30.50.93 | 3016.10 | u10-u13 + s94-t39-sut2 | up | t39 | 3n-snr | ? | ? | 10.30.51.94 | 10.30.50.94 | 3016.10 | u6-u9 + s89-t39t310-tg1 | up | t39 | 3n-snr | ? | ? | 10.30.51.89 | 10.30.50.89 | 3016.10 | u2-u5 + + +#### Rack YUL1-11 (3016.11) + + **name** | **oper-status** | **testbed-id** | **role** | **model** | **s/n** | **mgmt-ip4** | **ipmi-ip4** | **rackid** | **rackunit** +-----------------------|-----------------|----------------|------------|----------------|-----------------|--------------|--------------|------------|-------------- + yul1-11-lb6m | up | switch | arm-uplink | ? | ? | ? | ? | 3016.11 | u48 + yul1-11-lf-tor-switch | up | switch | uplink | ? | ? | ? | ? | 3016.11 | u47 + mtl1-6-7050QX-32 | up | switch | uplink | ? | ? | ? | ? | 3016.11 | u46 + fdio-marvell-dev | up | N/A | dev | ThunderX-88XX | N/A | 10.30.51.38 | 10.30.50.38 | 3016.11 | u45 + s56-t14-sut1 | up | t14 | 1n-tx2 | ThunderX2-9980 | N/A | 10.30.51.71 | 10.30.50.71 | 3016.11 | u41-u42 + s78-t38-sut1 | up | t38 | 3n-icx | SYS-740GP-TNRT | C7470KL03P50450 | 10.30.51.78 | 10.30.50.78 | 3016.11 | u31-u34 + s79-t38-sut2 | up | t38 | 3n-icx | SYS-740GP-TNRT | C7470KL07P50297 | 10.30.51.79 | 10.30.50.79 | 3016.11 | u27-u30 + s80-t38-tg1 | up | t38 | 3n-icx | SYS-740GP-TNRT | C7470KL03P50454 | 10.30.51.80 | 10.30.50.80 | 3016.11 | u23-u26 + s55-t13-sut1 | up | t13 | 1n-tx2 | ThunderX2-9980 | N/A | 10.30.51.70 | 10.30.50.70 | 3016.11 | u11-u12 + s62-t34-sut1 | up | t34 | 3n-alt | WIWYNN | 04000059N0SC | 10.30.51.72 | 10.30.50.72 | 3016.11 | u9-u10 + s63-t34-sut2 | up | t34 | 3n-alt | WIWYNN | 0390003EN0SC | 10.30.51.73 | 10.30.50.73 | 3016.11 | u7-u8 + s64-t34-tg1 | up | t34 | 3n-alt | SYS-740GP-TNRT | C7470KK40P50249 | 10.30.51.74 | 10.30.50.74 | 3016.11 | u3-u6 + +#### Rack YUL1-12 (3016.12) + + **name** | **oper-status** | **testbed-id** | **role** | **model** | **s/n** | **mgmt-ip4** | **ipmi-ip4** | **rackid** | **rackunit** +---------------|-----------------|-----------------------------|----------------|-----------------|-----------------|--------------|--------------|------------|-------------- + yul1-12-lb4m | up | switch | uplink | ? | ? | ? | ? | 3016.12 | u47 + s28-nomad | up | nomad-cluster1.nomad-client | SYS-7049GP-TRT | C7470KH06A20196 | 10.30.51.28 | 10.30.50.28 | 3016.12 | u41-u44 | + s27-nomad | up | nomad-cluster1 | nomad-client | SYS-7049GP-TRT | C7470KH06A20055 | 10.30.51.27 | 10.30.50.27 | 3016.12 | u37-u40 + s91-nomad | up | nomad-cluster1 | nomad-client | R152-P30-00 | GLG4P9912A0016 | 10.30.51.91 | 10.30.50.91 | 3016.12 | u36 + s92-nomad | up | nomad-cluster1 | nomad-client | R152-P30-00 | GLG4P9912A0004 | 10.30.51.92 | 10.30.50.92 | 3016.12 | u35 + s23-nomad | up | nomad-cluster1 | nomad-server | SYS-1029P-WTRT | C1160LI12NM0256 | 10.30.51.23 | 10.30.51.23 | 3016.12 | u34 + s24-nomad | up | nomad-cluster1 | nomad-server | SYS-1029P-WTRT | C1160LI12NM0241 | 10.30.51.24 | 10.30.51.24 | 3016.12 | u33 + s25-nomad | up | nomad-cluster1 | nomad-server | SYS-1029P-WTRT | C1160LI12NM0540 | 10.30.51.25 | 10.30.51.25 | 3016.12 | u32 + s61-t210-tg1 | up | t210 | 2n-zn2 | AS-1014S-WTRT | C8150LI50NS2689 | 10.32.8.25 | 10.30.55.25 | 3016.12 | u31 + s60-t210-sut1 | up | t210 | 2n-zn2 | AS-1114S-WTRT | N/A | 10.32.8.24 | 10.30.55.24 | 3016.12 | u30 + s26-nomad | up | nomad-cluster1 | nomad-server | SYS-7049GP-TRT | C7470KH37A30505 | 10.30.51.26 | 10.30.51.26 | 3016.12 | u26-u29 + s33-t27-sut1 | up | t27 | 2n-clx | SYS-7049GP-TRT | C7470KH37A30567 | 10.32.8.18 | 10.30.55.18 | 3016.12 | u22-u25 + s34-t27-tg1 | up | t27 | 2n-clx | SYS-7049GP-TRT | C7470KH37A30565 | 10.32.8.19 | 10.30.55.19 | 3016.12 | u18-u21 + s35-t28-sut1 | up | t28 | 2n-clx | SYS-7049GP-TRT | C7470KH37A30509 | 10.32.8.20 | 10.30.55.20 | 3016.12 | u14-u17 + s36-t28-tg1 | up | t28 | 2n-clx | SYS-7049GP-TRT | C7470KH37A30511 | 10.32.8.21 | 10.30.55.21 | 3016.12 | u10-u13 + s37-t29-sut1 | up | t29 | 2n-clx | SYS-7049GP-TRT | C7470KH37A30566 | 10.32.8.22 | 10.30.55.22 | 3016.12 | u6-u9 + s38-t29-tg1 | up | t29 | 2n-clx | SYS-7049GP-TRT | C7470KH37A30506 | 10.32.8.23 | 10.30.55.23 | 3016.12 | u2-u5
\ No newline at end of file diff --git a/docs/content/infrastructure/testbed_configuration/_index.md b/docs/content/infrastructure/testbed_configuration/_index.md new file mode 100644 index 0000000000..ce023237c7 --- /dev/null +++ b/docs/content/infrastructure/testbed_configuration/_index.md @@ -0,0 +1,5 @@ +--- +bookFlatSection: false +title: "FD.io CSIT Testbed Configuration" +weight: 3 +---
\ No newline at end of file diff --git a/docs/content/infrastructure/testbed_configuration/sm_clx_hw_bios_cfg.md b/docs/content/infrastructure/testbed_configuration/sm_clx_hw_bios_cfg.md new file mode 100644 index 0000000000..b2c859b11f --- /dev/null +++ b/docs/content/infrastructure/testbed_configuration/sm_clx_hw_bios_cfg.md @@ -0,0 +1,1419 @@ +--- +bookToc: true +title: "SuperMicro Cascadelake" +--- + +# SuperMicro Cascadelake + +## Linux lscpu + +``` +Architecture: x86_64 +CPU op-mode(s): 32-bit, 64-bit +Byte Order: Little Endian +CPU(s): 112 +On-line CPU(s) list: 0-111 +Thread(s) per core: 2 +Core(s) per socket: 28 +Socket(s): 2 +NUMA node(s): 2 +Vendor ID: GenuineIntel +CPU family: 6 +Model: 85 +Model name: Intel(R) Xeon(R) Platinum 8280 CPU @ 2.70GHz +Stepping: 7 +CPU MHz: 3299.609 +BogoMIPS: 5400.00 +Virtualization: VT-x +L1d cache: 32K +L1i cache: 32K +L2 cache: 1024K +L3 cache: 39424K +NUMA node0 CPU(s): 0-27,56-83 +NUMA node1 CPU(s): 28-55,84-111 +Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca +cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx +pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology +nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est +tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt +tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch +cpuid_fault epb cat_l3 cdp_l3 invpcid_single ssbd mba ibrs ibpb stibp +ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 +hle avx2 smep bmi2 erms invpcid rtm cqm mpx rdt_a avx512f avx512dq rdseed adx +smap clflushopt clwb intel_pt avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 +xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts +pku ospke avx512_vnni md_clear flush_l1d arch_capabilities +``` + +``` +Architecture: x86_64 +CPU op-mode(s): 32-bit, 64-bit +Byte Order: Little Endian +CPU(s): 96 +On-line CPU(s) list: 0-95 +Thread(s) per core: 2 +Core(s) per socket: 24 +Socket(s): 2 +NUMA node(s): 2 +Vendor ID: GenuineIntel +CPU family: 6 +Model: 85 +Model name: Intel(R) Xeon(R) Gold 6252N CPU @ 2.30GHz +Stepping: 7 +CPU MHz: 3000.989 +BogoMIPS: 4600.00 +Virtualization: VT-x +L1d cache: 32K +L1i cache: 32K +L2 cache: 1024K +L3 cache: 36608K +NUMA node0 CPU(s): 0-23,48-71 +NUMA node1 CPU(s): 24-47,72-95 +Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca +cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx +pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology +nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 +ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt +tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch +cpuid_fault epb cat_l3 cdp_l3 invpcid_single ssbd mba ibrs ibpb stibp +ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle +avx2 smep bmi2 erms invpcid rtm cqm mpx rdt_a avx512f avx512dq rdseed adx smap +clflushopt clwb intel_pt avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 +xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts +pku ospke avx512_vnni md_clear flush_l1d arch_capabilities +``` + +## Linux dmidecode + +``` + # dmidecode 3.1 + Getting SMBIOS data from sysfs. + SMBIOS 3.1.2 present. + Table at 0x6EB92000. + + Handle 0x0000, DMI type 0, 26 bytes + BIOS Information + Vendor: American Megatrends Inc. + Version: 3.0c + Release Date: 03/27/2019 + Address: 0xF0000 + Runtime Size: 64 kB + ROM Size: 32 MB + Characteristics: + PCI is supported + BIOS is upgradeable + BIOS shadowing is allowed + Boot from CD is supported + Selectable boot is supported + BIOS ROM is socketed + EDD is supported + 5.25"/1.2 MB floppy services are supported (int 13h) + 3.5"/720 kB floppy services are supported (int 13h) + 3.5"/2.88 MB floppy services are supported (int 13h) + Print screen service is supported (int 5h) + Serial services are supported (int 14h) + Printer services are supported (int 17h) + ACPI is supported + USB legacy is supported + BIOS boot specification is supported + Targeted content distribution is supported + UEFI is supported + BIOS Revision: 5.14 + + Handle 0x0001, DMI type 1, 27 bytes + System Information + Manufacturer: Supermicro + Product Name: SYS-7049GP-TRT + Version: 0123456789 + Serial Number: S291427X9525476 + UUID: 00000000-0000-0000-0000-AC1F6BACD7BA + Wake-up Type: Power Switch + SKU Number: To be filled by O.E.M. + Family: To be filled by O.E.M. + + Handle 0x0002, DMI type 2, 15 bytes + Base Board Information + Manufacturer: Supermicro + Product Name: X11DPG-QT + Version: 1.10A + Serial Number: VM189S007860 + Asset Tag: To be filled by O.E.M. + Features: + Board is a hosting board + Board is replaceable + Location In Chassis: To be filled by O.E.M. + Chassis Handle: 0x0003 + Type: Motherboard + Contained Object Handles: 0 + + Handle 0x0003, DMI type 3, 22 bytes + Chassis Information + Manufacturer: Supermicro + Type: Other + Lock: Not Present + Version: 0123456789 + Serial Number: C7470KH37A30566 + Asset Tag: To be filled by O.E.M. + Boot-up State: Safe + Power Supply State: Safe + Thermal State: Safe + Security Status: None + OEM Information: 0x00000000 + Height: Unspecified + Number Of Power Cords: 1 + Contained Elements: 0 + SKU Number: To be filled by O.E.M. + + Handle 0x0055, DMI type 4, 48 bytes + Processor Information + Socket Designation: CPU1 + Type: Central Processor + Family: Xeon + Manufacturer: Intel(R) Corporation + ID: 57 06 05 00 FF FB EB BF + Signature: Type 0, Family 6, Model 85, Stepping 7 + Flags: + FPU (Floating-point unit on-chip) + VME (Virtual mode extension) + DE (Debugging extension) + PSE (Page size extension) + TSC (Time stamp counter) + MSR (Model specific registers) + PAE (Physical address extension) + MCE (Machine check exception) + CX8 (CMPXCHG8 instruction supported) + APIC (On-chip APIC hardware supported) + SEP (Fast system call) + MTRR (Memory type range registers) + PGE (Page global enable) + MCA (Machine check architecture) + CMOV (Conditional move instruction supported) + PAT (Page attribute table) + PSE-36 (36-bit page size extension) + CLFSH (CLFLUSH instruction supported) + DS (Debug store) + ACPI (ACPI supported) + MMX (MMX technology supported) + FXSR (FXSAVE and FXSTOR instructions supported) + SSE (Streaming SIMD extensions) + SSE2 (Streaming SIMD extensions 2) + SS (Self-snoop) + HTT (Multi-threading) + TM (Thermal monitor supported) + PBE (Pending break enabled) + Version: Intel(R) Xeon(R) Gold 6252N CPU @ 2.30GHz + Voltage: 1.6 V + External Clock: 100 MHz + Max Speed: 4500 MHz + Current Speed: 2300 MHz + Status: Populated, Enabled + Upgrade: Socket LGA3647-1 + L1 Cache Handle: 0x0052 + L2 Cache Handle: 0x0053 + L3 Cache Handle: 0x0054 + Serial Number: Not Specified + Asset Tag: UNKNOWN + Part Number: Not Specified + Core Count: 24 + Core Enabled: 24 + Thread Count: 48 + Characteristics: + 64-bit capable + Multi-Core + Hardware Thread + Execute Protection + Enhanced Virtualization + Power/Performance Control + + Handle 0x0059, DMI type 4, 48 bytes + Processor Information + Socket Designation: CPU2 + Type: Central Processor + Family: Xeon + Manufacturer: Intel(R) Corporation + ID: 57 06 05 00 FF FB EB BF + Signature: Type 0, Family 6, Model 85, Stepping 7 + Flags: + FPU (Floating-point unit on-chip) + VME (Virtual mode extension) + DE (Debugging extension) + PSE (Page size extension) + TSC (Time stamp counter) + MSR (Model specific registers) + PAE (Physical address extension) + MCE (Machine check exception) + CX8 (CMPXCHG8 instruction supported) + APIC (On-chip APIC hardware supported) + SEP (Fast system call) + MTRR (Memory type range registers) + PGE (Page global enable) + MCA (Machine check architecture) + CMOV (Conditional move instruction supported) + PAT (Page attribute table) + PSE-36 (36-bit page size extension) + CLFSH (CLFLUSH instruction supported) + DS (Debug store) + ACPI (ACPI supported) + MMX (MMX technology supported) + FXSR (FXSAVE and FXSTOR instructions supported) + SSE (Streaming SIMD extensions) + SSE2 (Streaming SIMD extensions 2) + SS (Self-snoop) + HTT (Multi-threading) + TM (Thermal monitor supported) + PBE (Pending break enabled) + Version: Intel(R) Xeon(R) Gold 6252N CPU @ 2.30GHz + Voltage: 1.6 V + External Clock: 100 MHz + Max Speed: 4500 MHz + Current Speed: 2300 MHz + Status: Populated, Enabled + Upgrade: Socket LGA3647-1 + L1 Cache Handle: 0x0056 + L2 Cache Handle: 0x0057 + L3 Cache Handle: 0x0058 + Serial Number: Not Specified + Asset Tag: UNKNOWN + Part Number: Not Specified + Core Count: 24 + Core Enabled: 24 + Thread Count: 48 + Characteristics: + 64-bit capable + Multi-Core + Hardware Thread + Execute Protection + Enhanced Virtualization + Power/Performance Control +``` + +## Linux dmidecode pci + +``` + Handle 0x000B, DMI type 9, 17 bytes + System Slot Information + Designation: CPU1 SLOT2 PCI-E 3.0 X16 + Type: x16 PCI Express 3 x16 + Current Usage: In Use + Length: Long + ID: 2 + Characteristics: + 3.3 V is provided + Opening is shared + PME signal is supported + Bus Address: 0000:18:00.0 + + Handle 0x000C, DMI type 9, 17 bytes + System Slot Information + Designation: CPU1 SLOT4 PCI-E 3.0 X16 + Type: x16 PCI Express 3 x16 + Current Usage: In Use + Length: Short + ID: 4 + Characteristics: + 3.3 V is provided + Opening is shared + PME signal is supported + Bus Address: 0000:3b:00.0 + + Handle 0x000D, DMI type 9, 17 bytes + System Slot Information + Designation: CPU2 SLOT6 PCI-E 3.0 X16 + Type: x16 PCI Express 3 x16 + Current Usage: Available + Length: Short + ID: 6 + Characteristics: + 3.3 V is provided + Opening is shared + PME signal is supported + Bus Address: 0000:ff:00.0 + + Handle 0x000E, DMI type 9, 17 bytes + System Slot Information + Designation: CPU2 SLOT8 PCI-E 3.0 X16 + Type: x16 PCI Express 3 x16 + Current Usage: Available + Length: Short + ID: 8 + Characteristics: + 3.3 V is provided + Opening is shared + PME signal is supported + Bus Address: 0000:ff:00.0 + + Handle 0x000F, DMI type 9, 17 bytes + System Slot Information + Designation: CPU1 SLOT9 PCI-E 3.0 X16 + Type: x16 PCI Express 3 x16 + Current Usage: Available + Length: Short + ID: 9 + Characteristics: + 3.3 V is provided + Opening is shared + PME signal is supported + Bus Address: 0000:ff:00.0 + + Handle 0x0010, DMI type 9, 17 bytes + System Slot Information + Designation: CPU2 SLOT10 PCI-E 3.0 X16 + Type: x16 PCI Express 3 x16 + Current Usage: Available + Length: Short + ID: 10 + Characteristics: + 3.3 V is provided + Opening is shared + PME signal is supported + Bus Address: 0000:ff:00.0 + + Handle 0x0011, DMI type 9, 17 bytes + System Slot Information + Designation: CPU2 SLOT11 PCI-E 3.0 X4(IN X8) + Type: x4 PCI Express 3 x8 + Current Usage: Available + Length: Short + ID: 11 + Characteristics: + 3.3 V is provided + Opening is shared + PME signal is supported + Bus Address: 0000:ff:00.0 + + Handle 0x0012, DMI type 9, 17 bytes + System Slot Information + Designation: M.2 CONNECTOR + Type: x4 M.2 Socket 2 + Current Usage: Available + Length: Short + Characteristics: + 3.3 V is provided + Opening is shared + PME signal is supported + Bus Address: 0000:ff:00.0 +``` + +## Linux dmidecode memory + +``` + Handle 0x0023, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0021 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P1-DIMMA1 + Bank Locator: P0_Node0_Channel0_Dimm0 + Type: DDR4 + Type Detail: Synchronous + Speed: 2933 MT/s + Manufacturer: SK Hynix + Serial Number: 93275F0E + Asset Tag: P1-DIMMA1_AssetTag (date:19/22) + Part Number: HMA82GR7CJR8N-WM + Rank: 2 + Configured Clock Speed: 2934 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x0024, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0021 + Error Information Handle: Not Provided + Total Width: Unknown + Data Width: Unknown + Size: No Module Installed + Form Factor: DIMM + Set: None + Locator: P1-DIMMA2 + Bank Locator: P0_Node0_Channel0_Dimm1 + Type: Unknown + Type Detail: Unknown + Speed: Unknown + Manufacturer: NO DIMM + Serial Number: NO DIMM + Asset Tag: NO DIMM + Part Number: NO DIMM + Rank: Unknown + Configured Clock Speed: Unknown + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x0025, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0021 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P1-DIMMB1 + Bank Locator: P0_Node0_Channel1_Dimm0 + Type: DDR4 + Type Detail: Synchronous + Speed: 2933 MT/s + Manufacturer: SK Hynix + Serial Number: 93275F1F + Asset Tag: P1-DIMMB1_AssetTag (date:19/22) + Part Number: HMA82GR7CJR8N-WM + Rank: 2 + Configured Clock Speed: 2934 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x0027, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0021 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P1-DIMMC1 + Bank Locator: P0_Node0_Channel2_Dimm0 + Type: DDR4 + Type Detail: Synchronous + Speed: 2933 MT/s + Manufacturer: SK Hynix + Serial Number: 93275F07 + Asset Tag: P1-DIMMC1_AssetTag (date:19/22) + Part Number: HMA82GR7CJR8N-WM + Rank: 2 + Configured Clock Speed: 2934 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x002B, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0029 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P1-DIMMD1 + Bank Locator: P0_Node1_Channel0_Dimm0 + Type: DDR4 + Type Detail: Synchronous + Speed: 2933 MT/s + Manufacturer: SK Hynix + Serial Number: 93275F02 + Asset Tag: P1-DIMMD1_AssetTag (date:19/22) + Part Number: HMA82GR7CJR8N-WM + Rank: 2 + Configured Clock Speed: 2934 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x002C, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0029 + Error Information Handle: Not Provided + Total Width: Unknown + Data Width: Unknown + Size: No Module Installed + Form Factor: DIMM + Set: None + Locator: P1-DIMMD2 + Bank Locator: P0_Node1_Channel0_Dimm1 + Type: Unknown + Type Detail: Unknown + Speed: Unknown + Manufacturer: NO DIMM + Serial Number: NO DIMM + Asset Tag: NO DIMM + Part Number: NO DIMM + Rank: Unknown + Configured Clock Speed: Unknown + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x002D, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0029 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P1-DIMME1 + Bank Locator: P0_Node1_Channel1_Dimm0 + Type: DDR4 + Type Detail: Synchronous + Speed: 2933 MT/s + Manufacturer: SK Hynix + Serial Number: 93275F19 + Asset Tag: P1-DIMME1_AssetTag (date:19/22) + Part Number: HMA82GR7CJR8N-WM + Rank: 2 + Configured Clock Speed: 2934 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x002F, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0029 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P1-DIMMF1 + Bank Locator: P0_Node1_Channel2_Dimm0 + Type: DDR4 + Type Detail: Synchronous + Speed: 2933 MT/s + Manufacturer: SK Hynix + Serial Number: 93275FD3 + Asset Tag: P1-DIMMF1_AssetTag (date:19/22) + Part Number: HMA82GR7CJR8N-WM + Rank: 2 + Configured Clock Speed: 2934 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x0031, DMI type 16, 23 bytes + Physical Memory Array + Location: System Board Or Motherboard + Use: System Memory + Error Correction Type: Single-bit ECC + Maximum Capacity: 2304 GB + Error Information Handle: Not Provided + Number Of Devices: 4 + + Handle 0x0033, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0031 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P2-DIMMA1 + Bank Locator: P1_Node0_Channel0_Dimm0 + Type: DDR4 + Type Detail: Synchronous + Speed: 2933 MT/s + Manufacturer: SK Hynix + Serial Number: 93275FE2 + Asset Tag: P2-DIMMA1_AssetTag (date:19/22) + Part Number: HMA82GR7CJR8N-WM + Rank: 2 + Configured Clock Speed: 2934 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x0034, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0031 + Error Information Handle: Not Provided + Total Width: Unknown + Data Width: Unknown + Size: No Module Installed + Form Factor: DIMM + Set: None + Locator: P2-DIMMA2 + Bank Locator: P1_Node0_Channel0_Dimm1 + Type: Unknown + Type Detail: Unknown + Speed: Unknown + Manufacturer: NO DIMM + Serial Number: NO DIMM + Asset Tag: NO DIMM + Part Number: NO DIMM + Rank: Unknown + Configured Clock Speed: Unknown + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x0035, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0031 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P2-DIMMB1 + Bank Locator: P1_Node0_Channel1_Dimm0 + Type: DDR4 + Type Detail: Synchronous + Speed: 2933 MT/s + Manufacturer: SK Hynix + Serial Number: 93276001 + Asset Tag: P2-DIMMB1_AssetTag (date:19/22) + Part Number: HMA82GR7CJR8N-WM + Rank: 2 + Configured Clock Speed: 2934 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x0037, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0031 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P2-DIMMC1 + Bank Locator: P1_Node0_Channel2_Dimm0 + Type: DDR4 + Type Detail: Synchronous + Speed: 2933 MT/s + Manufacturer: SK Hynix + Serial Number: 93276005 + Asset Tag: P2-DIMMC1_AssetTag (date:19/22) + Part Number: HMA82GR7CJR8N-WM + Rank: 2 + Configured Clock Speed: 2934 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x0039, DMI type 16, 23 bytes + Physical Memory Array + Location: System Board Or Motherboard + Use: System Memory + Error Correction Type: Single-bit ECC + Maximum Capacity: 2304 GB + Error Information Handle: Not Provided + Number Of Devices: 4 + + Handle 0x003B, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0039 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P2-DIMMD1 + Bank Locator: P1_Node1_Channel0_Dimm0 + Type: DDR4 + Type Detail: Synchronous + Speed: 2933 MT/s + Manufacturer: SK Hynix + Serial Number: 93275F44 + Asset Tag: P2-DIMMD1_AssetTag (date:19/22) + Part Number: HMA82GR7CJR8N-WM + Rank: 2 + Configured Clock Speed: 2934 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x003C, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0039 + Error Information Handle: Not Provided + Total Width: Unknown + Data Width: Unknown + Size: No Module Installed + Form Factor: DIMM + Set: None + Locator: P2-DIMMD2 + Bank Locator: P1_Node1_Channel0_Dimm1 + Type: Unknown + Type Detail: Unknown + Speed: Unknown + Manufacturer: NO DIMM + Serial Number: NO DIMM + Asset Tag: NO DIMM + Part Number: NO DIMM + Rank: Unknown + Configured Clock Speed: Unknown + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x003D, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0039 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P2-DIMME1 + Bank Locator: P1_Node1_Channel1_Dimm0 + Type: DDR4 + Type Detail: Synchronous + Speed: 2933 MT/s + Manufacturer: SK Hynix + Serial Number: 93275FDF + Asset Tag: P2-DIMME1_AssetTag (date:19/22) + Part Number: HMA82GR7CJR8N-WM + Rank: 2 + Configured Clock Speed: 2934 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x003F, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0039 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P2-DIMMF1 + Bank Locator: P1_Node1_Channel2_Dimm0 + Type: DDR4 + Type Detail: Synchronous + Speed: 2933 MT/s + Manufacturer: SK Hynix + Serial Number: 93275FDD + Asset Tag: P2-DIMMF1_AssetTag (date:19/22) + Part Number: HMA82GR7CJR8N-WM + Rank: 2 + Configured Clock Speed: 2934 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V +``` + +## Xeon CLX Server BIOS Configuration - TG + +### Boot Feature + +``` + | Quiet Boot [Enabled] |Boot option | + | | | + | Option ROM Messages [Force BIOS] | | + | Bootup NumLock State [On] | | + | Wait For "F1" If Error [Enabled] | | + | INT19 Trap Response [Immediate] | | + | Re-try Boot [Disabled] | | + | Install Windows 7 USB support [Disabled] | | + | Port 61h Bit-4 Emulation [Disabled] | | + | | | + | Power Configuration | | + | Watch Dog Function [Disabled] | | + | Restore on AC Power Loss [Last State] | | + | Power Button Function [Instant Off] | | +``` + +### CPU Configuration + +``` + | Processor Configuration ^|Enables Hyper Threading | + | -------------------------------------------------- *|(Software Method to | + | Processor BSP Revision 50657 - CLX B1 *|Enable/Disable Logical | + | Processor Socket CPU1 | CPU2 *|Processor threads. | + | Processor ID 00050657* | 00050657 *| | + | Processor Frequency 2.700GHz | 2.700GHz *| | + | Processor Max Ratio 1BH | 1BH *| | + | Processor Min Ratio 0AH | 0AH *| | + | Microcode Revision 0500002C | 0500002C *| | + | L1 Cache RAM 64KB | 64KB *| | + | L2 Cache RAM 1024KB | 1024KB *| | + | L3 Cache RAM 39424KB | 39424KB *| | + | Processor 0 Version *| | + | Intel(R) Xeon(R) Platinum 8280 CPU @ 2.70GHz *| | + | Processor 1 Version *| | + | Intel(R) Xeon(R) Platinum 8280 CPU @ 2.70GHz *| | + | *|-----------------------------| + | Hyper-Threading [ALL] [Enable] *|><: Select Screen | + | Cores Enabled 0 *|^v: Select Item | + | Monitor/Mwait [Auto] *|Enter: Select | + | Execute Disable Bit [Enable] +|+/-: Change Opt. | + | Intel Virtualization Technology [Enable] +|F1: General Help | + | PPIN Control [Unlock/Enable] +|F2: Previous Values | + | Hardware Prefetcher [Enable] +|F3: Optimized Defaults | + | Adjacent Cache Prefetch [Enable] v|F4: Save & Exit | + | DCU Streamer Prefetcher [Enable] | | + | DCU IP Prefetcher [Enable] | | + | LLC Prefetch [Disable] | | + | Extended APIC [Disable] | | + | AES-NI [Enable] | | + |> Advanced Power Management Configuration | | +``` + +#### Advanced Power Management Configuration + +``` + | Advanced Power Management Configuration |Switch CPU Power Management | + | -------------------------------------------------- |profile | + | Power Technology [Custom] | | + | Power Performance Tuning [BIOS Controls EPB] | | + | ENERGY_PERF_BIAS_CFG mode [Maximum Performance] | | + |> CPU P State Control | | + |> Hardware PM State Control | | + |> CPU C State Control | | + |> Package C State Control | | + |> CPU T State Control | | +``` + +##### CPU P State Control + +``` + | CPU P State Control |EIST allows the processor | + | |to dynamically adjust | + | SpeedStep (P-States) [Disable] |frequency and voltage based | + | EIST PSD Function [HW_ALL] |on power versus performance | + | |needs. | + | | | +``` + +##### Hardware PM State Control + +``` + | Hardware PM State Control |If set to Disable, hardware ^| + | |will choose a P-state *| + | Hardware P-States [Disable] |setting for the system *| + | |based on an OS request. *| + | |If set to Native Mode, *| + | |hardware will choose a *| + | |P-state setting based on OS *| + | |guidance. *| + | |If set to Native Mode with *| + | |No Legacy Support, hardware *| + | |will choose a P-state *| + | |setting independently *| + | |without OS guidance. +| + | |If set to Out of Band Mode, +| + | |hardware autonomously v| +``` + +##### CPU C State Control + +``` + | CPU C State Control |Select Enable to support | + | |Autonomous Core C-State | + | Autonomous Core C-State [Disable] |control which will allow | + | CPU C6 report [Disable] |the processor core to | + | Enhanced Halt State (C1E) [Disable] |control its C-State setting | + | |automatically and | + | |independently. | +``` + +##### Package C State Control + +``` + | Package C State Control |Limit the lowest package | + | |level C-State to | + | Package C State [C0/C1 state] |processors. Lower package | + | |C-State lower processor | + | |power consumption upon idle. | +``` + +##### CPU T State Control + +``` + | CPU T State Control |Enable/Disable CPU | + | |throttling by OS. | + | Software Controlled T-States [Disable] |Throttling reduces power | + | |consumption | +``` + +#### Chipset Configuration + +``` + | WARNING: Setting wrong values in below sections may cause |North Bridge Parameters | + | system to malfunction. | | + |> North Bridge | | + |> South Bridge | | +``` + +##### North Bridge + +``` + |> UPI Configuration |Displays and provides | + |> Memory Configuration |option to change the UPI | + |> IIO Configuration |Settings | +``` + +##### UPI Configuration + +``` + | UPI Configuration |Use this feature to select | + | -------------------------------------------------- |the degrading precedence | + | Number of CPU 2 |option for Ultra Path | + | Number of Active UPI Link 3 |Interconnect connections. | + | Current UPI Link Speed Fast |Select Topology Precedent | + | Current UPI Link Frequency 10.4 GT/s |to degrade UPI features if | + | UPI Global MMIO Low Base / Limit 90000000 / FBFFFFFF |system options are in | + | UPI Global MMIO High Base / Limit 0000000000000000 / |conflict. Select Feature | + | 00000000FFFFFFFF |Precedent to degrade UPI | + | UPI Pci-e Configuration Base / Size 80000000 / 10000000 |topology if system options | + | Degrade Precedence [Topology Precedence] |are in conflict. | + | Link L0p Enable [Disable] | | + | Link L1 Enable [Disable] | | + | IO Directory Cache (IODC) [Auto] | | + | SNC [Disable] | | + | XPT Prefetch [Disable] | | + | KTI Prefetch [Enable] |-----------------------------| + | Local/Remote Threshold [Auto] |><: Select Screen | + | Stale AtoS [Auto] |^v: Select Item | + | LLC Dead Line Alloc [Enable] |Enter: Select | + | Isoc Mode [Auto] |+/-: Change Opt. | +``` + +##### Memory Configuration + +``` + | |Select POR to enforce POR | + | -------------------------------------------------- |restrictions for DDR4 | + | Integrated Memory Controller (iMC) |frequency and voltage | + | -------------------------------------------------- |programming | + | | | + | Enforce POR [POR] | | + | PPR Type [Hard PPR] | | + | Enhanced PPR [Disable] | | + | Operation Mode [Test and Repair] | | + | Memory Frequency [2933] | | + | Data Scrambling for DDR4 [Auto] | | + | tCCD_L Relaxation [Auto] | | + | tRWSR Relaxation [Disable] | | + | tRFC Optimization for 16Gb Based DIMM [Force 550ns] | | + | 2x Refresh [Auto] | | + | Page Policy [Auto] | | + | IMC Interleaving [2-way Interleave] |-----------------------------| + |> Memory Topology |><: Select Screen | + |> Memory RAS Configuration |^v: Select Item | +``` + +##### IIO Configuration + +``` + | IIO Configuration |Expose IIO DFX devices and | + | -------------------------------------------------- |other CPU devices like PMON | + | | | + | EV DFX Features [Disable] | | + |> CPU1 Configuration | | + |> CPU2 Configuration | | + |> IOAT Configuration | | + |> Intel. VT for Directed I/O (VT-d) | | + |> Intel. VMD technology | | + | | | + | IIO-PCIE Express Global Options | | + | ======================================== | | + | PCI-E Completion Timeout Disable [No] | | +``` + +##### CPU1 Configuration + +``` + | IOU0 (IIO PCIe Br1) [Auto] |Selects PCIe port | + | IOU1 (IIO PCIe Br2) [Auto] |Bifurcation for selected | + | IOU2 (IIO PCIe Br3) [Auto] |slot(s) | + |> CPU1 SLOT2 PCI-E 3.0 X16 | | + |> CPU1 SLOT4 PCI-E 3.0 X16 | | + |> CPU1 SLOT9 PCI-E 3.0 X16 | | +``` + +##### CPU2 Configuration + +``` + | IOU0 (IIO PCIe Br1) [Auto] |Selects PCIe port | + | IOU1 (IIO PCIe Br2) [Auto] |Bifurcation for selected | + | IOU2 (IIO PCIe Br3) [Auto] |slot(s) | + |> CPU2 SLOT6 PCI-E 3.0 X16 | | + |> CPU2 SLOT8 PCI-E 3.0 X16 | | + |> CPU2 SLOT10 PCI-E 3.0 X16 | | +``` + +#### South Bridge + +``` + | |Enables Legacy USB support. | + | USB Module Version 21 |AUTO option disables legacy | + | |support if no USB devices | + | USB Devices: |are connected. DISABLE | + | 1 Keyboard, 1 Mouse, 1 Hub |option will keep USB | + | |devices available only for | + | Legacy USB Support [Enabled] |EFI applications. | + | XHCI Hand-off [Enabled] | | + | Port 60/64 Emulation [Enabled] | | + | PCIe PLL SSC [Disable] | | + | Real USB Wake Up [Enabled] | | + | Front USB Wake Up [Enabled] | | + | | | + | Azalia [Auto] | | + | Azalia PME Enable [Disabled] | | +``` + +### PCIe/PCI/PnP Configuration + +``` + | PCI Bus Driver Version A5.01.18 ^|Enables or Disables 64bit | + | *|capable Devices to be | + | PCI Devices Common Settings: *|Decoded in Above 4G Address | + | Above 4G Decoding [Enabled] *|Space (Only if System | + | SR-IOV Support [Enabled] *|Supports 64 bit PCI | + | ARI Support [Enabled] *|Decoding). | + | MMIO High Base [56T] *| | + | MMIO High Granularity Size [256G] *| | + | Maximum Read Request [Auto] *| | + | MMCFG Base [2G] *| | + | NVMe Firmware Source [Vendor Defined *| | + | Firmware] *| | + | VGA Priority [Onboard] *| | + | CPU1 SLOT2 PCI-E 3.0 X16 OPROM [Legacy] *| | + | CPU1 SLOT4 PCI-E 3.0 X16 OPROM [Legacy] *| | + | CPU2 SLOT6 PCI-E 3.0 X16 OPROM [Legacy] *| | + | CPU2 SLOT8 PCI-E 3.0 X16 OPROM [Legacy] *|-----------------------------| + | CPU1 SLOT9 PCI-E 3.0 X16 OPROM [Legacy] *|><: Select Screen | + | CPU2 SLOT10 PCI-E 3.0 X16 OPROM [Legacy] *|^v: Select Item | + | CPU2 SLOT11 PCI-E 3.0 X4(IN X8) OPROM [Legacy] *|Enter: Select | + | M.2 CONNECTOR OPROM [Legacy] *|+/-: Change Opt. | + | Bus Master Enable [Enabled] +|F1: General Help | + | Onboard LAN1 Option ROM [Legacy] +|F2: Previous Values | + | Onboard LAN2 Option ROM [Disabled] +|F3: Optimized Defaults | + | Onboard Video Option ROM [Legacy] v|F4: Save & Exit | + |> Network Stack Configuration | | +``` + +### ACPI Settings + +``` + | ACPI Settings |Enable or Disable Non | + | |uniform Memory Access | + | NUMA [Enabled] |(NUMA). | + | WHEA Support [Enabled] | | + | High Precision Event Timer [Enabled] | | +``` + +## Xeon CLX Server BIOS Configuration - DUT + +### Boot Feature + +``` + | Quiet Boot [Enabled] |Boot option | + | | | + | Option ROM Messages [Force BIOS] | | + | Bootup NumLock State [On] | | + | Wait For "F1" If Error [Enabled] | | + | INT19 Trap Response [Immediate] | | + | Re-try Boot [Disabled] | | + | Install Windows 7 USB support [Disabled] | | + | Port 61h Bit-4 Emulation [Disabled] | | + | | | + | Power Configuration | | + | Watch Dog Function [Disabled] | | + | Restore on AC Power Loss [Last State] | | + | Power Button Function [Instant Off] | | +``` + +### CPU Configuration + +``` + |--------------------------------------------------------------------+-----------------------------\ + | Processor Configuration ^|Enables Hyper Threading | + | -------------------------------------------------- *|(Software Method to | + | Processor BSP Revision 50657 - CLX B1 *|Enable/Disable Logical | + | Processor Socket CPU1 | CPU2 *|Processor threads. | + | Processor ID 00050657* | 00050657 *| | + | Processor Frequency 2.300GHz | 2.300GHz *| | + | Processor Max Ratio 17H | 17H *| | + | Processor Min Ratio 0AH | 0AH *| | + | Microcode Revision 0500002C | 0500002C *| | + | L1 Cache RAM 64KB | 64KB *| | + | L2 Cache RAM 1024KB | 1024KB *| | + | L3 Cache RAM 36608KB | 36608KB *| | + | Processor 0 Version *| | + | Intel(R) Xeon(R) Gold 6252N CPU @ 2.30GHz *| | + | Processor 1 Version *| | + | Intel(R) Xeon(R) Gold 6252N CPU @ 2.30GHz *| | + | *|-----------------------------| + | Hyper-Threading [ALL] [Enable] *|><: Select Screen | + | Cores Enabled 0 *|^v: Select Item | + | Monitor/Mwait [Auto] *|Enter: Select | + | Execute Disable Bit [Enable] +|+/-: Change Opt. | + | Intel Virtualization Technology [Enable] +|F1: General Help | + | PPIN Control [Unlock/Enable] +|F2: Previous Values | + | Hardware Prefetcher [Enable] +|F3: Optimized Defaults | + | Adjacent Cache Prefetch [Enable] v|F4: Save & Exit | + | DCU Streamer Prefetcher [Enable] | | + | DCU IP Prefetcher [Enable] | | + | LLC Prefetch [Disable] | | + | Extended APIC [Disable] | | + | AES-NI [Enable] | | + |> Advanced Power Management Configuration | | +``` + +#### Advanced Power Management Configuration + +``` + | Advanced Power Management Configuration |Switch CPU Power Management | + | -------------------------------------------------- |profile | + | Power Technology [Custom] | | + | Power Performance Tuning [BIOS Controls EPB] | | + | ENERGY_PERF_BIAS_CFG mode [Maximum Performance] | | + |> CPU P State Control | | + |> Hardware PM State Control | | + |> CPU C State Control | | + |> Package C State Control | | + |> CPU T State Control | | +``` + +##### CPU P State Control + +``` + | CPU P State Control |EIST allows the processor | + | |to dynamically adjust | + | SpeedStep (P-States) [Disable] |frequency and voltage based | + | Activate PBF [Disable] |on power versus performance | + | Configure PBF [Enable] |needs. | + | EIST PSD Function [HW_ALL] | | +``` + +##### Hardware PM State Control + +``` + | Hardware PM State Control |If set to Disable, hardware ^| + | |will choose a P-state *| + | Hardware P-States [Disable] |setting for the system *| + | |based on an OS request. *| + | |If set to Native Mode, *| + | |hardware will choose a *| + | |P-state setting based on OS *| + | |guidance. *| + | |If set to Native Mode with *| + | |No Legacy Support, hardware *| + | |will choose a P-state *| + | |setting independently *| + | |without OS guidance. +| + | |If set to Out of Band Mode, +| + | |hardware autonomously v| +``` + +##### CPU C State Control + +``` + | CPU C State Control |Select Enable to support | + | |Autonomous Core C-State | + | Autonomous Core C-State [Disable] |control which will allow | + | CPU C6 report [Disable] |the processor core to | + | Enhanced Halt State (C1E) [Disable] |control its C-State setting | + | |automatically and | + | |independently. | +``` + +##### Package C State Control + +``` + | Package C State Control |Limit the lowest package | + | |level C-State to | + | Package C State [C0/C1 state] |processors. Lower package | + | |C-State lower processor | + | |power consumption upon idle. | +``` + +##### CPU T State Control + +``` + | CPU T State Control |Enable/Disable CPU | + | |throttling by OS. | + | Software Controlled T-States [Disable] |Throttling reduces power | + | |consumption | +``` + +#### Chipset Configuration + +``` + | WARNING: Setting wrong values in below sections may cause |North Bridge Parameters | + | system to malfunction. | | + |> North Bridge | | + |> South Bridge | | +``` + +##### North Bridge + +``` + |> UPI Configuration |Displays and provides | + |> Memory Configuration |option to change the UPI | + |> IIO Configuration |Settings | +``` + +##### UPI Configuration + +``` + | UPI Configuration |Use this feature to select | + | -------------------------------------------------- |the degrading precedence | + | Number of CPU 2 |option for Ultra Path | + | Number of Active UPI Link 3 |Interconnect connections. | + | Current UPI Link Speed Fast |Select Topology Precedent | + | Current UPI Link Frequency 10.4 GT/s |to degrade UPI features if | + | UPI Global MMIO Low Base / Limit 90000000 / FBFFFFFF |system options are in | + | UPI Global MMIO High Base / Limit 0000000000000000 / |conflict. Select Feature | + | 00000000FFFFFFFF |Precedent to degrade UPI | + | UPI Pci-e Configuration Base / Size 80000000 / 10000000 |topology if system options | + | Degrade Precedence [Topology Precedence] |are in conflict. | + | Link L0p Enable [Disable] | | + | Link L1 Enable [Disable] | | + | IO Directory Cache (IODC) [Auto] | | + | SNC [Disable] | | + | XPT Prefetch [Disable] | | + | KTI Prefetch [Enable] |-----------------------------| + | Local/Remote Threshold [Auto] |><: Select Screen | + | Stale AtoS [Auto] |^v: Select Item | + | LLC Dead Line Alloc [Enable] |Enter: Select | + | Isoc Mode [Auto] |+/-: Change Opt. | +``` + +##### Memory Configuration + +``` + | |Select POR to enforce POR | + | -------------------------------------------------- |restrictions for DDR4 | + | Integrated Memory Controller (iMC) |frequency and voltage | + | -------------------------------------------------- |programming | + | | | + | Enforce POR [POR] | | + | PPR Type [Hard PPR] | | + | Enhanced PPR [Disable] | | + | Operation Mode [Test and Repair] | | + | Memory Frequency [2933] | | + | Data Scrambling for DDR4 [Auto] | | + | tCCD_L Relaxation [Auto] | | + | tRWSR Relaxation [Disable] | | + | tRFC Optimization for 16Gb Based DIMM [Force 550ns] | | + | 2x Refresh [Auto] | | + | Page Policy [Auto] | | + | IMC Interleaving [2-way Interleave] |-----------------------------| + |> Memory Topology |><: Select Screen | + |> Memory RAS Configuration |^v: Select Item | +``` + +##### IIO Configuration + +``` + | IIO Configuration |Expose IIO DFX devices and | + | -------------------------------------------------- |other CPU devices like PMON | + | | | + | EV DFX Features [Disable] | | + |> CPU1 Configuration | | + |> CPU2 Configuration | | + |> IOAT Configuration | | + |> Intel. VT for Directed I/O (VT-d) | | + |> Intel. VMD technology | | + | | | + | IIO-PCIE Express Global Options | | + | ======================================== | | + | PCI-E Completion Timeout Disable [No] | | +``` + +##### CPU1 Configuration + +``` + | IOU0 (IIO PCIe Br1) [Auto] |Selects PCIe port | + | IOU1 (IIO PCIe Br2) [Auto] |Bifurcation for selected | + | IOU2 (IIO PCIe Br3) [Auto] |slot(s) | + |> CPU1 SLOT2 PCI-E 3.0 X16 | | + |> CPU1 SLOT4 PCI-E 3.0 X16 | | + |> CPU1 SLOT9 PCI-E 3.0 X16 | | +``` + +##### CPU2 Configuration + +``` + | IOU0 (IIO PCIe Br1) [Auto] |Selects PCIe port | + | IOU1 (IIO PCIe Br2) [Auto] |Bifurcation for selected | + | IOU2 (IIO PCIe Br3) [Auto] |slot(s) | + |> CPU2 SLOT6 PCI-E 3.0 X16 | | + |> CPU2 SLOT8 PCI-E 3.0 X16 | | + |> CPU2 SLOT10 PCI-E 3.0 X16 | | +``` + +#### South Bridge + +``` + | |Enables Legacy USB support. | + | USB Module Version 21 |AUTO option disables legacy | + | |support if no USB devices | + | USB Devices: |are connected. DISABLE | + | 1 Keyboard, 1 Mouse, 1 Hub |option will keep USB | + | |devices available only for | + | Legacy USB Support [Enabled] |EFI applications. | + | XHCI Hand-off [Enabled] | | + | Port 60/64 Emulation [Enabled] | | + | PCIe PLL SSC [Disable] | | + | Real USB Wake Up [Enabled] | | + | Front USB Wake Up [Enabled] | | + | | | + | Azalia [Auto] | | + | Azalia PME Enable [Disabled] | | +``` + +### PCIe/PCI/PnP Configuration + +``` + | PCI Bus Driver Version A5.01.18 ^|Enables or Disables 64bit | + | *|capable Devices to be | + | PCI Devices Common Settings: *|Decoded in Above 4G Address | + | Above 4G Decoding [Enabled] *|Space (Only if System | + | SR-IOV Support [Enabled] *|Supports 64 bit PCI | + | ARI Support [Enabled] *|Decoding). | + | MMIO High Base [56T] *| | + | MMIO High Granularity Size [256G] *| | + | Maximum Read Request [Auto] *| | + | MMCFG Base [2G] *| | + | NVMe Firmware Source [Vendor Defined *| | + | Firmware] *| | + | VGA Priority [Onboard] *| | + | CPU1 SLOT2 PCI-E 3.0 X16 OPROM [Legacy] *| | + | CPU1 SLOT4 PCI-E 3.0 X16 OPROM [Legacy] *| | + | CPU2 SLOT6 PCI-E 3.0 X16 OPROM [Legacy] *| | + | CPU2 SLOT8 PCI-E 3.0 X16 OPROM [Legacy] *|-----------------------------| + | CPU1 SLOT9 PCI-E 3.0 X16 OPROM [Legacy] *|><: Select Screen | + | CPU2 SLOT10 PCI-E 3.0 X16 OPROM [Legacy] *|^v: Select Item | + | CPU2 SLOT11 PCI-E 3.0 X4(IN X8) OPROM [Legacy] *|Enter: Select | + | M.2 CONNECTOR OPROM [Legacy] *|+/-: Change Opt. | + | Bus Master Enable [Enabled] +|F1: General Help | + | Onboard LAN1 Option ROM [Legacy] +|F2: Previous Values | + | Onboard LAN2 Option ROM [Disabled] +|F3: Optimized Defaults | + | Onboard Video Option ROM [Legacy] v|F4: Save & Exit | + |> Network Stack Configuration | | +``` + +### ACPI Settings + +``` + | ACPI Settings |Enable or Disable Non | + | |uniform Memory Access | + | NUMA [Enabled] |(NUMA). | + | WHEA Support [Enabled] | | + | High Precision Event Timer [Enabled] | | +``` + + +## Xeon Clx Server Firmware Inventory + +``` +Host. IPMI IP. BMC. BIOS. CPLD. CPU Microcode. PCI Bus. X710 Firmware. XXV710 Firmware. i40e. MLX5 Firmware. mlx5_core E810 Firmware. ice. +s33-t27-sut1. 10.30.55.18. 1.67. 3.2. 03.B1.05. 0500002C. A5.01.18. 8.70. 8.70. 2.19.3. 16.32.1010. 5.6-1.0.3.3. 4.00. 1.9.7. +s34-t27-tg1. 10.30.55.19. 1.67. 3.2. 03.B1.05. 0500002C. A5.01.18. 8.30. 8.30. 2.19.3. 16.32.1010. 5.6-1.0.3.3. N/A. N/A. +s35-t28-sut1. 10.30.55.20. 1.67. 3.2. 03.B1.05. 0500002C. A5.01.18. 8.70. 8.70. 2.19.3. 16.32.1010. 5.6-1.0.3.3. 4.00. 1.9.7. +s36-t28-tg1. 10.30.55.21. 1.67. 3.2. 03.B1.05. 0500002C. A5.01.18. 8.30. 8.30. 2.19.3. 16.32.1010. 5.6-1.0.3.3. N/A. N/A. +s37-t29-sut1. 10.30.55.22. 1.67. 3.2. 03.B1.05. 0500002C. A5.01.18. 8.70. 8.70. 2.19.3. 16.32.1010. 5.6-1.0.3.3. 4.00. 1.9.7. +s38-t29-tg1. 10.30.55.23. 1.67. 3.2. 03.B1.05. 0500002C. A5.01.18. 8.30. 8.30. 2.19.3. 16.32.1010. 5.6-1.0.3.3. N/A. N/A. +``` diff --git a/docs/content/infrastructure/testbed_configuration/sm_icx_hw_bios_cfg.md b/docs/content/infrastructure/testbed_configuration/sm_icx_hw_bios_cfg.md new file mode 100644 index 0000000000..854d3d0418 --- /dev/null +++ b/docs/content/infrastructure/testbed_configuration/sm_icx_hw_bios_cfg.md @@ -0,0 +1,1113 @@ +--- +bookToc: true +title: "SuperMicro Icelake" +--- + +# SuperMicro Icelake + +## Linux lscpu + +``` +$ lscpu +Architecture: x86_64 +CPU op-mode(s): 32-bit, 64-bit +Byte Order: Little Endian +Address sizes: 46 bits physical, 57 bits virtual +CPU(s): 128 +On-line CPU(s) list: 0-127 +Thread(s) per core: 2 +Core(s) per socket: 32 +Socket(s): 2 +NUMA node(s): 2 +Vendor ID: GenuineIntel +CPU family: 6 +Model: 106 +Model name: Intel(R) Xeon(R) Platinum 8358 CPU @ 2.60GHz +Stepping: 6 +CPU MHz: 3283.980 +BogoMIPS: 5200.00 +Virtualization: VT-x +L1d cache: 3 MiB +L1i cache: 2 MiB +L2 cache: 80 MiB +L3 cache: 96 MiB +NUMA node0 CPU(s): 0-31,64-95 +NUMA node1 CPU(s): 32-63,96-127 +Vulnerability Itlb multihit: Not affected +Vulnerability L1tf: Not affected +Vulnerability Mds: Not affected +Vulnerability Meltdown: Not affected +Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp +Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization +Vulnerability Spectre v2: Mitigation; Enhanced IBRS, IBPB conditional, RSB filling +Vulnerability Srbds: Not affected +Vulnerability Tsx async abort: Not affected +Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe sysca + ll nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmu + lqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadl + ine_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 invpcid_single ssbd mba ibrs ibpb stibp + ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a avx + 512f avx512dq rdseed adx smap avx512ifma clflushopt clwb intel_pt avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 x + saves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local wbnoinvd dtherm ida arat pln pts avx512vbmi umip pku ospke avx512_vbm + i2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg tme avx512_vpopcntdq rdpid md_clear pconfig flush_l1d arch_capabilities +``` + +## Linux dmidecode + +``` +# dmidecode 3.2 +Getting SMBIOS data from sysfs. +SMBIOS 3.3.0 present. +# SMBIOS implementations newer than version 3.2.0 are not +# fully supported by this version of dmidecode. +Table at 0x6BAEE000. + +Handle 0x0000, DMI type 0, 26 bytes +BIOS Information + Vendor: American Megatrends International, LLC. + Version: 1.1 + Release Date: 04/09/2021 + Address: 0xF0000 + Runtime Size: 64 kB + ROM Size: 32 MB + Characteristics: + PCI is supported + BIOS is upgradeable + BIOS shadowing is allowed + Boot from CD is supported + Selectable boot is supported + BIOS ROM is socketed + EDD is supported + Japanese floppy for NEC 9800 1.2 MB is supported (int 13h) + Japanese floppy for Toshiba 1.2 MB is supported (int 13h) + 5.25"/360 kB floppy services are supported (int 13h) + 5.25"/1.2 MB floppy services are supported (int 13h) + 3.5"/720 kB floppy services are supported (int 13h) + 3.5"/2.88 MB floppy services are supported (int 13h) + Print screen service is supported (int 5h) + Serial services are supported (int 14h) + Printer services are supported (int 17h) + CGA/mono video services are supported (int 10h) + USB legacy is supported + BIOS boot specification is supported + Targeted content distribution is supported + UEFI is supported + BIOS Revision: 5.22 + +Handle 0x0001, DMI type 1, 27 bytes +System Information + Manufacturer: Supermicro + Product Name: SYS-740GP-TNRT + Version: 0123456789 + Serial Number: S424016X1B00510 + UUID: 0698ae00-2383-11ec-8000-3cecefb9a6ba + Wake-up Type: Power Switch + SKU Number: To be filled by O.E.M. + Family: Family + +Handle 0x0002, DMI type 2, 15 bytes +Base Board Information + Manufacturer: Supermicro + Product Name: X12DPG-QT6 + Version: 1.00 + Serial Number: UM219S003392 + Asset Tag: Base Board Asset Tag + Features: + Board is a hosting board + Board is replaceable + Location In Chassis: Part Component + Chassis Handle: 0x0003 + Type: Motherboard + Contained Object Handles: 0 + +Handle 0x0003, DMI type 3, 22 bytes +Chassis Information + Manufacturer: Supermicro + Type: Other + Lock: Not Present + Version: 0123456789 + Serial Number: C7470KK25P50098 + Asset Tag: Chassis Asset Tag + Boot-up State: Safe + Power Supply State: Safe + Thermal State: Safe + Security Status: None + OEM Information: 0x00000000 + Height: Unspecified + Number Of Power Cords: 1 + Contained Elements: 0 + SKU Number: To be filled by O.E.M. + +Handle 0x001B, DMI type 38, 18 bytes +IPMI Device Information + Interface Type: KCS (Keyboard Control Style) + Specification Version: 2.0 + I2C Slave Address: 0x10 + NV Storage Device: Not Present + Base Address: 0x0000000000000CA2 (I/O) + Register Spacing: Successive Byte Boundaries + +Handle 0x002A, DMI type 4, 48 bytes +Processor Information + Socket Designation: CPU1 + Type: Central Processor + Family: Xeon + Manufacturer: Intel(R) Corporation + ID: A6 06 06 00 FF FB EB BF + Signature: Type 0, Family 6, Model 106, Stepping 6 + Flags: + FPU (Floating-point unit on-chip) + VME (Virtual mode extension) + DE (Debugging extension) + PSE (Page size extension) + TSC (Time stamp counter) + MSR (Model specific registers) + PAE (Physical address extension) + MCE (Machine check exception) + CX8 (CMPXCHG8 instruction supported) + APIC (On-chip APIC hardware supported) + SEP (Fast system call) + MTRR (Memory type range registers) + PGE (Page global enable) + MCA (Machine check architecture) + CMOV (Conditional move instruction supported) + PAT (Page attribute table) + PSE-36 (36-bit page size extension) + CLFSH (CLFLUSH instruction supported) + DS (Debug store) + ACPI (ACPI supported) + MMX (MMX technology supported) + FXSR (FXSAVE and FXSTOR instructions supported) + SSE (Streaming SIMD extensions) + SSE2 (Streaming SIMD extensions 2) + SS (Self-snoop) + HTT (Multi-threading) + TM (Thermal monitor supported) + PBE (Pending break enabled) + Version: Intel(R) Xeon(R) Platinum 8358 CPU @ 2.60GHz + Voltage: 1.6 V + External Clock: 100 MHz + Max Speed: 4500 MHz + Current Speed: 2600 MHz + Status: Populated, Enabled + Upgrade: <OUT OF SPEC> + L1 Cache Handle: 0x0027 + L2 Cache Handle: 0x0028 + L3 Cache Handle: 0x0029 + Serial Number: Not Specified + Asset Tag: UNKNOWN + Part Number: Not Specified + Core Count: 32 + Core Enabled: 32 + Thread Count: 64 + Characteristics: + 64-bit capable + Multi-Core + Hardware Thread + Execute Protection + Enhanced Virtualization + Power/Performance Control + +Handle 0x002E, DMI type 4, 48 bytes +Processor Information + Socket Designation: CPU2 + Type: Central Processor + Family: Xeon + Manufacturer: Intel(R) Corporation + ID: A6 06 06 00 FF FB EB BF + Signature: Type 0, Family 6, Model 106, Stepping 6 + Flags: + FPU (Floating-point unit on-chip) + VME (Virtual mode extension) + DE (Debugging extension) + PSE (Page size extension) + TSC (Time stamp counter) + MSR (Model specific registers) + PAE (Physical address extension) + MCE (Machine check exception) + CX8 (CMPXCHG8 instruction supported) + APIC (On-chip APIC hardware supported) + SEP (Fast system call) + MTRR (Memory type range registers) + PGE (Page global enable) + MCA (Machine check architecture) + CMOV (Conditional move instruction supported) + PAT (Page attribute table) + PSE-36 (36-bit page size extension) + CLFSH (CLFLUSH instruction supported) + DS (Debug store) + ACPI (ACPI supported) + MMX (MMX technology supported) + FXSR (FXSAVE and FXSTOR instructions supported) + SSE (Streaming SIMD extensions) + SSE2 (Streaming SIMD extensions 2) + SS (Self-snoop) + HTT (Multi-threading) + TM (Thermal monitor supported) + PBE (Pending break enabled) + Version: Intel(R) Xeon(R) Platinum 8358 CPU @ 2.60GHz + Voltage: 1.6 V + External Clock: 100 MHz + Max Speed: 4500 MHz + Current Speed: 2600 MHz + Status: Populated, Enabled + Upgrade: <OUT OF SPEC> + L1 Cache Handle: 0x002B + L2 Cache Handle: 0x002C + L3 Cache Handle: 0x002D + Serial Number: Not Specified + Asset Tag: UNKNOWN + Part Number: Not Specified + Core Count: 32 + Core Enabled: 32 + Thread Count: 64 + Characteristics: + 64-bit capable + Multi-Core + Hardware Thread + Execute Protection + Enhanced Virtualization + Power/Performance Control +``` + +## Linux dmidecode memory + +``` +Handle 0x0034, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0033 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P1-DIMMA1 + Bank Locator: P0_Node0_Channel0_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: H0MK0001304705C2E3 + Asset Tag: P1-DIMMA1_AssetTag (date:21/30) + Part Number: M393A2K43DB3-CWE + Rank: 2 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 16 GB + Cache Size: None + Logical Size: None + +Handle 0x0036, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0033 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P1-DIMMB1 + Bank Locator: P0_Node0_Channel1_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: H0MK0001304705CE60 + Asset Tag: P1-DIMMB1_AssetTag (date:21/30) + Part Number: M393A2K43DB3-CWE + Rank: 2 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 16 GB + Cache Size: None + Logical Size: None + +Handle 0x0038, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0033 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P1-DIMMC1 + Bank Locator: P0_Node0_Channel2_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: H0MK0001304705C59E + Asset Tag: P1-DIMMC1_AssetTag (date:21/30) + Part Number: M393A2K43DB3-CWE + Rank: 2 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 16 GB + Cache Size: None + Logical Size: None + +Handle 0x003A, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0033 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P1-DIMMD1 + Bank Locator: P0_Node0_Channel3_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: H0MK0001304705D12D + Asset Tag: P1-DIMMD1_AssetTag (date:21/30) + Part Number: M393A2K43DB3-CWE + Rank: 2 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 16 GB + Cache Size: None + Logical Size: None + +Handle 0x003C, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0033 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P1-DIMME1 + Bank Locator: P0_Node1_Channel0_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: H0MK0001304705C69C + Asset Tag: P1-DIMME1_AssetTag (date:21/30) + Part Number: M393A2K43DB3-CWE + Rank: 2 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 16 GB + Cache Size: None + Logical Size: None + +Handle 0x003E, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0033 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P1-DIMMF1 + Bank Locator: P0_Node1_Channel1_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: H0MK0001304705C33A + Asset Tag: P1-DIMMF1_AssetTag (date:21/30) + Part Number: M393A2K43DB3-CWE + Rank: 2 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 16 GB + Cache Size: None + Logical Size: None + +Handle 0x0040, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0033 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P1-DIMMG1 + Bank Locator: P0_Node1_Channel2_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: H0MK0001304705C59F + Asset Tag: P1-DIMMG1_AssetTag (date:21/30) + Part Number: M393A2K43DB3-CWE + Rank: 2 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 16 GB + Cache Size: None + Logical Size: None + +Handle 0x0042, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0033 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P1-DIMMH1 + Bank Locator: P0_Node1_Channel3_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: H0MK0001304705CA16 + Asset Tag: P1-DIMMH1_AssetTag (date:21/30) + Part Number: M393A2K43DB3-CWE + Rank: 2 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 16 GB + Cache Size: None + Logical Size: None + +Handle 0x0044, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0033 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P2-DIMMA1 + Bank Locator: P1_Node0_Channel0_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: H0MK0001304705CBFE + Asset Tag: P2-DIMMA1_AssetTag (date:21/30) + Part Number: M393A2K43DB3-CWE + Rank: 2 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 16 GB + Cache Size: None + Logical Size: None + +Handle 0x0046, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0033 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P2-DIMMB1 + Bank Locator: P1_Node0_Channel1_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: H0MK0001304705CFC8 + Asset Tag: P2-DIMMB1_AssetTag (date:21/30) + Part Number: M393A2K43DB3-CWE + Rank: 2 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 16 GB + Cache Size: None + Logical Size: None + +Handle 0x0048, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0033 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P2-DIMMC1 + Bank Locator: P1_Node0_Channel2_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: H0MK0001304705CC02 + Asset Tag: P2-DIMMC1_AssetTag (date:21/30) + Part Number: M393A2K43DB3-CWE + Rank: 2 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 16 GB + Cache Size: None + Logical Size: None + +Handle 0x004A, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0033 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P2-DIMMD1 + Bank Locator: P1_Node0_Channel3_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: H0MK0001304705CB5A + Asset Tag: P2-DIMMD1_AssetTag (date:21/30) + Part Number: M393A2K43DB3-CWE + Rank: 2 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 16 GB + Cache Size: None + Logical Size: None + +Handle 0x004C, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0033 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P2-DIMME1 + Bank Locator: P1_Node1_Channel0_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: H0MK0001304705CB30 + Asset Tag: P2-DIMME1_AssetTag (date:21/30) + Part Number: M393A2K43DB3-CWE + Rank: 2 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 16 GB + Cache Size: None + Logical Size: None + +Handle 0x004E, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0033 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P2-DIMMF1 + Bank Locator: P1_Node1_Channel1_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: H0MK0001304705CB87 + Asset Tag: P2-DIMMF1_AssetTag (date:21/30) + Part Number: M393A2K43DB3-CWE + Rank: 2 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 16 GB + Cache Size: None + Logical Size: None + +Handle 0x0050, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0033 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P2-DIMMG1 + Bank Locator: P1_Node1_Channel2_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: H0MK0001304705CB08 + Asset Tag: P2-DIMMG1_AssetTag (date:21/30) + Part Number: M393A2K43DB3-CWE + Rank: 2 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 16 GB + Cache Size: None + Logical Size: None + +Handle 0x0052, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x0033 + Error Information Handle: Not Provided + Total Width: 72 bits + Data Width: 64 bits + Size: 16384 MB + Form Factor: DIMM + Set: None + Locator: P2-DIMMH1 + Bank Locator: P1_Node1_Channel3_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: Samsung + Serial Number: H0MK0001304705CC01 + Asset Tag: P2-DIMMH1_AssetTag (date:21/30) + Part Number: M393A2K43DB3-CWE + Rank: 2 + Configured Memory Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xCE + Module Product ID: Unknown + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 16 GB + Cache Size: None + Logical Size: None +``` + +## Xeon ICX Server BIOS Configuration + +### Boot Feature + +``` + | |Enables or disables Quiet | + | Quiet Boot [Enabled] |Boot option | + | | | + | Option ROM Messages [Force BIOS] | | + | Bootup NumLock State [On] | | + | Wait For "F1" If Error [Disabled] | | + | INT19 Trap Response [Immediate] | | + | Re-try Boot [Disabled] | | + | | | + | Power Configuration | | + | | | + | Watch Dog Function [Disabled] | | + | Restore on AC Power Loss [Last State] | | + | Power Button Function [Instant Off] | | + | Deep Sleep Mode [Disabled] | | +``` + +### CPU Configuration + +``` + | Processor Configuration ^|Enables Hyper Threading | + | -------------------------------------------------- *|(Software Method to | + | Processor BSP Revision 606A6 - ICX D2 *|Enable/Disable Logical | + | Processor Socket CPU1 CPU2 *|Processor threads. | + | Processor ID 000606A6* | 000606A6 *| | + | Processor Frequency 2.600GHz | 2.600GHz *| | + | Processor Max Ratio 1AH | 1AH *| | + | Processor Min Ratio 08H | 08H *| | + | Microcode Revision 0D000280 | 0D000280 *| | + | L1 Cache RAM(Per Core) 80KB | 80KB *| | + | L2 Cache RAM(Per Core) 1280KB | 1280KB *| | + | L3 Cache RAM(Per Package) 49152KB | 49152KB *| | + | Processor 0 Version Intel(R) Xeon(R) *| | + | Platinum 8358 CPU @ *| | + | 2.60GHz *| | + | Processor 1 Version Intel(R) Xeon(R) *| | + | Platinum 8358 CPU @ *| | + | 2.60GHz *| | + | +| | + |> CPU1 Core Disable Bitmap +| | + |> CPU2 Core Disable Bitmap +|-----------------------------| + | Hyper-Threading [ALL] [Enable] +|><: Select Screen | + | Hardware Prefetcher [Enable] +|^v: Select Item | + | Adjacent Cache Prefetch [Enable] +|Enter: Select | + | DCU Streamer Prefetcher [Enable] +|+/-: Change Opt. | + | DCU IP Prefetcher [Enable] +|F1: General Help | + | LLC Prefetch [Enable] +|F2: Previous Values | + | Extended APIC [Disable] +|F3: Optimized Defaults | + | VMX [Enable] v|F4: Save & Exit | + | Enable SMX [Disable] +| | + | PPIN Control [Unlock/Enable] *| | + | AES-NI [Enable] *| | + | -------------------------------------------------- *| | + | TME, TME-MT, TDX *| | + | -------------------------------------------------- *| | + | Total Memory Encryption (TME) [Disabled] *| | + | -------------------------------------------------- *|-----------------------------| + | Software Guard Extension (SGX) *|><: Select Screen | + | -------------------------------------------------- *|^v: Select Item | + | SGX Factory Reset [Disabled] *|Enter: Select | + | SW Guard Extensions (SGX) [Disabled] *|+/-: Change Opt. | + | SGX Package Info In-Band Access [Disabled] *|F1: General Help | + | -------------------------------------------------- *|F2: Previous Values | + | Limit CPU PA to 46 Bits [Enable] *|F3: Optimized Defaults | + |> Advanced Power Management Configuration v|F4: Save & Exit | +``` + +#### Advanced Power Management Configuration + +``` + | Advanced Power Management Configuration |Enable processor power | + | -------------------------------------------------- |management features. | + | Power Technology [Custom] | | + | Power Performance Tuning [BIOS Controls EPB] | | + | ENERGY_PERF_BIAS_CFG Mode [Maximum Performance] | | +``` + +##### CPU P State Control + +``` + | CPU P State Control |EIST allows the processor | + | |to dynamically adjust | + | SpeedStep (P-States) [Disable] |frequency and voltage based | + | Activate SST-BF [Disable] |on power versus performance | + | Configure SST-BF [Enable] |needs. | + | EIST PSD Function [HW_ALL] | | +``` + +##### Hardware PM State Control + +``` + | Hardware PM State Control |If set to Disable, hardware ^| + | |will choose a P-state *| + | Hardware P-States [Disable] |setting for the system *| + | |based on an OS request. *| + + | Frequency Prioritization |This knob controls whether | + | |RAPL balancer is enabled. | + | RAPL Prioritization [Disable] |When enabled it activates | +``` + +##### CPU C State Control + +``` + | CPU C State Control |Allows Monitor and MWAIT | + | |instructions. | + | Enable Monitor MWAIT [Enable] | | + | CPU C6 Report [Disable] | | + | Enhanced Halt State (C1E) [Disable] | | +``` + +##### Package C State Control + +``` + | Package C State Control |Limit the lowest package | + | |level C-State to | + | Package C State [C0/C1 state] |processors. Lower package | +``` + +##### CPU T State Control + +``` + | CPU T State Control |Enable/Disable CPU | + | |throttling by OS. | + | Software Controlled T-States [Disable] |Throttling reduces power | +``` + +##### UPI Configuration + +``` + | Uncore Configuration |Choose Topology Precedence | + | -------------------------------------------------- |to degrade features if | + | Number of CPU 2 |system options are in | + | Number of IIO 2 |conflict or choose Feature | + | Current UPI Link Speed Fast |Precedence to degrade | + | Current UPI Link Frequency 11.2 GT/s |topology if system options | + | Global MMIO Low Base / Limit 90000000 / FBFFFFFF |are in conflict. | + | Global MMIO High Base / Limit 0000200000000000 / | | + | 0000204FFFFFFFFF | | + | Pci-e Configuration Base / Size 80000000 / 10000000 | | + | Degrade Precedence [Topology Precedence] | | + | Link L0p Enable [Disable] | | + | Link L1 Enable [Disable] | | + | XPT Remote Prefetch [Auto] | | + | KTI Prefetch [Auto] |-----------------------------| + | Local/Remote Threshold [Auto] |><: Select Screen | + | IO Directory Cache (IODC) [Auto] |^v: Select Item | + | SNC (Sub NUMA) [Disable] |Enter: Select | + | XPT Prefetch [Auto] |+/-: Change Opt. | + | Snoop Throttle Configuration [Auto] |F1: General Help | + | PCIe Remote P2P Relaxed Ordering [Disable] |F2: Previous Values | + | Stale AtoS [Auto] |F3: Optimized Defaults | + | LLC Dead Line Alloc [Enable] |F4: Save & Exit | +``` + +##### Memory Configuration + +``` + | |Set Enable or Disable | + | -------------------------------------------------- |STEP(Samsung TestBIOS & | + | Integrated Memory Controller (iMC) |Enhanced PPR)function | + | -------------------------------------------------- | | + | | | + | STEP DRAM Test [Disable] | | + | Operation Mode [Test and Repair] | | + | Enforce POR [POR] | | + | PPR Type [Hard PPR] | | + | Memory Frequency [Auto] | | + | Data Scrambling for DDR4 [Enable] | | + | 2x Refresh Enable [Auto] | | +``` + +##### IIO Configuration + +``` + | IIO Configuration |Press <Enter> to bring up | + | -------------------------------------------------- |the Intel. Virtualization | + | |for Directed I/O (VT-d) | + |> CPU1 Configuration |Configuration menu. | + |> CPU2 Configuration | | + |> IOAT Configuration | | + |> Intel. VT for Directed I/O (VT-d) | | + |> Intel. VMD Technology | | + | PCI-E ASPM Support (Global) [Disable] | | + | IIO eDPC Support [Disable] | | +``` + +##### CPU1 Configuration + +``` + | IOU0 (IIO PCIe Port 1) [Auto] |Selects PCIe port | + | IOU1 (IIO PCIe Port 2) [Auto] |Bifurcation for selected | + | IOU3 (IIO PCIe Port 4) [Auto] |slot(s) | + | IOU4 (IIO PCIe Port 5) [Auto] | | +``` + +##### CPU2 Configuration + +``` + | IOU0 (IIO PCIe Port 1) [Auto] |Selects PCIe port | + | IOU1 (IIO PCIe Port 2) [Auto] |Bifurcation for selected | + | IOU3 (IIO PCIe Port 4) [Auto] |slot(s) | + | IOU4 (IIO PCIe Port 5) [Auto] | | +``` + +#### South Bridge + +``` + | USB Module Version 26 |AUTO option disables legacy | + | |support if no USB devices | + | USB Devices: |are connected. DISABLE | + | 1 Drive, 2 Keyboards, 2 Mice, 1 Hub |option will keep USB | + | |devices available only for | + | Legacy USB Support [Enabled] |EFI applications. | + | XHCI Hand-off [Enabled] | | + | Port 60/64 Emulation [Disabled] | | + | PCIe PLL SSC [Disabled] | | + | Port 61h Bit-4 Emulation [Disabled] | | +``` + +### PCIe/PCI/PnP Configuration + +``` + | PCI Bus Driver Version A5.01.24 ^|Enables or Disables 64bit | + | *|capable Devices to be | + | PCI Devices Common Settings: *|Decoded in Above 4G Address | + | Above 4G Decoding [Enabled] *|Space (Only if System | + | SR-IOV Support [Enabled] *|Supports 64 bit PCI | + | ARI Support [Enabled] *|Decoding). | + | Bus Master Enable [Enabled] *| | + | Consistent Device Name Support [Disabled] *| | + | MMIO High Base [32T] *| | + | MMIO High Granularity Size [64G] *| | + | Maximum Read Request [Auto] *| | + | MMCFG Base [Auto] *| | + | NVMe Firmware Source [Vendor Defined *| | + | Firmware] *| | + | VGA Priority [Onboard] *| | + | CPU1 SLOT2 PCI-E 3.0 X16 OPROM [EFI] *| | + | CPU1 SLOT4 PCI-E 3.0 X16 OPROM [EFI] *| | + | CPU2 SLOT6 PCI-E 3.0 X16 OPROM [EFI] *| | + | CPU2 SLOT8 PCI-E 3.0 X16 OPROM [EFI] *|-----------------------------| + | CPU1 SLOT9 PCI-E 3.0 X16 OPROM [EFI] *|><: Select Screen | + | CPU2 SLOT10 PCI-E 3.0 X16 OPROM [EFI] *|^v: Select Item | + | CPU2 SLOT11 PCI-E 3.0 X4(IN X8) OPROM [EFI] *|Enter: Select | + | M.2 CONNECTOR OPROM [EFI] *|+/-: Change Opt. | + | Bus Master Enable [Enabled] +|F1: General Help | + | Onboard LAN1 Option ROM [EFI] +|F2: Previous Values | + | Onboard LAN2 Option ROM [Disabled] +|F3: Optimized Defaults | + | Onboard Video Option ROM [EFI] v|F4: Save & Exit | + |> Network Stack Configuration | | +``` + +## Xeon ICX Server Firmware Inventory + +``` +Host. IPMI IP. BMC. BIOS. CPLD. CPU Microcode. PCI Bus. X710 Firmware. i40e. E810 Firmware. ice. +s65-t37-sut1. 10.30.50.75. 1.00.21. 1.1. F1.00.07. 0D000280. A5.01.24. 8.70. 2.19.3. 4.00. 1.9.7. +s66-t37-sut2. 10.30.50.76. 1.00.21. 1.1. F1.00.07. 0D000280. A5.01.24. 8.70. 2.19.3. 4.00. 1.9.7. +s67-t37-tg1. 10.30.50.77. 1.00.21. 1.1. F1.00.07. 0D000280. A5.01.24. 8.30. 2.19.3. 3.20. 1.9.7. +s78-t38-sut1. 10.30.50.78. 1.00.21. 1.1. F1.00.07. 0D000280. A5.01.24. 8.70. 2.19.3. 4.00. 1.9.7. +s79-t38-sut2. 10.30.50.79. 1.00.21. 1.1. F1.00.07. 0D000280. A5.01.24. 8.70. 2.19.3. 4.00. 1.9.7. +s80-t38-tg1. 10.30.50.80. 1.00.21. 1.1. F1.00.07. 0D000280. A5.01.24. 8.30. 2.19.3. 3.20. 1.9.7. +s71-t212-sut1. 10.30.50.81. 1.00.21. 1.1. F1.00.07. 0D000280. A5.01.24. 8.70. 2.19.3. 4.00. 1.9.7. +s72-t212-tg1. 10.30.50.82. 1.00.21. 1.1. F1.00.07. 0D000280. A5.01.24. 8.30. 2.19.3. 3.20. 1.9.7. +s83-t213-sut1. 10.30.50.83. 1.00.21. 1.1. F1.00.07. 0D000280. A5.01.24. 8.70. 2.19.3. 4.00. 1.9.7. +s84-t213-tg1. 10.30.50.84. 1.00.21. 1.1. F1.00.07. 0D000280. A5.01.24. 8.30. 2.19.3. 3.20. 1.9.7. +s85-t214-sut1. 10.30.50.85. 1.00.21. 1.1. F1.00.07. 0D000280. A5.01.24. 8.70. 2.19.3. 4.00. 1.9.7. +s86-t214-tg1. 10.30.50.86. 1.00.21. 1.1. F1.00.07. 0D000280. A5.01.24. 8.30. 2.19.3. 3.20. 1.9.7. +s87-t215-sut1. 10.30.50.87. 1.00.21. 1.1. F1.00.07. 0D000280. A5.01.24. 8.70. 2.19.3. 4.00. 1.9.7. +s88-t215-tg1. 10.30.50.88. 1.00.21. 1.1. F1.00.07. 0D000280. A5.01.24. 8.30. 2.19.3. 3.20. 1.9.7. +``` diff --git a/docs/content/infrastructure/testbed_configuration/sm_spr_hw_bios_cfg.md b/docs/content/infrastructure/testbed_configuration/sm_spr_hw_bios_cfg.md new file mode 100644 index 0000000000..c2bf8fb795 --- /dev/null +++ b/docs/content/infrastructure/testbed_configuration/sm_spr_hw_bios_cfg.md @@ -0,0 +1,845 @@ +--- +bookToc: true +title: "SuperMicro SapphireRapids" +--- + +# SuperMicro SapphireRapids + +## Linux lscpu + +``` +Architecture: x86_64 + CPU op-mode(s): 32-bit, 64-bit + Address sizes: 46 bits physical, 57 bits virtual + Byte Order: Little Endian +CPU(s): 128 + On-line CPU(s) list: 0-127 +Vendor ID: GenuineIntel + Model name: Intel(R) Xeon(R) Platinum 8462Y+ + CPU family: 6 + Model: 143 + Thread(s) per core: 2 + Core(s) per socket: 32 + Socket(s): 2 + Stepping: 8 + BogoMIPS: 5600.00 + Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1 + gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dt + es64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsav + e avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cat_l2 cdp_l3 invpcid_single cdp_l2 ssbd mba ibrs ibpb stibp ibrs_enh + anced tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a avx512f avx512dq rdsee + d adx smap avx512ifma clflushopt clwb intel_pt avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cq + m_mbm_total cqm_mbm_local split_lock_detect avx_vnni avx512_bf16 wbnoinvd dtherm arat pln pts hwp hwp_act_window hwp_epp hwp_pkg_req avx + 512vbmi umip pku ospke waitpkg avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg tme avx512_vpopcntdq la57 rdpid bus_lock_dete + ct cldemote movdiri movdir64b enqcmd fsrm md_clear serialize tsxldtrk pconfig arch_lbr amx_bf16 avx512_fp16 amx_tile amx_int8 flush_l1d + arch_capabilities +Virtualization features: + Virtualization: VT-x +Caches (sum of all): + L1d: 3 MiB (64 instances) + L1i: 2 MiB (64 instances) + L2: 128 MiB (64 instances) + L3: 120 MiB (2 instances) +NUMA: + NUMA node(s): 2 + NUMA node0 CPU(s): 0-31,64-95 + NUMA node1 CPU(s): 32-63,96-127 +Vulnerabilities: + Itlb multihit: Not affected + L1tf: Not affected + Mds: Not affected + Meltdown: Not affected + Mmio stale data: Not affected + Retbleed: Not affected + Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp + Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization + Spectre v2: Mitigation; Enhanced IBRS, IBPB conditional, RSB filling + Srbds: Not affected + Tsx async abort: Not affected +``` + +## Linux dmidecode + +``` +# dmidecode 3.3 +Getting SMBIOS data from sysfs. +SMBIOS 3.5.0 present. +Table at 0x000E6E00. + +Handle 0x0000, DMI type 0, 26 bytes +BIOS Information + Vendor: American Megatrends International, LLC. + Version: 1.0 + Release Date: 11/16/2022 + Address: 0xF0000 + Runtime Size: 64 kB + ROM Size: 32 MB + Characteristics: + PCI is supported + BIOS is upgradeable + BIOS shadowing is allowed + Boot from CD is supported + Selectable boot is supported + BIOS ROM is socketed + EDD is supported + Japanese floppy for NEC 9800 1.2 MB is supported (int 13h) + Japanese floppy for Toshiba 1.2 MB is supported (int 13h) + 5.25"/360 kB floppy services are supported (int 13h) + 5.25"/1.2 MB floppy services are supported (int 13h) + 3.5"/720 kB floppy services are supported (int 13h) + 3.5"/2.88 MB floppy services are supported (int 13h) + Print screen service is supported (int 5h) + Serial services are supported (int 14h) + Printer services are supported (int 17h) + CGA/mono video services are supported (int 10h) + ACPI is supported + USB legacy is supported + BIOS boot specification is supported + Targeted content distribution is supported + UEFI is supported + BIOS Revision: 5.29 + +Handle 0x0001, DMI type 1, 27 bytes +System Information + Manufacturer: Supermicro + Product Name: SYS-741GE-TNRT + Version: 0123456789 + Serial Number: S512539X3109946 + UUID: 00000000-0000-0000-0000-7cc255275836 + Wake-up Type: Power Switch + SKU Number: To be filled by O.E.M. + Family: Family + +Handle 0x0002, DMI type 2, 15 bytes +Base Board Information + Manufacturer: Supermicro + Product Name: X13DEG-QT + Version: 1.10 + Serial Number: OM22CS039806 + Asset Tag: Base Board Asset Tag + Features: + Board is a hosting board + Board is replaceable + Location In Chassis: Part Component + Chassis Handle: 0x0003 + Type: Motherboard + Contained Object Handles: 0 + +Handle 0x0003, DMI type 3, 22 bytes +Chassis Information + Manufacturer: Supermicro + Type: Other + Lock: Not Present + Version: 0123456789 + Serial Number: C7490FL36A40118 + Asset Tag: Chassis Asset Tag + Boot-up State: Safe + Power Supply State: Safe + Thermal State: Safe + Security Status: None + OEM Information: 0x00000000 + Height: Unspecified + Number Of Power Cords: 1 + Contained Elements: 0 + SKU Number: 0123456789 + +Handle 0x0032, DMI type 4, 50 bytes +Processor Information + Socket Designation: CPU1 + Type: Central Processor + Family: Xeon + Manufacturer: Intel(R) Corporation + ID: F8 06 08 00 FF FB EB BF + Signature: Type 0, Family 6, Model 143, Stepping 8 + Flags: + FPU (Floating-point unit on-chip) + VME (Virtual mode extension) + DE (Debugging extension) + PSE (Page size extension) + TSC (Time stamp counter) + MSR (Model specific registers) + PAE (Physical address extension) + MCE (Machine check exception) + CX8 (CMPXCHG8 instruction supported) + APIC (On-chip APIC hardware supported) + SEP (Fast system call) + MTRR (Memory type range registers) + PGE (Page global enable) + MCA (Machine check architecture) + CMOV (Conditional move instruction supported) + PAT (Page attribute table) + PSE-36 (36-bit page size extension) + CLFSH (CLFLUSH instruction supported) + DS (Debug store) + ACPI (ACPI supported) + MMX (MMX technology supported) + FXSR (FXSAVE and FXSTOR instructions supported) + SSE (Streaming SIMD extensions) + SSE2 (Streaming SIMD extensions 2) + SS (Self-snoop) + HTT (Multi-threading) + TM (Thermal monitor supported) + PBE (Pending break enabled) + Version: Intel(R) Xeon(R) Platinum 8462Y+ + Voltage: 1.6 V + External Clock: 100 MHz + Max Speed: 4000 MHz + Current Speed: 2800 MHz + Status: Populated, Enabled + Upgrade: Socket LGA4677 + L1 Cache Handle: 0x002F + L2 Cache Handle: 0x0030 + L3 Cache Handle: 0x0031 + Serial Number: Not Specified + Asset Tag: UNKNOWN + Part Number: Not Specified + Core Count: 32 + Core Enabled: 32 + Thread Count: 64 + Characteristics: + 64-bit capable + Multi-Core + Hardware Thread + Execute Protection + Enhanced Virtualization + Power/Performance Control + +Handle 0x0036, DMI type 4, 50 bytes +Processor Information + Socket Designation: CPU2 + Type: Central Processor + Family: Xeon + Manufacturer: Intel(R) Corporation + ID: F8 06 08 00 FF FB EB BF + Signature: Type 0, Family 6, Model 143, Stepping 8 + Flags: + FPU (Floating-point unit on-chip) + VME (Virtual mode extension) + DE (Debugging extension) + PSE (Page size extension) + TSC (Time stamp counter) + MSR (Model specific registers) + PAE (Physical address extension) + MCE (Machine check exception) + CX8 (CMPXCHG8 instruction supported) + APIC (On-chip APIC hardware supported) + SEP (Fast system call) + MTRR (Memory type range registers) + PGE (Page global enable) + MCA (Machine check architecture) + CMOV (Conditional move instruction supported) + PAT (Page attribute table) + PSE-36 (36-bit page size extension) + CLFSH (CLFLUSH instruction supported) + DS (Debug store) + ACPI (ACPI supported) + MMX (MMX technology supported) + FXSR (FXSAVE and FXSTOR instructions supported) + SSE (Streaming SIMD extensions) + SSE2 (Streaming SIMD extensions 2) + SS (Self-snoop) + HTT (Multi-threading) + TM (Thermal monitor supported) + PBE (Pending break enabled) + Version: Intel(R) Xeon(R) Platinum 8462Y+ + Voltage: 1.6 V + External Clock: 100 MHz + Max Speed: 4000 MHz + Current Speed: 2800 MHz + Status: Populated, Enabled + Upgrade: Socket LGA4677 + L1 Cache Handle: 0x0033 + L2 Cache Handle: 0x0034 + L3 Cache Handle: 0x0035 + Serial Number: Not Specified + Asset Tag: UNKNOWN + Part Number: Not Specified + Core Count: 32 + Core Enabled: 32 + Thread Count: 64 + Characteristics: + 64-bit capable + Multi-Core + Hardware Thread + Execute Protection + Enhanced Virtualization + Power/Performance Control +``` + +## Linux dmidecode memory + +``` +Handle 0x003D, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x003C + Error Information Handle: Not Provided + Total Width: 80 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: P1-DIMMA1 + Bank Locator: P0_Node0_Channel0_Dimm0 + Type: DDR5 + Type Detail: Synchronous Registered (Buffered) + Speed: 4800 MT/s + Manufacturer: SK Hynix + Serial Number: 80AD0122149612A27F + Asset Tag: P1-DIMMA1_AssetTag (date:22/14) + Part Number: HMCG88MEBRA107N + Rank: 2 + Configured Memory Speed: 4800 MT/s + Minimum Voltage: 1.1 V + Maximum Voltage: 1.1 V + Configured Voltage: 1.1 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xAD + Module Product ID: 0xAD00 + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 32 GB + Cache Size: None + Logical Size: None + +Handle 0x003E, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x003C + Error Information Handle: Not Provided + Total Width: 80 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: P1-DIMMB1 + Bank Locator: P0_Node0_Channel1_Dimm0 + Type: DDR5 + Type Detail: Synchronous Registered (Buffered) + Speed: 4800 MT/s + Manufacturer: SK Hynix + Serial Number: 80AD0122149612A1CC + Asset Tag: P1-DIMMB1_AssetTag (date:22/14) + Part Number: HMCG88MEBRA107N + Rank: 2 + Configured Memory Speed: 4800 MT/s + Minimum Voltage: 1.1 V + Maximum Voltage: 1.1 V + Configured Voltage: 1.1 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xAD + Module Product ID: 0xAD00 + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 32 GB + Cache Size: None + Logical Size: None + +Handle 0x003F, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x003C + Error Information Handle: Not Provided + Total Width: 80 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: P1-DIMMC1 + Bank Locator: P0_Node0_Channel2_Dimm0 + Type: DDR5 + Type Detail: Synchronous Registered (Buffered) + Speed: 4800 MT/s + Manufacturer: SK Hynix + Serial Number: 80AD0122149612A1B7 + Asset Tag: P1-DIMMC1_AssetTag (date:22/14) + Part Number: HMCG88MEBRA107N + Rank: 2 + Configured Memory Speed: 4800 MT/s + Minimum Voltage: 1.1 V + Maximum Voltage: 1.1 V + Configured Voltage: 1.1 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xAD + Module Product ID: 0xAD00 + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 32 GB + Cache Size: None + Logical Size: None + +Handle 0x0040, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x003C + Error Information Handle: Not Provided + Total Width: 80 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: P1-DIMMD1 + Bank Locator: P0_Node0_Channel3_Dimm0 + Type: DDR5 + Type Detail: Synchronous Registered (Buffered) + Speed: 4800 MT/s + Manufacturer: SK Hynix + Serial Number: 80AD0122149612927A + Asset Tag: P1-DIMMD1_AssetTag (date:22/14) + Part Number: HMCG88MEBRA107N + Rank: 2 + Configured Memory Speed: 4800 MT/s + Minimum Voltage: 1.1 V + Maximum Voltage: 1.1 V + Configured Voltage: 1.1 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xAD + Module Product ID: 0xAD00 + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 32 GB + Cache Size: None + Logical Size: None + +Handle 0x0041, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x003C + Error Information Handle: Not Provided + Total Width: 80 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: P1-DIMME1 + Bank Locator: P0_Node1_Channel0_Dimm0 + Type: DDR5 + Type Detail: Synchronous Registered (Buffered) + Speed: 4800 MT/s + Manufacturer: SK Hynix + Serial Number: 80AD0122149612A2B2 + Asset Tag: P1-DIMME1_AssetTag (date:22/14) + Part Number: HMCG88MEBRA107N + Rank: 2 + Configured Memory Speed: 4800 MT/s + Minimum Voltage: 1.1 V + Maximum Voltage: 1.1 V + Configured Voltage: 1.1 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xAD + Module Product ID: 0xAD00 + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 32 GB + Cache Size: None + Logical Size: None + +Handle 0x0042, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x003C + Error Information Handle: Not Provided + Total Width: 80 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: P1-DIMMF1 + Bank Locator: P0_Node1_Channel1_Dimm0 + Type: DDR5 + Type Detail: Synchronous Registered (Buffered) + Speed: 4800 MT/s + Manufacturer: SK Hynix + Serial Number: 80AD0122149612A7F0 + Asset Tag: P1-DIMMF1_AssetTag (date:22/14) + Part Number: HMCG88MEBRA107N + Rank: 2 + Configured Memory Speed: 4800 MT/s + Minimum Voltage: 1.1 V + Maximum Voltage: 1.1 V + Configured Voltage: 1.1 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xAD + Module Product ID: 0xAD00 + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 32 GB + Cache Size: None + Logical Size: None + +Handle 0x0043, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x003C + Error Information Handle: Not Provided + Total Width: 80 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: P1-DIMMG1 + Bank Locator: P0_Node1_Channel2_Dimm0 + Type: DDR5 + Type Detail: Synchronous Registered (Buffered) + Speed: 4800 MT/s + Manufacturer: SK Hynix + Serial Number: 80AD0122149612A1B0 + Asset Tag: P1-DIMMG1_AssetTag (date:22/14) + Part Number: HMCG88MEBRA107N + Rank: 2 + Configured Memory Speed: 4800 MT/s + Minimum Voltage: 1.1 V + Maximum Voltage: 1.1 V + Configured Voltage: 1.1 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xAD + Module Product ID: 0xAD00 + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 32 GB + Cache Size: None + Logical Size: None + +Handle 0x0044, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x003C + Error Information Handle: Not Provided + Total Width: 80 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: P1-DIMMH1 + Bank Locator: P0_Node1_Channel3_Dimm0 + Type: DDR5 + Type Detail: Synchronous Registered (Buffered) + Speed: 4800 MT/s + Manufacturer: SK Hynix + Serial Number: 80AD012214961292F4 + Asset Tag: P1-DIMMH1_AssetTag (date:22/14) + Part Number: HMCG88MEBRA107N + Rank: 2 + Configured Memory Speed: 4800 MT/s + Minimum Voltage: 1.1 V + Maximum Voltage: 1.1 V + Configured Voltage: 1.1 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xAD + Module Product ID: 0xAD00 + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 32 GB + Cache Size: None + Logical Size: None + +Handle 0x0045, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x003C + Error Information Handle: Not Provided + Total Width: 80 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: P2-DIMMA1 + Bank Locator: P1_Node0_Channel0_Dimm0 + Type: DDR5 + Type Detail: Synchronous Registered (Buffered) + Speed: 4800 MT/s + Manufacturer: SK Hynix + Serial Number: 80AD01221496129322 + Asset Tag: P2-DIMMA1_AssetTag (date:22/14) + Part Number: HMCG88MEBRA107N + Rank: 2 + Configured Memory Speed: 4800 MT/s + Minimum Voltage: 1.1 V + Maximum Voltage: 1.1 V + Configured Voltage: 1.1 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xAD + Module Product ID: 0xAD00 + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 32 GB + Cache Size: None + Logical Size: None + +Handle 0x0046, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x003C + Error Information Handle: Not Provided + Total Width: 80 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: P2-DIMMB1 + Bank Locator: P1_Node0_Channel1_Dimm0 + Type: DDR5 + Type Detail: Synchronous Registered (Buffered) + Speed: 4800 MT/s + Manufacturer: SK Hynix + Serial Number: 80AD0122149612A282 + Asset Tag: P2-DIMMB1_AssetTag (date:22/14) + Part Number: HMCG88MEBRA107N + Rank: 2 + Configured Memory Speed: 4800 MT/s + Minimum Voltage: 1.1 V + Maximum Voltage: 1.1 V + Configured Voltage: 1.1 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xAD + Module Product ID: 0xAD00 + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 32 GB + Cache Size: None + Logical Size: None + +Handle 0x0047, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x003C + Error Information Handle: Not Provided + Total Width: 80 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: P2-DIMMC1 + Bank Locator: P1_Node0_Channel2_Dimm0 + Type: DDR5 + Type Detail: Synchronous Registered (Buffered) + Speed: 4800 MT/s + Manufacturer: SK Hynix + Serial Number: 80AD0122149612936B + Asset Tag: P2-DIMMC1_AssetTag (date:22/14) + Part Number: HMCG88MEBRA107N + Rank: 2 + Configured Memory Speed: 4800 MT/s + Minimum Voltage: 1.1 V + Maximum Voltage: 1.1 V + Configured Voltage: 1.1 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xAD + Module Product ID: 0xAD00 + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 32 GB + Cache Size: None + Logical Size: None + +Handle 0x0048, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x003C + Error Information Handle: Not Provided + Total Width: 80 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: P2-DIMMD1 + Bank Locator: P1_Node0_Channel3_Dimm0 + Type: DDR5 + Type Detail: Synchronous Registered (Buffered) + Speed: 4800 MT/s + Manufacturer: SK Hynix + Serial Number: 80AD012214961292FA + Asset Tag: P2-DIMMD1_AssetTag (date:22/14) + Part Number: HMCG88MEBRA107N + Rank: 2 + Configured Memory Speed: 4800 MT/s + Minimum Voltage: 1.1 V + Maximum Voltage: 1.1 V + Configured Voltage: 1.1 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xAD + Module Product ID: 0xAD00 + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 32 GB + Cache Size: None + Logical Size: None + +Handle 0x0049, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x003C + Error Information Handle: Not Provided + Total Width: 80 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: P2-DIMME1 + Bank Locator: P1_Node1_Channel0_Dimm0 + Type: DDR5 + Type Detail: Synchronous Registered (Buffered) + Speed: 4800 MT/s + Manufacturer: SK Hynix + Serial Number: 80AD012214961292ED + Asset Tag: P2-DIMME1_AssetTag (date:22/14) + Part Number: HMCG88MEBRA107N + Rank: 2 + Configured Memory Speed: 4800 MT/s + Minimum Voltage: 1.1 V + Maximum Voltage: 1.1 V + Configured Voltage: 1.1 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xAD + Module Product ID: 0xAD00 + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 32 GB + Cache Size: None + Logical Size: None + +Handle 0x004A, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x003C + Error Information Handle: Not Provided + Total Width: 80 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: P2-DIMMF1 + Bank Locator: P1_Node1_Channel1_Dimm0 + Type: DDR5 + Type Detail: Synchronous Registered (Buffered) + Speed: 4800 MT/s + Manufacturer: SK Hynix + Serial Number: 80AD0122149612A888 + Asset Tag: P2-DIMMF1_AssetTag (date:22/14) + Part Number: HMCG88MEBRA107N + Rank: 2 + Configured Memory Speed: 4800 MT/s + Minimum Voltage: 1.1 V + Maximum Voltage: 1.1 V + Configured Voltage: 1.1 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xAD + Module Product ID: 0xAD00 + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 32 GB + Cache Size: None + Logical Size: None + +Handle 0x004B, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x003C + Error Information Handle: Not Provided + Total Width: 80 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: P2-DIMMG1 + Bank Locator: P1_Node1_Channel2_Dimm0 + Type: DDR5 + Type Detail: Synchronous Registered (Buffered) + Speed: 4800 MT/s + Manufacturer: SK Hynix + Serial Number: 80AD0122149612A299 + Asset Tag: P2-DIMMG1_AssetTag (date:22/14) + Part Number: HMCG88MEBRA107N + Rank: 2 + Configured Memory Speed: 4800 MT/s + Minimum Voltage: 1.1 V + Maximum Voltage: 1.1 V + Configured Voltage: 1.1 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xAD + Module Product ID: 0xAD00 + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 32 GB + Cache Size: None + Logical Size: None + +Handle 0x004C, DMI type 17, 92 bytes +Memory Device + Array Handle: 0x003C + Error Information Handle: Not Provided + Total Width: 80 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: P2-DIMMH1 + Bank Locator: P1_Node1_Channel3_Dimm0 + Type: DDR5 + Type Detail: Synchronous Registered (Buffered) + Speed: 4800 MT/s + Manufacturer: SK Hynix + Serial Number: 80AD0122149612A195 + Asset Tag: P2-DIMMH1_AssetTag (date:22/14) + Part Number: HMCG88MEBRA107N + Rank: 2 + Configured Memory Speed: 4800 MT/s + Minimum Voltage: 1.1 V + Maximum Voltage: 1.1 V + Configured Voltage: 1.1 V + Memory Technology: DRAM + Memory Operating Mode Capability: Volatile memory + Firmware Version: 0000 + Module Manufacturer ID: Bank 1, Hex 0xAD + Module Product ID: 0xAD00 + Memory Subsystem Controller Manufacturer ID: Unknown + Memory Subsystem Controller Product ID: Unknown + Non-Volatile Size: None + Volatile Size: 32 GB + Cache Size: None + Logical Size: None +``` + +## Xeon ICX Server Firmware Inventory + +``` +Host. IPMI IP. BMC. BIOS. CPLD. CPU Microcode. Cx-7 Firmware. mlx5. E810 Firmware. ice. +s52-t21-sut1. 10.30.50.52. 1.00.2. 1.0. F2.43.09. 0x2b0000c0. 28.34.1002. 5.9-0.5.6.0. -. -. +s53-t21-tg1. 10.30.50.53. 1.00.2. 1.0. F2.43.09. 0x2b0000c0. 28.34.1002. 5.9-0.5.6.0. -. -. +s54-t22-sut1. 10.30.50.54. 1.00.2. 1.0. F2.43.09. 0x2b0000c0. -. -. 4.00. 1.9.7. +s55-t22-tg1. 10.30.50.55. 1.00.2. 1.0. F2.43.09. 0x2b0000c0. -. -. 3.20. 1.9.7. +s56-t23-sut1. 10.30.50.56. 1.00.2. 1.0. F2.43.09. 0x2b0000c0. 28.34.1002. 5.9-0.5.6.0. 4.00. 1.9.7. +s57-t23-tg1. 10.30.50.57. 1.00.2. 1.0. F2.43.09. 0x2b0000c0. 28.34.1002. 5.9-0.5.6.0. 3.20. 1.9.7. +s58-t24-sut1. 10.30.50.58. 1.00.2. 1.0. F2.43.09. 0x2b0000c0. 28.34.1002. 5.9-0.5.6.0. 4.00. 1.9.7. +s59-t24-tg1. 10.30.50.59. 1.00.2. 1.0. F2.43.09. 0x2b0000c0. 28.34.1002. 5.9-0.5.6.0. 3.20. 1.9.7. +``` diff --git a/docs/content/infrastructure/testbed_configuration/sm_zn2_hw_bios_cfg.md b/docs/content/infrastructure/testbed_configuration/sm_zn2_hw_bios_cfg.md new file mode 100644 index 0000000000..31335d5cc7 --- /dev/null +++ b/docs/content/infrastructure/testbed_configuration/sm_zn2_hw_bios_cfg.md @@ -0,0 +1,614 @@ +--- +bookToc: true +title: "SuperMicro EPYC Zen2" +--- + +# SuperMicro EPYC Zen2 + +## Linux lscpu + +``` +$ lscpu +Architecture: x86_64 +CPU op-mode(s): 32-bit, 64-bit +Byte Order: Little Endian +CPU(s): 64 +On-line CPU(s) list: 0-63 +Thread(s) per core: 2 +Core(s) per socket: 32 +Socket(s): 1 +NUMA node(s): 2 +Vendor ID: AuthenticAMD +CPU family: 23 +Model: 49 +Model name: AMD EPYC 7532 32-Core Processor +Stepping: 0 +CPU MHz: 1981.470 +CPU max MHz: 2400.0000 +CPU min MHz: 1500.0000 +BogoMIPS: 4800.05 +Virtualization: AMD-V +L1d cache: 32K +L1i cache: 32K +L2 cache: 512K +L3 cache: 16384K +NUMA node0 CPU(s): 0-15,32-47 +NUMA node1 CPU(s): 16-31,48-63 +Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate sme ssbd ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif umip rdpid overflow_recov succor smca +``` + +``` +$ lscpu +Architecture: x86_64 +CPU op-mode(s): 32-bit, 64-bit +Byte Order: Little Endian +CPU(s): 64 +On-line CPU(s) list: 0-63 +Thread(s) per core: 2 +Core(s) per socket: 32 +Socket(s): 1 +NUMA node(s): 2 +Vendor ID: AuthenticAMD +CPU family: 23 +Model: 49 +Model name: AMD EPYC 7532 32-Core Processor +Stepping: 0 +CPU MHz: 1981.470 +CPU max MHz: 2400.0000 +CPU min MHz: 1500.0000 +BogoMIPS: 4800.05 +Virtualization: AMD-V +L1d cache: 32K +L1i cache: 32K +L2 cache: 512K +L3 cache: 16384K +NUMA node0 CPU(s): 0-15,32-47 +NUMA node1 CPU(s): 16-31,48-63 +Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate sme ssbd ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif umip rdpid overflow_recov succor smca +``` + +## Linux dmidecode + +``` + $ dmidecode -t slot + Handle 0x0026, DMI type 7, 27 bytes + Cache Information + Socket Designation: L1 Cache + Configuration: Enabled, Not Socketed, Level 1 + Operational Mode: Write Back + Location: Internal + Installed Size: 2048 kB + Maximum Size: 2048 kB + Supported SRAM Types: + Pipeline Burst + Installed SRAM Type: Pipeline Burst + Speed: 1 ns + Error Correction Type: Multi-bit ECC + System Type: Unified + Associativity: 8-way Set-associative + + Handle 0x0027, DMI type 7, 27 bytes + Cache Information + Socket Designation: L2 Cache + Configuration: Enabled, Not Socketed, Level 2 + Operational Mode: Write Back + Location: Internal + Installed Size: 16384 kB + Maximum Size: 16384 kB + Supported SRAM Types: + Pipeline Burst + Installed SRAM Type: Pipeline Burst + Speed: 1 ns + Error Correction Type: Multi-bit ECC + System Type: Unified + Associativity: 8-way Set-associative + + Handle 0x0028, DMI type 7, 27 bytes + Cache Information + Socket Designation: L3 Cache + Configuration: Enabled, Not Socketed, Level 3 + Operational Mode: Write Back + Location: Internal + Installed Size: 262144 kB + Maximum Size: 262144 kB + Supported SRAM Types: + Pipeline Burst + Installed SRAM Type: Pipeline Burst + Speed: 1 ns + Error Correction Type: Multi-bit ECC + System Type: Unified + Associativity: 16-way Set-associative + + Handle 0x0029, DMI type 4, 48 bytes + Processor Information + Socket Designation: CPU + Type: Central Processor + Family: Zen + Manufacturer: Advanced Micro Devices, Inc. + ID: 10 0F 83 00 FF FB 8B 17 + Signature: Family 23, Model 49, Stepping 0 + Flags: + FPU (Floating-point unit on-chip) + VME (Virtual mode extension) + DE (Debugging extension) + PSE (Page size extension) + TSC (Time stamp counter) + MSR (Model specific registers) + PAE (Physical address extension) + MCE (Machine check exception) + CX8 (CMPXCHG8 instruction supported) + APIC (On-chip APIC hardware supported) + SEP (Fast system call) + MTRR (Memory type range registers) + PGE (Page global enable) + MCA (Machine check architecture) + CMOV (Conditional move instruction supported) + PAT (Page attribute table) + PSE-36 (36-bit page size extension) + CLFSH (CLFLUSH instruction supported) + MMX (MMX technology supported) + FXSR (FXSAVE and FXSTOR instructions supported) + SSE (Streaming SIMD extensions) + SSE2 (Streaming SIMD extensions 2) + HTT (Multi-threading) + Version: AMD EPYC 7532 32-Core Processor + Voltage: 1.1 V + External Clock: 100 MHz + Max Speed: 3300 MHz + Current Speed: 2400 MHz + Status: Populated, Enabled + Upgrade: Socket SP3 + L1 Cache Handle: 0x0026 + L2 Cache Handle: 0x0027 + L3 Cache Handle: 0x0028 + Serial Number: Unknown + Asset Tag: Unknown + Part Number: Unknown + Core Count: 32 + Core Enabled: 32 + Thread Count: 64 + Characteristics: + 64-bit capable + Multi-Core + Hardware Thread + Execute Protection + Enhanced Virtualization + Power/Performance Control +``` + +## Linux dmidecode pci + +``` + $ dmidecode -t slot + Getting SMBIOS data from sysfs. + SMBIOS 3.2.0 present. + # SMBIOS implementations newer than version 3.1.1 are not + # fully supported by this version of dmidecode. + + Handle 0x000A, DMI type 9, 17 bytes + System Slot Information + Designation: M.2-HC1 CPU PCI-E 4.0 X4/X2 + Type: x4 PCI Express 3 x4 + Current Usage: Available + Length: Short + ID: 1 + Characteristics: + 3.3 V is provided + Opening is shared + PME signal is supported + Bus Address: 0000:ff:00.0 + + Handle 0x000B, DMI type 9, 17 bytes + System Slot Information + Designation: M.2-HC2 CPU PCI-E 4.0 X2 + Type: x2 PCI Express 3 x2 + Current Usage: Available + Length: Short + ID: 2 + Characteristics: + 3.3 V is provided + Opening is shared + PME signal is supported + Bus Address: 0000:ff:00.0 + + Handle 0x0042, DMI type 9, 17 bytes + System Slot Information + Designation: RSC-W-66G4 SLOT1 PCI-E 4.0 X16 + Type: x16 PCI Express 3 x16 + Current Usage: In Use + Length: Long + ID: 1 + Characteristics: + 3.3 V is provided + PME signal is supported + Bus Address: 0000:41:00.0 + + Handle 0x0043, DMI type 9, 17 bytes + System Slot Information + Designation: RSC-W-66G4 SLOT2 PCI-E 4.0 X16 + Type: x16 PCI Express 3 x16 + Current Usage: In Use + Length: Long + ID: 2 + Characteristics: + 3.3 V is provided + PME signal is supported + Bus Address: 0000:81:00.0 + + Handle 0x0045, DMI type 9, 17 bytes + System Slot Information + Designation: RSC-WR-6 SLOT1 PCI-E 4.0 X16 + Type: x16 PCI Express 3 x16 + Current Usage: In Use + Length: Long + ID: 1 + Characteristics: + 3.3 V is provided + PME signal is supported + Bus Address: 0000:01:00.0 + +``` + +## Linux dmidecode memory + +``` + $ dmidecode -t memory + # dmidecode 3.1 + Getting SMBIOS data from sysfs. + SMBIOS 3.2.0 present. + # SMBIOS implementations newer than version 3.1.1 are not + # fully supported by this version of dmidecode. + + Handle 0x0023, DMI type 16, 23 bytes + Physical Memory Array + Location: System Board Or Motherboard + Use: System Memory + Error Correction Type: Multi-bit ECC + Maximum Capacity: 2 TB + Error Information Handle: 0x0022 + Number Of Devices: 8 + + Handle 0x002B, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0023 + Error Information Handle: 0x002A + Total Width: 72 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: DIMMA1 + Bank Locator: P0_Node0_Channel0_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: SK Hynix + Serial Number: 431E9216 + Asset Tag: P1-DIMMA1_AssetTag (date:19/00) + Part Number: HMA84GR7CJR4N-XN + Rank: 2 + Configured Clock Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x002E, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0023 + Error Information Handle: 0x002D + Total Width: 72 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: DIMMB1 + Bank Locator: P0_Node0_Channel1_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: SK Hynix + Serial Number: 431E91D2 + Asset Tag: P1-DIMMB1_AssetTag (date:19/00) + Part Number: HMA84GR7CJR4N-XN + Rank: 2 + Configured Clock Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x0031, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0023 + Error Information Handle: 0x0030 + Total Width: 72 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: DIMMC1 + Bank Locator: P0_Node0_Channel2_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: SK Hynix + Serial Number: 431E918A + Asset Tag: P1-DIMMC1_AssetTag (date:19/00) + Part Number: HMA84GR7CJR4N-XN + Rank: 2 + Configured Clock Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x0034, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0023 + Error Information Handle: 0x0033 + Total Width: 72 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: DIMMD1 + Bank Locator: P0_Node0_Channel3_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: SK Hynix + Serial Number: 431E9187 + Asset Tag: P1-DIMMD1_AssetTag (date:19/00) + Part Number: HMA84GR7CJR4N-XN + Rank: 2 + Configured Clock Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x0037, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0023 + Error Information Handle: 0x0036 + Total Width: 72 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: DIMME1 + Bank Locator: P0_Node0_Channel4_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: SK Hynix + Serial Number: 431E9178 + Asset Tag: P1-DIMME1_AssetTag (date:19/00) + Part Number: HMA84GR7CJR4N-XN + Rank: 2 + Configured Clock Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x003A, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0023 + Error Information Handle: 0x0039 + Total Width: 72 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: DIMMF1 + Bank Locator: P0_Node0_Channel5_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: SK Hynix + Serial Number: 431E9206 + Asset Tag: P1-DIMMF1_AssetTag (date:19/00) + Part Number: HMA84GR7CJR4N-XN + Rank: 2 + Configured Clock Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x003D, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0023 + Error Information Handle: 0x003C + Total Width: 72 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: DIMMG1 + Bank Locator: P0_Node0_Channel6_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: SK Hynix + Serial Number: 431E9207 + Asset Tag: P1-DIMMG1_AssetTag (date:19/00) + Part Number: HMA84GR7CJR4N-XN + Rank: 2 + Configured Clock Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V + + Handle 0x0040, DMI type 17, 84 bytes + Memory Device + Array Handle: 0x0023 + Error Information Handle: 0x003F + Total Width: 72 bits + Data Width: 64 bits + Size: 32 GB + Form Factor: DIMM + Set: None + Locator: DIMMH1 + Bank Locator: P0_Node0_Channel7_Dimm0 + Type: DDR4 + Type Detail: Synchronous Registered (Buffered) + Speed: 3200 MT/s + Manufacturer: SK Hynix + Serial Number: 431E9209 + Asset Tag: P1-DIMMH1_AssetTag (date:19/00) + Part Number: HMA84GR7CJR4N-XN + Rank: 2 + Configured Clock Speed: 3200 MT/s + Minimum Voltage: 1.2 V + Maximum Voltage: 1.2 V + Configured Voltage: 1.2 V +``` + +## EPYC zn2 Server BIOS Configuration - TG + +### Boot Feature + +``` + | Quiet Boot [Enabled] |Boot option | + | | | + | Option ROM Messages [Force BIOS] | | + | Bootup NumLock State [On] | | + | Wait For "F1" If Error [Enabled] | | + | INT19 Trap Response [Immediate] | | + | Re-try Boot [Disabled] | | + | | | + | Power Configuration | | + | Watch Dog Function [Disabled] | | + | Restore on AC Power Loss [Last State] | | + | Power Button Function [Instant Off] | | +``` + +### CPU Configuration + +``` + | ACPI Settings ^| | + | -------------------------------------------------- *| | + | PCI AER Support [Disabled] *| | + | High Precision Event Timer [Disabled] *| | + | NUMA Nodes Per Socket [NPS2] *| | + | ACPI SRAT L3 Cache As NUMA Domain [Auto] *| | + | *| | + | CPU Configuration ^| | + | -------------------------------------------------- *| | + | SMT Control [Auto] *| | + | Core Performance Boost [Auto] *| | + | Global C-state control [Disabled] *| | + | Local APIC Mode [Auto] *| | + | CCD Control [Auto] *| | + | Core Control [Auto] *| | + | Core Control [Auto] *| | + | L1 Stream HW Prefetcher [Enabled] *| | + | L2 Stream HW Prefetcher [Enabled] *| | + | SVM Mode [Enabled] *| | + | SMEE [Disabled] *| | + | *| | + |> CPU1 Information *| | + | *| | + | NB Configuration ^| | + | -------------------------------------------------- *| | + | Determinism Control [Manual] *| | + | Determinism Slider [Performance] *| | + | cTDP Control [Disabled] *| | + | IOMMU [Disabled] *| | + | ACS Enable [Auto] *| | + | Package Power Limit Control [Auto] *| | + | APBDIS [1] *| | + | Fixed SOC Pstate [P0] *| | + | DF Cstates [Enabled] *| | + | Preferred IO [Manual] *| | + | Preferred IO Bus [##] *| | + | *| | + | *|-----------------------------| + | *|><: Select Screen | + | *|^v: Select Item | + | *|Enter: Select | + | +|+/-: Change Opt. | + | +|F1: General Help | + | +|F2: Previous Values | + | +|F3: Optimized Defaults | + | v|F4: Save & Exit | + | | | +``` + + +## EPYC zn2 Server BIOS Configuration - DUT + +### Boot Feature + +``` + | Quiet Boot [Enabled] |Boot option | + | | | + | Option ROM Messages [Force BIOS] | | + | Bootup NumLock State [On] | | + | Wait For "F1" If Error [Enabled] | | + | INT19 Trap Response [Immediate] | | + | Re-try Boot [Disabled] | | + | | | + | Power Configuration | | + | Watch Dog Function [Disabled] | | + | Restore on AC Power Loss [Last State] | | + | Power Button Function [Instant Off] | | +``` + +### CPU Configuration + +``` + | ACPI Settings ^| | + | -------------------------------------------------- *| | + | PCI AER Support [Disabled] *| | + | High Precision Event Timer [Disabled] *| | + | NUMA Nodes Per Socket [NPS2] *| | + | ACPI SRAT L3 Cache As NUMA Domain [Auto] *| | + | *| | + | CPU Configuration ^| | + | -------------------------------------------------- *| | + | SMT Control [Auto] *| | + | Core Performance Boost [Auto] *| | + | Global C-state control [Disabled] *| | + | Local APIC Mode [Auto] *| | + | CCD Control [Auto] *| | + | Core Control [Auto] *| | + | Core Control [Auto] *| | + | L1 Stream HW Prefetcher [Enabled] *| | + | L2 Stream HW Prefetcher [Enabled] *| | + | SVM Mode [Enabled] *| | + | SMEE [Disabled] *| | + | *| | + |> CPU1 Information *| | + | *| | + | NB Configuration ^| | + | -------------------------------------------------- *| | + | Determinism Control [Manual] *| | + | Determinism Slider [Performance] *| | + | cTDP Control [Disabled] *| | + | IOMMU [Disabled] *| | + | ACS Enable [Auto] *| | + | Package Power Limit Control [Auto] *| | + | APBDIS [1] *| | + | Fixed SOC Pstate [P0] *| | + | DF Cstates [Enabled] *| | + | Preferred IO [Manual] *| | + | Preferred IO Bus [##] *| | + | *| | + | *|-----------------------------| + | *|><: Select Screen | + | *|^v: Select Item | + | *|Enter: Select | + | +|+/-: Change Opt. | + | +|F1: General Help | + | +|F2: Previous Values | + | +|F3: Optimized Defaults | + | v|F4: Save & Exit | + | | | +``` + + +## EPYC zn2 Server Firmware Inventory + +``` +Host. IPMI IP. BMC. BIOS. CPLD. CPU Microcode. PCI Bus. X710 Firmware. XXV710 Firmware. i40e. MLX5 Firmware. mlx5_core +s60-t210-sut1. 10.30.55.24. 03.10.04. 1.1a. 02.c2.00. 0x8301038. ?. 8.70. 8.70. 2.19.3. 16.32.1010. 5.6-1.0.3.3. +s61-t210-tg1. 10.30.55.25. 03.10.04. 1.1a. 02.c2.00. 0x8301038. ?. 8.30. 8.30. 2.19.3. 16.32.1010. 5.6-1.0.3.3. +```
\ No newline at end of file diff --git a/docs/content/introduction/_index.md b/docs/content/introduction/_index.md new file mode 100644 index 0000000000..e028786bd1 --- /dev/null +++ b/docs/content/introduction/_index.md @@ -0,0 +1,5 @@ +--- +bookFlatSection: true +title: "Introduction" +weight: 1 +---
\ No newline at end of file diff --git a/docs/content/introduction/automating_vpp_api_flag_day.md b/docs/content/introduction/automating_vpp_api_flag_day.md new file mode 100644 index 0000000000..131adeab9d --- /dev/null +++ b/docs/content/introduction/automating_vpp_api_flag_day.md @@ -0,0 +1,303 @@ +--- +bookHidden: true +title: "VPP API Flag Day Algorithms" +--- + +# VPP API Flag Day Algorithm + +## Abstract + +This document describes the current solution to the problem of +automating the detection of VPP API changes which are not backwards +compatible with existing CSIT tests, by defining the "Flag Day" +process of deploying a new set of CSIT tests which are compatible +with the new version of the VPP API without causing a halt to the +normal VPP/CSIT operational CI process. This is initially +limited to changes in \*.api files contained in the vpp repo. +Eventually the detection algorithm could be extended to include +other integration points such as "directory" structure of stats +segment or PAPI python library dependencies. + +## Motivation + +Aside of per-release activities (release report), CSIT also provides testing +that requires somewhat tight coupling to the latest (merged but not released) +VPP code. Currently, HEAD of one project is run against somewhat older codebase +of the other project. Definition of what is the older codebase to use +is maintained by CSIT project. For older CSIT codebase, there are so-called +"oper" branches. For older VPP codebase, CSIT master HEAD contains identifiers +for "stable" VPP builds. Such older codebases are also used for verify jobs, +where HEAD of the other project is replaced by the commit under review. + +One particular type of jobs useful for VPP development is trending jobs. +They test latests VPP build with latest oper branch of CSIT, +and analytics is applied to detect regressions in preformance. +For this to work properly, VPP project needs a warning against breaking +the assumptions the current oper branch makes about VPP behavior. +In the past, the most frequent type of such breakage was API change. + +Earlier attempts to create a process to minimize breakage have focused +on creating a new verify job for VPP (called api-crc job) that +votes -1 on a change that affects CRC values for API messages CSIT uses. +The list of messages and CRC values (multiple "collections" are allowed) +is maintained in CSIT repository (in oper branch). +The process was less explicit on how should CSIT project maintain such list. +As CSIT was not willing to support two incpompatible API messages +by the same codebase (commit), there were unavoidable windows +where either trenging jobs, or CSIT verify jobs were failing. + +Practice showed that human (or infra) errors can create two kinds of breakages. +Either the unavoidable short window gets long, affecting a trending job run +or two, or the api-crc job starts giving -1 to innocent changes +because oper branch went out of sync with VPP HEAD codebase. +This second type of failure prevents any merges to VPP for a long time +(12 hours is the typical time, give time zone differences). + +The current version of this document introduces two new requirements. +Firstly, the api-crc job should not give false -1, under any +(reasonable) circumstances. That means, if a VPP change +(nor any of its unmerged ancestor commits) does not affect any CRC values +for messages used by CSIT, -1 should only mean "rebase is needed", +and rebasing to HEAD should result in +1 from the api-crc job. +Secondly, no more than one VPP change is allowed to be processed +(at the same time). + +## Naming + +It is easier to define the process after chosing shorter names +for notions that need long definition. + +Note: Everytime a single job is mentioned, +in practice it can be a set of jobs covering parts of functionality. +A "run" of the set of jobs passes only if each job within the set +has been run (again) and passed. + +## Jobs + ++ A *vpp verify* job: Any job run automatically, and voting on open VPP changes. + Some verify jobs compile and package VPP for target operating system + and processor architecture, the packages are NOT archived (currently). + They should be cached somewhere in future to speed up in downstream jobs, + but currently each such downstream job can clone and build. + ++ The *api-crc* job: Quick verify job for VPP changes, that accesses + CSIT repository (checkout latest oper branch HEAD) to figure out + whether merging the change is safe from CSIT point of view. + Here, -1 means CSIT is not ready. +1 means CSIT looks to be ready + for the new CRC values, but there still may be failures on real tests. + ++ A *trending* job: Any job that is started by timer and performs testing. + It checkouts CSIT latest oper branch HEAD, downloads the most recent + completely uploaded VPP package, and unconditionally runs the tests. + CRC checks are optional, ideally only written to console log + without otherwise affecting the test cases. + ++ A *vpp-csit* job: A slower verify job for VPP changes, that accesses CSIT + repository and runs tests from the correct CSIT commit (chosen as in trending) + against the VPP (built from the VPP patch under review). + Vote -1 means there were test failures. +1 means no test failures, meaning + there either was no API change, or it was backward compatible. + ++ A *csit-vpp* job: Verify job for open CSIT changes. Downloads the + (completely uploaded) VPP package marked as "stable", and runs a selection + of tests (from the CSIT patch under review). + Vote +1 means all tests have passed, so it is safe to merge + the patch under review. + ++ A *patch-on-patch* job: Manually triggered non-voting job + for open CSIT changes. Compiles and packages from VPP source + (usually of an unmerged change). Then runs the same tests as csit-vpp job. + This job is used to prove the CSIT patch under review is supporting + the specified VPP code. + In practice, this can be a vpp-csit job started with CSIT_REF set. + ++ A *manual verification* is done by a CSIT committer, locally executing steps + equivalent to the patch-on-patch job. This can to save time and resources. + +## CRC Collections + +Any commit in/for the CSIT repository contains a file (supported_crcs.yaml), +which contains either one or two collections. A collection is a mapping +that maps API message name to its CRC value. + +A collection name specifies which VPP build is this collection for. +An API message name is present in a collection if and only if +it is used by a test implementation (can be in different CSIT commit) +targeted at the VPP build (pointed out by the collection name). + ++ The *stable collection*: Usually required, listed first, has comments and name + pointing to the VPP build this CSIT commit marks as stable. + The stable collection is only missing in deactivating changes (see below) + when not mergeable yet. + ++ The *active collection*: Optional, listed second, has comments and name + pointing to the VPP Gerrit (including patch set number) + the currently active API process is processing. + The patch set number part can be behind the actual Gerrit state. + This is safe, because api-crc job on the active API change will fail + if the older patch is no longer API-equivalent to the newer patch. + +## Changes + ++ An *API change*: The name for any Gerrit Change for VPP repository + that does not pass api-crc job right away, and needs this whole process. + This usually means .api files are edited, but a patch that affects + the way CRC values are computed is also an API change. + + Full name could be VPP API Change, but as no CSIT change is named "API change" + (and this document does not talk about other FD.io or external projects), + "API change" is shorter. + ++ A *blocked change*: The name for open Gerrit Change for VPP repository + that got -1 from some of voting verify jobs. + ++ A *VPP-blocked change": A blocked change which got -1 from some "pure VPP" + verify job, meaning no CSIT code has been involved in the vote. + Example: "make test" fails. + + VPP contributor is expected to fix the change, or VPP developers + are expected to found a cause in an earlier VPP change, and fix it. + No interaction with CSIT developers is necessary. + ++ A *CSIT-blocked change*: A blocked change which is not VPP-blocked, + but does not pass some vpp-csit job. + To fix a CSIT-blocked change, an interaction with a CSIT committer + is usually necessary. Even if a VPP developer is experienced enough + to identify the cause of the failure, a merge to CSIT is usually needed + for a full fix. + + This process does not specify what to do with CSIT-blocked changes + that are not also API changes. + ++ A *candidate API change*: An API change that meets all requirements + to become active (see below). Currently, the requirements are: + + + No -1 nor -2 from from any human reviewer. + + + All verify jobs (except vpp-csit ones) pass. + + + +1 from a VPP committer. + + The reason is to avoid situations where an API change becomes active, + but the VPP committers are unwilling to merge it for some reason. + ++ The *active API change*: The candidate API change currently being processed + by the API Flag Day Algorithm. + While many API changes can be candidates at the same time, + only one is allowed be active at a time. + ++ The *activating change*: The name for a Gerrit Change for CSIT repository + that does not change the test code, but adds the active CRC collection. + Merge of the opening change (to latest CSIT oper branch) defines + which API change has become active. + ++ The *deactivating change*: The name for Gerrit Change for CSIT repository + that only supports tests and CRC values for VPP with the active API change. + That implies the previously stable CRC collection is deleted, + and any edits to the test implementation are done here. + ++ The *mergeable deactivating change*: The deactivating change with additional + requirements. Details on the requirements are listed in the next section. + Merging this change finishes the process for the active API change. + +It is possible for a single CSIT change to act both as a mergeable +deactivating change for one API change, and as an activating change +for another API change. As English lacks a good adjective for such a thing, +this document does not name this change. +When this documents says a change is activating or deactivating, +it allows the possibility for the change to fullfill also other purposes +(e.g. acting as deactivating / activating change for another API change). + +## Algorithm Steps + +The following steps describe the application of the API "Flag Day" algorithm: + +#. A VPP patch for an API change is submitted to + gerrit for review. +#. The api-crc job detects the API CRC values have changed + for some messages used by CSIT. +#. The api-crc job runs in parallel with any other vpp-csit verify job, + so those other jobs can hint at the impact on CSIT. + Currently, any such vpp-csit job is non-voting, + as the current process does not guarantee such jobs passes + when the API change is merged. +#. If the api-crc job fails, an email with the appropriate reason + is sent to the VPP patch submitter and vpp-api-dev@lists.fd.io + including the VPP patch information and .api files that are edited. +#. The VPP patch developer works with a VPP committer + to ensure the patch meets requirements to become a candidate (see above). +#. The VPP patch developer and CSIT team create a CSIT JIRA ticket + to identify the work required to support the new VPP API version. +#. CSIT developer creates a patch of the deactivating change + (upload to Gerrit not required yet). +#. CSIT developer runs patch-on-patch job (or manual verification). + Both developers iterate until the verification passes. + Note that in this phase csit-vpp job is expected to vote -1, + as the deactivating change is not mergeable yet. +#. CSIT developer creates the activating change, uploads to Gerrit, + waits for vote (usual review cycle applies). +#. When CSIT committer is satisfied, the activating change is merged + to CSIT master branch and cherry-picked to the latest oper branch. + This enters a "critical section" of the process. + Merges of other activating changes are not allowed from now on. + The targeted API change becomes the active API change. + This does not break any jobs. +#. VPP developer (or CSIT committer) issues a recheck on the VPP patch. +#. On failure, VPP and CSIT committers analyze what went wrong. + Typically, the active CRC collection is matching only an older patch set, + but a newer patch set needs different CRC values. + Either due to improvements on the VPP change in question, + or due to a rebase over previously merged (unrelated) API change. + VPP perhaps needs to rebase, and CSIT definitely needs + to merge edits to the active collection. Then issue a recheck again, + and iterate until success. +#. On success, VPP Committer merges the active API change patch. + (This is also a delayed verification of the current active CRC collection.) +#. VPP committer sends an e-mail to vpp-api-dev stating the support for + the previous CRC values will soon be removed, implying other changes + (whether API or not) should be rebased soon. +#. VPP merge jobs create and upload new VPP packages. + This breaks trending jobs, but both VPP and CSIT verify jobs still work. +#. CSIT developer makes the deactivating change mergeable: + The stable VPP build indicator is bumped to the build + that contains the active API change. The active CRC collection + (added by the activating change) is renamed to the new stable collection. + (The previous stable collection has already been deleted.) + At this time, the deactivating change should be uploaded to Gerrit and + csit verify jobs should be triggered. +#. CSIT committer reviews the code, perhaps triggering any additional jobs + needed to verify the tests using the edited APIs are still working. +#. When satisfied, CSIT committer merges the mergeable deactivating change + (to both master and oper). + The merge fixes trending jobs. VPP and CSIT verify jobs continue to work. + The merge also breaks some verify jobs for old changes in VPP, + as announced when the active API change was merged. + The merge is the point where the process leaves the "critical section", + thus allowing merges of activating changes for other API changes. +#. CSIT committer sends an e-mail to vpp-api-dev stating the support for + the previous CRC values has been removed, and rebase is needed + for all affected VPP changes. +#. Recheck of existing VPP patches in gerrit may cause the "VPP + API Incompatible Change Test" to send an email to the patch + submitter to rebase the patch to pick up the compatible VPP API + version files. + +### Real life examples + +Simple API change: https://gerrit.fd.io/r/c/vpp/+/23829 + +Activating change: https://gerrit.fd.io/r/c/csit/+/23956 + +Mergeable deactivating change: https://gerrit.fd.io/r/c/csit/+/24280 + +Less straightforward mergeable deactivating change: +https://gerrit.fd.io/r/c/csit/+/22526 +It shows: + ++ Crc edits: supported_crcs.yaml ++ Version bump: VPP_STABLE_VER_UBUNTU_BIONIC ++ And even a way to work around failing tests: + eth2p-ethicmpv4-ip4base-eth-1tap-dev.robot + +Simple change that is both deactivating and activating: +https://gerrit.fd.io/r/c/csit/+/23969 diff --git a/docs/content/introduction/bash_code_style.md b/docs/content/introduction/bash_code_style.md new file mode 100644 index 0000000000..1cc1464056 --- /dev/null +++ b/docs/content/introduction/bash_code_style.md @@ -0,0 +1,651 @@ +--- +bookHidden: true +title: "Bash Code Style" +--- + +The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", +"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", +"MAY", and "OPTIONAL" in this document are to be interpreted as +described in `BCP 14 <https://tools.ietf.org/html/bcp14>`_ +`[RFC2119] <https://tools.ietf.org/html/rfc2119>`_ +`[RFC8174] <https://tools.ietf.org/html/rfc8174>`_ +when, and only when, they appear in all capitals, as shown here. + +This document SHALL describe guidelines for writing reliable, maintainable, +reusable and readable code for CSIT. + +# Proposed Style + +# File Types + +Bash files SHOULD NOT be monolithic. Generally, this document +considers two types of bash files: + ++ Entry script: Assumed to be called by user, + or a script "external" in some way. + + + Sources bash libraries and calls functions defined there. + ++ Library file: To be sourced by entry scipts, possibly also by other libraries. + + + Sources other libraries for functions it needs. + + + Or relies on a related file already having sourced that. + + + Documentation SHALL imply which case it is. + + + Defines multiple functions other scripts can call. + +# Safety + ++ Variable expansions MUST be quoted, to prevent word splitting. + + + This includes special "variables" such as "${1}". + + + RECOMMENDED even if the value is safe, as in "$?" and "$#". + + + It is RECOMMENDED to quote strings in general, + so text editors can syntax-highlight them. + + + Even if the string is a numeric value. + + + Commands and known options can get their own highlight, no need to quote. + + + Example: You do not need to quote every word of + "pip install --upgrade virtualenv". + + + Code SHALL NOT quote glob characters you need to expand (obviously). + + + OPTIONALLY do not quote adjacent characters (such as dot or fore-slash), + so that syntax highlighting makes them stand out compared to surrounding + ordinary strings. + + + Example: cp "logs"/*."log" "."/ + + + Command substitution on right hand side of assignment are safe + without quotes. + + + Note that command substitution limits the scope for quotes, + so it is NOT REQUIRED to escape the quotes in deeper levels. + + + Both backtics and "dollar round-bracket" provide command substitution. + The folowing rules are RECOMMENDED: + + + For simple constructs, use "dollar round-bracket". + + + If there are round brackets in the surrounding text, use backticks, + as some editor highlighting logic can get confused. + + + Avoid nested command substitution. + + + Put intermediate results into local variables, + use "|| die" on each step of command substitution. + + + Code SHOULD NOT be structured in a way where + word splitting is intended. + + + Example: Variable holding string of multiple command lines arguments. + + + Solution: Array variable should be used in this case. + + + Expansion MUST use quotes then: "${name[@]}". + + + Word splitting MAY be used when creating arrays from command substitution. + ++ Code MUST always check the exit code of commands. + + + Traditionally, error code checking is done either by "set -e" + or by appending "|| die" after each command. + The first is unreliable, due to many rules affecting "set -e" behavior + (see <https://mywiki.wooledge.org/BashFAQ/105>), but "|| die" + relies on humans identifying each command, which is also unreliable. + When was the last time you checked error code of "echo" command, + for example? + + + Another example: "set -e" in your function has no effect + if any ancestor call is done with logical or, + for example in "func || code=$?" construct. + + + As there is no reliable method of error detection, and there are two + largely independent unreliable methods, the best what we can do + is to apply both. So, code SHOULD explicitly + check each command (with "|| die" and similar) AND have "set -e" applied. + + + Code MUST explicitly check each command, unless the command is well known, + and considered safe (such as the aforementioned "echo"). + + + The well known commands MUST still be checked implicitly via "set -e". + + + See below for specific "set -e" recommendations. + ++ Code SHOULD use "readlink -e" (or "-f" if target does not exist yet) + to normalize any path value to absolute path without symlinks. + It helps with debugging and identifies malformed paths. + ++ Code SHOULD use such normalized paths for sourcing. + ++ When exiting on a known error, code MUST print a longer, helpful message, + in order for the user to fix their situation if possible. + ++ When error happens at an unexpected place, it is RECOMMENDED for the message + to be short and generic, instead of speculative. + +# Bash Options + ++ Code MUST apply "-x" to make debugging easier. + + + Code MAY temporarily supress such output in order to avoid spam + (e.g. in long busy loops), but it is still NOT RECOMMENDED to do so. + ++ Code MUST apply "-e" for early error detection. + + + But code still SHOULD use "|| die" for most commands, + as "-e" has numerous rules and exceptions. + + + Code MAY apply "+e" temporarily for commands which (possibly nonzero) + exit code it interested in. + + + Code MUST to store "$?" and call "set -e" immediatelly afterwards. + + + Code MUST NOT use this approach when calling functions. + + + That is because functions are instructed to apply "set -e" on their own + which (when triggered) will exit the whole entry script. + + + Unless overriden by ERR trap. + But code SHOULD NOT set any ERR trap. + + + If code needs exit code of a function, it is RECOMMENDED to use + pattern 'code="0"; called_function || code="${?}"'. + + + In this case, contributor MUST make sure nothing in the + called_function sub-graph relies on "set -e" behavior, + because the call being part of "or construct" disables it. + + + Code MAY append "|| true" for benign commands, + when it is clear non-zero exit codes make no difference. + + + Also in this case, the contributor MUST make sure nothing within + the called sub-graph depends on "set -e", as it is disabled. + ++ Code MUST apply "-u" as unset variable is generally a typo, thus an error. + + + Code MAY temporarily apply "+u" if a command needs that to pass. + + + Virtualenv activation is the only known example so far. + ++ Code MUST apply "-o pipefail" to make sure "-e" picks errors + inside piped construct. + + + Code MAY use "|| true" inside a pipe construct, in the (inprobable) case + when non-zero exit code still results in a meaningful pipe output. + ++ All together: "set -exuo pipefail". + + + Code MUST put that line near start of every file, so we are sure + the options are applied no matter what. + + + "Near start" means "before any nontrivial code". + + + Basically only copyright is RECOMMENDED to appear before. + + + Also code MUST put the line near start of function bodies + and subshell invocations. + +# Functions + +There are (at least) two possibilities how a code from an external file +can be executed. Either the file contains a code block to execute +on each "source" invocation, or the file just defines functions +which have to be called separately. + +This document considers the "function way" to be better, +here are some pros and cons: + ++ Cons: + + + The function way takes more space. Files have more lines, + and the code in function body is one indent deeper. + + + It is not easy to create functions for low-level argument manipulation, + as "shift" command in the function code does not affect the caller context. + + + Call sites frequently refer to code two times, + when sourcing the definition and when executing the function. + + + It is not clear when a library can rely on its relative + to have performed the sourcing already. + + + Ideally, each library should detect if it has been sourced already + and return early, which takes even more space. + ++ Pros: + + + Some code blocks are more useful when used as function, + to make call site shorter. + + + Examples: Trap functions, "die" function. + + + The "import" part and "function" part usually have different side effects, + making the documentation more focused (even if longer overall). + + + There is zero risk of argument-less invocation picking arguments + from parent context. + + + This safety feature is the main reason for chosing the "function way". + + + This allows code blocks to support optional arguments. + ++ Rules: + + + Library files MUST be only "source"d. For example if "tox" calls a script, + it is an entry script. + + + Library files (upon sourcing) MUST minimize size effect. + + + The only permitted side effects MUST by directly related to: + + + Defining functions (without executing them). + + + Sourcing sub-library files. + + + If a bash script indirectly call another bash script, + it is not a "source" operation, variables are not shared, + so the called script MUST be considered an entry script, + even if it implements logic fitting into a single function. + + + Entry scripts SHOULD avoid duplicating any logic. + + + Clear duplicated blocks MUST be moved into libraries as functions. + + + Blocks with low amount of duplication MAY remain in entry scripts. + + + Usual motives for not creating functions are: + + + The extracted function would have too much logic for processing + arguments (instead of hardcoding values as in entry script). + + + The arguments needed would be too verbose. + + + And using "set +x" would take too much vertical space + (when compared to entry script implementation). + +# Variables + +This document describes two kinds of variables: called "local" and "global". + ++ Local variables: + + + Variable name MUST contain only lower case letters, digits and underscores. + + + Code MUST NOT export local variables. + + + Code MUST NOT rely on local variables set in different contexts. + + + Documentation is NOT REQUIRED. + + + Variable name SHOULD be descriptive enough. + + + Local variable MUST be initialized before first use. + + + Code SHOULD have a comment if a reader might have missed + the initialization. + + + Unset local variables when leaving the function. + + + Explicitly typeset by "local" builtin command. + + + Require strict naming convention, e.g. function_name__variable_name. + ++ Global variables: + + + Variable name MUST contain only upper case letters, digits and underscores. + + + They SHOULD NOT be exported, unless external commands need them + (e.g. PYTHONPATH). + + + Code MUST document if a function (or its inner call) + reads a global variable. + + + Code MUST document if a function (or its inner call) + sets or rewrites a global variable. + + + If a function "wants to return a value", it SHOULD be implemented + as the function setting (or rewriting) a global variable, + and the call sites reading that variable. + + + If a function "wants to accept an argument", it IS RECOMMENDED + to be implemented as the call sites setting or rewriting global variables, + and the function reading that variables. + But see below for direct arguments. + ++ Code MUST use curly brackets when referencing variables, + e.g. "${my_variable}". + + + It makes related constructs (such as ${name:-default}) less surprising. + + + It looks more similar to Robot Framework variables (which is good). + +# Arguments + +Bash scripts and functions MAY accept arguments, named "${1}", "${2}" and so on. +As a whole available via "$@". +You MAY use "shift" command to consume an argument. + +## Contexts + +Functions never have access to parent arguments, but they can read and write +variables set or read by parent contexts. + +### Arguments Or Variables + ++ Both arguments and global variables MAY act as an input. + ++ In general, if the caller is likely to supply the value already placed + in a global variable of known name, it is RECOMMENDED + to use that global variable. + ++ Construct "${NAME:-value}" can be used equally well for arguments, + so default values are possible for both input methods. + ++ Arguments are positional, so there are restrictions on which input + is optional. + ++ Functions SHOULD either look at arguments (possibly also + reading global variables to use as defaults), or look at variables only. + ++ Code MUST NOT rely on "${0}", it SHOULD use "${BASH_SOURCE[0]}" instead + (and apply "readlink -e") to get the current block location. + ++ For entry scripts, it is RECOMMENDED to use standard parsing capabilities. + + + For most Linux distros, "getopt" is RECOMMENDED. + +# Working Directory Handling + ++ Functions SHOULD act correctly without neither assuming + what the currect working directory is, nor changing it. + + + That is why global variables and arguments SHOULD contain + (normalized) full paths. + + + Motivation: Different call sites MAY rely on different working directories. + ++ A function MAY return (also with nonzero exit code) when working directory + is changed. + + + In this case the function documentation MUST clearly state where (and when) + is the working directory changed. + + + Exception: Functions with undocumented exit code. + + + Those functions MUST return nonzero code only on "set -e" or "die". + + + Note that both "set -e" and "die" by default result in exit of the whole + entry script, but the caller MAY have altered that behavior + (by registering ERR trap, or redefining die function). + + + Any callers which use "set +e" or "|| true" MUST make sure + their (and their caller ancestors') assumption on working directory + are not affected. + + + Such callers SHOULD do that by restoring the original working directory + either in their code, + + + or contributors SHOULD do such restoration in the function code, + (see below) if that is more convenient. + + + Motivation: Callers MAY rely on this side effect to simplify their logic. + ++ A function MAY assume a particular directory is already set + as the working directory (to save space). + + + In this case function documentation MUST clearly state what the assumed + working directory is. + + + Motivation: Callers MAY call several functions with common + directory of interest. + + + Example: Several dowload actions to execute in sequence, + implemented as functions assuming ${DOWNLOAD_DIR} + is the working directory. + ++ A function MAY change the working directory transiently, + before restoring it back before return. + + + Such functions SHOULD use command "pushd" to change the working directory. + + + Such functions SHOULD use "trap 'trap - RETURN; popd' RETURN" + imediately after the pushd. + + + In that case, the "trap - RETURN" part MUST be included, + to restore any trap set by ancestor. + + + Functions MAY call "trap - RETURN; popd" exlicitly. + + + Such functions MUST NOT call another pushd (before an explicit popd), + as traps do not stack within a function. + ++ If entry scripts also use traps to restore working directory (or other state), + they SHOULD use EXIT traps instead. + + + That is because "exit" command, as well as the default behavior + of "die" or "set -e" cause direct exit (without skipping function returns). + +# Function Size + ++ In general, code SHOULD follow reasoning similar to how pylint + limits code complexity. + ++ It is RECOMMENDED to have functions somewhat simpler than Python functions, + as Bash is generally more verbose and less readable. + ++ If code contains comments in order to partition a block + into sub-blocks, the sub-blocks SHOULD be moved into separate functions. + + + Unless the sub-blocks are essentially one-liners, + not readable just because external commands do not have + obvious enough parameters. Use common sense. + +# Documentation + ++ The library path and filename is visible from source sites. It SHOULD be + descriptive enough, so reader do not need to look inside to determine + how and why is the sourced file used. + + + If code would use several functions with similar names, + it is RECOMMENDED to create a (well-named) sub-library for them. + + + Code MAY create deep library trees if needed, it SHOULD store + common path prefixes into global variables to make sourcing easier. + + + Contributors, look at other files in the subdirectory. You SHOULD + improve their filenames when adding-removing other filenames. + + + Library files SHOULD NOT have executable flag set. + + + Library files SHOULD have an extension .sh (or perhaps .bash). + + + It is RECOMMENDED for entry scripts to also have executable flag unset + and have .sh extension. + ++ Each entry script MUST start with a shebang. + + + "#!/bin/usr/env bash" is RECOMMENDED. + + + Code SHOULD put an empty line after shebang. + + + Library files SHOULD NOT contain a shebang, as "source" is the primary + method to include them. + ++ Following that, there SHOULD be a block of comment lines with copyright. + + + It is a boilerplate, but human eyes are good at ignoring it. + + + Overhead for git is also negligible. + ++ Following that, there MUST be "set -exuo pipefail". + + + It acts as an anchor for humans to start paying attention. + +Then it depends on script type. + +## Library Documentation + ++ Following "set -exuo pipefail" SHALL come the "import part" documentation. + ++ Then SHALL be the import code + ("source" commands and a bare minimum they need). + ++ Then SHALL be the function definitions, and inside: + + + The body SHALL sart with the function documentation explaining API contract. + Similar to Robot [Documentation] or Python function-level docstring. + + + See below. + + + "set -exuo pipefail" SHALL be the first executable line + in the function body, except functions which legitimely need + different flags. Those SHALL also start with appropriate "set" command(s). + + + Lines containing code itself SHALL follow. + + + "Code itself" SHALL include comment lines + explaining any non-obvious logic. + + + There SHALL be two empty lines between function definitions. + +More details on function documentation: + +Generally, code SHOULD use comments to explain anything +not obvious from the funtion name. + ++ Function documentation SHOULD start with short description of function + operation or motivation, but only if not obvious from function name. + ++ Documentation SHOULD continue with listing any non-obvious side effect: + + + Documentation MUST list all read global variables. + + + Documentation SHOULD include descriptions of semantics + of global variable values. + It is RECOMMENDED to mention which function is supposed to set them. + + + The "include descriptions" part SHOULD apply to other items as well. + + + Documentation MUST list all global variables set, unset, reset, + or otherwise updated. + + + It is RECOMMENDED to list all hardcoded values used in code. + + + Not critical, but can hint at future improvements. + + + Documentation MUST list all files or directories read + (so caller can make sure their content is ready). + + + Documentation MUST list all files or directories updated + (created, deleted, emptied, otherwise edited). + + + Documentation SHOULD list all functions called (so reader can look them up). + + + Documentation SHOULD mention where are the functions defined, + if not in the current file. + + + Documentation SHOULD list all external commands executed. + + + Because their behavior can change "out of bounds", meaning + the contributor changing the implementation of the extrenal command + can be unaware of this particular function interested in its side effects. + + + Documentation SHOULD explain exit code (coming from + the last executed command). + + + Usually, most functions SHOULD be "pass or die", + but some callers MAY be interested in nonzero exit codes + without using global variables to store them. + + + Remember, "exit 1" ends not only the function, but all scripts + in the source chain, so code MUST NOT use it for other purposes. + + + Code SHOULD call "die" function instead. This way the caller can + redefine that function, if there is a good reason for not exiting + on function failure. + +## Entry Script Documentation + ++ After "set -exuo pipefail", high-level description SHALL come. + + + Entry scripts are rarely reused, so detailed side effects + are OPTIONAL to document. + + + But code SHOULD document the primary side effects. + ++ Then SHALL come few commented lines to import the library with "die" function. + ++ Then block of "source" commands for sourcing other libraries needed SHALL be. + + + In alphabetical order, any "special" library SHOULD be + in the previous block (for "die"). + ++ Then block os commands processing arguments SHOULD be (if needed). + ++ Then SHALL come block of function calls (with parameters as needed). + +# Other General Recommendations + ++ Code SHOULD NOT not repeat itself, even in documentation: + + + For hardcoded values, a general description SHOULD be written + (instead of copying the value), so when someone edits the value + in the code, the description still applies. + + + If affected directory name is taken from a global variable, + documentation MAY distribute the directory description + over the two items. + + + If most of side effects come from an inner call, + documentation MAY point the reader to the documentation + of the called function (instead of listing all the side effects). + ++ But documentation SHOULD repeat it if the information crosses functions. + + + Item description MUST NOT be skipped just because the reader + should have read parent/child documentation already. + + + Frequently it is RECOMMENDED to copy&paste item descriptions + between functions. + + + But sometimes it is RECOMMENDED to vary the descriptions. For example: + + + A global variable setter MAY document how does it figure out the value + (without caring about what it will be used for by other functions). + + + A global variable reader MAY document how does it use the value + (without caring about how has it been figured out by the setter). + ++ When possible, Bash code SHOULD be made to look like Python + (or Robot Framework). Those are three primary languages CSIT code relies on, + so it is nicer for the readers to see similar expressions when possible. + Examples: + + + Code MUST use indentation, 1 level is 4 spaces. + + + Code SHOULD use "if" instead of "&&" constructs. + + + For comparisons, code SHOULD use operators such as "!=" (needs "[["). + ++ Code MUST NOT use more than 80 characters per line. + + + If long external command invocations are needed, + code SHOULD use array variables to shorten them. + + + If long strings (or arrays) are needed, code SHOULD use "+=" operator + to grow the value over multiple lines. + + + If "|| die" does not fit with the command, code SHOULD use curly braces: + + + Current line has "|| {", + + + Next line has the die commands (indented one level deeper), + + + Final line closes with "}" at original intent level. diff --git a/docs/content/introduction/branches.md b/docs/content/introduction/branches.md new file mode 100644 index 0000000000..20759b9c78 --- /dev/null +++ b/docs/content/introduction/branches.md @@ -0,0 +1,192 @@ +--- +bookHidden: true +title: "Git Branches in CSIT" +--- + +# Git Branches in CSIT + +## Overview + +This document describes how to create and remove git branches in CSIT project. + +To be able to perform everything described in this file, you must be **logged +in as a committer**. + +## Operational Branches + +For more information about operational branches see +[CSIT/Branching Strategy](https://wiki.fd.io/view/CSIT/Branching_Strategy) and +[CSIT/Jobs](https://wiki.fd.io/view/CSIT/Jobs) on +[fd.io](https://fd.io) [wiki](https://wiki.fd.io/view/CSIT) pages. + +> Note: The branch `rls2009_lts` is used here only as an example. + +### Pre-requisites + +1. The last builds of weekly and semiweekly jobs must finish with status + *"Success"*. +1. If any of watched jobs failed, try to find the root cause, fix it and run it + again. + +The watched jobs are: + +- master: + - [csit-vpp-device-master-ubuntu1804-1n-skx-weekly](https://jenkins.fd.io/view/csit/job/csit-vpp-device-master-ubuntu1804-1n-skx-weekly) + - [csit-vpp-device-master-ubuntu1804-1n-skx-semiweekly](https://jenkins.fd.io/view/csit/job/csit-vpp-device-master-ubuntu1804-1n-skx-semiweekly) +- 2009_lts: + - [csit-vpp-device-2009_lts-ubuntu1804-1n-skx-weekly](https://jenkins.fd.io/view/csit/job/csit-vpp-device-2009_lts-ubuntu1804-1n-skx-weekly) + - [csit-vpp-device-2009_lts-ubuntu1804-1n-skx-semiweekly](https://jenkins.fd.io/view/csit/job/csit-vpp-device-2009_lts-ubuntu1804-1n-skx-semiweekly) + +### Procedure + +**A. CSIT Operational Branch** +1. Take the revision string from the last successful build of the **weekly** + job, e.g. **Revision**: 0f9b20775b4a656b67c7039e2dda4cf676af2b21. +1. Open [Gerrit](https://gerrit.fd.io). +1. Go to + [Browse --> Repositories --> csit --> Branches](https://gerrit.fd.io/r/admin/repos/csit,branches). +1. Click `CREATE NEW`. +1. Fill in the revision number and the name of the new operational branch. Its + format is: `oper-YYMMDD` for master and `oper-rls{RELEASE}-{YYMMDD}` or + `oper-rls{RELEASE}_lts-{YYMMDD}` for release branches. +1. Click "CREATE". +1. If needed, delete old operational branches by clicking "DELETE". + +**B. VPP Stable version** +1. Open the console log of the last successful **semiweekly** build and search + for VPP version (e.g. vpp_21 ...). +1. You should find the string with this structure: + `vpp_21.01-rc0~469-g7acab3790~b368_amd64.deb` +1. Modify [VPP_STABLE_VER_UBUNTU_BIONIC](../../VPP_STABLE_VER_UBUNTU_BIONIC) + and [VPP_STABLE_VER_CENTOS](../../VPP_STABLE_VER_CENTOS) files. +1. Use a string with the build number, e.g. `21.01-rc0~469_g7acab3790~b129` + for [VPP_STABLE_VER_CENTOS](../../VPP_STABLE_VER_CENTOS) and a string + without the build number, e.g. `21.01-rc0~469_g7acab3790` for + [VPP_STABLE_VER_UBUNTU_BIONIC](../../VPP_STABLE_VER_UBUNTU_BIONIC). +1. Update the stable versions in master and in all LTS branches. + +## Release Branches + +> Note: VPP release 21.01 is used here only as an example. + +### Pre-requisites + +1. VPP release manager sends the information email to announce that the RC1 + milestone for VPP {release}, e.g. 21.01, is complete, and the artifacts are + available. +1. The artifacts (*.deb and *.rpm) should be available at + `https://packagecloud.io/fdio/{release}`. For example see artifacts for the + [VPP release 20.01](https://packagecloud.io/fdio/2101). The last available + build is to be used. +1. All CSIT patches for the release are merged in CSIT master branch. + +### Procedure + +**A. Release branch** + +1. Open [Gerrit](https://gerrit.fd.io). +1. Go to + [Browse --> Repositories --> csit --> Branches](https://gerrit.fd.io/r/admin/repos/csit,branches). +1. Save the revision string of master for further use. +1. Click `CREATE NEW`. +1. Fill in the revision number and the name of the new release branch. Its + format is: `rlsYYMM`, e.g. rls2101. +1. Click "CREATE". + +**B. Jenkins jobs** + +See ["Add CSIT rls2101 branch"](https://gerrit.fd.io/r/c/ci-management/+/30439) +and ["Add report jobs to csit rls2101 branch"](https://gerrit.fd.io/r/c/ci-management/+/30462) +patches as an example. + +1. [csit.yaml](https://github.com/FDio/ci-management/blob/master/jjb/csit/csit.yaml): + Documentation of the source code and the Report + - Add release branch (rls2101) for `csit-docs-merge-{stream}` and + `csit-report-merge-{stream}` (project --> stream). +1. [csit-perf.yaml](https://github.com/FDio/ci-management/blob/master/jjb/csit/csit-perf.yaml): + Verify jobs + - Add release branch (rls2101) to `project --> jobs --> + csit-vpp-perf-verify-{stream}-{node-arch} --> stream`. + - Add release branch (rls2101) to `project --> project: 'csit' --> stream`. + - Add release branch (rls2101) to `project --> project: 'csit' --> stream_report`. +1. [csit-tox.yaml](https://github.com/FDio/ci-management/blob/master/jjb/csit/csit-tox.yaml): + tox + - Add release branch (rls2101) to `project --> stream`. +1. [csit-vpp-device.yaml](https://github.com/FDio/ci-management/blob/master/jjb/csit/csit-vpp-device.yaml): + csit-vpp-device + - Add release branch (rls2101) to `project --> jobs (weekly / semiweekly) --> stream`. + - Add release branch (rls2101) to `project --> project: 'csit' --> stream`. + +**C. VPP Stable version** + +See the patch +[Update of VPP_REPO_URL and VPP_STABLE_VER files](https://gerrit.fd.io/r/c/csit/+/30461) +and / or +[rls2101: Update VPP_STABLE_VER files to release version](https://gerrit.fd.io/r/c/csit/+/30976) +as an example. + +1. Find the last successful build on the + [Package Cloud](https://packagecloud.io) for the release, e.g. + [VPP release 20.01](https://packagecloud.io/fdio/2101). +1. Clone the release branch to your PC: + `git clone --depth 1 ssh://<user>@gerrit.fd.io:29418/csit --branch rls{RELEASE}` +1. Modify [VPP_STABLE_VER_UBUNTU_BIONIC](../../VPP_STABLE_VER_UBUNTU_BIONIC) + and [VPP_STABLE_VER_CENTOS](../../VPP_STABLE_VER_CENTOS) files with the last + successful build. +1. Modify [VPP_REPO_URL](../../VPP_REPO_URL) to point to the new release, e.g. + `https://packagecloud.io/install/repositories/fdio/2101`. +1. You can also modify the [.gitreview](../../.gitreview) file and set the new + default branch. +1. Wait until the verify jobs + - [csit-vpp-device-2101-ubuntu1804-1n-skx](https://jenkins.fd.io/job/csit-vpp-device-2101-ubuntu1804-1n-skx) + - [csit-vpp-device-2101-ubuntu1804-1n-tx2](https://jenkins.fd.io/job/csit-vpp-device-2101-ubuntu1804-1n-tx2) + + successfully finish and merge the patch. + +**D. CSIT Operational Branch** + +1. Manually start (Build with Parameters) the weekly job + [csit-vpp-device-2101-ubuntu1804-1n-skx-weekly](https://jenkins.fd.io/view/csit/job/csit-vpp-device-2101-ubuntu1804-1n-skx-weekly) +1. When it successfully finishes, take the revision string e.g. **Revision**: + 876b6c1ae05bfb1ad54ff253ea021f3b46780fd4 to create a new operational branch + for the new release. +1. Open [Gerrit](https://gerrit.fd.io). +1. Go to + [Browse --> Repositories --> csit --> Branches](https://gerrit.fd.io/r/admin/repos/csit,branches). +1. Click `CREATE NEW`. +1. Fill in the revision number and the name of the new operational branch. Its + format is: `oper-rls{RELEASE}-YYMMDD` e.g. `oper-rls2101-201217`. +1. Click "CREATE". +1. Manually start (Build with Parameters) the semiweekly job + [csit-vpp-device-2101-ubuntu1804-1n-skx-semiweekly](https://jenkins.fd.io/view/csit/job/csit-vpp-device-2101-ubuntu1804-1n-skx-semiweekly) +1. When it successfully finishes check in console log if it used the right VPP + version (search for `VPP_VERSION=`) from the right repository (search for + `REPO_URL=`). + +**E. Announcement** + +If everything is as it should be, send the announcement email to +`csit-dev@lists.fd.io` mailing list. + +*Example:* + +Subject: +```text +CSIT rls2101 branch pulled out +``` + +Body: +```text +CSIT rls2101 branch [0] is created and fully functional. + +Corresponding operational branch (oper-rls2101-201217) has been created too. + +We are starting dry runs for performance ndrpdr iterative tests to get initial +ndrpdr values with available rc1 packages as well as to test all the infra +before starting report data collection runs. + +Regards, +<signature> + +[0] https://git.fd.io/csit/log/?h=rls2101 +``` diff --git a/docs/content/introduction/dashboard_history.md b/docs/content/introduction/dashboard_history.md new file mode 100644 index 0000000000..f7f9db576a --- /dev/null +++ b/docs/content/introduction/dashboard_history.md @@ -0,0 +1,17 @@ +--- +title: "Dashboard History" +weight: 1 +--- + +# Dashboard History + +FD.io {{< release_csit >}} Dashboard History and per .[ww] revision changes are +listed below. + + **.[ww] Revision** | **Changes** +--------------------|------------------ + .10 | Initial revision + +FD.io CSIT Revision follow CSIT-[yy][mm].[ww] numbering format, with version +denoted by concatenation of two digit year [yy] and two digit month [mm], and +maintenance revision identified by two digit calendar week number [ww]. diff --git a/docs/content/introduction/design.md b/docs/content/introduction/design.md new file mode 100644 index 0000000000..e236b47d6f --- /dev/null +++ b/docs/content/introduction/design.md @@ -0,0 +1,148 @@ +--- +title: "Design" +weight: 3 +--- + +# Design + +FD.io CSIT system design needs to meet continuously expanding requirements of +FD.io projects including VPP, related sub-systems (e.g. plugin applications, +DPDK drivers) and FD.io applications (e.g. DPDK applications), as well as +growing number of compute platforms running those applications. With CSIT +project scope and charter including both FD.io continuous testing AND +performance trending/comparisons, those evolving requirements further amplify +the need for CSIT framework modularity, flexibility and usability. + +## Design Hierarchy + +CSIT follows a hierarchical system design with SUTs and DUTs at the bottom level +of the hierarchy, presentation level at the top level and a number of functional +layers in-between. The current CSIT system design including CSIT framework is +depicted in the figure below. + +{{< svg "static/csit_design_picture.svg" >}} + +A brief bottom-up description is provided here: + +1. SUTs, DUTs, TGs + - SUTs - Systems Under Test; + - DUTs - Devices Under Test; + - TGs - Traffic Generators; +2. Level-1 libraries - Robot and Python + - Lowest level CSIT libraries abstracting underlying test environment, SUT, + DUT and TG specifics; + - Used commonly across multiple L2 KWs; + - Performance and functional tests: + - L1 KWs (KeyWords) are implemented as RF libraries and Python + libraries; + - Performance TG L1 KWs: + - All L1 KWs are implemented as Python libraries: + - Support for TRex only today; + - CSIT IXIA drivers in progress; + - Performance data plane traffic profiles: + - TG-specific stream profiles provide full control of: + - Packet definition - layers, MACs, IPs, ports, combinations thereof + e.g. IPs and UDP ports; + - Stream definitions - different streams can run together, delayed, + one after each other; + - Stream profiles are independent of CSIT framework and can be used + in any T-rex setup, can be sent anywhere to repeat tests with + exactly the same setup; + - Easily extensible - one can create a new stream profile that meets + tests requirements; + - Same stream profile can be used for different tests with the same + traffic needs; + - Functional data plane traffic scripts: + - Scapy specific traffic scripts; +3. Level-2 libraries - Robot resource files: + - Higher level CSIT libraries abstracting required functions for executing + tests; + - L2 KWs are classified into the following functional categories: + - Configuration, test, verification, state report; + - Suite setup, suite teardown; + - Test setup, test teardown; +4. Tests - Robot: + - Test suites with test cases; + - Performance tests using physical testbed environment: + - VPP; + - DPDK-Testpmd; + - DPDK-L3Fwd; + - Tools: + - Documentation generator; + - Report generator; + - Testbed environment setup ansible playbooks; + - Operational debugging scripts; + +5. Test Lifecycle Abstraction + +A well coded test must follow a disciplined abstraction of the test +lifecycles that includes setup, configuration, test and verification. In +addition to improve test execution efficiency, the commmon aspects of +test setup and configuration shared across multiple test cases should be +done only once. Translating these high-level guidelines into the Robot +Framework one arrives to definition of a well coded RF tests for FD.io +CSIT. Anatomy of Good Tests for CSIT: + +1. Suite Setup - Suite startup Configuration common to all Test Cases in suite: + uses Configuration KWs, Verification KWs, StateReport KWs; +2. Test Setup - Test startup Configuration common to multiple Test Cases: uses + Configuration KWs, StateReport KWs; +3. Test Case - uses L2 KWs with RF Gherkin style: + - prefixed with {Given} - Verification of Test setup, reading state: uses + Configuration KWs, Verification KWs, StateReport KWs; + - prefixed with {When} - Test execution: Configuration KWs, Test KWs; + - prefixed with {Then} - Verification of Test execution, reading state: uses + Verification KWs, StateReport KWs; +4. Test Teardown - post Test teardown with Configuration cleanup and + Verification common to multiple Test Cases - uses: Configuration KWs, + Verification KWs, StateReport KWs; +5. Suite Teardown - Suite post-test Configuration cleanup: uses Configuration + KWs, Verification KWs, StateReport KWs; + +## RF Keywords Functional Classification + +CSIT RF KWs are classified into the functional categories matching the test +lifecycle events described earlier. All CSIT RF L2 and L1 KWs have been grouped +into the following functional categories: + +1. Configuration; +2. Test; +3. Verification; +4. StateReport; +5. SuiteSetup; +6. TestSetup; +7. SuiteTeardown; +8. TestTeardown; + +## RF Keywords Naming Guidelines + +Readability counts: "..code is read much more often than it is written." +Hence following a good and consistent grammar practice is important when +writing Robot Framework KeyWords and Tests. All CSIT test cases +are coded using Gherkin style and include only L2 KWs references. L2 KWs are +coded using simple style and include L2 KWs, L1 KWs, and L1 python references. +To improve readability, the proposal is to use the same grammar for both +Robot Framework KW styles, and to formalize the grammar of English +sentences used for naming the Robot Framework KWs. Robot +Framework KWs names are short sentences expressing functional description of +the command. They must follow English sentence grammar in one of the following +forms: + +1. **Imperative** - verb-object(s): *"Do something"*, verb in base form. +2. **Declarative** - subject-verb-object(s): *"Subject does something"*, verb in + a third-person singular present tense form. +3. **Affirmative** - modal_verb-verb-object(s): *"Subject should be something"*, + *"Object should exist"*, verb in base form. +4. **Negative** - modal_verb-Not-verb-object(s): *"Subject should not be + something"*, *"Object should not exist"*, verb in base form. + +Passive form MUST NOT be used. However a usage of past participle as an +adjective is okay. See usage examples provided in the Coding guidelines +section below. Following sections list applicability of the above +grammar forms to different Robot Framework KW categories. Usage +examples are provided, both good and bad. + +## Coding Guidelines + +Coding guidelines can be found on +[Design optimizations wiki page](https://wiki.fd.io/view/CSIT/Design_Optimizations).
\ No newline at end of file diff --git a/docs/content/introduction/model_schema.md b/docs/content/introduction/model_schema.md new file mode 100644 index 0000000000..ae3ba38fd7 --- /dev/null +++ b/docs/content/introduction/model_schema.md @@ -0,0 +1,60 @@ +--- +bookHidden: true +title: "Model Schema" +--- + +# Model Schema + +This document describes what is currently implemented in CSIT, +especially the export side (UTI), not import side (PAL). + +## Version + +This document is valid for CSIT model version 1.4.0. + +It is recommended to use semantic versioning: https://semver.org/ +That means, if the new model misses a field present in the old model, +bump the major version. If the new model adds a field +not present in the old model, bump the minor version. +Any other edit in the implmenetation (or documentation) bumps the patch version. +If you change value type or formatting, +consider whether the parser (PAL) understands the new value correctly. +Renaming a field is the same as adding a new one and removing the old one. +Parser (PAL) has to know exact major version and minimal minor version, +and unless bugs, it can ignore patch version and bumped minor version. + +## UTI + +UTI stands for Unified Test Interface. +It mainly focuses on exporting information gathered during test run +into JSON output files. + +### Output Structure + +UTI outputs come in filesystem tree structure (single tree), where directories +correspond to suite levels and files correspond to suite setup, suite teardown +or any test case at this level of suite. +The directory name comes from SUITE_NAME Robot variable (the last part +as the previous parts are higher level suites), converted to lowercase. +If the suite name contains spaces (Robot converts underscores to spaces), +they are replaced with underscores. + +The filesystem tree is rooted under tests/ (as suites in git are there), +and for each component (test case, suite setup, suite teardown). + +Although we expect only ASCII text in the exported files, +we manipulate files using UTF-8 encoding, +so if Robot Framework uses a non-ascii character, it will be handled. + +### JSON schemas + +CSIT model is formally defined as a collection of JSON schema documents, +one for each output file type. + +The current version specifies only one output file type: +Info output for test case. + +The authoritative JSON schema documents are in JSON format. +Git repository also contains YAML formatted document and conversion utility, +which simplifies maintaining of the JSON document +(no need to track brackets and commas), but are not authoritative. diff --git a/docs/content/introduction/perf_triggers_design.md b/docs/content/introduction/perf_triggers_design.md new file mode 100644 index 0000000000..445846f4d9 --- /dev/null +++ b/docs/content/introduction/perf_triggers_design.md @@ -0,0 +1,44 @@ +--- +bookHidden: true +title: "Performance Triggers Design" +--- + +# Performance Triggers Design + +*Syntax* + trigger_keyword [{tag1} {tag2}AND{tag3} !{tag4} !{tag5}] + +*Inputs* + - trigger_keyword for vpp-* jobs: 'perftest' + - trigger_keyword for csit-* jobs: 'csit-perftest' + - tags: existing CSIT tags [4]_ i.e. ip4base, ip6base, iacldst, memif + +Set of default tags appended to user input, under control by CSIT + - always-on for vpp-csit*.job: 'mrr' 'nic_intel_x710-da2' '1t1c' + - if input with no tags, following set applied: + - 'mrrANDnic_intel-x710AND1t1cAND64bANDip4base' + - 'mrrANDnic_intel-x710AND1t1cAND78bANDip6base' + - 'mrrANDnic_intel-x710AND1t1cAND64bANDl2bdbase' + +Examples + input: 'perftest' + expanded: 'mrrANDnic_intel_x710-da2AND1t1cAND64bANDl2bdbase mrrANDnic_intel_x710-da2AND1t1cAND64bANDip4base mrrANDnic_intel_x710-da2AND1t1cAND78bANDip6base' + input: 'perftest l2bdbase l2xcbase' + expanded: 'mrrANDnic_intel_x710-da2ANDl2bdbase mrrANDnic_intel_x710-da2ANDl2xcbase' + input: 'perftest ip4base !feature' + expanded: 'mrrANDnic_intel_x710-da2ANDip4base' not 'feature' + input: 'perftest ip4base !feature !lbond_dpdk' + expanded: 'mrrANDnic_intel_x710-da2ANDip4base' not 'feature' not 'lbond_dpdk' + input: 'perftestxyx ip4base !feature !lbond_dpdk' + invalid: detected as error + input: 'perftestip4base !feature !lbond_dpdk' + invalid: detected as error + input: 'perftest ip4base!feature!lbond_dpdk' + invalid expand: 'mrrANDnic_intel_x710-da2ANDip4base!feature!lbond_dpdk' + execution of RobotFramework will fail + +Constrains + Trigger keyword must be different for every job to avoid running multiple jobs + at once. Trigger keyword must not be substring of job name or any other + message printed by JJB bach to gerrit message which can lead to recursive + execution. diff --git a/docs/content/introduction/test_code_guidelines.md b/docs/content/introduction/test_code_guidelines.md new file mode 100644 index 0000000000..9707d63ea6 --- /dev/null +++ b/docs/content/introduction/test_code_guidelines.md @@ -0,0 +1,294 @@ +--- +bookHidden: true +title: "CSIT Test Code Guidelines" +--- + +# CSIT Test Code Guidelines + +The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", +"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", +"MAY", and "OPTIONAL" in this document are to be interpreted as +described in [BCP 14](https://tools.ietf.org/html/bcp14), +[RFC2119](https://tools.ietf.org/html/rfc2119), +[RFC8174](https://tools.ietf.org/html/rfc8174) +when, and only when, they appear in all capitals, as shown here. + +This document SHALL describe guidelines for writing reliable, maintainable, +reusable and readable code for CSIT. + +# RobotFramework test case files and resource files + ++ General + + + Contributors SHOULD look at requirements.txt in root CSIT directory + for the currently used Robot Framework version. + Contributors SHOULD read + [Robot Framework User Guide](http://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html) + for more details. + + + RobotFramework test case files and resource files + SHALL use special extension .robot + + + Pipe and space separated file format (without trailing pipe + and without pipe aligning) SHALL be used. + Tabs are invisible characters, which are error prone. + 4-spaces separation is prone to accidental double space + acting as a separator. + + + Files SHALL be encoded in UTF-8 (the default Robot source file encoding). + Usage of non-ASCII characters SHOULD be avoided if possible. + It is RECOMMENDED to + [escape](http://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#escaping) + non-ASCII characters. + + + Line length SHALL be limited to 80 characters. + + + There SHALL be licence text present at the beginning of each file. + + + Copy-pasting of the code NOT RECOMMENDED practice, any code that could be + re-used SHOULD be put into a library (Robot resource, Python library, ...). + ++ Test cases + + + It is RECOMMENDED to use data-driven test case definitions + anytime suite contains test cases similar in structure. + Typically, a suite SHOULD define a Template keyword, and test cases + SHOULD only specify tags and argument values + + *** Settings *** + | Test Template | Local Template + ... + + *** Test Cases *** + | tc01-64B-1c-eth-l2patch-mrr + | | [Tags] | 64B | 1C + | | framesize=${64} | phy_cores=${1} + + + Test case templates (or testcases) SHALL be written in Behavior-driven style + i.e. in readable English, so that even non-technical project stakeholders + can understand it + + *** Keywords *** + | Local Template + | | [Documentation] + | | ... | [Cfg] DUT runs L2 patch config with ${phy_cores} phy core(s). + | | ... | [Ver] Measure NDR and PDR values using MLRsearch algorithm.\ + | | ... + | | ... | *Arguments:* + | | ... | - frame_size - Framesize in Bytes in integer + | | ... | or string (IMIX_v4_1). Type: integer, string + | | ... | - phy_cores - Number of physical cores. Type: integer + | | ... | - rxq - Number of RX queues, default value: ${None}. + | | ... | Type: integer + | | ... + | | [Arguments] | ${frame_size} | ${phy_cores} | ${rxq}=${None} + | | ... + | | Set Test Variable | \${frame_size} + | | ... + | | Given Add worker threads and rxqueues to all DUTs + | | ... | ${phy_cores} | ${rxq} + | | And Add PCI devices to all DUTs + | | Set Max Rate And Jumbo And Handle Multi Seg + | | And Apply startup configuration on all VPP DUTs + | | When Initialize L2 patch + | | Then Find NDR and PDR intervals using optimized search + + + Every suite and test case template (or testcase) + SHALL contain short documentation. + Generated CSIT web pages display the documentation. + + + You SHOULD NOT use hard-coded constants. + It is RECOMMENDED to use the variable table + (\*\*\*Variables\*\*\*) to define test case specific values. + You SHALL use the assignment sign = after the variable name + to make assigning variables slightly more explicit + + *** Variables *** + | ${traffic_profile}= | trex-stl-2n-ethip4-ip4src254 + + + Common test case specific settings of the test environment SHALL be done + in Test Setup keyword defined in the Setting table. + + + Run Keywords construction is RECOMMENDED if it is more readable + than a keyword. + + + Separate keyword is RECOMMENDED if the construction is less readable. + + + Post-test cleaning and processing actions SHALL be done in Test Teardown + part of the Setting table (e.g. download statistics from VPP nodes). + This part is executed even if the test case has failed. On the other hand + it is possible to disable the tear-down from command line, thus leaving + the system in “broken” state for investigation. + + + Every testcase SHALL be correctly tagged. List of defined tags is in + csit/docs/introduction/test_tag_documentation.rst + + + Whenever possible, common tags SHALL be set using Force Tags + in Settings table. + + + User high-level keywords specific for the particular test suite + SHOULD be implemented in the Keywords table of suitable Robot resource file + to enable readability and code-reuse. + + + Such keywords MAY be implemented in Keywords table of the suite instead, + if the contributor believes no other test will use such keywords. + But this is NOT RECOMMENDED in general, as keywords in Resources + are easier to maintain. + + + All test case names (and suite names) SHALL conform + to current naming convention. + https://wiki.fd.io/view/CSIT/csit-test-naming + + + Frequently, different suites use the same test case layout. + It is RECOMMENDED to use autogeneration scripts available, + possibly extending them if their current functionality is not sufficient. + ++ Resource files + + + SHALL be used to implement higher-level keywords that are used in test cases + or other higher-level (or medium-level) keywords. + + + Every keyword SHALL contain Documentation where the purpose and arguments + of the keyword are described. Also document types, return values, + and any specific assumptions the particular keyword relies on. + + + A keyword usage example SHALL be the part of the Documentation. + The example SHALL use pipe and space separated format + (with escaped pipes and) with a trailing pipe. + + + The reason was possbile usage of Robot's libdoc tool + to generate tests and resources documentation. In that case + example keyword usage would be rendered in table. + + + Keyword name SHALL describe what the keyword does, + specifically and in a reasonable length (“short sentence”). + + + Keyword names SHALL be short enough for call sites + to fit within line length limit. + + + If a keyword argument has a most commonly used value, it is RECOMMENDED + to set it as default. This makes keyword code longer, + but suite code shorter, and readability (and maintainability) + of suites SHALL always more important. + + + If there is intermediate data (created by one keyword, to be used + by another keyword) of singleton semantics (it is clear that the test case + can have at most one instance of such data, even if the instance + is complex, for example ${nodes}), it is RECOMMENDED to store it + in test variables. You SHALL document test variables read or written + by a keyword. This makes the test template code less verbose. + As soon as the data instance is not unique, you SHALL pass it around + via arguments and return values explicitly (this makes lower level keywords + more reusable and less bug prone). + + + It is RECOMMENDED to pass arguments explicitly via [Arguments] line. + Setting test variables takes more space and is less explicit. + Using arguments embedded in keyword name makes them less visible, + and it makes it harder for the line containing the resulting long name + to fit into the maximum character limit, so you SHOULD NOT use them. + +# Python library files + ++ General + + + SHALL be used to implement low-level keywords that are called from + resource files (of higher-level keywords) or from test cases. + + + Higher-level keywords MAY be implemented in python library file too. + it is RECOMMENDED especially in the case that their implementation + in resource file would be too difficult or impossible, + e.g. complex data structures or functional programming. + + + Every keyword, Python module, class, method, enum SHALL contain + docstring with the short description and used input parameters + and possible return value(s) or raised exceptions. + + + The docstrings SHOULD conform to + [PEP 257](https://www.python.org/dev/peps/pep-0257/) + and other quality standards. + + + CSIT contributions SHALL use a specific formatting for documenting + arguments, return values and similar. + + + Keyword usage examples MAY be grouped and used + in the class/module documentation string, to provide better overview + of the usage and relationships between keywords. + + + Keyword name SHALL describe what the keyword does, + specifically and in a reasonable length (“short sentence”). + See https://wiki.fd.io/view/CSIT/csit-test-naming + + + Python implementation of a keyword is a function, + so its name in the python library should be lowercase_with_underscores. + Robot call sites should usename with first letter capitalized, and spaces. + ++ Coding + + + It is RECOMMENDED to use some standard development tool + (e.g. PyCharm Community Edition) and follow + [PEP-8](https://www.python.org/dev/peps/pep-0008/) recommendations. + + + All python code (not only Robot libraries) SHALL adhere to PEP-8 standard. + This is reported by CSIT Jenkins verify job. + + + Indentation: You SHALL NOT use tab for indents! + Indent is defined as four spaces. + + + Line length: SHALL be limited to 80 characters. + + + CSIT Python code assumes PYTHONPATH is set + to the root of cloned CSIT git repository, creating a tree of sub-packages. + You SHALL use that tree for importing, for example + + from resources.libraries.python.ssh import exec_cmd_no_error + + + Imports SHALL be grouped in the following order: + + 1. standard library imports, + 2. related third party imports, + 3. local application/library specific imports. + + You SHALL put a blank line between each group of imports. + + + You SHALL use two blank lines between top-level definitions, + one blank line between method definitions. + + + You SHALL NOT execute any active code on library import. + + + You SHALL NOT use global variables inside library files. + + + You MAY define constants inside library files. + + + It is NOT RECOMMENDED to use hard-coded constants (e.g. numbers, + paths without any description). It is RECOMMENDED to use + configuration file(s), like /csit/resources/libraries/python/Constants.py, + with appropriate comments. + + + The code SHALL log at the lowest possible level of implementation, + for debugging purposes. You SHALL use same style for similar events. + You SHALL keep logging as verbose as necessary. + + + You SHALL use the most appropriate exception not general one (Exception) + if possible. You SHOULD create your own exception + if necessary and implement there logging, level debug. + + + You MAY use RuntimeException for generally unexpected failures. + + + It is RECOMMENDED to use RuntimeError also for + infrastructure failures, e.g. losing SSH connection to SUT. + + + You MAY use EnvironmentError and its cublasses instead, + if the distinction is informative for callers. + + + It is RECOMMENDED to use AssertionError when SUT is at fault. + + + For each class (e.g. exception) it is RECOMMENDED to implement __repr__() + which SHALL return a string usable as a constructor call + (including repr()ed arguments). + When logging, you SHOULD log the repr form, unless the internal structure + of the object in question would likely result in too long output. + This is helpful for debugging. + + + For composing and formatting strings, you SHOULD use .format() + with named arguments. + Example: "repr() of name: {name!r}".format(name=name) diff --git a/docs/content/introduction/test_naming.md b/docs/content/introduction/test_naming.md new file mode 100644 index 0000000000..22e2c0bf8a --- /dev/null +++ b/docs/content/introduction/test_naming.md @@ -0,0 +1,107 @@ +--- +title: "Test Naming" +weight: 4 +--- + +# Test Naming + +## Background + +{{< release_csit >}} follows a common structured naming convention for all +performance and system functional tests, introduced in CSIT 17.01. + +The naming should be intuitive for majority of the tests. Complete +description of CSIT test naming convention is provided on +[CSIT test naming wiki page](https://wiki.fd.io/view/CSIT/csit-test-naming). +Below few illustrative examples of the naming usage for test suites across CSIT +performance, functional and Honeycomb management test areas. + +## Naming Convention + +The CSIT approach is to use tree naming convention and to encode following +testing information into test suite and test case names: + +1. packet network port configuration + * port type, physical or virtual; + * number of ports; + * NIC model, if applicable; + * port-NIC locality, if applicable; +2. packet encapsulations; +3. VPP packet processing + * packet forwarding mode; + * packet processing function(s); +4. packet forwarding path + * if present, network functions (processes, containers, VMs) and their + topology within the computer; +5. main measured variable, type of test. + +Proposed convention is to encode ports and NICs on the left (underlay), +followed by outer-most frame header, then other stacked headers up to the +header processed by vSwitch-VPP, then VPP forwarding function, then encap on +vhost interface, number of vhost interfaces, number of VMs. If chained VMs +present, they get added on the right. Test topology is expected to be +symmetric, in other words packets enter and leave SUT through ports specified +on the left of the test name. Here some examples to illustrate the convention +followed by the complete legend, and tables mapping the new test filenames to +old ones. + +## Naming Examples + +CSIT test suite naming examples (filename.robot) for common tested VPP +topologies: + +1. **Physical port to physical port - a.k.a. NIC-to-NIC, Phy-to-Phy, P2P** + * *PortNICConfig-WireEncapsulation-PacketForwardingFunction- + PacketProcessingFunction1-...-PacketProcessingFunctionN-TestType* + * *10ge2p1x520-dot1q-l2bdbasemaclrn-ndrdisc.robot* => 2 ports of 10GE on Intel + x520 NIC, dot1q tagged Ethernet, L2 bridge-domain baseline switching with + MAC learning, NDR throughput discovery. + * *10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrchk.robot* => 2 ports of 10GE on + Intel x520 NIC, IPv4 VXLAN Ethernet, L2 bridge-domain baseline switching + with MAC learning, NDR throughput discovery. + * *10ge2p1x520-ethip4-ip4base-ndrdisc.robot* => 2 ports of 10GE on Intel x520 + NIC, IPv4 baseline routed forwarding, NDR throughput discovery. + * *10ge2p1x520-ethip6-ip6scale200k-ndrdisc.robot* => 2 ports of 10GE on Intel + x520 NIC, IPv6 scaled up routed forwarding, NDR throughput discovery. + * *10ge2p1x520-ethip4-ip4base-iacldstbase-ndrdisc.robot* => 2 ports of 10GE on + Intel x520 NIC, IPv4 baseline routed forwarding, ingress Access Control + Lists baseline matching on destination, NDR throughput discovery. + * *40ge2p1vic1385-ethip4-ip4base-ndrdisc.robot* => 2 ports of 40GE on Cisco + vic1385 NIC, IPv4 baseline routed forwarding, NDR throughput discovery. + * *eth2p-ethip4-ip4base-func.robot* => 2 ports of Ethernet, IPv4 baseline + routed forwarding, functional tests. +2. **Physical port to VM (or VM chain) to physical port - a.k.a. NIC2VM2NIC, + P2V2P, NIC2VMchain2NIC, P2V2V2P** + * *PortNICConfig-WireEncapsulation-PacketForwardingFunction- + PacketProcessingFunction1-...-PacketProcessingFunctionN-VirtEncapsulation- + VirtPortConfig-VMconfig-TestType* + * *10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot* => 2 ports + of 10GE on Intel x520 NIC, dot1q tagged Ethernet, L2 bridge-domain switching + to/from two vhost interfaces and one VM, NDR throughput discovery. + * *10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrdisc.robot* => 2 + ports of 10GE on Intel x520 NIC, IPv4 VXLAN Ethernet, L2 bridge-domain + switching to/from two vhost interfaces and one VM, NDR throughput discovery. + * *10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-4vhost-2vm-ndrdisc.robot* => 2 + ports of 10GE on Intel x520 NIC, IPv4 VXLAN Ethernet, L2 bridge-domain + switching to/from four vhost interfaces and two VMs, NDR throughput + discovery. + * *eth2p-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-func.robot* => 2 ports of + Ethernet, IPv4 VXLAN Ethernet, L2 bridge-domain switching to/from two vhost + interfaces and one VM, functional tests. +3. **API CRUD tests - Create (Write), Read (Retrieve), Update (Modify), Delete + (Destroy) operations for configuration and operational data** + * *ManagementTestKeyword-ManagementOperation-ManagedFunction1-...- + ManagedFunctionN-ManagementAPI1-ManagementAPIN-TestType* + * *mgmt-cfg-lisp-apivat-func* => configuration of LISP with VAT API calls, + functional tests. + * *mgmt-cfg-l2bd-apihc-apivat-func* => configuration of L2 Bridge-Domain with + Honeycomb API and VAT API calls, functional tests. + * *mgmt-oper-int-apihcnc-func* => reading status and operational data of + interface with Honeycomb NetConf API calls, functional tests. + * *mgmt-cfg-int-tap-apihcnc-func* => configuration of tap interfaces with + Honeycomb NetConf API calls, functional tests. + * *mgmt-notif-int-subint-apihcnc-func* => notifications of interface and + sub-interface events with Honeycomb NetConf Notifications, functional tests. + +For complete description of CSIT test naming convention please refer to +[CSIT test naming wiki page](https://wiki.fd.io/view/CSIT/csit-test-naming>). diff --git a/docs/content/introduction/test_scenarios_overview.md b/docs/content/introduction/test_scenarios_overview.md new file mode 100644 index 0000000000..415ee3403f --- /dev/null +++ b/docs/content/introduction/test_scenarios_overview.md @@ -0,0 +1,61 @@ +--- +title: "Test Scenarios Overview" +weight: 2 +--- + +# Test Scenarios Overview + +FD.io CSIT Dashboard includes multiple test scenarios of VPP +centric applications, topologies and use cases. In addition it also +covers baseline tests of DPDK sample applications. Tests are executed in +physical (performance tests) and virtual environments (functional +tests). + +Brief overview of test scenarios covered in this documentation: + +1. **VPP Performance**: VPP performance tests are executed in physical + FD.io testbeds, focusing on VPP network data plane performance in + NIC-to-NIC switching topologies. VPP application runs in + bare-metal host user-mode handling NICs. TRex is used as a traffic generator. +2. **VPP Vhostuser Performance with KVM VMs**: VPP VM service switching + performance tests using vhostuser virtual interface for + interconnecting multiple NF-in-VM instances. VPP vswitch + instance runs in bare-metal user-mode handling NICs and connecting + over vhost-user interfaces to VM instances each running VPP with virtio + virtual interfaces. Similarly to VPP Performance, tests are run across a + range of configurations. TRex is used as a traffic generator. +3. **VPP Memif Performance with LXC and Docker Containers**: VPP + Container service switching performance tests using memif virtual + interface for interconnecting multiple VPP-in-container instances. + VPP vswitch instance runs in bare-metal user-mode handling NICs and + connecting over memif (Slave side) interfaces to more instances of + VPP running in LXC or in Docker Containers, both with memif + interfaces (Master side). Similarly to VPP Performance, tests are + run across a range of configurations. TRex is used as a traffic + generator. +4. **DPDK Performance**: VPP uses DPDK to drive the NICs and physical + interfaces. DPDK performance tests are used as a baseline to + profile performance of the DPDK sub-system. Two DPDK applications + are tested: Testpmd and L3fwd. DPDK tests are executed in the same + testing environment as VPP tests. DPDK Testpmd and L3fwd + applications run in host user-mode. TRex is used as a traffic + generator. +5. **T-Rex Performance**: T-Rex perfomance tests are executed in physical + FD.io testbeds, focusing on T-Rex data plane performance in NIC-to-NIC + loopback topologies. +6. **VPP Functional**: VPP functional tests are executed in virtual + FD.io testbeds, focusing on VPP packet processing functionality, + including both network data plane and in-line control plane. Tests + cover vNIC-to-vNIC vNIC-to-nestedVM-to-vNIC forwarding topologies. + Scapy is used as a traffic generator. + +All CSIT test data included in this report is auto-generated from Robot +Framework json output files produced by Linux Foundation FD.io Jenkins jobs +executed against {{< release_vpp >}} artifacts. + +FD.io CSIT system is developed using two main coding platforms: Robot +Framework and Python. {{< release_csit >}} source code for the executed test +suites is available in corresponding CSIT branch in the directory +`./tests/<name_of_the_test_suite>`. A local copy of CSIT source code +can be obtained by cloning CSIT git repository - `git clone +https://gerrit.fd.io/r/csit`. diff --git a/docs/content/introduction/test_tag_description.md b/docs/content/introduction/test_tag_description.md new file mode 100644 index 0000000000..630afa864e --- /dev/null +++ b/docs/content/introduction/test_tag_description.md @@ -0,0 +1,863 @@ +--- +title: "Test Tags Descriptions" +weight: 5 +--- + +# Test Tags Descriptions + +All CSIT test cases are labelled with Robot Framework tags used to allow for +easy test case type identification, test case grouping and selection for +execution. Following sections list currently used CSIT tags and their +descriptions. + +## Testbed Topology Tags + +**2_NODE_DOUBLE_LINK_TOPO** + + 2 nodes connected in a circular topology with two links interconnecting + the devices. + +**2_NODE_SINGLE_LINK_TOPO** + + 2 nodes connected in a circular topology with at least one link + interconnecting devices. + +**3_NODE_DOUBLE_LINK_TOPO** + + 3 nodes connected in a circular topology with two links interconnecting + the devices. + +**3_NODE_SINGLE_LINK_TOPO** + + 3 nodes connected in a circular topology with at least one link + interconnecting devices. + +## Objective Tags + +**SKIP_PATCH** + + Test case(s) marked to not run in case of vpp-csit-verify (i.e. VPP patch) + and csit-vpp-verify jobs (i.e. CSIT patch). + +**SKIP_VPP_PATCH** + + Test case(s) marked to not run in case of vpp-csit-verify (i.e. VPP patch). + +## Environment Tags + +**HW_ENV** + + DUTs and TGs are running on bare metal. + +**VM_ENV** + + DUTs and TGs are running in virtual environment. + +**VPP_VM_ENV** + + DUTs with VPP and capable of running Virtual Machine. + +## NIC Model Tags + +**NIC_Intel-X520-DA2** + + Intel X520-DA2 NIC. + +**NIC_Intel-XL710** + + Intel XL710 NIC. + +**NIC_Intel-X710** + + Intel X710 NIC. + +**NIC_Intel-XXV710** + + Intel XXV710 NIC. + +**NIC_Cisco-VIC-1227** + + VIC-1227 by Cisco. + +**NIC_Cisco-VIC-1385** + + VIC-1385 by Cisco. + +**NIC_Amazon-Nitro-50G** + + Amazon EC2 ENA NIC. + +## Scaling Tags + +**FIB_20K** + + 2x10,000 entries in single fib table + +**FIB_200K** + + 2x100,000 entries in single fib table + +**FIB_1M** + + 2x500,000 entries in single fib table + +**FIB_2M** + + 2x1,000,000 entries in single fib table + +**L2BD_1** + + Test with 1 L2 bridge domain. + +**L2BD_10** + + Test with 10 L2 bridge domains. + +**L2BD_100** + + Test with 100 L2 bridge domains. + +**L2BD_1K** + + Test with 1000 L2 bridge domains. + +**VLAN_1** + + Test with 1 VLAN sub-interface. + +**VLAN_10** + + Test with 10 VLAN sub-interfaces. + +**VLAN_100** + + Test with 100 VLAN sub-interfaces. + +**VLAN_1K** + + Test with 1000 VLAN sub-interfaces. + +**VXLAN_1** + + Test with 1 VXLAN tunnel. + +**VXLAN_10** + + Test with 10 VXLAN tunnels. + +**VXLAN_100* + + Test with 100 VXLAN tunnels. + +**VXLAN_1K** + + Test with 1000 VXLAN tunnels. + +**TNL_{t}** + + IPSec in tunnel mode - {t} tunnels. + +**SRC_USER_{u}** + + Traffic flow with {u} unique IPs (users) in one direction. + {u}=(1,10,100,1000,2000,4000). + +**100_FLOWS** + + Traffic stream with 100 unique flows (10 IPs/users x 10 UDP ports) in one + direction. + +**10k_FLOWS** + + Traffic stream with 10 000 unique flows (10 IPs/users x 1000 UDP ports) in + one direction. + +**100k_FLOWS** + + Traffic stream with 100 000 unique flows (100 IPs/users x 1000 UDP ports) in + one direction. + +**HOSTS_{h}** + + Stateless or stateful traffic stream with {h} client source IP4 addresses, + usually with 63 flow differing in source port number. Could be UDP or TCP. + If NAT is used, the clients are inside. Outside IP range can differ. + {h}=(1024,4096,16384,65536,262144). + +**GENEVE4_{t}TUN** + + Test with {t} GENEVE IPv4 tunnel. {t}=(1,4,16,64,256,1024) + +## Test Category Tags + +**DEVICETEST* + + All vpp_device functional test cases. + +**PERFTEST** + + All performance test cases. + +## VPP Device Type Tags + +**SCAPY** + + All test cases that uses Scapy for packet generation and validation. + +## erformance Type Tags + +**NDRPDR** + + Single test finding both No Drop Rate and Partial Drop Rate simultaneously. + The search is done by optimized algorithm which performs + multiple trial runs at different durations and transmit rates. + The results come from the final trials, which have duration of 30 seconds. + +**MRR** + + Performance tests where TG sends the traffic at maximum rate (line rate) + and reports total sent/received packets over trial duration. + The result is an average of 10 trials of 1 second duration. + +**SOAK** + + Performance tests using PLRsearch to find the critical load. + +**RECONF** + + Performance tests aimed to measure lost packets (time) when performing + reconfiguration while full throughput offered load is applied. + +## Ethernet Frame Size Tags + +These are describing the traffic offered by Traffic Generator, +"primary" traffic in case of asymmetric load. +For traffic between DUTs, or for "secondary" traffic, see ${overhead} value. + +**{b}B** + + {b} Bytes frames used for test. + +**IMIX** + + IMIX frame sequence (28x 64B, 16x 570B, 4x 1518B) used for test. + +## Test Type Tags + +**BASE** + + Baseline test cases, no encapsulation, no feature(s) configured in tests. + No scaling whatsoever, beyond minimum needed for RSS. + +**IP4BASE** + + IPv4 baseline test cases, no encapsulation, no feature(s) configured in + tests. Minimal number of routes. Other quantities may be scaled. + +**IP6BASE** + + IPv6 baseline test cases, no encapsulation, no feature(s) configured in + tests. + +**L2XCBASE** + + L2XC baseline test cases, no encapsulation, no feature(s) configured in + tests. + +**L2BDBASE** + + L2BD baseline test cases, no encapsulation, no feature(s) configured in + tests. + +**L2PATCH** + + L2PATCH baseline test cases, no encapsulation, no feature(s) configured in + tests. + +**SCALE** + + Scale test cases. Other tags specify which quantities are scaled. + Also applies if scaling is set on TG only (e.g. DUT works as IP4BASE). + +**ENCAP** + + Test cases where encapsulation is used. Use also encapsulation tag(s). + +**FEATURE** + + At least one feature is configured in test cases. Use also feature tag(s). + +**UDP** + + Tests which use any kind of UDP traffic (STL or ASTF profile). + +**TCP** + + Tests which use any kind of TCP traffic (STL or ASTF profile). + +**TREX** + + Tests which test trex traffic without any software DUTs in the traffic path. + +**UDP_UDIR** + + Tests which use unidirectional UDP traffic (STL profile only). + +**UDP_BIDIR** + + Tests which use bidirectional UDP traffic (STL profile only). + +**UDP_CPS** + + Tests which measure connections per second on minimal UDP pseudoconnections. + This implies ASTF traffic profile is used. + This tag selects specific output processing in PAL. + +**TCP_CPS** + + Tests which measure connections per second on empty TCP connections. + This implies ASTF traffic profile is used. + This tag selects specific output processing in PAL. + +**TCP_RPS** + + Tests which measure requests per second on empty TCP connections. + This implies ASTF traffic profile is used. + This tag selects specific output processing in PAL. + +**UDP_PPS** + + Tests which measure packets per second on lightweight UDP transactions. + This implies ASTF traffic profile is used. + This tag selects specific output processing in PAL. + +**TCP_PPS** + + Tests which measure packets per second on lightweight TCP transactions. + This implies ASTF traffic profile is used. + This tag selects specific output processing in PAL. + +**HTTP** + + Tests which use traffic formed of valid HTTP requests (and responses). + +**LDP_NGINX** + + LDP NGINX is un-modified NGINX with VPP via LD_PRELOAD. + +**NF_DENSITY** + + Performance tests that measure throughput of multiple VNF and CNF + service topologies at different service densities. + +## NF Service Density Tags + +**CHAIN** + + NF service density tests with VNF or CNF service chain topology(ies). + +**PIPE** + + NF service density tests with CNF service pipeline topology(ies). + +**NF_L3FWDIP4** + + NF service density tests with DPDK l3fwd IPv4 routing as NF workload. + +**NF_VPPIP4** + + NF service density tests with VPP IPv4 routing as NF workload. + +**{r}R{c}C** + + Service density matrix locator {r}R{c}C, {r}Row denoting number of + service instances, {c}Column denoting number of NFs per service + instance. {r}=(1,2,4,6,8,10), {c}=(1,2,4,6,8,10). + +**{n}VM{t}T** + + Service density {n}VM{t}T, {n}Number of NF Qemu VMs, {t}Number of threads + per NF. + +**{n}DCRt}T** + + Service density {n}DCR{t}T, {n}Number of NF Docker containers, {t}Number of + threads per NF. + +**{n}_ADDED_CHAINS** + + {n}Number of chains (or pipelines) added (and/or removed) + during RECONF test. + +## Forwarding Mode Tags + +**L2BDMACSTAT** + + VPP L2 bridge-domain, L2 MAC static. + +**L2BDMACLRN** + + VPP L2 bridge-domain, L2 MAC learning. + +**L2XCFWD** + + VPP L2 point-to-point cross-connect. + +**IP4FWD** + + VPP IPv4 routed forwarding. + +**IP6FWD** + + VPP IPv6 routed forwarding. + +**LOADBALANCER_MAGLEV** + + VPP Load balancer maglev mode. + +**LOADBALANCER_L3DSR** + + VPP Load balancer l3dsr mode. + +**LOADBALANCER_NAT4** + + VPP Load balancer nat4 mode. + +**N2N** + + Mode, where NICs from the same physical server are directly + connected with a cable. + +## Underlay Tags + +**IP4UNRLAY** + + IPv4 underlay. + +**IP6UNRLAY** + + IPv6 underlay. + +**MPLSUNRLAY** + + MPLS underlay. + +## Overlay Tags + +**L2OVRLAY** + + L2 overlay. + +**IP4OVRLAY** + + IPv4 overlay (IPv4 payload). + +**IP6OVRLAY** + + IPv6 overlay (IPv6 payload). + +## Tagging Tags + +**DOT1Q** + + All test cases with dot1q. + +**DOT1AD** + + All test cases with dot1ad. + +## Encapsulation Tags + +**ETH** + + All test cases with base Ethernet (no encapsulation). + +**LISP** + + All test cases with LISP. + +**LISPGPE** + + All test cases with LISP-GPE. + +**LISP_IP4o4** + + All test cases with LISP_IP4o4. + +**LISPGPE_IP4o4** + + All test cases with LISPGPE_IP4o4. + +**LISPGPE_IP6o4** + + All test cases with LISPGPE_IP6o4. + +**LISPGPE_IP4o6** + + All test cases with LISPGPE_IP4o6. + +**LISPGPE_IP6o6** + + All test cases with LISPGPE_IP6o6. + +**VXLAN** + + All test cases with Vxlan. + +**VXLANGPE** + + All test cases with VXLAN-GPE. + +**GRE** + + All test cases with GRE. + +**GTPU** + + All test cases with GTPU. + +**GTPU_HWACCEL** + + All test cases with GTPU_HWACCEL. + +**IPSEC** + + All test cases with IPSEC. + +**WIREGUARD** + + All test cases with WIREGUARD. + +**SRv6** + + All test cases with Segment routing over IPv6 dataplane. + +**SRv6_1SID** + + All SRv6 test cases with single SID. + +**SRv6_2SID_DECAP** + + All SRv6 test cases with two SIDs and with decapsulation. + +**SRv6_2SID_NODECAP** + + All SRv6 test cases with two SIDs and without decapsulation. + +**GENEVE** + + All test cases with GENEVE. + +**GENEVE_L3MODE** + + All test cases with GENEVE tunnel in L3 mode. + +**FLOW** + + All test cases with FLOW. + +**FLOW_DIR** + + All test cases with FLOW_DIR. + +**FLOW_RSS** + + All test cases with FLOW_RSS. + +**NTUPLE** + + All test cases with NTUPLE. + +**L2TPV3** + + All test cases with L2TPV3. + +## Interface Tags + +**PHY** + + All test cases which use physical interface(s). + +**GSO** + + All test cases which uses Generic Segmentation Offload. + +**VHOST** + + All test cases which uses VHOST. + +**VHOST_1024** + + All test cases which uses VHOST DPDK driver with qemu queue size set + to 1024. + +**VIRTIO** + + All test cases which uses VIRTIO native VPP driver. + +**VIRTIO_1024** + + All test cases which uses VIRTIO native VPP driver with qemu queue size set + to 1024. + +**CFS_OPT** + + All test cases which uses VM with optimised scheduler policy. + +**TUNTAP* + + All test cases which uses TUN and TAP. + +**AFPKT** + + All test cases which uses AFPKT. + +**NETMAP** + + All test cases which uses Netmap. + +**MEMIF** + + All test cases which uses Memif. + +**SINGLE_MEMIF** + + All test cases which uses only single Memif connection per DUT. One DUT + instance is running in container having one physical interface exposed to + container. + +**LBOND** + + All test cases which uses link bonding (BondEthernet interface). + +**LBOND_DPDK** + + All test cases which uses DPDK link bonding. + +**LBOND_VPP** + + All test cases which uses VPP link bonding. + +**LBOND_MODE_XOR** + + All test cases which uses link bonding with mode XOR. + +**LBOND_MODE_LACP** + + All test cases which uses link bonding with mode LACP. + +**LBOND_LB_L34** + + All test cases which uses link bonding with load-balance mode l34. + +**LBOND_{n}L** + + All test cases which use {n} link(s) for link bonding. + +**DRV_{d}** + + All test cases which NIC Driver for DUT is set to {d}. Default is VFIO_PCI. + {d}=(AVF, RDMA_CORE, VFIO_PCI, AF_XDP). + +**TG_DRV_{d}** + + All test cases which NIC Driver for TG is set to {d}. Default is IGB_UIO. + {d}=(RDMA_CORE, IGB_UIO). + +**RXQ_SIZE_{n}** + + All test cases which RXQ size (RX descriptors) are set to {n}. Default is 0, + which means VPP (API) default. + +**TXQ_SIZE_{n}** + + All test cases which TXQ size (TX descriptors) are set to {n}. Default is 0, + which means VPP (API) default. + +## Feature Tags + +**IACLDST** + + iACL destination. + +**ADLALWLIST** + + ADL allowlist. + +**NAT44** + + NAT44 configured and tested. + +**NAT64** + + NAT44 configured and tested. + +**ACL** + + ACL plugin configured and tested. + +**IACL** + + ACL plugin configured and tested on input path. + +**OACL** + + ACL plugin configured and tested on output path. + +**ACL_STATELESS** + + ACL plugin configured and tested in stateless mode (permit action). + +**ACL_STATEFUL** + + ACL plugin configured and tested in stateful mode (permit+reflect action). + +**ACL1** + + ACL plugin configured and tested with 1 not-hitting ACE. + +**ACL10** + + ACL plugin configured and tested with 10 not-hitting ACEs. + +**ACL50** + + ACL plugin configured and tested with 50 not-hitting ACEs. + +**SRv6_PROXY** + + SRv6 endpoint to SR-unaware appliance via proxy. + +**SRv6_PROXY_STAT** + + SRv6 endpoint to SR-unaware appliance via static proxy. + +**SRv6_PROXY_DYN** + + SRv6 endpoint to SR-unaware appliance via dynamic proxy. + +**SRv6_PROXY_MASQ** + + SRv6 endpoint to SR-unaware appliance via masquerading proxy. + +## Encryption Tags + +**IPSECSW** + + Crypto in software. + +**IPSECHW** + + Crypto in hardware. + +**IPSECTRAN** + + IPSec in transport mode. + +**IPSECTUN** + + IPSec in tunnel mode. + +**IPSECINT** + + IPSec in interface mode. + +**AES** + + IPSec using AES algorithms. + +**AES_128_CBC** + + IPSec using AES 128 CBC algorithms. + +**AES_128_GCM** + + IPSec using AES 128 GCM algorithms. + +**AES_256_GCM** + + IPSec using AES 256 GCM algorithms. + +**HMAC** + + IPSec using HMAC integrity algorithms. + +**HMAC_SHA_256** + + IPSec using HMAC SHA 256 integrity algorithms. + +**HMAC_SHA_512** + + IPSec using HMAC SHA 512 integrity algorithms. + +**SCHEDULER** + + IPSec using crypto sw scheduler engine. + +**FASTPATH** + + IPSec policy mode with spd fast path enabled. + +## Client-Workload Tags + +**VM** + + All test cases which use at least one virtual machine. + +**LXC** + + All test cases which use Linux container and LXC utils. + +**DRC** + + All test cases which use at least one Docker container. + +**DOCKER** + + All test cases which use Docker as container manager. + +**APP** + + All test cases with specific APP use. + +## Container Orchestration Tags + +**{n}VSWITCH** + + {n} VPP running in {n} Docker container(s) acting as a VSWITCH. + {n}=(1). + +**{n}VNF** + + {n} VPP running in {n} Docker container(s) acting as a VNF work load. + {n}=(1). + +## Multi-Threading Tags + +**STHREAD** + + *Dynamic tag*. + All test cases using single poll mode thread. + +**MTHREAD** + + *Dynamic tag*. + All test cases using more then one poll mode driver thread. + +**{n}NUMA** + + All test cases with packet processing on {n} socket(s). {n}=(1,2). + +**{c}C** + + {c} worker thread pinned to {c} dedicated physical core; or if + HyperThreading is enabled, {c}*2 worker threads each pinned to a separate + logical core within 1 dedicated physical core. Main thread pinned to core 1. + {t}=(1,2,4). + +**{t}T{c}C** + + *Dynamic tag*. + {t} worker threads pinned to {c} dedicated physical cores. Main thread + pinned to core 1. By default CSIT is configuring same amount of receive + queues per interface as worker threads. {t}=(1,2,4,8), {t}=(1,2,4). diff --git a/docs/content/introduction/testing_in_vagrant.md b/docs/content/introduction/testing_in_vagrant.md new file mode 100644 index 0000000000..ef1e231393 --- /dev/null +++ b/docs/content/introduction/testing_in_vagrant.md @@ -0,0 +1,85 @@ +--- +bookHidden: true +title: "Running CSIT locally in Vagrant" +--- + +# Running CSIT locally in Vagrant + +## Install prerequisites + +Run all commands from command line. + +1. Download and install virtualbox from + [official page](https://www.virtualbox.org/wiki/Downloads). + To verify the installation, run VBoxManage + + - on windows + + "C:\Program Files\Oracle\VirtualBox\VBoxManage.exe" --version + + - on nix + + VBoxManage --version + Tested version: 6.1.16r140961 + +2. Download and install latest vagrant + [from official page](https://www.vagrantup.com/downloads.html). + To verify the installtion, run + + vagrant -v + Tested version: Vagrant 2.2.15 + +3. Install vagrant plugins:: + + vagrant plugin install vagrant-vbguest + vagrant plugin install vagrant-cachier + + If you are behind a proxy, install proxyconf plugin and update proxy + settings in Vagrantfile:: + + vagrant plugin install vagrant-proxyconf + +## Set up and run Vagrant virtualbox + +Before running following commands change working directory to Vagrant specific directory +(from within root CSIT directory) + + cd csit.infra.vagrant + +This allows Vagrant to automatically find Vagrantfile and corresponding Vagrant environment. + +Start the provisioning + + vagrant up --provider virtualbox + +Your new VPP Device virtualbox machine will be created and configured. +Master branch of csit project will be cloned inside virtual machine into +/home/vagrant/csit folder. + +Once the process is finished, you can login to the box using + + vagrant ssh + +In case you need to completely rebuild the box and start from scratch, +run these commands + + vagrant destroy -f + vagrant up --provider virtualbox + +## Run tests + +From within the box run the tests using + + cd /home/vagrant/csit/resources/libraries/bash/entry + ./bootstrap_vpp_device.sh csit-vpp-device-master-ubuntu2004-1n-vbox + +To run only selected tests based on TAGS, export environment variables before +running the test suite + + export GERRIT_EVENT_TYPE="comment-added" + export GERRIT_EVENT_COMMENT_TEXT="devicetest memif" + + # now it will run tests, selected based on tags + ./bootstrap_vpp_device.sh csit-vpp-device-master-ubuntu2004-1n-vbox + + diff --git a/docs/content/methodology/_index.md b/docs/content/methodology/_index.md new file mode 100644 index 0000000000..0959bf089a --- /dev/null +++ b/docs/content/methodology/_index.md @@ -0,0 +1,5 @@ +--- +bookFlatSection: true +title: "Methodology" +weight: 2 +---
\ No newline at end of file diff --git a/docs/content/release_notes/_index.md b/docs/content/release_notes/_index.md new file mode 100644 index 0000000000..c08254e068 --- /dev/null +++ b/docs/content/release_notes/_index.md @@ -0,0 +1,5 @@ +--- +bookFlatSection: true +title: "Release notes" +weight: 2 +---
\ No newline at end of file diff --git a/docs/content/release_notes/dpdk.md b/docs/content/release_notes/dpdk.md new file mode 100644 index 0000000000..facefe4b23 --- /dev/null +++ b/docs/content/release_notes/dpdk.md @@ -0,0 +1,31 @@ +--- +title: "DPDK Performance" +weight: 2 +--- + +# Changes in {{< release_csit >}} + +1. TEST FRAMEWORK + - **CSIT test environment** version has been updated to ver. 11, see + [Environment Versioning]({{< ref "infrastructure#Release Notes" >}}). +2. DPDK PERFORMANCE TESTS + - No updates +3. DPDK RELEASE VERSION CHANGE + - {{< release_csit >}} tested {{< release_dpdk >}}, as used by + {{< release_vpp >}}. + +# Known Issues + +List of known issues in {{< release_csit >}} for DPDK performance tests: + + **#** | **JiraID** | **Issue Description** +-------|--------------------------------------------------|--------------------------------------------------------------------------- + 1 | [CSIT-1848](https://jira.fd.io/browse/CSIT-1848) | 2n-clx, 3n-alt: sporadic testpmd/l3fwd tests fail with no or low traffic. + + +## New + +List of new issues in {{< release_csit >}} for DPDK performance tests: + + **#** | **JiraID** | **Issue Description** +-------|--------------------------------------------------|---------------------------------------------------------------------------
\ No newline at end of file diff --git a/docs/content/release_notes/trex.md b/docs/content/release_notes/trex.md new file mode 100644 index 0000000000..3794dc159c --- /dev/null +++ b/docs/content/release_notes/trex.md @@ -0,0 +1,26 @@ +--- +title: "TRex Performance" +weight: 3 +--- + +# Changes in {{< release_csit >}} + +1. TEST FRAMEWORK + - **CSIT test environment** version has been updated to ver. 11, see + [Environment Versioning]({{< ref "infrastructure#Release Notes" >}}). + +# Known Issues + +List of known issues in {{< release_csit >}} for TRex performance tests + + **#** | **JiraID** | **Issue Description** +-------|--------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------- + 1 | [CSIT-1876](https://jira.fd.io/browse/CSIT-1876) | 1n-aws: TRex NDR PDR ALL IP4 scale and L2 scale tests failing with 50% packet loss. CSIT removed ip4scale and l2scale except ip4scale2m where it's still failing. + + +## New + +List of new issues in {{< release_csit >}} for TRex performance tests: + + **#** | **JiraID** | **Issue Description** +-------|--------------------------------------------------|---------------------------------------------------------------------------
\ No newline at end of file diff --git a/docs/content/release_notes/vpp.md b/docs/content/release_notes/vpp.md new file mode 100644 index 0000000000..48805ba574 --- /dev/null +++ b/docs/content/release_notes/vpp.md @@ -0,0 +1,95 @@ +--- +title: "VPP Performance" +weight: 1 +--- + +# Changes in {{< release_csit >}} + +1. VPP PERFORMANCE TESTS + - **Enhanced and added VPP hoststack tests** to daily and weekly + trending including: Quic VPP Echo, UDP+TCP LD_PRELOAD iPerf3, + LD_PRELOAD NGINX. + - **Added Nvidia/Mellanox DPDK tests** to daily and weekly trending + and report, in addition to RDMA_CORE ones that were already + there. + - **Jumbo frames tests** got fixed and re-added number of to report + coverage tests. + - **Intel Xeon SKX performance testbeds** got decommissioned and + removed from FD.io performance lab. +2. TEST FRAMEWORK + - **CSIT test environment** version has not changed from ver. 11 used + in previous release, see + [Environment Versioning]({{< ref "infrastructure#Release Notes" >}}). + - **CSIT PAPI optimizations for scale** got applied improving PAPI + programming speed especially for large scale tests. VAT has been + now completely deprecated from CSIT. + - **General Code Housekeeping**: Ongoing code optimizations and bug + fixes. +3. PRESENTATION AND ANALYTICS LAYER + - [Performance dashboard](https://csit.fd.io/) got updated with + addition of VPP telemetry trending across all VPP tests. A number + of code and AWS resource usage optimizations got applied to the + data processing pipeline and UI frontend and backend. + - Examples of release iterative data visualisation: + + - [Packet throughput 2n-icx-e810cq-ip4-base-scale-pdr](https://csit.fd.io/report/#eNrdVcluwjAQ_Zr0ggbZDml64QDkP5BxhhJlwYxNVPr1OAhpYiGO7cEHb3pv1qeRnT8T7h1266zYZuU2U2VThy3LN4twUOdULhSM1oLKl-FG2KF2CGqAxvyAFOIblZX4JYW5gB6P0NgVfK4OIA2gP02vsA6Tja1pcq12T9cvcRitr57RED1CRiQGo7SYZk-3GeddsszXhJoNQsYMeXSzZOKamHUk3aNrfpGpoQuMm9BohqSJ_fubnaHPRpXVg_F3qjijO1RCtEBDnZo8UXFJ6NQmKlGbgjp9ujPU_8cEFdXHcKb-8Q8V1R2PI8PX) + - [Speedup Multi-Core throughput graph for 2n-icx-e810cq-ip4-base-pdr](https://csit.fd.io/report/#eNrtlM8OgjAMxp8GL6aGFRAvHlTew8xRhAR1bpOoT-8wJIUYEg8mXjjsX35fu65fMusuhvaW6nWQbIN0G2Ba5X4Kos3cL6a2GIUIjdaA0cLvDNUkLQGeoVJ3EGF4JNSCViJUV5BNAZWOYRkfQCggV7YnPw5tjM5Nmxp3XeqPe5jmN8fU3z4gDRmGg7JYpstHTzNWLOulIckBvmJGjmyvmOGbWFUYeSJbPYmlvgvMlW80I6GG-d1D92jXqDR7K37qCk6ujLuC_3IlnlwZdyX-0pUkm50v5vT-yZLsBXP6Swk>) + - [MRR, NDR and PDR comparison for 2n-icx-e810cq-ip4-base](https://csit.fd.io/report/#eNrtVMsOgjAQ_Bq8mDW0gHjxoPIfppZVSQDrthLx6y2GuBBj4kVPHvrKzG6nM0mtOxFuLZbLIFkH6TqQaZH7KYhWU79QaWUUSmiMARnN_I6wRGURZA2FvoIIwwNKI3AhQn0G1eyhMDHM4x0IDeiO3cmPXVdTEXWt5aZv_XIPo_nFMepvHyENEoMjWUwzx3bAeSeW-YpQcYFXzJBDOxAzfhOz9qQqtMUNmepdYFx7oxkSetzftWaA9kal2YPx5VTq_J_KR6n0Rv0mFfNP5bNUzDOVJJvUJ6oeP1mS3QG2H0sT>) + - [Normalized throughput architecture comparison for 2n-[icx|clx]-e810cq-ip4-base-pdr](https://csit.fd.io/report/#eNrVk00OgjAQhU-DGzOGFhA3LlTuYUoZhKRibSsRT28hJANRF-500b98rzOvM6l1F4NHi2obJPsg3Qc8rQs_BdFu6RejLI9CDq3WwKOV3xlUKCwCb0CqO7AwPCHXDDcslFcQbQm1jmEd58AkoKv6kx95f0cXpg_ND2PolzxEi5sj6rPPSIuG4MwWyXTVTTSfzJJeGBR0wTsm5NBOzMzfRKrSiDPa-oEk9VUgLn2hCTE5j-86PaFjodJsUHzXlVr-UVfem_35riTZormY8_BneNpvhRpzJNkT6FzkMw>) + - [NICs comparison for 2n-icx-ip4-base-pdr](https://csit.fd.io/report/#eNrll99ugyAUh5_G3SxnESx1N7to53s0FI6rmbYMnKF7-qFrcmRmV7vReuG__A74wSckuvZi8eCwfknEPsn3Cc8rHU5JtnsMF1s7nqUcOmOAZ0_hzmKN0iHwM6jaA0vTN-SGKS_EVkJTewGV2cB2cwSmANtT_xSOY9_IaNv3zV9vfU9eRKn-bCkNr4-SDi2FEReVmdN1VPMnLTWQFiW1CMgUtehGNPGgqKq0skFXfSGVhmmgXIWppoipuP_2akbpbabyYqj4txerG7kcLz3tnXvBZ5aqD5BduQAtBLsOK9ro9-Vo6Wnv1sswUJ-zdPZLJSJdgY_ZL5IY9U6NcPEzTN8NX14JXpsZW_mNewi46zAz691rwroKJzPfwaaws7ciiofzxTbDv6QovgETwNPp>) + +# Known Issues + +Editing Note: below listed known issues need to be updated to reflect the +current state as tracked on +[CSIT TestFailuresTracking wiki](https://wiki.fd.io/view/CSIT/TestFailuresTracking). + +## New + + **#** | **JiraID** | **Issue Description** +-------|--------------------------------------------------|------------------------------------------------------ + 1 | [CSIT-1890](https://jira.fd.io/browse/CSIT-1890) | 3n-alt: Tests failing until 40Ge Interface comes up. + + +## Previous + +Issues reported in previous releases which still affect the current results. + + **#** | **JiraID** | **Issue Description** +-------|-------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + 1 | [CSIT-1782](https://jira.fd.io/browse/CSIT-1782) | Multicore AVF tests are failing when trying to create interface. Frequency is reduced by CSIT workaround, but occasional failures do still happen. + 2 | [CSIT-1785](https://jira.fd.io/browse/CSIT-1785) [VPP-1972](https://jira.fd.io/browse/VPP-1972) | NAT44ED tests failing to establish all TCP sessions. At least for max scale, in allotted time (limited by session 500s timeout) due to worse slow path performance than previously measured and calibrated for. CSIT removed the max scale NAT tests to avoid this issue. + 3 | [CSIT-1799](https://jira.fd.io/browse/CSIT-1799) | All NAT44-ED 16M sessions CPS scale tests fail while setting NAT44 address range. + 4 | [CSIT-1800](https://jira.fd.io/browse/CSIT-1800) | All Geneve L3 mode scale tests (1024 tunnels) are failing. + 5 | [CSIT-1801](https://jira.fd.io/browse/CSIT-1801) | 9000B payload frames not forwarded over tunnels due to violating supported Max Frame Size (VxLAN, LISP, + 6 | [CSIT-1802](https://jira.fd.io/browse/CSIT-1802) | all testbeds: AF-XDP - NDR tests failing from time to time. + 7 | [CSIT-1804](https://jira.fd.io/browse/CSIT-1804) | All testbeds: NDR tests failing from time to time. + 8 | [CSIT-1808](https://jira.fd.io/browse/CSIT-1808) | All tests with 9000B payload frames not forwarded over memif interfaces. + 9 | [CSIT-1827](https://jira.fd.io/browse/CSIT-1827) | 3n-icx, 3n-skx: all AVF crypto tests sporadically fail. 1518B with no traffic, IMIX with excessive + 10 | [CSIT-1835](https://jira.fd.io/browse/CSIT-1835) | 3n-icx: QUIC vppecho BPS tests failing on timeout when checking hoststack finished. + 11 | [CSIT-1849](https://jira.fd.io/browse/CSIT-1849) | 2n-skx, 2n-clx, 2n-icx: UDP 16m TPUT tests fail to create all sessions. + 12 | [CSIT-1864](https://jira.fd.io/browse/CSIT-1864) | 2n-clx: half of the packets lost on PDR tests. + 13 | [CSIT-1877](https://jira.fd.io/browse/CSIT-1877) | 3n-tsh: all VM tests failing to boot VM. + 14 | [CSIT-1883](https://jira.fd.io/browse/CSIT-1883) | 3n-snr: All hwasync wireguard tests failing when trying to verify device. + 15 | [CSIT-1884](https://jira.fd.io/browse/CSIT-1884) | 2n-clx, 2n-icx: All NAT44DET NDR PDR IMIX over 1M sessions BIDIR tests failing to create enough sessions. + 16 | [CSIT-1885](https://jira.fd.io/browse/CSIT-1885) | 3n-icx: 9000b ip4 ip6 l2 NDRPDR AVF tests are failing to forward traffic. + 17 | [CSIT-1886](https://jira.fd.io/browse/CSIT-1886) | 3n-icx: Wireguard tests with 100 and more tunnels are failing PDR criteria. + +## Fixed + +Issues reported in previous releases which were fixed in this release: + + **#** | **JiraID** | **Issue Description** +-------|--------------------------------------------------|--------------------------------------------------------------------- + 1 | [CSIT-1868](https://jira.fd.io/browse/CSIT-1868) | 2n-clx: ALL ldpreload-nginx tests fails when trying to start nginx. + 2 | [CSIT-1871](https://jira.fd.io/browse/CSIT-1871) | 3n-snr: 25GE interface between SUT and TG/TRex goes down randomly. + +# Root Cause Analysis for Performance Changes + +List of RCAs in {{< release_csit >}} for VPP performance changes: + + **#** | **JiraID** | **Issue Description** +-------|--------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------- + 1 | [CSIT-1887](https://jira.fd.io/browse/CSIT-1887) | rls2210 RCA: ASTF tests TRex upgrade decreased TRex performance. NAT results not affected, except on Denverton due to interference from VPP-2010. + 2 | [CSIT-1888](https://jira.fd.io/browse/CSIT-1888) | rls2210 RCA: testbed differences, especially for ipsec. Not caused by VPP code nor CSIT code. Most probable cause is clang-14 behavior. + 3 | [CSIT-1889](https://jira.fd.io/browse/CSIT-1889) | rls2210 RCA: policy-outbound-nocrypto. When VPP added spd fast path matching (Gerrit 36097), it decreased MRR of the corresponding tests, at least on 3-alt. diff --git a/docs/content/release_notes/vpp_device.md b/docs/content/release_notes/vpp_device.md new file mode 100644 index 0000000000..2f1f6d34b5 --- /dev/null +++ b/docs/content/release_notes/vpp_device.md @@ -0,0 +1,24 @@ +--- +title: "VPP Device" +weight: 4 +--- + +# Changes in {{< release_csit >}} + +1. TEST FRAMEWORK + - **CSIT test environment** version has been updated to ver. 11, see + [Environment Versioning]({{< ref "infrastructure#Release Notes" >}}). + +# Known Issues + +List of known issues in {{< release_csit >}} for VPP functional tests in VPP Device: + + **#** | **JiraID** | **Issue Description** +-------|--------------------------------------------------|--------------------------------------------------------------------------- + +## New + +List of new issues in {{< release_csit >}} for VPP functional tests in VPP Device: + + **#** | **JiraID** | **Issue Description** +-------|--------------------------------------------------|---------------------------------------------------------------------------
\ No newline at end of file |