aboutsummaryrefslogtreecommitdiffstats
path: root/docs/lab/testbed_specifications.md
diff options
context:
space:
mode:
Diffstat (limited to 'docs/lab/testbed_specifications.md')
-rw-r--r--docs/lab/testbed_specifications.md153
1 files changed, 145 insertions, 8 deletions
diff --git a/docs/lab/testbed_specifications.md b/docs/lab/testbed_specifications.md
index 6aa6bfc72a..e8ea4011d0 100644
--- a/docs/lab/testbed_specifications.md
+++ b/docs/lab/testbed_specifications.md
@@ -16,6 +16,8 @@
1. [3-Node-Altra Arm Ampere (3n-alt)](#3-node-altra-arm-armpere-3n-alt)
1. [3-Node-Icelake Xeon Intel (3n-icx)](#3-node-icelake-xeon-intel-3n-icx)
1. [3-Node-SnowRidge Atom Intel (3n-snr)](#3-node-snowridge-atom-intel-3n-snr)
+ 1. [2-Node-Full-SapphireRapids Xeon Intel (2nf-spr)](#2-node-full-sapphirerapids-xeon-intel-2nf-spr)
+ 1. [2-Node-SapphireRapids Xeon Intel (2n-spr)](#2-Node-SapphireRapids-Xeon-Intel-2n-spr)
1. [Server Management](#server-management)
1. [Requirements](#requirements)
1. [Addressing](#addressing)
@@ -39,6 +41,8 @@
1. [3-Node-Altra Servers (3n-alt) PROD](#3-node-altra-servers-3n-alt-prod)
1. [3-Node-Icelake Servers (3n-icx) PROD](#3-node-icelake-servers-3n-icx-prod)
1. [3-Node-SnowRidge Servers (3n-snr) PROD](#3-node-snowridge-servers-3n-snr-prod)
+ 1. ...
+ 1. ...
1. [Per Testbed Wiring](#per-testbed-wiring)
1. [1-Node-Skylake Wiring (1n-skx) PROD](#1-node-skylake-wiring-1n-skx-prod)
1. [1-Node-ThunderX2 Wiring (1n-tx2) PROD](#1-node-thunderx2-wiring-1n-tx2-prod)
@@ -53,6 +57,8 @@
1. [3-Node-Altra Wiring (3n-alt) PROD](#3-node-altra-wiring-3n-alt-prod)
1. [3-Node-Icelake Wiring (3n-icx) PROD](#3-node-icelake-wiring-3n-icx-prod)
1. [3-Node-SnowRidge Wiring (3n-snr) PROD](#3-node-snowridge-wiring-3n-snr-prod)
+ 1. ...
+ 1. ...
1. [Inventory](#inventory)
1. [Appliances](#appliances)
1. [Arm Servers](#arm-servers)
@@ -177,6 +183,17 @@ TG (Server-Type-F4) connected in a 3-node circular topology
([Server Types](#server-types)).
Used for FD.io performance tests.
+### 2-Node-Full-SapphireRapids Xeon Intel (2nf-spr)
+
+One 2-Node-Full-SapphireRapids testbed includes one SUT (Server-Type-TODO) and
+one TG (Server-Type-TODO) connected in a 2-node physical topology with NUMA (socket) daisy chaining. For more detail see [Server Types](#server-types) and [Testbed Topology-TODO](#TODO).
+Used for FD.io performance tests in a full system SUT setup with all PCIe Gen5 x16 lane slots populated with 2p200GbE NICs.
+
+### 2-Node-SapphireRapids Xeon Intel (2n-spr)
+
+Each 2-Node-SapphireRapids testbed includes one SUT (Server-Type-TODO) and
+one TG (Server-Type-TODO) connected in a 2-node circular topology. For more detail see [Server Types](#server-types) and [Testbed Topology-TODO](#TODO).
+Used for FD.io performance tests.
## Server Management
@@ -513,11 +530,11 @@ FD.io CSIT lab contains following server types:
- RAM Memory: 16* 16GB DDR4-3200.
- Disks: 2* 960GB SATA SSD.
- NICs configuration:
- - Numa0: (x16, x16, x16 PCIe3.0 lanes)
+ - Numa0: (x16, x16, x16 PCIe4.0 lanes)
- PCIe Slot2 18:00.xx: xxv710-DA2-2p25GE Intel.
- PCIe Slot4 3b:00.xx: e810-XXVDA4-4p25GE Intel.
- PCIe Slot9 5e:00.xx: e810-2CQDA2-2p100GE Intel.
- - Numa1: (x16, x16, x16 PCIe3.0 lanes)
+ - Numa1: (x16, x16, x16 PCIe4.0 lanes)
- PCIe Slot6 86:00.xx: empty.
- PCIe Slot8 af:00.xx: empty.
- PCIe Slot10 d8:00.xx: empty.
@@ -533,11 +550,11 @@ FD.io CSIT lab contains following server types:
- RAM Memory: 16* 16GB DDR4-3200.
- Disks: 2* 960GB SATA SSD.
- NICs configuration:
- - Numa0: (x16, x16, x16 PCIe3.0 lanes)
+ - Numa0: (x16, x16, x16 PCIe4.0 lanes)
- PCIe Slot2 18:00.xx: xxv710-DA2-2p25GE Intel.
- PCIe Slot4 3b:00.xx: e810-XXVDA4-4p25GE Intel.
- PCIe Slot9 5e:00.xx: e810-2CQDA2-2p100GE Intel.
- - Numa1: (x16, x16, x16 PCIe3.0 lanes)
+ - Numa1: (x16, x16, x16 PCIe4.0 lanes)
- PCIe Slot6 86:00.xx: e810-2CQDA2-2p100GE Intel.
- PCIe Slot8 af:00.xx: empty.
- PCIe Slot10 d8:00.xx: empty.
@@ -553,11 +570,11 @@ FD.io CSIT lab contains following server types:
- RAM Memory: 16* 16GB DDR4-3200.
- Disks: 2* 960GB SATA SSD.
- NICs configuration:
- - Numa0: (x16, x16, x16 PCIe3.0 lanes)
+ - Numa0: (x16, x16, x16 PCIe4.0 lanes)
- PCIe Slot2 18:00.xx: xxv710-DA2-2p25GE Intel.
- PCIe Slot4 3b:00.xx: e810-XXVDA4-4p25GE Intel.
- PCIe Slot9 5e:00.xx: e810-2CQDA2-2p100GE Intel.
- - Numa1: (x16, x16, x16 PCIe3.0 lanes)
+ - Numa1: (x16, x16, x16 PCIe4.0 lanes)
- PCIe Slot6 86:00.xx: empty.
- PCIe Slot8 af:00.xx: empty.
- PCIe Slot10 d8:00.xx: empty.
@@ -573,11 +590,11 @@ FD.io CSIT lab contains following server types:
- RAM Memory: 16* 16GB DDR4-3200.
- Disks: 2* 960GB SATA SSD.
- NICs configuration:
- - Numa0: (x16, x16, x16 PCIe3.0 lanes)
+ - Numa0: (x16, x16, x16 PCIe4.0 lanes)
- PCIe Slot2 18:00.xx: xxv710-DA2-2p25GE Intel.
- PCIe Slot4 3b:00.xx: empty.
- PCIe Slot9 5e:00.xx: empty.
- - Numa1: (x16, x16, x16 PCIe3.0 lanes)
+ - Numa1: (x16, x16, x16 PCIe4.0 lanes)
- PCIe Slot6 86:00.xx: e810-XXVDA4-4p25GE Intel.
- PCIe Slot8 af:00.xx: e810-2CQDA2-2p100GE Intel.
- PCIe Slot10 d8:00.xx: empty.
@@ -595,6 +612,126 @@ FD.io CSIT lab contains following server types:
- NICs configuration:
- Numa0: (x16, PCIe3.0 lane)
- PCIe BuiltIn ec:00.xx: e810-XXVDA4-4p25GE Intel.
+22. Server-Type-H1: Purpose - SapphireRapids Xeon SUT for FD.io full system performance testing.
+ - Quantity: 1.
+ - Physical connectivity:
+ - IPMI and host management ports.
+ - NIC ports connected into 3-numa-node testbed topologies.
+ - Main HW configuration:
+ - Chassis: SuperMicro SYS-741GE-TNRT.
+ - Motherboard: Super X13DEG-QT-P.
+ - Processors: 2* Intel Platinum 8462Y+ 32 core 2.8 GHz 300W TDP.
+ - RAM Memory: 16* 32GB DDR5-4800.
+ - Disks: 2* 960GB SATA SSD.
+ - NICs configuration:
+ - Numa0: (x16, x16, x16 PCIe5.0 lanes)
+ - PCIe Slot2 18:00.xx: MCX713106AS-VEAT ConnectX7-2p200GE Nvidia.
+ - PCIe Slot4 3b:00.xx: MCX713106AS-VEAT ConnectX7-2p200GE Nvidia.
+ - PCIe Slot9 5e:00.xx: MCX713106AS-VEAT ConnectX7-2p200GE Nvidia.
+ - Numa1: (x16, x16, x16 PCIe5.0 lanes)
+ - PCIe Slot6 86:00.xx: MCX713106AS-VEAT ConnectX7-2p200GE Nvidia.
+ - PCIe Slot8 af:00.xx: MCX713106AS-VEAT ConnectX7-2p200GE Nvidia.
+ - PCIe Slot10 d8:00.xx: MCX713106AS-VEAT ConnectX7-2p200GE Nvidia.
+23. TODO-REVIEW Server-Type-H2: Purpose - SapphireRapids Xeon TG for FD.io full system performance testing.
+ - Quantity: 1.
+ - Physical connectivity:
+ - IPMI and host management ports.
+ - NIC ports connected into 3-numa-node testbed topologies.
+ - Main HW configuration:
+ - Chassis: SuperMicro SYS-741GE-TNRT.
+ - Motherboard: Super X13DEG-QT-P.
+ - Processors: 2* Intel Platinum 8462Y+ 32 core 2.8 GHz 300W TDP.
+ - RAM Memory: 16* 32GB DDR5-4800.
+ - Disks: 2* 960GB SATA SSD.
+ - NICs configuration:
+ - Numa0: (x16, x16, x16 PCIe5.0 lanes)
+ - PCIe Slot2 18:00.xx: MCX713106AS-VEAT ConnectX7-2p200GE Nvidia.
+ - PCIe Slot4 3b:00.xx: MCX713106AS-VEAT ConnectX7-2p200GE Nvidia.
+ - PCIe Slot9 5e:00.xx: MCX713106AS-VEAT ConnectX7-2p200GE Nvidia.
+ - Numa1: (x16, x16, x16 PCIe5.0 lanes)
+ - PCIe Slot6 86:00.xx: MCX713106AS-VEAT ConnectX7-2p200GE Nvidia.
+ - PCIe Slot8 af:00.xx: MCX713106AS-VEAT ConnectX7-2p200GE Nvidia.
+ - PCIe Slot10 d8:00.xx: empty.
+24. Server-Type-H3: Purpose - SapphireRapids Xeon SUT for FD.io performance testing.
+ - Quantity: 1.
+ - Physical connectivity:
+ - IPMI and host management ports.
+ - NIC ports connected into 3-numa-node testbed topologies.
+ - Main HW configuration:
+ - Chassis: SuperMicro SYS-741GE-TNRT.
+ - Motherboard: Super X13DEG-QT-P.
+ - Processors: 2* Intel Platinum 8462Y+ 32 core 2.8 GHz 300W TDP.
+ - RAM Memory: 16* 32GB DDR5-4800.
+ - Disks: 2* 960GB SATA SSD.
+ - NICs configuration:
+ - Numa0: (x16, x16, x16 PCIe5.0 lanes)
+ - PCIe Slot2 18:00.xx: e810-2CQDA2-2p100GE Intel.
+ - PCIe Slot4 3b:00.xx: e810-XXVDA4-4p25GE Intel.
+ - PCIe Slot9 5e:00.xx:
+ - Numa1: (x16, x16, x16 PCIe5.0 lanes)
+ - PCIe Slot6 86:00.xx: e810-2CQDA2-2p100GE Intel.
+ - PCIe Slot8 af:00.xx: e810-XXVDA4-4p25GE Intel.
+ - PCIe Slot10 d8:00.xx: empty.
+25. Server-Type-H4: Purpose - SapphireRapids Xeon TG for FD.io performance testing.
+ - Quantity: 1.
+ - Physical connectivity:
+ - IPMI and host management ports.
+ - NIC ports connected into 3-numa-node testbed topologies.
+ - Main HW configuration:
+ - Chassis: SuperMicro SYS-741GE-TNRT.
+ - Motherboard: Super X13DEG-QT-P.
+ - Processors: 2* Intel Platinum 8462Y+ 32 core 2.8 GHz 300W TDP.
+ - RAM Memory: 16* 32GB DDR5-4800.
+ - Disks: 2* 960GB SATA SSD.
+ - NICs configuration:
+ - Numa0: (x16, x16, x16 PCIe5.0 lanes)
+ - PCIe Slot2 18:00.xx: e810-2CQDA2-2p100GE Intel.
+ - PCIe Slot4 3b:00.xx: e810-XXVDA4-4p25GE Intel.
+ - PCIe Slot9 5e:00.xx: empty.
+ - Numa1: (x16, x16, x16 PCIe5.0 lanes)
+ - PCIe Slot6 86:00.xx: empty.
+ - PCIe Slot8 af:00.xx: empty.
+ - PCIe Slot10 d8:00.xx: empty.
+26. Server-Type-H5: Purpose - SapphireRapids Xeon SUT for FD.io performance testing.
+ - Quantity: 2.
+ - Physical connectivity:
+ - IPMI and host management ports.
+ - NIC ports connected into 2-node testbed topologies.
+ - Main HW configuration:
+ - Chassis: SuperMicro SYS-741GE-TNRT.
+ - Motherboard: Super X13DEG-QT-P.
+ - Processors: 2* Intel Platinum 8462Y+ 32 core 2.8 GHz 300W TDP.
+ - RAM Memory: 16* 32GB DDR5-4800.
+ - Disks: 2* 960GB SATA SSD.
+ - NICs configuration:
+ - Numa0: (x16, x16, x16 PCIe5.0 lanes)
+ - PCIe Slot2 18:00.xx: MCX713106AS-VEAT ConnectX7-2p200GE Nvidia.
+ - PCIe Slot4 3b:00.xx: e810-2CQDA2-2p100GE Intel.
+ - PCIe Slot9 5e:00.xx: e810-XXVDA4-4p25GE Intel.
+ - Numa1: (x16, x16, x16 PCIe5.0 lanes)
+ - PCIe Slot6 86:00.xx: empty.
+ - PCIe Slot8 af:00.xx: empty.
+ - PCIe Slot10 d8:00.xx: empty.
+27. Server-Type-H6: Purpose - SapphireRapids Xeon TG for FD.io performance testing.
+ - Quantity: 2.
+ - Physical connectivity:
+ - IPMI and host management ports.
+ - NIC ports connected into 2-node testbed topologies plus loopbacks in Numa1 for TG self-test.
+ - Main HW configuration:
+ - Chassis: SuperMicro SYS-741GE-TNRT.
+ - Motherboard: Super X13DEG-QT-P.
+ - Processors: 2* Intel Platinum 8462Y+ 32 core 2.8 GHz 300W TDP.
+ - RAM Memory: 16* 32GB DDR5-4800.
+ - Disks: 2* 960GB SATA SSD.
+ - NICs configuration:
+ - Numa0: (x16, x16, x16 PCIe5.0 lanes)
+ - PCIe Slot2 18:00.xx: MCX713106AS-VEAT ConnectX7-2p200GE Nvidia.
+ - PCIe Slot4 3b:00.xx: e810-2CQDA2-2p100GE Intel.
+ - PCIe Slot9 5e:00.xx: e810-XXVDA4-4p25GE Intel.
+ - Numa1: (x16, x16, x16 PCIe5.0 lanes)
+ - PCIe Slot6 86:00.xx: MCX713106AS-VEAT ConnectX7-2p200GE Nvidia.
+ - PCIe Slot8 af:00.xx: e810-2CQDA2-2p100GE Intel.
+ - PCIe Slot10 d8:00.xx: empty.
```
### Naming Convention