diff options
Diffstat (limited to 'docs/lab')
| -rw-r--r-- | docs/lab/Testbeds_Xeon_Skx_Arm_Atom.md | 1560 | ||||
| -rw-r--r-- | docs/lab/testbed_specifications.md | 1469 | ||||
| -rw-r--r-- | docs/lab/testbeds_sm_skx_hw_bios_cfg.md | 544 | ||||
| -rw-r--r-- | docs/lab/testbeds_ucs_hsw_hw_bios_cfg.md (renamed from docs/lab/Testbeds_Xeon_Hsw_VIRL.md) | 455 |
4 files changed, 2021 insertions, 2007 deletions
diff --git a/docs/lab/Testbeds_Xeon_Skx_Arm_Atom.md b/docs/lab/Testbeds_Xeon_Skx_Arm_Atom.md deleted file mode 100644 index 3227ee09d4..0000000000 --- a/docs/lab/Testbeds_Xeon_Skx_Arm_Atom.md +++ /dev/null @@ -1,1560 +0,0 @@ -## FD.io CSIT testbeds - Xeon Skylake, Arm, Atom - -This is a low-level design implemented as an extensions of FD.io CSIT lab to -accommodate the new Intel Xeon Skylake, Arm AArch64 and Atom devices. - -## Testbeds Overview - -### Testbeds Type Breakdown - -``` - #. CSIT_tb. Purpose. SUT. TG. #tb. #SUTs. #TGs. #skx_node. - 1. 1-node Xeon. func. skx. n/a. 2. 2. 0. 2. - 2. 2-node Xeon. perf. skx. skx. 4. 4. 4. 8. - 3. 3-node Xeon. perf. skx. skx. 2. 4. 2. 6. - 4. tcp-l47. tcp-stack. skx. ps1. 1. 1. 1. 1. - 5. atom-netgate. perf+func. net. skx. 1. 3. 1. 1. - 6. aarch64-d05 perf arm. skx. 1. 2. 1. 1/2. - 7. aarch64-mcbin perf arm. skx. 1. 2. 1. 1/2. - 8. 1-node aarch64 func arm. n/a. 1. 1. 0. 0. - Total skx_node: 19. -``` - -### 1-Node Xeon Testbeds - -One 1-node Xeon testbed for VPP_Device tests is built using one SUT (Type-6 -server), with NIC ports connected back-to-back. - -### 2-Node Xeon Testbeds - -Four 2-node Xeon testbeds (are expected to be built|are built), with each -testbed using one SUTs (Type-1 server) and one TG (Type-2 server) connected -back-to-back. NIC cards placement into slots and NIC ports connectivity is -following the testbed specification included in next sections. - -### 3-Node Xeon Testbeds - -Two 3-node Xeon testbeds (are expected to be built|are built), with each testbed -using two SUTs (Type-1 server) and one TG (Type-2 server) connected in full-mesh -triangle. NIC cards placement into slots and NIC ports connectivity is following -the testbed specification included in next sections. - -### 1-Node Arm Marvell ThunderX2 Testbed - -One 1-node ThunderX2 testbed for VPP_Device tests is expected to be built using -one SUT (Type-9 server), with NIC ports connected back-to-back. - -### 3-Node Arm Huawei TaiShan Testbed - -One 3-node TaiShan testbed is built, with each testbed using two SUTs (Type-3 -server) and one TG (Type-2 server) connected in full-mesh triangle. NIC cards -placement into slots and NIC ports connectivity is following the testbed -specification included in next sections. - -### 3-Node Arm Marvell MACCHIATObin Testbed - -One 3-node MACCHIATObin testbeds is built, with each testbed using two SUTs -(Type-4 server) and one TG (Type-2 server) connected in full-mesh triangle. -Built-in NIC ports connectivity is following the testbed specification included -in next sections. - -### TCP/IP and L47 Testbeds - -One 2-node Ixia PS One and Xeon server testbed, for TCP/IP host stack tests. - -### Atom Testbeds -One 3-node Atom (Netgate based) testbed is built consisting of three SUTs -(Type-5 Netgate device.) NIC cards placement into slots and NIC ports -connectivity is following the testbed specification included in the next -section. - -## Inventory - -### Appliances - -``` -1. Ixia PerfectStorm One Appliance - - 1 * PS10GE4NG - - Chassis: PS10GE4NG. - - Motherboard: SuperMicro X11DPG-QT. - - Processors: Quad-Core, Intel Processor. - - HW accelerators: FPGA offload. - - RAM Memory: 64GB. - - Disks: 1 * 1 TB, Enterprise Class, High MTBF. - - Physical Interfaces: 4 * 10GE SFP+. - - Operating System: Native IxOS. -``` - -### Arm Servers - -``` -1. Arm Cortex A-72 servers - - 1 * ThunderX2 - - Chassis: Marvell ThunderX2 - - Processors: 2* ThunderX2 CN9975 ~ 112* ThunderX2. - - RAM Memory: 4* 32GB RDIMM - - Disks: 1* 480GB SSD Micron, 1* 1000GB HDD Seagate_25 - - 2 * Huawei TaiShan 2280. - - Chassis: Huawei TaiShan 2280. - - Processors: 1* hip07-d05 ~ 64* Arm Cortex-A72. - - RAM Memory: 8* 16GB DDR4-2400MT/s. - - Disks: 1* 4TB SATA HDD. - - 3 * MACCHIATObin - - Chassis: MACCHIATObin. - - Processors: 1* Armada 8040 ~ 4* Arm Cortex-A72. - - RAM Memory: 1* 16GB DDR4. - - Disks: 1* 128GB(?) SATA SDD. -``` - -Platform Name and Specification | Role | Status | Hostname | IP | IPMI | Cores | RAM | Ethernet | Distro -------------------------------- | ---- | ------ | -------- | -- | ---- | ----- | --- | -------- | ------ -[SoftIron OverDrive 1000](https://softiron.com/development-tools/overdrive-1000/) | CI build server | Up, Not Running Jobs | softiron-1 | 10.30.51.12 | N/A | 4 | 8GB | | openSUSE - | | CI build server | Up, Not Running Jobs | softiron-2 | 10.30.51.13 | N/A | 4 | 8GB | | openSUSE - | | CI build server | Up, Not Running Jobs | softiron-3 | 10.30.51.14 | N/A | 4 | 8GB | | openSUSE -[Marvell ThunderX](https://www.marvell.com/server-processors/thunderx-arm-processors/) | CI build server | Up, Running VPP CI | nomad3arm | 10.30.51.38 | 10.30.50.38 | 96 | 128GB | 3x40GbE QSFP+ / 4x10GbE SFP+ | Ubuntu 16.04 - | | CI build server | Up, Running VPP CI | nomad4arm | 10.30.51.39 | 10.30.50.39 | 96 | 128GB | 3x40GbE QSFP+ / 4x10GbE SFP+ | Ubuntu 16.04 - | | CI build server | Up, Running VPP CI | nomad5arm | 10.30.51.40 | 10.30.50.40 | 96 | 128GB | 3x40GbE QSFP+ / 4x10GbE SFP+ | Ubuntu 16.04 - | | CI build server | Up, Not Running Jobs, USB_NIC broken, QSFP wiring to be added | nomad6arm | 10.30.51.65 | 10.30.50.65 | 96 | 256GB | 2xQSFP+ / USB Ethernet | Ubuntu 18.04.1 - | | VPP dev debug | Up | nomad7arm | 10.30.51.66 | 10.30.50.66 | 96 | 256GB | 2xQSFP+ / USB Ethernet | Ubuntu 18.04.1 - | | CI build server | Up, Not Running Jobs, USB_NIC broken, QSFP wiring to be added | nomad8arm | 10.30.51.67 | 10.30.50.67 | 96 | 256GB | 2xQSFP+ / USB Ethernet | Ubuntu 16.04.1 - | | VPP dev debug | Up | nomad9arm | 10.30.51.68 | 10.30.50.68 | 96 | 256GB | 2xQSFP+ / USB Ethernet | Ubuntu 16.04.1 -[Marvell ThunderX2](https://www.marvell.com/server-processors/thunderx2-arm-processors/) | VPP device server | Being Provisioned | s27-t13-sut1 | 10.30.51.69 | 10.30.50.69 | 112 | 128GB | 3x40GbE QSFP+ XL710-QDA2 | Ubuntu 18.04.1 -Huawei TaiShan 2280 | CSIT Performance | Up, Manual perf experiments | s17-t33-sut1 | 10.30.51.36 | 10.30.50.36 | 64 | 128GB | 2x10GbE SFP+ Intel 82599 / 2x25GbE SFP28 Mellanox CX-4 | Ubuntu 17.10 - | | CSIT Performance | Up, Manual perf experiments | s18-t33-sut2 | 10.30.51.37 | 10.30.50.37 | 64 | 128GB | 2x10GbE SFP+ Intel 82599 / 2x25GbE SFP28 Mellanox CX-4 | Ubuntu 17.10 -[Marvell MACCHIATObin](http://macchiatobin.net/) | CSIT Performance | Up, Manual experiments, Full Skx TG too much for it - suggest to use LXC/DRC TG(!) | s20-t34-sut1 | 10.30.51.41 | 10.30.51.49, then connect to /dev/ttyUSB0 | 4 | 16GB | 2x10GbE SFP+ | Ubuntu 16.04.4 - | | CSIT Performance | Up, Manual experiments, Full Skx TG too much for it - suggest to use LXC/DRC TG(!) | s21-t34-sut2 | 10.30.51.42 | 10.30.51.49, then connect to /dev/ttyUSB1 | 4 | 16GB | 2x10GbE SFP+ | Ubuntu 16.04.5 - | | VPP dev debug | Up, Manual VPP Device experiments, Full Skx TG too much for it - suggest to use LXC/DRC TG(!) | fdio-mcbin3 | 10.30.51.43 | 10.30.51.49, then connect to /dev/ttyUSB2 | 4 | 16GB | 2x10GbE SFP+ | Ubuntu 16.04.5 - -### Xeon and Atom Servers - -``` -1. Intel Xeon servers: - - 20 * SuperMicro SYS-7049GP-TRT with Xeon Skylake processors. - - Chassis: SuperMicro SYS-7049GP-TRT. - - Motherboard: SuperMicro X11DPG-QT. - - Processors: 2* Intel Platinum 8180 2.3 GHz. - - RAM Memory: 16* 16GB DDR4-2666MHz. - - Disks: 2* 1.6TB 6G SATA SSD. -2. Intel Atom servers with Rangely processors. - - 3 * Netgate XG-2758-1u - - Chassis: Netgate XG-2758-1u - - Processors: 1* Rangely (Atom) C2758 2.4 GHz - - RAM Memory: 16GB ECC - - Disks: 150 GB -``` - -### Network Interface Cards - -``` -1. 10GE NICs - - 14 * Intel® Ethernet Converged Network Adapter X710-DA4 - - 6 * Intel® Ethernet Converged Network Adapter X710-DA2 - - 6 * Intel® Ethernet Converged Network Adapter X520-DA2 -2. 25GE NICs - - 12 * Intel® Ethernet Network Adapter XXV710-DA2 -3. 40GE NICs - - 2 * Intel® Ethernet Converged Network Adapter XL710-QDA2 -4. 100GE NICs - - 4 * mcx556a-edat NICs (not on site yet, in transit) -``` - -### Pluggables and Cables - -Pluggables: - -``` -1. 10GE SFP+ - - 16 * Intel E10GSFPSR Ethernet SFP+ SR Optics - - 80 * 10G SR optic (generic, "Intel" compatible branded) -2. 25GE SFP28 - - None -3. 40GE QSFP+ - - None -4. 100GE - - 8 * mcp1600-c002 qsfp28 pluggables and cables (not on site yet, in transit) -``` - -Standalone cables: - -``` -1. 10GE - - None -2. 25GE - - None -3. 40GE QSFP+ - - 20 * Intel XLDACBL5 40G QSFP+ Passive DAC Cable -4. 100GE - - None -``` - -### Other Network Cards - -Any QATs? - -## Installation Status - -Lab installation status is tracked by LF IT team in -[FD.io Server Status](https://docs.google.com/document/d/16TdvGC73wuNQjkP355MTXRckv7yqnaxJkWwX7G7izEo/edit?ts=5b10411b#heading=h.dprb64shku8u). - -## Server/Device Management and Naming - -### Server Management Requirements - -Total of 20 SM SYS-7049GP-TRT servers are made available for FD.IO CSIT testbed. -For management purposes, each server must have following two ports connected to -the management network: - -``` -- 1GE IPMI port - - IPMI - Intelligent Platform Management Interface. - - Required for access to embedded server management with WebUI, CLI, SNMPv3, - IPMIv2.0, for firmware (BIOS) and OS updates. -- 1GE/10GE management port - - hostOS management port for general system management. -``` - -### Server and Port Naming Convention - -Following naming convention is used within this page to specify physical -connectivity and wiring across defined CSIT testbeds: - -``` -- testbedname: testbedN. -- hostname: - - traffic-generator: tN-tgW. - - system-under-testX: tN-sutX. -- portnames: - - tN-tgW-cY/pZ. - - tN-sutX-cY/pZ. -- where: - - N - testbed number. - - tgW - server acts as traffic-generator with W index. - - sutX - server acts as system-under-test with X index. - - Y - PCIe slot number denoting a NIC card number within the host. - - Y=2,4,9 - slots connected to NUMA node 0. - - Y=6,8,10 - slots connected to NUMA node 1. - - Z - port number on the NIC card. -``` - -### Server Management - Addressing - -Each server has a LOM (Lights-Out-Management e.g. SM IPMI) and a Management -port, which are connected to two different VLANs. - -``` -1. LOM (IPMI) VLAN: - - Subnet: 10.30.50.0/24 - - Gateway: 10.30.50.1 - - Broadcast: 10.30.50.255 - - DNS1: 199.204.44.24 - - DNS2: 199.204.47.54 -2. Management Vlan: - - Subnet: 10.30.51.0/24 - - Gateway: 10.30.51.1 - - Broadcast: 10.30.51.255 - - DNS1: 199.204.44.24 - - DNS2: 199.204.47.54 -``` - -To access these hosts, an VPN connection is required. - -### LOM (IPMI) VLAN IP Addresses -.. - -### Management VLAN IP Addresses -.. - -## Testbeds Specification - Target Build - -### Server/Ports Naming, NIC Placement - -#### 1-Node Xeon - -Each server in 1-node Xeon topology has its NIC cards placed, and NIC cards and -ports indexed per following specification: - -``` -- Server1 [Type-6]: - - testbedname: testbed11. - - hostname: s1-t11-sut1. - - IPMI IP: 10.30.50.47 - - Host IP: 10.30.51.50 - - portnames: - - s1-t11-sut1-c2/p1 - 10GE-port1 x710-4p10GE. - - s1-t11-sut1-c2/p2 - 10GE-port2 x710-4p10GE. - - s1-t11-sut1-c2/p3 - 10GE-port3 x710-4p10GE. - - s1-t11-sut1-c2/p4 - 10GE-port4 x710-4p10GE. - - s1-t11-sut1-c4/p1 - 10GE-port1 x710-4p10GE. - - s1-t11-sut1-c4/p2 - 10GE-port2 x710-4p10GE. - - s1-t11-sut1-c4/p3 - 10GE-port3 x710-4p10GE. - - s1-t11-sut1-c4/p4 - 10GE-port4 x710-4p10GE. -- Server2 [Type-6]: - - testbedname: testbed12. - - hostname: s2-t12-sut1. - - IPMI IP: 10.30.50.48 - - Host IP: 10.30.51.51 - - portnames: - - s2-t12-sut1-c2/p1 - 10GE-port1 x710-4p10GE. - - s2-t12-sut1-c2/p2 - 10GE-port2 x710-4p10GE. - - s2-t12-sut1-c2/p3 - 10GE-port3 x710-4p10GE. - - s2-t12-sut1-c2/p4 - 10GE-port4 x710-4p10GE. - - s2-t12-sut1-c4/p1 - 10GE-port1 x710-4p10GE. - - s2-t12-sut1-c4/p2 - 10GE-port2 x710-4p10GE. - - s2-t12-sut1-c4/p3 - 10GE-port3 x710-4p10GE. - - s2-t12-sut1-c4/p4 - 10GE-port4 x710-4p10GE. -``` - -#### 2-Node Xeon - -Each server in 2-node Xeon topology has its NIC cards placed, and NIC cards and -ports indexed per following specification: - -``` -- Server3 [Type-1]: - - testbedname: testbed21. - - hostname: s3-t21-sut1. - - IPMI IP: 10.30.50.41 - - Host IP: 10.30.51.44 - - portnames: - - s3-t21-sut1-c2/p1 - 10GE-port1 x710-4p10GE. - - s3-t21-sut1-c2/p2 - 10GE-port2 x710-4p10GE. - - s3-t21-sut1-c2/p3 - 10GE-port3 x710-4p10GE. - - s3-t21-sut1-c2/p4 - 10GE-port4 x710-4p10GE. - - s3-t21-sut1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. - - s3-t21-sut1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. - - s3-t21-sut1-c9/p1 - FUTURE 100GE-port1 ConnectX5-2p100GE. - - s3-t21-sut1-c9/p2 - FUTURE 100GE-port2 ConnectX5-2p100GE. -- Server4 [Type-2]: - - testbedname: testbed21. - - hostname: s4-t21-tg1. - - IPMI IP: 10.30.50.42 - - Host IP: 10.30.51.45 - - portnames: - - s4-t21-tg1-c2/p1 - 10GE-port1 x710-4p10GE. - - s4-t21-tg1-c2/p2 - 10GE-port2 x710-4p10GE. - - s4-t21-tg1-c2/p3 - 10GE-port3 x710-4p10GE. - - s4-t21-tg1-c2/p4 - 10GE-port4 x710-4p10GE. - - s4-t21-tg1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. - - s4-t21-tg1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. - - s4-t21-tg1-c9/p1 - FUTURE 100GE-port1 ConnectX5-2p100GE. - - s4-t21-tg1-c9/p2 - FUTURE 100GE-port2 ConnectX5-2p100GE. -- Server5 [Type-1]: - - testbedname: testbed22. - - hostname: s5-t22-sut1. - - IPMI IP: 10.30.50.49 - - Host IP: 10.30.51.52 - - portnames: - - s5-t22-sut1-c2/p1 - 10GE-port1 x710-4p10GE. - - s5-t22-sut1-c2/p2 - 10GE-port2 x710-4p10GE. - - s5-t22-sut1-c2/p3 - 10GE-port3 x710-4p10GE. - - s5-t22-sut1-c2/p4 - 10GE-port4 x710-4p10GE. - - s5-t22-sut1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. - - s5-t22-sut1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. - - s5-t22-sut1-c9/p1 - FUTURE 100GE-port1 ConnectX5-2p100GE. - - s5-t22-sut1-c9/p2 - FUTURE 100GE-port2 ConnectX5-2p100GE. -- Server6 [Type-2]: - - testbedname: testbed22. - - hostname: s6-t22-tg1. - - IPMI IP: 10.30.50.50 - - Host IP: 10.30.51.53 - - portnames: - - s6-t22-tg1-c2/p1 - 10GE-port1 x710-4p10GE. - - s6-t22-tg1-c2/p2 - 10GE-port2 x710-4p10GE. - - s6-t22-tg1-c2/p3 - 10GE-port3 x710-4p10GE. - - s6-t22-tg1-c2/p4 - 10GE-port4 x710-4p10GE. - - s6-t22-tg1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. - - s6-t22-tg1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. - - s6-t22-tg1-c9/p1 - FUTURE 100GE-port1 ConnectX5-2p100GE. - - s6-t22-tg1-c9/p2 - FUTURE 100GE-port2 ConnectX5-2p100GE. -- Server7 [Type-1]: - - testbedname: testbed23. - - hostname: s7-t23-sut1. - - IPMI IP: 10.30.50.51 - - Host IP: 10.30.51.54 - - portnames: - - s7-t23-sut1-c2/p1 - 10GE-port1 x710-4p10GE. - - s7-t23-sut1-c2/p2 - 10GE-port2 x710-4p10GE. - - s7-t23-sut1-c2/p3 - 10GE-port3 x710-4p10GE. - - s7-t23-sut1-c2/p4 - 10GE-port4 x710-4p10GE. - - s7-t23-sut1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. - - s7-t23-sut1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. - - s7-t23-sut1-c9/p1 - FUTURE 100GE-port1 ConnectX5-2p100GE. - - s7-t23-sut1-c9/p2 - FUTURE 100GE-port2 ConnectX5-2p100GE. -- Server8 [Type-2]: - - testbedname: testbed23. - - hostname: s8-t23-tg1. - - IPMI IP: 10.30.50.52 - - Host IP: 10.30.51.55 - - portnames: - - s8-t23-tg1-c2/p1 - 10GE-port1 x710-4p10GE. - - s8-t23-tg1-c2/p2 - 10GE-port2 x710-4p10GE. - - s8-t23-tg1-c2/p3 - 10GE-port3 x710-4p10GE. - - s8-t23-tg1-c2/p4 - 10GE-port4 x710-4p10GE. - - s8-t23-tg1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. - - s8-t23-tg1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. - - s8-t23-tg1-c9/p1 - FUTURE 100GE-port1 ConnectX5-2p100GE. - - s8-t23-tg1-c9/p2 - FUTURE 100GE-port2 ConnectX5-2p100GE. -- Server9 [Type-1]: - - testbedname: testbed24. - - hostname: s9-t24-sut1. - - IPMI IP: 10.30.50.53 - - Host IP: 10.30.51.56 - - portnames: - - s9-t24-sut1-c2/p1 - 10GE-port1 x710-4p10GE. - - s9-t24-sut1-c2/p2 - 10GE-port2 x710-4p10GE. - - s9-t24-sut1-c2/p3 - 10GE-port3 x710-4p10GE. - - s9-t24-sut1-c2/p4 - 10GE-port4 x710-4p10GE. - - s9-t24-sut1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. - - s9-t24-sut1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. - - s9-t24-sut1-c9/p1 - FUTURE 100GE-port1 ConnectX5-2p100GE. - - s9-t24-sut1-c9/p2 - FUTURE 100GE-port2 ConnectX5-2p100GE. -- Server10 [Type-2]: - - testbedname: testbed24. - - hostname: s10-t24-tg1. - - IPMI IP: 10.30.50.54 - - Host IP: 10.30.51.57 - - portnames: - - s10-t24-tg1-c2/p1 - 10GE-port1 x710-4p10GE. - - s10-t24-tg1-c2/p2 - 10GE-port2 x710-4p10GE. - - s10-t24-tg1-c2/p3 - 10GE-port3 x710-4p10GE. - - s10-t24-tg1-c2/p4 - 10GE-port4 x710-4p10GE. - - s10-t24-tg1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. - - s10-t24-tg1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. - - s10-t24-tg1-c9/p1 - FUTURE 100GE-port1 ConnectX5-2p100GE. - - s10-t24-tg1-c9/p2 - FUTURE 100GE-port2 ConnectX5-2p100GE. -``` - -#### 3-Node Xeon - -Each server in 3-node Xeon topology has its NIC cards placed, and NIC cards and -ports indexed per following specification: - -``` -- Server11 [Type-1]: - - testbedname: testbed31. - - hostname: s11-t31-sut1. - - IPMI IP: 10.30.50.43 - - Host IP: 10.30.51.46 - - portnames: - - s11-t31-sut1-c2/p1 - 10GE-port1 x710-4p10GE. - - s11-t31-sut1-c2/p2 - 10GE-port2 x710-4p10GE. - - s11-t31-sut1-c2/p3 - 10GE-port3 x710-4p10GE. - - s11-t31-sut1-c2/p4 - 10GE-port4 x710-4p10GE. - - s11-t31-sut1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. - - s11-t31-sut1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. - - s11-t31-sut1-c9/p1 - FUTURE 100GE-port1 ConnectX5-2p100GE. - - s11-t31-sut1-c9/p2 - FUTURE 100GE-port2 ConnectX5-2p100GE. -- Server12 [Type-1]: - - testbedname: testbed31. - - hostname: s12-t31-sut2. - - IPMI IP: 10.30.50.44 - - Host IP: 10.30.51.47 - - portnames: - - s12-t31-sut2-c2/p1 - 10GE-port1 x710-4p10GE. - - s12-t31-sut2-c2/p2 - 10GE-port2 x710-4p10GE. - - s12-t31-sut2-c2/p3 - 10GE-port3 x710-4p10GE. - - s12-t31-sut2-c2/p4 - 10GE-port4 x710-4p10GE. - - s12-t31-sut2-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. - - s12-t31-sut2-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. - - s12-t31-sut2-c9/p1 - FUTURE 100GE-port1 ConnectX5-2p100GE. - - s12-t31-sut2-c9/p2 - FUTURE 100GE-port2 ConnectX5-2p100GE. -- Server13 [Type-2]: - - testbedname: testbed31. - - hostname: s13-t31-tg1. - - IPMI IP: 10.30.50.45 - - Host IP: 10.30.51.48 - - portnames: - - s13-t31-tg1-c2/p1 - 10GE-port1 x710-4p10GE. - - s13-t31-tg1-c2/p2 - 10GE-port2 x710-4p10GE. - - s13-t31-tg1-c2/p3 - 10GE-port3 x710-4p10GE. - - s13-t31-tg1-c2/p4 - 10GE-port4 x710-4p10GE. - - s13-t31-tg1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. - - s13-t31-tg1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. - - s13-t31-tg1-c9/p1 - FUTURE 100GE-port1 ConnectX5-2p100GE. - - s13-t31-tg1-c9/p2 - FUTURE 100GE-port2 ConnectX5-2p100GE. -- Server14 [Type-1]: - - testbedname: testbed32. - - hostname: s14-t32-sut1. - - IPMI IP: 10.30.50.55 - - Host IP: 10.30.51.58 - - portnames: - - s14-t32-sut1-c2/p1 - 10GE-port1 x710-4p10GE. - - s14-t32-sut1-c2/p2 - 10GE-port2 x710-4p10GE. - - s14-t32-sut1-c2/p3 - 10GE-port3 x710-4p10GE. - - s14-t32-sut1-c2/p4 - 10GE-port4 x710-4p10GE. - - s14-t32-sut1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. - - s14-t32-sut1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. - - s14-t32-sut1-c9/p1 - FUTURE 100GE-port1 ConnectX5-2p100GE. - - s14-t32-sut1-c9/p2 - FUTURE 100GE-port2 ConnectX5-2p100GE. -- Server15 [Type-1]: - - testbedname: testbed32. - - hostname: s15-t32-sut2. - - IPMI IP: 10.30.50.56 - - Host IP: 10.30.51.59 - - portnames: - - s15-t32-sut2-c2/p1 - 10GE-port1 x710-4p10GE. - - s15-t32-sut2-c2/p2 - 10GE-port2 x710-4p10GE. - - s15-t32-sut2-c2/p3 - 10GE-port3 x710-4p10GE. - - s15-t32-sut2-c2/p4 - 10GE-port4 x710-4p10GE. - - s15-t32-sut2-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. - - s15-t32-sut2-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. - - s15-t32-sut2-c9/p1 - FUTURE 100GE-port1 ConnectX5-2p100GE. - - s15-t32-sut2-c9/p2 - FUTURE 100GE-port2 ConnectX5-2p100GE. -- Server16 [Type-2]: - - testbedname: testbed32. - - hostname: s16-t32-tg1. - - IPMI IP: 10.30.50.57 - - Host IP: 10.30.51.60 - - portnames: - - s16-t32-tg1-c2/p1 - 10GE-port1 x710-4p10GE. - - s16-t32-tg1-c2/p2 - 10GE-port2 x710-4p10GE. - - s16-t32-tg1-c2/p3 - 10GE-port3 x710-4p10GE. - - s16-t32-tg1-c2/p4 - 10GE-port4 x710-4p10GE. - - s16-t32-tg1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. - - s16-t32-tg1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. - - s16-t32-tg1-c9/p1 - FUTURE 100GE-port1 ConnectX5-2p100GE. - - s16-t32-tg1-c9/p2 - FUTURE 100GE-port2 ConnectX5-2p100GE. -``` - -#### 1-Node Arm - -``` -- Server17 [Type-3]: - - testbedname: testbed13. - - hostname: s27-t13-sut1. - - IPMI IP: 10.30.50.69 - - Host IP: 10.30.51.69 - - portnames: - - s27-t13-sut1-c1/p1 - 40GE-port1 XL710-QDA2-2p40GE. - - s27-t13-sut1-c1/p2 - 40GE-port2 XL710-QDA2-2p40GE. - - s27-t13-sut1-c3/p1 - 40GE-port1 XL710-QDA2-2p40GE. - - s27-t13-sut1-c3/p2 - 40GE-port2 XL710-QDA2-2p40GE. - - s27-t13-sut1-c6/p1 - 40GE-port1 XL710-QDA2-2p40GE. - - s27-t13-sut1-c6/p2 - 40GE-port2 XL710-QDA2-2p40GE. -``` - -#### 3-Node Arm - -Note: Server19 (TG) is shared between testbed33 & testbed34 - -``` -- Server17 [Type-3]: - - testbedname: testbed33. - - hostname: s17-t33-sut1. - - IPMI IP: 10.30.50.36 - - Host IP: 10.30.51.36 - - portnames: - - s17-t33-sut1-c6/p1 - 10GE-port1 82599-2p10GE. - - s17-t33-sut1-c6/p2 - 10GE-port2 82599-2p10GE. - - s17-t33-sut1-c4/p1 - 25GE-port1 cx4-2p25GE. - - s17-t33-sut1-c4/p2 - 25GE-port2 cx4-2p25GE. -- Server18 [Type-3]: - - testbedname: testbed33. - - hostname: s18-t33-sut2. - - IPMI IP: 10.30.50.37 - - Host IP: 10.30.51.37 - - portnames: - - s18-t33-sut2-c6/p1 - 10GE-port1 82599-2p10GE. - - s18-t33-sut2-c6/p2 - 10GE-port2 82599-2p10GE. - - s18-t33-sut2-c4/p1 - 25GE-port1 cx4-2p25GE. - - s18-t33-sut2-c4/p2 - 25GE-port2 cx4-2p25GE. -- Server19 [Type-2]: - - testbednames: testbed33 and testbed34. - - hostname: s19-t33t34-tg1. - - IPMI IP: 10.30.50.46 - - Host IP: 10.30.51.49 - - portnames: - - s19-t33t34-tg1-c2/p1 - 10GE-port1 x710-4p10GE. - - s19-t33t34-tg1-c2/p2 - 10GE-port2 x710-4p10GE. - - s19-t33t34-tg1-c2/p3 - 10GE-port3 x710-4p10GE. - - s19-t33t34-tg1-c2/p4 - 10GE-port4 x710-4p10GE. - - s19-t33t34-tg1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. - - s19-t33t34-tg1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. - - s19-t33t34-tg1-c10/p1 - 10GE-port1 x710-4p10GE. - - s19-t33t34-tg1-c10/p2 - 10GE-port2 x710-4p10GE. - - s19-t33t34-tg1-c10/p3 - 10GE-port3 x710-4p10GE. - - s19-t33t34-tg1-c10/p4 - 10GE-port4 x710-4p10GE. -- Server20 [Type-4]: - - testbedname: testbed34. - - hostname: s20-t34-sut1. - - IPMI IP: N/A - - Host IP: 10.30.51.41 - - portnames: - - s20-t34-sut1-ca/p1 - 10GE-port1 Marvell. - - s20-t34-sut1-ca/p2 - 10GE-port2 Marvell. -- Server21 [Type-4]: - - testbedname: testbed34. - - hostname: s21-t34-sut2. - - IPMI IP: N/A - - Host IP: 10.30.51.42 - - portnames: - - s21-t34-sut2-ca/p1 - 10GE-port1 Marvell. - - s21-t34-sut2-ca/p2 - 10GE-port2 Marvell. -``` - -#### TCP/IP and L47 - -Each server (appliance) in 2-node TCP/IP topology has its NIC cards placed, and -NIC cards and ports indexed per following specification: - -``` -- Server25 [Type-8]: - - testbedname: testbed25. - - hostname: s25-t25-sut1. - - IPMI IP: 10.30.50.58 - - Host IP: 10.30.51.61 - - portnames: - - s25-t25-sut1-c2/p1 - 10GE-port1 x710-4p10GE. - - s25-t25-sut1-c2/p2 - 10GE-port2 x710-4p10GE. - - s25-t25-sut1-c2/p3 - 10GE-port3 x710-4p10GE. - - s25-t25-sut1-c2/p4 - 10GE-port4 x710-4p10GE. -- Server26 [Type-7]: - - testbedname: testbed25. - - hostname: s26-t25-tg1. - - IPMI IP: 10.30.50.59 - - Host IP: 10.30.51.62 - - portnames: - - s26-t25-tg1-p1 - 10GE-port1. - - s26-t25-tg1-p2 - 10GE-port2. - - s26-t25-tg1-p3 - 10GE-port3. - - s26-t25-tg1-p4 - 10GE-port4. -``` - -#### 3-Node Atom - -Note: There is no IPMI. Serial console is accessible via VIRL2 and VIRL3 USB. - -``` -- Server22 [Type-5]: - - testbedname: testbed35. - - hostname: s22-t35-sut1 (vex-yul-rot-netgate-1). - - IPMI IP: 10.30.51.29 - screen -r /dev/ttyUSB0 - - Host IP: 10.30.51.9 - - portnames: - - s22-t35-sut1-p1 - 10GE-port1 ix0 82599. - - s22-t35-sut1-p2 - 10GE-port2 ix1 82599. - - 1GB ports (tbd) -- Server23 [Type-5]: - - testbedname: testbed35. - - hostname: s23-t35-sut2 (vex-yul-rot-netgate-2). - - IPMI IP: 10.30.51.30 - screen -r /dev/ttyUSB1 - - Host IP: 10.30.51.10 - - portnames: - - s23-t35-sut1-p1 - 10GE-port1 ix0 82599. - - s23-t35-sut1-p2 - 10GE-port2 ix1 82599. - - 1GB ports (tbd) -- Server24 [Type-5]: - - testbedname: testbed35. - - hostname: s24-t35-sut3 (vex-yul-rot-netgate-3). - - IPMI IP: 10.30.51.30 - screen -r /dev/ttyUSB2 - - Host IP: 10.30.51.11 - - portnames: - - s24-t35-sut1-p1 - 10GE-port1 ix0 82599. - - s24-t35-sut1-p2 - 10GE-port2 ix1 82599. - - 1GB ports (tbd) -``` - -### Physical Connectivity within Testbeds - -#### 1-Node Xeon - -Two 1-Node testbeds are constructed by connecting 2 Xeon servers using below -specification: - -``` -- testbed11: - - ring1 10GE-ports x710-4p10GE: - - s1-t11-sut1-c2/p1 to s1-t11-sut1-c4/p1. - - ring2 10GE-ports x710-4p10GE: - - s1-t11-sut1-c2/p2 to s1-t11-sut1-c4/p2. - - ring3 10GE-ports x710-4p10GE: - - s1-t11-sut1-c2/p3 to s1-t11-sut1-c4/p3. - - ring4 10GE-ports x710-4p10GE: - - s1-t11-sut1-c2/p3 to s1-t11-sut1-c4/p3. -- testbed12: - - ring1 10GE-ports x710-4p10GE: - - s2-t12-sut1-c2/p1 to s2-t12-sut1-c4/p1. - - ring2 10GE-ports x710-4p10GE: - - s2-t12-sut1-c2/p2 to s2-t12-sut1-c4/p2. - - ring3 10GE-ports x710-4p10GE: - - s2-t12-sut1-c2/p3 to s2-t12-sut1-c4/p3. - - ring4 10GE-ports x710-4p10GE: - - s2-t12-sut1-c2/p3 to s2-t12-sut1-c4/p3. -``` - -#### 2-Node Xeon - -Four 2-Node testbeds are constructed by connecting 8 Xeon servers using below -specification: - -``` -- testbed21: - - ring1 10GE-ports x710-4p10GE on SUT: - - s4-t21-tg1-c2/p1 to s3-t21-sut1-c2/p1. - - s3-t21-sut1-c2/p2 to s4-t21-tg1-c2/p2. - - ring2 10GE-ports x710-4p10GE on SUT: - - s4-t21-tg1-c2/p3 to s3-t21-sut1-c2/p3. - - s3-t21-sut1-c2/p4 to s4-t21-tg1-c2/p4. - - ring3 25GE-ports xxv710-DA2-2p25GE on SUT - - s4-t21-tg1-c4/p1 to s3-t21-sut1-c4/p1. - - s3-t21-sut1-c4/p2 to s4-t21-tg1-c4/p2. - - FUTURE ring4 100GE-ports ConnectX5-2p100GE on SUT: - - s4-t21-tg1-c9/p1 to s3-t21-sut1-c9/p1. - - s3-t21-sut1-c9/p2 to s4-t21-tg1-c9/p2. - - ring5 10GE-ports x710-4p10GE loopbacks on TG for self-tests: - - s4-t21-tg1-c10/p1 to s4-t21-tg1-c10/p2. - - s4-t21-tg1-c10/p3 to s4-t21-tg1-c10/p4. -- testbed22: - - ring1 10GE-ports x710-4p10GE on SUT: - - s6-t22-tg1-c2/p1 to s5-t22-sut1-c2/p1. - - s5-t22-sut1-c2/p2 to s6-t22-tg1-c2/p2. - - ring2 10GE-ports x710-4p10GE on SUT: - - s6-t22-tg1-c2/p3 to s5-t22-sut1-c2/p3. - - s5-t22-sut1-c2/p4 to s6-t22-tg1-c2/p4. - - ring3 25GE-ports xxv710-DA2-2p25GE on SUT - - s6-t22-tg1-c4/p1 to s5-t22-sut1-c4/p1. - - s5-t22-sut1-c4/p2 to s6-t22-tg1-c4/p2. - - FUTURE ring4 100GE-ports ConnectX5-2p100GE on SUT: - - t22-tg1-c9/p1 to s5-t22-sut1-c9/p1. - - s5-t22-sut1-c9/p2 to s6-t22-tg1-c9/p2. - - ring5 10GE-ports x710-4p10GE loopbacks on TG for self-tests: - - s6-t22-tg1-c10/p1 to s6-t22-tg1-c10/p2. - - s6-t22-tg1-c10/p3 to s6-t22-tg1-c10/p4. -- testbed23: - - ring1 10GE-ports x710-4p10GE on SUT: - - s8-t23-tg1-c2/p1 to s7-t23-sut1-c2/p1. - - s7-t23-sut1-c2/p2 to s8-t23-tg1-c2/p2. - - ring2 10GE-ports x710-4p10GE on SUT: - - s8-t23-tg1-c2/p3 to s7-t23-sut1-c2/p3. - - s7-t23-sut1-c2/p4 to s8-t23-tg1-c2/p4. - - ring3 25GE-ports xxv710-DA2-2p25GE on SUT - - s8-t23-tg1-c4/p1 to s7-t23-sut1-c4/p1. - - s7-t23-sut1-c4/p2 to s8-t23-tg1-c4/p2. - - FUTURE ring4 100GE-ports ConnectX5-2p100GE on SUT: - - s8-t23-tg1-c9/p1 to s7-t23-sut1-c9/p1. - - s7-t23-sut1-c9/p2 to s8-t23-tg1-c9/p2. - - ring5 10GE-ports x710-4p10GE loopbacks on TG for self-tests: - - s8-t23-tg1-c10/p1 to s8-t23-tg1-c10/p2. - - s8-t23-tg1-c10/p3 to s8-t23-tg1-c10/p4. -- testbed24: - - ring1 10GE-ports x710-4p10GE on SUT: - - s10-t24-tg1-c2/p1 to s9-t24-sut1-c2/p1. - - s9-t24-sut1-c2/p2 to s10-t24-tg1-c2/p2. - - ring2 10GE-ports x710-4p10GE on SUT: - - s10-t24-tg1-c2/p3 to s9-t24-sut1-c2/p3. - - s9-t24-sut1-c2/p4 to s10-t24-tg1-c2/p4. - - ring3 25GE-ports xxv710-DA2-2p25GE on SUT - - s10-t24-tg1-c4/p1 to s9-t24-sut1-c4/p1. - - s9-t24-sut1-c4/p2 to s10-t24-tg1-c4/p2. - - FUTURE ring4 100GE-ports ConnectX5-2p100GE on SUT: - - s10-t24-tg1-c9/p1 to s9-t24-sut1-c9/p1. - - s9-t24-sut1-c9/p2 to s10-t24-tg1-c9/p2. - - ring5 10GE-ports x710-4p10GE loopbacks on TG for self-tests: - - s10-t24-tg1-c10/p1 to s10-t24-tg1-c10/p2. - - s10-t24-tg1-c10/p3 to s10-t24-tg1-c10/p4. -``` - -#### 3-Node Xeon - -Two 3-Node testbeds are constructed by connecting 6 Xeon servers using below -specification: - -``` -- testbed31: - - ring1 10GE-ports x710-4p10GE on SUTs: - - s13-t31-tg1-c2/p1 to s11-t31-sut1-c2/p1. - - s11-t31-sut1-c2/p2 to s12-t31-sut2-c2/p2. - - s12-t31-sut2-c2/p1 to s13-t31-tg1-c2/p2. - - ring2 10GE-ports x710-4p10GE on SUT: - - s13-t31-tg1-c2/p3 to s11-t31-sut1-c2/p3. - - s11-t31-sut1-c2/p4 to s12-t31-sut2-c2/p4. - - s12-t31-sut2-c2/p3 to s13-t31-tg1-c2/p4. - - ring3 25GE-ports xxv710-DA2-2p25GE on SUT - - s13-t31-tg1-c4/p1 to s11-t31-sut1-c4/p1. - - s11-t31-sut1-c4/p2 to s12-t31-sut2-c4/p2. - - s12-t31-sut2-c4/p1 to s13-t31-tg1-c4/p2. - - FUTURE ring4 100GE-ports ConnectX5-2p100GE on SUT: - - s13-t31-tg1-c9/p1 to s11-t31-sut1-c9/p1. - - s11-t31-sut1-c9/p2 to s12-t31-sut2-c9/p2. - - s12-t31-sut2-c9/p1 to s13-t31-tg1-c9/p2. - - ring5 10GE-ports x710-4p10GE loopbacks on TG for self-tests: - - s13-t31-tg1-c10/p1 to s13-t31-tg1-c10/p2. - - s13-t31-tg1-c10/p3 to s13-t31-tg1-c10/p4. -- testbed32: - - ring1 10GE-ports x710-4p10GE on SUTs: - - s16-t32-tg1-c2/p1 to s14-t32-sut1-c2/p1. - - s14-t32-sut1-c2/p2 to s15-t32-sut2-c2/p2. - - s15-t32-sut2-c2/p1 to s16-t32-tg1-c2/p2. - - ring2 10GE-ports x710-4p10GE on SUT: - - s16-t32-tg1-c2/p3 to s14-t32-sut1-c2/p3. - - s14-t32-sut1-c2/p4 to s15-t32-sut2-c2/p4. - - s15-t32-sut2-c2/p3 to s16-t32-tg1-c2/p4. - - ring3 25GE-ports xxv710-DA2-2p25GE on SUT - - s16-t32-tg1-c4/p1 to s14-t32-sut1-c4/p1. - - s14-t32-sut1-c4/p2 to s15-t32-sut2-c4/p2. - - s15-t32-sut2-c4/p1 to s16-t32-tg1-c4/p2. - - FUTURE ring4 100GE-ports ConnectX5-2p100GE on SUT: - - s16-t32-tg1-c9/p1 to s14-t32-sut1-c9/p1. - - s14-t32-sut1-c9/p2 to s15-t32-sut2-c9/p2. - - s15-t32-sut2-c9/p1 to s16-t32-tg1-c9/p2. - - ring5 10GE-ports x710-4p10GE loopbacks on TG for self-tests: - - s16-t32-tg1-c10/p1 to s16-t32-tg1-c10/p2. - - s16-t32-tg1-c10/p3 to s16-t32-tg1-c10/p4. -``` - - -#### 1-Node Arm - -One 1-Node testbed is constructed by connecting 1 Arm server using below -specification: - -``` -- testbed13: - - ring1 40GE-ports XL710-QDA2-2p40GE on SUTs: - - s27-t13-sut1-c1/p2 - s27-t13-sut1-c3/p1. - - ring2 40GE-ports XL710-QDA2-2p40GE on SUTs: - - s27-t13-sut1-c3/p2 - s27-t13-sut1-c6/p1. - - ring3 40GE-ports XL710-QDA2-2p40GE on SUTs: - - s27-t13-sut1-c6/p2 - s27-t13-sut1-c1/p1. -``` - -#### 3-Node Arm - -One 3-Node testbed is constructed by connecting 2 TaiShan servers and one Xeon -server using below specification: - -``` -- testbed33: - - ring1 10GE-ports 82599-2p10GE on SUTs: - - t33t34-tg1-c2/p2 - t33-sut1-c6/p2. - - t33-sut1-c6/p1 - t33-sut2-c6/p2. - - t33-sut2-c6/p1 - t33t34-tg1-c2/p1. - - ring2 25GE-ports cx4-2p25GE on SUTs: - - t33t34-tg1-c4/p2 - t33-sut1-c4/p2. - - t33-sut1-c4/p1 - t33-sut2-c4/p2. - - t33-sut2-c4/p1 - t33t34-tg1-c4/p1. -``` - -One 3-Node testbed is constructed by connecting 2 MACCHIATObin servers and one -Xeon server using below specification: - -``` -- testbed34: - - ring1 10GE-ports Marvell on SUTs: - - t33t34-tg1-c2/p3 - t34-sut1-ca/p1. - - t34-sut1-ca/p2 - t34-sut2-ca/p1. - - t34-sut2-ca/p2 - t33t34-tg1-c2/p4. -``` - -#### TCP/IP and L47 - -One 2-Node TCP/IP testbed is constructed by connecting Ixia PSOne and 1 Xeon -server using below specification: - -``` -- testbed25: - - link1 10GE-port x710-4p10GE on SUT: - - t25-tg1-p1 to t25-sut1-c2/p1. - - link2 10GE-port x710-4p10GE on SUT: - - t25-tg1-p2 to t25-sut1-c2/p2. - - link3 10GE-port x710-4p10GE on SUT: - - t25-tg1-p3 to t25-sut1-c2/p3. - - link4 10GE-port x710-4p10GE on SUT: - - t25-tg1-p4 to t25-sut1-c2/p4. -``` - -#### 3-Node Atom - -.. - -## Server Specification - -### Hardware Configuration - -The new FD.io CSIT-CPL lab (is expected to contain|contains) following hardware -server configurations: - -``` -1. Type-1: Purpose - (Intel Xeon Processor) SUT for SW Data Plane Workload i.e. - VPP, testpmd. - - Quantity: TBD based on testbed allocation. - - Physical connectivity: - - IPMI and host management ports. - - NIC ports connected into 2-node and 3-node topologies. - - Main HW configuration: - - Chassis: SuperMicro SYS-7049GP-TRT. - - Motherboard: SuperMicro X11DPG-QT. - - Processors: 2* Intel Platinum 8180 2.3 GHz. - - RAM Memory: 16* 16GB DDR4-2666MHz. - - Disks: 2* 1.6TB 6G SATA SSD. - - NICs configuration: - - Numa0: (x16, x16, x16 PCIe3.0 lanes) - - PCIe Slot2 18:00.xx: x710-4p10GE Intel. - - PCIe Slot4 3b:00.xx: xxv710-DA2-2p25GE Intel. - - PCIe Slot9 5e:00.xx: FUTURE ConnectX5-2p100GE Mellanox. - - Numa1: (x16, x16, x16 PCIe3.0 lanes) - - PCIe Slot6 86:00.xx: empty. - - PCIe Slot8 af:00.xx: empty. - - PCIe Slot10 d8:00.xx: empty. -2. Type-2: Purpose - (Intel Xeon Processor) TG for T-Rex. - - Quantity: TBD based on testbed allocation. - - Physical connectivity: - - IPMI and host management ports. - - NIC ports connected into 2-node and 3-node topologies. - - Main HW configuration: - - Chassis: SuperMicro SYS-7049GP-TRT. - - Motherboard: SuperMicro X11DPG-QT. - - Processors: 2* Intel Platinum 8180 2.3 GHz. - - RAM Memory: 16* 16GB DDR4-2666MHz. - - Disks: 2* 1.6TB 6G SATA SSD. - - NICs configuration: - - Numa0: (x16, x16, x16 PCIe3.0 lanes) - - PCIe Slot2 18:00.xx: x710-4p10GE Intel. - - PCIe Slot4 3b:00.xx: xxv710-DA2 2p25GE Intel. - - PCIe Slot9 5e:00.xx: FUTURE ConnectX5-2p100GE Mellanox. - - Numa1: (x16, x16, x16 PCIe3.0 lanes) - - PCIe Slot6 86:00.xx: empty. - - PCIe Slot8 af:00.xx: empty. - - PCIe Slot10 d8:00.xx: x710-4p10GE Intel. -3. Type-3: Purpose - (Arm hip07-d05 Processor) SUT for SW Data Plane Workload - i.e. VPP, testpmd. - - Quantity: 2 - - Physical connectivity: - - IPMI(?) and host management ports. - - NIC ports connected into 3-node topology. - - Main HW configuration: - - Chassis: Huawei TaiShan 2280. - - Processors: 1* hip07-d05 ~ 64* Arm Cortex-A72 - - RAM Memory: 8* 16GB DDR4-2400MT/s - - Disks: 1* 4TB SATA HDD - - NICs configuration: - - PCIe Slot4 e9:00.xx: connectx4-2p25GE Mellanox. - - PCIe Slot6 11:00.xx: 82599-2p10GE Intel. -4. Type-4: Purpose - (Arm Armada 8040 Processor) SUT for SW Data Plane Workload - i.e. VPP, testpmd. - - Quantity: 3 - - Physical connectivity: - - Host management ports. - - NIC ports connected into 2-node and 3-node topologies. - - Main HW configuration: - - Chassis: MACCHIATObin. - - Processors: 1* Armada 8040 ~ 4* Arm Cortex-A72 - - RAM Memory: 1* 16GB DDR4 - - Disks: 1* 128GB(?) SATA SDD - - NICs configuration: - - pp2-2p10GE Marvell (on-chip Ethernet ports ; marvell plugin in VPP) -5. Type-5: Purpose - (Intel Atom Processor) SUT for SW Data Plane Workload i.e. - VPP, testpmd. - - Quantity: TBD based on testbed allocation. - - Physical connectivity: - - Management: serial Port (usb) for console - - NIC ports connected into 2-node. - - Main HW configuration: - - Chassis: Netgate XG-2758-1u - - Processors: 1* Rangeley (Atom) C2758 2.4 GHz - - RAM Memory: 16GB ECC - - Disks: 150 GB - - NICs configuration: - - 2x 10Gb Intel 82599ES - - 4x 1GB Intel I354 -6. Type-6: Purpose - (Intel Xeon Processor) SUT for VPP_Device functional tests. - - Quantity: 1. - - Physical connectivity: - - IPMI and host management ports. - - NIC ports connected into 2-node and 3-node topologies. - - Main HW configuration: - - Chassis: SuperMicro SYS-7049GP-TRT. - - Motherboard: SuperMicro X11DPG-QT. - - Processors: 2* Intel Platinum 8180 2.3 GHz. - - RAM Memory: 16* 16GB DDR4-2666MHz. - - Disks: 2* 1.6TB 6G SATA SSD. - - NICs configuration: - - Numa0: (x16, x16, x16 PCIe3.0 lanes) - - PCIe Slot2 18:00.xx: x710-4p10GE Intel. - - PCIe Slot4 3b:00.xx: x710-4p10GE Intel. - - PCIe Slot9 5e:00.xx: empty. - - Numa1: (x16, x16, x16 PCIe3.0 lanes) - - PCIe Slot6 86:00.xx: empty. - - PCIe Slot8 af:00.xx: empty. - - PCIe Slot10 d8:00.xx: empty. -7. Type-7: Purpose - Ixia PerfectStorm One Appliance TG for TCP/IP performance - tests. - - Quantity: 1. - - Physical connectivity: - - Host management interface: 10/100/1000-BaseT. - - 8-port 10GE SFP+ integrated NIC. - - Main HW configuration: - - Chassis: PS10GE4NG. - - Motherboard: SuperMicro X11DPG-QT. - - Processors: Quad-Core, Intel Processor. - - HW accelerators: FPGA offload. - - RAM Memory: 64GB. - - Disks: 1 * 1 TB, Enterprise Class, High MTBF. - - Physical Interfaces: 4 * 10GE SFP+. - - Operating System: Native IxOS. - - Interface configuration: - - Port-1: 10GE SFP+. - - Port-2: 10GE SFP+. - - Port-3: 10GE SFP+. - - Port-4: 10GE SFP+. -8. Type-8: Purpose - (Intel Xeon Processor) SUT for TCP/IP host stack tests. - - Quantity: 1. - - Physical connectivity: - - IPMI and host management ports. - - NIC ports. - - Main HW configuration: - - Chassis: SuperMicro SYS-7049GP-TRT. - - Motherboard: SuperMicro X11DPG-QT. - - Processors: 2* Intel Platinum 8180 2.3 GHz. - - RAM Memory: 16* 16GB DDR4-2666MHz. - - Disks: 2* 1.6TB 6G SATA SSD. - - NICs configuration: - - Numa0: (x16, x16, x16 PCIe3.0 lanes) - - PCIe Slot2 18:00.xx: x710-4p10GE Intel. - - PCIe Slot4 3b:00.xx: empty. - - PCIe Slot9 5e:00.xx: empty. - - Numa1: (x16, x16, x16 PCIe3.0 lanes) - - PCIe Slot6 86:00.xx: empty. - - PCIe Slot8 af:00.xx: empty. - - PCIe Slot10 d8:00.xx: empty. -9. Type-9: Purpose - (Marvell ThunderX2 Processor) SUT for VPP_Device functional - tests. - - Quantity: 1 - - Physical connectivity: - - IPMI and host management ports. - - NIC ports connected into 1-node topologies. - - Main HW configuration: - - Chassis: Gigabyte R181-T90 1U - - Motheboard: MT91-FS1 - - Processors: 1* ThunderX2 ARMv8 CN9975 2.0 GHz - - RAM Memory: 4* 32GB RDIMM - - Disks: 1* 480GB SSD Micron, 1* 1000GB HDD Seagate_25 - - NICs configuration: - - Numa0: - - PCIe Slot1 05:00.xx: XL710-QDA2. - - PCIe Slot3 08:00.xx: XL710-QDA2. - - Numa1: - - PCIe Slot6 85:00.xx: XL710-QDA2. -10. Type-10: Purpose - (Intel Atom C3000 Processor) SUT for SW Data Plane - Workload i.e. VPP, testpmd. - - Quantity: 4 - - Physical connectivity: - - IPMI and host management ports. - - NIC ports. - - Main HW configuration: - - Chassis: SuperMicro SYS-E300-9A - - Processors: 1* Intel(R) Atom(TM) CPU C3858 @ 2.00GHz - - RAM Memory: 32GB ECC - - Disks: 480 GB - - NICs configuration: - - 2x 10Gb Intel x553 fiber ports - - 2x 10Gb Intel x553 copper ports - - 4x 1GB Intel I350 ports -``` - -### Xeon Skx Server BIOS Configuration - -#### Boot Feature - -``` - | Quiet Boot [Enabled] |Boot option | - | | | - | Option ROM Messages [Force BIOS] | | - | Bootup NumLock State [On] | | - | Wait For "F1" If Error [Enabled] | | - | INT19 Trap Response [Immediate] | | - | Re-try Boot [Disabled] | | - | Install Windows 7 USB support [Disabled] | | - | Port 61h Bit-4 Emulation [Disabled] | | - | | | - | Power Configuration | | - | Watch Dog Function [Disabled] | | - | Restore on AC Power Loss [Last State] | | - | Power Button Function [Instant Off] | | - | Throttle on Power Fail [Disabled] | | -``` - -#### CPU Configuration - -``` - | Processor Configuration |Enables Hyper Threading | - | -------------------------------------------------- |(Software Method to | - | Processor BSP Revision 50654 - SKX H0 |Enable/Disable Logical | - | Processor Socket CPU1 | CPU2 |Processor threads. | - | Processor ID 00050654* | 000506... | | - | Processor Frequency 2.500GHz | 2.500GHz | | - | Processor Max Ratio 19H | 19H | | - | Processor Min Ratio 0AH | 0AH | | - | Microcode Revision 02000030 | | - | L1 Cache RAM 64KB | 64KB | | - | L2 Cache RAM 1024KB | 1024KB | | - | L3 Cache RAM 39424KB | 39424KB | | - | Processor 0 Version | | - | Intel(R) Xeon(R) Platinum 8180 CPU @ 2.50GHz | | - | Processor 1 Version | | - | Intel(R) Xeon(R) Platinum 8180 CPU @ 2.50GHz | | - | | | - | Hyper-Threading [ALL] [Enable] | | - | Core Disable Bitmap(Hex) 0 | | - | Execute Disable Bit [Enable] | | - | Intel Virtualization Technology [Enable] | | - | PPIN Control [Unlock/Enable] | | - | Hardware Prefetcher [Enable] | | - | Adjacent Cache Prefetch [Enable] | | - | DCU Streamer Prefetcher [Enable] | | - | DCU IP Prefetcher [Enable] | | - | LLC Prefetch [Disable] | | - | Extended APIC [Disable] | | - | AES-NI [Enable] | | - |> Advanced Power Management Configuration | | -``` - -##### Advanced Power Management Configuration - -``` - | Advanced Power Management Configuration |Switch CPU Power Management | - | -------------------------------------------------- |profile | - | Power Technology [Custom] | | - | Power Performance Tuning [BIOS Controls EPB] | | - | ENERGY_PERF_BIAS_CFG mode [Maximum Performance] | | - |> CPU P State Control | | - |> Hardware PM State Control | | - |> CPU C State Control | | - |> Package C State Control | | - |> CPU T State Control | | -``` - -###### CPU P State Control - -``` - | CPU P State Control |Enable/Disable EIST | - | |(P-States) | - | SpeedStep (Pstates) [Disable] | | - | EIST PSD Function [HW_ALL] | | -``` - -###### Hardware PM State Control - -``` - | Hardware PM State Control |Disable: Hardware chooses a | - | |P-state based on OS Request | - | Hardware P-States [Disable] |(Legacy P-States) | - | |Native Mode:Hardware | - | |chooses a P-state based on | - | |OS guidance | - | |Out of Band Mode:Hardware | - | |autonomously chooses a | - | |P-state (no OS guidance) | -``` - -###### CPU C State Control - -``` - | CPU C State Control |Autonomous Core C-State | - | |Control | - | Autonomous Core C-State [Disable] | | - | CPU C6 report [Disable] | | - | Enhanced Halt State (C1E) [Disable] | | -``` - -###### Package C State Control - -``` - | Package C State Control |Package C State limit | - | | | - | Package C State [C0/C1 state] | | -``` - -###### CPU T State Control - -``` - | CPU T State Control |Enable/Disable Software | - | |Controlled T-States | - | Software Controlled T-States [Disable] | | -``` - -##### Chipset Configuration - -``` - | WARNING: Setting wrong values in below sections may cause |North Bridge Parameters | - | system to malfunction. | | - |> North Bridge | | - |> South Bridge | | -``` - -###### North Bridge - -``` - |> UPI Configuration |Displays and provides | - |> Memory Configuration |option to change the UPI | - |> IIO Configuration |Settings | -``` - -###### UPI Configuration - -``` - | UPI Configuration |Choose Topology Precedence | - | -------------------------------------------------- |to degrade features if | - | Number of CPU 2 |system options are in | - | Number of Active UPI Link 3 |conflict or choose Feature | - | Current UPI Link Speed Fast |Precedence to degrade | - | Current UPI Link Frequency 10.4 GT/s |topology if system options | - | UPI Global MMIO Low Base / Limit 90000000 / FBFFFFFF |are in conflict. | - | UPI Global MMIO High Base / Limit 0000000000000000 / ... | | - | UPI Pci-e Configuration Base / Size 80000000 / 10000000 | | - | Degrade Precedence [Topology Precedence] | | - | Link L0p Enable [Disable] | | - | Link L1 Enable [Disable] | | - | IO Directory Cache (IODC) [Auto] | | - | SNC [Disable] | | - | XPT Prefetch [Disable] | | - | KTI Prefetch [Enable] | | - | Local/Remote Threshold [Auto] | | - | Stale AtoS [Disable] | | - | LLC dead line alloc [Enable] | | - | Isoc Mode [Auto] | | -``` - -###### Memory Configuration - -``` - | |POR - Enforces Plan Of | - | -------------------------------------------------- |Record restrictions for | - | Integrated Memory Controller (iMC) |DDR4 frequency and voltage | - | -------------------------------------------------- |programming. Disable - | - | |Disables this feature. | - | Enforce POR [Disable] | | - | Memory Frequency [2666] | | - | Data Scrambling for NVMDIMM [Auto] | | - | Data Scrambling for DDR4 [Auto] | | - | tCCD_L Relaxation [Auto] | | - | Memory tRWSR Relaxation [Enable] | | - | 2X REFRESH [Auto] | | - | Page Policy [Auto] | | - | IMC Interleaving [2-way Interleave] | | - |> Memory Topology | | - |> Memory RAS Configuration | | -``` - -###### IIO Configuration - -``` - | IIO Configuration |Expose IIO DFX devices and | - | -------------------------------------------------- |other CPU devices like PMON | - | | | - | EV DFX Features [Disable] | | - |> CPU1 Configuration | | - |> CPU2 Configuration | | - |> IOAT Configuration | | - |> Intel. VT for Directed I/O (VT-d) | | - |> Intel. VMD technology | | - | | | - | IIO-PCIE Express Global Options | | - | ======================================== | | - | PCI-E Completion Timeout Disable [No] | | -``` - -###### CPU1 Configuration - -``` - | IOU0 (IIO PCIe Br1) [Auto] |Selects PCIe port | - | IOU1 (IIO PCIe Br2) [Auto] |Bifurcation for selected | - | IOU2 (IIO PCIe Br3) [Auto] |slot(s) | - |> CPU1 SLOT2 PCI-E 3.0 X16 | | - |> CPU1 SLOT4 PCI-E 3.0 X16 | | - |> CPU1 SLOT9 PCI-E 3.0 X16 | | -``` - -###### CPU2 Configuration - -``` - | IOU0 (IIO PCIe Br1) [Auto] |Selects PCIe port | - | IOU1 (IIO PCIe Br2) [Auto] |Bifurcation for selected | - | IOU2 (IIO PCIe Br3) [Auto] |slot(s) | - |> CPU2 SLOT6 PCI-E 3.0 X16 | | - |> CPU2 SLOT8 PCI-E 3.0 X16 | | - |> CPU2 SLOT10 PCI-E 3.0 X16 | | -``` - -##### South Bridge - -``` - | |Enables Legacy USB support. | - | USB Module Version 17 |AUTO option disables legacy | - | |support if no USB devices | - | USB Devices: |are connected. DISABLE | - | 1 Keyboard, 1 Mouse, 1 Hub |option will keep USB | - | |devices available only for | - | Legacy USB Support [Enabled] |EFI applications. | - | XHCI Hand-off [Disabled] | | - | Port 60/64 Emulation [Enabled] | | - | PCIe PLL SSC [Disable] | | - | Real USB Wake Up [Enabled] | | - | Front USB Wake Up [Enabled] | | - | | | - | Azalia [Auto] | | - | Azalia PME Enable [Disabled] | | -``` - -#### PCIe/PCI/PnP Configuration - -``` - | PCI Bus Driver Version A5.01.12 |Enables or Disables 64bit | - | |capable Devices to be | - | PCI Devices Common Settings: |Decoded in Above 4G Address | - | Above 4G Decoding [Enabled] |Space (Only if System | - | SR-IOV Support [Enabled] |Supports 64 bit PCI | - | MMIO High Base [56T] |Decoding). | - | MMIO High Granularity Size [256G] | | - | Maximum Read Request [Auto] | | - | MMCFG Base [2G] | | - | NVMe Firmware Source [Vendor Defined Fi...] | | - | VGA Priority [Onboard] | | - | CPU1 SLOT2 PCI-E 3.0 X16 OPROM [Legacy] | | - | CPU1 SLOT4 PCI-E 3.0 X16 OPROM [Legacy] | | - | CPU2 SLOT6 PCI-E 3.0 X16 OPROM [Legacy] | | - | CPU2 SLOT8 PCI-E 3.0 X16 OPROM [Legacy] | | - | CPU1 SLOT9 PCI-E 3.0 X16 OPROM [Legacy] | | - | CPU2 SLOT10 PCI-E 3.0 X16 OPROM [Legacy] | | - | CPU2 SLOT11 PCI-E 3.0 X4(IN X8) OPROM [Legacy] | | - | M.2 CONNECTOR OPROM [Legacy] | | - | Onboard LAN1 Option ROM [Legacy] | | - | Onboard LAN2 Option ROM [Disabled] | | - | Onboard Video Option ROM [Legacy] | | - |> Network Stack Configuration | | -``` - -#### ACPI Settings - -``` - | ACPI Settings |Enable or Disable Non | - | |uniform Memory Access | - | NUMA [Enabled] |(NUMA). | - | WHEA Support [Enabled] | | - | High Precision Event Timer [Enabled] | | - | ACPI Sleep State [S3 (Suspend to RAM)] | | -``` - -#### DMIDECODE - -``` - # dmidecode 3.1 - Getting SMBIOS data from sysfs. - SMBIOS 3.1.1 present. - Table at 0x000E89C0. - - Handle 0x0000, DMI type 0, 26 bytes - BIOS Information - Vendor: American Megatrends Inc. - Version: 2.0 - Release Date: 11/29/2017 - Address: 0xF0000 - Runtime Size: 64 kB - ROM Size: 64 MB - Characteristics: - PCI is supported - BIOS is upgradeable - BIOS shadowing is allowed - Boot from CD is supported - Selectable boot is supported - BIOS ROM is socketed - EDD is supported - 5.25"/1.2 MB floppy services are supported (int 13h) - 3.5"/720 kB floppy services are supported (int 13h) - 3.5"/2.88 MB floppy services are supported (int 13h) - Print screen service is supported (int 5h) - Serial services are supported (int 14h) - Printer services are supported (int 17h) - ACPI is supported - USB legacy is supported - BIOS boot specification is supported - Targeted content distribution is supported - UEFI is supported - BIOS Revision: 5.12 - - Handle 0x0001, DMI type 1, 27 bytes - System Information - Manufacturer: Supermicro - Product Name: SYS-7049GP-TRT - Version: 0123456789 - Serial Number: S291427X8332242 - UUID: 00000000-0000-0000-0000-AC1F6B8A8DB6 - Wake-up Type: Power Switch - SKU Number: To be filled by O.E.M. - Family: To be filled by O.E.M. - - Handle 0x0002, DMI type 2, 15 bytes - Base Board Information - Manufacturer: Supermicro - Product Name: X11DPG-QT - Version: 1.02 - Serial Number: VM183S014930 - Asset Tag: To be filled by O.E.M. - Features: - Board is a hosting board - Board is replaceable - Location In Chassis: To be filled by O.E.M. - Chassis Handle: 0x0003 - Type: Motherboard - Contained Object Handles: 0 - - Handle 0x0003, DMI type 3, 22 bytes - Chassis Information - Manufacturer: Supermicro - Type: Other - Lock: Not Present - Version: 0123456789 - Serial Number: C7470KH06A20167 - Asset Tag: To be filled by O.E.M. - Boot-up State: Safe - Power Supply State: Safe - Thermal State: Safe - Security Status: None - OEM Information: 0x00000000 - - Handle 0x0050, DMI type 4, 48 bytes - Processor Information - Socket Designation: CPU1 - Type: Central Processor - Family: Xeon - Manufacturer: Intel(R) Corporation - ID: 54 06 05 00 FF FB EB BF - Signature: Type 0, Family 6, Model 85, Stepping 4 - Flags: - FPU (Floating-point unit on-chip) - VME (Virtual mode extension) - DE (Debugging extension) - PSE (Page size extension) - TSC (Time stamp counter) - MSR (Model specific registers) - PAE (Physical address extension) - MCE (Machine check exception) - CX8 (CMPXCHG8 instruction supported) - APIC (On-chip APIC hardware supported) - SEP (Fast system call) - MTRR (Memory type range registers) - PGE (Page global enable) - MCA (Machine check architecture) - CMOV (Conditional move instruction supported) - PAT (Page attribute table) - PSE-36 (36-bit page size extension) - CLFSH (CLFLUSH instruction supported) - DS (Debug store) - ACPI (ACPI supported) - MMX (MMX technology supported) - FXSR (FXSAVE and FXSTOR instructions supported) - SSE (Streaming SIMD extensions) - SSE2 (Streaming SIMD extensions 2) - SS (Self-snoop) - HTT (Multi-threading) - TM (Thermal monitor supported) - PBE (Pending break enabled) - Version: Intel(R) Xeon(R) Platinum 8180 CPU @ 2.50GHz - Voltage: 1.6 V - External Clock: 100 MHz - Max Speed: 4000 MHz - Current Speed: 2500 MHz - Status: Populated, Enabled - Upgrade: Other - L1 Cache Handle: 0x004D - L2 Cache Handle: 0x004E - L3 Cache Handle: 0x004F - Serial Number: Not Specified - Asset Tag: UNKNOWN - Part Number: Not Specified - Core Count: 28 - Core Enabled: 28 - Thread Count: 56 - Characteristics: - 64-bit capable - Multi-Core - Hardware Thread - Execute Protection - Enhanced Virtualization - Power/Performance Control - - - Handle 0x0054, DMI type 4, 48 bytes - Processor Information - Socket Designation: CPU2 - Type: Central Processor - Family: Xeon - Manufacturer: Intel(R) Corporation - ID: 54 06 05 00 FF FB EB BF - Signature: Type 0, Family 6, Model 85, Stepping 4 - Flags: - FPU (Floating-point unit on-chip) - VME (Virtual mode extension) - DE (Debugging extension) - PSE (Page size extension) - TSC (Time stamp counter) - MSR (Model specific registers) - PAE (Physical address extension) - MCE (Machine check exception) - CX8 (CMPXCHG8 instruction supported) - APIC (On-chip APIC hardware supported) - SEP (Fast system call) - MTRR (Memory type range registers) - PGE (Page global enable) - MCA (Machine check architecture) - CMOV (Conditional move instruction supported) - PAT (Page attribute table) - PSE-36 (36-bit page size extension) - CLFSH (CLFLUSH instruction supported) - DS (Debug store) - ACPI (ACPI supported) - MMX (MMX technology supported) - FXSR (FXSAVE and FXSTOR instructions supported) - SSE (Streaming SIMD extensions) - SSE2 (Streaming SIMD extensions 2) - SS (Self-snoop) - HTT (Multi-threading) - TM (Thermal monitor supported) - PBE (Pending break enabled) - Version: Intel(R) Xeon(R) Platinum 8180 CPU @ 2.50GHz - Voltage: 1.6 V - External Clock: 100 MHz - Max Speed: 4000 MHz - Current Speed: 2500 MHz - Status: Populated, Enabled - Upgrade: Other - L1 Cache Handle: 0x0051 - L2 Cache Handle: 0x0052 - L3 Cache Handle: 0x0053 - Serial Number: Not Specified - Asset Tag: UNKNOWN - Part Number: Not Specified - Core Count: 28 - Core Enabled: 28 - Thread Count: 56 - Characteristics: - 64-bit capable - Multi-Core - Hardware Thread - Execute Protection - Enhanced Virtualization - Power/Performance Control -``` - -### Xeon Skx Server Firmware Inventory - -``` -Host. IPMI IP. BIOS. CPLD. Aptio SU. CPU Microcode. PCI Bus. ME Operation FW. X710 Firmware. XXV710 Firmware. i40e. -s1-t11-sut1. 10.30.50.47. 2.1. 03.B1.03. 2.19.1268. 02000043. A5.01.12. 4.0.4.294. 6.01 0x80003554 1.1747.0. 6.01 0x80003554 1.1747.0. 2.1.14-k. -s2-t12-sut1. 10.30.50.48. 2.1. 03.B1.03. 2.19.1268. 02000043. A5.01.12. 4.0.4.294. 6.01 0x80003554 1.1747.0. 6.01 0x80003554 1.1747.0. 2.1.14-k. -s3-t21-sut1. 10.30.50.41. 2.0b. 03.B1.03. 2.19.1268. 02000043. A5.01.12. 4.0.4.294. 6.01 0x80003554 1.1747.0. 6.01 0x80003554 1.1747.0. 2.1.14-k. -s4-t21-tg1. 10.30.50.42. 2.0b. 03.B1.03. 2.19.1268. 02000043. A5.01.12. 4.0.4.294. 6.01 0x80003554 1.1747.0. 6.01 0x80003554 1.1747.0. 2.1.14-k. -s5-t22-sut1. 10.30.50.49. 2.0b. 03.B1.03. 2.19.1268. 02000043. A5.01.12. 4.0.4.294. 6.01 0x80003554 1.1747.0. 6.01 0x80003554 1.1747.0. 2.1.14-k. -s6-t22-tg1. 10.30.50.50. 2.0b. 03.B1.03. 2.19.1268. 02000043. A5.01.12. 4.0.4.294. 6.01 0x80003554 1.1747.0. 6.01 0x80003554 1.1747.0. 2.1.14-k. -s7-t23-sut1. 10.30.50.51. 2.0b. 03.B1.03. 2.19.1268. 02000043. A5.01.12. 4.0.4.294. 6.01 0x80003554 1.1747.0. 6.01 0x80003554 1.1747.0. 2.1.14-k. -s8-t23-tg1. 10.30.50.52. 2.0b. 03.B1.03. 2.19.1268. 02000043. A5.01.12. 4.0.4.294. 6.01 0x80003554 1.1747.0. 6.01 0x80003554 1.1747.0. 2.1.14-k. -s9-t24-sut1. 10.30.50.53. 2.0b. 03.B1.03. 2.19.1268. 02000043. A5.01.12. 4.0.4.294. 6.01 0x80003554 1.1747.0. 6.01 0x80003554 1.1747.0. 2.1.14-k. -s10-t24-tg1. 10.30.50.54. 2.0b. 03.B1.03. 2.19.1268. 02000043. A5.01.12. 4.0.4.294. 6.01 0x80003554 1.1747.0. 6.01 0x80003554 1.1747.0. 2.1.14-k. -s11-t31-sut1. 10.30.50.43. 2.0b. 03.B1.03. 2.19.1268. 02000043. A5.01.12. 4.0.4.294. 6.01 0x80003554 1.1747.0. 6.01 0x80003554 1.1747.0. 2.1.14-k. -s12-t31-sut2. 10.30.50.44. 2.0b. 03.B1.03. 2.19.1268. 02000043. A5.01.12. 4.0.4.294. 6.01 0x80003554 1.1747.0. 6.01 0x80003554 1.1747.0. 2.1.14-k. -s13-t31-tg1. 10.30.50.45. 2.0b. 03.B1.03. 2.19.1268. 02000043. A5.01.12. 4.0.4.294. 6.01 0x80003554 1.1747.0. 6.01 0x80003554 1.1747.0. 2.1.14-k. -s14-t32-sut1. 10.30.50.55. 2.0b. 03.B1.03. 2.19.1268. 02000043. A5.01.12. 4.0.4.294. 6.01 0x80003554 1.1747.0. 6.01 0x80003554 1.1747.0. 2.1.14-k. -s15-t32-sut2. 10.30.50.56. 2.0b. 03.B1.03. 2.19.1268. 02000043. A5.01.12. 4.0.4.294. 6.01 0x80003554 1.1747.0. 6.01 0x80003554 1.1747.0. 2.1.14-k. -s16-t32-tg1. 10.30.50.57. 2.0b. 03.B1.03. 2.19.1268. 02000043. A5.01.12. 4.0.4.294. 6.01 0x80003554 1.1747.0. 6.01 0x80003554 1.1747.0. 2.1.14-k. -s19-t33t34-tg1. 10.30.50.46. 2.0b. 03.B1.03. 2.19.1268. 02000043. A5.01.12. 4.0.4.294. 6.01 0x80003554 1.1747.0. 6.01 0x80003554 1.1747.0. 2.1.14-k. -``` diff --git a/docs/lab/testbed_specifications.md b/docs/lab/testbed_specifications.md new file mode 100644 index 0000000000..b12d42d22f --- /dev/null +++ b/docs/lab/testbed_specifications.md @@ -0,0 +1,1469 @@ +<!-- TOC depthFrom:1 depthTo:6 withLinks:1 updateOnSave:1 orderedList:0 --> + + - [FD.io CSIT Testbed Specifications](#fdio-csit-testbed-specifications) + - [Testbeds Overview](#testbeds-overview) + - [Summary List](#summary-list) + - [1-Node-Skylake Xeon Intel (1n-skx)](#1-node-skylake-xeon-intel-1n-skx) + - [1-Node-ThunderX2 Arm Marvell (1n-tx2)](#1-node-thunderx2-arm-marvell-1n-tx2) + - [2-Node-Skylake Xeon Intel (2n-skx)](#2-node-skylake-xeon-intel-2n-skx) + - [2-Node-Denverton Atom Intel (2n-dnv)](#2-node-denverton-atom-intel-2n-dnv) + - [2-Node-IxiaPS1L47 Ixia PSOne L47 (2n-ps1)](#2-node-ixiaps1l47-ixia-psone-l47-2n-ps1) + - [3-Node-Haswell Xeon Intel (3n-skx)](#3-node-haswell-xeon-intel-3n-skx) + - [3-Node-Skylake Xeon Intel (3n-skx)](#3-node-skylake-xeon-intel-3n-skx) + - [3-Node-TaiShan Arm Huawei (3n-tsh)](#3-node-taishan-arm-huawei-3n-tsh) + - [3-Node-MACCHIATObin Arm Marvell](#3-node-macchiatobin-arm-marvell) + - [3-Node-Rangeley Atom Testbeds](#3-node-rangeley-atom-testbeds) + - [Server Management](#server-management) + - [Requirements](#requirements) + - [Addressing](#addressing) + - [LOM (IPMI) VLAN IP Addresses](#lom-ipmi-vlan-ip-addresses) + - [Management VLAN IP Addresses](#management-vlan-ip-addresses) + - [Server Type Specification](#server-type-specification) + - [Server and Port Naming](#server-and-port-naming) + - [Testbeds Configuration](#testbeds-configuration) + - [Per Testbed Server Allocation and Naming](#per-testbed-server-allocation-and-naming) + - [1-Node-Skylake (1n-skx) PROD](#1-node-skylake-1n-skx-prod) + - [1-Node-Thunderx2 (1n-tx2) WIP](#1-node-thunderx2-1n-tx2-wip) + - [2-Node-Skylake (2n-skx) PROD](#2-node-skylake-2n-skx-prod) + - [2-Node-Denverton (2n-dnv) TODO](#2-node-denverton-2n-dnv-todo) + - [2-Node-IxiaPS1L47 (2n-ps1) VERIFY](#2-node-ixiaps1l47-2n-ps1-verify) + - [3-Node-Haswell (3n-hsw) PROD](#3-node-haswell-3n-hsw-prod) + - [3-Node-Skylake (3n-skx) PROD](#3-node-skylake-3n-skx-prod) + - [3-Node-Rangeley (3n-rng) VERIFY](#3-node-rangeley-3n-rng-verify) + - [3-Node-Taishan (3n-tsh) WIP](#3-node-taishan-3n-tsh-wip) + - [3-Node-Mcbin (3n-mcb) TODO](#3-node-mcbin-3n-mcb-todo) + - [Per Testbed Wiring](#per-testbed-wiring) + - [1-Node-Skylake (1n-skx) PROD](#1-node-skylake-1n-skx-prod) + - [1-Node-Thunderx2 (1n-tx2) WIP](#1-node-thunderx2-1n-tx2-wip) + - [2-Node-Skylake (2n-skx) PROD](#2-node-skylake-2n-skx-prod) + - [2-Node-Denverton (2n-dnv) TODO](#2-node-denverton-2n-dnv-todo) + - [2-Node-IxiaPS1L47 (2n-ps1) VERIFY](#2-node-ixiaps1l47-2n-ps1-verify) + - [3-Node-Haswell (3n-hsw) PROD](#3-node-haswell-3n-hsw-prod) + - [3-Node-Skylake (3n-skx) PROD](#3-node-skylake-3n-skx-prod) + - [3-Node-Rangeley (3n-rng) TODO](#3-node-rangeley-3n-rng-todo) + - [3-Node-Taishan (3n-tsh) WIP](#3-node-taishan-3n-tsh-wip) + - [3-Node-Mcbin (3n-mcb) WIP](#3-node-mcbin-3n-mcb-wip) + - [Inventory](#inventory) + - [Appliances](#appliances) + - [Arm Servers](#arm-servers) + - [Xeon and Atom Servers](#xeon-and-atom-servers) + - [Network Interface Cards](#network-interface-cards) + - [Pluggables and Cables](#pluggables-and-cables) + - [Other Parts](#other-parts) + +<!-- /TOC --> +## FD.io CSIT Testbed Specifications + +This note includes specification of the physical testbed infrastructure +hosted by LFN FD.io CSIT project. + +## Testbeds Overview + +### Summary List + +``` + #. CSIT_tb Purpose SUT TG #TB #SUT #TG #hsw #skx #ps1 #rng #dnv #tx2 #tsh #mcb + 1. 1-Node-Skylake func skx na 2 2 0 0 2 0 0 0 0 0 0 + 2. 1-Node-Thunderx2 func tx2 na 1 1 0 0 0 0 0 0 1 0 0 + 3. 2-Node-Skylake perf skx skx 4 4 4 0 8 0 0 0 0 0 0 + 4. 2-Node-Denverton perf dnv skx 1 1 1 0 .5 0 0 1 0 0 0 + 5. 2-Node-IxiaPS1L47 tcp skx ps1 1 1 1 0 1 1 0 0 0 0 0 + 6. 3-Node-Haswell perf hsw hsw 3 6 3 9 0 0 0 0 0 0 0 + 7. 3-Node-Skylake perf skx skx 2 4 2 0 6 0 0 0 0 0 0 + 8. 3-Node-Rangeley perf rng skx 1 3 1 0 0 0 2 0 0 0 0 + 9. 3-Node-Taishan perf tsh skx 1 2 1 0 .5 0 0 0 0 2 0 +10. 3-Node-Mcbin perf mcb skx 1 2 1 0 .5 0 0 0 0 0 2 +11. 1-Node-VIRL func hsw --- 3 3 0 3 0 0 0 0 0 0 0 + Totals: 17 26 14 12 19 1 2 1 1 2 2 +``` + +### 1-Node-Skylake Xeon Intel (1n-skx) + +Each 1-Node-Skylake testbed includes one SUT (Type-B6 server) with NIC +ports connected back-to-back. +Used for FD.io VPP_Device functional driver tests. + +### 1-Node-ThunderX2 Arm Marvell (1n-tx2) + +Each 1-Node-ThunderX2 testbed includes one SUT (Type-B9 server) with NIC +ports connected back-to-back. +Used for FD.io VPP_Device functional driver tests. + +### 2-Node-Skylake Xeon Intel (2n-skx) + +Each 2-Node-Skylake testbed includes one SUT (Type-B1 server) and one TG +(Type-B2 server) connected back-to-back. +Used for FD.io performance tests. + +### 2-Node-Denverton Atom Intel (2n-dnv) + +Each 2-Node-Skylake testbed includes one SUT (Type-B10 server) and one TG +(Type-B2 server) connected back-to-back. +Used for FD.io performance tests. + +### 2-Node-IxiaPS1L47 Ixia PSOne L47 (2n-ps1) + +Each 2-Node-IxiaPS1L47 testbed includes one SUT (Type-B1 server) and one +TG (Ixia PSOne appliance) with 10GE interfaces connected back-to-back. +Used for FD.io TCP/IP and HTTP performance tests. + +### 3-Node-Haswell Xeon Intel (3n-skx) + +### 3-Node-Skylake Xeon Intel (3n-skx) + +Each 3-Node-Skylake testbed includes two SUTs (Type-B1 servers) and one +TG (Type-B2 server) connected in full-mesh triangle. +Used for FD.io performance tests. + +### 3-Node-TaiShan Arm Huawei (3n-tsh) + +Each 3-Node-TaiShan testbed includes two SUTs (Type-B3 server) and one TG +(Type-B2 server) connected in full-mesh triangle. +Used for FD.io performance tests. + +### 3-Node-MACCHIATObin Arm Marvell + +Each 3-Node-MACCHIATObin testbed includes two SUTs (Type-B4 server) and +one TG (Type-B2 server) connected in full-mesh triangle. +Used for FD.io performance tests. + +### 3-Node-Rangeley Atom Testbeds + +Each 3-Node-Rangeley testbed includes two SUTs (Type-B5 Netgate device) +and one TG (Type-2 server) connected in full-mesh triangle. +Used for FD.io performance tests. + +## Server Management + +### Requirements + +For management purposes, each server must have following two ports +connected to the management network: + +``` +- 1GE IPMI port + - IPMI - Intelligent Platform Management Interface. + - Required for access to embedded server management with WebUI, CLI, SNMPv3, + IPMIv2.0, for firmware (BIOS) and OS updates. +- 1GE/10GE/40GE management port + - hostOS management port for general system management. +``` + +### Addressing + +Each server has a LOM (Lights-Out-Management e.g. SM IPMI) and a Management +port, which are connected to two different VLANs. + +``` +1. LOM (IPMI) VLAN: + - Subnet: 10.30.50.0/24 + - Gateway: 10.30.50.1 + - Broadcast: 10.30.50.255 + - DNS1: 199.204.44.24 + - DNS2: 199.204.47.54 +2. Management Vlan: + - Subnet: 10.30.51.0/24 + - Gateway: 10.30.51.1 + - Broadcast: 10.30.51.255 + - DNS1: 199.204.44.24 + - DNS2: 199.204.47.54 +``` + +To access these hosts, VPN connection is required. + +### LOM (IPMI) VLAN IP Addresses + +Name | Comment +------------ | ------- +10.30.50.0 | network +10.30.51.1 | Router +10.30.50.2 | LF Reserved +10.30.50.3 | LF Reserved +10.30.50.4 | LF Reserved +10.30.50.5 | LF Reserved +10.30.50.6 | LF Reserved +10.30.50.7 | LF Reserved +10.30.50.8 | LF Reserved +10.30.50.9 | LF Reserved +10.30.50.10 | LF Reserved +10.30.50.11 | LF Reserved +10.30.50.12 | LF Reserved +10.30.50.13 | LF Reserved +10.30.50.14 | LF Reserved +10.30.50.15 | LF Reserved +10.30.50.16 | t1-tg1 +10.30.50.17 | t1-sut1 +10.30.50.18 | t1-sut2 +10.30.50.20 | t2-tg1 +10.30.50.21 | t2-sut1 +10.30.50.22 | t2-sut2 +10.30.50.24 | t3-tg1 +10.30.50.25 | t3-sut1 +10.30.50.26 | t3-sut-2 +10.30.50.28 | t4-sut1 +10.30.50.29 | t4-sut2 +10.30.50.30 | t4-sut3 +10.30.50.36 | s17-t33-sut1 +10.30.50.37 | s18-t33-sut2 +10.30.50.41 | s3-t21-sut1 +10.30.50.42 | s4-t21-tg1 +10.30.50.43 | s11-t31-sut1 +10.30.50.44 | s12-t31-sut2 +10.30.50.45 | s13-t31-tg1 +10.30.50.46 | s19-t33t34-tg1 +10.30.50.47 | s1-t11-sut1 +10.30.50.48 | s2-t12-sut1 +10.30.50.49 | s5-t22-sut1 +10.30.50.50 | s6-t22-tg1 +10.30.50.51 | s7-t23-sut1 +10.30.50.52 | s8-t23-tg1 +10.30.50.53 | s9-t24-sut1 +10.30.50.54 | s10-t24-tg1 +10.30.50.55 | s14-t32-sut1 +10.30.50.56 | s15-t32-sut2 +10.30.50.57 | s16-t32-tg1 +10.30.50.58 | s25-t25-sut1 +10.30.50.59 | s26-t25-tg1 +10.30.50.69 | s27-t13-sut +10.30.50.255 | Broadcast + +### Management VLAN IP Addresses + +Name | Comment +------------------------- | ------- +10.30.51.0 | network +10.30.51.1 | Router +10.30.51.2 | LF Reserved +10.30.51.3 | LF Reserved +10.30.51.4 | LF Reserved +10.30.51.5 | LF Reserved +10.30.51.6 | LF Reserved +10.30.51.7 | LF Reserved +10.30.51.8 | LF Reserved +10.30.51.9 | s22-t35-sut1 (netgate-1) +10.30.51.10 | s23-t35-sut2 (netgate-2) +10.30.51.11 | s24-t35-sut3 (netgate-3) +10.30.51.12 | softiron-1 +10.30.51.13 | softiron-2 +10.30.51.14 | softiron-3 +10.30.51.15 | LF Reserved +10.30.51.16 | t1-tg1 +10.30.51.17 | t1-sut1 +10.30.51.18 | t1-sut2 +10.30.51.20 | t2-tg1 +10.30.51.21 | t2-sut1 +10.30.51.22 | t2-sut2 +10.30.51.24 | t3-tg1 +10.30.51.25 | t3-sut1 +10.30.51.26 | t3-sut-2 +10.30.51.28 | t4-sut1 +10.30.51.29 | t4-sut2 +10.30.51.29 | s22-t35-sut1 screen -r /dev/ttyUSB0, TO BE VERIFIED +10.30.51.30 | t4-sut3 +10.30.51.30 | s23-t35-sut2 screen -r /dev/ttyUSB1, TO BE VERIFIED +10.30.51.30 | s24-t35-sut3 screen -r /dev/ttyUSB2, TO BE VERIFIED +10.30.51.36 | s17-t33-sut1 +10.30.51.37 | s18-t33-sut2 +10.30.51.41 | s20-t34-sut1 +10.30.51.42 | s21-t34-sut2 +10.30.51.44 | s3-t21-sut1 +10.30.51.45 | s4-t21-tg1 +10.30.51.46 | s11-t31-sut1 +10.30.51.47 | s12-t31-sut2 +10.30.51.48 | s13-t31-tg1 +10.30.51.49 | s19-t33t34-tg1 +10.30.51.50 | s1-t11-sut1 +10.30.51.51 | s2-t12-sut1 +10.30.51.52 | s5-t22-sut1 +10.30.51.53 | s6-t22-tg1 +10.30.51.54 | s7-t23-sut1 +10.30.51.55 | s8-t23-tg1 +10.30.51.56 | s9-t24-sut1 +10.30.51.57 | s10-t24-tg1 +10.30.51.58 | s14-t32-sut1 +10.30.51.59 | s15-t32-sut2 +10.30.51.60 | s16-t32-tg1 +10.30.51.61 | s25-t25-sut1 +10.30.51.62 | s26-t25-tg1 +10.30.51.69 | s27-t13-sut1 +10.30.51.70-10.30.51.105 | VIRL1 TO BE VERIFIED +10.30.51.106-10.30.51.180 | VIRL2 +10.30.51.181-10.30.51.254 | VIRL3 +10.30.51.255 | Broadcast + +## Server Type Specification + +FD.io CSIT lab contains following server types: +``` +1. Type-A1: Purpose - Haswell Xeon SUT (Systems Under Test) for FD.io performance testing. + - Quantity: 6 servers. + - Physical connectivity: + - CIMC and host management ports. + - NIC ports connected in 3-node topologies. + - Main HW configuration: + - Chassis: UCSC-C240-M4SX with 6 PCIe3.0 slots. + - Processors: 2* E5-2699v3 2.3 GHz. + - RAM Memory: 16* 32GB DDR4-2133MHz. + - Disks: 2* 2TB 12G SAS 7.2K RPM SFF HDD. + - NICs configuration: + - Numa0: Right PCIe Riser Board (Riser 1) (x8, x8, x8 PCIe3.0 lanes) + - PCIe Slot1: Cisco VIC 1385 2p40GE. + - PCIe Slot2: Intel NIC x520 2p10GE. + - PCIe Slot3: empty. + - Numa1: Left PCIe Riser Board (Riser 2) (x8, x16, x8 PCIe3.0 lanes) + - PCIe Slot4: Intel NIC xl710 2p40GE. + - PCIe Slot5: Intel NIC x710 2p10GE. + - PCIe Slot6: Intel QAT 8950 50G (Walnut Hill) + - MLOM slot: Cisco VIC 1227 2p10GE (x8 PCIe2.0 lanes). +2. Type-A2: Purpose - Haswell Xeon TG (Traffic Generators) for FD.io performance testing. + - Quantity: 3 servers. + - Physical connectivity: + - CIMC and host management ports. + - NIC ports connected in 3-node topologies. + - Main HW configuration: + - Chassis: UCSC-C240-M4SX with 6 PCIe3.0 slots. + - Processors: 2* E5-2699v3 2.3 GHz. + - RAM Memory: 16* 32GB DDR4-2133MHz. + - Disks: 2* 2TB 12G SAS 7.2K RPM SFF HDD. + - NICs configuration: + - Numa0: Right PCIe Riser Board (Riser 1) (x8, x8, x8 lanes) + - PCIe Slot1: Intel NIC xl710 2p40GE. + - PCIe Slot2: Intel NIC x710 2p10GE. + - PCIe Slot3: Intel NIC x710 2p10GE. + - Numa1: Left PCIe Riser Board (Riser 2) (x8, x16, x8 lanes) + - PCIe Slot4: Intel NIC xl710 2p40GE. + - PCIe Slot5: Intel NIC x710 2p10GE. + - PCIe Slot6: Intel NIC x710 2p10GE. + - MLOM slot: empty. +3. Type-A3: Purpose - Haswell Xeon VIRL hosts for FD.io functional testing. + - Quantity: 3 servers. + - Physical connectivity: + - CIMC and host management ports. + - no NIC ports, standalone setup. + - Main HW configuration: + - Chassis: UCSC-C240-M4SX with 6 PCIe3.0 slots. + - Processors: 2* E5-2699v3 2.3 GHz. + - RAM Memory: 16* 32GB DDR4-2133MHz. + - Disks: 2* 480 GB 2.5inch 6G SATA SSD. + - NICs configuration: + - Numa0: Right PCIe Riser Board (Riser 1) (x8, x8, x8 lanes) + - no cards. + - Numa1: Left PCIe Riser Board (Riser 2) (x8, x16, x8 lanes) + - no cards. + - MLOM slot: empty. +4. Type-B1: Purpose - Skylake Xeon SUT for FD.io performance testing. + - Quantity: --- + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 2-node and 3-node topologies. + - Main HW configuration: + - Chassis: SuperMicro SYS-7049GP-TRT. + - Motherboard: SuperMicro X11DPG-QT. + - Processors: 2* Intel Platinum 8180 2.3 GHz. + - RAM Memory: 16* 16GB DDR4-2666MHz. + - Disks: 2* 1.6TB 6G SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe3.0 lanes) + - PCIe Slot2 18:00.xx: x710-4p10GE Intel. + - PCIe Slot4 3b:00.xx: xxv710-DA2-2p25GE Intel. + - PCIe Slot9 5e:00.xx: FUTURE ConnectX5-2p100GE Mellanox. + - Numa1: (x16, x16, x16 PCIe3.0 lanes) + - PCIe Slot6 86:00.xx: empty. + - PCIe Slot8 af:00.xx: empty. + - PCIe Slot10 d8:00.xx: empty. +2. Type-B2: Purpose - Skylake Xeon TG for FD.io performance testing. + - Quantity: --- + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 2-node and 3-node topologies. + - Main HW configuration: + - Chassis: SuperMicro SYS-7049GP-TRT. + - Motherboard: SuperMicro X11DPG-QT. + - Processors: 2* Intel Platinum 8180 2.3 GHz. + - RAM Memory: 16* 16GB DDR4-2666MHz. + - Disks: 2* 1.6TB 6G SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe3.0 lanes) + - PCIe Slot2 18:00.xx: x710-4p10GE Intel. + - PCIe Slot4 3b:00.xx: xxv710-DA2 2p25GE Intel. + - PCIe Slot9 5e:00.xx: FUTURE ConnectX5-2p100GE Mellanox. + - Numa1: (x16, x16, x16 PCIe3.0 lanes) + - PCIe Slot6 86:00.xx: empty. + - PCIe Slot8 af:00.xx: empty. + - PCIe Slot10 d8:00.xx: x710-4p10GE Intel. +3. Type-B3: Purpose - TaiShan Arm Huawei SUT for FD.io performance testing. + - Quantity: 2 + - Physical connectivity: + - IPMI(?) and host management ports. + - NIC ports connected into 3-node topology. + - Main HW configuration: + - Chassis: Huawei TaiShan 2280. + - Processors: 1* hip07-d05 ~ 64* Arm Cortex-A72 + - RAM Memory: 8* 16GB DDR4-2400MT/s + - Disks: 1* 4TB SATA HDD + - NICs configuration: + - PCIe Slot4 e9:00.xx: connectx4-2p25GE Mellanox. + - PCIe Slot6 11:00.xx: 82599-2p10GE Intel. +4. Type-B4: Purpose - MACCHIATObin Arm Marvell SUT for FD.io performance testing. + - Quantity: 3 + - Physical connectivity: + - Host management ports. + - NIC ports connected into 2-node and 3-node topologies. + - Main HW configuration: + - Chassis: MACCHIATObin. + - Processors: 1* Armada 8040 ~ 4* Arm Cortex-A72 + - RAM Memory: 1* 16GB DDR4 + - Disks: 1* 128GB(?) SATA SDD + - NICs configuration: + - pp2-2p10GE Marvell (on-chip Ethernet ports ; marvell plugin in VPP) +5. Type-B5: Purpose - Rangeley Atom SUT for FD.io performance testing. + - Quantity: TBD based on testbed allocation. + - Physical connectivity: + - Management: serial Port (usb) for console + - NIC ports connected into 2-node. + - Main HW configuration: + - Chassis: Netgate XG-2758-1u + - Processors: 1* Rangeley (Atom) C2758 2.4 GHz + - RAM Memory: 16GB ECC + - Disks: 150 GB + - NICs configuration: + - 2x 10Gb Intel 82599ES + - 4x 1GB Intel I354 +6. Type-B6: Purpose - Skylake Xeon SUT for FD.io VPP_Device functional tests. + - Quantity: 2. + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 2-node and 3-node topologies. + - Main HW configuration: + - Chassis: SuperMicro SYS-7049GP-TRT. + - Motherboard: SuperMicro X11DPG-QT. + - Processors: 2* Intel Platinum 8180 2.3 GHz. + - RAM Memory: 16* 16GB DDR4-2666MHz. + - Disks: 2* 1.6TB 6G SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe3.0 lanes) + - PCIe Slot2 18:00.xx: x710-4p10GE Intel. + - PCIe Slot4 3b:00.xx: x710-4p10GE Intel. + - PCIe Slot9 5e:00.xx: empty. + - Numa1: (x16, x16, x16 PCIe3.0 lanes) + - PCIe Slot6 86:00.xx: empty. + - PCIe Slot8 af:00.xx: empty. + - PCIe Slot10 d8:00.xx: empty. +7. Type-B7: Purpose - Ixia PerfectStorm One Appliance TG for FD.io TCP/IP performance tests. + - Quantity: 1. + - Physical connectivity: + - Host management interface: 10/100/1000-BaseT. + - 8-port 10GE SFP+ integrated NIC. + - Main HW configuration: + - Chassis: PS10GE4NG. + - Motherboard: SuperMicro X11DPG-QT. + - Processors: Quad-Core, Intel Processor. + - HW accelerators: FPGA offload. + - RAM Memory: 64GB. + - Disks: 1 * 1 TB, Enterprise Class, High MTBF. + - Physical Interfaces: 4 * 10GE SFP+. + - Operating System: Native IxOS. + - Interface configuration: + - Port-1: 10GE SFP+. + - Port-2: 10GE SFP+. + - Port-3: 10GE SFP+. + - Port-4: 10GE SFP+. +8. Type-B8: Purpose - Skylake Xeon SUT for TCP/IP host stack tests. + - Quantity: 1. + - Physical connectivity: + - IPMI and host management ports. + - NIC ports. + - Main HW configuration: + - Chassis: SuperMicro SYS-7049GP-TRT. + - Motherboard: SuperMicro X11DPG-QT. + - Processors: 2* Intel Platinum 8180 2.3 GHz. + - RAM Memory: 16* 16GB DDR4-2666MHz. + - Disks: 2* 1.6TB 6G SATA SSD. + - NICs configuration: + - Numa0: (x16, x16, x16 PCIe3.0 lanes) + - PCIe Slot2 18:00.xx: x710-4p10GE Intel. + - PCIe Slot4 3b:00.xx: empty. + - PCIe Slot9 5e:00.xx: empty. + - Numa1: (x16, x16, x16 PCIe3.0 lanes) + - PCIe Slot6 86:00.xx: empty. + - PCIe Slot8 af:00.xx: empty. + - PCIe Slot10 d8:00.xx: empty. +9. Type-B9: Purpose - ThunderX2 Arm Marvell SUT for FD.io VPP_Device functional tests. + - Quantity: 1 + - Physical connectivity: + - IPMI and host management ports. + - NIC ports connected into 1-node topologies. + - Main HW configuration: + - Chassis: Gigabyte R181-T90 1U + - Motheboard: MT91-FS1 + - Processors: 1* ThunderX2 ARMv8 CN9975 2.0 GHz + - RAM Memory: 4* 32GB RDIMM + - Disks: 1* 480GB SSD Micron, 1* 1000GB HDD Seagate_25 + - NICs configuration: + - Numa0: + - PCIe Slot1 05:00.xx: XL710-QDA2. + - PCIe Slot3 08:00.xx: XL710-QDA2. + - Numa1: + - PCIe Slot6 85:00.xx: XL710-QDA2. +10. Type-B10: Purpose - Denverton Atom SUT for FD.io performance testing. + - Quantity: 4 + - Physical connectivity: + - IPMI and host management ports. + - NIC ports. + - Main HW configuration: + - Chassis: SuperMicro SYS-E300-9A + - Processors: 1* Intel(R) Atom(TM) CPU C3858 @ 2.00GHz + - RAM Memory: 32GB ECC + - Disks: 480 GB + - NICs configuration: + - 2x 10Gb Intel x553 fiber ports + - 2x 10Gb Intel x553 copper ports + - 4x 1GB Intel I350 ports +``` + +### Server and Port Naming + +Following naming convention is used within this page to specify physical +connectivity and wiring across defined CSIT testbeds: + +``` +- testbedname: testbedN. +- hostname: + - traffic-generator: tN-tgW. + - system-under-testX: tN-sutX. +- portnames: + - tN-tgW-cY/pZ. + - tN-sutX-cY/pZ. +- where: + - N - testbed number. + - tgW - server acts as traffic-generator with W index. + - sutX - server acts as system-under-test with X index. + - Y - PCIe slot number denoting a NIC card number within the host. + - Z - port number on the NIC card. +``` + +## Testbeds Configuration + +### Per Testbed Server Allocation and Naming + +#### 1-Node-Skylake (1n-skx) PROD + +``` +- ServerB1 [Type-B6]: + - testbedname: testbed11. + - hostname: s1-t11-sut1. + - IPMI IP: 10.30.50.47 + - Host IP: 10.30.51.50 + - portnames: + - s1-t11-sut1-c2/p1 - 10GE-port1 x710-4p10GE. + - s1-t11-sut1-c2/p2 - 10GE-port2 x710-4p10GE. + - s1-t11-sut1-c2/p3 - 10GE-port3 x710-4p10GE. + - s1-t11-sut1-c2/p4 - 10GE-port4 x710-4p10GE. + - s1-t11-sut1-c4/p1 - 10GE-port1 x710-4p10GE. + - s1-t11-sut1-c4/p2 - 10GE-port2 x710-4p10GE. + - s1-t11-sut1-c4/p3 - 10GE-port3 x710-4p10GE. + - s1-t11-sut1-c4/p4 - 10GE-port4 x710-4p10GE. +- ServerB2 [Type-B6]: + - testbedname: testbed12. + - hostname: s2-t12-sut1. + - IPMI IP: 10.30.50.48 + - Host IP: 10.30.51.51 + - portnames: + - s2-t12-sut1-c2/p1 - 10GE-port1 x710-4p10GE. + - s2-t12-sut1-c2/p2 - 10GE-port2 x710-4p10GE. + - s2-t12-sut1-c2/p3 - 10GE-port3 x710-4p10GE. + - s2-t12-sut1-c2/p4 - 10GE-port4 x710-4p10GE. + - s2-t12-sut1-c4/p1 - 10GE-port1 x710-4p10GE. + - s2-t12-sut1-c4/p2 - 10GE-port2 x710-4p10GE. + - s2-t12-sut1-c4/p3 - 10GE-port3 x710-4p10GE. + - s2-t12-sut1-c4/p4 - 10GE-port4 x710-4p10GE. +``` + +#### 1-Node-Thunderx2 (1n-tx2) WIP + +``` +- ServerB17 [Type-B9]: + - testbedname: testbed13. + - hostname: s27-t13-sut1. + - IPMI IP: 10.30.50.69 + - Host IP: 10.30.51.69 + - portnames: + - s27-t13-sut1-c1/p1 - 40GE-port1 XL710-QDA2-2p40GE. + - s27-t13-sut1-c1/p2 - 40GE-port2 XL710-QDA2-2p40GE. + - s27-t13-sut1-c3/p1 - 40GE-port1 XL710-QDA2-2p40GE. + - s27-t13-sut1-c3/p2 - 40GE-port2 XL710-QDA2-2p40GE. + - s27-t13-sut1-c6/p1 - 40GE-port1 XL710-QDA2-2p40GE. + - s27-t13-sut1-c6/p2 - 40GE-port2 XL710-QDA2-2p40GE. +``` + +#### 2-Node-Skylake (2n-skx) PROD + +``` +- ServerB3 [Type-B1]: + - testbedname: testbed21. + - hostname: s3-t21-sut1. + - IPMI IP: 10.30.50.41 + - Host IP: 10.30.51.44 + - portnames: + - s3-t21-sut1-c2/p1 - 10GE-port1 x710-4p10GE. + - s3-t21-sut1-c2/p2 - 10GE-port2 x710-4p10GE. + - s3-t21-sut1-c2/p3 - 10GE-port3 x710-4p10GE. + - s3-t21-sut1-c2/p4 - 10GE-port4 x710-4p10GE. + - s3-t21-sut1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s3-t21-sut1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s3-t21-sut1-c9/p1 - FUTURE 100GE-port1 ConnectX5-2p100GE. + - s3-t21-sut1-c9/p2 - FUTURE 100GE-port2 ConnectX5-2p100GE. +- ServerB4 [Type-B2]: + - testbedname: testbed21. + - hostname: s4-t21-tg1. + - IPMI IP: 10.30.50.42 + - Host IP: 10.30.51.45 + - portnames: + - s4-t21-tg1-c2/p1 - 10GE-port1 x710-4p10GE. + - s4-t21-tg1-c2/p2 - 10GE-port2 x710-4p10GE. + - s4-t21-tg1-c2/p3 - 10GE-port3 x710-4p10GE. + - s4-t21-tg1-c2/p4 - 10GE-port4 x710-4p10GE. + - s4-t21-tg1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s4-t21-tg1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s4-t21-tg1-c9/p1 - FUTURE 100GE-port1 ConnectX5-2p100GE. + - s4-t21-tg1-c9/p2 - FUTURE 100GE-port2 ConnectX5-2p100GE. +- ServerB5 [Type-B1]: + - testbedname: testbed22. + - hostname: s5-t22-sut1. + - IPMI IP: 10.30.50.49 + - Host IP: 10.30.51.52 + - portnames: + - s5-t22-sut1-c2/p1 - 10GE-port1 x710-4p10GE. + - s5-t22-sut1-c2/p2 - 10GE-port2 x710-4p10GE. + - s5-t22-sut1-c2/p3 - 10GE-port3 x710-4p10GE. + - s5-t22-sut1-c2/p4 - 10GE-port4 x710-4p10GE. + - s5-t22-sut1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s5-t22-sut1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s5-t22-sut1-c9/p1 - FUTURE 100GE-port1 ConnectX5-2p100GE. + - s5-t22-sut1-c9/p2 - FUTURE 100GE-port2 ConnectX5-2p100GE. +- ServerB6 [Type-B2]: + - testbedname: testbed22. + - hostname: s6-t22-tg1. + - IPMI IP: 10.30.50.50 + - Host IP: 10.30.51.53 + - portnames: + - s6-t22-tg1-c2/p1 - 10GE-port1 x710-4p10GE. + - s6-t22-tg1-c2/p2 - 10GE-port2 x710-4p10GE. + - s6-t22-tg1-c2/p3 - 10GE-port3 x710-4p10GE. + - s6-t22-tg1-c2/p4 - 10GE-port4 x710-4p10GE. + - s6-t22-tg1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s6-t22-tg1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s6-t22-tg1-c9/p1 - FUTURE 100GE-port1 ConnectX5-2p100GE. + - s6-t22-tg1-c9/p2 - FUTURE 100GE-port2 ConnectX5-2p100GE. +- ServerB7 [Type-B1]: + - testbedname: testbed23. + - hostname: s7-t23-sut1. + - IPMI IP: 10.30.50.51 + - Host IP: 10.30.51.54 + - portnames: + - s7-t23-sut1-c2/p1 - 10GE-port1 x710-4p10GE. + - s7-t23-sut1-c2/p2 - 10GE-port2 x710-4p10GE. + - s7-t23-sut1-c2/p3 - 10GE-port3 x710-4p10GE. + - s7-t23-sut1-c2/p4 - 10GE-port4 x710-4p10GE. + - s7-t23-sut1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s7-t23-sut1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s7-t23-sut1-c9/p1 - FUTURE 100GE-port1 ConnectX5-2p100GE. + - s7-t23-sut1-c9/p2 - FUTURE 100GE-port2 ConnectX5-2p100GE. +- ServerB8 [Type-B2]: + - testbedname: testbed23. + - hostname: s8-t23-tg1. + - IPMI IP: 10.30.50.52 + - Host IP: 10.30.51.55 + - portnames: + - s8-t23-tg1-c2/p1 - 10GE-port1 x710-4p10GE. + - s8-t23-tg1-c2/p2 - 10GE-port2 x710-4p10GE. + - s8-t23-tg1-c2/p3 - 10GE-port3 x710-4p10GE. + - s8-t23-tg1-c2/p4 - 10GE-port4 x710-4p10GE. + - s8-t23-tg1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s8-t23-tg1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s8-t23-tg1-c9/p1 - FUTURE 100GE-port1 ConnectX5-2p100GE. + - s8-t23-tg1-c9/p2 - FUTURE 100GE-port2 ConnectX5-2p100GE. +- ServerB9 [Type-B1]: + - testbedname: testbed24. + - hostname: s9-t24-sut1. + - IPMI IP: 10.30.50.53 + - Host IP: 10.30.51.56 + - portnames: + - s9-t24-sut1-c2/p1 - 10GE-port1 x710-4p10GE. + - s9-t24-sut1-c2/p2 - 10GE-port2 x710-4p10GE. + - s9-t24-sut1-c2/p3 - 10GE-port3 x710-4p10GE. + - s9-t24-sut1-c2/p4 - 10GE-port4 x710-4p10GE. + - s9-t24-sut1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s9-t24-sut1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s9-t24-sut1-c9/p1 - FUTURE 100GE-port1 ConnectX5-2p100GE. + - s9-t24-sut1-c9/p2 - FUTURE 100GE-port2 ConnectX5-2p100GE. +- ServerB10 [Type-B2]: + - testbedname: testbed24. + - hostname: s10-t24-tg1. + - IPMI IP: 10.30.50.54 + - Host IP: 10.30.51.57 + - portnames: + - s10-t24-tg1-c2/p1 - 10GE-port1 x710-4p10GE. + - s10-t24-tg1-c2/p2 - 10GE-port2 x710-4p10GE. + - s10-t24-tg1-c2/p3 - 10GE-port3 x710-4p10GE. + - s10-t24-tg1-c2/p4 - 10GE-port4 x710-4p10GE. + - s10-t24-tg1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s10-t24-tg1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s10-t24-tg1-c9/p1 - FUTURE 100GE-port1 ConnectX5-2p100GE. + - s10-t24-tg1-c9/p2 - FUTURE 100GE-port2 ConnectX5-2p100GE. +``` + +#### 2-Node-Denverton (2n-dnv) TODO + +``` +To be completed. +``` + +#### 2-Node-IxiaPS1L47 (2n-ps1) VERIFY + +``` +- ServerB25 [Type-B8]: + - testbedname: testbed25. + - hostname: s25-t25-sut1. + - IPMI IP: 10.30.50.58 + - Host IP: 10.30.51.61 + - portnames: + - s25-t25-sut1-c2/p1 - 10GE-port1 x710-4p10GE. + - s25-t25-sut1-c2/p2 - 10GE-port2 x710-4p10GE. + - s25-t25-sut1-c2/p3 - 10GE-port3 x710-4p10GE. + - s25-t25-sut1-c2/p4 - 10GE-port4 x710-4p10GE. +- ServerB26 [Type-B7]: + - testbedname: testbed25. + - hostname: s26-t25-tg1. + - IPMI IP: 10.30.50.59 + - Host IP: 10.30.51.62 + - portnames: + - s26-t25-tg1-p1 - 10GE-port1. + - s26-t25-tg1-p2 - 10GE-port2. + - s26-t25-tg1-p3 - 10GE-port3. + - s26-t25-tg1-p4 - 10GE-port4. +``` + +#### 3-Node-Haswell (3n-hsw) PROD + +``` + 1. Server1 of Type-1: + - testbedname: testbed1. + - hostname: t1-sut1. + - CIMC IP: 10.30.50.17 + - Host IP: 10.30.51.17 + - portnames: + - t1-sut1-c1/p1 - 10GE port1 on Intel NIC x520 2p10GE. + - t1-sut1-c1/p2 - 10GE port2 on Intel NIC x520 2p10GE. + - t1-sut1-c2/p1 - 40GE port1 on Cisco VIC 1385 2p40GE. + - t1-sut1-c2/p2 - 40GE port2 on Cisco VIC 1385 2p40GE. + - t1-sut1-c4/p1 - 40GE port1 on Intel NIC xl710 2p40GE. + - t1-sut1-c4/p2 - 40GE port2 on Intel NIC xl710 2p40GE. + - t1-sut1-c5/p1 - 10GE port1 on Intel NIC x710 2p10GE. + - t1-sut1-c5/p2 - 10GE port2 on Intel NIC x710 2p10GE. + - t1-sut1-cm/p1 - 10GE port1 on Cisco VIC 1227 2p10GE. + - t1-sut1-cm/p2 - 10GE port2 on Cisco VIC 1227 2p10GE. + 2. Server2 of Type-1: + - testbedname: testbed1. + - hostname: t1-sut2. + - CIMC IP: 10.30.50.18 + - Host IP: 10.30.51.18 + - portnames: + - t1-sut2-c1/p1 - 10GE port1 on Intel NIC x520 2p10GE. + - t1-sut2-c1/p2 - 10GE port2 on Intel NIC x520 2p10GE. + - t1-sut2-c2/p1 - 40GE port1 on Cisco VIC 1385 2p40GE. + - t1-sut2-c2/p2 - 40GE port2 on Cisco VIC 1385 2p40GE. + - t1-sut2-c4/p1 - 40GE port1 on Intel NIC xl710 2p40GE. + - t1-sut2-c4/p2 - 40GE port2 on Intel NIC xl710 2p40GE. + - t1-sut2-c5/p1 - 10GE port1 on Intel NIC x710 2p10GE. + - t1-sut2-c5/p2 - 10GE port2 on Intel NIC x710 2p10GE. + - t1-sut2-cm/p1 - 10GE port1 on Cisco VIC 1227 2p10GE. + - t1-sut2-cm/p2 - 10GE port2 on Cisco VIC 1227 2p10GE. + 3. Server3 of Type-2: + - testbedname: testbed1. + - hostname: t1-tg1. + - CIMC IP: 10.30.50.16 + - Host IP: 10.30.51.16 + - portnames: + - t1-tg1-c1/p1 - 40GE port1 on Intel NIC xl710 2p40GE. + - t1-tg1-c1/p2 - 40GE port2 on Intel NIC xl710 2p40GE. + - t1-tg1-c2/p1 - 10GE port1 on Intel NIC x710 2p10GE. + - t1-tg1-c2/p2 - 10GE port2 on Intel NIC x710 2p10GE. + - t1-tg1-c3/p1 - 10GE port1 on Intel NIC x710 2p10GE. + - t1-tg1-c3/p2 - 10GE port2 on Intel NIC x710 2p10GE. + - t1-tg1-c4/p1 - 40GE port1 on Intel NIC xl710 2p40GE. + - t1-tg1-c4/p2 - 40GE port2 on Intel NIC xl710 2p40GE. + - t1-tg1-c5/p1 - 10GE port1 on Intel NIC x710 2p10GE. + - t1-tg1-c5/p2 - 10GE port2 on Intel NIC x710 2p10GE. + - t1-tg1-c6/p1 - 10GE port1 on Intel NIC x710 2p10GE. + - t1-tg1-c6/p2 - 10GE port2 on Intel NIC x710 2p10GE. + 4. Server4 of Type-1: + - testbedname: testbed2. + - hostname: t2-sut1. + - CIMC IP: 10.30.50.21 + - Host IP: 10.30.51.21 + - portnames: + - t2-sut1-c1/p1 - 10GE port1 on Intel NIC x520 2p10GE. + - t2-sut1-c1/p2 - 10GE port2 on Intel NIC x520 2p10GE. + - t2-sut1-c2/p1 - 40GE port1 on Cisco VIC 1385 2p40GE. + - t2-sut1-c2/p2 - 40GE port2 on Cisco VIC 1385 2p40GE. + - t2-sut1-c4/p1 - 40GE port1 on Intel NIC xl710 2p40GE. + - t2-sut1-c4/p2 - 40GE port2 on Intel NIC xl710 2p40GE. + - t2-sut1-c5/p1 - 10GE port1 on Intel NIC x710 2p10GE. + - t2-sut1-c5/p2 - 10GE port2 on Intel NIC x710 2p10GE. + - t2-sut1-cm/p1 - 10GE port1 on Cisco VIC 1227 2p10GE. + - t2-sut1-cm/p2 - 10GE port2 on Cisco VIC 1227 2p10GE. + 5. Server5 of Type-1: + - testbedname: testbed2. + - hostname: t2-sut2. + - CIMC IP: 10.30.50.22 + - Host IP: 10.30.51.22 + - portnames: + - t2-sut2-c1/p1 - 10GE port1 on Intel NIC x520 2p10GE. + - t2-sut2-c1/p2 - 10GE port2 on Intel NIC x520 2p10GE. + - t2-sut2-c2/p1 - 40GE port1 on Cisco VIC 1385 2p40GE. + - t2-sut2-c2/p2 - 40GE port2 on Cisco VIC 1385 2p40GE. + - t2-sut2-c4/p1 - 40GE port1 on Intel NIC xl710 2p40GE. + - t2-sut2-c4/p2 - 40GE port2 on Intel NIC xl710 2p40GE. + - t2-sut2-c5/p1 - 10GE port1 on Intel NIC x710 2p10GE. + - t2-sut2-c5/p2 - 10GE port2 on Intel NIC x710 2p10GE. + - t2-sut2-cm/p1 - 10GE port1 on Cisco VIC 1227 2p10GE. + - t2-sut2-cm/p2 - 10GE port2 on Cisco VIC 1227 2p10GE. + 6. Server6 of Type-2: + - testbedname: testbed2. + - hostname: t2-tg1. + - CIMC IP: 10.30.50.20 + - Host IP: 10.30.51.20 + - portnames: + - t2-tg1-c1/p1 - 40GE port1 on Intel NIC xl710 2p40GE. + - t2-tg1-c1/p2 - 40GE port2 on Intel NIC xl710 2p40GE. + - t2-tg1-c2/p1 - 10GE port1 on Intel NIC x710 2p10GE. + - t2-tg1-c2/p2 - 10GE port2 on Intel NIC x710 2p10GE. + - t2-tg1-c3/p1 - 10GE port1 on Intel NIC x710 2p10GE. + - t2-tg1-c3/p2 - 10GE port2 on Intel NIC x710 2p10GE. + - t2-tg1-c4/p1 - 40GE port1 on Intel NIC xl710 2p40GE. + - t2-tg1-c4/p2 - 40GE port2 on Intel NIC xl710 2p40GE. + - t2-tg1-c5/p1 - 10GE port1 on Intel NIC x710 2p10GE. + - t2-tg1-c5/p2 - 10GE port2 on Intel NIC x710 2p10GE. + - t2-tg1-c6/p1 - 10GE port1 on Intel NIC x710 2p10GE. + - t2-tg1-c6/p2 - 10GE port2 on Intel NIC x710 2p10GE. + 7. Server7 of Type-1: + - testbedname: testbed3. + - hostname: t3-sut1. + - CIMC IP: 10.30.50.25 + - Host IP: 10.30.51.25 + - portnames: + - t3-sut1-c1/p1 - 10GE port1 on Intel NIC x520 2p10GE. + - t3-sut1-c1/p2 - 10GE port2 on Intel NIC x520 2p10GE. + - t3-sut1-c2/p1 - 40GE port1 on Cisco VIC 1385 2p40GE. + - t3-sut1-c2/p2 - 40GE port2 on Cisco VIC 1385 2p40GE. + - t3-sut1-c4/p1 - 40GE port1 on Intel NIC xl710 2p40GE. + - t3-sut1-c4/p2 - 40GE port2 on Intel NIC xl710 2p40GE. + - t3-sut1-c5/p1 - 10GE port1 on Intel NIC x710 2p10GE. + - t3-sut1-c5/p2 - 10GE port2 on Intel NIC x710 2p10GE. + - t3-sut1-cm/p1 - 10GE port1 on Cisco VIC 1227 2p10GE. + - t3-sut1-cm/p2 - 10GE port2 on Cisco VIC 1227 2p10GE. + 8. Server8 of Type-1: + - testbedname: testbed3. + - hostname: t3-sut2. + - CIMC IP: 10.30.50.26 + - Host IP: 10.30.51.26 + - portnames: + - t3-sut2-c1/p1 - 10GE port1 on Intel NIC x520 2p10GE. + - t3-sut2-c1/p2 - 10GE port2 on Intel NIC x520 2p10GE. + - t3-sut2-c2/p1 - 40GE port1 on Cisco VIC 1385 2p40GE. + - t3-sut2-c2/p2 - 40GE port2 on Cisco VIC 1385 2p40GE. + - t3-sut2-c4/p1 - 40GE port1 on Intel NIC xl710 2p40GE. + - t3-sut2-c4/p2 - 40GE port2 on Intel NIC xl710 2p40GE. + - t3-sut2-c5/p1 - 10GE port1 on Intel NIC x710 2p10GE. + - t3-sut2-c5/p2 - 10GE port2 on Intel NIC x710 2p10GE. + - t3-sut2-cm/p1 - 10GE port1 on Cisco VIC 1227 2p10GE. + - t3-sut2-cm/p2 - 10GE port2 on Cisco VIC 1227 2p10GE. + 9. Server9 of Type-2: + - testbedname: testbed3. + - hostname: t3-tg1. + - CIMC IP: 10.30.50.24 + - Host IP: 10.30.51.24 + - portnames: + - t3-tg1-c1/p1 - 40GE port1 on Intel NIC xl710 2p40GE. + - t3-tg1-c1/p2 - 40GE port2 on Intel NIC xl710 2p40GE. + - t3-tg1-c2/p1 - 10GE port1 on Intel NIC x710 2p10GE. + - t3-tg1-c2/p2 - 10GE port2 on Intel NIC x710 2p10GE. + - t3-tg1-c3/p1 - 10GE port1 on Intel NIC x710 2p10GE. + - t3-tg1-c3/p2 - 10GE port2 on Intel NIC x710 2p10GE. + - t3-tg1-c4/p1 - 40GE port1 on Intel NIC xl710 2p40GE. + - t3-tg1-c4/p2 - 40GE port2 on Intel NIC xl710 2p40GE. + - t3-tg1-c5/p1 - 10GE port1 on Intel NIC x710 2p10GE. + - t3-tg1-c5/p2 - 10GE port2 on Intel NIC x710 2p10GE. + - t3-tg1-c6/p1 - 10GE port1 on Intel NIC x710 2p10GE. + - t3-tg1-c6/p2 - 10GE port2 on Intel NIC x710 2p10GE. +``` + +#### 3-Node-Skylake (3n-skx) PROD + +``` +- ServerB11 [Type-B1]: + - testbedname: testbed31. + - hostname: s11-t31-sut1. + - IPMI IP: 10.30.50.43 + - Host IP: 10.30.51.46 + - portnames: + - s11-t31-sut1-c2/p1 - 10GE-port1 x710-4p10GE. + - s11-t31-sut1-c2/p2 - 10GE-port2 x710-4p10GE. + - s11-t31-sut1-c2/p3 - 10GE-port3 x710-4p10GE. + - s11-t31-sut1-c2/p4 - 10GE-port4 x710-4p10GE. + - s11-t31-sut1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s11-t31-sut1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s11-t31-sut1-c9/p1 - FUTURE 100GE-port1 ConnectX5-2p100GE. + - s11-t31-sut1-c9/p2 - FUTURE 100GE-port2 ConnectX5-2p100GE. +- ServerB12 [Type-B1]: + - testbedname: testbed31. + - hostname: s12-t31-sut2. + - IPMI IP: 10.30.50.44 + - Host IP: 10.30.51.47 + - portnames: + - s12-t31-sut2-c2/p1 - 10GE-port1 x710-4p10GE. + - s12-t31-sut2-c2/p2 - 10GE-port2 x710-4p10GE. + - s12-t31-sut2-c2/p3 - 10GE-port3 x710-4p10GE. + - s12-t31-sut2-c2/p4 - 10GE-port4 x710-4p10GE. + - s12-t31-sut2-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s12-t31-sut2-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s12-t31-sut2-c9/p1 - FUTURE 100GE-port1 ConnectX5-2p100GE. + - s12-t31-sut2-c9/p2 - FUTURE 100GE-port2 ConnectX5-2p100GE. +- ServerB13 [Type-B2]: + - testbedname: testbed31. + - hostname: s13-t31-tg1. + - IPMI IP: 10.30.50.45 + - Host IP: 10.30.51.48 + - portnames: + - s13-t31-tg1-c2/p1 - 10GE-port1 x710-4p10GE. + - s13-t31-tg1-c2/p2 - 10GE-port2 x710-4p10GE. + - s13-t31-tg1-c2/p3 - 10GE-port3 x710-4p10GE. + - s13-t31-tg1-c2/p4 - 10GE-port4 x710-4p10GE. + - s13-t31-tg1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s13-t31-tg1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s13-t31-tg1-c9/p1 - FUTURE 100GE-port1 ConnectX5-2p100GE. + - s13-t31-tg1-c9/p2 - FUTURE 100GE-port2 ConnectX5-2p100GE. +- ServerB14 [Type-B1]: + - testbedname: testbed32. + - hostname: s14-t32-sut1. + - IPMI IP: 10.30.50.55 + - Host IP: 10.30.51.58 + - portnames: + - s14-t32-sut1-c2/p1 - 10GE-port1 x710-4p10GE. + - s14-t32-sut1-c2/p2 - 10GE-port2 x710-4p10GE. + - s14-t32-sut1-c2/p3 - 10GE-port3 x710-4p10GE. + - s14-t32-sut1-c2/p4 - 10GE-port4 x710-4p10GE. + - s14-t32-sut1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s14-t32-sut1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s14-t32-sut1-c9/p1 - FUTURE 100GE-port1 ConnectX5-2p100GE. + - s14-t32-sut1-c9/p2 - FUTURE 100GE-port2 ConnectX5-2p100GE. +- ServerB15 [Type-B1]: + - testbedname: testbed32. + - hostname: s15-t32-sut2. + - IPMI IP: 10.30.50.56 + - Host IP: 10.30.51.59 + - portnames: + - s15-t32-sut2-c2/p1 - 10GE-port1 x710-4p10GE. + - s15-t32-sut2-c2/p2 - 10GE-port2 x710-4p10GE. + - s15-t32-sut2-c2/p3 - 10GE-port3 x710-4p10GE. + - s15-t32-sut2-c2/p4 - 10GE-port4 x710-4p10GE. + - s15-t32-sut2-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s15-t32-sut2-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s15-t32-sut2-c9/p1 - FUTURE 100GE-port1 ConnectX5-2p100GE. + - s15-t32-sut2-c9/p2 - FUTURE 100GE-port2 ConnectX5-2p100GE. +- ServerB16 [Type-B2]: + - testbedname: testbed32. + - hostname: s16-t32-tg1. + - IPMI IP: 10.30.50.57 + - Host IP: 10.30.51.60 + - portnames: + - s16-t32-tg1-c2/p1 - 10GE-port1 x710-4p10GE. + - s16-t32-tg1-c2/p2 - 10GE-port2 x710-4p10GE. + - s16-t32-tg1-c2/p3 - 10GE-port3 x710-4p10GE. + - s16-t32-tg1-c2/p4 - 10GE-port4 x710-4p10GE. + - s16-t32-tg1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s16-t32-tg1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s16-t32-tg1-c9/p1 - FUTURE 100GE-port1 ConnectX5-2p100GE. + - s16-t32-tg1-c9/p2 - FUTURE 100GE-port2 ConnectX5-2p100GE. +``` + +Note: ServerB19 (TG) is shared between testbed33 & testbed34 + +#### 3-Node-Rangeley (3n-rng) VERIFY + +Note: There is no IPMI. Serial console is accessible via VIRL2 and VIRL3 USB. + +``` +- ServerB22 [Type-B5]: + - testbedname: testbed35. + - hostname: s22-t35-sut1 (vex-yul-rot-netgate-1). + - IPMI IP: 10.30.51.29 - screen -r /dev/ttyUSB0 + - Host IP: 10.30.51.9 + - portnames: + - s22-t35-sut1-p1 - 10GE-port1 ix0 82599. + - s22-t35-sut1-p2 - 10GE-port2 ix1 82599. + - 1GB ports (tbd) +- ServerB23 [Type-B5]: + - testbedname: testbed35. + - hostname: s23-t35-sut2 (vex-yul-rot-netgate-2). + - IPMI IP: 10.30.51.30 - screen -r /dev/ttyUSB1 + - Host IP: 10.30.51.10 + - portnames: + - s23-t35-sut1-p1 - 10GE-port1 ix0 82599. + - s23-t35-sut1-p2 - 10GE-port2 ix1 82599. + - 1GB ports (tbd) +- ServerB24 [Type-B5]: + - testbedname: testbed35. + - hostname: s24-t35-sut3 (vex-yul-rot-netgate-3). + - IPMI IP: 10.30.51.30 - screen -r /dev/ttyUSB2 + - Host IP: 10.30.51.11 + - portnames: + - s24-t35-sut1-p1 - 10GE-port1 ix0 82599. + - s24-t35-sut1-p2 - 10GE-port2 ix1 82599. + - 1GB ports (tbd) +``` + +#### 3-Node-Taishan (3n-tsh) WIP + +``` +- ServerB17 [Type-B3]: + - testbedname: testbed33. + - hostname: s17-t33-sut1. + - IPMI IP: 10.30.50.36 + - Host IP: 10.30.51.36 + - portnames: + - s17-t33-sut1-c6/p1 - 10GE-port1 82599-2p10GE. + - s17-t33-sut1-c6/p2 - 10GE-port2 82599-2p10GE. + - s17-t33-sut1-c4/p1 - 25GE-port1 cx4-2p25GE. + - s17-t33-sut1-c4/p2 - 25GE-port2 cx4-2p25GE. +- ServerB18 [Type-B3]: + - testbedname: testbed33. + - hostname: s18-t33-sut2. + - IPMI IP: 10.30.50.37 + - Host IP: 10.30.51.37 + - portnames: + - s18-t33-sut2-c6/p1 - 10GE-port1 82599-2p10GE. + - s18-t33-sut2-c6/p2 - 10GE-port2 82599-2p10GE. + - s18-t33-sut2-c4/p1 - 25GE-port1 cx4-2p25GE. + - s18-t33-sut2-c4/p2 - 25GE-port2 cx4-2p25GE. +- ServerB19 [Type-B2]: + - testbednames: testbed33 and testbed34. + - hostname: s19-t33t34-tg1. + - IPMI IP: 10.30.50.46 + - Host IP: 10.30.51.49 + - portnames: + - s19-t33t34-tg1-c2/p1 - 10GE-port1 x710-4p10GE. + - s19-t33t34-tg1-c2/p2 - 10GE-port2 x710-4p10GE. + - s19-t33t34-tg1-c2/p3 - 10GE-port3 x710-4p10GE. + - s19-t33t34-tg1-c2/p4 - 10GE-port4 x710-4p10GE. + - s19-t33t34-tg1-c4/p1 - 25GE-port1 xxv710-DA2-2p25GE. + - s19-t33t34-tg1-c4/p2 - 25GE-port2 xxv710-DA2-2p25GE. + - s19-t33t34-tg1-c10/p1 - 10GE-port1 x710-4p10GE. + - s19-t33t34-tg1-c10/p2 - 10GE-port2 x710-4p10GE. + - s19-t33t34-tg1-c10/p3 - 10GE-port3 x710-4p10GE. + - s19-t33t34-tg1-c10/p4 - 10GE-port4 x710-4p10GE. +- ServerB20 [Type-B4]: + - testbedname: testbed34. + - hostname: s20-t34-sut1. + - IPMI IP: N/A + - Host IP: 10.30.51.41 + - portnames: + - s20-t34-sut1-ca/p1 - 10GE-port1 Marvell. + - s20-t34-sut1-ca/p2 - 10GE-port2 Marvell. +- ServerB21 [Type-B4]: + - testbedname: testbed34. + - hostname: s21-t34-sut2. + - IPMI IP: N/A + - Host IP: 10.30.51.42 + - portnames: + - s21-t34-sut2-ca/p1 - 10GE-port1 Marvell. + - s21-t34-sut2-ca/p2 - 10GE-port2 Marvell. +``` + +#### 3-Node-Mcbin (3n-mcb) TODO + +``` +To be completed. +``` + +### Per Testbed Wiring + +#### 1-Node-Skylake (1n-skx) PROD + +``` +- testbed11: + - ring1 10GE-ports x710-4p10GE: + - s1-t11-sut1-c2/p1 to s1-t11-sut1-c4/p1. + - ring2 10GE-ports x710-4p10GE: + - s1-t11-sut1-c2/p2 to s1-t11-sut1-c4/p2. + - ring3 10GE-ports x710-4p10GE: + - s1-t11-sut1-c2/p3 to s1-t11-sut1-c4/p3. + - ring4 10GE-ports x710-4p10GE: + - s1-t11-sut1-c2/p3 to s1-t11-sut1-c4/p3. +- testbed12: + - ring1 10GE-ports x710-4p10GE: + - s2-t12-sut1-c2/p1 to s2-t12-sut1-c4/p1. + - ring2 10GE-ports x710-4p10GE: + - s2-t12-sut1-c2/p2 to s2-t12-sut1-c4/p2. + - ring3 10GE-ports x710-4p10GE: + - s2-t12-sut1-c2/p3 to s2-t12-sut1-c4/p3. + - ring4 10GE-ports x710-4p10GE: + - s2-t12-sut1-c2/p3 to s2-t12-sut1-c4/p3. +``` + +#### 1-Node-Thunderx2 (1n-tx2) WIP + +``` +- testbed13: + - ring1 40GE-ports XL710-QDA2-2p40GE on SUTs: + - s27-t13-sut1-c1/p2 - s27-t13-sut1-c3/p1. + - ring2 40GE-ports XL710-QDA2-2p40GE on SUTs: + - s27-t13-sut1-c3/p2 - s27-t13-sut1-c6/p1. + - ring3 40GE-ports XL710-QDA2-2p40GE on SUTs: + - s27-t13-sut1-c6/p2 - s27-t13-sut1-c1/p1. +``` + +#### 2-Node-Skylake (2n-skx) PROD + +``` +- testbed21: + - ring1 10GE-ports x710-4p10GE on SUT: + - s4-t21-tg1-c2/p1 to s3-t21-sut1-c2/p1. + - s3-t21-sut1-c2/p2 to s4-t21-tg1-c2/p2. + - ring2 10GE-ports x710-4p10GE on SUT: + - s4-t21-tg1-c2/p3 to s3-t21-sut1-c2/p3. + - s3-t21-sut1-c2/p4 to s4-t21-tg1-c2/p4. + - ring3 25GE-ports xxv710-DA2-2p25GE on SUT + - s4-t21-tg1-c4/p1 to s3-t21-sut1-c4/p1. + - s3-t21-sut1-c4/p2 to s4-t21-tg1-c4/p2. + - FUTURE ring4 100GE-ports ConnectX5-2p100GE on SUT: + - s4-t21-tg1-c9/p1 to s3-t21-sut1-c9/p1. + - s3-t21-sut1-c9/p2 to s4-t21-tg1-c9/p2. + - ring5 10GE-ports x710-4p10GE loopbacks on TG for self-tests: + - s4-t21-tg1-c10/p1 to s4-t21-tg1-c10/p2. + - s4-t21-tg1-c10/p3 to s4-t21-tg1-c10/p4. +- testbed22: + - ring1 10GE-ports x710-4p10GE on SUT: + - s6-t22-tg1-c2/p1 to s5-t22-sut1-c2/p1. + - s5-t22-sut1-c2/p2 to s6-t22-tg1-c2/p2. + - ring2 10GE-ports x710-4p10GE on SUT: + - s6-t22-tg1-c2/p3 to s5-t22-sut1-c2/p3. + - s5-t22-sut1-c2/p4 to s6-t22-tg1-c2/p4. + - ring3 25GE-ports xxv710-DA2-2p25GE on SUT + - s6-t22-tg1-c4/p1 to s5-t22-sut1-c4/p1. + - s5-t22-sut1-c4/p2 to s6-t22-tg1-c4/p2. + - FUTURE ring4 100GE-ports ConnectX5-2p100GE on SUT: + - t22-tg1-c9/p1 to s5-t22-sut1-c9/p1. + - s5-t22-sut1-c9/p2 to s6-t22-tg1-c9/p2. + - ring5 10GE-ports x710-4p10GE loopbacks on TG for self-tests: + - s6-t22-tg1-c10/p1 to s6-t22-tg1-c10/p2. + - s6-t22-tg1-c10/p3 to s6-t22-tg1-c10/p4. +- testbed23: + - ring1 10GE-ports x710-4p10GE on SUT: + - s8-t23-tg1-c2/p1 to s7-t23-sut1-c2/p1. + - s7-t23-sut1-c2/p2 to s8-t23-tg1-c2/p2. + - ring2 10GE-ports x710-4p10GE on SUT: + - s8-t23-tg1-c2/p3 to s7-t23-sut1-c2/p3. + - s7-t23-sut1-c2/p4 to s8-t23-tg1-c2/p4. + - ring3 25GE-ports xxv710-DA2-2p25GE on SUT + - s8-t23-tg1-c4/p1 to s7-t23-sut1-c4/p1. + - s7-t23-sut1-c4/p2 to s8-t23-tg1-c4/p2. + - FUTURE ring4 100GE-ports ConnectX5-2p100GE on SUT: + - s8-t23-tg1-c9/p1 to s7-t23-sut1-c9/p1. + - s7-t23-sut1-c9/p2 to s8-t23-tg1-c9/p2. + - ring5 10GE-ports x710-4p10GE loopbacks on TG for self-tests: + - s8-t23-tg1-c10/p1 to s8-t23-tg1-c10/p2. + - s8-t23-tg1-c10/p3 to s8-t23-tg1-c10/p4. +- testbed24: + - ring1 10GE-ports x710-4p10GE on SUT: + - s10-t24-tg1-c2/p1 to s9-t24-sut1-c2/p1. + - s9-t24-sut1-c2/p2 to s10-t24-tg1-c2/p2. + - ring2 10GE-ports x710-4p10GE on SUT: + - s10-t24-tg1-c2/p3 to s9-t24-sut1-c2/p3. + - s9-t24-sut1-c2/p4 to s10-t24-tg1-c2/p4. + - ring3 25GE-ports xxv710-DA2-2p25GE on SUT + - s10-t24-tg1-c4/p1 to s9-t24-sut1-c4/p1. + - s9-t24-sut1-c4/p2 to s10-t24-tg1-c4/p2. + - FUTURE ring4 100GE-ports ConnectX5-2p100GE on SUT: + - s10-t24-tg1-c9/p1 to s9-t24-sut1-c9/p1. + - s9-t24-sut1-c9/p2 to s10-t24-tg1-c9/p2. + - ring5 10GE-ports x710-4p10GE loopbacks on TG for self-tests: + - s10-t24-tg1-c10/p1 to s10-t24-tg1-c10/p2. + - s10-t24-tg1-c10/p3 to s10-t24-tg1-c10/p4. +``` + +#### 2-Node-Denverton (2n-dnv) TODO + +``` +To be completed. +``` + +#### 2-Node-IxiaPS1L47 (2n-ps1) VERIFY + +``` +- testbed25: + - link1 10GE-port x710-4p10GE on SUT: + - t25-tg1-p1 to t25-sut1-c2/p1. + - link2 10GE-port x710-4p10GE on SUT: + - t25-tg1-p2 to t25-sut1-c2/p2. + - link3 10GE-port x710-4p10GE on SUT: + - t25-tg1-p3 to t25-sut1-c2/p3. + - link4 10GE-port x710-4p10GE on SUT: + - t25-tg1-p4 to t25-sut1-c2/p4. +``` + +#### 3-Node-Haswell (3n-hsw) PROD + +``` + 1. testbed1: + - ring of 40GE ports on Cisco VIC 1385 2p40GE on SUTs + - t1-tg1-c1/p1 to t1-sut1-c1/p2. + - t1-sut1-c1/p1 to t1-sut2-c1/p2. + - t1-sut2-c1/p1 to t1-tg1-c1/p2. + - ring of 10GE ports on Intel NIC x520 2p10GE on SUTs + - t1-tg1-c2/p1 to t1-sut1-c2/p2. + - t1-sut1-c2/p1 to t1-sut2-c2/p2. + - t1-sut2-c2/p1 to t1-tg1-c2/p2. + - ring of 40GE ports on Intel NIC xl710 2p40GE on SUTs + - t1-tg1-c4/p1 to t1-sut1-c4/p2. + - t1-sut1-c4/p1 to t1-sut2-c4/p2. + - t1-sut2-c4/p1 to t1-tg1-c4/p2. + - ring of 10GE ports on Intel NIC x710 2p10GE on SUTs + - t1-tg1-c5/p1 to t1-sut1-c5/p2. + - t1-sut1-c5/p1 to t1-sut2-c5/p2. + - t1-sut2-c5/p1 to t1-tg1-c5/p2. + - ring of 10GE ports on Cisco VIC 1227 2p10GE on SUTs + - t1-tg1-c2/p1 to t1-sut1-cm/p2. + - t1-sut1-cm/p1 to t1-sut2-cm/p2. + - t1-sut2-cm/p1 to t1-tg1-c2/p2. + - TG loopback ports Intel NIC x710 2p10GE + - t1-tg1-c6/p1 to t1-tg1-c6/p2. + + 2. testbed2: + - ring of 40GE ports on Cisco VIC 1385 2p40GE on SUTs + - t2-tg1-c1/p1 to t2-sut1-c1/p2. + - t2-sut1-c1/p1 to t2-sut2-c1/p2. + - t2-sut2-c1/p1 to t2-tg1-c1/p2. + - ring of 10GE ports on Intel NIC x520 2p10GE on SUTs + - t2-tg1-c2/p1 to t2-sut1-c2/p2. + - t2-sut1-c2/p1 to t2-sut2-c2/p2. + - t2-sut2-c2/p1 to t2-tg1-c2/p2. + - ring of 40GE ports on Intel NIC xl710 2p40GE on SUTs + - t2-tg1-c4/p1 to t2-sut1-c4/p2. + - t2-sut1-c4/p1 to t2-sut2-c4/p2. + - t2-sut2-c4/p1 to t2-tg1-c4/p2. + - ring of 10GE ports on Intel NIC x710 2p10GE on SUTs + - t2-tg1-c5/p1 to t2-sut1-c5/p2. + - t2-sut1-c5/p1 to t2-sut2-c5/p2. + - t2-sut2-c5/p1 to t2-tg1-c5/p2. + - ring of 10GE ports on Cisco VIC 1227 2p10GE on SUTs + - t2-tg1-c2/p1 to t2-sut1-cm/p2. + - t2-sut1-cm/p1 to t2-sut2-cm/p2. + - t2-sut2-cm/p1 to t2-tg1-c2/p2. + - TG loopback ports Intel NIC x710 2p10GE + - t2-tg1-c6/p1 to t2-tg1-c6/p2. + + 3. testbed3: + - ring of 40GE ports on Cisco VIC 1385 2p40GE on SUTs + - t3-tg1-c1/p1 to t3-sut1-c1/p2. + - t3-sut1-c1/p1 to t3-sut2-c1/p2. + - t3-sut2-c1/p1 to t3-tg1-c1/p2. + - ring of 10GE ports on Intel NIC x520 2p10GE on SUTs + - t3-tg1-c2/p1 to t3-sut1-c2/p2. + - t3-sut1-c2/p1 to t3-sut2-c2/p2. + - t3-sut2-c2/p1 to t3-tg1-c2/p2. + - ring of 40GE ports on Intel NIC xl710 2p40GE on SUTs + - t3-tg1-c4/p1 to t3-sut1-c4/p2. + - t3-sut1-c4/p1 to t3-sut2-c4/p2. + - t3-sut2-c4/p1 to t3-tg1-c4/p2. + - ring of 10GE ports on Intel NIC x710 2p10GE on SUTs + - t3-tg1-c5/p1 to t3-sut1-c5/p2. + - t3-sut1-c5/p1 to t3-sut2-c5/p2. + - t3-sut2-c5/p1 to t3-tg1-c5/p2. + - ring of 10GE ports on Cisco VIC 1227 2p10GE on SUTs + - t3-tg1-c2/p1 to t3-sut1-cm/p2. + - t3-sut1-cm/p1 to t3-sut2-cm/p2. + - t3-sut2-cm/p1 to t3-tg1-c2/p2. + - TG loopback ports Intel NIC x710 2p10GE + - t3-tg1-c6/p1 to t3-tg1-c6/p2. +``` + +#### 3-Node-Skylake (3n-skx) PROD + +``` +- testbed31: + - ring1 10GE-ports x710-4p10GE on SUTs: + - s13-t31-tg1-c2/p1 to s11-t31-sut1-c2/p1. + - s11-t31-sut1-c2/p2 to s12-t31-sut2-c2/p2. + - s12-t31-sut2-c2/p1 to s13-t31-tg1-c2/p2. + - ring2 10GE-ports x710-4p10GE on SUT: + - s13-t31-tg1-c2/p3 to s11-t31-sut1-c2/p3. + - s11-t31-sut1-c2/p4 to s12-t31-sut2-c2/p4. + - s12-t31-sut2-c2/p3 to s13-t31-tg1-c2/p4. + - ring3 25GE-ports xxv710-DA2-2p25GE on SUT + - s13-t31-tg1-c4/p1 to s11-t31-sut1-c4/p1. + - s11-t31-sut1-c4/p2 to s12-t31-sut2-c4/p2. + - s12-t31-sut2-c4/p1 to s13-t31-tg1-c4/p2. + - FUTURE ring4 100GE-ports ConnectX5-2p100GE on SUT: + - s13-t31-tg1-c9/p1 to s11-t31-sut1-c9/p1. + - s11-t31-sut1-c9/p2 to s12-t31-sut2-c9/p2. + - s12-t31-sut2-c9/p1 to s13-t31-tg1-c9/p2. + - ring5 10GE-ports x710-4p10GE loopbacks on TG for self-tests: + - s13-t31-tg1-c10/p1 to s13-t31-tg1-c10/p2. + - s13-t31-tg1-c10/p3 to s13-t31-tg1-c10/p4. +- testbed32: + - ring1 10GE-ports x710-4p10GE on SUTs: + - s16-t32-tg1-c2/p1 to s14-t32-sut1-c2/p1. + - s14-t32-sut1-c2/p2 to s15-t32-sut2-c2/p2. + - s15-t32-sut2-c2/p1 to s16-t32-tg1-c2/p2. + - ring2 10GE-ports x710-4p10GE on SUT: + - s16-t32-tg1-c2/p3 to s14-t32-sut1-c2/p3. + - s14-t32-sut1-c2/p4 to s15-t32-sut2-c2/p4. + - s15-t32-sut2-c2/p3 to s16-t32-tg1-c2/p4. + - ring3 25GE-ports xxv710-DA2-2p25GE on SUT + - s16-t32-tg1-c4/p1 to s14-t32-sut1-c4/p1. + - s14-t32-sut1-c4/p2 to s15-t32-sut2-c4/p2. + - s15-t32-sut2-c4/p1 to s16-t32-tg1-c4/p2. + - FUTURE ring4 100GE-ports ConnectX5-2p100GE on SUT: + - s16-t32-tg1-c9/p1 to s14-t32-sut1-c9/p1. + - s14-t32-sut1-c9/p2 to s15-t32-sut2-c9/p2. + - s15-t32-sut2-c9/p1 to s16-t32-tg1-c9/p2. + - ring5 10GE-ports x710-4p10GE loopbacks on TG for self-tests: + - s16-t32-tg1-c10/p1 to s16-t32-tg1-c10/p2. + - s16-t32-tg1-c10/p3 to s16-t32-tg1-c10/p4. +``` + +#### 3-Node-Rangeley (3n-rng) TODO + +``` +To be completed. +``` + +#### 3-Node-Taishan (3n-tsh) WIP + +``` +- testbed33: + - ring1 10GE-ports 82599-2p10GE on SUTs: + - t33t34-tg1-c2/p2 - t33-sut1-c6/p2. + - t33-sut1-c6/p1 - t33-sut2-c6/p2. + - t33-sut2-c6/p1 - t33t34-tg1-c2/p1. + - ring2 25GE-ports cx4-2p25GE on SUTs: + - t33t34-tg1-c4/p2 - t33-sut1-c4/p2. + - t33-sut1-c4/p1 - t33-sut2-c4/p2. + - t33-sut2-c4/p1 - t33t34-tg1-c4/p1. +``` + +#### 3-Node-Mcbin (3n-mcb) WIP + +``` +- testbed34: + - ring1 10GE-ports Marvell on SUTs: + - t33t34-tg1-c2/p3 - t34-sut1-ca/p1. + - t34-sut1-ca/p2 - t34-sut2-ca/p1. + - t34-sut2-ca/p2 - t33t34-tg1-c2/p4. +``` + +## Inventory + +### Appliances + +``` +1. Ixia PerfectStorm One Appliance + - 1 * PS10GE4NG + - Chassis: PS10GE4NG. + - Motherboard: SuperMicro X11DPG-QT. + - Processors: Quad-Core, Intel Processor. + - HW accelerators: FPGA offload. + - RAM Memory: 64GB. + - Disks: 1 * 1 TB, Enterprise Class, High MTBF. + - Physical Interfaces: 4 * 10GE SFP+. + - Operating System: Native IxOS. +``` + +### Arm Servers + +``` +1. Arm Cortex A-72 servers + - 1 * ThunderX2 + - Chassis: Marvell ThunderX2 + - Processors: 2* ThunderX2 CN9975 ~ 112* ThunderX2. + - RAM Memory: 4* 32GB RDIMM + - Disks: 1* 480GB SSD Micron, 1* 1000GB HDD Seagate_25 + - 2 * Huawei TaiShan 2280. + - Chassis: Huawei TaiShan 2280. + - Processors: 1* hip07-d05 ~ 64* Arm Cortex-A72. + - RAM Memory: 8* 16GB DDR4-2400MT/s. + - Disks: 1* 4TB SATA HDD. + - 3 * MACCHIATObin + - Chassis: MACCHIATObin. + - Processors: 1* Armada 8040 ~ 4* Arm Cortex-A72. + - RAM Memory: 1* 16GB DDR4. + - Disks: 1* 128GB(?) SATA SDD. +``` + +### Xeon and Atom Servers + +``` +1. Intel Xeon servers: + - 20 * SuperMicro SYS-7049GP-TRT with Xeon Skylake processors. + - Chassis: SuperMicro SYS-7049GP-TRT. + - Motherboard: SuperMicro X11DPG-QT. + - Processors: 2* Intel Platinum 8180 2.3 GHz. + - RAM Memory: 16* 16GB DDR4-2666MHz. + - Disks: 2* 1.6TB 6G SATA SSD. +2. Intel Atom servers with Rangely processors. + - 3 * Netgate XG-2758-1u + - Chassis: Netgate XG-2758-1u + - Processors: 1* Rangely (Atom) C2758 2.4 GHz + - RAM Memory: 16GB ECC + - Disks: 150 GB +``` + +### Network Interface Cards + +``` +1. 10GE NICs + - 14 * Intel® Ethernet Converged Network Adapter X710-DA4 + - 6 * Intel® Ethernet Converged Network Adapter X710-DA2 + - 6 * Intel® Ethernet Converged Network Adapter X520-DA2 +2. 25GE NICs + - 12 * Intel® Ethernet Network Adapter XXV710-DA2 +3. 40GE NICs + - 2 * Intel® Ethernet Converged Network Adapter XL710-QDA2 +4. 100GE NICs + - 4 * mcx556a-edat NICs (not on site yet, in transit) +``` + +### Pluggables and Cables + +Pluggables: + +``` +1. 10GE SFP+ + - 16 * Intel E10GSFPSR Ethernet SFP+ SR Optics + - 80 * 10G SR optic (generic, "Intel" compatible branded) +2. 25GE SFP28 + - None +3. 40GE QSFP+ + - None +4. 100GE + - 8 * mcp1600-c002 qsfp28 pluggables and cables (not on site yet, in transit) +``` + +Standalone cables: + +``` +1. 10GE + - None +2. 25GE + - None +3. 40GE QSFP+ + - 20 * Intel XLDACBL5 40G QSFP+ Passive DAC Cable +4. 100GE + - None +``` + +### Other Parts + +None. diff --git a/docs/lab/testbeds_sm_skx_hw_bios_cfg.md b/docs/lab/testbeds_sm_skx_hw_bios_cfg.md new file mode 100644 index 0000000000..6a2bfd782d --- /dev/null +++ b/docs/lab/testbeds_sm_skx_hw_bios_cfg.md @@ -0,0 +1,544 @@ +<!-- TOC depthFrom:1 depthTo:6 withLinks:1 updateOnSave:1 orderedList:0 --> + + - [SuperMicro Xeon Skylake Servers - Hardware and BIOS Configuration](#supermicro-xeon-skylake-servers-hardware-and-bios-configuration) + - [Linux lscpu TODO](#linux-lscpu-todo) + - [Linux dmidecode pci TODO](#linux-dmidecode-pci-todo) + - [Linux dmidecode memory TODO](#linux-dmidecode-memory-todo) + - [Xeon Skx Server BIOS Configuration](#xeon-skx-server-bios-configuration) + - [Boot Feature](#boot-feature) + - [CPU Configuration](#cpu-configuration) + - [Advanced Power Management Configuration](#advanced-power-management-configuration) + - [CPU P State Control](#cpu-p-state-control) + - [Hardware PM State Control](#hardware-pm-state-control) + - [CPU C State Control](#cpu-c-state-control) + - [Package C State Control](#package-c-state-control) + - [CPU T State Control](#cpu-t-state-control) + - [Chipset Configuration](#chipset-configuration) + - [North Bridge](#north-bridge) + - [UPI Configuration](#upi-configuration) + - [Memory Configuration](#memory-configuration) + - [IIO Configuration](#iio-configuration) + - [CPU1 Configuration](#cpu1-configuration) + - [CPU2 Configuration](#cpu2-configuration) + - [South Bridge](#south-bridge) + - [PCIe/PCI/PnP Configuration](#pciepcipnp-configuration) + - [ACPI Settings](#acpi-settings) + - [DMIDECODE](#dmidecode) + - [Xeon Skx Server Firmware Inventory](#xeon-skx-server-firmware-inventory) + +<!-- /TOC --> + +## SuperMicro Xeon Skylake Servers - Hardware and BIOS Configuration + +### Linux lscpu TODO + +### Linux dmidecode pci TODO + +### Linux dmidecode memory TODO + +### Xeon Skx Server BIOS Configuration + +#### Boot Feature + +``` + | Quiet Boot [Enabled] |Boot option | + | | | + | Option ROM Messages [Force BIOS] | | + | Bootup NumLock State [On] | | + | Wait For "F1" If Error [Enabled] | | + | INT19 Trap Response [Immediate] | | + | Re-try Boot [Disabled] | | + | Install Windows 7 USB support [Disabled] | | + | Port 61h Bit-4 Emulation [Disabled] | | + | | | + | Power Configuration | | + | Watch Dog Function [Disabled] | | + | Restore on AC Power Loss [Last State] | | + | Power Button Function [Instant Off] | | + | Throttle on Power Fail [Disabled] | | +``` + +#### CPU Configuration + +``` + | Processor Configuration |Enables Hyper Threading | + | -------------------------------------------------- |(Software Method to | + | Processor BSP Revision 50654 - SKX H0 |Enable/Disable Logical | + | Processor Socket CPU1 | CPU2 |Processor threads. | + | Processor ID 00050654* | 000506... | | + | Processor Frequency 2.500GHz | 2.500GHz | | + | Processor Max Ratio 19H | 19H | | + | Processor Min Ratio 0AH | 0AH | | + | Microcode Revision 02000030 | | + | L1 Cache RAM 64KB | 64KB | | + | L2 Cache RAM 1024KB | 1024KB | | + | L3 Cache RAM 39424KB | 39424KB | | + | Processor 0 Version | | + | Intel(R) Xeon(R) Platinum 8180 CPU @ 2.50GHz | | + | Processor 1 Version | | + | Intel(R) Xeon(R) Platinum 8180 CPU @ 2.50GHz | | + | | | + | Hyper-Threading [ALL] [Enable] | | + | Core Disable Bitmap(Hex) 0 | | + | Execute Disable Bit [Enable] | | + | Intel Virtualization Technology [Enable] | | + | PPIN Control [Unlock/Enable] | | + | Hardware Prefetcher [Enable] | | + | Adjacent Cache Prefetch [Enable] | | + | DCU Streamer Prefetcher [Enable] | | + | DCU IP Prefetcher [Enable] | | + | LLC Prefetch [Disable] | | + | Extended APIC [Disable] | | + | AES-NI [Enable] | | + |> Advanced Power Management Configuration | | +``` + +##### Advanced Power Management Configuration + +``` + | Advanced Power Management Configuration |Switch CPU Power Management | + | -------------------------------------------------- |profile | + | Power Technology [Custom] | | + | Power Performance Tuning [BIOS Controls EPB] | | + | ENERGY_PERF_BIAS_CFG mode [Maximum Performance] | | + |> CPU P State Control | | + |> Hardware PM State Control | | + |> CPU C State Control | | + |> Package C State Control | | + |> CPU T State Control | | +``` + +###### CPU P State Control + +``` + | CPU P State Control |Enable/Disable EIST | + | |(P-States) | + | SpeedStep (Pstates) [Disable] | | + | EIST PSD Function [HW_ALL] | | +``` + +###### Hardware PM State Control + +``` + | Hardware PM State Control |Disable: Hardware chooses a | + | |P-state based on OS Request | + | Hardware P-States [Disable] |(Legacy P-States) | + | |Native Mode:Hardware | + | |chooses a P-state based on | + | |OS guidance | + | |Out of Band Mode:Hardware | + | |autonomously chooses a | + | |P-state (no OS guidance) | +``` + +###### CPU C State Control + +``` + | CPU C State Control |Autonomous Core C-State | + | |Control | + | Autonomous Core C-State [Disable] | | + | CPU C6 report [Disable] | | + | Enhanced Halt State (C1E) [Disable] | | +``` + +###### Package C State Control + +``` + | Package C State Control |Package C State limit | + | | | + | Package C State [C0/C1 state] | | +``` + +###### CPU T State Control + +``` + | CPU T State Control |Enable/Disable Software | + | |Controlled T-States | + | Software Controlled T-States [Disable] | | +``` + +##### Chipset Configuration + +``` + | WARNING: Setting wrong values in below sections may cause |North Bridge Parameters | + | system to malfunction. | | + |> North Bridge | | + |> South Bridge | | +``` + +###### North Bridge + +``` + |> UPI Configuration |Displays and provides | + |> Memory Configuration |option to change the UPI | + |> IIO Configuration |Settings | +``` + +###### UPI Configuration + +``` + | UPI Configuration |Choose Topology Precedence | + | -------------------------------------------------- |to degrade features if | + | Number of CPU 2 |system options are in | + | Number of Active UPI Link 3 |conflict or choose Feature | + | Current UPI Link Speed Fast |Precedence to degrade | + | Current UPI Link Frequency 10.4 GT/s |topology if system options | + | UPI Global MMIO Low Base / Limit 90000000 / FBFFFFFF |are in conflict. | + | UPI Global MMIO High Base / Limit 0000000000000000 / ... | | + | UPI Pci-e Configuration Base / Size 80000000 / 10000000 | | + | Degrade Precedence [Topology Precedence] | | + | Link L0p Enable [Disable] | | + | Link L1 Enable [Disable] | | + | IO Directory Cache (IODC) [Auto] | | + | SNC [Disable] | | + | XPT Prefetch [Disable] | | + | KTI Prefetch [Enable] | | + | Local/Remote Threshold [Auto] | | + | Stale AtoS [Disable] | | + | LLC dead line alloc [Enable] | | + | Isoc Mode [Auto] | | +``` + +###### Memory Configuration + +``` + | |POR - Enforces Plan Of | + | -------------------------------------------------- |Record restrictions for | + | Integrated Memory Controller (iMC) |DDR4 frequency and voltage | + | -------------------------------------------------- |programming. Disable - | + | |Disables this feature. | + | Enforce POR [Disable] | | + | Memory Frequency [2666] | | + | Data Scrambling for NVMDIMM [Auto] | | + | Data Scrambling for DDR4 [Auto] | | + | tCCD_L Relaxation [Auto] | | + | Memory tRWSR Relaxation [Enable] | | + | 2X REFRESH [Auto] | | + | Page Policy [Auto] | | + | IMC Interleaving [2-way Interleave] | | + |> Memory Topology | | + |> Memory RAS Configuration | | +``` + +###### IIO Configuration + +``` + | IIO Configuration |Expose IIO DFX devices and | + | -------------------------------------------------- |other CPU devices like PMON | + | | | + | EV DFX Features [Disable] | | + |> CPU1 Configuration | | + |> CPU2 Configuration | | + |> IOAT Configuration | | + |> Intel. VT for Directed I/O (VT-d) | | + |> Intel. VMD technology | | + | | | + | IIO-PCIE Express Global Options | | + | ======================================== | | + | PCI-E Completion Timeout Disable [No] | | +``` + +###### CPU1 Configuration + +``` + | IOU0 (IIO PCIe Br1) [Auto] |Selects PCIe port | + | IOU1 (IIO PCIe Br2) [Auto] |Bifurcation for selected | + | IOU2 (IIO PCIe Br3) [Auto] |slot(s) | + |> CPU1 SLOT2 PCI-E 3.0 X16 | | + |> CPU1 SLOT4 PCI-E 3.0 X16 | | + |> CPU1 SLOT9 PCI-E 3.0 X16 | | +``` + +###### CPU2 Configuration + +``` + | IOU0 (IIO PCIe Br1) [Auto] |Selects PCIe port | + | IOU1 (IIO PCIe Br2) [Auto] |Bifurcation for selected | + | IOU2 (IIO PCIe Br3) [Auto] |slot(s) | + |> CPU2 SLOT6 PCI-E 3.0 X16 | | + |> CPU2 SLOT8 PCI-E 3.0 X16 | | + |> CPU2 SLOT10 PCI-E 3.0 X16 | | +``` + +##### South Bridge + +``` + | |Enables Legacy USB support. | + | USB Module Version 17 |AUTO option disables legacy | + | |support if no USB devices | + | USB Devices: |are connected. DISABLE | + | 1 Keyboard, 1 Mouse, 1 Hub |option will keep USB | + | |devices available only for | + | Legacy USB Support [Enabled] |EFI applications. | + | XHCI Hand-off [Disabled] | | + | Port 60/64 Emulation [Enabled] | | + | PCIe PLL SSC [Disable] | | + | Real USB Wake Up [Enabled] | | + | Front USB Wake Up [Enabled] | | + | | | + | Azalia [Auto] | | + | Azalia PME Enable [Disabled] | | +``` + +#### PCIe/PCI/PnP Configuration + +``` + | PCI Bus Driver Version A5.01.12 |Enables or Disables 64bit | + | |capable Devices to be | + | PCI Devices Common Settings: |Decoded in Above 4G Address | + | Above 4G Decoding [Enabled] |Space (Only if System | + | SR-IOV Support [Enabled] |Supports 64 bit PCI | + | MMIO High Base [56T] |Decoding). | + | MMIO High Granularity Size [256G] | | + | Maximum Read Request [Auto] | | + | MMCFG Base [2G] | | + | NVMe Firmware Source [Vendor Defined Fi...] | | + | VGA Priority [Onboard] | | + | CPU1 SLOT2 PCI-E 3.0 X16 OPROM [Legacy] | | + | CPU1 SLOT4 PCI-E 3.0 X16 OPROM [Legacy] | | + | CPU2 SLOT6 PCI-E 3.0 X16 OPROM [Legacy] | | + | CPU2 SLOT8 PCI-E 3.0 X16 OPROM [Legacy] | | + | CPU1 SLOT9 PCI-E 3.0 X16 OPROM [Legacy] | | + | CPU2 SLOT10 PCI-E 3.0 X16 OPROM [Legacy] | | + | CPU2 SLOT11 PCI-E 3.0 X4(IN X8) OPROM [Legacy] | | + | M.2 CONNECTOR OPROM [Legacy] | | + | Onboard LAN1 Option ROM [Legacy] | | + | Onboard LAN2 Option ROM [Disabled] | | + | Onboard Video Option ROM [Legacy] | | + |> Network Stack Configuration | | +``` + +#### ACPI Settings + +``` + | ACPI Settings |Enable or Disable Non | + | |uniform Memory Access | + | NUMA [Enabled] |(NUMA). | + | WHEA Support [Enabled] | | + | High Precision Event Timer [Enabled] | | + | ACPI Sleep State [S3 (Suspend to RAM)] | | +``` + +#### DMIDECODE + +``` + # dmidecode 3.1 + Getting SMBIOS data from sysfs. + SMBIOS 3.1.1 present. + Table at 0x000E89C0. + + Handle 0x0000, DMI type 0, 26 bytes + BIOS Information + Vendor: American Megatrends Inc. + Version: 2.0 + Release Date: 11/29/2017 + Address: 0xF0000 + Runtime Size: 64 kB + ROM Size: 64 MB + Characteristics: + PCI is supported + BIOS is upgradeable + BIOS shadowing is allowed + Boot from CD is supported + Selectable boot is supported + BIOS ROM is socketed + EDD is supported + 5.25"/1.2 MB floppy services are supported (int 13h) + 3.5"/720 kB floppy services are supported (int 13h) + 3.5"/2.88 MB floppy services are supported (int 13h) + Print screen service is supported (int 5h) + Serial services are supported (int 14h) + Printer services are supported (int 17h) + ACPI is supported + USB legacy is supported + BIOS boot specification is supported + Targeted content distribution is supported + UEFI is supported + BIOS Revision: 5.12 + + Handle 0x0001, DMI type 1, 27 bytes + System Information + Manufacturer: Supermicro + Product Name: SYS-7049GP-TRT + Version: 0123456789 + Serial Number: S291427X8332242 + UUID: 00000000-0000-0000-0000-AC1F6B8A8DB6 + Wake-up Type: Power Switch + SKU Number: To be filled by O.E.M. + Family: To be filled by O.E.M. + + Handle 0x0002, DMI type 2, 15 bytes + Base Board Information + Manufacturer: Supermicro + Product Name: X11DPG-QT + Version: 1.02 + Serial Number: VM183S014930 + Asset Tag: To be filled by O.E.M. + Features: + Board is a hosting board + Board is replaceable + Location In Chassis: To be filled by O.E.M. + Chassis Handle: 0x0003 + Type: Motherboard + Contained Object Handles: 0 + + Handle 0x0003, DMI type 3, 22 bytes + Chassis Information + Manufacturer: Supermicro + Type: Other + Lock: Not Present + Version: 0123456789 + Serial Number: C7470KH06A20167 + Asset Tag: To be filled by O.E.M. + Boot-up State: Safe + Power Supply State: Safe + Thermal State: Safe + Security Status: None + OEM Information: 0x00000000 + + Handle 0x0050, DMI type 4, 48 bytes + Processor Information + Socket Designation: CPU1 + Type: Central Processor + Family: Xeon + Manufacturer: Intel(R) Corporation + ID: 54 06 05 00 FF FB EB BF + Signature: Type 0, Family 6, Model 85, Stepping 4 + Flags: + FPU (Floating-point unit on-chip) + VME (Virtual mode extension) + DE (Debugging extension) + PSE (Page size extension) + TSC (Time stamp counter) + MSR (Model specific registers) + PAE (Physical address extension) + MCE (Machine check exception) + CX8 (CMPXCHG8 instruction supported) + APIC (On-chip APIC hardware supported) + SEP (Fast system call) + MTRR (Memory type range registers) + PGE (Page global enable) + MCA (Machine check architecture) + CMOV (Conditional move instruction supported) + PAT (Page attribute table) + PSE-36 (36-bit page size extension) + CLFSH (CLFLUSH instruction supported) + DS (Debug store) + ACPI (ACPI supported) + MMX (MMX technology supported) + FXSR (FXSAVE and FXSTOR instructions supported) + SSE (Streaming SIMD extensions) + SSE2 (Streaming SIMD extensions 2) + SS (Self-snoop) + HTT (Multi-threading) + TM (Thermal monitor supported) + PBE (Pending break enabled) + Version: Intel(R) Xeon(R) Platinum 8180 CPU @ 2.50GHz + Voltage: 1.6 V + External Clock: 100 MHz + Max Speed: 4000 MHz + Current Speed: 2500 MHz + Status: Populated, Enabled + Upgrade: Other + L1 Cache Handle: 0x004D + L2 Cache Handle: 0x004E + L3 Cache Handle: 0x004F + Serial Number: Not Specified + Asset Tag: UNKNOWN + Part Number: Not Specified + Core Count: 28 + Core Enabled: 28 + Thread Count: 56 + Characteristics: + 64-bit capable + Multi-Core + Hardware Thread + Execute Protection + Enhanced Virtualization + Power/Performance Control + + + Handle 0x0054, DMI type 4, 48 bytes + Processor Information + Socket Designation: CPU2 + Type: Central Processor + Family: Xeon + Manufacturer: Intel(R) Corporation + ID: 54 06 05 00 FF FB EB BF + Signature: Type 0, Family 6, Model 85, Stepping 4 + Flags: + FPU (Floating-point unit on-chip) + VME (Virtual mode extension) + DE (Debugging extension) + PSE (Page size extension) + TSC (Time stamp counter) + MSR (Model specific registers) + PAE (Physical address extension) + MCE (Machine check exception) + CX8 (CMPXCHG8 instruction supported) + APIC (On-chip APIC hardware supported) + SEP (Fast system call) + MTRR (Memory type range registers) + PGE (Page global enable) + MCA (Machine check architecture) + CMOV (Conditional move instruction supported) + PAT (Page attribute table) + PSE-36 (36-bit page size extension) + CLFSH (CLFLUSH instruction supported) + DS (Debug store) + ACPI (ACPI supported) + MMX (MMX technology supported) + FXSR (FXSAVE and FXSTOR instructions supported) + SSE (Streaming SIMD extensions) + SSE2 (Streaming SIMD extensions 2) + SS (Self-snoop) + HTT (Multi-threading) + TM (Thermal monitor supported) + PBE (Pending break enabled) + Version: Intel(R) Xeon(R) Platinum 8180 CPU @ 2.50GHz + Voltage: 1.6 V + External Clock: 100 MHz + Max Speed: 4000 MHz + Current Speed: 2500 MHz + Status: Populated, Enabled + Upgrade: Other + L1 Cache Handle: 0x0051 + L2 Cache Handle: 0x0052 + L3 Cache Handle: 0x0053 + Serial Number: Not Specified + Asset Tag: UNKNOWN + Part Number: Not Specified + Core Count: 28 + Core Enabled: 28 + Thread Count: 56 + Characteristics: + 64-bit capable + Multi-Core + Hardware Thread + Execute Protection + Enhanced Virtualization + Power/Performance Control +``` + +### Xeon Skx Server Firmware Inventory + +``` +Host. IPMI IP. BIOS. CPLD. Aptio SU. CPU Microcode. PCI Bus. ME Operation FW. X710 Firmware. XXV710 Firmware. i40e. +s1-t11-sut1. 10.30.50.47. 2.1. 03.B1.03. 2.19.1268. 02000043. A5.01.12. 4.0.4.294. 6.01 0x80003554 1.1747.0. 6.01 0x80003554 1.1747.0. 2.1.14-k. +s2-t12-sut1. 10.30.50.48. 2.1. 03.B1.03. 2.19.1268. 02000043. A5.01.12. 4.0.4.294. 6.01 0x80003554 1.1747.0. 6.01 0x80003554 1.1747.0. 2.1.14-k. +s3-t21-sut1. 10.30.50.41. 2.0b. 03.B1.03. 2.19.1268. 02000043. A5.01.12. 4.0.4.294. 6.01 0x80003554 1.1747.0. 6.01 0x80003554 1.1747.0. 2.1.14-k. +s4-t21-tg1. 10.30.50.42. 2.0b. 03.B1.03. 2.19.1268. 02000043. A5.01.12. 4.0.4.294. 6.01 0x80003554 1.1747.0. 6.01 0x80003554 1.1747.0. 2.1.14-k. +s5-t22-sut1. 10.30.50.49. 2.0b. 03.B1.03. 2.19.1268. 02000043. A5.01.12. 4.0.4.294. 6.01 0x80003554 1.1747.0. 6.01 0x80003554 1.1747.0. 2.1.14-k. +s6-t22-tg1. 10.30.50.50. 2.0b. 03.B1.03. 2.19.1268. 02000043. A5.01.12. 4.0.4.294. 6.01 0x80003554 1.1747.0. 6.01 0x80003554 1.1747.0. 2.1.14-k. +s7-t23-sut1. 10.30.50.51. 2.0b. 03.B1.03. 2.19.1268. 02000043. A5.01.12. 4.0.4.294. 6.01 0x80003554 1.1747.0. 6.01 0x80003554 1.1747.0. 2.1.14-k. +s8-t23-tg1. 10.30.50.52. 2.0b. 03.B1.03. 2.19.1268. 02000043. A5.01.12. 4.0.4.294. 6.01 0x80003554 1.1747.0. 6.01 0x80003554 1.1747.0. 2.1.14-k. +s9-t24-sut1. 10.30.50.53. 2.0b. 03.B1.03. 2.19.1268. 02000043. A5.01.12. 4.0.4.294. 6.01 0x80003554 1.1747.0. 6.01 0x80003554 1.1747.0. 2.1.14-k. +s10-t24-tg1. 10.30.50.54. 2.0b. 03.B1.03. 2.19.1268. 02000043. A5.01.12. 4.0.4.294. 6.01 0x80003554 1.1747.0. 6.01 0x80003554 1.1747.0. 2.1.14-k. +s11-t31-sut1. 10.30.50.43. 2.0b. 03.B1.03. 2.19.1268. 02000043. A5.01.12. 4.0.4.294. 6.01 0x80003554 1.1747.0. 6.01 0x80003554 1.1747.0. 2.1.14-k. +s12-t31-sut2. 10.30.50.44. 2.0b. 03.B1.03. 2.19.1268. 02000043. A5.01.12. 4.0.4.294. 6.01 0x80003554 1.1747.0. 6.01 0x80003554 1.1747.0. 2.1.14-k. +s13-t31-tg1. 10.30.50.45. 2.0b. 03.B1.03. 2.19.1268. 02000043. A5.01.12. 4.0.4.294. 6.01 0x80003554 1.1747.0. 6.01 0x80003554 1.1747.0. 2.1.14-k. +s14-t32-sut1. 10.30.50.55. 2.0b. 03.B1.03. 2.19.1268. 02000043. A5.01.12. 4.0.4.294. 6.01 0x80003554 1.1747.0. 6.01 0x80003554 1.1747.0. 2.1.14-k. +s15-t32-sut2. 10.30.50.56. 2.0b. 03.B1.03. 2.19.1268. 02000043. A5.01.12. 4.0.4.294. 6.01 0x80003554 1.1747.0. 6.01 0x80003554 1.1747.0. 2.1.14-k. +s16-t32-tg1. 10.30.50.57. 2.0b. 03.B1.03. 2.19.1268. 02000043. A5.01.12. 4.0.4.294. 6.01 0x80003554 1.1747.0. 6.01 0x80003554 1.1747.0. 2.1.14-k. +s19-t33t34-tg1. 10.30.50.46. 2.0b. 03.B1.03. 2.19.1268. 02000043. A5.01.12. 4.0.4.294. 6.01 0x80003554 1.1747.0. 6.01 0x80003554 1.1747.0. 2.1.14-k. +``` diff --git a/docs/lab/Testbeds_Xeon_Hsw_VIRL.md b/docs/lab/testbeds_ucs_hsw_hw_bios_cfg.md index 5ec6b59215..ed4b1f561d 100644 --- a/docs/lab/Testbeds_Xeon_Hsw_VIRL.md +++ b/docs/lab/testbeds_ucs_hsw_hw_bios_cfg.md @@ -1,452 +1,14 @@ -## FD.io CSIT Testbeds - Xeon Haswell +<!-- TOC depthFrom:1 depthTo:6 withLinks:1 updateOnSave:1 orderedList:0 --> -This is a low-level design implemented as an original FD.io CSIT lab based on -Cisco UCS-c240m4 servers based on Intel Xeon Haswell processors. + - [Cisco UCS c240m4 Xeon Haswell Servers - Hardware and BIOS Configuration](#cisco-ucs-c240m4-xeon-haswell-servers-hardware-and-bios-configuration) + - [Linux lscpu](#linux-lscpu) + - [Linux dmidecode pci](#linux-dmidecode-pci) + - [Linux dmidecode memory](#linux-dmidecode-memory) + - [Xeon Hsw Server BIOS Configuration](#xeon-hsw-server-bios-configuration) -### Server Hardware Configuration +<!-- /TOC --> -Total of 10 Cisco UCS-c240 servers with Intel Xeon Haswell processors have been -used to built FD.io CSIT 3-Node Haswell (3n-hsw) performance testbeds and VIRL -functional testbeds. Following three HW configuration types of UCS x86 servers -are used: - -``` - 1. Type-1: Purpose - VPP functional and performance conformance testing. - - Quantity: 6 computers as SUT hosts (Systems Under Test). - - Physical connectivity: - - CIMC and host management ports. - - NIC ports connected in 3-node topologies. - - Main HW configuration: - - Chassis: UCSC-C240-M4SX with 6 PCIe3.0 slots. - - Processors: 2* E5-2699v3 2.3 GHz. - - RAM Memory: 16* 32GB DDR4-2133MHz. - - Disks: 2* 2TB 12G SAS 7.2K RPM SFF HDD. - - NICs configuration: - - Right PCIe Riser Board (Riser 1) (x8, x8, x8 PCIe3.0 lanes) - - PCIe Slot1: Cisco VIC 1385 2p40GE. - - PCIe Slot2: Intel NIC x520 2p10GE. - - PCIe Slot3: empty. - - Left PCIe Riser Board (Riser 2) (x8, x16, x8 PCIe3.0 lanes) - - PCIe Slot4: Intel NIC xl710 2p40GE. - - PCIe Slot5: Intel NIC x710 2p10GE. - - PCIe Slot6: Intel QAT 8950 50G (Walnut Hill) - - MLOM slot: Cisco VIC 1227 2p10GE (x8 PCIe2.0 lanes). - 2. Type-2: Purpose - VPP functional and performance conformance testing. - - Quantity: 3 computers as TG hosts (Traffic Generators). - - Physical connectivity: - - CIMC and host management ports. - - NIC ports connected in 3-node topologies. - - Main HW configuration: - - Chassis: UCSC-C240-M4SX with 6 PCIe3.0 slots. - - Processors: 2* E5-2699v3 2.3 GHz. - - RAM Memory: 16* 32GB DDR4-2133MHz. - - Disks: 2* 2TB 12G SAS 7.2K RPM SFF HDD. - - NICs configuration: - - Right PCIe Riser Board (Riser 1) (x8, x8, x8 lanes) - - PCIe Slot1: Intel NIC xl710 2p40GE. - - PCIe Slot2: Intel NIC x710 2p10GE. - - PCIe Slot3: Intel NIC x710 2p10GE. - - Left PCIe Riser Board (Riser 2) (x8, x16, x8 lanes) - - PCIe Slot4: Intel NIC xl710 2p40GE. - - PCIe Slot5: Intel NIC x710 2p10GE. - - PCIe Slot6: Intel NIC x710 2p10GE. - - MLOM slot: empty. - 3. Type-3: Purpose - VIRL functional conformance. - - Quantity: 3 computers as VIRL hosts. - - Physical connectivity: - - CIMC and host management ports. - - no NIC ports, standalone setup. - - Main HW configuration: - - Chassis: UCSC-C240-M4SX with 6 PCIe3.0 slots. - - Processors: 2* E5-2699v3 2.3 GHz. - - RAM Memory: 16* 32GB DDR4-2133MHz. - - Disks: 2* 480 GB 2.5inch 6G SATA SSD. - - NICs configuration: - - Right PCIe Riser Board (Riser 1) (x8, x8, x8 lanes) - - no cards. - - Left PCIe Riser Board (Riser 2) (x8, x16, x8 lanes) - - no cards. - - MLOM slot: empty. -``` - -## Testbeds1,2,3 Connectivity - -### Management Ports - -Total of 10 UCSC-C240-M4SX servers is made available for FD.IO CSIT testbed. For -management purposes, each server must have following two ports connected to the -management network: - -``` - 1. 1GE CIMC port - - CIMC - Cisco Integrated Management Controller. - - Required for provides embedded server management with WebUI, CLI, SNMPv3, - IPMIv2.0. - 2. 1GE management port - - hostOS management port. -``` - -### Naming Convention - -Following naming convention is used within this page to specify physical -connectivity and wiring across defined CSIT testbeds: - -``` - - testbedname: testbedN. - - hostname: - - traffic-generator: tN-tgW. - - system-under-testX: tN-sutX. - - portnames: - - tN-tgW-cY/pZ. - - tN-sutX-cY/pZ. - - where: - - N - testbed number. - - tgW - server acts as traffic-generator with W index. - - sutX - server acts as system-under-test with X index. - - Y - PCIe slot number denoting a NIC card number within the host. - - Y=1,2,3 - slots in Riser 1, Right PCIe Riser Board, NUMA node 0. - - Y=4,5,6 - slots in Riser 2, Left PCIe Riser Board, NUMA node 1. - - Y=m - the MLOM slot. - - Z - port number on the NIC card. -``` - -### 3-node Topology Testbeds for Performance - -Nine servers are used to build three of 3-node topologies, with each topology -using two servers of Type-1 (SUT function) and one server of Type-2 -(TG function). Server NIC cards are placed and NIC ports are connected using the -scheme defined in next sections. - -### LOM (CIMC) and Management networks - -Each server has a LOM (Lights-Out-Management e.g. Cisco CIMC) and a Management -port, which are connected to two different VLANs. - -``` - 1. LOM (CIMC) VLAN: - - Subnet: 10.30.50.0/24 - - Gateway: 10.30.50.1 - - Broadcast: 10.30.50.255 - - DNS1: 199.204.44.24 - - DNS2: 199.204.47.54 - 2. Management Vlan: - - Subnet: 10.30.51.0/24 - - Gateway: 10.30.51.1 - - Broadcast: 10.30.51.255 - - DNS1: 199.204.44.24 - - DNS2: 199.204.47.54 -``` -To access these hosts, an VPN connection is required. - -#### LOM (CIMC) VLAN IP Addresses allocation - -Name | Comment ----- | ------- -10.30.50.0 | network -10.30.51.1 | Router -10.30.50.2 | LF Reserved -10.30.50.3 | LF Reserved -10.30.50.4 | LF Reserved -10.30.50.5 | LF Reserved -10.30.50.6 | LF Reserved -10.30.50.7 | LF Reserved -10.30.50.8 | LF Reserved -10.30.50.9 | LF Reserved -10.30.50.10 | LF Reserved -10.30.50.11 | LF Reserved -10.30.50.12 | LF Reserved -10.30.50.13 | LF Reserved -10.30.50.14 | LF Reserved -10.30.50.15 | LF Reserved -10.30.50.16 | t1-tg1 -10.30.50.17 | t1-sut1 -10.30.50.18 | t1-sut2 -10.30.50.20 | t2-tg1 -10.30.50.21 | t2-sut1 -10.30.50.22 | t2-sut2 -10.30.50.24 | t3-tg1 -10.30.50.25 | t3-sut1 -10.30.50.26 | t3-sut-2 -10.30.50.28 | t4-sut1 -10.30.50.29 | t4-sut2 -10.30.50.30 | t4-sut3 -10.30.50.255 | Broadcast - -#### Management VLAN IP Addresses allocation - -Name | Comment ----- | ------- -10.30.51.0 | network -10.30.51.1 | Router -10.30.51.2 | LF Reserved -10.30.51.3 | LF Reserved -10.30.51.4 | LF Reserved -10.30.51.5 | LF Reserved -10.30.51.6 | LF Reserved -10.30.51.7 | LF Reserved -10.30.51.8 | LF Reserved -10.30.51.9 | netgate-1 -10.30.51.10 | netgate-2 -10.30.51.11 | netgate-3 -10.30.51.12 | softiron-1 -10.30.51.13 | softiron-2 -10.30.51.14 | softiron-3 -10.30.51.15 | LF Reserved -10.30.51.16 | t1-tg1 -10.30.51.17 | t1-sut1 -10.30.51.18 | t1-sut2 -10.30.51.20 | t2-tg1 -10.30.51.21 | t2-sut1 -10.30.51.22 | t2-sut2 -10.30.51.24 | t3-tg1 -10.30.51.25 | t3-sut1 -10.30.51.26 | t3-sut-2 -10.30.51.28 | t4-sut1 -10.30.51.29 | t4-sut2 -10.30.51.30 | t4-sut3 -10.30.51.31-10.30.51.105 | VIRL1 -10.30.51.106-10.30.51.180 | VIRL2 -10.30.51.181-10.30.51.254 | VIRL3 -10.30.51.255 | Broadcast - -### Testbeds1,2,3 Naming: Servers, Ports - -Each server in 3-node Topology has its NIC cards placed, and NIC cards and ports -indexed using defined naming convention: - -``` - 1. Server1 of Type-1: - - testbedname: testbed1. - - hostname: t1-sut1. - - CIMC IP: 10.30.50.17 - - Host IP: 10.30.51.17 - - portnames: - - t1-sut1-c1/p1 - 10GE port1 on Intel NIC x520 2p10GE. - - t1-sut1-c1/p2 - 10GE port2 on Intel NIC x520 2p10GE. - - t1-sut1-c2/p1 - 40GE port1 on Cisco VIC 1385 2p40GE. - - t1-sut1-c2/p2 - 40GE port2 on Cisco VIC 1385 2p40GE. - - t1-sut1-c4/p1 - 40GE port1 on Intel NIC xl710 2p40GE. - - t1-sut1-c4/p2 - 40GE port2 on Intel NIC xl710 2p40GE. - - t1-sut1-c5/p1 - 10GE port1 on Intel NIC x710 2p10GE. - - t1-sut1-c5/p2 - 10GE port2 on Intel NIC x710 2p10GE. - - t1-sut1-cm/p1 - 10GE port1 on Cisco VIC 1227 2p10GE. - - t1-sut1-cm/p2 - 10GE port2 on Cisco VIC 1227 2p10GE. - 2. Server2 of Type-1: - - testbedname: testbed1. - - hostname: t1-sut2. - - CIMC IP: 10.30.50.18 - - Host IP: 10.30.51.18 - - portnames: - - t1-sut2-c1/p1 - 10GE port1 on Intel NIC x520 2p10GE. - - t1-sut2-c1/p2 - 10GE port2 on Intel NIC x520 2p10GE. - - t1-sut2-c2/p1 - 40GE port1 on Cisco VIC 1385 2p40GE. - - t1-sut2-c2/p2 - 40GE port2 on Cisco VIC 1385 2p40GE. - - t1-sut2-c4/p1 - 40GE port1 on Intel NIC xl710 2p40GE. - - t1-sut2-c4/p2 - 40GE port2 on Intel NIC xl710 2p40GE. - - t1-sut2-c5/p1 - 10GE port1 on Intel NIC x710 2p10GE. - - t1-sut2-c5/p2 - 10GE port2 on Intel NIC x710 2p10GE. - - t1-sut2-cm/p1 - 10GE port1 on Cisco VIC 1227 2p10GE. - - t1-sut2-cm/p2 - 10GE port2 on Cisco VIC 1227 2p10GE. - 3. Server3 of Type-2: - - testbedname: testbed1. - - hostname: t1-tg1. - - CIMC IP: 10.30.50.16 - - Host IP: 10.30.51.16 - - portnames: - - t1-tg1-c1/p1 - 40GE port1 on Intel NIC xl710 2p40GE. - - t1-tg1-c1/p2 - 40GE port2 on Intel NIC xl710 2p40GE. - - t1-tg1-c2/p1 - 10GE port1 on Intel NIC x710 2p10GE. - - t1-tg1-c2/p2 - 10GE port2 on Intel NIC x710 2p10GE. - - t1-tg1-c3/p1 - 10GE port1 on Intel NIC x710 2p10GE. - - t1-tg1-c3/p2 - 10GE port2 on Intel NIC x710 2p10GE. - - t1-tg1-c4/p1 - 40GE port1 on Intel NIC xl710 2p40GE. - - t1-tg1-c4/p2 - 40GE port2 on Intel NIC xl710 2p40GE. - - t1-tg1-c5/p1 - 10GE port1 on Intel NIC x710 2p10GE. - - t1-tg1-c5/p2 - 10GE port2 on Intel NIC x710 2p10GE. - - t1-tg1-c6/p1 - 10GE port1 on Intel NIC x710 2p10GE. - - t1-tg1-c6/p2 - 10GE port2 on Intel NIC x710 2p10GE. - 4. Server4 of Type-1: - - testbedname: testbed2. - - hostname: t2-sut1. - - CIMC IP: 10.30.50.21 - - Host IP: 10.30.51.21 - - portnames: - - t2-sut1-c1/p1 - 10GE port1 on Intel NIC x520 2p10GE. - - t2-sut1-c1/p2 - 10GE port2 on Intel NIC x520 2p10GE. - - t2-sut1-c2/p1 - 40GE port1 on Cisco VIC 1385 2p40GE. - - t2-sut1-c2/p2 - 40GE port2 on Cisco VIC 1385 2p40GE. - - t2-sut1-c4/p1 - 40GE port1 on Intel NIC xl710 2p40GE. - - t2-sut1-c4/p2 - 40GE port2 on Intel NIC xl710 2p40GE. - - t2-sut1-c5/p1 - 10GE port1 on Intel NIC x710 2p10GE. - - t2-sut1-c5/p2 - 10GE port2 on Intel NIC x710 2p10GE. - - t2-sut1-cm/p1 - 10GE port1 on Cisco VIC 1227 2p10GE. - - t2-sut1-cm/p2 - 10GE port2 on Cisco VIC 1227 2p10GE. - 5. Server5 of Type-1: - - testbedname: testbed2. - - hostname: t2-sut2. - - CIMC IP: 10.30.50.22 - - Host IP: 10.30.51.22 - - portnames: - - t2-sut2-c1/p1 - 10GE port1 on Intel NIC x520 2p10GE. - - t2-sut2-c1/p2 - 10GE port2 on Intel NIC x520 2p10GE. - - t2-sut2-c2/p1 - 40GE port1 on Cisco VIC 1385 2p40GE. - - t2-sut2-c2/p2 - 40GE port2 on Cisco VIC 1385 2p40GE. - - t2-sut2-c4/p1 - 40GE port1 on Intel NIC xl710 2p40GE. - - t2-sut2-c4/p2 - 40GE port2 on Intel NIC xl710 2p40GE. - - t2-sut2-c5/p1 - 10GE port1 on Intel NIC x710 2p10GE. - - t2-sut2-c5/p2 - 10GE port2 on Intel NIC x710 2p10GE. - - t2-sut2-cm/p1 - 10GE port1 on Cisco VIC 1227 2p10GE. - - t2-sut2-cm/p2 - 10GE port2 on Cisco VIC 1227 2p10GE. - 6. Server6 of Type-2: - - testbedname: testbed2. - - hostname: t2-tg1. - - CIMC IP: 10.30.50.20 - - Host IP: 10.30.51.20 - - portnames: - - t2-tg1-c1/p1 - 40GE port1 on Intel NIC xl710 2p40GE. - - t2-tg1-c1/p2 - 40GE port2 on Intel NIC xl710 2p40GE. - - t2-tg1-c2/p1 - 10GE port1 on Intel NIC x710 2p10GE. - - t2-tg1-c2/p2 - 10GE port2 on Intel NIC x710 2p10GE. - - t2-tg1-c3/p1 - 10GE port1 on Intel NIC x710 2p10GE. - - t2-tg1-c3/p2 - 10GE port2 on Intel NIC x710 2p10GE. - - t2-tg1-c4/p1 - 40GE port1 on Intel NIC xl710 2p40GE. - - t2-tg1-c4/p2 - 40GE port2 on Intel NIC xl710 2p40GE. - - t2-tg1-c5/p1 - 10GE port1 on Intel NIC x710 2p10GE. - - t2-tg1-c5/p2 - 10GE port2 on Intel NIC x710 2p10GE. - - t2-tg1-c6/p1 - 10GE port1 on Intel NIC x710 2p10GE. - - t2-tg1-c6/p2 - 10GE port2 on Intel NIC x710 2p10GE. - 7. Server7 of Type-1: - - testbedname: testbed3. - - hostname: t3-sut1. - - CIMC IP: 10.30.50.25 - - Host IP: 10.30.51.25 - - portnames: - - t3-sut1-c1/p1 - 10GE port1 on Intel NIC x520 2p10GE. - - t3-sut1-c1/p2 - 10GE port2 on Intel NIC x520 2p10GE. - - t3-sut1-c2/p1 - 40GE port1 on Cisco VIC 1385 2p40GE. - - t3-sut1-c2/p2 - 40GE port2 on Cisco VIC 1385 2p40GE. - - t3-sut1-c4/p1 - 40GE port1 on Intel NIC xl710 2p40GE. - - t3-sut1-c4/p2 - 40GE port2 on Intel NIC xl710 2p40GE. - - t3-sut1-c5/p1 - 10GE port1 on Intel NIC x710 2p10GE. - - t3-sut1-c5/p2 - 10GE port2 on Intel NIC x710 2p10GE. - - t3-sut1-cm/p1 - 10GE port1 on Cisco VIC 1227 2p10GE. - - t3-sut1-cm/p2 - 10GE port2 on Cisco VIC 1227 2p10GE. - 8. Server8 of Type-1: - - testbedname: testbed3. - - hostname: t3-sut2. - - CIMC IP: 10.30.50.26 - - Host IP: 10.30.51.26 - - portnames: - - t3-sut2-c1/p1 - 10GE port1 on Intel NIC x520 2p10GE. - - t3-sut2-c1/p2 - 10GE port2 on Intel NIC x520 2p10GE. - - t3-sut2-c2/p1 - 40GE port1 on Cisco VIC 1385 2p40GE. - - t3-sut2-c2/p2 - 40GE port2 on Cisco VIC 1385 2p40GE. - - t3-sut2-c4/p1 - 40GE port1 on Intel NIC xl710 2p40GE. - - t3-sut2-c4/p2 - 40GE port2 on Intel NIC xl710 2p40GE. - - t3-sut2-c5/p1 - 10GE port1 on Intel NIC x710 2p10GE. - - t3-sut2-c5/p2 - 10GE port2 on Intel NIC x710 2p10GE. - - t3-sut2-cm/p1 - 10GE port1 on Cisco VIC 1227 2p10GE. - - t3-sut2-cm/p2 - 10GE port2 on Cisco VIC 1227 2p10GE. - 9. Server9 of Type-2: - - testbedname: testbed3. - - hostname: t3-tg1. - - CIMC IP: 10.30.50.24 - - Host IP: 10.30.51.24 - - portnames: - - t3-tg1-c1/p1 - 40GE port1 on Intel NIC xl710 2p40GE. - - t3-tg1-c1/p2 - 40GE port2 on Intel NIC xl710 2p40GE. - - t3-tg1-c2/p1 - 10GE port1 on Intel NIC x710 2p10GE. - - t3-tg1-c2/p2 - 10GE port2 on Intel NIC x710 2p10GE. - - t3-tg1-c3/p1 - 10GE port1 on Intel NIC x710 2p10GE. - - t3-tg1-c3/p2 - 10GE port2 on Intel NIC x710 2p10GE. - - t3-tg1-c4/p1 - 40GE port1 on Intel NIC xl710 2p40GE. - - t3-tg1-c4/p2 - 40GE port2 on Intel NIC xl710 2p40GE. - - t3-tg1-c5/p1 - 10GE port1 on Intel NIC x710 2p10GE. - - t3-tg1-c5/p2 - 10GE port2 on Intel NIC x710 2p10GE. - - t3-tg1-c6/p1 - 10GE port1 on Intel NIC x710 2p10GE. - - t3-tg1-c6/p2 - 10GE port2 on Intel NIC x710 2p10GE. -``` - -### Testbeds1,2,3 Network Connectivity - -Listed nine servers are connected into the three of 3-node testbeds, testbed1, -testbed2 and testbed3, using defined naming convention as follows: - -``` - 1. testbed1: - - ring of 40GE ports on Cisco VIC 1385 2p40GE on SUTs - - t1-tg1-c1/p1 to t1-sut1-c1/p2. - - t1-sut1-c1/p1 to t1-sut2-c1/p2. - - t1-sut2-c1/p1 to t1-tg1-c1/p2. - - ring of 10GE ports on Intel NIC x520 2p10GE on SUTs - - t1-tg1-c2/p1 to t1-sut1-c2/p2. - - t1-sut1-c2/p1 to t1-sut2-c2/p2. - - t1-sut2-c2/p1 to t1-tg1-c2/p2. - - ring of 40GE ports on Intel NIC xl710 2p40GE on SUTs - - t1-tg1-c4/p1 to t1-sut1-c4/p2. - - t1-sut1-c4/p1 to t1-sut2-c4/p2. - - t1-sut2-c4/p1 to t1-tg1-c4/p2. - - ring of 10GE ports on Intel NIC x710 2p10GE on SUTs - - t1-tg1-c5/p1 to t1-sut1-c5/p2. - - t1-sut1-c5/p1 to t1-sut2-c5/p2. - - t1-sut2-c5/p1 to t1-tg1-c5/p2. - - ring of 10GE ports on Cisco VIC 1227 2p10GE on SUTs - - t1-tg1-c2/p1 to t1-sut1-cm/p2. - - t1-sut1-cm/p1 to t1-sut2-cm/p2. - - t1-sut2-cm/p1 to t1-tg1-c2/p2. - - TG loopback ports Intel NIC x710 2p10GE - - t1-tg1-c6/p1 to t1-tg1-c6/p2. - - 2. testbed2: - - ring of 40GE ports on Cisco VIC 1385 2p40GE on SUTs - - t2-tg1-c1/p1 to t2-sut1-c1/p2. - - t2-sut1-c1/p1 to t2-sut2-c1/p2. - - t2-sut2-c1/p1 to t2-tg1-c1/p2. - - ring of 10GE ports on Intel NIC x520 2p10GE on SUTs - - t2-tg1-c2/p1 to t2-sut1-c2/p2. - - t2-sut1-c2/p1 to t2-sut2-c2/p2. - - t2-sut2-c2/p1 to t2-tg1-c2/p2. - - ring of 40GE ports on Intel NIC xl710 2p40GE on SUTs - - t2-tg1-c4/p1 to t2-sut1-c4/p2. - - t2-sut1-c4/p1 to t2-sut2-c4/p2. - - t2-sut2-c4/p1 to t2-tg1-c4/p2. - - ring of 10GE ports on Intel NIC x710 2p10GE on SUTs - - t2-tg1-c5/p1 to t2-sut1-c5/p2. - - t2-sut1-c5/p1 to t2-sut2-c5/p2. - - t2-sut2-c5/p1 to t2-tg1-c5/p2. - - ring of 10GE ports on Cisco VIC 1227 2p10GE on SUTs - - t2-tg1-c2/p1 to t2-sut1-cm/p2. - - t2-sut1-cm/p1 to t2-sut2-cm/p2. - - t2-sut2-cm/p1 to t2-tg1-c2/p2. - - TG loopback ports Intel NIC x710 2p10GE - - t2-tg1-c6/p1 to t2-tg1-c6/p2. - - 3. testbed3: - - ring of 40GE ports on Cisco VIC 1385 2p40GE on SUTs - - t3-tg1-c1/p1 to t3-sut1-c1/p2. - - t3-sut1-c1/p1 to t3-sut2-c1/p2. - - t3-sut2-c1/p1 to t3-tg1-c1/p2. - - ring of 10GE ports on Intel NIC x520 2p10GE on SUTs - - t3-tg1-c2/p1 to t3-sut1-c2/p2. - - t3-sut1-c2/p1 to t3-sut2-c2/p2. - - t3-sut2-c2/p1 to t3-tg1-c2/p2. - - ring of 40GE ports on Intel NIC xl710 2p40GE on SUTs - - t3-tg1-c4/p1 to t3-sut1-c4/p2. - - t3-sut1-c4/p1 to t3-sut2-c4/p2. - - t3-sut2-c4/p1 to t3-tg1-c4/p2. - - ring of 10GE ports on Intel NIC x710 2p10GE on SUTs - - t3-tg1-c5/p1 to t3-sut1-c5/p2. - - t3-sut1-c5/p1 to t3-sut2-c5/p2. - - t3-sut2-c5/p1 to t3-tg1-c5/p2. - - ring of 10GE ports on Cisco VIC 1227 2p10GE on SUTs - - t3-tg1-c2/p1 to t3-sut1-cm/p2. - - t3-sut1-cm/p1 to t3-sut2-cm/p2. - - t3-sut2-cm/p1 to t3-tg1-c2/p2. - - TG loopback ports Intel NIC x710 2p10GE - - t3-tg1-c6/p1 to t3-tg1-c6/p2. -``` - -## Testbeds1,2,3 Server Specifications +## Cisco UCS c240m4 Xeon Haswell Servers - Hardware and BIOS Configuration ### Linux lscpu @@ -1181,4 +743,3 @@ testbed2 and testbed3, using defined naming convention as follows: Out-of-Band Management: Disabled C240-FCH1950V1H5 /bios/advanced # ``` - |