aboutsummaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
-rw-r--r--docs/lab/testbed_specifications.md169
1 files changed, 53 insertions, 116 deletions
diff --git a/docs/lab/testbed_specifications.md b/docs/lab/testbed_specifications.md
index 556a170a4e..69efbae272 100644
--- a/docs/lab/testbed_specifications.md
+++ b/docs/lab/testbed_specifications.md
@@ -14,7 +14,6 @@
1. [3-Node-Haswell Xeon Intel (3n-skx)](#3-node-haswell-xeon-intel-3n-skx)
1. [3-Node-Skylake Xeon Intel (3n-skx)](#3-node-skylake-xeon-intel-3n-skx)
1. [3-Node-TaiShan Arm Huawei (3n-tsh)](#3-node-taishan-arm-huawei-3n-tsh)
- 1. [3-Node-MACCHIATObin Arm Marvell](#3-node-macchiatobin-arm-marvell)
1. [3-Node-Rangeley Atom Testbeds](#3-node-rangeley-atom-testbeds)
1. [Server Management](#server-management)
1. [Requirements](#requirements)
@@ -37,8 +36,7 @@
1. [3-Node-Haswell Servers (3n-hsw) PROD](#3-node-haswell-servers-3n-hsw-prod)
1. [3-Node-Skylake Servers (3n-skx) PROD](#3-node-skylake-servers-3n-skx-prod)
1. [3-Node-Rangeley Servers (3n-rng) VERIFY](#3-node-rangeley-servers-3n-rng-verify)
- 1. [3-Node-Taishan Servers (3n-tsh) WIP](#3-node-taishan-servers-3n-tsh-wip)
- 1. [3-Node-Mcbin Servers (3n-mcb) TODO](#3-node-mcbin-servers-3n-mcb-todo)
+ 1. [3-Node-Taishan Servers (3n-tsh) PROD](#3-node-taishan-servers-3n-tsh-wip)
1. [Per Testbed Wiring](#per-testbed-wiring)
1. [1-Node-Skylake Wiring (1n-skx) PROD](#1-node-skylake-wiring-1n-skx-prod)
1. [1-Node-Thunderx2 Wiring (1n-tx2) WIP](#1-node-thunderx2-wiring-1n-tx2-wip)
@@ -51,8 +49,7 @@
1. [3-Node-Haswell Wiring (3n-hsw) PROD](#3-node-haswell-wiring-3n-hsw-prod)
1. [3-Node-Skylake Wiring (3n-skx) PROD](#3-node-skylake-wiring-3n-skx-prod)
1. [3-Node-Rangeley Wiring (3n-rng) TODO](#3-node-rangeley-wiring-3n-rng-todo)
- 1. [3-Node-Taishan Wiring (3n-tsh) WIP](#3-node-taishan-wiring-3n-tsh-wip)
- 1. [3-Node-Mcbin Wiring (3n-mcb) WIP](#3-node-mcbin-wiring-3n-mcb-wip)
+ 1. [3-Node-Taishan Wiring (3n-tsh) PROD](#3-node-taishan-wiring-3n-tsh-wip)
1. [Inventory](#inventory)
1. [Appliances](#appliances)
1. [Arm Servers](#arm-servers)
@@ -71,24 +68,23 @@ hosted by LFN FD.io CSIT project.
### Summary List
```
- #. CSIT_tb Purpose SUT TG #TB #SUT #TG #hsw #skx #ps1 #rng #dnv #tx2 #tsh #mcb #clx #zn2
- 1. 1-Node-VIRL dev hsw --- 3 3 0 3 0 0 0 0 0 0 0 0 0
- 2. 1-Node-Skylake dev skx na 2 2 0 0 2 0 0 0 0 0 0 0 0
- 3. 1-Node-Thunderx2 idle tx2 na 1 1 0 0 0 0 0 0 1 0 0 0 0
- 4. 1-Node-Thunderx2 dev tx2 na 2 2 0 0 0 0 0 0 2 0 0 0 0
- 5. 1-Node-Cascadelake dev clx lcx 1 1 0 0 0 0 0 0 0 0 0 1 0
- 6. 2-Node-Skylake perf skx skx 4 4 4 0 8 0 0 0 0 0 0 0 0
- 7. 2-Node-Denverton perf dnv skx 1 1 1 0 .5 0 0 1 0 0 0 0 0
- 8. 2-Node-IxiaPS1L47 tcp skx ps1 1 1 1 0 1 1 0 0 0 0 0 0 0
- 9. 2-Node-Cascadelake perf clx clx 3 3 3 0 0 0 0 0 0 0 0 6 0
-10. 3-Node-Haswell perf hsw hsw 3 6 3 9 0 0 0 0 0 0 0 0 0
-11. 3-Node-Skylake perf skx skx 2 4 2 0 6 0 0 0 0 0 0 0 0
-12. 3-Node-Rangeley perf rng skx 1 3 1 0 0 0 2 0 0 0 0 0 0
-13. 3-Node-Taishan perf tsh skx 1 2 1 0 .5 0 0 0 0 2 0 0 0
-14. 3-Node-Mcbin perf mcb skx 1 2 1 0 .5 0 0 0 0 0 2 0 0
-15. 3-Node-Denverton perf dnv skx 1 2 1 0 .5 0 0 2 0 0 0 0 0
-16. 2-Node-Zen2 perf zn2 zn2 1 1 1 0 0 0 0 0 0 0 0 0 2
- Totals: 28 38 18 12 19 1 2 3 3 2 2 7 2
+ #. CSIT_tb Purpose SUT TG #TB #SUT #TG #hsw #skx #ps1 #rng #dnv #tx2 #tsh #clx #zn2
+ 1. 1-Node-VIRL dev hsw --- 3 3 0 3 0 0 0 0 0 0 0 0
+ 2. 1-Node-Skylake dev skx na 2 2 0 0 2 0 0 0 0 0 0 0
+ 3. 1-Node-Thunderx2 idle tx2 na 1 1 0 0 0 0 0 0 1 0 0 0
+ 4. 1-Node-Thunderx2 dev tx2 na 2 2 0 0 0 0 0 0 2 0 0 0
+ 5. 1-Node-Cascadelake dev clx lcx 1 1 0 0 0 0 0 0 0 0 1 0
+ 6. 2-Node-Skylake perf skx skx 4 4 4 0 8 0 0 0 0 0 0 0
+ 7. 2-Node-Denverton perf dnv skx 1 1 1 0 .5 0 0 1 0 0 0 0
+ 8. 2-Node-IxiaPS1L47 tcp skx ps1 1 1 1 0 1 1 0 0 0 0 0 0
+ 9. 2-Node-Cascadelake perf clx clx 3 3 3 0 0 0 0 0 0 0 6 0
+10. 3-Node-Haswell perf hsw hsw 3 6 3 9 0 0 0 0 0 0 0 0
+11. 3-Node-Skylake perf skx skx 2 4 2 0 6 0 0 0 0 0 0 0
+12. 3-Node-Rangeley perf rng skx 1 3 1 0 0 0 2 0 0 0 0 0
+13. 3-Node-Taishan perf tsh skx 1 2 .5 0 .5 0 0 0 0 2 0 0
+14. 3-Node-Denverton perf dnv skx 1 2 1 0 .5 0 0 2 0 0 0 0
+15. 2-Node-Zen2 perf zn2 zn2 1 1 1 0 0 0 0 0 0 0 0 2
+ Totals: 27 36 16.5 12 18.5 1 2 3 3 2 7 2
```
### 1-Node-Skylake Xeon Intel (1n-skx)
@@ -165,13 +161,6 @@ TG (Server-Type-B2) connected in a 3-node circular topology
([Server Types](#server-types)).
Used for FD.io performance tests.
-### 3-Node-MACCHIATObin Arm Marvell
-
-Each 3-Node-MACCHIATObin testbed includes two SUTs (Server-Type-B4) and
-one TG (Server-Type-B2) connected in a 3-node circular topology
-([Server Types](#server-types)).
-Used for FD.io performance tests.
-
### 3-Node-Rangeley Atom Testbeds
Each 3-Node-Rangeley testbed includes two SUTs (Server-Type-B5) and one
@@ -281,9 +270,9 @@ Name | Comment
10.30.50.58 | s25-t25-sut1
10.30.50.59 | s26-t25-tg1
10.30.50.65 | s52-nomad
-10.30.50.66 | s51-nomad (out of order)
+10.30.50.66 | s51-nomad
10.30.50.67 | s49-nomad
-10.30.50.68 | s50-nomad (out of order)
+10.30.50.68 | s50-nomad
10.30.50.69 | s27-t13-sut
10.30.50.70 | s55-t36-sut1
10.30.50.71 | s56-t37-sut1
@@ -326,9 +315,9 @@ Name | Comment
10.30.51.9 | s22-t35-sut1 (netgate-1)
10.30.51.10 | s23-t35-sut2 (netgate-2)
10.30.51.11 | s24-t35-sut3 (netgate-3)
-10.30.51.12 | softiron-1
-10.30.51.13 | softiron-2
-10.30.51.14 | softiron-3
+10.30.51.12 | softiron-1 (decommissioned)
+10.30.51.13 | softiron-2 (decommissioned)
+10.30.51.14 | softiron-3 (decommissioned)
10.30.51.15 | LF Reserved
10.30.51.16 | t1-tg1
10.30.51.17 | t1-sut1
@@ -351,8 +340,9 @@ Name | Comment
10.30.51.38 | fdio-marvell-dev
10.30.51.39 | s53-nomad
10.30.51.40 | s54-nomad
-10.30.51.41 | s20-t34-sut1
-10.30.51.42 | s21-t34-sut2
+10.30.51.41 | s20-t34-sut1 (decommissioned)
+10.30.51.42 | s21-t34-sut2 (decommissioned)
+10.30.51.43 | fdio-mcbin3 (decommissioned)
10.30.51.44 | s3-t21-sut1
10.30.51.45 | s4-t21-tg1
10.30.51.46 | s11-t31-sut1
@@ -518,19 +508,7 @@ FD.io CSIT lab contains following server types:
- NICs configuration:
- PCIe Slot4 e9:00.xx: connectx4-2p25GE Mellanox.
- PCIe Slot6 11:00.xx: x520-2p10GE Intel.
-7. Server-Type-B4: Purpose - MACCHIATObin Arm Marvell SUT for FD.io performance testing.
- - Quantity: 3
- - Physical connectivity:
- - Host management ports.
- - NIC ports connected into 2-node and 3-node topologies.
- - Main HW configuration:
- - Chassis: MACCHIATObin.
- - Processors: 1* Armada 8040 ~ 4* Arm Cortex-A72
- - RAM Memory: 1* 16GB DDR4
- - Disks: 1* 128GB(?) SATA SDD
- - NICs configuration:
- - pp2-2p10GE Marvell (on-chip Ethernet ports ; marvell plugin in VPP)
-8. Server-Type-B5: Purpose - Rangeley Atom SUT for FD.io performance testing.
+7. Server-Type-B5: Purpose - Rangeley Atom SUT for FD.io performance testing.
- Quantity: TBD based on testbed allocation.
- Physical connectivity:
- Management: serial Port (usb) for console
@@ -543,7 +521,7 @@ FD.io CSIT lab contains following server types:
- NICs configuration:
- 2x 10Gb Intel 82599ES
- 4x 1GB Intel I354
-9. Server-Type-B6: Purpose - Skylake Xeon SUT for FD.io VPP_Device functional tests.
+8. Server-Type-B6: Purpose - Skylake Xeon SUT for FD.io VPP_Device functional tests.
- Quantity: 2.
- Physical connectivity:
- IPMI and host management ports.
@@ -563,7 +541,7 @@ FD.io CSIT lab contains following server types:
- PCIe Slot6 86:00.xx: empty.
- PCIe Slot8 af:00.xx: empty.
- PCIe Slot10 d8:00.xx: empty.
-10. Server-Type-B7: Purpose - Ixia PerfectStorm One Appliance TG for FD.io TCP/IP performance tests.
+9. Server-Type-B7: Purpose - Ixia PerfectStorm One Appliance TG for FD.io TCP/IP performance tests.
- Quantity: 1.
- Physical connectivity:
- Host management interface: 10/100/1000-BaseT.
@@ -582,7 +560,7 @@ FD.io CSIT lab contains following server types:
- Port-2: 10GE SFP+.
- Port-3: 10GE SFP+.
- Port-4: 10GE SFP+.
-11. Server-Type-B8: Purpose - Skylake Xeon SUT for TCP/IP host stack tests.
+10. Server-Type-B8: Purpose - Skylake Xeon SUT for TCP/IP host stack tests.
- Quantity: 1.
- Physical connectivity:
- IPMI and host management ports.
@@ -602,7 +580,7 @@ FD.io CSIT lab contains following server types:
- PCIe Slot6 86:00.xx: empty.
- PCIe Slot8 af:00.xx: empty.
- PCIe Slot10 d8:00.xx: empty.
-12. Server-Type-B9: Purpose - ThunderX2 Arm Marvell(Idle).
+11. Server-Type-B9: Purpose - ThunderX2 Arm Marvell(Idle).
- Quantity: 1
- Physical connectivity:
- IPMI and host management ports.
@@ -619,7 +597,7 @@ FD.io CSIT lab contains following server types:
- PCIe Slot3 08:00.xx: XL710-QDA2.
- Numa1:
- PCIe Slot6 85:00.xx: XL710-QDA2.
-13. Server-Type-B10: Purpose - Denverton Atom SUT for FD.io performance testing.
+12. Server-Type-B10: Purpose - Denverton Atom SUT for FD.io performance testing.
- Quantity: 4
- Physical connectivity:
- IPMI and host management ports.
@@ -633,8 +611,7 @@ FD.io CSIT lab contains following server types:
- 2x 10Gb Intel x553 fiber ports
- 2x 10Gb Intel x553 copper ports
- 4x 1GB Intel I350 ports
-
-14. Server-Type-B11: Purpose - Thunderx2 Arm Marvell SUT for FD.io VPP_Device functional tests.
+13. Server-Type-B11: Purpose - Thunderx2 Arm Marvell SUT for FD.io VPP_Device functional tests.
- Quantity: 2
- Physical connectivity:
- IPMI and host management ports.
@@ -653,7 +630,7 @@ FD.io CSIT lab contains following server types:
- PCIe Slot14 91:00.xx: XL710-QDA2-2p40GE Intel.
- PCIe Slot26 9a:00.xx: ConnectX5-2p10/25GE Mellanox.
-15. Server-Type-C1: Purpose - Cascadelake Xeon SUT for FD.io VPP_Device functional tests.
+14. Server-Type-C1: Purpose - Cascadelake Xeon SUT for FD.io VPP_Device functional tests.
- Quantity: 1.
- Physical connectivity:
- IPMI and host management ports.
@@ -673,8 +650,7 @@ FD.io CSIT lab contains following server types:
- PCIe Slot6 86:00.xx: empty.
- PCIe Slot8 af:00.xx: empty.
- PCIe Slot10 d8:00.xx: empty.
-
-16. Server-Type-C2: Purpose - Cascadelake Xeon SUT for FD.io performance testing.
+15. Server-Type-C2: Purpose - Cascadelake Xeon SUT for FD.io performance testing.
- Quantity: 3
- Physical connectivity:
- IPMI and host management ports.
@@ -694,7 +670,7 @@ FD.io CSIT lab contains following server types:
- PCIe Slot6 86:00.xx: empty.
- PCIe Slot8 af:00.xx: empty.
- PCIe Slot10 d8:00.xx: empty.
-17. Server-Type-C3: Purpose - Cascadelake Xeon TG for FD.io performance testing.
+16. Server-Type-C3: Purpose - Cascadelake Xeon TG for FD.io performance testing.
- Quantity: 3.
- Physical connectivity:
- IPMI and host management ports.
@@ -714,8 +690,7 @@ FD.io CSIT lab contains following server types:
- PCIe Slot6 86:00.xx: empty.
- PCIe Slot8 af:00.xx: empty.
- PCIe Slot10 d8:00.xx: empty.
-
-18. Server-Type-C4: Purpose - Cascadelake Xeon Backend hosts for FD.io builds and data processing.
+17. Server-Type-C4: Purpose - Cascadelake Xeon Backend hosts for FD.io builds and data processing.
- Quantity: 3.
- Physical connectivity:
- CIMC and host management ports.
@@ -731,7 +706,8 @@ FD.io CSIT lab contains following server types:
- no cards.
- Numa1:
- no cards.
-19. Server-Type-D1: Purpose - Zen2 EPYC SUT for FD.io performance testing.
+
+18. Server-Type-D1: Purpose - Zen2 EPYC SUT for FD.io performance testing.
- Quantity: 1.
- Physical connectivity:
- IPMI and host management ports.
@@ -746,7 +722,7 @@ FD.io CSIT lab contains following server types:
- PCIe Slot1 01:00.xx: x710-4p10GE Intel.
- PCIe Slot2 41:00.xx: xxv710-DA2-2p25GE Intel.
- PCIe Slot3 81:00.xx: mcx556a-edat ConnectX5-2p100GE Mellanox.
-20. Server-Type-D2: Purpose - Zen2 EPYC TG for FD.io performance testing.
+19. Server-Type-D2: Purpose - Zen2 EPYC TG for FD.io performance testing.
- Quantity: 1.
- Physical connectivity:
- IPMI and host management ports.
@@ -1435,9 +1411,7 @@ Note: There is no IPMI. Serial console is accessible via VIRL2 and VIRL3 USB.
- 1GB ports (tbd)
```
-#### 3-Node-Taishan Servers (3n-tsh) WIP
-
-Note: ServerB19 (TG) is shared between testbed33 & testbed34
+#### 3-Node-Taishan Servers (3n-tsh) PROD
```
- ServerB17 [Server-Type-B3]:
@@ -1461,7 +1435,7 @@ Note: ServerB19 (TG) is shared between testbed33 & testbed34
- s18-t33-sut2-c4/p1 - 25GE-port1 cx4-2p25GE.
- s18-t33-sut2-c4/p2 - 25GE-port2 cx4-2p25GE.
- ServerB19 [Server-Type-B2]:
- - testbednames: testbed33 and testbed34.
+ - testbedname: testbed33 and testbed34.
- hostname: s19-t33t34-tg1.
- IPMI IP: 10.30.50.46
- Host IP: 10.30.51.49
@@ -1476,22 +1450,6 @@ Note: ServerB19 (TG) is shared between testbed33 & testbed34
- s19-t33t34-tg1-c10/p2 - 10GE-port2 x710-4p10GE.
- s19-t33t34-tg1-c10/p3 - 10GE-port3 x710-4p10GE.
- s19-t33t34-tg1-c10/p4 - 10GE-port4 x710-4p10GE.
-- ServerB20 [Server-Type-B4]:
- - testbedname: testbed34.
- - hostname: s20-t34-sut1.
- - IPMI IP: N/A
- - Host IP: 10.30.51.41
- - portnames:
- - s20-t34-sut1-ca/p1 - 10GE-port1 Marvell.
- - s20-t34-sut1-ca/p2 - 10GE-port2 Marvell.
-- ServerB21 [Server-Type-B4]:
- - testbedname: testbed34.
- - hostname: s21-t34-sut2.
- - IPMI IP: N/A
- - Host IP: 10.30.51.42
- - portnames:
- - s21-t34-sut2-ca/p1 - 10GE-port1 Marvell.
- - s21-t34-sut2-ca/p2 - 10GE-port2 Marvell.
```
#### 3-Node-Denverton Servers (3n-dnv) WIP
@@ -1519,12 +1477,6 @@ Note: ServerB19 (TG) is shared between testbed33 & testbed34
- s31-t35-sut2-p4 - 10GE-port4 x553 fiber port.
```
-#### 3-Node-Mcbin Servers (3n-mcb) TODO
-
-```
-To be completed.
-```
-
### Per Testbed Wiring
#### 1-Node-Skylake Wiring (1n-skx) PROD
@@ -1848,28 +1800,18 @@ To be completed.
To be completed.
```
-#### 3-Node-Taishan Wiring (3n-tsh) WIP
+#### 3-Node-Taishan Wiring (3n-tsh) PROD
```
- testbed33:
- ring1 10GE-ports x520-2p10GE on SUTs:
- - t33t34-tg1-c2/p2 - t33-sut1-c6/p2.
- - t33-sut1-c6/p1 - t33-sut2-c6/p2.
- - t33-sut2-c6/p1 - t33t34-tg1-c2/p1.
+ - s19-t33t34-tg1-c2/p2 - s17-t33-sut1-c6/p2.
+ - s17-t33-sut1-c6/p1 - s18-t33-sut2-c6/p2.
+ - s18-t33-sut2-c6/p1 - s19-t33t34-tg1-c2/p1.
- ring2 25GE-ports cx4-2p25GE on SUTs:
- - t33t34-tg1-c4/p2 - t33-sut1-c4/p2.
- - t33-sut1-c4/p1 - t33-sut2-c4/p2.
- - t33-sut2-c4/p1 - t33t34-tg1-c4/p1.
-```
-
-#### 3-Node-Mcbin Wiring (3n-mcb) WIP
-
-```
-- testbed34:
- - ring1 10GE-ports Marvell on SUTs:
- - t33t34-tg1-c2/p3 - t34-sut1-ca/p1.
- - t34-sut1-ca/p2 - t34-sut2-ca/p1.
- - t34-sut2-ca/p2 - t33t34-tg1-c2/p4.
+ - s19-t33t34-tg1-c4/p2 - s17-t33-sut1-c4/p2.
+ - s17-t33-sut1-c4/p1 - s18-t33-sut2-c4/p2.
+ - s18-t33-sut2-c4/p1 - s19-t33t34-tg1-c4/p1.
```
#### 3-Node-Denverton Wiring (3n-dnv) WIP
@@ -1909,9 +1851,9 @@ To be completed.
1. Arm Cortex A-72 servers
- 2 * ThunderX2 (VPP Device)
- Chassis: GIGABYTE Rack Mount
- - Processors: 2* ThunderX2 CN9975 ~ 112* ThunderX2
- - RAM Memory: 250GB
- - Disks: 500GB
+ - Processors: 2* ThunderX2 CN9980 ~ 128* ThunderX2
+ - RAM Memory: 16* 16GB DIMM
+ - Disks: 2* 480GB 6G SATA SSD SAMSUNG MZ7LH480
- 1 * ThunderX2 (Idle)
- Chassis: Marvell ThunderX2
- Processors: 2* ThunderX2 CN9975 ~ 112* ThunderX2.
@@ -1922,11 +1864,6 @@ To be completed.
- Processors: 1* hip07-d05 ~ 64* Arm Cortex-A72.
- RAM Memory: 8* 16GB DDR4-2400MT/s.
- Disks: 1* 4TB SATA HDD.
- - 3 * MACCHIATObin
- - Chassis: MACCHIATObin.
- - Processors: 1* Armada 8040 ~ 4* Arm Cortex-A72.
- - RAM Memory: 1* 16GB DDR4.
- - Disks: 1* 128GB(?) SATA SDD.
```
### Xeon and Atom Servers