diff options
| -rw-r--r-- | docs/lab/testbed_specifications.md | 107 | ||||
| -rw-r--r-- | docs/lab/testbeds_sm_skx_hw_bios_cfg.md | 115 | ||||
| -rw-r--r-- | docs/lab/testbeds_ucs_hsw_hw_bios_cfg.md | 23 |
3 files changed, 117 insertions, 128 deletions
diff --git a/docs/lab/testbed_specifications.md b/docs/lab/testbed_specifications.md index b12d42d22f..73138ec6cf 100644 --- a/docs/lab/testbed_specifications.md +++ b/docs/lab/testbed_specifications.md @@ -1,57 +1,56 @@ -<!-- TOC depthFrom:1 depthTo:6 withLinks:1 updateOnSave:1 orderedList:0 --> - - - [FD.io CSIT Testbed Specifications](#fdio-csit-testbed-specifications) - - [Testbeds Overview](#testbeds-overview) - - [Summary List](#summary-list) - - [1-Node-Skylake Xeon Intel (1n-skx)](#1-node-skylake-xeon-intel-1n-skx) - - [1-Node-ThunderX2 Arm Marvell (1n-tx2)](#1-node-thunderx2-arm-marvell-1n-tx2) - - [2-Node-Skylake Xeon Intel (2n-skx)](#2-node-skylake-xeon-intel-2n-skx) - - [2-Node-Denverton Atom Intel (2n-dnv)](#2-node-denverton-atom-intel-2n-dnv) - - [2-Node-IxiaPS1L47 Ixia PSOne L47 (2n-ps1)](#2-node-ixiaps1l47-ixia-psone-l47-2n-ps1) - - [3-Node-Haswell Xeon Intel (3n-skx)](#3-node-haswell-xeon-intel-3n-skx) - - [3-Node-Skylake Xeon Intel (3n-skx)](#3-node-skylake-xeon-intel-3n-skx) - - [3-Node-TaiShan Arm Huawei (3n-tsh)](#3-node-taishan-arm-huawei-3n-tsh) - - [3-Node-MACCHIATObin Arm Marvell](#3-node-macchiatobin-arm-marvell) - - [3-Node-Rangeley Atom Testbeds](#3-node-rangeley-atom-testbeds) - - [Server Management](#server-management) - - [Requirements](#requirements) - - [Addressing](#addressing) - - [LOM (IPMI) VLAN IP Addresses](#lom-ipmi-vlan-ip-addresses) - - [Management VLAN IP Addresses](#management-vlan-ip-addresses) - - [Server Type Specification](#server-type-specification) - - [Server and Port Naming](#server-and-port-naming) - - [Testbeds Configuration](#testbeds-configuration) - - [Per Testbed Server Allocation and Naming](#per-testbed-server-allocation-and-naming) - - [1-Node-Skylake (1n-skx) PROD](#1-node-skylake-1n-skx-prod) - - [1-Node-Thunderx2 (1n-tx2) WIP](#1-node-thunderx2-1n-tx2-wip) - - [2-Node-Skylake (2n-skx) PROD](#2-node-skylake-2n-skx-prod) - - [2-Node-Denverton (2n-dnv) TODO](#2-node-denverton-2n-dnv-todo) - - [2-Node-IxiaPS1L47 (2n-ps1) VERIFY](#2-node-ixiaps1l47-2n-ps1-verify) - - [3-Node-Haswell (3n-hsw) PROD](#3-node-haswell-3n-hsw-prod) - - [3-Node-Skylake (3n-skx) PROD](#3-node-skylake-3n-skx-prod) - - [3-Node-Rangeley (3n-rng) VERIFY](#3-node-rangeley-3n-rng-verify) - - [3-Node-Taishan (3n-tsh) WIP](#3-node-taishan-3n-tsh-wip) - - [3-Node-Mcbin (3n-mcb) TODO](#3-node-mcbin-3n-mcb-todo) - - [Per Testbed Wiring](#per-testbed-wiring) - - [1-Node-Skylake (1n-skx) PROD](#1-node-skylake-1n-skx-prod) - - [1-Node-Thunderx2 (1n-tx2) WIP](#1-node-thunderx2-1n-tx2-wip) - - [2-Node-Skylake (2n-skx) PROD](#2-node-skylake-2n-skx-prod) - - [2-Node-Denverton (2n-dnv) TODO](#2-node-denverton-2n-dnv-todo) - - [2-Node-IxiaPS1L47 (2n-ps1) VERIFY](#2-node-ixiaps1l47-2n-ps1-verify) - - [3-Node-Haswell (3n-hsw) PROD](#3-node-haswell-3n-hsw-prod) - - [3-Node-Skylake (3n-skx) PROD](#3-node-skylake-3n-skx-prod) - - [3-Node-Rangeley (3n-rng) TODO](#3-node-rangeley-3n-rng-todo) - - [3-Node-Taishan (3n-tsh) WIP](#3-node-taishan-3n-tsh-wip) - - [3-Node-Mcbin (3n-mcb) WIP](#3-node-mcbin-3n-mcb-wip) - - [Inventory](#inventory) - - [Appliances](#appliances) - - [Arm Servers](#arm-servers) - - [Xeon and Atom Servers](#xeon-and-atom-servers) - - [Network Interface Cards](#network-interface-cards) - - [Pluggables and Cables](#pluggables-and-cables) - - [Other Parts](#other-parts) - -<!-- /TOC --> +# FD.io CSIT Testbed Specifications + +1. [FD.io CSIT Testbed Specifications](#fdio-csit-testbed-specifications) +1. [Testbeds Overview](#testbeds-overview) + 1. [Summary List](#summary-list) + 1. [1-Node-Skylake Xeon Intel (1n-skx)](#1-node-skylake-xeon-intel-1n-skx) + 1. [1-Node-ThunderX2 Arm Marvell (1n-tx2)](#1-node-thunderx2-arm-marvell-1n-tx2) + 1. [2-Node-Skylake Xeon Intel (2n-skx)](#2-node-skylake-xeon-intel-2n-skx) + 1. [2-Node-Denverton Atom Intel (2n-dnv)](#2-node-denverton-atom-intel-2n-dnv) + 1. [2-Node-IxiaPS1L47 Ixia PSOne L47 (2n-ps1)](#2-node-ixiaps1l47-ixia-psone-l47-2n-ps1) + 1. [3-Node-Haswell Xeon Intel (3n-skx)](#3-node-haswell-xeon-intel-3n-skx) + 1. [3-Node-Skylake Xeon Intel (3n-skx)](#3-node-skylake-xeon-intel-3n-skx) + 1. [3-Node-TaiShan Arm Huawei (3n-tsh)](#3-node-taishan-arm-huawei-3n-tsh) + 1. [3-Node-MACCHIATObin Arm Marvell](#3-node-macchiatobin-arm-marvell) + 1. [3-Node-Rangeley Atom Testbeds](#3-node-rangeley-atom-testbeds) +1. [Server Management](#server-management) + 1. [Requirements](#requirements) + 1. [Addressing](#addressing) + 1. [LOM (IPMI) VLAN IP Addresses](#lom-ipmi-vlan-ip-addresses) + 1. [Management VLAN IP Addresses](#management-vlan-ip-addresses) +1. [Server Type Specification](#server-type-specification) + 1. [Server and Port Naming](#server-and-port-naming) +1. [Testbeds Configuration](#testbeds-configuration) + 1. [Per Testbed Server Allocation and Naming](#per-testbed-server-allocation-and-naming) + 1 [1-Node-Skylake (1n-skx) PROD](#1-node-skylake-1n-skx-prod) + 1 [1-Node-Thunderx2 (1n-tx2) WIP](#1-node-thunderx2-1n-tx2-wip) + 1 [2-Node-Skylake (2n-skx) PROD](#2-node-skylake-2n-skx-prod) + 1 [2-Node-Denverton (2n-dnv) TODO](#2-node-denverton-2n-dnv-todo) + 1 [2-Node-IxiaPS1L47 (2n-ps1) VERIFY](#2-node-ixiaps1l47-2n-ps1-verify) + 1 [3-Node-Haswell (3n-hsw) PROD](#3-node-haswell-3n-hsw-prod) + 1 [3-Node-Skylake (3n-skx) PROD](#3-node-skylake-3n-skx-prod) + 1 [3-Node-Rangeley (3n-rng) VERIFY](#3-node-rangeley-3n-rng-verify) + 1 [3-Node-Taishan (3n-tsh) WIP](#3-node-taishan-3n-tsh-wip) + 1 [3-Node-Mcbin (3n-mcb) TODO](#3-node-mcbin-3n-mcb-todo) + 1. [Per Testbed Wiring](#per-testbed-wiring) + 1 [1-Node-Skylake (1n-skx) PROD](#1-node-skylake-1n-skx-prod) + 1 [1-Node-Thunderx2 (1n-tx2) WIP](#1-node-thunderx2-1n-tx2-wip) + 1 [2-Node-Skylake (2n-skx) PROD](#2-node-skylake-2n-skx-prod) + 1 [2-Node-Denverton (2n-dnv) TODO](#2-node-denverton-2n-dnv-todo) + 1 [2-Node-IxiaPS1L47 (2n-ps1) VERIFY](#2-node-ixiaps1l47-2n-ps1-verify) + 1 [3-Node-Haswell (3n-hsw) PROD](#3-node-haswell-3n-hsw-prod) + 1 [3-Node-Skylake (3n-skx) PROD](#3-node-skylake-3n-skx-prod) + 1 [3-Node-Rangeley (3n-rng) TODO](#3-node-rangeley-3n-rng-todo) + 1 [3-Node-Taishan (3n-tsh) WIP](#3-node-taishan-3n-tsh-wip) + 1 [3-Node-Mcbin (3n-mcb) WIP](#3-node-mcbin-3n-mcb-wip) +1. [Inventory](#inventory) + 1. [Appliances](#appliances) + 1. [Arm Servers](#arm-servers) + 1. [Xeon and Atom Servers](#xeon-and-atom-servers) + 1. [Network Interface Cards](#network-interface-cards) + 1. [Pluggables and Cables](#pluggables-and-cables) + 1. [Other Parts](#other-parts) + ## FD.io CSIT Testbed Specifications This note includes specification of the physical testbed infrastructure diff --git a/docs/lab/testbeds_sm_skx_hw_bios_cfg.md b/docs/lab/testbeds_sm_skx_hw_bios_cfg.md index 6a2bfd782d..5aecf594cb 100644 --- a/docs/lab/testbeds_sm_skx_hw_bios_cfg.md +++ b/docs/lab/testbeds_sm_skx_hw_bios_cfg.md @@ -1,44 +1,39 @@ -<!-- TOC depthFrom:1 depthTo:6 withLinks:1 updateOnSave:1 orderedList:0 --> - - - [SuperMicro Xeon Skylake Servers - Hardware and BIOS Configuration](#supermicro-xeon-skylake-servers-hardware-and-bios-configuration) - - [Linux lscpu TODO](#linux-lscpu-todo) - - [Linux dmidecode pci TODO](#linux-dmidecode-pci-todo) - - [Linux dmidecode memory TODO](#linux-dmidecode-memory-todo) - - [Xeon Skx Server BIOS Configuration](#xeon-skx-server-bios-configuration) - - [Boot Feature](#boot-feature) - - [CPU Configuration](#cpu-configuration) - - [Advanced Power Management Configuration](#advanced-power-management-configuration) - - [CPU P State Control](#cpu-p-state-control) - - [Hardware PM State Control](#hardware-pm-state-control) - - [CPU C State Control](#cpu-c-state-control) - - [Package C State Control](#package-c-state-control) - - [CPU T State Control](#cpu-t-state-control) - - [Chipset Configuration](#chipset-configuration) - - [North Bridge](#north-bridge) - - [UPI Configuration](#upi-configuration) - - [Memory Configuration](#memory-configuration) - - [IIO Configuration](#iio-configuration) - - [CPU1 Configuration](#cpu1-configuration) - - [CPU2 Configuration](#cpu2-configuration) - - [South Bridge](#south-bridge) - - [PCIe/PCI/PnP Configuration](#pciepcipnp-configuration) - - [ACPI Settings](#acpi-settings) - - [DMIDECODE](#dmidecode) - - [Xeon Skx Server Firmware Inventory](#xeon-skx-server-firmware-inventory) - -<!-- /TOC --> - -## SuperMicro Xeon Skylake Servers - Hardware and BIOS Configuration - -### Linux lscpu TODO - -### Linux dmidecode pci TODO - -### Linux dmidecode memory TODO - -### Xeon Skx Server BIOS Configuration - -#### Boot Feature +# SuperMicro Xeon Skylake Servers - Hardware and BIOS Configuration + +1. [Linux lscpu TODO](#linux-lscpu-todo) +1. [Linux dmidecode pci TODO](#linux-dmidecode-pci-todo) +1. [Linux dmidecode memory TODO](#linux-dmidecode-memory-todo) +1. [Xeon Skx Server BIOS Configuration](#xeon-skx-server-bios-configuration) + 1. [Boot Feature](#boot-feature) + 1. [CPU Configuration](#cpu-configuration) + 1. [Advanced Power Management Configuration](#advanced-power-management-configuration) + 1. [CPU P State Control](#cpu-p-state-control) + 1. [Hardware PM State Control](#hardware-pm-state-control) + 1. [CPU C State Control](#cpu-c-state-control) + 1. [Package C State Control](#package-c-state-control) + 1. [CPU T State Control](#cpu-t-state-control) + 1. [Chipset Configuration](#chipset-configuration) + 1. [North Bridge](#north-bridge) + 1. [UPI Configuration](#upi-configuration) + 1. [Memory Configuration](#memory-configuration) + 1. [IIO Configuration](#iio-configuration) + 1. [CPU1 Configuration](#cpu1-configuration) + 1. [CPU2 Configuration](#cpu2-configuration) + 1. [South Bridge](#south-bridge) + 1. [PCIe/PCI/PnP Configuration](#pciepcipnp-configuration) + 1. [ACPI Settings](#acpi-settings) + 1. [DMIDECODE](#dmidecode) +1. [Xeon Skx Server Firmware Inventory](#xeon-skx-server-firmware-inventory) + +## Linux lscpu TODO + +## Linux dmidecode pci TODO + +## Linux dmidecode memory TODO + +## Xeon Skx Server BIOS Configuration + +### Boot Feature ``` | Quiet Boot [Enabled] |Boot option | @@ -58,7 +53,7 @@ | Throttle on Power Fail [Disabled] | | ``` -#### CPU Configuration +### CPU Configuration ``` | Processor Configuration |Enables Hyper Threading | @@ -93,7 +88,7 @@ |> Advanced Power Management Configuration | | ``` -##### Advanced Power Management Configuration +#### Advanced Power Management Configuration ``` | Advanced Power Management Configuration |Switch CPU Power Management | @@ -108,7 +103,7 @@ |> CPU T State Control | | ``` -###### CPU P State Control +##### CPU P State Control ``` | CPU P State Control |Enable/Disable EIST | @@ -117,7 +112,7 @@ | EIST PSD Function [HW_ALL] | | ``` -###### Hardware PM State Control +##### Hardware PM State Control ``` | Hardware PM State Control |Disable: Hardware chooses a | @@ -131,7 +126,7 @@ | |P-state (no OS guidance) | ``` -###### CPU C State Control +##### CPU C State Control ``` | CPU C State Control |Autonomous Core C-State | @@ -141,7 +136,7 @@ | Enhanced Halt State (C1E) [Disable] | | ``` -###### Package C State Control +##### Package C State Control ``` | Package C State Control |Package C State limit | @@ -149,7 +144,7 @@ | Package C State [C0/C1 state] | | ``` -###### CPU T State Control +##### CPU T State Control ``` | CPU T State Control |Enable/Disable Software | @@ -157,7 +152,7 @@ | Software Controlled T-States [Disable] | | ``` -##### Chipset Configuration +#### Chipset Configuration ``` | WARNING: Setting wrong values in below sections may cause |North Bridge Parameters | @@ -166,7 +161,7 @@ |> South Bridge | | ``` -###### North Bridge +##### North Bridge ``` |> UPI Configuration |Displays and provides | @@ -174,7 +169,7 @@ |> IIO Configuration |Settings | ``` -###### UPI Configuration +##### UPI Configuration ``` | UPI Configuration |Choose Topology Precedence | @@ -199,7 +194,7 @@ | Isoc Mode [Auto] | | ``` -###### Memory Configuration +##### Memory Configuration ``` | |POR - Enforces Plan Of | @@ -220,7 +215,7 @@ |> Memory RAS Configuration | | ``` -###### IIO Configuration +##### IIO Configuration ``` | IIO Configuration |Expose IIO DFX devices and | @@ -238,7 +233,7 @@ | PCI-E Completion Timeout Disable [No] | | ``` -###### CPU1 Configuration +##### CPU1 Configuration ``` | IOU0 (IIO PCIe Br1) [Auto] |Selects PCIe port | @@ -249,7 +244,7 @@ |> CPU1 SLOT9 PCI-E 3.0 X16 | | ``` -###### CPU2 Configuration +##### CPU2 Configuration ``` | IOU0 (IIO PCIe Br1) [Auto] |Selects PCIe port | @@ -260,7 +255,7 @@ |> CPU2 SLOT10 PCI-E 3.0 X16 | | ``` -##### South Bridge +#### South Bridge ``` | |Enables Legacy USB support. | @@ -280,7 +275,7 @@ | Azalia PME Enable [Disabled] | | ``` -#### PCIe/PCI/PnP Configuration +### PCIe/PCI/PnP Configuration ``` | PCI Bus Driver Version A5.01.12 |Enables or Disables 64bit | @@ -308,7 +303,7 @@ |> Network Stack Configuration | | ``` -#### ACPI Settings +### ACPI Settings ``` | ACPI Settings |Enable or Disable Non | @@ -319,7 +314,7 @@ | ACPI Sleep State [S3 (Suspend to RAM)] | | ``` -#### DMIDECODE +### DMIDECODE ``` # dmidecode 3.1 @@ -520,7 +515,7 @@ Power/Performance Control ``` -### Xeon Skx Server Firmware Inventory +## Xeon Skx Server Firmware Inventory ``` Host. IPMI IP. BIOS. CPLD. Aptio SU. CPU Microcode. PCI Bus. ME Operation FW. X710 Firmware. XXV710 Firmware. i40e. diff --git a/docs/lab/testbeds_ucs_hsw_hw_bios_cfg.md b/docs/lab/testbeds_ucs_hsw_hw_bios_cfg.md index ed4b1f561d..9c330a125e 100644 --- a/docs/lab/testbeds_ucs_hsw_hw_bios_cfg.md +++ b/docs/lab/testbeds_ucs_hsw_hw_bios_cfg.md @@ -1,16 +1,11 @@ -<!-- TOC depthFrom:1 depthTo:6 withLinks:1 updateOnSave:1 orderedList:0 --> +# Cisco UCS c240m4 Xeon Haswell Servers - Hardware and BIOS Configuration - - [Cisco UCS c240m4 Xeon Haswell Servers - Hardware and BIOS Configuration](#cisco-ucs-c240m4-xeon-haswell-servers-hardware-and-bios-configuration) - - [Linux lscpu](#linux-lscpu) - - [Linux dmidecode pci](#linux-dmidecode-pci) - - [Linux dmidecode memory](#linux-dmidecode-memory) - - [Xeon Hsw Server BIOS Configuration](#xeon-hsw-server-bios-configuration) +1. [Linux lscpu](#linux-lscpu) +1. [Linux dmidecode pci](#linux-dmidecode-pci) +1. [Linux dmidecode memory](#linux-dmidecode-memory) +1. [Xeon Hsw Server BIOS Configuration](#xeon-hsw-server-bios-configuration) -<!-- /TOC --> - -## Cisco UCS c240m4 Xeon Haswell Servers - Hardware and BIOS Configuration - -### Linux lscpu +## Linux lscpu ``` $ lscpu @@ -42,7 +37,7 @@ cqm_llc cqm_occup_llc dtherm arat pln pts ``` -### Linux dmidecode pci +## Linux dmidecode pci ``` $ dmidecode --type 9 | grep 'Handle\|Slot\|Type\|Address' @@ -148,7 +143,7 @@ Bus Address: 0000:00:1f.7 ``` -### Linux dmidecode memory +## Linux dmidecode memory ``` $ dmidecode -t memory @@ -669,7 +664,7 @@ Configured Clock Speed: Unknown ``` -### Xeon Hsw Server BIOS Configuration +## Xeon Hsw Server BIOS Configuration ``` C240 / # scope bios |