CSIT-1437 Create portable host-setup document

- Ansible simplified and optimalized
- Porting global variables into single file
- Rework the README.md
- Tested outside LF environemnt

Change-Id: I124cffa5c4510edf365e8b558da0c8c498f55d50
Signed-off-by: Peter Mikus <pmikus@cisco.com>

1 files changed, 246 insertions, 0 deletions

diff --git a/resources/tools/testbed-setup/README.rst b/resources/tools/testbed-setup/README.rst
new file mode 100644
index 0000000000..738dffad49
--- /dev/null
+++ b/resources/tools/testbed-setup/README.rst
@@ -0,0 +1,246 @@
+Testbed Setup
+=============
+
+Introduction
+------------
+
+This directory contains the *high-level* process to set up a hardware machine
+as a CSIT testbed, either for use as a physical performance testbed host or as
+a vpp_device host.
+
+Code in this directory is NOT executed as part of a regular CSIT test case
+but is stored here for ad-hoc installation of HW, archiving and documentation
+purposes.
+
+Setting up a hardware host
+--------------------------
+
+Documentation below is step by step tutorial and assumes an understanding of PXE
+boot and Ansible and managing physical hardware via CIMC or IPMI.
+
+This process is not specific for LF lab, but associated files and code, is based
+on the assumption that it runs in LF environment. If run elsewhere, changes
+will be required in following files:
+
+#. Inventory directory: `ansible/inventories/sample_inventory/`
+#. Inventory files: `ansible/inventories/sample_inventory/hosts`
+#. Kickseed file: `pxe/ks.cfg`
+#. DHCPD file: `pxe/dhcpd.conf`
+#. Bootscreen file: `boot-screens_txt.cfg`
+
+The process below assumes that there is a host used for bootstrapping (referred
+to as "PXE bootstrap server" below).
+
+Prepare the PXE bootstrap server when there is no http server AMD64
+```````````````````````````````````````````````````````````````````
+
+#. Clone the csit repo:
+
+   .. code-block:: bash
+
+      git clone https://gerrit.fd.io/r/csit
+      cd csit/resources/tools/testbed-setup/pxe
+
+#. Setup prerequisities (isc-dhcp-server tftpd-hpa nginx-light ansible):
+
+   .. code-block:: bash
+
+      sudo apt-get install isc-dhcp-server tftpd-hpa nginx-light ansible
+
+#. Edit dhcpd.cfg:
+
+   .. code-block:: bash
+
+      sudo cp dhcpd.cfg /etc/dhcp/
+      sudo service isc-dhcp-server restart
+      sudo mkdir /mnt/cdrom
+
+#. Download Ubuntu 18.04 LTS - X86_64:
+
+   .. code-block:: bash
+
+      wget http://cdimage.ubuntu.com/ubuntu/releases/18.04/release/ubuntu-18.04-server-amd64.iso
+      sudo mount -o loop ubuntu-18.04-server-amd64.iso /mnt/cdrom/
+      sudo cp -r /mnt/cdrom/install/netboot/* /var/lib/tftpboot/
+
+      # Figure out root folder for NGINX webserver. The configuration is in one
+      # of the files in /etc/nginx/conf.d/, /etc/nginx/sites-enabled/ or in
+      # /etc/nginx/nginx.conf under section server/root. Save the path to
+      # variable WWW_ROOT.
+      sudo mkdir -p ${WWW_ROOT}/download/ubuntu
+      sudo cp -r /mnt/cdrom/* ${WWW_ROOT}/download/ubuntu/
+      sudo cp /mnt/cdrom/ubuntu/isolinux/ldlinux.c32 /var/lib/tftpboot
+      sudo cp /mnt/cdrom/ubuntu/isolinux/libcom32.c32 /var/lib/tftpboot
+      sudo cp /mnt/cdrom/ubuntu/isolinux/libutil.c32 /var/lib/tftpboot
+      sudo cp /mnt/cdrom/ubuntu/isolinux/chain.c32 /var/lib/tftpboot
+      sudo umount /mnt/cdrom
+
+#. Edit ks.cfg and replace IP address of PXE bootstrap server and subdir in
+   `/var/www` (in this case `/var/www/download`):
+
+   .. code-block:: bash
+
+      sudo cp ks.cfg ${WWW_ROOT}/download/ks.cfg
+
+#. Edit boot-screens_txt.cfg and replace IP address of PXE bootstrap server and
+   subdir in `/var/www` (in this case `/var/www/download`):
+
+   .. code-block:: bash
+
+      sudo cp boot-screens_txt.cfg /var/lib/tftpboot/ubuntu-installer/amd64/boot-screens/txt.cfg
+      sudo cp syslinux.cfg /var/lib/tftpboot/ubuntu-installer/amd64/boot-screens/syslinux.cfg
+
+New testbed host - manual preparation
+`````````````````````````````````````
+
+Set CIMC/IPMI address, username, password and hostname an BIOS.
+
+Bootstrap the host
+``````````````````
+
+Convenient way to re-stage host via script:
+
+.. code-block:: bash
+
+   sudo ./bootstrap_setup_testbed.sh <linux_ip> <mgmt_ip> <username> <pass>
+
+Optional: CIMC - From PXE boostrap server
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+#. Initialize args.ip: Power-Off, reset BIOS defaults, Enable console redir, get
+   LOM MAC addr:
+
+   .. code-block:: bash
+
+     ./cimc.py -u admin -p Cisco1234 $CIMC_ADDRESS -d -i
+
+#. Adjust BIOS settings:
+
+   .. code-block:: bash
+
+      ./cimc.py -u admin -p Cisco1234 $CIMC_ADDRESS -d -s '<biosVfIntelHyperThreadingTech rn="Intel-HyperThreading-Tech" vpIntelHyperThreadingTech="disabled" />' -s '<biosVfEnhancedIntelSpeedStepTech rn="Enhanced-Intel-SpeedStep-Tech" vpEnhancedIntelSpeedStepTech="disabled" />' -s '<biosVfIntelTurboBoostTech rn="Intel-Turbo-Boost-Tech" vpIntelTurboBoostTech="disabled" />'
+
+#. If RAID is not created in CIMC. Create RAID array. Reboot:
+
+   .. code-block:: bash
+
+      ./cimc.py -u admin -p Cisco1234 $CIMC_ADDRESS -d --wipe
+      ./cimc.py -u admin -p Cisco1234 $CIMC_ADDRESS -d -r -rl 1 -rs <disk size> -rd '[1,2]'
+
+#. Reboot server with boot from PXE (restart immediately):
+
+   .. code-block:: bash
+
+      ./cimc.py -u admin -p Cisco1234 $CIMC_ADDRESS -d -pxe
+
+#. Set the next boot from HDD (without restart). Execute while Ubuntu install
+   is running:
+
+   .. code-block:: bash
+
+      ./cimc.py -u admin -p Cisco1234 $CIMC_ADDRESS -d -hdd
+
+Optional: IPMI - From PXE boostrap server
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+#. Get MAC address of LAN0:
+
+   .. code-block:: bash
+
+      ipmitool -U ADMIN -H $HOST_ADDRESS raw 0x30 0x21 | tail -c 18
+
+#. Reboot into PXE for next boot only:
+
+   .. code-block:: bash
+
+      ipmitool -I lanplus -H $HOST_ADDRESS -U ADMIN chassis bootdev pxe
+      ipmitool -I lanplus -H $HOST_ADDRESS -U ADMIN power reset
+
+#. For live watching SOL (Serial-over-LAN console):
+
+   .. code-block:: bash
+
+      ipmitool -I lanplus -H $HOST_ADDRESS -U ADMIN sol activate
+      ipmitool -I lanplus -H $HOST_ADDRESS -U ADMIN sol deactivate
+
+Ansible machine
+~~~~~~~~~~~~~~~
+
+Prerequisities for running Ansible
+..................................
+
+- Ansible can run on any machine that has direct SSH connectivity to target
+  machines that will be provisioned (does not need to be PXE server).
+- User `testuser` with password `Csit1234` is created with home folder
+  initialized on all target machines that will be provisioned.
+- SSH keys for no pass access are copied to all target machines that will be
+  provisioned: `ssh-copy-id x.x.x.x`.
+- Inventory directory is created with same or similar content as
+  `inventories/lf_inventory` in `inventories/` directory (`sample_inventory`
+  can be used).
+- Group variables in `ansible/inventories/<inventory>/group_vars/all.yaml` are
+  adjusted per environment. Special attention to `proxy_env` variable.
+- Host variables in `ansible/inventories/<inventory>/host_vars/x.x.x.x.yaml` are
+  defined.
+
+Ansible structure
+.................
+
+Ansible is defining roles `TG` (Traffic Generator), `SUT` (System Under Test),
+`VPP_DEVICE` (vpp_device host for functional testing).
+
+Each Host has corresponding Ansible role mapped and is applied only if Host
+with that role is present in inventory file. As a part of optimization the role
+`common` contains Ansible tasks applied for all Hosts.
+
+.. note::
+
+   You may see `[WARNING]: Could not match supplied host pattern, ignoring:
+   <role>` in case you have not define hosts for that particular role.
+
+Ansible structure is described below:
+
+.. code-block:: bash
+
+   .
+   ├── inventories                     # Contains all inventories.
+   │   ├── sample_inventory            # Sample, free for edits outside of LF.
+   │   │   ├── group_vars              # Variables applied for all hosts.
+   │   │   │   └── all.yaml
+   │   │   ├── hosts                   # Inventory list with sample hosts.
+   │   │   └── host_vars               # Variables applied for single host only.
+   │   │       └── 1.1.1.1.yaml        # Sample host with IP 1.1.1.1
+   │   └── lf_inventory                # Linux Foundation inventory.
+   │       ├── group_vars
+   │       │   └── all.yaml
+   │       ├── hosts
+   │       └── host_vars
+   ├── roles                           # CSIT roles.
+   │   ├── common                      # Role applied for all hosts.
+   │   ├── sut                         # Role applied for all SUTs only.
+   │   ├── tg                          # Role applied for all TGs only.
+   │   ├── tg_sut                      # Role applied for TGs and SUTs only.
+   │   └── vpp_device                  # Role applied for vpp_device only.
+   ├── site.yaml                       # Main playbook.
+   ├── sut.yaml                        # SUT playbook.
+   ├── tg.yaml                         # TG playbook.
+   ├── vault_pass                      # Main password for vualt.
+   ├── vault.yml                       # Ansible vualt storage.
+   └── vpp_device.yaml                 # vpp_device playbook.
+
+Running Ansible
+...............
+
+#. Go to ansible directory: `cd csit/resources/tools/testbed-setup/ansible`
+#. Run ansible on selected hosts:
+   `ansible-playbook --vault-id vault_pass --extra-vars '@vault.yml' --inventory <inventory_file> site.yaml --limit x.x.x.x`
+
+.. note::
+
+   In case you want to provision only particular role. You can use tags: `tg`,
+   `sut`, `vpp_device`.
+
+Reboot hosts
+------------
+
+Manually reboot hosts after Ansible provisioning succeeded.