feat(Docs): Add AWS methodology

Signed-off-by: pmikus <pmikus@cisco.com>
Change-Id: I766c1b8ce85e174946503a3d65f8239d8fec1bcd

6 files changed, 290 insertions, 53 deletions

diff --git a/docs/report/introduction/methodology.rst b/docs/report/introduction/methodology.rst
index 158f541619..3e1030bf72 100644
--- a/docs/report/introduction/methodology.rst
+++ b/docs/report/introduction/methodology.rst
@@ -27,3 +27,4 @@ Test Methodology
     methodology_lxc_drc_container_memif
     methodology_nfv_service_density
     methodology_vpp_device_functional
+    methodology_aws/index
diff --git a/docs/report/introduction/methodology_aws/aws_ami.rst b/docs/report/introduction/methodology_aws/aws_ami.rst
new file mode 100644
index 0000000000..2a94a6b0cc
--- /dev/null
+++ b/docs/report/introduction/methodology_aws/aws_ami.rst
@@ -0,0 +1,46 @@
+AWS Amazon Machine Images
+-------------------------
+
+An Amazon Machine Image (AMI) provides the information required to launch an
+instance. CSIT is using Amazon EBS where the root device for an instance
+launched from the AMI is an Amazon Elastic Block Store (Amazon EBS) volume
+created from an Amazon EBS snapshot.
+
+A the TG and SUT instances have slightly different software requirements,
+we are defining two AMIs for TG and SUT separately. AMI details examples:
+
+- AMI Name: csit_c5n_ubuntu_focal_tg
+- Platform details: Linux/UNIX
+- Architecture: x86_64
+- Usage operation: RunInstances
+- Image Type: machine
+- Virtualization type: hvm
+- Description: CSIT TG image based on Ubuntu Focal
+- Root Device Name: /dev/sda1
+- Root Device Type: ebs
+
+- AMI Name: csit_c5n_ubuntu_focal_sut
+- Platform details: Linux/UNIX
+- Architecture: x86_64
+- Usage operation: RunInstances
+- Image Type: machine
+- Virtualization type: hvm
+- Description: CSIT SUT image based on Ubuntu Focal
+- Root Device Name: /dev/sda1
+- Root Device Type: ebs
+
+Both TG and SUT AMIs are created manually before launching topology and are not
+part of automated scripts. To create CSIT AMIs:
+
+::
+
+  cd csit/fdio.infra.packer/aws_c5n/
+  packer init
+  packer build
+
+Building AMIs requires Hashicorp Packer with Amazon plugin installed.
+
+Reference
+---------
+
+.. [aws_ami] `Amazon Machine Images <https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html>`_
diff --git a/docs/report/introduction/methodology_aws/aws_instances.rst b/docs/report/introduction/methodology_aws/aws_instances.rst
new file mode 100644
index 0000000000..204c15eb79
--- /dev/null
+++ b/docs/report/introduction/methodology_aws/aws_instances.rst
@@ -0,0 +1,59 @@
+AWS Instances
+-------------
+
+CSIT is using C5n instances for high compute applications that can take
+advantage of improved network throughput and packet rate performance. C5n
+instances offers up to 100 Gbps network bandwidth and increased memory over
+comparable C5 instances [aws_it]_.
+
+Features:
+
+- 3.0 GHz Intel Xeon Platinum processors with Intel Advanced Vector Extension
+  512 (AVX-512) instruction set.
+- Sustained all core Turbo frequency of up to 3.4GHz, and single core turbo
+  frequency of up to 3.5 GHz.
+- Requires HVM AMIs that include drivers for ENA and NVMe [aws_ena]_.
+- Network bandwidth increases to up to 100 Gbps, delivering increased
+  performance for network intensive applications.
+- Powered by the AWS Nitro System, a combination of dedicated hardware and
+  lightweight hypervisor.
+
++-------------+------+--------------+------------------------+-----------------------------+----------------------+
+| Model       | vCPU | Memory (GiB) | Instance Storage (GiB) | Network Bandwidth (Gbps)*** | EBS Bandwidth (Mbps) |
++=============+======+==============+========================+=============================+======================+
+|c5n.large    |    2 |         5.25 |        EBS-Only        |                    Up to 25 |          Up to 4,750 |
++-------------+------+--------------+------------------------+-----------------------------+----------------------+
+|c5n.xlarge   |    4 |        10.5  |        EBS-Only        |                    Up to 25 |          Up to 4,750 |
++-------------+------+--------------+------------------------+-----------------------------+----------------------+
+|c5n.2xlarge  |    8 |        21    |        EBS-Only        |                    Up to 25 |          Up to 4,750 |
++-------------+------+--------------+------------------------+-----------------------------+----------------------+
+|c5n.4xlarge  |   16 |        42    |        EBS-Only        |                    Up to 25 |                4,750 |
++-------------+------+--------------+------------------------+-----------------------------+----------------------+
+|c5n.9xlarge  |   36 |        96    |        EBS-Only        |                          50 |                9,500 |
++-------------+------+--------------+------------------------+-----------------------------+----------------------+
+|c5n.18xlarge |   72 |       192    |        EBS-Only        |                         100 |               19,000 |
++-------------+------+--------------+------------------------+-----------------------------+----------------------+
+|c5n.metal    |   72 |       192    |        EBS-Only        |                         100 |               19,000 |
++-------------+------+--------------+------------------------+-----------------------------+----------------------+
+
+CSIT is configured by default to use `c5n.4xlarge` in `eu-central-1` AWS region
+due to allocation stability issues with `c5n.9xlarge` in `eu-central-1` region.
+
+
+AWS Elastic Network Adapter (ENA)
+---------------------------------
+
+Enhanced networking uses single root I/O virtualization (SR-IOV) to provide
+high-performance networking capabilities on supported instance types. SR-IOV is
+a method of device virtualization that provides higher I/O performance and lower
+CPU utilization when compared to traditional virtualized network interfaces
+[aws_ena]_.
+
+Current advertised limits are captured in [aws_limits]_.
+
+Reference
+---------
+
+.. [aws_it] `Instance types <https://aws.amazon.com/ec2/instance-types/>`_
+.. [aws_ena] `Elastic Network Adapter <https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/enhanced-networking-ena.html>`_
+.. [aws_limits] `<https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/compute-optimized-instances.html>`_
\ No newline at end of file
diff --git a/docs/report/introduction/methodology_aws/aws_terraform.rst b/docs/report/introduction/methodology_aws/aws_terraform.rst
new file mode 100644
index 0000000000..7c73e04b20
--- /dev/null
+++ b/docs/report/introduction/methodology_aws/aws_terraform.rst
@@ -0,0 +1,176 @@
+Terraform-aws-csit modules
+--------------------------
+
+Terraform-aws-csit module is IaaC - infrastructure as a code. Module uses the
+Amazon Web Services (AWS) provider to interact with resources provided by AWS
+to orchestrate virtual environment for running CSIT tests.
+
+- `aws <https://registry.terraform.io/providers/hashicorp/aws/latest/>`_.
+
+Compatibility
+~~~~~~~~~~~~~
+
++-----------+----------------+
+| Software  | OSS Version    |
++===========+================+
+| Terraform | 1.0.3 or newer |
++-----------+----------------+
+| Vault     | 1.8.4 or newer |
++-----------+----------------+
+
+Requirements
+~~~~~~~~~~~~
+
+Required modules and provider
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+- `aws <https://registry.terraform.io/providers/hashicorp/aws/latest>`_.
+- `null <https://registry.terraform.io/providers/hashicorp/null/latest>`_.
+- `tls <https://registry.terraform.io/providers/hashicorp/tls>`_.
+- `vault <https://registry.terraform.io/providers/hashicorp/vault>`_.
+
+Required software
+^^^^^^^^^^^^^^^^^
+
+- `Vault <https://releases.hashicorp.com/vault/>`_ service available on
+  specified ip/port.
+
+Usage
+~~~~~
+
+- OPTIONAL: Enable logging
+  Terraform does not have logging enabled by default, to enable logging
+  to stderr, set up TF_LOG variable with specified loglevel:
+  Available loglevels: TRACE, DEBUG, INFO, WARN, ERROR
+
+  ::
+
+    export TF_LOG="LOGLEVEL"
+
+  It is also possible to store logged output to a file by setting up
+  TF_LOG_PATH variable:
+
+  ::
+
+    export TF_LOG_PATH="path/to/logfile"
+
+- Run Terraform in a given root module folder depending on chosen testbed
+  topology. Terraform will deploy and configure instances and other resources,
+  all of these resources can be later identified on AWS via Environment tag.
+  By default, Environment tag "CSIT-AWS" is used.
+  Example:
+
+  ::
+
+    cd fdio.infra.terraform/2n_aws_c5n/
+    terraform init
+    terraform plan
+    terraform apply
+
+  This will deploy environment with default values, you can check the defaults
+  in ./2n_aws_c5n/main.tf and ./2n_aws_c5n/variables.tf files
+
+  If you would like to change some of these values, you can:
+
+  1. Set up TF_VAR_* environment variables prior to running 'terraform apply':
+
+     ::
+
+       export TF_VAR_testbed_name="testbed1"
+
+  2. Use '-var=varname=value' flag when running 'terraform apply':
+
+     ::
+
+       terraform apply -var=testbed_name=testbed1
+
+  Note:
+  Only variables defined in variables.tf file of the root module can be
+  changed using these methods.
+
+- To clean up the AWS environment and remove all used resources, run:
+
+  ::
+
+    terraform destroy
+
+Example usage
+~~~~~~~~~~~~~
+
+These are the default values for the AWS modules. The following example is
+2n topology (3n topology variant is very similar). Few variables are defined in
+a `variable.tf` file.
+
+::
+
+  module "deploy" {
+    source = "./deploy"
+
+    # Parameters starting with var. can be set using "TF_VAR_*" environment
+    # variables or -var parameter when running "terraform apply", for default
+    # values see ./variables.tf
+    testbed_name          = var.testbed_name
+    topology_name         = var.topology_name
+    environment_name      = var.environment_name
+    resources_name_prefix = var.resources_name_prefix
+
+    # AWS general
+    region        = var.region
+    avail_zone    = var.avail_zone
+    instance_type = var.instance_type
+    ami_image_tg  = var.ami_image_tg
+    ami_image_sut = var.ami_image_sut
+
+    # AWS Network
+    vpc_cidr_mgmt = "192.168.0.0/24"
+    vpc_cidr_b    = "192.168.10.0/24"
+    vpc_cidr_c    = "200.0.0.0/24"
+    vpc_cidr_d    = "192.168.20.0/24"
+
+    tg_mgmt_ip   = "192.168.0.10"
+    dut1_mgmt_ip = "192.168.0.11"
+
+    tg_if1_ip   = "192.168.10.254"
+    tg_if2_ip   = "192.168.20.254"
+    dut1_if1_ip = "192.168.10.11"
+    dut1_if2_ip = "192.168.20.11"
+
+    trex_dummy_cidr_port_0 = "10.0.0.0/24"
+    trex_dummy_cidr_port_1 = "20.0.0.0/24"
+
+    # Ansible
+    ansible_python_executable = "/usr/bin/python3"
+    ansible_file_path         = "../../fdio.infra.ansible/site.yaml"
+    ansible_topology_path     = "../../fdio.infra.ansible/cloud_topology.yaml"
+    ansible_provision_pwd     = "Csit1234"
+
+    # First run
+    first_run_commands = [
+      "sudo sed -i 's/^PasswordAuthentication/#PasswordAuthentication/' /etc/ssh/sshd_config",
+      "sudo systemctl restart sshd",
+      "sudo useradd --create-home -s /bin/bash provisionuser",
+      "echo 'provisionuser:Csit1234' | sudo chpasswd",
+      "echo 'provisionuser ALL = (ALL) NOPASSWD: ALL' | sudo tee -a /etc/sudoers",
+      "sudo useradd --create-home -s /bin/bash testuser",
+      "echo 'testuser:Csit1234' | sudo chpasswd",
+      "echo 'testuser ALL = (ALL) NOPASSWD: ALL' | sudo tee -a /etc/sudoers"
+    ]
+  }
+
+Secrets & Credentials
+~~~~~~~~~~~~~~~~~~~~~
+
+Set credentials manually
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+To set the credentials manually you first need to tell the module to not fetch
+credentials from Vault. To do that, set `provider "aws"` `access_key` and
+`secret_key` to custom value or use credentials file as a source.
+
+::
+
+  provider "aws" {
+    region     = var.region
+    access_key = data.vault_aws_access_credentials.creds.access_key
+    secret_key = data.vault_aws_access_credentials.creds.secret_key
+  }
diff --git a/docs/report/introduction/methodology_aws/index.rst b/docs/report/introduction/methodology_aws/index.rst
new file mode 100644
index 0000000000..602f2c4b3b
--- /dev/null
+++ b/docs/report/introduction/methodology_aws/index.rst
@@ -0,0 +1,8 @@
+Amazon Web Services
+===================
+
+.. toctree::
+
+    aws_terraform
+    aws_ami
+    aws_instances
diff --git a/fdio.infra.terraform/README.md b/fdio.infra.terraform/README.md
index 92cdfb2389..7bad380b1e 100644
--- a/fdio.infra.terraform/README.md
+++ b/fdio.infra.terraform/README.md
@@ -13,59 +13,6 @@ Testbed deployment - Amazon AWS
 - ./2n_aws_c5n/
 - ./3n_aws_c5n/
 
-### Getting Started
-
-Tested setup:
-- Install and configure prerequisities as specified.
-
-- OPTIONAL: Enable logging
-  Terraform does not have logging enabled by default, to enable logging
-  to stderr, set up TF_LOG variable with specified loglevel:
-  Available loglevels: TRACE, DEBUG, INFO, WARN, ERROR
-
-    export TF_LOG="LOGLEVEL"
-
-  It is also possible to store logged output to a file by setting up
-  TF_LOG_PATH variable:
-    export TF_LOG_PATH="path/to/logfile"
-
-- Run Terraform in a given root module folder depending on chosen testbed
-  topology. Terraform will deploy and configure instances and other resources,
-  all of these resources can be later identified on AWS via Environment tag.
-  By default, Environment tag "CSIT-AWS" is used.
-  Example:
-    cd fdio.infra.terraform/2n_aws_c5n/
-    terraform init
-    terraform apply
-
-  This will deploy environment with default values, you can check the defaults
-  in ./2n_aws_c5n/main.tf and ./2n_aws_c5n/variables.tf files
-
-  If you would like to change some of these values, you can:
-  1) Set up TF_VAR_* environment variables prior to running 'terraform apply':
-      export TF_VAR_testbed_name="testbed1"
-
-  2) Use '-var=varname=value' flag when running 'terraform apply':
-      terraform apply -var=testbed_name=testbed1
-
-  Note:
-  Only variables defined in variables.tf file of the root module can be
-  changed using these methods.
-
-- Run the tests
-  1) To run some tests use the VPP performance bootstrap script, e.g:
-      cd ../resources/libraries/bash/entry
-      ./bootstrap_vpp_perf.sh csit-vpp-perf-mrr-daily-master-2n-aws
-
-  2) To run only selected tests based on TAGS, export environment variables
-     before running the test suite and run a verify JOB, e.g:
-      export GERRIT_EVENT_TYPE="comment-added"
-      export GERRIT_EVENT_COMMENT_TEXT="1cAND64bANDmrrANDethip4-ip4base"
-      ./bootstrap_vpp_perf.sh csit-vpp-perf-verify-master-2n-aws
-
-- To clean up the AWS environment and remove all used resources, run:
-    terraform destroy
-
 ## Azure
 
 Testbed deployment - Microsoft Azure