diff options
Diffstat (limited to 'docker/scripts/README.md')
| -rw-r--r-- | docker/scripts/README.md | 502 |
1 files changed, 10 insertions, 492 deletions
diff --git a/docker/scripts/README.md b/docker/scripts/README.md index 2c98aa95..5fb4cee4 100644 --- a/docker/scripts/README.md +++ b/docker/scripts/README.md @@ -24,8 +24,7 @@ the executor image is as follows: 2. For each OS package manager, there is a bash function which generates the Dockerfile for the specified OS which uses said package manager. For example, lib_apt.sh contains 'generate_apt_dockerfile()' which is executed for Ubuntu - and debian OS's. lib_yum.sh and lib_dnf.sh contain similar functions for yum - (centos-7) and dnf (centos-8). + and debian OS's. 3. The Dockerfiles contain the following sections: - a. Environment setup and copying of project workspace git repos @@ -56,11 +55,8 @@ bash functions for use by the operational scripts. - `lib_csit.sh`: CSIT specific functions and environment variables -- `lib_dnf.sh`: Dockerfile generation functions for dnf package manager. - - `lib_vpp.sh`: VPP specific functions and environment variables -- `lib_yum.sh`: Dockerfile generation functions for yum package manager. ## Bash Scripts @@ -115,9 +111,7 @@ The following commands are useful to build the initial builder images: `cd <ci-managment repository directory>` -`sudo ./docker/scripts/build_executor_docker_image.sh ubuntu-18.04 2>&1 | tee u1804-$(uname -m).log | grep -ve '^+'` - -`sudo ./docker/scripts/build_executor_docker_image.sh centos-7 2>&1 | tee centos7-$(uname -m).log | grep -ve '^+'` +`sudo ./docker/scripts/build_executor_docker_image.sh ubuntu-20.04 2>&1 | tee u2004-$(uname -m).log | grep -ve '^+'` `sudo ./docker/scripts/build_executor_docker_image.sh -apr sandbox 2>&1 | tee all-sandbox-$(uname -m).log | grep -ve '^+'` @@ -139,10 +133,10 @@ avoids the pitfalls encountered with Docker-In-Docker environments: The environment in the docker shell contains all of the necessary environment variable definitions so the docker scripts can be run directly on the cli. Here is an example command that would be used in a CI job -which automates the generation and testing of a new ubuntu-18.04 docker image -and push it to Docker Hub fdiotools/builder-ubuntu1804:test-<arch>: +which automates the generation and testing of a new ubuntu-20.04 docker image +and push it to Docker Hub fdiotools/builder-ubuntu2004:test-<arch>: -`build_executor_docker_image.sh -pr test ubuntu-18.04` +`build_executor_docker_image.sh -pr test ubuntu-20.04` In the future, a fully automated CI/CD pipeline may be created for production docker images. @@ -189,8 +183,7 @@ tags, but for now testing is a manual operation. 2. Edit jjb/vpp/vpp.yam (or other project yaml file) and replace '-prod-' with '-sandbox-' for all of the docker image -3. `jjb-sandbox-env` # This bash function currently lives in ../vpp/extras/bash/functions.sh - - TODO: move it to ci-management repo. +3. `jjb-sandbox-env` 4. For each job using one of the docker images: @@ -256,14 +249,11 @@ script will be written to automate the process. ## DockerHub Repositories -- fdiotools/builder-centos7 -- fdiotools/builder-centos8 -- fdiotools/builder-debian9 -- fdiotools/builder-debian10 -- fdiotools/builder-ubuntu1804 +- fdiotools/builder-debian11 - fdiotools/builder-ubuntu2004 -- fdiotools/csit_dut-ubuntu1804 -- fdiotools/csit_shim-ubuntu1804 +- fdiotools/builder-ubuntu2204 +- fdiotools/csit_dut-ubuntu2004 +- fdiotools/csit_shim-ubuntu2004 ## Docker Image Tags @@ -279,475 +269,3 @@ a production image to the previous image used in production. Jenkins-Nomad Label. - sandbox-aarch64: Tag used to select the aarch64 sandbox image by the associated Jenkins-Nomad Label. -- test-x86_64: Tag used to select the x86_64 sandbox image by the associated -Jenkins-Nomad Label. -- test-aarch64: Tag used to select the aarch64 sandbox image by the associated -Jenkins-Nomad Label. - -# Jenkins-Nomad Label Definitions - -<class>-<os>-<role>-<arch> (e.g. builder-ubuntu1804-prod-x86_64) - -- class --- builder --- csit_dut --- csit_shim - -- os --- ubuntu1804 --- centos7 --- ubuntu2004 --- centos8 --- debian9 --- debian10 - -- role --- prod --- test --- sandbox - -- arch --- x86_64 --- aarch64 - -## Jenkins Nomad Plugin Node Labels - -### Common Attributes of All Jenkins Nomad Plugin Nodes -- Disk: 3000 -- Priority: 50 -- Idle Termination Time: 10 -- Executors: 1 -- Usage: Only build jobs with label expressions matching this node -- Workspace root: /w -- Privileged: Y -- Network: bridge -- Force-pull: Y - -### Production (prod) Jenkins Nomad Plugin Nodes - -#### Node 'builder-ubuntu1804-prod-x86_64' -- Labels: builder-ubuntu1804-prod-x86_64 -- Job Prefix: builder-ubuntu1804-prod-x86_64 -- Image: fdiotools/builder-ubuntu1804:prod-x86_64 -- CPU: 14000 -- Memory: 14000 -- ${attr.cpu.arch}: amd64 -- ${node.class}: builder - -#### Node 'builder-ubuntu1804-prod-aarch64' -- Labels: builder-ubuntu1804-prod-aarch64 -- Job Prefix: builder-ubuntu1804-prod-aarch64 -- Image: fdiotools/builder-ubuntu1804:prod-aarch64 -- CPU: 6000 -- Memory: 10000 -- ${attr.cpu.arch}: arm64 -- ${node.class}: builder - -#### Node 'builder-centos7-prod-x86_64' -- Labels: builder-centos7-prod-x86_64 -- Job Prefix: builder-centos7-prod-x86_64 -- Image: fdiotools/builder-centos7:prod-x86_64 -- CPU: 14000 -- Memory: 14000 -- ${attr.cpu.arch}: amd64 -- ${node.class}: builder - -#### Node 'builder-centos7-prod-aarch64' -- Labels: builder-centos7-prod-aarch64 -- Job Prefix: builder-centos7-prod-aarch64 -- Image: fdiotools/builder-centos7:prod-aarch64 -- CPU: 6000 -- Memory: 10000 -- ${attr.cpu.arch}: arm64 -- ${node.class}: builder - -#### Node 'builder-ubuntu2004-prod-x86_64' -- Labels: builder-ubuntu2004-prod-x86_64 -- Job Prefix: builder-ubuntu2004-prod-x86_64 -- Image: fdiotools/builder-ubuntu2004:prod-x86_64 -- CPU: 14000 -- Memory: 14000 -- ${attr.cpu.arch}: amd64 -- ${node.class}: builder - -#### Node 'builder-ubuntu2004-prod-aarch64' -- Labels: builder-ubuntu2004-prod-aarch64 -- Job Prefix: builder-ubuntu2004-prod-aarch64 -- Image: fdiotools/builder-ubuntu2004:prod-aarch64 -- CPU: 6000 -- Memory: 10000 -- ${attr.cpu.arch}: arm64 -- ${node.class}: builder - -#### Node 'builder-centos8-prod-x86_64' -- Labels: builder-centos8-prod-x86_64 -- Job Prefix: builder-centos7-prod-x86_64 -- Image: fdiotools/builder-centos8:prod-x86_64 -- CPU: 14000 -- Memory: 14000 -- ${attr.cpu.arch}: amd64 -- ${node.class}: builder - -#### Node 'builder-centos8-prod-aarch64' -- Labels: builder-centos8-prod-aarch64 -- Job Prefix: builder-centos8-prod-aarch64 -- Image: fdiotools/builder-centos8:prod-aarch64 -- CPU: 6000 -- Memory: 10000 -- ${attr.cpu.arch}: arm64 -- ${node.class}: builder - -#### Node 'builder-debian9-prod-x86_64' -- Labels: builder-debian9-prod-x86_64 -- Job Prefix: builder-debian9-prod-x86_64 -- Image: fdiotools/builder-debian9:prod-x86_64 -- CPU: 14000 -- Memory: 14000 -- ${attr.cpu.arch}: amd64 -- ${node.class}: builder - -#### Node 'builder-debian9-prod-aarch64' -- Labels: builder-debian9-prod-aarch64 -- Job Prefix: builder-debian9-prod-aarch64 -- Image: fdiotools/builder-debian9:prod-aarch64 -- CPU: 6000 -- Memory: 10000 -- ${attr.cpu.arch}: arm64 -- ${node.class}: builder - -#### Node 'builder-debian10-prod-x86_64' -- Labels: builder-debian10-prod-x86_64 -- Job Prefix: builder-debian10-prod-x86_64 -- Image: fdiotools/builder-debian10:prod-x86_64 -- CPU: 14000 -- Memory: 14000 -- ${attr.cpu.arch}: amd64 -- ${node.class}: builder - -#### Node 'builder-debian10-prod-aarch64' -- Labels: builder-debian10-prod-aarch64 -- Job Prefix: builder-debian10-prod-aarch64 -- Image: fdiotools/builder-debian10:prod-aarch64 -- CPU: 6000 -- Memory: 10000 -- ${attr.cpu.arch}: arm64 -- ${node.class}: builder - -#### Node 'csit_dut-ubuntu1804-prod-x86_64' -- Labels: csit_dut-ubuntu1804-prod-x86_64 -- Job Prefix: csit_dut-ubuntu1804-prod-x86_64 -- Image: fdiotools/csit_dut-ubuntu1804:prod-x86_64 -- CPU: 10000 -- Memory: 18000 -- ${attr.cpu.arch}: amd64 -- ${node.class}: csit - -#### Node 'csit_dut-ubuntu1804-prod-aarch64' -- Labels: csit_dut-ubuntu1804-prod-aarch64 -- Job Prefix: csit_dut-ubuntu1804-prod-aarch64 -- Image: fdiotools/csit_dut-ubuntu1804:prod-aarch64 -- CPU: 6000 -- Memory: 10000 -- ${attr.cpu.arch}: arm64 -- ${node.class}: csitarm - -#### Node 'csit_shim-ubuntu1804-prod-x86_64' -- Labels: csit_shim-ubuntu1804-prod-x86_64 -- Job Prefix: csit_shim-ubuntu1804-prod-x86_64 -- Image: fdiotools/csit_shim-ubuntu1804:prod-x86_64 -- CPU: 10000 -- Memory: 18000 -- ${attr.cpu.arch}: amd64 -- ${node.class}: csit - -#### Node 'csit_shim-ubuntu1804-prod-aarch64' -- Labels: csit_shim-ubuntu1804-prod-aarch64 -- Job Prefix: csit_shim-ubuntu1804-prod-aarch64 -- Image: fdiotools/csit_shim-ubuntu1804:prod-aarch64 -- CPU: 6000 -- Memory: 10000 -- ${attr.cpu.arch}: arm64 -- ${node.class}: csitarm - -### Sandbox (sandbox) Jenkins Nomad Plugin Nodes - -#### Node 'builder-ubuntu1804-sandbox-x86_64' -- Labels: builder-ubuntu1804-sandbox-x86_64 -- Job Prefix: builder-ubuntu1804-sandbox-x86_64 -- Image: fdiotools/builder-ubuntu1804:sandbox-x86_64 -- CPU: 14000 -- Memory: 14000 -- ${attr.cpu.arch}: amd64 -- ${node.class}: builder - -#### Node 'builder-ubuntu1804-sandbox-aarch64' -- Labels: builder-ubuntu1804-sandbox-aarch64 -- Job Prefix: builder-ubuntu1804-sandbox-aarch64 -- Image: fdiotools/builder-ubuntu1804:sandbox-aarch64 -- CPU: 6000 -- Memory: 10000 -- ${attr.cpu.arch}: arm64 -- ${node.class}: builder - -#### Node 'builder-centos7-sandbox-x86_64' -- Labels: builder-centos7-sandbox-x86_64 -- Job Prefix: builder-centos7-sandbox-x86_64 -- Image: fdiotools/builder-centos7:sandbox-x86_64 -- CPU: 14000 -- Memory: 14000 -- ${attr.cpu.arch}: amd64 -- ${node.class}: builder - -#### Node 'builder-centos7-sandbox-aarch64' -- Labels: builder-centos7-sandbox-aarch64 -- Job Prefix: builder-centos7-sandbox-aarch64 -- Image: fdiotools/builder-centos7:sandbox-aarch64 -- CPU: 6000 -- Memory: 10000 -- ${attr.cpu.arch}: arm64 -- ${node.class}: builder - -#### Node 'builder-ubuntu2004-sandbox-x86_64' -- Labels: builder-ubuntu2004-sandbox-x86_64 -- Job Prefix: builder-ubuntu2004-sandbox-x86_64 -- Image: fdiotools/builder-ubuntu2004:sandbox-x86_64 -- CPU: 14000 -- Memory: 14000 -- ${attr.cpu.arch}: amd64 -- ${node.class}: builder - -#### Node 'builder-ubuntu2004-sandbox-aarch64' -- Labels: builder-ubuntu2004-sandbox-aarch64 -- Job Prefix: builder-ubuntu2004-sandbox-aarch64 -- Image: fdiotools/builder-ubuntu2004:sandbox-aarch64 -- CPU: 6000 -- Memory: 10000 -- ${attr.cpu.arch}: arm64 -- ${node.class}: builder - -#### Node 'builder-centos8-sandbox-x86_64' -- Labels: builder-centos8-sandbox-x86_64 -- Job Prefix: builder-centos8-sandbox-x86_64 -- Image: fdiotools/builder-centos8:sandbox-x86_64 -- CPU: 14000 -- Memory: 14000 -- ${attr.cpu.arch}: amd64 -- ${node.class}: builder - -#### Node 'builder-centos8-sandbox-aarch64' -- Labels: builder-centos8-sandbox-aarch64 -- Job Prefix: builder-centos8-sandbox-aarch64 -- Image: fdiotools/builder-centos8:sandbox-aarch64 -- CPU: 6000 -- Memory: 10000 -- ${attr.cpu.arch}: arm64 -- ${node.class}: builder - -#### Node 'builder-debian9-sandbox-x86_64' -- Labels: builder-debian9-sandbox-x86_64 -- Job Prefix: builder-debian9-sandbox-x86_64 -- Image: fdiotools/builder-debian9:sandbox-x86_64 -- CPU: 14000 -- Memory: 14000 -- ${attr.cpu.arch}: amd64 -- ${node.class}: builder - -#### Node 'builder-debian9-sandbox-aarch64' -- Labels: builder-debian9-sandbox-aarch64 -- Job Prefix: builder-debian9-sandbox-aarch64 -- Image: fdiotools/builder-debian9:sandbox-aarch64 -- CPU: 6000 -- Memory: 10000 -- ${attr.cpu.arch}: arm64 -- ${node.class}: builder - -#### Node 'builder-debian10-sandbox-x86_64' -- Labels: builder-debian10-sandbox-x86_64 -- Job Prefix: builder-debian10-sandbox-x86_64 -- Image: fdiotools/builder-debian10:sandbox-x86_64 -- CPU: 14000 -- Memory: 14000 -- ${attr.cpu.arch}: amd64 -- ${node.class}: builder - -#### Node 'builder-debian10-sandbox-aarch64' -- Labels: builder-debian10-sandbox-aarch64 -- Job Prefix: builder-debian10-sandbox-aarch64 -- Image: fdiotools/builder-debian10:sandbox-aarch64 -- CPU: 6000 -- Memory: 10000 -- ${attr.cpu.arch}: arm64 -- ${node.class}: builder - -#### Node 'csit_dut-ubuntu1804-sandbox-x86_64' -- Labels: csit_dut-ubuntu1804-sandbox-x86_64 -- Job Prefix: csit_dut-ubuntu1804-sandbox-x86_64 -- Image: fdiotools/csit_dut-ubuntu1804:sandbox-x86_64 -- CPU: 10000 -- Memory: 18000 -- ${attr.cpu.arch}: amd64 -- ${node.class}: csit - -#### Node 'csit_dut-ubuntu1804-sandbox-aarch64' -- Labels: csit_dut-ubuntu1804-sandbox-aarch64 -- Job Prefix: csit_dut-ubuntu1804-sandbox-aarch64 -- Image: fdiotools/csit_dut-ubuntu1804:sandbox-aarch64 -- CPU: 6000 -- Memory: 10000 -- ${attr.cpu.arch}: arm64 -- ${node.class}: csitarm - -#### Node 'csit_shim-ubuntu1804-sandbox-x86_64' -- Labels: csit_shim-ubuntu1804-sandbox-x86_64 -- Job Prefix: csit_shim-ubuntu1804-sandbox-x86_64 -- Image: fdiotools/csit_shim-ubuntu1804:sandbox-x86_64 -- CPU: 10000 -- Memory: 18000 -- ${attr.cpu.arch}: amd64 -- ${node.class}: csit - -#### Node 'csit_shim-ubuntu1804-sandbox-aarch64' -- Labels: csit_shim-ubuntu1804-sandbox-aarch64 -- Job Prefix: csit_shim-ubuntu1804-sandbox-aarch64 -- Image: fdiotools/csit_shim-ubuntu1804:sandbox-aarch64 -- CPU: 6000 -- Memory: 10000 -- ${attr.cpu.arch}: arm64 -- ${node.class}: csitarm - -### Automated Testing (test) Jenkins Nomad Plugin Nodes - -#### Node 'builder-ubuntu1804-test-x86_64' -- Labels: builder-ubuntu1804-test-x86_64 -- Job Prefix: builder-ubuntu1804-test-x86_64 -- Image: fdiotools/builder-ubuntu1804:test-x86_64 -- CPU: 14000 -- Memory: 14000 -- ${attr.cpu.arch}: amd64 -- ${node.class}: builder - -#### Node 'builder-ubuntu1804-test-aarch64' -- Labels: builder-ubuntu1804-test-aarch64 -- Job Prefix: builder-ubuntu1804-test-aarch64 -- Image: fdiotools/builder-ubuntu1804:test-aarch64 -- CPU: 6000 -- Memory: 10000 -- ${attr.cpu.arch}: arm64 -- ${node.class}: builder - -#### Node 'builder-centos7-test-x86_64' -- Labels: builder-centos7-test-x86_64 -- Job Prefix: builder-centos7-test-x86_64 -- Image: fdiotools/builder-centos7:test-x86_64 -- CPU: 14000 -- Memory: 14000 -- ${attr.cpu.arch}: amd64 -- ${node.class}: builder - -#### Node 'builder-centos7-test-aarch64' -- Labels: builder-centos7-test-aarch64 -- Job Prefix: builder-centos7-test-aarch64 -- Image: fdiotools/builder-centos7:test-aarch64 -- CPU: 6000 -- Memory: 10000 -- ${attr.cpu.arch}: arm64 -- ${node.class}: builder - -#### Node 'builder-ubuntu2004-test-x86_64' -- Labels: builder-ubuntu2004-test-x86_64 -- Job Prefix: builder-ubuntu2004-test-x86_64 -- Image: fdiotools/builder-ubuntu2004:test-x86_64 -- CPU: 14000 -- Memory: 14000 -- ${attr.cpu.arch}: amd64 -- ${node.class}: builder - -#### Node 'builder-ubuntu2004-test-aarch64' -- Labels: builder-ubuntu2004-test-aarch64 -- Job Prefix: builder-ubuntu2004-test-aarch64 -- Image: fdiotools/builder-ubuntu2004:test-aarch64 -- CPU: 6000 -- Memory: 10000 -- ${attr.cpu.arch}: arm64 -- ${node.class}: builder - -#### Node 'builder-centos8-test-x86_64' -- Labels: builder-centos8-test-x86_64 -- Job Prefix: builder-centos8-test-x86_64 -- Image: fdiotools/builder-centos8:test-x86_64 -- CPU: 14000 -- Memory: 14000 -- ${attr.cpu.arch}: amd64 -- ${node.class}: builder - -#### Node 'builder-centos8-test-aarch64' -- Labels: builder-centos8-test-aarch64 -- Job Prefix: builder-centos8-test-aarch64 -- Image: fdiotools/builder-centos8:test-aarch64 -- CPU: 6000 -- Memory: 10000 -- ${attr.cpu.arch}: arm64 -- ${node.class}: builder - -#### Node 'builder-debian9-test-x86_64' -- Labels: builder-debian9-test-x86_64 -- Job Prefix: builder-debian9-test-x86_64 -- Image: fdiotools/builder-debian9:test-x86_64 -- CPU: 14000 -- Memory: 14000 -- ${attr.cpu.arch}: amd64 -- ${node.class}: builder - -#### Node 'builder-debian9-test-aarch64' -- Labels: builder-debian9-test-aarch64 -- Job Prefix: builder-debian9-test-aarch64 -- Image: fdiotools/builder-debian9:test-aarch64 -- CPU: 6000 -- Memory: 10000 -- ${attr.cpu.arch}: arm64 -- ${node.class}: builder - -#### Node 'builder-debian10-test-x86_64' -- Labels: builder-debian10-test-x86_64 -- Job Prefix: builder-debian10-test-x86_64 -- Image: fdiotools/builder-debian10:test-x86_64 -- CPU: 14000 -- Memory: 14000 -- ${attr.cpu.arch}: amd64 -- ${node.class}: builder - -#### Node 'builder-debian10-test-aarch64' -- Labels: builder-debian10-test-aarch64 -- Job Prefix: builder-debian10-test-aarch64 -- Image: fdiotools/builder-debian10:test-aarch64 -- CPU: 6000 -- Memory: 10000 -- ${attr.cpu.arch}: arm64 -- ${node.class}: builder - -#### Node 'csit_dut-ubuntu1804-sandbox-x86_64' -- Labels: csit_dut-ubuntu1804-sandbox-x86_64 -- Job Prefix: csit_dut-ubuntu1804-sandbox-x86_64 -- Image: fdiotools/csit_dut-ubuntu1804:sandbox-x86_64 -- CPU: 10000 -- Memory: 18000 -- ${attr.cpu.arch}: amd64 -- ${node.class}: csit - -#### Node 'csit_dut-ubuntu1804-test-aarch64' -- Labels: csit_dut-ubuntu1804-test-aarch64 -- Job Prefix: csit_dut-ubuntu1804-test-aarch64 -- Image: fdiotools/csit_dut-ubuntu1804:test-aarch64 -- CPU: 6000 -- Memory: 10000 -- ${attr.cpu.arch}: arm64 -- ${node.class}: csitarm - -#### Node 'csit_shim-ubuntu1804-test-aarch64' -- Labels: csit_shim-ubuntu1804-test-aarch64 -- Job Prefix: csit_shim-ubuntu1804-test-aarch64 -- Image: fdiotools/csit_shim-ubuntu1804:test-aarch64 -- CPU: 6000 -- Memory: 10000 -- ${attr.cpu.arch}: arm64 -- ${node.class}: csitarm |