diff options
Diffstat (limited to 'README.md')
| -rw-r--r-- | README.md | 201 |
1 files changed, 201 insertions, 0 deletions
diff --git a/README.md b/README.md new file mode 100644 index 0000000000..f878980dfb --- /dev/null +++ b/README.md @@ -0,0 +1,201 @@ +# CSIT - Continuous System Integration Testing + +1. [Architecture](#architecture) +1. [Directory Structure](#directory-structure) + 1. [Tests](#tests) + 1. [Keywords](#keywords) + 1. [Other Resources](#other-resources) +1. [Quickstart](#quick-start) + 1. [Vagrant](#vagrant) + 1. [Physical Testbed](#physical-testbed) +1. [Report](#report) +1. [Trending](#trending) +1. [Code Documentation](#code-documentation) +1. [Coding Guidelines](#coding-guidelines) + +## Architecture + +FD.io CSIT system design needs to meet continuously expanding requirements of +FD.io projects including VPP, related sub-systems (e.g. plugin applications, +DPDK drivers) and FD.io applications (e.g. DPDK applications), as well as +growing number of compute platforms running those applications. With CSIT +project scope and charter including both FD.io continuous testing AND +performance trending/comparisons, those evolving requirements further amplify +the need for CSIT framework modularity, flexibility and usability. + +CSIT follows a hierarchical system design with SUTs and DUTs at the bottom level +of the hierarchy, presentation level at the top level and a number of functional +layers in-between. The current CSIT system design including CSIT framework is +depicted in the figure below. + + + +A brief bottom-up description is provided here: + +1. SUTs, DUTs, TGs + - SUTs - Systems Under Test; + - DUTs - Devices Under Test; + - TGs - Traffic Generators; +1. Level-1 libraries - Robot and Python + - Lowest level CSIT libraries abstracting underlying test environment, SUT, + DUT and TG specifics; + - Used commonly across multiple L2 KWs; + - Performance and functional tests: + - L1 KWs (KeyWords) are implemented as RF libraries and Python + libraries; + - Performance TG L1 KWs: + - All L1 KWs are implemented as Python libraries: + - Support for TRex only today; + - Performance data plane traffic profiles: + - TG-specific stream profiles provide full control of: + - Packet definition – layers, MACs, IPs, ports, combinations thereof + e.g. IPs and UDP ports; + - Stream definitions - different streams can run together, delayed, + one after each other; + - Stream profiles are independent of CSIT framework and can be used + in any T-rex setup, can be sent anywhere to repeat tests with + exactly the same setup; + - Easily extensible – one can create a new stream profile that meets + tests requirements; + - Same stream profile can be used for different tests with the same + traffic needs; + - Functional data plane traffic scripts: + - Scapy specific traffic scripts; +1. Level-2 libraries - Robot resource files + - Higher level CSIT libraries abstracting required functions for executing + tests; + - L2 KWs are classified into the following functional categories: + - Configuration, test, verification, state report; + - Suite setup, suite teardown; + - Test setup, test teardown; +1. Tests - Robot + - Test suites with test cases; + - Functional tests using VIRL environment: + - VPP; + - Honeycomb; + - NSH_SFC; + - DMM; + - TLDK; + - Performance tests using physical testbed environment: + - VPP; + - DPDK-Testpmd; + - DPDK-L3Fwd; + - Honeycomb; + - VPP Container K8s orchestrated topologies; + - Tools: + - Documentation generator; + - Report generator; + - Testbed environment setup ansible playbooks; + - Operational debugging scripts; + +## Directory Structure + +### Tests + +``` +. +└── tests + ├── dmm + │ └── func # DMM functional VIRL tests + ├── dpdk + │ ├── dpdk_scripts # DPDK helper scripts + │ └── perf # DPDK performance tests + ├── honeycomb + │ ├── func # Honeycomb functional VIRL tests + │ └── perf # Honeycomb functional performance tests + ├── kubernetes + │ └── perf # VPP K8S orchestration performance tests + ├── nsh_sfc + │ ├── func # NSH_SFC functional tests + │ └── sfc_scripts # NSH_SFC helper scripts + ├── tldk + │ ├── func # TLDK functional VIRL tests + │ ├── tldk_scripts # TLDK helper scripts + │ └── tldk_testconfig # TLDK test configuration + └── vpp + ├── device # VPP device tests + ├── func # VPP functional VIRL tests + └── perf # VPP performance tests +``` + +### Keywords + +``` +. +resources +└── libraries + ├── bash + │ ├── config + │ ├── entry # Main bootstrap entry directory + │ ├── function # Bootstrap function library + │ ├── qemu_patches # Custom QEMU patches (see KVM methodology) + │ └── shell # Various functions + ├── python # Python L1 KWs + └── robot # Robot Framework L2 KWs +``` + +### Other Resources + +``` +. +├── docs # Main documentaion +├── PyPI # PyPI packages provided by CSIT +│ ├── jumpavg +│ └── MLRsearch +├── resources +│ ├── templates # Templates (vpp_api_test, kubernetes, ...) +│ ├── test_data # Robot Test configuration +│ ├── tools +│ │ ├── disk-image-builder # Utilities for building (DCR, VM) images +│ │ ├── doc_gen # Code documentation generator +│ │ ├── papi # PAPI driver +│ │ ├── presentation # Report generator +│ │ ├── scripts # Various tools +│ │ ├── testbed-setup # Physical testbed setup scripts +│ │ ├── topology # Helper scripts for topology manipulation +│ │ ├── trex # TRex driver +│ │ ├── vagrant # VPP device vagrant environment +│ │ ├── virl # VIRL helper scripts +│ │ └── wrk # WRK driver +│ ├── topology_schemas +│ ├── traffic_profiles # Performance tests traffic profiles +│ │ ├── trex +│ │ └── wrk +│ └── traffic_scripts # Functional tests traffic profiles +│ ├── dhcp +│ ├── honeycomb +│ └── lisp +└── topologies # Linux Foundation topology files + ├── available + └── enabled +``` + +## Quickstart + +### Vagrant + +[Vagrant environment preparation](docs/testing_in_vagrant.rst) documentaion is +describing local VPP Device functional testing. + +### Physical Testbed + +[Physical testbed preparation](resources/tools/testbed-setup/README.md) +documentation is describing PXE and Ansible setup process. All the software +requirements for running Performance Teste are part of Ansible playbooks. + +## Report + +[CSIT Report](https://docs.fd.io/csit/master/report/). + +## Trending + +[CSIT Trending](https://docs.fd.io/csit/master/trending/). + +## Code Documentation + +[CSIT Code Documentation](https://docs.fd.io/csit/master/doc/). + +## Coding Guidelines + +If you are interested in contributing, please see the +[coding guidelines](docs/test_code_guidelines.rst). |