diff options
| -rw-r--r-- | README | 51 | ||||
| -rw-r--r-- | README.md | 201 | ||||
| -rw-r--r-- | docs/directory_structure.md | 537 | ||||
| -rw-r--r-- | docs/qemu_vpp_vm | 49 | ||||
| -rw-r--r-- | docs/report/introduction/methodology_kvm_vms_vhost_user.rst | 18 | ||||
| -rw-r--r-- | docs/tg_interface_driver | 14 | ||||
| -rw-r--r-- | docs/topology_schemas | 2 |
7 files changed, 219 insertions, 653 deletions
diff --git a/README b/README deleted file mode 100644 index b14834f07b..0000000000 --- a/README +++ /dev/null @@ -1,51 +0,0 @@ -# STEPS TO START DEVELOPING TESTS LOCALLY - - install virtualenv - - generate environment using virtualenv: - # cd $ROOT - # virtualenv env - # source env/bin/activate - - install python requirements for this project by executing: - # pip install -r requirements.txt - - make sure user mentioned in topology.py has NOPASSWD sudo access to - vpp_api_test - -Note: -You can alternatively create the virtualenv with the flag --system-site-packages -It give access to the global site-packages dir to the virtual environment, which -is faster, but you may end up with some conflicts. - - - Done. - -# STEPS TO START THE TESTS -export PYTHONPATH=. - -# create topology, edit ip addresses -cp topologies/available/3_node_hw_topo1.yaml.example topologies/available/topology.yaml -ln -s ../available/topology.yaml topologies/enabled/topology.yaml - -pybot -L TRACE -v TOPOLOGY_PATH:topologies/enabled/topology.yaml tests - or -./main.py -t topologies/enabled/topology.yaml -i test_tag - or -./main.py - - -# Dependencies on Nodes - - - virtualenv - - pip - - python2.7 - - python-dev package - - gcc (pycrypto) - - libpcap-devel (pypcap) - -``` bash -# on fedora26 -yum install -y python-virtualenv python-pip python python-devel libpcap-devel gcc - -# if you have the following error during pycrypto -# gcc: error: /usr/lib/rpm/redhat/redhat-hardened-cc1 No such file or directory -yum install -y redhat-rpm-config -``` - diff --git a/README.md b/README.md new file mode 100644 index 0000000000..f878980dfb --- /dev/null +++ b/README.md @@ -0,0 +1,201 @@ +# CSIT - Continuous System Integration Testing + +1. [Architecture](#architecture) +1. [Directory Structure](#directory-structure) + 1. [Tests](#tests) + 1. [Keywords](#keywords) + 1. [Other Resources](#other-resources) +1. [Quickstart](#quick-start) + 1. [Vagrant](#vagrant) + 1. [Physical Testbed](#physical-testbed) +1. [Report](#report) +1. [Trending](#trending) +1. [Code Documentation](#code-documentation) +1. [Coding Guidelines](#coding-guidelines) + +## Architecture + +FD.io CSIT system design needs to meet continuously expanding requirements of +FD.io projects including VPP, related sub-systems (e.g. plugin applications, +DPDK drivers) and FD.io applications (e.g. DPDK applications), as well as +growing number of compute platforms running those applications. With CSIT +project scope and charter including both FD.io continuous testing AND +performance trending/comparisons, those evolving requirements further amplify +the need for CSIT framework modularity, flexibility and usability. + +CSIT follows a hierarchical system design with SUTs and DUTs at the bottom level +of the hierarchy, presentation level at the top level and a number of functional +layers in-between. The current CSIT system design including CSIT framework is +depicted in the figure below. + + + +A brief bottom-up description is provided here: + +1. SUTs, DUTs, TGs + - SUTs - Systems Under Test; + - DUTs - Devices Under Test; + - TGs - Traffic Generators; +1. Level-1 libraries - Robot and Python + - Lowest level CSIT libraries abstracting underlying test environment, SUT, + DUT and TG specifics; + - Used commonly across multiple L2 KWs; + - Performance and functional tests: + - L1 KWs (KeyWords) are implemented as RF libraries and Python + libraries; + - Performance TG L1 KWs: + - All L1 KWs are implemented as Python libraries: + - Support for TRex only today; + - Performance data plane traffic profiles: + - TG-specific stream profiles provide full control of: + - Packet definition – layers, MACs, IPs, ports, combinations thereof + e.g. IPs and UDP ports; + - Stream definitions - different streams can run together, delayed, + one after each other; + - Stream profiles are independent of CSIT framework and can be used + in any T-rex setup, can be sent anywhere to repeat tests with + exactly the same setup; + - Easily extensible – one can create a new stream profile that meets + tests requirements; + - Same stream profile can be used for different tests with the same + traffic needs; + - Functional data plane traffic scripts: + - Scapy specific traffic scripts; +1. Level-2 libraries - Robot resource files + - Higher level CSIT libraries abstracting required functions for executing + tests; + - L2 KWs are classified into the following functional categories: + - Configuration, test, verification, state report; + - Suite setup, suite teardown; + - Test setup, test teardown; +1. Tests - Robot + - Test suites with test cases; + - Functional tests using VIRL environment: + - VPP; + - Honeycomb; + - NSH_SFC; + - DMM; + - TLDK; + - Performance tests using physical testbed environment: + - VPP; + - DPDK-Testpmd; + - DPDK-L3Fwd; + - Honeycomb; + - VPP Container K8s orchestrated topologies; + - Tools: + - Documentation generator; + - Report generator; + - Testbed environment setup ansible playbooks; + - Operational debugging scripts; + +## Directory Structure + +### Tests + +``` +. +└── tests + ├── dmm + │ └── func # DMM functional VIRL tests + ├── dpdk + │ ├── dpdk_scripts # DPDK helper scripts + │ └── perf # DPDK performance tests + ├── honeycomb + │ ├── func # Honeycomb functional VIRL tests + │ └── perf # Honeycomb functional performance tests + ├── kubernetes + │ └── perf # VPP K8S orchestration performance tests + ├── nsh_sfc + │ ├── func # NSH_SFC functional tests + │ └── sfc_scripts # NSH_SFC helper scripts + ├── tldk + │ ├── func # TLDK functional VIRL tests + │ ├── tldk_scripts # TLDK helper scripts + │ └── tldk_testconfig # TLDK test configuration + └── vpp + ├── device # VPP device tests + ├── func # VPP functional VIRL tests + └── perf # VPP performance tests +``` + +### Keywords + +``` +. +resources +└── libraries + ├── bash + │ ├── config + │ ├── entry # Main bootstrap entry directory + │ ├── function # Bootstrap function library + │ ├── qemu_patches # Custom QEMU patches (see KVM methodology) + │ └── shell # Various functions + ├── python # Python L1 KWs + └── robot # Robot Framework L2 KWs +``` + +### Other Resources + +``` +. +├── docs # Main documentaion +├── PyPI # PyPI packages provided by CSIT +│ ├── jumpavg +│ └── MLRsearch +├── resources +│ ├── templates # Templates (vpp_api_test, kubernetes, ...) +│ ├── test_data # Robot Test configuration +│ ├── tools +│ │ ├── disk-image-builder # Utilities for building (DCR, VM) images +│ │ ├── doc_gen # Code documentation generator +│ │ ├── papi # PAPI driver +│ │ ├── presentation # Report generator +│ │ ├── scripts # Various tools +│ │ ├── testbed-setup # Physical testbed setup scripts +│ │ ├── topology # Helper scripts for topology manipulation +│ │ ├── trex # TRex driver +│ │ ├── vagrant # VPP device vagrant environment +│ │ ├── virl # VIRL helper scripts +│ │ └── wrk # WRK driver +│ ├── topology_schemas +│ ├── traffic_profiles # Performance tests traffic profiles +│ │ ├── trex +│ │ └── wrk +│ └── traffic_scripts # Functional tests traffic profiles +│ ├── dhcp +│ ├── honeycomb +│ └── lisp +└── topologies # Linux Foundation topology files + ├── available + └── enabled +``` + +## Quickstart + +### Vagrant + +[Vagrant environment preparation](docs/testing_in_vagrant.rst) documentaion is +describing local VPP Device functional testing. + +### Physical Testbed + +[Physical testbed preparation](resources/tools/testbed-setup/README.md) +documentation is describing PXE and Ansible setup process. All the software +requirements for running Performance Teste are part of Ansible playbooks. + +## Report + +[CSIT Report](https://docs.fd.io/csit/master/report/). + +## Trending + +[CSIT Trending](https://docs.fd.io/csit/master/trending/). + +## Code Documentation + +[CSIT Code Documentation](https://docs.fd.io/csit/master/doc/). + +## Coding Guidelines + +If you are interested in contributing, please see the +[coding guidelines](docs/test_code_guidelines.rst). diff --git a/docs/directory_structure.md b/docs/directory_structure.md deleted file mode 100644 index b4065989d2..0000000000 --- a/docs/directory_structure.md +++ /dev/null @@ -1,537 +0,0 @@ - -# Directory structure re-organization - -## Table of contents - -1. [The new structure](#the-new-structure) -1. [Tests](#tests) - 1. [Performance](#performance) - 1. [Functional](#functional) -1. [Keywords](#keywords) - 1. [L2 Robot keywords](#l2-robot-keywords) - 1. [L1 Python keywords](#l1-python-keywords) - -## The new structure - -### Tests -``` -$CSIT/ - tests/ - vpp/ - func/ - l2bd/ - l2xc/ - ip4/ - ip6/ - ip4_tunnels/ - ip6_tunnels/ - vm_vhost/ - crypto/ - interfaces/ - telemetry/ - honeycomb/ - perf/ - l2/ - ip4/ - ip6/ - ip4_tunnels/ - ip6_tunnels/ - vm_vhost/ - crypto/ - dpdk/ - func/ - perf/ - nsh_sfc/ - func/ - perf/ - tldk/ - func/ - perf/ -``` - -### Keywords -``` -$CSIT/ - resources/ - libraries/ - bash/ - python/ - packages/ (dirs): feature | area - modules/ - robot/ - shared/ - l2/ - ip/ - overlay/ - vm/ - crypto/ - dpdk/ - nsh_sfc/ - tldk/ - honeycomb/ - performance/ - telemetry/ - features/ - fds/ -``` - -*Notes:* -1. **l1/** - L1 will not be created, L1 KWs will be temporarily kept in L2 (their - current place) and asap refactored using Python -1. **l2/** - Not needed if we do not have l1/ - - -### Other resources -``` -$CSIT/ - resources/ - templates/ - vat/ - honeycomb/ - test_data/ - honeycomb/ - lisp/ - softwire/ - tools/ - disk_image_builder/ - doc_gen/ - report_gen/ - scripts/ - testbed_setup/ - topology/ - trex/ - vagrant/ - virl/ - topology_schemas/ - traffic_scripts/ - traffic_profiles/ - trex/ - ixia/ -``` - -## Tests - -### Performance - -#### L2 Ethernet Switching -``` -ls | grep -E "(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-(ndrpdrdisc|ndrchk|pdrchk)" -``` -``` -l2/ - 10ge2p1vic1227-eth-l2bdbasemaclrn-ndrpdrdisc.robot - 10ge2p1x520-dot1ad-l2xcbase-ndrchk.robot - 10ge2p1x520-dot1ad-l2xcbase-ndrpdrdisc.robot - 10ge2p1x520-dot1q-l2xcbase-ndrchk.robot - 10ge2p1x520-dot1q-l2xcbase-ndrpdrdisc.robot - 10ge2p1x520-eth-l2bdbasemaclrn-ndrchk.robot - 10ge2p1x520-eth-l2bdbasemaclrn-ndrpdrdisc.robot - 10ge2p1x520-eth-l2bdbasemaclrn-pdrchk.robot - 10ge2p1x520-eth-l2xcbase-ndrchk.robot - 10ge2p1x520-eth-l2xcbase-ndrpdrdisc.robot - 10ge2p1x520-eth-l2xcbase-pdrchk.robot - 10ge2p1x710-eth-l2bdbasemaclrn-ndrpdrdisc.robot - 40ge2p1vic1385-eth-l2bdbasemaclrn-ndrpdrdisc.robot - 40ge2p1xl710-eth-l2bdbasemaclrn-ndrpdrdisc.robot - 40ge2p1xl710-eth-l2xcbase-ndrpdrdisc.robot -``` - -#### IPv4 Routed-Forwarding -``` -ls | grep -P 'ethip4(udp|)-ip4(base|scale)[a-z0-9]*(?!-eth-[0-9]vhost).*-(ndrpdrdisc|ndrchk|pdrchk)' -``` -``` -ip4/ - 10ge2p1x520-ethip4-ip4base-copwhtlistbase-ndrchk.robot - 10ge2p1x520-ethip4-ip4base-copwhtlistbase-ndrpdrdisc.robot - 10ge2p1x520-ethip4-ip4base-iacldstbase-ndrchk.robot - 10ge2p1x520-ethip4-ip4base-iacldstbase-ndrpdrdisc.robot - 10ge2p1x520-ethip4-ip4base-ipolicemarkbase-ndrchk.robot - 10ge2p1x520-ethip4-ip4base-ipolicemarkbase-ndrpdrdisc.robot - 10ge2p1x520-ethip4-ip4base-ndrchk.robot - 10ge2p1x520-ethip4-ip4base-ndrpdrdisc.robot - 10ge2p1x520-ethip4-ip4base-pdrchk.robot - 10ge2p1x520-ethip4-ip4base-snat-ndrpdrdisc.robot - 10ge2p1x520-ethip4-ip4scale200k-ndrchk.robot - 10ge2p1x520-ethip4-ip4scale200k-ndrpdrdisc.robot - 10ge2p1x520-ethip4-ip4scale20k-ndrchk.robot - 10ge2p1x520-ethip4-ip4scale20k-ndrpdrdisc.robot - 10ge2p1x520-ethip4-ip4scale2m-ndrchk.robot - 10ge2p1x520-ethip4-ip4scale2m-ndrpdrdisc.robot - 10ge2p1x520-ethip4-ip4scale2m-pdrchk.robot - 10ge2p1x520-ethip4udp-ip4base-udpsrcscale15-snat-ndrpdrdisc.robot - 10ge2p1x520-ethip4udp-ip4scale1000-udpsrcscale15-snat-ndrpdrdisc.robot - 10ge2p1x520-ethip4udp-ip4scale100-udpsrcscale15-snat-ndrpdrdisc.robot - 10ge2p1x520-ethip4udp-ip4scale10-udpsrcscale15-snat-ndrpdrdisc.robot - 10ge2p1x520-ethip4udp-ip4scale2000-udpsrcscale15-snat-ndrpdrdisc.robot - 10ge2p1x520-ethip4udp-ip4scale4000-udpsrcscale15-snat-ndrpdrdisc.robot - 40ge2p1xl710-ethip4-ip4base-ndrpdrdisc.robot -``` - -#### IPv6 Routed-Forwarding -``` -ls | grep -E "ethip6-ip6(base|scale)[-a-z0-9]*-(ndrpdrdisc|ndrchk|pdrchk)" -``` -``` -ip6/ - 10ge2p1x520-ethip6-ip6base-copwhtlistbase-ndrchk.robot - 10ge2p1x520-ethip6-ip6base-copwhtlistbase-ndrpdrdisc.robot - 10ge2p1x520-ethip6-ip6base-iacldstbase-ndrchk.robot - 10ge2p1x520-ethip6-ip6base-iacldstbase-ndrpdrdisc.robot - 10ge2p1x520-ethip6-ip6base-ndrchk.robot - 10ge2p1x520-ethip6-ip6base-ndrpdrdisc.robot - 10ge2p1x520-ethip6-ip6base-pdrchk.robot - 10ge2p1x520-ethip6-ip6scale200k-ndrchk.robot - 10ge2p1x520-ethip6-ip6scale200k-ndrpdrdisc.robot - 10ge2p1x520-ethip6-ip6scale20k-ndrchk.robot - 10ge2p1x520-ethip6-ip6scale20k-ndrpdrdisc.robot - 10ge2p1x520-ethip6-ip6scale2m-ndrchk.robot - 10ge2p1x520-ethip6-ip6scale2m-ndrpdrdisc.robot - 10ge2p1x520-ethip6-ip6scale2m-pdrchk.robot - 40ge2p1xl710-ethip6-ip6base-ndrpdrdisc.robot -``` - -#### IPv4 Overlay Tunnels -``` -ls | grep -E "ethip4[a-z0-9]+-[a-z0-9]*-(ndrpdrdisc|ndrchk|pdrchk)" -``` -``` -ip4_tunnels/ - 10ge2p1x520-ethip4lispip4-ip4base-ndrchk.robot - 10ge2p1x520-ethip4lispip4-ip4base-ndrpdrdisc.robot - 10ge2p1x520-ethip4lispip4-ip4base-pdrchk.robot - 10ge2p1x520-ethip4lispip6-ip4base-ndrchk.robot - 10ge2p1x520-ethip4lispip6-ip4base-ndrpdrdisc.robot - 10ge2p1x520-ethip4lispip6-ip4base-pdrchk.robot - 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-ndrpdrdisc.robot - 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-pdrchk.robot - 10ge2p1x520-ethip4vxlan-l2xcbase-ndrchk.robot - 10ge2p1x520-ethip4vxlan-l2xcbase-ndrpdrdisc.robot - 10ge2p1x520-ethip4vxlan-l2xcbase-pdrchk.robot -``` - -#### IPv6 Overlay Tunnels -``` -ls | grep -E "ethip6[a-z0-9]+-[a-z0-9]*-(ndrpdrdisc|ndrchk|pdrchk)" -``` -``` -ip6_tunnels/ - 10ge2p1x520-ethip6lispip4-ip6base-ndrchk.robot - 10ge2p1x520-ethip6lispip4-ip6base-ndrpdrdisc.robot - 10ge2p1x520-ethip6lispip4-ip6base-pdrchk.robot - 10ge2p1x520-ethip6lispip6-ip6base-ndrchk.robot - 10ge2p1x520-ethip6lispip6-ip6base-ndrpdrdisc.robot - 10ge2p1x520-ethip6lispip6-ip6base-pdrchk.robot -``` - -#### VM vhost Connections -``` -ls | grep -E ".*vhost.*-(ndrpdrdisc|ndrchk|pdrchk)" -``` -``` -vm_vhost/ - 10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot - 10ge2p1x520-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-pdrchk.robot - 10ge2p1x520-dot1q-l2xcbase-eth-2vhost-1vm-ndrpdrdisc.robot - 10ge2p1x520-dot1q-l2xcbase-eth-2vhost-1vm-pdrchk.robot - 10ge2p1x520-ethip4-ip4base-eth-2vhost-1vm-ndrpdrdisc.robot - 10ge2p1x520-ethip4-ip4base-eth-2vhost-1vm-pdrchk.robot - 10ge2p1x520-ethip4-ip4base-eth-4vhost-2vm-ndrpdrdisc.robot - 10ge2p1x520-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot - 10ge2p1x520-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot - 10ge2p1x520-eth-l2bdbasemaclrn-eth-4vhost-2vm-ndrpdrdisc.robot - 10ge2p1x520-eth-l2xcbase-eth-2vhost-1vm-ndrpdrdisc.robot - 10ge2p1x520-eth-l2xcbase-eth-2vhost-1vm-pdrchk.robot - 10ge2p1x520-eth-l2xcbase-eth-4vhost-2vm-ndrpdrdisc.robot - 10ge2p1x710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot - 40ge2p1xl710-ethip4-ip4base-eth-4vhost-2vm-ndrpdrdisc.robot - 40ge2p1xl710-eth-l2bdbasemaclrn-eth-2vhost-1vm-ndrpdrdisc.robot - 40ge2p1xl710-eth-l2bdbasemaclrn-eth-4vhost-2vm-ndrpdrdisc.robot - 40ge2p1xl710-eth-l2xcbase-eth-4vhost-2vm-ndrpdrdisc.robot -``` - -#### IPSec Crypto HW: IP4 Routed-Forwarding -``` -ls | grep -E ".*ipsec.*-(ndrpdrdisc|ndrchk|pdrchk)" -``` -``` -crypto/ - 40ge2p1xl710-ethip4ipsecbasetnl-ip4base-int-aes-gcm-ndrpdrdisc.robot - 40ge2p1xl710-ethip4ipsecbasetnl-ip4base-int-cbc-sha1-ndrpdrdisc.robot - 40ge2p1xl710-ethip4ipsecbasetnl-ip4base-tnl-aes-gcm-ndrpdrdisc.robot - 40ge2p1xl710-ethip4ipsecbasetnl-ip4base-tnl-cbc-sha1-ndrpdrdisc.robot - 40ge2p1xl710-ethip4ipsecscale1000tnl-ip4base-int-aes-gcm-ndrpdrdisc.robot - 40ge2p1xl710-ethip4ipsecscale1000tnl-ip4base-int-cbc-sha1-ndrpdrdisc.robot - 40ge2p1xl710-ethip4ipsecscale1000tnl-ip4base-tnl-aes-gcm-ndrpdrdisc.robot - 40ge2p1xl710-ethip4ipsecscale1000tnl-ip4base-tnl-cbc-sha1-ndrpdrdisc.robot - 40ge2p1xl710-ethip4ipsectptlispgpe-ip4base-cbc-sha1-ndrpdrdisc.robot -``` - -### Functional - -#### L2 Ethernet Switching -``` -l2bd/ - eth2p-dot1ad--dot1q-l2bdbasemaclrn-vlantrans21-func.robot - eth2p-dot1ad-l2bdbasemaclrn-vlantrans22-func.robot - eth2p-dot1q--dot1ad-l2bdbasemaclrn-vlantrans12-func.robot - eth2p-dot1q-l2bdbasemaclrn-vlantrans11-func.robot - eth2p-eth-l2bdbasemaclrn-func.robot - eth2p-eth-l2bdbasemacstc-func.robot - eth4p-eth-l2bdbasemaclrn-l2shg-func.robot -``` - -``` -l2xc/ - eth2p-dot1ad--dot1q-l2xcbase-vlantrans21-func.robot - eth2p-dot1ad-l2xcbase-func.robot - eth2p-dot1ad-l2xcbase-vlantrans22-func.robot - eth2p-dot1q--dot1ad-l2xcbase-vlantrans12-func.robot - eth2p-dot1q-l2xcbase-vlantrans11-func.robot - eth2p-eth-l2xcbase-func.robot - eth2p-eth-l2xcbase-iaclbase-func.robot -``` - -#### IPv4 Routed-Forwarding -``` -ip4/ - eth2p-dot1q-ip4base-func.robot - eth2p-ethip4-ip4base-copblklistbase-func.robot - eth2p-ethip4-ip4base-copwhlistbase-func.robot - eth2p-ethip4-ip4base-func.robot - eth2p-ethip4-ip4base-iaclbase-func.robot - eth2p-ethip4-ip4base-ip4arp-func.robot - eth2p-ethip4-ip4base-ip4dhcpclient-func.robot - eth2p-ethip4-ip4base-ip4dhcpproxy-func.robot - eth2p-ethip4-ip4base-ip4ecmp-func.robot - eth2p-ethip4-ip4base-ip4proxyarp-func.robot - eth2p-ethip4-ip4base-ipolicemarkbase-func.robot - eth2p-ethip4-ip4base-rpf-func.robot - eth2p-ethip4-ip4basevrf-func.robot -``` - -#### IPv6 Routed-Forwarding -``` -ip6/ - eth2p-ethip6-ip6base-copblklistbase-func.robot - eth2p-ethip6-ip6base-copwhlistbase-func.robot - eth2p-ethip6-ip6base-func.robot - eth2p-ethip6-ip6base-iaclbase-func.robot - eth2p-ethip6-ip6base-ip6dhcpproxy-func.robot - eth2p-ethip6-ip6base-ip6ecmp-func.robot - eth2p-ethip6-ip6base-ip6ra-func.robot - eth2p-ethip6-ip6base-ipolicemarkbase-func.robot - eth2p-ethip6-ip6basevrf-func.robot -``` - -#### IPv4 Overlay Tunnels -``` -ip4_tunnels/ - gre/ - eth2p-ethip4gre-ip4base-func.robot - lisp/ - api-crud-lisp-func.robot - eth2p-ethip4lispgpe-ip4base-func.robot - eth2p-ethip4lispgpe-ip4basevrf-func.robot - eth2p-ethip4lispgpe-ip6base-func.robot - eth2p-ethip4lispgpe-ip6basevrf-func.robot - eth2p-ethip4lisp-ip4base-func.robot - eth2p-ethip4lisp-l2bdbasemaclrn-func.robot - softwire/ - eth2p-ethip4--ethip6ip4-ip4base--ip6base-swiremapt-func.robot - eth2p-ethip4--ethip6ip4-ip4base--ip6base-swirelw46-func.robot - eth2p-ethip4--ethip6ip4-ip4base--ip6base-swiremape-func.robot - vxlan/ - eth2p-dot1qip4vxlan-l2bdbasemaclrn-func.robot - eth4p-ethip4vxlan-l2bdbasemaclrn-l2shg-func.robot - eth2p-ethip4vxlan-l2bdbasemaclrn-func.robot - eth2p-ethip4vxlan-l2xcbase-func.robot -``` - -#### IPv6 Overlay Tunnels -``` -ip6_tunnels/ - lisp/ - eth2p-ethip6lispgpe-ip6base-func.robot - eth2p-ethip6lispgpe-ip6basevrf-func.robot - eth2p-ethip6lispgpe-ip4base-func.robot - eth2p-ethip6lisp-l2bdbasemaclrn-func.robot - vxlan/ - eth2p-ethip6vxlan-l2bdbasemaclrn-func.robot - eth4p-ethip6vxlan-l2bdbasemaclrn-l2shg-func.robot -``` - -#### VM vhost Connections -``` -vm_vhost/ - l2bd/ - eth2p-dot1q-l2bdbasemaclrn-eth-2vhost-1vm-func.robot - eth2p-dot1q-l2bdbasemaclrn-eth-4vhost-2vm-fds-provider-nets-func.robot - eth2p-ethip4-l2bdbase-vhost-client-reconnect-2vm-func.robot - eth2p-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-func.robot - eth2p-ethip4vxlan-l2bdbasemaclrn--eth-4vhost-2vm-fds-tenant-nets-func.robot - eth2p-ethip6vxlan-l2bdbasemaclrn-eth-2vhost-1vm-func.robot - eth2p-eth-l2bdbasemaclrn-eth-2vhost-1vm-func.robot - eth2p-eth-l2bdbasemacstc-eth-2vhost-1vm-func.robot - l2xc/ - eth2p-eth-l2xcbase-eth-2vhost-1vm-func.robot - ip4/ - eth2p-ethip4-ip4base-eth-2vhost-1vm.robot - eth2p-ethip4ipsectptlispgpe-ip4base-eth-2vhost-1vm-func.robot - eth2p-ethip4ipsectptlispgpe-ip6base-eth-2vhost-1vm-func.robot - eth2p-ethip4lispgpe-ip4base-eth-2vhost-1vm-func.robot - eth2p-ethip4lispgpe-ip4basevrf-eth-2vhost-1vm-func.robot - eth2p-ethip4lispgpe-ip6base-eth-2vhost-1vm-func.robot - ip6/ - eth2p-ethip6ipsectptlispgpe-ip4base-eth-2vhost-1vm-func.robot - eth2p-ethip6ipsectptlispgpe-ip6base-eth-2vhost-1vm-func.robot - eth2p-ethip6lispgpe-ip6base-eth-2vhost-1vm-func.robot - eth2p-ethip6lispgpe-ip6basevrf-eth-2vhost-1vm-func.robot -``` - -#### Crypto HW: IP4 Routed-Forwarding -``` -crypto/ - eth2p-ethip4ipsectnl-ip4base-func.robot - eth2p-ethip4ipsectpt-ip4base-func.robot - eth2p-ethip4ipsectptlispgpe-ip4base-func.robot - eth2p-ethip4ipsectptlispgpe-ip6base-func.robot - eth2p-ethip4ipsectptlispgpe-ip6basevrf-func.robot - eth2p-ethip6ipsectnl-ip6base-func.robot - eth2p-ethip6ipsectpt-ip6base-func.robot - eth2p-ethip6ipsectptlispgpe-ip4base-func.robot - eth2p-ethip6ipsectptlispgpe-ip6base-func.robot -``` - -#### Honecomb -``` -honeycomb/ - __init__.robot - mgmt-cfg-l2fib-apihc-apivat-func.robot - mgmt-cfg-slaac-apihc-func.robot - mgmt-cfg-l2bd-apihc-apivat-func.robot - mgmt-cfg-lisp-apihc-apivat-func.robot - mgmt-cfg-intip4-intip6-apihc-apivat-func.robot - mgmt-cfg-nsh-apihc-apivat-func.robot - mgmt-cfg-proxyarp-apihc-func.robot - mgmt-cfg-int-subint-apihc-apivat-func.robot - mgmt-cfg-snat44-apihc-apivat-func.robot - mgmt-cfg-vxlangpe-apihc-apivat-func.robot - mgmt-cfg-pluginacl-apihc-apivat-func.robot - mgmt-cfg-dhcp-apihc-apivat-func.robot - mgmt-cfg-inttap-apihc-apivat-func.robot - mgmt-cfg-routing-apihc-apivat-func.robot - mgmt-cfg-spanrx-apihc-apivat-func.robot - mgmt-cfg-vxlan-apihc-apivat-func.robot - mgmt-cfg-policer-apihc-func.robot - mgmt-cfg-intvhost-apihc-apivat-func.robot - mgmt-notif-apihcnc-func.robot - mgmt-cfg-proxynd6-apihc-func.robot - mgmt-cfg-pbb-apihc-apivat-func.robot - mgmt-statepersist-apihc-func.robot - mgmt-cfg-int-apihcnc-func.robot - mgmt-cfg-acl-apihc-apivat-func.robot -``` - -#### Telemetry -``` -telemetry/ - eth2p-ethip4-ip4base-ip4ipfixbase-func.robot - eth2p-ethip4-ip4base-ip4ipfixscale-func.robot - eth2p-ethip4-ip4base-spanrx-func.robot - eth2p-ethip6-ip6base-ip6ipfixbase-func.robot - eth2p-ethip6-ip6base-ip6ipfixscale-func.robot - eth2p-ethip6-ip6base-spanrx-func.robot -``` - -#### Interface -``` -interfaces/ - api-crud-tap-func.robot - eth2p-ethip4-ip4base-eth-1tap-func.robot - eth2p-eth-l2bdbasemaclrn-eth-2tap-func.robot - eth2p-eth-l2bdbasemaclrn-l2shg-eth-2tap-func.robot -``` - - -## Keywords - -### L2 Robot keywords -``` -shared/ - counters.robot - default.robot - interfaces.robot - traffic.robot - testing_path.robot - lxc.robot -l2/ - bridge_domain.robot - l2_traffic.robot - l2_xconnect.robot - tagging.robot -ip/ - ipv4.robot - ipv6.robot - snat.robot - map.robot -overlay/ - gre.robot - lisp_static_adjacency.robot - lispgpe.robot - l2lisp.robot - lisp_api.robot - vxlan.robot -vm/ - double_qemu_setup.robot - qemu.robot -crypto/ - ipsec.robot -dpdk/ - default.robot -nsh_sfc/ - default.robot -tldk/ - TLDKUtils.robot -honeycomb/ - policer.robot - nat.robot - port_mirroring.robot - vhost_user.robot - netconf.robot - nsh.robot - vxlan.robot - dhcp.robot - slaac.robot - notifications.robot - routing.robot - access_control_lists.robot - tap.robot - interfaces.robot - honeycomb.robot - persistence.robot - sub_interface.robot - provider_backbone_bridge.robot - vxlan_gpe.robot - proxyarp.robot - bridge_domain.robot - lisp.robot - l2_fib.robot -performance/ - performance_configuration.robot - performance_utils.robot - performance_setup.robot -telemetry/ - span.robot - ipfix.robot -features/ - policer.robot - dhcp_client.robot - dhcp_proxy.robot -fds/ - default.robot -``` - -### L1 Python keywords - -No changes at this stage. diff --git a/docs/qemu_vpp_vm b/docs/qemu_vpp_vm deleted file mode 100644 index 7f99231e15..0000000000 --- a/docs/qemu_vpp_vm +++ /dev/null @@ -1,49 +0,0 @@ -# Copyright (c) 2016 Cisco and/or its affiliates. -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at: -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -QEMU is used for VPP-VM testing enviroment. You need to run qemu-2.2.1 or newer -in order to Vhos-user support. In CSIT VIRL setup DUT has preinstalled QEMU and -have small VM image "/var/lib/vm/vhost-nested.img". QEMU binary path must -be "/opt/qemu/bin/qemu-system-x86_64". You can use following script to replicate -QEMU setup on DUT "resources/libraries/bash/qemu_build.sh" for local testing, -out of LF's VM setup. VM image must have installed at least qemu-guest-agent, -sshd, bridge-utils and VirtIO support. Note that VPP must be installed before -starting QEMU, because VPP will do the hugepages configuration as part of it's -installation process. Username/password for the VM must be cisco/cisco and -NOPASSWD sudo access. The interface naming is based on driver (management -interface type is Intel E1000), all E1000 interfaces will be named mgmt<n> and -all VirtIO interfaces will be named virtio<n>. In VM -"/etc/init.d/qemu-guest-agent" you must set "TRANSPORT=isa-serial:/dev/ttyS1" -because ttyS0 is used by serial console and ttyS1 is dedicated for -qemu-guest-agent in QEMU setup. -There is python library for QEMU setup, start and some utilities -"resources/libraries/python/QemuUtils.py" and keyword "Stop and Clear QEMU" for -teardown in resources/libraries/robot/vm/qemu.robot. "Qemu Start" setup one -management interface by default. You can add Vhost-user interfaces by -"Qemu Add Vhost User If" keyword. -Example usage in robot framework test: - -*** Settings *** -| Resource | resources/libraries/robot/vm/qemu.robot - -*** Test Cases *** -| VM test -| | [Tags] | VPP_VM_ENV -| | Qemu Set Node | ${nodes['DUT1']} -| | Qemu Add Vhost User If | /tmp/vhost_sock -| | ${vm}= | Set Variable | ${None} -| | ${vm}= | Qemu Start -| | [Teardown] | Stop and Clear QEMU | ${dut1} | ${vm} - -Note that VPP must be started and configured before starting the VM when using -Vhost-user interfaces. diff --git a/docs/report/introduction/methodology_kvm_vms_vhost_user.rst b/docs/report/introduction/methodology_kvm_vms_vhost_user.rst index 0a0ea1be1e..0a465cf0fb 100644 --- a/docs/report/introduction/methodology_kvm_vms_vhost_user.rst +++ b/docs/report/introduction/methodology_kvm_vms_vhost_user.rst @@ -1,6 +1,24 @@ KVM VMs vhost-user ------------------ +QEMU is used for VPP-VM testing enviroment. By default, standard QEMU version +preinstalled from OS repositories is used on VIRL/vpp_device functional testing +(qemu-2.11.x for Ubuntu 18.04, qemu-2.5.0 for Ubuntu 16.04). For perfomance +testing QEMU is downloaded from `project homepage <qemu.org>`_ and compiled +during testing. This allows framework to easily inject QEMU patches in case of +need. In QEMU version <2.8 we used it for increasing QEMU virtion queue size. +In CSIT setup DUTs have small VM image `/var/lib/vm/vhost-nested.img`. QEMU +binary can be adjusted in global settings. VM image must have installed at least +qemu-guest-agent, sshd, bridge-utils, VirtIO support and Testpmd/L3fwd +application. Username/password for the VM must be cisco/cisco and +NOPASSWD sudo access. The interface naming is based on driver (management +interface type is Intel E1000), all E1000 interfaces will be named mgmt<n> and +all VirtIO interfaces will be named virtio<n>. In VM +"/etc/init.d/qemu-guest-agent" you must set "TRANSPORT=isa-serial:/dev/ttyS1" +because ttyS0 is used by serial console and ttyS1 is dedicated for +qemu-guest-agent in QEMU setup. There is python library for QEMU setup, start +and some utilities "resources/libraries/python/QemuUtils.py" + FD.io CSIT performance lab is testing VPP vhost with KVM VMs using following environment settings: diff --git a/docs/tg_interface_driver b/docs/tg_interface_driver deleted file mode 100644 index a45999a819..0000000000 --- a/docs/tg_interface_driver +++ /dev/null @@ -1,14 +0,0 @@ -If using traffic scripts in test add "| Suite Setup | Setup all TGs before -traffic script" to test suite robot file, this bind TG interfaces to the kernel -driver specified in topology. Also add kernel driver name for TG interfaces to -topology YAML file. You can find driver name with following command where you -specify interface PCI address: -# lspci -vmmks 0000:00:07.0 -Slot: 00:07.0 -Class: Ethernet controller -Vendor: Red Hat, Inc -Device: Virtio network device -SVendor: Red Hat, Inc -SDevice: Device 0001 -PhySlot: 7 -Driver: virtio-pci diff --git a/docs/topology_schemas b/docs/topology_schemas deleted file mode 100644 index d25e99ad76..0000000000 --- a/docs/topology_schemas +++ /dev/null @@ -1,2 +0,0 @@ -http://www.kuwata-lab.com/kwalify/ruby/users-guide.html -http://www.kuwata-lab.com/kwalify/ruby/users-guide.02.html#tips-merge |