From da2aa127ee071e37bc4d6060dd39476179e65365 Mon Sep 17 00:00:00 2001 From: Maciek Konstantynowicz Date: Sun, 5 Feb 2017 16:33:28 +0000 Subject: csit rls1701 report edits: - simplified section structure for clarity and readability, - updated overview sections, - moved not rls related content from rls_notes to overview sections, - removed section title suffixes: HW, VIRL. - completed vpp_unit_tests and vpp_unittest_results sections. - updated all documentation sections. - updated rls_notes sections for vpp performance and vpp functional. Change-Id: Id2c2abbf9d3531ec9f63ecd353f385a0b55ae1ba Signed-off-by: Maciek Konstantynowicz Signed-off-by: pmikus Signed-off-by: Maciek Konstantynowicz Signed-off-by: pmikus Signed-off-by: Maciek Konstantynowicz --- .../csit_release_notes.rst | 68 ---------- .../documentation.rst | 4 - .../honeycomb_functional_tests_virl/index.rst | 10 -- .../honeycomb_functional_tests_virl/overview.rst | 137 --------------------- .../test_environment.rst | 33 ----- 5 files changed, 252 deletions(-) delete mode 100644 docs/report/honeycomb_functional_tests_virl/csit_release_notes.rst delete mode 100644 docs/report/honeycomb_functional_tests_virl/documentation.rst delete mode 100644 docs/report/honeycomb_functional_tests_virl/index.rst delete mode 100644 docs/report/honeycomb_functional_tests_virl/overview.rst delete mode 100644 docs/report/honeycomb_functional_tests_virl/test_environment.rst (limited to 'docs/report/honeycomb_functional_tests_virl') diff --git a/docs/report/honeycomb_functional_tests_virl/csit_release_notes.rst b/docs/report/honeycomb_functional_tests_virl/csit_release_notes.rst deleted file mode 100644 index 2708b05b0c..0000000000 --- a/docs/report/honeycomb_functional_tests_virl/csit_release_notes.rst +++ /dev/null @@ -1,68 +0,0 @@ -CSIT Release Notes -================== - -Changes in CSIT |release| -------------------------- - -#. Naming change for all Honeycomb functional test suites in VIRL - - - Honeycomb functional test case names stayed unchanged - -#. Added Honeycomb functional tests - - - NSH_SFC - - LISP - - NAT - - SPAN - -Functional Tests Naming ------------------------ - -CSIT |release| introduced a common structured naming convention for all -performance and functional tests. This change was driven by substantially -growing number and type of CSIT test cases. Firstly, the original practice did -not always follow any strict naming convention. Secondly test names did not -always clearly capture tested packet encapsulations, and the actual type or -content of the tests. Thirdly HW configurations in terms of NICs, ports and -their locality were not captured either. These were but few reasons that drove -the decision to change and define a new more complete and stricter test naming -convention, and to apply this to all existing and new test cases. - -The new naming should be intuitive for majority of the tests. The complete -description of CSIT test naming convention is provided on `CSIT test naming -page `_. - -Here few illustrative examples of the new naming usage for functional test -suites: - -#. **Physical port to physical port - a.k.a. NIC-to-NIC, Phy-to-Phy, P2P** - - - *eth2p-ethip4-ip4base-func.robot* => 2 ports of Ethernet, IPv4 baseline - routed forwarding, functional tests. - -#. **Physical port to VM (or VM chain) to physical port - a.k.a. NIC2VM2NIC, - P2V2P, NIC2VMchain2NIC, P2V2V2P** - - - *eth2p-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-func.robot* => 2 ports of - Ethernet, IPv4 VXLAN Ethernet, L2 bridge-domain switching to/from two vhost - interfaces and one VM, functional tests. - - -Known Issues ------------- - -Here is the list of known issues in CSIT |release| for Honeycomb functional -tests in VIRL: - -+-------------------------------------------------+-----------------------------------------------------------------+ -| Issue | Description | -+-------------------------------------------------+-----------------------------------------------------------------+ -| Operational data for Vhost-user interfaces | Honeycomb Operational data reports Vhost-user interfaces | -| "server" flag | as client, even if they are server. | -+-------------------------------------------------+-----------------------------------------------------------------+ -| Persistence of VxLAN tunnels | Configuration persistence often fails to restore | -| | Honeycomb's internal naming context for VxLAN interfaces. | -| | The interface is renamed to "vxlan_tunnel0" but is otherwise | -| | configured correctly. | -+-------------------------------------------------+-----------------------------------------------------------------+ - diff --git a/docs/report/honeycomb_functional_tests_virl/documentation.rst b/docs/report/honeycomb_functional_tests_virl/documentation.rst deleted file mode 100644 index 6904068516..0000000000 --- a/docs/report/honeycomb_functional_tests_virl/documentation.rst +++ /dev/null @@ -1,4 +0,0 @@ -Documentation -============= - -Documentation is located on nexus repository https://docs.fd.io/csit/master/doc/tests.func.html. diff --git a/docs/report/honeycomb_functional_tests_virl/index.rst b/docs/report/honeycomb_functional_tests_virl/index.rst deleted file mode 100644 index 6b58b06935..0000000000 --- a/docs/report/honeycomb_functional_tests_virl/index.rst +++ /dev/null @@ -1,10 +0,0 @@ -HoneyComb Functional Tests VIRL -=============================== - -.. toctree:: - - overview - csit_release_notes - test_environment - documentation - diff --git a/docs/report/honeycomb_functional_tests_virl/overview.rst b/docs/report/honeycomb_functional_tests_virl/overview.rst deleted file mode 100644 index 2bfc89a814..0000000000 --- a/docs/report/honeycomb_functional_tests_virl/overview.rst +++ /dev/null @@ -1,137 +0,0 @@ -Overview -======== - -Tested Topologies VIRL ----------------------- - -CSIT Honeycomb functional tests are executed on virtualized topologies created -using Virtual Internet Routing Lab (VIRL) simulation platform contributed by -Cisco. VIRL runs on physical baremetal servers hosted by LF FD.io project. All -tests are executed in two node logical test topology - Traffic Generator (TG) -node and Systems Under Test (SUT1) node connected in a loop. Logical test -topology is shown in the figure below. - -:: - - +------------------------+ - | | - | +------------------+ | - +---------------> <--------------+ - | | | | | | - | |------------> DUT1 <-----------+ | - | | | +------------------+ | | | - | | | | | | - | | | SUT1 | | | - | | +------------------------+ | | - | | | | - | | | | - | | +-----------+ | | - | +---------------> <---------------+ | - | | TG | | - +------------------> <------------------+ - +-----------+ - -SUT1 is a VM (Ubuntu or Centos, depending on the test suite), TG is a Traffic -Generator (TG, another Ubuntu VM). SUTs run Honeycomb and VPP SW applications -in Linux user-mode as a Device Under Test (DUT) within the VM. TG runs Scapy -SW application as a packet Traffic Generator. Logical connectivity between -SUTs and to TG is provided using virtual NICs using VMs' virtio driver. - -Virtual testbeds are created on-demand whenever a verification job is started -(e.g. triggered by the gerrit patch submission) and destroyed upon completion -of all functional tests. Each node is a Virtual Machine and each connection -that is drawn on the diagram is available for use in any test case. During the -test execution, all nodes are reachable thru the Management network connected -to every node via dedicated virtual NICs and virtual links (not shown above -for clarity). - -Honeycomb Functional Tests Overview ------------------------------------ - -The following Honeycomb functional test areas are included in the CSIT |release| -with results listed in this report: - -- **Basic interface management** - CRUD for interface state, - - ipv4/ipv6 address, ipv4 neighbor, MTU value. - - Test case count: 7 -- **L2BD** - CRUD for L2 Bridge-Domain, interface assignment. - - Create up to two bridge domains with all implemented functions turned on. - - (flooding, unknown-unicast flooding, forwarding, learning, arp-termination) - - Assign up to two physical interfaces to a single bridge domain. - - Remove interface assignments, remove bridge domains. - - Test case count: 5 -- **L2FIB** - CRD for L2-FIB entries. - - Create 4 FIB entries - - (one of each for filter/forward, static/dynamic combinations). - - Remove FIB entries. - - Test case count: 7 -- **VxLAN** - CRD for VxLAN tunnels. - - Create VxLAN interface. - - Disable VxLAN interface. - - Re-create a disabled VxLAN interface. - - Test case count: 6 -- **VxLAN-GPE** - CRD for VxLAN GPE tunnels. - - Create VxLAN GPE interface. - - Disable VxLAN interface. - - Re-create a disabled VxLAN interface. - - Test case count: 7 -- **Vhost-user** - CRUD for Vhost-user interfaces. - - Create, modify and delete Vhost-user interface, as client and server. - - Test case count: 8 -- **TAP** - CRUD for Tap interface management. - - Create, modify and delete TAP interface. - - Test case count: 3 -- **VLAN** - CRUD for VLAN sub-interface management. - - Create VLAN sub-interface over a physical interface. - - Toggle interface state separately for super-interface and sub-interface. - - Configure IP address and bridge domain assignment on sub-interface. - - Configure VLAN tag rewrite on sub-interface. - - Test case count: 17 -- **ACL** - CRD for low-level classifiers: table and session management, - - interface assignment. - - Configure up to 2 classify tables. - - Configure up to 2 classify sessions on one table. - - Assign classify session to a physical interface. - - Remove tables, sessions, interface assignments. - - Test case count: 9 -- **PBB** - CRD for provider backbone bridge sub-interface. - - Configure, modify and remove a PBB sub-interface over a physical interface. - - Test case count: 9 -- **NSH_SFC** - CRD for NSH maps and entries, using NSH_SFC plugin. - - Configure up to 2 NSH entries. - - Configure up to 2 NSH maps. - - Modify and delete NSH maps and entries. - - Test case count: 8 -- **LISP** - CRD for Lisp: mapping, locator set, adjacency, map resolver. - - Toggle Lisp feature status. - - Configure and delete Lisp mapping as local and remote. - - Configure and delete Lisp adjacency mapping - - Configure and delete Lisp map resolver, proxy ITR. - - Test case count: 11 -- **NAT** - CRD for NAT entries, interface assignment. - - Configure and delete up to two NAT entries. - - Assign NAT entries to a physical interface. - - Test case count: 6 -- **Port mirroring** - CRD for SPAN port mirroring, interface assignment. - - Configure SPAN port mirroring on a physical interface, mirroring - - up to 2 interfaces. - - Remove SPAN configuration from interfaces. - - Test case count: 3 -- **Honeycomb Infractructure** - configuration persistence, - - Netconf notifications for interface events, - - Netconf negative tests aimed at specific issues - -Total 111 Honeycomb tests in the CSIT |release|. - -Operational data in Honeycomb should mirror configuration data at all times. -Because of this, test cases follow this general pattern: - -#. read operational data of the feature using restconf. -#. read status of the feature using VPP API dump. -#. modify configuration of the feature using restconf. -#. verify changes to operational data using restconf. -#. verify changes using VPP API dump. - -Test cases involving network interfaces utilize the first two interfaces on -the DUT node. - diff --git a/docs/report/honeycomb_functional_tests_virl/test_environment.rst b/docs/report/honeycomb_functional_tests_virl/test_environment.rst deleted file mode 100644 index e706aa5573..0000000000 --- a/docs/report/honeycomb_functional_tests_virl/test_environment.rst +++ /dev/null @@ -1,33 +0,0 @@ -Test Environment -================ - -CSIT Honeycomb tests are currently executed in VIRL, as mentioned above. The -physical VIRL testbed infrastructure consists of three identical VIRL hosts, -each host being a Cisco UCS C240-M4 (2x Intel(R) Xeon(R) CPU E5-2699 v3 @ -2.30GHz, 18c, 512GB RAM) running Ubuntu 14.04.3 and the following VIRL software -versions: - - STD server version 0.10.24.7 - UWM server version 0.10.24.7 - -Current VPP tests have been executed on a single VM operating system and -version only, as described in the following paragraphs. - -In CSIT terminology, the VM operating system for both SUTs and TG that VPP 17.01 -has been tested with, is the following: - - ubuntu-14.04.4_2016-10-07_1.3 - -which implies Ubuntu 14.04.3 LTS, current as of 2016/10/07 (that is, package -versions are those that would have been installed by a "apt-get update", -"apt-get upgrade" on October 7), produced by CSIT disk image build scripts -version 1.6. - -The exact list of installed packages and their versions (including the Linux -kernel package version) are included in CSIT source repository: - - resources/tools/disk-image-builder/ubuntu/lists/ubuntu-14.04.4_2016-10-07_1.3 - -A replica of this VM image can be built by running the "build.sh" script in CSIT -repository resources/tools/disk-image-builder/. - -- cgit 1.2.3-korg