diff options
author | 2017-02-05 16:33:28 +0000 | |
---|---|---|
committer | 2017-02-06 16:27:37 +0000 | |
commit | da2aa127ee071e37bc4d6060dd39476179e65365 (patch) | |
tree | ea4fa5cd56fc6219efa290ba2b93288bc84ebe7e /docs/report/honeycomb_functional_tests_virl | |
parent | a1d482fcf9c221cda2f13429810bca945d9f09f4 (diff) |
csit rls1701 report edits:
- simplified section structure for clarity and readability,
- updated overview sections,
- moved not rls related content from rls_notes to overview sections,
- removed section title suffixes: HW, VIRL.
- completed vpp_unit_tests and vpp_unittest_results sections.
- updated all documentation sections.
- updated rls_notes sections for vpp performance and vpp functional.
Change-Id: Id2c2abbf9d3531ec9f63ecd353f385a0b55ae1ba
Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
Signed-off-by: pmikus <pmikus@cisco.com>
Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
Signed-off-by: pmikus <pmikus@cisco.com>
Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
Diffstat (limited to 'docs/report/honeycomb_functional_tests_virl')
5 files changed, 0 insertions, 252 deletions
diff --git a/docs/report/honeycomb_functional_tests_virl/csit_release_notes.rst b/docs/report/honeycomb_functional_tests_virl/csit_release_notes.rst deleted file mode 100644 index 2708b05b0c..0000000000 --- a/docs/report/honeycomb_functional_tests_virl/csit_release_notes.rst +++ /dev/null @@ -1,68 +0,0 @@ -CSIT Release Notes
-==================
-
-Changes in CSIT |release|
--------------------------
-
-#. Naming change for all Honeycomb functional test suites in VIRL
-
- - Honeycomb functional test case names stayed unchanged
-
-#. Added Honeycomb functional tests
-
- - NSH_SFC
- - LISP
- - NAT
- - SPAN
-
-Functional Tests Naming
------------------------
-
-CSIT |release| introduced a common structured naming convention for all
-performance and functional tests. This change was driven by substantially
-growing number and type of CSIT test cases. Firstly, the original practice did
-not always follow any strict naming convention. Secondly test names did not
-always clearly capture tested packet encapsulations, and the actual type or
-content of the tests. Thirdly HW configurations in terms of NICs, ports and
-their locality were not captured either. These were but few reasons that drove
-the decision to change and define a new more complete and stricter test naming
-convention, and to apply this to all existing and new test cases.
-
-The new naming should be intuitive for majority of the tests. The complete
-description of CSIT test naming convention is provided on `CSIT test naming
-page <https://wiki.fd.io/view/CSIT/csit-test-naming>`_.
-
-Here few illustrative examples of the new naming usage for functional test
-suites:
-
-#. **Physical port to physical port - a.k.a. NIC-to-NIC, Phy-to-Phy, P2P**
-
- - *eth2p-ethip4-ip4base-func.robot* => 2 ports of Ethernet, IPv4 baseline
- routed forwarding, functional tests.
-
-#. **Physical port to VM (or VM chain) to physical port - a.k.a. NIC2VM2NIC,
- P2V2P, NIC2VMchain2NIC, P2V2V2P**
-
- - *eth2p-ethip4vxlan-l2bdbasemaclrn-eth-2vhost-1vm-func.robot* => 2 ports of
- Ethernet, IPv4 VXLAN Ethernet, L2 bridge-domain switching to/from two vhost
- interfaces and one VM, functional tests.
-
-
-Known Issues
-------------
-
-Here is the list of known issues in CSIT |release| for Honeycomb functional
-tests in VIRL:
-
-+-------------------------------------------------+-----------------------------------------------------------------+
-| Issue | Description |
-+-------------------------------------------------+-----------------------------------------------------------------+
-| Operational data for Vhost-user interfaces | Honeycomb Operational data reports Vhost-user interfaces |
-| "server" flag | as client, even if they are server. |
-+-------------------------------------------------+-----------------------------------------------------------------+
-| Persistence of VxLAN tunnels | Configuration persistence often fails to restore |
-| | Honeycomb's internal naming context for VxLAN interfaces. |
-| | The interface is renamed to "vxlan_tunnel0" but is otherwise |
-| | configured correctly. |
-+-------------------------------------------------+-----------------------------------------------------------------+
-
diff --git a/docs/report/honeycomb_functional_tests_virl/documentation.rst b/docs/report/honeycomb_functional_tests_virl/documentation.rst deleted file mode 100644 index 6904068516..0000000000 --- a/docs/report/honeycomb_functional_tests_virl/documentation.rst +++ /dev/null @@ -1,4 +0,0 @@ -Documentation
-=============
-
-Documentation is located on nexus repository https://docs.fd.io/csit/master/doc/tests.func.html.
diff --git a/docs/report/honeycomb_functional_tests_virl/index.rst b/docs/report/honeycomb_functional_tests_virl/index.rst deleted file mode 100644 index 6b58b06935..0000000000 --- a/docs/report/honeycomb_functional_tests_virl/index.rst +++ /dev/null @@ -1,10 +0,0 @@ -HoneyComb Functional Tests VIRL -=============================== - -.. toctree:: - - overview - csit_release_notes - test_environment - documentation - diff --git a/docs/report/honeycomb_functional_tests_virl/overview.rst b/docs/report/honeycomb_functional_tests_virl/overview.rst deleted file mode 100644 index 2bfc89a814..0000000000 --- a/docs/report/honeycomb_functional_tests_virl/overview.rst +++ /dev/null @@ -1,137 +0,0 @@ -Overview
-========
-
-Tested Topologies VIRL
-----------------------
-
-CSIT Honeycomb functional tests are executed on virtualized topologies created
-using Virtual Internet Routing Lab (VIRL) simulation platform contributed by
-Cisco. VIRL runs on physical baremetal servers hosted by LF FD.io project. All
-tests are executed in two node logical test topology - Traffic Generator (TG)
-node and Systems Under Test (SUT1) node connected in a loop. Logical test
-topology is shown in the figure below.
-
-::
-
- +------------------------+
- | |
- | +------------------+ |
- +---------------> <--------------+
- | | | | | |
- | |------------> DUT1 <-----------+ |
- | | | +------------------+ | | |
- | | | | | |
- | | | SUT1 | | |
- | | +------------------------+ | |
- | | | |
- | | | |
- | | +-----------+ | |
- | +---------------> <---------------+ |
- | | TG | |
- +------------------> <------------------+
- +-----------+
-
-SUT1 is a VM (Ubuntu or Centos, depending on the test suite), TG is a Traffic
-Generator (TG, another Ubuntu VM). SUTs run Honeycomb and VPP SW applications
-in Linux user-mode as a Device Under Test (DUT) within the VM. TG runs Scapy
-SW application as a packet Traffic Generator. Logical connectivity between
-SUTs and to TG is provided using virtual NICs using VMs' virtio driver.
-
-Virtual testbeds are created on-demand whenever a verification job is started
-(e.g. triggered by the gerrit patch submission) and destroyed upon completion
-of all functional tests. Each node is a Virtual Machine and each connection
-that is drawn on the diagram is available for use in any test case. During the
-test execution, all nodes are reachable thru the Management network connected
-to every node via dedicated virtual NICs and virtual links (not shown above
-for clarity).
-
-Honeycomb Functional Tests Overview
------------------------------------
-
-The following Honeycomb functional test areas are included in the CSIT |release|
-with results listed in this report:
-
-- **Basic interface management** - CRUD for interface state,
- - ipv4/ipv6 address, ipv4 neighbor, MTU value.
- - Test case count: 7
-- **L2BD** - CRUD for L2 Bridge-Domain, interface assignment.
- - Create up to two bridge domains with all implemented functions turned on.
- - (flooding, unknown-unicast flooding, forwarding, learning, arp-termination)
- - Assign up to two physical interfaces to a single bridge domain.
- - Remove interface assignments, remove bridge domains.
- - Test case count: 5
-- **L2FIB** - CRD for L2-FIB entries.
- - Create 4 FIB entries
- - (one of each for filter/forward, static/dynamic combinations).
- - Remove FIB entries.
- - Test case count: 7
-- **VxLAN** - CRD for VxLAN tunnels.
- - Create VxLAN interface.
- - Disable VxLAN interface.
- - Re-create a disabled VxLAN interface.
- - Test case count: 6
-- **VxLAN-GPE** - CRD for VxLAN GPE tunnels.
- - Create VxLAN GPE interface.
- - Disable VxLAN interface.
- - Re-create a disabled VxLAN interface.
- - Test case count: 7
-- **Vhost-user** - CRUD for Vhost-user interfaces.
- - Create, modify and delete Vhost-user interface, as client and server.
- - Test case count: 8
-- **TAP** - CRUD for Tap interface management.
- - Create, modify and delete TAP interface.
- - Test case count: 3
-- **VLAN** - CRUD for VLAN sub-interface management.
- - Create VLAN sub-interface over a physical interface.
- - Toggle interface state separately for super-interface and sub-interface.
- - Configure IP address and bridge domain assignment on sub-interface.
- - Configure VLAN tag rewrite on sub-interface.
- - Test case count: 17
-- **ACL** - CRD for low-level classifiers: table and session management,
- - interface assignment.
- - Configure up to 2 classify tables.
- - Configure up to 2 classify sessions on one table.
- - Assign classify session to a physical interface.
- - Remove tables, sessions, interface assignments.
- - Test case count: 9
-- **PBB** - CRD for provider backbone bridge sub-interface.
- - Configure, modify and remove a PBB sub-interface over a physical interface.
- - Test case count: 9
-- **NSH_SFC** - CRD for NSH maps and entries, using NSH_SFC plugin.
- - Configure up to 2 NSH entries.
- - Configure up to 2 NSH maps.
- - Modify and delete NSH maps and entries.
- - Test case count: 8
-- **LISP** - CRD for Lisp: mapping, locator set, adjacency, map resolver.
- - Toggle Lisp feature status.
- - Configure and delete Lisp mapping as local and remote.
- - Configure and delete Lisp adjacency mapping
- - Configure and delete Lisp map resolver, proxy ITR.
- - Test case count: 11
-- **NAT** - CRD for NAT entries, interface assignment.
- - Configure and delete up to two NAT entries.
- - Assign NAT entries to a physical interface.
- - Test case count: 6
-- **Port mirroring** - CRD for SPAN port mirroring, interface assignment.
- - Configure SPAN port mirroring on a physical interface, mirroring
- - up to 2 interfaces.
- - Remove SPAN configuration from interfaces.
- - Test case count: 3
-- **Honeycomb Infractructure** - configuration persistence,
- - Netconf notifications for interface events,
- - Netconf negative tests aimed at specific issues
-
-Total 111 Honeycomb tests in the CSIT |release|.
-
-Operational data in Honeycomb should mirror configuration data at all times.
-Because of this, test cases follow this general pattern:
-
-#. read operational data of the feature using restconf.
-#. read status of the feature using VPP API dump.
-#. modify configuration of the feature using restconf.
-#. verify changes to operational data using restconf.
-#. verify changes using VPP API dump.
-
-Test cases involving network interfaces utilize the first two interfaces on
-the DUT node.
-
diff --git a/docs/report/honeycomb_functional_tests_virl/test_environment.rst b/docs/report/honeycomb_functional_tests_virl/test_environment.rst deleted file mode 100644 index e706aa5573..0000000000 --- a/docs/report/honeycomb_functional_tests_virl/test_environment.rst +++ /dev/null @@ -1,33 +0,0 @@ -Test Environment
-================
-
-CSIT Honeycomb tests are currently executed in VIRL, as mentioned above. The
-physical VIRL testbed infrastructure consists of three identical VIRL hosts,
-each host being a Cisco UCS C240-M4 (2x Intel(R) Xeon(R) CPU E5-2699 v3 @
-2.30GHz, 18c, 512GB RAM) running Ubuntu 14.04.3 and the following VIRL software
-versions:
-
- STD server version 0.10.24.7
- UWM server version 0.10.24.7
-
-Current VPP tests have been executed on a single VM operating system and
-version only, as described in the following paragraphs.
-
-In CSIT terminology, the VM operating system for both SUTs and TG that VPP 17.01
-has been tested with, is the following:
-
- ubuntu-14.04.4_2016-10-07_1.3
-
-which implies Ubuntu 14.04.3 LTS, current as of 2016/10/07 (that is, package
-versions are those that would have been installed by a "apt-get update",
-"apt-get upgrade" on October 7), produced by CSIT disk image build scripts
-version 1.6.
-
-The exact list of installed packages and their versions (including the Linux
-kernel package version) are included in CSIT source repository:
-
- resources/tools/disk-image-builder/ubuntu/lists/ubuntu-14.04.4_2016-10-07_1.3
-
-A replica of this VM image can be built by running the "build.sh" script in CSIT
-repository resources/tools/disk-image-builder/.
-
|