aboutsummaryrefslogtreecommitdiffstats
path: root/docs/report/honeycomb_functional_tests
diff options
context:
space:
mode:
authorselias <samelias@cisco.com>2017-07-20 09:50:58 +0200
committerselias <samelias@cisco.com>2017-07-24 15:22:02 +0200
commitcd8d4dc1498a69b4f9541fb4405fd31fd61d81d9 (patch)
tree2a00205af6500a085afcb8306beec29ab5d1b584 /docs/report/honeycomb_functional_tests
parent1911067935bdf6481fc9b89f40d79ca61f2448d0 (diff)
Static report content - Honeycomb
Change-Id: I39899747f99684b932bb9c88268391e55b6fc779 Signed-off-by: selias <samelias@cisco.com>
Diffstat (limited to 'docs/report/honeycomb_functional_tests')
-rw-r--r--docs/report/honeycomb_functional_tests/csit_release_notes.rst39
-rw-r--r--docs/report/honeycomb_functional_tests/index.rst2
-rw-r--r--docs/report/honeycomb_functional_tests/overview.rst18
3 files changed, 32 insertions, 27 deletions
diff --git a/docs/report/honeycomb_functional_tests/csit_release_notes.rst b/docs/report/honeycomb_functional_tests/csit_release_notes.rst
index a398fa41cf..38ec001418 100644
--- a/docs/report/honeycomb_functional_tests/csit_release_notes.rst
+++ b/docs/report/honeycomb_functional_tests/csit_release_notes.rst
@@ -4,16 +4,15 @@ CSIT Release Notes
Changes in CSIT |release|
-------------------------
-#. Added Honeycomb functional tests
+#. Added Honeycomb functional tests for the following features:
- - ACL plugin
- - Routing
- - SLAAC
- - Proxy ARP
- - DHCP Relay
- - Neighbor Discovery Proxy
+ - Policer
-#. Changed execution environment from Ubuntu14.04 to Ubuntu16.04
+#. Improved test coverage for the following features:
+
+ - Interface Management
+ - Vlan
+ - Port Mirroring
Known Issues
------------
@@ -25,18 +24,20 @@ tests in VIRL:
| # | Issue | Jira ID | Description |
+---+--------------------------------------------+------------+----------------------------------------------------------------------------+
| 1 | IP address subnet validation | VPP-649 | When configuring two IP addresses from the same subnet on an interface, |
-| | | | VPP refuses the configuration but returns OK. This can cause desync |
-| | | | between Honeycomb's config and operational data. |
+| | | | VPP refuses the configuration but returns code 200:OK. This can cause |
+| | | | desync between Honeycomb's config and operational data. |
+---+--------------------------------------------+------------+----------------------------------------------------------------------------+
-| 2 | Persistence of VxLAN tunnel naming context | HC2VPP-47 | When VPP restarts with Honeycomb running and a VxLan interface configured, |
-| | | | the interface is sometimes renamed to "vxlan_tunnel0". |
-| | | | It is otherwise configured correctly. |
+| 2 | Removal of ACP-plugin interface assignment | HC2VPP-173 | Attempting to remove all ACLs from an interface responds with OK but does |
+| | | | not remove the assignments. |
+---+--------------------------------------------+------------+----------------------------------------------------------------------------+
-| 3 | Classifier plugin for IPv6 cases | VPP-687 | Classifier ignores IPv6 packets with less than 8 bytes after last header. |
-| | | | Fixed in VPP 17.07. |
+| 3 | VxLAN GPE configuration crashes VPP | VPP-875 | Specific VxLAN GPE configurations cause VPP to crash and restart. |
+---+--------------------------------------------+------------+----------------------------------------------------------------------------+
-| 4 | Batch disable Lisp features | HC2VPP-131 | When removing complex Lisp configurations in a single request, |
-| | | | the operation fails due to a write ordering issue. |
+| 4 | Policer traffic test failure | CSIT- | Traffic test has begun to fail, likely due to VPP changes. There is more |
+| | | | information available yet. |
++---+--------------------------------------------+------------+----------------------------------------------------------------------------+
+| 5 | SPAN traffic test failure | CSIT- | Traffic test has begun to fail, likely due to VPP changes. There is more |
+| | | | information available yet. |
++---+--------------------------------------------+------------+----------------------------------------------------------------------------+
+| 6 | Unnumbered interface VIRL issue | CSIT- | CRUD for unnumbered interface appears to fail in VIRL, but not in local |
+| | | | test runs. Investigation pending. |
+---+--------------------------------------------+------------+----------------------------------------------------------------------------+
-
-
diff --git a/docs/report/honeycomb_functional_tests/index.rst b/docs/report/honeycomb_functional_tests/index.rst
index d5d1b6180b..424ad3640f 100644
--- a/docs/report/honeycomb_functional_tests/index.rst
+++ b/docs/report/honeycomb_functional_tests/index.rst
@@ -1,4 +1,4 @@
-HoneyComb Functional Tests
+Honeycomb Functional Tests
==========================
.. toctree::
diff --git a/docs/report/honeycomb_functional_tests/overview.rst b/docs/report/honeycomb_functional_tests/overview.rst
index c73e9706f8..1c992a5c2d 100644
--- a/docs/report/honeycomb_functional_tests/overview.rst
+++ b/docs/report/honeycomb_functional_tests/overview.rst
@@ -53,7 +53,7 @@ with results listed in this report:
- **Basic interface management** - CRUD for interface state,
- ipv4/ipv6 address, ipv4 neighbor, MTU value.
- - Test case count: 7
+ - Test case count: 14
- **L2BD** - CRUD for L2 Bridge-Domain, interface assignment.
- Create up to two bridge domains with all implemented functions turned on.
- (flooding, unknown-unicast flooding, forwarding, learning, arp-termination)
@@ -86,7 +86,7 @@ with results listed in this report:
- Toggle interface state separately for super-interface and sub-interface.
- Configure IP address and bridge domain assignment on sub-interface.
- Configure VLAN tag rewrite on sub-interface.
- - Test case count: 17
+ - Test case count: 24
- **ACL** - CRD for low-level classifiers: table and session management,
- interface assignment.
- Configure up to 2 classify tables.
@@ -96,7 +96,7 @@ with results listed in this report:
- Test case count: 9
- **PBB** - CRD for provider backbone bridge sub-interface.
- Configure, modify and remove a PBB sub-interface over a physical interface.
- - Test case count: 9
+ - Test case count: 8
- **NSH_SFC** - CRD for NSH maps and entries, using NSH_SFC plugin.
- Configure up to 2 NSH entries.
- Configure up to 2 NSH maps.
@@ -107,7 +107,7 @@ with results listed in this report:
- Configure and delete Lisp mapping as local and remote.
- Configure and delete Lisp adjacency mapping
- Configure and delete Lisp map resolver, proxy ITR.
- - Test case count: 11
+ - Test case count: 16
- **NAT** - CRD for NAT entries, interface assignment.
- Configure and delete up to two NAT entries.
- Assign NAT entries to a physical interface.
@@ -116,7 +116,7 @@ with results listed in this report:
- Configure SPAN port mirroring on a physical interface, mirroring
- up to 2 interfaces.
- Remove SPAN configuration from interfaces.
- - Test case count: 3
+ - Test case count: 14
- **ACL-PLUGIN** - CRD for high-level classifier
- MAC + IP address classification.
- IPv4, IPv6 address classification.
@@ -144,11 +144,15 @@ with results listed in this report:
- Configure blackhole route.
- IPv4 and IPv6 variants.
- Test case count: 6
+- **Policer** - CRD for traffic policing feature.
+ - Configure Policing rules.
+ - Assign to interface.
+ - Test case count: 6
- **Honeycomb Infractructure** - configuration persistence,
- Netconf notifications for interface events,
- Netconf negative tests aimed at specific issues
-Total 158 Honeycomb tests in the CSIT |release|.
+Total 173 Honeycomb functional tests in the CSIT |release|.
Operational data in Honeycomb should mirror configuration data at all times.
Because of this, test cases follow this general pattern:
@@ -158,7 +162,7 @@ Because of this, test cases follow this general pattern:
#. modify configuration of the feature using restconf.
#. verify changes to operational data using restconf.
#. verify changes using VPP API dump, OR
-#. send a packet to VPP node and observe behaviour to verify configuration
+#. send a packet to VPP node and observe behaviour to verify configuration.
Test cases involving network interfaces utilize the first two interfaces on
the DUT node.