1704 Report - Honeycomb section

Change-Id: I8006cf8187f943b8d880c8193b6b1bde25ad343c
Signed-off-by: selias <samelias@cisco.com>

3 files changed, 62 insertions, 28 deletions

diff --git a/docs/report/honeycomb_functional_tests/csit_release_notes.rst b/docs/report/honeycomb_functional_tests/csit_release_notes.rst
index 68b5eb6fb5..52525423f6 100644
--- a/docs/report/honeycomb_functional_tests/csit_release_notes.rst
+++ b/docs/report/honeycomb_functional_tests/csit_release_notes.rst
@@ -4,32 +4,38 @@ CSIT Release Notes
 Changes in CSIT |release|

 -------------------------

 

-#. Naming change for all Honeycomb functional test suites in VIRL

-

-    - Honeycomb functional test case names stayed unchanged

-

 #. Added Honeycomb functional tests

 

-    - NSH_SFC

-    - LISP

-    - NAT

-    - SPAN

+    - ACL plugin

+    - Routing

+    - SLAAC

+    - Proxy ARP

+    - DHCP Relay

+    - Neighbor Discovery Proxy

+

+#. Changed execution environment from Ubuntu14.04 to Ubuntu16.04

 

 Known Issues

 ------------

 

 Here is the list of known issues in CSIT |release| for Honeycomb functional

 tests in VIRL:

++---+--------------------------------------------+------------+----------------------------------------------------------------------------+

+| # | Issue                                      | Jira ID    | Description                                                                |

++---+--------------------------------------------+------------+----------------------------------------------------------------------------+

+| 1 | IP address subnet validation               | VPP-649    | When configuring two IP addresses from the same subnet on an interface,    |

+|   |                                            |            | VPP refuses the configuration but returns OK. This can cause desync        |

+|   |                                            |            | between Honeycomb's config and operational data.                           |

++---+--------------------------------------------+------------+----------------------------------------------------------------------------+

+| 2 | Persistence of VxLAN tunnel naming context | HC2VPP-47  | When VPP restarts with Honeycomb running and a VxLan interface configured, |

+|   |                                            |            | the interface is sometimes renamed to "vxlan_tunnel0".                     |

+|   |                                            |            | It is otherwise configured correctly.                                      |

++---+--------------------------------------------+------------+----------------------------------------------------------------------------+

+| 3 | Classifier plugin for IPv6 cases           | VPP-687    | Classifier ignores IPv6 packets with less than 8 bytes after last header.  |

+|   |                                            |            | Fixed in VPP 17.07.                                                        |

++---+--------------------------------------------+------------+----------------------------------------------------------------------------+

+| 4 | Batch disable Lisp features                | HC2VPP-131 | When removing complex Lisp configurations in a single request,             |

+|   |                                            |            | the operation fails due to a write ordering issue.                         |

++---+--------------------------------------------+------------+----------------------------------------------------------------------------+

 

-+---+-------------------------------------------------+-----------------------------------------------------------------+

-| # | Issue                                           | Description                                                     |

-+---+-------------------------------------------------+-----------------------------------------------------------------+

-| 1 | Operational data for Vhost-user interfaces      | Honeycomb Operational data reports Vhost-user interfaces        |

-|   | "server" flag                                   | as client, even if they are server.                             |

-+---+-------------------------------------------------+-----------------------------------------------------------------+

-| 2 | Persistence of VxLAN tunnels                    | Configuration persistence often fails to restore                |

-|   |                                                 | Honeycomb's internal naming context for VxLAN interfaces.       |

-|   |                                                 | The interface is renamed to "vxlan_tunnel0" but is otherwise    |

-|   |                                                 | configured correctly.                                           |

-+---+-------------------------------------------------+-----------------------------------------------------------------+

 

diff --git a/docs/report/honeycomb_functional_tests/overview.rst b/docs/report/honeycomb_functional_tests/overview.rst
index d93de132a9..c643f3ea78 100644
--- a/docs/report/honeycomb_functional_tests/overview.rst
+++ b/docs/report/honeycomb_functional_tests/overview.rst
@@ -117,6 +117,33 @@ with results listed in this report:
   - up to 2 interfaces.

   - Remove SPAN configuration from interfaces.

   - Test case count: 3

+- **ACL-PLUGIN** - CRD for high-level classifier

+  - MAC + IP address classification.

+  - IPv4, IPv6 address classification.

+  - TCP, UDP, ICMP, ICMPv6 protocol/next-header classification.

+  - port number classification.

+  - ICMP, ICMPv6 code and type classification.

+  - Test case count: 15

+- **ProxyARP** - CRD for proxyARP feature.

+  - Configure proxyARP.

+  - Assign to interface.

+  - Test case count: 3

+- **ProxyND6** - CRD for Neighbor Discovery Proxy.

+  - Configure ProxyND6 feature on interface.

+  - Test case count: 4

+- **DHCP Relay** - CRD for DHCP relay feature.

+  - Configure DHCP Relays.

+  - IPv4 and IPv6 variants.

+  - Test case count: 4

+- **SLAAC** - CRD for Stateless Address AutoConfiguration.

+  - Configure SLAAC feature on interfaces.

+  - Test case count: 7

+- **Routing** - CRD for routing.

+  - Configure single-hop route.

+  - Configure multi-hop routes.

+  - Configure blackhole route.

+  - IPv4 and IPv6 variants.

+    - Test case count: 6

 - **Honeycomb Infractructure** - configuration persistence,

   - Netconf notifications for interface events,

   - Netconf negative tests aimed at specific issues

@@ -130,7 +157,8 @@ Because of this, test cases follow this general pattern:
 #. read status of the feature using VPP API dump.

 #. modify configuration of the feature using restconf.

 #. verify changes to operational data using restconf.

-#. verify changes using VPP API dump.

+#. verify changes using VPP API dump, OR

+#. send a packet to VPP node and observe behaviour to verify configuration

 

 Test cases involving network interfaces utilize the first two interfaces on

 the DUT node.

diff --git a/docs/report/honeycomb_functional_tests/test_environment.rst b/docs/report/honeycomb_functional_tests/test_environment.rst
index e706aa5573..028911fd94 100644
--- a/docs/report/honeycomb_functional_tests/test_environment.rst
+++ b/docs/report/honeycomb_functional_tests/test_environment.rst
@@ -13,21 +13,21 @@ versions:
 Current VPP tests have been executed on a single VM operating system and

 version only, as described in the following paragraphs.

 

-In CSIT terminology, the VM operating system for both SUTs and TG that VPP 17.01

+In CSIT terminology, the VM operating system for both SUTs and TG that VPP 17.04

 has been tested with, is the following:

 

-  ubuntu-14.04.4_2016-10-07_1.3

+  ubuntu-16.04.1_2017-02-23_1.8

 

-which implies Ubuntu 14.04.3 LTS, current as of 2016/10/07 (that is, package

-versions are those that would have been installed by a "apt-get update",

-"apt-get upgrade" on October 7), produced by CSIT disk image build scripts

-version 1.6.

+This image implies Ubuntu 16.04.1 LTS, current as of 2017/02/23 (that is,

+package versions are those that would have been installed by a "apt-get update",

+"apt-get upgrade" on February 23), produced by CSIT disk image build scripts

+version 1.8.

 

 The exact list of installed packages and their versions (including the Linux

 kernel package version) are included in CSIT source repository:

 

-  resources/tools/disk-image-builder/ubuntu/lists/ubuntu-14.04.4_2016-10-07_1.3

+  resources/tools/disk-image-builder/ubuntu/lists/ubuntu-16.04.1_2017-02-23_1.8

 

 A replica of this VM image can be built by running the "build.sh" script in CSIT

-repository resources/tools/disk-image-builder/.

+repository resources/tools/disk-image-builder/ubuntu.