Static report content - Honeycomb

Change-Id: I39899747f99684b932bb9c88268391e55b6fc779
Signed-off-by: selias <samelias@cisco.com>

4 files changed, 83 insertions, 71 deletions

diff --git a/docs/report/introduction/csit_design.rst b/docs/report/introduction/csit_design.rst
index 374c9cdace..baba58f904 100644
--- a/docs/report/introduction/csit_design.rst
+++ b/docs/report/introduction/csit_design.rst
@@ -85,7 +85,7 @@ A brief bottom-up description is provided here:
    - Functional tests using VIRL environment:

 

      - VPP;

-     - HoneyComb;

+     - Honeycomb;

      - NSH_SFC;

 

    - Performance tests using physical testbed environment:

diff --git a/docs/report/introduction/csit_test_naming.rst b/docs/report/introduction/csit_test_naming.rst
index 13eab06df5..c88ec493a3 100644
--- a/docs/report/introduction/csit_test_naming.rst
+++ b/docs/report/introduction/csit_test_naming.rst
@@ -11,7 +11,7 @@ The naming should be intuitive for majority of the tests. Complete
 description of CSIT test naming convention is provided on

 `CSIT test naming wiki page <https://wiki.fd.io/view/CSIT/csit-test-naming>`_.

 Below few illustrative examples of the naming usage for test suites across CSIT

-performance, functional and HoneyComb management test areas.

+performance, functional and Honeycomb management test areas.

 

 Naming Convention

 -----------------

@@ -106,13 +106,13 @@ topologies:
    * *mgmt-cfg-lisp-apivat-func* => configuration of LISP with VAT API calls,

      functional tests.

    * *mgmt-cfg-l2bd-apihc-apivat-func* => configuration of L2 Bridge-Domain with

-     HoneyComb API and VAT API calls, functional tests.

+     Honeycomb API and VAT API calls, functional tests.

    * *mgmt-oper-int-apihcnc-func* => reading status and operational data of

-     interface with HoneyComb NetConf API calls, functional tests.

+     interface with Honeycomb NetConf API calls, functional tests.

    * *mgmt-cfg-int-tap-apihcnc-func* => configuration of tap interfaces with

-     HoneyComb NetConf API calls, functional tests.

+     Honeycomb NetConf API calls, functional tests.

    * *mgmt-notif-int-subint-apihcnc-func* => notifications of interface and

-     sub-interface events with HoneyComb NetConf Notifications, functional tests.

+     sub-interface events with Honeycomb NetConf Notifications, functional tests.

 

 For complete description of CSIT test naming convention please refer to `CSIT

 test naming wiki page <https://wiki.fd.io/view/CSIT/csit-test-naming>`_.

diff --git a/docs/report/introduction/general_notes.rst b/docs/report/introduction/general_notes.rst
index 64e6231443..380f109764 100644
--- a/docs/report/introduction/general_notes.rst
+++ b/docs/report/introduction/general_notes.rst
@@ -1,62 +1,67 @@
-General Notes

-=============

-

-All CSIT test results listed in this report are sourced and auto-generated

-from output.xml Robot Framework (RF) files resulting from LF FD.io Jenkins

-jobs execution against |vpp-release| release artifacts. References are

-provided to the original LF FD.io Jenkins job results. However, as LF FD.io

-Jenkins infrastructure does not automatically archive all jobs (history record

-is provided for the last 30 days or 40 jobs only), additional references are

-provided to the RF result files that got archived in FD.io nexus online

-storage system.

-

-FD.io CSIT project currently covers multiple FD.io system and sub-system

-testing areas and this is  reflected in this report, where each testing area

-is listed separately, as follows:

-

-#. **VPP Performance Tests** - VPP performance tests are executed in physical

-   FD.io testbeds, focusing on VPP network data plane performance at this stage,

-   both for Phy-to-Phy (NIC-to-NIC) and Phy-to-VM-to-Phy (NIC-to-VM-to-NIC)

-   forwarding topologies. Tested across a range of NICs, 10GE and 40GE

-   interfaces, range of multi-thread and multi-core configurations. VPP

-   application runs in host user- mode. TRex is used as a traffic generator.

-

-#. **DPDK Performance Tests** - VPP is using DPDK code to control and drive

-   the NICs and physical interfaces. Testpmd tests are used as a baseline to

-   profile the DPDK sub-system of VPP. DPDK performance tests executed in

-   physical FD.io testbeds, focusing on Testpmd/L3FWD data plane performance for

-   Phy-to-Phy (NIC-to-NIC). Tests cover a range of NICs, 10GE and 40GE

-   interfaces, range of multi-thread and multi-core configurations.

-   Testpmd/L3FWD application runs in host user-mode. TRex is used as a traffic

-   generator.

-

-#. **VPP Functional Tests** - VPP functional tests are executed in virtual

-   FD.io testbeds focusing on VPP packet processing functionality, including

-   network data plane and in -line control plane. Tests cover vNIC-to-vNIC

-   vNIC-to-VM-to-vNIC forwarding topologies. Scapy is used as a traffic

-   generator.

-

-#. **HoneyComb Functional Tests** - HoneyComb functional tests are executed in

-   virtual FD.io testbeds focusing on HoneyComb management and programming

-   functionality of VPP. Tests cover a range of CRUD operations executed

-   against VPP.

-

-#. **NSH_SFC Functional Tests** - NSH_SFC functional tests are executed in

-   virtual FD.io testbeds focusing on NSH_SFC of VPP. Tests cover a range of

-   CRUD operations executed against VPP.

-

-In addition to above, CSIT |release| report does also include VPP unit test

-results. VPP unit tests are developed within the FD.io VPP project and as they

-complement CSIT system functional tests, they are provided mainly as a reference

-and to provide a more complete view of automated testing executed against

-|vpp-release|.

-

-FD.io CSIT system is developed using two main coding platforms: Robot

-Framework (RF) and Python. CSIT |release| source code for the executed test

-suites is available in CSIT branch |release| in the directory

-"./tests/<name_of_the_test_suite>". A local copy of CSIT source code can be

-obtained by cloning CSIT git repository - "git clone

-https://gerrit.fd.io/r/csit". The CSIT testing virtual environment can be run

-on a local computer workstation (laptop, server) using Vagrant by following

-the instructions in `CSIT tutorials

-<https://wiki.fd.io/view/CSIT#Tutorials>`_.

+General Notes
+=============
+
+All CSIT test results listed in this report are sourced and auto-generated
+from output.xml Robot Framework (RF) files resulting from LF FD.io Jenkins
+jobs execution against |vpp-release| release artifacts. References are
+provided to the original LF FD.io Jenkins job results. However, as LF FD.io
+Jenkins infrastructure does not automatically archive all jobs (history record
+is provided for the last 30 days or 40 jobs only), additional references are
+provided to the RF result files that got archived in FD.io nexus online
+storage system.
+
+FD.io CSIT project currently covers multiple FD.io system and sub-system
+testing areas and this is  reflected in this report, where each testing area
+is listed separately, as follows:
+
+#. **VPP Performance Tests** - VPP performance tests are executed in physical
+   FD.io testbeds, focusing on VPP network data plane performance at this stage,
+   both for Phy-to-Phy (NIC-to-NIC) and Phy-to-VM-to-Phy (NIC-to-VM-to-NIC)
+   forwarding topologies. Tested across a range of NICs, 10GE and 40GE
+   interfaces, range of multi-thread and multi-core configurations. VPP
+   application runs in host user- mode. TRex is used as a traffic generator.
+
+#. **DPDK Performance Tests** - VPP is using DPDK code to control and drive
+   the NICs and physical interfaces. Testpmd tests are used as a baseline to
+   profile the DPDK sub-system of VPP. DPDK performance tests executed in
+   physical FD.io testbeds, focusing on Testpmd/L3FWD data plane performance for
+   Phy-to-Phy (NIC-to-NIC). Tests cover a range of NICs, 10GE and 40GE
+   interfaces, range of multi-thread and multi-core configurations.
+   Testpmd/L3FWD application runs in host user-mode. TRex is used as a traffic
+   generator.
+
+#. **VPP Functional Tests** - VPP functional tests are executed in virtual
+   FD.io testbeds focusing on VPP packet processing functionality, including
+   network data plane and in -line control plane. Tests cover vNIC-to-vNIC
+   vNIC-to-VM-to-vNIC forwarding topologies. Scapy is used as a traffic
+   generator.
+
+#. **Honeycomb Functional Tests** - Honeycomb functional tests are executed in
+   virtual FD.io testbeds, focusing on Honeycomb management and programming
+   functionality of VPP. Tests cover a range of CRUD operations executed
+   against VPP.
+
+#. **Honeycomb Performance Tests** - Honeycomb performance tests are executed in
+   physical FD.io testbeds, focusing on the performance of Honeycomb management and programming
+   functionality of VPP. Tests cover a range of CRUD operations executed
+   against VPP.
+
+#. **NSH_SFC Functional Tests** - NSH_SFC functional tests are executed in
+   virtual FD.io testbeds focusing on NSH_SFC of VPP. Tests cover a range of
+   CRUD operations executed against VPP.
+
+In addition to above, CSIT |release| report does also include VPP unit test
+results. VPP unit tests are developed within the FD.io VPP project and as they
+complement CSIT system functional tests, they are provided mainly as a reference
+and to provide a more complete view of automated testing executed against
+|vpp-release|.
+
+FD.io CSIT system is developed using two main coding platforms: Robot
+Framework (RF) and Python. CSIT |release| source code for the executed test
+suites is available in CSIT branch |release| in the directory
+"./tests/<name_of_the_test_suite>". A local copy of CSIT source code can be
+obtained by cloning CSIT git repository - "git clone
+https://gerrit.fd.io/r/csit". The CSIT testing virtual environment can be run
+on a local computer workstation (laptop, server) using Vagrant by following
+the instructions in `CSIT tutorials
+<https://wiki.fd.io/view/CSIT#Tutorials>`_.
diff --git a/docs/report/introduction/overview.rst b/docs/report/introduction/overview.rst
index 4aac352d0c..57b4cd4897 100644
--- a/docs/report/introduction/overview.rst
+++ b/docs/report/introduction/overview.rst
@@ -10,7 +10,7 @@ continuous execution delivered in CSIT |release|. A high-level overview is
 provided for each CSIT test environment running in Linux Foundation (LF) FD.io
 Continuous Performance Labs. This is followed by summary of all executed tests
 against the |vpp-release| and associated FD.io projects and sub-systems
-(HoneyComb, DPDK, NSH_SFC), CSIT |release| release notes, result highlights and
+(Honeycomb, DPDK, NSH_SFC), CSIT |release| release notes, result highlights and
 known issues discovered in CSIT. More detailed description of each environment,
 pointers to CSIT test code documentation and detailed test resuls with links to
 the source data files are also provided.
@@ -48,13 +48,20 @@ CSIT |release| report contains following main sections and sub-sections:
    added; *Test Environment* - environment description ; *Documentation* -
    source code documentation for VPP functional tests.
 
-#. **HoneyComb Functional Tests** - HoneyComb functional tests executed in
+#. **Honeycomb Functional Tests** - Honeycomb functional tests executed in
    virtual FD.io testbeds; *Overview* - tested virtual topologies, test
    coverage and naming specifics; *CSIT Release Notes* - changes in CSIT
    |release|, added tests, environment or methodology changes, known CSIT issues;
    *Test Environment* - environment description ;
    *Documentation* - source code documentation for Honeycomb functional tests.
 
+#. **Honeycomb Performance Tests** - Honeycomb performance tests executed in
+   physical FD.io testbeds; *Overview* - tested topologies, test
+   coverage and naming specifics; *CSIT Release Notes* - changes in CSIT
+   |release|, added tests, environment or methodology changes, known CSIT issues;
+   *Test Environment* - environment description ;
+   *Documentation* - source code documentation for Honeycomb performance tests.
+
 #. **VPP Unit Tests** - refers to VPP functional unit tests executed as
    part of vpp make test verify option within the FD.io VPP project; listed in
    this report to give a more complete view about executed VPP functional tests;
@@ -71,7 +78,7 @@ CSIT |release| report contains following main sections and sub-sections:
 #. **Detailed Test Results** - auto-generated results from CSIT jobs
    executions using CSIT Robot Framework output files as source data; *VPP
    Performance Results*, *DPDK Performance Results*, *VPP Functional
-   Results*, *HoneyComb Functional Results*, *VPPtest Functional Results*.
+   Results*, *Honeycomb Functional Results*, *VPPtest Functional Results*.
 
 #. **Test Configuration** - auto-generated DUT configuration data from CSIT jobs
    executions using CSIT Robot Framework output files as source data; *VPP