CSIT rls1704 report - update to handcraftec sections:

- introduction /overview /general_notes /csit_test_naming
- vpp_performance_tests /overview /csit_release_notes /test_environment /documentation /xx_graphs
- vpp_functional_tests /overview /csit_release_notes /documentation
- testpmd_performance_tests /overview /csit_release_notes /test_environment /documentation /xx_graphs

Change-Id: I10593d4155edc663ca07ae5cd29eb679c5f5d85f
Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>

3 files changed, 25 insertions, 36 deletions

diff --git a/docs/report/introduction/csit_test_naming.rst b/docs/report/introduction/csit_test_naming.rst
index 0415e5c5d6..a4de765e0a 100644
--- a/docs/report/introduction/csit_test_naming.rst
+++ b/docs/report/introduction/csit_test_naming.rst
@@ -4,21 +4,13 @@ CSIT Test Naming
 Background

 ----------

 

-CSIT |release| introduced a common structured naming convention for all

-performance and functional tests. This change was driven by substantially

-growing number and type of CSIT test cases. Firstly, the original practice did

-not always follow any strict naming convention. Secondly test names did not

-always clearly capture tested packet encapsulations, and the actual type or

-content of the tests. Thirdly HW configurations in terms of NICs, ports and

-their locality were not captured either. These were but few reasons that drove

-the decision to change and define a new more complete and stricter test naming

-convention. And to apply it to all existing and new test suites and test

-cases.

-

-The new naming should be intuitive for majority of the tests. Complete

-description of the newly adopted CSIT test naming convention is provided on

-`CSIT test naming page <https://wiki.fd.io/view/CSIT/csit-test-naming>`_. Here

-few illustrative examples of the new naming usage for test suites across CSIT

+CSIT |release| follows a common structured naming convention for all

+performance and system functional tests, introduced in CSIT rls1701.

+

+The naming should be intuitive for majority of the tests. Complete

+description of CSIT test naming convention is provided on

+`CSIT test naming wiki page <https://wiki.fd.io/view/CSIT/csit-test-naming>`_. Below

+few illustrative examples of the naming usage for test suites across CSIT

 performance, functional and HoneyComb management test areas.

 

 Naming Convention

@@ -123,4 +115,4 @@ topologies:
     sub-interface events with HoneyComb NetConf Notifications, functional tests.

 

 For complete description of CSIT test naming convention please refer to `CSIT

-test naming page <https://wiki.fd.io/view/CSIT/csit-test-naming>`_.

+test naming wiki page <https://wiki.fd.io/view/CSIT/csit-test-naming>`_.

diff --git a/docs/report/introduction/general_notes.rst b/docs/report/introduction/general_notes.rst
index c05401e4cc..ebb73cdc93 100644
--- a/docs/report/introduction/general_notes.rst
+++ b/docs/report/introduction/general_notes.rst
@@ -3,7 +3,7 @@ General Notes
 

 All CSIT test results listed in this report are sourced and auto-generated

 from output.xml Robot Framework (RF) files resulting from LF FD.io Jenkins

-jobs execution against VPP-17.01 release artifacts. References are provided

+jobs execution against VPP-17.04 release artifacts. References are provided

 to the original LF FD.io Jenkins job results. However, as LF FD.io Jenkins

 infrastructure does not automatically archive all jobs (history record is

 provided for the last 30 days or 40 jobs only), additional references are

@@ -12,10 +12,10 @@ storage system.
 

 FD.io CSIT project currently covers multiple FD.io system and sub-system

 testing areas and this is  reflected in this report, where each testing area

-is listed separately, as follows

+is listed separately, as follows:

 

-#. **VPP Performance Tests** - VPP performance are tests executed in physical

-   FD.io testbeds,  focusing on VPP network data plane performance at this stage,

+#. **VPP Performance Tests** - VPP performance tests are executed in physical

+   FD.io testbeds, focusing on VPP network data plane performance at this stage,

    both for Phy-to-Phy (NIC-to-NIC) and Phy-to-VM-to-Phy (NIC-to-VM-to-NIC)

    forwarding topologies. Tested across a range of NICs, 10GE and 40GE

    interfaces, range of multi-thread and multi-core configurations. VPP

@@ -40,11 +40,9 @@ is listed separately, as follows
    functionality of VPP. Tests cover a range of CRUD operations executed

    against VPP.

 

-CSIT |release| report does also include VPP unit test results. These tests

-have been developed within the FD.io VPP project, and not in CSIT, but they do

-complement tests done by CSIT. They are provided mainly as a reference, to

-give the reader a more complete view of automated testing executed against

-VPP-17.01 release.

+In addition to above, CSIT |release| report does also include VPP unit test results. VPP unit tests are developed within the FD.io VPP project and as they complement CSIT system functional tests, they are provided mainly as a reference and to

+provide a more complete view of automated testing executed against

+VPP-17.04 release.

 

 FD.io CSIT system is developed using two main coding platforms: Robot

 Framework (RF) and Python. CSIT |release| source code for the executed test

diff --git a/docs/report/introduction/overview.rst b/docs/report/introduction/overview.rst
index ee1a91c174..d781993198 100644
--- a/docs/report/introduction/overview.rst
+++ b/docs/report/introduction/overview.rst
@@ -3,14 +3,14 @@ Overview
 

 This is the **F**\ast **D**\ata **I**/**O** Project (**FD.io**) **C**\ontinuous

 **S**\ystem **I**\ntegration and **T**\esting (**CSIT**) project report for CSIT

-|release| system testing of VPP-17.01 release.

+|release| system testing of VPP-17.04 release.

 

 The report describes CSIT functional and performance tests and their

 continuous execution delivered in CSIT |release|. A high-level overview is

 provided for each CSIT test environment running in Linux Foundation (LF) FD.io

 Continuous Performance Labs. This is followed by summary of all executed tests

-against the VPP-17.01 release and associated sub-systems (HoneyComb, DPDK),

-CSIT |release| release notes, result highlights and known issues. More

+against the VPP-17.04 release and associated FD.io projects and sub-systems (HoneyComb, DPDK),

+CSIT |release| release notes, result highlights and known issues discovered in CSIT. More

 detailed description of each environment, pointers to CSIT test code

 documentation and detailed test resuls with links to the source data files are

 also provided.

@@ -28,19 +28,18 @@ CSIT |release| report contains following main sections and sub-sections:
    specifics, methodology for multi-core, packet throughput and latency, and

    KVM VM vhost tests; *CSIT Release Notes* - changes in CSIT |release|, added

    tests, performance changes, environment or methodology changes, known CSIT

-   issues, tests to be added; *Packet Throughput Graphs* and *Packet Latency

+   issues; *Packet Throughput Graphs* and *Packet Latency

    Graphs* - plotted NDR, PDR throughput and latency results from multiple

-   test job executions; *Test Environment* - environment description ;

-   *Documentation* - source code documentation for VPP performance tests.

+   test job executions; *Test Environment* - environment description;

+   *Documentation* - CSIT source code documentation for VPP performance tests.

 

 #. **Testpmd Performance Tests** - Testpmd performance tests executed in

    physical FD.io testbeds; *Overview* - tested topologies, test coverage;

    *CSIT Release Notes* - changes in CSIT |release|, any known CSIT issues;

-   *Tests to Be Added* - performance tests to be added in the next revision of

-   CSIT |release| report; *Packet Throughput Graphs* and *Packet Latency Graphs*

+   *Packet Throughput Graphs* and *Packet Latency Graphs*

    - plotted NDR, PDR throughput and latency results from multiple test job

    executions; *Test Environment* - environment description; *Documentation* -

-   source code documentation for Testpmd performance tests.

+   CSIT source code documentation for Testpmd performance tests.

 

 #. **VPP Functional Tests** - VPP functional tests executed in virtual

    FD.io testbeds; *Overview* - tested virtual topologies, test coverage and

@@ -52,8 +51,8 @@ CSIT |release| report contains following main sections and sub-sections:
 #. **HoneyComb Functional Tests** - HoneyComb functional tests executed in

    virtual FD.io testbeds; *Overview* - tested virtual topologies, test

    coverage and naming specifics; *CSIT Release Notes* - changes in CSIT

-   |release|, added tests, environment or methodology changes, known CSIT issues,

-   tests to be added; *Test Environment* - environment description ;

+   |release|, added tests, environment or methodology changes, known CSIT issues;

+   *Test Environment* - environment description ;

    *Documentation* - source code documentation for Honeycomb functional tests.

 

 #. **VPP Unit Tests** - refers to VPP functional unit tests executed as