CSIT-702 Update static content after optimizations

Update the mechanics of script for generating report to reflect the latest changes. Update some static content.

Change-Id: If60f7b74245138cc3044a49c46ff06d5850732de
Signed-off-by: Peter Mikus <pmikus@cisco.com>

3 files changed, 20 insertions, 424 deletions

diff --git a/docs/report/introduction/csit_tag_description.rst b/docs/report/introduction/csit_tag_description.rst
index 5ea6c7fc04..53f1eb357a 100644
--- a/docs/report/introduction/csit_tag_description.rst
+++ b/docs/report/introduction/csit_tag_description.rst
@@ -7,414 +7,5 @@ execution. Following sections list currently used CSIT TAGs and their
 documentation based on the content of
 `tag documentation rst file`_.
 
-Topology TAGs
--------------
-
-.. topic:: 3_NODE_DOUBLE_LINK_TOPO
-
-    3 nodes connected in a circular topology with two links interconnecting
-    the devices.
-
-.. topic:: 3_NODE_SINGLE_LINK_TOPO
-
-    3 nodes connected in a circular topology with at least one link
-    interconnecting devices.
-
-Objective TAGs
---------------
-
-.. topic:: SKIP_PATCH
-
-    Test case(s) marked to not run in case of vpp-csit-verify (i.e. VPP patch)
-    and csit-vpp-verify jobs (i.e. CSIT patch).
-
-
-Environment TAGs
-----------------
-
-.. topic:: HW_ENV
-
-    DUTs and TGs are running on bare metal.
-
-.. topic:: VM_ENV
-
-    DUTs and TGs are running in virtual environment.
-
-.. topic:: VPP_VM_ENV
-
-    DUTs with VPP and capable of running Virtual Machine.
-
-
-NIC model tags
---------------
-
-.. topic:: NIC_Intel-X520-DA2
-
-    Intel X520-DA2 NIC.
-
-.. topic:: NIC_Intel-XL710
-
-    Intel XL710 NIC.
-
-.. topic:: NIC_Intel-X710
-
-    Intel X710 NIC.
-
-.. topic:: NIC_Cisco-VIC-1227
-
-    VIC-1227 by Cisco.
-
-.. topic:: NIC_Cisco-VIC-1385
-
-    VIC-1385 by Cisco.
-
-
-Scaling TAGs
-------------
-
-.. topic:: FIB_20K
-
-    2x10,000 entries in single fib table
-
-.. topic:: FIB_200K
-
-    2x100,000 entries in single fib table
-
-.. topic:: FIB_2M
-
-    2x1,000,000 entries in single fib table
-
-
-Tags marking functional vs. performance of tests
-------------------------------------------------
-
-.. topic:: FUNCTEST
-
-    All functional test cases.
-
-.. topic:: PERFTEST
-
-    All performance test cases.
-
-
-Performance testing tags
-------------------------
-
-.. topic:: PDRDISC
-
-    Partial Drop Rate evaluation of single run result, with non-zero packet
-    loss tolerance (LT) expressed in percentage of packets transmitted.
-
-.. topic:: NDRDISC
-
-    Non Drop Rate evaluation of results. Loss acceptance of dropped packets is
-    set to zero lost packets.
-
-.. topic:: NDRCHK
-
-    Performance tests where TG verifies DUTs' throughput at ref-NDR (reference
-    Non Drop Rate) with zero packet loss tolerance.
-
-.. topic:: NDRPDRDISC
-
-    Find performance of DUT based on RFC2544 with linear / binary / combined
-    search. (Previous LONG tests.)
-
-Ethernet frame size tags for performance tests
-----------------------------------------------
-
-.. topic:: 64B
-
-    64B frames used for test.
-
-.. topic:: 78B
-
-    78B frames used for test.
-
-.. topic:: IMIX
-
-    IMIX frame sequence (28x 64B, 16x 570B, 4x 1518B) used for test.
-
-.. topic:: 1460B
-
-    1460B frames used for test.
-
-.. topic:: 1480B
-
-    1480B frames used for test.
-
-.. topic:: 1514B
-
-    1514B frames used for test.
-
-.. topic:: 1518B
-
-    1518B frames used for test.
-
-.. topic:: 9000B
-
-    9000B frames used for test.
-
-Test type tags
---------------
-
-.. topic:: BASE
-
-    Baseline test cases, no encapsulation, no feature(s) configured in tests.
-
-.. topic:: IP4BASE
-
-    IPv4 baseline test cases, no encapsulation, no feature(s) configured in
-    tests.
-
-.. topic:: IP6BASE
-
-    IPv6 baseline test cases, no encapsulation, no feature(s) configured in
-    tests.
-
-.. topic:: L2XCBASE
-
-    L2XC baseline test cases, no encapsulation, no feature(s) configured in
-    tests.
-
-.. topic:: L2BDBASE
-
-    L2BD baseline test cases, no encapsulation, no feature(s) configured in
-    tests.
-
-.. topic:: SCALE
-
-    Scale test cases.
-
-.. topic:: ENCAP
-
-    Test cases where encapsulation is used. Use also encapsulation tag(s).
-
-.. topic:: FEATURE
-
-    At least one feature is configured in test cases. Use also feature tag(s).
-
-
-Forwarding mode tags
---------------------
-
-.. topic:: L2BDMACSTAT
-
-    VPP L2 bridge-domain, L2 MAC static.
-
-.. topic:: L2BDMACLRN
-
-    VPP L2 bridge-domain, L2 MAC learning.
-
-.. topic:: L2XCFWD
-
-    VPP L2 point-to-point cross-connect.
-
-.. topic:: IP4FWD
-
-    VPP IPv4 routed forwarding.
-
-.. topic:: IP6FWD
-
-    VPP IPv6 routed forwarding.
-
-
-Underlay tags
--------------
-
-.. topic:: IP4UNRLAY
-
-    IPv4 underlay.
-
-.. topic:: IP6UNRLAY
-
-    IPv6 underlay.
-
-.. topic:: MPLSUNRLAY
-
-    MPLS underlay.
-
-
-Overlay tags
-------------
-
-.. topic:: L2OVRLAY
-
-    L2 overlay.
-
-.. topic:: IP4OVRLAY
-
-    IPv4 overlay (IPv4 payload).
-
-.. topic:: IP6OVRLAY
-
-    IPv6 overlay (IPv6 payload).
-
-
-Encapsulation tags
-------------------
-
-.. topic:: ETH
-
-    All test cases with base Ethernet (no encapsulation).
-
-.. topic:: DOT1Q
-
-    All test cases with dot1q.
-
-.. topic:: DOT1AD
-
-    All test cases with dot1ad.
-
-.. topic:: LISP
-
-    All test cases with LISP.
-
-.. topic:: LISPGPE
-
-    All test cases with LISP-GPE.
-
-.. topic:: VXLAN
-
-    All test cases with Vxlan.
-
-.. topic:: VXLANGPE
-
-    All test cases with VXLAN-GPE.
-
-.. topic:: GRE
-
-    All test cases with GRE.
-
-.. topic:: IPSEC
-
-    All test cases with IPSEC.
-
-
-Interface tags
---------------
-
-.. topic:: PHY
-
-    All test cases which use physical interface(s).
-
-.. topic:: VHOST
-
-    All test cases which uses VHOST.
-
-.. topic:: TUNTAP
-
-    All test cases which uses TUN and TAP.
-
-.. topic:: AFPKT
-
-    All test cases which uses AFPKT.
-
-.. topic:: NETMAP
-
-    All test cases which uses Netmap.
-
-
-Feature tags
-------------
-
-.. topic:: IACLDST
-
-    iACL destination.
-
-.. topic:: COPWHLIST
-
-    COP whitelist.
-
-.. topic:: SNAT
-
-    SNAT configured and tested.
-
-
-Encryption tags
----------------
-
-.. topic:: IPSECSW
-
-    Crypto in software.
-
-.. topic:: IPSECHW
-
-    Crypto in hardware.
-
-.. topic:: IPSECTRAN
-
-    IPSec in transport mode.
-
-.. topic:: IPSECTUN
-
-    IPSec in tunnel mode.
-
-
-Client-workload tags
---------------------
-
-.. topic:: VM
-
-    All test cases which use at least one virtual machine.
-
-.. topic:: LXC
-
-    All test cases which use Linux container.
-
-.. topic:: APP
-
-    All test cases with specific APP use.
-
-
-Multi-threading tags
---------------------
-
-.. topic:: STHREAD
-
-    All test cases using single poll mode thread.
-
-.. topic:: MTHREAD
-
-    All test cases using more then one poll mode driver thread.
-
-.. topic:: 1NUMA
-
-    All test cases with packet processing on single socket.
-
-.. topic:: 2NUMA
-
-    All test cases with packet processing on two sockets.
-
-.. topic:: SMT
-
-    All test cases with symmetric Multi-Threading (HyperThreading) enabled.
-
-.. topic:: NOSMT
-
-    All test cases with symmetric Multi-Threading (HyperThreading) disabled.
-
-.. topic:: 1T1C
-
-    1 worker thread pinned to 1 dedicated physical core. 1 receive queue per
-    interface. Main thread pinned to core 0.
-
-.. topic:: 2T2C
-
-    2 worker threads pinned to 2 dedicated physical cores. 1 receive queue per
-    interface. Main thread pinned to core 0.
-
-.. topic:: 4T4C
-
-    4 worker threads pinned to 4 dedicated physical cores. 2 receive queues per
-    interface. Main thread pinned to core 0.
-
-.. topic:: 6T6C
-
-    6 worker threads pinned to 6 dedicated physical cores. 3 receive queues per
-    interface. Main thread pinned to core 0.
-
-.. topic:: 8T8C
-
-    8 worker threads pinned to 8 dedicated physical cores. 4 receive queues per
-    interface. Main thread pinned to core 0.
+.. include:: ../../tag_documentation.rst
+   :start-line: 8
diff --git a/docs/report/introduction/general_notes.rst b/docs/report/introduction/general_notes.rst
index 5015b6bb16..d96dc00101 100644
--- a/docs/report/introduction/general_notes.rst
+++ b/docs/report/introduction/general_notes.rst
@@ -3,10 +3,10 @@ General Notes
 

 All CSIT test results listed in this report are sourced and auto-generated

 from output.xml Robot Framework (RF) files resulting from LF FD.io Jenkins

-jobs execution against VPP-17.04 release artifacts. References are provided

-to the original LF FD.io Jenkins job results. However, as LF FD.io Jenkins

-infrastructure does not automatically archive all jobs (history record is

-provided for the last 30 days or 40 jobs only), additional references are

+jobs execution against |vpp-release| release artifacts. References are

+provided to the original LF FD.io Jenkins job results. However, as LF FD.io

+Jenkins infrastructure does not automatically archive all jobs (history record

+is provided for the last 30 days or 40 jobs only), additional references are

 provided to the RF result files that got archived in FD.io nexus online

 storage system.

 

@@ -21,13 +21,14 @@ is listed separately, as follows:
    interfaces, range of multi-thread and multi-core configurations. VPP

    application runs in host user- mode. TRex is used as a traffic generator.

 

-#. **Testpmd Performance Tests** - VPP is using DPDK code to control and drive

+#. **DPDK Performance Tests** - VPP is using DPDK code to control and drive

    the NICs and physical interfaces. Testpmd tests are used as a baseline to

-   profile the DPDK sub-system of VPP. Testpmd performance tests executed in

-   physical FD.io testbeds, focusing on Testpmd data plane performance for Phy-

-   to-Phy (NIC-to-NIC). Tests cover a range of NICs, 10GE and 40GE interfaces,

-   range of multi-thread and multi-core configurations. Testpmd application runs

-   in host user-mode. TRex is used as a traffic generator.

+   profile the DPDK sub-system of VPP. DPDK performance tests executed in

+   physical FD.io testbeds, focusing on Testpmd/L3FWD data plane performance for

+   Phy-to-Phy (NIC-to-NIC). Tests cover a range of NICs, 10GE and 40GE

+   interfaces, range of multi-thread and multi-core configurations.

+   Testpmd/L3FWD application runs in host user-mode. TRex is used as a traffic

+   generator.

 

 #. **VPP Functional Tests** - VPP functional tests are executed in virtual

    FD.io testbeds focusing on VPP packet processing functionality, including

@@ -40,6 +41,10 @@ is listed separately, as follows:
    functionality of VPP. Tests cover a range of CRUD operations executed

    against VPP.

 

+#. **TLDK Tests** - TODO

+

+#. **NSH_SFC Tests** - TODO

+

 In addition to above, CSIT |release| report does also include VPP unit test

 results. VPP unit tests are developed within the FD.io VPP project and as they

 complement CSIT system functional tests, they are provided mainly as a reference

diff --git a/docs/report/introduction/overview.rst b/docs/report/introduction/overview.rst
index 30f62a90e2..1356ca6795 100644
--- a/docs/report/introduction/overview.rst
+++ b/docs/report/introduction/overview.rst
@@ -33,13 +33,13 @@ CSIT |release| report contains following main sections and sub-sections:
    test job executions; *Test Environment* - environment description;

    *Documentation* - CSIT source code documentation for VPP performance tests.

 

-#. **Testpmd Performance Tests** - Testpmd performance tests executed in

+#. **DPDK Performance Tests** - DPDK performance tests executed in

    physical FD.io testbeds; *Overview* - tested topologies, test coverage;

    *CSIT Release Notes* - changes in CSIT |release|, any known CSIT issues;

    *Packet Throughput Graphs* and *Packet Latency Graphs*

    - plotted NDR, PDR throughput and latency results from multiple test job

    executions; *Test Environment* - environment description; *Documentation* -

-   CSIT source code documentation for Testpmd performance tests.

+   CSIT source code documentation for DPDK performance tests.

 

 #. **VPP Functional Tests** - VPP functional tests executed in virtual

    FD.io testbeds; *Overview* - tested virtual topologies, test coverage and

@@ -63,7 +63,7 @@ CSIT |release| report contains following main sections and sub-sections:
 

 #. **Detailed Test Results** - auto-generated results from CSIT jobs

    executions using CSIT Robot Framework output files as source data; *VPP

-   Performance Results*, *Testpmd Performance Results*, *VPP Functional

+   Performance Results*, *DPDK Performance Results*, *VPP Functional

    Results*, *HoneyComb Functional Results*, *VPPtest Functional Results*.

 

 #. **Test Configuration** - auto-generated DUT configuration data from CSIT jobs