aboutsummaryrefslogtreecommitdiffstats
path: root/docs/report/introduction
diff options
context:
space:
mode:
authorMaciek Konstantynowicz <mkonstan@cisco.com>2017-04-20 22:42:01 +0100
committerPeter Mikus <pmikus@cisco.com>2017-04-21 12:27:38 +0000
commit5d294b7d4c501d971d67c4e3fe192a9dc504d76e (patch)
tree62602fa98b43a4a42d8a65fbffeb1d5f7d0b17fa /docs/report/introduction
parent2c1706d69455d8479b0b0318d4c2fa8b4ab806cb (diff)
CSIT rls1704 report - update to handcraftec sections:
- introduction /overview /general_notes /csit_test_naming - vpp_performance_tests /overview /csit_release_notes /test_environment /documentation /xx_graphs - vpp_functional_tests /overview /csit_release_notes /documentation - testpmd_performance_tests /overview /csit_release_notes /test_environment /documentation /xx_graphs Change-Id: I10593d4155edc663ca07ae5cd29eb679c5f5d85f Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>
Diffstat (limited to 'docs/report/introduction')
-rw-r--r--docs/report/introduction/csit_test_naming.rst24
-rw-r--r--docs/report/introduction/general_notes.rst16
-rw-r--r--docs/report/introduction/overview.rst21
3 files changed, 25 insertions, 36 deletions
diff --git a/docs/report/introduction/csit_test_naming.rst b/docs/report/introduction/csit_test_naming.rst
index 0415e5c5d6..a4de765e0a 100644
--- a/docs/report/introduction/csit_test_naming.rst
+++ b/docs/report/introduction/csit_test_naming.rst
@@ -4,21 +4,13 @@ CSIT Test Naming
Background
----------
-CSIT |release| introduced a common structured naming convention for all
-performance and functional tests. This change was driven by substantially
-growing number and type of CSIT test cases. Firstly, the original practice did
-not always follow any strict naming convention. Secondly test names did not
-always clearly capture tested packet encapsulations, and the actual type or
-content of the tests. Thirdly HW configurations in terms of NICs, ports and
-their locality were not captured either. These were but few reasons that drove
-the decision to change and define a new more complete and stricter test naming
-convention. And to apply it to all existing and new test suites and test
-cases.
-
-The new naming should be intuitive for majority of the tests. Complete
-description of the newly adopted CSIT test naming convention is provided on
-`CSIT test naming page <https://wiki.fd.io/view/CSIT/csit-test-naming>`_. Here
-few illustrative examples of the new naming usage for test suites across CSIT
+CSIT |release| follows a common structured naming convention for all
+performance and system functional tests, introduced in CSIT rls1701.
+
+The naming should be intuitive for majority of the tests. Complete
+description of CSIT test naming convention is provided on
+`CSIT test naming wiki page <https://wiki.fd.io/view/CSIT/csit-test-naming>`_. Below
+few illustrative examples of the naming usage for test suites across CSIT
performance, functional and HoneyComb management test areas.
Naming Convention
@@ -123,4 +115,4 @@ topologies:
sub-interface events with HoneyComb NetConf Notifications, functional tests.
For complete description of CSIT test naming convention please refer to `CSIT
-test naming page <https://wiki.fd.io/view/CSIT/csit-test-naming>`_.
+test naming wiki page <https://wiki.fd.io/view/CSIT/csit-test-naming>`_.
diff --git a/docs/report/introduction/general_notes.rst b/docs/report/introduction/general_notes.rst
index c05401e4cc..ebb73cdc93 100644
--- a/docs/report/introduction/general_notes.rst
+++ b/docs/report/introduction/general_notes.rst
@@ -3,7 +3,7 @@ General Notes
All CSIT test results listed in this report are sourced and auto-generated
from output.xml Robot Framework (RF) files resulting from LF FD.io Jenkins
-jobs execution against VPP-17.01 release artifacts. References are provided
+jobs execution against VPP-17.04 release artifacts. References are provided
to the original LF FD.io Jenkins job results. However, as LF FD.io Jenkins
infrastructure does not automatically archive all jobs (history record is
provided for the last 30 days or 40 jobs only), additional references are
@@ -12,10 +12,10 @@ storage system.
FD.io CSIT project currently covers multiple FD.io system and sub-system
testing areas and this is reflected in this report, where each testing area
-is listed separately, as follows
+is listed separately, as follows:
-#. **VPP Performance Tests** - VPP performance are tests executed in physical
- FD.io testbeds, focusing on VPP network data plane performance at this stage,
+#. **VPP Performance Tests** - VPP performance tests are executed in physical
+ FD.io testbeds, focusing on VPP network data plane performance at this stage,
both for Phy-to-Phy (NIC-to-NIC) and Phy-to-VM-to-Phy (NIC-to-VM-to-NIC)
forwarding topologies. Tested across a range of NICs, 10GE and 40GE
interfaces, range of multi-thread and multi-core configurations. VPP
@@ -40,11 +40,9 @@ is listed separately, as follows
functionality of VPP. Tests cover a range of CRUD operations executed
against VPP.
-CSIT |release| report does also include VPP unit test results. These tests
-have been developed within the FD.io VPP project, and not in CSIT, but they do
-complement tests done by CSIT. They are provided mainly as a reference, to
-give the reader a more complete view of automated testing executed against
-VPP-17.01 release.
+In addition to above, CSIT |release| report does also include VPP unit test results. VPP unit tests are developed within the FD.io VPP project and as they complement CSIT system functional tests, they are provided mainly as a reference and to
+provide a more complete view of automated testing executed against
+VPP-17.04 release.
FD.io CSIT system is developed using two main coding platforms: Robot
Framework (RF) and Python. CSIT |release| source code for the executed test
diff --git a/docs/report/introduction/overview.rst b/docs/report/introduction/overview.rst
index ee1a91c174..d781993198 100644
--- a/docs/report/introduction/overview.rst
+++ b/docs/report/introduction/overview.rst
@@ -3,14 +3,14 @@ Overview
This is the **F**\ast **D**\ata **I**/**O** Project (**FD.io**) **C**\ontinuous
**S**\ystem **I**\ntegration and **T**\esting (**CSIT**) project report for CSIT
-|release| system testing of VPP-17.01 release.
+|release| system testing of VPP-17.04 release.
The report describes CSIT functional and performance tests and their
continuous execution delivered in CSIT |release|. A high-level overview is
provided for each CSIT test environment running in Linux Foundation (LF) FD.io
Continuous Performance Labs. This is followed by summary of all executed tests
-against the VPP-17.01 release and associated sub-systems (HoneyComb, DPDK),
-CSIT |release| release notes, result highlights and known issues. More
+against the VPP-17.04 release and associated FD.io projects and sub-systems (HoneyComb, DPDK),
+CSIT |release| release notes, result highlights and known issues discovered in CSIT. More
detailed description of each environment, pointers to CSIT test code
documentation and detailed test resuls with links to the source data files are
also provided.
@@ -28,19 +28,18 @@ CSIT |release| report contains following main sections and sub-sections:
specifics, methodology for multi-core, packet throughput and latency, and
KVM VM vhost tests; *CSIT Release Notes* - changes in CSIT |release|, added
tests, performance changes, environment or methodology changes, known CSIT
- issues, tests to be added; *Packet Throughput Graphs* and *Packet Latency
+ issues; *Packet Throughput Graphs* and *Packet Latency
Graphs* - plotted NDR, PDR throughput and latency results from multiple
- test job executions; *Test Environment* - environment description ;
- *Documentation* - source code documentation for VPP performance tests.
+ test job executions; *Test Environment* - environment description;
+ *Documentation* - CSIT source code documentation for VPP performance tests.
#. **Testpmd Performance Tests** - Testpmd performance tests executed in
physical FD.io testbeds; *Overview* - tested topologies, test coverage;
*CSIT Release Notes* - changes in CSIT |release|, any known CSIT issues;
- *Tests to Be Added* - performance tests to be added in the next revision of
- CSIT |release| report; *Packet Throughput Graphs* and *Packet Latency Graphs*
+ *Packet Throughput Graphs* and *Packet Latency Graphs*
- plotted NDR, PDR throughput and latency results from multiple test job
executions; *Test Environment* - environment description; *Documentation* -
- source code documentation for Testpmd performance tests.
+ CSIT source code documentation for Testpmd performance tests.
#. **VPP Functional Tests** - VPP functional tests executed in virtual
FD.io testbeds; *Overview* - tested virtual topologies, test coverage and
@@ -52,8 +51,8 @@ CSIT |release| report contains following main sections and sub-sections:
#. **HoneyComb Functional Tests** - HoneyComb functional tests executed in
virtual FD.io testbeds; *Overview* - tested virtual topologies, test
coverage and naming specifics; *CSIT Release Notes* - changes in CSIT
- |release|, added tests, environment or methodology changes, known CSIT issues,
- tests to be added; *Test Environment* - environment description ;
+ |release|, added tests, environment or methodology changes, known CSIT issues;
+ *Test Environment* - environment description ;
*Documentation* - source code documentation for Honeycomb functional tests.
#. **VPP Unit Tests** - refers to VPP functional unit tests executed as