aboutsummaryrefslogtreecommitdiffstats
path: root/docs/report
diff options
context:
space:
mode:
authorTibor Frank <tifrank@cisco.com>2018-08-07 13:05:06 +0200
committerTibor Frank <tifrank@cisco.com>2018-08-09 14:42:16 +0200
commit468da48d3f3ced298ff8dccc76885157dff5cfe7 (patch)
tree0623e90f98dd7ed7c6eed69fcc62f0a5217ea730 /docs/report
parent1ba065be541936a73ba9a9113165583a8bdf36c3 (diff)
CSIT-1229: Add DMM to report
Change-Id: Ib3ea8c13371a29ea2b4661a8506190dcb15c8ced Signed-off-by: Tibor Frank <tifrank@cisco.com> Signed-off-by: sharath <sharathkumarboyanapally@gmail.com>
Diffstat (limited to 'docs/report')
-rw-r--r--docs/report/detailed_test_results/dmm_functional_results/index.rst12
-rw-r--r--docs/report/dmm_functional_tests/csit_release_notes.rst22
-rw-r--r--docs/report/dmm_functional_tests/documentation.rst6
-rw-r--r--docs/report/dmm_functional_tests/index.rst9
-rw-r--r--docs/report/dmm_functional_tests/overview.rst66
-rw-r--r--docs/report/dmm_functional_tests/test_environment.rst1
-rw-r--r--docs/report/index.html.template10
-rw-r--r--docs/report/index.pdf.template1
-rw-r--r--docs/report/introduction/overview.rst7
9 files changed, 134 insertions, 0 deletions
diff --git a/docs/report/detailed_test_results/dmm_functional_results/index.rst b/docs/report/detailed_test_results/dmm_functional_results/index.rst
new file mode 100644
index 0000000000..2acd888914
--- /dev/null
+++ b/docs/report/detailed_test_results/dmm_functional_results/index.rst
@@ -0,0 +1,12 @@
+DMM Functional
+==============
+
+.. note::
+
+ Data sources for reported test results: i) `FD.io test executor DMM
+ functional jobs`_, ii) archived FD.io jobs test result `output files
+ <../../_static/archive/>`_.
+
+.. toctree::
+
+ dmm_functional_results
diff --git a/docs/report/dmm_functional_tests/csit_release_notes.rst b/docs/report/dmm_functional_tests/csit_release_notes.rst
new file mode 100644
index 0000000000..04b41f3c60
--- /dev/null
+++ b/docs/report/dmm_functional_tests/csit_release_notes.rst
@@ -0,0 +1,22 @@
+Release Notes
+=============
+
+Changes in |csit-release|
+-------------------------
+
+#. Added DMM functional tests
+
+ - DMM base vs epoll test case
+
+Known Issues
+------------
+
+Here is the list of known issues in |csit-release| for DMM functional tests in
+VIRL:
+
++---+-------------------------------------------------+----------+------------------------------------------------------+
+| # | Issue | Jira ID | Description |
++---+-------------------------------------------------+----------+------------------------------------------------------+
+| 1 | DMM CentOs job is disabled | DMM-17 | DPDK version and relative kernel version issue, |
+| | | | CentOS job is temporarily disabled. |
++---+-------------------------------------------------+----------+------------------------------------------------------+
diff --git a/docs/report/dmm_functional_tests/documentation.rst b/docs/report/dmm_functional_tests/documentation.rst
new file mode 100644
index 0000000000..56fb357a60
--- /dev/null
+++ b/docs/report/dmm_functional_tests/documentation.rst
@@ -0,0 +1,6 @@
+Documentation
+=============
+
+`CSIT DMM Functional Tests Documentation`_ contains detailed
+functional description and input parameters for each test case.
+
diff --git a/docs/report/dmm_functional_tests/index.rst b/docs/report/dmm_functional_tests/index.rst
new file mode 100644
index 0000000000..defb9ddbc8
--- /dev/null
+++ b/docs/report/dmm_functional_tests/index.rst
@@ -0,0 +1,9 @@
+DMM Functional
+==============
+
+.. toctree::
+
+ overview
+ csit_release_notes
+ test_environment
+ documentation
diff --git a/docs/report/dmm_functional_tests/overview.rst b/docs/report/dmm_functional_tests/overview.rst
new file mode 100644
index 0000000000..d30398a282
--- /dev/null
+++ b/docs/report/dmm_functional_tests/overview.rst
@@ -0,0 +1,66 @@
+Overview
+========
+
+Virtual Topologies
+------------------
+
+CSIT DMM functional tests are executed on virtualized topologies created using
+:abbr:`VIRL (Virtual Internet Routing Lab)` simulation platform contributed by
+Cisco. VIRL runs on physical baremetal servers hosted by LF FD.io project.
+Based on the packet path through server SUTs, one logical topology type
+is used for DMM DUT data plane testing:
+
+#. NIC-to-NIC switching topologies.
+
+NIC-to-NIC Switching
+~~~~~~~~~~~~~~~~~~~~
+
+The simplest logical topology for software data plane application like
+DMM is NIC-to-NIC switching. Tested topologies for the 3-Node
+testbed is shown in the figure below.
+
+.. only:: latex
+
+ .. raw:: latex
+
+ \begin{figure}[H]
+ \centering
+ \graphicspath{{../_tmp/src/vpp_performance_tests/}}
+ \includegraphics[width=0.90\textwidth]{logical-3n-nic2nic}
+ \label{fig:logical-3n-nic2nic}
+ \end{figure}
+
+.. only:: html
+
+ .. figure:: ../vpp_performance_tests/logical-3n-nic2nic.svg
+ :alt: logical-3n-nic2nic
+ :align: center
+
+SUT1 and SUT2 are two VMs (Ubuntu), TG
+is a Traffic Generator (TG, another Ubuntu VM). SUTs run sample server-client
+application using the DMM libs in Linux user-mode as a Device Under Test (DUT)
+within the VM. Currently TG node is not being used in DMM-CSIT. Logical
+connectivity between SUTs is provided using virtual NICs using VMs' virtio
+driver.
+
+Virtual testbeds are created on-demand whenever a verification job is started
+(e.g. triggered by the gerrit patch submission) and destroyed upon completion
+of all functional tests. Each node is a Virtual Machine and each connection
+that is drawn on the diagram is available for use in any test case. During the
+test execution, all nodes are reachable through the Management network connected
+to every node via dedicated virtual NICs and virtual links (not shown above
+for clarity).
+
+DMM Functional Tests Coverage
+-----------------------------
+
+Following DMM functional test areas are covered in the |csit-release| with
+results listed in this report:
+
+- **DMM basic testcase** - DMM has only one test case right now.
+ The testcase demonstrates single server[DUT1] and single client[DUT2] scenario
+ using DMM framework and kernel tcp/ip stack.
+
+ - Test case count: 1
+
+Total 1 DMM functional test in the |csit-release|.
diff --git a/docs/report/dmm_functional_tests/test_environment.rst b/docs/report/dmm_functional_tests/test_environment.rst
new file mode 100644
index 0000000000..3b35fd432f
--- /dev/null
+++ b/docs/report/dmm_functional_tests/test_environment.rst
@@ -0,0 +1 @@
+.. include:: ../vpp_functional_tests/test_environment.rst
diff --git a/docs/report/index.html.template b/docs/report/index.html.template
index b5b55c0dd2..f262ea36e5 100644
--- a/docs/report/index.html.template
+++ b/docs/report/index.html.template
@@ -63,6 +63,15 @@ CSIT 18.07
.. toctree::
:maxdepth: 2
+ :caption: DMM Functional
+
+ dmm_functional_tests/overview
+ dmm_functional_tests/csit_release_notes
+ dmm_functional_tests/test_environment
+ dmm_functional_tests/documentation
+
+.. toctree::
+ :maxdepth: 2
:caption: Detailed Results
detailed_test_results/vpp_performance_results/index
@@ -72,6 +81,7 @@ CSIT 18.07
detailed_test_results/vpp_functional_results/index
detailed_test_results/honeycomb_functional_results/index
detailed_test_results/nshsfc_functional_results/index
+ detailed_test_results/dmm_functional_results/index
.. toctree::
:maxdepth: 2
diff --git a/docs/report/index.pdf.template b/docs/report/index.pdf.template
index 0a0ba48593..5db11eb388 100644
--- a/docs/report/index.pdf.template
+++ b/docs/report/index.pdf.template
@@ -10,4 +10,5 @@ CSIT 18.07
vpp_functional_tests/index
honeycomb_functional_tests/index
nsh_sfc_functional_tests/index
+ dmm_functional_tests/index
csit_framework_documentation/index
diff --git a/docs/report/introduction/overview.rst b/docs/report/introduction/overview.rst
index ea96c85b76..c3537a9f66 100644
--- a/docs/report/introduction/overview.rst
+++ b/docs/report/introduction/overview.rst
@@ -67,6 +67,13 @@ the source data files are also provided.
*Test Environment* - environment description;
*Documentation* - source code documentation for NSH_SFC functional tests.
+#. **DMM Functional** - DMM functional tests executed in
+ virtual FD.io testbeds; *Overview* - tested virtual topologies, test
+ coverage and naming specifics; *CSIT Release Notes* - changes in CSIT
+ |release|, added tests, environment or methodology changes, known CSIT issues;
+ *Test Environment* - environment description;
+ *Documentation* - source code documentation for DMM functional tests.
+
#. **Detailed Results** - auto-generated results from CSIT jobs
executions using CSIT Robot Framework output files as source data; *VPP
Performance Results*, *DPDK Performance Results*, *VPP Functional