aboutsummaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
-rw-r--r--docs/report/dmm_functional_tests/csit_release_notes.rst2
-rw-r--r--docs/report/dmm_functional_tests/overview.rst1
-rw-r--r--docs/report/dpdk_performance_tests/csit_release_notes.rst40
-rw-r--r--docs/report/honeycomb_functional_tests/overview.rst93
-rw-r--r--docs/report/nsh_sfc_functional_tests/overview.rst4
-rw-r--r--docs/report/vpp_functional_tests/csit_release_notes.rst14
-rw-r--r--docs/report/vpp_performance_tests/csit_release_notes.rst82
7 files changed, 104 insertions, 132 deletions
diff --git a/docs/report/dmm_functional_tests/csit_release_notes.rst b/docs/report/dmm_functional_tests/csit_release_notes.rst
index 352809c038..1987e218a5 100644
--- a/docs/report/dmm_functional_tests/csit_release_notes.rst
+++ b/docs/report/dmm_functional_tests/csit_release_notes.rst
@@ -6,7 +6,7 @@ Changes in |csit-release|
#. **DMM Functional Tests**
- - Added DMM base vs. epoll test case.
+ - Added DMM base vs. epoll test case.
Known Issues
------------
diff --git a/docs/report/dmm_functional_tests/overview.rst b/docs/report/dmm_functional_tests/overview.rst
index aa9e853817..562fa09eb7 100644
--- a/docs/report/dmm_functional_tests/overview.rst
+++ b/docs/report/dmm_functional_tests/overview.rst
@@ -49,6 +49,5 @@ environment:
| | testcase demonstrates single server[DUT1] |
| | and single client[DUT2] scenario using DMM |
| | framework and kernel tcp/ip stack. |
-| | |
| | Test case count: 1. |
+-----------------------+----------------------------------------------+
diff --git a/docs/report/dpdk_performance_tests/csit_release_notes.rst b/docs/report/dpdk_performance_tests/csit_release_notes.rst
index d84b4e259a..bd89657135 100644
--- a/docs/report/dpdk_performance_tests/csit_release_notes.rst
+++ b/docs/report/dpdk_performance_tests/csit_release_notes.rst
@@ -6,33 +6,33 @@ Changes in |csit-release|
#. **DPDK performance tests**
- - **MRR Throughput**: MRR (Maximum Receive Rate) test code has now
- configurable trial duration and number of consecutive executions.
- Coverage of MRR tests has been extended across more test
- scenarios. MRR tests are used for continuous performance trending
- and for comparison between DPDK releases.
+ - **MRR Throughput**: MRR (Maximum Receive Rate) test code has now
+ configurable trial duration and number of consecutive executions.
+ Coverage of MRR tests has been extended across more test
+ scenarios. MRR tests are used for continuous performance trending
+ and for comparison between DPDK releases.
- - **MLRsearch Throughput**: MLRsearch algorithm has been introduced
- for all NDR and PDR throughput tests. All tests that previously
- used binary search got converted to MLRsearch. Coverage of NDR/PDR
- tests has been extended across more test scenarios.
+ - **MLRsearch Throughput**: MLRsearch algorithm has been introduced
+ for all NDR and PDR throughput tests. All tests that previously
+ used binary search got converted to MLRsearch. Coverage of NDR/PDR
+ tests has been extended across more test scenarios.
- - **2-Node Tests**: A new set of 2-node tests covering DPDK Testpmd
- and L3fwd, running on new Xeon Skylake testbeds.
+ - **2-Node Tests**: A new set of 2-node tests covering DPDK Testpmd
+ and L3fwd, running on new Xeon Skylake testbeds.
- - **More NICs**: Increased coverage of NIC models tested: Intel-
- xxv710-da2, Intel-x710-da4.
+ - **More NICs**: Increased coverage of NIC models tested: Intel-
+ xxv710-da2, Intel-x710-da4.
- - **Generated tests**: Simplified and unified test structure, semi-
- autogenerated by generator script. Test generator is currently
- able to create test combinations with various frame size and
- cores combinations. All existing test cases were converted to new
- format.
+ - **Generated tests**: Simplified and unified test structure, semi-
+ autogenerated by generator script. Test generator is currently
+ able to create test combinations with various frame size and
+ cores combinations. All existing test cases were converted to new
+ format.
#. **Presentation and Analytics Layer**
- - **Performance trending**: Added continuous performance trending with
- anomaly detection and analysis for DPDK.
+ - **Performance trending**: Added continuous performance trending with
+ anomaly detection and analysis for DPDK.
Performance Changes
-------------------
diff --git a/docs/report/honeycomb_functional_tests/overview.rst b/docs/report/honeycomb_functional_tests/overview.rst
index b57d2b4094..29bd671ae3 100644
--- a/docs/report/honeycomb_functional_tests/overview.rst
+++ b/docs/report/honeycomb_functional_tests/overview.rst
@@ -31,7 +31,7 @@ figure below.
SUT (System Under Test) is a VM running Ubuntu Linux (or Centos,
depending on the test suite), TG (Traffic Generator) is another VM
-running Ubuntu Linux. SUT VMs run HoneyComb management agent and VPP in
+running Ubuntu Linux. SUT VMs run HoneyComb management agent and VPP in
Linux user-mode as a combined DUT (Device Under Test). TG runs Scapy
application as a packet Traffic Generator. Virtual connectivity between
SUT and TG is provided using virtual NICs using VMs' virtio drivers.
@@ -45,9 +45,8 @@ virtual VM environment:
+-----------------------+----------------------------------------------+
| Functionality | Description |
+=======================+==============================================+
-| ACL | CRD for low-level classifiers: table and |
-| | session management, interface assignment. |
-| | |
+| ACL | - CRD for low-level classifiers: table and |
+| | session management, interface assignment. |
| | - Configure up to 2 classify tables. |
| | - Configure up to 2 classify sessions on one |
| | table. |
@@ -57,8 +56,7 @@ virtual VM environment:
| | assignments. |
| | - Test case count: 9. |
+-----------------------+----------------------------------------------+
-| ACL-PLUGIN | CRD for high-level classifier. |
-| | |
+| ACL-PLUGIN | - CRD for high-level classifier. |
| | - MAC + IP address classification. |
| | - IPv4, IPv6 address classification. |
| | - TCP, UDP, ICMP, ICMPv6 protocol and |
@@ -67,36 +65,31 @@ virtual VM environment:
| | - ICMP, ICMPv6 code and type classification. |
| | - Test case count: 15. |
+-----------------------+----------------------------------------------+
-| Basic interface | CRUD for interface state. |
-| management | |
-| | - ipv4/ipv6 address, ipv4 neighbor, MTU |
+| Basic interface | - CRUD for interface state. |
+| management | - ipv4/ipv6 address, ipv4 neighbor, MTU |
| | value. |
| | - Test case count: 14. |
+-----------------------+----------------------------------------------+
-| Border Gateway | CRUD and functional tests for BGP. |
-| Protocol | |
-| | - Configure peers and routes |
+| Border Gateway | - CRUD and functional tests for BGP. |
+| Protocol | - Configure peers and routes |
| | - Check interactions with another BGP peer. |
| | - Test case count: 13. |
+-----------------------+----------------------------------------------+
-| DHCP Relay | CRD for DHCP relay feature. |
-| | |
+| DHCP Relay | - CRD for DHCP relay feature. |
| | - Configure DHCP Relays. |
| | - IPv4 and IPv6 variants. |
| | - Test case count: 4. |
+-----------------------+----------------------------------------------+
-| Honeycomb | Configuration persistence. |
-| Infractructure | |
-| | - Netconf notifications for interface |
+| Honeycomb | - Configuration persistence. |
+| Infractructure | - Netconf notifications for interface |
| | events. |
| | - Netconf negative tests aimed at specific |
| | issues. |
| | - Netconf/Restconf northbound over IPv6. |
| | - Test case count: 12. |
+-----------------------+----------------------------------------------+
-| L2BD | CRUD for L2 Bridge-Domain, interface |
-| | assignment. |
-| | |
+| L2BD | - CRUD for L2 Bridge-Domain, interface |
+| | assignment. |
| | - Create up to two bridge domains with all |
| | implemented functions turned on: |
| | flooding, unknown-unicast flooding, |
@@ -107,17 +100,15 @@ virtual VM environment:
| | bridge domains. |
| | - Test case count: 5. |
+-----------------------+----------------------------------------------+
-| L2FIB | CRD for L2-FIB entries. |
-| | |
+| L2FIB | - CRD for L2-FIB entries. |
| | - Create 4 FIB entries: |
| | one of each for filter/forward, |
| | static/dynamic combinations. |
| | - Remove FIB entries. |
| | - Test case count: 7. |
+-----------------------+----------------------------------------------+
-| LISP | CRD for Lisp: mapping, locator set, |
-| | adjacency, mapresolver. |
-| | |
+| LISP | - CRD for Lisp: mapping, locator set, |
+| | adjacency, mapresolver. |
| | - Toggle Lisp feature status. |
| | - Configure and delete Lisp mapping as local |
| | and remote. |
@@ -127,83 +118,72 @@ virtual VM environment:
| | proxy ITR. |
| | - Test case count: 18. |
+-----------------------+----------------------------------------------+
-| LISP GPE | CRUD for LISP GPE mappings. |
-| | |
+| LISP GPE | - CRUD for LISP GPE mappings. |
| | - Toggle Lisp GPE feature status. |
| | - Configure Lisp GPE mappings. |
| | - Traffic test verifying encapsulation. |
| | - Test case count: 12. |
+-----------------------+----------------------------------------------+
-| NAT | CRD for NAT entries, interface assignment. |
-| | |
+| NAT | - CRD for NAT entries, interface assignment. |
| | - Configure and delete up to two NAT |
| | entries. |
| | - Assign NAT entries to a physical |
| | interface. |
| | - Test case count: 6. |
+-----------------------+----------------------------------------------+
-| NSH_SFC | CRD for NSH maps and entries, using NSH_SFC |
-| | plugin. |
-| | |
+| NSH_SFC | - CRD for NSH maps and entries, using |
+| | NSH_SFC plugin. |
| | - Configure up to 2 NSH entries. |
| | - Configure up to 2 NSH maps. |
| | - Modify and delete NSH maps and entries. |
| | - Test case count: 8. |
+-----------------------+----------------------------------------------+
-| PBB | CRD for provider backbone bridge |
-| | sub-interface. |
-| | |
+| PBB | - CRD for provider backbone bridge |
+| | sub-interface. |
| | - Configure, modify and remove a PBB |
| | sub-interface over a physical interface. |
| | - Test case count: 8. |
+-----------------------+----------------------------------------------+
-| Policer | CRD for traffic policing feature. |
-| | |
+| Policer | - CRD for traffic policing feature. |
| | - Configure Policing rules. |
| | - Assign to interface. |
| | - Test case count: 6. |
+-----------------------+----------------------------------------------+
-| Port mirroring | CRD for SPAN port mirroring, interface |
-| | assignment. |
-| | |
+| Port mirroring | - CRD for SPAN port mirroring, interface |
+| | assignment. |
| | - Configure SPAN port mirroring on a |
| | physical interface, mirroring. |
| | - up to 2 interfaces. |
| | - Remove SPAN configuration from interfaces. |
| | - Test case count: 14. |
+-----------------------+----------------------------------------------+
-| ProxyARP | CRD for proxyARP feature. |
-| | |
+| ProxyARP | - CRD for proxyARP feature. |
| | - Configure proxyARP. |
| | - Assign to interface. |
| | - Test case count: 3. |
+-----------------------+----------------------------------------------+
-| ProxyND6 | CRD for Neighbor Discovery Proxy. |
-| | |
+| ProxyND6 | - CRD for Neighbor Discovery Proxy. |
| | - Configure ProxyND6 feature on interface. |
| | - Test case count: 4. |
+-----------------------+----------------------------------------------+
-| Routing | CRD for routing. |
-| | |
+| Routing | - CRD for routing. |
| | - Configure single-hop route. |
| | - Configure multi-hop routes. |
| | - Configure blackhole route. |
| | - IPv4 and IPv6 variants. |
| | - Test case count: 6. |
+-----------------------+----------------------------------------------+
-| SLAAC | CRD for Stateless Address AutoConfiguration. |
-| | |
+| SLAAC | - CRD for Stateless Address |
+| | AutoConfiguration. |
| | - Configure SLAAC feature on interfaces. |
| | - Test case count: 7. |
+-----------------------+----------------------------------------------+
-| Vhost-user | CRUD for Vhost-user interfaces. |
-| | |
+| Vhost-user | - CRUD for Vhost-user interfaces. |
| | - Create, modify and delete Vhost-user |
| | interface, as client and server. |
| | - Test case count: 8. |
+-----------------------+----------------------------------------------+
-| VLAN | CRUD for VLAN sub-interface management. |
-| | |
+| VLAN | - CRUD for VLAN sub-interface management. |
| | - Create VLAN sub-interface over a physical |
| | interface. |
| | - Toggle interface state separately for |
@@ -214,22 +194,19 @@ virtual VM environment:
| | sub-interface. |
| | - Test case count: 24. |
+-----------------------+----------------------------------------------+
-| VxLAN | CRD for VxLAN tunnels. |
-| | |
+| VxLAN | - CRD for VxLAN tunnels. |
| | - Create VxLAN interface. |
| | - Disable VxLAN interface. |
| | - Re-create a disabled VxLAN interface. |
| | - Test case count: 6. |
+-----------------------+----------------------------------------------+
-| VxLAN-GPE | CRD for VxLAN GPE tunnels. |
-| | |
+| VxLAN-GPE | - CRD for VxLAN GPE tunnels. |
| | - Create VxLAN GPE interface. |
| | - Disable VxLAN interface. |
| | - Re-create a disabled VxLAN interface. |
| | - Test case count: 7. |
+-----------------------+----------------------------------------------+
-| TAP | CRUD for Tap interface management. |
-| | |
+| TAP | - CRUD for Tap interface management. |
| | - Create, modify and delete TAP interface. |
| | - Test case count: 3. |
+-----------------------+----------------------------------------------+
diff --git a/docs/report/nsh_sfc_functional_tests/overview.rst b/docs/report/nsh_sfc_functional_tests/overview.rst
index 6d36f377ce..39b9dc6eac 100644
--- a/docs/report/nsh_sfc_functional_tests/overview.rst
+++ b/docs/report/nsh_sfc_functional_tests/overview.rst
@@ -50,7 +50,6 @@ virtual VM environment:
| | these packets from one NIC and loopback the |
| | VXLAN-GPE-NSH encapsulated packets to the TG |
| | from other NIC. |
-| | |
| | Test case count: 7. |
+-----------------------+----------------------------------------------+
| NSH SFC Proxy Inbound | TG sends some VXLAN-GPE-NSH encapsulated |
@@ -59,7 +58,6 @@ virtual VM environment:
| | from one NIC and loopback the VXLAN |
| | encapsulated packets to the TG from other |
| | NIC. |
-| | |
| | Test case count: 6. |
+-----------------------+----------------------------------------------+
| NSH SFC Proxy | TG sends some VXLAN encapsulated packets to |
@@ -67,7 +65,6 @@ virtual VM environment:
| | will receive these packets from one NIC and |
| | loopback the VXLAN-GPE-NSH encapsulated |
| | packets to the TG from other NIC. |
-| | |
| | Test case count: 6. |
+-----------------------+----------------------------------------------+
| NSH SFC Service | TG sends some VXLAN-GPE-NSH |
@@ -77,7 +74,6 @@ virtual VM environment:
| | the VXLAN-GPE-NSH header, after that DUT1 |
| | loopback the VXLAN-GPE-NSH encapsulated |
| | packets to the TG from other NIC. |
-| | |
| | Test case count: 6. |
+-----------------------+----------------------------------------------+
diff --git a/docs/report/vpp_functional_tests/csit_release_notes.rst b/docs/report/vpp_functional_tests/csit_release_notes.rst
index 7110cdab7e..84cb3b671a 100644
--- a/docs/report/vpp_functional_tests/csit_release_notes.rst
+++ b/docs/report/vpp_functional_tests/csit_release_notes.rst
@@ -6,17 +6,17 @@ Changes in |csit-release|
#. **CSIT Framework**
- - Few test case bug fixes.
+ - Few test case bug fixes.
#. **Test Case Porting to VPP_make_test**
- - Implementation of VIRL functional integration tests in
- VPP_make_test.
+ - Implementation of VIRL functional integration tests in
+ VPP_make_test.
- - All VIRL tests identified by FD.io VPP and CSIT projects as
- Priority-0 ported to VPP_make_test.
- Ref. `CSIT_VIRL migration progress
- <https://docs.google.com/spreadsheets/d/1PciV8XN9v1qHbIRUpFJoqyES29_vik7lcFDl73G1usc/edit?usp=sharing>`_.
+ - All VIRL tests identified by FD.io VPP and CSIT projects as
+ Priority-0 ported to VPP_make_test.
+ Ref. `CSIT_VIRL migration progress
+ <https://docs.google.com/spreadsheets/d/1PciV8XN9v1qHbIRUpFJoqyES29_vik7lcFDl73G1usc/edit?usp=sharing>`_.
Known Issues
------------
diff --git a/docs/report/vpp_performance_tests/csit_release_notes.rst b/docs/report/vpp_performance_tests/csit_release_notes.rst
index 03fca8f9a0..f6887e138f 100644
--- a/docs/report/vpp_performance_tests/csit_release_notes.rst
+++ b/docs/report/vpp_performance_tests/csit_release_notes.rst
@@ -6,53 +6,53 @@ Changes in |csit-release|
#. **VPP Performance Tests**
- - **MRR Throughput**: MRR (Maximum Receive Rate) test code has now
- configurable trial duration and number of consecutive executions.
- Coverage of MRR tests has been extended across more test
- scenarios. MRR tests are used for continuous performance trending
- and for comparison between VPP releases.
-
- - **MLRsearch Throughput**: MLRsearch algorithm has been introduced
- for all NDR and PDR throughput tests. All tests that previously
- used binary search got converted to MLRsearch. Coverage of NDR/PDR
- tests has been extended across more test scenarios.
-
- - **L2patch Tests**: Tests measure performance of VPP L2patch, the
- fastest L2 forwarding path implemented in VPP, that cross-links
- RX and TX of two physical interfaces.
-
- - **2-Node Tests**: A new baseline set of 2-node tests covering base
- ip4, ip6, l2patch, l2bd, l2xc, running on new Xeon Skylake
- testbeds.
-
- - **Generated tests**: Simplified and unified test structure, semi-
- autogenerated by generator script. Test generator is currently
- able to create test combinations with various frame size and
- cores combinations. All existing test cases were converted to new
- format.
-
- - **Simultaneous Multi-Threading**: SMT-aware detection of server
- processor operation mode (HyperThreading enabled/disabled) with
- associated compute resource configuration including thread
- affinity, number of Rx queues and DPDK I/O mbufs. Tests are
- automatically tagged during execution to indicate executed thread
- configuration.
-
- - **Intel Xeon Skylake Support**: Support for 2-Node and 3-Node
- physical testbed topologies based on the new SuperMirco servers
- each with two Intel Xeon Skylake Platinum processors. Full
- Ansible playbooks refactor for quick server (re)installation and
- reference pointers of configuration.
+ - **MRR Throughput**: MRR (Maximum Receive Rate) test code has now
+ configurable trial duration and number of consecutive executions.
+ Coverage of MRR tests has been extended across more test
+ scenarios. MRR tests are used for continuous performance trending
+ and for comparison between VPP releases.
+
+ - **MLRsearch Throughput**: MLRsearch algorithm has been introduced
+ for all NDR and PDR throughput tests. All tests that previously
+ used binary search got converted to MLRsearch. Coverage of NDR/PDR
+ tests has been extended across more test scenarios.
+
+ - **L2patch Tests**: Tests measure performance of VPP L2patch, the
+ fastest L2 forwarding path implemented in VPP, that cross-links
+ RX and TX of two physical interfaces.
+
+ - **2-Node Tests**: A new baseline set of 2-node tests covering base
+ ip4, ip6, l2patch, l2bd, l2xc, running on new Xeon Skylake
+ testbeds.
+
+ - **Generated tests**: Simplified and unified test structure, semi-
+ autogenerated by generator script. Test generator is currently
+ able to create test combinations with various frame size and
+ cores combinations. All existing test cases were converted to new
+ format.
+
+ - **Simultaneous Multi-Threading**: SMT-aware detection of server
+ processor operation mode (HyperThreading enabled/disabled) with
+ associated compute resource configuration including thread
+ affinity, number of Rx queues and DPDK I/O mbufs. Tests are
+ automatically tagged during execution to indicate executed thread
+ configuration.
+
+ - **Intel Xeon Skylake Support**: Support for 2-Node and 3-Node
+ physical testbed topologies based on the new SuperMirco servers
+ each with two Intel Xeon Skylake Platinum processors. Full
+ Ansible playbooks refactor for quick server (re)installation and
+ reference pointers of configuration.
#. **Presentation and Analytics Layer**
- - **Performance trending**: Further improved continuous performance
- trending with anomaly detection and analysis.
+ - **Performance trending**: Further improved continuous performance
+ trending with anomaly detection and analysis.
#. **Test Framework Optimizations**
- - **General Code Housekeeping**: Ongoing RF keywords optimizations,
- removal of redundant RF keywords.
+ - **General Code Housekeeping**: Ongoing RF keywords optimizations,
+ removal of redundant RF keywords.
Performance Changes
-------------------