diff options
5 files changed, 79 insertions, 72 deletions
diff --git a/docs/report/dmm_functional_tests/csit_release_notes.rst b/docs/report/dmm_functional_tests/csit_release_notes.rst index 04b41f3c60..352809c038 100644 --- a/docs/report/dmm_functional_tests/csit_release_notes.rst +++ b/docs/report/dmm_functional_tests/csit_release_notes.rst @@ -4,9 +4,9 @@ Release Notes Changes in |csit-release| ------------------------- -#. Added DMM functional tests +#. **DMM Functional Tests** - - DMM base vs epoll test case + - Added DMM base vs. epoll test case. Known Issues ------------ @@ -17,6 +17,6 @@ VIRL: +---+-------------------------------------------------+----------+------------------------------------------------------+ | # | Issue | Jira ID | Description | +---+-------------------------------------------------+----------+------------------------------------------------------+ -| 1 | DMM CentOs job is disabled | DMM-17 | DPDK version and relative kernel version issue, | +| 1 | DMM CentOs job disabled | DMM-17 | DPDK version and relative kernel version issue, | | | | | CentOS job is temporarily disabled. | +---+-------------------------------------------------+----------+------------------------------------------------------+ diff --git a/docs/report/dpdk_performance_tests/csit_release_notes.rst b/docs/report/dpdk_performance_tests/csit_release_notes.rst index d073c493fe..bd08c57bba 100644 --- a/docs/report/dpdk_performance_tests/csit_release_notes.rst +++ b/docs/report/dpdk_performance_tests/csit_release_notes.rst @@ -6,24 +6,33 @@ Changes in |csit-release| #. **DPDK performance tests** - - *MRR tests* - Maximum Receive Rate tests measure the packet forwarding rate - under the maximum load offered by traffic generator over a set trial - duration, regardless of packet loss. MRR tests are used for continuous - performance trending and for comparison between releases. + - **MRR Throughput**: MRR (Maximum Receive Rate) test code has now + configurable trial duration and number of consecutive executions. + Coverage of MRR tests has been extended across more test + scenarios. MRR tests are used for continuous performance trending + and for comparison between DPDK releases. - - *MLR tests* - NDR and PDR tests measure the packet forwarding rate using - MLRsearch library by traffic generator. All tests that previously used - binary search were converted to MLRsearch. + - **MLRsearch Throughput**: MLRsearch algorithm has been introduced + for all NDR and PDR throughput tests. All tests that previously + used binary search got converted to MLRsearch. Coverage of NDR/PDR + tests has been extended across more test scenarios. - - *2-node tests* - Set of 2-node tests covering testpmd and l3fwd. + - **2-Node Tests**: A new set of 2-node tests covering DPDK Testpmd + and L3fwd, running on new Xeon Skylake testbeds. - - Increased coverage of NIC specific tests (Intel-xxv710-da2, Intel-x710) + - **More NICs**: Increased coverage of NIC models tested: Intel- + xxv710-da2, Intel-x710-da4. - - *Generated tests* - Simplified and unified test structure, - semi-autogenerated by generator script. Test generator is currently able - to create test combinations with various frame size and cores combinations. - All existing test cases were converted to new format. + - **Generated tests**: Simplified and unified test structure, semi- + autogenerated by generator script. Test generator is currently + able to create test combinations with various frame size and + cores combinations. All existing test cases were converted to new + format. +#. **Presentation and Analytics Layer** + + - **Performance trending**: Added continuous performance trending with + anomaly detection and analysis for DPDK. Performance Changes ------------------- diff --git a/docs/report/nsh_sfc_functional_tests/csit_release_notes.rst b/docs/report/nsh_sfc_functional_tests/csit_release_notes.rst index fb0f9b93c0..4ba9d7f1a5 100644 --- a/docs/report/nsh_sfc_functional_tests/csit_release_notes.rst +++ b/docs/report/nsh_sfc_functional_tests/csit_release_notes.rst @@ -4,17 +4,9 @@ Release Notes Changes in |csit-release| ------------------------- -#. Added NSH SFC functional tests - - - None. +No changes. Known Issues ------------ -Here is the list of known issues in |csit-release| for NSH_SFC functional tests in VIRL: - -+---+-------------------------------------------------+----------+------------------------------------------------------+ -| # | Issue | Jira ID | Description | -+---+-------------------------------------------------+----------+------------------------------------------------------+ -| 1 | None | None | None | -+---+-------------------------------------------------+----------+------------------------------------------------------+ +No known issues.
\ No newline at end of file diff --git a/docs/report/vpp_functional_tests/csit_release_notes.rst b/docs/report/vpp_functional_tests/csit_release_notes.rst index 5d4f2817a3..7110cdab7e 100644 --- a/docs/report/vpp_functional_tests/csit_release_notes.rst +++ b/docs/report/vpp_functional_tests/csit_release_notes.rst @@ -4,18 +4,19 @@ Release Notes Changes in |csit-release| ------------------------- -#. Code updates in CSIT functional framework: +#. **CSIT Framework** - - few test case bug fixes. + - Few test case bug fixes. -#. VPP_Path: +#. **Test Case Porting to VPP_make_test** - - Implementation of new VPP-make_test VPP integration tests for functional - acceptance of VPP feature path(s) driven by use case(s). + - Implementation of VIRL functional integration tests in + VPP_make_test. - - All VIRL P0 tests (`CSIT_VIRL migration progress - <https://docs.google.com/spreadsheets/d/1PciV8XN9v1qHbIRUpFJoqyES29_vik7lcFDl73G1usc/edit?usp=sharing>`_) - migrated to VPP_Path using VPP make test. + - All VIRL tests identified by FD.io VPP and CSIT projects as + Priority-0 ported to VPP_make_test. + Ref. `CSIT_VIRL migration progress + <https://docs.google.com/spreadsheets/d/1PciV8XN9v1qHbIRUpFJoqyES29_vik7lcFDl73G1usc/edit?usp=sharing>`_. Known Issues ------------ diff --git a/docs/report/vpp_performance_tests/csit_release_notes.rst b/docs/report/vpp_performance_tests/csit_release_notes.rst index 6ac7fcdfc7..580ef4812a 100644 --- a/docs/report/vpp_performance_tests/csit_release_notes.rst +++ b/docs/report/vpp_performance_tests/csit_release_notes.rst @@ -4,50 +4,55 @@ Release Notes Changes in |csit-release| ------------------------- -#. **VPP performance tests** - - - *MRR tests* - Maximum Receive Rate tests measure the packet forwarding rate - under the maximum load offered by traffic generator over a set trial - duration, regardless of packet loss. MRR tests are used for continuous - performance trending and for comparison between releases. - - - *MLR tests* - NDR and PDR tests measure the packet forwarding rate using - MLRsearch library by traffic generator. All tests that previously used - binary search were converted to MLRsearch. - - - *L2patch tests* - Tests measure performance of L2patch, cross linking - RX and TX of two physical interfaces. - - - *2-node tests* - Baseline set of 2-node tests covering base ip4, ip6, - l2patch, l2bd, l2xc. - - - *Generated tests* - Simplified and unified test structure, - semi-autogenerated by generator script. Test generator is currently able - to create test combinations with various frame size and cores combinations. - All existing test cases were converted to new format. - - - *Simultaneous multithreading* - SMT-aware automatic detection and - resource configuration including thread affinity, number of rx queues and - DPDK I/O mbufs. Tests are automatically tagged during execution indicating - configuration. - - - *Intel Skylake* - Topologies consisting of 2-node and 3-node using - SuperMirco servers each equipped with two Intel Xeon Skylake Platinum - processors. Full Ansible playbooks refactor for quick server - (re)installation and reference pointers of configuration. +#. **VPP Performance Tests** + + - **MRR Throughput**: MRR (Maximum Receive Rate) test code has now + configurable trial duration and number of consecutive executions. + Coverage of MRR tests has been extended across more test + scenarios. MRR tests are used for continuous performance trending + and for comparison between VPP releases. + + - **MLRsearch Throughput**: MLRsearch algorithm has been introduced + for all NDR and PDR throughput tests. All tests that previously + used binary search got converted to MLRsearch. Coverage of NDR/PDR + tests has been extended across more test scenarios. + + - **L2patch Tests**: Tests measure performance of VPP L2patch, the + fastest L2 forwarding path implemented in VPP, that cross-links + RX and TX of two physical interfaces. + + - **2-Node Tests**: A new baseline set of 2-node tests covering base + ip4, ip6, l2patch, l2bd, l2xc, running on new Xeon Skylake + testbeds. + + - **Generated tests**: Simplified and unified test structure, semi- + autogenerated by generator script. Test generator is currently + able to create test combinations with various frame size and + cores combinations. All existing test cases were converted to new + format. + + - **Simultaneous Multi-Threading**: SMT-aware detection of server + processor operation mode (HyperThreading enabled/disabled) with + associated compute resource configuration including thread + affinity, number of Rx queues and DPDK I/O mbufs. Tests are + automatically tagged during execution to indicate executed thread + configuration. + + - **Intel Xeon Skylake Support**: Support for 2-Node and 3-Node + physical testbed topologies based on the new SuperMirco servers + each with two Intel Xeon Skylake Platinum processors. Full + Ansible playbooks refactor for quick server (re)installation and + reference pointers of configuration. #. **Presentation and Analytics Layer** - - *Performance trending* - Added continuous performance trending and - analysis. New Performance Trending and Performance Analysis jobs - executed regular throughput tests, with results being subsequently - analysed and trend and anomalies summarized and presented in VPP - Performance Dashboard and trendline graphs. + - **Performance trending**: Further improved continuous performance + trending with anomaly detection and analysis. #. **Test Framework Optimizations** - - *General code housekeeping* - ongoing RF keywords optimizations, - removal of redundant RF keywords. + - **General Code Housekeeping**: Ongoing RF keywords optimizations, + removal of redundant RF keywords. Performance Changes ------------------- |