aboutsummaryrefslogtreecommitdiffstats
path: root/docs/report/dpdk_performance_tests
diff options
context:
space:
mode:
Diffstat (limited to 'docs/report/dpdk_performance_tests')
-rw-r--r--docs/report/dpdk_performance_tests/csit_release_notes.rst94
-rw-r--r--docs/report/dpdk_performance_tests/overview.rst2
-rw-r--r--docs/report/dpdk_performance_tests/packet_latency_graphs/index.rst11
-rw-r--r--docs/report/dpdk_performance_tests/packet_throughput_graphs/index.rst9
4 files changed, 109 insertions, 7 deletions
diff --git a/docs/report/dpdk_performance_tests/csit_release_notes.rst b/docs/report/dpdk_performance_tests/csit_release_notes.rst
index 214eb4c4e7..363c2e70f1 100644
--- a/docs/report/dpdk_performance_tests/csit_release_notes.rst
+++ b/docs/report/dpdk_performance_tests/csit_release_notes.rst
@@ -6,6 +6,100 @@ Changes in |csit-release|
No code changes apart from bug fixes.
+Performance Changes
+-------------------
+
+Relative performance changes in measured packet throughput in |csit-release|
+are calculated against the results from |csit-release-1|
+report. Listed mean and standard deviation values are computed based on
+a series of the same tests executed against respective VPP releases to
+verify test results repeatability, with percentage change calculated for
+mean values. Note that the standard deviation is quite high for a small
+number of packet throughput tests, what indicates poor test results
+repeatability and makes the relative change of mean throughput value not
+fully representative for these tests. The root causes behind poor
+results repeatability vary between the test cases.
+
+NDR Changes
+~~~~~~~~~~~
+
+NDR throughput changes between releases are available in a
+CSV and pretty ASCII formats:
+
+ - `csv format for 1t1c <../_static/dpdk/performance-changes-1t1c-ndr.csv>`_,
+ - `csv format for 2t2c <../_static/dpdk/performance-changes-2t2c-ndr.csv>`_,
+ - `pretty ASCII format for 1t1c <../_static/dpdk/performance-changes-1t1c-ndr.txt>`_,
+ - `pretty ASCII format for 2t2c <../_static/dpdk/performance-changes-2t2c-ndr.txt>`_.
+
+.. note::
+
+ Test results have been generated by
+ `FD.io test executor dpdk performance job 3n-hsw`_
+ with Robot Framework result
+ files csit-vpp-perf-|srelease|-\*.zip
+ `archived here <../_static/archive/>`_.
+
+PDR Changes
+~~~~~~~~~~~
+
+PDR throughput changes between releases are available in a
+CSV and pretty ASCII formats:
+
+ - `csv format for 1t1c <../_static/dpdk/performance-changes-1t1c-pdr.csv>`_,
+ - `csv format for 2t2c <../_static/dpdk/performance-changes-2t2c-pdr.csv>`_,
+ - `pretty ASCII format for 1t1c <../_static/dpdk/performance-changes-1t1c-pdr.txt>`_,
+ - `pretty ASCII format for 2t2c <../_static/dpdk/performance-changes-2t2c-pdr.txt>`_.
+
+.. note::
+
+ Test results have been generated by
+ `FD.io test executor dpdk performance job 3n-hsw`_
+ with Robot Framework result
+ files csit-vpp-perf-|srelease|-\*.zip
+ `archived here <../_static/archive/>`_.
+
+Comparison Across Testbeds
+--------------------------
+
+Relative performance changes in measured packet throughputon 3-Node Skx testbed
+are calculated against the results measured on 3-Node Hsw testbed.
+
+NDR Changes
+~~~~~~~~~~~
+
+NDR changes between testbeds are available in a
+CSV and pretty ASCII formats:
+
+ - `csv format for ndr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-ndr.csv>`_,
+ - `pretty ASCII format for ndr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-ndr.txt>`_.
+
+.. note::
+
+ Test results have been generated by
+ `FD.io test executor dpdk performance job 3n-hsw`_ and
+ `FD.io test executor dpdk performance job 3n-skx`_
+ with Robot Framework result
+ files csit-vpp-perf-|srelease|-\*.zip
+ `archived here <../_static/archive/>`_.
+
+PDR Changes
+~~~~~~~~~~~
+
+PDR throughput changes between testbeds are available in a
+CSV and pretty ASCII formats:
+
+ - `csv format for pdr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-pdr.csv>`_,
+ - `pretty ASCII format for pdr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-pdr.txt>`_.
+
+.. note::
+
+ Test results have been generated by
+ `FD.io test executor dpdk performance job 3n-hsw`_ and
+ `FD.io test executor dpdk performance job 3n-skx`_
+ with Robot Framework result
+ files csit-vpp-perf-|srelease|-\*.zip
+ `archived here <../_static/archive/>`_.
+
Known Issues
------------
diff --git a/docs/report/dpdk_performance_tests/overview.rst b/docs/report/dpdk_performance_tests/overview.rst
index 037300b3ee..b38f9595be 100644
--- a/docs/report/dpdk_performance_tests/overview.rst
+++ b/docs/report/dpdk_performance_tests/overview.rst
@@ -116,7 +116,7 @@ Performance Tests Naming
------------------------
|csit-release| follows a common structured naming convention for all performance
-and system functional tests, introduced in CSIT rls1701.
+and system functional tests, introduced in CSIT-17.01.
The naming should be intuitive for majority of the tests. Complete description
of CSIT test naming convention is provided on :ref:`csit_test_naming`.
diff --git a/docs/report/dpdk_performance_tests/packet_latency_graphs/index.rst b/docs/report/dpdk_performance_tests/packet_latency_graphs/index.rst
index 1e59d1eae2..014d1095a7 100644
--- a/docs/report/dpdk_performance_tests/packet_latency_graphs/index.rst
+++ b/docs/report/dpdk_performance_tests/packet_latency_graphs/index.rst
@@ -8,7 +8,7 @@ latency per test.
*Title of each graph* is a regex (regular expression) matching all
throughput test cases plotted on this graph, *X-axis labels* are indices
of individual test suites executed by
-`FD.io test executor dpdk performance jobs`_ that created result output file
+FD.io test executor dpdk performance jobs that created result output file
used as data source for the graph, *Y-axis labels* are measured packet Latency
[uSec] values, and the *Graph legend* lists the plotted test suites and their
indices. Latency is reported for concurrent symmetric bi-directional flows,
@@ -19,8 +19,13 @@ TGint2-to-SUT2-to-SUT1-to-TGint1.
.. note::
Test results have been generated by
- `FD.io test executor dpdk performance jobs`_ with Robot Framework result
- files csit-dpdk-perf-\*.zip `archived here <../../_static/archive/>`_.
+ `FD.io test executor dpdk performance job 3n-hsw`_,
+ `FD.io test executor dpdk performance job 3n-skx`_ and
+ `FD.io test executor dpdk performance job 2n-skx`_ with Robot Framework
+ result files csit-vpp-perf-|srelease|-\*.zip
+ `archived here <../../_static/archive/>`_.
+ Plotted data set size per test case is equal to the number of job executions
+ presented in this report version: **10**.
.. toctree::
:maxdepth: 1
diff --git a/docs/report/dpdk_performance_tests/packet_throughput_graphs/index.rst b/docs/report/dpdk_performance_tests/packet_throughput_graphs/index.rst
index 42dfe4f572..a243922798 100644
--- a/docs/report/dpdk_performance_tests/packet_throughput_graphs/index.rst
+++ b/docs/report/dpdk_performance_tests/packet_throughput_graphs/index.rst
@@ -20,7 +20,7 @@ have the same value, only a horizontal line is plotted.
*Title of each graph* is a regex (regular expression) matching all
throughput test cases plotted on this graph, *X-axis labels* are indices
of individual test suites executed by
-`FD.io test executor dpdk performance jobs`_ jobs that created result output
+FD.io test executor dpdk performance jobs that created result output
files used as data sources for the graph, *Y-axis labels* are measured Packets
Per Second [pps] values, and the *Graph legend* lists the plotted test suites
and their indices.
@@ -28,8 +28,11 @@ and their indices.
.. note::
Test results have been generated by
- `FD.io test executor dpdk performance jobs`_ with Robot Framework result
- files csit-dpdk-perf-\*.zip `archived here <../../_static/archive/>`_.
+ `FD.io test executor dpdk performance job 3n-hsw`_,
+ `FD.io test executor dpdk performance job 3n-skx`_ and
+ `FD.io test executor dpdk performance job 2n-skx`_ with Robot Framework
+ result files csit-vpp-perf-|srelease|-\*.zip
+ `archived here <../../_static/archive/>`_.
Plotted data set size per test case is equal to the number of job executions
presented in this report version: **10**.