1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
|
Release Notes
=============
Changes in |csit-release|
-------------------------
#. **DPDK performance tests**
- *MRR tests* - Maximum Receive Rate tests measure the packet forwarding rate
under the maximum load offered by traffic generator over a set trial
duration, regardless of packet loss. MRR tests are used for continuous
performance trending and for comparison between releases.
- *MLR tests* - NDR and PDR tests measure the packet forwarding rate using
MLRsearch library by traffic generator. All tests that previously used
binary search were converted to MLRsearch.
- *2-node tests* - Set of 2-node tests covering testpmd and l3fwd.
- Increased coverage of NIC specific tests (Intel-xxv710-da2, Intel-x710)
- *Generated tests* - Simplified and unified test structure,
semi-autogenerated by generator script. Test generator is currently able
to create test combinations with various frame size and cores combinations.
All existing test cases were converted to new format.
Performance Changes
-------------------
Relative performance changes in measured packet throughput in |csit-release|
are calculated against the results from |csit-release-1|
report. Listed mean and standard deviation values are computed based on
a series of the same tests executed against respective DPDK releases to
verify test results repeatability, with percentage change calculated for
mean values.
NDR Changes
~~~~~~~~~~~
NDR throughput changes between releases are available in a CSV and pretty ASCII
formats:
- `csv format for 1t1c <../_static/dpdk/performance-changes-1t1c-ndr.csv>`_,
- `csv format for 2t2c <../_static/dpdk/performance-changes-2t2c-ndr.csv>`_,
- `pretty ASCII format for 1t1c <../_static/dpdk/performance-changes-1t1c-ndr.txt>`_,
- `pretty ASCII format for 2t2c <../_static/dpdk/performance-changes-2t2c-ndr.txt>`_.
.. note::
Test results have been generated by
`FD.io test executor dpdk performance job 3n-hsw`_
with Robot Framework result
files csit-dpdk-perf-|srelease|-\*.zip
`archived here <../_static/archive/>`_.
PDR Changes
~~~~~~~~~~~
PDR throughput changes between releases are available in a CSV and pretty ASCII
formats:
- `csv format for 1t1c <../_static/dpdk/performance-changes-1t1c-pdr.csv>`_,
- `csv format for 2t2c <../_static/dpdk/performance-changes-2t2c-pdr.csv>`_,
- `pretty ASCII format for 1t1c <../_static/dpdk/performance-changes-1t1c-pdr.txt>`_,
- `pretty ASCII format for 2t2c <../_static/dpdk/performance-changes-2t2c-pdr.txt>`_.
.. note::
Test results have been generated by
`FD.io test executor dpdk performance job 3n-hsw`_
with Robot Framework result
files csit-dpdk-perf-|srelease|-\*.zip
`archived here <../_static/archive/>`_.
MRR Changes
~~~~~~~~~~~
MRR small packet throughput changes between releases are available in a
CSV and pretty ASCII formats:
- `csv format for 1t1c <../_static/dpdk/performance-changes-1t1c-mrr.csv>`_,
- `csv format for 2t2c <../_static/dpdk/performance-changes-2t2c-mrr.csv>`_,
- `csv format for 4t4c <../_static/dpdk/performance-changes-4t4c-mrr.csv>`_,
- `pretty ASCII format for 1t1c <../_static/dpdk/performance-changes-1t1c-mrr.txt>`_,
- `pretty ASCII format for 2t2c <../_static/dpdk/performance-changes-2t2c-mrr.txt>`_,
- `pretty ASCII format for 4t4c <../_static/dpdk/performance-changes-4t4c-mrr.txt>`_.
.. note::
Test results have been generated by
`FD.io test executor dpdk performance job 3n-hsw`_,
with Robot Framework result
files csit-vpp-perf-|srelease|-\*.zip
`archived here <../_static/archive/>`_.
Comparison Across Testbeds
--------------------------
Relative performance changes in measured packet throughput on 3-Node Skx testbed
are calculated against the results measured on 3-Node Hsw testbed.
NDR Changes
~~~~~~~~~~~
NDR throughput changes between testbeds are available in a CSV and pretty ASCII
formats:
- `csv format for ndr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-ndr.csv>`_,
- `pretty ASCII format for ndr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-ndr.txt>`_.
.. note::
Test results have been generated by
`FD.io test executor dpdk performance job 3n-hsw`_ and
`FD.io test executor dpdk performance job 3n-skx`_
with Robot Framework result
files csit-dpdk-perf-|srelease|-\*.zip
`archived here <../_static/archive/>`_.
PDR Changes
~~~~~~~~~~~
PDR throughput changes between testbeds are available in a CSV and pretty ASCII
formats:
- `csv format for pdr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-pdr.csv>`_,
- `pretty ASCII format for pdr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-pdr.txt>`_.
.. note::
Test results have been generated by
`FD.io test executor dpdk performance job 3n-hsw`_ and
`FD.io test executor dpdk performance job 3n-skx`_
with Robot Framework result
files csit-dpdk-perf-|srelease|-\*.zip
`archived here <../_static/archive/>`_.
MRR Changes
~~~~~~~~~~~
MRR throughput changes between testbeds are available in a
CSV and pretty ASCII formats:
- `csv format for mrr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-mrr.csv>`_,
- `pretty ASCII format for mrr <../_static/dpdk/performance-compare-testbeds-3n-hsw-3n-skx-mrr.txt>`_.
.. note::
Test results have been generated by
`FD.io test executor dpdk performance job 3n-hsw`_ and
`FD.io test executor dpdk performance job 3n-skx`_
with Robot Framework result
files csit-vpp-perf-|srelease|-\*.zip
`archived here <../_static/archive/>`_.
Known Issues
------------
Here is the list of known issues in |csit-release| for Testpmd performance tests:
+---+---------------------------------------------------+------------+-----------------------------------------------------------------+
| # | Issue | Jira ID | Description |
+---+---------------------------------------------------+------------+-----------------------------------------------------------------+
| | No known issues | | |
+---+---------------------------------------------------+------------+-----------------------------------------------------------------+
|
