1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
|
Report Structure
================
FD.io |csit-release| report contains system performance and functional
testing data of |vpp-release|. `PDF version of this report`_ is
available for download.
|csit-release| report is structured as follows:
#. INTRODUCTION: General introduction to FD.io |csit-release|.
- **Introduction**: This section.
- **Test Scenarios Overview**: A brief overview of test scenarios
covered in this report.
- **Physical Testbeds**: Description of physical testbeds.
- **Test Methodology**: Performance benchmarking and functional test
methodologies.
#. VPP PERFORMANCE: VPP performance tests executed in physical
FD.io testbeds.
- **Overview**: Tested logical topologies, test coverage and naming
specifics.
- **Release Notes**: Changes in |csit-release|, added tests,
environment or methodology changes, known issues.
- **Packet Throughput**: NDR, PDR throughput graphs based on results
from repeated same test job executions to verify repeatibility of
measurements.
- **Speedup Multi-Core**: NDR, PDR throughput multi-core speedup
graphs based on results from test job executions.
- **Packet Latency**: Latency graphs based on results from test job
executions.
- **Soak Tests**: Long duration soak tests are executed using PLRsearch
algorithm.
- **NFV Service Density**: Network Function Virtualization (NFV) service
density tests focus on measuring total per server throughput at varied NFV
service "packing" densities with vswitch providing host dataplane.
- **Comparisons**: Performance comparisons between VPP releases and
between different testbed types.
- **Throughput Trending**: References to continuous VPP performance
trending.
- **Test Environment**: Performance test environment configuration.
- **Documentation**: Pointers to CSIT source code documentation for VPP
performance tests.
#. DPDK PERFORMANCE: DPDK performance tests executed in physical
FD.io testbeds.
- **Overview**: Tested logical topologies, test coverage.
- **Release Notes**: Changes in |csit-release|, known issues.
- **Packet Throughput**: NDR, PDR throughput graphs based on results
from repeated same test job executions to verify repeatibility of
measurements.
- **Packet Latency**: Latency graphs based on results from test job
executions.
- **Comparisons**: Performance comparisons between DPDK releases and
between different testbed types.
- **Throughput Trending**: References to regular DPDK performance
trending.
- **Test Environment**: Performance test environment configuration.
- **Documentation**: Pointers to CSIT source code documentation for
DPDK performance tests.
#. TREX PERFORMANCE: TREX performance tests executed in physical
FD.io testbeds.
- **Overview**: Tested logical topologies, test coverage.
- **Release Notes**: Changes in |csit-release|, known issues.
- **Packet Throughput**: NDR, PDR throughput graphs based on results
from repeated same test job executions to verify repeatibility of
measurements.
- **Packet Latency**: Latency graphs based on results from test job
executions.
- **Throughput Trending**: References to regular TREX performance
trending.
- **Test Environment**: Performance test environment configuration.
#. VPP DEVICE: VPP functional tests executed in physical FD.io
testbeds using containers.
- **Overview**: Tested virtual topologies, test coverage and naming
specifics;
- **Release Notes**: Changes in |csit-release|, added tests,
environment or methodology changes, known issues.
- **Integration Tests**: Functional test environment configuration.
- **Documentation**: Pointers to CSIT source code documentation for
VPP functional tests.
#. DETAILED RESULTS: Detailed result tables auto-generated from CSIT
test job executions using RF (Robot Framework) output files as
sources.
- **VPP Performance NDR/PDR**: VPP NDR/PDR throughput and latency.
- **VPP Performance MRR**: VPP MRR throughput.
- **DPDK Performance**: DPDK Testpmd and L3fwd NDR/PDR throughput
and latency.
#. TEST CONFIGURATION: VPP DUT configuration data based on VPP API
Test (VAT) Commands History auto-generated from CSIT test job
executions using RF output files as sources.
- **VPP Performance NDR/PDR**: Configuration data.
- **VPP Performance MRR**: Configuration data.
#. TEST OPERATIONAL DATA: VPP DUT operational data auto-generated
from CSIT test job executions using RFoutput files as sources.
- **VPP Performance NDR/PDR**: VPP `show run` outputs under test
load.
#. CSIT FRAMEWORK DOCUMENTATION: Description of the overall FD.io
CSIT framework.
- **Design**: Framework modular design hierarchy.
- **Test naming**: Test naming convention.
- **Presentation and Analytics Layer**: Description of PAL CSIT
analytics module.
- **CSIT RF Tags Descriptions**: CSIT RF Tags used for test suite and
test case grouping and selection.
|
