blob: dc86f8600ed5bc67fad89d763e94fd2b1cfe5c8b (
plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
|
Overview
========
This is the **F**\ast **D**\ata **I**/**O** Project (FD.io) **C**\ontinuous
**S**\ystem **I**\ntegration and **T**\esting (CSIT) project report for CSIT
|release| system testing of VPP-17.01 release. Initial sections of the report
describe FD.io functional and performance tests and their continuous execution
achieved in CSIT |release|. This is followed by high-level overview of FD.io
test environments running in Linux Foundation (LF) FD.io Continuous
Performance Labs), CSIT |release| release notes and result highlights.
Subsequent sections list the functional and performance detailed test results
generated by LF FD.io Jenkins system executing CSIT jobs running tests
against the VPP-17.01 release and associated sub-systems (HoneyComb, DPDK).
The exact structure of the report is as follows:
#. ``Introduction`` - Includes: ``Overview`` - this section;
``CSIT Test Naming`` - describing CSIT naming convention for test suites and
test cases, important to understand to correctly interpret the reported
results; ``General Remarks`` - additional notes related to this report.
#. ``VPP Performance Tests HW`` - Summary of VPP performance tests executed in
physical Hardware testbed. Includes: ``Overview`` - containing topology
description and list of executed tests; ``Performance Tests Naming`` - lists
CSIT naming specifics for performance tests; ``CSIT Release Notes`` - with
changes in CSIT |release| including major test additions, performance
improvements, changes in testing environment and methodology and known
issues; ``Performance Measurements Methodology`` - details testing
methodology followed for multi-thread multi-core, packet throughput and
latency, and KVM VM vhost tests; ``Report Addendum Targets`` - lists content
to be added to this report as part of CSIT |release|.
#. ``Testpmd Performance Tests HW`` - ...
#. ``VPP Functional Tests VIRL`` - ...
#. ``VPP Functional Tests VPPtest`` - ...
#. ``HoneyComb Functional Tests VIRL`` - ..
#. ``VPP Performance Results HW`` - ...
#. ``Testpmd Performance Results HW`` - ...
#. ``VPP Functional Results VIRL`` - ...
#. ``VPP Functional Results VPPtest`` - ...
#. ``HoneyComb Functional Results VIRL`` - ...
FD.io CSIT system is developed using two main coding platforms: Robot
Framework (RF) and Python. CSIT |release| source code for the executed test
suites is available in CSIT branch |release| in the directory
"./tests/<name_of_the_test_suite>". A local copy of CSIT source code can be
obtained by cloning CSIT git repository - "git clone
https://gerrit.fd.io/r/csit". The CSIT testing virtual environment can be run
on a local computer workstation (laptop, server) using Vagrant by following
the instructions in `CSIT tutorials
<https://wiki.fd.io/view/CSIT#Tutorials>`_.
|
