aboutsummaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorVratko Polak <vrpolak@cisco.com>2021-11-16 17:32:50 +0100
committerTibor Frank <tifrank@cisco.com>2021-11-18 12:15:38 +0000
commitcc58afc7de521e571fe4cbaa4c9841a2d3e9be52 (patch)
treeac21a20208b01cb9543ea2476f62aa59c30d0e15
parent1493f602debb12f1ba152b74c16d1a379a53e50f (diff)
Report: Improve docs related to TRex
Change-Id: I0ed346ff30c61d28b5a2232ef2f9d32d26c1ae2c Signed-off-by: Vratko Polak <vrpolak@cisco.com>
-rw-r--r--docs/report/introduction/test_environment_changes_tg.rst2
-rw-r--r--docs/report/introduction/test_environment_intro.rst5
-rw-r--r--docs/report/trex_performance_tests/overview.rst11
-rw-r--r--docs/report/trex_performance_tests/throughput_trending.rst6
4 files changed, 15 insertions, 9 deletions
diff --git a/docs/report/introduction/test_environment_changes_tg.rst b/docs/report/introduction/test_environment_changes_tg.rst
index 21175f2ba5..d21e43e04a 100644
--- a/docs/report/introduction/test_environment_changes_tg.rst
+++ b/docs/report/introduction/test_environment_changes_tg.rst
@@ -1,5 +1,5 @@
To identify performance changes due to TRex code development between previous
-and current TRex version, both have been tested in CSIT environment of latest
+and current TRex version, both have been tested in CSIT environment of latest
version and compared against each other. All substantial progressions and
regressions have been marked up with RCA analysis. See :ref:`trex_known_issues`.
diff --git a/docs/report/introduction/test_environment_intro.rst b/docs/report/introduction/test_environment_intro.rst
index 1999906669..c2feb1b4c4 100644
--- a/docs/report/introduction/test_environment_intro.rst
+++ b/docs/report/introduction/test_environment_intro.rst
@@ -10,9 +10,12 @@ CSIT test environment versioning has been introduced to track
modifications of the test environment.
Any benchmark anomalies (progressions, regressions) between releases of
-a DUT application (e.g. VPP, DPDK, TRex), are determined by testing it in the
+a DUT application (e.g. VPP, DPDK), are determined by testing it in the
same test environment, to avoid test environment changes clouding the
picture.
+To beter distinguish impact of test environment changes,
+we also execute tests without any SUT (just with TRex TG sending packets
+over a link looping back to TG).
A mirror approach is introduced to determine benchmarking anomalies due
to the test environment change. This is achieved by testing the same DUT
diff --git a/docs/report/trex_performance_tests/overview.rst b/docs/report/trex_performance_tests/overview.rst
index 9740179df7..8ad8850ea8 100644
--- a/docs/report/trex_performance_tests/overview.rst
+++ b/docs/report/trex_performance_tests/overview.rst
@@ -1,14 +1,14 @@
Overview
========
-TREX performance test results are reported for a range of processors.
-For description of physical testbeds used for TREX performance tests
+TRex performance test results are reported for a range of processors.
+For description of physical testbeds used for TRex performance tests
please refer to :ref:`tested_physical_topologies`.
Logical Topology
----------------
-CSIT TREX performance tests are executed on physical testbeds described
+CSIT TRex performance tests are executed on physical testbeds described
in :ref:`tested_physical_topologies`. Logical topology use 1 nic that has
loopback connected ports. See figure below.
@@ -52,11 +52,12 @@ topologies and configurations:
- 10% of discovered PDR throughput.
- Minimal offered load.
-|csit-release| includes following TRex data plane functionality
+|csit-release| includes tests using the following TRex traffic profiles
+(corresponding to data plane functionality when DUT is used)
performance tested across a range of NIC drivers and NIC models:
+-----------------------+----------------------------------------------+
-| Functionality | Description |
+| Traffic profile | Corresponding dataplane functionality |
+=======================+==============================================+
| IPv4 Base | IPv4 routing. |
+-----------------------+----------------------------------------------+
diff --git a/docs/report/trex_performance_tests/throughput_trending.rst b/docs/report/trex_performance_tests/throughput_trending.rst
index cb2751972f..9106cb4bdf 100644
--- a/docs/report/trex_performance_tests/throughput_trending.rst
+++ b/docs/report/trex_performance_tests/throughput_trending.rst
@@ -1,12 +1,14 @@
Throughput Trending
-------------------
-In addition to reporting throughput comparison between TRex releases,
CSIT provides continuous performance trending for master branch:
#. `TRex Trending Graphs <https://s3-docs.fd.io/csit/master/trending/ndrpdr_trending/trex.html>`_:
per TRex test case throughput trend, trend compliance and summary of
- detected anomalies.
+ detected anomalies. We expect TRex to hit the curently used bps or pps limit,
+ so no anomalies here (unless we change those limits in CSIT).
#. `TRex Latency Graphs <https://s3-docs.fd.io/csit/master/trending/ndrpdr_latency_trending/trex.html>`_:
per TRex build NDRPDR latency measurements against the trendline.
+ We have seen in past that the latency numbers can depend on TRex version,
+ NIC firmware, or driver used.