aboutsummaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorVratko Polak <vrpolak@cisco.com>2021-11-16 17:32:50 +0100
committerTibor Frank <tifrank@cisco.com>2021-11-18 12:16:01 +0000
commitfdf6fb175c4a8051ff75732b8b1bae139bacf9c8 (patch)
tree98907b81ffb51567db1b05552565fa73dfa3463a
parent37eba068fba30ebecadb8ea175273d057fcb6105 (diff)
Report: Improve docs related to TRex
Change-Id: I0ed346ff30c61d28b5a2232ef2f9d32d26c1ae2c Signed-off-by: Vratko Polak <vrpolak@cisco.com> (cherry picked from commit cc58afc7de521e571fe4cbaa4c9841a2d3e9be52)
-rw-r--r--docs/report/introduction/test_environment_changes_tg.rst2
-rw-r--r--docs/report/introduction/test_environment_intro.rst5
-rw-r--r--docs/report/trex_performance_tests/overview.rst11
-rw-r--r--docs/report/trex_performance_tests/throughput_trending.rst6
4 files changed, 15 insertions, 9 deletions
diff --git a/docs/report/introduction/test_environment_changes_tg.rst b/docs/report/introduction/test_environment_changes_tg.rst
index 21175f2ba5..d21e43e04a 100644
--- a/docs/report/introduction/test_environment_changes_tg.rst
+++ b/docs/report/introduction/test_environment_changes_tg.rst
@@ -1,5 +1,5 @@
To identify performance changes due to TRex code development between previous
-and current TRex version, both have been tested in CSIT environment of latest
+and current TRex version, both have been tested in CSIT environment of latest
version and compared against each other. All substantial progressions and
regressions have been marked up with RCA analysis. See :ref:`trex_known_issues`.
diff --git a/docs/report/introduction/test_environment_intro.rst b/docs/report/introduction/test_environment_intro.rst
index 1999906669..c2feb1b4c4 100644
--- a/docs/report/introduction/test_environment_intro.rst
+++ b/docs/report/introduction/test_environment_intro.rst
@@ -10,9 +10,12 @@ CSIT test environment versioning has been introduced to track
modifications of the test environment.
Any benchmark anomalies (progressions, regressions) between releases of
-a DUT application (e.g. VPP, DPDK, TRex), are determined by testing it in the
+a DUT application (e.g. VPP, DPDK), are determined by testing it in the
same test environment, to avoid test environment changes clouding the
picture.
+To beter distinguish impact of test environment changes,
+we also execute tests without any SUT (just with TRex TG sending packets
+over a link looping back to TG).
A mirror approach is introduced to determine benchmarking anomalies due
to the test environment change. This is achieved by testing the same DUT
diff --git a/docs/report/trex_performance_tests/overview.rst b/docs/report/trex_performance_tests/overview.rst
index 9740179df7..8ad8850ea8 100644
--- a/docs/report/trex_performance_tests/overview.rst
+++ b/docs/report/trex_performance_tests/overview.rst
@@ -1,14 +1,14 @@
Overview
========
-TREX performance test results are reported for a range of processors.
-For description of physical testbeds used for TREX performance tests
+TRex performance test results are reported for a range of processors.
+For description of physical testbeds used for TRex performance tests
please refer to :ref:`tested_physical_topologies`.
Logical Topology
----------------
-CSIT TREX performance tests are executed on physical testbeds described
+CSIT TRex performance tests are executed on physical testbeds described
in :ref:`tested_physical_topologies`. Logical topology use 1 nic that has
loopback connected ports. See figure below.
@@ -52,11 +52,12 @@ topologies and configurations:
- 10% of discovered PDR throughput.
- Minimal offered load.
-|csit-release| includes following TRex data plane functionality
+|csit-release| includes tests using the following TRex traffic profiles
+(corresponding to data plane functionality when DUT is used)
performance tested across a range of NIC drivers and NIC models:
+-----------------------+----------------------------------------------+
-| Functionality | Description |
+| Traffic profile | Corresponding dataplane functionality |
+=======================+==============================================+
| IPv4 Base | IPv4 routing. |
+-----------------------+----------------------------------------------+
diff --git a/docs/report/trex_performance_tests/throughput_trending.rst b/docs/report/trex_performance_tests/throughput_trending.rst
index cb2751972f..9106cb4bdf 100644
--- a/docs/report/trex_performance_tests/throughput_trending.rst
+++ b/docs/report/trex_performance_tests/throughput_trending.rst
@@ -1,12 +1,14 @@
Throughput Trending
-------------------
-In addition to reporting throughput comparison between TRex releases,
CSIT provides continuous performance trending for master branch:
#. `TRex Trending Graphs <https://s3-docs.fd.io/csit/master/trending/ndrpdr_trending/trex.html>`_:
per TRex test case throughput trend, trend compliance and summary of
- detected anomalies.
+ detected anomalies. We expect TRex to hit the curently used bps or pps limit,
+ so no anomalies here (unless we change those limits in CSIT).
#. `TRex Latency Graphs <https://s3-docs.fd.io/csit/master/trending/ndrpdr_latency_trending/trex.html>`_:
per TRex build NDRPDR latency measurements against the trendline.
+ We have seen in past that the latency numbers can depend on TRex version,
+ NIC firmware, or driver used.