aboutsummaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorVratko Polak <vrpolak@cisco.com>2024-04-25 13:39:05 +0200
committerTibor Frank <tifrank@cisco.com>2024-04-25 11:55:21 +0000
commit6bd1272528e0a383acd3c059bdc2230bc7d83446 (patch)
tree931d2d2b7a35bcca400c25ac90e85d88c618fa9b
parent98281bab4170d96f6618fa985895bc0c885251fe (diff)
fix(methodology): Update info for soak and ndrpdr
Change-Id: I58fa4b3494c6148735a992f2c0e8a7ba771660a8 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
-rw-r--r--docs/content/methodology/measurements/data_plane_throughput/data_plane_throughput.md82
-rw-r--r--docs/content/methodology/measurements/data_plane_throughput/plr_search.md5
2 files changed, 50 insertions, 37 deletions
diff --git a/docs/content/methodology/measurements/data_plane_throughput/data_plane_throughput.md b/docs/content/methodology/measurements/data_plane_throughput/data_plane_throughput.md
index 865405ba2f..c7dce24c1a 100644
--- a/docs/content/methodology/measurements/data_plane_throughput/data_plane_throughput.md
+++ b/docs/content/methodology/measurements/data_plane_throughput/data_plane_throughput.md
@@ -11,24 +11,39 @@ set of performance test cases implemented and executed within CSIT.
Following throughput test methods are used:
-- MLRsearch - Multiple Loss Ratio search
-- PLRsearch - Probabilistic Loss Ratio search
-- MRR - Maximum Receive Rate
+- MLRsearch - Multiple Loss Ratio search, used in NDRPDR tests.
+- PLRsearch - Probabilistic Loss Ratio search, used in SOAK tests.
+- MRR - Maximum Receive Rate tests, the method based on FRMOL from RFC 2285.
Description of each test method is followed by generic test properties
shared by all methods.
-## MLRsearch Tests
+## NDRPDR Tests
+
+These tests employ MLRsearch to find two conditional throughput values.
+NDR for zero loss ratio goal and PDR for 0.5% loss ratio goal.
+
+### Algorithm Details
+
+See [MLRSearch]({{< ref "mlr_search/#MLRsearch" >}}) section for more detail.
+MLRsearch is being standardized in IETF in
+[draft-ietf-bmwg-mlrsearch](https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-mlrsearch-06).
### Description
-Multiple Loss Ratio search (MLRsearch) tests discover multiple packet
-throughput rates in a single search, reducing the overall test execution
-time compared to a binary search. Each rate is associated with a
-distinct Packet Loss Ratio (PLR) criteria. In FD.io CSIT two throughput
-rates are discovered: Non-Drop Rate (NDR, with zero packet loss, PLR=0)
-and Partial Drop Rate (PDR, with PLR<0.5%). MLRsearch is compliant with
-RFC2544.
+Multiple Loss Ratio search (MLRsearch) algorithm can discover multiple
+conditional throughputs in a single search,
+reducing the overall test execution time compared to a binary search.
+In FD.io CSIT, conditional throughputs are discovered for two search goals:
+Non-Drop Rate (NDR, zero loss ratio goal)
+and Partial Drop Rate (PDR, 0.5% loss ratio goal).
+Other inputs are common for both goals:
+Goal width is 0.5%, trial duration is 1 second, duration sum goal is 21 seconds
+and exceed ratio is 50%.
+
+The main algorithm expresses the conditional throughput based on one-port load.
+The results presented in CSIT show aggregate load,
+(the value from the search is doubled if the tests uses bidirectional traffic).
### Usage
@@ -44,39 +59,43 @@ environments. NDR and PDR packet and bandwidth throughput results for
all frame sizes and for all tests are presented in detailed results
tables.
-### Details
+## SOAK Tests
-See [MLRSearch]({{< ref "mlr_search/#MLRsearch" >}}) section for more detail.
-MLRsearch is being standardized in IETF in
-[draft-ietf-bmwg-mlrsearch](https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-mlrsearch-01).
+These tests employ PLRsearch to find a critical load value.
-## PLRsearch Tests
+### Algorithm Details
+
+See [PLRSearch]({{< ref "plr_search/#PLRsearch" >}}) methodology section for
+more detail. PLRsearch is being standardized in IETF in
+[draft-vpolak-bmwg-plrsearch](https://tools.ietf.org/html/draft-vpolak-bmwg-plrsearch).
### Description
Probabilistic Loss Ratio search (PLRsearch) tests discovers a packet
throughput rate associated with configured Packet Loss Ratio (PLR)
-criteria for tests run over an extended period of time a.k.a. soak
+target for tests run over an extended period of time a.k.a. soak
testing. PLRsearch assumes that system under test is probabilistic in
nature, and not deterministic.
### Usage
-PLRsearch are run to discover a sustained throughput for PLR=10^-7^
+PLRsearch are run to discover a critical load for PLR=10^-7^
(close to NDR) for VPP release covered by CSIT report. Results for small
frame sizes (64B/78B) are presented in packet throughput graphs (Box
Plots) for a small subset of baseline tests.
-Each soak test lasts 30 minutes and is executed at least twice. Results are
-compared against NDR and PDR rates discovered with MLRsearch.
+Each soak test lasts 30 minutes and is executed at least twice.
-### Details
+## MRR Tests
-See [PLRSearch]({{< ref "plr_search/#PLRsearch" >}}) methodology section for
-more detail. PLRsearch is being standardized in IETF in
-[draft-vpolak-bmwg-plrsearch](https://tools.ietf.org/html/draft-vpolak-bmwg-plrsearch).
+### Algorithm Details
-## MRR Tests
+See [MRR Throughput]({{< ref "mrr/#MRR" >}})
+section for more detail about MRR tests configuration.
+
+FD.io CSIT performance dashboard includes complete description of
+[daily performance trending tests]({{< ref "../../trending/analysis" >}})
+and [VPP per patch tests]({{< ref "../../per_patch_testing.md" >}}).
### Description
@@ -91,8 +110,8 @@ specified Ethernet frame size is set to the bi-directional link rate.
### Usage
-MRR tests are much faster than MLRsearch as they rely on a single trial
-or a small set of trials with very short duration. It is this property
+MRR tests are much faster than MLRsearch as they rely on
+a small set of trials with very short duration. It is this property
that makes them suitable for continuous execution in daily performance
trending jobs enabling detection of performance anomalies (regressions,
progressions) resulting from data plane code changes.
@@ -102,15 +121,6 @@ patch performance vs parent. CSIT reports include MRR throughput
comparisons between releases and test environments. Small frame sizes
only (64B/78B, IMIX).
-### Details
-
-See [MRR Throughput]({{< ref "mrr/#MRR" >}})
-section for more detail about MRR tests configuration.
-
-FD.io CSIT performance dashboard includes complete description of
-[daily performance trending tests]({{< ref "../../trending/analysis" >}})
-and [VPP per patch tests]({{< ref "../../per_patch_testing.md" >}}).
-
## Generic Test Properties
All data plane throughput test methodologies share following generic
diff --git a/docs/content/methodology/measurements/data_plane_throughput/plr_search.md b/docs/content/methodology/measurements/data_plane_throughput/plr_search.md
index 529bac1f7f..6f208c1ece 100644
--- a/docs/content/methodology/measurements/data_plane_throughput/plr_search.md
+++ b/docs/content/methodology/measurements/data_plane_throughput/plr_search.md
@@ -66,6 +66,9 @@ the algorithm is agnostic to flows and directions,
it only cares about aggregate counts of packets sent and packets lost),
but debug output from traffic generator lists unidirectional values.
+In CSIT, tests that employ PLRsearch are identified as SOAK tests,
+the search time is set to 30 minuts.
+
### Measurement Delay
In a sample implemenation in FD.io CSIT project, there is roughly 0.5
@@ -347,7 +350,7 @@ agrees that the interval should be wider than that.
#### Summary
-The two graphs show the behavior of PLRsearch algorithm applied to soaking test
+The two graphs show the behavior of PLRsearch algorithm applied to soak test
when some of PLRsearch assumptions do not hold:
+ L2 patch measurement results violate the assumption