From 675b7d448ede5f954e97ce412b03f78b40a36a2b Mon Sep 17 00:00:00 2001 From: Vratko Polak Date: Thu, 26 Oct 2023 13:49:47 +0200 Subject: fix(methodology): MLRv9 updates + Also update reconf tests emthodology. Change-Id: If3b19d3d9f14da6b28e525e9023bccde1cf1c8a9 Signed-off-by: Vratko Polak --- .../measurements/data_plane_throughput/mlr_search.md | 12 ++++++++---- docs/content/methodology/test/reconfiguration.md | 4 +++- 2 files changed, 11 insertions(+), 5 deletions(-) diff --git a/docs/content/methodology/measurements/data_plane_throughput/mlr_search.md b/docs/content/methodology/measurements/data_plane_throughput/mlr_search.md index 72e68a5fc7..71e4471905 100644 --- a/docs/content/methodology/measurements/data_plane_throughput/mlr_search.md +++ b/docs/content/methodology/measurements/data_plane_throughput/mlr_search.md @@ -15,14 +15,16 @@ Two loss ratio goals are of interest in FD.io CSIT, leading to Non-Drop Rate (NDR, loss ratio goal is exact zero) and Partial Drop Rate (PDR, 0.5% loss ratio goal). Instead of a single long trial, a sequence of short (1s) trials is done. -Thus, instead of final trial duration, a duration sum (20s) is prescribed. +Thus, instead of final trial duration, a duration sum (21s) is prescribed. This allows the algorithm to make a decision sooner, when the results are quite one-sided. Also, only one half of the trial results is required to meet the loss ratio requirement, making the conditional throughput more stable. -The conditional throughput in this case is the median forwarding rate -among all full-length trials (including imaginary missing ones with high loss) -at the relevant lower bound intended load. +The conditional throughput in this case is in principle the median forwarding rate +among all trials at the relevant lower bound intended load. +In practice, the search stops when missing trial results cannot +disprove the load as a lower bound, so conditional throughput +is the worst forwarding rate among the measured good trials. MLRsearch discovers all the loads in single search, reducing required time duration compared to separate `binary search`es[^1] for each rate. Overall @@ -62,6 +64,8 @@ For every loss ratio goal, the relevant upper and lower bound Exit condition is given by that interval reaching low enough relative width. Small enough width is achieved by bisecting the current interval. The bisection can be uneven, to save measurements based on information theory. +The width value is 0.5%, the same as PDR goal loss ratio, +as smaller values may report PDR conditional throughput smaller than NDR. Switching to higher trial duration sum generally requires additional trials at a load from previous duration sum target. diff --git a/docs/content/methodology/test/reconfiguration.md b/docs/content/methodology/test/reconfiguration.md index 6dec4d918b..a0678103d9 100644 --- a/docs/content/methodology/test/reconfiguration.md +++ b/docs/content/methodology/test/reconfiguration.md @@ -32,7 +32,9 @@ the loss count divided by the offered load. Each reconf suite is based on a similar MLRsearch performance suite. MLRsearch parameters are changed to speed up the throughput discovery. -For example, PDR is not searched for, and the final trial duration is shorter. +For example, PDR is not searched for, and the goal duration sum is shorter. +Contrary to usual NDR, exceed ratio is set to zero in reconf tests +as load stability is less important than attribution of losses. The MLRsearch suite has to contain a configuration parameter that can be scaled up, e.g. number of tunnels or number of service chains. -- cgit 1.2.3-korg