feat(MLRsearch): MLRsearch v7

Replaces MLRv2, suitable for "big bang" upgrade across CSIT.
PyPI metadata updated only partially (full edits will come separately).
Pylint wants less complexity, but the differences are only minor.

+ Use the same (new CSIT) defaults everywhere, also in Python library.
+ Update also PLRsearch to use the new result class.
+ Make upper bound optional in UTI.
+ Fix ASTF approximate duration detection.
+ Do not keep approximated_receive_rate (for MRR) in result structure.

Change-Id: I03406f32d5c93f56b527cb3f93791b61955dfd74
Signed-off-by: Vratko Polak <vrpolak@cisco.com>

2 files changed, 56 insertions, 95 deletions

diff --git a/resources/libraries/robot/performance/performance_display.robot b/resources/libraries/robot/performance/performance_display.robot
index d230bcb35a..791a45db62 100644
--- a/resources/libraries/robot/performance/performance_display.robot
+++ b/resources/libraries/robot/performance/performance_display.robot
@@ -1,4 +1,4 @@
-# Copyright (c) 2022 Cisco and/or its affiliates.
+# Copyright (c) 2023 Cisco and/or its affiliates.
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at:
@@ -17,35 +17,6 @@
 | ... | This includes checks to fail test.
 
 *** Keywords ***
-| Check NDRPDR interval validity
-| | [Documentation]
-| | ... | Extract loss ratio of lower bound of the interval.
-| | ... | Fail if it does not reach the allowed value.
-| |
-| | ... | *Arguments:*
-| | ... | - interval - Measured interval. Type: ReceiveRateInterval
-| | ... | - packet_loss_ratio - Accepted loss (0.0 for NDR). Type: float
-| |
-| | ... | *Example:*
-| |
-| | ... | \| Check NDRPDR interval validity \| \${result.pdr_interval} \
-| | ... | \| \${0.005} \|
-| |
-| | [Arguments] | ${interval} | ${packet_loss_ratio}=${0.0}
-| |
-| | ${lower_bound} = | Set Variable | ${interval.measured_low}
-| | ${lower_bound_lr} = | Set Variable | ${lower_bound.loss_ratio}
-| | Return From Keyword If | ${lower_bound_lr} <= ${packet_loss_ratio}
-| | Set Test Variable | \${rate_for_teardown} | ${lower_bound.target_tr}
-| | ${message}= | Catenate | SEPARATOR=${SPACE}
-| | ... | Minimal rate loss ratio ${lower_bound_lr}
-| | ... | does not reach target ${packet_loss_ratio}.
-| | ${message_zero} = | Set Variable | Zero packets forwarded!
-| | ${message_other} = | Set Variable | ${lower_bound.loss_count} packets lost.
-| | ${message} = | Set Variable If | ${lower_bound_lr} >= 1.0
-| | ... | ${message}${\n}${message_zero} | ${message}${\n}${message_other}
-| | Fail | ${message}
-
 | Compute Bandwidth
 | | [Documentation]
 | | ... | Compute (bidir) bandwidth from given (unidir) transaction rate.
@@ -77,8 +48,8 @@
 | | ... | due to reconfiguration under traffic.
 | |
 | | ... | *Arguments:*
-| | ... | - result - Result of bidirectional measurement.
-| | ... | Type: ReceiveRateMeasurement
+| | ... | - result - Result of MLRsearch invocation for one search goal.
+| | ... | Type: StatInterval
 | |
 | | ... | *Example:*
 | |
@@ -86,7 +57,7 @@
 | |
 | | [Arguments] | ${result}
 | |
-| | ${bandwidth} | ${packet_rate}= | Compute Bandwidth | ${result.target_tr}
+| | ${bandwidth} | ${packet_rate}= | Compute Bandwidth | ${result.intended_load}
 | | ${packet_loss} = | Set Variable | ${result.loss_count}
 | | ${time_loss} = | Evaluate | ${packet_loss} / ${packet_rate}
 | | Set Test Message | Packets lost due to reconfig: ${packet_loss}
@@ -95,14 +66,11 @@
 
 | Display result of NDRPDR search
 | | [Documentation]
-| | ... | Display result of NDR+PDR search, both quantities, both bounds,
-| | ... | aggregated, in units given by trasaction type, e.g. by default
-| | ... | in packet per seconds and Gbps total bandwidth
+| | ... | Display result of NDR+PDR search, both quantities, aggregated,
+| | ... | conditional throughput only, in units given by trasaction type,
+| | ... | e.g. by default in packet per seconds and Gbps total bandwidth
 | | ... | (for initial packet size).
-| | ... |
-| | ... | The bound to display is encoded as target rate, it is assumed
-| | ... | it is in transactions per second. Bidirectional traffic
-| | ... | transaction is understood as having 2 packets, for this purpose.
+| | ... | The lower bounds in the result are assumed to be valid.
 | | ... |
 | | ... | Througput is calculated as:
 | | ... | Sum of measured rate over streams
@@ -115,8 +83,8 @@
 | | ... | - transaction_type - String identifier to determine how to count
 | | ... | transactions. Default is "packet".
 | | ... | *Arguments:*
-| | ... | - result - Measured result data. Aggregated rate, tps or pps.
-| | ... | Type: NdrPdrResult
+| | ... | - result - Measured result data Tps. Type: List[TrimmedStat]
+| | ... | *Returns:* NDR and PDR: Unidirectional intended load as tps float.
 | |
 | | ... | *Example:*
 | |
@@ -124,21 +92,18 @@
 | |
 | | [Arguments] | ${result}
 | |
-| | Display single bound | NDR_LOWER
-| | ... | ${result[0].measured_low.target_tr}
-| | ... | ${result[0].measured_low.latency}
-| | Display single bound | NDR_UPPER
-| | ... | ${result[0].measured_high.target_tr}
-| | Display single bound | PDR_LOWER
-| | ... | ${result[1].measured_low.target_tr}
-| | ... | ${result[1].measured_low.latency}
-| | Display single bound | PDR_UPPER
-| | ... | ${result[1].measured_high.target_tr}
+| | ${ndr} = | Convert To Number | ${result[0]}
+| | ${pdr} = | Convert To Number | ${result[1]}
+| | Display single bound | NDR | ${result[0].conditional_throughput}
+| | Display single bound | PDR | ${result[1].conditional_throughput}
+| | Return From Keyword | ${ndr} | ${pdr}
 
 | Display result of soak search
 | | [Documentation]
 | | ... | Display result of soak search, avg+-stdev, as upper/lower bounds.
 | | ... | See Display single bound for units used.
+| | ... | The displayed values are bidirectional, based on conditional
+| | ... | throughput. The returned
 | |
 | | ... | *Test (or broader scope) variables read:*
 | | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: integer or
@@ -148,7 +113,6 @@
 | | ... | *Arguments:*
 | | ... | - avg - Estimated average critical load [pps]. Type: float
 | | ... | - stdev - Standard deviation of critical load [pps]. Type: float
-| |
 | | ... | *Returns:*
 | | ... | - Lower and upper bound of critical load [pps]. Type: 2-tuple of float
 | |
@@ -169,14 +133,14 @@
 | Display single bound
 | | [Documentation]
 | | ... | Compute and display one bound of NDR+PDR (or soak) search result.
-| | ... | If the latency string is present, it is displayed as well.
 | | ... |
 | | ... | The bound to display is given as target transfer rate, it is assumed
-| | ... | it is in transactions per second. Bidirectional traffic
+| | ... | valid and in transactions per second. Bidirectional traffic
 | | ... | transaction is understood as having 2 packets, for this purpose.
 | | ... |
 | | ... | Pps values are aggregated, in packet per seconds
 | | ... | and Gbps total bandwidth (for initial packet size).
+| | ... | If the latency string is present, it is displayed as well.
 | | ... |
 | | ... | Througput is calculated as:
 | | ... | Sum of measured rate over streams
@@ -189,20 +153,22 @@
 | | ... | transactions. Default is "packet".
 | | ... | *Arguments:*
 | | ... | - text - Flavor text describing which bound is this. Type: string
-| | ... | - tps - Transaction rate [tps]. Type: float
+| | ... | - tps - Conditional throughput [tps]. Type: Union[float, DiscreteLoad]
 | | ... | - latency - Latency data to display if non-empty. Type: string
 | |
 | | ... | *Example:*
 | |
-| | ... | \| Display single bound \| NDR lower bound \| \${12345.67} \
+| | ... | \| Display single bound \| NDR \| \${12345.67} \
 | | ... | \| latency=\${EMPTY} \|
 | |
 | | [Arguments] | ${text} | ${tps} | ${latency}=${EMPTY}
 | |
+| | ${tps} = | Convert To Number | ${tps}
 | | ${transaction_type} = | Get Transaction Type
 | | Run Keyword And Return If | """_cps""" in """${transaction_type}"""
 | | ... | Display Single CPS Bound | ${text} | ${tps} | ${latency}
-| | Display Single PPS Bound | ${text} | ${tps} | ${latency}
+| | Run Keyword And Return
+| | ... | Display Single PPS Bound | ${text} | ${tps} | ${latency}
 
 | Display Single CPS Bound
 | | [Documentation]
@@ -225,9 +191,8 @@
 | | Set Test Message | ${\n}${text}: ${tps} CPS | append=yes
 | | ${bandwidth} | ${pps} = | Compute Bandwidth | ${tps}
 | | Export Search Bound | ${text} | ${tps} | cps | ${bandwidth * 1e9}
-| | Return From Keyword If | not """${latency}"""
-| | Set Test Message | ${\n}LATENCY [min/avg/max/hdrh] per stream: ${latency}
-| | ... | append=yes
+| | Run Keyword If | """${latency}""" | Set Test Message
+| | ... | ${\n}LATENCY [min/avg/max/hdrh] per stream: ${latency} | append=yes
 
 | Display Single PPS Bound
 | | [Documentation]
@@ -261,6 +226,5 @@
 | | Set Test Message | ${\n}${text}: ${pps} pps, | append=yes
 | | Set Test Message | ${bandwidth} Gbps (initial) | append=yes
 | | Export Search Bound | ${text} | ${pps} | pps | ${bandwidth * 1e9}
-| | Return From Keyword If | not """${latency}"""
-| | Set Test Message | ${\n}LATENCY [min/avg/max/hdrh] per stream: ${latency}
-| | ... | append=yes
+| | Run Keyword If | """${latency}""" | Set Test Message
+| | ... | ${\n}LATENCY [min/avg/max/hdrh] per stream: ${latency} | append=yes
diff --git a/resources/libraries/robot/performance/performance_utils.robot b/resources/libraries/robot/performance/performance_utils.robot
index 78a1fcc0f9..9392bc0e8c 100644
--- a/resources/libraries/robot/performance/performance_utils.robot
+++ b/resources/libraries/robot/performance/performance_utils.robot
@@ -93,8 +93,8 @@
 | | ${average} | ${stdev} = | Perform soak search
 | | ... | frame_size=${frame_size}
 | | ... | traffic_profile=${traffic_profile}
-| | ... | minimum_transmit_rate=${min_rate_soft}
-| | ... | maximum_transmit_rate=${max_rate}
+| | ... | min_load=${min_rate_soft}
+| | ... | max_load=${max_rate}
 | | ... | plr_target=${1e-7}
 | | ... | tdpt=${0.1}
 | | ... | initial_count=${50}
@@ -142,7 +142,7 @@
 | | ... | - traffic_profile - Name of module defining traffc for measurements.
 | | ... | Type: string
 | | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: integer or
-| | ... | string
+| | ... | string.
 | | ... | - max_rate - Calculated maximal unidirectional transmit rate [tps].
 | | ... | Type: float
 | | ... | - resetter - Callable to reset DUT state before each trial.
@@ -160,8 +160,7 @@
 | | ${disable_latency} = | Get Disable Latency
 | | ${max_rate} = | Get Max Rate
 | | ${min_rate_soft} = | Get Min Rate Soft
-| | # \${packet_loss_ratio} is used twice so it is worth a variable.
-| | ${packet_loss_ratio} = | Get Packet Loss Ratio
+| | ${loss_ratio} = | Get Packet Loss Ratio
 | | ${ppta} = | Get Packets Per Transaction Aggregated
 | | ${ramp_up_duration} = | Get Ramp Up Duration
 | | ${ramp_up_rate} = | Get Ramp Up Rate
@@ -171,17 +170,18 @@
 | | ${transaction_scale} = | Get Transaction Scale
 | | ${transaction_type} = | Get Transaction Type
 | | ${use_latency} = | Get Use Latency
-| | ${result} = | Perform optimized ndrpdr search
+| | ${result} = | Perform MLR Search
 | | ... | frame_size=${frame_size}
 | | ... | traffic_profile=${traffic_profile}
-| | ... | minimum_transmit_rate=${min_rate_soft}
-| | ... | maximum_transmit_rate=${max_rate}
-| | ... | packet_loss_ratio=${packet_loss_ratio}
-| | ... | final_relative_width=${0.005}
-| | ... | final_trial_duration=${30.0}
+| | ... | min_load=${min_rate_soft}
+| | ... | max_load=${max_rate}
+| | ... | loss_ratio=${loss_ratio}
+| | ... | relative_width=${0.005}
 | | ... | initial_trial_duration=${1.0}
-| | ... | number_of_intermediate_phases=${2}
-| | ... | timeout=${1200.0}
+| | ... | final_trial_duration=${1.0}
+| | ... | duration_sum=${20.0}
+| | ... | preceding_targets=${2}
+| | ... | search_duration_max=${1200.0}
 | | ... | ppta=${ppta}
 | | ... | resetter=${resetter}
 | | ... | traffic_directions=${traffic_directions}
@@ -191,13 +191,8 @@
 | | ... | use_latency=${use_latency}
 | | ... | ramp_up_duration=${ramp_up_duration}
 | | ... | ramp_up_rate=${ramp_up_rate}
-| | Display result of NDRPDR search | ${result}
-| | Check NDRPDR interval validity | ${result[1]}
-| | ... | ${packet_loss_ratio}
-| | Check NDRPDR interval validity | ${result[0]}
-| | ${pdr} = | Set Variable | ${result[1].measured_low.target_tr}
-| | ${ndr} = | Set Variable | ${result[0].measured_low.target_tr}
-| | # We expect NDR and PDR to have different-looking stats.
+| | ${ndr} | ${pdr} = | Display result of NDRPDR search | ${result}
+| | # We expect NDR and PDR to have different-looking telemetry.
 | | Set Test Variable | ${telemetry_rate} | pdr
 | | Set Test Variable | ${telemetry_export} | ${True}
 | | Send traffic at specified rate
@@ -262,17 +257,18 @@
 | | ${transaction_scale} = | Get Transaction Scale
 | | ${transaction_type} = | Get Transaction Type
 | | ${use_latency} = | Get Use Latency
-| | ${result} = | Perform optimized ndrpdr search
+| | ${result} = | Perform MLR Search
 | | ... | frame_size=${frame_size}
 | | ... | traffic_profile=${traffic_profile}
-| | ... | minimum_transmit_rate=${min_rate_soft}
-| | ... | maximum_transmit_rate=${max_rate}
-| | ... | packet_loss_ratio=${0.0}
-| | ... | final_relative_width=${0.001}
-| | ... | final_trial_duration=${10.0}
+| | ... | min_load=${min_rate_soft}
+| | ... | max_load=${max_rate}
+| | ... | loss_ratio=${0.0}
+| | ... | relative_width=${0.001}
 | | ... | initial_trial_duration=${1.0}
-| | ... | number_of_intermediate_phases=${1}
-| | ... | timeout=${1200}
+| | ... | final_trial_duration=${1.0}
+| | ... | duration_sum=${10.0}
+| | ... | preceding_targets=${1}
+| | ... | search_duration_max=${1200}
 | | ... | ppta=${ppta}
 | | ... | resetter=${resetter}
 | | ... | traffic_directions=${traffic_directions}
@@ -282,8 +278,8 @@
 | | ... | use_latency=${use_latency}
 | | ... | ramp_up_duration=${ramp_up_duration}
 | | ... | ramp_up_rate=${ramp_up_rate}
-| | Check NDRPDR interval validity | ${result[0]}
-| | Return From Keyword | ${result[0].measured_low.target_tr}
+| | ${ret} = | Convert To Number | ${result[0]}
+| | Return From Keyword | ${ret}
 
 | Measure and show latency at specified rate
 | | [Documentation]
@@ -450,7 +446,8 @@
 | | | ... | ramp_up_rate=${ramp_up_rate}
 | | | # Out of several quantities for aborted traffic (duration stretching),
 | | | # the approximated receive rate is the best estimate we have.
-| | | ${value} = | Set Variable | ${result.approximated_receive_rate}
+| | | ${value} = | Set Variable | ${result.forwarding_count}
+| | | ${value} = | Evaluate | ${value} / ${result.offered_duration}
 | | | ${bandwidth} | ${pps} = | Compute Bandwidth | ${value} / ${ppta}
 | | | Append Mrr Value | ${value} | ${export_mrr_unit} | ${bandwidth * 1e9}
 | | | Append To List | ${results} | ${value}