diff options
Diffstat (limited to 'resources/libraries/robot/performance/performance_utils.robot')
| -rw-r--r-- | resources/libraries/robot/performance/performance_utils.robot | 158 |
1 files changed, 97 insertions, 61 deletions
diff --git a/resources/libraries/robot/performance/performance_utils.robot b/resources/libraries/robot/performance/performance_utils.robot index dcabbb9fac..84113709cf 100644 --- a/resources/libraries/robot/performance/performance_utils.robot +++ b/resources/libraries/robot/performance/performance_utils.robot @@ -50,7 +50,8 @@ | | ... | *Test (or broader scope) variables read:* | | ... | - traffic_profile - Name of module defining traffc for measurements. | | ... | Type: string -| | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: int or str +| | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: integer or +| | ... | string | | ... | - max_rate - Calculated unidirectional maximal transmit rate [pps]. | | ... | Type: float | | @@ -59,12 +60,16 @@ | | ... | - final_relative_width - Maximal width multiple of upper. Type: float | | ... | - final_trial_duration - Duration of final trials [s]. Type: float | | ... | - initial_trial_duration - Duration of initial trials [s]. Type: float -| | ... | - intermediate phases - Number of intermediate phases [1]. Type: int +| | ... | - intermediate phases - Number of intermediate phases [1]. +| | ... | Type: integer | | ... | - timeout - Fail if search duration is longer [s]. Type: float -| | ... | - doublings - How many doublings to do when expanding [1]. Type: int +| | ... | - doublings - How many doublings to do when expanding [1]. +| | ... | Type: integer | | ... | - traffic_directions - Bi- (2) or uni- (1) directional traffic. -| | ... | Type: int +| | ... | Type: integer | | ... | - latency_duration - Duration for latency-specific trials. Type: float +| | ... | - latency - False to disable latency measurement; default value: True. +| | ... | Type: boolean | | | | ... | *Example:* | | @@ -78,6 +83,7 @@ | | ... | ${number_of_intermediate_phases}=${2} | ${timeout}=${720.0} | | ... | ${doublings}=${2} | ${traffic_directions}=${2} | | ... | ${latency_duration}=${PERF_TRIAL_LATENCY_DURATION} +| | ... | ${latency}=${True} | | | | # Latency measurements will need more than 9000 pps. | | ${result} = | Perform optimized ndrpdr search | ${frame_size} @@ -86,6 +92,7 @@ | | ... | ${final_trial_duration} | ${initial_trial_duration} | | ... | ${number_of_intermediate_phases} | timeout=${timeout} | | ... | doublings=${doublings} | traffic_directions=${traffic_directions} +| | ... | latency=${latency} | | Display result of NDRPDR search | ${result} | | Check NDRPDR interval validity | ${result.pdr_interval} | | ... | ${packet_loss_ratio} @@ -95,18 +102,22 @@ | | ${ndr_sum}= | Set Variable | ${result.ndr_interval.measured_low.target_tr} | | ${ndr_per_stream}= | Evaluate | ${ndr_sum} / float(${traffic_directions}) | | ${rate}= | Evaluate | 0.9 * ${pdr_per_stream} -| | Measure and show latency at specified rate | Latency at 90% PDR: +| | Run Keyword If | ${latency} +| | ... | Measure and show latency at specified rate | Latency at 90% PDR: | | ... | ${latency_duration} | ${rate} | ${framesize} | | ... | ${traffic_profile} | ${traffic_directions} | | ${rate}= | Evaluate | 0.5 * ${pdr_per_stream} -| | Measure and show latency at specified rate | Latency at 50% PDR: +| | Run Keyword If | ${latency} +| | ... | Measure and show latency at specified rate | Latency at 50% PDR: | | ... | ${latency_duration} | ${rate} | ${framesize} | | ... | ${traffic_profile} | ${traffic_directions} | | ${rate}= | Evaluate | 0.1 * ${pdr_per_stream} -| | Measure and show latency at specified rate | Latency at 10% PDR: +| | Run Keyword If | ${latency} +| | ... | Measure and show latency at specified rate | Latency at 10% PDR: | | ... | ${latency_duration} | ${rate} | ${framesize} | | ... | ${traffic_profile} | ${traffic_directions} -| | Measure and show latency at specified rate | Latency at 0% PDR: +| | Run Keyword If | ${latency} +| | ... | Measure and show latency at specified rate | Latency at 0% PDR: | | ... | ${latency_duration} | ${0} | ${framesize} | | ... | ${traffic_profile} | ${traffic_directions} | | # Finally, trials with runtime and other stats. @@ -128,7 +139,8 @@ | | ... | *Test (or broader scope) variables read:* | | ... | - traffic_profile - Name of module defining traffc for measurements. | | ... | Type: string -| | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: int or str +| | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: integer or +| | ... | string | | ... | - max_rate - Calculated unidirectional maximal transmit rate [pps]. | | ... | Type: float | | @@ -137,11 +149,15 @@ | | ... | - final_relative_width - Maximal width multiple of upper. Type: float | | ... | - final_trial_duration - Duration of final trials [s]. Type: float | | ... | - initial_trial_duration - Duration of initial trials [s]. Type: float -| | ... | - intermediate phases - Number of intermediate phases [1]. Type: int +| | ... | - intermediate phases - Number of intermediate phases [1]. +| | ... | Type: integer | | ... | - timeout - Fail if search duration is longer [s]. Type: float -| | ... | - doublings - How many doublings to do when expanding [1]. Type: int +| | ... | - doublings - How many doublings to do when expanding [1]. +| | ... | Type: integer | | ... | - traffic_directions - Bi- (2) or uni- (1) directional traffic. -| | ... | Type: int +| | ... | Type: integer +| | ... | - latency - True to enable latency measurement; default value: False. +| | ... | Type: boolean | | | | ... | *Returns:* | | ... | - Lower bound for bi-directional throughput at given PLR. Type: float @@ -156,7 +172,7 @@ | | ... | ${final_relative_width}=${0.001} | ${final_trial_duration}=${10.0} | | ... | ${initial_trial_duration}=${1.0} | | ... | ${number_of_intermediate_phases}=${1} | ${timeout}=${720.0} -| | ... | ${doublings}=${2} | ${traffic_directions}=${2} +| | ... | ${doublings}=${2} | ${traffic_directions}=${2} | ${latency}=${False} | | | | ${result} = | Perform optimized ndrpdr search | ${frame_size} | | ... | ${traffic_profile} | ${10000} | ${max_rate} @@ -164,6 +180,7 @@ | | ... | ${final_trial_duration} | ${initial_trial_duration} | | ... | ${number_of_intermediate_phases} | timeout=${timeout} | | ... | doublings=${doublings} | traffic_directions=${traffic_directions} +| | ... | latency=${latency} | | Check NDRPDR interval validity | ${result.pdr_interval} | | ... | ${packet_loss_ratio} | | Return From Keyword | ${result.pdr_interval.measured_low.target_tr} @@ -181,7 +198,8 @@ | | ... | *Test (or broader scope) variables read:* | | ... | - traffic_profile - Name of module defining traffc for measurements. | | ... | Type: string -| | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: int or str +| | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: integer or +| | ... | string | | ... | - max_rate - Calculated unidirectional maximal transmit rate [pps]. | | ... | Type: float | | @@ -189,7 +207,9 @@ | | ... | - packet_loss_ratio - Accepted loss during search. Type: float | | ... | - timeout - Stop when search duration is longer [s]. Type: float | | ... | - traffic_directions - Bi- (2) or uni- (1) directional traffic. -| | ... | Type: int +| | ... | Type: integer +| | ... | - latency - True to enable latency measurement; default value: False. +| | ... | Type: boolean | | | | ... | *Example:* | | @@ -197,13 +217,13 @@ | | ... | \| \${2} \| | | | | [Arguments] | ${packet_loss_ratio}=${1e-7} | ${timeout}=${1800.0} -| | ... | ${traffic_directions}=${2} +| | ... | ${traffic_directions}=${2} | ${latency}=${False} | | | | ${min_rate} = | Set Variable | ${10000} | | ${average} | ${stdev} = | Perform soak search | ${frame_size} | | ... | ${traffic_profile} | ${min_rate} | ${max_rate} | | ... | ${packet_loss_ratio} | timeout=${timeout} -| | ... | traffic_directions=${traffic_directions} +| | ... | traffic_directions=${traffic_directions} | latency=${latency} | | ${lower} | ${upper} = | Display result of soak search | | ... | ${average} | ${stdev} | | Should Not Be True | 1.1 * ${traffic_directions} * ${min_rate} > ${lower} @@ -229,7 +249,7 @@ | | ... | *Example:* | | | | ... | \| Display single bound \| NDR lower bound \| \${12345.67} \ -| | ... | \| \${64} \| show_latency=\${EMPTY} \| +| | ... | \| \${64} \| latency=\${EMPTY} \| | | | | [Arguments] | ${text} | ${rate_total} | ${frame_size} | ${latency}=${EMPTY} | | @@ -271,7 +291,8 @@ | | ... | The given result should contain latency data as well. | | | | ... | *Test (or broader scope) variables read:* -| | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: int or str +| | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: integer or +| | ... | string | | ... | *Arguments:* | | ... | - result - Measured result data per stream [pps]. Type: NdrPdrResult | | @@ -304,7 +325,8 @@ | | ... | (Throughput * (L2 Frame Size + IPG) * 8) | | | | ... | *Test (or broader scope) variables read:* -| | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: int or str +| | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: integer or +| | ... | string | | ... | *Arguments:* | | ... | - avg - Estimated average critical load [pps]. Type: float | | ... | - stdev - Standard deviation of critical load [pps]. Type: float @@ -360,36 +382,36 @@ | | ... | Send traffic at maximum rate. | | | | ... | *Test (or broader scope) variables read:* -| | ... | - traffic_profile - Name of module defining traffc for measurements. +| | ... | - traffic_profile - Name of module defining traffic for measurements. | | ... | Type: string -| | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: int or str +| | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: integer or +| | ... | string | | ... | - max_rate - Calculated unidirectional maximal transmit rate [pps]. | | ... | Type: float | | | | ... | *Arguments:* -| | ... | - trial_duration - Duration of single trial [s]. -| | ... | Type: float -| | ... | - fail_no_traffic - Whether to fail on zero receive count. -| | ... | Type: boolean +| | ... | - trial_duration - Duration of single trial [s]. Type: float +| | ... | - fail_no_traffic - Whether to fail on zero receive count; +| | ... | default value: True. Type: boolean | | ... | - trial_multiplicity - How many trials in this measurement. -| | ... | Type: int -| | ... | - traffic_directions - Bi- (2) or uni- (1) directional traffic. -| | ... | Type: int -| | ... | - tx_port - TX port of TG, default 0. -| | ... | Type: integer -| | ... | - rx_port - RX port of TG, default 1. | | ... | Type: integer +| | ... | - traffic_directions - Bi- (2) or uni- (1) directional traffic; +| | ... | default value: 2. Type: integer +| | ... | - tx_port - TX port of TG; default value: 0. Type: integer +| | ... | - rx_port - RX port of TG; default value: 1. Type: integer +| | ... | - latency - True to enable latency measurement; default value: False. +| | ... | Type: boolean | | | | ... | *Example:* | | -| | ... | \| Traffic should pass with maximum rate \| \${1} \| \${10.0} \ -| | ... | \| \${False} \| \${2} \| \${0} \| \${1} \| +| | ... | \| Traffic should pass with maximum rate \| \${1} \| \${False} \ +| | ... | \| \${10.0} \| \${2} \| \${0} \| \${1} \| \${True} \| | | | | [Arguments] | ${trial_duration}=${trial_duration} | | ... | ${fail_no_traffic}=${True} | | ... | ${trial_multiplicity}=${trial_multiplicity} | | ... | ${traffic_directions}=${2} | ${tx_port}=${0} | ${rx_port}=${1} -| | ... | ${latency}=${True} +| | ... | ${latency}=${False} | | | | ${results}= | Send traffic at specified rate | | ... | ${trial_duration} | ${max_rate} | ${frame_size} @@ -411,35 +433,39 @@ | | ... | - trial_duration - Duration of single trial [s]. Type: float | | ... | - rate - Target aggregate transmit rate [pps] / Connections per second | | ... | (CPS) for UDP/TCP flows. Type: float -| | ... | - frame_size - L2 Frame Size [B]. Type: integer/string +| | ... | - frame_size - L2 Frame Size [B]. Type: integer or string | | ... | - traffic_profile - Name of module defining traffc for measurements. | | ... | Type: string | | ... | - trial_multiplicity - How many trials in this measurement. -| | ... | Type: int -| | ... | - traffic_directions - Bi- (2) or uni- (1) directional traffic. -| | ... | Type: int -| | ... | - tx_port - TX port of TG, default 0. | | ... | Type: integer -| | ... | - rx_port - RX port of TG, default 1. +| | ... | - traffic_directions - Bi- (2) or uni- (1) directional traffic. | | ... | Type: integer -| | ... | - extended_debug- True to enable extended debug. +| | ... | - tx_port - TX port of TG; default value: 0. Type: integer +| | ... | - rx_port - RX port of TG; default value: 1. Type: integer +| | ... | - extended_debug - True to enable extended debug. +| | ... | Type: boolean +| | ... | - latency - True to enable latency measurement; default value: False. | | ... | Type: boolean | | | | ... | *Example:* | | | | ... | \| Send traffic at specified rate \| \${1.0} \| ${4000000.0} \ | | ... | \| \${64} \| 3-node-IPv4 \| \${10} \| \${2} \| \${0} \| \${1} \ -| | ... | \${False} \| +| | ... | \| ${False} \| ${True} \| | | | | [Arguments] | ${trial_duration} | ${rate} | ${frame_size} | | ... | ${traffic_profile} | ${trial_multiplicity}=${trial_multiplicity} | | ... | ${traffic_directions}=${2} | ${tx_port}=${0} | ${rx_port}=${1} -| | ... | ${extended_debug}=${extended_debug} | ${latency}=${True} +| | ... | ${extended_debug}=${extended_debug} | ${latency}=${False} | | | | Set Test Variable | ${extended_debug} -| | Clear and show runtime counters with running traffic | ${trial_duration} -| | ... | ${rate} | ${frame_size} | ${traffic_profile} -| | ... | ${traffic_directions} | ${tx_port} | ${rx_port} +| | # Following setting of test variables is needed as +| | # "Clear and show runtime counters with running traffic" has been moved to +| | # pre_stats actions. +| | Set Test Variable | ${rate} +| | Set Test Variable | ${traffic_directions} +| | Set Test Variable | ${tx_port} +| | Set Test Variable | ${rx_port} | | FOR | ${action} | IN | @{pre_stats} | | | Run Keyword | Additional Statistics Action For ${action} | | END @@ -473,15 +499,15 @@ | | ... | - rate - Rate [pps] for sending packets in case of T-Rex stateless | | ... | mode or multiplier of profile CPS in case of T-Rex astf mode. | | ... | Type: float -| | ... | - frame_size - L2 Frame Size [B]. Type: integer/string +| | ... | - frame_size - L2 Frame Size [B]. Type: integer or string | | ... | - traffic_profile - Name of module defining traffic for measurements. | | ... | Type: string | | ... | - traffic_directions - Bi- (2) or uni- (1) directional traffic. -| | ... | Type: int -| | ... | - tx_port - TX port of TG, default 0. Type: integer -| | ... | - rx_port - RX port of TG, default 1. Type: integer +| | ... | Type: integer +| | ... | - tx_port - TX port of TG; default value: 0. Type: integer +| | ... | - rx_port - RX port of TG; default value: 1. Type: integer | | ... | - safe_rate - To apply if rate is below this, as latency pps is fixed. -| | ... | In pps, type int. +| | ... | In pps. Type: integer. | | | | ... | *Example:* | | @@ -514,13 +540,13 @@ | | ... | - rate - Rate [pps] for sending packets in case of T-Rex stateless | | ... | mode or multiplier of profile CPS in case of T-Rex astf mode. | | ... | Type: float -| | ... | - frame_size - L2 Frame Size [B] or IMIX_v4_1. Type: integer/string +| | ... | - frame_size - L2 Frame Size [B] or IMIX_v4_1. Type: integer or string | | ... | - traffic_profile - Name of module defining traffc for measurements. | | ... | Type: string | | ... | - traffic_directions - Bi- (2) or uni- (1) directional traffic. -| | ... | Type: int -| | ... | - tx_port - TX port of TG, default 0. Type: integer -| | ... | - rx_port - RX port of TG, default 1. Type: integer +| | ... | Type: integer +| | ... | - tx_port - TX port of TG; default value: 0. Type: integer +| | ... | - rx_port - RX port of TG, default value: 1. Type: integer | | | | ... | *Example:* | | @@ -553,15 +579,16 @@ | | ... | *Test (or broader scope) variables read:* | | ... | - traffic_profile - Name of module defining traffc for measurements. | | ... | Type: string -| | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: int or str +| | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: integer or +| | ... | string | | ... | *Arguments:* | | ... | - rate - Rate [pps] for sending packets in case of T-Rex stateless | | ... | mode or multiplier of profile CPS in case of T-Rex astf mode. | | ... | Type: float | | ... | - traffic_directions - Bi- (2) or uni- (1) directional traffic. -| | ... | Type: int -| | ... | - tx_port - TX port of TG, default 0. Type: integer -| | ... | - rx_port - RX port of TG, default 1. Type: integer +| | ... | Type: integer +| | ... | - tx_port - TX port of TG; default value: 0. Type: integer +| | ... | - rx_port - RX port of TG; default value: 1. Type: integer | | | | ... | *Example:* | | @@ -647,4 +674,13 @@ | | ... | Additional Statistics Action for bash command "perf stat". | | | | Run Keyword If | ${extended_debug}==${True} -| | ... | Perf Stat On All DUTs | ${nodes}
\ No newline at end of file +| | ... | Perf Stat On All DUTs | ${nodes} + +| Additional Statistics Action For vpp-clear-show-runtime-with-traffic +| | [Documentation] +| | ... | Additional Statistics Action for clear and show runtime counters with +| | ... | running traffic. +| | +| | Clear and show runtime counters with running traffic +| | ... | ${trial_duration} | ${rate} | ${frame_size} | ${traffic_profile} +| | ... | ${traffic_directions} | ${tx_port} | ${rx_port} |