diff options
Diffstat (limited to 'resources/libraries/robot/performance')
4 files changed, 605 insertions, 337 deletions
diff --git a/resources/libraries/robot/performance/performance_actions.robot b/resources/libraries/robot/performance/performance_actions.robot index c28b2ebdc8..1d2ea52274 100644 --- a/resources/libraries/robot/performance/performance_actions.robot +++ b/resources/libraries/robot/performance/performance_actions.robot @@ -1,4 +1,4 @@ -# Copyright (c) 2021 Cisco and/or its affiliates. +# Copyright (c) 2023 Cisco and/or its affiliates. # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at: @@ -26,23 +26,177 @@ | | Run Keyword If | ${extended_debug}==${True} | | ... | Perf Stat On All DUTs | ${nodes} | cpu_list=${cpu_alloc_str} -| Additional Statistics Action For clear-show-runtime-with-traffic +| Additional Statistics Action For trex-runtime +| | [Documentation] +| | ... | Additional Statistics Action for T-Rex telemetry counters with +| | ... | running traffic. +| | +| | ... | See documentation of the called keyword for required test variables. +| | +| | ${ppta} = | Get Packets Per Transaction Aggregated +| | ${ramp_up_duration} = | Get Ramp Up Duration +| | ${ramp_up_rate} = | Get Ramp Up Rate +| | ${runtime_duration} = | Get Runtime Duration +| | ${runtime_rate} = | Get Runtime Rate +| | ${traffic_directions} = | Get Traffic Directions +| | ${transaction_duration} = | Get Transaction Duration +| | ${transaction_scale} = | Get Transaction Scale +| | ${transaction_type} = | Get Transaction Type +| | ${use_latency} = | Get Use Latency +| | Send traffic on tg +| | ... | duration=${-1} +| | ... | rate=${runtime_rate} +| | ... | frame_size=${frame_size} +| | ... | traffic_profile=${traffic_profile} +| | ... | async_call=${True} +| | ... | ppta=${ppta} +| | ... | use_latency=${use_latency} +| | ... | traffic_directions=${traffic_directions} +| | ... | transaction_duration=${transaction_duration} +| | ... | transaction_scale=${transaction_scale} +| | ... | transaction_type=${transaction_type} +| | ... | duration_limit=${0.0} +| | ... | ramp_up_duration=${ramp_up_duration} +| | ... | ramp_up_rate=${ramp_up_rate} +| | Sleep | 1s +| | Stop traffic on tg + +| Additional Statistics Action For vpp-runtime | | [Documentation] | | ... | Additional Statistics Action for clear and show runtime counters with | | ... | running traffic. | | | | ... | See documentation of the called keyword for required test variables. | | -| | Clear and show runtime counters with running traffic +| | ${ppta} = | Get Packets Per Transaction Aggregated +| | ${ramp_up_duration} = | Get Ramp Up Duration +| | ${ramp_up_rate} = | Get Ramp Up Rate +| | ${runtime_duration} = | Get Runtime Duration +| | ${runtime_rate} = | Get Runtime Rate +| | ${traffic_directions} = | Get Traffic Directions +| | ${transaction_duration} = | Get Transaction Duration +| | ${transaction_scale} = | Get Transaction Scale +| | ${transaction_type} = | Get Transaction Type +| | ${use_latency} = | Get Use Latency +| | Send traffic on tg +| | ... | duration=${-1} +| | ... | rate=${runtime_rate} +| | ... | frame_size=${frame_size} +| | ... | traffic_profile=${traffic_profile} +| | ... | async_call=${True} +| | ... | ppta=${ppta} +| | ... | use_latency=${use_latency} +| | ... | traffic_directions=${traffic_directions} +| | ... | transaction_duration=${transaction_duration} +| | ... | transaction_scale=${transaction_scale} +| | ... | transaction_type=${transaction_type} +| | ... | duration_limit=${0.0} +| | ... | ramp_up_duration=${ramp_up_duration} +| | ... | ramp_up_rate=${ramp_up_rate} +| | Run Telemetry On All DUTs +| | ... | ${nodes} | profile=vppctl_runtime.yaml +| | ... | rate=${telemetry_rate} | export=${telemetry_export} +| | Stop traffic on tg + +| Additional Statistics Action For bpf-runtime +| | [Documentation] +| | ... | Additional Statistics Action for linux bundle counters with +| | ... | running traffic. +| | +| | ... | See documentation of the called keyword for required test variables. +| | +| | ${ppta} = | Get Packets Per Transaction Aggregated +| | ${ramp_up_duration} = | Get Ramp Up Duration +| | ${ramp_up_rate} = | Get Ramp Up Rate +| | ${runtime_duration} = | Get Runtime Duration +| | ${runtime_rate} = | Get Runtime Rate +| | ${traffic_directions} = | Get Traffic Directions +| | ${transaction_duration} = | Get Transaction Duration +| | ${transaction_scale} = | Get Transaction Scale +| | ${transaction_type} = | Get Transaction Type +| | ${use_latency} = | Get Use Latency +| | Send traffic on tg +| | ... | duration=${-1} +| | ... | rate=${runtime_rate} +| | ... | frame_size=${frame_size} +| | ... | traffic_profile=${traffic_profile} +| | ... | async_call=${True} +| | ... | ppta=${ppta} +| | ... | use_latency=${use_latency} +| | ... | traffic_directions=${traffic_directions} +| | ... | transaction_duration=${transaction_duration} +| | ... | transaction_scale=${transaction_scale} +| | ... | transaction_type=${transaction_type} +| | ... | duration_limit=${0.0} +| | ... | ramp_up_duration=${ramp_up_duration} +| | ... | ramp_up_rate=${ramp_up_rate} +| | Run Telemetry On All DUTs +| | ... | ${nodes} | profile=bpf_runtime.yaml +| | ... | rate=${telemetry_rate} | export=${False} +| | Stop traffic on tg + +| Additional Statistics Action For perf-stat-runtime +| | [Documentation] +| | ... | Additional Statistics Action for linux bundle counters with +| | ... | running traffic. +| | +| | ... | See documentation of the called keyword for required test variables. +| | +| | ${ppta} = | Get Packets Per Transaction Aggregated +| | ${ramp_up_duration} = | Get Ramp Up Duration +| | ${ramp_up_rate} = | Get Ramp Up Rate +| | ${runtime_duration} = | Get Runtime Duration +| | ${runtime_rate} = | Get Runtime Rate +| | ${traffic_directions} = | Get Traffic Directions +| | ${transaction_duration} = | Get Transaction Duration +| | ${transaction_scale} = | Get Transaction Scale +| | ${transaction_type} = | Get Transaction Type +| | ${use_latency} = | Get Use Latency +| | Send traffic on tg +| | ... | duration=${-1} +| | ... | rate=${runtime_rate} +| | ... | frame_size=${frame_size} +| | ... | traffic_profile=${traffic_profile} +| | ... | async_call=${True} +| | ... | ppta=${ppta} +| | ... | use_latency=${use_latency} +| | ... | traffic_directions=${traffic_directions} +| | ... | transaction_duration=${transaction_duration} +| | ... | transaction_scale=${transaction_scale} +| | ... | transaction_type=${transaction_type} +| | ... | duration_limit=${0.0} +| | ... | ramp_up_duration=${ramp_up_duration} +| | ... | ramp_up_rate=${ramp_up_rate} +| | Run Telemetry On All DUTs +| | ... | ${nodes} | profile=perf_stat_runtime.yaml +| | ... | rate=${telemetry_rate} | export=${False} +| | Stop traffic on tg -| Additional Statistics Action For clear-show-runtime-with-iperf3 +| Additional Statistics Action For vpp-runtime-iperf3 | | [Documentation] | | ... | Additional Statistics Action for clear and show runtime counters with | | ... | iPerf3 running traffic. | | | | ... | See documentation of the called keyword for required test variables. | | -| | Clear and show runtime counters with running iperf3 +| | ${runtime_duration} = | Get Runtime Duration +| | ${pids}= | iPerf Client Start Remote Exec +| | | ... | ${nodes['${iperf_client_node}']} +| | | ... | duration=${-1} +| | | ... | rate=${None} +| | | ... | frame_size=${None} +| | | ... | async_call=True +| | | ... | warmup_time=0 +| | | ... | traffic_directions=${1} +| | | ... | namespace=${iperf_client_namespace} +| | | ... | udp=${iperf_client_udp} +| | | ... | host=${iperf_server_bind} +| | | ... | bind=${iperf_client_bind} +| | | ... | affinity=${iperf_client_affinity} +| | Run Telemetry On All DUTs +| | ... | ${nodes} | profile=vppctl_runtime.yaml +| | ... | rate=${telemetry_rate} | export=${telemetry_export} +| | iPerf Client Stop Remote Exec | ${nodes['${iperf_client_node}']} | ${pids} | Additional Statistics Action For noop | | [Documentation] @@ -50,23 +204,13 @@ | | | | No operation -| Additional Statistics Action For vpp-clear-runtime -| | [Documentation] -| | ... | Additional Statistics Action for clear VPP runtime. -| | -| | VPP Clear Runtime On All DUTs | ${nodes} - | Additional Statistics Action For vpp-clear-stats | | [Documentation] | | ... | Additional Statistics Action for clear VPP statistics. | | -| | Clear Statistics On All DUTs | ${nodes} - -| Additional Statistics Action For vpp-enable-elog -| | [Documentation] -| | ... | Additional Statistics Action for enable VPP elog trace. -| | -| | VPP Enable Elog Traces On All DUTs | ${nodes} +| | Run Telemetry On All DUTs +| | ... | ${nodes} | profile=vppctl_clear_stats.yaml +| | ... | export=${False} | Additional Statistics Action For vpp-enable-packettrace | | [Documentation] @@ -75,12 +219,6 @@ | | Run Keyword If | ${extended_debug}==${True} | | ... | VPP Enable Traces On All DUTs | ${nodes} | fail_on_error=${False} -| Additional Statistics Action For vpp-show-elog -| | [Documentation] -| | ... | Additional Statistics Action for show VPP elog trace. -| | -| | Show Event Logger On All DUTs | ${nodes} - | Additional Statistics Action For vpp-show-packettrace | | [Documentation] | | ... | Additional Statistics Action for show VPP packet trace. @@ -88,14 +226,10 @@ | | Run Keyword If | ${extended_debug}==${True} | | ... | Show Packet Trace On All Duts | ${nodes} | maximum=${100} -| Additional Statistics Action For vpp-show-runtime -| | [Documentation] -| | ... | Additional Statistics Action for show VPP runtime. -| | -| | VPP Show Runtime On All DUTs | ${nodes} - | Additional Statistics Action For vpp-show-stats | | [Documentation] | | ... | Additional Statistics Action for show VPP statistics. | | -| | Show Statistics On All DUTs | ${nodes} +| | Run Telemetry On All DUTs +| | ... | ${nodes} | profile=vppctl_show_stats.yaml +| | ... | export=${False} diff --git a/resources/libraries/robot/performance/performance_display.robot b/resources/libraries/robot/performance/performance_display.robot index e8dfdbbfbd..6ee493a17e 100644 --- a/resources/libraries/robot/performance/performance_display.robot +++ b/resources/libraries/robot/performance/performance_display.robot @@ -1,4 +1,4 @@ -# Copyright (c) 2020 Cisco and/or its affiliates. +# Copyright (c) 2023 Cisco and/or its affiliates. # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at: @@ -17,34 +17,30 @@ | ... | This includes checks to fail test. *** Keywords *** -| Check NDRPDR interval validity +| Compute Bandwidth | | [Documentation] -| | ... | Extract loss ratio of lower bound of the interval. -| | ... | Fail if it does not reach the allowed value. -| | +| | ... | Compute (bidir) bandwidth from given (unidir) transaction rate. +| | ... +| | ... | This keyword reads \${ppta} and \${avg_aggregated_frame_size} set +| | ... | elsewhere. The implementation should work for both pps and cps rates. +| | ... | | | ... | *Arguments:* -| | ... | - interval - Measured interval. Type: ReceiveRateInterval -| | ... | - packet_loss_ratio - Accepted loss (0.0 for NDR). Type: float +| | ... | - tps - Transaction rate (unidirectional) [tps]. Type: float +| | ... +| | ... | *Returns:* +| | ... | - Computed bandwidth in Gbps. +| | ... | - Computed aggregated packet rate in pps. | | | | ... | *Example:* | | -| | ... | \| Check NDRPDR interval validity \| \${result.pdr_interval} \ -| | ... | \| \${0.005} \| -| | -| | [Arguments] | ${interval} | ${packet_loss_ratio}=${0.0} -| | -| | ${lower_bound} = | Set Variable | ${interval.measured_low} -| | ${lower_bound_lf} = | Set Variable | ${lower_bound.loss_fraction} -| | Return From Keyword If | ${lower_bound_lf} <= ${packet_loss_ratio} -| | Set Test Variable | \${rate_for_teardown} | ${lower_bound_lf} -| | ${message}= | Catenate | SEPARATOR=${SPACE} -| | ... | Minimal rate loss fraction ${lower_bound_lf} -| | ... | does not reach target ${packet_loss_ratio}. -| | ${message_zero} = | Set Variable | Zero packets forwarded! -| | ${message_other} = | Set Variable | ${lower_bound.loss_count} packets lost. -| | ${message} = | Set Variable If | ${lower_bound_lf} >= 1.0 -| | ... | ${message}${\n}${message_zero} | ${message}${\n}${message_other} -| | Fail | ${message} +| | ... | |\ \${bandwidth} \| \${pps} = \| Compute Bandwidth \| \${12345.67} \| +| | +| | [Arguments] | ${tps} +| | +| | ${ppta} = | Get Packets Per Transaction Aggregated +| | ${pps} = | Evaluate | ${tps} * ${ppta} +| | ${bandwidth} = | Evaluate | ${pps} * (${avg_aggregated_frame_size}+20)*8/1e9 +| | Return From Keyword | ${bandwidth} | ${pps} | Display Reconfig Test Message | | [Documentation] @@ -52,8 +48,8 @@ | | ... | due to reconfiguration under traffic. | | | | ... | *Arguments:* -| | ... | - result - Result of bidirectional measurtement. -| | ... | Type: ReceiveRateMeasurement +| | ... | - result - Result of MLRsearch invocation for one search goal. +| | ... | Type: StatInterval | | | | ... | *Example:* | | @@ -61,23 +57,20 @@ | | | | [Arguments] | ${result} | | -| | ${ppta} = | Get Packets Per Transaction Aggregated -| | ${packet_rate} = | Evaluate | ${result.target_tr} * ${ppta} +| | ${bandwidth} | ${packet_rate}= | Compute Bandwidth | ${result.intended_load} | | ${packet_loss} = | Set Variable | ${result.loss_count} | | ${time_loss} = | Evaluate | ${packet_loss} / ${packet_rate} | | Set Test Message | Packets lost due to reconfig: ${packet_loss} | | Set Test Message | ${\n}Implied time lost: ${time_loss} | append=yes +| | Export Reconf Result | ${packet_rate} | ${packet_loss} | ${bandwidth * 1e9} | Display result of NDRPDR search | | [Documentation] -| | ... | Display result of NDR+PDR search, both quantities, both bounds, -| | ... | aggregate in units given by trasaction type, e.g. by default -| | ... | in packet per seconds and Gbps total bandwidth +| | ... | Display result of NDR+PDR search, both quantities, aggregated, +| | ... | conditional throughput only, in units given by trasaction type, +| | ... | e.g. by default in packet per seconds and Gbps total bandwidth | | ... | (for initial packet size). -| | ... | -| | ... | The bound to display is encoded as target rate, it is assumed -| | ... | it is in transactions per second. Bidirectional traffic -| | ... | transaction is understood as having 2 packets, for this purpose. +| | ... | The lower bounds in the result are assumed to be valid. | | ... | | | ... | Througput is calculated as: | | ... | Sum of measured rate over streams @@ -90,8 +83,8 @@ | | ... | - transaction_type - String identifier to determine how to count | | ... | transactions. Default is "packet". | | ... | *Arguments:* -| | ... | - result - Measured result data. Aggregate rate, tps or pps. -| | ... | Type: NdrPdrResult +| | ... | - result - Measured result data Tps. Type: List[TrimmedStat] +| | ... | *Returns:* NDR and PDR: Unidirectional intended load as tps float. | | | | ... | *Example:* | | @@ -99,21 +92,18 @@ | | | | [Arguments] | ${result} | | -| | Display single bound | NDR_LOWER -| | ... | ${result.ndr_interval.measured_low.target_tr} -| | ... | ${result.ndr_interval.measured_low.latency} -| | Display single bound | NDR_UPPER -| | ... | ${result.ndr_interval.measured_high.target_tr} -| | Display single bound | PDR_LOWER -| | ... | ${result.pdr_interval.measured_low.target_tr} -| | ... | ${result.pdr_interval.measured_low.latency} -| | Display single bound | PDR_UPPER -| | ... | ${result.pdr_interval.measured_high.target_tr} +| | ${ndr} = | Convert To Number | ${result[0].relevant_lower_bound} +| | ${pdr} = | Convert To Number | ${result[1].relevant_lower_bound} +| | Display single bound | NDR | ${result[0].conditional_throughput} +| | Display single bound | PDR | ${result[1].conditional_throughput} +| | Return From Keyword | ${ndr} | ${pdr} | Display result of soak search | | [Documentation] | | ... | Display result of soak search, avg+-stdev, as upper/lower bounds. | | ... | See Display single bound for units used. +| | ... | The displayed values are bidirectional, based on conditional +| | ... | throughput. The returned | | | | ... | *Test (or broader scope) variables read:* | | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: integer or @@ -123,7 +113,6 @@ | | ... | *Arguments:* | | ... | - avg - Estimated average critical load [pps]. Type: float | | ... | - stdev - Standard deviation of critical load [pps]. Type: float -| | | | ... | *Returns:* | | ... | - Lower and upper bound of critical load [pps]. Type: 2-tuple of float | | @@ -144,14 +133,14 @@ | Display single bound | | [Documentation] | | ... | Compute and display one bound of NDR+PDR (or soak) search result. -| | ... | If the latency string is present, it is displayed as well. | | ... | | | ... | The bound to display is given as target transfer rate, it is assumed -| | ... | it is in transactions per second. Bidirectional traffic +| | ... | valid and in transactions per second. Bidirectional traffic | | ... | transaction is understood as having 2 packets, for this purpose. | | ... | -| | ... | Pps values are aggregate in packet per seconds, +| | ... | Pps values are aggregated, in packet per seconds | | ... | and Gbps total bandwidth (for initial packet size). +| | ... | If the latency string is present, it is displayed as well. | | ... | | | ... | Througput is calculated as: | | ... | Sum of measured rate over streams @@ -164,22 +153,24 @@ | | ... | transactions. Default is "packet". | | ... | *Arguments:* | | ... | - text - Flavor text describing which bound is this. Type: string -| | ... | - tps - Transaction rate [tps]. Type: float +| | ... | - tps - Conditional throughput [tps]. Type: Union[float, DiscreteLoad] | | ... | - latency - Latency data to display if non-empty. Type: string | | | | ... | *Example:* | | -| | ... | \| Display single bound \| NDR lower bound \| \${12345.67} \ +| | ... | \| Display single bound \| NDR \| \${12345.67} \ | | ... | \| latency=\${EMPTY} \| | | | | [Arguments] | ${text} | ${tps} | ${latency}=${EMPTY} | | +| | ${tps} = | Convert To Number | ${tps} | | ${transaction_type} = | Get Transaction Type | | Run Keyword And Return If | """_cps""" in """${transaction_type}""" -| | ... | Display single cps bound | ${text} | ${tps} | ${latency} -| | Display single pps bound | ${text} | ${tps} | ${latency} +| | ... | Display Single CPS Bound | ${text} | ${tps} | ${latency} +| | Run Keyword And Return +| | ... | Display Single PPS Bound | ${text} | ${tps} | ${latency} -| Display single cps bound +| Display Single CPS Bound | | [Documentation] | | ... | Display one bound of NDR+PDR search for CPS tests. | | ... | The bounds are expressed as transactions per second. @@ -192,20 +183,21 @@ | | | | ... | *Example:* | | -| | ... | \| Display single cps bound \| NDR lower bound \| \${12345.67} \ +| | ... | \| Display Single CPS Bound \| NDR lower bound \| \${12345.67} \ | | ... | \| latency=\${EMPTY} \| | | | | [Arguments] | ${text} | ${tps} | ${latency}=${EMPTY} | | | | Set Test Message | ${\n}${text}: ${tps} CPS | append=yes -| | Return From Keyword If | not """${latency}""" -| | Set Test Message | ${\n}LATENCY [min/avg/max/hdrh] per stream: ${latency} -| | ... | append=yes +| | ${bandwidth} | ${pps} = | Compute Bandwidth | ${tps} +| | Export Search Bound | ${text} | ${tps} | cps | ${bandwidth * 1e9} +| | Run Keyword If | """${latency}""" | Set Test Message +| | ... | ${\n}LATENCY [min/avg/max/hdrh] per stream: ${latency} | append=yes -| Display single pps bound +| Display Single PPS Bound | | [Documentation] -| | ... | Display one pps bound of NDR+PDR search, -| | ... | aggregate in packet per seconds and Gbps total bandwidth +| | ... | Display one pps bound of NDR+PDR search, aggregated, +| | ... | in packet per seconds and Gbps total bandwidth | | ... | (for initial packet size). | | ... | | | ... | The bound to display is given as target transfer rate, it is assumed @@ -225,16 +217,14 @@ | | | | ... | *Example:* | | -| | ... | \| Display single pps bound \| NDR lower bound \| \${12345.67} \ +| | ... | \| Display Single PPS Bound \| NDR lower bound \| \${12345.67} \ | | ... | \| latency=\${EMPTY} \| | | | | [Arguments] | ${text} | ${tps} | ${latency}=${EMPTY} | | -| | ${ppta} = | Get Packets Per Transaction Aggregated -| | ${pps} = | Evaluate | ${tps} * ${ppta} -| | ${bandwidth} = | Evaluate | ${pps} * (${avg_frame_size}+20)*8 / 1e9 +| | ${bandwidth} | ${pps} = | Compute Bandwidth | ${tps} | | Set Test Message | ${\n}${text}: ${pps} pps, | append=yes | | Set Test Message | ${bandwidth} Gbps (initial) | append=yes -| | Return From Keyword If | not """${latency}""" -| | Set Test Message | ${\n}LATENCY [min/avg/max/hdrh] per stream: ${latency} -| | ... | append=yes +| | Export Search Bound | ${text} | ${pps} | pps | ${bandwidth * 1e9} +| | Run Keyword If | """${latency}""" | Set Test Message +| | ... | ${\n}LATENCY [min/avg/max/hdrh] per stream: ${latency} | append=yes diff --git a/resources/libraries/robot/performance/performance_utils.robot b/resources/libraries/robot/performance/performance_utils.robot index d45adc6cf9..4211486a19 100644 --- a/resources/libraries/robot/performance/performance_utils.robot +++ b/resources/libraries/robot/performance/performance_utils.robot @@ -1,4 +1,4 @@ -# Copyright (c) 2021 Cisco and/or its affiliates. +# Copyright (c) 2023 Cisco and/or its affiliates. # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at: @@ -13,11 +13,12 @@ *** Settings *** | Library | Collections +| Library | resources.libraries.python.model.ExportResult | Library | resources.libraries.python.topology.Topology | Library | resources.libraries.python.NodePath -| Library | resources.libraries.python.PerfUtil | Library | resources.libraries.python.InterfaceUtil | Library | resources.libraries.python.Iperf3 +| Library | resources.libraries.python.TelemetryUtil | Library | resources.libraries.python.TrafficGenerator | Library | resources.libraries.python.TrafficGenerator.OptimizedSearch | Library | resources.libraries.python.TrafficGenerator.TGDropRateSearchImpl @@ -38,58 +39,14 @@ | ${heap_size_mult}= | ${1} *** Keywords *** -| Clear and show runtime counters with running traffic -| | [Documentation] -| | ... | Start traffic at specified rate then clear runtime counters on all -| | ... | DUTs. Wait for specified amount of time and capture runtime counters -| | ... | on all DUTs. Finally stop traffic. -| | -| | ... | TODO: Support resetter if this is not the first trial-ish action? -| | -| | ... | *Example:* -| | -| | ... | \| Clear and show runtime counters with running traffic \| -| | -| | ${ppta} = | Get Packets Per Transaction Aggregated -| | ${runtime_duration} = | Get Runtime Duration -| | ${runtime_rate} = | Get Runtime Rate -| | ${traffic_directions} = | Get Traffic Directions -| | ${transaction_duration} = | Get Transaction Duration -| | ${transaction_scale} = | Get Transaction Scale -| | ${transaction_type} = | Get Transaction Type -| | ${use_latency} = | Get Use Latency -| | # Duration of -1 means we will stop traffic manually. -| | Send traffic on tg -| | ... | duration=${-1} -| | ... | rate=${runtime_rate} -| | ... | frame_size=${frame_size} -| | ... | traffic_profile=${traffic_profile} -| | ... | async_call=${True} -| | ... | ppta=${ppta} -| | ... | use_latency=${use_latency} -| | ... | traffic_directions=${traffic_directions} -| | ... | transaction_duration=${transaction_duration} -| | ... | transaction_scale=${transaction_scale} -| | ... | transaction_type=${transaction_type} -| | ... | duration_limit=${0.0} -| | FOR | ${action} | IN | @{pre_run_stats} -| | | Run Keyword | Additional Statistics Action For ${action} -| | END -| | Sleep | ${runtime_duration} -| | FOR | ${action} | IN | @{post_run_stats} -| | | Run Keyword | Additional Statistics Action For ${action} -| | END -| | Stop traffic on tg - | Find critical load using PLRsearch | | [Documentation] | | ... | Find boundaries for troughput (of hardcoded target loss ratio) | | ... | using PLRsearch algorithm. | | ... | Display results as formatted test message. | | ... | Fail if computed lower bound is 110% of the minimal rate or less. -| | ... | Input rates are understood as uni-directional, -| | ... | reported result contains aggregate rates. -| | ... | Currently, the min_rate value is hardcoded to match test teardowns. +| | ... | Input rates are unidirectional, in transaction per second. +| | ... | Reported result may contain aggregated pps rates, depending on test. | | ... | Call \${resetter} (if defined) to reset DUT state before each trial. | | | | ... | *Test (or broader scope) variables read:* @@ -106,19 +63,38 @@ | | | | # Get values via performance_vars. | | ${max_rate} = | Get Max Rate -| | ${min_rate} = | Get Min Rate +| | ${min_rate_soft} = | Get Min Rate Soft | | ${ppta} = | Get Packets Per Transaction Aggregated +| | ${ramp_up_duration} = | Get Ramp Up Duration +| | ${ramp_up_rate} = | Get Ramp Up Rate | | ${resetter} = | Get Resetter | | ${traffic_directions} = | Get Traffic Directions | | ${transaction_duration} = | Get Transaction Duration | | ${transaction_scale} = | Get Transaction Scale | | ${transaction_type} = | Get Transaction Type | | ${use_latency} = | Get Use Latency +| | # TRex needs a warmup to avoid unsent packets at half-max rate. +| | Send traffic on tg +| | ... | duration=1.0 +| | ... | rate=${max_rate} +| | ... | frame_size=${frame_size} +| | ... | traffic_profile=${traffic_profile} +| | ... | async_call=${False} +| | ... | duration_limit=${1.0} +| | ... | ppta=${ppta} +| | ... | traffic_directions=${traffic_directions} +| | ... | transaction_duration=${transaction_duration} +| | ... | transaction_scale=${transaction_scale} +| | ... | transaction_type=${transaction_type} +| | ... | use_latency=False +| | ... | ramp_up_duration=${0.0} +| | ... | ramp_up_rate=${0.0} +| | # Ready for main search. | | ${average} | ${stdev} = | Perform soak search | | ... | frame_size=${frame_size} | | ... | traffic_profile=${traffic_profile} -| | ... | minimum_transmit_rate=${min_rate} -| | ... | maximum_transmit_rate=${max_rate} +| | ... | min_load=${min_rate_soft} +| | ... | max_load=${max_rate} | | ... | plr_target=${1e-7} | | ... | tdpt=${0.1} | | ... | initial_count=${50} @@ -131,35 +107,43 @@ | | ... | transaction_duration=${transaction_duration} | | ... | transaction_type=${transaction_type} | | ... | use_latency=${use_latency} +| | ... | ramp_up_duration=${ramp_up_duration} +| | ... | ramp_up_rate=${ramp_up_rate} | | ${lower} | ${upper} = | Display result of soak search | | ... | ${average} | ${stdev} | | Set Test Variable | \${rate for teardown} | ${lower} -| | Should Not Be True | 1.1 * ${min_rate} > ${lower} -| | ... | Lower bound ${lower} too small for unidirectional minimum ${min_rate}. +| | # Stats at the discovered critical rate. +| | Set Test Variable | ${telemetry_rate} | plr +| | Set Test Variable | ${telemetry_export} | ${True} +| | Send traffic at specified rate +| | ... | rate=${lower} +| | ... | trial_duration=${1.0} +| | ... | trial_multiplicity=${1} +| | ... | use_latency=${use_latency} +| | ... | duration_limit=${1.0} +| | Should Not Be True | 1.1 * ${min_rate_soft} > ${lower} +| | ... | Lower bound ${lower} too small for unidir minimum ${min_rate_soft}. | Find NDR and PDR intervals using optimized search | | [Documentation] | | ... | Find boundaries for RFC2544 compatible NDR and PDR values | | ... | using an optimized search algorithm. | | ... | Display findings as a formatted test message. -| | ... | Fail if a resulting lower bound has too high loss fraction. -| | ... | Input rates are understood as uni-directional, -| | ... | reported result contains aggregate rates. +| | ... | Fail if a resulting lower bound has too high loss ratio. +| | ... | Input rates are unidirectional, in transaction per second. +| | ... | Reported result may contain aggregated pps rates, depending on test. | | ... | Additional latency measurements are performed for smaller loads, | | ... | even if latency stream is disabled in search. Their results | | ... | are also displayed. | | ... | Finally, two measurements for runtime stats are done (not displayed). -| | ... | Currently, the min_rate value is hardcoded to 90kpps, -| | ... | allowing measurement at 10% of the discovered rate -| | ... | without breaking latency streams. | | ... | Call \${resetter} (if defined) to reset DUT state before each trial. | | | | ... | *Test (or broader scope) variables read:* | | ... | - traffic_profile - Name of module defining traffc for measurements. | | ... | Type: string | | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: integer or -| | ... | string -| | ... | - max_rate - Calculated maximal transmit rate [tps]. +| | ... | string. +| | ... | - max_rate - Calculated maximal unidirectional transmit rate [tps]. | | ... | Type: float | | ... | - resetter - Callable to reset DUT state before each trial. | | ... | - transaction_scale - Number of ASTF transaction (zero if unlimited). @@ -175,28 +159,29 @@ | | # Get values via performance_vars. | | ${disable_latency} = | Get Disable Latency | | ${max_rate} = | Get Max Rate -| | ${min_rate} = | Get Min Rate -| | # \${packet_loss_ratio} is used twice so it is worth a variable. -| | ${packet_loss_ratio} = | Get Packet Loss Ratio +| | ${min_rate_soft} = | Get Min Rate Soft +| | ${loss_ratio} = | Get Packet Loss Ratio | | ${ppta} = | Get Packets Per Transaction Aggregated +| | ${ramp_up_duration} = | Get Ramp Up Duration +| | ${ramp_up_rate} = | Get Ramp Up Rate | | ${resetter} = | Get Resetter | | ${traffic_directions} = | Get Traffic Directions | | ${transaction_duration} = | Get Transaction Duration | | ${transaction_scale} = | Get Transaction Scale | | ${transaction_type} = | Get Transaction Type | | ${use_latency} = | Get Use Latency -| | ${result} = | Perform optimized ndrpdr search +| | ${result} = | Perform MLR Search | | ... | frame_size=${frame_size} | | ... | traffic_profile=${traffic_profile} -| | ... | minimum_transmit_rate=${min_rate} -| | ... | maximum_transmit_rate=${max_rate} -| | ... | packet_loss_ratio=${packet_loss_ratio} -| | ... | final_relative_width=${0.005} -| | ... | final_trial_duration=${30.0} +| | ... | min_load=${min_rate_soft} +| | ... | max_load=${max_rate} +| | ... | loss_ratio=${loss_ratio} +| | ... | relative_width=${0.005} | | ... | initial_trial_duration=${1.0} -| | ... | number_of_intermediate_phases=${2} -| | ... | timeout=${720.0} -| | ... | doublings=${2} +| | ... | final_trial_duration=${1.0} +| | ... | duration_sum=${21.0} +| | ... | preceding_targets=${2} +| | ... | search_duration_max=${1200.0} | | ... | ppta=${ppta} | | ... | resetter=${resetter} | | ... | traffic_directions=${traffic_directions} @@ -204,19 +189,20 @@ | | ... | transaction_scale=${transaction_scale} | | ... | transaction_type=${transaction_type} | | ... | use_latency=${use_latency} -| | Display result of NDRPDR search | ${result} -| | Check NDRPDR interval validity | ${result.pdr_interval} -| | ... | ${packet_loss_ratio} -| | Check NDRPDR interval validity | ${result.ndr_interval} -| | ${pdr} = | Set Variable | ${result.pdr_interval.measured_low.target_tr} -| | ${ndr} = | Set Variable | ${result.ndr_interval.measured_low.target_tr} -| | # We expect NDR and PDR to have different-looking stats. +| | ... | ramp_up_duration=${ramp_up_duration} +| | ... | ramp_up_rate=${ramp_up_rate} +| | ${ndr} | ${pdr} = | Display result of NDRPDR search | ${result} +| | # We expect NDR and PDR to have different-looking telemetry. +| | Set Test Variable | ${telemetry_rate} | pdr +| | Set Test Variable | ${telemetry_export} | ${True} | | Send traffic at specified rate | | ... | rate=${pdr} | | ... | trial_duration=${1.0} | | ... | trial_multiplicity=${1} | | ... | use_latency=${use_latency} | | ... | duration_limit=${1.0} +| | Set Test Variable | ${telemetry_rate} | ndr +| | Set Test Variable | ${telemetry_export} | ${True} | | Run Keyword If | ${ndr} != ${pdr} | | ... | Send traffic at specified rate | | ... | rate=${ndr} @@ -236,9 +222,8 @@ | Find Throughput Using MLRsearch | | [Documentation] | | ... | Find and return lower bound NDR (zero PLR) -| | ... | aggregate throughput using MLRsearch algorithm. -| | ... | Input rates are understood as uni-directional. -| | ... | Currently, the min_rate value is hardcoded to match test teardowns. +| | ... | throughput using MLRsearch algorithm. +| | ... | Input and output rates are understood as uni-directional, in tps. | | ... | Call \${resetter} (if defined) to reset DUT state before each trial. | | | | ... | *Test (or broader scope) variables read:* @@ -246,7 +231,7 @@ | | ... | Type: string | | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: integer or | | ... | string -| | ... | - max_rate - Calculated maximal transmit rate [tps]. +| | ... | - max_rate - Calculated maximal unidirectional transmit rate [tps]. | | ... | Type: float | | ... | - resetter - Callable to reset DUT state before each trial. | | ... | - transaction_scale - Number of ASTF transaction (zero if unlimited). @@ -254,33 +239,36 @@ | | ... | transactions. Default is "packet". | | | | ... | *Returns:* -| | ... | - Lower bound for bi-directional throughput at given PLR. Type: float +| | ... | - Lower bound for uni-directional tps throughput at given PLR. +| | ... | Type: float | | | | ... | *Example:* | | | | ... | \| \${throughpt}= \| Find Throughput Using MLRsearch \| | | | | ${max_rate} = | Get Max Rate -| | ${min_rate} = | Get Min Rate +| | ${min_rate_soft} = | Get Min Rate Soft | | ${ppta} = | Get Packets Per Transaction Aggregated +| | ${ramp_up_duration} = | Get Ramp Up Duration +| | ${ramp_up_rate} = | Get Ramp Up Rate | | ${resetter} = | Get Resetter | | ${traffic_directions} = | Get Traffic Directions | | ${transaction_duration} = | Get Transaction Duration | | ${transaction_scale} = | Get Transaction Scale | | ${transaction_type} = | Get Transaction Type | | ${use_latency} = | Get Use Latency -| | ${result} = | Perform optimized ndrpdr search +| | ${result} = | Perform MLR Search | | ... | frame_size=${frame_size} | | ... | traffic_profile=${traffic_profile} -| | ... | minimum_transmit_rate=${min_rate} -| | ... | maximum_transmit_rate=${max_rate} -| | ... | packet_loss_ratio=${0.0} -| | ... | final_relative_width=${0.001} -| | ... | final_trial_duration=${10.0} +| | ... | min_load=${min_rate_soft} +| | ... | max_load=${max_rate} +| | ... | loss_ratio=${0.0} +| | ... | relative_width=${0.001} | | ... | initial_trial_duration=${1.0} -| | ... | number_of_intermediate_phases=${1} -| | ... | timeout=${720} -| | ... | doublings=${2} +| | ... | final_trial_duration=${1.0} +| | ... | duration_sum=${11.0} +| | ... | preceding_targets=${1} +| | ... | search_duration_max=${1200} | | ... | ppta=${ppta} | | ... | resetter=${resetter} | | ... | traffic_directions=${traffic_directions} @@ -288,21 +276,22 @@ | | ... | transaction_scale=${transaction_scale} | | ... | transaction_type=${transaction_type} | | ... | use_latency=${use_latency} -| | Check NDRPDR interval validity | ${result.pdr_interval} -| | ... | ${0.0} -| | Return From Keyword | ${result.pdr_interval.measured_low.target_tr} +| | ... | ramp_up_duration=${ramp_up_duration} +| | ... | ramp_up_rate=${ramp_up_rate} +| | ${ret} = | Convert To Number | ${result[0].relevant_lower_bound} +| | Return From Keyword | ${ret} | Measure and show latency at specified rate | | [Documentation] | | ... | Send traffic at specified rate, single trial. | | ... | Extract latency information and append it to text message. -| | ... | The rate argument is int, so should not include "pps". +| | ... | The rate argument is float, so should not include "pps". | | ... | If the given rate is too low, a safe value is used instead. | | ... | Call \${resetter} (if defined) to reset DUT state before each trial. | | | | ... | *Arguments:* | | ... | - message_prefix - Preface to test message addition. Type: string -| | ... | - rate - Rate [tps] for sending packets. +| | ... | - rate - Unidirectional rate [tps] for sending packets. | | ... | Type: float | | | | ... | *Example:* @@ -312,9 +301,11 @@ | | | | [Arguments] | ${message_prefix} | ${rate} | | -| | ${min_rate} = | Get Min Rate +| | ${min_rate_hard} = | Get Min Rate Hard | | ${ppta} = | Get Packets Per Transaction Aggregated -| | ${real_rate} = | Evaluate | max(${rate}, ${min_rate}) +| | ${ramp_up_duration} = | Get Ramp Up Duration +| | ${ramp_up_rate} = | Get Ramp Up Rate +| | ${real_rate} = | Evaluate | max(${rate}, ${min_rate_hard}) | | ${traffic_directions} = | Get Traffic Directions | | ${transaction_duration} = | Get Transaction Duration | | ${transaction_scale} = | Get Transaction Scale @@ -333,12 +324,15 @@ | | ... | transaction_scale=${transaction_scale} | | ... | transaction_type=${transaction_type} | | ... | use_latency=${True} +| | ... | ramp_up_duration=${ramp_up_duration} +| | ... | ramp_up_rate=${ramp_up_rate} | | ${latency} = | Get Latency Int | | Set Test Message | ${\n}${message_prefix} ${latency} | append=${True} +| | Export Ndrpdr Latency | ${message_prefix} | ${latency} | Send ramp-up traffic | | [Documentation] -| | ... | Do nothing unless positive ramp-up duration is specified. +| | ... | Fail unless positive ramp-up rate is specified. | | ... | Else perform one trial with appropriate rate and duration. | | ... | This is useful for tests that set DUT state via traffic. | | ... | Rate has to bee low enough so packets are not lost, @@ -352,7 +346,7 @@ | | ... | string | | ... | - ramp_up_duration - Suitable traffic duration [s]. | | ... | Type: float -| | ... | - ramp_up_rate - Suitable transmit rate [tps]. +| | ... | - ramp_up_rate - Suitable unidirectional transmit rate [tps]. | | ... | Type: float | | ... | - transaction_type - String identifier to determine how to count | | ... | transactions. Default is "packet". @@ -361,9 +355,9 @@ | | | | ... | \| Send ramp-up traffic \| | | -| | ${ramp_up_duration} = | Get Ramp Up Duration -| | Run Keyword Unless | ${ramp_up_duration} > 0.0 | Return From Keyword | | ${ramp_up_rate} = | Get Ramp Up Rate +| | Run Keyword If | ${ramp_up_rate} <= 0.0 | Fail | Ramp up rate missing! +| | ${ramp_up_duration} = | Get Ramp Up Duration | | ${ppta} = | Get Packets Per Transaction Aggregated | | ${traffic_directions} = | Get Traffic Directions | | ${transaction_duration} = | Get Transaction Duration @@ -383,6 +377,9 @@ | | ... | transaction_duration=${transaction_duration} | | ... | transaction_scale=${transaction_scale} | | ... | transaction_type=${transaction_type} +| | ... | ramp_up_duration=${ramp_up_duration} +| | ... | ramp_up_rate=${ramp_up_rate} +| | ... | ramp_up_only=${True} | Send traffic at specified rate | | [Documentation] @@ -394,7 +391,7 @@ | | | | ... | *Arguments:* | | ... | - trial_duration - Duration of single trial [s]. Type: float -| | ... | - rate - Target transmit rate [tps]. Type: float +| | ... | - rate - Target unidirectional transmit rate [tps]. Type: float | | ... | Type: string | | ... | - trial_multiplicity - How many trials in this measurement. | | ... | Type: boolean @@ -402,28 +399,37 @@ | | ... | Type: boolean | | ... | - duration_limit - Hard limit for trial duration, overriding duration | | ... | computed from transaction_scale. Default 0.0 means no limit. +| | ... | - export_mrr_unit - Use this unit when exporting MRR values, +| | ... | or empty string for no export. | | | | ... | *Example:* | | | | ... | \| Send traffic at specified rate \| \${1.0} \| ${4000000.0} \ -| | ... | \| \${10} \| ${False} \| ${1.0} \| +| | ... | \| \${10} \| ${False} \| ${1.0} \| pps \| | | | | [Arguments] | ${trial_duration} | ${rate} | ${trial_multiplicity} | | ... | ${use_latency}=${False} | ${duration_limit}=${0.0} +| | ... | ${export_mrr_unit}=${Empty} | | | | ${ppta} = | Get Packets Per Transaction Aggregated +| | ${ramp_up_duration} = | Get Ramp Up Duration +| | ${ramp_up_rate} = | Get Ramp Up Rate | | ${traffic_directions} = | Get Traffic Directions | | ${transaction_duration} = | Get Transaction Duration | | ${transaction_scale} = | Get Transaction Scale | | ${transaction_type} = | Get Transaction Type | | Set Test Variable | \${rate_for_teardown} | ${rate} -| | FOR | ${action} | IN | @{pre_stats} +| | Set Test Variable | \${runtime_rate} | ${rate} +| | FOR | ${action} | IN | @{stat_runtime} +| | | Run Keyword | Additional Statistics Action For ${action} +| | END +| | FOR | ${action} | IN | @{stat_pre_trial} | | | Run Keyword | Additional Statistics Action For ${action} | | END | | ${results} = | Create List | | FOR | ${i} | IN RANGE | ${trial_multiplicity} | | | Call Resetter -| | | Send traffic on tg +| | | ${result} = | Send traffic on tg | | | ... | duration=${trial_duration} | | | ... | rate=${rate} | | | ... | frame_size=${frame_size} @@ -436,49 +442,21 @@ | | | ... | transaction_scale=${transaction_scale} | | | ... | transaction_type=${transaction_type} | | | ... | use_latency=${use_latency} -| | | ${result}= | Get Measurement Result +| | | ... | ramp_up_duration=${ramp_up_duration} +| | | ... | ramp_up_rate=${ramp_up_rate} | | | # Out of several quantities for aborted traffic (duration stretching), | | | # the approximated receive rate is the best estimate we have. -| | | Append To List | ${results} | ${result.approximated_receive_rate} +| | | ${value} = | Set Variable | ${result.forwarding_count} +| | | ${value} = | Evaluate | ${value} / ${result.offered_duration} +| | | ${bandwidth} | ${pps} = | Compute Bandwidth | ${value} / ${ppta} +| | | Append Mrr Value | ${value} | ${export_mrr_unit} | ${bandwidth * 1e9} +| | | Append To List | ${results} | ${value} | | END -| | FOR | ${action} | IN | @{post_stats} +| | FOR | ${action} | IN | @{stat_post_trial} | | | Run Keyword | Additional Statistics Action For ${action} | | END | | Return From Keyword | ${results} -| Clear and show runtime counters with running iperf3 -| | [Documentation] -| | ... | Start traffic at specified rate then clear runtime counters on all -| | ... | DUTs. Wait for specified amount of time and capture runtime counters -| | ... | on all DUTs. Finally stop traffic. -| | -| | ... | *Example:* -| | -| | ... | \| Clear and show runtime counters with running traffic \| -| | -| | ${runtime_duration} = | Get Runtime Duration -| | ${pids}= | iPerf Client Start Remote Exec -| | | ... | ${nodes['${iperf_client_node}']} -| | | ... | duration=${-1} -| | | ... | rate=${None} -| | | ... | frame_size=${None} -| | | ... | async_call=True -| | | ... | warmup_time=0 -| | | ... | traffic_directions=${1} -| | | ... | namespace=${iperf_client_namespace} -| | | ... | udp=${iperf_client_udp} -| | | ... | host=${iperf_server_bind} -| | | ... | bind=${iperf_client_bind} -| | | ... | affinity=${iperf_client_affinity} -| | FOR | ${action} | IN | @{pre_run_stats} -| | | Run Keyword | Additional Statistics Action For ${action} -| | END -| | Sleep | ${runtime_duration} -| | FOR | ${action} | IN | @{post_run_stats} -| | | Run Keyword | Additional Statistics Action For ${action} -| | END -| | iPerf Client Stop Remote Exec | ${nodes['${iperf_client_node}']} | ${pids} - | Traffic should pass with maximum rate on iPerf3 | | [Documentation] | | ... | Send traffic at maximum rate on iPerf3. @@ -500,9 +478,12 @@ | | ... | ${trial_multiplicity}=${trial_multiplicity} | | ... | ${traffic_directions}=${1} | | +| | Set Test Variable | ${telemetry_rate} | mrr +| | Set Test Variable | ${telemetry_export} | ${True} | | ${results}= | Send iPerf3 traffic at specified rate | | ... | ${trial_duration} | ${None} | ${None} | | ... | ${trial_multiplicity} | ${traffic_directions} +| | ... | export_mrr_unit=bps | | Set Test Message | ${\n}iPerf3 trial results | | Set Test Message | in Gbits per second: ${results} | | ... | append=yes @@ -517,7 +498,7 @@ | | ... | *Arguments:* | | ... | - trial_duration - Duration of single trial [s]. | | ... | Type: float -| | ... | - rate - Target aggregate transmit rate [bps] / Bits per second +| | ... | - rate - Target aggregated transmit rate [bps] / Bits per second. | | ... | Type: float | | ... | - frame_size - L2 Frame Size [B]. | | ... | Type: integer or string @@ -527,6 +508,9 @@ | | ... | Type: integer | | ... | - extended_debug - True to enable extended debug. | | ... | Type: boolean +| | ... | - export_mrr_unit - Use this unit when exporting MRR values, +| | ... | or empty string for no export. +| | ... | Type: string | | | | ... | *Example:* | | @@ -536,6 +520,7 @@ | | [Arguments] | ${trial_duration} | ${rate} | ${frame_size} | | ... | ${trial_multiplicity}=${trial_multiplicity} | | ... | ${traffic_directions}=${1} | ${extended_debug}=${extended_debug} +| | ... | ${export_mrr_unit}=${Empty} | | | | Set Test Variable | ${extended_debug} | | Set Test Variable | ${rate} @@ -544,7 +529,7 @@ | | ${smt_used}= | Is SMT enabled | ${nodes['${iperf_server_node}']['cpuinfo']} | | ${vm_status} | ${value}= | Run Keyword And Ignore Error | | ... | Get Library Instance | vnf_manager -| | ${vth}= | Evaluate | (${thr_count_int} + 1) +| | ${vth}= | Evaluate | (${dp_count_int} + 1) | | ${cpu_skip_cnt}= | Set Variable If | '${vm_status}' == 'PASS' | | ... | ${CPU_CNT_SYSTEM} | | ... | ${${CPU_CNT_SYSTEM} + ${CPU_CNT_MAIN} + ${cpu_count_int} + ${vth}} @@ -558,22 +543,26 @@ | | ... | bind_mask=${iperf_server_bind_mask} | | ... | namespace=${iperf_server_namespace} | | ... | cpu_skip_cnt=${cpu_skip_cnt} -| | Run Keyword If | '${iperf_client_namespace}' is not '${None}' +| | Run Keyword If | '${iperf_client_namespace}' != '${None}' | | ... | Set Linux Interface IP | | ... | ${nodes['${iperf_client_node}']} | | ... | interface=${iperf_client_interface} | | ... | ip_addr=${iperf_client_bind} | | ... | prefix=${iperf_client_bind_mask} | | ... | namespace=${iperf_client_namespace} -| | Run Keyword If | '${iperf_client_namespace}' is not '${None}' +| | Run Keyword If | '${iperf_client_namespace}' != '${None}' | | ... | Add Default Route To Namespace | | ... | ${nodes['${iperf_client_node}']} | | ... | namespace=${iperf_client_namespace} | | ... | default_route=${iperf_client_bind_gw} -| | ${pre_stats}= | Create List -| | ... | clear-show-runtime-with-iperf3 -| | ... | vpp-clear-stats | vpp-enable-packettrace | vpp-enable-elog -| | FOR | ${action} | IN | @{pre_stats} +| | ${stat_runtime}= | Create List +| | ... | vpp-runtime-iperf3 +| | ${stat_pre_trial}= | Create List +| | ... | vpp-runtime-iperf3 | vpp-clear-stats | vpp-enable-packettrace +| | FOR | ${action} | IN | @{stat_runtime} +| | | Run Keyword | Additional Statistics Action For ${action} +| | END +| | FOR | ${action} | IN | @{stat_pre_trial} | | | Run Keyword | Additional Statistics Action For ${action} | | END | | ${results} = | Create List @@ -592,12 +581,13 @@ | | | ... | bind=${iperf_client_bind} | | | ... | affinity=${iperf_client_affinity} | | | ${conv} = | Convert To Number | ${rr['sum_received']['bits_per_second']} +| | | Append Mrr Value | ${conv} | ${export_mrr_unit} | | | ${conv} = | Evaluate | ${conv} / ${1000} / ${1000} / ${1000} | | | ${conv} = | Evaluate | "{:.3f}".format(${conv}) | | | Append To List | | | ... | ${results} | ${conv} | | END -| | FOR | ${action} | IN | @{post_stats} +| | FOR | ${action} | IN | @{stat_post_trial} | | | Run Keyword | Additional Statistics Action For ${action} | | END | | Return From Keyword | ${results} @@ -614,7 +604,7 @@ | | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: integer or | | ... | string | | ... | *Arguments:* -| | ... | - rate - Rate [tps] for sending packets. +| | ... | - rate - Unidirectional rate [tps] for sending packets. | | ... | Type: float | | | | ... | *Example:* @@ -624,6 +614,8 @@ | | [Arguments] | ${rate} | | | | ${ppta} = | Get Packets Per Transaction Aggregated +| | ${ramp_up_duration} = | Get Ramp Up Duration +| | ${ramp_up_rate} = | Get Ramp Up Rate | | ${traffic_directions} = | Get Traffic Directions | | ${transaction_duration} = | Get Transaction Duration | | ${transaction_scale} = | Get Transaction Scale @@ -639,6 +631,8 @@ | | ... | async_call=${True} | | ... | duration_limit=${0.0} | | ... | ppta=${ppta} +| | ... | ramp_up_duration=${ramp_up_duration} +| | ... | ramp_up_rate=${ramp_up_rate} | | ... | traffic_directions=${traffic_directions} | | ... | transaction_duration=${transaction_duration} | | ... | transaction_scale=${transaction_scale} @@ -680,21 +674,25 @@ | | | | ... | \| Traffic should pass with maximum rate \| | | +| | Set Test Variable | ${telemetry_rate} | mrr +| | Set Test Variable | ${telemetry_export} | ${True} | | ${max_rate} = | Get Max Rate | | ${transaction_type} = | Get Transaction Type | | ${trial_duration} = | Get Mrr Trial Duration | | ${trial_multiplicity} = | Get Mrr Trial Multiplicity | | ${use_latency} = | Get Use Latency -| | # The following also sets \${rate_for_teardown} +| | ${unit} = | Set Variable If | """_cps""" in """${transaction_type}""" +| | ... | cps | pps | | ${results} = | Send traffic at specified rate | | ... | rate=${max_rate} | | ... | trial_duration=${trial_duration} | | ... | trial_multiplicity=${trial_multiplicity} | | ... | use_latency=${use_latency} | | ... | duration_limit=${0.0} -| | ${unit} = | Set Variable If | """_cps""" in """${transaction_type}""" +| | ... | export_mrr_unit=${unit} +| | ${unit_text} = | Set Variable If | """_cps""" in """${transaction_type}""" | | ... | estimated connections per second | packets per second | | Set Test Message | ${\n}Maximum Receive Rate trial results -| | Set Test Message | in ${unit}: ${results} +| | Set Test Message | in ${unit_text}: ${results} | | ... | append=yes | | Fail if no traffic forwarded diff --git a/resources/libraries/robot/performance/performance_vars.robot b/resources/libraries/robot/performance/performance_vars.robot index 4a6d385363..a0fa277e4c 100644 --- a/resources/libraries/robot/performance/performance_vars.robot +++ b/resources/libraries/robot/performance/performance_vars.robot @@ -1,4 +1,4 @@ -# Copyright (c) 2021 Cisco and/or its affiliates. +# Copyright (c) 2023 Cisco and/or its affiliates. # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at: @@ -60,25 +60,46 @@ | | Return From Keyword If | ${max_rate} | ${max_rate} | | Fail | \${max_rate} is not defined. Call Set Max Rate And Jumbo keyword. -| Get Min Rate +| Get Min Rate Hard | | [Documentation] -| | ... | Return a hardcoded value. This is an abstraction, useful in case -| | ... | we start allowing various other overrides or computations. -| | ... | Call this just before calling a Python keyword, -| | ... | as those have restricted access to Robot variables. -| | -| | ... | The return value controls the minimal unidirectional packet rate. -| | ... | The value is also usable for minimal TPS value for ASTF tests. +| | ... | Return a hardcoded value. +| | ... | The return value controls the minimal unidirectional packet rate, +| | ... | to be used anywhere, including latency measurements at 0% load. | | ... | The current value is the smallest one permitted | | ... | by STL profiles with latency streams. | | ... | Return type: float. | | | | ... | *Example:* | | -| | ... | \| \${min_rate} = \| Get Min Rate \| +| | ... | \| \${min_rate_hard} = \| Get Min Rate Hard \| | | | | Return From Keyword | ${9001.0} +| Get Min Rate Soft +| | [Documentation] +| | ... | If ramp up rate is not defined, return the hard min value. +| | ... | If ramp up rate is defined (and larger than hard min), return that. +| | ... | The reason is, ramp up rate should already guarantee no loss. +| | +| | ... | The return value controls the minimal unidirectional packet rate, +| | ... | to be used in various search algorithms. +| | ... | Latency measurements may want even lower loads, use hard min for that. +| | +| | ... | The value is also usable for minimal TPS value for ASTF tests. +| | ... | Return type: float. +| | +| | ... | Currently, undefined ramp up rate is reported as zero, +| | ... | so we return the maximum of ramp up rate and the hard min rate. +| | +| | ... | *Example:* +| | +| | ... | \| \${min_rate_soft} = \| Get Min Rate Soft \| +| | +| | ${min_rate_hard} = | Get Min Rate Hard +| | ${ramp_up_rate} = | Get Ramp Up Rate +| | ${min_rate_soft} = | Evaluate | max(${ramp_up_rate}, ${min_rate_hard}) +| | Return From Keyword | ${min_rate_soft} + | Get Mrr Trial Duration | | [Documentation] | | ... | Return value from Constants. This is an abstraction, useful in case @@ -134,11 +155,11 @@ | Get Packets Per Transaction Aggregated | | [Documentation] | | ... | Return value of \${packets_per_transaction_aggregated}; -| | ... | if not defined, assume traffic is symmetric and compute -| | ... | from unidirectional values. +| | ... | if not defined, assume traffic is symmetric (or unidirectional) +| | ... | and compute from unidirectional values. | | -| | ... | The return value is used when reporting PPS values from TPS found -| | ... | by some search (e.g. NDRPDR). +| | ... | The return value is used when reporting PPS (and bandwidth) values +| | ... | from TPS found by some search (e.g. NDRPDR). | | ... | Return type: integer. | | | | ... | *Example:* @@ -148,7 +169,6 @@ | | ${ppta} = | Get Variable Value | \${packets_per_transaction_aggregated} | | ... | ${0} | | Return From Keyword If | "${ppta}" != "0" | ${ppta} -| | # TODO: Insert TCP computation from packet size here. | | ${pptad} = | Get Packets Per Transaction And Direction | | ${traffic_directions} = | Get Traffic Directions | | # We do not support ASTF profiles with multiple transactions, yet. @@ -160,8 +180,9 @@ | | ... | Return value of \${packets_per_transaction_and_direction}, | | ... | or ${1} if not defined. | | -| | ... | The return value is used when computing max rate (TPS), -| | ... | so for asymmetric transaction use the more numerous direction. +| | ... | The return value is used when computing max rate (TPS) +| | ... | from packet level (pps or bps) limits. +| | ... | For asymmetric transactions, use the more numerous direction. | | ... | Return type: integer. | | | | ... | *Example:* @@ -179,7 +200,7 @@ | | | | ... | The return value determines the required duration of ramp-up phase. | | ... | Typically used to prepare a specific state on DUT. -| | ... | If the value is zero, ramp-up phase is skipped. +| | ... | If the value is zero, ramp-up phase is either skipped or size-limited. | | ... | Return type: float. | | | | ... | *Example:* @@ -192,12 +213,13 @@ | Get Ramp Up Rate | | [Documentation] | | ... | Return value of \${ramp_up_rate}, -| | ... | if not defined return \${max_rate}. +| | ... | if not defined, return zero. | | | | ... | The return value determines the rate for ramp-up phase. | | ... | Typically used to limit the rate when max rate | | ... | would lose packets in the ramp up phase, thus not setting | | ... | the DUT state correctly. +| | ... | If the value is zero, ramp-up phase should be skipped. | | ... | Return type: float. | | | | ... | *Example:* @@ -205,13 +227,12 @@ | | ... | \| \${ramp_up_rate} = \| Get Ramp Up Rate \| | | | | ${ramp_up_rate} = | Get Variable Value | \${ramp_up_rate} | ${0.0} -| | Return From Keyword If | ${ramp_up_rate} | ${ramp_up_rate} -| | Run Keyword And Return | Get Max Rate +| | Return From Keyword | ${ramp_up_rate} | Get Rate For Teardown | | [Documentation] | | ... | Return value of \${rate_for_teardown}, -| | ... | if not defined (or zero) return the min rate. +| | ... | if not defined (or zero) return the soft min rate. | | | | ... | The return value determines the rate for teardown trial, | | ... | that is executed if a perf test fails. @@ -226,7 +247,7 @@ | | | | ${rate_for_teardown} = | Get Variable Value | \${rate_for_teardown} | ${0.0} | | Return From Keyword If | ${rate_for_teardown} | ${rate_for_teardown} -| | Run Keyword And Return | Get Min Rate +| | Run Keyword And Return | Get Min Rate Soft | Get Resetter | | [Documentation] @@ -371,54 +392,44 @@ | Set Jumbo | | [Documentation] -| | ... | For jumbo frames detection, the maximal packet size is relevant, -| | ... | encapsulation overhead (if any) has effect. -| | -| | ... | This keyword computes jumbo boolean (some suites need that for -| | ... | configuration decisions). +| | ... | Call Set Numeric Frame Sizes and set jumbo based on max framesize. | | ... | To streamline suite autogeneration, both input and output values | | ... | are communicated as test (or broader scope) variables, | | ... | instead of explicit arguments and return values. | | | | ... | *Test (or broader scope) variables read:* | | ... | - overhead - Overhead in bytes; default value: 0. Type: integer -| | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: integer or -| | ... | string +| | ... | - frame_size - Framesize. Type: integer or string | | | | ... | *Test variables set:* | | ... | - jumbo - Jumbo boolean, true if jumbo packet support has to be | | ... | enabled. Type: boolean +| | ... | - recommended_mtu - Resonable value (with space for encap overhead) +| | ... | according to jumbo. Type: int | | | | ... | *Example:* | | | | ... | \| Set Jumbo \| | | -| | # Already called by Set Max Rate And Jumbo, but some suites (e.g. device) -| | # are calling this directly. +| | # Some suites (e.g. device) are not calling Set Max Rate And Jumbo. | | Set Numeric Frame Sizes -| | ${jumbo} = | Set Variable If | ${max_frame_size} < 1522 -| | ... | ${False} | ${True} +| | ${jumbo} = | Evaluate | ${max_frame_size} >= 1522 | | Set Test Variable | \${jumbo} +| | ${recommended_mtu} = | Set Variable If | ${jumbo} | ${9200} | ${1800} +| | Set Test Variable | \${recommended_mtu} | Set Max Rate And Jumbo | | [Documentation] -| | ... | Input framesize can be either integer in case of a single packet -| | ... | in stream, or IMIX string defining mix of packets. -| | ... | For jumbo frames detection, the maximal packet size is relevant. -| | ... | For maximal transmit rate, the average packet size is relevant. -| | ... | In both cases, encapsulation overhead (if any) has effect. -| | ... | The maximal rate is computed from NIC name. -| | ... | The implementation works by mapping from exact -| | ... | whitelisted NIC names. -| | ... | The mapping is hardcoded in nic_limits.yaml -| | ... | TODO: Make the mapping from NIC names case insensistive. -| | -| | ... | This keyword computes maximal unidirectional transmit rate -| | ... | and jumbo boolean (some suites need that for configuration decisions). +| | ... | This keyword computes maximal unidirectional transmit rate, +| | ... | jumbo boolean (some suites need that for configuration decisions), +| | ... | and recommended MTU value (depends on jumbo). | | ... | To streamline suite autogeneration, both input and output values | | ... | are communicated as test (or broader scope) variables, | | ... | instead of explicit arguments and return values. | | +| | ... | For correctly applying bandwidth limit, average frame size is used, +| | ... | see Set Numeric Frame Sizes keyword documentation for details. +| | | | ... | If this keyword detects the test is interested in (unidirectional) | | ... | transactons per second maximal rate (tps), that is returned (not pps). | | @@ -427,8 +438,13 @@ | | ... | - overhead - Overhead in bytes; default value: 0. Type: integer | | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: integer or | | ... | string -| | ... | - packets_per_transaction_and_direction - Pps-tps conversion. -| | ... | Optional, default 1. +| | ... | - ASTF_N_DATA_FRAMES - Number of data frames per transaction +| | ... | and direction. Type: integer +| | ... | - packets_per_transaction_and_direction - May be unset. +| | ... | See Get Packets Per Transaction And Direction keyword. Type: integer +| | ... | - packets_per_transaction_aggregated - May be unset. +| | ... | See Get Packets Per Transaction Aggregated keyword. Type: integer +| | ... | - TEST_TAGS - Robot tags of this test. Type: list of string | | | | ... | *Test variables set:* | | ... | - max_rate - Calculated unidirectional maximal transmit rate [pps]. @@ -436,8 +452,13 @@ | | ... | Type: float | | ... | - jumbo - Jumbo boolean, true if jumbo packet support has to be | | ... | enabled. Type: boolean -| | ... | avg_frame_size - Average frame size including overhead. Type: float -| | ... | max_frame_size - Maximal frame size including overhead. Type: float +| | ... | - recommended_mtu - Resonable value (with space for encap overhead) +| | ... | according to jumbo. Type: int +| | ... | - max_frame_size - Maximal frame size including overhead. Type: float +| | ... | - avg_directional_frame_size - Average frame size including overhead +| | ... | for the more loaded direction. Type: float +| | ... | - avg_aggregated_frame_size - Average frame size including overhead +| | ... | across both traffic directions. Type: float | | | | ... | *Example:* | | @@ -449,67 +470,192 @@ | | ... | ${NIC_NAME_TO_PPS_LIMIT} | ${nic_name} | | ${bps_limit} = | Get From Dictionary | | ... | ${NIC_NAME_TO_BPS_LIMIT} | ${nic_name} -| | Set Numeric Frame Sizes -| | # We need to add 20B (Ethernet preamble and inter-frame gap) -| | # to avg_frame_size -| | ${rate} = | Evaluate | ${bps_limit} / ((${avg_frame_size} + 20.0) * 8) +| | # Set Jumbo also calls Set Numeric Frame Sizes. +| | Set Jumbo +| | # We need to add 20B (Ethernet preamble and inter-frame gap). +| | ${adfs} = | Get Variable Value | \${avg_directional_frame_size} +| | ${rate} = | Evaluate | ${bps_limit} / ((${adfs} + 20.0) * 8) | | ${max_rate} = | Set Variable If | ${rate} > ${pps_limit} | | ... | ${pps_limit} | ${rate} | | ${pptad} = | Get Packets Per Transaction And Direction | | ${max_rate} = | Evaluate | ${max_rate} / ${pptad} | | Set Test Variable | \${max_rate} -| | Set Jumbo | Set Numeric Frame Sizes | | [Documentation] | | ... | Framesize can be either integer in case of a single packet | | ... | in stream, or set of packets in case of IMIX type or simmilar. | | ... | For jumbo decisions, we need a numeric size of the biggest packet. -| | ... | For max rate decisions, we need a numeric average packet size. -| | ... | This keyword computes both and sets them as test variables. +| | ... | For bandwidth limit decisions, we need a numeric average packet size +| | ... | in the more bit intensive direction if traffic is non-symmetric. +| | ... | Computation of max_rate assumes it is also the more pps direction +| | ... | (so it can have smaller average frame size than the aggregated one). +| | ... | Average (across both directions) frame size is also used +| | ... | for displaying the bidirectional bandwidth forwarded. +| | ... | This keyword computes all three values (accounting for overheads) +| | ... | and sets them as test variables. | | | | ... | Each suite sets a value named \${overhead}, | | ... | which describes by how many bytes the frames on DUT-DUT link | | ... | are larger (due to encapsulation) than those -| | ... | on the primary TG-DUT link. But for some suites that value +| | ... | on the primary TG-DUT link. For some suites that value | | ... | can be negaive (if TG-DUT is encapsulated more heavily). | | ... | For calculations in this keyword, we need largest sizes | | ... | across links, so zero is used if \${overhead} is negative. | | +| | ... | The other overhead is from TCP control packets (only IPv4 supported). +| | ... | TCP_CPS tests have SYN frames of length 78B and other frames 70B. +| | ... | The more loaded is client-to-server direction with 1 SYN and 3 other, +| | ... | across both directions it is 2 SYN and 5 other. +| | ... | TCP_PPS and TCP_TPUT tests have one other control packet less +| | ... | (in the less loaded direction), but they do contain data frames. +| | | | ... | *Test variables read:* | | ... | - frame_size - Framesize. Type: integer or string | | ... | - overhead - Overhead in bytes; default value: ${0}. Type: integer +| | ... | - ASTF_N_DATA_FRAMES - Number of data frames per transaction +| | ... | and direction. Type: integer +| | ... | - packets_per_transaction_and_direction - May be unset. +| | ... | See Get Packets Per Transaction And Direction keyword. Type: integer +| | ... | - packets_per_transaction_aggregated - May be unset. +| | ... | See Get Packets Per Transaction Aggregated keyword. Type: integer +| | ... | - TEST_TAGS - Robot tags of this test. Type: list of string | | | | ... | *Test variables set* -| | ... | avg_frame_size - Average frame size including overhead. Type: float -| | ... | max_frame_size - Maximal frame size including overhead. Type: float +| | ... | - max_frame_size - Maximal frame size including overhead. Type: float +| | ... | - avg_directional_frame_size - Average frame size including overhead +| | ... | for the more loaded direction. Type: float +| | ... | - avg_aggregated_frame_size - Average frame size including overhead +| | ... | across both traffic directions. Type: float | | | | ... | *Example:* | | | | ... | \| Set Numeric Frame Sizes \| | | +| | ${bare_max_frame_size} = | Run Keyword If +| | ... | '${frame_size}' == 'IMIX_v4_1' | Set Variable | ${1518.0} +| | ... | ELSE | Convert To Number | ${frame_size} +| | ${bafs} = | Run Keyword If +| | ... | '${frame_size}' == 'IMIX_v4_1' | Set Variable | ${353.8333333333333} +| | ... | ELSE | Convert To Number | ${frame_size} +| | # Long boolean formula in 2 lines. +| | ${is_cps} = | Evaluate | 'TCP_CPS' in ${TEST_TAGS} +| | ${is_pps} = | Evaluate | 'TCP_PPS' in ${TEST_TAGS} +| | ${is_tput} = | Evaluate | ${is_pps} or 'TCP_TPUT' in ${TEST_TAGS} +| | # TODO: Investigate impact of values on hoststack tests. +| | ${is_cps} = | Evaluate | ${is_cps} and 'HOSTSTACK' not in ${TEST_TAGS} +| | ${is_tput} = | Evaluate | ${is_tput} and 'HOSTSTACK' not in ${TEST_TAGS} +| | ${avg_dir_frame_size} | ${avg_agg_frame_size} = | Run Keyword If +| | ... | ${is_cps} | Apply Tcp Cps Proto Overhead | ${bafs} +| | ... | ELSE IF | ${is_tput} | Apply Tcp Tput Proto Overhead | ${bafs} +| | ... | ELSE | Set Variable | ${bafs} | ${bafs} | | ${max_overhead} = | Set Variable If | ${overhead} >= 0 | ${overhead} | ${0} -| | ${bare_avg_frame_size} = | Run Keyword If | '${frame_size}' == 'IMIX_v4_1' -| | ... | Set Variable | ${353.83333} -| | ... | ELSE -| | ... | Convert To Number | ${frame_size} -| | ${avg_frame_size} = | Evaluate | $bare_avg_frame_size + $max_overhead -| | Set Test Variable | \${avg_frame_size} -| | ${bare_max_frame_size} = | Run Keyword If | '${frame_size}' == 'IMIX_v4_1' -| | ... | Set Variable | ${1518} -| | ... | ELSE -| | ... | Convert To Number | ${frame_size} -| | ${max_frame_size} = | Evaluate | $bare_max_frame_size + $max_overhead -| | Set Test Variable | ${max_frame_size} +| | ${mfs} = | Evaluate | ${bare_max_frame_size} + ${max_overhead} +| | ${adfs} = | Evaluate | ${avg_dir_frame_size} + ${max_overhead} +| | ${aafs} = | Evaluate | ${avg_agg_frame_size} + ${max_overhead} +| | Set Test Variable | \${max_frame_size} | ${mfs} +| | Set Test Variable | \${avg_directional_frame_size} | ${adfs} +| | Set Test Variable | \${avg_aggregated_frame_size} | ${aafs} + +| Apply Tcp Cps Proto Overhead +| | [Documentation] +| | ... | Recompute average frame size for TCP CPS test cases. +| | +| | ... | This is contitionally called from Set Numeric Frame Sizes. +| | ... | In Robot Framework it is more convenient to wrap such a block +| | ... | as a standalone keyword to Run Keyword If. +| | +| | ... | *Test variables read:* +| | ... | - ASTF_N_DATA_FRAMES - Number of data frames per transaction +| | ... | and direction. Usually set globally. Type: integer +| | ... | - packets_per_transaction_and_direction - May be unset. +| | ... | See Get Packets Per Transaction And Direction keyword. Type: integer +| | ... | - packets_per_transaction_aggregated - May be unset. +| | ... | See Get Packets Per Transaction Aggregated keyword. Type: integer +| | +| | ... | *Arguments:* +| | ... | - bare_avg_frame_size - Average numeric framesize without overheads. +| | +| | ... | *Returns:* +| | ... | - avg_dir_frame_size - Average framesize for more loaded direction. +| | ... | - avg_agg_frame_size - Average framesize across both directions. +| | +| | ... | *Example:* +| | +| | ... | \| \${adfs} \| \${aafs} = \| Apply Tcp Cps Proto Overhead \| \${bafs} +| | +| | [Arguments] | ${bare_avg_frame_size} +| | +| | # Increase max_frame_size for TCP tests if used for more than just jumbo. +| | Run Keyword If | ${bare_avg_frame_size} != 64 +| | ... | Fail | TCP_CPS tests are only supported for (nominal) 64B frames. +| | # TODO: Unify with packets_per_transaction_* variables when adding PCAP. +| | ${pptad} = | Get Packets Per Transaction And Direction +| | ${ppta} = | Get Packets Per Transaction Aggregated +| | ${avg_dir_frame_size} = | Evaluate | (78.0 * 1 + 70.0 * 3) / (1 + 3) +| | Run Keyword If | '${pptad}' != '4' +| | ... | Fail | TCP CPS with pptad '${pptad}' != '4'. +| | ${avg_agg_frame_size} = | Evaluate | (78.0 * 2 + 70.0 * 5) / (2 + 5) +| | Run Keyword If | '${ppta}' != '7' +| | ... | Fail | TCP CPS with ppta '${ppta}' != '7'. +| | Return From Keyword | ${avg_dir_frame_size} | ${avg_agg_frame_size} + +| Apply Tcp Tput Proto Overhead +| | [Documentation] +| | ... | Recompute average frame size for TCP TPUT (or PPS) test cases. +| | +| | ... | This is contitionally called from Set Numeric Frame Sizes. +| | ... | In Robot Framework it is more convenient to wrap such a block +| | ... | as a standalone keyword to Run Keyword If. +| | +| | ... | *Test variables read:* +| | ... | - ASTF_N_DATA_FRAMES - Number of data frames per transaction +| | ... | and direction. Usually set globally. Type: integer +| | ... | - packets_per_transaction_and_direction - May be unset. +| | ... | See Get Packets Per Transaction And Direction keyword. Type: integer +| | ... | - packets_per_transaction_aggregated - May be unset. +| | ... | See Get Packets Per Transaction Aggregated keyword. Type: integer +| | +| | ... | *Arguments:* +| | ... | - bare_framesize - Average numeric framesize without overheads. +| | +| | ... | *Returns:* +| | ... | - avg_dir_frame_size - Average framesize for more loaded direction. +| | ... | - avg_agg_frame_size - Average framesize across both directions. +| | +| | ... | *Example:* +| | +| | ... | \| \${adfs} \| \${aafs} = \| Apply Tcp Cps Proto Overhead \| \${bafs} +| | +| | [Arguments] | ${bare_framesize} +| | +| | # TODO: Unify with packets_per_transaction_* variables when adding PCAP. +| | ${pptad} = | Get Packets Per Transaction And Direction +| | ${ppta} = | Get Packets Per Transaction Aggregated +| | # Long float formula in 4 lines. +| | ${numerator} = | Evaluate | ${bare_framesize} * ${ASTF_N_DATA_FRAMES} +| | ${numerator} = | Evaluate | 78.0 * 1 + 70.0 * 3 + ${numerator} +| | ${denominator} = | Evaluate | 1 + 3 + ${ASTF_N_DATA_FRAMES} +| | ${avg_dir_frame_size} = | Evaluate | ${numerator} / ${denominator} +| | Run Keyword If | '${pptad}' != '${denominator}' +| | ... | Fail | TCP TPUT with pptad '${pptad}' != '${denominator}'. +| | # Long float formula in 4 lines. +| | ${numerator} = | Evaluate | ${bare_framesize} * 2 * ${ASTF_N_DATA_FRAMES} +| | ${numerator} = | Evaluate | 78.0 * 2 + 70.0 * 4 + ${numerator} +| | ${denominator} = | Evaluate | 2 + 4 + 2 * ${ASTF_N_DATA_FRAMES} +| | ${avg_agg_frame_size} = | Evaluate | ${numerator} / ${denominator} +| | Run Keyword If | '${ppta}' != '${denominator}' +| | ... | Fail | TCP TPUT with ppta '${ppta}' != '${denominator}'. +| | Return From Keyword | ${avg_dir_frame_size} | ${avg_agg_frame_size} | Set Rates For Policer | | [Documentation] | | ... | Policer tests need these values, -| | ... | currently computed from \${avg_frame_size}. +| | ... | currently computed from \${avg_directional_frame_size}. | | ... | TODO: Verify the units match and computation is correct. | | | | ... | *Test (or broader scope) variables read:* -| | ... | - avg_frame_size - Average L2 Frame Size [B]. Type: float +| | ... | - avg_directional_frame_size - Average L2 Frame Size [B]. Type: float | | ... | Set by Set Max Rate And Jumbo keyword. | | | | ... | *Test variables set:* @@ -520,5 +666,5 @@ | | | | ... | \| Set Rates For Policer \| | | -| | Set Test Variable | \${eb} | ${avg_frame_size} -| | Set Test Variable | \${cb} | ${avg_frame_size} +| | Set Test Variable | \${eb} | ${avg_directional_frame_size} +| | Set Test Variable | \${cb} | ${avg_directional_frame_size} |