diff options
Diffstat (limited to 'resources/libraries/robot')
8 files changed, 1280 insertions, 714 deletions
diff --git a/resources/libraries/robot/ip/nat.robot b/resources/libraries/robot/ip/nat.robot index e80e1e1c85..0ebb58c2a0 100644 --- a/resources/libraries/robot/ip/nat.robot +++ b/resources/libraries/robot/ip/nat.robot @@ -40,10 +40,20 @@ | Initialize NAT44 endpoint-dependent mode in circular topology | | [Documentation] | Initialization of NAT44 endpoint-dependent mode on DUT1 | | +| | ... | This keyword also sets a test variable \${resetter} +| | ... | to hold a callable which resets VPP state. +| | ... | Keywords performing search will call it to get consistent trials. +| | ... | Tests which do not wish to reset NAT state should use ramp-up, +| | ... | so the resetter is not set if \${ramp_up_duration} variable +| | ... | is (defined and) nonzero. +| | | | Configure inside and outside interfaces | | ... | ${dut1} | ${DUT1_${int}1}[0] | ${DUT1_${int}2}[0] -| | Set NAT44 Address Range +| | ${resetter} = | Set NAT44 Address Range | | ... | ${dut1} | ${out_net} | ${out_net_end} +| | ${ramp_up_duration} = | Get Ramp Up Duration +| | Return From Keyword If | ${ramp_up_duration} +| | Set Test Variable | \${resetter} # TODO: Remove when 'ip4.Initialize IPv4 forwarding in circular topology' KW # adapted to use IP values from variables @@ -181,6 +191,13 @@ | Configure deterministic mode for NAT44 | | [Documentation] | Set deterministic behaviour of NAT44 (DET44). | | +| | ... | This keyword also sets a test variable \${resetter} +| | ... | to hold a callable which resets VPP state. +| | ... | Keywords performing search will call it to get consistent trials. +| | ... | Tests which do not wish to reset NAT state should use ramp-up, +| | ... | so the resetter is not set if \${ramp_up_duration} variable +| | ... | is (defined and) nonzero. +| | | | ... | *Arguments:* | | ... | - node - DUT node to set deterministic mode for NAT44 on. | | ... | Type: dictionary @@ -196,8 +213,11 @@ | | | | [Arguments] | ${node} | ${ip_in} | ${subnet_in} | ${ip_out} | ${subnet_out} | | -| | Set DET44 Mapping +| | ${resetter} = | Set DET44 Mapping | | ... | ${node} | ${ip_in} | ${subnet_in} | ${ip_out} | ${subnet_out} +| | ${ramp_up_duration} = | Get Ramp Up Duration +| | Return From Keyword If | ${ramp_up_duration} +| | Set Test Variable | \${resetter} | Initialize NAT44 deterministic mode in circular topology | | [Documentation] | Initialization of NAT44 deterministic mode (DET44) diff --git a/resources/libraries/robot/performance/performance_actions.robot b/resources/libraries/robot/performance/performance_actions.robot new file mode 100644 index 0000000000..2cf954a5b3 --- /dev/null +++ b/resources/libraries/robot/performance/performance_actions.robot @@ -0,0 +1,92 @@ +# Copyright (c) 2020 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Variables | resources/libraries/python/Constants.py +| Resource | resources/libraries/robot/performance/performance_utils.robot +| +| Documentation +| ... | Performance suite keywords - Actions related to performance tests. + +*** Keywords *** +| Additional Statistics Action For bash-perf-stat +| | [Documentation] +| | ... | Additional Statistics Action for bash command "perf stat". +| | +| | Run Keyword If | ${extended_debug}==${True} +| | ... | Perf Stat On All DUTs | ${nodes} | cpu_list=${cpu_alloc_str} + +| Additional Statistics Action For clear-show-runtime-with-traffic +| | [Documentation] +| | ... | Additional Statistics Action for clear and show runtime counters with +| | ... | running traffic. +| | +| | ... | See documentation of the called keyword for required test variables. +| | +| | Clear and show runtime counters with running traffic + +| Additional Statistics Action For noop +| | [Documentation] +| | ... | Additional Statistics Action for no operation. +| | +| | No operation + +| Additional Statistics Action For vpp-clear-runtime +| | [Documentation] +| | ... | Additional Statistics Action for clear VPP runtime. +| | +| | VPP Clear Runtime On All DUTs | ${nodes} + +| Additional Statistics Action For vpp-clear-stats +| | [Documentation] +| | ... | Additional Statistics Action for clear VPP statistics. +| | +| | Clear Statistics On All DUTs | ${nodes} + +| Additional Statistics Action For vpp-enable-elog +| | [Documentation] +| | ... | Additional Statistics Action for enable VPP elog trace. +| | +| | VPP Enable Elog Traces On All DUTs | ${nodes} + +| Additional Statistics Action For vpp-enable-packettrace +| | [Documentation] +| | ... | Additional Statistics Action for enable VPP packet trace. +| | +| | Run Keyword If | ${extended_debug}==${True} +| | ... | VPP Enable Traces On All DUTs | ${nodes} | fail_on_error=${False} + +| Additional Statistics Action For vpp-show-elog +| | [Documentation] +| | ... | Additional Statistics Action for show VPP elog trace. +| | +| | Show Event Logger On All DUTs | ${nodes} + +| Additional Statistics Action For vpp-show-packettrace +| | [Documentation] +| | ... | Additional Statistics Action for show VPP packet trace. +| | +| | Run Keyword If | ${extended_debug}==${True} +| | ... | Show Packet Trace On All Duts | ${nodes} | maximum=${100} + +| Additional Statistics Action For vpp-show-runtime +| | [Documentation] +| | ... | Additional Statistics Action for show VPP runtime. +| | +| | VPP Show Runtime On All DUTs | ${nodes} + +| Additional Statistics Action For vpp-show-stats +| | [Documentation] +| | ... | Additional Statistics Action for show VPP statistics. +| | +| | Show Statistics On All DUTs | ${nodes} diff --git a/resources/libraries/robot/performance/performance_display.robot b/resources/libraries/robot/performance/performance_display.robot new file mode 100644 index 0000000000..e8dfdbbfbd --- /dev/null +++ b/resources/libraries/robot/performance/performance_display.robot @@ -0,0 +1,240 @@ +# Copyright (c) 2020 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Documentation +| ... | Performance suite keywords - Displaying results as test messages. +| ... | This includes checks to fail test. + +*** Keywords *** +| Check NDRPDR interval validity +| | [Documentation] +| | ... | Extract loss ratio of lower bound of the interval. +| | ... | Fail if it does not reach the allowed value. +| | +| | ... | *Arguments:* +| | ... | - interval - Measured interval. Type: ReceiveRateInterval +| | ... | - packet_loss_ratio - Accepted loss (0.0 for NDR). Type: float +| | +| | ... | *Example:* +| | +| | ... | \| Check NDRPDR interval validity \| \${result.pdr_interval} \ +| | ... | \| \${0.005} \| +| | +| | [Arguments] | ${interval} | ${packet_loss_ratio}=${0.0} +| | +| | ${lower_bound} = | Set Variable | ${interval.measured_low} +| | ${lower_bound_lf} = | Set Variable | ${lower_bound.loss_fraction} +| | Return From Keyword If | ${lower_bound_lf} <= ${packet_loss_ratio} +| | Set Test Variable | \${rate_for_teardown} | ${lower_bound_lf} +| | ${message}= | Catenate | SEPARATOR=${SPACE} +| | ... | Minimal rate loss fraction ${lower_bound_lf} +| | ... | does not reach target ${packet_loss_ratio}. +| | ${message_zero} = | Set Variable | Zero packets forwarded! +| | ${message_other} = | Set Variable | ${lower_bound.loss_count} packets lost. +| | ${message} = | Set Variable If | ${lower_bound_lf} >= 1.0 +| | ... | ${message}${\n}${message_zero} | ${message}${\n}${message_other} +| | Fail | ${message} + +| Display Reconfig Test Message +| | [Documentation] +| | ... | Display the number of packets lost (bidirectionally) +| | ... | due to reconfiguration under traffic. +| | +| | ... | *Arguments:* +| | ... | - result - Result of bidirectional measurtement. +| | ... | Type: ReceiveRateMeasurement +| | +| | ... | *Example:* +| | +| | ... | \| Display Reconfig Test Message \| \${result} \| +| | +| | [Arguments] | ${result} +| | +| | ${ppta} = | Get Packets Per Transaction Aggregated +| | ${packet_rate} = | Evaluate | ${result.target_tr} * ${ppta} +| | ${packet_loss} = | Set Variable | ${result.loss_count} +| | ${time_loss} = | Evaluate | ${packet_loss} / ${packet_rate} +| | Set Test Message | Packets lost due to reconfig: ${packet_loss} +| | Set Test Message | ${\n}Implied time lost: ${time_loss} | append=yes + +| Display result of NDRPDR search +| | [Documentation] +| | ... | Display result of NDR+PDR search, both quantities, both bounds, +| | ... | aggregate in units given by trasaction type, e.g. by default +| | ... | in packet per seconds and Gbps total bandwidth +| | ... | (for initial packet size). +| | ... | +| | ... | The bound to display is encoded as target rate, it is assumed +| | ... | it is in transactions per second. Bidirectional traffic +| | ... | transaction is understood as having 2 packets, for this purpose. +| | ... | +| | ... | Througput is calculated as: +| | ... | Sum of measured rate over streams +| | ... | Bandwidth is calculated as: +| | ... | (Throughput * (L2 Frame Size + IPG) * 8) +| | ... | If the results contain latency data, display them for lower bounds. +| | +| | ... | *Test (or broader scope) variables read:* +| | ... | - frame_size_num - L2 Frame Size [B]. Type: integer or float +| | ... | - transaction_type - String identifier to determine how to count +| | ... | transactions. Default is "packet". +| | ... | *Arguments:* +| | ... | - result - Measured result data. Aggregate rate, tps or pps. +| | ... | Type: NdrPdrResult +| | +| | ... | *Example:* +| | +| | ... | \| Display result of NDRPDR search \| \${result} \| +| | +| | [Arguments] | ${result} +| | +| | Display single bound | NDR_LOWER +| | ... | ${result.ndr_interval.measured_low.target_tr} +| | ... | ${result.ndr_interval.measured_low.latency} +| | Display single bound | NDR_UPPER +| | ... | ${result.ndr_interval.measured_high.target_tr} +| | Display single bound | PDR_LOWER +| | ... | ${result.pdr_interval.measured_low.target_tr} +| | ... | ${result.pdr_interval.measured_low.latency} +| | Display single bound | PDR_UPPER +| | ... | ${result.pdr_interval.measured_high.target_tr} + +| Display result of soak search +| | [Documentation] +| | ... | Display result of soak search, avg+-stdev, as upper/lower bounds. +| | ... | See Display single bound for units used. +| | +| | ... | *Test (or broader scope) variables read:* +| | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: integer or +| | ... | string +| | ... | - transaction_type - String identifier to determine how to count +| | ... | transactions. Default is "packet". +| | ... | *Arguments:* +| | ... | - avg - Estimated average critical load [pps]. Type: float +| | ... | - stdev - Standard deviation of critical load [pps]. Type: float +| | +| | ... | *Returns:* +| | ... | - Lower and upper bound of critical load [pps]. Type: 2-tuple of float +| | +| | ... | *Example:* +| | +| | ... | \| Display result of soak search \| \${100000} \| \${100} \| +| | +| | [Arguments] | ${avg} | ${stdev} +| | +| | ${avg} = | Convert To Number | ${avg} +| | ${stdev} = | Convert To Number | ${stdev} +| | ${lower} = | Evaluate | ${avg} - ${stdev} +| | ${upper} = | Evaluate | ${avg} + ${stdev} +| | Display single bound | PLRsearch lower bound | ${lower} +| | Display single bound | PLRsearch upper bound | ${upper} +| | Return From Keyword | ${lower} | ${upper} + +| Display single bound +| | [Documentation] +| | ... | Compute and display one bound of NDR+PDR (or soak) search result. +| | ... | If the latency string is present, it is displayed as well. +| | ... | +| | ... | The bound to display is given as target transfer rate, it is assumed +| | ... | it is in transactions per second. Bidirectional traffic +| | ... | transaction is understood as having 2 packets, for this purpose. +| | ... | +| | ... | Pps values are aggregate in packet per seconds, +| | ... | and Gbps total bandwidth (for initial packet size). +| | ... | +| | ... | Througput is calculated as: +| | ... | Sum of measured rate over streams +| | ... | Bandwidth is calculated as: +| | ... | (Throughput * (L2 Frame Size + IPG) * 8) +| | ... | If the results contain latency data, display them for lower bounds. +| | +| | ... | *Test (or broader scope) variables read:* +| | ... | - transaction_type - String identifier to determine how to count +| | ... | transactions. Default is "packet". +| | ... | *Arguments:* +| | ... | - text - Flavor text describing which bound is this. Type: string +| | ... | - tps - Transaction rate [tps]. Type: float +| | ... | - latency - Latency data to display if non-empty. Type: string +| | +| | ... | *Example:* +| | +| | ... | \| Display single bound \| NDR lower bound \| \${12345.67} \ +| | ... | \| latency=\${EMPTY} \| +| | +| | [Arguments] | ${text} | ${tps} | ${latency}=${EMPTY} +| | +| | ${transaction_type} = | Get Transaction Type +| | Run Keyword And Return If | """_cps""" in """${transaction_type}""" +| | ... | Display single cps bound | ${text} | ${tps} | ${latency} +| | Display single pps bound | ${text} | ${tps} | ${latency} + +| Display single cps bound +| | [Documentation] +| | ... | Display one bound of NDR+PDR search for CPS tests. +| | ... | The bounds are expressed as transactions per second. +| | ... | If the latency string is present, it is displayed as well. +| | +| | ... | *Arguments:* +| | ... | - text - Flavor text describing which bound is this. Type: string +| | ... | - tps - Transaction rate [tps]. Type: float +| | ... | - latency - Latency data to display if non-empty. Type: string +| | +| | ... | *Example:* +| | +| | ... | \| Display single cps bound \| NDR lower bound \| \${12345.67} \ +| | ... | \| latency=\${EMPTY} \| +| | +| | [Arguments] | ${text} | ${tps} | ${latency}=${EMPTY} +| | +| | Set Test Message | ${\n}${text}: ${tps} CPS | append=yes +| | Return From Keyword If | not """${latency}""" +| | Set Test Message | ${\n}LATENCY [min/avg/max/hdrh] per stream: ${latency} +| | ... | append=yes + +| Display single pps bound +| | [Documentation] +| | ... | Display one pps bound of NDR+PDR search, +| | ... | aggregate in packet per seconds and Gbps total bandwidth +| | ... | (for initial packet size). +| | ... | +| | ... | The bound to display is given as target transfer rate, it is assumed +| | ... | it is in transactions per second. Bidirectional traffic +| | ... | transaction is understood as having 2 packets, for this purpose. +| | ... | +| | ... | Througput is calculated as: +| | ... | Sum of measured rates over streams +| | ... | Bandwidth is calculated as: +| | ... | (Throughput * (L2 Frame Size + IPG) * 8) +| | ... | If the latency string is present, it is displayed as well. +| | +| | ... | *Arguments:* +| | ... | - text - Flavor text describing which bound is this. Type: string +| | ... | - tps - Transaction rate [tps]. Type: float +| | ... | - latency - Latency data to display if non-empty. Type: string +| | +| | ... | *Example:* +| | +| | ... | \| Display single pps bound \| NDR lower bound \| \${12345.67} \ +| | ... | \| latency=\${EMPTY} \| +| | +| | [Arguments] | ${text} | ${tps} | ${latency}=${EMPTY} +| | +| | ${ppta} = | Get Packets Per Transaction Aggregated +| | ${pps} = | Evaluate | ${tps} * ${ppta} +| | ${bandwidth} = | Evaluate | ${pps} * (${avg_frame_size}+20)*8 / 1e9 +| | Set Test Message | ${\n}${text}: ${pps} pps, | append=yes +| | Set Test Message | ${bandwidth} Gbps (initial) | append=yes +| | Return From Keyword If | not """${latency}""" +| | Set Test Message | ${\n}LATENCY [min/avg/max/hdrh] per stream: ${latency} +| | ... | append=yes diff --git a/resources/libraries/robot/performance/performance_limits.robot b/resources/libraries/robot/performance/performance_limits.robot deleted file mode 100644 index 332ed9368b..0000000000 --- a/resources/libraries/robot/performance/performance_limits.robot +++ /dev/null @@ -1,153 +0,0 @@ -# Copyright (c) 2020 Cisco and/or its affiliates. -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at: -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -*** Settings *** -| Documentation | Performance suite keywords - Handling NIC and TG limits. -| Library | Collections -| Variables | ${CURDIR}/../../python/Constants.py - -*** Keywords *** -| Get Average Frame Size -| | [Documentation] -| | ... | Framesize can be either integer in case of a single packet -| | ... | in stream, or set of packets in case of IMIX type or simmilar. -| | -| | ... | *Arguments:* -| | ... | - frame_size - Framesize. Type: integer or string -| | ... | - overhead - Overhead in bytes; default value: ${0}. Type: integer -| | -| | ... | *Returns:* -| | ... | Average frame size including overhead. Type: float -| | -| | ... | *Example:* -| | -| | ... | \| Get Average Frame Size \| IMIX_v4_1 \| -| | -| | [Arguments] | ${frame_size} | ${overhead}=${0} -| | -| | ${frame_size} = | Run Keyword If | '${frame_size}' == 'IMIX_v4_1' -| | ... | Set Variable | ${353.83333} -| | ... | ELSE -| | ... | Convert To Number | ${frame_size} -| | ${avg_frame_size} = | Evaluate | ${frame_size} + ${overhead} -| | Return From Keyword | ${avg_frame_size} - -| Get Maximum Frame Size -| | [Documentation] -| | ... | Framesize can be either integer in case of a single packet -| | ... | in stream, or set of packets in case of IMIX type or simmilar. -| | -| | ... | *Arguments:* -| | ... | - frame_size - Framesize. Type: integer or string -| | ... | - overhead - Overhead in bytes; default value: ${0}. Type: integer -| | -| | ... | *Returns:* -| | ... | Maximum frame size including overhead. Type: float -| | -| | ... | *Example:* -| | -| | ... | \| Get Maximum Frame Size \| IMIX_v4_1 \| -| | -| | [Arguments] | ${frame_size} | ${overhead}=${0} -| | -| | ${frame_size} = | Run Keyword If | '${frame_size}' == 'IMIX_v4_1' -| | ... | Set Variable | ${1518} -| | ... | ELSE -| | ... | Convert To Number | ${frame_size} -| | ${max_frame_size} = | Evaluate | ${frame_size} + ${overhead} -| | Return From Keyword | ${max_frame_size} - -| Set Max Rate And Jumbo -| | [Documentation] -| | ... | Input framesize can be either integer in case of a single packet -| | ... | in stream, or IMIX string defining mix of packets. -| | ... | For jumbo frames detection, the maximal packet size is relevant. -| | ... | For maximal transmit rate, the average packet size is relevant. -| | ... | In both cases, encapsulation overhead (if any) has effect. -| | ... | The maximal rate is computed from NIC name. -| | ... | The implementation works by mapping from exact -| | ... | whitelisted NIC names. -| | ... | The mapping is hardcoded in nic_limits.yaml -| | ... | TODO: Make the mapping from NIC names case insensistive. -| | -| | ... | TODO: Make pps limit also definable per NIC. -| | -| | ... | This keyword computes maximal unidirectional transmit rate -| | ... | and jumbo boolean (some suites need that for configuration decisions). -| | ... | To streamline suite autogeneration, both input and output values -| | ... | are communicated as test (or broader scope) variables, -| | ... | instead of explicit arguments and return values. -| | -| | ... | *Test (or broader scope) variables read:* -| | ... | - nic_name - Name of bottleneck NIC. Type: string -| | ... | - overhead - Overhead in bytes; default value: 0. Type: integer -| | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: integer or -| | ... | string -| | -| | ... | *Test variables set:* -| | ... | - max_rate - Calculated unidirectional maximal transmit rate [pps]. -| | ... | Type: float -| | ... | - jumbo - Jumbo boolean, true if jumbo packet support has to be -| | ... | enabled. Type: boolean -| | -| | ... | *Example:* -| | -| | ... | \| Set Test Variable \| \${frame_size} \| IMIX_v4_1 \| -| | ... | \| Set Max Rate And Jumbo \| -| | -| | # Negative overhead is possible, if DUT-DUT traffic is less encapsulated -| | # than TG-DUT traffic. -| | # TODO: Re-check overhead values in suites with both traffics encapsulated. -| | # TODO: Improve layered setup to detect encap/decap and update overhead. -| | ${overhead} = | Set Variable If | ${overhead} >= 0 | ${overhead} | ${0} -| | ${pps_limit} = | Get From Dictionary -| | ... | ${NIC_NAME_TO_PPS_LIMIT} | ${nic_name} -| | ${bps_limit} = | Get From Dictionary -| | ... | ${NIC_NAME_TO_BPS_LIMIT} | ${nic_name} -| | # swo := size_with_overhead -| | ${avg_swo} = | Get Average Frame Size | ${frame_size} | ${overhead} -| | ${rate} = | Evaluate | ${bps_limit} / ((${avg_swo} + 20.0) * 8) -| | ${max_rate} = | Set Variable If | ${rate} > ${pps_limit} -| | ... | ${pps_limit} | ${rate} -| | Set Test Variable | \${max_rate} -| | Set Jumbo - -| Set Jumbo -| | [Documentation] -| | ... | For jumbo frames detection, the maximal packet size is relevant, -| | ... | encapsulation overhead (if any) has effect. -| | -| | ... | This keyword computes jumbo boolean (some suites need that for -| | ... | configuration decisions). -| | ... | To streamline suite autogeneration, both input and output values -| | ... | are communicated as test (or broader scope) variables, -| | ... | instead of explicit arguments and return values. -| | -| | ... | *Test (or broader scope) variables read:* -| | ... | - overhead - Overhead in bytes; default value: 0. Type: integer -| | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: integer or -| | ... | string -| | -| | ... | *Test variables set:* -| | ... | - jumbo - Jumbo boolean, true if jumbo packet support has to be -| | ... | enabled. Type: boolean -| | -| | ... | *Example:* -| | -| | ... | \| Set Jumnbo \| -| | -| | ${overhead} = | Set Variable If | ${overhead} >= 0 | ${overhead} | ${0} -| | ${max_swo} = | Get Maximum Frame Size | ${frame_size} | ${overhead} -| | ${jumbo} = | Set Variable If | ${max_swo} < 1522 -| | ... | ${False} | ${True} -| | Set Test Variable | \${jumbo} diff --git a/resources/libraries/robot/performance/performance_utils.robot b/resources/libraries/robot/performance/performance_utils.robot index b9c4dc08d1..ffafd40854 100644 --- a/resources/libraries/robot/performance/performance_utils.robot +++ b/resources/libraries/robot/performance/performance_utils.robot @@ -22,16 +22,116 @@ | Library | resources.libraries.python.TrafficGenerator.TGDropRateSearchImpl | Library | resources.libraries.python.Trace | Variables | resources/libraries/python/Constants.py +| Resource | resources/libraries/robot/performance/performance_actions.robot +| Resource | resources/libraries/robot/performance/performance_display.robot +| Resource | resources/libraries/robot/performance/performance_vars.robot | | Documentation | ... | Performance suite keywords - utilities to find and verify NDR and PDR. +| ... | See performance_vars.robot for values accessed via there. -*** Variables *** -| ${trial_duration}= | ${PERF_TRIAL_DURATION} -| ${trial_multiplicity}= | ${PERF_TRIAL_MULTIPLICITY} -| ${extended_debug}= | ${EXTENDED_DEBUG} +# Library/suite scope variables are maybe defined in performance_vars.robot. *** Keywords *** +| Clear and show runtime counters with running traffic +| | [Documentation] +| | ... | Start traffic at specified rate then clear runtime counters on all +| | ... | DUTs. Wait for specified amount of time and capture runtime counters +| | ... | on all DUTs. Finally stop traffic. +| | +| | ... | TODO: Support resetter if this is not the first trial-ish action? +| | +| | ... | *Example:* +| | +| | ... | \| Clear and show runtime counters with running traffic \| +| | +| | ${ppta} = | Get Packets Per Transaction Aggregated +| | ${runtime_duration} = | Get Runtime Duration +| | ${runtime_rate} = | Get Runtime Rate +| | ${traffic_directions} = | Get Traffic Directions +| | ${transaction_duration} = | Get Transaction Duration +| | ${transaction_scale} = | Get Transaction Scale +| | ${transaction_type} = | Get Transaction Type +| | ${use_latency} = | Get Use Latency +| | # Duration of -1 means we will stop traffic manually. +| | Send traffic on tg +| | ... | duration=${-1} +| | ... | rate=${runtime_rate} +| | ... | frame_size=${frame_size} +| | ... | traffic_profile=${traffic_profile} +| | ... | async_call=${True} +| | ... | ppta=${ppta} +| | ... | use_latency=${use_latency} +| | ... | traffic_directions=${traffic_directions} +| | ... | transaction_duration=${transaction_duration} +| | ... | transaction_scale=${transaction_scale} +| | ... | transaction_type=${transaction_type} +| | ... | duration_limit=${0.0} +| | FOR | ${action} | IN | @{pre_run_stats} +| | | Run Keyword | Additional Statistics Action For ${action} +| | END +| | Sleep | ${runtime_duration} +| | FOR | ${action} | IN | @{post_run_stats} +| | | Run Keyword | Additional Statistics Action For ${action} +| | END +| | Stop traffic on tg + +| Find critical load using PLRsearch +| | [Documentation] +| | ... | Find boundaries for troughput (of hardcoded target loss ratio) +| | ... | using PLRsearch algorithm. +| | ... | Display results as formatted test message. +| | ... | Fail if computed lower bound is 110% of the minimal rate or less. +| | ... | Input rates are understood as uni-directional, +| | ... | reported result contains aggregate rates. +| | ... | Currently, the min_rate value is hardcoded to match test teardowns. +| | ... | Call \${resetter} (if defined) to reset DUT state before each trial. +| | +| | ... | *Test (or broader scope) variables read:* +| | ... | - traffic_profile - Name of module defining traffc for measurements. +| | ... | Type: string +| | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: integer or +| | ... | string +| | ... | - max_rate - Calculated unidirectional maximal transmit rate [pps]. +| | ... | Type: float +| | +| | ... | *Example:* +| | +| | ... | \| Find critical load using PLR search \| +| | +| | # Get values via performance_vars. +| | ${max_rate} = | Get Max Rate +| | ${min_rate} = | Get Min Rate +| | ${ppta} = | Get Packets Per Transaction Aggregated +| | ${resetter} = | Get Resetter +| | ${traffic_directions} = | Get Traffic Directions +| | ${transaction_duration} = | Get Transaction Duration +| | ${transaction_scale} = | Get Transaction Scale +| | ${transaction_type} = | Get Transaction Type +| | ${use_latency} = | Get Use Latency +| | ${average} | ${stdev} = | Perform soak search +| | ... | frame_size=${frame_size} +| | ... | traffic_profile=${traffic_profile} +| | ... | minimum_transmit_rate=${min_rate} +| | ... | maximum_transmit_rate=${max_rate} +| | ... | plr_target=${1e-7} +| | ... | tdpt=${0.1} +| | ... | initial_count=${50} +| | ... | ppta=${ppta} +| | ... | resetter=${resetter} +| | ... | timeout=${1800.0} +| | ... | trace_enabled=${False} +| | ... | traffic_directions=${traffic_directions} +| | ... | transaction_scale=${transaction_scale} +| | ... | transaction_duration=${transaction_duration} +| | ... | transaction_type=${transaction_type} +| | ... | use_latency=${use_latency} +| | ${lower} | ${upper} = | Display result of soak search +| | ... | ${average} | ${stdev} +| | Set Test Variable | \${rate for teardown} | ${lower} +| | Should Not Be True | 1.1 * ${min_rate} > ${lower} +| | ... | Lower bound ${lower} too small for unidirectional minimum ${min_rate}. + | Find NDR and PDR intervals using optimized search | | [Documentation] | | ... | Find boundaries for RFC2544 compatible NDR and PDR values @@ -41,386 +141,243 @@ | | ... | Input rates are understood as uni-directional, | | ... | reported result contains aggregate rates. | | ... | Additional latency measurements are performed for smaller loads, -| | ... | their results are also displayed. +| | ... | even if latency stream is disabled in search. Their results +| | ... | are also displayed. | | ... | Finally, two measurements for runtime stats are done (not displayed). | | ... | Currently, the min_rate value is hardcoded to 90kpps, | | ... | allowing measurement at 10% of the discovered rate | | ... | without breaking latency streams. +| | ... | Call \${resetter} (if defined) to reset DUT state before each trial. | | | | ... | *Test (or broader scope) variables read:* | | ... | - traffic_profile - Name of module defining traffc for measurements. | | ... | Type: string | | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: integer or | | ... | string -| | ... | - max_rate - Calculated unidirectional maximal transmit rate [pps]. +| | ... | - max_rate - Calculated maximal transmit rate [tps]. | | ... | Type: float -| | -| | ... | *Arguments:* -| | ... | - packet_loss_ratio - Accepted loss during search. Type: float -| | ... | - final_relative_width - Maximal width multiple of upper. Type: float -| | ... | - final_trial_duration - Duration of final trials [s]. Type: float -| | ... | - initial_trial_duration - Duration of initial trials [s]. Type: float -| | ... | - intermediate phases - Number of intermediate phases [1]. -| | ... | Type: integer -| | ... | - timeout - Fail if search duration is longer [s]. Type: float -| | ... | - doublings - How many doublings to do when expanding [1]. -| | ... | Type: integer -| | ... | - traffic_directions - Bi- (2) or uni- (1) directional traffic. -| | ... | Type: integer -| | ... | - latency_duration - Duration for latency-specific trials. Type: float -| | ... | - latency - False to disable latency measurement; default value: True. -| | ... | Type: boolean +| | ... | - resetter - Callable to reset DUT state before each trial. +| | ... | - transaction_scale - Number of ASTF transaction (zero if unlimited). +| | ... | - transaction_type - String identifier to determine how to count +| | ... | transactions. Default is "packet". +| | ... | - disable_latency - If true, skip anything related to latency. +| | ... | Useful if transaction_scale is high and TPS is low. Default: false. | | | | ... | *Example:* | | -| | ... | \| Find NDR and PDR intervals using optimized search \| \${0.005} \ -| | ... | \| \${0.005} \| \${30.0} \| \${1.0} \| \${2} \| \${600.0} \| \${2} \ -| | ... | \| \${2} \| ${5.0} \| -| | -| | [Arguments] | ${packet_loss_ratio}=${0.005} -| | ... | ${final_relative_width}=${0.005} | ${final_trial_duration}=${30.0} -| | ... | ${initial_trial_duration}=${1.0} -| | ... | ${number_of_intermediate_phases}=${2} | ${timeout}=${720.0} -| | ... | ${doublings}=${2} | ${traffic_directions}=${2} -| | ... | ${latency_duration}=${PERF_TRIAL_LATENCY_DURATION} -| | ... | ${latency}=${True} -| | -| | # Latency measurements will need more than 9000 pps. -| | ${result} = | Perform optimized ndrpdr search | ${frame_size} -| | ... | ${traffic_profile} | ${9001} | ${max_rate} -| | ... | ${packet_loss_ratio} | ${final_relative_width} -| | ... | ${final_trial_duration} | ${initial_trial_duration} -| | ... | ${number_of_intermediate_phases} | timeout=${timeout} -| | ... | doublings=${doublings} | traffic_directions=${traffic_directions} -| | ... | latency=${latency} +| | ... | \| Find NDR and PDR intervals using optimized search \| +| | +| | # Get values via performance_vars. +| | ${disable_latency} = | Get Disable Latency +| | ${max_rate} = | Get Max Rate +| | ${min_rate} = | Get Min Rate +| | # \${packet_loss_ratio} is used twice so it is worth a variable. +| | ${packet_loss_ratio} = | Get Packet Loss Ratio +| | ${ppta} = | Get Packets Per Transaction Aggregated +| | ${resetter} = | Get Resetter +| | ${traffic_directions} = | Get Traffic Directions +| | ${transaction_duration} = | Get Transaction Duration +| | ${transaction_scale} = | Get Transaction Scale +| | ${transaction_type} = | Get Transaction Type +| | ${use_latency} = | Get Use Latency +| | ${result} = | Perform optimized ndrpdr search +| | ... | frame_size=${frame_size} +| | ... | traffic_profile=${traffic_profile} +| | ... | minimum_transmit_rate=${min_rate} +| | ... | maximum_transmit_rate=${max_rate} +| | ... | packet_loss_ratio=${packet_loss_ratio} +| | ... | final_relative_width=${0.005} +| | ... | final_trial_duration=${30.0} +| | ... | initial_trial_duration=${1.0} +| | ... | number_of_intermediate_phases=${2} +| | ... | timeout=${720.0} +| | ... | doublings=${2} +| | ... | ppta=${ppta} +| | ... | resetter=${resetter} +| | ... | traffic_directions=${traffic_directions} +| | ... | transaction_duration=${transaction_duration} +| | ... | transaction_scale=${transaction_scale} +| | ... | transaction_type=${transaction_type} +| | ... | use_latency=${use_latency} | | Display result of NDRPDR search | ${result} | | Check NDRPDR interval validity | ${result.pdr_interval} | | ... | ${packet_loss_ratio} | | Check NDRPDR interval validity | ${result.ndr_interval} -| | ${pdr_sum}= | Set Variable | ${result.pdr_interval.measured_low.target_tr} -| | ${pdr_per_stream}= | Evaluate | ${pdr_sum} / float(${traffic_directions}) -| | ${ndr_sum}= | Set Variable | ${result.ndr_interval.measured_low.target_tr} -| | ${ndr_per_stream}= | Evaluate | ${ndr_sum} / float(${traffic_directions}) -| | ${rate}= | Evaluate | 0.9 * ${pdr_per_stream} -| | Run Keyword If | ${latency} -| | ... | Measure and show latency at specified rate | Latency at 90% PDR: -| | ... | ${latency_duration} | ${rate} | ${framesize} -| | ... | ${traffic_profile} | ${traffic_directions} -| | ${rate}= | Evaluate | 0.5 * ${pdr_per_stream} -| | Run Keyword If | ${latency} -| | ... | Measure and show latency at specified rate | Latency at 50% PDR: -| | ... | ${latency_duration} | ${rate} | ${framesize} -| | ... | ${traffic_profile} | ${traffic_directions} -| | ${rate}= | Evaluate | 0.1 * ${pdr_per_stream} -| | Run Keyword If | ${latency} -| | ... | Measure and show latency at specified rate | Latency at 10% PDR: -| | ... | ${latency_duration} | ${rate} | ${framesize} -| | ... | ${traffic_profile} | ${traffic_directions} -| | Run Keyword If | ${latency} -| | ... | Measure and show latency at specified rate | Latency at 0% PDR: -| | ... | ${latency_duration} | ${0} | ${framesize} -| | ... | ${traffic_profile} | ${traffic_directions} -| | # Finally, trials with runtime and other stats. +| | ${pdr} = | Set Variable | ${result.pdr_interval.measured_low.target_tr} +| | ${ndr} = | Set Variable | ${result.ndr_interval.measured_low.target_tr} | | # We expect NDR and PDR to have different-looking stats. | | Send traffic at specified rate -| | ... | ${1.0} | ${pdr_per_stream} | ${framesize} | ${traffic_profile} -| | ... | traffic_directions=${traffic_directions} -| | Send traffic at specified rate -| | ... | ${1.0} | ${ndr_per_stream} | ${framesize} | ${traffic_profile} -| | ... | traffic_directions=${traffic_directions} +| | ... | rate=${pdr} +| | ... | trial_duration=${1.0} +| | ... | trial_multiplicity=${1} +| | ... | use_latency=${use_latency} +| | ... | duration_limit=${1.0} +| | Run Keyword If | ${ndr} != ${pdr} +| | ... | Send traffic at specified rate +| | ... | rate=${ndr} +| | ... | trial_duration=${1.0} +| | ... | trial_multiplicity=${1} +| | ... | use_latency=${use_latency} +| | ... | duration_limit=${1.0} +| | Return From Keyword If | ${disable_latency} +| | ${rate} = | Evaluate | 0.9 * ${pdr} +| | Measure and show latency at specified rate | Latency at 90% PDR: | ${rate} +| | ${rate} = | Evaluate | 0.5 * ${pdr} +| | Measure and show latency at specified rate | Latency at 50% PDR: | ${rate} +| | ${rate} = | Evaluate | 0.1 * ${pdr} +| | Measure and show latency at specified rate | Latency at 10% PDR: | ${rate} +| | Measure and show latency at specified rate | Latency at 0% PDR: | ${0.0} | Find Throughput Using MLRsearch | | [Documentation] -| | ... | Find and return lower bound PDR (zero PLR by default) +| | ... | Find and return lower bound NDR (zero PLR) | | ... | aggregate throughput using MLRsearch algorithm. | | ... | Input rates are understood as uni-directional. | | ... | Currently, the min_rate value is hardcoded to match test teardowns. +| | ... | Call \${resetter} (if defined) to reset DUT state before each trial. | | | | ... | *Test (or broader scope) variables read:* | | ... | - traffic_profile - Name of module defining traffc for measurements. | | ... | Type: string | | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: integer or | | ... | string -| | ... | - max_rate - Calculated unidirectional maximal transmit rate [pps]. +| | ... | - max_rate - Calculated maximal transmit rate [tps]. | | ... | Type: float -| | -| | ... | *Arguments:* -| | ... | - packet_loss_ratio - Accepted loss during search. Type: float -| | ... | - final_relative_width - Maximal width multiple of upper. Type: float -| | ... | - final_trial_duration - Duration of final trials [s]. Type: float -| | ... | - initial_trial_duration - Duration of initial trials [s]. Type: float -| | ... | - intermediate phases - Number of intermediate phases [1]. -| | ... | Type: integer -| | ... | - timeout - Fail if search duration is longer [s]. Type: float -| | ... | - doublings - How many doublings to do when expanding [1]. -| | ... | Type: integer -| | ... | - traffic_directions - Bi- (2) or uni- (1) directional traffic. -| | ... | Type: integer -| | ... | - latency - True to enable latency measurement; default value: False. -| | ... | Type: boolean +| | ... | - resetter - Callable to reset DUT state before each trial. +| | ... | - transaction_scale - Number of ASTF transaction (zero if unlimited). +| | ... | - transaction_type - String identifier to determine how to count +| | ... | transactions. Default is "packet". | | | | ... | *Returns:* | | ... | - Lower bound for bi-directional throughput at given PLR. Type: float | | | | ... | *Example:* | | -| | ... | \| \${throughpt}= \| Find Throughput Using MLRsearch \| \${0} \ -| | ... | \| \${0.001} \| \${10.0}\| \${1.0} \| \${1} \| \${720.0} \| \${2} \ -| | ... | \| \${2} \| -| | -| | [Arguments] | ${packet_loss_ratio}=${0.0} -| | ... | ${final_relative_width}=${0.001} | ${final_trial_duration}=${10.0} -| | ... | ${initial_trial_duration}=${1.0} -| | ... | ${number_of_intermediate_phases}=${1} | ${timeout}=${720.0} -| | ... | ${doublings}=${2} | ${traffic_directions}=${2} | ${latency}=${False} -| | -| | ${result} = | Perform optimized ndrpdr search | ${frame_size} -| | ... | ${traffic_profile} | ${10000} | ${max_rate} -| | ... | ${packet_loss_ratio} | ${final_relative_width} -| | ... | ${final_trial_duration} | ${initial_trial_duration} -| | ... | ${number_of_intermediate_phases} | timeout=${timeout} -| | ... | doublings=${doublings} | traffic_directions=${traffic_directions} -| | ... | latency=${latency} +| | ... | \| \${throughpt}= \| Find Throughput Using MLRsearch \| +| | +| | ${max_rate} = | Get Max Rate +| | ${min_rate} = | Get Min Rate +| | ${ppta} = | Get Packets Per Transaction Aggregated +| | ${resetter} = | Get Resetter +| | ${traffic_directions} = | Get Traffic Directions +| | ${transaction_duration} = | Get Transaction Duration +| | ${transaction_scale} = | Get Transaction Scale +| | ${transaction_type} = | Get Transaction Type +| | ${use_latency} = | Get Use Latency +| | ${result} = | Perform optimized ndrpdr search +| | ... | frame_size=${frame_size} +| | ... | traffic_profile=${traffic_profile} +| | ... | minimum_transmit_rate=${min_rate} +| | ... | maximum_transmit_rate=${max_rate} +| | ... | packet_loss_ratio=${0.0} +| | ... | final_relative_width=${0.001} +| | ... | final_trial_duration=${10.0} +| | ... | initial_trial_duration=${1.0} +| | ... | number_of_intermediate_phases=${1} +| | ... | timeout=${720} +| | ... | doublings=${2} +| | ... | ppta=${ppta} +| | ... | resetter=${resetter} +| | ... | traffic_directions=${traffic_directions} +| | ... | transaction_duration=${transaction_duration} +| | ... | transaction_scale=${transaction_scale} +| | ... | transaction_type=${transaction_type} +| | ... | use_latency=${use_latency} | | Check NDRPDR interval validity | ${result.pdr_interval} -| | ... | ${packet_loss_ratio} +| | ... | ${0.0} | | Return From Keyword | ${result.pdr_interval.measured_low.target_tr} -| Find critical load using PLRsearch -| | [Documentation] -| | ... | Find boundaries for troughput (of given target loss ratio) -| | ... | using PLRsearch algorithm. -| | ... | Display results as formatted test message. -| | ... | Fail if computed lower bound 110% of the minimal rate or less. -| | ... | Input rates are understood as uni-directional, -| | ... | reported result contains aggregate rates. -| | ... | Currently, the min_rate value is hardcoded to match test teardowns. -| | -| | ... | *Test (or broader scope) variables read:* -| | ... | - traffic_profile - Name of module defining traffc for measurements. -| | ... | Type: string -| | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: integer or -| | ... | string -| | ... | - max_rate - Calculated unidirectional maximal transmit rate [pps]. -| | ... | Type: float -| | -| | ... | *Arguments:* -| | ... | - packet_loss_ratio - Accepted loss during search. Type: float -| | ... | - timeout - Stop when search duration is longer [s]. Type: float -| | ... | - traffic_directions - Bi- (2) or uni- (1) directional traffic. -| | ... | Type: integer -| | ... | - latency - True to enable latency measurement; default value: False. -| | ... | Type: boolean -| | -| | ... | *Example:* -| | -| | ... | \| Find critical load using PLR search \| \${1e-7} \| \${120} \ -| | ... | \| \${2} \| -| | -| | [Arguments] | ${packet_loss_ratio}=${1e-7} | ${timeout}=${1800.0} -| | ... | ${traffic_directions}=${2} | ${latency}=${False} -| | -| | ${min_rate} = | Set Variable | ${10000} -| | ${average} | ${stdev} = | Perform soak search | ${frame_size} -| | ... | ${traffic_profile} | ${min_rate} | ${max_rate} -| | ... | ${packet_loss_ratio} | timeout=${timeout} -| | ... | traffic_directions=${traffic_directions} | latency=${latency} -| | ${lower} | ${upper} = | Display result of soak search -| | ... | ${average} | ${stdev} -| | Should Not Be True | 1.1 * ${traffic_directions} * ${min_rate} > ${lower} -| | ... | Lower bound ${lower} too small for unidirectional minimum ${min_rate}. - -| Display single bound -| | [Documentation] -| | ... | Display one bound of NDR+PDR search, -| | ... | in packet per seconds (total and per stream) -| | ... | and Gbps total bandwidth (for initial packet size). -| | ... | Througput is calculated as: -| | ... | Sum of measured rates over streams -| | ... | Bandwidth is calculated as: -| | ... | (Throughput * (L2 Frame Size + IPG) * 8) -| | ... | The given result should contain latency data as well. -| | -| | ... | *Arguments:* -| | ... | - text - Flavor text describing which bound is this. Type: string -| | ... | - rate_total - Total (not per stream) measured Tr [pps]. Type: float -| | ... | - frame_size - L2 Frame Size [B]. Type: integer -| | ... | - latency - Latency data to display if non-empty. Type: string -| | -| | ... | *Example:* -| | -| | ... | \| Display single bound \| NDR lower bound \| \${12345.67} \ -| | ... | \| \${64} \| latency=\${EMPTY} \| -| | -| | [Arguments] | ${text} | ${rate_total} | ${frame_size} | ${latency}=${EMPTY} -| | -| | ${bandwidth_total} = | Evaluate | ${rate_total} * (${frame_size}+20)*8 / 1e9 -| | Set Test Message | ${\n}${text}: ${rate_total} pps, | append=yes -| | Set Test Message | ${bandwidth_total} Gbps (initial) | append=yes -| | Return From Keyword If | not """${latency}""" -| | Set Test Message | ${\n}LATENCY [min/avg/max/hdrh] per stream: ${latency} -| | ... | append=yes - -| Display Reconfig Test Message -| | [Documentation] -| | ... | Display the number of packets lost (bidirectionally) -| | ... | due to reconfiguration under traffic. -| | -| | ... | *Arguments:* -| | ... | - result - Result of bidirectional measurtement. -| | ... | Type: ReceiveRateMeasurement -| | -| | ... | *Example:* -| | -| | ... | \| Display Reconfig Test Message \| \${result} \| -| | -| | [Arguments] | ${result} -| | -| | Set Test Message | Packets lost due to reconfig: ${result.loss_count} -| | ${time_lost} = | Evaluate | ${result.loss_count} / ${result.target_tr} -| | Set Test Message | ${\n}Implied time lost: ${time_lost} | append=yes - -| Display result of NDRPDR search -| | [Documentation] -| | ... | Display result of NDR+PDR search, both quantities, both bounds, -| | ... | aggregate in packet per seconds -| | ... | and Gbps total bandwidth (for initial packet size). -| | ... | Througput is calculated as: -| | ... | Sum of measured rate over streams -| | ... | Bandwidth is calculated as: -| | ... | (Throughput * (L2 Frame Size + IPG) * 8) -| | ... | The given result should contain latency data as well. -| | -| | ... | *Test (or broader scope) variables read:* -| | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: integer or -| | ... | string -| | ... | *Arguments:* -| | ... | - result - Measured result data per stream [pps]. Type: NdrPdrResult -| | -| | ... | *Example:* -| | -| | ... | \| Display result of NDRPDR search \| \${result} \| -| | -| | [Arguments] | ${result} -| | -| | ${frame_size} = | Get Average Frame Size | ${frame_size} -| | Display single bound | NDR_LOWER -| | ... | ${result.ndr_interval.measured_low.target_tr} | ${frame_size} -| | ... | ${result.ndr_interval.measured_low.latency} -| | Display single bound | NDR_UPPER -| | ... | ${result.ndr_interval.measured_high.target_tr} | ${frame_size} -| | Display single bound | PDR_LOWER -| | ... | ${result.pdr_interval.measured_low.target_tr} | ${frame_size} -| | ... | ${result.pdr_interval.measured_low.latency} -| | Display single bound | PDR_UPPER -| | ... | ${result.pdr_interval.measured_high.target_tr} | ${frame_size} - -| Display result of soak search -| | [Documentation] -| | ... | Display result of soak search, avg+-stdev, as upper/lower bounds, -| | ... | in aggregate packets per seconds -| | ... | and Gbps total bandwidth (for initial packet size). -| | ... | Througput is calculated as: -| | ... | Sum of measured rates over streams -| | ... | Bandwidth is calculated as: -| | ... | (Throughput * (L2 Frame Size + IPG) * 8) -| | -| | ... | *Test (or broader scope) variables read:* -| | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: integer or -| | ... | string -| | ... | *Arguments:* -| | ... | - avg - Estimated average critical load [pps]. Type: float -| | ... | - stdev - Standard deviation of critical load [pps]. Type: float -| | -| | ... | *Returns:* -| | ... | - Lower and upper bound of critical load [pps]. Type: 2-tuple of float -| | -| | ... | *Example:* -| | -| | ... | \| Display result of soak search \| \${100000} \| \${100} \| -| | -| | [Arguments] | ${avg} | ${stdev} -| | -| | ${frame_size} = | Get Average Frame Size | ${frame_size} -| | ${avg} = | Convert To Number | ${avg} -| | ${stdev} = | Convert To Number | ${stdev} -| | ${lower} = | Evaluate | ${avg} - ${stdev} -| | ${upper} = | Evaluate | ${avg} + ${stdev} -| | Display single bound | PLRsearch lower bound | ${lower} | ${frame_size} -| | Display single bound | PLRsearch upper bound | ${upper} | ${frame_size} -| | Return From Keyword | ${lower} | ${upper} - -| Check NDRPDR interval validity +| Measure and show latency at specified rate | | [Documentation] -| | ... | Extract loss ratio of lower bound of the interval. -| | ... | Fail if it does not reach the allowed value. +| | ... | Send traffic at specified rate, single trial. +| | ... | Extract latency information and append it to text message. +| | ... | The rate argument is int, so should not include "pps". +| | ... | If the given rate is too low, a safe value is used instead. +| | ... | Call \${resetter} (if defined) to reset DUT state before each trial. | | | | ... | *Arguments:* -| | ... | - interval - Measured interval. Type: ReceiveRateInterval -| | ... | - packet_loss_ratio - Accepted loss (0.0 for NDR). Type: float +| | ... | - message_prefix - Preface to test message addition. Type: string +| | ... | - rate - Rate [tps] for sending packets. +| | ... | Type: float | | | | ... | *Example:* | | -| | ... | \| Check NDRPDR interval validity \| \${result.pdr_interval} \ -| | ... | \| \${0.005} \| -| | -| | [Arguments] | ${interval} | ${packet_loss_ratio}=${0.0} -| | -| | ${lower_bound} = | Set Variable | ${interval.measured_low} -| | ${lower_bound_lf} = | Set Variable | ${lower_bound.loss_fraction} -| | Return From Keyword If | ${lower_bound_lf} <= ${packet_loss_ratio} -| | ${message}= | Catenate | SEPARATOR=${SPACE} -| | ... | Minimal rate loss fraction ${lower_bound_lf} -| | ... | does not reach target ${packet_loss_ratio}. -| | ${message_zero} = | Set Variable | Zero packets forwarded! -| | ${message_other} = | Set Variable | ${lower_bound.loss_count} packets lost. -| | ${message} = | Set Variable If | ${lower_bound_lf} >= 1.0 -| | ... | ${message}${\n}${message_zero} | ${message}${\n}${message_other} -| | Fail | ${message} +| | ... | \| Measure and show latency at specified rate \| Latency at 90% NDR \ +| | ... | \| ${10000000} \| +| | +| | [Arguments] | ${message_prefix} | ${rate} +| | +| | ${min_rate} = | Get Min Rate +| | ${ppta} = | Get Packets Per Transaction Aggregated +| | ${real_rate} = | Evaluate | max(${rate}, ${min_rate}) +| | ${traffic_directions} = | Get Traffic Directions +| | ${transaction_duration} = | Get Transaction Duration +| | ${transaction_scale} = | Get Transaction Scale +| | ${transaction_type} = | Get Transaction Type +| | Call Resetter +| | Send traffic on tg +| | ... | duration=${PERF_TRIAL_LATENCY_DURATION} +| | ... | rate=${real_rate} +| | ... | frame_size=${frame_size} +| | ... | traffic_profile=${traffic_profile} +| | ... | async_call=${False} +| | ... | duration_limit=${PERF_TRIAL_LATENCY_DURATION} +| | ... | ppta=${ppta} +| | ... | traffic_directions=${traffic_directions} +| | ... | transaction_duration=${transaction_duration} +| | ... | transaction_scale=${transaction_scale} +| | ... | transaction_type=${transaction_type} +| | ... | use_latency=${True} +| | ${latency} = | Get Latency Int +| | Set Test Message | ${\n}${message_prefix} ${latency} | append=${True} -| Traffic should pass with maximum rate +| Send ramp-up traffic | | [Documentation] -| | ... | Send traffic at maximum rate. +| | ... | Do nothing unless positive ramp-up duration is specified. +| | ... | Else perform one trial with appropriate rate and duration. +| | ... | This is useful for tests that set DUT state via traffic. +| | ... | Rate has to bee low enough so packets are not lost, +| | ... | Duration has to be long enough to set all the state. +| | ... | The trial results are discarded. | | | | ... | *Test (or broader scope) variables read:* | | ... | - traffic_profile - Name of module defining traffic for measurements. | | ... | Type: string | | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: integer or | | ... | string -| | ... | - max_rate - Calculated unidirectional maximal transmit rate [pps]. +| | ... | - ramp_up_duration - Suitable traffic duration [s]. | | ... | Type: float -| | -| | ... | *Arguments:* -| | ... | - trial_duration - Duration of single trial [s]. Type: float -| | ... | - fail_no_traffic - Whether to fail on zero receive count; -| | ... | default value: True. Type: boolean -| | ... | - trial_multiplicity - How many trials in this measurement. -| | ... | Type: integer -| | ... | - traffic_directions - Bi- (2) or uni- (1) directional traffic; -| | ... | default value: 2. Type: integer -| | ... | - tx_port - TX port of TG; default value: 0. Type: integer -| | ... | - rx_port - RX port of TG; default value: 1. Type: integer -| | ... | - latency - True to enable latency measurement; default value: False. -| | ... | Type: boolean +| | ... | - ramp_up_rate - Suitable transmit rate [tps]. +| | ... | Type: float +| | ... | - transaction_type - String identifier to determine how to count +| | ... | transactions. Default is "packet". | | | | ... | *Example:* | | -| | ... | \| Traffic should pass with maximum rate \| \${1} \| \${False} \ -| | ... | \| \${10.0} \| \${2} \| \${0} \| \${1} \| \${True} \| -| | -| | [Arguments] | ${trial_duration}=${trial_duration} -| | ... | ${fail_no_traffic}=${True} -| | ... | ${trial_multiplicity}=${trial_multiplicity} -| | ... | ${traffic_directions}=${2} | ${tx_port}=${0} | ${rx_port}=${1} -| | ... | ${latency}=${False} -| | -| | ${results}= | Send traffic at specified rate -| | ... | ${trial_duration} | ${max_rate} | ${frame_size} -| | ... | ${traffic_profile} | ${trial_multiplicity} -| | ... | ${traffic_directions} | ${tx_port} | ${rx_port} | latency=${latency} -| | Set Test Message | ${\n}Maximum Receive Rate trial results -| | Set Test Message | in packets per second: ${results} -| | ... | append=yes -| | Run Keyword If | ${fail_no_traffic} | Fail if no traffic forwarded +| | ... | \| Send ramp-up traffic \| +| | +| | ${ramp_up_duration} = | Get Ramp Up Duration +| | Run Keyword Unless | ${ramp_up_duration} > 0.0 | Return From Keyword +| | ${ramp_up_rate} = | Get Ramp Up Rate +| | ${ppta} = | Get Packets Per Transaction Aggregated +| | ${traffic_directions} = | Get Traffic Directions +| | ${transaction_duration} = | Get Transaction Duration +| | ${transaction_scale} = | Get Transaction Scale +| | ${transaction_type} = | Get Transaction Type +| | ${use_latency} = | Get Use Latency +| | Send traffic on tg +| | ... | duration=${ramp_up_duration} +| | ... | rate=${ramp_up_rate} +| | ... | frame_size=${frame_size} +| | ... | traffic_profile=${traffic_profile} +| | ... | async_call=${False} +| | ... | duration_limit=${0.0} +| | ... | ppta=${ppta} +| | ... | use_latency=${use_latency} +| | ... | traffic_directions=${traffic_directions} +| | ... | transaction_duration=${transaction_duration} +| | ... | transaction_scale=${transaction_scale} +| | ... | transaction_type=${transaction_type} | Send traffic at specified rate | | [Documentation] @@ -428,178 +385,62 @@ | | ... | Then send traffic at specified rate, possibly multiple trials. | | ... | Show various DUT stats, optionally also packet trace. | | ... | Return list of measured receive rates. +| | ... | Call \${resetter} (if defined) to reset DUT state before each trial. | | | | ... | *Arguments:* | | ... | - trial_duration - Duration of single trial [s]. Type: float -| | ... | - rate - Target aggregate transmit rate [pps] / Connections per second -| | ... | (CPS) for UDP/TCP flows. Type: float -| | ... | - frame_size - L2 Frame Size [B]. Type: integer or string -| | ... | - traffic_profile - Name of module defining traffc for measurements. +| | ... | - rate - Target transmit rate [tps]. Type: float | | ... | Type: string | | ... | - trial_multiplicity - How many trials in this measurement. -| | ... | Type: integer -| | ... | - traffic_directions - Bi- (2) or uni- (1) directional traffic. -| | ... | Type: integer -| | ... | - tx_port - TX port of TG; default value: 0. Type: integer -| | ... | - rx_port - RX port of TG; default value: 1. Type: integer -| | ... | - extended_debug - True to enable extended debug. | | ... | Type: boolean -| | ... | - latency - True to enable latency measurement; default value: False. +| | ... | - use_latency - Use latency stream in search; default value: False. | | ... | Type: boolean +| | ... | - duration_limit - Hard limit for trial duration, overriding duration +| | ... | computed from transaction_scale. Default 0.0 means no limit. | | | | ... | *Example:* | | | | ... | \| Send traffic at specified rate \| \${1.0} \| ${4000000.0} \ -| | ... | \| \${64} \| 3-node-IPv4 \| \${10} \| \${2} \| \${0} \| \${1} \ -| | ... | \| ${False} \| ${True} \| +| | ... | \| \${10} \| ${False} \| ${1.0} \| | | -| | [Arguments] | ${trial_duration} | ${rate} | ${frame_size} -| | ... | ${traffic_profile} | ${trial_multiplicity}=${trial_multiplicity} -| | ... | ${traffic_directions}=${2} | ${tx_port}=${0} | ${rx_port}=${1} -| | ... | ${extended_debug}=${extended_debug} | ${latency}=${False} -| | -| | Set Test Variable | ${extended_debug} -| | # Following setting of test variables is needed for some pre_stats actions. -| | Set Test Variable | ${rate} -| | Set Test Variable | ${traffic_directions} -| | Set Test Variable | ${tx_port} -| | Set Test Variable | ${rx_port} +| | [Arguments] | ${trial_duration} | ${rate} | ${trial_multiplicity} +| | ... | ${use_latency}=${False} | ${duration_limit}=${0.0} | | +| | ${ppta} = | Get Packets Per Transaction Aggregated +| | ${traffic_directions} = | Get Traffic Directions +| | ${transaction_duration} = | Get Transaction Duration +| | ${transaction_scale} = | Get Transaction Scale +| | ${transaction_type} = | Get Transaction Type +| | Set Test Variable | \${rate_for_teardown} | ${rate} | | FOR | ${action} | IN | @{pre_stats} | | | Run Keyword | Additional Statistics Action For ${action} | | END | | ${results} = | Create List | | FOR | ${i} | IN RANGE | ${trial_multiplicity} -| | | # The following line is skipping some default arguments, -| | | # that is why subsequent arguments have to be named. +| | | Call Resetter | | | Send traffic on tg -| | | ... | ${trial_duration} | ${rate} | ${frame_size} | ${traffic_profile} -| | | ... | warmup_time=${0} | traffic_directions=${traffic_directions} -| | | ... | tx_port=${tx_port} | rx_port=${rx_port} | latency=${latency} -| | | ${rx} = | Get Received -| | | ${rr} = | Evaluate | ${rx} / ${trial_duration} -| | | Append To List | ${results} | ${rr} +| | | ... | duration=${trial_duration} +| | | ... | rate=${rate} +| | | ... | frame_size=${frame_size} +| | | ... | traffic_profile=${traffic_profile} +| | | ... | async_call=${False} +| | | ... | duration_limit=${duration_limit} +| | | ... | ppta=${ppta} +| | | ... | traffic_directions=${traffic_directions} +| | | ... | transaction_duration=${transaction_duration} +| | | ... | transaction_scale=${transaction_scale} +| | | ... | transaction_type=${transaction_type} +| | | ... | use_latency=${use_latency} +| | | ${result}= | Get Measurement Result +| | | # Out of several quantities for aborted traffic (duration stretching), +| | | # the approximated receive rate is the best estimate we have. +| | | Append To List | ${results} | ${result.approximated_receive_rate} | | END | | FOR | ${action} | IN | @{post_stats} | | | Run Keyword | Additional Statistics Action For ${action} | | END | | Return From Keyword | ${results} -| Measure and show latency at specified rate -| | [Documentation] -| | ... | Send traffic at specified rate, single trial. -| | ... | Extract latency information and append it to text message. -| | ... | The rate argument is int, so should not include "pps". -| | ... | If the given rate is too low, a safe value is used instead. -| | -| | ... | *Arguments:* -| | ... | - message_prefix - Preface to test message addition. Type: string -| | ... | - trial_duration - Duration of single trial [s]. Type: float -| | ... | - rate - Rate [pps] for sending packets in case of T-Rex stateless -| | ... | mode or multiplier of profile CPS in case of T-Rex astf mode. -| | ... | Type: float -| | ... | - frame_size - L2 Frame Size [B]. Type: integer or string -| | ... | - traffic_profile - Name of module defining traffic for measurements. -| | ... | Type: string -| | ... | - traffic_directions - Bi- (2) or uni- (1) directional traffic. -| | ... | Type: integer -| | ... | - tx_port - TX port of TG; default value: 0. Type: integer -| | ... | - rx_port - RX port of TG; default value: 1. Type: integer -| | ... | - safe_rate - To apply if rate is below this, as latency pps is fixed. -| | ... | In pps. Type: integer. -| | -| | ... | *Example:* -| | -| | ... | \| Measure and show latency at specified rate \| Latency at 90% NDR \ -| | ... | \| \${1.0} \| ${10000000} \| \${64} \| 3-node-IPv4 \| \${2} \ -| | ... | \| \${0} \| \${1} \| ${9500} \| -| | -| | [Arguments] | ${message_prefix} | ${trial_duration} | ${rate} -| | ... | ${frame_size} | ${traffic_profile} | ${traffic_directions}=${2} -| | ... | ${tx_port}=${0} | ${rx_port}=${1} | ${safe_rate}=${9001} -| | -| | ${real_rate} = | Evaluate | max(${rate}, ${safe_rate}) -| | # The following line is skipping some default arguments, -| | # that is why subsequent arguments have to be named. -| | Send traffic on tg | ${trial_duration} | ${real_rate} | ${frame_size} -| | ... | ${traffic_profile} | warmup_time=${0} -| | ... | traffic_directions=${traffic_directions} | tx_port=${tx_port} -| | ... | rx_port=${rx_port} | latency=${True} -| | ${latency} = | Get Latency Int -| | Set Test Message | ${\n}${message_prefix} ${latency} | append=${True} - -| Clear and show runtime counters with running traffic -| | [Documentation] -| | ... | Start traffic at specified rate then clear runtime counters on all -| | ... | DUTs. Wait for specified amount of time and capture runtime counters -| | ... | on all DUTs. Finally stop traffic -| | -| | ... | *Arguments:* -| | ... | - duration - Duration of traffic run [s]. Type: integer -| | ... | - rate - Rate [pps] for sending packets in case of T-Rex stateless -| | ... | mode or multiplier of profile CPS in case of T-Rex astf mode. -| | ... | Type: float -| | ... | - frame_size - L2 Frame Size [B] or IMIX_v4_1. Type: integer or string -| | ... | - traffic_profile - Name of module defining traffc for measurements. -| | ... | Type: string -| | ... | - traffic_directions - Bi- (2) or uni- (1) directional traffic. -| | ... | Type: integer -| | ... | - tx_port - TX port of TG; default value: 0. Type: integer -| | ... | - rx_port - RX port of TG, default value: 1. Type: integer -| | -| | ... | *Example:* -| | -| | ... | \| Clear and show runtime counters with running traffic \| \${10} \ -| | ... | \| ${4000000.0} \| \${64} \| 3-node-IPv4 \| \${2} \| \${0} \| \${1} \| -| | -| | [Arguments] | ${duration} | ${rate} | ${frame_size} | ${traffic_profile} -| | ... | ${traffic_directions}=${2} | ${tx_port}=${0} | ${rx_port}=${1} -| | -| | # Duration of -1 means we will stop traffic manually. -| | Send traffic on tg | ${-1} | ${rate} | ${frame_size} | ${traffic_profile} -| | ... | warmup_time=${0} | async_call=${True} | latency=${False} -| | ... | traffic_directions=${traffic_directions} | tx_port=${tx_port} -| | ... | rx_port=${rx_port} -| | FOR | ${action} | IN | @{pre_run_stats} -| | | Run Keyword | Additional Statistics Action For ${action} -| | END -| | Sleep | ${duration} -| | FOR | ${action} | IN | @{post_run_stats} -| | | Run Keyword | Additional Statistics Action For ${action} -| | END -| | Stop traffic on tg - -| Send ramp-up traffic -| | [Documentation] -| | ... | Start ramp-up traffic at specified rate for defined duration. -| | -| | ... | *Arguments:* -| | ... | - duration - Duration of traffic run [s]. Type: integer -| | ... | - rate - Rate [pps] for sending packets in case of T-Rex stateless -| | ... | mode or multiplier of profile CPS in case of T-Rex astf mode. -| | ... | Type: float -| | ... | - frame_size - L2 Frame Size [B] or IMIX_v4_1. Type: integer or string -| | ... | - traffic_profile - Name of module defining traffc for measurements. -| | ... | Type: string -| | ... | - traffic_directions - Bi- (2) or uni- (1) directional traffic. -| | ... | Type: integer -| | ... | - tx_port - TX port of TG; default value: 0. Type: integer -| | ... | - rx_port - RX port of TG, default value: 1. Type: integer -| | -| | ... | *Example:* -| | -| | ... | \| Send ramp-up traffic \| \${10} \| ${400000.0} \| ${64} \ -| | ... | \| ${2} \| ${0} \| ${1} \| -| | -| | [Arguments] | ${duration}=${ramp_up_duration} | ${rate}=${ramp_up_rate} -| | ... | ${frame_size}=${frame_size} | ${traffic_profile}=${traffic_profile} -| | ... | ${traffic_directions}=${2} | ${tx_port}=${0} | ${rx_port}=${1} -| | -| | Send traffic on tg -| | ... | ${duration} | ${rate} | ${frame_size} | ${traffic_profile} -| | ... | warmup_time=${0} | traffic_directions=${traffic_directions} -| | ... | tx_port=${tx_port} | rx_port=${rx_port} | latency=${False} - | Start Traffic on Background | | [Documentation] | | ... | Start traffic at specified rate then return control to Robot. @@ -612,27 +453,36 @@ | | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: integer or | | ... | string | | ... | *Arguments:* -| | ... | - rate - Rate [pps] for sending packets in case of T-Rex stateless -| | ... | mode or multiplier of profile CPS in case of T-Rex astf mode. +| | ... | - rate - Rate [tps] for sending packets. | | ... | Type: float -| | ... | - traffic_directions - Bi- (2) or uni- (1) directional traffic. -| | ... | Type: integer -| | ... | - tx_port - TX port of TG; default value: 0. Type: integer -| | ... | - rx_port - RX port of TG; default value: 1. Type: integer | | | | ... | *Example:* | | -| | ... | \| Start Traffic on Background \| ${4000000.0} \| \${2} \| \${0} \ -| | ... | \| \${1} \| +| | ... | \| Start Traffic on Background \| ${4000000.0} \| | | -| | [Arguments] | ${rate} | ${traffic_directions}=${2} | ${tx_port}=${0} -| | ... | ${rx_port}=${1} +| | [Arguments] | ${rate} | | +| | ${ppta} = | Get Packets Per Transaction Aggregated +| | ${traffic_directions} = | Get Traffic Directions +| | ${transaction_duration} = | Get Transaction Duration +| | ${transaction_scale} = | Get Transaction Scale +| | ${transaction_type} = | Get Transaction Type +| | ${use_latency} = | Get Use Latency +| | Call Resetter | | # Duration of -1 means we will stop traffic manually. -| | Send traffic on tg | ${-1} | ${rate} | ${frame_size} | ${traffic_profile} -| | ... | warmup_time=${0} | async_call=${True} | latency=${False} -| | ... | traffic_directions=${traffic_directions} | tx_port=${tx_port} -| | ... | rx_port=${rx_port} +| | Send traffic on tg +| | ... | duration=${-1} +| | ... | rate=${rate} +| | ... | frame_size=${frame_size} +| | ... | traffic_profile=${traffic_profile} +| | ... | async_call=${True} +| | ... | duration_limit=${0.0} +| | ... | ppta=${ppta} +| | ... | traffic_directions=${traffic_directions} +| | ... | transaction_duration=${transaction_duration} +| | ... | transaction_scale=${transaction_scale} +| | ... | transaction_type=${transaction_type} +| | ... | use_latency=${use_latency} | Stop Running Traffic | | [Documentation] @@ -649,75 +499,41 @@ | | ${result}= | Stop traffic on tg | | Return From Keyword | ${result} -| Additional Statistics Action For vpp-clear-stats -| | [Documentation] -| | ... | Additional Statistics Action for clear VPP statistics. -| | -| | Clear Statistics On All DUTs | ${nodes} - -| Additional Statistics Action For vpp-show-stats -| | [Documentation] -| | ... | Additional Statistics Action for show VPP statistics. -| | -| | Show Statistics On All DUTs | ${nodes} - -| Additional Statistics Action For vpp-clear-runtime -| | [Documentation] -| | ... | Additional Statistics Action for clear VPP runtime. -| | -| | VPP Clear Runtime On All DUTs | ${nodes} - -| Additional Statistics Action For vpp-show-runtime -| | [Documentation] -| | ... | Additional Statistics Action for show VPP runtime. -| | -| | VPP Show Runtime On All DUTs | ${nodes} - -| Additional Statistics Action For vpp-enable-packettrace -| | [Documentation] -| | ... | Additional Statistics Action for enable VPP packet trace. -| | -| | Run Keyword If | ${extended_debug}==${True} -| | ... | VPP Enable Traces On All DUTs | ${nodes} | fail_on_error=${False} - -| Additional Statistics Action For vpp-show-packettrace -| | [Documentation] -| | ... | Additional Statistics Action for show VPP packet trace. -| | -| | Run Keyword If | ${extended_debug}==${True} -| | ... | Show Packet Trace On All Duts | ${nodes} | maximum=${100} - -| Additional Statistics Action For vpp-enable-elog -| | [Documentation] -| | ... | Additional Statistics Action for enable VPP elog trace. -| | -| | VPP Enable Elog Traces On All DUTs | ${nodes} - -| Additional Statistics Action For vpp-show-elog -| | [Documentation] -| | ... | Additional Statistics Action for show VPP elog trace. -| | -| | Show Event Logger On All DUTs | ${nodes} - -| Additional Statistics Action For bash-perf-stat +| Traffic should pass with maximum rate | | [Documentation] -| | ... | Additional Statistics Action for bash command "perf stat". +| | ... | Send traffic at maximum rate. +| | ... | Call \${resetter} (if defined) to reset DUT state before each trial. +| | ... | Fail if no packets were forwarded. | | -| | Run Keyword If | ${extended_debug}==${True} -| | ... | Perf Stat On All DUTs | ${nodes} | cpu_list=${cpu_alloc_str} - -| Additional Statistics Action For clear-show-runtime-with-traffic -| | [Documentation] -| | ... | Additional Statistics Action for clear and show runtime counters with -| | ... | running traffic. +| | ... | *Test (or broader scope) variables read:* +| | ... | - traffic_profile - Name of module defining traffic for measurements. +| | ... | Type: string +| | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: integer or +| | ... | string +| | ... | - max_rate - Calculated maximal transmit rate [tps]. +| | ... | Type: float +| | ... | - transaction_type - String identifier to determine how to count +| | ... | transactions. Default is "packet". | | -| | Clear and show runtime counters with running traffic -| | ... | ${trial_duration} | ${rate} -| | ... | ${frame_size} | ${traffic_profile} | ${traffic_directions} -| | ... | ${tx_port} | ${rx_port} - -| Additional Statistics Action For noop -| | [Documentation] -| | ... | Additional Statistics Action for no operation. +| | ... | *Example:* | | -| | No operation +| | ... | \| Traffic should pass with maximum rate \| +| | +| | ${max_rate} = | Get Max Rate +| | ${transaction_type} = | Get Transaction Type +| | ${trial_duration} = | Get Mrr Trial Duration +| | ${trial_multiplicity} = | Get Mrr Trial Multiplicity +| | ${use_latency} = | Get Use Latency +| | # The following also sets \${rate_for_teardown} +| | ${results} = | Send traffic at specified rate +| | ... | rate=${max_rate} +| | ... | trial_duration=${trial_duration} +| | ... | trial_multiplicity=${trial_multiplicity} +| | ... | use_latency=${use_latency} +| | ... | duration_limit=${0.0} +| | ${unit} = | Set Variable If | """_cps""" in """${transaction_type}""" +| | ... | estimated connections per second | packets per second +| | Set Test Message | ${\n}Maximum Receive Rate trial results +| | Set Test Message | in ${unit}: ${results} +| | ... | append=yes +| | Fail if no traffic forwarded diff --git a/resources/libraries/robot/performance/performance_vars.robot b/resources/libraries/robot/performance/performance_vars.robot new file mode 100644 index 0000000000..130237ed82 --- /dev/null +++ b/resources/libraries/robot/performance/performance_vars.robot @@ -0,0 +1,522 @@ +# Copyright (c) 2020 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Documentation | Performance suite keywords - Handling of various values +| ... | to allow autodetection, computation and overriding by suite variables. +| Library | Collections +| Variables | ${CURDIR}/../../python/Constants.py + +*** Variables *** +| ${extended_debug}= | ${EXTENDED_DEBUG} + +*** Keywords *** +| Get Disable Latency +| | [Documentation] +| | ... | If Get Use Latency returns true, return false. +| | ... | Otherwise return value of \${disable_latency} variable, +| | ... | or \${False} if not defined. +| | +| | ... | The return value controls whether latency trials in NDRPDR tests +| | ... | are executed. For example, ASTF tests do not support latency +| | ... | measurements yet, so executing the trials just wastes time. +| | ... | Return type: bool. +| | +| | ... | *Example:* +| | +| | ... | \| \${disable_latency} = \| Get Disable Latency \| +| | +| | ${use_latency} = | Get Use Latency +| | Return From Keyword If | ${use_latency} | ${False} +| | ${disable_latency} = | Get Variable Value | \${disable_latency} | ${False} +| | Return From Keyword | ${disable_latency} + +| Get Max Rate +| | [Documentation] +| | ... | Return value of \${max_rate} variable, +| | ... | fail if it is not defined. +| | ... | Call this just before calling a Python keyword, +| | ... | as those have restricted access to Robot variables. +| | +| | ... | The return value controls the maximal unidirectional packet rate. +| | ... | The value is also usable for minimal TPS value for ASTF tests. +| | ... | Return type: float. +| | +| | ... | *Example:* +| | +| | ... | \| \${max_rate} = \| Get Max Rate \| +| | +| | ${max_rate} = | Get Variable Value | \${max_rate} | ${0.0} +| | Return From Keyword If | ${max_rate} | ${max_rate} +| | Fail | \${max_rate} is not defined. Call Set Max Rate And Jumbo keyword. + +| Get Min Rate +| | [Documentation] +| | ... | Return a hardcoded value. This is an abstraction, useful in case +| | ... | we start allowing various other overrides or computations. +| | ... | Call this just before calling a Python keyword, +| | ... | as those have restricted access to Robot variables. +| | +| | ... | The return value controls the minimal unidirectional packet rate. +| | ... | The value is also usable for minimal TPS value for ASTF tests. +| | ... | The current value is the smallest one permitted +| | ... | by STL profiles with latency streams. +| | ... | Return type: float. +| | +| | ... | *Example:* +| | +| | ... | \| \${min_rate} = \| Get Min Rate \| +| | +| | Return From Keyword | ${9001.0} + +| Get Mrr Trial Duration +| | [Documentation] +| | ... | Return value from Constants. This is an abstraction, useful in case +| | ... | we start allowing various other overrides or computations. +| | ... | Call this just before calling a Python keyword, +| | ... | as those have restricted access to Robot variables. +| | +| | ... | The return value controls the duration of main trial measurement +| | ... | for MRR type tests. +| | ... | Return type: float. +| | +| | ... | *Example:* +| | +| | ... | \| \${mrr_trial_duration} = \| Get Mrr Trial Duration \| +| | +| | Return From Keyword | ${PERF_TRIAL_DURATION} + +| Get Mrr Trial Multiplicity +| | [Documentation] +| | ... | Return value from Constants. This is an abstraction, useful in case +| | ... | we start allowing various other overrides or computations. +| | ... | Call this just before calling a Python keyword, +| | ... | as those have restricted access to Robot variables. +| | +| | ... | The return value controls the number of main trial measurement +| | ... | for (B)MRR type tests. +| | ... | Return type: integer. +| | +| | ... | *Example:* +| | +| | ... | \| \${mrr_trial_multiplicity} = \| Get Mrr Trial Multiplicity \| +| | +| | Return From Keyword | ${PERF_TRIAL_MULTIPLICITY} + +| Get Packet Loss Ratio +| | [Documentation] +| | ... | Return a hardcoded value. This is an abstraction, useful in case +| | ... | we start allowing various other overrides or computations. +| | ... | Call this just before calling a Python keyword, +| | ... | as those have restricted access to Robot variables. +| | +| | ... | The return value controls the default packet loss ration for PDR +| | ... | in NDRPDR tests. Some other usages of MLRsearch (e.g. reconf tests) +| | ... | may use a different value. +| | ... | Return type: float. +| | +| | ... | *Example:* +| | +| | ... | \| \${packet_loss_ratio} = \| Get Packet Loss Ratio \| +| | +| | Return From Keyword | ${0.005} + +| Get Packets Per Transaction Aggregated +| | [Documentation] +| | ... | Return value of \${packets_per_transaction_aggregated}; +| | ... | if not defined, assume traffic is symmetric and compute +| | ... | from unidirectional values. +| | +| | ... | The return value is used when reporting PPS values from TPS found +| | ... | by some search (e.g. NDRPDR). +| | ... | Return type: integer. +| | +| | ... | *Example:* +| | +| | ... | \| \${ppta} = \| Get Packets Per Transaction Aggregated \| +| | +| | ${ppta} = | Get Variable Value | \${packets_per_transaction_aggregated} +| | ... | ${0} +| | Return From Keyword If | "${ppta}" != "0" | ${ppta} +| | # TODO: Insert TCP computation from packet size here. +| | ${pptad} = | Get Packets Per Transaction And Direction +| | ${traffic_directions} = | Get Traffic Directions +| | # We do not support ASTF profiles with multiple transactions, yet. +| | ${ppta} = | Evaluate | ${pptad} * ${traffic_directions} +| | Return From Keyword | ${ppta} + +| Get Packets Per Transaction And Direction +| | [Documentation] +| | ... | Return value of \${packets_per_transaction_and_direction}, +| | ... | or ${1} if not defined. +| | +| | ... | The return value is used when computing max rate (TPS), +| | ... | so for asymmetric transaction use the more numerous direction. +| | ... | Return type: integer. +| | +| | ... | *Example:* +| | +| | ... | \| \${pptad} = \| Get Packets Per Transaction And Direction \| +| | +| | ${pptad} = | Get Variable Value | \${packets_per_transaction_and_direction} +| | ... | ${1} +| | Return From Keyword | ${pptad} + +| Get Ramp Up Duration +| | [Documentation] +| | ... | Return value of \${ramp_up_duration}, +| | ... | or ${0.0} if not defined. +| | +| | ... | The return value determines the required duration of ramp-up phase. +| | ... | Typically used to prepare a specific state on DUT. +| | ... | If the value is zero, ramp-up phase is skipped. +| | ... | Return type: float. +| | +| | ... | *Example:* +| | +| | ... | \| \${ramp_up_duration} = \| Get Ramp Up Duration \| +| | +| | ${ramp_up_duration} = | Get Variable Value | \${ramp_up_duration} | ${0.0} +| | Return From Keyword | ${ramp_up_duration} + +| Get Ramp Up Rate +| | [Documentation] +| | ... | Return value of \${ramp_up_rate}, +| | ... | if not defined return \${max_rate}. +| | +| | ... | The return value determines the rate for ramp-up phase. +| | ... | Typically used to limit the rate when max rate +| | ... | would lose packets in the ramp up phase, thus not setting +| | ... | the DUT state correctly. +| | ... | Return type: float. +| | +| | ... | *Example:* +| | +| | ... | \| \${ramp_up_rate} = \| Get Ramp Up Rate \| +| | +| | ${ramp_up_rate} = | Get Variable Value | \${ramp_up_rate} | ${0.0} +| | Return From Keyword If | ${ramp_up_rate} | ${ramp_up_rate} +| | Run Keyword And Return | Get Max Rate + +| Get Rate For Teardown +| | [Documentation] +| | ... | Return value of \${rate_for_teardown}, +| | ... | if not defined (or zero) return the min rate. +| | +| | ... | The return value determines the rate for teardown trial, +| | ... | that is executed if a perf test fails. +| | ... | The \${rate_for_teardown} is usually not defined in suite, +| | ... | but search keywords set it in places where failure can occur, +| | ... | so the trial is done at the rate interesting for the failure. +| | ... | Return type: float. +| | +| | ... | *Example:* +| | +| | ... | \| \${rate_for_teardown} = \| Get Rate For Teardown \| +| | +| | ${rate_for_teardown} = | Get Variable Value | \${rate_for_teardown} | ${0.0} +| | Return From Keyword If | ${rate_for_teardown} | ${rate_for_teardown} +| | Run Keyword And Return | Get Min Rate + +| Get Resetter +| | [Documentation] +| | ... | Return value of \${resetter} variable, +| | ... | or \${None} if not defined. +| | +| | ... | If not \${None}, the returned value is callable. +| | ... | Its use is to reset DUT to initial conditions, +| | ... | for example to remove NAT sessions created in the previous trial. +| | +| | ... | *Example:* +| | +| | ... | \| \${resetter} = \| Get Resetter \| +| | +| | ${resetter} = | Get Variable Value | \${resetter} | ${None} +| | Return From Keyword | ${resetter} + +| Get Runtime Duration +| | [Documentation] +| | ... | Return value of \${runtime_duration} variable, +| | ... | if not defined return ${1.0}. +| | +| | ... | The return value controls the duration of runtime trial, +| | ... | which also acts as a warmup. Usually one second is enough, +| | ... | but some suites need longer time to set up state on DUT. +| | ... | Return type: float. +| | +| | ... | *Example:* +| | +| | ... | \| \${runtime_duration} = \| Get Runtime Duration \| +| | +| | ${runtime_duration} = | Get Variable Value | \${runtime_duration} | ${1.0} +| | Return From Keyword | ${runtime_duration} + +| Get Runtime Rate +| | [Documentation] +| | ... | Return value of \${runtime_rate} variable, +| | ... | if not defined return the max rate. +| | +| | ... | The return value controls the rate (TPS unidir) of runtime trial, +| | ... | which also acts as a warmup. No plans to ever use a different rate, +| | ... | but keywords look better if access to such values is uniform. +| | ... | Return type: float. +| | +| | ... | *Example:* +| | +| | ... | \| \${runtime_rate} = \| Get Runtime Rate \| +| | +| | ${runtime_rate} = | Get Variable Value | \${runtime_rate} | ${0.0} +| | Return From Keyword If | ${runtime_rate} | ${runtime_rate} +| | Run Keyword And Return | Get Max Rate + +| Get Traffic Directions +| | [Documentation] +| | ... | Return value of \${traffic_directions}, +| | ... | or ${2} if not defined. +| | +| | ... | The return value used when parsing for measurement results. +| | ... | This needs to be known already in profile driver, +| | ... | as bidirectional parsing may fail on unidirectional traffic. +| | ... | Return type: integer. +| | +| | ... | *Example:* +| | +| | ... | \| \${traffic_directions} = \| Get Traffic Directions \| +| | +| | ${traffic_directions} = | Get Variable Value | \${traffic_directions} | ${2} +| | Return From Keyword | ${traffic_directions} + +| Get Transaction Duration +| | [Documentation] +| | ... | Return value of \${transaction_duration} variable, +| | ... | or \${0.0} if not defined. +| | +| | ... | The return value is the expected duration of single (ASTF) transaction +| | ... | if it is not negligible for overall trial duration computation. +| | ... | Most tests use very short transactions (without explicit delays), +| | ... | so the zero default works (and suite saves one line +| | ... | of Variables table). +| | ... | Return type: float. +| | +| | ... | *Example:* +| | +| | ... | \| \${transaction_duration} = \| Get Transaction Duration \| +| | +| | ${transaction_duration} = | Get Variable Value | \${transaction_duration} +| | ... | ${0.0} +| | Return From Keyword | ${transaction_duration} + +| Get Transaction Scale +| | [Documentation] +| | ... | Return value of \${transaction_scale} variable, +| | ... | or \${0} if not defined. +| | +| | ... | Zero return value means the number of transactions is not limited, +| | ... | which is true for most STL TRex profiles (transaction is a packet). +| | ... | Nonzero return value means the number of transactions is fixed, +| | ... | for example in stateful NAT scale tests. +| | ... | Return type: integer. +| | +| | ... | *Example:* +| | +| | ... | \| \${transaction_scale} = \| Get Transaction Scale \| +| | +| | ${transaction_scale} = | Get Variable Value | \${transaction_scale} | ${0} +| | Return From Keyword | ${transaction_scale} + +| Get Transaction Type +| | [Documentation] +| | ... | Return value of \${transaction_type} variable, +| | ... | or "packet" if not defined. +| | +| | ... | The return value describes the type of transaction +| | ... | the test is executed. For example "packet" means a transaction +| | ... | is just a single packet. For more sophisticated transactions, +| | ... | the logic to determine the number of passed transactions +| | ... | is different from merely counting the packets received from DUT. +| | ... | Return type: string. +| | +| | ... | *Example:* +| | +| | ... | \| \${transaction_type} = \| Get Transaction Type \| +| | +| | ${transaction_type} = | Get Variable Value | \${transaction_type} | packet +| | Return From Keyword | ${transaction_type} + +| Get Use Latency +| | [Documentation] +| | ... | Return value of \${use_latency} variable, +| | ... | if not defined return the value from Constants. +| | +| | ... | The return value controls whether latency streams are active +| | ... | during the main search. +| | ... | Return type: bool. +| | +| | ... | *Example:* +| | +| | ... | \| \${use_latency} = \| Get Use Latency \| +| | +| | ${use_latency} = | Get Variable Value | ${use_latency} | ${PERF_USE_LATENCY} +| | Return From Keyword | ${use_latency} + +| Set Jumbo +| | [Documentation] +| | ... | For jumbo frames detection, the maximal packet size is relevant, +| | ... | encapsulation overhead (if any) has effect. +| | +| | ... | This keyword computes jumbo boolean (some suites need that for +| | ... | configuration decisions). +| | ... | To streamline suite autogeneration, both input and output values +| | ... | are communicated as test (or broader scope) variables, +| | ... | instead of explicit arguments and return values. +| | +| | ... | *Test (or broader scope) variables read:* +| | ... | - overhead - Overhead in bytes; default value: 0. Type: integer +| | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: integer or +| | ... | string +| | +| | ... | *Test variables set:* +| | ... | - jumbo - Jumbo boolean, true if jumbo packet support has to be +| | ... | enabled. Type: boolean +| | +| | ... | *Example:* +| | +| | ... | \| Set Jumbo \| +| | +| | # Already called by Set Max Rate And Jumbo, but some suites (e.g. device) +| | # are calling this directly. +| | Set Numeric Frame Sizes +| | ${jumbo} = | Set Variable If | ${max_frame_size} < 1522 +| | ... | ${False} | ${True} +| | Set Test Variable | \${jumbo} + +| Set Max Rate And Jumbo +| | [Documentation] +| | ... | Input framesize can be either integer in case of a single packet +| | ... | in stream, or IMIX string defining mix of packets. +| | ... | For jumbo frames detection, the maximal packet size is relevant. +| | ... | For maximal transmit rate, the average packet size is relevant. +| | ... | In both cases, encapsulation overhead (if any) has effect. +| | ... | The maximal rate is computed from NIC name. +| | ... | The implementation works by mapping from exact +| | ... | whitelisted NIC names. +| | ... | The mapping is hardcoded in nic_limits.yaml +| | ... | TODO: Make the mapping from NIC names case insensistive. +| | +| | ... | This keyword computes maximal unidirectional transmit rate +| | ... | and jumbo boolean (some suites need that for configuration decisions). +| | ... | To streamline suite autogeneration, both input and output values +| | ... | are communicated as test (or broader scope) variables, +| | ... | instead of explicit arguments and return values. +| | +| | ... | If this keyword detects the test is interested in (unidirectional) +| | ... | transactons per second maximal rate (tps), that is returned (not pps). +| | +| | ... | *Test (or broader scope) variables read:* +| | ... | - nic_name - Name of bottleneck NIC. Type: string +| | ... | - overhead - Overhead in bytes; default value: 0. Type: integer +| | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: integer or +| | ... | string +| | ... | - packets_per_transaction_and_direction - Pps-tps conversion. +| | ... | Optional, default 1. +| | +| | ... | *Test variables set:* +| | ... | - max_rate - Calculated unidirectional maximal transmit rate [pps]. +| | ... | This never exceeds bandwidth on TG-DUT nor DUT-DUT links. +| | ... | Type: float +| | ... | - jumbo - Jumbo boolean, true if jumbo packet support has to be +| | ... | enabled. Type: boolean +| | ... | avg_frame_size - Average frame size including overhead. Type: float +| | ... | max_frame_size - Maximal frame size including overhead. Type: float +| | +| | ... | *Example:* +| | +| | ... | \| Set Max Rate And Jumbo \| +| | +| | # TODO: Re-check overhead values in suites with both traffics encapsulated. +| | # TODO: Improve layered setup to detect encap/decap and update overhead. +| | ${pps_limit} = | Get From Dictionary +| | ... | ${NIC_NAME_TO_PPS_LIMIT} | ${nic_name} +| | ${bps_limit} = | Get From Dictionary +| | ... | ${NIC_NAME_TO_BPS_LIMIT} | ${nic_name} +| | Set Numeric Frame Sizes +| | ${rate} = | Evaluate | ${bps_limit} / ((${avg_frame_size} + 20.0) * 8) +| | ${max_rate} = | Set Variable If | ${rate} > ${pps_limit} +| | ... | ${pps_limit} | ${rate} +| | ${pptad} = | Get Packets Per Transaction And Direction +| | ${max_rate} = | Evaluate | ${max_rate} / ${pptad} +| | Set Test Variable | \${max_rate} +| | Set Jumbo + +| Set Numeric Frame Sizes +| | [Documentation] +| | ... | Framesize can be either integer in case of a single packet +| | ... | in stream, or set of packets in case of IMIX type or simmilar. +| | ... | For jumbo decisions, we need a numeric size of the biggest packet. +| | ... | For max rate decisions, we need a numeric average packet size. +| | ... | This keyword computes both and sets them as test variables. +| | +| | ... | Each suite sets a value named \${overhead}, +| | ... | which describes by how many bytes the frames on DUT-DUT link +| | ... | are larger (due to encapsulation) than those +| | ... | on the primary TG-DUT link. But for some suites that value +| | ... | can be negaive (if TG-DUT is encapsulated more heavily). +| | ... | For calculations in this keyword, we need largest sizes +| | ... | across links, so zero is used if \${overhead} is negative. +| | +| | ... | *Test variables read:* +| | ... | - frame_size - Framesize. Type: integer or string +| | ... | - overhead - Overhead in bytes; default value: ${0}. Type: integer +| | +| | ... | *Test variables set* +| | ... | avg_frame_size - Average frame size including overhead. Type: float +| | ... | max_frame_size - Maximal frame size including overhead. Type: float +| | +| | ... | *Example:* +| | +| | ... | \| Set Numeric Frame Sizes \| +| | +| | ${max_overhead} = | Set Variable If | ${overhead} >= 0 | ${overhead} | ${0} +| | ${bare_avg_frame_size} = | Run Keyword If | '${frame_size}' == 'IMIX_v4_1' +| | ... | Set Variable | ${353.83333} +| | ... | ELSE +| | ... | Convert To Number | ${frame_size} +| | ${avg_frame_size} = | Evaluate | $bare_avg_frame_size + $max_overhead +| | Set Test Variable | \${avg_frame_size} +| | ${bare_max_frame_size} = | Run Keyword If | '${frame_size}' == 'IMIX_v4_1' +| | ... | Set Variable | ${1518} +| | ... | ELSE +| | ... | Convert To Number | ${frame_size} +| | ${max_frame_size} = | Evaluate | $bare_max_frame_size + $max_overhead +| | Set Test Variable | ${max_frame_size} + +| Set Rates For Policer +| | [Documentation] +| | ... | Policer tests need these values, +| | ... | currently computed from \${avg_frame_size}. +| | ... | TODO: Verify the units match and computation is correct. +| | +| | ... | *Test (or broader scope) variables read:* +| | ... | - avg_frame_size - Average L2 Frame Size [B]. Type: float +| | ... | Set by Set Max Rate And Jumbo keyword. +| | +| | ... | *Test variables set:* +| | ... | - eb - Excess burst rate for policer. Type: float +| | ... | - cb - Committed burst rate for policer. Type: float +| | +| | ... | *Example:* +| | +| | ... | \| Set Rates For Policer \| +| | +| | Set Test Variable | \${eb} | ${avg_frame_size} +| | Set Test Variable | \${cb} | ${avg_frame_size} diff --git a/resources/libraries/robot/shared/default.robot b/resources/libraries/robot/shared/default.robot index cf6b599a29..fdc4da475e 100644 --- a/resources/libraries/robot/shared/default.robot +++ b/resources/libraries/robot/shared/default.robot @@ -58,7 +58,6 @@ | Resource | resources/libraries/robot/overlay/lisp.robot | Resource | resources/libraries/robot/overlay/lispgpe.robot | Resource | resources/libraries/robot/overlay/lisp_api.robot -| Resource | resources/libraries/robot/performance/performance_limits.robot | Resource | resources/libraries/robot/performance/performance_utils.robot | Resource | resources/libraries/robot/shared/interfaces.robot | Resource | resources/libraries/robot/shared/container.robot @@ -74,6 +73,26 @@ | ${cpu_alloc_str}= | ${0} *** Keywords *** +# TODO: Sort keywords alphabetically. + +| Call Resetter +| | [Documentation] +| | ... | Check for a presence of test variable \${resetter}. +| | ... | If it exists (and not None), call the resetter (as a Python callable). +| | ... | This is usually used to reset any state on DUT before next trial. +| | +| | ... | TODO: Move to a more specific library if needed. +| | +| | ... | *Example:* +| | +| | ... | \| Call Resetter \| +| | +| | ${resetter} = | Get Resetter +| | # See http://robotframework.org/robotframework/3.1.2/libraries/BuiltIn.html +| | # #Evaluating%20expressions for $variable (without braces) syntax. +| | # Parens are there to perform the call. +| | Run Keyword If | $resetter | Evaluate | $resetter() + | Configure crypto device on all DUTs | | [Documentation] | Verify if Crypto QAT device virtual functions are | | ... | initialized on all DUTs. If parameter force_init is set to True, then diff --git a/resources/libraries/robot/shared/test_teardown.robot b/resources/libraries/robot/shared/test_teardown.robot index 93c4574c83..09ebd9d01e 100644 --- a/resources/libraries/robot/shared/test_teardown.robot +++ b/resources/libraries/robot/shared/test_teardown.robot @@ -64,11 +64,21 @@ | Additional Test Tear Down Action For performance | | [Documentation] | | ... | Additional teardown for tests which uses performance measurement. -| | -| | Run Keyword If Test Failed -| | ... | Send traffic at specified rate | ${1.0} | 10000 -| | ... | ${frame_size} | ${traffic_profile} | trial_multiplicity=${1} -| | ... | extended_debug=${True} +| | ... | Optionally, call \${resetter} (if defined) to reset DUT state. +| | +| | ... | TODO: Document what test variables are required or optional. +| | +| | Run Keyword If Test Passed | Return From Keyword +| | ${use_latency} = | Get Use Latency +| | ${rate_for_teardown} = | Get Rate For Teardown +| | Call Resetter +| | Set Test Variable | \${extended_debug} | ${True} +| | Send traffic at specified rate +| | ... | trial_duration=${1.0} +| | ... | rate=${rate_for_teardown} +| | ... | trial_multiplicity=${1} +| | ... | use_latency=${use_latency} +| | ... | duration_limit=${1.0} | Additional Test Tear Down Action For packet_trace | | [Documentation] |