diff options
Diffstat (limited to 'resources/libraries/robot')
4 files changed, 223 insertions, 60 deletions
diff --git a/resources/libraries/robot/l2/l2_bridge_domain.robot b/resources/libraries/robot/l2/l2_bridge_domain.robot index e15f8b4338..6c16ed447f 100644 --- a/resources/libraries/robot/l2/l2_bridge_domain.robot +++ b/resources/libraries/robot/l2/l2_bridge_domain.robot @@ -186,16 +186,18 @@ | | ... | *Arguments:* | | ... | - nf_chains - Number of chains of NFs. Type: integer | | ... | - nf_nodes - Number of NFs nodes per chain. Type: integer +| | ... | - start - Id of first chain, allows to add chains during test. +| | ... | Type: integer | | ... | | ... | *Example:* | | ... | | ... | \| Initialize L2 bridge domains for multiple chains with Vhost-User \ -| | ... | \| 1 \| 1 \| +| | ... | \| 3 \| 1 \| 2 \| | | ... -| | [Arguments] | ${nf_chains}=${1} | ${nf_nodes}=${1} +| | [Arguments] | ${nf_chains}=${1} | ${nf_nodes}=${1} | ${start}=${1} | | ... | | Set interfaces in path up -| | :FOR | ${nf_chain} | IN RANGE | 1 | ${nf_chains} + 1 +| | :FOR | ${nf_chain} | IN RANGE | ${start} | ${nf_chains} + 1 | | | Initialize L2 bridge domains with Vhost-User | | | ... | nf_chain=${nf_chain} | nf_nodes=${nf_nodes} diff --git a/resources/libraries/robot/performance/performance_limits.robot b/resources/libraries/robot/performance/performance_limits.robot index 759ee5c28f..67ef3cf5b0 100644 --- a/resources/libraries/robot/performance/performance_limits.robot +++ b/resources/libraries/robot/performance/performance_limits.robot @@ -75,6 +75,11 @@ | | ... | \| Set test Variable \| \${frame_size} \| IMIX_v4_1 \| | | ... | \| Set Max Rate And Jumbo \| | | ... +| | # Negative overhead is possible, if DUT-DUT traffic is less encapsulated +| | # than TG-DUT traffic. +| | # TODO: Re-check overhead values in suites with both traffics encapsulated. +| | # TODO: Improve layered setup to detect encap/decap and update overhead. +| | ${overhead} = | Set Variable If | ${overhead} >= 0 | ${overhead} | ${0} | | ${pps_limit} = | Set Variable | ${18750000.0} | | ${bps_limit} = | Get From Dictionary | ${NIC_NAME_TO_LIMIT} | ${nic_name} | | ${avg_size} = | Get Average Frame Size | ${frame_size} diff --git a/resources/libraries/robot/performance/performance_utils.robot b/resources/libraries/robot/performance/performance_utils.robot index 54b20cba2d..d809376c0f 100644 --- a/resources/libraries/robot/performance/performance_utils.robot +++ b/resources/libraries/robot/performance/performance_utils.robot @@ -32,7 +32,7 @@ | | [Documentation] | | ... | Find boundaries for RFC2544 compatible NDR and PDR values | | ... | using an optimized search algorithm. -| | ... | Display results as formatted test message. +| | ... | Display findings as a formatted test message. | | ... | Fail if a resulting lower bound has too high loss fraction. | | ... | Input rates are understood as uni-directional, | | ... | reported result contains bi-directional rates. @@ -61,7 +61,7 @@ | | ... | | ... | *Example:* | | ... -| | ... | \| Find NDR and PDR intervals using optimized search \| \${0.005} +| | ... | \| Find NDR and PDR intervals using optimized search \| \${0.005} \ | | ... | \| \${0.005} \| \${30.0} \| \${1.0} \| \${2} \| ${600.0} \| ${2} \| | | ... | | [Arguments] | ${packet_loss_ratio}=${0.005} @@ -76,13 +76,83 @@ | | ... | ${final_trial_duration} | ${initial_trial_duration} | | ... | ${number_of_intermediate_phases} | timeout=${timeout} | | ... | doublings=${doublings} -| | Display result of NDRPDR search | ${result} | ${frame_size} +| | Display result of NDRPDR search | ${result} | | Check NDRPDR interval validity | ${result.pdr_interval} | | ... | ${packet_loss_ratio} | | Check NDRPDR interval validity | ${result.ndr_interval} | | Perform additional measurements based on NDRPDR result | | ... | ${result} | ${frame_size} | ${traffic_profile} +| Display Reconfig Test Message +| | [Documentation] +| | ... | Display the number of packets lost (bidirectionally) +| | ... | due to reconfiguration under traffic. +| | ... +| | ... | *Arguments:* +| | ... | - result - Result of bidirectional measurtement. +| | ... | Type: ReceiveRateMeasurement +| | ... +| | ... | *Example:* +| | ... +| | ... | \| Display Reconfig Test Message \| \${result} \| +| | ... +| | [Arguments] | ${result} +| | ... +| | Set Test Message | Packets lost due to reconfig: ${result.loss_count} +| | ${time_lost} = | Evaluate | ${result.loss_count} / ${result.target_tr} +| | Set Test Message | ${\n}Implied time lost: ${time_lost} | append=yes + +| Find Throughput Using MLRsearch +| | [Documentation] +| | ... | Find and return lower bound PDR (zero PLR by default) +| | ... | bi-directional throughput using MLRsearch algorithm. +| | ... | Input rates are understood as uni-directional, +| | ... | reported result contains bi-directional rates. +| | ... | Currently, the min_rate value is hardcoded to match test teardowns. +| | ... +| | ... | TODO: Should the trial duration of the additional +| | ... | measurements be configurable? +| | ... +| | ... | Some inputs are read from variables to streamline suites. +| | ... +| | ... | *Test (or broader scope) variables read:* +| | ... | - traffic_profile - Name of module defining traffc for measurements. +| | ... | Type: string +| | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: int or str +| | ... | - max_rate - Calculated unidirectional maximal transmit rate [pps]. +| | ... | Type: float +| | ... +| | ... | *Arguments:* +| | ... | - packet_loss_ratio - Accepted loss during search. Type: float +| | ... | - final_relative_width - Maximal width multiple of upper. Type: float +| | ... | - final_trial_duration - Duration of final trials [s]. Type: float +| | ... | - initial_trial_duration - Duration of initial trials [s]. Type: float +| | ... | - intermediate phases - Number of intermediate phases [1]. Type: int +| | ... | - timeout - Fail if search duration is longer [s]. Type: float +| | ... | - doublings - How many doublings to do when expanding [1]. Type: int +| | ... +| | ... | *Returns:* +| | ... | - Lower bound for bi-directional throughput at given PLR. Type: float +| | ... +| | ... | *Example:* +| | ... +| | ... | \| \${throughpt}= \| Find Throughput Using MLRsearch \| \${0} \ +| | ... | \| \${0.001} \| \${10.0}\| \${1.0} \| \${1} \| ${720.0} \| ${2} \| +| | ... +| | [Arguments] | ${packet_loss_ratio}=${0.0} +| | ... | ${final_relative_width}=${0.001} | ${final_trial_duration}=${10.0} +| | ... | ${initial_trial_duration}=${1.0} +| | ... | ${number_of_intermediate_phases}=${1} | ${timeout}=${720.0} +| | ... | ${doublings}=${2} +| | ... +| | ${result} = | Perform optimized ndrpdr search | ${frame_size} +| | ... | ${traffic_profile} | ${20000} | ${max_rate*2} +| | ... | ${packet_loss_ratio} | ${final_relative_width} +| | ... | ${final_trial_duration} | ${initial_trial_duration} +| | ... | ${number_of_intermediate_phases} | timeout=${timeout} +| | ... | doublings=${doublings} +| | Return From Keyword | ${result.pdr_interval.measured_low.transmit_rate} + | Find critical load using PLRsearch | | [Documentation] | | ... | Find boundaries for troughput (of given target loss ratio) @@ -116,7 +186,7 @@ | | ... | ${traffic_profile} | ${min_rate} | ${max_rate*2} | | ... | ${packet_loss_ratio} | timeout=${timeout} | | ${lower} | ${upper} = | Display result of soak search -| | ... | ${average} | ${stdev} | ${frame_size} +| | ... | ${average} | ${stdev} | | Should Not Be True | ${lower} < ${min_rate} | | ... | Lower bound ${lower} is below bidirectional minimum ${min_rate}. @@ -162,15 +232,16 @@ | | ... | (Throughput * (L2 Frame Size + IPG) * 8) / Max bitrate of NIC | | ... | The given result should contain latency data as well. | | ... +| | ... | *Test (or broader scope) variables read:* +| | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: int or str | | ... | *Arguments:* | | ... | - result - Measured result data per stream [pps]. Type: NdrPdrResult -| | ... | - frame_size - L2 Frame Size [B]. Type: integer | | ... | | ... | *Example:* | | ... -| | ... | \| Display result of NDRPDR search \| \${result} \| \${64} \| +| | ... | \| Display result of NDRPDR search \| \${result} \| | | ... -| | [Arguments] | ${result} | ${frame_size} +| | [Arguments] | ${result} | | ... | | ${frame_size} = | Get Average Frame Size | ${frame_size} | | Display single bound | NDR_LOWER @@ -196,19 +267,20 @@ | | ... | TODO: Do we want to report some latency data, | | ... | even if not measured at the reported bounds?. | | ... +| | ... | *Test (or broader scope) variables read:* +| | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: int or str | | ... | *Arguments:* | | ... | - avg - Estimated average critical load [pps]. Type: float | | ... | - stdev - Standard deviation of critical load [pps]. Type: float -| | ... | - frame_size - L2 Frame Size [B]. Type: integer | | ... | | ... | *Returns:* | | ... | - Lower and upper bound of critical load [pps]. Type: 2-tuple of float | | ... | | ... | *Example:* | | ... -| | ... | \| Display result of soak search \| \${100000} \| \${100} \| \${64} \| +| | ... | \| Display result of soak search \| \${100000} \| \${100} \| | | ... -| | [Arguments] | ${avg} | ${stdev} | ${frame_size} +| | [Arguments] | ${avg} | ${stdev} | | ... | | ${frame_size} = | Get Average Frame Size | ${frame_size} | | ${avg} = | Convert To Number | ${avg} @@ -321,7 +393,7 @@ | | ... | | ... | *Example:* | | ... -| | ... | \| Traffic should pass with maximum rate \| ${1} \| ${10.0} \| +| | ... | \| Traffic should pass with maximum rate \| ${1} \| ${10.0} \ | | ... | \| ${False} \| ${False} \| ${0} \| ${1} \| | | ... | | [Arguments] | ${trial_duration}=${perf_trial_duration} @@ -399,7 +471,7 @@ | | ... | | ... | *Arguments:* | | ... | - duration - Duration of traffic run [s]. Type: integer -| | ... | - rate - Rate for sending packets. Type: string +| | ... | - rate - Unidirectional rate for sending packets. Type: string | | ... | - frame_size - L2 Frame Size [B] or IMIX_v4_1. Type: integer/string | | ... | - traffic_profile - Name of module defining traffc for measurements. | | ... | Type: string @@ -425,3 +497,50 @@ | | Run Keyword If | ${dut_stats}==${True} | | ... | Show runtime counters on all DUTs | ${nodes} | | Stop traffic on tg + +| Start Traffic on Background +| | [Documentation] +| | ... | Start traffic at specified rate then return control to Robot. +| | ... | Useful if the test needs to do something while traffic is running. +| | ... | Just a wrapper around L1 keyword. +| | ... | +| | ... | TODO: How to make sure the traffic is stopped on any failure? +| | ... +| | ... | *Test (or broader scope) variables read:* +| | ... | - traffic_profile - Name of module defining traffc for measurements. +| | ... | Type: string +| | ... | - frame_size - L2 Frame Size [B] or IMIX string. Type: int or str +| | ... | *Arguments:* +| | ... | - rate - Unidirectional rate for sending packets. Type: string +| | ... | - unidirection - False if traffic is bidirectional. Type: boolean +| | ... | - tx_port - TX port of TG, default 0. Type: integer +| | ... | - rx_port - RX port of TG, default 1. Type: integer +| | ... +| | ... | *Example:* +| | ... +| | ... | \| Start Traffic on Background \| 4.0mpps \| ${False} \| ${0} \ +| | ... | \| ${1} \| +| | ... +| | [Arguments] | ${rate} | ${unidirection}=${False} | ${tx_port}=${0} +| | ... | ${rx_port}=${1} +| | ... +| | # Duration of -1 means we will stop traffic manually. +| | Send traffic on tg | ${-1} | ${rate} | ${frame_size} | ${traffic_profile} +| | ... | warmup_time=${0} | async_call=${True} | latency=${False} +| | ... | unidirection=${unidirection} | tx_port=${tx_port} | rx_port=${rx_port} + +| Stop Running Traffic +| | [Documentation] +| | ... | Stop the running traffic, return measurement result. +| | ... | For bidirectional traffic, the reported values are bi-directional. +| | ... | Just a wrapper around L1 keyword. +| | ... | +| | ... | TODO: Tolerate if traffic was not started. +| | ... +| | ... | *Example:* +| | ... +| | ... | \${result}= \| Stop Running Traffic \| +| | ... +| | Stop traffic on tg +| | ${result}= | Get Measurement Result +| | Return From Keyword | ${result} diff --git a/resources/libraries/robot/shared/interfaces.robot b/resources/libraries/robot/shared/interfaces.robot index 55d0a3d844..f3fb3decd5 100644 --- a/resources/libraries/robot/shared/interfaces.robot +++ b/resources/libraries/robot/shared/interfaces.robot @@ -233,53 +233,82 @@ | | | ... | count=${count} | | Set Test Variable | ${prev_layer} | bond +| Initialize layer dot1q on node for chain +| | [Documentation] +| | ... | Optionally create tag popping subinterface per chain. +| | ... | Return interface indices for dot1q layer interfaces, +| | ... | or Nones if subinterfaces are not created. +| | ... +| | ... | *Arguments:* +| | ... | - dut - DUT node. Type: string +| | ... | - id - Positive index of the chain. Type: integer +| | ... | - vlan_per_chain - Whether to create vlan subinterface for each chain. +| | ... | Type: boolean +| | ... +| | ... | *Example:* +| | ... +| | ... | \| Initialize layer dot1q on node for chain \| DUT1 \| 1 \| True \| +| | ... +| | [Arguments] | ${dut} | ${id} | ${vlan_per_chain}=${True} +| | ... +| | ${dut_str}= | Convert To Lowercase | ${dut} +| | Return From Keyword If | ${id} != ${1} and not ${vlan_per_chain} +| | ... | ${NONE} | ${NONE} +| | # TODO: Is it worth creating Get Variable Value If Not None keyword? +| | ${default}= | Evaluate | ${100} + ${id} - ${1} +| | ${if1_vlan}= | Get Variable Value | \${${dut_str}_vlan1} +| | ${if1_vlan}= | Set Variable If | '${if1_vlan}' != '${NONE}' +| | ... | ${if1_vlan} | ${default} +| | ${default}= | Evaluate | ${200} + ${id} - ${1} +| | ${if2_vlan}= | Get Variable Value | \${${dut_str}_vlan2} +| | ${if2_vlan}= | Set Variable If | '${if2_vlan}' != '${NONE}' +| | ... | ${if2_vlan} | ${default} +| | ${if1_name} | ${if1_index}= | Create Vlan Subinterface +| | ... | ${nodes['${dut}']} | ${${dut_str}_${prev_layer}_${id}_1} +| | ... | ${if1_vlan} +| | ${if2_name} | ${if2_index}= | Create Vlan Subinterface +| | ... | ${nodes['${dut}']} | ${${dut_str}_${prev_layer}_${id}_2} +| | ... | ${if2_vlan} +| | Set Interface State | ${nodes['${dut}']} | ${if1_index} | up +| | Set Interface State | ${nodes['${dut}']} | ${if2_index} | up +| | Configure L2 tag rewrite method on interfaces +| | ... | ${nodes['${dut}']} | ${if1_index} | TAG_REWRITE_METHOD=pop-1 +| | Configure L2 tag rewrite method on interfaces +| | ... | ${nodes['${dut}']} | ${if2_index} | TAG_REWRITE_METHOD=pop-1 +| | Return From Keyword | ${if1_index} | ${if2_index} + | Initialize layer dot1q on node | | [Documentation] | | ... | Dot1q interfaces and variables to be created on all DUT's node | | ... | interfaces. | | ... +| | ... | TODO: Unify names for number of chains/pipelines/instances/interfaces. +| | ... | Chose names and descriptions that makes sense for both +| | ... | nf_density and older tests. +| | ... | Note that with vlan_per_chain=False it is not a number of interfaces. +| | ... | At least not number of real interfaces, just number of aliases. +| | ... | This TODO applies also to all keywords with nf_chains argument. +| | ... | | ... | *Arguments:* | | ... | - dut - DUT node. Type: string -| | ... | - count - Number of tagged interfaces. Type: integer -| | ... | - create - Whether to create vlan subinterface for each chain. +| | ... | - count - Number of chains. Type: integer +| | ... | - vlan_per_chain - Whether to create vlan subinterface for each chain. | | ... | Type: boolean +| | ... | - start - Id of first chain, allows adding chains during test. +| | ... | Type: integer | | ... | | ... | *Example:* | | ... -| | ... | \| Initialize layer dot1q on node \| DUT1 \| 1 \| True \| +| | ... | \| Initialize layer dot1q on node \| DUT1 \| 3 \| True \| 2 \| | | ... -| | [Arguments] | ${dut} | ${count}=${1} | ${create}=${True} +| | [Arguments] | ${dut} | ${count}=${1} | ${vlan_per_chain}=${True} +| | ... | ${start}=${1} | | ... | | ${dut_str}= | Convert To Lowercase | ${dut} -| | :FOR | ${id} | IN RANGE | 1 | ${count} + 1 -| | | ${if1_vlan}= | Get Variable Value | \${${dut_str}_vlan1} -| | | ${default}= | Evaluate | ${100} + ${id} - ${1} -| | | ${if1_vlan}= | Set Variable If | '${if1_vlan}' == '${NONE}' -| | | ... | ${default} | ${if1_vlan} -| | | ${if2_vlan}= | Get Variable Value | \${${dut_str}_vlan2} -| | | ${default}= | Evaluate | ${200} + ${id} - ${1} -| | | ${if2_vlan}= | Set Variable If | '${if2_vlan}' == '${NONE}' -| | | ... | ${default} | ${if2_vlan} -| | | ${if1_name} | ${if1_index}= | Run Keyword If -| | | ... | ${create} or ${id} == ${1} -| | | ... | Create Vlan Subinterface -| | | ... | ${nodes['${dut}']} | ${${dut_str}_${prev_layer}_${id}_1} -| | | ... | ${if1_vlan} -| | | ${if2_name} | ${if2_index}= | Run Keyword If -| | | ... | ${create} or ${id} == ${1} -| | | ... | Create Vlan Subinterface -| | | ... | ${nodes['${dut}']} | ${${dut_str}_${prev_layer}_${id}_2} -| | | ... | ${if2_vlan} -| | | Run Keyword If | ${create} or ${id} == ${1} -| | | ... | Set Interface State | ${nodes['${dut}']} | ${if1_index} | up -| | | Run Keyword If | ${create} or ${id} == ${1} -| | | ... | Set Interface State | ${nodes['${dut}']} | ${if2_index} | up -| | | Run Keyword If | ${create} or ${id} == ${1} -| | | ... | Configure L2 tag rewrite method on interfaces -| | | ... | ${nodes['${dut}']} | ${if1_index} | TAG_REWRITE_METHOD=pop-1 -| | | Run Keyword If | ${create} or ${id} == ${1} -| | | ... | Configure L2 tag rewrite method on interfaces -| | | ... | ${nodes['${dut}']} | ${if2_index} | TAG_REWRITE_METHOD=pop-1 +| | :FOR | ${id} | IN RANGE | ${start} | ${count} + 1 +| | | ${if1_index} | ${if2_index}= | Initialize layer dot1q on node for chain +| | | ... | dut=${dut} | id=${id} | vlan_per_chain=${vlan_per_chain} +| | | # First id results in non-None indices, after that _1_ are defined. | | | ${if1_index}= | Set Variable If | '${if1_index}' == '${NONE}' | | | ... | ${${dut_str}_dot1q_1_1} | ${if1_index} | | | ${if2_index}= | Set Variable If | '${if2_index}' == '${NONE}' @@ -292,16 +321,19 @@ | | ... | Dot1q interfaces and variables to be created on all DUT's interfaces. | | ... | | ... | *Arguments:* -| | ... | - count - Number of tagged interfaces. Type: integer -| | ... | - create - Whether to create vlan for each chain. Type: boolean +| | ... | - count - Number of chains. Type: integer +| | ... | - vlan_per_chain - Whether to create vlan subinterface for each chain. +| | ... | Type: boolean +| | ... | - start - Id of first chain, allows adding chains during test. +| | ... | Type: integer | | ... -| | ... | \| Initialize layer dot1q \| 1 \| True \| +| | ... | \| Initialize layer dot1q \| 3 \| True \| 2 \| | | ... -| | [Arguments] | ${count}=${1} | ${create}=${True} +| | [Arguments] | ${count}=${1} | ${vlan_per_chain}=${True} | ${start}=${1} | | ... | | :FOR | ${dut} | IN | @{duts} | | | Initialize layer dot1q on node | ${dut} | count=${count} -| | | ... | create=${create} +| | | ... | vlan_per_chain=${vlan_per_chain} | start=${start} | | Set Test Variable | ${prev_layer} | dot1q | Initialize layer ip4vxlan on node @@ -314,21 +346,23 @@ | | ... | *Arguments:* | | ... | - dut - DUT node. Type: string | | ... | - count - Number of vxlan interfaces. Type: integer +| | ... | - start - Id of first chain, allows adding chains during test. +| | ... | Type: integer | | ... | | ... | *Example:* | | ... -| | ... | \| Initialize layer ip4vxlan on node \| DUT1 \| 1 \| +| | ... | \| Initialize layer ip4vxlan on node \| DUT1 \| 3 \| 2 \| | | ... -| | [Arguments] | ${dut} | ${count}=${1} +| | [Arguments] | ${dut} | ${count}=${1} | ${start}=${1} | | ... | | ${dut_str}= | Convert To Lowercase | ${dut} -| | VPP Interface Set IP Address +| | Run Keyword If | "${start}" == "1" | VPP Interface Set IP Address | | ... | ${nodes['${dut}']} | ${${dut_str}_${prev_layer}_1_1} | | ... | 172.16.0.1 | 24 -| | VPP Interface Set IP Address +| | Run Keyword If | "${start}" == "1" | VPP Interface Set IP Address | | ... | ${nodes['${dut}']} | ${${dut_str}_${prev_layer}_1_2} | | ... | 172.26.0.1 | 24 -| | :FOR | ${id} | IN RANGE | 1 | ${count} + 1 +| | :FOR | ${id} | IN RANGE | ${start} | ${count} + 1 | | | ${subnet}= | Evaluate | ${id} - 1 | | | ${vni}= | Evaluate | ${id} - 1 | | | ${ip4vxlan_1}= | Create VXLAN interface @@ -368,13 +402,16 @@ | | ... | | ... | *Arguments:* | | ... | - count - Number of vxlan interfaces. Type: integer +| | ... | - start - Id of first chain, allows adding chains during test. +| | ... | Type: integer | | ... -| | ... | \| Initialize layer ip4vxlan \| 1 \| +| | ... | \| Initialize layer ip4vxlan \| 3 \| 2 \| | | ... -| | [Arguments] | ${count}=${1} +| | [Arguments] | ${count}=${1} | ${start}=${1} | | ... | | :FOR | ${dut} | IN | @{duts} | | | Initialize layer ip4vxlan on node | ${dut} | count=${count} +| | ... | start=${start} | | Set Test Variable | ${prev_layer} | ip4vxlan | Configure vhost interfaces |