diff options
| author |   Jan Gelety <jgelety@cisco.com> | 2017-07-07 15:05:33 +0200 |
|---|---|---|
| committer | Jan Gelety <jgelety@cisco.com> | 2017-07-19 13:07:07 +0200 |
| commit | cb6b9fd6499a6e2f0efbb87ee84a8e768a7df869 (patch) | |
| tree | 4c6de8a7075ad57850c1aa329d52998142558a38 | |
| parent | ebf71b8ec8863ac82becfcdee97d4e7164fb2fcb (diff) | |
CSIT-704: Add ACL IP4BASE perf tests
Change-Id: I1fada28f2855534faf3b3051a3efe1cbb4dcdd03
Signed-off-by: Jan Gelety <jgelety@cisco.com>
41 files changed, 5399 insertions, 42 deletions
diff --git a/resources/libraries/robot/performance/performance_configuration.robot b/resources/libraries/robot/performance/performance_configuration.robot index adf233d4ba..4d0e53f896 100644 --- a/resources/libraries/robot/performance/performance_configuration.robot +++ b/resources/libraries/robot/performance/performance_configuration.robot @@ -21,6 +21,7 @@ | Library | resources.libraries.python.VhostUser | Library | resources.libraries.python.TrafficGenerator | Library | resources.libraries.python.TrafficGenerator.TGDropRateSearchImpl +| Library | resources.libraries.python.Classify | Resource | resources/libraries/robot/shared/default.robot | Resource | resources/libraries/robot/shared/interfaces.robot | Resource | resources/libraries/robot/shared/counters.robot @@ -851,6 +852,48 @@ | | All Vpp Interfaces Ready Wait | ${nodes} | | Configure IPv4 ACLs | ${dut1} | ${dut1_if1} | ${dut1_if2} +| Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | [Documentation] +| | ... | Set UP state on VPP interfaces in path on nodes in 3-node circular +| | ... | topology. Get the interface MAC addresses and setup ARP on all VPP +| | ... | interfaces. Setup IPv4 addresses with /24 prefix on DUT-TG links and +| | ... | /30 prefix on DUT1-DUT2 link. Set routing on both DUT nodes with +| | ... | prefix /24 and next hops of neighbour DUT interface IPv4 address. +| | ... | Apply required ACL rules to DUT1 interfaces. +| | ... +| | ... | _NOTE:_ This KW uses following test case variables: +| | ... | - ${dut1} - DUT1 node. +| | ... | - ${dut2} - DUT2 node. +| | ... | - ${dut1_if1} - DUT1 interface towards TG. +| | ... | - ${dut1_if2} - DUT1 interface towards DUT2. +| | ... | - ${dut2_if1} - DUT2 interface towards DUT1. +| | ... | - ${dut2_if2} - DUT2 interface towards TG. +| | ... +| | Set Interface State | ${dut1} | ${dut1_if1} | up +| | Set Interface State | ${dut1} | ${dut1_if2} | up +| | Set Interface State | ${dut2} | ${dut2_if1} | up +| | Set Interface State | ${dut2} | ${dut2_if2} | up +| | ${tg1_if1_mac}= | Get Interface MAC | ${tg} | ${tg_if1} +| | ${tg1_if2_mac}= | Get Interface MAC | ${tg} | ${tg_if2} +| | ${dut1_if2_mac}= | Get Interface MAC | ${dut1} | ${dut1_if2} +| | ${dut2_if1_mac}= | Get Interface MAC | ${dut2} | ${dut2_if1} +| | :FOR | ${number} | IN RANGE | 2 | ${ip_nr}+2 +| | | Add arp on dut | ${dut1} | ${dut1_if1} | 10.10.10.${number} +| | | ... | ${tg1_if1_mac} +| | | Add arp on dut | ${dut2} | ${dut2_if2} | 20.20.20.${number} +| | | ... | ${tg1_if2_mac} +| | Add arp on dut | ${dut1} | ${dut1_if2} | 1.1.1.2 | ${dut2_if1_mac} +| | Add arp on dut | ${dut2} | ${dut2_if1} | 1.1.1.1 | ${dut1_if2_mac} +| | Configure IP addresses on interfaces +| | ... | ${dut1} | ${dut1_if1} | 10.10.10.1 | 24 +| | ... | ${dut1} | ${dut1_if2} | 1.1.1.1 | 30 +| | ... | ${dut2} | ${dut2_if1} | 1.1.1.2 | 30 +| | ... | ${dut2} | ${dut2_if2} | 20.20.20.1 | 24 +| | Vpp Route Add | ${dut1} | 20.20.20.0 | 24 | 1.1.1.2 | ${dut1_if2} +| | Vpp Route Add | ${dut2} | 10.10.10.0 | 24 | 1.1.1.1 | ${dut2_if1} +| | All Vpp Interfaces Ready Wait | ${nodes} +| | Configure IPv4 ACLs | ${dut1} | ${dut1_if1} | ${dut1_if2} + | Initialize L2 bridge domains with Vhost-User in 3-node circular topology | | [Documentation] | | ... | Create two Vhost-User interfaces on all defined VPP nodes. Add each diff --git a/resources/libraries/robot/performance/performance_setup.robot b/resources/libraries/robot/performance/performance_setup.robot index de50fd3254..6e46e3bc0d 100644 --- a/resources/libraries/robot/performance/performance_setup.robot +++ b/resources/libraries/robot/performance/performance_setup.robot @@ -531,16 +531,38 @@ | | ... | *Arguments:* | | ... | - rate - Rate for sending packets. Type: string | | ... | - framesize - L2 Frame Size [B]. Type: integer -| | ... | - topology_type - Topology type. Type: string +| | ... | - traffic_profile - Traffic profile. Type: string | | ... | | ... | *Example:* | | ... -| | ... | \| Tear down performance discovery test with SNAT \| 4.0mpps \| 64 \ +| | ... | \| Tear down performance discovery test with SNAT \| 100000pps \| 64 \ | | ... | \| ${traffic_profile} \| | | ... | | [Arguments] | ${rate} | ${framesize} | ${traffic_profile} | | ... -| | Tear down performance discovery test | ${rate}pps | ${framesize} +| | Tear down performance discovery test | ${rate} | ${framesize} | | ... | ${traffic_profile} | | Show SNAT verbose | ${dut1} | | Show SNAT verbose | ${dut2} + +| Tear down performance test with ACL +| | [Documentation] | Common test teardown for ndrdisc and pdrdisc performance \ +| | ... | tests with ACL feature used. +| | ... +| | ... | *Arguments:* +| | ... | - rate - Rate for sending packets. Type: string +| | ... | - framesize - L2 Frame Size [B]. Type: integer +| | ... | - traffic_profile - Traffic profile. Type: string +| | ... +| | ... | *Example:* +| | ... +| | ... | \| Tear down performance test with ACL \| 100000pps \| 64 \ +| | ... | \| ${traffic_profile} \| +| | ... +| | [Arguments] | ${rate} | ${framesize} | ${traffic_profile} +| | ... +| | Tear down performance discovery test | ${rate} | ${framesize} +| ... | ${traffic_profile} +| | Vpp Log Plugin Acl Settings | ${dut1} +| | Run Keyword And Ignore Error | Vpp Log Plugin Acl Interface Assignment +| ... | ${dut1} diff --git a/resources/traffic_profiles/trex/trex-sl-3n-ethip4udp-100u1000p-conc.py b/resources/traffic_profiles/trex/trex-sl-3n-ethip4udp-100u1000p-conc.py index 2f8b782ca6..02d9cda075 100755 --- a/resources/traffic_profiles/trex/trex-sl-3n-ethip4udp-100u1000p-conc.py +++ b/resources/traffic_profiles/trex/trex-sl-3n-ethip4udp-100u1000p-conc.py @@ -17,13 +17,13 @@ Stream profile: - Two streams sent in directions 0 --> 1 and 1 --> 0 at the same time. - Packet: ETH / IP / UDP - Direction 0 --> 1: - - Source IP address range: 10.10.10.0 - 10.10.10.99 - - Destination IP address range: 20.20.20.0 - 20.20.20.99 + - Source IP address range: 10.10.10.2 - 10.10.10.101 + - Destination IP address range: 20.20.20.2 - 20.20.20.101 - Source UDP port range: 1001 - 2000 - Destination UDP port range: 2001 - 3000 - Direction 1 --> 0: - - Source IP address range: 20.20.20.0 - 20.20.20.99 - - Destination IP address range: 10.10.10.0 - 10.10.10.99 + - Source IP address range: 20.20.20.2 - 20.20.20.101 + - Destination IP address range: 10.10.10.2 - 10.10.10.101 - Source UDP port range: 2001-3000 - Destination UDP port range: 1001 - 2000 """ @@ -41,15 +41,15 @@ class TrafficStreams(TrafficStreamsBaseClass): super(TrafficStreamsBaseClass, self).__init__() # IPs used in packet headers. - self.p1_src_start_ip = '10.10.10.0' - self.p1_src_end_ip = '10.10.10.99' - self.p1_dst_start_ip = '20.20.20.0' - self.p1_dst_end_ip = '20.20.20.99' - - self.p2_src_start_ip = '20.20.20.0' - self.p2_src_end_ip = '20.20.20.99' - self.p2_dst_start_ip = '10.10.10.0' - self.p2_dst_end_ip = '10.10.10.99' + self.p1_src_start_ip = '10.10.10.2' + self.p1_src_end_ip = '10.10.10.101' + self.p1_dst_start_ip = '20.20.20.2' + self.p1_dst_end_ip = '20.20.20.101' + + self.p2_src_start_ip = '20.20.20.2' + self.p2_src_end_ip = '20.20.20.101' + self.p2_dst_start_ip = '10.10.10.2' + self.p2_dst_end_ip = '10.10.10.101' # UDP ports used in packet headers. self.p1_src_start_udp_port = 1001 diff --git a/resources/traffic_profiles/trex/trex-sl-3n-ethip4udp-10u1000p-conc.py b/resources/traffic_profiles/trex/trex-sl-3n-ethip4udp-10u1000p-conc.py index b0362aedcd..847eccdf0a 100755 --- a/resources/traffic_profiles/trex/trex-sl-3n-ethip4udp-10u1000p-conc.py +++ b/resources/traffic_profiles/trex/trex-sl-3n-ethip4udp-10u1000p-conc.py @@ -17,13 +17,13 @@ Stream profile: - Two streams sent in directions 0 --> 1 and 1 --> 0 at the same time. - Packet: ETH / IP / UDP - Direction 0 --> 1: - - Source IP address range: 10.10.10.0 - 10.10.10.9 - - Destination IP address range: 20.20.20.0 - 20.20.20.9 + - Source IP address range: 10.10.10.2 - 10.10.10.11 + - Destination IP address range: 20.20.20.2 - 20.20.20.11 - Source UDP port range: 1001 - 2000 - Destination UDP port range: 2001 - 3000 - Direction 1 --> 0: - - Source IP address range: 20.20.20.0 - 20.20.20.9 - - Destination IP address range: 10.10.10.0 - 10.10.10.9 + - Source IP address range: 20.20.20.2 - 20.20.20.11 + - Destination IP address range: 10.10.10.2 - 10.10.10.11 - Source UDP port range: 2001-3000 - Destination UDP port range: 1001 - 2000 """ @@ -41,15 +41,15 @@ class TrafficStreams(TrafficStreamsBaseClass): super(TrafficStreamsBaseClass, self).__init__() # IPs used in packet headers. - self.p1_src_start_ip = '10.10.10.0' - self.p1_src_end_ip = '10.10.10.9' - self.p1_dst_start_ip = '20.20.20.0' - self.p1_dst_end_ip = '20.20.20.9' - - self.p2_src_start_ip = '20.20.20.0' - self.p2_src_end_ip = '20.20.20.9' - self.p2_dst_start_ip = '10.10.10.0' - self.p2_dst_end_ip = '10.10.10.9' + self.p1_src_start_ip = '10.10.10.2' + self.p1_src_end_ip = '10.10.10.11' + self.p1_dst_start_ip = '20.20.20.2' + self.p1_dst_end_ip = '20.20.20.11' + + self.p2_src_start_ip = '20.20.20.2' + self.p2_src_end_ip = '20.20.20.11' + self.p2_dst_start_ip = '10.10.10.2' + self.p2_dst_end_ip = '10.10.10.11' # UDP ports used in packet headers. self.p1_src_start_udp_port = 1001 diff --git a/resources/traffic_profiles/trex/trex-sl-3n-ethip4udp-10u10p-conc.py b/resources/traffic_profiles/trex/trex-sl-3n-ethip4udp-10u10p-conc.py index 76ccb53259..81d113350b 100755 --- a/resources/traffic_profiles/trex/trex-sl-3n-ethip4udp-10u10p-conc.py +++ b/resources/traffic_profiles/trex/trex-sl-3n-ethip4udp-10u10p-conc.py @@ -17,13 +17,13 @@ Stream profile: - Two streams sent in directions 0 --> 1 and 1 --> 0 at the same time. - Packet: ETH / IP / UDP - Direction 0 --> 1: - - Source IP address range: 10.10.10.0 - 10.10.10.9 - - Destination IP address range: 20.20.20.0 - 20.20.20.9 + - Source IP address range: 10.10.10.2 - 10.10.10.11 + - Destination IP address range: 20.20.20.2 - 20.20.2011 - Source UDP port range: 1001 - 1010 - Destination UDP port range: 2001 - 2010 - Direction 1 --> 0: - - Source IP address range: 20.20.20.0 - 20.20.20.9 - - Destination IP address range: 10.10.10.0 - 10.10.10.9 + - Source IP address range: 20.20.20.2 - 20.20.20.11 + - Destination IP address range: 10.10.10.2 - 10.10.10.11 - Source UDP port range: 2001-2010 - Destination UDP port range: 1001 - 1010 """ @@ -41,15 +41,15 @@ class TrafficStreams(TrafficStreamsBaseClass): super(TrafficStreamsBaseClass, self).__init__() # IPs used in packet headers. - self.p1_src_start_ip = '10.10.10.0' - self.p1_src_end_ip = '10.10.10.9' - self.p1_dst_start_ip = '20.20.20.0' - self.p1_dst_end_ip = '20.20.20.9' - - self.p2_src_start_ip = '20.20.20.0' - self.p2_src_end_ip = '20.20.20.9' - self.p2_dst_start_ip = '10.10.10.0' - self.p2_dst_end_ip = '10.10.10.9' + self.p1_src_start_ip = '10.10.10.2' + self.p1_src_end_ip = '10.10.10.11' + self.p1_dst_start_ip = '20.20.20.2' + self.p1_dst_end_ip = '20.20.20.11' + + self.p2_src_start_ip = '20.20.20.2' + self.p2_src_end_ip = '20.20.20.11' + self.p2_dst_start_ip = '10.10.10.2' + self.p2_dst_end_ip = '10.10.10.11' # UDP ports used in packet headers. self.p1_src_start_udp_port = 1001 diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sf-100flows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sf-100flows-ndrpdrdisc.robot new file mode 100644 index 0000000000..14c47124ea --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sf-100flows-ndrpdrdisc.robot @@ -0,0 +1,147 @@ +# Copyright (c) 2017 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL +| ... | IACL | ACL10 | 100_FLOWS +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| Test Teardown | Tear down performance test with ACL | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\ +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ +| ... | of packets transmitted. NDR and PDR are discovered for different\ +| ... | Ethernet L2 frame sizes using either binary search or linear search\ +| ... | algorithms with configured starting rate and final step that determines\ +| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit+reflect +| ${acl_apply_type}= | input +| ${no_hit_aces_number}= | 10 +| ${flows_per_dir}= | 100 + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +*** Keywords *** +| Discover NDR or PDR for IPv4 routing with ACLs +| | [Arguments] | ${wt} | ${rxq} | ${framesize} | ${min_rate} | ${search_type} +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} +| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | ${binary_min}= | Set Variable | ${min_rate} +| | ${binary_max}= | Set Variable | ${max_rate} +| | ${threshold}= | Set Variable | ${min_rate} +| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology +| | And Add PCI devices to DUTs in 3-node single link topology +| | ${get_framesize}= | Get Frame Size | ${framesize} +| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u10p-conc +| | Set Test Variable | ${traffic_profile} +| | Then Run Keyword If | '${search_type}' == 'NDR' +| | ... | Find NDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ELSE IF | '${search_type}' == 'PDR' +| | ... | Find PDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} + +*** Test Cases *** +| tc01-64B-1t1c-ethip4udp-ip4base-iacl10-stateful-flows100-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc02-64B-1t1c-ethip4udp-ip4base-iacl10-stateful-flows100-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR + +| tc03-64B-2t2c-ethip4udp-ip4base-iacl10-stateful-flows100-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc04-64B-2t2c-ethip4udp-ip4base-iacl10-stateful-flows100-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sf-100kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sf-100kflows-ndrpdrdisc.robot new file mode 100644 index 0000000000..967677d3fd --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sf-100kflows-ndrpdrdisc.robot @@ -0,0 +1,147 @@ +# Copyright (c) 2017 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL +| ... | IACL | ACL10 | 100k_FLOWS +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| Test Teardown | Tear down performance test with ACL | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\ +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ +| ... | of packets transmitted. NDR and PDR are discovered for different\ +| ... | Ethernet L2 frame sizes using either binary search or linear search\ +| ... | algorithms with configured starting rate and final step that determines\ +| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit+reflect +| ${acl_apply_type}= | input +| ${no_hit_aces_number}= | 10 +| ${flows_per_dir}= | 100k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +*** Keywords *** +| Discover NDR or PDR for IPv4 routing with ACLs +| | [Arguments] | ${wt} | ${rxq} | ${framesize} | ${min_rate} | ${search_type} +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} +| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | ${binary_min}= | Set Variable | ${min_rate} +| | ${binary_max}= | Set Variable | ${max_rate} +| | ${threshold}= | Set Variable | ${min_rate} +| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology +| | And Add PCI devices to DUTs in 3-node single link topology +| | ${get_framesize}= | Get Frame Size | ${framesize} +| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 100 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-100u1000p-conc +| | Set Test Variable | ${traffic_profile} +| | Then Run Keyword If | '${search_type}' == 'NDR' +| | ... | Find NDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ELSE IF | '${search_type}' == 'PDR' +| | ... | Find PDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} + +*** Test Cases *** +| tc01-64B-1t1c-ethip4udp-ip4base-iacl10-stateful-flows100k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc02-64B-1t1c-ethip4udp-ip4base-iacl10-stateful-flows100k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR + +| tc03-64B-2t2c-ethip4udp-ip4base-iacl10-stateful-flows100k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc04-64B-2t2c-ethip4udp-ip4base-iacl10-stateful-flows100k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sf-10kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sf-10kflows-ndrpdrdisc.robot new file mode 100644 index 0000000000..4e28e6997f --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sf-10kflows-ndrpdrdisc.robot @@ -0,0 +1,147 @@ +# Copyright (c) 2017 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL +| ... | IACL | ACL10 | 10k_FLOWS +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| Test Teardown | Tear down performance test with ACL | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\ +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ +| ... | of packets transmitted. NDR and PDR are discovered for different\ +| ... | Ethernet L2 frame sizes using either binary search or linear search\ +| ... | algorithms with configured starting rate and final step that determines\ +| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit+reflect +| ${acl_apply_type}= | input +| ${no_hit_aces_number}= | 10 +| ${flows_per_dir}= | 10k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +*** Keywords *** +| Discover NDR or PDR for IPv4 routing with ACLs +| | [Arguments] | ${wt} | ${rxq} | ${framesize} | ${min_rate} | ${search_type} +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} +| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | ${binary_min}= | Set Variable | ${min_rate} +| | ${binary_max}= | Set Variable | ${max_rate} +| | ${threshold}= | Set Variable | ${min_rate} +| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology +| | And Add PCI devices to DUTs in 3-node single link topology +| | ${get_framesize}= | Get Frame Size | ${framesize} +| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u1000p-conc +| | Set Test Variable | ${traffic_profile} +| | Then Run Keyword If | '${search_type}' == 'NDR' +| | ... | Find NDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ELSE IF | '${search_type}' == 'PDR' +| | ... | Find PDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} + +*** Test Cases *** +| tc01-64B-1t1c-ethip4udp-ip4base-iacl10-stateful-flows10k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc02-64B-1t1c-ethip4udp-ip4base-iacl10-stateful-flows10k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR + +| tc03-64B-2t2c-ethip4udp-ip4base-iacl10-stateful-flows10k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc04-64B-2t2c-ethip4udp-ip4base-iacl10-stateful-flows10k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sl-100flows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sl-100flows-ndrpdrdisc.robot new file mode 100644 index 0000000000..987c810c22 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sl-100flows-ndrpdrdisc.robot @@ -0,0 +1,147 @@ +# Copyright (c) 2017 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS +| ... | IACL | ACL10 | 100_FLOWS +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| Test Teardown | Tear down performance test with ACL | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\ +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ +| ... | of packets transmitted. NDR and PDR are discovered for different\ +| ... | Ethernet L2 frame sizes using either binary search or linear search\ +| ... | algorithms with configured starting rate and final step that determines\ +| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit +| ${acl_apply_type}= | input +| ${no_hit_aces_number}= | 10 +| ${flows_per_dir}= | 100 + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +*** Keywords *** +| Discover NDR or PDR for IPv4 routing with ACLs +| | [Arguments] | ${wt} | ${rxq} | ${framesize} | ${min_rate} | ${search_type} +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} +| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | ${binary_min}= | Set Variable | ${min_rate} +| | ${binary_max}= | Set Variable | ${max_rate} +| | ${threshold}= | Set Variable | ${min_rate} +| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology +| | And Add PCI devices to DUTs in 3-node single link topology +| | ${get_framesize}= | Get Frame Size | ${framesize} +| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u10p-conc +| | Set Test Variable | ${traffic_profile} +| | Then Run Keyword If | '${search_type}' == 'NDR' +| | ... | Find NDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ELSE IF | '${search_type}' == 'PDR' +| | ... | Find PDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} + +*** Test Cases *** +| tc01-64B-1t1c-ethip4udp-ip4base-iacl10-stateless-flows100-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc02-64B-1t1c-ethip4udp-ip4base-iacl10-stateless-flows100-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR + +| tc03-64B-2t2c-ethip4udp-ip4base-iacl10-stateless-flows100-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc04-64B-2t2c-ethip4udp-ip4base-iacl10-stateless-flows100-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sl-100kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sl-100kflows-ndrpdrdisc.robot new file mode 100644 index 0000000000..b500865533 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sl-100kflows-ndrpdrdisc.robot @@ -0,0 +1,147 @@ +# Copyright (c) 2017 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS +| ... | IACL | ACL10 | 100k_FLOWS +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| Test Teardown | Tear down performance test with ACL | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\ +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ +| ... | of packets transmitted. NDR and PDR are discovered for different\ +| ... | Ethernet L2 frame sizes using either binary search or linear search\ +| ... | algorithms with configured starting rate and final step that determines\ +| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit +| ${acl_apply_type}= | input +| ${no_hit_aces_number}= | 10 +| ${flows_per_dir}= | 100k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +*** Keywords *** +| Discover NDR or PDR for IPv4 routing with ACLs +| | [Arguments] | ${wt} | ${rxq} | ${framesize} | ${min_rate} | ${search_type} +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} +| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | ${binary_min}= | Set Variable | ${min_rate} +| | ${binary_max}= | Set Variable | ${max_rate} +| | ${threshold}= | Set Variable | ${min_rate} +| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology +| | And Add PCI devices to DUTs in 3-node single link topology +| | ${get_framesize}= | Get Frame Size | ${framesize} +| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 100 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-100u1000p-conc +| | Set Test Variable | ${traffic_profile} +| | Then Run Keyword If | '${search_type}' == 'NDR' +| | ... | Find NDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ELSE IF | '${search_type}' == 'PDR' +| | ... | Find PDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} + +*** Test Cases *** +| tc01-64B-1t1c-ethip4udp-ip4base-iacl10-stateless-flows100k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc02-64B-1t1c-ethip4udp-ip4base-iacl10-stateless-flows100k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR + +| tc03-64B-2t2c-ethip4udp-ip4base-iacl10-stateless-flows100k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc04-64B-2t2c-ethip4udp-ip4base-iacl10-stateless-flows100k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sl-10kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sl-10kflows-ndrpdrdisc.robot new file mode 100644 index 0000000000..72cc901399 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sl-10kflows-ndrpdrdisc.robot @@ -0,0 +1,147 @@ +# Copyright (c) 2017 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS +| ... | IACL | ACL10 | 10k_FLOWS +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| Test Teardown | Tear down performance test with ACL | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\ +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ +| ... | of packets transmitted. NDR and PDR are discovered for different\ +| ... | Ethernet L2 frame sizes using either binary search or linear search\ +| ... | algorithms with configured starting rate and final step that determines\ +| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit +| ${acl_apply_type}= | input +| ${no_hit_aces_number}= | 10 +| ${flows_per_dir}= | 10k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +*** Keywords *** +| Discover NDR or PDR for IPv4 routing with ACLs +| | [Arguments] | ${wt} | ${rxq} | ${framesize} | ${min_rate} | ${search_type} +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} +| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | ${binary_min}= | Set Variable | ${min_rate} +| | ${binary_max}= | Set Variable | ${max_rate} +| | ${threshold}= | Set Variable | ${min_rate} +| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology +| | And Add PCI devices to DUTs in 3-node single link topology +| | ${get_framesize}= | Get Frame Size | ${framesize} +| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u1000p-conc +| | Set Test Variable | ${traffic_profile} +| | Then Run Keyword If | '${search_type}' == 'NDR' +| | ... | Find NDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ELSE IF | '${search_type}' == 'PDR' +| | ... | Find PDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} + +*** Test Cases *** +| tc01-64B-1t1c-ethip4udp-ip4base-iacl10-stateless-flows10k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc02-64B-1t1c-ethip4udp-ip4base-iacl10-stateless-flows10k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR + +| tc03-64B-2t2c-ethip4udp-ip4base-iacl10-stateless-flows10k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc04-64B-2t2c-ethip4udp-ip4base-iacl10-stateless-flows10k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sf-100flows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sf-100flows-ndrpdrdisc.robot new file mode 100644 index 0000000000..6ad814f6af --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sf-100flows-ndrpdrdisc.robot @@ -0,0 +1,147 @@ +# Copyright (c) 2017 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL +| ... | IACL | ACL1 | 100_FLOWS +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| Test Teardown | Tear down performance test with ACL | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\ +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ +| ... | of packets transmitted. NDR and PDR are discovered for different\ +| ... | Ethernet L2 frame sizes using either binary search or linear search\ +| ... | algorithms with configured starting rate and final step that determines\ +| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit+reflect +| ${acl_apply_type}= | input +| ${no_hit_aces_number}= | 1 +| ${flows_per_dir}= | 100 + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +*** Keywords *** +| Discover NDR or PDR for IPv4 routing with ACLs +| | [Arguments] | ${wt} | ${rxq} | ${framesize} | ${min_rate} | ${search_type} +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} +| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | ${binary_min}= | Set Variable | ${min_rate} +| | ${binary_max}= | Set Variable | ${max_rate} +| | ${threshold}= | Set Variable | ${min_rate} +| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology +| | And Add PCI devices to DUTs in 3-node single link topology +| | ${get_framesize}= | Get Frame Size | ${framesize} +| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u10p-conc +| | Set Test Variable | ${traffic_profile} +| | Then Run Keyword If | '${search_type}' == 'NDR' +| | ... | Find NDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ELSE IF | '${search_type}' == 'PDR' +| | ... | Find PDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} + +*** Test Cases *** +| tc01-64B-1t1c-ethip4udp-ip4base-iacl1-stateful-flows100-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc02-64B-1t1c-ethip4udp-ip4base-iacl1-stateful-flows100-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR + +| tc03-64B-2t2c-ethip4udp-ip4base-iacl1-stateful-flows100-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc04-64B-2t2c-ethip4udp-ip4base-iacl1-stateful-flows100-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sf-100kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sf-100kflows-ndrpdrdisc.robot new file mode 100644 index 0000000000..07229d8fd2 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sf-100kflows-ndrpdrdisc.robot @@ -0,0 +1,147 @@ +# Copyright (c) 2017 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL +| ... | IACL | ACL1 | 100k_FLOWS +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| Test Teardown | Tear down performance test with ACL | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\ +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ +| ... | of packets transmitted. NDR and PDR are discovered for different\ +| ... | Ethernet L2 frame sizes using either binary search or linear search\ +| ... | algorithms with configured starting rate and final step that determines\ +| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit+reflect +| ${acl_apply_type}= | input +| ${no_hit_aces_number}= | 1 +| ${flows_per_dir}= | 100k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +*** Keywords *** +| Discover NDR or PDR for IPv4 routing with ACLs +| | [Arguments] | ${wt} | ${rxq} | ${framesize} | ${min_rate} | ${search_type} +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} +| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | ${binary_min}= | Set Variable | ${min_rate} +| | ${binary_max}= | Set Variable | ${max_rate} +| | ${threshold}= | Set Variable | ${min_rate} +| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology +| | And Add PCI devices to DUTs in 3-node single link topology +| | ${get_framesize}= | Get Frame Size | ${framesize} +| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 100 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-100u1000p-conc +| | Set Test Variable | ${traffic_profile} +| | Then Run Keyword If | '${search_type}' == 'NDR' +| | ... | Find NDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ELSE IF | '${search_type}' == 'PDR' +| | ... | Find PDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} + +*** Test Cases *** +| tc01-64B-1t1c-ethip4udp-ip4base-iacl1-stateful-flows100k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc02-64B-1t1c-ethip4udp-ip4base-iacl1-stateful-flows100k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR + +| tc03-64B-2t2c-ethip4udp-ip4base-iacl1-stateful-flows100k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc04-64B-2t2c-ethip4udp-ip4base-iacl1-stateful-flows100k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sf-10kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sf-10kflows-ndrpdrdisc.robot new file mode 100644 index 0000000000..bf9974ad55 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sf-10kflows-ndrpdrdisc.robot @@ -0,0 +1,147 @@ +# Copyright (c) 2017 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL +| ... | IACL | ACL1 | 10k_FLOWS +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| Test Teardown | Tear down performance test with ACL | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\ +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ +| ... | of packets transmitted. NDR and PDR are discovered for different\ +| ... | Ethernet L2 frame sizes using either binary search or linear search\ +| ... | algorithms with configured starting rate and final step that determines\ +| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit+reflect +| ${acl_apply_type}= | input +| ${no_hit_aces_number}= | 1 +| ${flows_per_dir}= | 10k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +*** Keywords *** +| Discover NDR or PDR for IPv4 routing with ACLs +| | [Arguments] | ${wt} | ${rxq} | ${framesize} | ${min_rate} | ${search_type} +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} +| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | ${binary_min}= | Set Variable | ${min_rate} +| | ${binary_max}= | Set Variable | ${max_rate} +| | ${threshold}= | Set Variable | ${min_rate} +| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology +| | And Add PCI devices to DUTs in 3-node single link topology +| | ${get_framesize}= | Get Frame Size | ${framesize} +| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u1000p-conc +| | Set Test Variable | ${traffic_profile} +| | Then Run Keyword If | '${search_type}' == 'NDR' +| | ... | Find NDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ELSE IF | '${search_type}' == 'PDR' +| | ... | Find PDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} + +*** Test Cases *** +| tc01-64B-1t1c-ethip4udp-ip4base-iacl1-stateful-flows10k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc02-64B-1t1c-ethip4udp-ip4base-iacl1-stateful-flows10k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR + +| tc03-64B-2t2c-ethip4udp-ip4base-iacl1-stateful-flows10k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc04-64B-2t2c-ethip4udp-ip4base-iacl1-stateful-flows10k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sl-100flows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sl-100flows-ndrpdrdisc.robot new file mode 100644 index 0000000000..2eb3ae7734 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sl-100flows-ndrpdrdisc.robot @@ -0,0 +1,147 @@ +# Copyright (c) 2017 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS +| ... | IACL | ACL1 | 100_FLOWS +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| Test Teardown | Tear down performance test with ACL | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\ +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ +| ... | of packets transmitted. NDR and PDR are discovered for different\ +| ... | Ethernet L2 frame sizes using either binary search or linear search\ +| ... | algorithms with configured starting rate and final step that determines\ +| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit +| ${acl_apply_type}= | input +| ${no_hit_aces_number}= | 1 +| ${flows_per_dir}= | 100 + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +*** Keywords *** +| Discover NDR or PDR for IPv4 routing with ACLs +| | [Arguments] | ${wt} | ${rxq} | ${framesize} | ${min_rate} | ${search_type} +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} +| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | ${binary_min}= | Set Variable | ${min_rate} +| | ${binary_max}= | Set Variable | ${max_rate} +| | ${threshold}= | Set Variable | ${min_rate} +| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology +| | And Add PCI devices to DUTs in 3-node single link topology +| | ${get_framesize}= | Get Frame Size | ${framesize} +| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u10p-conc +| | Set Test Variable | ${traffic_profile} +| | Then Run Keyword If | '${search_type}' == 'NDR' +| | ... | Find NDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ELSE IF | '${search_type}' == 'PDR' +| | ... | Find PDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} + +*** Test Cases *** +| tc01-64B-1t1c-ethip4udp-ip4base-iacl1-stateless-flows100-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc02-64B-1t1c-ethip4udp-ip4base-iacl1-stateless-flows100-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR + +| tc03-64B-2t2c-ethip4udp-ip4base-iacl1-stateless-flows100-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc04-64B-2t2c-ethip4udp-ip4base-iacl1-stateless-flows100-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sl-100kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sl-100kflows-ndrpdrdisc.robot new file mode 100644 index 0000000000..3ef6e0474c --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sl-100kflows-ndrpdrdisc.robot @@ -0,0 +1,147 @@ +# Copyright (c) 2017 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS +| ... | IACL | ACL1 | 100k_FLOWS +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| Test Teardown | Tear down performance test with ACL | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\ +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ +| ... | of packets transmitted. NDR and PDR are discovered for different\ +| ... | Ethernet L2 frame sizes using either binary search or linear search\ +| ... | algorithms with configured starting rate and final step that determines\ +| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit +| ${acl_apply_type}= | input +| ${no_hit_aces_number}= | 1 +| ${flows_per_dir}= | 100k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +*** Keywords *** +| Discover NDR or PDR for IPv4 routing with ACLs +| | [Arguments] | ${wt} | ${rxq} | ${framesize} | ${min_rate} | ${search_type} +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} +| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | ${binary_min}= | Set Variable | ${min_rate} +| | ${binary_max}= | Set Variable | ${max_rate} +| | ${threshold}= | Set Variable | ${min_rate} +| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology +| | And Add PCI devices to DUTs in 3-node single link topology +| | ${get_framesize}= | Get Frame Size | ${framesize} +| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 100 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-100u1000p-conc +| | Set Test Variable | ${traffic_profile} +| | Then Run Keyword If | '${search_type}' == 'NDR' +| | ... | Find NDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ELSE IF | '${search_type}' == 'PDR' +| | ... | Find PDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} + +*** Test Cases *** +| tc01-64B-1t1c-ethip4udp-ip4base-iacl1-stateless-flows100k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc02-64B-1t1c-ethip4udp-ip4base-iacl1-stateless-flows100k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR + +| tc03-64B-2t2c-ethip4udp-ip4base-iacl1-stateless-flows100k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc04-64B-2t2c-ethip4udp-ip4base-iacl1-stateless-flows100k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sl-10kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sl-10kflows-ndrpdrdisc.robot new file mode 100644 index 0000000000..3aa0f972d0 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sl-10kflows-ndrpdrdisc.robot @@ -0,0 +1,147 @@ +# Copyright (c) 2017 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| Library | resources.libraries.python.Classify +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS +| ... | IACL | ACL1 | 10k_FLOWS +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| Test Teardown | Tear down performance test with ACL | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\ +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ +| ... | of packets transmitted. NDR and PDR are discovered for different\ +| ... | Ethernet L2 frame sizes using either binary search or linear search\ +| ... | algorithms with configured starting rate and final step that determines\ +| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit +| ${acl_apply_type}= | input +| ${no_hit_aces_number}= | 1 +| ${flows_per_dir}= | 10k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +*** Keywords *** +| Discover NDR or PDR for IPv4 routing with ACLs +| | [Arguments] | ${wt} | ${rxq} | ${framesize} | ${min_rate} | ${search_type} +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} +| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | ${binary_min}= | Set Variable | ${min_rate} +| | ${binary_max}= | Set Variable | ${max_rate} +| | ${threshold}= | Set Variable | ${min_rate} +| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology +| | And Add PCI devices to DUTs in 3-node single link topology +| | ${get_framesize}= | Get Frame Size | ${framesize} +| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u1000p-conc +| | Set Test Variable | ${traffic_profile} +| | Then Run Keyword If | '${search_type}' == 'NDR' +| | ... | Find NDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ELSE IF | '${search_type}' == 'PDR' +| | ... | Find PDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} + +*** Test Cases *** +| tc01-64B-1t1c-ethip4udp-ip4base-iacl1-stateless-flows10k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc02-64B-1t1c-ethip4udp-ip4base-iacl1-stateless-flows10k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR + +| tc03-64B-2t2c-ethip4udp-ip4base-iacl1-stateless-flows10k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc04-64B-2t2c-ethip4udp-ip4base-iacl1-stateless-flows10k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sf-100flows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sf-100flows-ndrpdrdisc.robot new file mode 100644 index 0000000000..fc27dcc59d --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sf-100flows-ndrpdrdisc.robot @@ -0,0 +1,147 @@ +# Copyright (c) 2017 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL +| ... | IACL | ACL50 | 100_FLOWS +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| Test Teardown | Tear down performance test with ACL | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\ +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ +| ... | of packets transmitted. NDR and PDR are discovered for different\ +| ... | Ethernet L2 frame sizes using either binary search or linear search\ +| ... | algorithms with configured starting rate and final step that determines\ +| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit+reflect +| ${acl_apply_type}= | input +| ${no_hit_aces_number}= | 50 +| ${flows_per_dir}= | 100 + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +*** Keywords *** +| Discover NDR or PDR for IPv4 routing with ACLs +| | [Arguments] | ${wt} | ${rxq} | ${framesize} | ${min_rate} | ${search_type} +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} +| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | ${binary_min}= | Set Variable | ${min_rate} +| | ${binary_max}= | Set Variable | ${max_rate} +| | ${threshold}= | Set Variable | ${min_rate} +| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology +| | And Add PCI devices to DUTs in 3-node single link topology +| | ${get_framesize}= | Get Frame Size | ${framesize} +| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u10p-conc +| | Set Test Variable | ${traffic_profile} +| | Then Run Keyword If | '${search_type}' == 'NDR' +| | ... | Find NDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ELSE IF | '${search_type}' == 'PDR' +| | ... | Find PDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} + +*** Test Cases *** +| tc01-64B-1t1c-ethip4udp-ip4base-iacl50-stateful-flows100-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc02-64B-1t1c-ethip4udp-ip4base-iacl50-stateful-flows100-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR + +| tc03-64B-2t2c-ethip4udp-ip4base-iacl50-stateful-flows100-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc04-64B-2t2c-ethip4udp-ip4base-iacl50-stateful-flows100-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sf-100kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sf-100kflows-ndrpdrdisc.robot new file mode 100644 index 0000000000..73d5a1e93a --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sf-100kflows-ndrpdrdisc.robot @@ -0,0 +1,147 @@ +# Copyright (c) 2017 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL +| ... | IACL | ACL50 | 100k_FLOWS +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| Test Teardown | Tear down performance test with ACL | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\ +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ +| ... | of packets transmitted. NDR and PDR are discovered for different\ +| ... | Ethernet L2 frame sizes using either binary search or linear search\ +| ... | algorithms with configured starting rate and final step that determines\ +| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit+reflect +| ${acl_apply_type}= | input +| ${no_hit_aces_number}= | 50 +| ${flows_per_dir}= | 100k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +*** Keywords *** +| Discover NDR or PDR for IPv4 routing with ACLs +| | [Arguments] | ${wt} | ${rxq} | ${framesize} | ${min_rate} | ${search_type} +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} +| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | ${binary_min}= | Set Variable | ${min_rate} +| | ${binary_max}= | Set Variable | ${max_rate} +| | ${threshold}= | Set Variable | ${min_rate} +| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology +| | And Add PCI devices to DUTs in 3-node single link topology +| | ${get_framesize}= | Get Frame Size | ${framesize} +| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 100 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-100u1000p-conc +| | Set Test Variable | ${traffic_profile} +| | Then Run Keyword If | '${search_type}' == 'NDR' +| | ... | Find NDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ELSE IF | '${search_type}' == 'PDR' +| | ... | Find PDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} + +*** Test Cases *** +| tc01-64B-1t1c-ethip4udp-ip4base-iacl50-stateful-flows100k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc02-64B-1t1c-ethip4udp-ip4base-iacl50-stateful-flows100k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| [Tags] | 64B | 1T1C | STHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR + +| tc03-64B-2t2c-ethip4udp-ip4base-iacl50-stateful-flows100k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc04-64B-2t2c-ethip4udp-ip4base-iacl50-stateful-flows100k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sf-10kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sf-10kflows-ndrpdrdisc.robot new file mode 100644 index 0000000000..a3cd4215b5 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sf-10kflows-ndrpdrdisc.robot @@ -0,0 +1,147 @@ +# Copyright (c) 2017 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL +| ... | IACL | ACL50 | 10k_FLOWS +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| Test Teardown | Tear down performance test with ACL | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\ +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ +| ... | of packets transmitted. NDR and PDR are discovered for different\ +| ... | Ethernet L2 frame sizes using either binary search or linear search\ +| ... | algorithms with configured starting rate and final step that determines\ +| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit+reflect +| ${acl_apply_type}= | input +| ${no_hit_aces_number}= | 50 +| ${flows_per_dir}= | 10k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +*** Keywords *** +| Discover NDR or PDR for IPv4 routing with ACLs +| | [Arguments] | ${wt} | ${rxq} | ${framesize} | ${min_rate} | ${search_type} +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} +| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | ${binary_min}= | Set Variable | ${min_rate} +| | ${binary_max}= | Set Variable | ${max_rate} +| | ${threshold}= | Set Variable | ${min_rate} +| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology +| | And Add PCI devices to DUTs in 3-node single link topology +| | ${get_framesize}= | Get Frame Size | ${framesize} +| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u1000p-conc +| | Set Test Variable | ${traffic_profile} +| | Then Run Keyword If | '${search_type}' == 'NDR' +| | ... | Find NDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ELSE IF | '${search_type}' == 'PDR' +| | ... | Find PDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} + +*** Test Cases *** +| tc01-64B-1t1c-ethip4udp-ip4base-iacl50-stateful-flows10k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc02-64B-1t1c-ethip4udp-ip4base-iacl50-stateful-flows10k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR + +| tc03-64B-2t2c-ethip4udp-ip4base-iacl50-stateful-flows10k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc04-64B-2t2c-ethip4udp-ip4base-iacl50-stateful-flows10k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sl-100flows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sl-100flows-ndrpdrdisc.robot new file mode 100644 index 0000000000..2bb5910dfa --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sl-100flows-ndrpdrdisc.robot @@ -0,0 +1,147 @@ +# Copyright (c) 2017 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS +| ... | IACL | ACL50 | 100_FLOWS +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| Test Teardown | Tear down performance test with ACL | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\ +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ +| ... | of packets transmitted. NDR and PDR are discovered for different\ +| ... | Ethernet L2 frame sizes using either binary search or linear search\ +| ... | algorithms with configured starting rate and final step that determines\ +| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit +| ${acl_apply_type}= | input +| ${no_hit_aces_number}= | 50 +| ${flows_per_dir}= | 100 + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +*** Keywords *** +| Discover NDR or PDR for IPv4 routing with ACLs +| | [Arguments] | ${wt} | ${rxq} | ${framesize} | ${min_rate} | ${search_type} +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} +| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | ${binary_min}= | Set Variable | ${min_rate} +| | ${binary_max}= | Set Variable | ${max_rate} +| | ${threshold}= | Set Variable | ${min_rate} +| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology +| | And Add PCI devices to DUTs in 3-node single link topology +| | ${get_framesize}= | Get Frame Size | ${framesize} +| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u10p-conc +| | Set Test Variable | ${traffic_profile} +| | Then Run Keyword If | '${search_type}' == 'NDR' +| | ... | Find NDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ELSE IF | '${search_type}' == 'PDR' +| | ... | Find PDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} + +*** Test Cases *** +| tc01-64B-1t1c-ethip4udp-ip4base-iacl50-stateless-flows100-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc02-64B-1t1c-ethip4udp-ip4base-iacl50-stateless-flows100-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR + +| tc03-64B-2t2c-ethip4udp-ip4base-iacl50-stateless-flows100-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc04-64B-2t2c-ethip4udp-ip4base-iacl50-stateless-flows100-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sl-100kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sl-100kflows-ndrpdrdisc.robot new file mode 100644 index 0000000000..b421314cdb --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sl-100kflows-ndrpdrdisc.robot @@ -0,0 +1,147 @@ +# Copyright (c) 2017 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS +| ... | IACL | ACL50 | 100k_FLOWS +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| Test Teardown | Tear down performance test with ACL | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\ +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ +| ... | of packets transmitted. NDR and PDR are discovered for different\ +| ... | Ethernet L2 frame sizes using either binary search or linear search\ +| ... | algorithms with configured starting rate and final step that determines\ +| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit +| ${acl_apply_type}= | input +| ${no_hit_aces_number}= | 50 +| ${flows_per_dir}= | 100k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +*** Keywords *** +| Discover NDR or PDR for IPv4 routing with ACLs +| | [Arguments] | ${wt} | ${rxq} | ${framesize} | ${min_rate} | ${search_type} +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} +| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | ${binary_min}= | Set Variable | ${min_rate} +| | ${binary_max}= | Set Variable | ${max_rate} +| | ${threshold}= | Set Variable | ${min_rate} +| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology +| | And Add PCI devices to DUTs in 3-node single link topology +| | ${get_framesize}= | Get Frame Size | ${framesize} +| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 100 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-100u1000p-conc +| | Set Test Variable | ${traffic_profile} +| | Then Run Keyword If | '${search_type}' == 'NDR' +| | ... | Find NDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ELSE IF | '${search_type}' == 'PDR' +| | ... | Find PDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} + +*** Test Cases *** +| tc01-64B-1t1c-ethip4udp-ip4base-iacl50-stateless-flows100k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc02-64B-1t1c-ethip4udp-ip4base-iacl50-stateless-flows100k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR + +| tc03-64B-2t2c-ethip4udp-ip4base-iacl50-stateless-flows100k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc04-64B-2t2c-ethip4udp-ip4base-iacl50-stateless-flows100k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sl-10kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sl-10kflows-ndrpdrdisc.robot new file mode 100644 index 0000000000..98f3b445d4 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sl-10kflows-ndrpdrdisc.robot @@ -0,0 +1,147 @@ +# Copyright (c) 2017 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS +| ... | IACL | ACL50 | 10k_FLOWS +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| Test Teardown | Tear down performance test with ACL | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\ +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ +| ... | of packets transmitted. NDR and PDR are discovered for different\ +| ... | Ethernet L2 frame sizes using either binary search or linear search\ +| ... | algorithms with configured starting rate and final step that determines\ +| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit +| ${acl_apply_type}= | input +| ${no_hit_aces_number}= | 50 +| ${flows_per_dir}= | 10k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +*** Keywords *** +| Discover NDR or PDR for IPv4 routing with ACLs +| | [Arguments] | ${wt} | ${rxq} | ${framesize} | ${min_rate} | ${search_type} +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} +| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | ${binary_min}= | Set Variable | ${min_rate} +| | ${binary_max}= | Set Variable | ${max_rate} +| | ${threshold}= | Set Variable | ${min_rate} +| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology +| | And Add PCI devices to DUTs in 3-node single link topology +| | ${get_framesize}= | Get Frame Size | ${framesize} +| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u1000p-conc +| | Set Test Variable | ${traffic_profile} +| | Then Run Keyword If | '${search_type}' == 'NDR' +| | ... | Find NDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ELSE IF | '${search_type}' == 'PDR' +| | ... | Find PDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} + +*** Test Cases *** +| tc01-64B-1t1c-ethip4udp-ip4base-iacl50-stateless-flows10k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc02-64B-1t1c-ethip4udp-ip4base-iacl50-stateless-flows10k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR + +| tc03-64B-2t2c-ethip4udp-ip4base-iacl50-stateless-flows10k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc04-64B-2t2c-ethip4udp-ip4base-iacl50-stateless-flows10k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sf-100flows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sf-100flows-ndrpdrdisc.robot new file mode 100644 index 0000000000..2f3031023b --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sf-100flows-ndrpdrdisc.robot @@ -0,0 +1,147 @@ +# Copyright (c) 2017 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL +| ... | OACL | ACL10 | 100_FLOWS +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| Test Teardown | Tear down performance test with ACL | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\ +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ +| ... | of packets transmitted. NDR and PDR are discovered for different\ +| ... | Ethernet L2 frame sizes using either binary search or linear search\ +| ... | algorithms with configured starting rate and final step that determines\ +| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit+reflect +| ${acl_apply_type}= | output +| ${no_hit_aces_number}= | 10 +| ${flows_per_dir}= | 100 + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +*** Keywords *** +| Discover NDR or PDR for IPv4 routing with ACLs +| | [Arguments] | ${wt} | ${rxq} | ${framesize} | ${min_rate} | ${search_type} +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} +| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | ${binary_min}= | Set Variable | ${min_rate} +| | ${binary_max}= | Set Variable | ${max_rate} +| | ${threshold}= | Set Variable | ${min_rate} +| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology +| | And Add PCI devices to DUTs in 3-node single link topology +| | ${get_framesize}= | Get Frame Size | ${framesize} +| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u10p-conc +| | Set Test Variable | ${traffic_profile} +| | Then Run Keyword If | '${search_type}' == 'NDR' +| | ... | Find NDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ELSE IF | '${search_type}' == 'PDR' +| | ... | Find PDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} + +*** Test Cases *** +| tc01-64B-1t1c-ethip4udp-ip4base-oacl10-stateful-flows100-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc02-64B-1t1c-ethip4udp-ip4base-oacl10-stateful-flows100-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR + +| tc03-64B-2t2c-ethip4udp-ip4base-oacl10-stateful-flows100-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc04-64B-2t2c-ethip4udp-ip4base-oacl10-stateful-flows100-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sf-100kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sf-100kflows-ndrpdrdisc.robot new file mode 100644 index 0000000000..0675f7517e --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sf-100kflows-ndrpdrdisc.robot @@ -0,0 +1,147 @@ +# Copyright (c) 2017 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL +| ... | OACL | ACL10 | 100k_FLOWS +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| Test Teardown | Tear down performance test with ACL | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\ +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ +| ... | of packets transmitted. NDR and PDR are discovered for different\ +| ... | Ethernet L2 frame sizes using either binary search or linear search\ +| ... | algorithms with configured starting rate and final step that determines\ +| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit+reflect +| ${acl_apply_type}= | output +| ${no_hit_aces_number}= | 10 +| ${flows_per_dir}= | 100k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +*** Keywords *** +| Discover NDR or PDR for IPv4 routing with ACLs +| | [Arguments] | ${wt} | ${rxq} | ${framesize} | ${min_rate} | ${search_type} +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} +| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | ${binary_min}= | Set Variable | ${min_rate} +| | ${binary_max}= | Set Variable | ${max_rate} +| | ${threshold}= | Set Variable | ${min_rate} +| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology +| | And Add PCI devices to DUTs in 3-node single link topology +| | ${get_framesize}= | Get Frame Size | ${framesize} +| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 100 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-100u1000p-conc +| | Set Test Variable | ${traffic_profile} +| | Then Run Keyword If | '${search_type}' == 'NDR' +| | ... | Find NDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ELSE IF | '${search_type}' == 'PDR' +| | ... | Find PDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} + +*** Test Cases *** +| tc01-64B-1t1c-ethip4udp-ip4base-oacl10-stateful-flows100k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc02-64B-1t1c-ethip4udp-ip4base-oacl10-stateful-flows100k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR + +| tc03-64B-2t2c-ethip4udp-ip4base-oacl10-stateful-flows100k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc04-64B-2t2c-ethip4udp-ip4base-oacl10-stateful-flows100k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sf-10kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sf-10kflows-ndrpdrdisc.robot new file mode 100644 index 0000000000..64f033cd82 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sf-10kflows-ndrpdrdisc.robot @@ -0,0 +1,147 @@ +# Copyright (c) 2017 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL +| ... | OACL | ACL10 | 10k_FLOWS +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| Test Teardown | Tear down performance test with ACL | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\ +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ +| ... | of packets transmitted. NDR and PDR are discovered for different\ +| ... | Ethernet L2 frame sizes using either binary search or linear search\ +| ... | algorithms with configured starting rate and final step that determines\ +| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit+reflect +| ${acl_apply_type}= | output +| ${no_hit_aces_number}= | 10 +| ${flows_per_dir}= | 10k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +*** Keywords *** +| Discover NDR or PDR for IPv4 routing with ACLs +| | [Arguments] | ${wt} | ${rxq} | ${framesize} | ${min_rate} | ${search_type} +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} +| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | ${binary_min}= | Set Variable | ${min_rate} +| | ${binary_max}= | Set Variable | ${max_rate} +| | ${threshold}= | Set Variable | ${min_rate} +| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology +| | And Add PCI devices to DUTs in 3-node single link topology +| | ${get_framesize}= | Get Frame Size | ${framesize} +| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u1000p-conc +| | Set Test Variable | ${traffic_profile} +| | Then Run Keyword If | '${search_type}' == 'NDR' +| | ... | Find NDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ELSE IF | '${search_type}' == 'PDR' +| | ... | Find PDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} + +*** Test Cases *** +| tc01-64B-1t1c-ethip4udp-ip4base-oacl10-stateful-flows10k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc02-64B-1t1c-ethip4udp-ip4base-oacl10-stateful-flows10k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR + +| tc03-64B-2t2c-ethip4udp-ip4base-oacl10-stateful-flows10k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc04-64B-2t2c-ethip4udp-ip4base-oacl10-stateful-flows10k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sl-100flows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sl-100flows-ndrpdrdisc.robot new file mode 100644 index 0000000000..3985aa066d --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sl-100flows-ndrpdrdisc.robot @@ -0,0 +1,147 @@ +# Copyright (c) 2017 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS +| ... | OACL | ACL10 | 100_FLOWS +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| Test Teardown | Tear down performance test with ACL | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\ +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ +| ... | of packets transmitted. NDR and PDR are discovered for different\ +| ... | Ethernet L2 frame sizes using either binary search or linear search\ +| ... | algorithms with configured starting rate and final step that determines\ +| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit +| ${acl_apply_type}= | output +| ${no_hit_aces_number}= | 10 +| ${flows_per_dir}= | 100 + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +*** Keywords *** +| Discover NDR or PDR for IPv4 routing with ACLs +| | [Arguments] | ${wt} | ${rxq} | ${framesize} | ${min_rate} | ${search_type} +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} +| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | ${binary_min}= | Set Variable | ${min_rate} +| | ${binary_max}= | Set Variable | ${max_rate} +| | ${threshold}= | Set Variable | ${min_rate} +| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology +| | And Add PCI devices to DUTs in 3-node single link topology +| | ${get_framesize}= | Get Frame Size | ${framesize} +| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u10p-conc +| | Set Test Variable | ${traffic_profile} +| | Then Run Keyword If | '${search_type}' == 'NDR' +| | ... | Find NDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ELSE IF | '${search_type}' == 'PDR' +| | ... | Find PDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} + +*** Test Cases *** +| tc01-64B-1t1c-ethip4udp-ip4base-oacl10-stateless-flows100-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc02-64B-1t1c-ethip4udp-ip4base-oacl10-stateless-flows100-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR + +| tc03-64B-2t2c-ethip4udp-ip4base-oacl10-stateless-flows100-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc04-64B-2t2c-ethip4udp-ip4base-oacl10-stateless-flows100-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sl-100kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sl-100kflows-ndrpdrdisc.robot new file mode 100644 index 0000000000..f1e25dff14 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sl-100kflows-ndrpdrdisc.robot @@ -0,0 +1,147 @@ +# Copyright (c) 2017 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS +| ... | OACL | ACL10 | 100k_FLOWS +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| Test Teardown | Tear down performance test with ACL | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\ +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ +| ... | of packets transmitted. NDR and PDR are discovered for different\ +| ... | Ethernet L2 frame sizes using either binary search or linear search\ +| ... | algorithms with configured starting rate and final step that determines\ +| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit +| ${acl_apply_type}= | output +| ${no_hit_aces_number}= | 10 +| ${flows_per_dir}= | 100k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +*** Keywords *** +| Discover NDR or PDR for IPv4 routing with ACLs +| | [Arguments] | ${wt} | ${rxq} | ${framesize} | ${min_rate} | ${search_type} +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} +| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | ${binary_min}= | Set Variable | ${min_rate} +| | ${binary_max}= | Set Variable | ${max_rate} +| | ${threshold}= | Set Variable | ${min_rate} +| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology +| | And Add PCI devices to DUTs in 3-node single link topology +| | ${get_framesize}= | Get Frame Size | ${framesize} +| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 100 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-100u1000p-conc +| | Set Test Variable | ${traffic_profile} +| | Then Run Keyword If | '${search_type}' == 'NDR' +| | ... | Find NDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ELSE IF | '${search_type}' == 'PDR' +| | ... | Find PDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} + +*** Test Cases *** +| tc01-64B-1t1c-ethip4udp-ip4base-oacl10-stateless-flows100k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc02-64B-1t1c-ethip4udp-ip4base-oacl10-stateless-flows100k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR + +| tc03-64B-2t2c-ethip4udp-ip4base-oacl10-stateless-flows100k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc04-64B-2t2c-ethip4udp-ip4base-oacl10-stateless-flows100k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sl-10kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sl-10kflows-ndrpdrdisc.robot new file mode 100644 index 0000000000..1f1f3b1e65 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sl-10kflows-ndrpdrdisc.robot @@ -0,0 +1,147 @@ +# Copyright (c) 2017 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS +| ... | OACL | ACL10 | 10k_FLOWS +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| Test Teardown | Tear down performance test with ACL | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\ +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ +| ... | of packets transmitted. NDR and PDR are discovered for different\ +| ... | Ethernet L2 frame sizes using either binary search or linear search\ +| ... | algorithms with configured starting rate and final step that determines\ +| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit +| ${acl_apply_type}= | output +| ${no_hit_aces_number}= | 10 +| ${flows_per_dir}= | 10k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +*** Keywords *** +| Discover NDR or PDR for IPv4 routing with ACLs +| | [Arguments] | ${wt} | ${rxq} | ${framesize} | ${min_rate} | ${search_type} +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} +| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | ${binary_min}= | Set Variable | ${min_rate} +| | ${binary_max}= | Set Variable | ${max_rate} +| | ${threshold}= | Set Variable | ${min_rate} +| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology +| | And Add PCI devices to DUTs in 3-node single link topology +| | ${get_framesize}= | Get Frame Size | ${framesize} +| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u1000p-conc +| | Set Test Variable | ${traffic_profile} +| | Then Run Keyword If | '${search_type}' == 'NDR' +| | ... | Find NDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ELSE IF | '${search_type}' == 'PDR' +| | ... | Find PDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} + +*** Test Cases *** +| tc01-64B-1t1c-ethip4udp-ip4base-oacl10-stateless-flows10k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc02-64B-1t1c-ethip4udp-ip4base-oacl10-stateless-flows10k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR + +| tc03-64B-2t2c-ethip4udp-ip4base-oacl10-stateless-flows10k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc04-64B-2t2c-ethip4udp-ip4base-oacl10-stateless-flows10k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sf-100flows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sf-100flows-ndrpdrdisc.robot new file mode 100644 index 0000000000..bdb16aade7 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sf-100flows-ndrpdrdisc.robot @@ -0,0 +1,147 @@ +# Copyright (c) 2017 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL +| ... | OACL | ACL1 | 100_FLOWS +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| Test Teardown | Tear down performance test with ACL | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\ +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ +| ... | of packets transmitted. NDR and PDR are discovered for different\ +| ... | Ethernet L2 frame sizes using either binary search or linear search\ +| ... | algorithms with configured starting rate and final step that determines\ +| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit+reflect +| ${acl_apply_type}= | output +| ${no_hit_aces_number}= | 1 +| ${flows_per_dir}= | 100 + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +*** Keywords *** +| Discover NDR or PDR for IPv4 routing with ACLs +| | [Arguments] | ${wt} | ${rxq} | ${framesize} | ${min_rate} | ${search_type} +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} +| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | ${binary_min}= | Set Variable | ${min_rate} +| | ${binary_max}= | Set Variable | ${max_rate} +| | ${threshold}= | Set Variable | ${min_rate} +| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology +| | And Add PCI devices to DUTs in 3-node single link topology +| | ${get_framesize}= | Get Frame Size | ${framesize} +| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u10p-conc +| | Set Test Variable | ${traffic_profile} +| | Then Run Keyword If | '${search_type}' == 'NDR' +| | ... | Find NDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ELSE IF | '${search_type}' == 'PDR' +| | ... | Find PDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} + +*** Test Cases *** +| tc01-64B-1t1c-ethip4udp-ip4base-oacl1-stateful-flows100-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc02-64B-1t1c-ethip4udp-ip4base-oacl1-stateful-flows100-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR + +| tc03-64B-2t2c-ethip4udp-ip4base-oacl1-stateful-flows100-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc04-64B-2t2c-ethip4udp-ip4base-oacl1-stateful-flows100-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sf-100kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sf-100kflows-ndrpdrdisc.robot new file mode 100644 index 0000000000..09fac074b9 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sf-100kflows-ndrpdrdisc.robot @@ -0,0 +1,147 @@ +# Copyright (c) 2017 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL +| ... | OACL | ACL1 | 100k_FLOWS +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| Test Teardown | Tear down performance test with ACL | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\ +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ +| ... | of packets transmitted. NDR and PDR are discovered for different\ +| ... | Ethernet L2 frame sizes using either binary search or linear search\ +| ... | algorithms with configured starting rate and final step that determines\ +| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit+reflect +| ${acl_apply_type}= | output +| ${no_hit_aces_number}= | 1 +| ${flows_per_dir}= | 100k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +*** Keywords *** +| Discover NDR or PDR for IPv4 routing with ACLs +| | [Arguments] | ${wt} | ${rxq} | ${framesize} | ${min_rate} | ${search_type} +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} +| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | ${binary_min}= | Set Variable | ${min_rate} +| | ${binary_max}= | Set Variable | ${max_rate} +| | ${threshold}= | Set Variable | ${min_rate} +| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology +| | And Add PCI devices to DUTs in 3-node single link topology +| | ${get_framesize}= | Get Frame Size | ${framesize} +| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 100 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-100u1000p-conc +| | Set Test Variable | ${traffic_profile} +| | Then Run Keyword If | '${search_type}' == 'NDR' +| | ... | Find NDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ELSE IF | '${search_type}' == 'PDR' +| | ... | Find PDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} + +*** Test Cases *** +| tc01-64B-1t1c-ethip4udp-ip4base-oacl1-stateful-flows100k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc02-64B-1t1c-ethip4udp-ip4base-oacl1-stateful-flows100k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR + +| tc03-64B-2t2c-ethip4udp-ip4base-oacl1-stateful-flows100k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc04-64B-2t2c-ethip4udp-ip4base-oacl1-stateful-flows100k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sf-10kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sf-10kflows-ndrpdrdisc.robot new file mode 100644 index 0000000000..074c07cdcd --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sf-10kflows-ndrpdrdisc.robot @@ -0,0 +1,147 @@ +# Copyright (c) 2017 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL +| ... | OACL | ACL1 | 10k_FLOWS +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| Test Teardown | Tear down performance test with ACL | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\ +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ +| ... | of packets transmitted. NDR and PDR are discovered for different\ +| ... | Ethernet L2 frame sizes using either binary search or linear search\ +| ... | algorithms with configured starting rate and final step that determines\ +| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit+reflect +| ${acl_apply_type}= | output +| ${no_hit_aces_number}= | 1 +| ${flows_per_dir}= | 10k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +*** Keywords *** +| Discover NDR or PDR for IPv4 routing with ACLs +| | [Arguments] | ${wt} | ${rxq} | ${framesize} | ${min_rate} | ${search_type} +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} +| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | ${binary_min}= | Set Variable | ${min_rate} +| | ${binary_max}= | Set Variable | ${max_rate} +| | ${threshold}= | Set Variable | ${min_rate} +| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology +| | And Add PCI devices to DUTs in 3-node single link topology +| | ${get_framesize}= | Get Frame Size | ${framesize} +| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u1000p-conc +| | Set Test Variable | ${traffic_profile} +| | Then Run Keyword If | '${search_type}' == 'NDR' +| | ... | Find NDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ELSE IF | '${search_type}' == 'PDR' +| | ... | Find PDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} + +*** Test Cases *** +| tc01-64B-1t1c-ethip4udp-ip4base-oacl1-stateful-flows10k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc02-64B-1t1c-ethip4udp-ip4base-oacl1-stateful-flows10k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR + +| tc03-64B-2t2c-ethip4udp-ip4base-oacl1-stateful-flows10k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc04-64B-2t2c-ethip4udp-ip4base-oacl1-stateful-flows10k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sl-100flows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sl-100flows-ndrpdrdisc.robot new file mode 100644 index 0000000000..a03b5cc31b --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sl-100flows-ndrpdrdisc.robot @@ -0,0 +1,147 @@ +# Copyright (c) 2017 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS +| ... | OACL | ACL1 | 100_FLOWS +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| Test Teardown | Tear down performance test with ACL | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\ +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ +| ... | of packets transmitted. NDR and PDR are discovered for different\ +| ... | Ethernet L2 frame sizes using either binary search or linear search\ +| ... | algorithms with configured starting rate and final step that determines\ +| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit +| ${acl_apply_type}= | output +| ${no_hit_aces_number}= | 1 +| ${flows_per_dir}= | 100 + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +*** Keywords *** +| Discover NDR or PDR for IPv4 routing with ACLs +| | [Arguments] | ${wt} | ${rxq} | ${framesize} | ${min_rate} | ${search_type} +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} +| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | ${binary_min}= | Set Variable | ${min_rate} +| | ${binary_max}= | Set Variable | ${max_rate} +| | ${threshold}= | Set Variable | ${min_rate} +| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology +| | And Add PCI devices to DUTs in 3-node single link topology +| | ${get_framesize}= | Get Frame Size | ${framesize} +| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u10p-conc +| | Set Test Variable | ${traffic_profile} +| | Then Run Keyword If | '${search_type}' == 'NDR' +| | ... | Find NDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ELSE IF | '${search_type}' == 'PDR' +| | ... | Find PDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} + +*** Test Cases *** +| tc01-64B-1t1c-ethip4udp-ip4base-oacl1-stateless-flows100-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc02-64B-1t1c-ethip4udp-ip4base-oacl1-stateless-flows100-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR + +| tc03-64B-2t2c-ethip4udp-ip4base-oacl1-stateless-flows100-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc04-64B-2t2c-ethip4udp-ip4base-oacl1-stateless-flows100-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sl-100kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sl-100kflows-ndrpdrdisc.robot new file mode 100644 index 0000000000..5bca34e36e --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sl-100kflows-ndrpdrdisc.robot @@ -0,0 +1,147 @@ +# Copyright (c) 2017 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS +| ... | OACL | ACL1 | 100k_FLOWS +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| Test Teardown | Tear down performance test with ACL | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\ +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ +| ... | of packets transmitted. NDR and PDR are discovered for different\ +| ... | Ethernet L2 frame sizes using either binary search or linear search\ +| ... | algorithms with configured starting rate and final step that determines\ +| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit +| ${acl_apply_type}= | output +| ${no_hit_aces_number}= | 1 +| ${flows_per_dir}= | 100k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +*** Keywords *** +| Discover NDR or PDR for IPv4 routing with ACLs +| | [Arguments] | ${wt} | ${rxq} | ${framesize} | ${min_rate} | ${search_type} +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} +| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | ${binary_min}= | Set Variable | ${min_rate} +| | ${binary_max}= | Set Variable | ${max_rate} +| | ${threshold}= | Set Variable | ${min_rate} +| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology +| | And Add PCI devices to DUTs in 3-node single link topology +| | ${get_framesize}= | Get Frame Size | ${framesize} +| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 100 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-100u1000p-conc +| | Set Test Variable | ${traffic_profile} +| | Then Run Keyword If | '${search_type}' == 'NDR' +| | ... | Find NDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ELSE IF | '${search_type}' == 'PDR' +| | ... | Find PDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} + +*** Test Cases *** +| tc01-64B-1t1c-ethip4udp-ip4base-oacl1-stateless-flows100k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc02-64B-1t1c-ethip4udp-ip4base-oacl1-stateless-flows100k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR + +| tc03-64B-2t2c-ethip4udp-ip4base-oacl1-stateless-flows100k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc04-64B-2t2c-ethip4udp-ip4base-oacl1-stateless-flows100k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sl-10kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sl-10kflows-ndrpdrdisc.robot new file mode 100644 index 0000000000..c1bb104d6f --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sl-10kflows-ndrpdrdisc.robot @@ -0,0 +1,147 @@ +# Copyright (c) 2017 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS +| ... | OACL | ACL1 | 10k_FLOWS +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| Test Teardown | Tear down performance test with ACL | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\ +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ +| ... | of packets transmitted. NDR and PDR are discovered for different\ +| ... | Ethernet L2 frame sizes using either binary search or linear search\ +| ... | algorithms with configured starting rate and final step that determines\ +| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit +| ${acl_apply_type}= | output +| ${no_hit_aces_number}= | 1 +| ${flows_per_dir}= | 10k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +*** Keywords *** +| Discover NDR or PDR for IPv4 routing with ACLs +| | [Arguments] | ${wt} | ${rxq} | ${framesize} | ${min_rate} | ${search_type} +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} +| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | ${binary_min}= | Set Variable | ${min_rate} +| | ${binary_max}= | Set Variable | ${max_rate} +| | ${threshold}= | Set Variable | ${min_rate} +| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology +| | And Add PCI devices to DUTs in 3-node single link topology +| | ${get_framesize}= | Get Frame Size | ${framesize} +| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u1000p-conc +| | Set Test Variable | ${traffic_profile} +| | Then Run Keyword If | '${search_type}' == 'NDR' +| | ... | Find NDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ELSE IF | '${search_type}' == 'PDR' +| | ... | Find PDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} + +*** Test Cases *** +| tc01-64B-1t1c-ethip4udp-ip4base-oacl1-stateless-flows10k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc02-64B-1t1c-ethip4udp-ip4base-oacl1-stateless-flows10k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR + +| tc03-64B-2t2c-ethip4udp-ip4base-oacl1-stateless-flows10k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc04-64B-2t2c-ethip4udp-ip4base-oacl1-stateless-flows10k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sf-100flows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sf-100flows-ndrpdrdisc.robot new file mode 100644 index 0000000000..47b877ba4a --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sf-100flows-ndrpdrdisc.robot @@ -0,0 +1,147 @@ +# Copyright (c) 2017 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL +| ... | OACL | ACL50 | 100_FLOWS +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| Test Teardown | Tear down performance test with ACL | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\ +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ +| ... | of packets transmitted. NDR and PDR are discovered for different\ +| ... | Ethernet L2 frame sizes using either binary search or linear search\ +| ... | algorithms with configured starting rate and final step that determines\ +| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit+reflect +| ${acl_apply_type}= | output +| ${no_hit_aces_number}= | 50 +| ${flows_per_dir}= | 100 + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +*** Keywords *** +| Discover NDR or PDR for IPv4 routing with ACLs +| | [Arguments] | ${wt} | ${rxq} | ${framesize} | ${min_rate} | ${search_type} +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} +| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | ${binary_min}= | Set Variable | ${min_rate} +| | ${binary_max}= | Set Variable | ${max_rate} +| | ${threshold}= | Set Variable | ${min_rate} +| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology +| | And Add PCI devices to DUTs in 3-node single link topology +| | ${get_framesize}= | Get Frame Size | ${framesize} +| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u10p-conc +| | Set Test Variable | ${traffic_profile} +| | Then Run Keyword If | '${search_type}' == 'NDR' +| | ... | Find NDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ELSE IF | '${search_type}' == 'PDR' +| | ... | Find PDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} + +*** Test Cases *** +| tc01-64B-1t1c-ethip4udp-ip4base-oacl50-stateful-flows100-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc02-64B-1t1c-ethip4udp-ip4base-oacl50-stateful-flows100-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR + +| tc03-64B-2t2c-ethip4udp-ip4base-oacl50-stateful-flows100-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc04-64B-2t2c-ethip4udp-ip4base-oacl50-stateful-flows100-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sf-100kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sf-100kflows-ndrpdrdisc.robot new file mode 100644 index 0000000000..ef31d67e55 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sf-100kflows-ndrpdrdisc.robot @@ -0,0 +1,147 @@ +# Copyright (c) 2017 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL +| ... | OACL | ACL50 | 100k_FLOWS +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| Test Teardown | Tear down performance test with ACL | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\ +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ +| ... | of packets transmitted. NDR and PDR are discovered for different\ +| ... | Ethernet L2 frame sizes using either binary search or linear search\ +| ... | algorithms with configured starting rate and final step that determines\ +| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit+reflect +| ${acl_apply_type}= | output +| ${no_hit_aces_number}= | 50 +| ${flows_per_dir}= | 100k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +*** Keywords *** +| Discover NDR or PDR for IPv4 routing with ACLs +| | [Arguments] | ${wt} | ${rxq} | ${framesize} | ${min_rate} | ${search_type} +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} +| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | ${binary_min}= | Set Variable | ${min_rate} +| | ${binary_max}= | Set Variable | ${max_rate} +| | ${threshold}= | Set Variable | ${min_rate} +| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology +| | And Add PCI devices to DUTs in 3-node single link topology +| | ${get_framesize}= | Get Frame Size | ${framesize} +| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 100 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-100u1000p-conc +| | Set Test Variable | ${traffic_profile} +| | Then Run Keyword If | '${search_type}' == 'NDR' +| | ... | Find NDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ELSE IF | '${search_type}' == 'PDR' +| | ... | Find PDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} + +*** Test Cases *** +| tc01-64B-1t1c-ethip4udp-ip4base-oacl50-stateful-flows100k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc02-64B-1t1c-ethip4udp-ip4base-oacl50-stateful-flows100k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR + +| tc03-64B-2t2c-ethip4udp-ip4base-oacl50-stateful-flows100k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc04-64B-2t2c-ethip4udp-ip4base-oacl50-stateful-flows100k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sf-10kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sf-10kflows-ndrpdrdisc.robot new file mode 100644 index 0000000000..b0a9f72730 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sf-10kflows-ndrpdrdisc.robot @@ -0,0 +1,147 @@ +# Copyright (c) 2017 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL +| ... | OACL | ACL50 | 10k_FLOWS +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| Test Teardown | Tear down performance test with ACL | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\ +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ +| ... | of packets transmitted. NDR and PDR are discovered for different\ +| ... | Ethernet L2 frame sizes using either binary search or linear search\ +| ... | algorithms with configured starting rate and final step that determines\ +| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit+reflect +| ${acl_apply_type}= | output +| ${no_hit_aces_number}= | 50 +| ${flows_per_dir}= | 10k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +*** Keywords *** +| Discover NDR or PDR for IPv4 routing with ACLs +| | [Arguments] | ${wt} | ${rxq} | ${framesize} | ${min_rate} | ${search_type} +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} +| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | ${binary_min}= | Set Variable | ${min_rate} +| | ${binary_max}= | Set Variable | ${max_rate} +| | ${threshold}= | Set Variable | ${min_rate} +| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology +| | And Add PCI devices to DUTs in 3-node single link topology +| | ${get_framesize}= | Get Frame Size | ${framesize} +| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u1000p-conc +| | Set Test Variable | ${traffic_profile} +| | Then Run Keyword If | '${search_type}' == 'NDR' +| | ... | Find NDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ELSE IF | '${search_type}' == 'PDR' +| | ... | Find PDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} + +*** Test Cases *** +| tc01-64B-1t1c-ethip4udp-ip4base-oacl50-stateful-flows10k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc02-64B-1t1c-ethip4udp-ip4base-oacl50-stateful-flows10k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR + +| tc03-64B-2t2c-ethip4udp-ip4base-oacl50-stateful-flows10k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc04-64B-2t2c-ethip4udp-ip4base-oacl50-stateful-flows10k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sl-100flows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sl-100flows-ndrpdrdisc.robot new file mode 100644 index 0000000000..3da5957ea7 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sl-100flows-ndrpdrdisc.robot @@ -0,0 +1,147 @@ +# Copyright (c) 2017 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS +| ... | OACL | ACL50 | 100_FLOWS +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| Test Teardown | Tear down performance test with ACL | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\ +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ +| ... | of packets transmitted. NDR and PDR are discovered for different\ +| ... | Ethernet L2 frame sizes using either binary search or linear search\ +| ... | algorithms with configured starting rate and final step that determines\ +| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit +| ${acl_apply_type}= | output +| ${no_hit_aces_number}= | 50 +| ${flows_per_dir}= | 100 + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +*** Keywords *** +| Discover NDR or PDR for IPv4 routing with ACLs +| | [Arguments] | ${wt} | ${rxq} | ${framesize} | ${min_rate} | ${search_type} +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} +| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | ${binary_min}= | Set Variable | ${min_rate} +| | ${binary_max}= | Set Variable | ${max_rate} +| | ${threshold}= | Set Variable | ${min_rate} +| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology +| | And Add PCI devices to DUTs in 3-node single link topology +| | ${get_framesize}= | Get Frame Size | ${framesize} +| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u10p-conc +| | Set Test Variable | ${traffic_profile} +| | Then Run Keyword If | '${search_type}' == 'NDR' +| | ... | Find NDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ELSE IF | '${search_type}' == 'PDR' +| | ... | Find PDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} + +*** Test Cases *** +| tc01-64B-1t1c-ethip4udp-ip4base-oacl50-stateless-flows100-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc02-64B-1t1c-ethip4udp-ip4base-oacl50-stateless-flows100-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR + +| tc03-64B-2t2c-ethip4udp-ip4base-oacl50-stateless-flows100-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc04-64B-2t2c-ethip4udp-ip4base-oacl50-stateless-flows100-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sl-100kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sl-100kflows-ndrpdrdisc.robot new file mode 100644 index 0000000000..d11a282d1a --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sl-100kflows-ndrpdrdisc.robot @@ -0,0 +1,147 @@ +# Copyright (c) 2017 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS +| ... | OACL | ACL50 | 100k_FLOWS +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| Test Teardown | Tear down performance test with ACL | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\ +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ +| ... | of packets transmitted. NDR and PDR are discovered for different\ +| ... | Ethernet L2 frame sizes using either binary search or linear search\ +| ... | algorithms with configured starting rate and final step that determines\ +| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit +| ${acl_apply_type}= | output +| ${no_hit_aces_number}= | 50 +| ${flows_per_dir}= | 100k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +*** Keywords *** +| Discover NDR or PDR for IPv4 routing with ACLs +| | [Arguments] | ${wt} | ${rxq} | ${framesize} | ${min_rate} | ${search_type} +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} +| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | ${binary_min}= | Set Variable | ${min_rate} +| | ${binary_max}= | Set Variable | ${max_rate} +| | ${threshold}= | Set Variable | ${min_rate} +| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology +| | And Add PCI devices to DUTs in 3-node single link topology +| | ${get_framesize}= | Get Frame Size | ${framesize} +| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 100 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-100u1000p-conc +| | Set Test Variable | ${traffic_profile} +| | Then Run Keyword If | '${search_type}' == 'NDR' +| | ... | Find NDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ELSE IF | '${search_type}' == 'PDR' +| | ... | Find PDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} + +*** Test Cases *** +| tc01-64B-1t1c-ethip4udp-ip4base-oacl50-stateless-flows100k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc02-64B-1t1c-ethip4udp-ip4base-oacl50-stateless-flows100k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR + +| tc03-64B-2t2c-ethip4udp-ip4base-oacl50-stateless-flows100k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc04-64B-2t2c-ethip4udp-ip4base-oacl50-stateless-flows100k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sl-10kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sl-10kflows-ndrpdrdisc.robot new file mode 100644 index 0000000000..f3d2004fcb --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sl-10kflows-ndrpdrdisc.robot @@ -0,0 +1,147 @@ +# Copyright (c) 2017 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS +| ... | OACL | ACL50 | 10k_FLOWS +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| Test Teardown | Tear down performance test with ACL | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\ +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ +| ... | of packets transmitted. NDR and PDR are discovered for different\ +| ... | Ethernet L2 frame sizes using either binary search or linear search\ +| ... | algorithms with configured starting rate and final step that determines\ +| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit +| ${acl_apply_type}= | output +| ${no_hit_aces_number}= | 50 +| ${flows_per_dir}= | 10k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +*** Keywords *** +| Discover NDR or PDR for IPv4 routing with ACLs +| | [Arguments] | ${wt} | ${rxq} | ${framesize} | ${min_rate} | ${search_type} +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} +| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | ${binary_min}= | Set Variable | ${min_rate} +| | ${binary_max}= | Set Variable | ${max_rate} +| | ${threshold}= | Set Variable | ${min_rate} +| | Given Add '${wt}' worker threads and '${rxq}' rxqueues in 3-node single-link circular topology +| | And Add PCI devices to DUTs in 3-node single link topology +| | ${get_framesize}= | Get Frame Size | ${framesize} +| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u1000p-conc +| | Set Test Variable | ${traffic_profile} +| | Then Run Keyword If | '${search_type}' == 'NDR' +| | ... | Find NDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ELSE IF | '${search_type}' == 'PDR' +| | ... | Find PDR using binary search and pps +| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} +| | ... | ${min_rate} | ${max_rate} | ${threshold} +| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} + +*** Test Cases *** +| tc01-64B-1t1c-ethip4udp-ip4base-oacl50-stateless-flows10k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc02-64B-1t1c-ethip4udp-ip4base-oacl50-stateless-flows10k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 1 thread, 1 phy core, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 1T1C | STHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=1 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR + +| tc03-64B-2t2c-ethip4udp-ip4base-oacl50-stateless-flows10k-ndrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find NDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | NDRDISC +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=NDR + +| tc04-64B-2t2c-ethip4udp-ip4base-oacl50-stateless-flows10k-pdrdisc +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL with\ +| | ... | 2 threads, 2 phy cores, 1 receive queue per NIC port. +| | ... | [Ver] Find PDR for 64 Byte frames using binary search start at 10GE\ +| | ... | linerate, step 10kpps, LT=0.5%. +| | ... +| | [Tags] | 64B | 2T2C | MTHREAD | PDRDISC | SKIP_PATCH +| | ... +| | [Template] | Discover NDR or PDR for IPv4 routing with ACLs +| | wt=2 | rxq=1 | framesize=${64} | min_rate=${10000} | search_type=PDR |