diff options
| author |   Vratko Polak <vrpolak@cisco.com> | 2018-07-19 19:14:26 +0200 |
|---|---|---|
| committer | Vratko Polak <vrpolak@cisco.com> | 2018-07-24 10:05:38 +0200 |
| commit | 74771ecb4e2742efd3ab40bb8dd9cf9ac1bb40c8 (patch) | |
| tree | 1f4b958215f272b2d79e63e32c8b341b9845ae99 /tests | |
| parent | f7ad4e08157eecc91d4be2a05c891f06bca9f966 (diff) | |
CSIT-1097: Migrate ip4 to NDRPDR and edit MRR
Change-Id: I9e3b2973c14f818247658a2ae34456c9fc2bae4e
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
Diffstat (limited to 'tests')
152 files changed, 12460 insertions, 6875 deletions
diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4base-copwhtlistbase-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4base-copwhtlistbase-mrr.robot index e0c7941aa2..0b8842b9d1 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4base-copwhtlistbase-mrr.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4base-copwhtlistbase-mrr.robot @@ -26,7 +26,7 @@ | ... | Test Teardown | Tear down performance mrr test | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *Raw results IPv4 whitelist test cases* | ... @@ -53,7 +53,7 @@ | ${traffic_profile}= | trex-sl-3n-ethip4-ip4src253 *** Keywords *** -| Local template +| Local Template | | ... | | [Documentation] | | ... | [Cfg] DUT runs IPv4 routing config. @@ -67,16 +67,12 @@ | | ... | - phy_cores - Number of physical cores. Type: integer | | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer | | ... -| | [Arguments] | ${phy_cores} | ${framesize} | ${rxq}=${None} -| | ... -| | Set Test Variable | ${framesize} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${get_framesize} +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | When Initialize IPv4 forwarding in 3-node circular topology | | And Add Fib Table | ${dut1} | 1 @@ -91,50 +87,50 @@ | | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} *** Test Cases *** -| tc01-64B-1t1c-ethip4-ip4base-copwhtlistbase-mrr +| tc01-64B-1c-ethip4-ip4base-copwhtlistbase-mrr | | [Tags] | 64B | 1C -| | phy_cores=${1} | framesize=${64} - -| tc02-1518B-1t1c-ethip4-ip4base-copwhtlistbase-mrr -| | [Tags] | 1518B | 1C -| | phy_cores=${1} | framesize=${1518} +| | framesize=${64} | phy_cores=${1} -| tc03-9000B-1t1c-ethip4-ip4base-copwhtlistbase-mrr -| | [Tags] | 9000B | 1C -| | phy_cores=${1} | framesize=${9000} +| tc02-64B-2c-ethip4-ip4base-copwhtlistbase-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} -| tc04-IMIX-1t1c-ethip4-ip4base-copwhtlistbase-mrr -| | [Tags] | IMIX | 1C -| | phy_cores=${1} | framesize=IMIX_v4_1 +| tc03-64B-4c-ethip4-ip4base-copwhtlistbase-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} -| tc05-64B-2t2c-ethip4-ip4base-copwhtlistbase-mrr -| | [Tags] | 64B | 2C -| | phy_cores=${2} | framesize=${64} +| tc04-1518B-1c-ethip4-ip4base-copwhtlistbase-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} -| tc06-1518B-2t2c-ethip4-ip4base-copwhtlistbase-mrr +| tc05-1518B-2c-ethip4-ip4base-copwhtlistbase-mrr | | [Tags] | 1518B | 2C -| | phy_cores=${2} | framesize=${1518} - -| tc07-9000B-2t2c-ethip4-ip4base-copwhtlistbase-mrr -| | [Tags] | 9000B | 2C -| | phy_cores=${2} | framesize=${9000} +| | framesize=${1518} | phy_cores=${2} -| tc08-IMIX-2t2c-ethip4-ip4base-copwhtlistbase-mrr -| | [Tags] | IMIX | 2C -| | phy_cores=${2} | framesize=IMIX_v4_1 +| tc06-1518B-4c-ethip4-ip4base-copwhtlistbase-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} -| tc09-64B-4t4c-ethip4-ip4base-copwhtlistbase-mrr -| | [Tags] | 64B | 4C -| | phy_cores=${4} | framesize=${64} +| tc07-9000B-1c-ethip4-ip4base-copwhtlistbase-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc10-1518B-4t4c-ethip4-ip4base-copwhtlistbase-mrr -| | [Tags] | 1518B | 4C -| | phy_cores=${4} | framesize=${1518} +| tc08-9000B-2c-ethip4-ip4base-copwhtlistbase-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc11-9000B-4t4c-ethip4-ip4base-copwhtlistbase-mrr +| tc09-9000B-4c-ethip4-ip4base-copwhtlistbase-mrr | | [Tags] | 9000B | 4C -| | phy_cores=${4} | framesize=${9000} +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4-ip4base-copwhtlistbase-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4-ip4base-copwhtlistbase-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc12-IMIX-4t4c-ethip4-ip4base-copwhtlistbase-mrr +| tc12-IMIX-4c-ethip4-ip4base-copwhtlistbase-mrr | | [Tags] | IMIX | 4C -| | phy_cores=${4} | framesize=IMIX_v4_1 +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4base-copwhtlistbase-ndrpdr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4base-copwhtlistbase-ndrpdr.robot new file mode 100644 index 0000000000..5e7fd80616 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4base-copwhtlistbase-ndrpdr.robot @@ -0,0 +1,142 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| Library | resources.libraries.python.Cop +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | COPWHLIST +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down performance discovery test | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Test Template | Local Template +| ... +| Documentation | *RFC2544: Pkt throughput IPv4 whitelist test cases* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4 for IPv4 routing. +| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 +| ... | routing, two static IPv4 /24 routes and IPv4 COP security whitelist +| ... | ingress /24 filter entries applied on links TG - DUT1 and DUT2 - TG. +| ... | DUT1 and DUT2 tested with 2p10GE NIC X520 Niantic by Intel. +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage +| ... | of packets transmitted. NDR and PDR are discovered for different +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on links to DUTs. TG traffic profile +| ... | contains two L3 flow-groups (flow-group per direction, 253 flows per +| ... | flow-group) with all packets containing Ethernet header, IPv4 header +| ... | with IP protocol=61 and static payload. MAC addresses are matching MAC +| ... | addresses of the TG node interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} +# Traffic profile: +| ${traffic_profile}= | trex-sl-3n-ethip4-ip4src253 + +*** Keywords *** +| Local Template +| | ... +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config. +| | ... | Each DUT uses ${phy_cores} physical core(s) for worker threads. +| | ... | [Ver] Measure NDR and PDR values using MLRsearch algorithm.\ +| | ... +| | ... | *Arguments:* +| | ... | - framesize - Framesize in Bytes in integer or string (IMIX_v4_1). +| | ... | Type: integer, string +| | ... | - phy_cores - Number of physical cores. Type: integer +| | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer +| | ... +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | When Initialize IPv4 forwarding in 3-node circular topology +| | And Add Fib Table | ${dut1} | 1 +| | And Vpp Route Add | ${dut1} | 10.10.10.0 | 24 |Â vrf=1 | local=${TRUE} +| | And Add Fib Table | ${dut2} | 1 +| | And Vpp Route Add | ${dut2} | 20.20.20.0 | 24 |Â vrf=1 | local=${TRUE} +| | And COP Add whitelist Entry | ${dut1} | ${dut1_if1} | ip4 | 1 +| | And COP Add whitelist Entry | ${dut2} | ${dut2_if2} | ip4 | 1 +| | And COP interface enable or disable | ${dut1} | ${dut1_if1} | enable +| | And COP interface enable or disable | ${dut2} | ${dut2_if2} | enable +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} + +*** Test Cases *** +| tc01-64B-1c-ethip4-ip4base-copwhtlistbase-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4-ip4base-copwhtlistbase-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4-ip4base-copwhtlistbase-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4-ip4base-copwhtlistbase-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4-ip4base-copwhtlistbase-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4-ip4base-copwhtlistbase-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4-ip4base-copwhtlistbase-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4-ip4base-copwhtlistbase-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4-ip4base-copwhtlistbase-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4-ip4base-copwhtlistbase-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4-ip4base-copwhtlistbase-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4-ip4base-copwhtlistbase-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4base-copwhtlistbase-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4base-copwhtlistbase-ndrpdrdisc.robot deleted file mode 100644 index 5c6b3bfca8..0000000000 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4base-copwhtlistbase-ndrpdrdisc.robot +++ /dev/null @@ -1,174 +0,0 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at: -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -*** Settings *** -| Resource | resources/libraries/robot/performance/performance_setup.robot -| Library | resources.libraries.python.Cop -| ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC -| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | COPWHLIST -| ... -| Suite Setup | Set up 3-node performance topology with DUT's NIC model -| ... | L3 | Intel-X520-DA2 -| Suite Teardown | Tear down 3-node performance topology -| ... -| Test Setup | Set up performance test -| ... -| Test Teardown | Tear down performance discovery test | ${min_rate}pps -| ... | ${framesize} | ${traffic_profile} -| ... -| Test Template | Local template -| ... -| Documentation | *RFC2544: Pkt throughput IPv4 whitelist test cases* -| ... -| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology -| ... | with single links between nodes. -| ... | *[Enc] Packet Encapsulations:* Eth-IPv4 for IPv4 routing. -| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 -| ... | routing, two static IPv4 /24 routes and IPv4 COP security whitelist -| ... | ingress /24 filter entries applied on links TG - DUT1 and DUT2 - TG. -| ... | DUT1 and DUT2 tested with 2p10GE NIC X520 Niantic by Intel. -| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop -| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop -| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage -| ... | of packets transmitted. NDR and PDR are discovered for different -| ... | Ethernet L2 frame sizes using either binary search or linear search -| ... | algorithms with configured starting rate and final step that determines -| ... | throughput measurement resolution. Test packets are generated by TG on -| ... | links to DUTs. TG traffic profile contains two L3 flow-groups -| ... | (flow-group per direction, 253 flows per flow-group) with all packets -| ... | containing Ethernet header, IPv4 header with IP protocol=61 and static -| ... | payload. MAC addresses are matching MAC addresses of the TG node -| ... | interfaces. -| ... | *[Ref] Applicable standard specifications:* RFC2544. - -*** Variables *** -# X520-DA2 bandwidth limit -| ${s_limit} | ${10000000000} -# Traffic profile: -| ${traffic_profile} | trex-sl-3n-ethip4-ip4src253 - -*** Keywords *** -| Local template -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with ${wt} thread(s), ${wt}\ -| | ... | phy core(s), ${rxq} receive queue(s) per NIC port. -| | ... | [Ver] Find NDR or PDR for ${framesize} frames using binary search\ -| | ... | start at 10GE linerate. -| | ... -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} -| | ... -| | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${get_framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} -| | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | When Initialize IPv4 forwarding in 3-node circular topology -| | And Add Fib Table | ${dut1} | 1 -| | And Vpp Route Add | ${dut1} | 10.10.10.0 | 24 |Â vrf=1 | local=${TRUE} -| | And Add Fib Table | ${dut2} | 1 -| | And Vpp Route Add | ${dut2} | 20.20.20.0 | 24 |Â vrf=1 | local=${TRUE} -| | And COP Add whitelist Entry | ${dut1} | ${dut1_if1} | ip4 | 1 -| | And COP Add whitelist Entry | ${dut2} | ${dut2_if2} | ip4 | 1 -| | And COP interface enable or disable | ${dut1} | ${dut1_if1} | enable -| | And COP interface enable or disable | ${dut2} | ${dut2_if2} | enable -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} - -*** Test Cases *** -| tc01-64B-1t1c-ethip4-ip4base-copwhtlistbase-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR - -| tc02-64B-1t1c-ethip4-ip4base-copwhtlistbase-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR - -| tc03-1518B-1t1c-ethip4-ip4base-copwhtlistbase-ndrdisc -| | [Tags] | 1518B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${1518} | search_type=NDR - -| tc04-1518B-1t1c-ethip4-ip4base-copwhtlistbase-pdrdisc -| | [Tags] | 1518B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${1518} | search_type=PDR - -| tc05-9000B-1t1c-ethip4-ip4base-copwhtlistbase-ndrdisc -| | [Tags] | 9000B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${9000} | search_type=NDR - -| tc06-9000B-1t1c-ethip4-ip4base-copwhtlistbase-pdrdisc -| | [Tags] | 9000B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${9000} | search_type=PDR - -| tc07-64B-2t2c-ethip4-ip4base-copwhtlistbase-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR - -| tc08-64B-2t2c-ethip4-ip4base-copwhtlistbase-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR - -| tc09-1518B-2t2c-ethip4-ip4base-copwhtlistbase-ndrdisc -| | [Tags] | 1518B | 2C | NDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${1518} | search_type=NDR - -| tc10-1518B-2t2c-ethip4-ip4base-copwhtlistbase-pdrdisc -| | [Tags] | 1518B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${1518} | search_type=PDR - -| tc11-9000B-2t2c-ethip4-ip4base-copwhtlistbase-ndrdisc -| | [Tags] | 9000B | 2C | NDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${9000} | search_type=NDR - -| tc12-9000B-2t2c-ethip4-ip4base-copwhtlistbase-pdrdisc -| | [Tags] | 9000B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${9000} | search_type=PDR - -| tc13-64B-4t4c-ethip4-ip4base-copwhtlistbase-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR - -| tc14-64B-4t4c-ethip4-ip4base-copwhtlistbase-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR - -| tc15-1518B-4t4c-ethip4-ip4base-copwhtlistbase-ndrdisc -| | [Tags] | 1518B | 4C | NDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${1518} | search_type=NDR - -| tc16-1518B-4t4c-ethip4-ip4base-copwhtlistbase-pdrdisc -| | [Tags] | 1518B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${1518} | search_type=PDR - -| tc17-9000B-4t4c-ethip4-ip4base-copwhtlistbase-ndrdisc -| | [Tags] | 9000B | 4C | NDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${9000} | search_type=NDR - -| tc18-9000B-4t4c-ethip4-ip4base-copwhtlistbase-pdrdisc -| | [Tags] | 9000B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${9000} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4base-iacldstbase-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4base-iacldstbase-mrr.robot index 7253731de1..fcaf232b72 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4base-iacldstbase-mrr.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4base-iacldstbase-mrr.robot @@ -25,7 +25,7 @@ | ... | Test Teardown | Tear down performance mrr test | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *Raw results IPv4 iAcl whitelist test cases* | ... @@ -52,7 +52,7 @@ | ${traffic_profile}= | trex-sl-3n-ethip4-ip4src253 *** Keywords *** -| Local template +| Local Template | | [Documentation] | | ... | [Cfg] DUT runs IPv4 routing config. | | ... | Each DUT uses ${phy_cores} physical core(s) for worker threads. @@ -65,16 +65,12 @@ | | ... | - phy_cores - Number of physical cores. Type: integer | | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer | | ... -| | [Arguments] | ${phy_cores} | ${framesize} | ${rxq}=${None} -| | ... -| | Set Test Variable | ${framesize} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${get_framesize} +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | When Initialize IPv4 forwarding in 3-node circular topology | | ${table_idx} | ${skip_n} | ${match_n}= | And Vpp Creates Classify Table L3 @@ -95,50 +91,50 @@ | | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} *** Test Cases *** -| tc01-64B-1t1c-ethip4-ip4base-iacldstbase-mrr +| tc01-64B-1c-ethip4-ip4base-iacldstbase-mrr | | [Tags] | 64B | 1C -| | phy_cores=${1} | framesize=${64} - -| tc02-1518B-1t1c-ethip4-ip4base-iacldstbase-mrr -| | [Tags] | 1518B | 1C -| | phy_cores=${1} | framesize=${1518} +| | framesize=${64} | phy_cores=${1} -| tc03-9000B-1t1c-ethip4-ip4base-iacldstbase-mrr -| | [Tags] | 9000B | 1CC -| | phy_cores=${1} | framesize=${9000} +| tc02-64B-2c-ethip4-ip4base-iacldstbase-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} -| tc04-IMIX-1t1c-ethip4-ip4base-iacldstbase-mrr -| | [Tags] | IMIX | 1C -| | phy_cores=${1} | framesize=IMIX_v4_1 +| tc03-64B-4c-ethip4-ip4base-iacldstbase-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} -| tc05-64B-2t2c-ethip4-ip4base-iacldstbase-mrr -| | [Tags] | 64B | 2C -| | phy_cores=${2} | framesize=${64} +| tc04-1518B-1c-ethip4-ip4base-iacldstbase-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} -| tc06-1518B-2t2c-ethip4-ip4base-iacldstbase-mrr +| tc05-1518B-2c-ethip4-ip4base-iacldstbase-mrr | | [Tags] | 1518B | 2C -| | phy_cores=${2} | framesize=${1518} - -| tc07-9000B-2t2c-ethip4-ip4base-iacldstbase-mrr -| | [Tags] | 9000B | 2C -| | phy_cores=${2} | framesize=${9000} +| | framesize=${1518} | phy_cores=${2} -| tc08-IMIX-2t2c-ethip4-ip4base-iacldstbase-mrr -| | [Tags] | IMIX | 2C -| | phy_cores=${2} | framesize=IMIX_v4_1 +| tc06-1518B-4c-ethip4-ip4base-iacldstbase-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} -| tc09-64B-4t4c-ethip4-ip4base-iacldstbase-mrr -| | [Tags] | 64B | 4C -| | phy_cores=${4} | framesize=${64} +| tc07-9000B-1c-ethip4-ip4base-iacldstbase-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc10-1518B-4t4c-ethip4-ip4base-iacldstbase-mrr -| | [Tags] | 1518B | 4C -| | phy_cores=${4} | framesize=${1518} +| tc08-9000B-2c-ethip4-ip4base-iacldstbase-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc11-9000B-4t4c-ethip4-ip4base-iacldstbase-mrr +| tc09-9000B-4c-ethip4-ip4base-iacldstbase-mrr | | [Tags] | 9000B | 4C -| | phy_cores=${4} | framesize=${9000} +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4-ip4base-iacldstbase-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4-ip4base-iacldstbase-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc12-IMIX-4t4c-ethip4-ip4base-iacldstbase-mrr +| tc12-IMIX-4c-ethip4-ip4base-iacldstbase-mrr | | [Tags] | IMIX | 4C -| | phy_cores=${4} | framesize=IMIX_v4_1 +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4base-iacldstbase-ndrpdr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4base-iacldstbase-ndrpdr.robot new file mode 100644 index 0000000000..e9192ae974 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4base-iacldstbase-ndrpdr.robot @@ -0,0 +1,146 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | IACLDST +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down performance discovery test | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Test Template | Local Template +| ... +| Documentation | *RFC2544: Pkt throughput IPv4 iAcl whitelist test cases* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4 for IPv4 routing. +| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 +| ... | routing, two static IPv4 /24 routes and IPv4 iAcl security whitelist +| ... | ingress /24 filter entries applied on links TG - DUT1 and DUT2 - TG. +| ... | DUT1 and DUT2 tested with 2p10GE NIC X520 Niantic by Intel. +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage +| ... | of packets transmitted. NDR and PDR are discovered for different +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on links to DUTs. TG traffic profile +| ... | contains two L3 flow-groups (flow-group per direction, 253 flows per +| ... | flow-group) with all packets containing Ethernet header, IPv6 header +| ... | and generated payload. MAC addresses are matching MAC addresses +| ... | of the TG node interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} +# Traffic profile: +| ${traffic_profile}= | trex-sl-3n-ethip4-ip4src253 + +*** Keywords *** +| Local Template +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config. +| | ... | Each DUT uses ${phy_cores} physical core(s) for worker threads. +| | ... | [Ver] Measure NDR and PDR values using MLRsearch algorithm.\ +| | ... +| | ... | *Arguments:* +| | ... | - framesize - Framesize in Bytes in integer or string (IMIX_v4_1). +| | ... | Type: integer, string +| | ... | - phy_cores - Number of physical cores. Type: integer +| | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer +| | ... +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | When Initialize IPv4 forwarding in 3-node circular topology +| | ${table_idx} | ${skip_n} | ${match_n}= | And Vpp Creates Classify Table L3 +| | ... | ${dut1} | ip4 | dst +| | And Vpp Configures Classify Session L3 +| | ... | ${dut1} | permit | ${table_idx} | ${skip_n} | ${match_n} +| | ... | ip4 | dst | 20.20.20.2 +| | And Vpp Enable Input Acl Interface +| | ... | ${dut1} | ${dut1_if1} | ip4 | ${table_idx} +| | ${table_idx} | ${skip_n} | ${match_n}= | And Vpp Creates Classify Table L3 +| | ... | ${dut2} | ip4 | dst +| | And Vpp Configures Classify Session L3 +| | ... | ${dut2} | permit | ${table_idx} | ${skip_n} | ${match_n} +| | ... | ip4 | dst | 10.10.10.2 +| | And Vpp Enable Input Acl Interface +| | ... | ${dut2} | ${dut2_if2} | ip4 | ${table_idx} +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} + +*** Test Cases *** +| tc01-64B-1c-ethip4-ip4base-iacldstbase-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4-ip4base-iacldstbase-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4-ip4base-iacldstbase-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4-ip4base-iacldstbase-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4-ip4base-iacldstbase-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4-ip4base-iacldstbase-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4-ip4base-iacldstbase-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4-ip4base-iacldstbase-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4-ip4base-iacldstbase-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4-ip4base-iacldstbase-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4-ip4base-iacldstbase-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4-ip4base-iacldstbase-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4base-iacldstbase-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4base-iacldstbase-ndrpdrdisc.robot deleted file mode 100644 index 93a67b677c..0000000000 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4base-iacldstbase-ndrpdrdisc.robot +++ /dev/null @@ -1,180 +0,0 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at: -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -*** Settings *** -| Resource | resources/libraries/robot/performance/performance_setup.robot -| ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC -| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | IACLDST -| ... -| Suite Setup | Set up 3-node performance topology with DUT's NIC model -| ... | L3 | Intel-X520-DA2 -| Suite Teardown | Tear down 3-node performance topology -| ... -| Test Setup | Set up performance test -| ... -| Test Teardown | Tear down performance discovery test | ${min_rate}pps -| ... | ${framesize} | ${traffic_profile} -| ... -| Test Template | Local template -| ... -| Documentation | *RFC2544: Pkt throughput IPv4 iAcl whitelist test cases* -| ... -| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology -| ... | with single links between nodes. -| ... | *[Enc] Packet Encapsulations:* Eth-IPv4 for IPv4 routing. -| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 -| ... | routing, two static IPv4 /24 routes and IPv4 iAcl security whitelist -| ... | ingress /24 filter entries applied on links TG - DUT1 and DUT2 - TG. -| ... | DUT1 and DUT2 tested with 2p10GE NIC X520 Niantic by Intel. -| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop -| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop -| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage -| ... | of packets transmitted. NDR and PDR are discovered for different -| ... | Ethernet L2 frame sizes using either binary search or linear search -| ... | algorithms with configured starting rate and final step that determines -| ... | throughput measurement resolution. Test packets are generated by TG on -| ... | links to DUTs. TG traffic profile contains two L3 flow-groups -| ... | (flow-group per direction, 253 flows per flow-group) with all packets -| ... | containing Ethernet header, IPv4 header with IP protocol=61 and -| ... | static payload. MAC addresses are matching MAC addresses of the TG -| ... | node interfaces. -| ... | *[Ref] Applicable standard specifications:* RFC2544. - -*** Variables *** -# X520-DA2 bandwidth limit -| ${s_limit} | ${10000000000} -# Traffic profile: -| ${traffic_profile} | trex-sl-3n-ethip4-ip4src253 - -*** Keywords *** -| Local template -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with ${wt} thread(s), ${wt}\ -| | ... | phy core(s), ${rxq} receive queue(s) per NIC port. -| | ... | [Ver] Find NDR or PDR for ${framesize} frames using binary search\ -| | ... | start at 10GE linerate. -| | ... -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} -| | ... -| | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${get_framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} -| | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | When Initialize IPv4 forwarding in 3-node circular topology -| | ${table_idx} | ${skip_n} | ${match_n}= | And Vpp Creates Classify Table L3 -| | ... | ${dut1} | ip4 | dst -| | And Vpp Configures Classify Session L3 -| | ... | ${dut1} | permit | ${table_idx} | ${skip_n} | ${match_n} -| | ... | ip4 | dst | 20.20.20.2 -| | And Vpp Enable Input Acl Interface -| | ... | ${dut1} | ${dut1_if1} | ip4 | ${table_idx} -| | ${table_idx} | ${skip_n} | ${match_n}= | And Vpp Creates Classify Table L3 -| | ... | ${dut2} | ip4 | dst -| | And Vpp Configures Classify Session L3 -| | ... | ${dut2} | permit | ${table_idx} | ${skip_n} | ${match_n} -| | ... | ip4 | dst | 10.10.10.2 -| | And Vpp Enable Input Acl Interface -| | ... | ${dut2} | ${dut2_if2} | ip4 | ${table_idx} -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} - -*** Test Cases *** -| tc01-64B-1t1c-ethip4-ip4base-iacldstbase-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR - -| tc02-64B-1t1c-ethip4-ip4base-iacldstbase-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR - -| tc03-1518B-1t1c-ethip4-ip4base-iacldstbase-ndrdisc -| | [Tags] | 1518B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${1518} | search_type=NDR - -| tc04-1518B-1t1c-ethip4-ip4base-iacldstbase-pdrdisc -| | [Tags] | 1518B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${1518} | search_type=PDR - -| tc05-9000B-1t1c-ethip4-ip4base-iacldstbase-ndrdisc -| | [Tags] | 9000B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${9000} | search_type=NDR - -| tc06-9000B-1t1c-ethip4-ip4base-iacldstbase-pdrdisc -| | [Tags] | 9000B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${9000} | search_type=PDR - -| tc07-64B-2t2c-ethip4-ip4base-iacldstbase-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR - -| tc08-64B-2t2c-ethip4-ip4base-iacldstbase-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR - -| tc09-1518B-2t2c-ethip4-ip4base-iacldstbase-ndrdisc -| | [Tags] | 1518B | 2C | NDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${1518} | search_type=NDR - -| tc10-1518B-2t2c-ethip4-ip4base-iacldstbase-pdrdisc -| | [Tags] | 1518B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${1518} | search_type=PDR - -| tc11-9000B-2t2c-ethip4-ip4base-iacldstbase-ndrdisc -| | [Tags] | 9000B | 2C | NDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${9000} | search_type=NDR - -| tc12-9000B-2t2c-ethip4-ip4base-iacldstbase-pdrdisc -| | [Tags] | 9000B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${9000} | search_type=PDR - -| tc13-64B-4t4c-ethip4-ip4base-iacldstbase-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR - -| tc14-64B-4t4c-ethip4-ip4base-iacldstbase-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR - -| tc15-1518B-4t4c-ethip4-ip4base-iacldstbase-ndrdisc -| | [Tags] | 1518B | 4C | NDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${1518} | search_type=NDR - -| tc16-1518B-4t4c-ethip4-ip4base-iacldstbase-pdrdisc -| | [Tags] | 1518B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${1518} | search_type=PDR - -| tc17-9000B-4t4c-ethip4-ip4base-iacldstbase-ndrdisc -| | [Tags] | 9000B | 4C | NDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${9000} | search_type=NDR - -| tc18-9000B-4t4c-ethip4-ip4base-iacldstbase-pdrdisc -| | [Tags] | 9000B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${9000} | search_type=PDR - diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4base-ipolicemarkbase-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4base-ipolicemarkbase-mrr.robot index 4bb4af4be1..5955c8b8e6 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4base-ipolicemarkbase-mrr.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4base-ipolicemarkbase-mrr.robot @@ -26,7 +26,7 @@ | ... | Test Teardown | Tear down performance mrr test | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *Raw results IPv4 policer test cases* | ... @@ -55,7 +55,7 @@ | ${traffic_profile}= | trex-sl-3n-ethip4-ip4src253 *** Keywords *** -| Local template +| Local Template | | ... | | [Documentation] | | ... | [Cfg] DUT runs IPv4 routing config. @@ -69,22 +69,15 @@ | | ... | - phy_cores - Number of physical cores. Type: integer | | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer | | ... -| | [Arguments] | ${phy_cores} | ${framesize} | ${rxq}=${None} +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... -| | Set Test Variable | ${framesize} -| | ${cb}= | Set Variable If -| | ... | '${framesize}' == 'IMIX_v4_1' | ${1518} | ${framesize} -| | Set Test Variable | ${cb} -| | ${eb}= | Set Variable If -| | ... | '${framesize}' == 'IMIX_v4_1' | ${1518} | ${framesize} -| | Set Test Variable | ${eb} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${get_framesize} +| | Set Test Variable | ${cb} | ${framesize} +| | Set Test Variable | ${eb} | ${framesize} | | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | When Initialize IPv4 forwarding in 3-node circular topology | | And Initialize IPv4 policer 2r3c-'ca' in 3-node circular topology @@ -92,50 +85,50 @@ | | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} *** Test Cases *** -| tc01-64B-1t1c-ethip4-ip4base-ipolicemarkbase-mrr +| tc01-64B-1c-ethip4-ip4base-ipolicemarkbase-mrr | | [Tags] | 64B | 1C -| | phy_cores=${1} | framesize=${64} +| | framesize=${64} | phy_cores=${1} -| tc02-1518B-1t1c-ethip4-ip4base-ipolicemarkbase-mrr -| | [Tags] | 1518B | 1C -| | phy_cores=${1} | framesize=${1518} - -| tc03-9000B-1t1c-ethip4-ip4base-ipolicemarkbase-mrr -| | [Tags] | 9000B | 1C -| | phy_cores=${1} | framesize=${9000} +| tc02-64B-2c-ethip4-ip4base-ipolicemarkbase-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} -| tc04-IMIX-1t1c-ethip4-ip4base-ipolicemarkbase-mrr -| | [Tags] | IMIX | 1C -| | phy_cores=${1} | framesize=IMIX_v4_1 +| tc03-64B-4c-ethip4-ip4base-ipolicemarkbase-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} -| tc05-64B-2t2c-ethip4-ip4base-ipolicemarkbase-mrr -| | [Tags] | 64B | 2C -| | phy_cores=${2} | framesize=${64} +| tc04-1518B-1c-ethip4-ip4base-ipolicemarkbase-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} -| tc06-1518B-2t2c-ethip4-ip4base-ipolicemarkbase-mrr +| tc05-1518B-2c-ethip4-ip4base-ipolicemarkbase-mrr | | [Tags] | 1518B | 2C -| | phy_cores=${2} | framesize=${1518} - -| tc07-9000B-2t2c-ethip4-ip4base-ipolicemarkbase-mrr -| | [Tags] | 9000B | 2C -| | phy_cores=${2} | framesize=${9000} +| | framesize=${1518} | phy_cores=${2} -| tc08-IMIX-2t2c-ethip4-ip4base-ipolicemarkbase-mrr -| | [Tags] | IMIX | 2C -| | phy_cores=${2} | framesize=IMIX_v4_1 +| tc06-1518B-4c-ethip4-ip4base-ipolicemarkbase-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} -| tc09-64B-4t4c-ethip4-ip4base-ipolicemarkbase-mrr -| | [Tags] | 64B | 4C -| | phy_cores=${4} | framesize=${64} +| tc07-9000B-1c-ethip4-ip4base-ipolicemarkbase-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc10-1518B-4t4c-ethip4-ip4base-ipolicemarkbase-mrr -| | [Tags] | 1518B | 4C -| | phy_cores=${4} | framesize=${1518} +| tc08-9000B-2c-ethip4-ip4base-ipolicemarkbase-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc11-9000B-4t4c-ethip4-ip4base-ipolicemarkbase-mrr +| tc09-9000B-4c-ethip4-ip4base-ipolicemarkbase-mrr | | [Tags] | 9000B | 4C -| | phy_cores=${4} | framesize=${9000} +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4-ip4base-ipolicemarkbase-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4-ip4base-ipolicemarkbase-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc12-IMIX-4t4c-ethip4-ip4base-ipolicemarkbase-mrr +| tc12-IMIX-4c-ethip4-ip4base-ipolicemarkbase-mrr | | [Tags] | IMIX | 4C -| | phy_cores=${4} | framesize=IMIX_v4_1 +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4base-ipolicemarkbase-ndrpdr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4base-ipolicemarkbase-ndrpdr.robot new file mode 100644 index 0000000000..4bcfd8e78c --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4base-ipolicemarkbase-ndrpdr.robot @@ -0,0 +1,139 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| Library | resources.libraries.python.Policer +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR +| ... | NIC_Intel-X520-DA2 | IP4FWD | BASE | POLICE_MARK +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down performance discovery test | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Test Template | Local Template +| ... +| Documentation | *RFC2544: Pkt throughput IPv4 policer test cases* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4 for IPv4 routing. +| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 +| ... | routing and two static IPv4 /24 route entries and policer with 2R3C +| ... | color-aware configuration. Policer is applied on links TG - DUT1 and +| ... | DUT2 - TG. DUT1 and DUT2 tested with 2p10GE NIC X520 Niantic by Intel. +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage +| ... | of packets transmitted. NDR and PDR are discovered for different +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on links to DUTs. TG traffic profile +| ... | contains two L3 flow-groups (flow-group per direction, 253 flows per +| ... | flow-group) with all packets containing Ethernet header, IPv6 header +| ... | and generated payload. MAC addresses are matching MAC addresses +| ... | of the TG node interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544, RFC2698. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} +| ${cir}= | ${100} +| ${eir}= | ${150} +# Traffic profile: +| ${traffic_profile}= | trex-sl-3n-ethip4-ip4src253 + +*** Keywords *** +| Local Template +| | ... +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config. +| | ... | Each DUT uses ${phy_cores} physical core(s) for worker threads. +| | ... | [Ver] Measure NDR and PDR values using MLRsearch algorithm.\ +| | ... +| | ... | *Arguments:* +| | ... | - framesize - Framesize in Bytes in integer or string (IMIX_v4_1). +| | ... | Type: integer, string +| | ... | - phy_cores - Number of physical cores. Type: integer +| | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer +| | ... +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} | ${10000} +| | Set Test Variable | ${cb} | ${framesize} +| | Set Test Variable | ${eb} | ${framesize} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | When Initialize IPv4 forwarding in 3-node circular topology +| | And Initialize IPv4 policer 2r3c-'ca' in 3-node circular topology +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} + +*** Test Cases *** +| tc01-64B-1c-ethip4-ip4base-ipolicemarkbase-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4-ip4base-ipolicemarkbase-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4-ip4base-ipolicemarkbase-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4-ip4base-ipolicemarkbase-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4-ip4base-ipolicemarkbase-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4-ip4base-ipolicemarkbase-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4-ip4base-ipolicemarkbase-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4-ip4base-ipolicemarkbase-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4-ip4base-ipolicemarkbase-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4-ip4base-ipolicemarkbase-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4-ip4base-ipolicemarkbase-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4-ip4base-ipolicemarkbase-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4base-ipolicemarkbase-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4base-ipolicemarkbase-ndrpdrdisc.robot deleted file mode 100644 index 1236a95be5..0000000000 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4base-ipolicemarkbase-ndrpdrdisc.robot +++ /dev/null @@ -1,171 +0,0 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at: -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -*** Settings *** -| Resource | resources/libraries/robot/performance/performance_setup.robot -| Library | resources.libraries.python.Policer -| ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC -| ... | NIC_Intel-X520-DA2 | IP4FWD | BASE | POLICE_MARK -| ... -| Suite Setup | Set up 3-node performance topology with DUT's NIC model -| ... | L3 | Intel-X520-DA2 -| Suite Teardown | Tear down 3-node performance topology -| ... -| Test Setup | Set up performance test -| ... -| Test Teardown | Tear down performance discovery test | ${min_rate}pps -| ... | ${framesize} | ${traffic_profile} -| ... -| Test Template | Local template -| ... -| Documentation | *RFC2544: Pkt throughput IPv4 policer test cases* -| ... -| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology -| ... | with single links between nodes. -| ... | *[Enc] Packet Encapsulations:* Eth-IPv4 for IPv4 routing. -| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 -| ... | routing and two static IPv4 /24 route entries and policer with 2R3C -| ... | color-aware configuration. Policer is applied on links TG - DUT1 and -| ... | DUT2 - TG. DUT1 and DUT2 tested with 2p10GE NIC X520 Niantic by Intel. -| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop -| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop -| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage -| ... | of packets transmitted. NDR and PDR are discovered for different -| ... | Ethernet L2 frame sizes using either binary search or linear search -| ... | algorithms with configured starting rate and final step that determines -| ... | throughput measurement resolution. Test packets are generated by TG on -| ... | links to DUTs. TG traffic profile contains two L3 flow-groups -| ... | (flow-group per direction, 253 flows per flow-group) with all packets -| ... | containing Ethernet header, IPv4 header with IP protocol=61 and -| ... | generated payload. MAC addresses are matching MAC addresses of the TG -| ... | node interfaces. -| ... | *[Ref] Applicable standard specifications:* RFC2544, RFC2698. - -*** Variables *** -# X520-DA2 bandwidth limit -| ${s_limit} | ${10000000000} -| ${cir}= | ${100} -| ${eir}= | ${150} -# Traffic profile: -| ${traffic_profile} | trex-sl-3n-ethip4-ip4src253 - -*** Keywords *** -| Local template -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with ${wt} thread(s), ${wt}\ -| | ... | phy core(s), ${rxq} receive queue(s) per NIC port. -| | ... | [Ver] Find NDR or PDR for ${framesize} frames using binary search\ -| | ... | start at 10GE linerate. -| | ... -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} -| | ... -| | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | Set Test Variable | ${cb} | ${framesize} -| | Set Test Variable | ${eb} | ${framesize} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${get_framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} -| | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | When Initialize IPv4 forwarding in 3-node circular topology -| | And Initialize IPv4 policer 2r3c-'ca' in 3-node circular topology -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} - -*** Test Cases *** -| tc01-64B-1t1c-ethip4-ip4base-ipolicemarkbase-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR - -| tc02-64B-1t1c-ethip4-ip4base-ipolicemarkbase-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR - -| tc03-1518B-1t1c-ethip4-ip4base-ipolicemarkbase-ndrdisc -| | [Tags] | 1518B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${1518} | search_type=NDR - -| tc04-1518B-1t1c-ethip4-ip4base-ipolicemarkbase-pdrdisc -| | [Tags] | 1518B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${1518} | search_type=PDR - -| tc05-9000B-1t1c-ethip4-ip4base-ipolicemarkbase-ndrdisc -| | [Tags] | 9000B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${9000} | search_type=NDR - -| tc06-9000B-1t1c-ethip4-ip4base-ipolicemarkbase-pdrdisc -| | [Tags] | 9000B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${9000} | search_type=PDR - -| tc07-64B-2t2c-ethip4-ip4base-ipolicemarkbase-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR - -| tc08-64B-2t2c-ethip4-ip4base-ipolicemarkbase-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR - -| tc09-1518B-2t2c-ethip4-ip4base-ipolicemarkbase-ndrdisc -| | [Tags] | 1518B | 2C | NDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${1518} | search_type=NDR - -| tc10-1518B-2t2c-ethip4-ip4base-ipolicemarkbase-pdrdisc -| | [Tags] | 1518B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${1518} | search_type=PDR - -| tc11-9000B-2t2c-ethip4-ip4base-ipolicemarkbase-ndrdisc -| | [Tags] | 9000B | 2C | NDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${9000} | search_type=NDR - -| tc12-9000B-2t2c-ethip4-ip4base-ipolicemarkbase-pdrdisc -| | [Tags] | 9000B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${9000} | search_type=PDR - -| tc13-64B-4t4c-ethip4-ip4base-ipolicemarkbase-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR - -| tc14-64B-4t4c-ethip4-ip4base-ipolicemarkbase-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR - -| tc15-1518B-4t4c-ethip4-ip4base-ipolicemarkbase-ndrdisc -| | [Tags] | 1518B | 4C | NDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${1518} | search_type=NDR - -| tc16-1518B-4t4c-ethip4-ip4base-ipolicemarkbase-pdrdisc -| | [Tags] | 1518B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${1518} | search_type=PDR - -| tc17-9000B-4t4c-ethip4-ip4base-ipolicemarkbase-ndrdisc -| | [Tags] | 9000B | 4C | NDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${9000} | search_type=NDR - -| tc18-9000B-4t4c-ethip4-ip4base-ipolicemarkbase-pdrdisc -| | [Tags] | 9000B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${9000} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4base-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4base-mrr.robot index 17c3e77a04..b763d9d9cb 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4base-mrr.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4base-mrr.robot @@ -24,7 +24,7 @@ | Test Setup | Set up performance test | Test Teardown | Tear down performance mrr test | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *Raw results IPv4 routing test cases* | ... @@ -50,7 +50,7 @@ | ${traffic_profile}= | trex-sl-3n-ethip4-ip4src253 *** Keywords *** -| Local template +| Local Template | | [Documentation] | | ... | [Cfg] DUT runs IPv4 routing config. | | ... | Each DUT uses ${phy_cores} physical core(s) for worker threads. @@ -63,66 +63,62 @@ | | ... | - phy_cores - Number of physical cores. Type: integer | | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer | | ... -| | [Arguments] | ${phy_cores} | ${framesize} | ${rxq}=${None} -| | ... -| | Set Test Variable | ${framesize} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | When Initialize IPv4 forwarding in 3-node circular topology | | Then Traffic should pass with maximum rate | | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} *** Test Cases *** -| tc01-64B-1t1c-ethip4-ip4base-mrr +| tc01-64B-1c-ethip4-ip4base-mrr | | [Tags] | 64B | 1C -| | phy_cores=${1} | framesize=${64} - -| tc02-1518B-1t1c-ethip4-ip4base-mrr -| | [Tags] | 1518B | 1C -| | phy_cores=${1} | framesize=${1518} +| | framesize=${64} | phy_cores=${1} -| tc03-9000B-1t1c-ethip4-ip4base-mrr -| | [Tags] | 9000B | 1C -| | phy_cores=${1} | framesize=${9000} +| tc02-64B-2c-ethip4-ip4base-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} -| tc04-IMIX-1t1c-ethip4-ip4base-mrr -| | [Tags] | IMIX | 1C -| | phy_cores=${1} | framesize=IMIX_v4_1 +| tc03-64B-4c-ethip4-ip4base-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} -| tc05-64B-2t2c-ethip4-ip4base-mrr -| | [Tags] | 64B | 2C -| | phy_cores=${2} | framesize=${64} +| tc04-1518B-1c-ethip4-ip4base-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} -| tc06-1518B-2t2c-ethip4-ip4base-mrr +| tc05-1518B-2c-ethip4-ip4base-mrr | | [Tags] | 1518B | 2C -| | phy_cores=${2} | framesize=${1518} - -| tc07-9000B-2t2c-ethip4-ip4base-mrr -| | [Tags] | 9000B | 2C -| | phy_cores=${2} | framesize=${9000} +| | framesize=${1518} | phy_cores=${2} -| tc08-IMIX-2t2c-ethip4-ip4base-mrr -| | [Tags] | IMIX | 2C -| | phy_cores=${2} | framesize=IMIX_v4_1 +| tc06-1518B-4c-ethip4-ip4base-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} -| tc09-64B-4t4c-ethip4-ip4base-mrr -| | [Tags] | 64B | 4C -| | phy_cores=${4} | framesize=${64} +| tc07-9000B-1c-ethip4-ip4base-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc10-1518B-4t4c-ethip4-ip4base-mrr -| | [Tags] | 1518B | 4C -| | phy_cores=${4} | framesize=${1518} +| tc08-9000B-2c-ethip4-ip4base-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc11-9000B-4t4c-ethip4-ip4base-mrr +| tc09-9000B-4c-ethip4-ip4base-mrr | | [Tags] | 9000B | 4C -| | phy_cores=${4} | framesize=${9000} +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4-ip4base-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4-ip4base-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc12-IMIX-4t4c-ethip4-ip4base-mrr +| tc12-IMIX-4c-ethip4-ip4base-mrr | | [Tags] | IMIX | 4C -| | phy_cores=${4} | framesize=IMIX_v4_1 +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4base-ndrpdr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4base-ndrpdr.robot index c58515121e..a6579e0fb2 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4base-ndrpdr.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4base-ndrpdr.robot @@ -12,28 +12,6 @@ # limitations under the License. *** Settings *** -| Documentation | *RFC2544: Pkt throughput IPv4 routing test cases* -| ... -| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology -| ... | with single links between nodes. -| ... | *[Enc] Packet Encapsulations:* Eth-IPv4 for IPv4 routing. -| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 -| ... | routing and two static IPv4 /24 route entries. DUT1 and DUT2 tested with -| ... | 2p10GE NIC X520 Niantic by Intel. -| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop -| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop -| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage -| ... | of packets transmitted. NDR and PDR are discovered for different -| ... | Ethernet L2 frame sizes using either binary search or linear search -| ... | algorithms with configured starting rate and final step that determines -| ... | throughput measurement resolution. Test packets are generated by TG on -| ... | links to DUTs. TG traffic profile contains two L3 flow-groups -| ... | (flow-group per direction, 253 flows per flow-group) with all packets -| ... | containing Ethernet header, IPv4 header with IP protocol=61 and static -| ... | payload. MAC addresses are matching MAC addresses of the TG node -| ... | interfaces. -| ... | *[Ref] Applicable standard specifications:* RFC2544. -| ... | Resource | resources/libraries/robot/performance/performance_setup.robot | ... | Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR @@ -44,87 +22,109 @@ | Suite Teardown | Tear down 3-node performance topology | ... | Test Setup | Set up performance test -| ... | Test Teardown | Tear down performance discovery test | ${min_rate}pps | ... | ${framesize} | ${traffic_profile} | ... -| Test Template | Local template +| Test Template | Local Template +| ... +| Documentation | *RFC2544: Pkt throughput IPv4 routing test cases* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4 for IPv4 routing. +| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 +| ... | routing and two static IPv4 /24 route entries. DUT1 and DUT2 tested with +| ... | 2p10GE NIC X520 Niantic by Intel. +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage +| ... | of packets transmitted. NDR and PDR are discovered for different +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on links to DUTs. TG traffic profile +| ... | contains two L3 flow-groups (flow-group per direction, 253 flows per +| ... | flow-group) with all packets containing Ethernet header, IPv4 header +| ... | with IP protocol=61 and static payload. MAC addresses are matching MAC +| ... | addresses of the TG node interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. *** Variables *** # X520-DA2 bandwidth limit -| ${s_limit} | ${10000000000} +| ${s_limit}= | ${10000000000} # Traffic profile: -| ${traffic_profile} | trex-sl-3n-ethip4-ip4src253 +| ${traffic_profile}= | trex-sl-3n-ethip4-ip4src253 *** Keywords *** -| Local template +| Local Template | | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with ${wt} thread(s), ${wt}\ -| | ... | phy core(s), ${rxq} receive queue(s) per NIC port. -| | ... | [Ver] Find NDR and PDR for ${framesize} frames using optimized search\ -| | ... | start at 10GE linerate. +| | ... | [Cfg] DUT runs IPv4 routing config. +| | ... | Each DUT uses ${phy_cores} physical core(s) for worker threads. +| | ... | [Ver] Measure NDR and PDR values using MLRsearch algorithm.\ +| | ... +| | ... | *Arguments:* +| | ... | - framesize - Framesize in Bytes in integer or string (IMIX_v4_1). +| | ... | Type: integer, string +| | ... | - phy_cores - Number of physical cores. Type: integer +| | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer | | ... -| | [Arguments] | ${phy_cores} | ${framesize} | ${rxq}=${None} +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} | ${20000} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ${max_uniditrectional_rate}= | Calculate pps | ${s_limit} | ${get_framesize} -| | ${max_rate}= | Evaluate | 2*${max_uniditrectional_rate} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding in 3-node circular topology +| | When Initialize IPv4 forwarding in 3-node circular topology | | Then Find NDR and PDR intervals using optimized search | | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} *** Test Cases *** -| tc01-64B-1t1c-ethip4-ip4base-ndrpdr +| tc01-64B-1c-ethip4-ip4base-ndrpdr | | [Tags] | 64B | 1C -| | phy_cores=${1} | framesize=${64} +| | framesize=${64} | phy_cores=${1} -| tc02-1518B-1t1c-ethip4-ip4base-ndrpdr -| | [Tags] | 1518B | 1C -| | phy_cores=${1} | framesize=${1518} +| tc02-64B-2c-ethip4-ip4base-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} -| tc03-9000B-1t1c-ethip4-ip4base-ndrpdr +| tc03-64B-4c-ethip4-ip4base-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4-ip4base-ndrpdr | | [Tags] | 1518B | 1C -| | phy_cores=${1} | framesize=${9000} +| | framesize=${1518} | phy_cores=${1} -| tc04-IMIX-1t1c-ethip4-ip4base-ndrpdr -| | [Tags] | IMIX | 1C -| | phy_cores=${1} | framesize=IMIX_v4_1 +| tc05-1518B-2c-ethip4-ip4base-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} -| tc05-64B-2t2c-ethip4-ip4base-ndrpdr -| | [Tags] | 64B | 2C -| | phy_cores=${2} | framesize=${64} +| tc06-1518B-4c-ethip4-ip4base-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} -| tc06-1518B-2t2c-ethip4-ip4base-ndrpdr -| | [Tags] | 1518B | 2C -| | phy_cores=${2} | framesize=${1518} +| tc07-9000B-1c-ethip4-ip4base-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc07-9000B-2t2c-ethip4-ip4base-ndrpdr +| tc08-9000B-2c-ethip4-ip4base-ndrpdr | | [Tags] | 9000B | 2C -| | phy_cores=${2} | framesize=${9000} - -| tc08-IMIX-2t2c-ethip4-ip4base-ndrpdr -| | [Tags] | IMIX | 2C -| | phy_cores=${2} | framesize=IMIX_v4_1 +| | framesize=${9000} | phy_cores=${2} -| tc09-64B-4t4c-ethip4-ip4base-ndrpdr -| | [Tags] | 64B | 4C -| | phy_cores=${4} | framesize=${64} +| tc09-9000B-4c-ethip4-ip4base-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} -| tc10-1518B-4t4c-ethip4-ip4base-ndrpdr -| | [Tags] | 1518B | 4C -| | phy_cores=${4} | framesize=${1518} +| tc10-IMIX-1c-ethip4-ip4base-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} -| tc11-9000B-4t4c-ethip4-ip4base-ndrpdr -| | [Tags] | 9000B | 4C -| | phy_cores=${4} | framesize=${9000} +| tc11-IMIX-2c-ethip4-ip4base-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc12-IMIX-4t4c-ethip4-ip4base-ndrpdr +| tc12-IMIX-4c-ethip4-ip4base-ndrpdr | | [Tags] | IMIX | 4C -| | phy_cores=${4} | framesize=IMIX_v4_1 +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4base-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4base-ndrpdrdisc.robot deleted file mode 100644 index 26c9e473eb..0000000000 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4base-ndrpdrdisc.robot +++ /dev/null @@ -1,164 +0,0 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at: -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -*** Settings *** -| Resource | resources/libraries/robot/performance/performance_setup.robot -| ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC -| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | BASE | IP4BASE -| ... -| Suite Setup | Set up 3-node performance topology with DUT's NIC model -| ... | L3 | Intel-X520-DA2 -| Suite Teardown | Tear down 3-node performance topology -| ... -| Test Setup | Set up performance test -| ... -| Test Teardown | Tear down performance discovery test | ${min_rate}pps -| ... | ${framesize} | ${traffic_profile} -| ... -| Test Template | Local template -| ... -| Documentation | *RFC2544: Pkt throughput IPv4 routing test cases* -| ... -| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology -| ... | with single links between nodes. -| ... | *[Enc] Packet Encapsulations:* Eth-IPv4 for IPv4 routing. -| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 -| ... | routing and two static IPv4 /24 route entries. DUT1 and DUT2 tested with -| ... | 2p10GE NIC X520 Niantic by Intel. -| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop -| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop -| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage -| ... | of packets transmitted. NDR and PDR are discovered for different -| ... | Ethernet L2 frame sizes using either binary search or linear search -| ... | algorithms with configured starting rate and final step that determines -| ... | throughput measurement resolution. Test packets are generated by TG on -| ... | links to DUTs. TG traffic profile contains two L3 flow-groups -| ... | (flow-group per direction, 253 flows per flow-group) with all packets -| ... | containing Ethernet header, IPv4 header with IP protocol=61 and static -| ... | payload. MAC addresses are matching MAC addresses of the TG node -| ... | interfaces. -| ... | *[Ref] Applicable standard specifications:* RFC2544. - -*** Variables *** -# X520-DA2 bandwidth limit -| ${s_limit} | ${10000000000} -# Traffic profile: -| ${traffic_profile} | trex-sl-3n-ethip4-ip4src253 - -*** Keywords *** -| Local template -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with ${wt} thread(s), ${wt}\ -| | ... | phy core(s), ${rxq} receive queue(s) per NIC port. -| | ... | [Ver] Find NDR or PDR for ${framesize} frames using binary search\ -| | ... | start at 10GE linerate. -| | ... -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} -| | ... -| | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${get_framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} -| | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding in 3-node circular topology -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} - -*** Test Cases *** -| tc01-64B-1t1c-ethip4-ip4base-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR - -| tc02-64B-1t1c-ethip4-ip4base-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR - -| tc03-1518B-1t1c-ethip4-ip4base-ndrdisc -| | [Tags] | 1518B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${1518} | search_type=NDR - -| tc04-1518B-1t1c-ethip4-ip4base-pdrdisc -| | [Tags] | 1518B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${1518} | search_type=PDR - -| tc05-9000B-1t1c-ethip4-ip4base-ndrdisc -| | [Tags] | 9000B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${9000} | search_type=NDR - -| tc06-9000B-1t1c-ethip4-ip4base-pdrdisc -| | [Tags] | 9000B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${9000} | search_type=PDR - -| tc07-64B-2t2c-ethip4-ip4base-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR - -| tc08-64B-2t2c-ethip4-ip4base-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR - -| tc09-1518B-2t2c-ethip4-ip4base-ndrdisc -| | [Tags] | 1518B | 2C | NDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${1518} | search_type=NDR - -| tc10-1518B-2t2c-ethip4-ip4base-pdrdisc -| | [Tags] | 1518B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${1518} | search_type=PDR - -| tc11-9000B-2t2c-ethip4-ip4base-ndrdisc -| | [Tags] | 9000B | 2C | NDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${9000} | search_type=NDR - -| tc12-9000B-2t2c-ethip4-ip4base-pdrdisc -| | [Tags] | 9000B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${9000} | search_type=PDR - -| tc13-64B-4t4c-ethip4-ip4base-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR - -| tc14-64B-4t4c-ethip4-ip4base-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR - -| tc15-1518B-4t4c-ethip4-ip4base-ndrdisc -| | [Tags] | 1518B | 4C | NDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${1518} | search_type=NDR - -| tc16-1518B-4t4c-ethip4-ip4base-pdrdisc -| | [Tags] | 1518B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${1518} | search_type=PDR - -| tc17-9000B-4t4c-ethip4-ip4base-ndrdisc -| | [Tags] | 9000B | 4C | NDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${9000} | search_type=NDR - -| tc18-9000B-4t4c-ethip4-ip4base-pdrdisc -| | [Tags] | 9000B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${9000} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4scale200k-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4scale200k-mrr.robot index e1f57f09fd..9579381dec 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4scale200k-mrr.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4scale200k-mrr.robot @@ -25,7 +25,7 @@ | ... | Test Teardown | Tear down performance mrr test | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *Raw results IPv4 routing test cases* | ... @@ -52,7 +52,7 @@ | ${traffic_profile}= | trex-sl-3n-ethip4-ip4dst${rts_per_flow} *** Keywords *** -| Local template +| Local Template | | ... | | [Documentation] | | ... | [Cfg] DUT runs IPv4 routing config. @@ -66,16 +66,12 @@ | | ... | - phy_cores - Number of physical cores. Type: integer | | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer | | ... -| | [Arguments] | ${phy_cores} | ${framesize} | ${rxq}=${None} -| | ... -| | Set Test Variable | ${framesize} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | When Initialize IPv4 forwarding with scaling in 3-node circular topology | | ... | ${rts_per_flow} @@ -83,50 +79,50 @@ | | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} *** Test Cases *** -| tc01-64B-1t1c-ethip4-ip4scale200k-mrr +| tc01-64B-1c-ethip4-ip4scale200k-mrr | | [Tags] | 64B | 1C -| | phy_cores=${1} | framesize=${64} - -| tc02-1518B-1t1c-ethip4-ip4scale200k-mrr -| | [Tags] | 1518B | 1C -| | phy_cores=${1} | framesize=${1518} +| | framesize=${64} | phy_cores=${1} -| tc03-9000B-1t1c-ethip4-ip4scale200k-mrr -| | [Tags] | 9000B | 1C -| | phy_cores=${1} | framesize=${9000} +| tc02-64B-2c-ethip4-ip4scale200k-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} -| tc04-IMIX-1t1c-ethip4-ip4scale200k-mrr -| | [Tags] | IMIX | 1C -| | phy_cores=${1} | framesize=IMIX_v4_1 +| tc03-64B-4c-ethip4-ip4scale200k-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} -| tc05-64B-2t2c-ethip4-ip4scale200k-mrr -| | [Tags] | 64B | 2C -| | phy_cores=${2} | framesize=${64} +| tc04-1518B-1c-ethip4-ip4scale200k-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} -| tc06-1518B-2t2c-ethip4-ip4scale200k-mrr +| tc05-1518B-2c-ethip4-ip4scale200k-mrr | | [Tags] | 1518B | 2C -| | phy_cores=${2} | framesize=${1518} - -| tc07-9000B-2t2c-ethip4-ip4scale200k-mrr -| | [Tags] | 9000B | 2C -| | phy_cores=${2} | framesize=${9000} +| | framesize=${1518} | phy_cores=${2} -| tc08-IMIX-2t2c-ethip4-ip4scale200k-mrr -| | [Tags] | IMIX | 2C -| | phy_cores=${2} | framesize=IMIX_v4_1 +| tc06-1518B-4c-ethip4-ip4scale200k-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} -| tc09-64B-4t4c-ethip4-ip4scale200k-mrr -| | [Tags] | 64B | 4C -| | phy_cores=${4} | framesize=${64} +| tc07-9000B-1c-ethip4-ip4scale200k-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc10-1518B-4t4c-ethip4-ip4scale200k-mrr -| | [Tags] | 1518B | 4C -| | phy_cores=${4} | framesize=${1518} +| tc08-9000B-2c-ethip4-ip4scale200k-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc11-9000B-4t4c-ethip4-ip4scale200k-mrr +| tc09-9000B-4c-ethip4-ip4scale200k-mrr | | [Tags] | 9000B | 4C -| | phy_cores=${4} | framesize=${9000} +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4-ip4scale200k-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4-ip4scale200k-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc12-IMIX-4t4c-ethip4-ip4scale200k-mrr +| tc12-IMIX-4c-ethip4-ip4scale200k-mrr | | [Tags] | IMIX | 4C -| | phy_cores=${4} | framesize=IMIX_v4_1 +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4scale200k-ndrpdr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4scale200k-ndrpdr.robot new file mode 100644 index 0000000000..3b4aa816d3 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4scale200k-ndrpdr.robot @@ -0,0 +1,134 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | SCALE | FIB_200K +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down performance discovery test | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Test Template | Local Template +| ... +| Documentation | *RFC2544: Pkt throughput IPv4 routing test cases* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4 for IPv4 routing. +| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 +| ... | routing and 2x100k static IPv4 /32 route entries. DUT1 and DUT2 tested +| ... | with 2p10GE NIC X520 Niantic by Intel. +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage +| ... | of packets transmitted. NDR and PDR are discovered for different +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on links to DUTs. TG traffic profile +| ... | contains two L3 flow-groups (flow-group per direction, 253 flows per +| ... | flow-group) with all packets containing Ethernet header, IPv4 header +| ... | with IP protocol=61 and static payload. MAC addresses are matching MAC +| ... | addresses of the TG node interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} +| ${rts_per_flow}= | ${100000} +# Traffic profile: +| ${traffic_profile}= | trex-sl-3n-ethip4-ip4dst${rts_per_flow} + +*** Keywords *** +| Local Template +| | ... +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config. +| | ... | Each DUT uses ${phy_cores} physical core(s) for worker threads. +| | ... | [Ver] Measure NDR and PDR values using MLRsearch algorithm.\ +| | ... +| | ... | *Arguments:* +| | ... | - framesize - Framesize in Bytes in integer or string (IMIX_v4_1). +| | ... | Type: integer, string +| | ... | - phy_cores - Number of physical cores. Type: integer +| | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer +| | ... +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | When Initialize IPv4 forwarding with scaling in 3-node circular topology +| | ... | ${rts_per_flow} +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} + +*** Test Cases *** +| tc01-64B-1c-ethip4-ip4scale200k-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4-ip4scale200k-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4-ip4scale200k-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4-ip4scale200k-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4-ip4scale200k-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4-ip4scale200k-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4-ip4scale200k-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4-ip4scale200k-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4-ip4scale200k-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4-ip4scale200k-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4-ip4scale200k-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4-ip4scale200k-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4scale200k-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4scale200k-ndrpdrdisc.robot deleted file mode 100644 index b686f5fb37..0000000000 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4scale200k-ndrpdrdisc.robot +++ /dev/null @@ -1,457 +0,0 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at: -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -*** Settings *** -| Resource | resources/libraries/robot/performance/performance_setup.robot -| ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC -| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | SCALE | FIB_200K -| ... -| Suite Setup | Set up 3-node performance topology with DUT's NIC model -| ... | L3 | Intel-X520-DA2 -| Suite Teardown | Tear down 3-node performance topology -| ... -| Test Setup | Set up performance test -| ... -| Test Teardown | Tear down performance discovery test | ${min_rate}pps -| ... | ${framesize} | ${traffic_profile} -| ... -| Documentation | *RFC2544: Pkt throughput IPv4 routing test cases* -| ... -| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology -| ... | with single links between nodes. -| ... | *[Enc] Packet Encapsulations:* Eth-IPv4 for IPv4 routing. -| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 -| ... | routing and 2x100k static IPv4 /32 route entries. DUT1 and DUT2 tested -| ... | with 2p10GE NIC X520 Niantic by Intel. -| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop -| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop -| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage -| ... | of packets transmitted. NDR and PDR are discovered for different -| ... | Ethernet L2 frame sizes using either binary search or linear search -| ... | algorithms with configured starting rate and final step that determines -| ... | throughput measurement resolution. Test packets are generated by TG on -| ... | links to DUTs. TG traffic profile contains two L3 flow-groups -| ... | (flow-group per direction, 100k flows per flow-group) with all packets -| ... | containing Ethernet header, IPv4 header with IP protocol=61 and -| ... | static payload. MAC addresses are matching MAC addresses of the TG -| ... | node interfaces. Incrementing of IP.dst (IPv4 destination address) field -| ... | is applied to both streams. -| ... | *[Ref] Applicable standard specifications:* RFC2544. - -*** Variables *** -# X520-DA2 bandwidth limit -| ${s_limit} | ${10000000000} -| ${rts_per_flow}= | 100000 -# Traffic profile: -| ${traffic_profile} | trex-sl-3n-ethip4-ip4dst${rts_per_flow} - -*** Test Cases *** -| tc01-64B-1t1c-ethip4-ip4scale200k-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 1 phy core, \ -| | ... | 1 receive queue per NIC port. [Ver] Find NDR for 64 Byte frames -| | ... | using binary search start at 10GE linerate, step 50kpps. -| | [Tags] | 64B | 1C | NDRDISC -| | ${framesize}= | Set Variable | ${64} -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 1 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find NDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} - -| tc02-64B-1t1c-ethip4-ip4scale200k-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 1 phy core, \ -| | ... | 1 receive queue per NIC port. [Ver] Find PDR for 64 Byte frames -| | ... | using binary search start at 10GE linerate, step 50kpps, LT=0.5%. -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | ${64} -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 1 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find PDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} - -| tc03-1518B-1t1c-ethip4-ip4scale200k-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 1 phy core, \ -| | ... | 1 receive queue per NIC port. [Ver] Find NDR for 1518 Byte frames -| | ... | using binary search start at 10GE linerate, step 50kpps. -| | [Tags] | 1518B | 1C | NDRDISC -| | ${framesize}= | Set Variable | ${1518} -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 1 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find NDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} - -| tc04-1518B-1t1c-ethip4-ip4scale200k-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 1 phy core, \ -| | ... | 1 receive queue per NIC port. [Ver] Find PDR for 1518 Byte frames -| | ... | using binary search start at 10GE linerate, step 50kpps, LT=0.5%. -| | [Tags] | 1518B | 1C | PDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | ${1518} -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 1 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find PDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} - -| tc05-9000B-1t1c-ethip4-ip4scale200k-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 1 phy core, \ -| | ... | 1 receive queue per NIC port. [Ver] Find NDR for 9000 Byte frames -| | ... | using binary search start at 10GE linerate, step 10kpps. -| | [Tags] | 9000B | 1C | NDRDISC -| | ${framesize}= | Set Variable | ${9000} -| | ${min_rate}= | Set Variable | ${10000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 1 -| | And Add PCI devices to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find NDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} - -| tc06-9000B-1t1c-ethip4-ip4scale200k-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 1 phy core, \ -| | ... | 1 receive queue per NIC port. [Ver] Find PDR for 9000 Byte frames -| | ... | using binary search start at 10GE linerate, step 10kpps, LT=0.5%. -| | [Tags] | 9000B | 1C | PDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | ${9000} -| | ${min_rate}= | Set Variable | ${10000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 1 -| | And Add PCI devices to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find PDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} - -| tc07-64B-2t2c-ethip4-ip4scale200k-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 2 phy cores, \ -| | ... | 1 receive queue per NIC port. [Ver] Find NDR for 64 Byte frames -| | ... | using binary search start at 10GE linerate, step 50kpps. -| | [Tags] | 64B | 2C | NDRDISC -| | ${framesize}= | Set Variable | ${64} -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 2 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find NDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} - -| tc08-64B-2t2c-ethip4-ip4scale200k-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 2 phy cores, \ -| | ... | 1 receive queue per NIC port. [Ver] Find PDR for 64 Byte frames -| | ... | using binary search start at 10GE linerate, step 50kpps, LT=0.5%. -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | ${64} -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 2 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find PDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} - -| tc09-1518B-2t2c-ethip4-ip4scale200k-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 2 phy cores, \ -| | ... | 1 receive queue per NIC port. [Ver] Find NDR for 1518 Byte frames -| | ... | using binary search start at 10GE linerate, step 50kpps. -| | [Tags] | 1518B | 2C | NDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | ${1518} -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 2 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find NDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} - -| tc10-1518B-2t2c-ethip4-ip4scale200k-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 2 phy cores, \ -| | ... | 1 receive queue per NIC port. [Ver] Find PDR for 1518 Byte frames -| | ... | using binary search start at 10GE linerate, step 50kpps, LT=0.5%. -| | [Tags] | 1518B | 2C | PDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | ${1518} -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 2 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find PDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} - -| tc11-9000B-2t2c-ethip4-ip4scale200k-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 2 phy cores, \ -| | ... | 1 receive queue per NIC port. [Ver] Find NDR for 9000 Byte frames -| | ... | using binary search start at 10GE linerate, step 10kpps. -| | [Tags] | 9000B | 2C | NDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | ${9000} -| | ${min_rate}= | Set Variable | ${10000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 2 -| | And Add PCI devices to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find NDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} - -| tc12-9000B-2t2c-ethip4-ip4scale200k-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 2 phy cores, \ -| | ... | 1 receive queue per NIC port. [Ver] Find PDR for 9000 Byte frames -| | ... | using binary search start at 10GE linerate, step 10kpps, LT=0.5%. -| | [Tags] | 9000B | 2C | PDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | ${9000} -| | ${min_rate}= | Set Variable | ${10000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 2 -| | And Add PCI devices to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find PDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} - -| tc13-64B-4t4c-ethip4-ip4scale200k-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 4 phy cores, \ -| | ... | 2 receive queues per NIC port. [Ver] Find NDR for 64 Byte frames -| | ... | using binary search start at 10GE linerate, step 50kpps. -| | [Tags] | 64B | 4C | NDRDISC -| | ${framesize}= | Set Variable | ${64} -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 4 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find NDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} - -| tc14-64B-4t4c-ethip4-ip4scale200k-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 4 phy cores, \ -| | ... | 2 receive queues per NIC port. [Ver] Find PDR for 64 Byte frames -| | ... | using binary search start at 10GE linerate, step 50kpps, LT=0.5%. -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | ${64} -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 4 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find PDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} - -| tc15-1518B-4t4c-ethip4-ip4scale200k-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 4 phy cores, \ -| | ... | 2 receive queues per NIC port. [Ver] Find NDR for 1518 Byte frames -| | ... | using binary search start at 10GE linerate, step 50kpps. -| | [Tags] | 1518B | 4C | NDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | ${1518} -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 4 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find NDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} - -| tc16-1518B-4t4c-ethip4-ip4scale200k-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 4 phy cores, \ -| | ... | 2 receive queues per NIC port. [Ver] Find PDR for 1518 Byte frames -| | ... | using binary search start at 10GE linerate, step 50kpps, LT=0.5%. -| | [Tags] | 1518B | 4C | PDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | ${1518} -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 4 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find PDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} - -| tc17-9000B-4t4c-ethip4-ip4scale200k-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 4 phy cores, \ -| | ... | 2 receive queues per NIC port. [Ver] Find NDR for 9000 Byte frames -| | ... | using binary search start at 10GE linerate, step 10kpps. -| | [Tags] | 9000B | 4C | NDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | ${9000} -| | ${min_rate}= | Set Variable | ${10000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 4 -| | And Add PCI devices to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find NDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} - -| tc18-9000B-4t4c-ethip4-ip4scale200k-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 4 phy cores, \ -| | ... | 2 receive queues per NIC port. [Ver] Find PDR for 9000 Byte frames -| | ... | using binary search start at 10GE linerate, step 10kpps, LT=0.5%. -| | [Tags] | 9000B | 4C | PDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | ${9000} -| | ${min_rate}= | Set Variable | ${10000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 4 -| | And Add PCI devices to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find PDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4scale20k-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4scale20k-mrr.robot index 9624c9ca89..9c1583fbde 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4scale20k-mrr.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4scale20k-mrr.robot @@ -25,7 +25,7 @@ | ... | Test Teardown | Tear down performance mrr test | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *Raw results IPv4 routing test cases* | ... @@ -52,7 +52,7 @@ | ${traffic_profile}= | trex-sl-3n-ethip4-ip4dst${rts_per_flow} *** Keywords *** -| Local template +| Local Template | | [Documentation] | | ... | [Cfg] DUT runs IPv4 routing config. | | ... | Each DUT uses ${phy_cores} physical core(s) for worker threads. @@ -65,16 +65,12 @@ | | ... | - phy_cores - Number of physical cores. Type: integer | | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer | | ... -| | [Arguments] | ${phy_cores} | ${framesize} | ${rxq}=${None} -| | ... -| | Set Test Variable | ${framesize} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | When Initialize IPv4 forwarding with scaling in 3-node circular topology | | ... | ${rts_per_flow} @@ -82,50 +78,50 @@ | | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} *** Test Cases *** -| tc01-64B-1t1c-ethip4-ip4scale20k-mrr +| tc01-64B-1c-ethip4-ip4scale20k-mrr | | [Tags] | 64B | 1C -| | phy_cores=${1} | framesize=${64} - -| tc02-1518B-1t1c-ethip4-ip4scale20k-mrr -| | [Tags] | 1518B | 1C -| | phy_cores=${1} | framesize=${1518} +| | framesize=${64} | phy_cores=${1} -| tc03-9000B-1t1c-ethip4-ip4scale20k-mrr -| | [Tags] | 9000B | 1C -| | phy_cores=${1} | framesize=${9000} +| tc02-64B-2c-ethip4-ip4scale20k-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} -| tc04-IMIX-1t1c-ethip4-ip4scale20k-mrr -| | [Tags] | IMIX | 1C -| | phy_cores=${1} | framesize=IMIX_v4_1 +| tc03-64B-4c-ethip4-ip4scale20k-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} -| tc05-64B-2t2c-ethip4-ip4scale20k-mrr -| | [Tags] | 64B | 2C -| | phy_cores=${2} | framesize=${64} +| tc04-1518B-1c-ethip4-ip4scale20k-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} -| tc06-1518B-2t2c-ethip4-ip4scale20k-mrr +| tc05-1518B-2c-ethip4-ip4scale20k-mrr | | [Tags] | 1518B | 2C -| | phy_cores=${2} | framesize=${1518} - -| tc07-9000B-2t2c-ethip4-ip4scale20k-mrr -| | [Tags] | 9000B | 2C -| | phy_cores=${2} | framesize=${9000} +| | framesize=${1518} | phy_cores=${2} -| tc08-IMIX-2t2c-ethip4-ip4scale20k-mrr -| | [Tags] | IMIX | 2C -| | phy_cores=${2} | framesize=IMIX_v4_1 +| tc06-1518B-4c-ethip4-ip4scale20k-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} -| tc09-64B-4t4c-ethip4-ip4scale20k-mrr -| | [Tags] | 64B | 4C -| | phy_cores=${4} | framesize=${64} +| tc07-9000B-1c-ethip4-ip4scale20k-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc10-1518B-4t4c-ethip4-ip4scale20k-mrr -| | [Tags] | 1518B | 4C -| | phy_cores=${4} | framesize=${1518} +| tc08-9000B-2c-ethip4-ip4scale20k-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc11-9000B-4t4c-ethip4-ip4scale20k-mrr +| tc09-9000B-4c-ethip4-ip4scale20k-mrr | | [Tags] | 9000B | 4C -| | phy_cores=${4} | framesize=${9000} +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4-ip4scale20k-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4-ip4scale20k-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc12-IMIX-4t4c-ethip4-ip4scale20k-mrr +| tc12-IMIX-4c-ethip4-ip4scale20k-mrr | | [Tags] | IMIX | 4C -| | phy_cores=${4} | framesize=IMIX_v4_1 +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4scale20k-ndrpdr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4scale20k-ndrpdr.robot new file mode 100644 index 0000000000..63f7003012 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4scale20k-ndrpdr.robot @@ -0,0 +1,133 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | SCALE | FIB_20K +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down performance discovery test | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Test Template | Local Template +| ... +| Documentation | *RFC2544: Pkt throughput IPv4 routing test cases* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4 for IPv4 routing. +| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 +| ... | routing and 2x100k static IPv4 /32 route entries. DUT1 and DUT2 tested +| ... | with 2p10GE NIC X520 Niantic by Intel. +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage +| ... | of packets transmitted. NDR and PDR are discovered for different +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on links to DUTs. TG traffic profile +| ... | contains two L3 flow-groups (flow-group per direction, 253 flows per +| ... | flow-group) with all packets containing Ethernet header, IPv4 header +| ... | with IP protocol=61 and static payload. MAC addresses are matching MAC +| ... | addresses of the TG node interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} +| ${rts_per_flow}= | ${10000} +# Traffic profile: +| ${traffic_profile}= | trex-sl-3n-ethip4-ip4dst${rts_per_flow} + +*** Keywords *** +| Local Template +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config. +| | ... | Each DUT uses ${phy_cores} physical core(s) for worker threads. +| | ... | [Ver] Measure NDR and PDR values using MLRsearch algorithm.\ +| | ... +| | ... | *Arguments:* +| | ... | - framesize - Framesize in Bytes in integer or string (IMIX_v4_1). +| | ... | Type: integer, string +| | ... | - phy_cores - Number of physical cores. Type: integer +| | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer +| | ... +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | When Initialize IPv4 forwarding with scaling in 3-node circular topology +| | ... | ${rts_per_flow} +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} + +*** Test Cases *** +| tc01-64B-1c-ethip4-ip4scale20k-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4-ip4scale20k-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4-ip4scale20k-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4-ip4scale20k-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4-ip4scale20k-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4-ip4scale20k-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4-ip4scale20k-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4-ip4scale20k-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4-ip4scale20k-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4-ip4scale20k-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4-ip4scale20k-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4-ip4scale20k-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4scale20k-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4scale20k-ndrpdrdisc.robot deleted file mode 100644 index 7f1918b7a3..0000000000 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4scale20k-ndrpdrdisc.robot +++ /dev/null @@ -1,457 +0,0 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at: -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -*** Settings *** -| Resource | resources/libraries/robot/performance/performance_setup.robot -| ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC -| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | SCALE | FIB_20K -| ... -| Suite Setup | Set up 3-node performance topology with DUT's NIC model -| ... | L3 | Intel-X520-DA2 -| Suite Teardown | Tear down 3-node performance topology -| ... -| Test Setup | Set up performance test -| ... -| Test Teardown | Tear down performance discovery test | ${min_rate}pps -| ... | ${framesize} | ${traffic_profile} -| ... -| Documentation | *RFC2544: Pkt throughput IPv4 routing test cases* -| ... -| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology -| ... | with single links between nodes. -| ... | *[Enc] Packet Encapsulations:* Eth-IPv4 for IPv4 routing. -| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 -| ... | routing and 2x10k static IPv4 /32 route entries. DUT1 and DUT2 tested -| ... | with 2p10GE NIC X520 Niantic by Intel. -| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop -| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop -| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage -| ... | of packets transmitted. NDR and PDR are discovered for different -| ... | Ethernet L2 frame sizes using either binary search or linear search -| ... | algorithms with configured starting rate and final step that determines -| ... | throughput measurement resolution. Test packets are generated by TG on -| ... | links to DUTs. TG traffic profile contains two L3 flow-groups -| ... | (flow-group per direction, 10k flows per flow-group) with all packets -| ... | containing Ethernet header, IPv4 header with IP protocol=61 and -| ... | static payload. MAC addresses are matching MAC addresses of the TG -| ... | node interfaces. Incrementing of IP.dst (IPv4 destination address) field -| ... | is applied to both streams. -| ... | *[Ref] Applicable standard specifications:* RFC2544. - -*** Variables *** -# X520-DA2 bandwidth limit -| ${s_limit} | ${10000000000} -| ${rts_per_flow}= | ${10000} -# Traffic profile: -| ${traffic_profile} | trex-sl-3n-ethip4-ip4dst${rts_per_flow} - -*** Test Cases *** -| tc01-64B-1t1c-ethip4-ip4scale20k-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 1 phy core, \ -| | ... | 1 receive queue per NIC port. [Ver] Find NDR for 64 Byte frames -| | ... | using binary search start at 10GE linerate, step 50kpps. -| | [Tags] | 64B | 1C | NDRDISC -| | ${framesize}= | Set Variable | ${64} -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 1 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find NDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} - -| tc02-64B-1t1c-ethip4-ip4scale20k-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 1 phy core, \ -| | ... | 1 receive queue per NIC port. [Ver] Find PDR for 64 Byte frames -| | ... | using binary search start at 10GE linerate, step 50kpps, LT=0.5%. -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | ${64} -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 1 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find PDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} - -| tc03-1518B-1t1c-ethip4-ip4scale20k-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 1 phy core, \ -| | ... | 1 receive queue per NIC port. [Ver] Find NDR for 1518 Byte frames -| | ... | using binary search start at 10GE linerate, step 50kpps. -| | [Tags] | 1518B | 1C | NDRDISC -| | ${framesize}= | Set Variable | ${1518} -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 1 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find NDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} - -| tc04-1518B-1t1c-ethip4-ip4scale20k-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 1 phy core, \ -| | ... | 1 receive queue per NIC port. [Ver] Find PDR for 1518 Byte frames -| | ... | using binary search start at 10GE linerate, step 50kpps, LT=0.5%. -| | [Tags] | 1518B | 1C | PDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | ${1518} -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 1 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find PDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} - -| tc05-9000B-1t1c-ethip4-ip4scale20k-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 1 phy core, \ -| | ... | 1 receive queue per NIC port. [Ver] Find NDR for 9000 Byte frames -| | ... | using binary search start at 10GE linerate, step 10kpps. -| | [Tags] | 9000B | 1C | NDRDISC -| | ${framesize}= | Set Variable | ${9000} -| | ${min_rate}= | Set Variable | ${10000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 1 -| | And Add PCI devices to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find NDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} - -| tc06-9000B-1t1c-ethip4-ip4scale20k-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 1 phy core, \ -| | ... | 1 receive queue per NIC port. [Ver] Find PDR for 9000 Byte frames -| | ... | using binary search start at 10GE linerate, step 10kpps, LT=0.5%. -| | [Tags] | 9000B | 1C | PDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | ${9000} -| | ${min_rate}= | Set Variable | ${10000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 1 -| | And Add PCI devices to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find PDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} - -| tc07-64B-2t2c-ethip4-ip4scale20k-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 2 phy cores, \ -| | ... | 1 receive queue per NIC port. [Ver] Find NDR for 64 Byte frames -| | ... | using binary search start at 10GE linerate, step 50kpps. -| | [Tags] | 64B | 2C | NDRDISC -| | ${framesize}= | Set Variable | ${64} -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 2 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find NDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} - -| tc08-64B-2t2c-ethip4-ip4scale20k-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 2 phy cores, \ -| | ... | 1 receive queue per NIC port. [Ver] Find PDR for 64 Byte frames -| | ... | using binary search start at 10GE linerate, step 50kpps, LT=0.5%. -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | ${64} -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 2 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find PDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} - -| tc09-1518B-2t2c-ethip4-ip4scale20k-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 2 phy cores, \ -| | ... | 1 receive queue per NIC port. [Ver] Find NDR for 1518 Byte frames -| | ... | using binary search start at 10GE linerate, step 50kpps. -| | [Tags] | 1518B | 2C | NDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | ${1518} -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 2 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find NDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} - -| tc10-1518B-2t2c-ethip4-ip4scale20k-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 2 phy cores, \ -| | ... | 1 receive queue per NIC port. [Ver] Find PDR for 1518 Byte frames -| | ... | using binary search start at 10GE linerate, step 50kpps, LT=0.5%. -| | [Tags] | 1518B | 2C | PDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | ${1518} -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 2 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find PDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} - -| tc11-9000B-2t2c-ethip4-ip4scale20k-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 2 phy cores, \ -| | ... | 1 receive queue per NIC port. [Ver] Find NDR for 9000 Byte frames -| | ... | using binary search start at 10GE linerate, step 10kpps. -| | [Tags] | 9000B | 2C | NDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | ${9000} -| | ${min_rate}= | Set Variable | ${10000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 2 -| | And Add PCI devices to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find NDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} - -| tc12-9000B-2t2c-ethip4-ip4scale20k-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 2 phy cores, \ -| | ... | 1 receive queue per NIC port. [Ver] Find PDR for 9000 Byte frames -| | ... | using binary search start at 10GE linerate, step 10kpps, LT=0.5%. -| | [Tags] | 9000B | 2C | PDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | ${9000} -| | ${min_rate}= | Set Variable | ${10000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 2 -| | And Add PCI devices to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find PDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} - -| tc13-64B-4t4c-ethip4-ip4scale20k-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 4 phy cores, \ -| | ... | 2 receive queues per NIC port. [Ver] Find NDR for 64 Byte frames -| | ... | using binary search start at 10GE linerate, step 50kpps. -| | [Tags] | 64B | 4C | NDRDISC -| | ${framesize}= | Set Variable | ${64} -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 4 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find NDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} - -| tc14-64B-4t4c-ethip4-ip4scale20k-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 4 phy cores, \ -| | ... | 2 receive queues per NIC port. [Ver] Find PDR for 64 Byte frames -| | ... | using binary search start at 10GE linerate, step 50kpps, LT=0.5%. -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | ${64} -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 4 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find PDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} - -| tc15-1518B-4t4c-ethip4-ip4scale20k-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 4 phy cores, \ -| | ... | 2 receive queues per NIC port. [Ver] Find NDR for 1518 Byte frames -| | ... | using binary search start at 10GE linerate, step 50kpps. -| | [Tags] | 1518B | 4C | NDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | ${1518} -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 4 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find NDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} - -| tc16-1518B-4t4c-ethip4-ip4scale20k-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 4 phy cores, \ -| | ... | 2 receive queues per NIC port. [Ver] Find PDR for 1518 Byte frames -| | ... | using binary search start at 10GE linerate, step 50kpps, LT=0.5%. -| | [Tags] | 1518B | 4C | PDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | ${1518} -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 4 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find PDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} - -| tc17-9000B-4t4c-ethip4-ip4scale20k-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 4 phy cores, \ -| | ... | 2 receive queues per NIC port. [Ver] Find NDR for 9000 Byte frames -| | ... | using binary search start at 10GE linerate, step 10kpps. -| | [Tags] | 9000B | 4C | NDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | ${9000} -| | ${min_rate}= | Set Variable | ${10000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 4 -| | And Add PCI devices to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find NDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} - -| tc18-9000B-4t4c-ethip4-ip4scale20k-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 4 phy cores, \ -| | ... | 2 receive queues per NIC port. [Ver] Find PDR for 9000 Byte frames -| | ... | using binary search start at 10GE linerate, step 10kpps, LT=0.5%. -| | [Tags] | 9000B | 4C | PDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | ${9000} -| | ${min_rate}= | Set Variable | ${10000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 4 -| | And Add PCI devices to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find PDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4scale2m-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4scale2m-mrr.robot index 4db6f4c363..038dfa1f40 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4scale2m-mrr.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4scale2m-mrr.robot @@ -25,7 +25,7 @@ | ... | Test Teardown | Tear down performance mrr test | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *Raw results IPv4 routing test cases* | ... @@ -52,7 +52,7 @@ | ${traffic_profile}= | trex-sl-3n-ethip4-ip4dst${rts_per_flow} *** Keywords *** -| Local template +| Local Template | | ... | | [Documentation] | | ... | [Cfg] DUT runs IPv4 routing config. @@ -66,16 +66,12 @@ | | ... | - phy_cores - Number of physical cores. Type: integer | | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer | | ... -| | [Arguments] | ${phy_cores} | ${framesize} | ${rxq}=${None} -| | ... -| | Set Test Variable | ${framesize} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | When Initialize IPv4 forwarding with scaling in 3-node circular topology | | ... | ${rts_per_flow} @@ -83,50 +79,50 @@ | | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} *** Test Cases *** -| tc01-64B-1t1c-ethip4-ip4scale2m-mrr +| tc01-64B-1c-ethip4-ip4scale2m-mrr | | [Tags] | 64B | 1C -| | phy_cores=${1} | framesize=${64} - -| tc02-1518B-1t1c-ethip4-ip4scale2m-mrr -| | [Tags] | 1518B | 1C -| | phy_cores=${1} | framesize=${1518} +| | framesize=${64} | phy_cores=${1} -| tc03-9000B-1t1c-ethip4-ip4scale2m-mrr -| | [Tags] | 9000B | 1C -| | phy_cores=${1} | framesize=${9000} +| tc02-64B-2c-ethip4-ip4scale2m-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} -| tc04-IMIX-1t1c-ethip4-ip4scale2m-mrr -| | [Tags] | IMIX | 1C -| | phy_cores=${1} | framesize=IMIX_v4_1 +| tc03-64B-4c-ethip4-ip4scale2m-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} -| tc05-64B-2t2c-ethip4-ip4scale2m-mrr -| | [Tags] | 64B | 2C -| | phy_cores=${2} | framesize=${64} +| tc04-1518B-1c-ethip4-ip4scale2m-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} -| tc06-1518B-2t2c-ethip4-ip4scale2m-mrr +| tc05-1518B-2c-ethip4-ip4scale2m-mrr | | [Tags] | 1518B | 2C -| | phy_cores=${2} | framesize=${1518} - -| tc07-9000B-2t2c-ethip4-ip4scale2m-mrr -| | [Tags] | 9000B | 2C -| | phy_cores=${2} | framesize=${9000} +| | framesize=${1518} | phy_cores=${2} -| tc08-IMIX-2t2c-ethip4-ip4scale2m-mrr -| | [Tags] | IMIX | 2C -| | phy_cores=${2} | framesize=IMIX_v4_1 +| tc06-1518B-4c-ethip4-ip4scale2m-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} -| tc09-64B-4t4c-ethip4-ip4scale2m-mrr -| | [Tags] | 64B | 4C -| | phy_cores=${4} | framesize=${64} +| tc07-9000B-1c-ethip4-ip4scale2m-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc10-1518B-4t4c-ethip4-ip4scale2m-mrr -| | [Tags] | 1518B | 4C -| | phy_cores=${4} | framesize=${1518} +| tc08-9000B-2c-ethip4-ip4scale2m-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc11-9000B-4t4c-ethip4-ip4scale2m-mrr +| tc09-9000B-4c-ethip4-ip4scale2m-mrr | | [Tags] | 9000B | 4C -| | phy_cores=${4} | framesize=${9000} +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4-ip4scale2m-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4-ip4scale2m-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc12-IMIX-4t4c-ethip4-ip4scale2m-mrr +| tc12-IMIX-4c-ethip4-ip4scale2m-mrr | | [Tags] | IMIX | 4C -| | phy_cores=${4} | framesize=IMIX_v4_1 +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4scale2m-ndrpdr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4scale2m-ndrpdr.robot new file mode 100644 index 0000000000..40a4923eac --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4scale2m-ndrpdr.robot @@ -0,0 +1,134 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | SCALE | FIB_2M +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X520-DA2 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down performance discovery test | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Test Template | Local Template +| ... +| Documentation | *RFC2544: Pkt throughput IPv4 routing test cases* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4 for IPv4 routing. +| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 +| ... | routing and 2x100k static IPv4 /32 route entries. DUT1 and DUT2 tested +| ... | with 2p10GE NIC X520 Niantic by Intel. +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage +| ... | of packets transmitted. NDR and PDR are discovered for different +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on links to DUTs. TG traffic profile +| ... | contains two L3 flow-groups (flow-group per direction, 253 flows per +| ... | flow-group) with all packets containing Ethernet header, IPv4 header +| ... | with IP protocol=61 and static payload. MAC addresses are matching MAC +| ... | addresses of the TG node interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} +| ${rts_per_flow}= | ${1000000} +# Traffic profile: +| ${traffic_profile}= | trex-sl-3n-ethip4-ip4dst${rts_per_flow} + +*** Keywords *** +| Local Template +| | ... +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config. +| | ... | Each DUT uses ${phy_cores} physical core(s) for worker threads. +| | ... | [Ver] Measure NDR and PDR values using MLRsearch algorithm.\ +| | ... +| | ... | *Arguments:* +| | ... | - framesize - Framesize in Bytes in integer or string (IMIX_v4_1). +| | ... | Type: integer, string +| | ... | - phy_cores - Number of physical cores. Type: integer +| | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer +| | ... +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | When Initialize IPv4 forwarding with scaling in 3-node circular topology +| | ... | ${rts_per_flow} +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} + +*** Test Cases *** +| tc01-64B-1c-ethip4-ip4scale2m-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4-ip4scale2m-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4-ip4scale2m-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4-ip4scale2m-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4-ip4scale2m-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4-ip4scale2m-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4-ip4scale2m-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4-ip4scale2m-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4-ip4scale2m-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4-ip4scale2m-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4-ip4scale2m-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4-ip4scale2m-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4scale2m-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4scale2m-ndrpdrdisc.robot deleted file mode 100644 index 4cd00b1dd4..0000000000 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4-ip4scale2m-ndrpdrdisc.robot +++ /dev/null @@ -1,457 +0,0 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at: -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -*** Settings *** -| Resource | resources/libraries/robot/performance/performance_setup.robot -| ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC -| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | SCALE | FIB_2M -| ... -| Suite Setup | Set up 3-node performance topology with DUT's NIC model -| ... | L3 | Intel-X520-DA2 -| Suite Teardown | Tear down 3-node performance topology -| ... -| Test Setup | Set up performance test -| ... -| Test Teardown | Tear down performance discovery test | ${min_rate}pps -| ... | ${framesize} | ${traffic_profile} -| ... -| Documentation | *RFC2544: Pkt throughput IPv4 routing test cases* -| ... -| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology -| ... | with single links between nodes. -| ... | *[Enc] Packet Encapsulations:* Eth-IPv4 for IPv4 routing. -| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 -| ... | routing and 2x1M static IPv4 /32 route entries. DUT1 and DUT2 tested -| ... | with 2p10GE NIC X520 Niantic by Intel. -| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop -| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop -| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage -| ... | of packets transmitted. NDR and PDR are discovered for different -| ... | Ethernet L2 frame sizes using either binary search or linear search -| ... | algorithms with configured starting rate and final step that determines -| ... | throughput measurement resolution. Test packets are generated by TG on -| ... | links to DUTs. TG traffic profile contains two L3 flow-groups -| ... | (flow-group per direction, 1M flows per flow-group) with all packets -| ... | containing Ethernet header, IPv4 header with IP protocol=61 and -| ... | static payload. MAC addresses are matching MAC addresses of the TG -| ... | node interfaces. Incrementing of IP.dst (IPv4 destination address) field -| ... | is applied to both streams. -| ... | *[Ref] Applicable standard specifications:* RFC2544. - -*** Variables *** -# X520-DA2 bandwidth limit -| ${s_limit} | ${10000000000} -| ${rts_per_flow}= | ${1000000} -# Traffic profile: -| ${traffic_profile} | trex-sl-3n-ethip4-ip4dst${rts_per_flow} - -*** Test Cases *** -| tc01-64B-1t1c-ethip4-ip4scale2m-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 1 phy core, \ -| | ... | 1 receive queue per NIC port. [Ver] Find NDR for 64 Byte frames -| | ... | using binary search start at 10GE linerate, step 50kpps. -| | [Tags] | 64B | 1C | NDRDISC -| | ${framesize}= | Set Variable | ${64} -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 1 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find NDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} - -| tc02-64B-1t1c-ethip4-ip4scale2m-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 1 phy core, \ -| | ... | 1 receive queue per NIC port. [Ver] Find PDR for 64 Byte frames -| | ... | using binary search start at 10GE linerate, step 50kpps, LT=0.5%. -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | ${64} -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 1 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find PDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} - -| tc03-1518B-1t1c-ethip4-ip4scale2m-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 1 phy core, \ -| | ... | 1 receive queue per NIC port. [Ver] Find NDR for 1518 Byte frames -| | ... | using binary search start at 10GE linerate, step 50kpps. -| | [Tags] | 1518B | 1C | NDRDISC -| | ${framesize}= | Set Variable | ${1518} -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 1 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find NDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} - -| tc04-1518B-1t1c-ethip4-ip4scale2m-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 1 phy core, \ -| | ... | 1 receive queue per NIC port. [Ver] Find PDR for 1518 Byte frames -| | ... | using binary search start at 10GE linerate, step 50kpps, LT=0.5%. -| | [Tags] | 1518B | 1C | PDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | ${1518} -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 1 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find PDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} - -| tc05-9000B-1t1c-ethip4-ip4scale2m-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 1 phy core, \ -| | ... | 1 receive queue per NIC port. [Ver] Find NDR for 9000 Byte frames -| | ... | using binary search start at 10GE linerate, step 10kpps. -| | [Tags] | 9000B | 1C | NDRDISC -| | ${framesize}= | Set Variable | ${9000} -| | ${min_rate}= | Set Variable | ${10000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 1 -| | And Add PCI devices to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find NDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} - -| tc06-9000B-1t1c-ethip4-ip4scale2m-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 1 phy core, \ -| | ... | 1 receive queue per NIC port. [Ver] Find PDR for 9000 Byte frames -| | ... | using binary search start at 10GE linerate, step 10kpps, LT=0.5%. -| | [Tags] | 9000B | 1C | PDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | ${9000} -| | ${min_rate}= | Set Variable | ${10000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 1 -| | And Add PCI devices to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find PDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} - -| tc07-64B-2t2c-ethip4-ip4scale2m-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 2 phy cores, \ -| | ... | 1 receive queue per NIC port. [Ver] Find NDR for 64 Byte frames -| | ... | using binary search start at 10GE linerate, step 50kpps. -| | [Tags] | 64B | 2C | NDRDISC -| | ${framesize}= | Set Variable | ${64} -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 2 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find NDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} - -| tc08-64B-2t2c-ethip4-ip4scale2m-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 2 phy cores, \ -| | ... | 1 receive queue per NIC port. [Ver] Find PDR for 64 Byte frames -| | ... | using binary search start at 10GE linerate, step 50kpps, LT=0.5%. -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | ${64} -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 2 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find PDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} - -| tc09-1518B-2t2c-ethip4-ip4scale2m-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 2 phy cores, \ -| | ... | 1 receive queue per NIC port. [Ver] Find NDR for 1518 Byte frames -| | ... | using binary search start at 10GE linerate, step 50kpps. -| | [Tags] | 1518B | 2C | NDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | ${1518} -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 2 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find NDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} - -| tc10-1518B-2t2c-ethip4-ip4scale2m-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 2 phy cores, \ -| | ... | 1 receive queue per NIC port. [Ver] Find PDR for 1518 Byte frames -| | ... | using binary search start at 10GE linerate, step 50kpps, LT=0.5%. -| | [Tags] | 1518B | 2C | PDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | ${1518} -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 2 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find PDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} - -| tc11-9000B-2t2c-ethip4-ip4scale2m-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 2 phy cores, \ -| | ... | 1 receive queue per NIC port. [Ver] Find NDR for 9000 Byte frames -| | ... | using binary search start at 10GE linerate, step 10kpps. -| | [Tags] | 9000B | 2C | NDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | ${9000} -| | ${min_rate}= | Set Variable | ${10000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 2 -| | And Add PCI devices to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find NDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} - -| tc12-9000B-2t2c-ethip4-ip4scale2m-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 2 phy cores, \ -| | ... | 1 receive queue per NIC port. [Ver] Find PDR for 9000 Byte frames -| | ... | using binary search start at 10GE linerate, step 10kpps, LT=0.5%. -| | [Tags] | 9000B | 2C | PDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | ${9000} -| | ${min_rate}= | Set Variable | ${10000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 2 -| | And Add PCI devices to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find PDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} - -| tc13-64B-4t4c-ethip4-ip4scale2m-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 4 phy cores, \ -| | ... | 2 receive queues per NIC port. [Ver] Find NDR for 64 Byte frames -| | ... | using binary search start at 10GE linerate, step 50kpps. -| | [Tags] | 64B | 4C | NDRDISC -| | ${framesize}= | Set Variable | ${64} -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 4 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find NDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} - -| tc14-64B-4t4c-ethip4-ip4scale2m-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 4 phy cores, \ -| | ... | 2 receive queues per NIC port. [Ver] Find PDR for 64 Byte frames -| | ... | using binary search start at 10GE linerate, step 50kpps, LT=0.5%. -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | ${64} -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 4 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find PDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} - -| tc15-1518B-4t4c-ethip4-ip4scale2m-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 4 phy cores, \ -| | ... | 2 receive queues per NIC port. [Ver] Find NDR for 1518 Byte frames -| | ... | using binary search start at 10GE linerate, step 50kpps. -| | [Tags] | 1518B | 4C | NDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | ${1518} -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 4 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find NDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} - -| tc16-1518B-4t4c-ethip4-ip4scale2m-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 4 phy cores, \ -| | ... | 2 receive queues per NIC port. [Ver] Find PDR for 1518 Byte frames -| | ... | using binary search start at 10GE linerate, step 50kpps, LT=0.5%. -| | [Tags] | 1518B | 4C | PDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | ${1518} -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 4 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find PDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} - -| tc17-9000B-4t4c-ethip4-ip4scale2m-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 4 phy cores, \ -| | ... | 2 receive queues per NIC port. [Ver] Find NDR for 9000 Byte frames -| | ... | using binary search start at 10GE linerate, step 10kpps. -| | [Tags] | 9000B | 4C | NDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | ${9000} -| | ${min_rate}= | Set Variable | ${10000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 4 -| | And Add PCI devices to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find NDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} - -| tc18-9000B-4t4c-ethip4-ip4scale2m-pdrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 4 phy cores, \ -| | ... | 2 receive queues per NIC port. [Ver] Find PDR for 9000 Byte frames -| | ... | using binary search start at 10GE linerate, step 10kpps, LT=0.5%. -| | [Tags] | 9000B | 4C | PDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | ${9000} -| | ${min_rate}= | Set Variable | ${10000} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 4 -| | And Add PCI devices to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding with scaling in 3-node circular topology -| | ... | ${rts_per_flow} -| | Then Find PDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sf-100flows-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sf-100flows-mrr.robot new file mode 100644 index 0000000000..b3a1aef99a --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sf-100flows-mrr.robot @@ -0,0 +1,136 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | MRR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL +| ... | IACL | ACL10 | 100_FLOWS +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with ACL +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down mrr test with ACL +| ... +| Test Template | Local Template +| ... +| Documentation | *Raw results IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* In MaxReceivedRate test TG sends traffic +| ... | at line rate and reports total received/sent packets over trial period. +| ... | Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit+reflect +| ${acl_apply_type}= | input +| ${no_hit_aces_number}= | 10 +| ${flows_per_dir}= | 100 + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u10p-conc + +*** Keywords *** +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | Then Traffic should pass with maximum rate +| | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-iacl10sf-100flows-mrr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-iacl10sf-100flows-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-iacl10sf-100flows-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-iacl10sf-100flows-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-iacl10sf-100flows-mrr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-iacl10sf-100flows-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-iacl10sf-100flows-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-iacl10sf-100flows-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-iacl10sf-100flows-mrr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-iacl10sf-100flows-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-iacl10sf-100flows-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-iacl10sf-100flows-mrr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sf-100flows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sf-100flows-ndrpdr.robot index bff07ea2aa..28bdd66fd8 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sf-100flows-ndrpdrdisc.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sf-100flows-ndrpdr.robot @@ -1,4 +1,4 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. +# Copyright (c) 2018 Cisco and/or its affiliates. # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at: @@ -14,7 +14,7 @@ *** Settings *** | Resource | resources/libraries/robot/performance/performance_setup.robot | ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR | ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL | ... | IACL | ACL10 | 100_FLOWS | ... @@ -29,7 +29,7 @@ | Test Teardown | Tear down performance test with ACL | ... | ${min_rate}pps | ${framesize} | ${traffic_profile} | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* | ... @@ -44,9 +44,8 @@ | ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ | ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ | ... | of packets transmitted. NDR and PDR are discovered for different\ -| ... | Ethernet L2 frame sizes using either binary search or linear search\ -| ... | algorithms with configured starting rate and final step that determines\ -| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on\ | ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ | ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ | ... | all packets containing Ethernet header, IPv4 header with UDP header and\ @@ -74,57 +73,71 @@ | ${trex_stream1_subnet}= | 10.10.10.0/24 | ${trex_stream2_subnet}= | 20.20.20.0/24 +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u10p-conc + *** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | ${ip_nr}= | Set Variable | 10 | | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u10p-conc -| | Set Test Variable | ${traffic_profile} -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-iacl10-stateful-flows100-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR +| tc01-64B-1c-ethip4udp-ip4base-iacl10sf-100flows-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-iacl10sf-100flows-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-iacl10sf-100flows-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-iacl10sf-100flows-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-iacl10sf-100flows-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-iacl10sf-100flows-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-iacl10sf-100flows-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc02-64B-1t1c-ethip4udp-ip4base-iacl10-stateful-flows100-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR +| tc08-9000B-2c-ethip4udp-ip4base-iacl10sf-100flows-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc03-64B-2t2c-ethip4udp-ip4base-iacl10-stateful-flows100-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR +| tc09-9000B-4c-ethip4udp-ip4base-iacl10sf-100flows-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} -| tc04-64B-2t2c-ethip4udp-ip4base-iacl10-stateful-flows100-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR +| tc10-IMIX-1c-ethip4udp-ip4base-iacl10sf-100flows-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} -| tc05-64B-4t4c-ethip4udp-ip4base-iacl10-stateful-flows100-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR +| tc11-IMIX-2c-ethip4udp-ip4base-iacl10sf-100flows-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc06-64B-4t4c-ethip4udp-ip4base-iacl10-stateful-flows100-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR +| tc12-IMIX-4c-ethip4udp-ip4base-iacl10sf-100flows-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sf-100kflows-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sf-100kflows-mrr.robot new file mode 100644 index 0000000000..5e54ef3511 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sf-100kflows-mrr.robot @@ -0,0 +1,136 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | MRR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL +| ... | IACL | ACL10 | 100k_FLOWS +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with ACL +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down mrr test with ACL +| ... +| Test Template | Local Template +| ... +| Documentation | *Raw results IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* In MaxReceivedRate test TG sends traffic +| ... | at line rate and reports total received/sent packets over trial period. +| ... | Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit+reflect +| ${acl_apply_type}= | input +| ${no_hit_aces_number}= | 10 +| ${flows_per_dir}= | 100k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +| ${traffic_profile}= | trex-sl-3n-ethip4udp-100u1000p-conc + +*** Keywords *** +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 100 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | Then Traffic should pass with maximum rate +| | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-iacl10sf-100kflows-mrr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-iacl10sf-100kflows-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-iacl10sf-100kflows-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-iacl10sf-100kflows-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-iacl10sf-100kflows-mrr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-iacl10sf-100kflows-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-iacl10sf-100kflows-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-iacl10sf-100kflows-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-iacl10sf-100kflows-mrr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-iacl10sf-100kflows-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-iacl10sf-100kflows-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-iacl10sf-100kflows-mrr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sf-100kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sf-100kflows-ndrpdr.robot index 91170e6da3..3dc8dd9957 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sf-100kflows-ndrpdrdisc.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sf-100kflows-ndrpdr.robot @@ -1,4 +1,4 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. +# Copyright (c) 2018 Cisco and/or its affiliates. # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at: @@ -14,7 +14,7 @@ *** Settings *** | Resource | resources/libraries/robot/performance/performance_setup.robot | ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR | ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL | ... | IACL | ACL10 | 100k_FLOWS | ... @@ -29,7 +29,7 @@ | Test Teardown | Tear down performance test with ACL | ... | ${min_rate}pps | ${framesize} | ${traffic_profile} | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* | ... @@ -44,9 +44,8 @@ | ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ | ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ | ... | of packets transmitted. NDR and PDR are discovered for different\ -| ... | Ethernet L2 frame sizes using either binary search or linear search\ -| ... | algorithms with configured starting rate and final step that determines\ -| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on\ | ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ | ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ | ... | all packets containing Ethernet header, IPv4 header with UDP header and\ @@ -74,57 +73,71 @@ | ${trex_stream1_subnet}= | 10.10.10.0/24 | ${trex_stream2_subnet}= | 20.20.20.0/24 +| ${traffic_profile}= | trex-sl-3n-ethip4udp-100u1000p-conc + *** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | ${ip_nr}= | Set Variable | 100 | | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-100u1000p-conc -| | Set Test Variable | ${traffic_profile} -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-iacl10-stateful-flows100k-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR +| tc01-64B-1c-ethip4udp-ip4base-iacl10sf-100kflows-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-iacl10sf-100kflows-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-iacl10sf-100kflows-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-iacl10sf-100kflows-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-iacl10sf-100kflows-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-iacl10sf-100kflows-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-iacl10sf-100kflows-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc02-64B-1t1c-ethip4udp-ip4base-iacl10-stateful-flows100k-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR +| tc08-9000B-2c-ethip4udp-ip4base-iacl10sf-100kflows-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc03-64B-2t2c-ethip4udp-ip4base-iacl10-stateful-flows100k-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR +| tc09-9000B-4c-ethip4udp-ip4base-iacl10sf-100kflows-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} -| tc04-64B-2t2c-ethip4udp-ip4base-iacl10-stateful-flows100k-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR +| tc10-IMIX-1c-ethip4udp-ip4base-iacl10sf-100kflows-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} -| tc05-64B-4t4c-ethip4udp-ip4base-iacl10-stateful-flows100k-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR +| tc11-IMIX-2c-ethip4udp-ip4base-iacl10sf-100kflows-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc06-64B-4t4c-ethip4udp-ip4base-iacl10-stateful-flows100k-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR +| tc12-IMIX-4c-ethip4udp-ip4base-iacl10sf-100kflows-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sf-10kflows-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sf-10kflows-mrr.robot new file mode 100644 index 0000000000..46a507a270 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sf-10kflows-mrr.robot @@ -0,0 +1,136 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | MRR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL +| ... | IACL | ACL10 | 10k_FLOWS +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with ACL +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down mrr test with ACL +| ... +| Test Template | Local Template +| ... +| Documentation | *Raw results IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* In MaxReceivedRate test TG sends traffic +| ... | at line rate and reports total received/sent packets over trial period. +| ... | Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit+reflect +| ${acl_apply_type}= | input +| ${no_hit_aces_number}= | 10 +| ${flows_per_dir}= | 10k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u1000p-conc + +*** Keywords *** +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | Then Traffic should pass with maximum rate +| | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-iacl10sf-10kflows-mrr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-iacl10sf-10kflows-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-iacl10sf-10kflows-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-iacl10sf-10kflows-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-iacl10sf-10kflows-mrr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-iacl10sf-10kflows-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-iacl10sf-10kflows-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-iacl10sf-10kflows-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-iacl10sf-10kflows-mrr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-iacl10sf-10kflows-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-iacl10sf-10kflows-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-iacl10sf-10kflows-mrr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sf-10kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sf-10kflows-ndrpdr.robot index 1695a4dd06..68f236b9a2 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sf-10kflows-ndrpdrdisc.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sf-10kflows-ndrpdr.robot @@ -1,4 +1,4 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. +# Copyright (c) 2018 Cisco and/or its affiliates. # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at: @@ -14,7 +14,7 @@ *** Settings *** | Resource | resources/libraries/robot/performance/performance_setup.robot | ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR | ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL | ... | IACL | ACL10 | 10k_FLOWS | ... @@ -29,7 +29,7 @@ | Test Teardown | Tear down performance test with ACL | ... | ${min_rate}pps | ${framesize} | ${traffic_profile} | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* | ... @@ -44,9 +44,8 @@ | ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ | ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ | ... | of packets transmitted. NDR and PDR are discovered for different\ -| ... | Ethernet L2 frame sizes using either binary search or linear search\ -| ... | algorithms with configured starting rate and final step that determines\ -| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on\ | ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ | ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ | ... | all packets containing Ethernet header, IPv4 header with UDP header and\ @@ -74,57 +73,71 @@ | ${trex_stream1_subnet}= | 10.10.10.0/24 | ${trex_stream2_subnet}= | 20.20.20.0/24 +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u1000p-conc + *** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | ${ip_nr}= | Set Variable | 10 | | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u1000p-conc -| | Set Test Variable | ${traffic_profile} -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-iacl10-stateful-flows10k-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR +| tc01-64B-1c-ethip4udp-ip4base-iacl10sf-10kflows-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-iacl10sf-10kflows-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-iacl10sf-10kflows-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-iacl10sf-10kflows-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-iacl10sf-10kflows-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-iacl10sf-10kflows-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-iacl10sf-10kflows-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc02-64B-1t1c-ethip4udp-ip4base-iacl10-stateful-flows10k-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR +| tc08-9000B-2c-ethip4udp-ip4base-iacl10sf-10kflows-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc03-64B-2t2c-ethip4udp-ip4base-iacl10-stateful-flows10k-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR +| tc09-9000B-4c-ethip4udp-ip4base-iacl10sf-10kflows-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} -| tc04-64B-2t2c-ethip4udp-ip4base-iacl10-stateful-flows10k-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR +| tc10-IMIX-1c-ethip4udp-ip4base-iacl10sf-10kflows-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} -| tc05-64B-4t4c-ethip4udp-ip4base-iacl10-stateful-flows10k-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR +| tc11-IMIX-2c-ethip4udp-ip4base-iacl10sf-10kflows-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc06-64B-4t4c-ethip4udp-ip4base-iacl10-stateful-flows10k-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR +| tc12-IMIX-4c-ethip4udp-ip4base-iacl10sf-10kflows-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sl-100flows-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sl-100flows-mrr.robot new file mode 100644 index 0000000000..d567e3cd43 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sl-100flows-mrr.robot @@ -0,0 +1,136 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | MRR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS +| ... | IACL | ACL10 | 100_FLOWS +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with ACL +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down mrr test with ACL +| ... +| Test Template | Local Template +| ... +| Documentation | *Raw results IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* In MaxReceivedRate test TG sends traffic +| ... | at line rate and reports total received/sent packets over trial period. +| ... | Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit +| ${acl_apply_type}= | input +| ${no_hit_aces_number}= | 10 +| ${flows_per_dir}= | 100 + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u10p-conc + +*** Keywords *** +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | Then Traffic should pass with maximum rate +| | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-iacl10sl-100flows-mrr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-iacl10sl-100flows-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-iacl10sl-100flows-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-iacl10sl-100flows-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-iacl10sl-100flows-mrr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-iacl10sl-100flows-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-iacl10sl-100flows-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-iacl10sl-100flows-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-iacl10sl-100flows-mrr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-iacl10sl-100flows-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-iacl10sl-100flows-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-iacl10sl-100flows-mrr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sl-100flows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sl-100flows-ndrpdr.robot index aca5177542..0c9b0bd213 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sl-100flows-ndrpdrdisc.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sl-100flows-ndrpdr.robot @@ -1,4 +1,4 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. +# Copyright (c) 2018 Cisco and/or its affiliates. # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at: @@ -14,7 +14,7 @@ *** Settings *** | Resource | resources/libraries/robot/performance/performance_setup.robot | ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR | ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS | ... | IACL | ACL10 | 100_FLOWS | ... @@ -29,7 +29,7 @@ | Test Teardown | Tear down performance test with ACL | ... | ${min_rate}pps | ${framesize} | ${traffic_profile} | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* | ... @@ -44,9 +44,8 @@ | ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ | ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ | ... | of packets transmitted. NDR and PDR are discovered for different\ -| ... | Ethernet L2 frame sizes using either binary search or linear search\ -| ... | algorithms with configured starting rate and final step that determines\ -| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on\ | ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ | ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ | ... | all packets containing Ethernet header, IPv4 header with UDP header and\ @@ -74,57 +73,71 @@ | ${trex_stream1_subnet}= | 10.10.10.0/24 | ${trex_stream2_subnet}= | 20.20.20.0/24 +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u10p-conc + *** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | ${ip_nr}= | Set Variable | 10 | | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u10p-conc -| | Set Test Variable | ${traffic_profile} -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-iacl10-stateless-flows100-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR +| tc01-64B-1c-ethip4udp-ip4base-iacl10sl-100flows-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-iacl10sl-100flows-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-iacl10sl-100flows-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-iacl10sl-100flows-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-iacl10sl-100flows-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-iacl10sl-100flows-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-iacl10sl-100flows-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc02-64B-1t1c-ethip4udp-ip4base-iacl10-stateless-flows100-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR +| tc08-9000B-2c-ethip4udp-ip4base-iacl10sl-100flows-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc03-64B-2t2c-ethip4udp-ip4base-iacl10-stateless-flows100-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR +| tc09-9000B-4c-ethip4udp-ip4base-iacl10sl-100flows-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} -| tc04-64B-2t2c-ethip4udp-ip4base-iacl10-stateless-flows100-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR +| tc10-IMIX-1c-ethip4udp-ip4base-iacl10sl-100flows-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} -| tc05-64B-4t4c-ethip4udp-ip4base-iacl10-stateless-flows100-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR +| tc11-IMIX-2c-ethip4udp-ip4base-iacl10sl-100flows-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc06-64B-4t4c-ethip4udp-ip4base-iacl10-stateless-flows100-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR +| tc12-IMIX-4c-ethip4udp-ip4base-iacl10sl-100flows-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sl-100kflows-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sl-100kflows-mrr.robot new file mode 100644 index 0000000000..a1e9ef8f3f --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sl-100kflows-mrr.robot @@ -0,0 +1,136 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | MRR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS +| ... | IACL | ACL10 | 100k_FLOWS +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with ACL +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down mrr test with ACL +| ... +| Test Template | Local Template +| ... +| Documentation | *Raw results IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* In MaxReceivedRate test TG sends traffic +| ... | at line rate and reports total received/sent packets over trial period. +| ... | Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit +| ${acl_apply_type}= | input +| ${no_hit_aces_number}= | 10 +| ${flows_per_dir}= | 100k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +| ${traffic_profile}= | trex-sl-3n-ethip4udp-100u1000p-conc + +*** Keywords *** +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 100 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | Then Traffic should pass with maximum rate +| | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-iacl10sl-100kflows-mrr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-iacl10sl-100kflows-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-iacl10sl-100kflows-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-iacl10sl-100kflows-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-iacl10sl-100kflows-mrr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-iacl10sl-100kflows-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-iacl10sl-100kflows-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-iacl10sl-100kflows-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-iacl10sl-100kflows-mrr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-iacl10sl-100kflows-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-iacl10sl-100kflows-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-iacl10sl-100kflows-mrr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sl-100kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sl-100kflows-ndrpdr.robot index 53f609874f..19657b49fa 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sl-100kflows-ndrpdrdisc.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sl-100kflows-ndrpdr.robot @@ -1,4 +1,4 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. +# Copyright (c) 2018 Cisco and/or its affiliates. # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at: @@ -14,7 +14,7 @@ *** Settings *** | Resource | resources/libraries/robot/performance/performance_setup.robot | ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR | ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS | ... | IACL | ACL10 | 100k_FLOWS | ... @@ -29,7 +29,7 @@ | Test Teardown | Tear down performance test with ACL | ... | ${min_rate}pps | ${framesize} | ${traffic_profile} | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* | ... @@ -44,9 +44,8 @@ | ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ | ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ | ... | of packets transmitted. NDR and PDR are discovered for different\ -| ... | Ethernet L2 frame sizes using either binary search or linear search\ -| ... | algorithms with configured starting rate and final step that determines\ -| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on\ | ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ | ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ | ... | all packets containing Ethernet header, IPv4 header with UDP header and\ @@ -74,57 +73,71 @@ | ${trex_stream1_subnet}= | 10.10.10.0/24 | ${trex_stream2_subnet}= | 20.20.20.0/24 +| ${traffic_profile}= | trex-sl-3n-ethip4udp-100u1000p-conc + *** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | ${ip_nr}= | Set Variable | 100 | | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-100u1000p-conc -| | Set Test Variable | ${traffic_profile} -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-iacl10-stateless-flows100k-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR +| tc01-64B-1c-ethip4udp-ip4base-iacl10sl-100kflows-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-iacl10sl-100kflows-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-iacl10sl-100kflows-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-iacl10sl-100kflows-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-iacl10sl-100kflows-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-iacl10sl-100kflows-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-iacl10sl-100kflows-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc02-64B-1t1c-ethip4udp-ip4base-iacl10-stateless-flows100k-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR +| tc08-9000B-2c-ethip4udp-ip4base-iacl10sl-100kflows-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc03-64B-2t2c-ethip4udp-ip4base-iacl10-stateless-flows100k-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR +| tc09-9000B-4c-ethip4udp-ip4base-iacl10sl-100kflows-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} -| tc04-64B-2t2c-ethip4udp-ip4base-iacl10-stateless-flows100k-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR +| tc10-IMIX-1c-ethip4udp-ip4base-iacl10sl-100kflows-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} -| tc05-64B-4t4c-ethip4udp-ip4base-iacl10-stateless-flows100k-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR +| tc11-IMIX-2c-ethip4udp-ip4base-iacl10sl-100kflows-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc06-64B-4t4c-ethip4udp-ip4base-iacl10-stateless-flows100k-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR +| tc12-IMIX-4c-ethip4udp-ip4base-iacl10sl-100kflows-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sl-10kflows-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sl-10kflows-mrr.robot new file mode 100644 index 0000000000..e778a47f25 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sl-10kflows-mrr.robot @@ -0,0 +1,136 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | MRR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS +| ... | IACL | ACL10 | 10k_FLOWS +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with ACL +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down mrr test with ACL +| ... +| Test Template | Local Template +| ... +| Documentation | *Raw results IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* In MaxReceivedRate test TG sends traffic +| ... | at line rate and reports total received/sent packets over trial period. +| ... | Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit +| ${acl_apply_type}= | input +| ${no_hit_aces_number}= | 10 +| ${flows_per_dir}= | 10k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u1000p-conc + +*** Keywords *** +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | Then Traffic should pass with maximum rate +| | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-iacl10sl-10kflows-mrr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-iacl10sl-10kflows-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-iacl10sl-10kflows-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-iacl10sl-10kflows-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-iacl10sl-10kflows-mrr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-iacl10sl-10kflows-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-iacl10sl-10kflows-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-iacl10sl-10kflows-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-iacl10sl-10kflows-mrr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-iacl10sl-10kflows-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-iacl10sl-10kflows-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-iacl10sl-10kflows-mrr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sl-10kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sl-10kflows-ndrpdr.robot index 04f691dbef..2e26f5be3d 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sl-10kflows-ndrpdrdisc.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl10sl-10kflows-ndrpdr.robot @@ -1,4 +1,4 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. +# Copyright (c) 2018 Cisco and/or its affiliates. # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at: @@ -14,7 +14,7 @@ *** Settings *** | Resource | resources/libraries/robot/performance/performance_setup.robot | ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR | ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS | ... | IACL | ACL10 | 10k_FLOWS | ... @@ -29,7 +29,7 @@ | Test Teardown | Tear down performance test with ACL | ... | ${min_rate}pps | ${framesize} | ${traffic_profile} | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* | ... @@ -44,9 +44,8 @@ | ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ | ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ | ... | of packets transmitted. NDR and PDR are discovered for different\ -| ... | Ethernet L2 frame sizes using either binary search or linear search\ -| ... | algorithms with configured starting rate and final step that determines\ -| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on\ | ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ | ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ | ... | all packets containing Ethernet header, IPv4 header with UDP header and\ @@ -74,57 +73,71 @@ | ${trex_stream1_subnet}= | 10.10.10.0/24 | ${trex_stream2_subnet}= | 20.20.20.0/24 +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u1000p-conc + *** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | ${ip_nr}= | Set Variable | 10 | | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u1000p-conc -| | Set Test Variable | ${traffic_profile} -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-iacl10-stateless-flows10k-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR +| tc01-64B-1c-ethip4udp-ip4base-iacl10sl-10kflows-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-iacl10sl-10kflows-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-iacl10sl-10kflows-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-iacl10sl-10kflows-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-iacl10sl-10kflows-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-iacl10sl-10kflows-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-iacl10sl-10kflows-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc02-64B-1t1c-ethip4udp-ip4base-iacl10-stateless-flows10k-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR +| tc08-9000B-2c-ethip4udp-ip4base-iacl10sl-10kflows-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc03-64B-2t2c-ethip4udp-ip4base-iacl10-stateless-flows10k-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR +| tc09-9000B-4c-ethip4udp-ip4base-iacl10sl-10kflows-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} -| tc04-64B-2t2c-ethip4udp-ip4base-iacl10-stateless-flows10k-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR +| tc10-IMIX-1c-ethip4udp-ip4base-iacl10sl-10kflows-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} -| tc05-64B-4t4c-ethip4udp-ip4base-iacl10-stateless-flows10k-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR +| tc11-IMIX-2c-ethip4udp-ip4base-iacl10sl-10kflows-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc06-64B-4t4c-ethip4udp-ip4base-iacl10-stateless-flows10k-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR +| tc12-IMIX-4c-ethip4udp-ip4base-iacl10sl-10kflows-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sf-100flows-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sf-100flows-mrr.robot new file mode 100644 index 0000000000..21e03b98a3 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sf-100flows-mrr.robot @@ -0,0 +1,136 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | MRR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL +| ... | IACL | ACL1 | 100_FLOWS +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with ACL +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down mrr test with ACL +| ... +| Test Template | Local Template +| ... +| Documentation | *Raw results IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* In MaxReceivedRate test TG sends traffic +| ... | at line rate and reports total received/sent packets over trial period. +| ... | Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit+reflect +| ${acl_apply_type}= | input +| ${no_hit_aces_number}= | 1 +| ${flows_per_dir}= | 100 + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u10p-conc + +*** Keywords *** +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | Then Traffic should pass with maximum rate +| | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-iacl1sf-100flows-mrr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-iacl1sf-100flows-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-iacl1sf-100flows-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-iacl1sf-100flows-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-iacl1sf-100flows-mrr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-iacl1sf-100flows-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-iacl1sf-100flows-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-iacl1sf-100flows-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-iacl1sf-100flows-mrr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-iacl1sf-100flows-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-iacl1sf-100flows-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-iacl1sf-100flows-mrr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sf-100flows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sf-100flows-ndrpdr.robot index 629fc3baa4..2ec8f1c1f7 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sf-100flows-ndrpdrdisc.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sf-100flows-ndrpdr.robot @@ -1,4 +1,4 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. +# Copyright (c) 2018 Cisco and/or its affiliates. # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at: @@ -14,7 +14,7 @@ *** Settings *** | Resource | resources/libraries/robot/performance/performance_setup.robot | ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR | ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL | ... | IACL | ACL1 | 100_FLOWS | ... @@ -29,7 +29,7 @@ | Test Teardown | Tear down performance test with ACL | ... | ${min_rate}pps | ${framesize} | ${traffic_profile} | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* | ... @@ -44,9 +44,8 @@ | ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ | ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ | ... | of packets transmitted. NDR and PDR are discovered for different\ -| ... | Ethernet L2 frame sizes using either binary search or linear search\ -| ... | algorithms with configured starting rate and final step that determines\ -| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on\ | ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ | ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ | ... | all packets containing Ethernet header, IPv4 header with UDP header and\ @@ -74,57 +73,71 @@ | ${trex_stream1_subnet}= | 10.10.10.0/24 | ${trex_stream2_subnet}= | 20.20.20.0/24 +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u10p-conc + *** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | ${ip_nr}= | Set Variable | 10 | | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u10p-conc -| | Set Test Variable | ${traffic_profile} -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-iacl1-stateful-flows100-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR +| tc01-64B-1c-ethip4udp-ip4base-iacl1sf-100flows-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-iacl1sf-100flows-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-iacl1sf-100flows-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-iacl1sf-100flows-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-iacl1sf-100flows-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-iacl1sf-100flows-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-iacl1sf-100flows-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc02-64B-1t1c-ethip4udp-ip4base-iacl1-stateful-flows100-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR +| tc08-9000B-2c-ethip4udp-ip4base-iacl1sf-100flows-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc03-64B-2t2c-ethip4udp-ip4base-iacl1-stateful-flows100-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR +| tc09-9000B-4c-ethip4udp-ip4base-iacl1sf-100flows-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} -| tc04-64B-2t2c-ethip4udp-ip4base-iacl1-stateful-flows100-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR +| tc10-IMIX-1c-ethip4udp-ip4base-iacl1sf-100flows-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} -| tc05-64B-4t4c-ethip4udp-ip4base-iacl1-stateful-flows100-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR +| tc11-IMIX-2c-ethip4udp-ip4base-iacl1sf-100flows-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc06-64B-4t4c-ethip4udp-ip4base-iacl1-stateful-flows100-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR +| tc12-IMIX-4c-ethip4udp-ip4base-iacl1sf-100flows-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sf-100kflows-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sf-100kflows-mrr.robot new file mode 100644 index 0000000000..132e3ed260 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sf-100kflows-mrr.robot @@ -0,0 +1,136 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | MRR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL +| ... | IACL | ACL1 | 100k_FLOWS +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with ACL +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down mrr test with ACL +| ... +| Test Template | Local Template +| ... +| Documentation | *Raw results IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* In MaxReceivedRate test TG sends traffic +| ... | at line rate and reports total received/sent packets over trial period. +| ... | Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit+reflect +| ${acl_apply_type}= | input +| ${no_hit_aces_number}= | 1 +| ${flows_per_dir}= | 100k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +| ${traffic_profile}= | trex-sl-3n-ethip4udp-100u1000p-conc + +*** Keywords *** +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 100 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | Then Traffic should pass with maximum rate +| | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-iacl1sf-100kflows-mrr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-iacl1sf-100kflows-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-iacl1sf-100kflows-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-iacl1sf-100kflows-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-iacl1sf-100kflows-mrr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-iacl1sf-100kflows-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-iacl1sf-100kflows-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-iacl1sf-100kflows-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-iacl1sf-100kflows-mrr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-iacl1sf-100kflows-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-iacl1sf-100kflows-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-iacl1sf-100kflows-mrr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sf-100kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sf-100kflows-ndrpdr.robot index 2568f5237e..f66baff435 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sf-100kflows-ndrpdrdisc.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sf-100kflows-ndrpdr.robot @@ -1,4 +1,4 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. +# Copyright (c) 2018 Cisco and/or its affiliates. # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at: @@ -14,7 +14,7 @@ *** Settings *** | Resource | resources/libraries/robot/performance/performance_setup.robot | ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR | ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL | ... | IACL | ACL1 | 100k_FLOWS | ... @@ -29,7 +29,7 @@ | Test Teardown | Tear down performance test with ACL | ... | ${min_rate}pps | ${framesize} | ${traffic_profile} | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* | ... @@ -44,9 +44,8 @@ | ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ | ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ | ... | of packets transmitted. NDR and PDR are discovered for different\ -| ... | Ethernet L2 frame sizes using either binary search or linear search\ -| ... | algorithms with configured starting rate and final step that determines\ -| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on\ | ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ | ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ | ... | all packets containing Ethernet header, IPv4 header with UDP header and\ @@ -74,57 +73,71 @@ | ${trex_stream1_subnet}= | 10.10.10.0/24 | ${trex_stream2_subnet}= | 20.20.20.0/24 +| ${traffic_profile}= | trex-sl-3n-ethip4udp-100u1000p-conc + *** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | ${ip_nr}= | Set Variable | 100 | | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-100u1000p-conc -| | Set Test Variable | ${traffic_profile} -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-iacl1-stateful-flows100k-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR +| tc01-64B-1c-ethip4udp-ip4base-iacl1sf-100kflows-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-iacl1sf-100kflows-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-iacl1sf-100kflows-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-iacl1sf-100kflows-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-iacl1sf-100kflows-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-iacl1sf-100kflows-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-iacl1sf-100kflows-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc02-64B-1t1c-ethip4udp-ip4base-iacl1-stateful-flows100k-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR +| tc08-9000B-2c-ethip4udp-ip4base-iacl1sf-100kflows-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc03-64B-2t2c-ethip4udp-ip4base-iacl1-stateful-flows100k-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR +| tc09-9000B-4c-ethip4udp-ip4base-iacl1sf-100kflows-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} -| tc04-64B-2t2c-ethip4udp-ip4base-iacl1-stateful-flows100k-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR +| tc10-IMIX-1c-ethip4udp-ip4base-iacl1sf-100kflows-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} -| tc05-64B-4t4c-ethip4udp-ip4base-iacl1-stateful-flows100k-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR +| tc11-IMIX-2c-ethip4udp-ip4base-iacl1sf-100kflows-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc06-64B-4t4c-ethip4udp-ip4base-iacl1-stateful-flows100k-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR +| tc12-IMIX-4c-ethip4udp-ip4base-iacl1sf-100kflows-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sf-10kflows-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sf-10kflows-mrr.robot index 1d048bc35d..9553789d58 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sf-10kflows-mrr.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sf-10kflows-mrr.robot @@ -28,7 +28,7 @@ | ... | Test Teardown | Tear down performance mrr test | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *Raw results IPv4 test cases with ACL* | ... @@ -68,8 +68,10 @@ | ${trex_stream1_subnet}= | 10.10.10.0/24 | ${trex_stream2_subnet}= | 20.20.20.0/24 +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u1000p-conc + *** Keywords *** -| Local template +| Local Template | | [Documentation] | | ... | [Cfg] DUT runs IPv4 routing config with ACL. | | ... | Each DUT uses ${phy_cores} physical core(s) for worker threads. @@ -82,69 +84,63 @@ | | ... | - phy_cores - Number of physical cores. Type: integer | | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer | | ... -| | [Arguments] | ${phy_cores} | ${framesize} | ${rxq}=${None} -| | ... -| | Set Test Variable | ${framesize} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | ${ip_nr}= | Set Variable | 10 | | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u1000p-conc -| | Set Test Variable | ${traffic_profile} | | Then Traffic should pass with maximum rate | | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-iacl1-stateful-flows10k-mrr +| tc01-64B-1c-ethip4udp-ip4base-iacl1sf-10kflows-mrr | | [Tags] | 64B | 1C -| | phy_cores=${1} | framesize=${64} - -| tc02-1518B-1t1c-ethip4udp-ip4base-iacl1-stateful-flows10k-mrr -| | [Tags] | 1518B | 1C -| | phy_cores=${1} | framesize=${1518} +| | framesize=${64} | phy_cores=${1} -| tc03-9000B-1t1c-ethip4udp-ip4base-iacl1-stateful-flows10k-mrr -| | [Tags] | 9000B | 1C -| | phy_cores=${1} | framesize=${9000} +| tc02-64B-2c-ethip4udp-ip4base-iacl1sf-10kflows-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} -| tc04-IMIX-1t1c-ethip4udp-ip4base-iacl1-stateful-flows10k-mrr -| | [Tags] | IMIX | 1C -| | phy_cores=${1} | framesize=IMIX_v4_1 +| tc03-64B-4c-ethip4udp-ip4base-iacl1sf-10kflows-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} -| tc05-64B-2t2c-ethip4udp-ip4base-iacl1-stateful-flows10k-mrr -| | [Tags] | 64B | 2C -| | phy_cores=${2} | framesize=${64} +| tc04-1518B-1c-ethip4udp-ip4base-iacl1sf-10kflows-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} -| tc06-1518B-2t2c-ethip4udp-ip4base-iacl1-stateful-flows10k-mrr +| tc05-1518B-2c-ethip4udp-ip4base-iacl1sf-10kflows-mrr | | [Tags] | 1518B | 2C -| | phy_cores=${2} | framesize=${1518} - -| tc07-9000B-2t2c-ethip4udp-ip4base-iacl1-stateful-flows10k-mrr -| | [Tags] | 9000B | 2C -| | phy_cores=${2} | framesize=${9000} +| | framesize=${1518} | phy_cores=${2} -| tc08-IMIX-2t2c-ethip4udp-ip4base-iacl1-stateful-flows10k-mrr -| | [Tags] | IMIX | 2C -| | phy_cores=${2} | framesize=IMIX_v4_1 +| tc06-1518B-4c-ethip4udp-ip4base-iacl1sf-10kflows-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} -| tc09-64B-4t4c-ethip4udp-ip4base-iacl1-stateful-flows10k-mrr -| | [Tags] | 64B | 4C -| | phy_cores=${4} | framesize=${64} +| tc07-9000B-1c-ethip4udp-ip4base-iacl1sf-10kflows-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc10-1518B-4t4c-ethip4udp-ip4base-iacl1-stateful-flows10k-mrr -| | [Tags] | 1518B | 4C -| | phy_cores=${4} | framesize=${1518} +| tc08-9000B-2c-ethip4udp-ip4base-iacl1sf-10kflows-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc11-9000B-4t4c-ethip4udp-ip4base-iacl1-stateful-flows10k-mrr +| tc09-9000B-4c-ethip4udp-ip4base-iacl1sf-10kflows-mrr | | [Tags] | 9000B | 4C -| | phy_cores=${4} | framesize=${9000} +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-iacl1sf-10kflows-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-iacl1sf-10kflows-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc12-IMIX-4t4c-ethip4udp-ip4base-iacl1-stateful-flows10k-mrr +| tc12-IMIX-4c-ethip4udp-ip4base-iacl1sf-10kflows-mrr | | [Tags] | IMIX | 4C -| | phy_cores=${4} | framesize=IMIX_v4_1 +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sf-10kflows-ndrpdr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sf-10kflows-ndrpdr.robot new file mode 100644 index 0000000000..cc555eb44b --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sf-10kflows-ndrpdr.robot @@ -0,0 +1,152 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL +| ... | IACL | ACL1 | 10k_FLOWS +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with ACL +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down performance discovery test | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Test Template | Local Template +| ... +| Documentation | *RFC2544: Pkt throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage +| ... | of packets transmitted. NDR and PDR are discovered for different +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on links to DUTs. TG traffic profile +| ... | contains two L3 flow-groups (flow-group per direction, 253 flows per +| ... | flow-group) with all packets containing Ethernet header, IPv4 header +| ... | with IP protocol=61 and static payload. MAC addresses are matching MAC +| ... | addresses of the TG node interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit+reflect +| ${acl_apply_type}= | input +| ${no_hit_aces_number}= | 1 +| ${flows_per_dir}= | 10k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u1000p-conc + +*** Keywords *** +| Local Template +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL. +| | ... | Each DUT uses ${phy_cores} physical core(s) for worker threads. +| | ... | [Ver] Measure NDR and PDR values using MLRsearch algorithm.\ +| | ... +| | ... | *Arguments:* +| | ... | - framesize - Framesize in Bytes in integer or string (IMIX_v4_1). +| | ... | Type: integer, string +| | ... | - phy_cores - Number of physical cores. Type: integer +| | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer +| | ... +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-iacl1sf-10kflows-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-iacl1sf-10kflows-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-iacl1sf-10kflows-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-iacl1sf-10kflows-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-iacl1sf-10kflows-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-iacl1sf-10kflows-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-iacl1sf-10kflows-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-iacl1sf-10kflows-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-iacl1sf-10kflows-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-iacl1sf-10kflows-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-iacl1sf-10kflows-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-iacl1sf-10kflows-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sf-10kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sf-10kflows-ndrpdrdisc.robot deleted file mode 100644 index ec6c2a5865..0000000000 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sf-10kflows-ndrpdrdisc.robot +++ /dev/null @@ -1,130 +0,0 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at: -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -*** Settings *** -| Resource | resources/libraries/robot/performance/performance_setup.robot -| ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC -| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL -| ... | IACL | ACL1 | 10k_FLOWS -| ... -| Suite Setup | Run Keywords -| ... | Set up 3-node performance topology with DUT's NIC model | L3 -| ... | Intel-X520-DA2 -| ... | AND | Set up performance test suite with ACL -| Suite Teardown | Tear down 3-node performance topology -| ... -| Test Setup | Set up performance test -| ... -| Test Teardown | Tear down performance test with ACL -| ... | ${min_rate}pps | ${framesize} | ${traffic_profile} -| ... -| Test Template | Local template -| ... -| Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* -| ... -| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ -| ... | with single links between nodes. -| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. -| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ -| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ -| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ -| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ -| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\ -| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ -| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ -| ... | of packets transmitted. NDR and PDR are discovered for different\ -| ... | Ethernet L2 frame sizes using either binary search or linear search\ -| ... | algorithms with configured starting rate and final step that determines\ -| ... | throughput measurement resolution. Test packets are generated by TG on\ -| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ -| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ -| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ -| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ -| ... | interfaces. -| ... | *[Ref] Applicable standard specifications:* RFC2544. - -*** Variables *** -# X520-DA2 bandwidth limit -| ${s_limit}= | ${10000000000} - -# ACL test setup -| ${acl_action}= | permit+reflect -| ${acl_apply_type}= | input -| ${no_hit_aces_number}= | 1 -| ${flows_per_dir}= | 10k - -# starting points for non-hitting ACLs -| ${src_ip_start}= | 30.30.30.1 -| ${dst_ip_start}= | 40.40.40.1 -| ${ip_step}= | ${1} -| ${sport_start}= | ${1000} -| ${dport_start}= | ${1000} -| ${port_step}= | ${1} -| ${trex_stream1_subnet}= | 10.10.10.0/24 -| ${trex_stream2_subnet}= | 20.20.20.0/24 - -*** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} -| | ... -| | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} -| | And Add PCI devices to all DUTs -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | ${ip_nr}= | Set Variable | 10 -| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u1000p-conc -| | Set Test Variable | ${traffic_profile} -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} - -*** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-iacl1-stateful-flows10k-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR - -| tc02-64B-1t1c-ethip4udp-ip4base-iacl1-stateful-flows10k-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR - -| tc03-64B-2t2c-ethip4udp-ip4base-iacl1-stateful-flows10k-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR - -| tc04-64B-2t2c-ethip4udp-ip4base-iacl1-stateful-flows10k-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR - -| tc05-64B-4t4c-ethip4udp-ip4base-iacl1-stateful-flows10k-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR - -| tc06-64B-4t4c-ethip4udp-ip4base-iacl1-stateful-flows10k-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sl-100flows-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sl-100flows-mrr.robot new file mode 100644 index 0000000000..2d82ff8d1e --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sl-100flows-mrr.robot @@ -0,0 +1,136 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | MRR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS +| ... | IACL | ACL1 | 100_FLOWS +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with ACL +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down mrr test with ACL +| ... +| Test Template | Local Template +| ... +| Documentation | *Raw results IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* In MaxReceivedRate test TG sends traffic +| ... | at line rate and reports total received/sent packets over trial period. +| ... | Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit +| ${acl_apply_type}= | input +| ${no_hit_aces_number}= | 1 +| ${flows_per_dir}= | 100 + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u10p-conc + +*** Keywords *** +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | Then Traffic should pass with maximum rate +| | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-iacl1sl-100flows-mrr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-iacl1sl-100flows-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-iacl1sl-100flows-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-iacl1sl-100flows-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-iacl1sl-100flows-mrr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-iacl1sl-100flows-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-iacl1sl-100flows-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-iacl1sl-100flows-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-iacl1sl-100flows-mrr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-iacl1sl-100flows-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-iacl1sl-100flows-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-iacl1sl-100flows-mrr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sl-100flows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sl-100flows-ndrpdr.robot index 3b43f90b6f..e9b16564fa 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sl-100flows-ndrpdrdisc.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sl-100flows-ndrpdr.robot @@ -1,4 +1,4 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. +# Copyright (c) 2018 Cisco and/or its affiliates. # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at: @@ -14,7 +14,7 @@ *** Settings *** | Resource | resources/libraries/robot/performance/performance_setup.robot | ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR | ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS | ... | IACL | ACL1 | 100_FLOWS | ... @@ -29,7 +29,7 @@ | Test Teardown | Tear down performance test with ACL | ... | ${min_rate}pps | ${framesize} | ${traffic_profile} | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* | ... @@ -44,9 +44,8 @@ | ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ | ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ | ... | of packets transmitted. NDR and PDR are discovered for different\ -| ... | Ethernet L2 frame sizes using either binary search or linear search\ -| ... | algorithms with configured starting rate and final step that determines\ -| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on\ | ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ | ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ | ... | all packets containing Ethernet header, IPv4 header with UDP header and\ @@ -74,57 +73,71 @@ | ${trex_stream1_subnet}= | 10.10.10.0/24 | ${trex_stream2_subnet}= | 20.20.20.0/24 +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u10p-conc + *** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | ${ip_nr}= | Set Variable | 10 | | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u10p-conc -| | Set Test Variable | ${traffic_profile} -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-iacl1-stateless-flows100-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR +| tc01-64B-1c-ethip4udp-ip4base-iacl1sl-100flows-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-iacl1sl-100flows-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-iacl1sl-100flows-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-iacl1sl-100flows-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-iacl1sl-100flows-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-iacl1sl-100flows-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-iacl1sl-100flows-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc02-64B-1t1c-ethip4udp-ip4base-iacl1-stateless-flows100-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR +| tc08-9000B-2c-ethip4udp-ip4base-iacl1sl-100flows-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc03-64B-2t2c-ethip4udp-ip4base-iacl1-stateless-flows100-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR +| tc09-9000B-4c-ethip4udp-ip4base-iacl1sl-100flows-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} -| tc04-64B-2t2c-ethip4udp-ip4base-iacl1-stateless-flows100-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR +| tc10-IMIX-1c-ethip4udp-ip4base-iacl1sl-100flows-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} -| tc05-64B-4t4c-ethip4udp-ip4base-iacl1-stateless-flows100-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR +| tc11-IMIX-2c-ethip4udp-ip4base-iacl1sl-100flows-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc06-64B-4t4c-ethip4udp-ip4base-iacl1-stateless-flows100-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR +| tc12-IMIX-4c-ethip4udp-ip4base-iacl1sl-100flows-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sl-100kflows-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sl-100kflows-mrr.robot new file mode 100644 index 0000000000..0ee32eff23 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sl-100kflows-mrr.robot @@ -0,0 +1,136 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | MRR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS +| ... | IACL | ACL1 | 100k_FLOWS +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with ACL +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down mrr test with ACL +| ... +| Test Template | Local Template +| ... +| Documentation | *Raw results IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* In MaxReceivedRate test TG sends traffic +| ... | at line rate and reports total received/sent packets over trial period. +| ... | Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit +| ${acl_apply_type}= | input +| ${no_hit_aces_number}= | 1 +| ${flows_per_dir}= | 100k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +| ${traffic_profile}= | trex-sl-3n-ethip4udp-100u1000p-conc + +*** Keywords *** +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 100 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | Then Traffic should pass with maximum rate +| | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-iacl1sl-100kflows-mrr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-iacl1sl-100kflows-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-iacl1sl-100kflows-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-iacl1sl-100kflows-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-iacl1sl-100kflows-mrr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-iacl1sl-100kflows-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-iacl1sl-100kflows-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-iacl1sl-100kflows-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-iacl1sl-100kflows-mrr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-iacl1sl-100kflows-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-iacl1sl-100kflows-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-iacl1sl-100kflows-mrr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sl-100kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sl-100kflows-ndrpdr.robot index d76e4ee792..6b775a5664 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sl-100kflows-ndrpdrdisc.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sl-100kflows-ndrpdr.robot @@ -1,4 +1,4 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. +# Copyright (c) 2018 Cisco and/or its affiliates. # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at: @@ -14,7 +14,7 @@ *** Settings *** | Resource | resources/libraries/robot/performance/performance_setup.robot | ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR | ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS | ... | IACL | ACL1 | 100k_FLOWS | ... @@ -29,7 +29,7 @@ | Test Teardown | Tear down performance test with ACL | ... | ${min_rate}pps | ${framesize} | ${traffic_profile} | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* | ... @@ -44,9 +44,8 @@ | ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ | ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ | ... | of packets transmitted. NDR and PDR are discovered for different\ -| ... | Ethernet L2 frame sizes using either binary search or linear search\ -| ... | algorithms with configured starting rate and final step that determines\ -| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on\ | ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ | ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ | ... | all packets containing Ethernet header, IPv4 header with UDP header and\ @@ -74,57 +73,71 @@ | ${trex_stream1_subnet}= | 10.10.10.0/24 | ${trex_stream2_subnet}= | 20.20.20.0/24 +| ${traffic_profile}= | trex-sl-3n-ethip4udp-100u1000p-conc + *** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | ${ip_nr}= | Set Variable | 100 | | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-100u1000p-conc -| | Set Test Variable | ${traffic_profile} -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-iacl1-stateless-flows100k-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR +| tc01-64B-1c-ethip4udp-ip4base-iacl1sl-100kflows-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-iacl1sl-100kflows-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-iacl1sl-100kflows-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-iacl1sl-100kflows-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-iacl1sl-100kflows-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-iacl1sl-100kflows-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-iacl1sl-100kflows-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc02-64B-1t1c-ethip4udp-ip4base-iacl1-stateless-flows100k-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR +| tc08-9000B-2c-ethip4udp-ip4base-iacl1sl-100kflows-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc03-64B-2t2c-ethip4udp-ip4base-iacl1-stateless-flows100k-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR +| tc09-9000B-4c-ethip4udp-ip4base-iacl1sl-100kflows-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} -| tc04-64B-2t2c-ethip4udp-ip4base-iacl1-stateless-flows100k-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR +| tc10-IMIX-1c-ethip4udp-ip4base-iacl1sl-100kflows-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} -| tc05-64B-4t4c-ethip4udp-ip4base-iacl1-stateless-flows100k-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR +| tc11-IMIX-2c-ethip4udp-ip4base-iacl1sl-100kflows-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc06-64B-4t4c-ethip4udp-ip4base-iacl1-stateless-flows100k-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR +| tc12-IMIX-4c-ethip4udp-ip4base-iacl1sl-100kflows-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sl-10kflows-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sl-10kflows-mrr.robot index 9f2fb7cb0a..e9d8f6e7ca 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sl-10kflows-mrr.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sl-10kflows-mrr.robot @@ -28,7 +28,7 @@ | ... | Test Teardown | Tear down performance mrr test | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *Raw results IPv4 test cases with ACL* | ... @@ -68,8 +68,10 @@ | ${trex_stream1_subnet}= | 10.10.10.0/24 | ${trex_stream2_subnet}= | 20.20.20.0/24 +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u1000p-conc + *** Keywords *** -| Local template +| Local Template | | ... | | [Documentation] | | ... | [Cfg] DUT runs IPv4 routing config with ACL. @@ -83,69 +85,63 @@ | | ... | - phy_cores - Number of physical cores. Type: integer | | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer | | ... -| | [Arguments] | ${phy_cores} | ${framesize} | ${rxq}=${None} -| | ... -| | Set Test Variable | ${framesize} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | ${ip_nr}= | Set Variable | 10 | | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u1000p-conc -| | Set Test Variable | ${traffic_profile} | | Then Traffic should pass with maximum rate | | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-iacl1-stateless-flows10k-mrr +| tc01-64B-1c-ethip4udp-ip4base-iacl1sl-10kflows-mrr | | [Tags] | 64B | 1C -| | phy_cores=${1} | framesize=${64} - -| tc02-1518B-1t1c-ethip4udp-ip4base-iacl1-stateless-flows10k-mrr -| | [Tags] | 1518B | 1C -| | phy_cores=${1} | framesize=${1518} +| | framesize=${64} | phy_cores=${1} -| tc03-9000B-1t1c-ethip4udp-ip4base-iacl1-stateless-flows10k-mrr -| | [Tags] | 9000B | 1C -| | phy_cores=${1} | framesize=${9000} +| tc02-64B-2c-ethip4udp-ip4base-iacl1sl-10kflows-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} -| tc04-IMIX-1t1c-ethip4udp-ip4base-iacl1-stateless-flows10k-mrr -| | [Tags] | IMIX | 1C -| | phy_cores=${1} | framesize=IMIX_v4_1 +| tc03-64B-4c-ethip4udp-ip4base-iacl1sl-10kflows-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} -| tc05-64B-2t2c-ethip4udp-ip4base-iacl1-stateless-flows10k-mrr -| | [Tags] | 64B | 2C -| | phy_cores=${2} | framesize=${64} +| tc04-1518B-1c-ethip4udp-ip4base-iacl1sl-10kflows-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} -| tc06-1518B-2t2c-ethip4udp-ip4base-iacl1-stateless-flows10k-mrr +| tc05-1518B-2c-ethip4udp-ip4base-iacl1sl-10kflows-mrr | | [Tags] | 1518B | 2C -| | phy_cores=${2} | framesize=${1518} - -| tc07-9000B-2t2c-ethip4udp-ip4base-iacl1-stateless-flows10k-mrr -| | [Tags] | 9000B | 2C -| | phy_cores=${2} | framesize=${9000} +| | framesize=${1518} | phy_cores=${2} -| tc08-IMIX-2t2c-ethip4udp-ip4base-iacl1-stateless-flows10k-mrr -| | [Tags] | IMIX | 2C -| | phy_cores=${2} | framesize=IMIX_v4_1 +| tc06-1518B-4c-ethip4udp-ip4base-iacl1sl-10kflows-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} -| tc09-64B-4t4c-ethip4udp-ip4base-iacl1-stateless-flows10k-mrr -| | [Tags] | 64B | 4C -| | phy_cores=${4} | framesize=${64} +| tc07-9000B-1c-ethip4udp-ip4base-iacl1sl-10kflows-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc10-1518B-4t4c-ethip4udp-ip4base-iacl1-stateless-flows10k-mrr -| | [Tags] | 1518B | 4C -| | phy_cores=${4} | framesize=${1518} +| tc08-9000B-2c-ethip4udp-ip4base-iacl1sl-10kflows-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc11-9000B-4t4c-ethip4udp-ip4base-iacl1-stateless-flows10k-mrr +| tc09-9000B-4c-ethip4udp-ip4base-iacl1sl-10kflows-mrr | | [Tags] | 9000B | 4C -| | phy_cores=${4} | framesize=${9000} +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-iacl1sl-10kflows-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-iacl1sl-10kflows-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc12-IMIX-4t4c-ethip4udp-ip4base-iacl1-stateless-flows10k-mrr +| tc12-IMIX-4c-ethip4udp-ip4base-iacl1sl-10kflows-mrr | | [Tags] | IMIX | 4C -| | phy_cores=${4} | framesize=IMIX_v4_1 +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sl-10kflows-ndrpdr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sl-10kflows-ndrpdr.robot new file mode 100644 index 0000000000..5a24a74816 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sl-10kflows-ndrpdr.robot @@ -0,0 +1,153 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS +| ... | IACL | ACL1 | 10k_FLOWS +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with ACL +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down performance discovery test | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Test Template | Local Template +| ... +| Documentation | *RFC2544: Pkt throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage +| ... | of packets transmitted. NDR and PDR are discovered for different +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on links to DUTs. TG traffic profile +| ... | contains two L3 flow-groups (flow-group per direction, 253 flows per +| ... | flow-group) with all packets containing Ethernet header, IPv4 header +| ... | with IP protocol=61 and static payload. MAC addresses are matching MAC +| ... | addresses of the TG node interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit +| ${acl_apply_type}= | input +| ${no_hit_aces_number}= | 1 +| ${flows_per_dir}= | 10k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u1000p-conc + +*** Keywords *** +| Local Template +| | ... +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL. +| | ... | Each DUT uses ${phy_cores} physical core(s) for worker threads. +| | ... | [Ver] Measure NDR and PDR values using MLRsearch algorithm.\ +| | ... +| | ... | *Arguments:* +| | ... | - framesize - Framesize in Bytes in integer or string (IMIX_v4_1). +| | ... | Type: integer, string +| | ... | - phy_cores - Number of physical cores. Type: integer +| | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer +| | ... +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-iacl1sl-10kflows-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-iacl1sl-10kflows-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-iacl1sl-10kflows-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-iacl1sl-10kflows-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-iacl1sl-10kflows-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-iacl1sl-10kflows-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-iacl1sl-10kflows-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-iacl1sl-10kflows-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-iacl1sl-10kflows-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-iacl1sl-10kflows-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-iacl1sl-10kflows-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-iacl1sl-10kflows-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sl-10kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sl-10kflows-ndrpdrdisc.robot deleted file mode 100644 index 89004d6832..0000000000 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl1sl-10kflows-ndrpdrdisc.robot +++ /dev/null @@ -1,129 +0,0 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at: -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -*** Settings *** -| Resource | resources/libraries/robot/performance/performance_setup.robot -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC -| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS -| ... | IACL | ACL1 | 10k_FLOWS -| ... -| Suite Setup | Run Keywords -| ... | Set up 3-node performance topology with DUT's NIC model | L3 -| ... | Intel-X520-DA2 -| ... | AND | Set up performance test suite with ACL -| Suite Teardown | Tear down 3-node performance topology -| ... -| Test Setup | Set up performance test -| ... -| Test Teardown | Tear down performance test with ACL -| ... | ${min_rate}pps | ${framesize} | ${traffic_profile} -| ... -| Test Template | Local template -| ... -| Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* -| ... -| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ -| ... | with single links between nodes. -| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. -| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ -| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ -| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ -| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ -| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\ -| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ -| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ -| ... | of packets transmitted. NDR and PDR are discovered for different\ -| ... | Ethernet L2 frame sizes using either binary search or linear search\ -| ... | algorithms with configured starting rate and final step that determines\ -| ... | throughput measurement resolution. Test packets are generated by TG on\ -| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ -| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ -| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ -| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ -| ... | interfaces. -| ... | *[Ref] Applicable standard specifications:* RFC2544. - -*** Variables *** -# X520-DA2 bandwidth limit -| ${s_limit}= | ${10000000000} - -# ACL test setup -| ${acl_action}= | permit -| ${acl_apply_type}= | input -| ${no_hit_aces_number}= | 1 -| ${flows_per_dir}= | 10k - -# starting points for non-hitting ACLs -| ${src_ip_start}= | 30.30.30.1 -| ${dst_ip_start}= | 40.40.40.1 -| ${ip_step}= | ${1} -| ${sport_start}= | ${1000} -| ${dport_start}= | ${1000} -| ${port_step}= | ${1} -| ${trex_stream1_subnet}= | 10.10.10.0/24 -| ${trex_stream2_subnet}= | 20.20.20.0/24 - -*** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} -| | ... -| | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} -| | And Add PCI devices to all DUTs -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | ${ip_nr}= | Set Variable | 10 -| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u1000p-conc -| | Set Test Variable | ${traffic_profile} -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} - -*** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-iacl1-stateless-flows10k-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR - -| tc02-64B-1t1c-ethip4udp-ip4base-iacl1-stateless-flows10k-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR - -| tc03-64B-2t2c-ethip4udp-ip4base-iacl1-stateless-flows10k-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR - -| tc04-64B-2t2c-ethip4udp-ip4base-iacl1-stateless-flows10k-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR - -| tc05-64B-4t4c-ethip4udp-ip4base-iacl1-stateless-flows10k-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR - -| tc06-64B-4t4c-ethip4udp-ip4base-iacl1-stateless-flows10k-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sf-100flows-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sf-100flows-mrr.robot new file mode 100644 index 0000000000..647d3454d1 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sf-100flows-mrr.robot @@ -0,0 +1,136 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | MRR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL +| ... | IACL | ACL50 | 100_FLOWS +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with ACL +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down mrr test with ACL +| ... +| Test Template | Local Template +| ... +| Documentation | *Raw results IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* In MaxReceivedRate test TG sends traffic +| ... | at line rate and reports total received/sent packets over trial period. +| ... | Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit+reflect +| ${acl_apply_type}= | input +| ${no_hit_aces_number}= | 50 +| ${flows_per_dir}= | 100 + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u10p-conc + +*** Keywords *** +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | Then Traffic should pass with maximum rate +| | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-iacl50sf-100flows-mrr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-iacl50sf-100flows-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-iacl50sf-100flows-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-iacl50sf-100flows-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-iacl50sf-100flows-mrr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-iacl50sf-100flows-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-iacl50sf-100flows-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-iacl50sf-100flows-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-iacl50sf-100flows-mrr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-iacl50sf-100flows-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-iacl50sf-100flows-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-iacl50sf-100flows-mrr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sf-100flows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sf-100flows-ndrpdr.robot index 4f2c474795..221d4f79cf 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sf-100flows-ndrpdrdisc.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sf-100flows-ndrpdr.robot @@ -1,4 +1,4 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. +# Copyright (c) 2018 Cisco and/or its affiliates. # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at: @@ -14,7 +14,7 @@ *** Settings *** | Resource | resources/libraries/robot/performance/performance_setup.robot | ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR | ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL | ... | IACL | ACL50 | 100_FLOWS | ... @@ -29,7 +29,7 @@ | Test Teardown | Tear down performance test with ACL | ... | ${min_rate}pps | ${framesize} | ${traffic_profile} | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* | ... @@ -44,9 +44,8 @@ | ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ | ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ | ... | of packets transmitted. NDR and PDR are discovered for different\ -| ... | Ethernet L2 frame sizes using either binary search or linear search\ -| ... | algorithms with configured starting rate and final step that determines\ -| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on\ | ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ | ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ | ... | all packets containing Ethernet header, IPv4 header with UDP header and\ @@ -74,57 +73,71 @@ | ${trex_stream1_subnet}= | 10.10.10.0/24 | ${trex_stream2_subnet}= | 20.20.20.0/24 +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u10p-conc + *** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | ${ip_nr}= | Set Variable | 10 | | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u10p-conc -| | Set Test Variable | ${traffic_profile} -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-iacl50-stateful-flows100-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR +| tc01-64B-1c-ethip4udp-ip4base-iacl50sf-100flows-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-iacl50sf-100flows-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-iacl50sf-100flows-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-iacl50sf-100flows-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-iacl50sf-100flows-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-iacl50sf-100flows-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-iacl50sf-100flows-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc02-64B-1t1c-ethip4udp-ip4base-iacl50-stateful-flows100-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR +| tc08-9000B-2c-ethip4udp-ip4base-iacl50sf-100flows-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc03-64B-2t2c-ethip4udp-ip4base-iacl50-stateful-flows100-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR +| tc09-9000B-4c-ethip4udp-ip4base-iacl50sf-100flows-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} -| tc04-64B-2t2c-ethip4udp-ip4base-iacl50-stateful-flows100-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR +| tc10-IMIX-1c-ethip4udp-ip4base-iacl50sf-100flows-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} -| tc05-64B-4t4c-ethip4udp-ip4base-ip4base-iacl50-stateful-flows100-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR +| tc11-IMIX-2c-ethip4udp-ip4base-iacl50sf-100flows-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc06-64B-4t4c-ethip4udp-ip4base-ip4base-iacl50-stateful-flows100-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR +| tc12-IMIX-4c-ethip4udp-ip4base-iacl50sf-100flows-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sf-100kflows-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sf-100kflows-mrr.robot new file mode 100644 index 0000000000..16b158d052 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sf-100kflows-mrr.robot @@ -0,0 +1,136 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | MRR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL +| ... | IACL | ACL50 | 100k_FLOWS +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with ACL +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down mrr test with ACL +| ... +| Test Template | Local Template +| ... +| Documentation | *Raw results IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* In MaxReceivedRate test TG sends traffic +| ... | at line rate and reports total received/sent packets over trial period. +| ... | Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit+reflect +| ${acl_apply_type}= | input +| ${no_hit_aces_number}= | 50 +| ${flows_per_dir}= | 100k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +| ${traffic_profile}= | trex-sl-3n-ethip4udp-100u1000p-conc + +*** Keywords *** +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 100 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | Then Traffic should pass with maximum rate +| | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-iacl50sf-100kflows-mrr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-iacl50sf-100kflows-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-iacl50sf-100kflows-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-iacl50sf-100kflows-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-iacl50sf-100kflows-mrr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-iacl50sf-100kflows-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-iacl50sf-100kflows-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-iacl50sf-100kflows-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-iacl50sf-100kflows-mrr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-iacl50sf-100kflows-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-iacl50sf-100kflows-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-iacl50sf-100kflows-mrr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sf-100kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sf-100kflows-ndrpdr.robot index 0b9af35dd6..c7ea5c4c39 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sf-100kflows-ndrpdrdisc.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sf-100kflows-ndrpdr.robot @@ -1,4 +1,4 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. +# Copyright (c) 2018 Cisco and/or its affiliates. # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at: @@ -14,7 +14,7 @@ *** Settings *** | Resource | resources/libraries/robot/performance/performance_setup.robot | ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR | ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL | ... | IACL | ACL50 | 100k_FLOWS | ... @@ -29,7 +29,7 @@ | Test Teardown | Tear down performance test with ACL | ... | ${min_rate}pps | ${framesize} | ${traffic_profile} | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* | ... @@ -44,9 +44,8 @@ | ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ | ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ | ... | of packets transmitted. NDR and PDR are discovered for different\ -| ... | Ethernet L2 frame sizes using either binary search or linear search\ -| ... | algorithms with configured starting rate and final step that determines\ -| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on\ | ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ | ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ | ... | all packets containing Ethernet header, IPv4 header with UDP header and\ @@ -74,57 +73,71 @@ | ${trex_stream1_subnet}= | 10.10.10.0/24 | ${trex_stream2_subnet}= | 20.20.20.0/24 +| ${traffic_profile}= | trex-sl-3n-ethip4udp-100u1000p-conc + *** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | ${ip_nr}= | Set Variable | 100 | | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-100u1000p-conc -| | Set Test Variable | ${traffic_profile} -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-iacl50-stateful-flows100k-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR +| tc01-64B-1c-ethip4udp-ip4base-iacl50sf-100kflows-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-iacl50sf-100kflows-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-iacl50sf-100kflows-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-iacl50sf-100kflows-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-iacl50sf-100kflows-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-iacl50sf-100kflows-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-iacl50sf-100kflows-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc02-64B-1t1c-ethip4udp-ip4base-iacl50-stateful-flows100k-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR +| tc08-9000B-2c-ethip4udp-ip4base-iacl50sf-100kflows-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc03-64B-2t2c-ethip4udp-ip4base-iacl50-stateful-flows100k-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR +| tc09-9000B-4c-ethip4udp-ip4base-iacl50sf-100kflows-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} -| tc04-64B-2t2c-ethip4udp-ip4base-iacl50-stateful-flows100k-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR +| tc10-IMIX-1c-ethip4udp-ip4base-iacl50sf-100kflows-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} -| tc05-64B-4t4c-ethip4udp-ip4base-iacl50-stateful-flows100k-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR +| tc11-IMIX-2c-ethip4udp-ip4base-iacl50sf-100kflows-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc06-64B-4t4c-ethip4udp-ip4base-iacl50-stateful-flows100k-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR +| tc12-IMIX-4c-ethip4udp-ip4base-iacl50sf-100kflows-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sf-10kflows-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sf-10kflows-mrr.robot index 7f06ed2d0b..bc070f0902 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sf-10kflows-mrr.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sf-10kflows-mrr.robot @@ -28,7 +28,7 @@ | ... | Test Teardown | Tear down performance mrr test | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *Raw results IPv4 test cases with ACL* | ... @@ -68,8 +68,10 @@ | ${trex_stream1_subnet}= | 10.10.10.0/24 | ${trex_stream2_subnet}= | 20.20.20.0/24 +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u1000p-conc + *** Keywords *** -| Local template +| Local Template | | ... | | [Documentation] | | ... | [Cfg] DUT runs IPv4 routing config with ACL. @@ -83,69 +85,63 @@ | | ... | - phy_cores - Number of physical cores. Type: integer | | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer | | ... -| | [Arguments] | ${phy_cores} | ${framesize} | ${rxq}=${None} -| | ... -| | Set Test Variable | ${framesize} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | ${ip_nr}= | Set Variable | 10 | | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u1000p-conc -| | Set Test Variable | ${traffic_profile} | | Then Traffic should pass with maximum rate | | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-iacl50-stateful-flows10k-mrr +| tc01-64B-1c-ethip4udp-ip4base-iacl50sf-10kflows-mrr | | [Tags] | 64B | 1C -| | phy_cores=${1} | framesize=${64} - -| tc02-1518B-1t1c-ethip4udp-ip4base-iacl50-stateful-flows10k-mrr -| | [Tags] | 1518B | 1C -| | phy_cores=${1} | framesize=${1518} +| | framesize=${64} | phy_cores=${1} -| tc03-9000B-1t1c-ethip4udp-ip4base-iacl50-stateful-flows10k-mrr -| | [Tags] | 9000B | 1C -| | phy_cores=${1} | framesize=${9000} +| tc02-64B-2c-ethip4udp-ip4base-iacl50sf-10kflows-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} -| tc04-IMIX-1t1c-ethip4udp-ip4base-iacl50-stateful-flows10k-mrr -| | [Tags] | IMIX | 1C -| | phy_cores=${1} | framesize=IMIX_v4_1 +| tc03-64B-4c-ethip4udp-ip4base-iacl50sf-10kflows-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} -| tc05-64B-2t2c-ethip4udp-ip4base-iacl50-stateful-flows10k-mrr -| | [Tags] | 64B | 2C -| | phy_cores=${2} | framesize=${64} +| tc04-1518B-1c-ethip4udp-ip4base-iacl50sf-10kflows-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} -| tc06-1518B-2t2c-ethip4udp-ip4base-iacl50-stateful-flows10k-mrr +| tc05-1518B-2c-ethip4udp-ip4base-iacl50sf-10kflows-mrr | | [Tags] | 1518B | 2C -| | phy_cores=${2} | framesize=${1518} - -| tc07-9000B-2t2c-ethip4udp-ip4base-iacl50-stateful-flows10k-mrr -| | [Tags] | 9000B | 2C -| | phy_cores=${2} | framesize=${9000} +| | framesize=${1518} | phy_cores=${2} -| tc08-IMIX-2t2c-ethip4udp-ip4base-iacl50-stateful-flows10k-mrr -| | [Tags] | IMIX | 2C -| | phy_cores=${2} | framesize=IMIX_v4_1 +| tc06-1518B-4c-ethip4udp-ip4base-iacl50sf-10kflows-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} -| tc09-64B-4t4c-ethip4udp-ip4base-iacl50-stateful-flows10k-mrr -| | [Tags] | 64B | 4C -| | phy_cores=${4} | framesize=${64} +| tc07-9000B-1c-ethip4udp-ip4base-iacl50sf-10kflows-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc10-1518B-4t4c-ethip4udp-ip4base-iacl50-stateful-flows10k-mrr -| | [Tags] | 1518B | 4C -| | phy_cores=${4} | framesize=${1518} +| tc08-9000B-2c-ethip4udp-ip4base-iacl50sf-10kflows-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc11-9000B-4t4c-ethip4udp-ip4base-iacl50-stateful-flows10k-mrr +| tc09-9000B-4c-ethip4udp-ip4base-iacl50sf-10kflows-mrr | | [Tags] | 9000B | 4C -| | phy_cores=${4} | framesize=${9000} +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-iacl50sf-10kflows-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-iacl50sf-10kflows-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc12-IMIX-4t4c-ethip4udp-ip4base-iacl50-stateful-flows10k-mrr +| tc12-IMIX-4c-ethip4udp-ip4base-iacl50sf-10kflows-mrr | | [Tags] | IMIX | 4C -| | phy_cores=${4} | framesize=IMIX_v4_1 +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sf-10kflows-ndrpdr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sf-10kflows-ndrpdr.robot new file mode 100644 index 0000000000..977bbfa591 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sf-10kflows-ndrpdr.robot @@ -0,0 +1,153 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL +| ... | IACL | ACL50 | 10k_FLOWS +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with ACL +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down performance discovery test | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Test Template | Local Template +| ... +| Documentation | *RFC2544: Pkt throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage +| ... | of packets transmitted. NDR and PDR are discovered for different +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on links to DUTs. TG traffic profile +| ... | contains two L3 flow-groups (flow-group per direction, 253 flows per +| ... | flow-group) with all packets containing Ethernet header, IPv4 header +| ... | with IP protocol=61 and static payload. MAC addresses are matching MAC +| ... | addresses of the TG node interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit+reflect +| ${acl_apply_type}= | input +| ${no_hit_aces_number}= | 50 +| ${flows_per_dir}= | 10k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u1000p-conc + +*** Keywords *** +| Local Template +| | ... +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL. +| | ... | Each DUT uses ${phy_cores} physical core(s) for worker threads. +| | ... | [Ver] Measure NDR and PDR values using MLRsearch algorithm.\ +| | ... +| | ... | *Arguments:* +| | ... | - framesize - Framesize in Bytes in integer or string (IMIX_v4_1). +| | ... | Type: integer, string +| | ... | - phy_cores - Number of physical cores. Type: integer +| | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer +| | ... +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-iacl50sf-10kflows-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-iacl50sf-10kflows-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-iacl50sf-10kflows-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-iacl50sf-10kflows-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-iacl50sf-10kflows-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-iacl50sf-10kflows-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-iacl50sf-10kflows-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-iacl50sf-10kflows-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-iacl50sf-10kflows-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-iacl50sf-10kflows-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-iacl50sf-10kflows-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-iacl50sf-10kflows-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sf-10kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sf-10kflows-ndrpdrdisc.robot deleted file mode 100644 index 534c77c48d..0000000000 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sf-10kflows-ndrpdrdisc.robot +++ /dev/null @@ -1,130 +0,0 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at: -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -*** Settings *** -| Resource | resources/libraries/robot/performance/performance_setup.robot -| ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC -| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL -| ... | IACL | ACL50 | 10k_FLOWS -| ... -| Suite Setup | Run Keywords -| ... | Set up 3-node performance topology with DUT's NIC model | L3 -| ... | Intel-X520-DA2 -| ... | AND | Set up performance test suite with ACL -| Suite Teardown | Tear down 3-node performance topology -| ... -| Test Setup | Set up performance test -| ... -| Test Teardown | Tear down performance test with ACL -| ... | ${min_rate}pps | ${framesize} | ${traffic_profile} -| ... -| Test Template | Local template -| ... -| Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* -| ... -| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ -| ... | with single links between nodes. -| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. -| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ -| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ -| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ -| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ -| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\ -| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ -| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ -| ... | of packets transmitted. NDR and PDR are discovered for different\ -| ... | Ethernet L2 frame sizes using either binary search or linear search\ -| ... | algorithms with configured starting rate and final step that determines\ -| ... | throughput measurement resolution. Test packets are generated by TG on\ -| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ -| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ -| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ -| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ -| ... | interfaces. -| ... | *[Ref] Applicable standard specifications:* RFC2544. - -*** Variables *** -# X520-DA2 bandwidth limit -| ${s_limit}= | ${10000000000} - -# ACL test setup -| ${acl_action}= | permit+reflect -| ${acl_apply_type}= | input -| ${no_hit_aces_number}= | 50 -| ${flows_per_dir}= | 10k - -# starting points for non-hitting ACLs -| ${src_ip_start}= | 30.30.30.1 -| ${dst_ip_start}= | 40.40.40.1 -| ${ip_step}= | ${1} -| ${sport_start}= | ${1000} -| ${dport_start}= | ${1000} -| ${port_step}= | ${1} -| ${trex_stream1_subnet}= | 10.10.10.0/24 -| ${trex_stream2_subnet}= | 20.20.20.0/24 - -*** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} -| | ... -| | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} -| | And Add PCI devices to all DUTs -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | ${ip_nr}= | Set Variable | 10 -| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u1000p-conc -| | Set Test Variable | ${traffic_profile} -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} - -*** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-iacl50-stateful-flows10k-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR - -| tc02-64B-1t1c-ethip4udp-ip4base-iacl50-stateful-flows10k-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR - -| tc03-64B-2t2c-ethip4udp-ip4base-iacl50-stateful-flows10k-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR - -| tc04-64B-2t2c-ethip4udp-ip4base-iacl50-stateful-flows10k-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR - -| tc05-64B-4t4c-ethip4udp-ip4base-iacl50-stateful-flows10k-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR - -| tc06-64B-4t4c-ethip4udp-ip4base-iacl50-stateful-flows10k-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sl-100flows-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sl-100flows-mrr.robot new file mode 100644 index 0000000000..28177ead2a --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sl-100flows-mrr.robot @@ -0,0 +1,136 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | MRR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS +| ... | IACL | ACL50 | 100_FLOWS +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with ACL +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down mrr test with ACL +| ... +| Test Template | Local Template +| ... +| Documentation | *Raw results IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* In MaxReceivedRate test TG sends traffic +| ... | at line rate and reports total received/sent packets over trial period. +| ... | Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit +| ${acl_apply_type}= | input +| ${no_hit_aces_number}= | 50 +| ${flows_per_dir}= | 100 + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u10p-conc + +*** Keywords *** +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | Then Traffic should pass with maximum rate +| | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-iacl50sl-100flows-mrr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-iacl50sl-100flows-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-iacl50sl-100flows-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-iacl50sl-100flows-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-iacl50sl-100flows-mrr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-iacl50sl-100flows-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-iacl50sl-100flows-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-iacl50sl-100flows-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-iacl50sl-100flows-mrr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-iacl50sl-100flows-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-iacl50sl-100flows-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-iacl50sl-100flows-mrr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sl-100flows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sl-100flows-ndrpdr.robot index 5aae40a044..708ebec8db 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sl-100flows-ndrpdrdisc.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sl-100flows-ndrpdr.robot @@ -1,4 +1,4 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. +# Copyright (c) 2018 Cisco and/or its affiliates. # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at: @@ -14,7 +14,7 @@ *** Settings *** | Resource | resources/libraries/robot/performance/performance_setup.robot | ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR | ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS | ... | IACL | ACL50 | 100_FLOWS | ... @@ -29,7 +29,7 @@ | Test Teardown | Tear down performance test with ACL | ... | ${min_rate}pps | ${framesize} | ${traffic_profile} | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* | ... @@ -44,9 +44,8 @@ | ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ | ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ | ... | of packets transmitted. NDR and PDR are discovered for different\ -| ... | Ethernet L2 frame sizes using either binary search or linear search\ -| ... | algorithms with configured starting rate and final step that determines\ -| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on\ | ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ | ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ | ... | all packets containing Ethernet header, IPv4 header with UDP header and\ @@ -74,57 +73,71 @@ | ${trex_stream1_subnet}= | 10.10.10.0/24 | ${trex_stream2_subnet}= | 20.20.20.0/24 +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u10p-conc + *** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | ${ip_nr}= | Set Variable | 10 | | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u10p-conc -| | Set Test Variable | ${traffic_profile} -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-iacl50-stateless-flows100-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR +| tc01-64B-1c-ethip4udp-ip4base-iacl50sl-100flows-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-iacl50sl-100flows-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-iacl50sl-100flows-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-iacl50sl-100flows-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-iacl50sl-100flows-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-iacl50sl-100flows-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-iacl50sl-100flows-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc02-64B-1t1c-ethip4udp-ip4base-iacl50-stateless-flows100-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR +| tc08-9000B-2c-ethip4udp-ip4base-iacl50sl-100flows-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc03-64B-2t2c-ethip4udp-ip4base-iacl50-stateless-flows100-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR +| tc09-9000B-4c-ethip4udp-ip4base-iacl50sl-100flows-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} -| tc04-64B-2t2c-ethip4udp-ip4base-iacl50-stateless-flows100-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR +| tc10-IMIX-1c-ethip4udp-ip4base-iacl50sl-100flows-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} -| tc05-64B-4t4c-ethip4udp-ip4base-iacl50-stateless-flows100-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR +| tc11-IMIX-2c-ethip4udp-ip4base-iacl50sl-100flows-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc06-64B-4t4c-ethip4udp-ip4base-iacl50-stateless-flows100-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR +| tc12-IMIX-4c-ethip4udp-ip4base-iacl50sl-100flows-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sl-100kflows-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sl-100kflows-mrr.robot new file mode 100644 index 0000000000..197cabfa9f --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sl-100kflows-mrr.robot @@ -0,0 +1,136 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | MRR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS +| ... | IACL | ACL50 | 100k_FLOWS +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with ACL +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down mrr test with ACL +| ... +| Test Template | Local Template +| ... +| Documentation | *Raw results IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* In MaxReceivedRate test TG sends traffic +| ... | at line rate and reports total received/sent packets over trial period. +| ... | Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit +| ${acl_apply_type}= | input +| ${no_hit_aces_number}= | 50 +| ${flows_per_dir}= | 100k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +| ${traffic_profile}= | trex-sl-3n-ethip4udp-100u1000p-conc + +*** Keywords *** +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 100 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | Then Traffic should pass with maximum rate +| | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-iacl50sl-100kflows-mrr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-iacl50sl-100kflows-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-iacl50sl-100kflows-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-iacl50sl-100kflows-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-iacl50sl-100kflows-mrr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-iacl50sl-100kflows-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-iacl50sl-100kflows-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-iacl50sl-100kflows-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-iacl50sl-100kflows-mrr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-iacl50sl-100kflows-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-iacl50sl-100kflows-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-iacl50sl-100kflows-mrr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sl-100kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sl-100kflows-ndrpdr.robot index 69847c9326..715384d6e7 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sl-100kflows-ndrpdrdisc.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sl-100kflows-ndrpdr.robot @@ -1,4 +1,4 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. +# Copyright (c) 2018 Cisco and/or its affiliates. # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at: @@ -14,7 +14,7 @@ *** Settings *** | Resource | resources/libraries/robot/performance/performance_setup.robot | ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR | ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS | ... | IACL | ACL50 | 100k_FLOWS | ... @@ -29,7 +29,7 @@ | Test Teardown | Tear down performance test with ACL | ... | ${min_rate}pps | ${framesize} | ${traffic_profile} | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* | ... @@ -44,9 +44,8 @@ | ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ | ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ | ... | of packets transmitted. NDR and PDR are discovered for different\ -| ... | Ethernet L2 frame sizes using either binary search or linear search\ -| ... | algorithms with configured starting rate and final step that determines\ -| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on\ | ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ | ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ | ... | all packets containing Ethernet header, IPv4 header with UDP header and\ @@ -74,57 +73,71 @@ | ${trex_stream1_subnet}= | 10.10.10.0/24 | ${trex_stream2_subnet}= | 20.20.20.0/24 +| ${traffic_profile}= | trex-sl-3n-ethip4udp-100u1000p-conc + *** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | ${ip_nr}= | Set Variable | 100 | | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-100u1000p-conc -| | Set Test Variable | ${traffic_profile} -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-iacl50-stateless-flows100k-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR +| tc01-64B-1c-ethip4udp-ip4base-iacl50sl-100kflows-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-iacl50sl-100kflows-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-iacl50sl-100kflows-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-iacl50sl-100kflows-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-iacl50sl-100kflows-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-iacl50sl-100kflows-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-iacl50sl-100kflows-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc02-64B-1t1c-ethip4udp-ip4base-iacl50-stateless-flows100k-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR +| tc08-9000B-2c-ethip4udp-ip4base-iacl50sl-100kflows-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc03-64B-2t2c-ethip4udp-ip4base-iacl50-stateless-flows100k-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR +| tc09-9000B-4c-ethip4udp-ip4base-iacl50sl-100kflows-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} -| tc04-64B-2t2c-ethip4udp-ip4base-iacl50-stateless-flows100k-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR +| tc10-IMIX-1c-ethip4udp-ip4base-iacl50sl-100kflows-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} -| tc05-64B-4t4c-ethip4udp-ip4base-iacl50-stateless-flows100k-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR +| tc11-IMIX-2c-ethip4udp-ip4base-iacl50sl-100kflows-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc06-64B-4t4c-ethip4udp-ip4base-iacl50-stateless-flows100k-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR +| tc12-IMIX-4c-ethip4udp-ip4base-iacl50sl-100kflows-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sl-10kflows-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sl-10kflows-mrr.robot index 4b4a92087e..bf8d1bd49b 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sl-10kflows-mrr.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sl-10kflows-mrr.robot @@ -28,7 +28,7 @@ | ... | Test Teardown | Tear down performance mrr test | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *Raw results IPv4 test cases with ACL* | ... @@ -68,8 +68,10 @@ | ${trex_stream1_subnet}= | 10.10.10.0/24 | ${trex_stream2_subnet}= | 20.20.20.0/24 +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u1000p-conc + *** Keywords *** -| Local template +| Local Template | | ... | | [Documentation] | | ... | [Cfg] DUT runs IPv4 routing config with ACL. @@ -83,69 +85,63 @@ | | ... | - phy_cores - Number of physical cores. Type: integer | | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer | | ... -| | [Arguments] | ${phy_cores} | ${framesize} | ${rxq}=${None} -| | ... -| | Set Test Variable | ${framesize} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | ${ip_nr}= | Set Variable | 10 | | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u1000p-conc -| | Set Test Variable | ${traffic_profile} | | Then Traffic should pass with maximum rate | | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-iacl50-stateless-flows10k-mrr +| tc01-64B-1c-ethip4udp-ip4base-iacl50sl-10kflows-mrr | | [Tags] | 64B | 1C -| | phy_cores=${1} | framesize=${64} - -| tc02-1518B-1t1c-ethip4udp-ip4base-iacl50-stateless-flows10k-mrr -| | [Tags] | 1518B | 1C -| | phy_cores=${1} | framesize=${1518} +| | framesize=${64} | phy_cores=${1} -| tc03-9000B-1t1c-ethip4udp-ip4base-iacl50-stateless-flows10k-mrr -| | [Tags] | 9000B | 1C -| | phy_cores=${1} | framesize=${9000} +| tc02-64B-2c-ethip4udp-ip4base-iacl50sl-10kflows-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} -| tc04-IMIX-1t1c-ethip4udp-ip4base-iacl50-stateless-flows10k-mrr -| | [Tags] | IMIX | 1C -| | phy_cores=${1} | framesize=IMIX_v4_1 +| tc03-64B-4c-ethip4udp-ip4base-iacl50sl-10kflows-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} -| tc05-64B-2t2c-ethip4udp-ip4base-iacl50-stateless-flows10k-mrr -| | [Tags] | 64B | 2C -| | phy_cores=${2} | framesize=${64} +| tc04-1518B-1c-ethip4udp-ip4base-iacl50sl-10kflows-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} -| tc06-1518B-2t2c-ethip4udp-ip4base-iacl50-stateless-flows10k-mrr +| tc05-1518B-2c-ethip4udp-ip4base-iacl50sl-10kflows-mrr | | [Tags] | 1518B | 2C -| | phy_cores=${2} | framesize=${1518} - -| tc07-9000B-2t2c-ethip4udp-ip4base-iacl50-stateless-flows10k-mrr -| | [Tags] | 9000B | 2C -| | phy_cores=${2} | framesize=${9000} +| | framesize=${1518} | phy_cores=${2} -| tc08-IMIX-2t2c-ethip4udp-ip4base-iacl50-stateless-flows10k-mrr -| | [Tags] | IMIX | 2C -| | phy_cores=${2} | framesize=IMIX_v4_1 +| tc06-1518B-4c-ethip4udp-ip4base-iacl50sl-10kflows-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} -| tc09-64B-4t4c-ethip4udp-ip4base-iacl50-stateless-flows10k-mrr -| | [Tags] | 64B | 4C -| | phy_cores=${4} | framesize=${64} +| tc07-9000B-1c-ethip4udp-ip4base-iacl50sl-10kflows-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc10-1518B-4t4c-ethip4udp-ip4base-iacl50-stateless-flows10k-mrr -| | [Tags] | 1518B | 4C -| | phy_cores=${4} | framesize=${1518} +| tc08-9000B-2c-ethip4udp-ip4base-iacl50sl-10kflows-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc11-9000B-4t4c-ethip4udp-ip4base-iacl50-stateless-flows10k-mrr +| tc09-9000B-4c-ethip4udp-ip4base-iacl50sl-10kflows-mrr | | [Tags] | 9000B | 4C -| | phy_cores=${4} | framesize=${9000} +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-iacl50sl-10kflows-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-iacl50sl-10kflows-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc12-IMIX-4t4c-ethip4udp-ip4base-iacl50-stateless-flows10k-mrr +| tc12-IMIX-4c-ethip4udp-ip4base-iacl50sl-10kflows-mrr | | [Tags] | IMIX | 4C -| | phy_cores=${4} | framesize=IMIX_v4_1 +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sl-10kflows-ndrpdr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sl-10kflows-ndrpdr.robot new file mode 100644 index 0000000000..ae10e2c74d --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sl-10kflows-ndrpdr.robot @@ -0,0 +1,153 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS +| ... | IACL | ACL50 | 10k_FLOWS +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with ACL +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down performance discovery test | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Test Template | Local Template +| ... +| Documentation | *RFC2544: Pkt throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage +| ... | of packets transmitted. NDR and PDR are discovered for different +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on links to DUTs. TG traffic profile +| ... | contains two L3 flow-groups (flow-group per direction, 253 flows per +| ... | flow-group) with all packets containing Ethernet header, IPv4 header +| ... | with IP protocol=61 and static payload. MAC addresses are matching MAC +| ... | addresses of the TG node interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit +| ${acl_apply_type}= | input +| ${no_hit_aces_number}= | 50 +| ${flows_per_dir}= | 10k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u1000p-conc + +*** Keywords *** +| Local Template +| | ... +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config with ACL. +| | ... | Each DUT uses ${phy_cores} physical core(s) for worker threads. +| | ... | [Ver] Measure NDR and PDR values using MLRsearch algorithm.\ +| | ... +| | ... | *Arguments:* +| | ... | - framesize - Framesize in Bytes in integer or string (IMIX_v4_1). +| | ... | Type: integer, string +| | ... | - phy_cores - Number of physical cores. Type: integer +| | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer +| | ... +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-iacl50sl-10kflows-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-iacl50sl-10kflows-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-iacl50sl-10kflows-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-iacl50sl-10kflows-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-iacl50sl-10kflows-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-iacl50sl-10kflows-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-iacl50sl-10kflows-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-iacl50sl-10kflows-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-iacl50sl-10kflows-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-iacl50sl-10kflows-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-iacl50sl-10kflows-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-iacl50sl-10kflows-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sl-10kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sl-10kflows-ndrpdrdisc.robot deleted file mode 100644 index 2eec70b130..0000000000 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-iacl50sl-10kflows-ndrpdrdisc.robot +++ /dev/null @@ -1,130 +0,0 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at: -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -*** Settings *** -| Resource | resources/libraries/robot/performance/performance_setup.robot -| ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC -| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS -| ... | IACL | ACL50 | 10k_FLOWS -| ... -| Suite Setup | Run Keywords -| ... | Set up 3-node performance topology with DUT's NIC model | L3 -| ... | Intel-X520-DA2 -| ... | AND | Set up performance test suite with ACL -| Suite Teardown | Tear down 3-node performance topology -| ... -| Test Setup | Set up performance test -| ... -| Test Teardown | Tear down performance test with ACL -| ... | ${min_rate}pps | ${framesize} | ${traffic_profile} -| ... -| Test Template | Local template -| ... -| Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* -| ... -| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ -| ... | with single links between nodes. -| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. -| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ -| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ -| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ -| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ -| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\ -| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ -| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ -| ... | of packets transmitted. NDR and PDR are discovered for different\ -| ... | Ethernet L2 frame sizes using either binary search or linear search\ -| ... | algorithms with configured starting rate and final step that determines\ -| ... | throughput measurement resolution. Test packets are generated by TG on\ -| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ -| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ -| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ -| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ -| ... | interfaces. -| ... | *[Ref] Applicable standard specifications:* RFC2544. - -*** Variables *** -# X520-DA2 bandwidth limit -| ${s_limit}= | ${10000000000} - -# ACL test setup -| ${acl_action}= | permit -| ${acl_apply_type}= | input -| ${no_hit_aces_number}= | 50 -| ${flows_per_dir}= | 10k - -# starting points for non-hitting ACLs -| ${src_ip_start}= | 30.30.30.1 -| ${dst_ip_start}= | 40.40.40.1 -| ${ip_step}= | ${1} -| ${sport_start}= | ${1000} -| ${dport_start}= | ${1000} -| ${port_step}= | ${1} -| ${trex_stream1_subnet}= | 10.10.10.0/24 -| ${trex_stream2_subnet}= | 20.20.20.0/24 - -*** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} -| | ... -| | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} -| | And Add PCI devices to all DUTs -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | ${ip_nr}= | Set Variable | 10 -| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u1000p-conc -| | Set Test Variable | ${traffic_profile} -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} - -*** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-iacl50-stateless-flows10k-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR - -| tc02-64B-1t1c-ethip4udp-ip4base-iacl50-stateless-flows10k-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR - -| tc03-64B-2t2c-ethip4udp-ip4base-iacl50-stateless-flows10k-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR - -| tc04-64B-2t2c-ethip4udp-ip4base-iacl50-stateless-flows10k-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR - -| tc05-64B-4t4c-ethip4udp-ip4base-iacl50-stateless-flows10k-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR - -| tc06-64B-4t4c-ethip4udp-ip4base-iacl50-stateless-flows10k-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-nat44-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-nat44-mrr.robot index c3d31813cc..7f92dc1302 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-nat44-mrr.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-nat44-mrr.robot @@ -29,9 +29,9 @@ | ... | Test Teardown | Tear down performance mrr test | ... -| Test Template | Local template +| Test Template | Local Template | ... -| Documentation | *NAT44 performance test cases* +| Documentation | *Raw results NAT44 performance* | ... | ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology | ... | with single links between nodes. @@ -55,7 +55,7 @@ | ${traffic_profile}= | trex-sl-3n-ethip4udp-1u1p *** Keywords *** -| Local template +| Local Template | | ... | | [Documentation] | | ... | [Cfg] DUT runs IPv4 routing config. @@ -69,16 +69,12 @@ | | ... | - phy_cores - Number of physical cores. Type: integer | | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer | | ... -| | [Arguments] | ${phy_cores} | ${framesize} | ${rxq}=${None} -| | ... -| | Set Test Variable | ${framesize} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${get_framesize}= | Get Frame Size | ${framesize} +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Add NAT to all DUTs | | And Apply startup configuration on all VPP DUTs | | When Initialize NAT44 in 3-node circular topology @@ -86,50 +82,50 @@ | | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} *** Test Cases *** -| tc01-64B-1t1c-ethip4-ip4base-snat-1u-1p-mrr +| tc01-64B-1c-ethip4udp-ip4base-nat44-mrr | | [Tags] | 64B | 1C -| | phy_cores=${1} | framesize=${64} - -| tc02-1518B-1t1c-ethip4-ip4base-snat-1u-1p-mrr -| | [Tags] | 1518B | 1C -| | phy_cores=${1} | framesize=${1518} +| | framesize=${64} | phy_cores=${1} -| tc03-9000B-1t1c-ethip4-ip4base-snat-1u-1p-mrr -| | [Tags] | 9000B | 1C -| | phy_cores=${1} | framesize=${9000} +| tc02-64B-2c-ethip4udp-ip4base-nat44-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} -| tc04-IMIX-1t1c-ethip4-ip4base-snat-1u-1p-mrr -| | [Tags] | IMIX | 1C -| | phy_cores=${1} | framesize=IMIX_v4_1 +| tc03-64B-4c-ethip4udp-ip4base-nat44-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} -| tc05-64B-2t2c-ethip4-ip4base-snat-1u-1p-mrr -| | [Tags] | 64B | 2C -| | phy_cores=${2} | framesize=${64} +| tc04-1518B-1c-ethip4udp-ip4base-nat44-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} -| tc06-1518B-2t2c-ethip4-ip4base-snat-1u-1p-mrr +| tc05-1518B-2c-ethip4udp-ip4base-nat44-mrr | | [Tags] | 1518B | 2C -| | phy_cores=${2} | framesize=${1518} - -| tc07-9000B-2t2c-ethip4-ip4base-snat-1u-1p-mrr -| | [Tags] | 9000B | 2C -| | phy_cores=${2} | framesize=${9000} +| | framesize=${1518} | phy_cores=${2} -| tc08-IMIX-2t2c-ethip4-ip4base-snat-1u-1p-mrr -| | [Tags] | IMIX | 2C -| | phy_cores=${2} | framesize=IMIX_v4_1 +| tc06-1518B-4c-ethip4udp-ip4base-nat44-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} -| tc09-64B-4t4c-ethip4-ip4base-snat-1u-1p-mrr -| | [Tags] | 64B | 4C -| | phy_cores=${4} | framesize=${64} +| tc07-9000B-1c-ethip4udp-ip4base-nat44-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc10-1518B-4t4c-ethip4-ip4base-snat-1u-1p-mrr -| | [Tags] | 1518B | 4C -| | phy_cores=${4} | framesize=${1518} +| tc08-9000B-2c-ethip4udp-ip4base-nat44-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc11-9000B-4t4c-ethip4-ip4base-snat-1u-1p-mrr +| tc09-9000B-4c-ethip4udp-ip4base-nat44-mrr | | [Tags] | 9000B | 4C -| | phy_cores=${4} | framesize=${9000} +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-nat44-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-nat44-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc12-IMIX-4t4c-ethip4-ip4base-snat-1u-1p-mrr +| tc12-IMIX-4c-ethip4udp-ip4base-nat44-mrr | | [Tags] | IMIX | 4C -| | phy_cores=${4} | framesize=IMIX_v4_1 +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-nat44-ndrpdr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-nat44-ndrpdr.robot new file mode 100644 index 0000000000..00f4e19cf1 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-nat44-ndrpdr.robot @@ -0,0 +1,137 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| Resource | resources/libraries/robot/ip/nat.robot +| Resource | resources/libraries/robot/shared/traffic.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | NAT44 | BASE +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with NAT +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down performance discovery test | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Test Template | Local Template +| ... +| Documentation | *RFC2544: Pkt throughput NAT44 performance* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for IPv4 routing. +| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 +| ... | routing and two static IPv4 /24 and IPv4/20 route entries. DUT1 and DUT2 +| ... | tested with 2p10GE NIC X520 Niantic by Intel. +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage +| ... | of packets transmitted. NDR and PDR are discovered for different +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on links to DUTs. TG traffic profile +| ... | contains two L3 flow-groups (flow-group per direction, 253 flows per +| ... | flow-group) with all packets containing Ethernet header, IPv4 header +| ... | with IP protocol=61 and static payload. MAC addresses are matching MAC +| ... | addresses of the TG node interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} +# Traffic profile: +| ${traffic_profile}= | trex-sl-3n-ethip4udp-1u1p + +*** Keywords *** +| Local Template +| | ... +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config. +| | ... | Each DUT uses ${phy_cores} physical core(s) for worker threads. +| | ... | [Ver] Measure NDR and PDR values using MLRsearch algorithm.\ +| | ... +| | ... | *Arguments:* +| | ... | - framesize - Framesize in Bytes in integer or string (IMIX_v4_1). +| | ... | Type: integer, string +| | ... | - phy_cores - Number of physical cores. Type: integer +| | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer +| | ... +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Add NAT to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | When Initialize NAT44 in 3-node circular topology +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-nat44-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-nat44-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-nat44-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-nat44-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-nat44-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-nat44-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-nat44-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-nat44-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-nat44-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-nat44-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-nat44-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-nat44-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-nat44-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-nat44-ndrpdrdisc.robot deleted file mode 100644 index 0fa9706a77..0000000000 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-nat44-ndrpdrdisc.robot +++ /dev/null @@ -1,164 +0,0 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at: -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -*** Settings *** -| Resource | resources/libraries/robot/performance/performance_setup.robot -| Resource | resources/libraries/robot/ip/nat.robot -| Resource | resources/libraries/robot/shared/traffic.robot -| ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC -| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | NAT44 | BASE -| ... -| Suite Setup | Run Keywords -| ... | Set up 3-node performance topology with DUT's NIC model | L3 -| ... | Intel-X520-DA2 -| ... | AND | Set up performance test suite with NAT -| Suite Teardown | Tear down 3-node performance topology -| ... -| Test Setup | Set up performance test -| ... -| Test Teardown | Tear down performance discovery test with NAT -| ... | ${min_rate}pps | ${framesize} | ${traffic_profile} -| ... -| Test Template | Local template -| ... -| Documentation | *NAT44 performance test cases* -| ... -| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology -| ... | with single links between nodes. -| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for IPv4 routing. -| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 -| ... | routing and two static IPv4 /24 and IPv4/20 route entries. DUT1 and DUT2 -| ... | tested with 2p10GE NIC X520 Niantic by Intel. -| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop -| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop -| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage -| ... | of packets transmitted. NDR and PDR are discovered for different -| ... | Ethernet L2 frame sizes using either binary search or linear search -| ... | algorithms with configured starting rate and final step that determines -| ... | throughput measurement resolution. Test packets are generated by TG on -| ... | links to DUTs. TG traffic profile contains two L3 flow-groups -| ... | (flow-group per direction, one flow per flow-group) with all packets -| ... | containing Ethernet header, IPv4 header with UDP header and static -| ... | payload. MAC addresses are matching MAC addresses of the TG node -| ... | interfaces. -| ... | *[Ref] Applicable standard specifications:* RFC2544. - -*** Variables *** -# X520-DA2 bandwidth limit -| ${s_limit} | ${10000000000} -# Traffic profile: -| ${traffic_profile} | trex-sl-3n-ethip4udp-1u1p - -*** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} -| | ... -| | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ... -| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} -| | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs -| | And Add NAT to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | When Initialize NAT44 in 3-node circular topology -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} - -*** Test Cases *** -| tc01-64B-1t1c-ethip4-ip4base-snat-1u-1p-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR - -| tc02-64B-1t1c-ethip4-ip4base-snat-1u-1p-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR - -| tc03-1518B-1t1c-ethip4-ip4base-snat-1u-1p-ndrdisc -| | [Tags] | 1518B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${1518} | search_type=NDR - -| tc04-1518B-1t1c-ethip4-ip4base-snat-1u-1p-pdrdisc -| | [Tags] | 1518B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${1518} | search_type=PDR - -| tc05-IMIX-1t1c-ethip4-ip4base-snat-1u-1p-ndrdisc -| | [Tags] | IMIX | 1C | NDRDISC -| | phy_cores=${1} | framesize=IMIX_v4_1 | search_type=NDR - -| tc06-IMIX-1t1c-ethip4-ip4base-snat-1u-1p-pdrdisc -| | [Tags] | IMIX | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=IMIX_v4_1 | search_type=PDR - -| tc07-64B-2t2c-ethip4-ip4base-snat-1u-1p-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR - -| tc08-64B-2t2c-ethip4-ip4base-snat-1u-1p-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR - -| tc09-1518B-2t2c-ethip4-ip4base-snat-1u-1p-ndrdisc -| | [Tags] | 1518B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${1518} | search_type=NDR - -| tc10-1518B-2t2c-ethip4-ip4base-snat-1u-1p-pdrdisc -| | [Tags] | 1518B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${1518} | search_type=PDR - -| tc11-IMIX-2t2c-ethip4-ip4base-snat-1u-1p-ndrdisc -| | [Tags] | IMIX | 2C | NDRDISC -| | phy_cores=${2} | framesize=IMIX_v4_1 | search_type=NDR - -| tc12-IMIX-2t2c-ethip4-ip4base-snat-1u-1p-pdrdisc -| | [Tags] | IMIX | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=IMIX_v4_1 | search_type=PDR - -| tc13-64B-4t4c-ethip4-ip4base-snat-1u-1p-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR - -| tc14-64B-4t4c-ethip4-ip4base-snat-1u-1p-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR - -| tc15-1518B-4t4c-ethip4-ip4base-snat-1u-1p-ndrdisc -| | [Tags] | 1518B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${1518} | search_type=NDR - -| tc16-1518B-4t4c-ethip4-ip4base-snat-1u-1p-pdrdisc -| | [Tags] | 1518B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${1518} | search_type=PDR - -| tc17-IMIX-4t4c-ethip4-ip4base-snat-1u-1p-ndrdisc -| | [Tags] | IMIX | 4C | NDRDISC -| | phy_cores=${4} | framesize=IMIX_v4_1 | search_type=NDR - -| tc18-IMIX-4t4c-ethip4-ip4base-snat-1u-1p-pdrdisc -| | [Tags] | IMIX | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=IMIX_v4_1 | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sf-100flows-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sf-100flows-mrr.robot new file mode 100644 index 0000000000..441c6bbc8e --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sf-100flows-mrr.robot @@ -0,0 +1,136 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | MRR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL +| ... | OACL | ACL10 | 100_FLOWS +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with ACL +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down mrr test with ACL +| ... +| Test Template | Local Template +| ... +| Documentation | *Raw results IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* In MaxReceivedRate test TG sends traffic +| ... | at line rate and reports total received/sent packets over trial period. +| ... | Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit+reflect +| ${acl_apply_type}= | output +| ${no_hit_aces_number}= | 10 +| ${flows_per_dir}= | 100 + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u10p-conc + +*** Keywords *** +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | Then Traffic should pass with maximum rate +| | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-oacl10sf-100flows-mrr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-oacl10sf-100flows-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-oacl10sf-100flows-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-oacl10sf-100flows-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-oacl10sf-100flows-mrr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-oacl10sf-100flows-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-oacl10sf-100flows-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-oacl10sf-100flows-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-oacl10sf-100flows-mrr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-oacl10sf-100flows-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-oacl10sf-100flows-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-oacl10sf-100flows-mrr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sf-100flows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sf-100flows-ndrpdr.robot index 0a816de129..d7be49236b 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sf-100flows-ndrpdrdisc.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sf-100flows-ndrpdr.robot @@ -1,4 +1,4 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. +# Copyright (c) 2018 Cisco and/or its affiliates. # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at: @@ -14,7 +14,7 @@ *** Settings *** | Resource | resources/libraries/robot/performance/performance_setup.robot | ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR | ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL | ... | OACL | ACL10 | 100_FLOWS | ... @@ -29,7 +29,7 @@ | Test Teardown | Tear down performance test with ACL | ... | ${min_rate}pps | ${framesize} | ${traffic_profile} | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* | ... @@ -44,9 +44,8 @@ | ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ | ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ | ... | of packets transmitted. NDR and PDR are discovered for different\ -| ... | Ethernet L2 frame sizes using either binary search or linear search\ -| ... | algorithms with configured starting rate and final step that determines\ -| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on\ | ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ | ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ | ... | all packets containing Ethernet header, IPv4 header with UDP header and\ @@ -74,57 +73,71 @@ | ${trex_stream1_subnet}= | 10.10.10.0/24 | ${trex_stream2_subnet}= | 20.20.20.0/24 +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u10p-conc + *** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | ${ip_nr}= | Set Variable | 10 | | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u10p-conc -| | Set Test Variable | ${traffic_profile} -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-oacl10-stateful-flows100-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR +| tc01-64B-1c-ethip4udp-ip4base-oacl10sf-100flows-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-oacl10sf-100flows-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-oacl10sf-100flows-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-oacl10sf-100flows-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-oacl10sf-100flows-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-oacl10sf-100flows-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-oacl10sf-100flows-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc02-64B-1t1c-ethip4udp-ip4base-oacl10-stateful-flows100-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR +| tc08-9000B-2c-ethip4udp-ip4base-oacl10sf-100flows-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc03-64B-2t2c-ethip4udp-ip4base-oacl10-stateful-flows100-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR +| tc09-9000B-4c-ethip4udp-ip4base-oacl10sf-100flows-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} -| tc04-64B-2t2c-ethip4udp-ip4base-oacl10-stateful-flows100-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR +| tc10-IMIX-1c-ethip4udp-ip4base-oacl10sf-100flows-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} -| tc05-64B-4t4c-ethip4udp-ip4base-oacl10-stateful-flows100-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR +| tc11-IMIX-2c-ethip4udp-ip4base-oacl10sf-100flows-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc06-64B-4t4c-ethip4udp-ip4base-oacl10-stateful-flows100-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR +| tc12-IMIX-4c-ethip4udp-ip4base-oacl10sf-100flows-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sf-100kflows-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sf-100kflows-mrr.robot new file mode 100644 index 0000000000..b11f56748b --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sf-100kflows-mrr.robot @@ -0,0 +1,136 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | MRR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL +| ... | OACL | ACL10 | 100k_FLOWS +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with ACL +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down mrr test with ACL +| ... +| Test Template | Local Template +| ... +| Documentation | *Raw results IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* In MaxReceivedRate test TG sends traffic +| ... | at line rate and reports total received/sent packets over trial period. +| ... | Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit+reflect +| ${acl_apply_type}= | output +| ${no_hit_aces_number}= | 10 +| ${flows_per_dir}= | 100k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +| ${traffic_profile}= | trex-sl-3n-ethip4udp-100u1000p-conc + +*** Keywords *** +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 100 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | Then Traffic should pass with maximum rate +| | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-oacl10sf-100kflows-mrr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-oacl10sf-100kflows-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-oacl10sf-100kflows-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-oacl10sf-100kflows-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-oacl10sf-100kflows-mrr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-oacl10sf-100kflows-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-oacl10sf-100kflows-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-oacl10sf-100kflows-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-oacl10sf-100kflows-mrr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-oacl10sf-100kflows-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-oacl10sf-100kflows-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-oacl10sf-100kflows-mrr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sf-100kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sf-100kflows-ndrpdr.robot index ba11329750..815585dba6 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sf-100kflows-ndrpdrdisc.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sf-100kflows-ndrpdr.robot @@ -1,4 +1,4 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. +# Copyright (c) 2018 Cisco and/or its affiliates. # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at: @@ -14,7 +14,7 @@ *** Settings *** | Resource | resources/libraries/robot/performance/performance_setup.robot | ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR | ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL | ... | OACL | ACL10 | 100k_FLOWS | ... @@ -29,7 +29,7 @@ | Test Teardown | Tear down performance test with ACL | ... | ${min_rate}pps | ${framesize} | ${traffic_profile} | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* | ... @@ -44,9 +44,8 @@ | ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ | ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ | ... | of packets transmitted. NDR and PDR are discovered for different\ -| ... | Ethernet L2 frame sizes using either binary search or linear search\ -| ... | algorithms with configured starting rate and final step that determines\ -| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on\ | ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ | ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ | ... | all packets containing Ethernet header, IPv4 header with UDP header and\ @@ -74,57 +73,71 @@ | ${trex_stream1_subnet}= | 10.10.10.0/24 | ${trex_stream2_subnet}= | 20.20.20.0/24 +| ${traffic_profile}= | trex-sl-3n-ethip4udp-100u1000p-conc + *** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | ${ip_nr}= | Set Variable | 100 | | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-100u1000p-conc -| | Set Test Variable | ${traffic_profile} -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-oacl10-stateful-flows100k-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR +| tc01-64B-1c-ethip4udp-ip4base-oacl10sf-100kflows-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-oacl10sf-100kflows-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-oacl10sf-100kflows-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-oacl10sf-100kflows-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-oacl10sf-100kflows-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-oacl10sf-100kflows-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-oacl10sf-100kflows-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc02-64B-1t1c-ethip4udp-ip4base-oacl10-stateful-flows100k-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR +| tc08-9000B-2c-ethip4udp-ip4base-oacl10sf-100kflows-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc03-64B-2t2c-ethip4udp-ip4base-oacl10-stateful-flows100k-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR +| tc09-9000B-4c-ethip4udp-ip4base-oacl10sf-100kflows-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} -| tc04-64B-2t2c-ethip4udp-ip4base-oacl10-stateful-flows100k-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR +| tc10-IMIX-1c-ethip4udp-ip4base-oacl10sf-100kflows-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} -| tc05-64B-4t4c-ethip4udp-ip4base-oacl10-stateful-flows100k-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR +| tc11-IMIX-2c-ethip4udp-ip4base-oacl10sf-100kflows-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc06-64B-4t4c-ethip4udp-ip4base-oacl10-stateful-flows100k-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR +| tc12-IMIX-4c-ethip4udp-ip4base-oacl10sf-100kflows-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sf-10kflows-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sf-10kflows-mrr.robot new file mode 100644 index 0000000000..c654bef291 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sf-10kflows-mrr.robot @@ -0,0 +1,136 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | MRR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL +| ... | OACL | ACL10 | 10k_FLOWS +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with ACL +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down mrr test with ACL +| ... +| Test Template | Local Template +| ... +| Documentation | *Raw results IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* In MaxReceivedRate test TG sends traffic +| ... | at line rate and reports total received/sent packets over trial period. +| ... | Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit+reflect +| ${acl_apply_type}= | output +| ${no_hit_aces_number}= | 10 +| ${flows_per_dir}= | 10k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u1000p-conc + +*** Keywords *** +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | Then Traffic should pass with maximum rate +| | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-oacl10sf-10kflows-mrr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-oacl10sf-10kflows-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-oacl10sf-10kflows-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-oacl10sf-10kflows-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-oacl10sf-10kflows-mrr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-oacl10sf-10kflows-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-oacl10sf-10kflows-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-oacl10sf-10kflows-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-oacl10sf-10kflows-mrr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-oacl10sf-10kflows-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-oacl10sf-10kflows-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-oacl10sf-10kflows-mrr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sf-10kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sf-10kflows-ndrpdr.robot index 0df726440c..92f7461527 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sf-10kflows-ndrpdrdisc.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sf-10kflows-ndrpdr.robot @@ -1,4 +1,4 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. +# Copyright (c) 2018 Cisco and/or its affiliates. # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at: @@ -14,7 +14,7 @@ *** Settings *** | Resource | resources/libraries/robot/performance/performance_setup.robot | ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR | ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL | ... | OACL | ACL10 | 10k_FLOWS | ... @@ -29,7 +29,7 @@ | Test Teardown | Tear down performance test with ACL | ... | ${min_rate}pps | ${framesize} | ${traffic_profile} | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* | ... @@ -44,9 +44,8 @@ | ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ | ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ | ... | of packets transmitted. NDR and PDR are discovered for different\ -| ... | Ethernet L2 frame sizes using either binary search or linear search\ -| ... | algorithms with configured starting rate and final step that determines\ -| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on\ | ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ | ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ | ... | all packets containing Ethernet header, IPv4 header with UDP header and\ @@ -74,57 +73,71 @@ | ${trex_stream1_subnet}= | 10.10.10.0/24 | ${trex_stream2_subnet}= | 20.20.20.0/24 +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u1000p-conc + *** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | ${ip_nr}= | Set Variable | 10 | | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u1000p-conc -| | Set Test Variable | ${traffic_profile} -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-oacl10-stateful-flows10k-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR +| tc01-64B-1c-ethip4udp-ip4base-oacl10sf-10kflows-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-oacl10sf-10kflows-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-oacl10sf-10kflows-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-oacl10sf-10kflows-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-oacl10sf-10kflows-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-oacl10sf-10kflows-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-oacl10sf-10kflows-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc02-64B-1t1c-ethip4udp-ip4base-oacl10-stateful-flows10k-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR +| tc08-9000B-2c-ethip4udp-ip4base-oacl10sf-10kflows-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc03-64B-2t2c-ethip4udp-ip4base-oacl10-stateful-flows10k-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR +| tc09-9000B-4c-ethip4udp-ip4base-oacl10sf-10kflows-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} -| tc04-64B-2t2c-ethip4udp-ip4base-oacl10-stateful-flows10k-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR +| tc10-IMIX-1c-ethip4udp-ip4base-oacl10sf-10kflows-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} -| tc05-64B-4t4c-ethip4udp-ip4base-oacl10-stateful-flows10k-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR +| tc11-IMIX-2c-ethip4udp-ip4base-oacl10sf-10kflows-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc06-64B-4t4c-ethip4udp-ip4base-oacl10-stateful-flows10k-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR +| tc12-IMIX-4c-ethip4udp-ip4base-oacl10sf-10kflows-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sl-100flows-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sl-100flows-mrr.robot new file mode 100644 index 0000000000..cf2bdece39 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sl-100flows-mrr.robot @@ -0,0 +1,136 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | MRR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS +| ... | OACL | ACL10 | 100_FLOWS +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with ACL +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down mrr test with ACL +| ... +| Test Template | Local Template +| ... +| Documentation | *Raw results IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* In MaxReceivedRate test TG sends traffic +| ... | at line rate and reports total received/sent packets over trial period. +| ... | Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit +| ${acl_apply_type}= | output +| ${no_hit_aces_number}= | 10 +| ${flows_per_dir}= | 100 + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u10p-conc + +*** Keywords *** +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | Then Traffic should pass with maximum rate +| | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-oacl10sl-100flows-mrr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-oacl10sl-100flows-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-oacl10sl-100flows-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-oacl10sl-100flows-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-oacl10sl-100flows-mrr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-oacl10sl-100flows-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-oacl10sl-100flows-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-oacl10sl-100flows-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-oacl10sl-100flows-mrr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-oacl10sl-100flows-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-oacl10sl-100flows-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-oacl10sl-100flows-mrr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sl-100flows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sl-100flows-ndrpdr.robot index d38cd33232..5b27c367b9 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sl-100flows-ndrpdrdisc.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sl-100flows-ndrpdr.robot @@ -1,4 +1,4 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. +# Copyright (c) 2018 Cisco and/or its affiliates. # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at: @@ -14,7 +14,7 @@ *** Settings *** | Resource | resources/libraries/robot/performance/performance_setup.robot | ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR | ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS | ... | OACL | ACL10 | 100_FLOWS | ... @@ -29,7 +29,7 @@ | Test Teardown | Tear down performance test with ACL | ... | ${min_rate}pps | ${framesize} | ${traffic_profile} | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* | ... @@ -44,9 +44,8 @@ | ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ | ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ | ... | of packets transmitted. NDR and PDR are discovered for different\ -| ... | Ethernet L2 frame sizes using either binary search or linear search\ -| ... | algorithms with configured starting rate and final step that determines\ -| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on\ | ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ | ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ | ... | all packets containing Ethernet header, IPv4 header with UDP header and\ @@ -74,57 +73,71 @@ | ${trex_stream1_subnet}= | 10.10.10.0/24 | ${trex_stream2_subnet}= | 20.20.20.0/24 +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u10p-conc + *** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | ${ip_nr}= | Set Variable | 10 | | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u10p-conc -| | Set Test Variable | ${traffic_profile} -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-oacl10-stateless-flows100-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR +| tc01-64B-1c-ethip4udp-ip4base-oacl10sl-100flows-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-oacl10sl-100flows-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-oacl10sl-100flows-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-oacl10sl-100flows-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-oacl10sl-100flows-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-oacl10sl-100flows-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-oacl10sl-100flows-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc02-64B-1t1c-ethip4udp-ip4base-oacl10-stateless-flows100-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR +| tc08-9000B-2c-ethip4udp-ip4base-oacl10sl-100flows-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc03-64B-2t2c-ethip4udp-ip4base-oacl10-stateless-flows100-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR +| tc09-9000B-4c-ethip4udp-ip4base-oacl10sl-100flows-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} -| tc04-64B-2t2c-ethip4udp-ip4base-oacl10-stateless-flows100-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR +| tc10-IMIX-1c-ethip4udp-ip4base-oacl10sl-100flows-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} -| tc05-64B-4t4c-ethip4udp-ip4base-oacl10-stateless-flows100-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR +| tc11-IMIX-2c-ethip4udp-ip4base-oacl10sl-100flows-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc06-64B-4t4c-ethip4udp-ip4base-oacl10-stateless-flows100-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR +| tc12-IMIX-4c-ethip4udp-ip4base-oacl10sl-100flows-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sl-100kflows-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sl-100kflows-mrr.robot new file mode 100644 index 0000000000..808f3b2c4d --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sl-100kflows-mrr.robot @@ -0,0 +1,136 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | MRR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS +| ... | OACL | ACL10 | 100k_FLOWS +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with ACL +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down mrr test with ACL +| ... +| Test Template | Local Template +| ... +| Documentation | *Raw results IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* In MaxReceivedRate test TG sends traffic +| ... | at line rate and reports total received/sent packets over trial period. +| ... | Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit +| ${acl_apply_type}= | output +| ${no_hit_aces_number}= | 10 +| ${flows_per_dir}= | 100k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +| ${traffic_profile}= | trex-sl-3n-ethip4udp-100u1000p-conc + +*** Keywords *** +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 100 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | Then Traffic should pass with maximum rate +| | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-oacl10sl-100kflows-mrr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-oacl10sl-100kflows-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-oacl10sl-100kflows-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-oacl10sl-100kflows-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-oacl10sl-100kflows-mrr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-oacl10sl-100kflows-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-oacl10sl-100kflows-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-oacl10sl-100kflows-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-oacl10sl-100kflows-mrr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-oacl10sl-100kflows-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-oacl10sl-100kflows-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-oacl10sl-100kflows-mrr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sl-100kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sl-100kflows-ndrpdr.robot index 3b99ce3825..bda4ec6749 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sl-100kflows-ndrpdrdisc.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sl-100kflows-ndrpdr.robot @@ -1,4 +1,4 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. +# Copyright (c) 2018 Cisco and/or its affiliates. # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at: @@ -14,7 +14,7 @@ *** Settings *** | Resource | resources/libraries/robot/performance/performance_setup.robot | ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR | ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS | ... | OACL | ACL10 | 100k_FLOWS | ... @@ -29,7 +29,7 @@ | Test Teardown | Tear down performance test with ACL | ... | ${min_rate}pps | ${framesize} | ${traffic_profile} | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* | ... @@ -44,9 +44,8 @@ | ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ | ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ | ... | of packets transmitted. NDR and PDR are discovered for different\ -| ... | Ethernet L2 frame sizes using either binary search or linear search\ -| ... | algorithms with configured starting rate and final step that determines\ -| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on\ | ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ | ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ | ... | all packets containing Ethernet header, IPv4 header with UDP header and\ @@ -74,57 +73,71 @@ | ${trex_stream1_subnet}= | 10.10.10.0/24 | ${trex_stream2_subnet}= | 20.20.20.0/24 +| ${traffic_profile}= | trex-sl-3n-ethip4udp-100u1000p-conc + *** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | ${ip_nr}= | Set Variable | 100 | | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-100u1000p-conc -| | Set Test Variable | ${traffic_profile} -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-oacl10-stateless-flows100k-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR +| tc01-64B-1c-ethip4udp-ip4base-oacl10sl-100kflows-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-oacl10sl-100kflows-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-oacl10sl-100kflows-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-oacl10sl-100kflows-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-oacl10sl-100kflows-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-oacl10sl-100kflows-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-oacl10sl-100kflows-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc02-64B-1t1c-ethip4udp-ip4base-oacl10-stateless-flows100k-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR +| tc08-9000B-2c-ethip4udp-ip4base-oacl10sl-100kflows-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc03-64B-2t2c-ethip4udp-ip4base-oacl10-stateless-flows100k-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR +| tc09-9000B-4c-ethip4udp-ip4base-oacl10sl-100kflows-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} -| tc04-64B-2t2c-ethip4udp-ip4base-oacl10-stateless-flows100k-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR +| tc10-IMIX-1c-ethip4udp-ip4base-oacl10sl-100kflows-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} -| tc05-64B-4t4c-ethip4udp-ip4base-oacl10-stateless-flows100k-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR +| tc11-IMIX-2c-ethip4udp-ip4base-oacl10sl-100kflows-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc06-64B-4t4c-ethip4udp-ip4base-oacl10-stateless-flows100k-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR +| tc12-IMIX-4c-ethip4udp-ip4base-oacl10sl-100kflows-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sl-10kflows-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sl-10kflows-mrr.robot new file mode 100644 index 0000000000..334bb39eb3 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sl-10kflows-mrr.robot @@ -0,0 +1,136 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | MRR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS +| ... | OACL | ACL10 | 10k_FLOWS +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with ACL +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down mrr test with ACL +| ... +| Test Template | Local Template +| ... +| Documentation | *Raw results IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* In MaxReceivedRate test TG sends traffic +| ... | at line rate and reports total received/sent packets over trial period. +| ... | Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit +| ${acl_apply_type}= | output +| ${no_hit_aces_number}= | 10 +| ${flows_per_dir}= | 10k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u1000p-conc + +*** Keywords *** +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | Then Traffic should pass with maximum rate +| | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-oacl10sl-10kflows-mrr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-oacl10sl-10kflows-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-oacl10sl-10kflows-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-oacl10sl-10kflows-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-oacl10sl-10kflows-mrr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-oacl10sl-10kflows-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-oacl10sl-10kflows-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-oacl10sl-10kflows-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-oacl10sl-10kflows-mrr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-oacl10sl-10kflows-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-oacl10sl-10kflows-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-oacl10sl-10kflows-mrr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sl-10kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sl-10kflows-ndrpdr.robot index cca06bc1ab..135f124e19 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sl-10kflows-ndrpdrdisc.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl10sl-10kflows-ndrpdr.robot @@ -1,4 +1,4 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. +# Copyright (c) 2018 Cisco and/or its affiliates. # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at: @@ -14,7 +14,7 @@ *** Settings *** | Resource | resources/libraries/robot/performance/performance_setup.robot | ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR | ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS | ... | OACL | ACL10 | 10k_FLOWS | ... @@ -29,7 +29,7 @@ | Test Teardown | Tear down performance test with ACL | ... | ${min_rate}pps | ${framesize} | ${traffic_profile} | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* | ... @@ -44,9 +44,8 @@ | ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ | ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ | ... | of packets transmitted. NDR and PDR are discovered for different\ -| ... | Ethernet L2 frame sizes using either binary search or linear search\ -| ... | algorithms with configured starting rate and final step that determines\ -| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on\ | ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ | ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ | ... | all packets containing Ethernet header, IPv4 header with UDP header and\ @@ -74,57 +73,71 @@ | ${trex_stream1_subnet}= | 10.10.10.0/24 | ${trex_stream2_subnet}= | 20.20.20.0/24 +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u1000p-conc + *** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | ${ip_nr}= | Set Variable | 10 | | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u1000p-conc -| | Set Test Variable | ${traffic_profile} -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-oacl10-stateless-flows10k-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR +| tc01-64B-1c-ethip4udp-ip4base-oacl10sl-10kflows-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-oacl10sl-10kflows-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-oacl10sl-10kflows-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-oacl10sl-10kflows-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-oacl10sl-10kflows-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-oacl10sl-10kflows-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-oacl10sl-10kflows-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc02-64B-1t1c-ethip4udp-ip4base-oacl10-stateless-flows10k-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR +| tc08-9000B-2c-ethip4udp-ip4base-oacl10sl-10kflows-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc03-64B-2t2c-ethip4udp-ip4base-oacl10-stateless-flows10k-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR +| tc09-9000B-4c-ethip4udp-ip4base-oacl10sl-10kflows-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} -| tc04-64B-2t2c-ethip4udp-ip4base-oacl10-stateless-flows10k-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR +| tc10-IMIX-1c-ethip4udp-ip4base-oacl10sl-10kflows-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} -| tc05-64B-4t4c-ethip4udp-ip4base-oacl10-stateless-flows10k-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR +| tc11-IMIX-2c-ethip4udp-ip4base-oacl10sl-10kflows-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc06-64B-4t4c-ethip4udp-ip4base-oacl10-stateless-flows10k-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR +| tc12-IMIX-4c-ethip4udp-ip4base-oacl10sl-10kflows-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sf-100flows-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sf-100flows-mrr.robot new file mode 100644 index 0000000000..cb04a4a48b --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sf-100flows-mrr.robot @@ -0,0 +1,136 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | MRR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL +| ... | OACL | ACL1 | 100_FLOWS +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with ACL +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down mrr test with ACL +| ... +| Test Template | Local Template +| ... +| Documentation | *Raw results IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* In MaxReceivedRate test TG sends traffic +| ... | at line rate and reports total received/sent packets over trial period. +| ... | Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit+reflect +| ${acl_apply_type}= | output +| ${no_hit_aces_number}= | 1 +| ${flows_per_dir}= | 100 + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u10p-conc + +*** Keywords *** +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | Then Traffic should pass with maximum rate +| | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-oacl1sf-100flows-mrr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-oacl1sf-100flows-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-oacl1sf-100flows-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-oacl1sf-100flows-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-oacl1sf-100flows-mrr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-oacl1sf-100flows-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-oacl1sf-100flows-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-oacl1sf-100flows-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-oacl1sf-100flows-mrr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-oacl1sf-100flows-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-oacl1sf-100flows-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-oacl1sf-100flows-mrr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sf-100flows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sf-100flows-ndrpdr.robot index 3bb25e1546..f984d9e967 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sf-100flows-ndrpdrdisc.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sf-100flows-ndrpdr.robot @@ -1,4 +1,4 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. +# Copyright (c) 2018 Cisco and/or its affiliates. # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at: @@ -14,7 +14,7 @@ *** Settings *** | Resource | resources/libraries/robot/performance/performance_setup.robot | ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR | ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL | ... | OACL | ACL1 | 100_FLOWS | ... @@ -29,7 +29,7 @@ | Test Teardown | Tear down performance test with ACL | ... | ${min_rate}pps | ${framesize} | ${traffic_profile} | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* | ... @@ -44,9 +44,8 @@ | ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ | ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ | ... | of packets transmitted. NDR and PDR are discovered for different\ -| ... | Ethernet L2 frame sizes using either binary search or linear search\ -| ... | algorithms with configured starting rate and final step that determines\ -| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on\ | ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ | ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ | ... | all packets containing Ethernet header, IPv4 header with UDP header and\ @@ -74,57 +73,71 @@ | ${trex_stream1_subnet}= | 10.10.10.0/24 | ${trex_stream2_subnet}= | 20.20.20.0/24 +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u10p-conc + *** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | ${ip_nr}= | Set Variable | 10 | | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u10p-conc -| | Set Test Variable | ${traffic_profile} -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-oacl1-stateful-flows100-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR +| tc01-64B-1c-ethip4udp-ip4base-oacl1sf-100flows-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-oacl1sf-100flows-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-oacl1sf-100flows-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-oacl1sf-100flows-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-oacl1sf-100flows-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-oacl1sf-100flows-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-oacl1sf-100flows-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc02-64B-1t1c-ethip4udp-ip4base-oacl1-stateful-flows100-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR +| tc08-9000B-2c-ethip4udp-ip4base-oacl1sf-100flows-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc03-64B-2t2c-ethip4udp-ip4base-oacl1-stateful-flows100-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR +| tc09-9000B-4c-ethip4udp-ip4base-oacl1sf-100flows-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} -| tc04-64B-2t2c-ethip4udp-ip4base-oacl1-stateful-flows100-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR +| tc10-IMIX-1c-ethip4udp-ip4base-oacl1sf-100flows-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} -| tc05-64B-4t4c-ethip4udp-ip4base-oacl1-stateful-flows100-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR +| tc11-IMIX-2c-ethip4udp-ip4base-oacl1sf-100flows-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc06-64B-4t4c-ethip4udp-ip4base-oacl1-stateful-flows100-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR +| tc12-IMIX-4c-ethip4udp-ip4base-oacl1sf-100flows-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sf-100kflows-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sf-100kflows-mrr.robot new file mode 100644 index 0000000000..85254c1175 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sf-100kflows-mrr.robot @@ -0,0 +1,136 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | MRR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL +| ... | OACL | ACL1 | 100k_FLOWS +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with ACL +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down mrr test with ACL +| ... +| Test Template | Local Template +| ... +| Documentation | *Raw results IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* In MaxReceivedRate test TG sends traffic +| ... | at line rate and reports total received/sent packets over trial period. +| ... | Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit+reflect +| ${acl_apply_type}= | output +| ${no_hit_aces_number}= | 1 +| ${flows_per_dir}= | 100k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +| ${traffic_profile}= | trex-sl-3n-ethip4udp-100u1000p-conc + +*** Keywords *** +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 100 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | Then Traffic should pass with maximum rate +| | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-oacl1sf-100kflows-mrr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-oacl1sf-100kflows-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-oacl1sf-100kflows-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-oacl1sf-100kflows-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-oacl1sf-100kflows-mrr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-oacl1sf-100kflows-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-oacl1sf-100kflows-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-oacl1sf-100kflows-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-oacl1sf-100kflows-mrr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-oacl1sf-100kflows-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-oacl1sf-100kflows-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-oacl1sf-100kflows-mrr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sf-100kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sf-100kflows-ndrpdr.robot index cda66cc035..07d6421e1b 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sf-100kflows-ndrpdrdisc.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sf-100kflows-ndrpdr.robot @@ -1,4 +1,4 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. +# Copyright (c) 2018 Cisco and/or its affiliates. # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at: @@ -14,7 +14,7 @@ *** Settings *** | Resource | resources/libraries/robot/performance/performance_setup.robot | ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR | ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL | ... | OACL | ACL1 | 100k_FLOWS | ... @@ -29,7 +29,7 @@ | Test Teardown | Tear down performance test with ACL | ... | ${min_rate}pps | ${framesize} | ${traffic_profile} | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* | ... @@ -44,9 +44,8 @@ | ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ | ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ | ... | of packets transmitted. NDR and PDR are discovered for different\ -| ... | Ethernet L2 frame sizes using either binary search or linear search\ -| ... | algorithms with configured starting rate and final step that determines\ -| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on\ | ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ | ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ | ... | all packets containing Ethernet header, IPv4 header with UDP header and\ @@ -74,57 +73,71 @@ | ${trex_stream1_subnet}= | 10.10.10.0/24 | ${trex_stream2_subnet}= | 20.20.20.0/24 +| ${traffic_profile}= | trex-sl-3n-ethip4udp-100u1000p-conc + *** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | ${ip_nr}= | Set Variable | 100 | | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-100u1000p-conc -| | Set Test Variable | ${traffic_profile} -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-oacl1-stateful-flows100k-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR +| tc01-64B-1c-ethip4udp-ip4base-oacl1sf-100kflows-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-oacl1sf-100kflows-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-oacl1sf-100kflows-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-oacl1sf-100kflows-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-oacl1sf-100kflows-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-oacl1sf-100kflows-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-oacl1sf-100kflows-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc02-64B-1t1c-ethip4udp-ip4base-oacl1-stateful-flows100k-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR +| tc08-9000B-2c-ethip4udp-ip4base-oacl1sf-100kflows-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc03-64B-2t2c-ethip4udp-ip4base-oacl1-stateful-flows100k-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR +| tc09-9000B-4c-ethip4udp-ip4base-oacl1sf-100kflows-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} -| tc04-64B-2t2c-ethip4udp-ip4base-oacl1-stateful-flows100k-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR +| tc10-IMIX-1c-ethip4udp-ip4base-oacl1sf-100kflows-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} -| tc05-64B-4t4c-ethip4udp-ip4base-oacl1-stateful-flows100k-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR +| tc11-IMIX-2c-ethip4udp-ip4base-oacl1sf-100kflows-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc06-64B-4t4c-ethip4udp-ip4base-oacl1-stateful-flows100k-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR +| tc12-IMIX-4c-ethip4udp-ip4base-oacl1sf-100kflows-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sf-10kflows-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sf-10kflows-mrr.robot new file mode 100644 index 0000000000..4a95dc140f --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sf-10kflows-mrr.robot @@ -0,0 +1,135 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | MRR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL +| ... | OACL | ACL1 | 10k_FLOWS +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with ACL +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down mrr test with ACL +| ... +| Test Template | Local Template +| ... +| Documentation | *Raw results IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* In MaxReceivedRate test TG sends traffic +| ... | at line rate and reports total received/sent packets over trial period. +| ... | Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit+reflect +| ${acl_apply_type}= | output +| ${no_hit_aces_number}= | 1 +| ${flows_per_dir}= | 10k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u1000p-conc + +*** Keywords *** +| Local Template +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | Then Traffic should pass with maximum rate +| | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-oacl1sf-10kflows-mrr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-oacl1sf-10kflows-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-oacl1sf-10kflows-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-oacl1sf-10kflows-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-oacl1sf-10kflows-mrr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-oacl1sf-10kflows-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-oacl1sf-10kflows-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-oacl1sf-10kflows-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-oacl1sf-10kflows-mrr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-oacl1sf-10kflows-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-oacl1sf-10kflows-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-oacl1sf-10kflows-mrr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sf-10kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sf-10kflows-ndrpdr.robot index 685f188da8..bd55f20a3c 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sf-10kflows-ndrpdrdisc.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sf-10kflows-ndrpdr.robot @@ -1,4 +1,4 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. +# Copyright (c) 2018 Cisco and/or its affiliates. # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at: @@ -14,7 +14,7 @@ *** Settings *** | Resource | resources/libraries/robot/performance/performance_setup.robot | ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR | ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL | ... | OACL | ACL1 | 10k_FLOWS | ... @@ -29,7 +29,7 @@ | Test Teardown | Tear down performance test with ACL | ... | ${min_rate}pps | ${framesize} | ${traffic_profile} | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* | ... @@ -44,9 +44,8 @@ | ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ | ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ | ... | of packets transmitted. NDR and PDR are discovered for different\ -| ... | Ethernet L2 frame sizes using either binary search or linear search\ -| ... | algorithms with configured starting rate and final step that determines\ -| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on\ | ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ | ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ | ... | all packets containing Ethernet header, IPv4 header with UDP header and\ @@ -74,57 +73,70 @@ | ${trex_stream1_subnet}= | 10.10.10.0/24 | ${trex_stream2_subnet}= | 20.20.20.0/24 +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u1000p-conc + *** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} +| Local Template +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | ${ip_nr}= | Set Variable | 10 | | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u1000p-conc -| | Set Test Variable | ${traffic_profile} -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-oacl1-stateful-flows10k-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR +| tc01-64B-1c-ethip4udp-ip4base-oacl1sf-10kflows-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-oacl1sf-10kflows-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-oacl1sf-10kflows-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-oacl1sf-10kflows-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-oacl1sf-10kflows-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-oacl1sf-10kflows-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-oacl1sf-10kflows-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc02-64B-1t1c-ethip4udp-ip4base-oacl1-stateful-flows10k-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR +| tc08-9000B-2c-ethip4udp-ip4base-oacl1sf-10kflows-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc03-64B-2t2c-ethip4udp-ip4base-oacl1-stateful-flows10k-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR +| tc09-9000B-4c-ethip4udp-ip4base-oacl1sf-10kflows-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} -| tc04-64B-2t2c-ethip4udp-ip4base-oacl1-stateful-flows10k-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR +| tc10-IMIX-1c-ethip4udp-ip4base-oacl1sf-10kflows-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} -| tc05-64B-4t4c-ethip4udp-ip4base-oacl1-stateful-flows10k-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR +| tc11-IMIX-2c-ethip4udp-ip4base-oacl1sf-10kflows-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc06-64B-4t4c-ethip4udp-ip4base-oacl1-stateful-flows10k-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR +| tc12-IMIX-4c-ethip4udp-ip4base-oacl1sf-10kflows-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sl-100flows-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sl-100flows-mrr.robot new file mode 100644 index 0000000000..63f447fbff --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sl-100flows-mrr.robot @@ -0,0 +1,136 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | MRR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS +| ... | OACL | ACL1 | 100_FLOWS +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with ACL +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down mrr test with ACL +| ... +| Test Template | Local Template +| ... +| Documentation | *Raw results IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* In MaxReceivedRate test TG sends traffic +| ... | at line rate and reports total received/sent packets over trial period. +| ... | Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit +| ${acl_apply_type}= | output +| ${no_hit_aces_number}= | 1 +| ${flows_per_dir}= | 100 + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u10p-conc + +*** Keywords *** +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | Then Traffic should pass with maximum rate +| | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-oacl1sl-100flows-mrr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-oacl1sl-100flows-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-oacl1sl-100flows-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-oacl1sl-100flows-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-oacl1sl-100flows-mrr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-oacl1sl-100flows-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-oacl1sl-100flows-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-oacl1sl-100flows-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-oacl1sl-100flows-mrr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-oacl1sl-100flows-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-oacl1sl-100flows-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-oacl1sl-100flows-mrr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sl-100flows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sl-100flows-ndrpdr.robot index e6b67c7ab7..5075519045 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sl-100flows-ndrpdrdisc.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sl-100flows-ndrpdr.robot @@ -1,4 +1,4 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. +# Copyright (c) 2018 Cisco and/or its affiliates. # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at: @@ -14,7 +14,7 @@ *** Settings *** | Resource | resources/libraries/robot/performance/performance_setup.robot | ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR | ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS | ... | OACL | ACL1 | 100_FLOWS | ... @@ -29,7 +29,7 @@ | Test Teardown | Tear down performance test with ACL | ... | ${min_rate}pps | ${framesize} | ${traffic_profile} | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* | ... @@ -44,9 +44,8 @@ | ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ | ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ | ... | of packets transmitted. NDR and PDR are discovered for different\ -| ... | Ethernet L2 frame sizes using either binary search or linear search\ -| ... | algorithms with configured starting rate and final step that determines\ -| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on\ | ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ | ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ | ... | all packets containing Ethernet header, IPv4 header with UDP header and\ @@ -74,57 +73,71 @@ | ${trex_stream1_subnet}= | 10.10.10.0/24 | ${trex_stream2_subnet}= | 20.20.20.0/24 +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u10p-conc + *** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | ${ip_nr}= | Set Variable | 10 | | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u10p-conc -| | Set Test Variable | ${traffic_profile} -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-oacl1-stateless-flows100-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR +| tc01-64B-1c-ethip4udp-ip4base-oacl1sl-100flows-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-oacl1sl-100flows-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-oacl1sl-100flows-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-oacl1sl-100flows-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-oacl1sl-100flows-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-oacl1sl-100flows-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-oacl1sl-100flows-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc02-64B-1t1c-ethip4udp-ip4base-oacl1-stateless-flows100-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR +| tc08-9000B-2c-ethip4udp-ip4base-oacl1sl-100flows-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc03-64B-2t2c-ethip4udp-ip4base-oacl1-stateless-flows100-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR +| tc09-9000B-4c-ethip4udp-ip4base-oacl1sl-100flows-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} -| tc04-64B-2t2c-ethip4udp-ip4base-oacl1-stateless-flows100-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR +| tc10-IMIX-1c-ethip4udp-ip4base-oacl1sl-100flows-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} -| tc05-64B-4t4c-ethip4udp-ip4base-oacl1-stateless-flows100-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR +| tc11-IMIX-2c-ethip4udp-ip4base-oacl1sl-100flows-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc06-64B-4t4c-ethip4udp-ip4base-oacl1-stateless-flows100-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR +| tc12-IMIX-4c-ethip4udp-ip4base-oacl1sl-100flows-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sl-100kflows-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sl-100kflows-mrr.robot new file mode 100644 index 0000000000..8ee8a5bb4e --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sl-100kflows-mrr.robot @@ -0,0 +1,136 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | MRR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS +| ... | OACL | ACL1 | 100k_FLOWS +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with ACL +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down mrr test with ACL +| ... +| Test Template | Local Template +| ... +| Documentation | *Raw results IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* In MaxReceivedRate test TG sends traffic +| ... | at line rate and reports total received/sent packets over trial period. +| ... | Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit +| ${acl_apply_type}= | output +| ${no_hit_aces_number}= | 1 +| ${flows_per_dir}= | 100k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +| ${traffic_profile}= | trex-sl-3n-ethip4udp-100u1000p-conc + +*** Keywords *** +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 100 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | Then Traffic should pass with maximum rate +| | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-oacl1sl-100kflows-mrr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-oacl1sl-100kflows-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-oacl1sl-100kflows-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-oacl1sl-100kflows-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-oacl1sl-100kflows-mrr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-oacl1sl-100kflows-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-oacl1sl-100kflows-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-oacl1sl-100kflows-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-oacl1sl-100kflows-mrr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-oacl1sl-100kflows-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-oacl1sl-100kflows-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-oacl1sl-100kflows-mrr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sl-100kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sl-100kflows-ndrpdr.robot index 19bca2c4b9..37dd338ca5 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sl-100kflows-ndrpdrdisc.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sl-100kflows-ndrpdr.robot @@ -1,4 +1,4 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. +# Copyright (c) 2018 Cisco and/or its affiliates. # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at: @@ -14,7 +14,7 @@ *** Settings *** | Resource | resources/libraries/robot/performance/performance_setup.robot | ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR | ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS | ... | OACL | ACL1 | 100k_FLOWS | ... @@ -29,7 +29,7 @@ | Test Teardown | Tear down performance test with ACL | ... | ${min_rate}pps | ${framesize} | ${traffic_profile} | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* | ... @@ -44,9 +44,8 @@ | ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ | ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ | ... | of packets transmitted. NDR and PDR are discovered for different\ -| ... | Ethernet L2 frame sizes using either binary search or linear search\ -| ... | algorithms with configured starting rate and final step that determines\ -| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on\ | ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ | ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ | ... | all packets containing Ethernet header, IPv4 header with UDP header and\ @@ -74,57 +73,71 @@ | ${trex_stream1_subnet}= | 10.10.10.0/24 | ${trex_stream2_subnet}= | 20.20.20.0/24 +| ${traffic_profile}= | trex-sl-3n-ethip4udp-100u1000p-conc + *** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | ${ip_nr}= | Set Variable | 100 | | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-100u1000p-conc -| | Set Test Variable | ${traffic_profile} -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-oacl1-stateless-flows100k-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR +| tc01-64B-1c-ethip4udp-ip4base-oacl1sl-100kflows-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-oacl1sl-100kflows-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-oacl1sl-100kflows-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-oacl1sl-100kflows-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-oacl1sl-100kflows-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-oacl1sl-100kflows-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-oacl1sl-100kflows-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc02-64B-1t1c-ethip4udp-ip4base-oacl1-stateless-flows100k-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR +| tc08-9000B-2c-ethip4udp-ip4base-oacl1sl-100kflows-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc03-64B-2t2c-ethip4udp-ip4base-oacl1-stateless-flows100k-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR +| tc09-9000B-4c-ethip4udp-ip4base-oacl1sl-100kflows-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} -| tc04-64B-2t2c-ethip4udp-ip4base-oacl1-stateless-flows100k-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR +| tc10-IMIX-1c-ethip4udp-ip4base-oacl1sl-100kflows-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} -| tc05-64B-4t4c-ethip4udp-ip4base-oacl1-stateless-flows100k-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR +| tc11-IMIX-2c-ethip4udp-ip4base-oacl1sl-100kflows-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc06-64B-4t4c-ethip4udp-ip4base-oacl1-stateless-flows100k-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR +| tc12-IMIX-4c-ethip4udp-ip4base-oacl1sl-100kflows-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sl-10kflows-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sl-10kflows-mrr.robot new file mode 100644 index 0000000000..29a45a7f6a --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sl-10kflows-mrr.robot @@ -0,0 +1,136 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | MRR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS +| ... | OACL | ACL1 | 10k_FLOWS +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with ACL +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down mrr test with ACL +| ... +| Test Template | Local Template +| ... +| Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* In MaxReceivedRate test TG sends traffic +| ... | at line rate and reports total received/sent packets over trial period. +| ... | Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit +| ${acl_apply_type}= | output +| ${no_hit_aces_number}= | 1 +| ${flows_per_dir}= | 10k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u1000p-conc + +*** Keywords *** +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | Then Traffic should pass with maximum rate +| | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-oacl1sl-10kflows-mrr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-oacl1sl-10kflows-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-oacl1sl-10kflows-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-oacl1sl-10kflows-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-oacl1sl-10kflows-mrr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-oacl1sl-10kflows-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-oacl1sl-10kflows-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-oacl1sl-10kflows-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-oacl1sl-10kflows-mrr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-oacl1sl-10kflows-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-oacl1sl-10kflows-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-oacl1sl-10kflows-mrr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sl-10kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sl-10kflows-ndrpdr.robot index 321d2ef780..212299ce9c 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sl-10kflows-ndrpdrdisc.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl1sl-10kflows-ndrpdr.robot @@ -1,4 +1,4 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. +# Copyright (c) 2018 Cisco and/or its affiliates. # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at: @@ -14,7 +14,7 @@ *** Settings *** | Resource | resources/libraries/robot/performance/performance_setup.robot | ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR | ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS | ... | OACL | ACL1 | 10k_FLOWS | ... @@ -29,7 +29,7 @@ | Test Teardown | Tear down performance test with ACL | ... | ${min_rate}pps | ${framesize} | ${traffic_profile} | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* | ... @@ -44,9 +44,8 @@ | ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ | ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ | ... | of packets transmitted. NDR and PDR are discovered for different\ -| ... | Ethernet L2 frame sizes using either binary search or linear search\ -| ... | algorithms with configured starting rate and final step that determines\ -| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on\ | ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ | ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ | ... | all packets containing Ethernet header, IPv4 header with UDP header and\ @@ -74,57 +73,71 @@ | ${trex_stream1_subnet}= | 10.10.10.0/24 | ${trex_stream2_subnet}= | 20.20.20.0/24 +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u1000p-conc + *** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | ${ip_nr}= | Set Variable | 10 | | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u1000p-conc -| | Set Test Variable | ${traffic_profile} -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-oacl1-stateless-flows10k-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR +| tc01-64B-1c-ethip4udp-ip4base-oacl1sl-10kflows-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-oacl1sl-10kflows-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-oacl1sl-10kflows-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-oacl1sl-10kflows-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-oacl1sl-10kflows-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-oacl1sl-10kflows-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-oacl1sl-10kflows-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc02-64B-1t1c-ethip4udp-ip4base-oacl1-stateless-flows10k-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR +| tc08-9000B-2c-ethip4udp-ip4base-oacl1sl-10kflows-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc03-64B-2t2c-ethip4udp-ip4base-oacl1-stateless-flows10k-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR +| tc09-9000B-4c-ethip4udp-ip4base-oacl1sl-10kflows-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} -| tc04-64B-2t2c-ethip4udp-ip4base-oacl1-stateless-flows10k-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR +| tc10-IMIX-1c-ethip4udp-ip4base-oacl1sl-10kflows-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} -| tc05-64B-4t4c-ethip4udp-ip4base-oacl1-stateless-flows10k-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR +| tc11-IMIX-2c-ethip4udp-ip4base-oacl1sl-10kflows-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc06-64B-4t4c-ethip4udp-ip4base-oacl1-stateless-flows10k-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR +| tc12-IMIX-4c-ethip4udp-ip4base-oacl1sl-10kflows-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sf-100flows-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sf-100flows-mrr.robot new file mode 100644 index 0000000000..b33f96a6dd --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sf-100flows-mrr.robot @@ -0,0 +1,136 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | MRR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL +| ... | OACL | ACL50 | 100_FLOWS +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with ACL +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down mrr test with ACL +| ... +| Test Template | Local Template +| ... +| Documentation | *Raw results IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* In MaxReceivedRate test TG sends traffic +| ... | at line rate and reports total received/sent packets over trial period. +| ... | Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit+reflect +| ${acl_apply_type}= | output +| ${no_hit_aces_number}= | 50 +| ${flows_per_dir}= | 100 + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u10p-conc + +*** Keywords *** +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | Then Traffic should pass with maximum rate +| | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-oacl50sf-100flows-mrr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-oacl50sf-100flows-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-oacl50sf-100flows-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-oacl50sf-100flows-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-oacl50sf-100flows-mrr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-oacl50sf-100flows-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-oacl50sf-100flows-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-oacl50sf-100flows-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-oacl50sf-100flows-mrr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-oacl50sf-100flows-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-oacl50sf-100flows-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-oacl50sf-100flows-mrr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sf-100flows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sf-100flows-ndrpdr.robot index 141ad893dd..4ee35f4a19 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sf-100flows-ndrpdrdisc.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sf-100flows-ndrpdr.robot @@ -1,4 +1,4 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. +# Copyright (c) 2018 Cisco and/or its affiliates. # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at: @@ -14,7 +14,7 @@ *** Settings *** | Resource | resources/libraries/robot/performance/performance_setup.robot | ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR | ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL | ... | OACL | ACL50 | 100_FLOWS | ... @@ -29,7 +29,7 @@ | Test Teardown | Tear down performance test with ACL | ... | ${min_rate}pps | ${framesize} | ${traffic_profile} | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* | ... @@ -44,9 +44,8 @@ | ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ | ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ | ... | of packets transmitted. NDR and PDR are discovered for different\ -| ... | Ethernet L2 frame sizes using either binary search or linear search\ -| ... | algorithms with configured starting rate and final step that determines\ -| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on\ | ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ | ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ | ... | all packets containing Ethernet header, IPv4 header with UDP header and\ @@ -74,57 +73,71 @@ | ${trex_stream1_subnet}= | 10.10.10.0/24 | ${trex_stream2_subnet}= | 20.20.20.0/24 +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u10p-conc + *** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | ${ip_nr}= | Set Variable | 10 | | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u10p-conc -| | Set Test Variable | ${traffic_profile} -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-oacl50-stateful-flows100-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR +| tc01-64B-1c-ethip4udp-ip4base-oacl50sf-100flows-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-oacl50sf-100flows-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-oacl50sf-100flows-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-oacl50sf-100flows-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-oacl50sf-100flows-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-oacl50sf-100flows-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-oacl50sf-100flows-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc02-64B-1t1c-ethip4udp-ip4base-oacl50-stateful-flows100-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR +| tc08-9000B-2c-ethip4udp-ip4base-oacl50sf-100flows-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc03-64B-2t2c-ethip4udp-ip4base-oacl50-stateful-flows100-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR +| tc09-9000B-4c-ethip4udp-ip4base-oacl50sf-100flows-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} -| tc04-64B-2t2c-ethip4udp-ip4base-oacl50-stateful-flows100-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR +| tc10-IMIX-1c-ethip4udp-ip4base-oacl50sf-100flows-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} -| tc05-64B-4t4c-ethip4udp-ip4base-oacl50-stateful-flows100-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR +| tc11-IMIX-2c-ethip4udp-ip4base-oacl50sf-100flows-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc06-64B-4t4c-ethip4udp-ip4base-oacl50-stateful-flows100-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR +| tc12-IMIX-4c-ethip4udp-ip4base-oacl50sf-100flows-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sf-100kflows-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sf-100kflows-mrr.robot new file mode 100644 index 0000000000..c1a0bc03dc --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sf-100kflows-mrr.robot @@ -0,0 +1,136 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | MRR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL +| ... | OACL | ACL50 | 100k_FLOWS +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with ACL +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down mrr test with ACL +| ... +| Test Template | Local Template +| ... +| Documentation | *Raw results IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* In MaxReceivedRate test TG sends traffic +| ... | at line rate and reports total received/sent packets over trial period. +| ... | Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit+reflect +| ${acl_apply_type}= | output +| ${no_hit_aces_number}= | 50 +| ${flows_per_dir}= | 100k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +| ${traffic_profile}= | trex-sl-3n-ethip4udp-100u1000p-conc + +*** Keywords *** +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 100 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | Then Traffic should pass with maximum rate +| | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-oacl50sf-100kflows-mrr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-oacl50sf-100kflows-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-oacl50sf-100kflows-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-oacl50sf-100kflows-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-oacl50sf-100kflows-mrr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-oacl50sf-100kflows-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-oacl50sf-100kflows-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-oacl50sf-100kflows-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-oacl50sf-100kflows-mrr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-oacl50sf-100kflows-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-oacl50sf-100kflows-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-oacl50sf-100kflows-mrr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sf-100kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sf-100kflows-ndrpdr.robot index 94d73f84fe..5cebb45d41 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sf-100kflows-ndrpdrdisc.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sf-100kflows-ndrpdr.robot @@ -1,4 +1,4 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. +# Copyright (c) 2018 Cisco and/or its affiliates. # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at: @@ -14,7 +14,7 @@ *** Settings *** | Resource | resources/libraries/robot/performance/performance_setup.robot | ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR | ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL | ... | OACL | ACL50 | 100k_FLOWS | ... @@ -29,7 +29,7 @@ | Test Teardown | Tear down performance test with ACL | ... | ${min_rate}pps | ${framesize} | ${traffic_profile} | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* | ... @@ -44,9 +44,8 @@ | ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ | ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ | ... | of packets transmitted. NDR and PDR are discovered for different\ -| ... | Ethernet L2 frame sizes using either binary search or linear search\ -| ... | algorithms with configured starting rate and final step that determines\ -| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on\ | ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ | ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ | ... | all packets containing Ethernet header, IPv4 header with UDP header and\ @@ -74,57 +73,71 @@ | ${trex_stream1_subnet}= | 10.10.10.0/24 | ${trex_stream2_subnet}= | 20.20.20.0/24 +| ${traffic_profile}= | trex-sl-3n-ethip4udp-100u1000p-conc + *** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | ${ip_nr}= | Set Variable | 100 | | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-100u1000p-conc -| | Set Test Variable | ${traffic_profile} -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-oacl50-stateful-flows100k-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR +| tc01-64B-1c-ethip4udp-ip4base-oacl50sf-100kflows-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-oacl50sf-100kflows-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-oacl50sf-100kflows-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-oacl50sf-100kflows-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-oacl50sf-100kflows-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-oacl50sf-100kflows-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-oacl50sf-100kflows-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc02-64B-1t1c-ethip4udp-ip4base-oacl50-stateful-flows100k-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR +| tc08-9000B-2c-ethip4udp-ip4base-oacl50sf-100kflows-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc03-64B-2t2c-ethip4udp-ip4base-oacl50-stateful-flows100k-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR +| tc09-9000B-4c-ethip4udp-ip4base-oacl50sf-100kflows-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} -| tc04-64B-2t2c-ethip4udp-ip4base-oacl50-stateful-flows100k-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR +| tc10-IMIX-1c-ethip4udp-ip4base-oacl50sf-100kflows-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} -| tc05-64B-4t4c-ethip4udp-ip4base-oacl50-stateful-flows100k-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR +| tc11-IMIX-2c-ethip4udp-ip4base-oacl50sf-100kflows-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc06-64B-4t4c-ethip4udp-ip4base-oacl50-stateful-flows100k-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR +| tc12-IMIX-4c-ethip4udp-ip4base-oacl50sf-100kflows-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sf-10kflows-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sf-10kflows-mrr.robot new file mode 100644 index 0000000000..1db6de2cce --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sf-10kflows-mrr.robot @@ -0,0 +1,136 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | MRR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL +| ... | OACL | ACL50 | 10k_FLOWS +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with ACL +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down mrr test with ACL +| ... +| Test Template | Local Template +| ... +| Documentation | *Raw results IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* In MaxReceivedRate test TG sends traffic +| ... | at line rate and reports total received/sent packets over trial period. +| ... | Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit+reflect +| ${acl_apply_type}= | output +| ${no_hit_aces_number}= | 50 +| ${flows_per_dir}= | 10k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u1000p-conc + +*** Keywords *** +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | Then Traffic should pass with maximum rate +| | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-oacl50sf-10kflows-mrr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-oacl50sf-10kflows-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-oacl50sf-10kflows-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-oacl50sf-10kflows-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-oacl50sf-10kflows-mrr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-oacl50sf-10kflows-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-oacl50sf-10kflows-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-oacl50sf-10kflows-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-oacl50sf-10kflows-mrr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-oacl50sf-10kflows-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-oacl50sf-10kflows-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-oacl50sf-10kflows-mrr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sf-10kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sf-10kflows-ndrpdr.robot index 316d2135b5..82ec769241 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sf-10kflows-ndrpdrdisc.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sf-10kflows-ndrpdr.robot @@ -1,4 +1,4 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. +# Copyright (c) 2018 Cisco and/or its affiliates. # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at: @@ -14,7 +14,7 @@ *** Settings *** | Resource | resources/libraries/robot/performance/performance_setup.robot | ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR | ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL | ... | OACL | ACL50 | 10k_FLOWS | ... @@ -29,7 +29,7 @@ | Test Teardown | Tear down performance test with ACL | ... | ${min_rate}pps | ${framesize} | ${traffic_profile} | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* | ... @@ -44,9 +44,8 @@ | ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ | ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ | ... | of packets transmitted. NDR and PDR are discovered for different\ -| ... | Ethernet L2 frame sizes using either binary search or linear search\ -| ... | algorithms with configured starting rate and final step that determines\ -| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on\ | ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ | ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ | ... | all packets containing Ethernet header, IPv4 header with UDP header and\ @@ -74,57 +73,71 @@ | ${trex_stream1_subnet}= | 10.10.10.0/24 | ${trex_stream2_subnet}= | 20.20.20.0/24 +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u1000p-conc + *** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | ${ip_nr}= | Set Variable | 10 | | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u1000p-conc -| | Set Test Variable | ${traffic_profile} -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-oacl50-stateful-flows10k-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR +| tc01-64B-1c-ethip4udp-ip4base-oacl50sf-10kflows-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-oacl50sf-10kflows-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-oacl50sf-10kflows-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-oacl50sf-10kflows-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-oacl50sf-10kflows-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-oacl50sf-10kflows-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-oacl50sf-10kflows-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc02-64B-1t1c-ethip4udp-ip4base-oacl50-stateful-flows10k-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR +| tc08-9000B-2c-ethip4udp-ip4base-oacl50sf-10kflows-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc03-64B-2t2c-ethip4udp-ip4base-oacl50-stateful-flows10k-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR +| tc09-9000B-4c-ethip4udp-ip4base-oacl50sf-10kflows-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} -| tc04-64B-2t2c-ethip4udp-ip4base-oacl50-stateful-flows10k-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR +| tc10-IMIX-1c-ethip4udp-ip4base-oacl50sf-10kflows-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} -| tc05-64B-4t4c-ethip4udp-ip4base-oacl50-stateful-flows10k-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR +| tc11-IMIX-2c-ethip4udp-ip4base-oacl50sf-10kflows-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc06-64B-4t4c-ethip4udp-ip4base-oacl50-stateful-flows10k-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR +| tc12-IMIX-4c-ethip4udp-ip4base-oacl50sf-10kflows-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sl-100flows-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sl-100flows-mrr.robot new file mode 100644 index 0000000000..fcfab06341 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sl-100flows-mrr.robot @@ -0,0 +1,136 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | MRR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS +| ... | OACL | ACL50 | 100_FLOWS +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with ACL +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down mrr test with ACL +| ... +| Test Template | Local Template +| ... +| Documentation | *Raw results IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* In MaxReceivedRate test TG sends traffic +| ... | at line rate and reports total received/sent packets over trial period. +| ... | Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit +| ${acl_apply_type}= | output +| ${no_hit_aces_number}= | 50 +| ${flows_per_dir}= | 100 + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u10p-conc + +*** Keywords *** +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | Then Traffic should pass with maximum rate +| | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-oacl50sl-100flows-mrr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-oacl50sl-100flows-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-oacl50sl-100flows-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-oacl50sl-100flows-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-oacl50sl-100flows-mrr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-oacl50sl-100flows-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-oacl50sl-100flows-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-oacl50sl-100flows-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-oacl50sl-100flows-mrr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-oacl50sl-100flows-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-oacl50sl-100flows-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-oacl50sl-100flows-mrr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sl-100flows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sl-100flows-ndrpdr.robot index c9899ce2d4..f323fd5a2d 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sl-100flows-ndrpdrdisc.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sl-100flows-ndrpdr.robot @@ -1,4 +1,4 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. +# Copyright (c) 2018 Cisco and/or its affiliates. # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at: @@ -14,7 +14,7 @@ *** Settings *** | Resource | resources/libraries/robot/performance/performance_setup.robot | ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR | ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS | ... | OACL | ACL50 | 100_FLOWS | ... @@ -29,7 +29,7 @@ | Test Teardown | Tear down performance test with ACL | ... | ${min_rate}pps | ${framesize} | ${traffic_profile} | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* | ... @@ -44,9 +44,8 @@ | ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ | ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ | ... | of packets transmitted. NDR and PDR are discovered for different\ -| ... | Ethernet L2 frame sizes using either binary search or linear search\ -| ... | algorithms with configured starting rate and final step that determines\ -| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on\ | ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ | ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ | ... | all packets containing Ethernet header, IPv4 header with UDP header and\ @@ -74,57 +73,71 @@ | ${trex_stream1_subnet}= | 10.10.10.0/24 | ${trex_stream2_subnet}= | 20.20.20.0/24 +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u10p-conc + *** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | ${ip_nr}= | Set Variable | 10 | | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u10p-conc -| | Set Test Variable | ${traffic_profile} -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-oacl50-stateless-flows100-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR +| tc01-64B-1c-ethip4udp-ip4base-oacl50sl-100flows-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-oacl50sl-100flows-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-oacl50sl-100flows-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-oacl50sl-100flows-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-oacl50sl-100flows-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-oacl50sl-100flows-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-oacl50sl-100flows-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc02-64B-1t1c-ethip4udp-ip4base-oacl50-stateless-flows100-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR +| tc08-9000B-2c-ethip4udp-ip4base-oacl50sl-100flows-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc03-64B-2t2c-ethip4udp-ip4base-oacl50-stateless-flows100-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR +| tc09-9000B-4c-ethip4udp-ip4base-oacl50sl-100flows-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} -| tc04-64B-2t2c-ethip4udp-ip4base-oacl50-stateless-flows100-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR +| tc10-IMIX-1c-ethip4udp-ip4base-oacl50sl-100flows-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} -| tc05-64B-4t4c-ethip4udp-ip4base-oacl50-stateless-flows100-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR +| tc11-IMIX-2c-ethip4udp-ip4base-oacl50sl-100flows-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc06-64B-4t4c-ethip4udp-ip4base-oacl50-stateless-flows100-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR +| tc12-IMIX-4c-ethip4udp-ip4base-oacl50sl-100flows-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sl-100kflows-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sl-100kflows-mrr.robot new file mode 100644 index 0000000000..6662bd09fb --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sl-100kflows-mrr.robot @@ -0,0 +1,136 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | MRR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS +| ... | OACL | ACL50 | 100k_FLOWS +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with ACL +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down mrr test with ACL +| ... +| Test Template | Local Template +| ... +| Documentation | *Raw results IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* In MaxReceivedRate test TG sends traffic +| ... | at line rate and reports total received/sent packets over trial period. +| ... | Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit +| ${acl_apply_type}= | output +| ${no_hit_aces_number}= | 50 +| ${flows_per_dir}= | 100k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +| ${traffic_profile}= | trex-sl-3n-ethip4udp-100u1000p-conc + +*** Keywords *** +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 100 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | Then Traffic should pass with maximum rate +| | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-oacl50sl-100kflows-mrr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-oacl50sl-100kflows-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-oacl50sl-100kflows-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-oacl50sl-100kflows-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-oacl50sl-100kflows-mrr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-oacl50sl-100kflows-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-oacl50sl-100kflows-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-oacl50sl-100kflows-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-oacl50sl-100kflows-mrr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-oacl50sl-100kflows-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-oacl50sl-100kflows-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-oacl50sl-100kflows-mrr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sl-100kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sl-100kflows-ndrpdr.robot index 124810b7f9..760ff4e9e4 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sl-100kflows-ndrpdrdisc.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sl-100kflows-ndrpdr.robot @@ -1,4 +1,4 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. +# Copyright (c) 2018 Cisco and/or its affiliates. # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at: @@ -14,7 +14,7 @@ *** Settings *** | Resource | resources/libraries/robot/performance/performance_setup.robot | ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR | ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS | ... | OACL | ACL50 | 100k_FLOWS | ... @@ -29,7 +29,7 @@ | Test Teardown | Tear down performance test with ACL | ... | ${min_rate}pps | ${framesize} | ${traffic_profile} | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* | ... @@ -44,9 +44,8 @@ | ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ | ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ | ... | of packets transmitted. NDR and PDR are discovered for different\ -| ... | Ethernet L2 frame sizes using either binary search or linear search\ -| ... | algorithms with configured starting rate and final step that determines\ -| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on\ | ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ | ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ | ... | all packets containing Ethernet header, IPv4 header with UDP header and\ @@ -74,57 +73,71 @@ | ${trex_stream1_subnet}= | 10.10.10.0/24 | ${trex_stream2_subnet}= | 20.20.20.0/24 +| ${traffic_profile}= | trex-sl-3n-ethip4udp-100u1000p-conc + *** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | ${ip_nr}= | Set Variable | 100 | | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-100u1000p-conc -| | Set Test Variable | ${traffic_profile} -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-oacl50-stateless-flows100k-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR +| tc01-64B-1c-ethip4udp-ip4base-oacl50sl-100kflows-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-oacl50sl-100kflows-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-oacl50sl-100kflows-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-oacl50sl-100kflows-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-oacl50sl-100kflows-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-oacl50sl-100kflows-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-oacl50sl-100kflows-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc02-64B-1t1c-ethip4udp-ip4base-oacl50-stateless-flows100k-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR +| tc08-9000B-2c-ethip4udp-ip4base-oacl50sl-100kflows-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc03-64B-2t2c-ethip4udp-ip4base-oacl50-stateless-flows100k-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR +| tc09-9000B-4c-ethip4udp-ip4base-oacl50sl-100kflows-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} -| tc04-64B-2t2c-ethip4udp-ip4base-oacl50-stateless-flows100k-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR +| tc10-IMIX-1c-ethip4udp-ip4base-oacl50sl-100kflows-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} -| tc05-64B-4t4c-ethip4udp-ip4base-oacl50-stateless-flows100k-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR +| tc11-IMIX-2c-ethip4udp-ip4base-oacl50sl-100kflows-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc06-64B-4t4c-ethip4udp-ip4base-oacl50-stateless-flows100k-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR +| tc12-IMIX-4c-ethip4udp-ip4base-oacl50sl-100kflows-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sl-10kflows-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sl-10kflows-mrr.robot new file mode 100644 index 0000000000..587b269966 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sl-10kflows-mrr.robot @@ -0,0 +1,136 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | MRR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS +| ... | OACL | ACL50 | 10k_FLOWS +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with ACL +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down mrr test with ACL +| ... +| Test Template | Local Template +| ... +| Documentation | *Raw results IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X520 Niantic by Intel.\ +| ... | *[Ver] TG verification:* In MaxReceivedRate test TG sends traffic +| ... | at line rate and reports total received/sent packets over trial period. +| ... | Test packets are generated by TG on\ +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ +| ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ +| ... | all packets containing Ethernet header, IPv4 header with UDP header and\ +| ... | static payload. MAC addresses are matching MAC addresses of the TG node\ +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit +| ${acl_apply_type}= | output +| ${no_hit_aces_number}= | 50 +| ${flows_per_dir}= | 10k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u1000p-conc + +*** Keywords *** +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | Then Traffic should pass with maximum rate +| | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-oacl50sl-10kflows-mrr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-oacl50sl-10kflows-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-oacl50sl-10kflows-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-oacl50sl-10kflows-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-oacl50sl-10kflows-mrr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-oacl50sl-10kflows-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-oacl50sl-10kflows-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-oacl50sl-10kflows-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-oacl50sl-10kflows-mrr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-oacl50sl-10kflows-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-oacl50sl-10kflows-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-oacl50sl-10kflows-mrr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sl-10kflows-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sl-10kflows-ndrpdr.robot index d8fbc85198..40c23b8f11 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sl-10kflows-ndrpdrdisc.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-oacl50sl-10kflows-ndrpdr.robot @@ -1,4 +1,4 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. +# Copyright (c) 2018 Cisco and/or its affiliates. # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at: @@ -14,7 +14,7 @@ *** Settings *** | Resource | resources/libraries/robot/performance/performance_setup.robot | ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR | ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS | ... | OACL | ACL50 | 10k_FLOWS | ... @@ -29,7 +29,7 @@ | Test Teardown | Tear down performance test with ACL | ... | ${min_rate}pps | ${framesize} | ${traffic_profile} | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *RFC2544: Packet throughput IPv4 test cases with ACL* | ... @@ -44,9 +44,8 @@ | ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ | ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ | ... | of packets transmitted. NDR and PDR are discovered for different\ -| ... | Ethernet L2 frame sizes using either binary search or linear search\ -| ... | algorithms with configured starting rate and final step that determines\ -| ... | throughput measurement resolution. Test packets are generated by TG on\ +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on\ | ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ | ... | (flow-group per direction, ${flows_per_dir} flows per flow-group) with\ | ... | all packets containing Ethernet header, IPv4 header with UDP header and\ @@ -74,57 +73,71 @@ | ${trex_stream1_subnet}= | 10.10.10.0/24 | ${trex_stream2_subnet}= | 20.20.20.0/24 +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u1000p-conc + *** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | And Run Keyword If | ${get_framesize} < ${1522} | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | ${ip_nr}= | Set Variable | 10 | | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u1000p-conc -| | Set Test Variable | ${traffic_profile} -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-oacl50-stateless-flows10k-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR +| tc01-64B-1c-ethip4udp-ip4base-oacl50sl-10kflows-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-oacl50sl-10kflows-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-oacl50sl-10kflows-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-oacl50sl-10kflows-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-oacl50sl-10kflows-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-oacl50sl-10kflows-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-oacl50sl-10kflows-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc02-64B-1t1c-ethip4udp-ip4base-oacl50-stateless-flows10k-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR +| tc08-9000B-2c-ethip4udp-ip4base-oacl50sl-10kflows-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc03-64B-2t2c-ethip4udp-ip4base-oacl50-stateless-flows10k-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR +| tc09-9000B-4c-ethip4udp-ip4base-oacl50sl-10kflows-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} -| tc04-64B-2t2c-ethip4udp-ip4base-oacl50-stateless-flows10k-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR +| tc10-IMIX-1c-ethip4udp-ip4base-oacl50sl-10kflows-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} -| tc05-64B-4t4c-ethip4udp-ip4base-oacl50-stateless-flows10k-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR +| tc11-IMIX-2c-ethip4udp-ip4base-oacl50sl-10kflows-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc06-64B-4t4c-ethip4udp-ip4base-oacl50-stateless-flows10k-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR +| tc12-IMIX-4c-ethip4udp-ip4base-oacl50sl-10kflows-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-udpsrcscale15-nat44-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-udpsrcscale15-nat44-mrr.robot new file mode 100644 index 0000000000..45617ee3b3 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-udpsrcscale15-nat44-mrr.robot @@ -0,0 +1,120 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| Resource | resources/libraries/robot/ip/nat.robot +| Resource | resources/libraries/robot/shared/traffic.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | MRR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | NAT44 | BASE +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with NAT +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down mrr test with NAT +| ... +| Test Template | Local Template +| ... +| Documentation | *Raw results NAT44 performance test cases* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for IPv4 routing. +| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 +| ... | routing and two static IPv4 /24 and IPv4/20 route entries. DUT1 and DUT2 +| ... | tested with 2p10GE NIC X520 Niantic by Intel. +| ... | *[Ver] TG verification:* In MaxReceivedRate test TG sends traffic +| ... | at line rate and reports total received/sent packets over trial period. +| ... | Test packets are generated by TG on +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups +| ... | (flow-group per direction, one flow per flow-group) with all packets +| ... | containing Ethernet header, IPv4 header with UDP header and static +| ... | payload. MAC addresses are matching MAC addresses of the TG node +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit} | ${10000000000} +# Traffic profile: +| ${traffic_profile} | trex-sl-3n-ethip4udp-1u15p + +*** Keywords *** +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Add NAT to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | When Initialize NAT44 in 3-node circular topology +| | Then Traffic should pass with maximum rate +| | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-udpsrcscale15-nat44-mrr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-udpsrcscale15-nat44-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-udpsrcscale15-nat44-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-udpsrcscale15-nat44-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-udpsrcscale15-nat44-mrr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-udpsrcscale15-nat44-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-udpsrcscale15-nat44-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-udpsrcscale15-nat44-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-udpsrcscale15-nat44-mrr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-udpsrcscale15-nat44-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-udpsrcscale15-nat44-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-udpsrcscale15-nat44-mrr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-udpsrcscale15-nat44-ndrpdr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-udpsrcscale15-nat44-ndrpdr.robot new file mode 100644 index 0000000000..6f70d163f3 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-udpsrcscale15-nat44-ndrpdr.robot @@ -0,0 +1,127 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| Resource | resources/libraries/robot/ip/nat.robot +| Resource | resources/libraries/robot/shared/traffic.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | NAT44 | BASE +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with NAT +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down performance discovery test with NAT +| ... | ${min_rate}pps | ${framesize} | ${traffic_profile} +| ... +| Test Template | Local Template +| ... +| Documentation | *RFC2544: Pkt throughput NAT44 performance test cases* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for IPv4 routing. +| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 +| ... | routing and two static IPv4 /24 and IPv4/20 route entries. DUT1 and DUT2 +| ... | tested with 2p10GE NIC X520 Niantic by Intel. +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage +| ... | of packets transmitted. NDR and PDR are discovered for different +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups +| ... | (flow-group per direction, one flow per flow-group) with all packets +| ... | containing Ethernet header, IPv4 header with UDP header and static +| ... | payload. MAC addresses are matching MAC addresses of the TG node +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit} | ${10000000000} +# Traffic profile: +| ${traffic_profile} | trex-sl-3n-ethip4udp-1u15p + +*** Keywords *** +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Add NAT to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | When Initialize NAT44 in 3-node circular topology +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-udpsrcscale15-nat44-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-udpsrcscale15-nat44-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-udpsrcscale15-nat44-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-udpsrcscale15-nat44-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-udpsrcscale15-nat44-ndrpdrdisc.robot deleted file mode 100644 index 46a2b39271..0000000000 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4base-udpsrcscale15-nat44-ndrpdrdisc.robot +++ /dev/null @@ -1,164 +0,0 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at: -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -*** Settings *** -| Resource | resources/libraries/robot/performance/performance_setup.robot -| Resource | resources/libraries/robot/ip/nat.robot -| Resource | resources/libraries/robot/shared/traffic.robot -| ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC -| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | NAT44 | BASE -| ... -| Suite Setup | Run Keywords -| ... | Set up 3-node performance topology with DUT's NIC model | L3 -| ... | Intel-X520-DA2 -| ... | AND | Set up performance test suite with NAT -| Suite Teardown | Tear down 3-node performance topology -| ... -| Test Setup | Set up performance test -| ... -| Test Teardown | Tear down performance discovery test with NAT -| ... | ${min_rate}pps | ${framesize} | ${traffic_profile} -| ... -| Test Template | Local template -| ... -| Documentation | *NAT44 performance test cases* -| ... -| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology -| ... | with single links between nodes. -| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for IPv4 routing. -| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 -| ... | routing and two static IPv4 /24 and IPv4/20 route entries. DUT1 and DUT2 -| ... | tested with 2p10GE NIC X520 Niantic by Intel. -| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop -| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop -| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage -| ... | of packets transmitted. NDR and PDR are discovered for different -| ... | Ethernet L2 frame sizes using either binary search or linear search -| ... | algorithms with configured starting rate and final step that determines -| ... | throughput measurement resolution. Test packets are generated by TG on -| ... | links to DUTs. TG traffic profile contains two L3 flow-groups -| ... | (flow-group per direction, one flow per flow-group) with all packets -| ... | containing Ethernet header, IPv4 header with UDP header and static -| ... | payload. MAC addresses are matching MAC addresses of the TG node -| ... | interfaces. -| ... | *[Ref] Applicable standard specifications:* RFC2544. - -*** Variables *** -# X520-DA2 bandwidth limit -| ${s_limit} | ${10000000000} -# Traffic profile: -| ${traffic_profile} | trex-sl-3n-ethip4udp-1u15p - -*** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} -| | ... -| | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ... -| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} -| | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs -| | And Add NAT to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | When Initialize NAT44 in 3-node circular topology -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} - -*** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-udpsrcscale15-snat-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR - -| tc02-64B-1t1c-ethip4udp-ip4base-udpsrcscale15-snat-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR - -| tc03-1518B-1t1c-ethip4udp-ip4base-udpsrcscale15-snat-ndrdisc -| | [Tags] | 1518B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${1518} | search_type=NDR - -| tc04-1518B-1t1c-ethip4udp-ip4base-udpsrcscale15-snat-pdrdisc -| | [Tags] | 1518B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${1518} | search_type=PDR - -| tc05-IMIX-1t1c-ethip4udp-ip4base-udpsrcscale15-snat-ndrdisc -| | [Tags] | IMIX | 1C | NDRDISC -| | phy_cores=${1} | framesize=IMIX_v4_1 | search_type=NDR - -| tc06-IMIX-1t1c-ethip4udp-ip4base-udpsrcscale15-snat-pdrdisc -| | [Tags] | IMIX | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=IMIX_v4_1 | search_type=PDR - -| tc07-64B-2t2c-ethip4udp-ip4base-udpsrcscale15-snat-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR - -| tc08-64B-2t2c-ethip4udp-ip4base-udpsrcscale15-snat-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR - -| tc09-1518B-2t2c-ethip4udp-ip4base-udpsrcscale15-snat-ndrdisc -| | [Tags] | 1518B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${1518} | search_type=NDR - -| tc10-1518B-2t2c-ethip4udp-ip4base-udpsrcscale15-snat-pdrdisc -| | [Tags] | 1518B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${1518} | search_type=PDR - -| tc11-IMIX-2t2c-ethip4udp-ip4base-udpsrcscale15-snat-ndrdisc -| | [Tags] | IMIX | 2C | NDRDISC -| | phy_cores=${2} | framesize=IMIX_v4_1 | search_type=NDR - -| tc12-IMIX-2t2c-ethip4udp-ip4base-udpsrcscale15-snat-pdrdisc -| | [Tags] | IMIX | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=IMIX_v4_1 | search_type=PDR - -| tc13-64B-4t4c-ethip4udp-ip4base-udpsrcscale15-snat-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR - -| tc14-64B-4t4c-ethip4udp-ip4base-udpsrcscale15-snat-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR - -| tc15-1518B-4t4c-ethip4udp-ip4base-udpsrcscale15-snat-ndrdisc -| | [Tags] | 1518B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${1518} | search_type=NDR - -| tc16-1518B-4t4c-ethip4udp-ip4base-udpsrcscale15-snat-pdrdisc -| | [Tags] | 1518B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${1518} | search_type=PDR - -| tc17-IMIX-4t4c-ethip4udp-ip4base-udpsrcscale15-snat-ndrdisc -| | [Tags] | IMIX | 4C | NDRDISC -| | phy_cores=${4} | framesize=IMIX_v4_1 | search_type=NDR - -| tc18-IMIX-4t4c-ethip4udp-ip4base-udpsrcscale15-snat-pdrdisc -| | [Tags] | IMIX | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=IMIX_v4_1 | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale10-udpsrcscale15-nat44-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale10-udpsrcscale15-nat44-mrr.robot new file mode 100644 index 0000000000..074545deb2 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale10-udpsrcscale15-nat44-mrr.robot @@ -0,0 +1,121 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| Resource | resources/libraries/robot/ip/nat.robot +| Resource | resources/libraries/robot/shared/traffic.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | MRR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | NAT44 | SRC_USER_10 +| ... | SCALE +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with NAT +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down mrr test with NAT +| ... +| Test Template | Local Template +| ... +| Documentation | *Raw results NAT44 performance test cases* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for IPv4 routing. +| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 +| ... | routing and two static IPv4 /24 and IPv4/20 route entries. DUT1 and DUT2 +| ... | tested with 2p10GE NIC X520 Niantic by Intel. +| ... | *[Ver] TG verification:* In MaxReceivedRate test TG sends traffic +| ... | at line rate and reports total received/sent packets over trial period. +| ... | Test packets are generated by TG on +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups +| ... | (flow-group per direction, one flow per flow-group) with all packets +| ... | containing Ethernet header, IPv4 header with UDP header and static +| ... | payload. MAC addresses are matching MAC addresses of the TG node +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit} | ${10000000000} +# Traffic profile: +| ${traffic_profile} | trex-sl-3n-ethip4udp-10u15p + +*** Keywords *** +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Add NAT to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | When Initialize NAT44 in 3-node circular topology +| | Then Traffic should pass with maximum rate +| | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4scale10-udpsrcscale15-nat44-mrr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4scale10-udpsrcscale15-nat44-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4scale10-udpsrcscale15-nat44-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4scale10-udpsrcscale15-nat44-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4scale10-udpsrcscale15-nat44-mrr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4scale10-udpsrcscale15-nat44-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4scale10-udpsrcscale15-nat44-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4scale10-udpsrcscale15-nat44-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4scale10-udpsrcscale15-nat44-mrr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4scale10-udpsrcscale15-nat44-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4scale10-udpsrcscale15-nat44-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4scale10-udpsrcscale15-nat44-mrr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale10-udpsrcscale15-nat44-ndrpdr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale10-udpsrcscale15-nat44-ndrpdr.robot new file mode 100644 index 0000000000..71ea75768b --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale10-udpsrcscale15-nat44-ndrpdr.robot @@ -0,0 +1,128 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| Resource | resources/libraries/robot/ip/nat.robot +| Resource | resources/libraries/robot/shared/traffic.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | NAT44 | SRC_USER_10 +| ... | SCALE +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with NAT +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down performance discovery test with NAT +| ... | ${min_rate}pps | ${framesize} | ${traffic_profile} +| ... +| Test Template | Local Template +| ... +| Documentation | *RFC2544: Pkt throughput NAT44 performance test cases* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for IPv4 routing. +| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 +| ... | routing and two static IPv4 /24 and IPv4/20 route entries. DUT1 and DUT2 +| ... | tested with 2p10GE NIC X520 Niantic by Intel. +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage +| ... | of packets transmitted. NDR and PDR are discovered for different +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups +| ... | (flow-group per direction, one flow per flow-group) with all packets +| ... | containing Ethernet header, IPv4 header with UDP header and static +| ... | payload. MAC addresses are matching MAC addresses of the TG node +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit} | ${10000000000} +# Traffic profile: +| ${traffic_profile} | trex-sl-3n-ethip4udp-10u15p + +*** Keywords *** +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Add NAT to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | When Initialize NAT44 in 3-node circular topology +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4scale10-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4scale10-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4scale10-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4scale10-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4scale10-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4scale10-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4scale10-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4scale10-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4scale10-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4scale10-udpsrcscale15-nat44-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4scale10-udpsrcscale15-nat44-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4scale10-udpsrcscale15-nat44-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale10-udpsrcscale15-nat44-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale10-udpsrcscale15-nat44-ndrpdrdisc.robot deleted file mode 100644 index cedfea3c50..0000000000 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale10-udpsrcscale15-nat44-ndrpdrdisc.robot +++ /dev/null @@ -1,165 +0,0 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at: -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -*** Settings *** -| Resource | resources/libraries/robot/performance/performance_setup.robot -| Resource | resources/libraries/robot/ip/nat.robot -| Resource | resources/libraries/robot/shared/traffic.robot -| ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC -| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | NAT44 | SRC_USER_10 -| ... | SCALE -| ... -| Suite Setup | Run Keywords -| ... | Set up 3-node performance topology with DUT's NIC model | L3 -| ... | Intel-X520-DA2 -| ... | AND | Set up performance test suite with NAT -| Suite Teardown | Tear down 3-node performance topology -| ... -| Test Setup | Set up performance test -| ... -| Test Teardown | Tear down performance discovery test with NAT -| ... | ${min_rate}pps | ${framesize} | ${traffic_profile} -| ... -| Test Template | Local template -| ... -| Documentation | *NAT44 performance test cases* -| ... -| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology -| ... | with single links between nodes. -| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for IPv4 routing. -| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 -| ... | routing and two static IPv4 /24 and IPv4/20 route entries. DUT1 and DUT2 -| ... | tested with 2p10GE NIC X520 Niantic by Intel. -| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop -| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop -| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage -| ... | of packets transmitted. NDR and PDR are discovered for different -| ... | Ethernet L2 frame sizes using either binary search or linear search -| ... | algorithms with configured starting rate and final step that determines -| ... | throughput measurement resolution. Test packets are generated by TG on -| ... | links to DUTs. TG traffic profile contains two L3 flow-groups -| ... | (flow-group per direction, one flow per flow-group) with all packets -| ... | containing Ethernet header, IPv4 header with UDP header and static -| ... | payload. MAC addresses are matching MAC addresses of the TG node -| ... | interfaces. -| ... | *[Ref] Applicable standard specifications:* RFC2544. - -*** Variables *** -# X520-DA2 bandwidth limit -| ${s_limit} | ${10000000000} -# Traffic profile: -| ${traffic_profile} | trex-sl-3n-ethip4udp-10u15p - -*** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} -| | ... -| | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ... -| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} -| | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs -| | And Add NAT to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | When Initialize NAT44 in 3-node circular topology -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} - -*** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4scale10-udpsrcscale15-snat-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR - -| tc02-64B-1t1c-ethip4udp-ip4scale10-udpsrcscale15-snat-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR - -| tc03-1518B-1t1c-ethip4udp-ip4scale10-udpsrcscale15-snat-ndrdisc -| | [Tags] | 1518B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${1518} | search_type=NDR - -| tc04-1518B-1t1c-ethip4udp-ip4scale10-udpsrcscale15-snat-pdrdisc -| | [Tags] | 1518B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${1518} | search_type=PDR - -| tc05-IMIX-1t1c-ethip4udp-ip4scale10-udpsrcscale15-snat-ndrdisc -| | [Tags] | IMIX | 1C | NDRDISC -| | phy_cores=${1} | framesize=IMIX_v4_1 | search_type=NDR - -| tc06-IMIX-1t1c-ethip4udp-ip4scale10-udpsrcscale15-snat-pdrdisc -| | [Tags] | IMIX | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=IMIX_v4_1 | search_type=PDR - -| tc07-64B-2t2c-ethip4udp-ip4scale10-udpsrcscale15-snat-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR - -| tc08-64B-2t2c-ethip4udp-ip4scale10-udpsrcscale15-snat-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR - -| tc09-1518B-2t2c-ethip4udp-ip4scale10-udpsrcscale15-snat-ndrdisc -| | [Tags] | 1518B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${1518} | search_type=NDR - -| tc10-1518B-2t2c-ethip4udp-ip4scale10-udpsrcscale15-snat-pdrdisc -| | [Tags] | 1518B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${1518} | search_type=PDR - -| tc11-IMIX-2t2c-ethip4udp-ip4scale10-udpsrcscale15-snat-ndrdisc -| | [Tags] | IMIX | 2C | NDRDISC -| | phy_cores=${2} | framesize=IMIX_v4_1 | search_type=NDR - -| tc12-IMIX-2t2c-ethip4udp-ip4scale10-udpsrcscale15-snat-pdrdisc -| | [Tags] | IMIX | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=IMIX_v4_1 | search_type=PDR - -| tc13-64B-4t4c-ethip4udp-ip4scale10-udpsrcscale15-snat-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR - -| tc14-64B-4t4c-ethip4udp-ip4scale10-udpsrcscale15-snat-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR - -| tc15-1518B-4t4c-ethip4udp-ip4scale10-udpsrcscale15-snat-ndrdisc -| | [Tags] | 1518B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${1518} | search_type=NDR - -| tc16-1518B-4t4c-ethip4udp-ip4scale10-udpsrcscale15-snat-pdrdisc -| | [Tags] | 1518B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${1518} | search_type=PDR - -| tc17-IMIX-4t4c-ethip4udp-ip4scale10-udpsrcscale15-snat-ndrdisc -| | [Tags] | IMIX | 4C | NDRDISC -| | phy_cores=${4} | framesize=IMIX_v4_1 | search_type=NDR - -| tc18-IMIX-4t4c-ethip4udp-ip4scale10-udpsrcscale15-snat-pdrdisc -| | [Tags] | IMIX | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=IMIX_v4_1 | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale100-udpsrcscale15-nat44-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale100-udpsrcscale15-nat44-mrr.robot new file mode 100644 index 0000000000..56daf568f2 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale100-udpsrcscale15-nat44-mrr.robot @@ -0,0 +1,121 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| Resource | resources/libraries/robot/ip/nat.robot +| Resource | resources/libraries/robot/shared/traffic.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | MRR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | NAT44 | SRC_USER_100 +| ... | SCALE +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with NAT +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down mrr test with NAT +| ... +| Test Template | Local Template +| ... +| Documentation | *Raw results NAT44 performance test cases* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for IPv4 routing. +| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 +| ... | routing and two static IPv4 /24 and IPv4/20 route entries. DUT1 and DUT2 +| ... | tested with 2p10GE NIC X520 Niantic by Intel. +| ... | *[Ver] TG verification:* In MaxReceivedRate test TG sends traffic +| ... | at line rate and reports total received/sent packets over trial period. +| ... | Test packets are generated by TG on +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups +| ... | (flow-group per direction, one flow per flow-group) with all packets +| ... | containing Ethernet header, IPv4 header with UDP header and static +| ... | payload. MAC addresses are matching MAC addresses of the TG node +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit} | ${10000000000} +# Traffic profile: +| ${traffic_profile} | trex-sl-3n-ethip4udp-100u15p + +*** Keywords *** +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Add NAT to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | When Initialize NAT44 in 3-node circular topology +| | Then Traffic should pass with maximum rate +| | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4scale100-udpsrcscale15-nat44-mrr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4scale100-udpsrcscale15-nat44-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4scale100-udpsrcscale15-nat44-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4scale100-udpsrcscale15-nat44-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4scale100-udpsrcscale15-nat44-mrr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4scale100-udpsrcscale15-nat44-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4scale100-udpsrcscale15-nat44-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4scale100-udpsrcscale15-nat44-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4scale100-udpsrcscale15-nat44-mrr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4scale100-udpsrcscale15-nat44-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4scale100-udpsrcscale15-nat44-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4scale100-udpsrcscale15-nat44-mrr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale100-udpsrcscale15-nat44-ndrpdr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale100-udpsrcscale15-nat44-ndrpdr.robot new file mode 100644 index 0000000000..ec926874fd --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale100-udpsrcscale15-nat44-ndrpdr.robot @@ -0,0 +1,128 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| Resource | resources/libraries/robot/ip/nat.robot +| Resource | resources/libraries/robot/shared/traffic.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | NAT44 | SRC_USER_100 +| ... | SCALE +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with NAT +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down performance discovery test with NAT +| ... | ${min_rate}pps | ${framesize} | ${traffic_profile} +| ... +| Test Template | Local Template +| ... +| Documentation | *RFC2544: Pkt throughput NAT44 performance test cases* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for IPv4 routing. +| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 +| ... | routing and two static IPv4 /24 and IPv4/20 route entries. DUT1 and DUT2 +| ... | tested with 2p10GE NIC X520 Niantic by Intel. +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage +| ... | of packets transmitted. NDR and PDR are discovered for different +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups +| ... | (flow-group per direction, one flow per flow-group) with all packets +| ... | containing Ethernet header, IPv4 header with UDP header and static +| ... | payload. MAC addresses are matching MAC addresses of the TG node +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit} | ${10000000000} +# Traffic profile: +| ${traffic_profile} | trex-sl-3n-ethip4udp-100u15p + +*** Keywords *** +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Add NAT to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | When Initialize NAT44 in 3-node circular topology +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4scale100-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4scale100-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4scale100-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4scale100-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4scale100-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4scale100-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4scale100-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4scale100-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4scale100-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4scale100-udpsrcscale15-nat44-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4scale100-udpsrcscale15-nat44-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4scale100-udpsrcscale15-nat44-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale100-udpsrcscale15-nat44-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale100-udpsrcscale15-nat44-ndrpdrdisc.robot deleted file mode 100644 index 487528d3ef..0000000000 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale100-udpsrcscale15-nat44-ndrpdrdisc.robot +++ /dev/null @@ -1,165 +0,0 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at: -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -*** Settings *** -| Resource | resources/libraries/robot/performance/performance_setup.robot -| Resource | resources/libraries/robot/ip/nat.robot -| Resource | resources/libraries/robot/shared/traffic.robot -| ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC -| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | NAT44 | SRC_USER_100 -| ... | SCALE -| ... -| Suite Setup | Run Keywords -| ... | Set up 3-node performance topology with DUT's NIC model | L3 -| ... | Intel-X520-DA2 -| ... | AND | Set up performance test suite with NAT -| Suite Teardown | Tear down 3-node performance topology -| ... -| Test Setup | Set up performance test -| ... -| Test Teardown | Tear down performance discovery test with NAT -| ... | ${min_rate}pps | ${framesize} | ${traffic_profile} -| ... -| Test Template | Local template -| ... -| Documentation | *NAT44 performance test cases* -| ... -| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology -| ... | with single links between nodes. -| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for IPv4 routing. -| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 -| ... | routing and two static IPv4 /24 and IPv4/20 route entries. DUT1 and DUT2 -| ... | tested with 2p10GE NIC X520 Niantic by Intel. -| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop -| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop -| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage -| ... | of packets transmitted. NDR and PDR are discovered for different -| ... | Ethernet L2 frame sizes using either binary search or linear search -| ... | algorithms with configured starting rate and final step that determines -| ... | throughput measurement resolution. Test packets are generated by TG on -| ... | links to DUTs. TG traffic profile contains two L3 flow-groups -| ... | (flow-group per direction, one flow per flow-group) with all packets -| ... | containing Ethernet header, IPv4 header with UDP header and static -| ... | payload. MAC addresses are matching MAC addresses of the TG node -| ... | interfaces. -| ... | *[Ref] Applicable standard specifications:* RFC2544. - -*** Variables *** -# X520-DA2 bandwidth limit -| ${s_limit} | ${10000000000} -# Traffic profile: -| ${traffic_profile} | trex-sl-3n-ethip4udp-100u15p - -*** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} -| | ... -| | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ... -| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} -| | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs -| | And Add NAT to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | When Initialize NAT44 in 3-node circular topology -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} - -*** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4scale100-udpsrcscale15-snat-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR - -| tc02-64B-1t1c-ethip4udp-ip4scale100-udpsrcscale15-snat-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR - -| tc03-1518B-1t1c-ethip4udp-ip4scale100-udpsrcscale15-snat-ndrdisc -| | [Tags] | 1518B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${1518} | search_type=NDR - -| tc04-1518B-1t1c-ethip4udp-ip4scale100-udpsrcscale15-snat-pdrdisc -| | [Tags] | 1518B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${1518} | search_type=PDR - -| tc05-IMIX-1t1c-ethip4udp-ip4scale100-udpsrcscale15-snat-ndrdisc -| | [Tags] | IMIX | 1C | NDRDISC -| | phy_cores=${1} | framesize=IMIX_v4_1 | search_type=NDR - -| tc06-IMIX-1t1c-ethip4udp-ip4scale100-udpsrcscale15-snat-pdrdisc -| | [Tags] | IMIX | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=IMIX_v4_1 | search_type=PDR - -| tc07-64B-2t2c-ethip4udp-ip4scale100-udpsrcscale15-snat-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR - -| tc08-64B-2t2c-ethip4udp-ip4scale100-udpsrcscale15-snat-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR - -| tc09-1518B-2t2c-ethip4udp-ip4scale100-udpsrcscale15-snat-ndrdisc -| | [Tags] | 1518B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${1518} | search_type=NDR - -| tc10-1518B-2t2c-ethip4udp-ip4scale100-udpsrcscale15-snat-pdrdisc -| | [Tags] | 1518B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${1518} | search_type=PDR - -| tc11-IMIX-2t2c-ethip4udp-ip4scale100-udpsrcscale15-snat-ndrdisc -| | [Tags] | IMIX | 2C | NDRDISC -| | phy_cores=${2} | framesize=IMIX_v4_1 | search_type=NDR - -| tc12-IMIX-2t2c-ethip4udp-ip4scale100-udpsrcscale15-snat-pdrdisc -| | [Tags] | IMIX | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=IMIX_v4_1 | search_type=PDR - -| tc13-64B-4t4c-ethip4udp-ip4scale100-udpsrcscale15-snat-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR - -| tc14-64B-4t4c-ethip4udp-ip4scale100-udpsrcscale15-snat-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR - -| tc15-1518B-4t4c-ethip4udp-ip4scale100-udpsrcscale15-snat-ndrdisc -| | [Tags] | 1518B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${1518} | search_type=NDR - -| tc16-1518B-4t4c-ethip4udp-ip4scale100-udpsrcscale15-snat-pdrdisc -| | [Tags] | 1518B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${1518} | search_type=PDR - -| tc17-IMIX-4t4c-ethip4udp-ip4scale100-udpsrcscale15-snat-ndrdisc -| | [Tags] | IMIX | 4C | NDRDISC -| | phy_cores=${4} | framesize=IMIX_v4_1 | search_type=NDR - -| tc18-IMIX-4t4c-ethip4udp-ip4scale100-udpsrcscale15-snat-pdrdisc -| | [Tags] | IMIX | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=IMIX_v4_1 | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale1000-udpsrcscale15-nat44-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale1000-udpsrcscale15-nat44-mrr.robot index ea4096b172..c5272f2ece 100644 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale1000-udpsrcscale15-nat44-mrr.robot +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale1000-udpsrcscale15-nat44-mrr.robot @@ -30,7 +30,7 @@ | ... | Test Teardown | Tear down performance mrr test | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *Raw results NAT44 performance test cases* | ... @@ -56,7 +56,7 @@ | ${traffic_profile}= | trex-sl-3n-ethip4udp-1000u15p *** Keywords *** -| Local template +| Local Template | | ... | | [Documentation] | | ... | [Cfg] DUT runs IPv4 routing config. @@ -70,16 +70,12 @@ | | ... | - phy_cores - Number of physical cores. Type: integer | | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer | | ... -| | [Arguments] | ${phy_cores} | ${framesize} | ${rxq}=${None} -| | ... -| | Set Test Variable | ${framesize} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${get_framesize}= | Get Frame Size | ${framesize} +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Add NAT to all DUTs | | And Apply startup configuration on all VPP DUTs | | When Initialize NAT44 in 3-node circular topology @@ -87,50 +83,50 @@ | | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4scale1000-udpsrcscale15-snat-mrr +| tc01-64B-1c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-mrr | | [Tags] | 64B | 1C -| | phy_cores=${1} | framesize=${64} - -| tc02-1518B-1t1c-ethip4udp-ip4scale1000-udpsrcscale15-snat-mrr -| | [Tags] | 1518B | 1C -| | phy_cores=${1} | framesize=${1518} +| | framesize=${64} | phy_cores=${1} -| tc03-9000B-1t1c-ethip4udp-ip4scale1000-udpsrcscale15-snat-mrr -| | [Tags] | 9000B | 1C -| | phy_cores=${1} | framesize=${9000} +| tc02-64B-2c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} -| tc04-IMIX-1t1c-ethip4udp-ip4scale1000-udpsrcscale15-snat-mrr -| | [Tags] | IMIX | 1C -| | phy_cores=${1} | framesize=IMIX_v4_1 +| tc03-64B-4c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} -| tc05-64B-2t2c-ethip4udp-ip4scale1000-udpsrcscale15-snat-mrr -| | [Tags] | 64B | 2C -| | phy_cores=${2} | framesize=${64} +| tc04-1518B-1c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} -| tc06-1518B-2t2c-ethip4udp-ip4scale1000-udpsrcscale15-snat-mrr +| tc05-1518B-2c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-mrr | | [Tags] | 1518B | 2C -| | phy_cores=${2} | framesize=${1518} - -| tc07-9000B-2t2c-ethip4udp-ip4scale1000-udpsrcscale15-snat-mrr -| | [Tags] | 9000B | 2C -| | phy_cores=${2} | framesize=${9000} +| | framesize=${1518} | phy_cores=${2} -| tc08-IMIX-2t2c-ethip4udp-ip4scale1000-udpsrcscale15-snat-mrr -| | [Tags] | IMIX | 2C -| | phy_cores=${2} | framesize=IMIX_v4_1 +| tc06-1518B-4c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} -| tc09-64B-4t4c-ethip4udp-ip4scale1000-udpsrcscale15-snat-mrr -| | [Tags] | 64B | 4C -| | phy_cores=${4} | framesize=${64} +| tc07-9000B-1c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc10-1518B-4t4c-ethip4udp-ip4scale1000-udpsrcscale15-snat-mrr -| | [Tags] | 1518B | 4C -| | phy_cores=${4} | framesize=${1518} +| tc08-9000B-2c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc11-9000B-4t4c-ethip4udp-ip4scale1000-udpsrcscale15-snat-mrr +| tc09-9000B-4c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-mrr | | [Tags] | 9000B | 4C -| | phy_cores=${4} | framesize=${9000} +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc12-IMIX-4t4c-ethip4udp-ip4scale1000-udpsrcscale15-snat-mrr +| tc12-IMIX-4c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-mrr | | [Tags] | IMIX | 4C -| | phy_cores=${4} | framesize=IMIX_v4_1 +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale1000-udpsrcscale15-nat44-ndrpdr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale1000-udpsrcscale15-nat44-ndrpdr.robot new file mode 100644 index 0000000000..c03a636ba5 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale1000-udpsrcscale15-nat44-ndrpdr.robot @@ -0,0 +1,138 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| Resource | resources/libraries/robot/ip/nat.robot +| Resource | resources/libraries/robot/shared/traffic.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | NAT44 | SRC_USER_1000 +| ... | SCALE +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with NAT +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down performance discovery test | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Test Template | Local Template +| ... +| Documentation | *RFC2544: Pkt throughput NAT44 performance test cases* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for IPv4 routing. +| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 +| ... | routing and two static IPv4 /24 and IPv4/20 route entries. DUT1 and DUT2 +| ... | tested with 2p10GE NIC X520 Niantic by Intel. +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage +| ... | of packets transmitted. NDR and PDR are discovered for different +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on links to DUTs. TG traffic profile +| ... | contains two L3 flow-groups (flow-group per direction, 253 flows per +| ... | flow-group) with all packets containing Ethernet header, IPv4 header +| ... | with IP protocol=61 and static payload. MAC addresses are matching MAC +| ... | addresses of the TG node interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit}= | ${10000000000} +# Traffic profile: +| ${traffic_profile}= | trex-sl-3n-ethip4udp-1000u15p + +*** Keywords *** +| Local Template +| | ... +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config. +| | ... | Each DUT uses ${phy_cores} physical core(s) for worker threads. +| | ... | [Ver] Measure NDR and PDR values using MLRsearch algorithm.\ +| | ... +| | ... | *Arguments:* +| | ... | - framesize - Framesize in Bytes in integer or string (IMIX_v4_1). +| | ... | Type: integer, string +| | ... | - phy_cores - Number of physical cores. Type: integer +| | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer +| | ... +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Add NAT to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | When Initialize NAT44 in 3-node circular topology +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale1000-udpsrcscale15-nat44-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale1000-udpsrcscale15-nat44-ndrpdrdisc.robot deleted file mode 100644 index 7d1df407f3..0000000000 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale1000-udpsrcscale15-nat44-ndrpdrdisc.robot +++ /dev/null @@ -1,165 +0,0 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at: -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -*** Settings *** -| Resource | resources/libraries/robot/performance/performance_setup.robot -| Resource | resources/libraries/robot/ip/nat.robot -| Resource | resources/libraries/robot/shared/traffic.robot -| ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC -| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | NAT44 | SRC_USER_1000 -| ... | SCALE -| ... -| Suite Setup | Run Keywords -| ... | Set up 3-node performance topology with DUT's NIC model | L3 -| ... | Intel-X520-DA2 -| ... | AND | Set up performance test suite with NAT -| Suite Teardown | Tear down 3-node performance topology -| ... -| Test Setup | Set up performance test -| ... -| Test Teardown | Tear down performance discovery test with NAT -| ... | ${min_rate}pps | ${framesize} | ${traffic_profile} -| ... -| Test Template | Local template -| ... -| Documentation | *NAT44 performance test cases* -| ... -| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology -| ... | with single links between nodes. -| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for IPv4 routing. -| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 -| ... | routing and two static IPv4 /24 and IPv4/20 route entries. DUT1 and DUT2 -| ... | tested with 2p10GE NIC X520 Niantic by Intel. -| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop -| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop -| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage -| ... | of packets transmitted. NDR and PDR are discovered for different -| ... | Ethernet L2 frame sizes using either binary search or linear search -| ... | algorithms with configured starting rate and final step that determines -| ... | throughput measurement resolution. Test packets are generated by TG on -| ... | links to DUTs. TG traffic profile contains two L3 flow-groups -| ... | (flow-group per direction, one flow per flow-group) with all packets -| ... | containing Ethernet header, IPv4 header with UDP header and static -| ... | payload. MAC addresses are matching MAC addresses of the TG node -| ... | interfaces. -| ... | *[Ref] Applicable standard specifications:* RFC2544. - -*** Variables *** -# X520-DA2 bandwidth limit -| ${s_limit} | ${10000000000} -# Traffic profile: -| ${traffic_profile} | trex-sl-3n-ethip4udp-1000u15p - -*** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} -| | ... -| | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ... -| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} -| | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs -| | And Add NAT to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | When Initialize NAT44 in 3-node circular topology -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} - -*** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4scale1000-udpsrcscale15-snat-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR - -| tc02-64B-1t1c-ethip4udp-ip4scale1000-udpsrcscale15-snat-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR - -| tc03-1518B-1t1c-ethip4udp-ip4scale1000-udpsrcscale15-snat-ndrdisc -| | [Tags] | 1518B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${1518} | search_type=NDR - -| tc04-1518B-1t1c-ethip4udp-ip4scale1000-udpsrcscale15-snat-pdrdisc -| | [Tags] | 1518B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${1518} | search_type=PDR - -| tc05-IMIX-1t1c-ethip4udp-ip4scale1000-udpsrcscale15-snat-ndrdisc -| | [Tags] | IMIX | 1C | NDRDISC -| | phy_cores=${1} | framesize=IMIX_v4_1 | search_type=NDR - -| tc06-IMIX-1t1c-ethip4udp-ip4scale1000-udpsrcscale15-snat-pdrdisc -| | [Tags] | IMIX | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=IMIX_v4_1 | search_type=PDR - -| tc07-64B-2t2c-ethip4udp-ip4scale1000-udpsrcscale15-snat-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR - -| tc08-64B-2t2c-ethip4udp-ip4scale1000-udpsrcscale15-snat-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR - -| tc09-1518B-2t2c-ethip4udp-ip4scale1000-udpsrcscale15-snat-ndrdisc -| | [Tags] | 1518B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${1518} | search_type=NDR - -| tc10-1518B-2t2c-ethip4udp-ip4scale1000-udpsrcscale15-snat-pdrdisc -| | [Tags] | 1518B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${1518} | search_type=PDR - -| tc11-IMIX-2t2c-ethip4udp-ip4scale1000-udpsrcscale15-snat-ndrdisc -| | [Tags] | IMIX | 2C | NDRDISC -| | phy_cores=${2} | framesize=IMIX_v4_1 | search_type=NDR - -| tc12-IMIX-2t2c-ethip4udp-ip4scale1000-udpsrcscale15-snat-pdrdisc -| | [Tags] | IMIX | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=IMIX_v4_1 | search_type=PDR - -| tc13-64B-4t4c-ethip4udp-ip4scale1000-udpsrcscale15-snat-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR - -| tc14-64B-4t4c-ethip4udp-ip4scale1000-udpsrcscale15-snat-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR - -| tc15-1518B-4t4c-ethip4udp-ip4scale1000-udpsrcscale15-snat-ndrdisc -| | [Tags] | 1518B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${1518} | search_type=NDR - -| tc16-1518B-4t4c-ethip4udp-ip4scale1000-udpsrcscale15-snat-pdrdisc -| | [Tags] | 1518B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${1518} | search_type=PDR - -| tc17-IMIX-4t4c-ethip4udp-ip4scale1000-udpsrcscale15-snat-ndrdisc -| | [Tags] | IMIX | 4C | NDRDISC -| | phy_cores=${4} | framesize=IMIX_v4_1 | search_type=NDR - -| tc18-IMIX-4t4c-ethip4udp-ip4scale1000-udpsrcscale15-snat-pdrdisc -| | [Tags] | IMIX | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=IMIX_v4_1 | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale2000-udpsrcscale15-nat44-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale2000-udpsrcscale15-nat44-mrr.robot new file mode 100644 index 0000000000..7405c02675 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale2000-udpsrcscale15-nat44-mrr.robot @@ -0,0 +1,121 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| Resource | resources/libraries/robot/ip/nat.robot +| Resource | resources/libraries/robot/shared/traffic.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | MRR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | NAT44 | SRC_USER_2000 +| ... | SCALE +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with NAT +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down mrr test with NAT +| ... +| Test Template | Local Template +| ... +| Documentation | *Raw results NAT44 performance test cases* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for IPv4 routing. +| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 +| ... | routing and two static IPv4 /24 and IPv4/20 route entries. DUT1 and DUT2 +| ... | tested with 2p10GE NIC X520 Niantic by Intel. +| ... | *[Ver] TG verification:* In MaxReceivedRate test TG sends traffic +| ... | at line rate and reports total received/sent packets over trial period. +| ... | Test packets are generated by TG on +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups +| ... | (flow-group per direction, one flow per flow-group) with all packets +| ... | containing Ethernet header, IPv4 header with UDP header and static +| ... | payload. MAC addresses are matching MAC addresses of the TG node +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit} | ${10000000000} +# Traffic profile: +| ${traffic_profile} | trex-sl-3n-ethip4udp-2000u15p + +*** Keywords *** +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Add NAT to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | When Initialize NAT44 in 3-node circular topology +| | Then Traffic should pass with maximum rate +| | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4scale2000-udpsrcscale15-nat44-mrr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4scale2000-udpsrcscale15-nat44-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4scale2000-udpsrcscale15-nat44-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4scale2000-udpsrcscale15-nat44-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4scale2000-udpsrcscale15-nat44-mrr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4scale2000-udpsrcscale15-nat44-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4scale2000-udpsrcscale15-nat44-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4scale2000-udpsrcscale15-nat44-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4scale2000-udpsrcscale15-nat44-mrr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4scale2000-udpsrcscale15-nat44-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4scale2000-udpsrcscale15-nat44-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4scale2000-udpsrcscale15-nat44-mrr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale2000-udpsrcscale15-nat44-ndrpdr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale2000-udpsrcscale15-nat44-ndrpdr.robot new file mode 100644 index 0000000000..e2ac029088 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale2000-udpsrcscale15-nat44-ndrpdr.robot @@ -0,0 +1,128 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| Resource | resources/libraries/robot/ip/nat.robot +| Resource | resources/libraries/robot/shared/traffic.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | NAT44 | SRC_USER_2000 +| ... | SCALE +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with NAT +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down performance discovery test with NAT +| ... | ${min_rate}pps | ${framesize} | ${traffic_profile} +| ... +| Test Template | Local Template +| ... +| Documentation | *RFC2544: Pkt throughput NAT44 performance test cases* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for IPv4 routing. +| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 +| ... | routing and two static IPv4 /24 and IPv4/20 route entries. DUT1 and DUT2 +| ... | tested with 2p10GE NIC X520 Niantic by Intel. +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage +| ... | of packets transmitted. NDR and PDR are discovered for different +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups +| ... | (flow-group per direction, one flow per flow-group) with all packets +| ... | containing Ethernet header, IPv4 header with UDP header and static +| ... | payload. MAC addresses are matching MAC addresses of the TG node +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit} | ${10000000000} +# Traffic profile: +| ${traffic_profile} | trex-sl-3n-ethip4udp-2000u15p + +*** Keywords *** +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Add NAT to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | When Initialize NAT44 in 3-node circular topology +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4scale2000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4scale2000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4scale2000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4scale2000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4scale2000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4scale2000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4scale2000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4scale2000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4scale2000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4scale2000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4scale2000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4scale2000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale2000-udpsrcscale15-nat44-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale2000-udpsrcscale15-nat44-ndrpdrdisc.robot deleted file mode 100644 index d4befa97c4..0000000000 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale2000-udpsrcscale15-nat44-ndrpdrdisc.robot +++ /dev/null @@ -1,165 +0,0 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at: -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -*** Settings *** -| Resource | resources/libraries/robot/performance/performance_setup.robot -| Resource | resources/libraries/robot/ip/nat.robot -| Resource | resources/libraries/robot/shared/traffic.robot -| ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC -| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | NAT44 | SRC_USER_2000 -| ... | SCALE -| ... -| Suite Setup | Run Keywords -| ... | Set up 3-node performance topology with DUT's NIC model | L3 -| ... | Intel-X520-DA2 -| ... | AND | Set up performance test suite with NAT -| Suite Teardown | Tear down 3-node performance topology -| ... -| Test Setup | Set up performance test -| ... -| Test Teardown | Tear down performance discovery test with NAT -| ... | ${min_rate}pps | ${framesize} | ${traffic_profile} -| ... -| Test Template | Local template -| ... -| Documentation | *NAT44 performance test cases* -| ... -| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology -| ... | with single links between nodes. -| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for IPv4 routing. -| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 -| ... | routing and two static IPv4 /24 and IPv4/20 route entries. DUT1 and DUT2 -| ... | tested with 2p10GE NIC X520 Niantic by Intel. -| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop -| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop -| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage -| ... | of packets transmitted. NDR and PDR are discovered for different -| ... | Ethernet L2 frame sizes using either binary search or linear search -| ... | algorithms with configured starting rate and final step that determines -| ... | throughput measurement resolution. Test packets are generated by TG on -| ... | links to DUTs. TG traffic profile contains two L3 flow-groups -| ... | (flow-group per direction, one flow per flow-group) with all packets -| ... | containing Ethernet header, IPv4 header with UDP header and static -| ... | payload. MAC addresses are matching MAC addresses of the TG node -| ... | interfaces. -| ... | *[Ref] Applicable standard specifications:* RFC2544. - -*** Variables *** -# X520-DA2 bandwidth limit -| ${s_limit} | ${10000000000} -# Traffic profile: -| ${traffic_profile} | trex-sl-3n-ethip4udp-2000u15p - -*** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} -| | ... -| | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ... -| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} -| | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs -| | And Add NAT to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | When Initialize NAT44 in 3-node circular topology -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} - -*** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4scale2000-udpsrcscale15-snat-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR - -| tc02-64B-1t1c-ethip4udp-ip4scale2000-udpsrcscale15-snat-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR - -| tc03-1518B-1t1c-ethip4udp-ip4scale2000-udpsrcscale15-snat-ndrdisc -| | [Tags] | 1518B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${1518} | search_type=NDR - -| tc04-1518B-1t1c-ethip4udp-ip4scale2000-udpsrcscale15-snat-pdrdisc -| | [Tags] | 1518B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${1518} | search_type=PDR - -| tc05-IMIX-1t1c-ethip4udp-ip4scale2000-udpsrcscale15-snat-ndrdisc -| | [Tags] | IMIX | 1C | NDRDISC -| | phy_cores=${1} | framesize=IMIX_v4_1 | search_type=NDR - -| tc06-IMIX-1t1c-ethip4udp-ip4scale2000-udpsrcscale15-snat-pdrdisc -| | [Tags] | IMIX | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=IMIX_v4_1 | search_type=PDR - -| tc07-64B-2t2c-ethip4udp-ip4scale2000-udpsrcscale15-snat-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR - -| tc08-64B-2t2c-ethip4udp-ip4scale2000-udpsrcscale15-snat-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR - -| tc09-1518B-2t2c-ethip4udp-ip4scale2000-udpsrcscale15-snat-ndrdisc -| | [Tags] | 1518B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${1518} | search_type=NDR - -| tc10-1518B-2t2c-ethip4udp-ip4scale2000-udpsrcscale15-snat-pdrdisc -| | [Tags] | 1518B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${1518} | search_type=PDR - -| tc11-IMIX-2t2c-ethip4udp-ip4scale2000-udpsrcscale15-snat-ndrdisc -| | [Tags] | IMIX | 2C | NDRDISC -| | phy_cores=${2} | framesize=IMIX_v4_1 | search_type=NDR - -| tc12-IMIX-2t2c-ethip4udp-ip4scale2000-udpsrcscale15-snat-pdrdisc -| | [Tags] | IMIX | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=IMIX_v4_1 | search_type=PDR - -| tc13-64B-4t4c-ethip4udp-ip4scale2000-udpsrcscale15-snat-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR - -| tc14-64B-4t4c-ethip4udp-ip4scale2000-udpsrcscale15-snat-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR - -| tc15-1518B-4t4c-ethip4udp-ip4scale2000-udpsrcscale15-snat-ndrdisc -| | [Tags] | 1518B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${1518} | search_type=NDR - -| tc16-1518B-4t4c-ethip4udp-ip4scale2000-udpsrcscale15-snat-pdrdisc -| | [Tags] | 1518B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${1518} | search_type=PDR - -| tc17-IMIX-4t4c-ethip4udp-ip4scale2000-udpsrcscale15-snat-ndrdisc -| | [Tags] | IMIX | 4C | NDRDISC -| | phy_cores=${4} | framesize=IMIX_v4_1 | search_type=NDR - -| tc18-IMIX-4t4c-ethip4udp-ip4scale2000-udpsrcscale15-snat-pdrdisc -| | [Tags] | IMIX | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=IMIX_v4_1 | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale4000-udpsrcscale15-nat44-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale4000-udpsrcscale15-nat44-mrr.robot new file mode 100644 index 0000000000..c835c5bbf6 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale4000-udpsrcscale15-nat44-mrr.robot @@ -0,0 +1,121 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| Resource | resources/libraries/robot/ip/nat.robot +| Resource | resources/libraries/robot/shared/traffic.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | MRR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | NAT44 | SRC_USER_4000 +| ... | SCALE +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with NAT +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down mrr test with NAT +| ... +| Test Template | Local Template +| ... +| Documentation | *Raw results NAT44 performance test cases* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for IPv4 routing. +| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 +| ... | routing and two static IPv4 /24 and IPv4/20 route entries. DUT1 and DUT2 +| ... | tested with 2p10GE NIC X520 Niantic by Intel. +| ... | *[Ver] TG verification:* In MaxReceivedRate test TG sends traffic +| ... | at line rate and reports total received/sent packets over trial period. +| ... | Test packets are generated by TG on +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups +| ... | (flow-group per direction, one flow per flow-group) with all packets +| ... | containing Ethernet header, IPv4 header with UDP header and static +| ... | payload. MAC addresses are matching MAC addresses of the TG node +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit} | ${10000000000} +# Traffic profile: +| ${traffic_profile} | trex-sl-3n-ethip4udp-4000u15p + +*** Keywords *** +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Add NAT to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | When Initialize NAT44 in 3-node circular topology +| | Then Traffic should pass with maximum rate +| | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4scale4000-udpsrcscale15-nat44-mrr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4scale4000-udpsrcscale15-nat44-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4scale4000-udpsrcscale15-nat44-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4scale4000-udpsrcscale15-nat44-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4scale4000-udpsrcscale15-nat44-mrr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4scale4000-udpsrcscale15-nat44-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4scale4000-udpsrcscale15-nat44-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4scale4000-udpsrcscale15-nat44-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4scale4000-udpsrcscale15-nat44-mrr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4scale4000-udpsrcscale15-nat44-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4scale4000-udpsrcscale15-nat44-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4scale4000-udpsrcscale15-nat44-mrr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale4000-udpsrcscale15-nat44-ndrpdr.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale4000-udpsrcscale15-nat44-ndrpdr.robot new file mode 100644 index 0000000000..0e4a800ecf --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale4000-udpsrcscale15-nat44-ndrpdr.robot @@ -0,0 +1,128 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| Resource | resources/libraries/robot/ip/nat.robot +| Resource | resources/libraries/robot/shared/traffic.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR +| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | NAT44 | SRC_USER_4000 +| ... | SCALE +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X520-DA2 +| ... | AND | Set up performance test suite with NAT +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down performance discovery test with NAT +| ... | ${min_rate}pps | ${framesize} | ${traffic_profile} +| ... +| Test Template | Local Template +| ... +| Documentation | *RFC2544: Pkt throughput NAT44 performance test cases* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for IPv4 routing. +| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 +| ... | routing and two static IPv4 /24 and IPv4/20 route entries. DUT1 and DUT2 +| ... | tested with 2p10GE NIC X520 Niantic by Intel. +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage +| ... | of packets transmitted. NDR and PDR are discovered for different +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on +| ... | links to DUTs. TG traffic profile contains two L3 flow-groups +| ... | (flow-group per direction, one flow per flow-group) with all packets +| ... | containing Ethernet header, IPv4 header with UDP header and static +| ... | payload. MAC addresses are matching MAC addresses of the TG node +| ... | interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X520-DA2 bandwidth limit +| ${s_limit} | ${10000000000} +# Traffic profile: +| ${traffic_profile} | trex-sl-3n-ethip4udp-4000u15p + +*** Keywords *** +| Local Template +| | [Documentation] | FIXME. +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Add NAT to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | When Initialize NAT44 in 3-node circular topology +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4scale4000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4scale4000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4scale4000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4scale4000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4scale4000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4scale4000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4scale4000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4scale4000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4scale4000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4scale4000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4scale4000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4scale4000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale4000-udpsrcscale15-nat44-ndrpdrdisc.robot b/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale4000-udpsrcscale15-nat44-ndrpdrdisc.robot deleted file mode 100644 index da11430106..0000000000 --- a/tests/vpp/perf/ip4/10ge2p1x520-ethip4udp-ip4scale4000-udpsrcscale15-nat44-ndrpdrdisc.robot +++ /dev/null @@ -1,165 +0,0 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at: -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -*** Settings *** -| Resource | resources/libraries/robot/performance/performance_setup.robot -| Resource | resources/libraries/robot/ip/nat.robot -| Resource | resources/libraries/robot/shared/traffic.robot -| ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC -| ... | NIC_Intel-X520-DA2 | ETH | IP4FWD | FEATURE | NAT44 | SRC_USER_4000 -| ... | SCALE -| ... -| Suite Setup | Run Keywords -| ... | Set up 3-node performance topology with DUT's NIC model | L3 -| ... | Intel-X520-DA2 -| ... | AND | Set up performance test suite with NAT -| Suite Teardown | Tear down 3-node performance topology -| ... -| Test Setup | Set up performance test -| ... -| Test Teardown | Tear down performance discovery test with NAT -| ... | ${min_rate}pps | ${framesize} | ${traffic_profile} -| ... -| Test Template | Local template -| ... -| Documentation | *NAT44 performance test cases* -| ... -| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology -| ... | with single links between nodes. -| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for IPv4 routing. -| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 -| ... | routing and two static IPv4 /24 and IPv4/20 route entries. DUT1 and DUT2 -| ... | tested with 2p10GE NIC X520 Niantic by Intel. -| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop -| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop -| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage -| ... | of packets transmitted. NDR and PDR are discovered for different -| ... | Ethernet L2 frame sizes using either binary search or linear search -| ... | algorithms with configured starting rate and final step that determines -| ... | throughput measurement resolution. Test packets are generated by TG on -| ... | links to DUTs. TG traffic profile contains two L3 flow-groups -| ... | (flow-group per direction, one flow per flow-group) with all packets -| ... | containing Ethernet header, IPv4 header with UDP header and static -| ... | payload. MAC addresses are matching MAC addresses of the TG node -| ... | interfaces. -| ... | *[Ref] Applicable standard specifications:* RFC2544. - -*** Variables *** -# X520-DA2 bandwidth limit -| ${s_limit} | ${10000000000} -# Traffic profile: -| ${traffic_profile} | trex-sl-3n-ethip4udp-4000u15p - -*** Keywords *** -| Local template -| | [Arguments] | ${phy_cores} | ${framesize} | ${search_type} -| | ... | ${rxq}=${None} | ${min_rate}=${50000} -| | ... -| | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ... -| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} -| | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs -| | And Add NAT to all DUTs -| | And Apply startup configuration on all VPP DUTs -| | When Initialize NAT44 in 3-node circular topology -| | Then Run Keyword If | '${search_type}' == 'NDR' -| | ... | Find NDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ELSE IF | '${search_type}' == 'PDR' -| | ... | Find PDR using binary search and pps -| | ... | ${framesize} | ${binary_min} | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} -| | ... | ${perf_pdr_loss_acceptance} | ${perf_pdr_loss_acceptance_type} - -*** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4scale4000-udpsrcscale15-snat-ndrdisc -| | [Tags] | 64B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${64} | search_type=NDR - -| tc02-64B-1t1c-ethip4udp-ip4scale4000-udpsrcscale15-snat-pdrdisc -| | [Tags] | 64B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${64} | search_type=PDR - -| tc03-1518B-1t1c-ethip4udp-ip4scale4000-udpsrcscale15-snat-ndrdisc -| | [Tags] | 1518B | 1C | NDRDISC -| | phy_cores=${1} | framesize=${1518} | search_type=NDR - -| tc04-1518B-1t1c-ethip4udp-ip4scale4000-udpsrcscale15-snat-pdrdisc -| | [Tags] | 1518B | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=${1518} | search_type=PDR - -| tc05-IMIX-1t1c-ethip4udp-ip4scale4000-udpsrcscale15-snat-ndrdisc -| | [Tags] | IMIX | 1C | NDRDISC -| | phy_cores=${1} | framesize=IMIX_v4_1 | search_type=NDR - -| tc06-IMIX-1t1c-ethip4udp-ip4scale4000-udpsrcscale15-snat-pdrdisc -| | [Tags] | IMIX | 1C | PDRDISC | SKIP_PATCH -| | phy_cores=${1} | framesize=IMIX_v4_1 | search_type=PDR - -| tc07-64B-2t2c-ethip4udp-ip4scale4000-udpsrcscale15-snat-ndrdisc -| | [Tags] | 64B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${64} | search_type=NDR - -| tc08-64B-2t2c-ethip4udp-ip4scale4000-udpsrcscale15-snat-pdrdisc -| | [Tags] | 64B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${64} | search_type=PDR - -| tc09-1518B-2t2c-ethip4udp-ip4scale4000-udpsrcscale15-snat-ndrdisc -| | [Tags] | 1518B | 2C | NDRDISC -| | phy_cores=${2} | framesize=${1518} | search_type=NDR - -| tc10-1518B-2t2c-ethip4udp-ip4scale4000-udpsrcscale15-snat-pdrdisc -| | [Tags] | 1518B | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=${1518} | search_type=PDR - -| tc11-IMIX-2t2c-ethip4udp-ip4scale4000-udpsrcscale15-snat-ndrdisc -| | [Tags] | IMIX | 2C | NDRDISC -| | phy_cores=${2} | framesize=IMIX_v4_1 | search_type=NDR - -| tc12-IMIX-2t2c-ethip4udp-ip4scale4000-udpsrcscale15-snat-pdrdisc -| | [Tags] | IMIX | 2C | PDRDISC | SKIP_PATCH -| | phy_cores=${2} | framesize=IMIX_v4_1 | search_type=PDR - -| tc13-64B-4t4c-ethip4udp-ip4scale4000-udpsrcscale15-snat-ndrdisc -| | [Tags] | 64B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${64} | search_type=NDR - -| tc14-64B-4t4c-ethip4udp-ip4scale4000-udpsrcscale15-snat-pdrdisc -| | [Tags] | 64B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${64} | search_type=PDR - -| tc15-1518B-4t4c-ethip4udp-ip4scale4000-udpsrcscale15-snat-ndrdisc -| | [Tags] | 1518B | 4C | NDRDISC -| | phy_cores=${4} | framesize=${1518} | search_type=NDR - -| tc16-1518B-4t4c-ethip4udp-ip4scale4000-udpsrcscale15-snat-pdrdisc -| | [Tags] | 1518B | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=${1518} | search_type=PDR - -| tc17-IMIX-4t4c-ethip4udp-ip4scale4000-udpsrcscale15-snat-ndrdisc -| | [Tags] | IMIX | 4C | NDRDISC -| | phy_cores=${4} | framesize=IMIX_v4_1 | search_type=NDR - -| tc18-IMIX-4t4c-ethip4udp-ip4scale4000-udpsrcscale15-snat-pdrdisc -| | [Tags] | IMIX | 4C | PDRDISC | SKIP_PATCH -| | phy_cores=${4} | framesize=IMIX_v4_1 | search_type=PDR diff --git a/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4base-copwhtlistbase-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4base-copwhtlistbase-mrr.robot index 5650f473f7..d0b3be4ca6 100644 --- a/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4base-copwhtlistbase-mrr.robot +++ b/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4base-copwhtlistbase-mrr.robot @@ -26,7 +26,7 @@ | ... | Test Teardown | Tear down performance mrr test | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *Raw results IPv4 whitelist test cases* | ... @@ -53,7 +53,7 @@ | ${traffic_profile}= | trex-sl-3n-ethip4-ip4src253 *** Keywords *** -| Local template +| Local Template | | ... | | [Documentation] | | ... | [Cfg] DUT runs IPv4 routing config. @@ -67,16 +67,12 @@ | | ... | - phy_cores - Number of physical cores. Type: integer | | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer | | ... -| | [Arguments] | ${phy_cores} | ${framesize} | ${rxq}=${None} -| | ... -| | Set Test Variable | ${framesize} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${get_framesize} +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | When Initialize IPv4 forwarding in 3-node circular topology | | And Add Fib Table | ${dut1} | 1 @@ -91,50 +87,50 @@ | | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} *** Test Cases *** -| tc01-64B-1t1c-ethip4-ip4base-copwhtlistbase-mrr +| tc01-64B-1c-ethip4-ip4base-copwhtlistbase-mrr | | [Tags] | 64B | 1C -| | phy_cores=${1} | framesize=${64} - -| tc02-1518B-1t1c-ethip4-ip4base-copwhtlistbase-mrr -| | [Tags] | 1518B | 1C -| | phy_cores=${1} | framesize=${1518} +| | framesize=${64} | phy_cores=${1} -| tc03-9000B-1t1c-ethip4-ip4base-copwhtlistbase-mrr -| | [Tags] | 9000B | 1C -| | phy_cores=${1} | framesize=${9000} +| tc02-64B-2c-ethip4-ip4base-copwhtlistbase-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} -| tc04-IMIX-1t1c-ethip4-ip4base-copwhtlistbase-mrr -| | [Tags] | IMIX | 1C -| | phy_cores=${1} | framesize=IMIX_v4_1 +| tc03-64B-4c-ethip4-ip4base-copwhtlistbase-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} -| tc05-64B-2t2c-ethip4-ip4base-copwhtlistbase-mrr -| | [Tags] | 64B | 2C -| | phy_cores=${2} | framesize=${64} +| tc04-1518B-1c-ethip4-ip4base-copwhtlistbase-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} -| tc06-1518B-2t2c-ethip4-ip4base-copwhtlistbase-mrr +| tc05-1518B-2c-ethip4-ip4base-copwhtlistbase-mrr | | [Tags] | 1518B | 2C -| | phy_cores=${2} | framesize=${1518} - -| tc07-9000B-2t2c-ethip4-ip4base-copwhtlistbase-mrr -| | [Tags] | 9000B | 2C -| | phy_cores=${2} | framesize=${9000} +| | framesize=${1518} | phy_cores=${2} -| tc08-IMIX-2t2c-ethip4-ip4base-copwhtlistbase-mrr -| | [Tags] | IMIX | 2C -| | phy_cores=${2} | framesize=IMIX_v4_1 +| tc06-1518B-4c-ethip4-ip4base-copwhtlistbase-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} -| tc09-64B-4t4c-ethip4-ip4base-copwhtlistbase-mrr -| | [Tags] | 64B | 4C -| | phy_cores=${4} | framesize=${64} +| tc07-9000B-1c-ethip4-ip4base-copwhtlistbase-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc10-1518B-4t4c-ethip4-ip4base-copwhtlistbase-mrr -| | [Tags] | 1518B | 4C -| | phy_cores=${4} | framesize=${1518} +| tc08-9000B-2c-ethip4-ip4base-copwhtlistbase-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc11-9000B-4t4c-ethip4-ip4base-copwhtlistbase-mrr +| tc09-9000B-4c-ethip4-ip4base-copwhtlistbase-mrr | | [Tags] | 9000B | 4C -| | phy_cores=${4} | framesize=${9000} +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4-ip4base-copwhtlistbase-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4-ip4base-copwhtlistbase-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc12-IMIX-4t4c-ethip4-ip4base-copwhtlistbase-mrr +| tc12-IMIX-4c-ethip4-ip4base-copwhtlistbase-mrr | | [Tags] | IMIX | 4C -| | phy_cores=${4} | framesize=IMIX_v4_1 +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4base-copwhtlistbase-ndrpdr.robot b/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4base-copwhtlistbase-ndrpdr.robot new file mode 100644 index 0000000000..ad4a4b66c4 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4base-copwhtlistbase-ndrpdr.robot @@ -0,0 +1,142 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| Library | resources.libraries.python.Cop +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR +| ... | NIC_Intel-X710 | ETH | IP4FWD | FEATURE | COPWHLIST +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X710 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down performance discovery test | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Test Template | Local Template +| ... +| Documentation | *RFC2544: Pkt throughput IPv4 whitelist test cases* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4 for IPv4 routing. +| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 +| ... | routing, two static IPv4 /24 routes and IPv4 COP security whitelist +| ... | ingress /24 filter entries applied on links TG - DUT1 and DUT2 - TG. +| ... | DUT1 and DUT2 tested with 2p10GE NIC X710 by Intel. +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage +| ... | of packets transmitted. NDR and PDR are discovered for different +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on links to DUTs. TG traffic profile +| ... | contains two L3 flow-groups (flow-group per direction, 253 flows per +| ... | flow-group) with all packets containing Ethernet header, IPv4 header +| ... | with IP protocol=61 and static payload. MAC addresses are matching MAC +| ... | addresses of the TG node interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X710 bandwidth limit +| ${s_limit}= | ${10000000000} +# Traffic profile: +| ${traffic_profile}= | trex-sl-3n-ethip4-ip4src253 + +*** Keywords *** +| Local Template +| | ... +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config. +| | ... | Each DUT uses ${phy_cores} physical core(s) for worker threads. +| | ... | [Ver] Measure NDR and PDR values using MLRsearch algorithm.\ +| | ... +| | ... | *Arguments:* +| | ... | - framesize - Framesize in Bytes in integer or string (IMIX_v4_1). +| | ... | Type: integer, string +| | ... | - phy_cores - Number of physical cores. Type: integer +| | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer +| | ... +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | When Initialize IPv4 forwarding in 3-node circular topology +| | And Add Fib Table | ${dut1} | 1 +| | And Vpp Route Add | ${dut1} | 10.10.10.0 | 24 |Â vrf=1 | local=${TRUE} +| | And Add Fib Table | ${dut2} | 1 +| | And Vpp Route Add | ${dut2} | 20.20.20.0 | 24 |Â vrf=1 | local=${TRUE} +| | And COP Add whitelist Entry | ${dut1} | ${dut1_if1} | ip4 | 1 +| | And COP Add whitelist Entry | ${dut2} | ${dut2_if2} | ip4 | 1 +| | And COP interface enable or disable | ${dut1} | ${dut1_if1} | enable +| | And COP interface enable or disable | ${dut2} | ${dut2_if2} | enable +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} + +*** Test Cases *** +| tc01-64B-1c-ethip4-ip4base-copwhtlistbase-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4-ip4base-copwhtlistbase-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4-ip4base-copwhtlistbase-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4-ip4base-copwhtlistbase-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4-ip4base-copwhtlistbase-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4-ip4base-copwhtlistbase-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4-ip4base-copwhtlistbase-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4-ip4base-copwhtlistbase-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4-ip4base-copwhtlistbase-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4-ip4base-copwhtlistbase-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4-ip4base-copwhtlistbase-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4-ip4base-copwhtlistbase-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4base-iacldstbase-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4base-iacldstbase-mrr.robot index d4937e700d..daf3acdca1 100644 --- a/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4base-iacldstbase-mrr.robot +++ b/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4base-iacldstbase-mrr.robot @@ -25,7 +25,7 @@ | ... | Test Teardown | Tear down performance mrr test | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *Raw results IPv4 iAcl whitelist test cases* | ... @@ -52,7 +52,7 @@ | ${traffic_profile}= | trex-sl-3n-ethip4-ip4src253 *** Keywords *** -| Local template +| Local Template | | ... | | [Documentation] | | ... | [Cfg] DUT runs IPv4 routing config. @@ -66,16 +66,12 @@ | | ... | - phy_cores - Number of physical cores. Type: integer | | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer | | ... -| | [Arguments] | ${phy_cores} | ${framesize} | ${rxq}=${None} -| | ... -| | Set Test Variable | ${framesize} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${get_framesize} +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | When Initialize IPv4 forwarding in 3-node circular topology | | ${table_idx} | ${skip_n} | ${match_n}= | And Vpp Creates Classify Table L3 @@ -96,50 +92,50 @@ | | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} *** Test Cases *** -| tc01-64B-1t1c-ethip4-ip4base-iacldstbase-mrr +| tc01-64B-1c-ethip4-ip4base-iacldstbase-mrr | | [Tags] | 64B | 1C -| | phy_cores=${1} | framesize=${64} - -| tc02-1518B-1t1c-ethip4-ip4base-iacldstbase-mrr -| | [Tags] | 1518B | 1C -| | phy_cores=${1} | framesize=${1518} +| | framesize=${64} | phy_cores=${1} -| tc03-9000B-1t1c-ethip4-ip4base-iacldstbase-mrr -| | [Tags] | 9000B | 1CC -| | phy_cores=${1} | framesize=${9000} +| tc02-64B-2c-ethip4-ip4base-iacldstbase-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} -| tc04-IMIX-1t1c-ethip4-ip4base-iacldstbase-mrr -| | [Tags] | IMIX | 1C -| | phy_cores=${1} | framesize=IMIX_v4_1 +| tc03-64B-4c-ethip4-ip4base-iacldstbase-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} -| tc05-64B-2t2c-ethip4-ip4base-iacldstbase-mrr -| | [Tags] | 64B | 2C -| | phy_cores=${2} | framesize=${64} +| tc04-1518B-1c-ethip4-ip4base-iacldstbase-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} -| tc06-1518B-2t2c-ethip4-ip4base-iacldstbase-mrr +| tc05-1518B-2c-ethip4-ip4base-iacldstbase-mrr | | [Tags] | 1518B | 2C -| | phy_cores=${2} | framesize=${1518} - -| tc07-9000B-2t2c-ethip4-ip4base-iacldstbase-mrr -| | [Tags] | 9000B | 2C -| | phy_cores=${2} | framesize=${9000} +| | framesize=${1518} | phy_cores=${2} -| tc08-IMIX-2t2c-ethip4-ip4base-iacldstbase-mrr -| | [Tags] | IMIX | 2C -| | phy_cores=${2} | framesize=IMIX_v4_1 +| tc06-1518B-4c-ethip4-ip4base-iacldstbase-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} -| tc09-64B-4t4c-ethip4-ip4base-iacldstbase-mrr -| | [Tags] | 64B | 4C -| | phy_cores=${4} | framesize=${64} +| tc07-9000B-1c-ethip4-ip4base-iacldstbase-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc10-1518B-4t4c-ethip4-ip4base-iacldstbase-mrr -| | [Tags] | 1518B | 4C -| | phy_cores=${4} | framesize=${1518} +| tc08-9000B-2c-ethip4-ip4base-iacldstbase-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc11-9000B-4t4c-ethip4-ip4base-iacldstbase-mrr +| tc09-9000B-4c-ethip4-ip4base-iacldstbase-mrr | | [Tags] | 9000B | 4C -| | phy_cores=${4} | framesize=${9000} +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4-ip4base-iacldstbase-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4-ip4base-iacldstbase-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc12-IMIX-4t4c-ethip4-ip4base-iacldstbase-mrr +| tc12-IMIX-4c-ethip4-ip4base-iacldstbase-mrr | | [Tags] | IMIX | 4C -| | phy_cores=${4} | framesize=IMIX_v4_1 +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4base-iacldstbase-ndrpdr.robot b/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4base-iacldstbase-ndrpdr.robot new file mode 100644 index 0000000000..38f7cd7dd3 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4base-iacldstbase-ndrpdr.robot @@ -0,0 +1,147 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR +| ... | NIC_Intel-X710 | ETH | IP4FWD | FEATURE | IACLDST +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X710 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down performance discovery test | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Test Template | Local Template +| ... +| Documentation | *RFC2544: Pkt throughput IPv4 iAcl whitelist test cases* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4 for IPv4 routing. +| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 +| ... | routing, two static IPv4 /24 routes and IPv4 iAcl security whitelist +| ... | ingress /24 filter entries applied on links TG - DUT1 and DUT2 - TG. +| ... | DUT1 and DUT2 tested with 2p10GE NIC X710 by Intel. +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage +| ... | of packets transmitted. NDR and PDR are discovered for different +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on links to DUTs. TG traffic profile +| ... | contains two L3 flow-groups (flow-group per direction, 253 flows per +| ... | flow-group) with all packets containing Ethernet header, IPv6 header +| ... | and generated payload. MAC addresses are matching MAC addresses +| ... | of the TG node interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X710 bandwidth limit +| ${s_limit}= | ${10000000000} +# Traffic profile: +| ${traffic_profile}= | trex-sl-3n-ethip4-ip4src253 + +*** Keywords *** +| Local Template +| | ... +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config. +| | ... | Each DUT uses ${phy_cores} physical core(s) for worker threads. +| | ... | [Ver] Measure NDR and PDR values using MLRsearch algorithm.\ +| | ... +| | ... | *Arguments:* +| | ... | - framesize - Framesize in Bytes in integer or string (IMIX_v4_1). +| | ... | Type: integer, string +| | ... | - phy_cores - Number of physical cores. Type: integer +| | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer +| | ... +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | When Initialize IPv4 forwarding in 3-node circular topology +| | ${table_idx} | ${skip_n} | ${match_n}= | And Vpp Creates Classify Table L3 +| | ... | ${dut1} | ip4 | dst +| | And Vpp Configures Classify Session L3 +| | ... | ${dut1} | permit | ${table_idx} | ${skip_n} | ${match_n} +| | ... | ip4 | dst | 20.20.20.2 +| | And Vpp Enable Input Acl Interface +| | ... | ${dut1} | ${dut1_if1} | ip4 | ${table_idx} +| | ${table_idx} | ${skip_n} | ${match_n}= | And Vpp Creates Classify Table L3 +| | ... | ${dut2} | ip4 | dst +| | And Vpp Configures Classify Session L3 +| | ... | ${dut2} | permit | ${table_idx} | ${skip_n} | ${match_n} +| | ... | ip4 | dst | 10.10.10.2 +| | And Vpp Enable Input Acl Interface +| | ... | ${dut2} | ${dut2_if2} | ip4 | ${table_idx} +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} + +*** Test Cases *** +| tc01-64B-1c-ethip4-ip4base-iacldstbase-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4-ip4base-iacldstbase-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4-ip4base-iacldstbase-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4-ip4base-iacldstbase-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4-ip4base-iacldstbase-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4-ip4base-iacldstbase-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4-ip4base-iacldstbase-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4-ip4base-iacldstbase-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4-ip4base-iacldstbase-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4-ip4base-iacldstbase-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4-ip4base-iacldstbase-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4-ip4base-iacldstbase-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4base-ipolicemarkbase-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4base-ipolicemarkbase-mrr.robot index 277932a18c..f521200a46 100644 --- a/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4base-ipolicemarkbase-mrr.robot +++ b/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4base-ipolicemarkbase-mrr.robot @@ -26,7 +26,7 @@ | ... | Test Teardown | Tear down performance mrr test | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *Raw results IPv4 policer test cases* | ... @@ -55,7 +55,7 @@ | ${traffic_profile}= | trex-sl-3n-ethip4-ip4src253 *** Keywords *** -| Local template +| Local Template | | ... | | [Documentation] | | ... | [Cfg] DUT runs IPv4 routing config. @@ -69,22 +69,15 @@ | | ... | - phy_cores - Number of physical cores. Type: integer | | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer | | ... -| | [Arguments] | ${phy_cores} | ${framesize} | ${rxq}=${None} +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... -| | Set Test Variable | ${framesize} -| | ${cb}= | Set Variable If -| | ... | '${framesize}' == 'IMIX_v4_1' | ${1518} | ${framesize} -| | Set Test Variable | ${cb} -| | ${eb}= | Set Variable If -| | ... | '${framesize}' == 'IMIX_v4_1' | ${1518} | ${framesize} -| | Set Test Variable | ${eb} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${get_framesize} +| | Set Test Variable | ${cb} | ${framesize} +| | Set Test Variable | ${eb} | ${framesize} | | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | When Initialize IPv4 forwarding in 3-node circular topology | | And Initialize IPv4 policer 2r3c-'ca' in 3-node circular topology @@ -92,50 +85,50 @@ | | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} *** Test Cases *** -| tc01-64B-1t1c-ethip4-ip4base-ipolicemarkbase-mrr +| tc01-64B-1c-ethip4-ip4base-ipolicemarkbase-mrr | | [Tags] | 64B | 1C -| | phy_cores=${1} | framesize=${64} +| | framesize=${64} | phy_cores=${1} -| tc02-1518B-1t1c-ethip4-ip4base-ipolicemarkbase-mrr -| | [Tags] | 1518B | 1C -| | phy_cores=${1} | framesize=${1518} - -| tc03-9000B-1t1c-ethip4-ip4base-ipolicemarkbase-mrr -| | [Tags] | 9000B | 1C -| | phy_cores=${1} | framesize=${9000} +| tc02-64B-2c-ethip4-ip4base-ipolicemarkbase-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} -| tc04-IMIX-1t1c-ethip4-ip4base-ipolicemarkbase-mrr -| | [Tags] | IMIX | 1C -| | phy_cores=${1} | framesize=IMIX_v4_1 +| tc03-64B-4c-ethip4-ip4base-ipolicemarkbase-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} -| tc05-64B-2t2c-ethip4-ip4base-ipolicemarkbase-mrr -| | [Tags] | 64B | 2C -| | phy_cores=${2} | framesize=${64} +| tc04-1518B-1c-ethip4-ip4base-ipolicemarkbase-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} -| tc06-1518B-2t2c-ethip4-ip4base-ipolicemarkbase-mrr +| tc05-1518B-2c-ethip4-ip4base-ipolicemarkbase-mrr | | [Tags] | 1518B | 2C -| | phy_cores=${2} | framesize=${1518} - -| tc07-9000B-2t2c-ethip4-ip4base-ipolicemarkbase-mrr -| | [Tags] | 9000B | 2C -| | phy_cores=${2} | framesize=${9000} +| | framesize=${1518} | phy_cores=${2} -| tc08-IMIX-2t2c-ethip4-ip4base-ipolicemarkbase-mrr -| | [Tags] | IMIX | 2C -| | phy_cores=${2} | framesize=IMIX_v4_1 +| tc06-1518B-4c-ethip4-ip4base-ipolicemarkbase-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} -| tc09-64B-4t4c-ethip4-ip4base-ipolicemarkbase-mrr -| | [Tags] | 64B | 4C -| | phy_cores=${4} | framesize=${64} +| tc07-9000B-1c-ethip4-ip4base-ipolicemarkbase-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc10-1518B-4t4c-ethip4-ip4base-ipolicemarkbase-mrr -| | [Tags] | 1518B | 4C -| | phy_cores=${4} | framesize=${1518} +| tc08-9000B-2c-ethip4-ip4base-ipolicemarkbase-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc11-9000B-4t4c-ethip4-ip4base-ipolicemarkbase-mrr +| tc09-9000B-4c-ethip4-ip4base-ipolicemarkbase-mrr | | [Tags] | 9000B | 4C -| | phy_cores=${4} | framesize=${9000} +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4-ip4base-ipolicemarkbase-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4-ip4base-ipolicemarkbase-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc12-IMIX-4t4c-ethip4-ip4base-ipolicemarkbase-mrr +| tc12-IMIX-4c-ethip4-ip4base-ipolicemarkbase-mrr | | [Tags] | IMIX | 4C -| | phy_cores=${4} | framesize=IMIX_v4_1 +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4base-ipolicemarkbase-ndrpdr.robot b/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4base-ipolicemarkbase-ndrpdr.robot new file mode 100644 index 0000000000..92a3b17fde --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4base-ipolicemarkbase-ndrpdr.robot @@ -0,0 +1,139 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| Library | resources.libraries.python.Policer +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR +| ... | NIC_Intel-X710 | IP4FWD | BASE | DOT1Q +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X710 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down performance discovery test | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Test Template | Local Template +| ... +| Documentation | *RFC2544: Pkt throughput IPv4 policer test cases* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4 for IPv4 routing. +| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 +| ... | routing and two static IPv4 /24 route entries and policer with 2R3C +| ... | color-aware configuration. Policer is applied on links TG - DUT1 and +| ... | DUT2 - TG. DUT1 and DUT2 tested with 2p10GE NIC X710 by Intel. +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage +| ... | of packets transmitted. NDR and PDR are discovered for different +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on links to DUTs. TG traffic profile +| ... | contains two L3 flow-groups (flow-group per direction, 253 flows per +| ... | flow-group) with all packets containing Ethernet header, IPv6 header +| ... | and generated payload. MAC addresses are matching MAC addresses +| ... | of the TG node interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544, RFC2698. + +*** Variables *** +# X710 bandwidth limit +| ${s_limit}= | ${10000000000} +| ${cir}= | ${100} +| ${eir}= | ${150} +# Traffic profile: +| ${traffic_profile}= | trex-sl-3n-ethip4-ip4src253 + +*** Keywords *** +| Local Template +| | ... +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config. +| | ... | Each DUT uses ${phy_cores} physical core(s) for worker threads. +| | ... | [Ver] Measure NDR and PDR values using MLRsearch algorithm.\ +| | ... +| | ... | *Arguments:* +| | ... | - framesize - Framesize in Bytes in integer or string (IMIX_v4_1). +| | ... | Type: integer, string +| | ... | - phy_cores - Number of physical cores. Type: integer +| | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer +| | ... +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} | ${10000} +| | Set Test Variable | ${cb} | ${framesize} +| | Set Test Variable | ${eb} | ${framesize} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | When Initialize IPv4 forwarding in 3-node circular topology +| | And Initialize IPv4 policer 2r3c-'ca' in 3-node circular topology +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} + +*** Test Cases *** +| tc01-64B-1c-ethip4-ip4base-ipolicemarkbase-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4-ip4base-ipolicemarkbase-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4-ip4base-ipolicemarkbase-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4-ip4base-ipolicemarkbase-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4-ip4base-ipolicemarkbase-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4-ip4base-ipolicemarkbase-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4-ip4base-ipolicemarkbase-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4-ip4base-ipolicemarkbase-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4-ip4base-ipolicemarkbase-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4-ip4base-ipolicemarkbase-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4-ip4base-ipolicemarkbase-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4-ip4base-ipolicemarkbase-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4base-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4base-mrr.robot index 3e76b7027e..dced5b1e41 100644 --- a/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4base-mrr.robot +++ b/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4base-mrr.robot @@ -24,7 +24,7 @@ | Test Setup | Set up performance test | Test Teardown | Tear down performance mrr test | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *Raw results IPv4 routing test cases* | ... @@ -50,7 +50,7 @@ | ${traffic_profile}= | trex-sl-3n-ethip4-ip4src253 *** Keywords *** -| Local template +| Local Template | | ... | | [Documentation] | | ... | [Cfg] DUT runs IPv4 routing config. @@ -64,66 +64,62 @@ | | ... | - phy_cores - Number of physical cores. Type: integer | | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer | | ... -| | [Arguments] | ${phy_cores} | ${framesize} | ${rxq}=${None} -| | ... -| | Set Test Variable | ${framesize} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | When Initialize IPv4 forwarding in 3-node circular topology | | Then Traffic should pass with maximum rate | | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} *** Test Cases *** -| tc01-64B-1t1c-ethip4-ip4base-mrr +| tc01-64B-1c-ethip4-ip4base-mrr | | [Tags] | 64B | 1C -| | phy_cores=${1} | framesize=${64} - -| tc02-1518B-1t1c-ethip4-ip4base-mrr -| | [Tags] | 1518B | 1C -| | phy_cores=${1} | framesize=${1518} +| | framesize=${64} | phy_cores=${1} -| tc03-9000B-1t1c-ethip4-ip4base-mrr -| | [Tags] | 9000B | 1C -| | phy_cores=${1} | framesize=${9000} +| tc02-64B-2c-ethip4-ip4base-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} -| tc04-IMIX-1t1c-ethip4-ip4base-mrr -| | [Tags] | IMIX | 1C -| | phy_cores=${1} | framesize=IMIX_v4_1 +| tc03-64B-4c-ethip4-ip4base-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} -| tc05-64B-2t2c-ethip4-ip4base-mrr -| | [Tags] | 64B | 2C -| | phy_cores=${2} | framesize=${64} +| tc04-1518B-1c-ethip4-ip4base-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} -| tc06-1518B-2t2c-ethip4-ip4base-mrr +| tc05-1518B-2c-ethip4-ip4base-mrr | | [Tags] | 1518B | 2C -| | phy_cores=${2} | framesize=${1518} - -| tc07-9000B-2t2c-ethip4-ip4base-mrr -| | [Tags] | 9000B | 2C -| | phy_cores=${2} | framesize=${9000} +| | framesize=${1518} | phy_cores=${2} -| tc08-IMIX-2t2c-ethip4-ip4base-mrr -| | [Tags] | IMIX | 2C -| | phy_cores=${2} | framesize=IMIX_v4_1 +| tc06-1518B-4c-ethip4-ip4base-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} -| tc09-64B-4t4c-ethip4-ip4base-mrr -| | [Tags] | 64B | 4C -| | phy_cores=${4} | framesize=${64} +| tc07-9000B-1c-ethip4-ip4base-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc10-1518B-4t4c-ethip4-ip4base-mrr -| | [Tags] | 1518B | 4C -| | phy_cores=${4} | framesize=${1518} +| tc08-9000B-2c-ethip4-ip4base-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc11-9000B-4t4c-ethip4-ip4base-mrr +| tc09-9000B-4c-ethip4-ip4base-mrr | | [Tags] | 9000B | 4C -| | phy_cores=${4} | framesize=${9000} +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4-ip4base-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4-ip4base-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc12-IMIX-4t4c-ethip4-ip4base-mrr +| tc12-IMIX-4c-ethip4-ip4base-mrr | | [Tags] | IMIX | 4C -| | phy_cores=${4} | framesize=IMIX_v4_1 +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4base-ndrpdr.robot b/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4base-ndrpdr.robot new file mode 100644 index 0000000000..151fc6184c --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4base-ndrpdr.robot @@ -0,0 +1,131 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR +| ... | NIC_Intel-X710 | ETH | IP4FWD | BASE | IP4BASE +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X710 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| Test Teardown | Tear down performance discovery test | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Test Template | Local Template +| ... +| Documentation | *RFC2544: Pkt throughput IPv4 routing test cases* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4 for IPv4 routing. +| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 +| ... | routing and two static IPv4 /24 route entries. DUT1 and DUT2 tested with +| ... | 2p10GE NIC X710 by Intel. +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage +| ... | of packets transmitted. NDR and PDR are discovered for different +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on links to DUTs. TG traffic profile +| ... | contains two L3 flow-groups (flow-group per direction, 253 flows per +| ... | flow-group) with all packets containing Ethernet header, IPv4 header +| ... | with IP protocol=61 and static payload. MAC addresses are matching MAC +| ... | addresses of the TG node interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X710 bandwidth limit +| ${s_limit}= | ${10000000000} +# Traffic profile: +| ${traffic_profile}= | trex-sl-3n-ethip4-ip4src253 + +*** Keywords *** +| Local Template +| | ... +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config. +| | ... | Each DUT uses ${phy_cores} physical core(s) for worker threads. +| | ... | [Ver] Measure NDR and PDR values using MLRsearch algorithm.\ +| | ... +| | ... | *Arguments:* +| | ... | - framesize - Framesize in Bytes in integer or string (IMIX_v4_1). +| | ... | Type: integer, string +| | ... | - phy_cores - Number of physical cores. Type: integer +| | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer +| | ... +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | When Initialize IPv4 forwarding in 3-node circular topology +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} + +*** Test Cases *** +| tc01-64B-1c-ethip4-ip4base-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4-ip4base-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4-ip4base-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4-ip4base-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4-ip4base-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4-ip4base-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4-ip4base-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4-ip4base-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4-ip4base-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4-ip4base-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4-ip4base-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4-ip4base-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4scale200k-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4scale200k-mrr.robot index e3461bcf97..7215aa23fc 100644 --- a/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4scale200k-mrr.robot +++ b/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4scale200k-mrr.robot @@ -25,7 +25,7 @@ | ... | Test Teardown | Tear down performance mrr test | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *Raw results IPv4 routing test cases* | ... @@ -52,7 +52,7 @@ | ${traffic_profile}= | trex-sl-3n-ethip4-ip4dst${rts_per_flow} *** Keywords *** -| Local template +| Local Template | | ... | | [Documentation] | | ... | [Cfg] DUT runs IPv4 routing config. @@ -66,16 +66,12 @@ | | ... | - phy_cores - Number of physical cores. Type: integer | | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer | | ... -| | [Arguments] | ${phy_cores} | ${framesize} | ${rxq}=${None} -| | ... -| | Set Test Variable | ${framesize} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | When Initialize IPv4 forwarding with scaling in 3-node circular topology | | ... | ${rts_per_flow} @@ -83,50 +79,50 @@ | | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} *** Test Cases *** -| tc01-64B-1t1c-ethip4-ip4scale200k-mrr +| tc01-64B-1c-ethip4-ip4scale200k-mrr | | [Tags] | 64B | 1C -| | phy_cores=${1} | framesize=${64} - -| tc02-1518B-1t1c-ethip4-ip4scale200k-mrr -| | [Tags] | 1518B | 1C -| | phy_cores=${1} | framesize=${1518} +| | framesize=${64} | phy_cores=${1} -| tc03-9000B-1t1c-ethip4-ip4scale200k-mrr -| | [Tags] | 9000B | 1C -| | phy_cores=${1} | framesize=${9000} +| tc02-64B-2c-ethip4-ip4scale200k-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} -| tc04-IMIX-1t1c-ethip4-ip4scale200k-mrr -| | [Tags] | IMIX | 1C -| | phy_cores=${1} | framesize=IMIX_v4_1 +| tc03-64B-4c-ethip4-ip4scale200k-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} -| tc05-64B-2t2c-ethip4-ip4scale200k-mrr -| | [Tags] | 64B | 2C -| | phy_cores=${2} | framesize=${64} +| tc04-1518B-1c-ethip4-ip4scale200k-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} -| tc06-1518B-2t2c-ethip4-ip4scale200k-mrr +| tc05-1518B-2c-ethip4-ip4scale200k-mrr | | [Tags] | 1518B | 2C -| | phy_cores=${2} | framesize=${1518} - -| tc07-9000B-2t2c-ethip4-ip4scale200k-mrr -| | [Tags] | 9000B | 2C -| | phy_cores=${2} | framesize=${9000} +| | framesize=${1518} | phy_cores=${2} -| tc08-IMIX-2t2c-ethip4-ip4scale200k-mrr -| | [Tags] | IMIX | 2C -| | phy_cores=${2} | framesize=IMIX_v4_1 +| tc06-1518B-4c-ethip4-ip4scale200k-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} -| tc09-64B-4t4c-ethip4-ip4scale200k-mrr -| | [Tags] | 64B | 4C -| | phy_cores=${4} | framesize=${64} +| tc07-9000B-1c-ethip4-ip4scale200k-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc10-1518B-4t4c-ethip4-ip4scale200k-mrr -| | [Tags] | 1518B | 4C -| | phy_cores=${4} | framesize=${1518} +| tc08-9000B-2c-ethip4-ip4scale200k-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc11-9000B-4t4c-ethip4-ip4scale200k-mrr +| tc09-9000B-4c-ethip4-ip4scale200k-mrr | | [Tags] | 9000B | 4C -| | phy_cores=${4} | framesize=${9000} +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4-ip4scale200k-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4-ip4scale200k-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc12-IMIX-4t4c-ethip4-ip4scale200k-mrr +| tc12-IMIX-4c-ethip4-ip4scale200k-mrr | | [Tags] | IMIX | 4C -| | phy_cores=${4} | framesize=IMIX_v4_1 +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4scale200k-ndrpdr.robot b/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4scale200k-ndrpdr.robot new file mode 100644 index 0000000000..0c09691951 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4scale200k-ndrpdr.robot @@ -0,0 +1,134 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR +| ... | NIC_Intel-X710 | ETH | IP4FWD | SCALE | FIB_200K +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X710 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down performance discovery test | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Test Template | Local Template +| ... +| Documentation | *RFC2544: Pkt throughput IPv4 routing test cases* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4 for IPv4 routing. +| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 +| ... | routing and 2x100k static IPv4 /32 route entries. DUT1 and DUT2 tested +| ... | with 2p10GE NIC X710 by Intel. +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage +| ... | of packets transmitted. NDR and PDR are discovered for different +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on links to DUTs. TG traffic profile +| ... | contains two L3 flow-groups (flow-group per direction, 253 flows per +| ... | flow-group) with all packets containing Ethernet header, IPv4 header +| ... | with IP protocol=61 and static payload. MAC addresses are matching MAC +| ... | addresses of the TG node interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X710 bandwidth limit +| ${s_limit}= | ${10000000000} +| ${rts_per_flow}= | ${100000} +# Traffic profile: +| ${traffic_profile}= | trex-sl-3n-ethip4-ip4dst${rts_per_flow} + +*** Keywords *** +| Local Template +| | ... +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config. +| | ... | Each DUT uses ${phy_cores} physical core(s) for worker threads. +| | ... | [Ver] Measure NDR and PDR values using MLRsearch algorithm.\ +| | ... +| | ... | *Arguments:* +| | ... | - framesize - Framesize in Bytes in integer or string (IMIX_v4_1). +| | ... | Type: integer, string +| | ... | - phy_cores - Number of physical cores. Type: integer +| | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer +| | ... +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | When Initialize IPv4 forwarding with scaling in 3-node circular topology +| | ... | ${rts_per_flow} +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} + +*** Test Cases *** +| tc01-64B-1c-ethip4-ip4scale200k-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4-ip4scale200k-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4-ip4scale200k-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4-ip4scale200k-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4-ip4scale200k-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4-ip4scale200k-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4-ip4scale200k-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4-ip4scale200k-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4-ip4scale200k-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4-ip4scale200k-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4-ip4scale200k-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4-ip4scale200k-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4scale20k-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4scale20k-mrr.robot index 95767a4b00..3ee50448bb 100644 --- a/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4scale20k-mrr.robot +++ b/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4scale20k-mrr.robot @@ -25,7 +25,7 @@ | ... | Test Teardown | Tear down performance mrr test | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *Raw results IPv4 routing test cases* | ... @@ -52,7 +52,7 @@ | ${traffic_profile}= | trex-sl-3n-ethip4-ip4dst${rts_per_flow} *** Keywords *** -| Local template +| Local Template | | ... | | [Documentation] | | ... | [Cfg] DUT runs IPv4 routing config. @@ -66,16 +66,12 @@ | | ... | - phy_cores - Number of physical cores. Type: integer | | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer | | ... -| | [Arguments] | ${phy_cores} | ${framesize} | ${rxq}=${None} -| | ... -| | Set Test Variable | ${framesize} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | When Initialize IPv4 forwarding with scaling in 3-node circular topology | | ... | ${rts_per_flow} @@ -83,50 +79,50 @@ | | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} *** Test Cases *** -| tc01-64B-1t1c-ethip4-ip4scale20k-mrr +| tc01-64B-1c-ethip4-ip4scale20k-mrr | | [Tags] | 64B | 1C -| | phy_cores=${1} | framesize=${64} - -| tc02-1518B-1t1c-ethip4-ip4scale20k-mrr -| | [Tags] | 1518B | 1C -| | phy_cores=${1} | framesize=${1518} +| | framesize=${64} | phy_cores=${1} -| tc03-9000B-1t1c-ethip4-ip4scale20k-mrr -| | [Tags] | 9000B | 1C -| | phy_cores=${1} | framesize=${9000} +| tc02-64B-2c-ethip4-ip4scale20k-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} -| tc04-IMIX-1t1c-ethip4-ip4scale20k-mrr -| | [Tags] | IMIX | 1C -| | phy_cores=${1} | framesize=IMIX_v4_1 +| tc03-64B-4c-ethip4-ip4scale20k-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} -| tc05-64B-2t2c-ethip4-ip4scale20k-mrr -| | [Tags] | 64B | 2C -| | phy_cores=${2} | framesize=${64} +| tc04-1518B-1c-ethip4-ip4scale20k-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} -| tc06-1518B-2t2c-ethip4-ip4scale20k-mrr +| tc05-1518B-2c-ethip4-ip4scale20k-mrr | | [Tags] | 1518B | 2C -| | phy_cores=${2} | framesize=${1518} - -| tc07-9000B-2t2c-ethip4-ip4scale20k-mrr -| | [Tags] | 9000B | 2C -| | phy_cores=${2} | framesize=${9000} +| | framesize=${1518} | phy_cores=${2} -| tc08-IMIX-2t2c-ethip4-ip4scale20k-mrr -| | [Tags] | IMIX | 2C -| | phy_cores=${2} | framesize=IMIX_v4_1 +| tc06-1518B-4c-ethip4-ip4scale20k-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} -| tc09-64B-4t4c-ethip4-ip4scale20k-mrr -| | [Tags] | 64B | 4C -| | phy_cores=${4} | framesize=${64} +| tc07-9000B-1c-ethip4-ip4scale20k-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc10-1518B-4t4c-ethip4-ip4scale20k-mrr -| | [Tags] | 1518B | 4C -| | phy_cores=${4} | framesize=${1518} +| tc08-9000B-2c-ethip4-ip4scale20k-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc11-9000B-4t4c-ethip4-ip4scale20k-mrr +| tc09-9000B-4c-ethip4-ip4scale20k-mrr | | [Tags] | 9000B | 4C -| | phy_cores=${4} | framesize=${9000} +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4-ip4scale20k-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4-ip4scale20k-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc12-IMIX-4t4c-ethip4-ip4scale20k-mrr +| tc12-IMIX-4c-ethip4-ip4scale20k-mrr | | [Tags] | IMIX | 4C -| | phy_cores=${4} | framesize=IMIX_v4_1 +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4scale20k-ndrpdr.robot b/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4scale20k-ndrpdr.robot new file mode 100644 index 0000000000..4ef934f7af --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4scale20k-ndrpdr.robot @@ -0,0 +1,134 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR +| ... | NIC_Intel-X710 | ETH | IP4FWD | SCALE | FIB_20K +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X710 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down performance discovery test | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Test Template | Local Template +| ... +| Documentation | *RFC2544: Pkt throughput IPv4 routing test cases* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4 for IPv4 routing. +| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 +| ... | routing and 2x100k static IPv4 /32 route entries. DUT1 and DUT2 tested +| ... | with 2p10GE NIC X710 by Intel. +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage +| ... | of packets transmitted. NDR and PDR are discovered for different +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on links to DUTs. TG traffic profile +| ... | contains two L3 flow-groups (flow-group per direction, 253 flows per +| ... | flow-group) with all packets containing Ethernet header, IPv4 header +| ... | with IP protocol=61 and static payload. MAC addresses are matching MAC +| ... | addresses of the TG node interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X710 bandwidth limit +| ${s_limit}= | ${10000000000} +| ${rts_per_flow}= | ${10000} +# Traffic profile: +| ${traffic_profile}= | trex-sl-3n-ethip4-ip4dst${rts_per_flow} + +*** Keywords *** +| Local Template +| | ... +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config. +| | ... | Each DUT uses ${phy_cores} physical core(s) for worker threads. +| | ... | [Ver] Measure NDR and PDR values using MLRsearch algorithm.\ +| | ... +| | ... | *Arguments:* +| | ... | - framesize - Framesize in Bytes in integer or string (IMIX_v4_1). +| | ... | Type: integer, string +| | ... | - phy_cores - Number of physical cores. Type: integer +| | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer +| | ... +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | When Initialize IPv4 forwarding with scaling in 3-node circular topology +| | ... | ${rts_per_flow} +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} + +*** Test Cases *** +| tc01-64B-1c-ethip4-ip4scale20k-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4-ip4scale20k-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4-ip4scale20k-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4-ip4scale20k-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4-ip4scale20k-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4-ip4scale20k-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4-ip4scale20k-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4-ip4scale20k-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4-ip4scale20k-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4-ip4scale20k-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4-ip4scale20k-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4-ip4scale20k-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4scale2m-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4scale2m-mrr.robot index 968b5fe309..7e9e9774b4 100644 --- a/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4scale2m-mrr.robot +++ b/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4scale2m-mrr.robot @@ -25,7 +25,7 @@ | ... | Test Teardown | Tear down performance mrr test | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *Raw results IPv4 routing test cases* | ... @@ -52,7 +52,7 @@ | ${traffic_profile}= | trex-sl-3n-ethip4-ip4dst${rts_per_flow} *** Keywords *** -| Local template +| Local Template | | ... | | [Documentation] | | ... | [Cfg] DUT runs IPv4 routing config. @@ -66,16 +66,12 @@ | | ... | - phy_cores - Number of physical cores. Type: integer | | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer | | ... -| | [Arguments] | ${phy_cores} | ${framesize} | ${rxq}=${None} -| | ... -| | Set Test Variable | ${framesize} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | When Initialize IPv4 forwarding with scaling in 3-node circular topology | | ... | ${rts_per_flow} @@ -83,50 +79,50 @@ | | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} *** Test Cases *** -| tc01-64B-1t1c-ethip4-ip4scale2m-mrr +| tc01-64B-1c-ethip4-ip4scale2m-mrr | | [Tags] | 64B | 1C -| | phy_cores=${1} | framesize=${64} - -| tc02-1518B-1t1c-ethip4-ip4scale2m-mrr -| | [Tags] | 1518B | 1C -| | phy_cores=${1} | framesize=${1518} +| | framesize=${64} | phy_cores=${1} -| tc03-9000B-1t1c-ethip4-ip4scale2m-mrr -| | [Tags] | 9000B | 1C -| | phy_cores=${1} | framesize=${9000} +| tc02-64B-2c-ethip4-ip4scale2m-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} -| tc04-IMIX-1t1c-ethip4-ip4scale2m-mrr -| | [Tags] | IMIX | 1C -| | phy_cores=${1} | framesize=IMIX_v4_1 +| tc03-64B-4c-ethip4-ip4scale2m-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} -| tc05-64B-2t2c-ethip4-ip4scale2m-mrr -| | [Tags] | 64B | 2C -| | phy_cores=${2} | framesize=${64} +| tc04-1518B-1c-ethip4-ip4scale2m-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} -| tc06-1518B-2t2c-ethip4-ip4scale2m-mrr +| tc05-1518B-2c-ethip4-ip4scale2m-mrr | | [Tags] | 1518B | 2C -| | phy_cores=${2} | framesize=${1518} - -| tc07-9000B-2t2c-ethip4-ip4scale2m-mrr -| | [Tags] | 9000B | 2C -| | phy_cores=${2} | framesize=${9000} +| | framesize=${1518} | phy_cores=${2} -| tc08-IMIX-2t2c-ethip4-ip4scale2m-mrr -| | [Tags] | IMIX | 2C -| | phy_cores=${2} | framesize=IMIX_v4_1 +| tc06-1518B-4c-ethip4-ip4scale2m-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} -| tc09-64B-4t4c-ethip4-ip4scale2m-mrr -| | [Tags] | 64B | 4C -| | phy_cores=${4} | framesize=${64} +| tc07-9000B-1c-ethip4-ip4scale2m-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc10-1518B-4t4c-ethip4-ip4scale2m-mrr -| | [Tags] | 1518B | 4C -| | phy_cores=${4} | framesize=${1518} +| tc08-9000B-2c-ethip4-ip4scale2m-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc11-9000B-4t4c-ethip4-ip4scale2m-mrr +| tc09-9000B-4c-ethip4-ip4scale2m-mrr | | [Tags] | 9000B | 4C -| | phy_cores=${4} | framesize=${9000} +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4-ip4scale2m-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4-ip4scale2m-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc12-IMIX-4t4c-ethip4-ip4scale2m-mrr +| tc12-IMIX-4c-ethip4-ip4scale2m-mrr | | [Tags] | IMIX | 4C -| | phy_cores=${4} | framesize=IMIX_v4_1 +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4scale2m-ndrpdr.robot b/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4scale2m-ndrpdr.robot new file mode 100644 index 0000000000..86de7c4dd9 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x710-ethip4-ip4scale2m-ndrpdr.robot @@ -0,0 +1,134 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR +| ... | NIC_Intel-X710 | ETH | IP4FWD | SCALE | FIB_2M +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-X710 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down performance discovery test | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Test Template | Local Template +| ... +| Documentation | *RFC2544: Pkt throughput IPv4 routing test cases* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4 for IPv4 routing. +| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 +| ... | routing and 2x100k static IPv4 /32 route entries. DUT1 and DUT2 tested +| ... | with 2p10GE NIC X710 by Intel. +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage +| ... | of packets transmitted. NDR and PDR are discovered for different +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on links to DUTs. TG traffic profile +| ... | contains two L3 flow-groups (flow-group per direction, 253 flows per +| ... | flow-group) with all packets containing Ethernet header, IPv4 header +| ... | with IP protocol=61 and static payload. MAC addresses are matching MAC +| ... | addresses of the TG node interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X710 bandwidth limit +| ${s_limit}= | ${10000000000} +| ${rts_per_flow}= | ${1000000} +# Traffic profile: +| ${traffic_profile}= | trex-sl-3n-ethip4-ip4dst${rts_per_flow} + +*** Keywords *** +| Local Template +| | ... +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config. +| | ... | Each DUT uses ${phy_cores} physical core(s) for worker threads. +| | ... | [Ver] Measure NDR and PDR values using MLRsearch algorithm.\ +| | ... +| | ... | *Arguments:* +| | ... | - framesize - Framesize in Bytes in integer or string (IMIX_v4_1). +| | ... | Type: integer, string +| | ... | - phy_cores - Number of physical cores. Type: integer +| | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer +| | ... +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | When Initialize IPv4 forwarding with scaling in 3-node circular topology +| | ... | ${rts_per_flow} +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} + +*** Test Cases *** +| tc01-64B-1c-ethip4-ip4scale2m-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4-ip4scale2m-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4-ip4scale2m-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4-ip4scale2m-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4-ip4scale2m-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4-ip4scale2m-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4-ip4scale2m-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4-ip4scale2m-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4-ip4scale2m-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4-ip4scale2m-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4-ip4scale2m-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4-ip4scale2m-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4base-iacl1sf-10kflows-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4base-iacl1sf-10kflows-mrr.robot index fe1e260629..2c541aaa2d 100644 --- a/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4base-iacl1sf-10kflows-mrr.robot +++ b/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4base-iacl1sf-10kflows-mrr.robot @@ -28,7 +28,7 @@ | ... | Test Teardown | Tear down performance mrr test | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *Raw results IPv4 test cases with ACL* | ... @@ -68,8 +68,10 @@ | ${trex_stream1_subnet}= | 10.10.10.0/24 | ${trex_stream2_subnet}= | 20.20.20.0/24 +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u1000p-conc + *** Keywords *** -| Local template +| Local Template | | ... | | [Documentation] | | ... | [Cfg] DUT runs IPv4 routing config. @@ -83,69 +85,63 @@ | | ... | - phy_cores - Number of physical cores. Type: integer | | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer | | ... -| | [Arguments] | ${phy_cores} | ${framesize} | ${rxq}=${None} -| | ... -| | Set Test Variable | ${framesize} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | ${ip_nr}= | Set Variable | 10 | | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u1000p-conc -| | Set Test Variable | ${traffic_profile} | | Then Traffic should pass with maximum rate | | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-iacl1-stateful-flows10k-mrr +| tc01-64B-1c-ethip4udp-ip4base-iacl1sf-10kflows-mrr | | [Tags] | 64B | 1C -| | phy_cores=${1} | framesize=${64} - -| tc02-1518B-1t1c-ethip4udp-ip4base-iacl1-stateful-flows10k-mrr -| | [Tags] | 1518B | 1C -| | phy_cores=${1} | framesize=${1518} +| | framesize=${64} | phy_cores=${1} -| tc03-9000B-1t1c-ethip4udp-ip4base-iacl1-stateful-flows10k-mrr -| | [Tags] | 9000B | 1C -| | phy_cores=${1} | framesize=${9000} +| tc02-64B-2c-ethip4udp-ip4base-iacl1sf-10kflows-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} -| tc04-IMIX-1t1c-ethip4udp-ip4base-iacl1-stateful-flows10k-mrr -| | [Tags] | IMIX | 1C -| | phy_cores=${1} | framesize=IMIX_v4_1 +| tc03-64B-4c-ethip4udp-ip4base-iacl1sf-10kflows-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} -| tc05-64B-2t2c-ethip4udp-ip4base-iacl1-stateful-flows10k-mrr -| | [Tags] | 64B | 2C -| | phy_cores=${2} | framesize=${64} +| tc04-1518B-1c-ethip4udp-ip4base-iacl1sf-10kflows-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} -| tc06-1518B-2t2c-ethip4udp-ip4base-iacl1-stateful-flows10k-mrr +| tc05-1518B-2c-ethip4udp-ip4base-iacl1sf-10kflows-mrr | | [Tags] | 1518B | 2C -| | phy_cores=${2} | framesize=${1518} - -| tc07-9000B-2t2c-ethip4udp-ip4base-iacl1-stateful-flows10k-mrr -| | [Tags] | 9000B | 2C -| | phy_cores=${2} | framesize=${9000} +| | framesize=${1518} | phy_cores=${2} -| tc08-IMIX-2t2c-ethip4udp-ip4base-iacl1-stateful-flows10k-mrr -| | [Tags] | IMIX | 2C -| | phy_cores=${2} | framesize=IMIX_v4_1 +| tc06-1518B-4c-ethip4udp-ip4base-iacl1sf-10kflows-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} -| tc09-64B-4t4c-ethip4udp-ip4base-iacl1-stateful-flows10k-mrr -| | [Tags] | 64B | 4C -| | phy_cores=${4} | framesize=${64} +| tc07-9000B-1c-ethip4udp-ip4base-iacl1sf-10kflows-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc10-1518B-4t4c-ethip4udp-ip4base-iacl1-stateful-flows10k-mrr -| | [Tags] | 1518B | 4C -| | phy_cores=${4} | framesize=${1518} +| tc08-9000B-2c-ethip4udp-ip4base-iacl1sf-10kflows-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc11-9000B-4t4c-ethip4udp-ip4base-iacl1-stateful-flows10k-mrr +| tc09-9000B-4c-ethip4udp-ip4base-iacl1sf-10kflows-mrr | | [Tags] | 9000B | 4C -| | phy_cores=${4} | framesize=${9000} +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-iacl1sf-10kflows-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-iacl1sf-10kflows-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc12-IMIX-4t4c-ethip4udp-ip4base-iacl1-stateful-flows10k-mrr +| tc12-IMIX-4c-ethip4udp-ip4base-iacl1sf-10kflows-mrr | | [Tags] | IMIX | 4C -| | phy_cores=${4} | framesize=IMIX_v4_1 +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4base-iacl1sf-10kflows-ndrpdr.robot b/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4base-iacl1sf-10kflows-ndrpdr.robot new file mode 100644 index 0000000000..aab43d051f --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4base-iacl1sf-10kflows-ndrpdr.robot @@ -0,0 +1,153 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR +| ... | NIC_Intel-X710 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL +| ... | IACL | ACL1 | 10k_FLOWS +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X710 +| ... | AND | Set up performance test suite with ACL +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down performance discovery test | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Test Template | Local Template +| ... +| Documentation | *RFC2544: Pkt throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X710 by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage +| ... | of packets transmitted. NDR and PDR are discovered for different +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on links to DUTs. TG traffic profile +| ... | contains two L3 flow-groups (flow-group per direction, 253 flows per +| ... | flow-group) with all packets containing Ethernet header, IPv4 header +| ... | with IP protocol=61 and static payload. MAC addresses are matching MAC +| ... | addresses of the TG node interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X710 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit+reflect +| ${acl_apply_type}= | input +| ${no_hit_aces_number}= | 1 +| ${flows_per_dir}= | 10k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u1000p-conc + +*** Keywords *** +| Local Template +| | ... +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config. +| | ... | Each DUT uses ${phy_cores} physical core(s) for worker threads. +| | ... | [Ver] Measure NDR and PDR values using MLRsearch algorithm.\ +| | ... +| | ... | *Arguments:* +| | ... | - framesize - Framesize in Bytes in integer or string (IMIX_v4_1). +| | ... | Type: integer, string +| | ... | - phy_cores - Number of physical cores. Type: integer +| | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer +| | ... +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-iacl1sf-10kflows-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-iacl1sf-10kflows-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-iacl1sf-10kflows-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-iacl1sf-10kflows-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-iacl1sf-10kflows-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-iacl1sf-10kflows-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-iacl1sf-10kflows-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-iacl1sf-10kflows-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-iacl1sf-10kflows-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-iacl1sf-10kflows-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-iacl1sf-10kflows-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-iacl1sf-10kflows-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4base-iacl1sl-10kflows-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4base-iacl1sl-10kflows-mrr.robot index 67e498d2ae..d5dbcde990 100644 --- a/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4base-iacl1sl-10kflows-mrr.robot +++ b/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4base-iacl1sl-10kflows-mrr.robot @@ -28,7 +28,7 @@ | ... | Test Teardown | Tear down performance mrr test | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *Raw results IPv4 test cases with ACL* | ... @@ -68,8 +68,10 @@ | ${trex_stream1_subnet}= | 10.10.10.0/24 | ${trex_stream2_subnet}= | 20.20.20.0/24 +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u1000p-conc + *** Keywords *** -| Local template +| Local Template | | ... | | [Documentation] | | ... | [Cfg] DUT runs IPv4 routing config. @@ -83,69 +85,63 @@ | | ... | - phy_cores - Number of physical cores. Type: integer | | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer | | ... -| | [Arguments] | ${phy_cores} | ${framesize} | ${rxq}=${None} -| | ... -| | Set Test Variable | ${framesize} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | ${ip_nr}= | Set Variable | 10 | | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u1000p-conc -| | Set Test Variable | ${traffic_profile} | | Then Traffic should pass with maximum rate | | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-iacl1-stateless-flows10k-mrr +| tc01-64B-1c-ethip4udp-ip4base-iacl1sl-10kflows-mrr | | [Tags] | 64B | 1C -| | phy_cores=${1} | framesize=${64} - -| tc02-1518B-1t1c-ethip4udp-ip4base-iacl1-stateless-flows10k-mrr -| | [Tags] | 1518B | 1C -| | phy_cores=${1} | framesize=${1518} +| | framesize=${64} | phy_cores=${1} -| tc03-9000B-1t1c-ethip4udp-ip4base-iacl1-stateless-flows10k-mrr -| | [Tags] | 9000B | 1C -| | phy_cores=${1} | framesize=${9000} +| tc02-64B-2c-ethip4udp-ip4base-iacl1sl-10kflows-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} -| tc04-IMIX-1t1c-ethip4udp-ip4base-iacl1-stateless-flows10k-mrr -| | [Tags] | IMIX | 1C -| | phy_cores=${1} | framesize=IMIX_v4_1 +| tc03-64B-4c-ethip4udp-ip4base-iacl1sl-10kflows-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} -| tc05-64B-2t2c-ethip4udp-ip4base-iacl1-stateless-flows10k-mrr -| | [Tags] | 64B | 2C -| | phy_cores=${2} | framesize=${64} +| tc04-1518B-1c-ethip4udp-ip4base-iacl1sl-10kflows-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} -| tc06-1518B-2t2c-ethip4udp-ip4base-iacl1-stateless-flows10k-mrr +| tc05-1518B-2c-ethip4udp-ip4base-iacl1sl-10kflows-mrr | | [Tags] | 1518B | 2C -| | phy_cores=${2} | framesize=${1518} - -| tc07-9000B-2t2c-ethip4udp-ip4base-iacl1-stateless-flows10k-mrr -| | [Tags] | 9000B | 2C -| | phy_cores=${2} | framesize=${9000} +| | framesize=${1518} | phy_cores=${2} -| tc08-IMIX-2t2c-ethip4udp-ip4base-iacl1-stateless-flows10k-mrr -| | [Tags] | IMIX | 2C -| | phy_cores=${2} | framesize=IMIX_v4_1 +| tc06-1518B-4c-ethip4udp-ip4base-iacl1sl-10kflows-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} -| tc09-64B-4t4c-ethip4udp-ip4base-iacl1-stateless-flows10k-mrr -| | [Tags] | 64B | 4C -| | phy_cores=${4} | framesize=${64} +| tc07-9000B-1c-ethip4udp-ip4base-iacl1sl-10kflows-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc10-1518B-4t4c-ethip4udp-ip4base-iacl1-stateless-flows10k-mrr -| | [Tags] | 1518B | 4C -| | phy_cores=${4} | framesize=${1518} +| tc08-9000B-2c-ethip4udp-ip4base-iacl1sl-10kflows-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc11-9000B-4t4c-ethip4udp-ip4base-iacl1-stateless-flows10k-mrr +| tc09-9000B-4c-ethip4udp-ip4base-iacl1sl-10kflows-mrr | | [Tags] | 9000B | 4C -| | phy_cores=${4} | framesize=${9000} +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-iacl1sl-10kflows-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-iacl1sl-10kflows-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc12-IMIX-4t4c-ethip4udp-ip4base-iacl1-stateless-flows10k-mrr +| tc12-IMIX-4c-ethip4udp-ip4base-iacl1sl-10kflows-mrr | | [Tags] | IMIX | 4C -| | phy_cores=${4} | framesize=IMIX_v4_1 +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4base-iacl1sl-10kflows-ndrpdr.robot b/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4base-iacl1sl-10kflows-ndrpdr.robot new file mode 100644 index 0000000000..c5f384c32e --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4base-iacl1sl-10kflows-ndrpdr.robot @@ -0,0 +1,153 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR +| ... | NIC_Intel-X710 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS +| ... | IACL | ACL1 | 10k_FLOWS +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X710 +| ... | AND | Set up performance test suite with ACL +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down performance discovery test | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Test Template | Local Template +| ... +| Documentation | *RFC2544: Pkt throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X710 by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage +| ... | of packets transmitted. NDR and PDR are discovered for different +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on links to DUTs. TG traffic profile +| ... | contains two L3 flow-groups (flow-group per direction, 253 flows per +| ... | flow-group) with all packets containing Ethernet header, IPv4 header +| ... | with IP protocol=61 and static payload. MAC addresses are matching MAC +| ... | addresses of the TG node interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X710 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit +| ${acl_apply_type}= | input +| ${no_hit_aces_number}= | 1 +| ${flows_per_dir}= | 10k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u1000p-conc + +*** Keywords *** +| Local Template +| | ... +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config. +| | ... | Each DUT uses ${phy_cores} physical core(s) for worker threads. +| | ... | [Ver] Measure NDR and PDR values using MLRsearch algorithm.\ +| | ... +| | ... | *Arguments:* +| | ... | - framesize - Framesize in Bytes in integer or string (IMIX_v4_1). +| | ... | Type: integer, string +| | ... | - phy_cores - Number of physical cores. Type: integer +| | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer +| | ... +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-iacl1sl-10kflows-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-iacl1sl-10kflows-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-iacl1sl-10kflows-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-iacl1sl-10kflows-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-iacl1sl-10kflows-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-iacl1sl-10kflows-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-iacl1sl-10kflows-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-iacl1sl-10kflows-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-iacl1sl-10kflows-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-iacl1sl-10kflows-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-iacl1sl-10kflows-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-iacl1sl-10kflows-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4base-iacl50sf-10kflows-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4base-iacl50sf-10kflows-mrr.robot index 8e3d74361f..efdcd06a72 100644 --- a/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4base-iacl50sf-10kflows-mrr.robot +++ b/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4base-iacl50sf-10kflows-mrr.robot @@ -28,7 +28,7 @@ | ... | Test Teardown | Tear down performance mrr test | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *Raw results IPv4 test cases with ACL* | ... @@ -68,8 +68,10 @@ | ${trex_stream1_subnet}= | 10.10.10.0/24 | ${trex_stream2_subnet}= | 20.20.20.0/24 +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u1000p-conc + *** Keywords *** -| Local template +| Local Template | | ... | | [Documentation] | | ... | [Cfg] DUT runs IPv4 routing config. @@ -83,69 +85,63 @@ | | ... | - phy_cores - Number of physical cores. Type: integer | | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer | | ... -| | [Arguments] | ${phy_cores} | ${framesize} | ${rxq}=${None} -| | ... -| | Set Test Variable | ${framesize} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | ${ip_nr}= | Set Variable | 10 | | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u1000p-conc -| | Set Test Variable | ${traffic_profile} | | Then Traffic should pass with maximum rate | | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-iacl50-stateful-flows10k-mrr +| tc01-64B-1c-ethip4udp-ip4base-iacl50sf-10kflows-mrr | | [Tags] | 64B | 1C -| | phy_cores=${1} | framesize=${64} - -| tc02-1518B-1t1c-ethip4udp-ip4base-iacl50-stateful-flows10k-mrr -| | [Tags] | 1518B | 1C -| | phy_cores=${1} | framesize=${1518} +| | framesize=${64} | phy_cores=${1} -| tc03-9000B-1t1c-ethip4udp-ip4base-iacl50-stateful-flows10k-mrr -| | [Tags] | 9000B | 1C -| | phy_cores=${1} | framesize=${9000} +| tc02-64B-2c-ethip4udp-ip4base-iacl50sf-10kflows-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} -| tc04-IMIX-1t1c-ethip4udp-ip4base-iacl50-stateful-flows10k-mrr -| | [Tags] | IMIX | 1C -| | phy_cores=${1} | framesize=IMIX_v4_1 +| tc03-64B-4c-ethip4udp-ip4base-iacl50sf-10kflows-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} -| tc05-64B-2t2c-ethip4udp-ip4base-iacl50-stateful-flows10k-mrr -| | [Tags] | 64B | 2C -| | phy_cores=${2} | framesize=${64} +| tc04-1518B-1c-ethip4udp-ip4base-iacl50sf-10kflows-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} -| tc06-1518B-2t2c-ethip4udp-ip4base-iacl50-stateful-flows10k-mrr +| tc05-1518B-2c-ethip4udp-ip4base-iacl50sf-10kflows-mrr | | [Tags] | 1518B | 2C -| | phy_cores=${2} | framesize=${1518} - -| tc07-9000B-2t2c-ethip4udp-ip4base-iacl50-stateful-flows10k-mrr -| | [Tags] | 9000B | 2C -| | phy_cores=${2} | framesize=${9000} +| | framesize=${1518} | phy_cores=${2} -| tc08-IMIX-2t2c-ethip4udp-ip4base-iacl50-stateful-flows10k-mrr -| | [Tags] | IMIX | 2C -| | phy_cores=${2} | framesize=IMIX_v4_1 +| tc06-1518B-4c-ethip4udp-ip4base-iacl50sf-10kflows-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} -| tc09-64B-4t4c-ethip4udp-ip4base-iacl50-stateful-flows10k-mrr -| | [Tags] | 64B | 4C -| | phy_cores=${4} | framesize=${64} +| tc07-9000B-1c-ethip4udp-ip4base-iacl50sf-10kflows-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc10-1518B-4t4c-ethip4udp-ip4base-iacl50-stateful-flows10k-mrr -| | [Tags] | 1518B | 4C -| | phy_cores=${4} | framesize=${1518} +| tc08-9000B-2c-ethip4udp-ip4base-iacl50sf-10kflows-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc11-9000B-4t4c-ethip4udp-ip4base-iacl50-stateful-flows10k-mrr +| tc09-9000B-4c-ethip4udp-ip4base-iacl50sf-10kflows-mrr | | [Tags] | 9000B | 4C -| | phy_cores=${4} | framesize=${9000} +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-iacl50sf-10kflows-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-iacl50sf-10kflows-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc12-IMIX-4t4c-ethip4udp-ip4base-iacl50-stateful-flows10k-mrr +| tc12-IMIX-4c-ethip4udp-ip4base-iacl50sf-10kflows-mrr | | [Tags] | IMIX | 4C -| | phy_cores=${4} | framesize=IMIX_v4_1 +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4base-iacl50sf-10kflows-ndrpdr.robot b/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4base-iacl50sf-10kflows-ndrpdr.robot new file mode 100644 index 0000000000..7f16474858 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4base-iacl50sf-10kflows-ndrpdr.robot @@ -0,0 +1,153 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR +| ... | NIC_Intel-X710 | ETH | IP4FWD | FEATURE | ACL | ACL_STATEFUL +| ... | IACL | ACL50 | 10k_FLOWS +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X710 +| ... | AND | Set up performance test suite with ACL +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down performance discovery test | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Test Template | Local Template +| ... +| Documentation | *RFC2544: Pkt throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X710 by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage +| ... | of packets transmitted. NDR and PDR are discovered for different +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on links to DUTs. TG traffic profile +| ... | contains two L3 flow-groups (flow-group per direction, 253 flows per +| ... | flow-group) with all packets containing Ethernet header, IPv4 header +| ... | with IP protocol=61 and static payload. MAC addresses are matching MAC +| ... | addresses of the TG node interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X710 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit+reflect +| ${acl_apply_type}= | input +| ${no_hit_aces_number}= | 50 +| ${flows_per_dir}= | 10k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u1000p-conc + +*** Keywords *** +| Local Template +| | ... +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config. +| | ... | Each DUT uses ${phy_cores} physical core(s) for worker threads. +| | ... | [Ver] Measure NDR and PDR values using MLRsearch algorithm.\ +| | ... +| | ... | *Arguments:* +| | ... | - framesize - Framesize in Bytes in integer or string (IMIX_v4_1). +| | ... | Type: integer, string +| | ... | - phy_cores - Number of physical cores. Type: integer +| | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer +| | ... +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-iacl50sf-10kflows-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-iacl50sf-10kflows-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-iacl50sf-10kflows-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-iacl50sf-10kflows-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-iacl50sf-10kflows-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-iacl50sf-10kflows-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-iacl50sf-10kflows-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-iacl50sf-10kflows-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-iacl50sf-10kflows-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-iacl50sf-10kflows-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-iacl50sf-10kflows-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-iacl50sf-10kflows-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4base-iacl50sl-10kflows-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4base-iacl50sl-10kflows-mrr.robot index 8f56bbad83..a966e6ff72 100644 --- a/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4base-iacl50sl-10kflows-mrr.robot +++ b/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4base-iacl50sl-10kflows-mrr.robot @@ -28,7 +28,7 @@ | ... | Test Teardown | Tear down performance mrr test | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *Raw results IPv4 test cases with ACL* | ... @@ -68,8 +68,10 @@ | ${trex_stream1_subnet}= | 10.10.10.0/24 | ${trex_stream2_subnet}= | 20.20.20.0/24 +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u1000p-conc + *** Keywords *** -| Local template +| Local Template | | ... | | [Documentation] | | ... | [Cfg] DUT runs IPv4 routing config. @@ -83,69 +85,63 @@ | | ... | - phy_cores - Number of physical cores. Type: integer | | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer | | ... -| | [Arguments] | ${phy_cores} | ${framesize} | ${rxq}=${None} -| | ... -| | Set Test Variable | ${framesize} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | ${ip_nr}= | Set Variable | 10 | | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology -| | ${traffic_profile}= | Set Variable | trex-sl-3n-ethip4udp-10u1000p-conc -| | Set Test Variable | ${traffic_profile} | | Then Traffic should pass with maximum rate | | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4base-iacl50-stateless-flows10k-mrr +| tc01-64B-1c-ethip4udp-ip4base-iacl50sl-10kflows-mrr | | [Tags] | 64B | 1C -| | phy_cores=${1} | framesize=${64} - -| tc02-1518B-1t1c-ethip4udp-ip4base-iacl50-stateless-flows10k-mrr -| | [Tags] | 1518B | 1C -| | phy_cores=${1} | framesize=${1518} +| | framesize=${64} | phy_cores=${1} -| tc03-9000B-1t1c-ethip4udp-ip4base-iacl50-stateless-flows10k-mrr -| | [Tags] | 9000B | 1C -| | phy_cores=${1} | framesize=${9000} +| tc02-64B-2c-ethip4udp-ip4base-iacl50sl-10kflows-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} -| tc04-IMIX-1t1c-ethip4udp-ip4base-iacl50-stateless-flows10k-mrr -| | [Tags] | IMIX | 1C -| | phy_cores=${1} | framesize=IMIX_v4_1 +| tc03-64B-4c-ethip4udp-ip4base-iacl50sl-10kflows-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} -| tc05-64B-2t2c-ethip4udp-ip4base-iacl50-stateless-flows10k-mrr -| | [Tags] | 64B | 2C -| | phy_cores=${2} | framesize=${64} +| tc04-1518B-1c-ethip4udp-ip4base-iacl50sl-10kflows-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} -| tc06-1518B-2t2c-ethip4udp-ip4base-iacl50-stateless-flows10k-mrr +| tc05-1518B-2c-ethip4udp-ip4base-iacl50sl-10kflows-mrr | | [Tags] | 1518B | 2C -| | phy_cores=${2} | framesize=${1518} - -| tc07-9000B-2t2c-ethip4udp-ip4base-iacl50-stateless-flows10k-mrr -| | [Tags] | 9000B | 2C -| | phy_cores=${2} | framesize=${9000} +| | framesize=${1518} | phy_cores=${2} -| tc08-IMIX-2t2c-ethip4udp-ip4base-iacl50-stateless-flows10k-mrr -| | [Tags] | IMIX | 2C -| | phy_cores=${2} | framesize=IMIX_v4_1 +| tc06-1518B-4c-ethip4udp-ip4base-iacl50sl-10kflows-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} -| tc09-64B-4t4c-ethip4udp-ip4base-iacl50-stateless-flows10k-mrr -| | [Tags] | 64B | 4C -| | phy_cores=${4} | framesize=${64} +| tc07-9000B-1c-ethip4udp-ip4base-iacl50sl-10kflows-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc10-1518B-4t4c-ethip4udp-ip4base-iacl50-stateless-flows10k-mrr -| | [Tags] | 1518B | 4C -| | phy_cores=${4} | framesize=${1518} +| tc08-9000B-2c-ethip4udp-ip4base-iacl50sl-10kflows-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc11-9000B-4t4c-ethip4udp-ip4base-iacl50-stateless-flows10k-mrr +| tc09-9000B-4c-ethip4udp-ip4base-iacl50sl-10kflows-mrr | | [Tags] | 9000B | 4C -| | phy_cores=${4} | framesize=${9000} +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-iacl50sl-10kflows-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-iacl50sl-10kflows-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc12-IMIX-4t4c-ethip4udp-ip4base-iacl50-stateless-flows10k-mrr +| tc12-IMIX-4c-ethip4udp-ip4base-iacl50sl-10kflows-mrr | | [Tags] | IMIX | 4C -| | phy_cores=${4} | framesize=IMIX_v4_1 +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4base-iacl50sl-10kflows-ndrpdr.robot b/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4base-iacl50sl-10kflows-ndrpdr.robot new file mode 100644 index 0000000000..0f1839bd09 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4base-iacl50sl-10kflows-ndrpdr.robot @@ -0,0 +1,153 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR +| ... | NIC_Intel-X710 | ETH | IP4FWD | FEATURE | ACL | ACL_STATELESS +| ... | IACL | ACL50 | 10k_FLOWS +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X710 +| ... | AND | Set up performance test suite with ACL +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down performance discovery test | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Test Template | Local Template +| ... +| Documentation | *RFC2544: Pkt throughput IPv4 test cases with ACL* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology\ +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for L2 switching of IPv4. +| ... | *[Cfg] DUT configuration:* DUT1 is configured with L2 bridge domain\ +| ... | and MAC learning enabled. DUT2 is configured with L2 cross-connects.\ +| ... | Required ACL rules are applied to input paths of both DUT1 intefaces.\ +| ... | DUT1 and DUT2 are tested with 2p10GE NIC X710 by Intel.\ +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage +| ... | of packets transmitted. NDR and PDR are discovered for different +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on links to DUTs. TG traffic profile +| ... | contains two L3 flow-groups (flow-group per direction, 253 flows per +| ... | flow-group) with all packets containing Ethernet header, IPv4 header +| ... | with IP protocol=61 and static payload. MAC addresses are matching MAC +| ... | addresses of the TG node interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X710 bandwidth limit +| ${s_limit}= | ${10000000000} + +# ACL test setup +| ${acl_action}= | permit +| ${acl_apply_type}= | input +| ${no_hit_aces_number}= | 50 +| ${flows_per_dir}= | 10k + +# starting points for non-hitting ACLs +| ${src_ip_start}= | 30.30.30.1 +| ${dst_ip_start}= | 40.40.40.1 +| ${ip_step}= | ${1} +| ${sport_start}= | ${1000} +| ${dport_start}= | ${1000} +| ${port_step}= | ${1} +| ${trex_stream1_subnet}= | 10.10.10.0/24 +| ${trex_stream2_subnet}= | 20.20.20.0/24 + +| ${traffic_profile}= | trex-sl-3n-ethip4udp-10u1000p-conc + +*** Keywords *** +| Local Template +| | ... +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config. +| | ... | Each DUT uses ${phy_cores} physical core(s) for worker threads. +| | ... | [Ver] Measure NDR and PDR values using MLRsearch algorithm.\ +| | ... +| | ... | *Arguments:* +| | ... | - framesize - Framesize in Bytes in integer or string (IMIX_v4_1). +| | ... | Type: integer, string +| | ... | - phy_cores - Number of physical cores. Type: integer +| | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer +| | ... +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Apply startup configuration on all VPP DUTs +| | ${ip_nr}= | Set Variable | 10 +| | When Initialize IPv4 routing for '${ip_nr}' addresses with IPv4 ACLs on DUT1 in 3-node circular topology +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-iacl50sl-10kflows-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-iacl50sl-10kflows-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-iacl50sl-10kflows-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-iacl50sl-10kflows-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-iacl50sl-10kflows-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-iacl50sl-10kflows-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-iacl50sl-10kflows-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-iacl50sl-10kflows-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-iacl50sl-10kflows-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-iacl50sl-10kflows-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-iacl50sl-10kflows-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-iacl50sl-10kflows-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4base-nat44-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4base-nat44-mrr.robot index a64e2c1e48..daa1620ced 100644 --- a/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4base-nat44-mrr.robot +++ b/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4base-nat44-mrr.robot @@ -29,9 +29,9 @@ | ... | Test Teardown | Tear down performance mrr test | ... -| Test Template | Local template +| Test Template | Local Template | ... -| Documentation | *NAT44 performance test cases* +| Documentation | *Raw results NAT44 performance test cases* | ... | ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology | ... | with single links between nodes. @@ -55,7 +55,7 @@ | ${traffic_profile}= | trex-sl-3n-ethip4udp-1u1p *** Keywords *** -| Local template +| Local Template | | ... | | [Documentation] | | ... | [Cfg] DUT runs IPv4 routing config. @@ -69,16 +69,12 @@ | | ... | - phy_cores - Number of physical cores. Type: integer | | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer | | ... -| | [Arguments] | ${phy_cores} | ${framesize} | ${rxq}=${None} -| | ... -| | Set Test Variable | ${framesize} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${get_framesize}= | Get Frame Size | ${framesize} +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Add NAT to all DUTs | | And Apply startup configuration on all VPP DUTs | | When Initialize NAT44 in 3-node circular topology @@ -86,50 +82,50 @@ | | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} *** Test Cases *** -| tc01-64B-1t1c-ethip4-ip4base-snat-1u-1p-mrr +| tc01-64B-1c-ethip4udp-ip4base-nat44-mrr | | [Tags] | 64B | 1C -| | phy_cores=${1} | framesize=${64} - -| tc02-1518B-1t1c-ethip4-ip4base-snat-1u-1p-mrr -| | [Tags] | 1518B | 1C -| | phy_cores=${1} | framesize=${1518} +| | framesize=${64} | phy_cores=${1} -| tc03-9000B-1t1c-ethip4-ip4base-snat-1u-1p-mrr -| | [Tags] | 9000B | 1C -| | phy_cores=${1} | framesize=${9000} +| tc02-64B-2c-ethip4udp-ip4base-nat44-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} -| tc04-IMIX-1t1c-ethip4-ip4base-snat-1u-1p-mrr -| | [Tags] | IMIX | 1C -| | phy_cores=${1} | framesize=IMIX_v4_1 +| tc03-64B-4c-ethip4udp-ip4base-nat44-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} -| tc05-64B-2t2c-ethip4-ip4base-snat-1u-1p-mrr -| | [Tags] | 64B | 2C -| | phy_cores=${2} | framesize=${64} +| tc04-1518B-1c-ethip4udp-ip4base-nat44-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} -| tc06-1518B-2t2c-ethip4-ip4base-snat-1u-1p-mrr +| tc05-1518B-2c-ethip4udp-ip4base-nat44-mrr | | [Tags] | 1518B | 2C -| | phy_cores=${2} | framesize=${1518} - -| tc07-9000B-2t2c-ethip4-ip4base-snat-1u-1p-mrr -| | [Tags] | 9000B | 2C -| | phy_cores=${2} | framesize=${9000} +| | framesize=${1518} | phy_cores=${2} -| tc08-IMIX-2t2c-ethip4-ip4base-snat-1u-1p-mrr -| | [Tags] | IMIX | 2C -| | phy_cores=${2} | framesize=IMIX_v4_1 +| tc06-1518B-4c-ethip4udp-ip4base-nat44-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} -| tc09-64B-4t4c-ethip4-ip4base-snat-1u-1p-mrr -| | [Tags] | 64B | 4C -| | phy_cores=${4} | framesize=${64} +| tc07-9000B-1c-ethip4udp-ip4base-nat44-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc10-1518B-4t4c-ethip4-ip4base-snat-1u-1p-mrr -| | [Tags] | 1518B | 4C -| | phy_cores=${4} | framesize=${1518} +| tc08-9000B-2c-ethip4udp-ip4base-nat44-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc11-9000B-4t4c-ethip4-ip4base-snat-1u-1p-mrr +| tc09-9000B-4c-ethip4udp-ip4base-nat44-mrr | | [Tags] | 9000B | 4C -| | phy_cores=${4} | framesize=${9000} +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-nat44-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-nat44-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc12-IMIX-4t4c-ethip4-ip4base-snat-1u-1p-mrr +| tc12-IMIX-4c-ethip4udp-ip4base-nat44-mrr | | [Tags] | IMIX | 4C -| | phy_cores=${4} | framesize=IMIX_v4_1 +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4base-nat44-ndrpdr.robot b/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4base-nat44-ndrpdr.robot new file mode 100644 index 0000000000..303abd5e23 --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4base-nat44-ndrpdr.robot @@ -0,0 +1,137 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| Resource | resources/libraries/robot/ip/nat.robot +| Resource | resources/libraries/robot/shared/traffic.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR +| ... | NIC_Intel-X710 | ETH | IP4FWD | FEATURE | NAT44 | BASE +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X710 +| ... | AND | Set up performance test suite with NAT +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down performance discovery test | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Test Template | Local Template +| ... +| Documentation | *RFC2544: Pkt throughput NAT44 performance test cases* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for IPv4 routing. +| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 +| ... | routing and two static IPv4 /24 and IPv4/20 route entries. DUT1 and DUT2 +| ... | tested with 2p10GE NIC X710 by Intel. +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage +| ... | of packets transmitted. NDR and PDR are discovered for different +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on links to DUTs. TG traffic profile +| ... | contains two L3 flow-groups (flow-group per direction, 253 flows per +| ... | flow-group) with all packets containing Ethernet header, IPv4 header +| ... | with IP protocol=61 and static payload. MAC addresses are matching MAC +| ... | addresses of the TG node interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X710 bandwidth limit +| ${s_limit}= | ${10000000000} +# Traffic profile: +| ${traffic_profile}= | trex-sl-3n-ethip4udp-1u1p + +*** Keywords *** +| Local Template +| | ... +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config. +| | ... | Each DUT uses ${phy_cores} physical core(s) for worker threads. +| | ... | [Ver] Measure NDR and PDR values using MLRsearch algorithm.\ +| | ... +| | ... | *Arguments:* +| | ... | - framesize - Framesize in Bytes in integer or string (IMIX_v4_1). +| | ... | Type: integer, string +| | ... | - phy_cores - Number of physical cores. Type: integer +| | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer +| | ... +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Add NAT to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | When Initialize NAT44 in 3-node circular topology +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4base-nat44-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4base-nat44-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4base-nat44-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4base-nat44-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4base-nat44-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4base-nat44-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4base-nat44-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4base-nat44-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4base-nat44-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4base-nat44-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4base-nat44-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4base-nat44-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4scale1000-udpsrcscale15-nat44-mrr.robot b/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4scale1000-udpsrcscale15-nat44-mrr.robot index 386b742adb..59383d75c2 100644 --- a/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4scale1000-udpsrcscale15-nat44-mrr.robot +++ b/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4scale1000-udpsrcscale15-nat44-mrr.robot @@ -30,7 +30,7 @@ | ... | Test Teardown | Tear down performance mrr test | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *Raw results NAT44 performance test cases* | ... @@ -56,7 +56,7 @@ | ${traffic_profile}= | trex-sl-3n-ethip4udp-1000u15p *** Keywords *** -| Local template +| Local Template | | ... | | [Documentation] | | ... | [Cfg] DUT runs IPv4 routing config. @@ -70,16 +70,12 @@ | | ... | - phy_cores - Number of physical cores. Type: integer | | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer | | ... -| | [Arguments] | ${phy_cores} | ${framesize} | ${rxq}=${None} -| | ... -| | Set Test Variable | ${framesize} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} -| | ${get_framesize}= | Get Frame Size | ${framesize} +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Add NAT to all DUTs | | And Apply startup configuration on all VPP DUTs | | When Initialize NAT44 in 3-node circular topology @@ -87,50 +83,50 @@ | | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} *** Test Cases *** -| tc01-64B-1t1c-ethip4udp-ip4scale1000-udpsrcscale15-snat-mrr +| tc01-64B-1c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-mrr | | [Tags] | 64B | 1C -| | phy_cores=${1} | framesize=${64} - -| tc02-1518B-1t1c-ethip4udp-ip4scale1000-udpsrcscale15-snat-mrr -| | [Tags] | 1518B | 1C -| | phy_cores=${1} | framesize=${1518} +| | framesize=${64} | phy_cores=${1} -| tc03-9000B-1t1c-ethip4udp-ip4scale1000-udpsrcscale15-snat-mrr -| | [Tags] | 9000B | 1C -| | phy_cores=${1} | framesize=${9000} +| tc02-64B-2c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} -| tc04-IMIX-1t1c-ethip4udp-ip4scale1000-udpsrcscale15-snat-mrr -| | [Tags] | IMIX | 1C -| | phy_cores=${1} | framesize=IMIX_v4_1 +| tc03-64B-4c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} -| tc05-64B-2t2c-ethip4udp-ip4scale1000-udpsrcscale15-snat-mrr -| | [Tags] | 64B | 2C -| | phy_cores=${2} | framesize=${64} +| tc04-1518B-1c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} -| tc06-1518B-2t2c-ethip4udp-ip4scale1000-udpsrcscale15-snat-mrr +| tc05-1518B-2c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-mrr | | [Tags] | 1518B | 2C -| | phy_cores=${2} | framesize=${1518} - -| tc07-9000B-2t2c-ethip4udp-ip4scale1000-udpsrcscale15-snat-mrr -| | [Tags] | 9000B | 2C -| | phy_cores=${2} | framesize=${9000} +| | framesize=${1518} | phy_cores=${2} -| tc08-IMIX-2t2c-ethip4udp-ip4scale1000-udpsrcscale15-snat-mrr -| | [Tags] | IMIX | 2C -| | phy_cores=${2} | framesize=IMIX_v4_1 +| tc06-1518B-4c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} -| tc09-64B-4t4c-ethip4udp-ip4scale1000-udpsrcscale15-snat-mrr -| | [Tags] | 64B | 4C -| | phy_cores=${4} | framesize=${64} +| tc07-9000B-1c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc10-1518B-4t4c-ethip4udp-ip4scale1000-udpsrcscale15-snat-mrr -| | [Tags] | 1518B | 4C -| | phy_cores=${4} | framesize=${1518} +| tc08-9000B-2c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc11-9000B-4t4c-ethip4udp-ip4scale1000-udpsrcscale15-snat-mrr +| tc09-9000B-4c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-mrr | | [Tags] | 9000B | 4C -| | phy_cores=${4} | framesize=${9000} +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc12-IMIX-4t4c-ethip4udp-ip4scale1000-udpsrcscale15-snat-mrr +| tc12-IMIX-4c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-mrr | | [Tags] | IMIX | 4C -| | phy_cores=${4} | framesize=IMIX_v4_1 +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4scale1000-udpsrcscale15-nat44-ndrpdr.robot b/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4scale1000-udpsrcscale15-nat44-ndrpdr.robot new file mode 100644 index 0000000000..8354abcb7b --- /dev/null +++ b/tests/vpp/perf/ip4/10ge2p1x710-ethip4udp-ip4scale1000-udpsrcscale15-nat44-ndrpdr.robot @@ -0,0 +1,137 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| Resource | resources/libraries/robot/ip/nat.robot +| Resource | resources/libraries/robot/shared/traffic.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR +| ... | NIC_Intel-X710 | ETH | IP4FWD | FEATURE | NAT44 | SRC_USER_1000 +| ... | SCALE +| ... +| Suite Setup | Run Keywords +| ... | Set up 3-node performance topology with DUT's NIC model | L3 +| ... | Intel-X710 +| ... | AND | Set up performance test suite with NAT +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| ... +| Test Teardown | Tear down performance discovery test | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Test Template | Local Template +| ... +| Documentation | *RFC2544: Pkt throughput NAT44 performance test cases* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-UDP for IPv4 routing. +| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 +| ... | routing and two static IPv4 /24 and IPv4/20 route entries. DUT1 and DUT2 +| ... | tested with 2p10GE NIC X710 by Intel. +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage +| ... | of packets transmitted. NDR and PDR are discovered for different +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on links to DUTs. TG traffic profile +| ... | contains two L3 flow-groups (flow-group per direction, 253 flows per +| ... | flow-group) with all packets containing Ethernet header, IPv4 header +| ... | with IP protocol=61 and static payload. MAC addresses are matching MAC +| ... | addresses of the TG node interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# X710 bandwidth limit +| ${s_limit}= | ${10000000000} +# Traffic profile: +| ${traffic_profile}= | trex-sl-3n-ethip4udp-1000u15p + +*** Keywords *** +| Local Template +| | ... +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config. +| | ... | Each DUT uses ${phy_cores} physical core(s) for worker threads. +| | ... | [Ver] Measure NDR and PDR values using MLRsearch algorithm.\ +| | ... +| | ... | *Arguments:* +| | ... | - framesize - Framesize in Bytes in integer or string (IMIX_v4_1). +| | ... | Type: integer, string +| | ... | - phy_cores - Number of physical cores. Type: integer +| | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer +| | ... +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} +| | And Add NAT to all DUTs +| | And Apply startup configuration on all VPP DUTs +| | When Initialize NAT44 in 3-node circular topology +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} + +*** Test Cases *** +| tc01-64B-1c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4udp-ip4scale1000-udpsrcscale15-nat44-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/2n1l-10ge2p1x710-ethip4-ip4base-mrr.robot b/tests/vpp/perf/ip4/2n1l-10ge2p1x710-ethip4-ip4base-mrr.robot index 30fdf44519..1e9a36d7c4 100644 --- a/tests/vpp/perf/ip4/2n1l-10ge2p1x710-ethip4-ip4base-mrr.robot +++ b/tests/vpp/perf/ip4/2n1l-10ge2p1x710-ethip4-ip4base-mrr.robot @@ -24,7 +24,7 @@ | Test Setup | Set up performance test | Test Teardown | Tear down performance mrr test | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *Raw results IPv4 routing test cases* | ... @@ -50,7 +50,7 @@ | ${traffic_profile}= | trex-sl-2n-ethip4-ip4src253 *** Keywords *** -| Local template +| Local Template | | [Documentation] | | ... | [Cfg] DUT runs IPv4 routing config.\ | | ... | Each DUT uses ${phy_cores} physical core(s) for worker threads. @@ -65,63 +65,60 @@ | | ... | | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... -| | Set Test Variable | ${framesize} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | When Initialize IPv4 forwarding in 2-node circular topology | | Then Traffic should pass with maximum rate | | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} *** Test Cases *** -| tc01-64B-1t1c-ethip4-ip4base-mrr +| tc01-64B-1c-ethip4-ip4base-mrr | | [Tags] | 64B | 1C | | framesize=${64} | phy_cores=${1} -| tc02-1518B-1t1c-ethip4-ip4base-mrr -| | [Tags] | 1518B | 1C -| | framesize=${1518} | phy_cores=${1} - -| tc03-9000B-1t1c-ethip4-ip4base-mrr -| | [Tags] | 9000B | 1C -| | framesize=${9000} | phy_cores=${1} - -| tc04-IMIX-1t1c-ethip4-ip4base-mrr -| | [Tags] | IMIX | 1C -| | framesize=IMIX_v4_1 | phy_cores=${1} - -| tc05-64B-2t2c-ethip4-ip4base-mrr +| tc02-64B-2c-ethip4-ip4base-mrr | | [Tags] | 64B | 2C | | framesize=${64} | phy_cores=${2} -| tc06-1518B-2t2c-ethip4-ip4base-mrr -| | [Tags] | 1518B | 2C -| | framesize=${1518} | phy_cores=${2} - -| tc07-9000B-2t2c-ethip4-ip4base-mrr -| | [Tags] | 9000B | 2C -| | framesize=${9000} | phy_cores=${2} - -| tc08-IMIX-2t2c-ethip4-ip4base-mrr -| | [Tags] | IMIX | 2C -| | framesize=IMIX_v4_1 | phy_cores=${2} - -| tc09-64B-4t4c-ethip4-ip4base-mrr +| tc03-64B-4c-ethip4-ip4base-mrr | | [Tags] | 64B | 4C | | framesize=${64} | phy_cores=${4} -| tc10-1518B-4t4c-ethip4-ip4base-mrr +| tc04-1518B-1c-ethip4-ip4base-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4-ip4base-mrr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4-ip4base-mrr | | [Tags] | 1518B | 4C | | framesize=${1518} | phy_cores=${4} -| tc11-9000B-4t4c-ethip4-ip4base-mrr +| tc07-9000B-1c-ethip4-ip4base-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4-ip4base-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4-ip4base-mrr | | [Tags] | 9000B | 4C | | framesize=${9000} | phy_cores=${4} -| tc12-IMIX-4t4c-ethip4-ip4base-mrr +| tc10-IMIX-1c-ethip4-ip4base-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4-ip4base-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4-ip4base-mrr | | [Tags] | IMIX | 4C | | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/2n1l-10ge2p1x710-ethip4-ip4base-ndrpdr.robot b/tests/vpp/perf/ip4/2n1l-10ge2p1x710-ethip4-ip4base-ndrpdr.robot index 92fef78eda..c583cb91dd 100644 --- a/tests/vpp/perf/ip4/2n1l-10ge2p1x710-ethip4-ip4base-ndrpdr.robot +++ b/tests/vpp/perf/ip4/2n1l-10ge2p1x710-ethip4-ip4base-ndrpdr.robot @@ -22,13 +22,12 @@ | Suite Teardown | Tear down 2-node performance topology | ... | Test Setup | Set up performance test -| ... | Test Teardown | Tear down performance discovery test | ${min_rate}pps | ... | ${framesize} | ${traffic_profile} | ... -| Test Template | Local template +| Test Template | Local Template | ... -| Documentation | *Packet throughput IPv4 routing test cases* +| Documentation | *RFC2544: Pkt throughput IPv4 routing test cases* | ... | ... | *[Top] Network Topologies:* TG-DUT1-TG 2-node circular topology | ... | with single links between nodes. @@ -36,16 +35,16 @@ | ... | *[Cfg] DUT configuration:* DUT1 is configured with IPv4 | ... | routing and two static IPv4 /24 route entries. DUT1 tested with | ... | 2p10GE NIC X710 by Intel. -| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\ -| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop\ -| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\ -| ... | of packets transmitted. NDR and PDR are discovered for different\ -| ... | Ethernet L2 frame sizes using MLRsearch library\ -| ... | Test packets are generated by TG on\ -| ... | links to DUTs. TG traffic profile contains two L3 flow-groups\ -| ... | (flow-group per direction, 254 flows per flow-group) with\ -| ... | all packets containing Ethernet header,IPv4 header with static payload.\ -| ... | MAC addresses are matching MAC addresses of the TG node interfaces. +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage +| ... | of packets transmitted. NDR and PDR are discovered for different +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on links to DUTs. TG traffic profile +| ... | contains two L3 flow-groups (flow-group per direction, 253 flows per +| ... | flow-group) with all packets containing Ethernet header, IPv4 header +| ... | with IP protocol=61 and static payload. MAC addresses are matching MAC +| ... | addresses of the TG node interfaces. | ... | *[Ref] Applicable standard specifications:* RFC2544. *** Variables *** @@ -55,11 +54,11 @@ | ${traffic_profile}= | trex-sl-2n-ethip4-ip4src253 *** Keywords *** -| Local template +| Local Template | | [Documentation] | | ... | [Cfg] DUT runs IPv4 routing config.\ | | ... | Each DUT uses ${phy_cores} physical core(s) for worker threads. -| | ... | [Ver] Measure NDR and PDR values using MLRsearch algorithm. +| | ... | [Ver] Measure NDR and PDR values using MLRsearch algorithm.\ | | ... | | ... | *Arguments:* | | ... | - framesize - Framesize in Bytes in integer or string (IMIX_v4_1). @@ -70,63 +69,62 @@ | | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} | ${20000} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ${max_rate}= | Calculate pps | ${s_limit} | ${framesize} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_limit} | ${framesize} | | And Apply startup configuration on all VPP DUTs | | When Initialize IPv4 forwarding in 2-node circular topology | | Then Find NDR and PDR intervals using optimized search | | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} *** Test Cases *** -| tc01-64B-1t1c-ethip4-ip4base-ndrpdr +| tc01-64B-1c-ethip4-ip4base-ndrpdr | | [Tags] | 64B | 1C | | framesize=${64} | phy_cores=${1} -| tc02-1518B-1t1c-ethip4-ip4base-ndrpdr -| | [Tags] | 1518B | 1C -| | framesize=${1518} | phy_cores=${1} - -| tc03-9000B-1t1c-ethip4-ip4base-ndrpdr -| | [Tags] | 9000B | 1C -| | framesize=${9000} | phy_cores=${1} - -| tc04-IMIX-1t1c-ethip4-ip4base-ndrpdr -| | [Tags] | IMIX | 1C -| | framesize=IMIX_v4_1 | phy_cores=${1} - -| tc05-64B-2t2c-ethip4-ip4base-ndrpdr +| tc02-64B-2c-ethip4-ip4base-ndrpdr | | [Tags] | 64B | 2C | | framesize=${64} | phy_cores=${2} -| tc06-1518B-2t2c-ethip4-ip4base-ndrpdr -| | [Tags] | 1518B | 2C -| | framesize=${1518} | phy_cores=${2} - -| tc07-9000B-2t2c-ethip4-ip4base-ndrpdr -| | [Tags] | 9000B | 2C -| | framesize=${9000} | phy_cores=${2} - -| tc08-IMIX-2t2c-ethip4-ip4base-ndrpdr -| | [Tags] | IMIX | 2C -| | framesize=IMIX_v4_1 | phy_cores=${2} - -| tc09-64B-4t4c-ethip4-ip4base-ndrpdr +| tc03-64B-4c-ethip4-ip4base-ndrpdr | | [Tags] | 64B | 4C | | framesize=${64} | phy_cores=${4} -| tc10-1518B-4t4c-ethip4-ip4base-ndrpdr +| tc04-1518B-1c-ethip4-ip4base-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4-ip4base-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4-ip4base-ndrpdr | | [Tags] | 1518B | 4C | | framesize=${1518} | phy_cores=${4} -| tc11-9000B-4t4c-ethip4-ip4base-ndrpdr +| tc07-9000B-1c-ethip4-ip4base-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4-ip4base-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4-ip4base-ndrpdr | | [Tags] | 9000B | 4C | | framesize=${9000} | phy_cores=${4} -| tc12-IMIX-4t4c-ethip4-ip4base-ndrpdr +| tc10-IMIX-1c-ethip4-ip4base-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4-ip4base-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4-ip4base-ndrpdr | | [Tags] | IMIX | 4C | | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/2n1l-25ge2p1xxv710-ethip4-ip4base-mrr.robot b/tests/vpp/perf/ip4/2n1l-25ge2p1xxv710-ethip4-ip4base-mrr.robot index fe3c67ec54..aebe1e85b8 100644 --- a/tests/vpp/perf/ip4/2n1l-25ge2p1xxv710-ethip4-ip4base-mrr.robot +++ b/tests/vpp/perf/ip4/2n1l-25ge2p1xxv710-ethip4-ip4base-mrr.robot @@ -24,7 +24,7 @@ | Test Setup | Set up performance test | Test Teardown | Tear down performance mrr test | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *Raw results IPv4 routing test cases* | ... @@ -52,7 +52,7 @@ | ${traffic_profile}= | trex-sl-2n-ethip4-ip4src253 *** Keywords *** -| Local template +| Local Template | | [Documentation] | | ... | [Cfg] DUT runs IPv4 routing config.\ | | ... | Each DUT uses ${phy_cores} physical core(s) for worker threads. @@ -67,65 +67,60 @@ | | ... | | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... -| | Set Test Variable | ${framesize} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ${max_rate}= | Calculate pps | ${s_25G} | ${get_framesize} -| | ${max_rate}= | Set Variable If -| | ... | ${max_rate} > ${s_18.75Mpps} | ${s_18.75Mpps} | ${max_rate} | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo +| | ... | ${s_24.5G} | ${framesize} | pps_limit=${s_18.75Mpps} | | And Apply startup configuration on all VPP DUTs | | When Initialize IPv4 forwarding in 2-node circular topology | | Then Traffic should pass with maximum rate | | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} *** Test Cases *** -| tc01-64B-1t1c-ethip4-ip4base-mrr +| tc01-64B-1c-ethip4-ip4base-mrr | | [Tags] | 64B | 1C | | framesize=${64} | phy_cores=${1} -| tc02-1518B-1t1c-ethip4-ip4base-mrr -| | [Tags] | 1518B | 1C -| | framesize=${1518} | phy_cores=${1} - -| tc03-9000B-1t1c-ethip4-ip4base-mrr -| | [Tags] | 9000B | 1C -| | framesize=${9000} | phy_cores=${1} - -| tc04-IMIX-1t1c-ethip4-ip4base-mrr -| | [Tags] | IMIX | 1C -| | framesize=IMIX_v4_1 | phy_cores=${1} - -| tc05-64B-2t2c-ethip4-ip4base-mrr +| tc02-64B-2c-ethip4-ip4base-mrr | | [Tags] | 64B | 2C | | framesize=${64} | phy_cores=${2} -| tc06-1518B-2t2c-ethip4-ip4base-mrr -| | [Tags] | 1518B | 2C -| | framesize=${1518} | phy_cores=${2} - -| tc07-9000B-2t2c-ethip4-ip4base-mrr -| | [Tags] | 9000B | 2C -| | framesize=${9000} | phy_cores=${2} - -| tc08-IMIX-2t2c-ethip4-ip4base-mrr -| | [Tags] | IMIX | 2C -| | framesize=IMIX_v4_1 | phy_cores=${2} - -| tc09-64B-4t4c-ethip4-ip4base-mrr +| tc03-64B-4c-ethip4-ip4base-mrr | | [Tags] | 64B | 4C | | framesize=${64} | phy_cores=${4} -| tc10-1518B-4t4c-ethip4-ip4base-mrr +| tc04-1518B-1c-ethip4-ip4base-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4-ip4base-mrr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4-ip4base-mrr | | [Tags] | 1518B | 4C | | framesize=${1518} | phy_cores=${4} -| tc11-9000B-4t4c-ethip4-ip4base-mrr +| tc07-9000B-1c-ethip4-ip4base-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4-ip4base-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4-ip4base-mrr | | [Tags] | 9000B | 4C | | framesize=${9000} | phy_cores=${4} -| tc12-IMIX-4t4c-ethip4-ip4base-mrr +| tc10-IMIX-1c-ethip4-ip4base-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4-ip4base-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4-ip4base-mrr | | [Tags] | IMIX | 4C | | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/2n1l-25ge2p1xxv710-ethip4-ip4base-ndrpdr.robot b/tests/vpp/perf/ip4/2n1l-25ge2p1xxv710-ethip4-ip4base-ndrpdr.robot index d041a9d5da..a1d6aaa16a 100644 --- a/tests/vpp/perf/ip4/2n1l-25ge2p1xxv710-ethip4-ip4base-ndrpdr.robot +++ b/tests/vpp/perf/ip4/2n1l-25ge2p1xxv710-ethip4-ip4base-ndrpdr.robot @@ -26,7 +26,7 @@ | Test Teardown | Tear down performance discovery test | ${min_rate}pps | ... | ${framesize} | ${traffic_profile} | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *Packet throughput IPv4 routing test cases* | ... @@ -57,7 +57,7 @@ | ${traffic_profile}= | trex-sl-2n-ethip4-ip4src253 *** Keywords *** -| Local template +| Local Template | | [Documentation] | | ... | [Cfg] DUT runs IPv4 routing config.\ | | ... | Each DUT uses ${phy_cores} physical core(s) for worker threads. @@ -72,65 +72,62 @@ | | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Set Test Variable | ${framesize} -| | Set Test Variable | ${min_rate} | ${20000} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ${max_rate}= | Calculate pps | ${s_25G} | ${get_framesize} -| | ${max_rate}= | Set Variable If -| | ... | ${max_rate} > ${s_18.75Mpps} | ${s_18.75Mpps} | ${max_rate} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo And Handle Multi Seg +| | ... | ${s_25G} | ${framesize} | pps_limit=${s_18.75Mpps} | | And Apply startup configuration on all VPP DUTs | | When Initialize IPv4 forwarding in 2-node circular topology | | Then Find NDR and PDR intervals using optimized search | | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} *** Test Cases *** -| tc01-64B-1t1c-ethip4-ip4base-ndrpdr +| tc01-64B-1c-ethip4-ip4base-ndrpdr | | [Tags] | 64B | 1C | | framesize=${64} | phy_cores=${1} -| tc02-1518B-1t1c-ethip4-ip4base-ndrpdr -| | [Tags] | 1518B | 1C -| | framesize=${1518} | phy_cores=${1} - -| tc03-9000B-1t1c-ethip4-ip4base-ndrpdr -| | [Tags] | 9000B | 1C -| | framesize=${9000} | phy_cores=${1} - -| tc04-IMIX-1t1c-ethip4-ip4base-ndrpdr -| | [Tags] | IMIX | 1C -| | framesize=IMIX_v4_1 | phy_cores=${1} - -| tc05-64B-2t2c-ethip4-ip4base-ndrpdr +| tc02-64B-2c-ethip4-ip4base-ndrpdr | | [Tags] | 64B | 2C | | framesize=${64} | phy_cores=${2} -| tc06-1518B-2t2c-ethip4-ip4base-ndrpdr -| | [Tags] | 1518B | 2C -| | framesize=${1518} | phy_cores=${2} - -| tc07-9000B-2t2c-ethip4-ip4base-ndrpdr -| | [Tags] | 9000B | 2C -| | framesize=${9000} | phy_cores=${2} - -| tc08-IMIX-2t2c-ethip4-ip4base-ndrpdr -| | [Tags] | IMIX | 2C -| | framesize=IMIX_v4_1 | phy_cores=${2} - -| tc09-64B-4t4c-ethip4-ip4base-ndrpdr +| tc03-64B-4c-ethip4-ip4base-ndrpdr | | [Tags] | 64B | 4C | | framesize=${64} | phy_cores=${4} -| tc10-1518B-4t4c-ethip4-ip4base-ndrpdr +| tc04-1518B-1c-ethip4-ip4base-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4-ip4base-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4-ip4base-ndrpdr | | [Tags] | 1518B | 4C | | framesize=${1518} | phy_cores=${4} -| tc11-9000B-4t4c-ethip4-ip4base-ndrpdr +| tc07-9000B-1c-ethip4-ip4base-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4-ip4base-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4-ip4base-ndrpdr | | [Tags] | 9000B | 4C | | framesize=${9000} | phy_cores=${4} -| tc12-IMIX-4t4c-ethip4-ip4base-ndrpdr +| tc10-IMIX-1c-ethip4-ip4base-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4-ip4base-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4-ip4base-ndrpdr | | [Tags] | IMIX | 4C | | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/40ge2p1xl710-ethip4-ip4base-mrr.robot b/tests/vpp/perf/ip4/40ge2p1xl710-ethip4-ip4base-mrr.robot index 9cb24f6701..2e025ff69c 100644 --- a/tests/vpp/perf/ip4/40ge2p1xl710-ethip4-ip4base-mrr.robot +++ b/tests/vpp/perf/ip4/40ge2p1xl710-ethip4-ip4base-mrr.robot @@ -24,7 +24,7 @@ | Test Setup | Set up performance test | Test Teardown | Tear down performance mrr test | ... -| Test Template | Local template +| Test Template | Local Template | ... | Documentation | *Raw results IPv4 routing test cases* | ... @@ -52,7 +52,7 @@ | ${traffic_profile}= | trex-sl-3n-ethip4-ip4src253 *** Keywords *** -| Local template +| Local Template | | ... | | [Documentation] | | ... | [Cfg] DUT runs IPv4 routing config. @@ -66,68 +66,62 @@ | | ... | - phy_cores - Number of physical cores. Type: integer | | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer | | ... -| | [Arguments] | ${phy_cores} | ${framesize} | ${rxq}=${None} -| | ... -| | Set Test Variable | ${framesize} -| | ${get_framesize}= | Get Frame Size | ${framesize} -| | ${max_rate}= | Calculate pps | ${s_24.5G} | ${get_framesize} -| | ${max_rate}= | Set Variable If -| | ... | ${max_rate} > ${s_18.75Mpps} | ${s_18.75Mpps} | ${max_rate} +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} | | ... | | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} | | And Add PCI devices to all DUTs -| | And Run Keyword If | ${get_framesize} < ${1522} -| | ... | Add no multi seg to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo +| | ... | ${s_24.5G} | ${framesize} | pps_limit=${s_18.75Mpps} | | And Apply startup configuration on all VPP DUTs | | When Initialize IPv4 forwarding in 3-node circular topology | | Then Traffic should pass with maximum rate | | ... | ${max_rate}pps | ${framesize} | ${traffic_profile} *** Test Cases *** -| tc01-64B-1t1c-ethip4-ip4base-mrr +| tc01-64B-1c-ethip4-ip4base-mrr | | [Tags] | 64B | 1C -| | phy_cores=${1} | framesize=${64} - -| tc02-1518B-1t1c-ethip4-ip4base-mrr -| | [Tags] | 1518B | 1C -| | phy_cores=${1} | framesize=${1518} +| | framesize=${64} | phy_cores=${1} -| tc03-9000B-1t1c-ethip4-ip4base-mrr -| | [Tags] | 9000B | 1C -| | phy_cores=${1} | framesize=${9000} +| tc02-64B-2c-ethip4-ip4base-mrr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} -| tc04-IMIX-1t1c-ethip4-ip4base-mrr -| | [Tags] | IMIX | 1C -| | phy_cores=${1} | framesize=IMIX_v4_1 +| tc03-64B-4c-ethip4-ip4base-mrr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} -| tc05-64B-2t2c-ethip4-ip4base-mrr -| | [Tags] | 64B | 2C -| | phy_cores=${2} | framesize=${64} +| tc04-1518B-1c-ethip4-ip4base-mrr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} -| tc06-1518B-2t2c-ethip4-ip4base-mrr +| tc05-1518B-2c-ethip4-ip4base-mrr | | [Tags] | 1518B | 2C -| | phy_cores=${2} | framesize=${1518} - -| tc07-9000B-2t2c-ethip4-ip4base-mrr -| | [Tags] | 9000B | 2C -| | phy_cores=${2} | framesize=${9000} +| | framesize=${1518} | phy_cores=${2} -| tc08-IMIX-2t2c-ethip4-ip4base-mrr -| | [Tags] | IMIX | 2C -| | phy_cores=${2} | framesize=IMIX_v4_1 +| tc06-1518B-4c-ethip4-ip4base-mrr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} -| tc09-64B-4t4c-ethip4-ip4base-mrr -| | [Tags] | 64B | 4C -| | phy_cores=${4} | framesize=${64} +| tc07-9000B-1c-ethip4-ip4base-mrr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} -| tc10-1518B-4t4c-ethip4-ip4base-mrr -| | [Tags] | 1518B | 4C -| | phy_cores=${4} | framesize=${1518} +| tc08-9000B-2c-ethip4-ip4base-mrr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} -| tc11-9000B-4t4c-ethip4-ip4base-mrr +| tc09-9000B-4c-ethip4-ip4base-mrr | | [Tags] | 9000B | 4C -| | phy_cores=${4} | framesize=${9000} +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4-ip4base-mrr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4-ip4base-mrr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} -| tc12-IMIX-4t4c-ethip4-ip4base-mrr +| tc12-IMIX-4c-ethip4-ip4base-mrr | | [Tags] | IMIX | 4C -| | phy_cores=${4} | framesize=IMIX_v4_1 +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/40ge2p1xl710-ethip4-ip4base-ndrpdr.robot b/tests/vpp/perf/ip4/40ge2p1xl710-ethip4-ip4base-ndrpdr.robot new file mode 100644 index 0000000000..703b7c747c --- /dev/null +++ b/tests/vpp/perf/ip4/40ge2p1xl710-ethip4-ip4base-ndrpdr.robot @@ -0,0 +1,133 @@ +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Resource | resources/libraries/robot/performance/performance_setup.robot +| ... +| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR +| ... | NIC_Intel-XL710 | ETH | IP4FWD | BASE | IP4BASE +| ... +| Suite Setup | Set up 3-node performance topology with DUT's NIC model +| ... | L3 | Intel-XL710 +| Suite Teardown | Tear down 3-node performance topology +| ... +| Test Setup | Set up performance test +| Test Teardown | Tear down performance discovery test | ${min_rate}pps +| ... | ${framesize} | ${traffic_profile} +| ... +| Test Template | Local Template +| ... +| Documentation | *RFC2544: Pkt throughput IPv4 routing test cases* +| ... +| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology +| ... | with single links between nodes. +| ... | *[Enc] Packet Encapsulations:* Eth-IPv4 for IPv4 routing. +| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 +| ... | routing and two static IPv4 /24 route entries. DUT1 and DUT2 tested with +| ... | 2p10GE NIC XL710 by Intel. +| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop +| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop +| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage +| ... | of packets transmitted. NDR and PDR are discovered for different +| ... | Ethernet L2 frame sizes using MLRsearch library. +| ... | Test packets are generated by TG on links to DUTs. TG traffic profile +| ... | contains two L3 flow-groups (flow-group per direction, 253 flows per +| ... | flow-group) with all packets containing Ethernet header, IPv4 header +| ... | with IP protocol=61 and static payload. MAC addresses are matching MAC +| ... | addresses of the TG node interfaces. +| ... | *[Ref] Applicable standard specifications:* RFC2544. + +*** Variables *** +# XL710-DA2 bandwidth limit ~49Gbps/2=24.5Gbps +| ${s_24.5G} | ${24500000000} +# XL710-DA2 Mpps limit 37.5Mpps/2=18.75Mpps +| ${s_18.75Mpps} | ${18750000} +# Traffic profile: +| ${traffic_profile}= | trex-sl-3n-ethip4-ip4src253 + +*** Keywords *** +| Local Template +| | ... +| | [Documentation] +| | ... | [Cfg] DUT runs IPv4 routing config. +| | ... | Each DUT uses ${phy_cores} physical core(s) for worker threads. +| | ... | [Ver] Measure NDR and PDR values using MLRsearch algorithm.\ +| | ... +| | ... | *Arguments:* +| | ... | - framesize - Framesize in Bytes in integer or string (IMIX_v4_1). +| | ... | Type: integer, string +| | ... | - phy_cores - Number of physical cores. Type: integer +| | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer +| | ... +| | [Arguments] | ${framesize} | ${phy_cores} | ${rxq}=${None} +| | ... +| | Set Test Variable | ${framesize} +| | Set Test Variable | ${min_rate} | ${10000} +| | ... +| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq} +| | And Add PCI devices to all DUTs +| | ${max_rate} | ${jumbo} = | Get Max Rate And Jumbo +| | ... | ${s_24.5G} | ${framesize} | pps_limit=${s_18.75Mpps} +| | And Apply startup configuration on all VPP DUTs +| | When Initialize IPv4 forwarding in 3-node circular topology +| | Then Find NDR and PDR intervals using optimized search +| | ... | ${framesize} | ${traffic_profile} | ${min_rate} | ${max_rate} + +*** Test Cases *** +| tc01-64B-1c-ethip4-ip4base-ndrpdr +| | [Tags] | 64B | 1C +| | framesize=${64} | phy_cores=${1} + +| tc02-64B-2c-ethip4-ip4base-ndrpdr +| | [Tags] | 64B | 2C +| | framesize=${64} | phy_cores=${2} + +| tc03-64B-4c-ethip4-ip4base-ndrpdr +| | [Tags] | 64B | 4C +| | framesize=${64} | phy_cores=${4} + +| tc04-1518B-1c-ethip4-ip4base-ndrpdr +| | [Tags] | 1518B | 1C +| | framesize=${1518} | phy_cores=${1} + +| tc05-1518B-2c-ethip4-ip4base-ndrpdr +| | [Tags] | 1518B | 2C +| | framesize=${1518} | phy_cores=${2} + +| tc06-1518B-4c-ethip4-ip4base-ndrpdr +| | [Tags] | 1518B | 4C +| | framesize=${1518} | phy_cores=${4} + +| tc07-9000B-1c-ethip4-ip4base-ndrpdr +| | [Tags] | 9000B | 1C +| | framesize=${9000} | phy_cores=${1} + +| tc08-9000B-2c-ethip4-ip4base-ndrpdr +| | [Tags] | 9000B | 2C +| | framesize=${9000} | phy_cores=${2} + +| tc09-9000B-4c-ethip4-ip4base-ndrpdr +| | [Tags] | 9000B | 4C +| | framesize=${9000} | phy_cores=${4} + +| tc10-IMIX-1c-ethip4-ip4base-ndrpdr +| | [Tags] | IMIX | 1C +| | framesize=IMIX_v4_1 | phy_cores=${1} + +| tc11-IMIX-2c-ethip4-ip4base-ndrpdr +| | [Tags] | IMIX | 2C +| | framesize=IMIX_v4_1 | phy_cores=${2} + +| tc12-IMIX-4c-ethip4-ip4base-ndrpdr +| | [Tags] | IMIX | 4C +| | framesize=IMIX_v4_1 | phy_cores=${4} diff --git a/tests/vpp/perf/ip4/40ge2p1xl710-ethip4-ip4base-ndrpdrdisc.robot b/tests/vpp/perf/ip4/40ge2p1xl710-ethip4-ip4base-ndrpdrdisc.robot deleted file mode 100644 index b913ed6c8e..0000000000 --- a/tests/vpp/perf/ip4/40ge2p1xl710-ethip4-ip4base-ndrpdrdisc.robot +++ /dev/null @@ -1,275 +0,0 @@ -# Copyright (c) 2017 Cisco and/or its affiliates. -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at: -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -*** Settings *** -| Resource | resources/libraries/robot/performance/performance_setup.robot -| Library | resources.libraries.python.topology.Topology -| Library | resources.libraries.python.NodePath -| Library | resources.libraries.python.InterfaceUtil -| Library | resources.libraries.python.IPv4Setup.Dut | ${nodes['DUT1']} -| ... | WITH NAME | dut1_v4 -| Library | resources.libraries.python.IPv4Setup.Dut | ${nodes['DUT2']} -| ... | WITH NAME | dut2_v4 -| ... -| Force Tags | 3_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDRDISC -| ... | NIC_Intel-XL710 | ETH | IP4FWD | BASE -| ... -| Suite Setup | Set up 3-node performance topology with DUT's NIC model -| ... | L3 | Intel-XL710 -| Suite Teardown | Tear down 3-node performance topology -| ... -| Test Setup | Set up performance test -| ... -| Test Teardown | Tear down performance discovery test | ${min_rate}pps -| ... | ${framesize} | ${traffic_profile} -| ... -| Documentation | *RFC2544: Pkt throughput IPv4 routing test cases* -| ... -| ... | *[Top] Network Topologies:* TG-DUT1-DUT2-TG 3-node circular topology -| ... | with single links between nodes. -| ... | *[Enc] Packet Encapsulations:* Eth-IPv4 for IPv4 routing. -| ... | *[Cfg] DUT configuration:* DUT1 and DUT2 are configured with IPv4 -| ... | routing and two static IPv4 /24 route entries. DUT1 and DUT2 tested with -| ... | 2p40GE NIC XL710 by Intel. -| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop -| ... | Rate) with zero packet loss tolerance or throughput PDR (Partial Drop -| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage -| ... | of packets transmitted. NDR and PDR are discovered for different -| ... | Ethernet L2 frame sizes using either binary search or linear search -| ... | algorithms with configured starting rate and final step that determines -| ... | throughput measurement resolution. Test packets are generated by TG on -| ... | links to DUTs. TG traffic profile contains two L3 flow-groups -| ... | (flow-group per direction, 253 flows per flow-group) with all packets -| ... | containing Ethernet header, IPv4 header with IP protocol=61 and static -| ... | payload. MAC addresses are matching MAC addresses of the TG node -| ... | interfaces. -| ... | *[Ref] Applicable standard specifications:* RFC2544. - -*** Variables *** -# XL710-DA2 bandwidth limit ~49Gbps/2=24.5Gbps -| ${s_24.5G} | ${24500000000} -# XL710-DA2 Mpps limit 37.5Mpps/2=18.75Mpps -| ${s_18.75Mpps} | ${18750000} -# Traffic profile: -| ${traffic_profile} | trex-sl-3n-ethip4-ip4src253 - -*** Test Cases *** -| tc01-64B-1t1c-ethip4-ip4base-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 1 phy core, \ -| | ... | 1 receive queue per NIC port. [Ver] Find NDR for 64 Byte frames -| | ... | using binary search start at 18.75Mpps rate, step 50kpps. -| | [Tags] | 64B | 1C | NDRDISC -| | ${framesize}= | Set Variable | ${64} -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Set Variable | ${s_18.75Mpps} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 1 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Add DPDK dev default RXD to all DUTs | 2048 -| | And Add DPDK dev default TXD to all DUTs | 2048 -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding in 3-node circular topology -| | Then Find NDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} - -| tc03-1518B-1t1c-ethip4-ip4base-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 1 phy core, \ -| | ... | 1 receive queue per NIC port. [Ver] Find NDR for 1518 Byte frames -| | ... | using binary search start at 24.5G rate, step 10kpps. -| | [Tags] | 1518B | 1C | NDRDISC -| | ${framesize}= | Set Variable | ${1518} -| | ${min_rate}= | Set Variable | ${10000} -| | ${max_rate}= | Calculate pps | ${s_24.5G} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 1 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Add DPDK dev default RXD to all DUTs | 2048 -| | And Add DPDK dev default TXD to all DUTs | 2048 -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding in 3-node circular topology -| | Then Find NDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} - -| tc07-64B-2t2c-ethip4-ip4base-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 2 phy cores, \ -| | ... | 1 receive queue per NIC port. [Ver] Find NDR for 64 Byte frames -| | ... | using binary search start at 18.75Mpps rate, step 50kpps. -| | [Tags] | 64B | 2C | NDRDISC -| | ${framesize}= | Set Variable | ${64} -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Set Variable | ${s_18.75Mpps} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 2 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Add DPDK dev default RXD to all DUTs | 2048 -| | And Add DPDK dev default TXD to all DUTs | 2048 -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding in 3-node circular topology -| | Then Find NDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} - -| tc09-1518B-2t2c-ethip4-ip4base-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 2 phy cores, \ -| | ... | 1 receive queue per NIC port. [Ver] Find NDR for 1518 Byte frames -| | ... | using binary search start at 24.5G rate, step 10kpps. -| | [Tags] | 1518B | 2C | NDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | ${1518} -| | ${min_rate}= | Set Variable | ${10000} -| | ${max_rate}= | Calculate pps | ${s_24.5G} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 2 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Add DPDK dev default RXD to all DUTs | 2048 -| | And Add DPDK dev default TXD to all DUTs | 2048 -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding in 3-node circular topology -| | Then Find NDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} - -| tc13-64B-4t4c-ethip4-ip4base-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 4 phy cores, \ -| | ... | 2 receive queues per NIC port. [Ver] Find NDR for 64 Byte frames -| | ... | using binary search start at 18.75Mpps rate, step 50kpps. -| | [Tags] | 64B | 4C | NDRDISC -| | ${framesize}= | Set Variable | ${64} -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Set Variable | ${s_18.75Mpps} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 4 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Add DPDK dev default RXD to all DUTs | 2048 -| | And Add DPDK dev default TXD to all DUTs | 2048 -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding in 3-node circular topology -| | Then Find NDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} - -| tc15-1518B-4t4c-ethip4-ip4base-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 4 phy cores, \ -| | ... | 2 receive queues per NIC port. [Ver] Find NDR for 1518 Byte frames -| | ... | using binary search start at 24.5G rate, step 10kpps. -| | [Tags] | 1518B | 4C | NDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | ${1518} -| | ${min_rate}= | Set Variable | ${10000} -| | ${max_rate}= | Calculate pps | ${s_24.5G} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 4 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Add DPDK dev default RXD to all DUTs | 2048 -| | And Add DPDK dev default TXD to all DUTs | 2048 -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding in 3-node circular topology -| | Then Find NDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} - -| tc19-IMIX-1t1c-ethip4-ip4base-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 1 phy core, \ -| | ... | 1 receive queue per NIC port. [Ver] Find NDR for IMIX_v4_1 frame size -| | ... | using binary search start at 24.5G rate, step 50kpps. -| | ... | IMIX_v4_1 = (28x64B;16x570B;4x1518B) -| | [Tags] | IMIX | 1C | NDRDISC -| | ${framesize}= | Set Variable | IMIX_v4_1 -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Calculate pps | ${s_24.5G} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 1 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Add DPDK dev default RXD to all DUTs | 2048 -| | And Add DPDK dev default TXD to all DUTs | 2048 -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding in 3-node circular topology -| | Then Find NDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} - -| tc20-IMIX-2t2c-ethip4-ip4base-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 2 phy core, \ -| | ... | 1 receive queue per NIC port. [Ver] Find NDR for IMIX_v4_1 frame size -| | ... | using binary search start at 24.5G rate, step 50kpps. -| | ... | IMIX_v4_1 = (28x64B;16x570B;4x1518B) -| | [Tags] | IMIX | 2C | NDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | IMIX_v4_1 -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Calculate pps | ${s_24.5G} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 2 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Add DPDK dev default RXD to all DUTs | 2048 -| | And Add DPDK dev default TXD to all DUTs | 2048 -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding in 3-node circular topology -| | Then Find NDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} - -| tc21-IMIX-4t4c-ethip4-ip4base-ndrdisc -| | [Documentation] -| | ... | [Cfg] DUT runs IPv4 routing config with 4 thread, 4 phy core, \ -| | ... | 2 receive queue per NIC port. [Ver] Find NDR for IMIX_v4_1 frame size -| | ... | using binary search start at 24.5G rate, step 50kpps. -| | ... | IMIX_v4_1 = (28x64B;16x570B;4x1518B) -| | [Tags] | IMIX | 4C | NDRDISC | SKIP_PATCH -| | ${framesize}= | Set Variable | IMIX_v4_1 -| | ${min_rate}= | Set Variable | ${50000} -| | ${max_rate}= | Calculate pps | ${s_24.5G} | ${framesize} -| | ${binary_min}= | Set Variable | ${min_rate} -| | ${binary_max}= | Set Variable | ${max_rate} -| | ${threshold}= | Set Variable | ${min_rate} -| | Given Add worker threads and rxqueues to all DUTs | 4 -| | And Add PCI devices to all DUTs -| | And Add no multi seg to all DUTs -| | And Add DPDK dev default RXD to all DUTs | 2048 -| | And Add DPDK dev default TXD to all DUTs | 2048 -| | And Apply startup configuration on all VPP DUTs -| | And Initialize IPv4 forwarding in 3-node circular topology -| | Then Find NDR using binary search and pps | ${framesize} | ${binary_min} -| | ... | ${binary_max} | ${traffic_profile} -| | ... | ${min_rate} | ${max_rate} | ${threshold} diff --git a/tests/vpp/perf/ip4/regenerate_testcases.py b/tests/vpp/perf/ip4/regenerate_testcases.py new file mode 100755 index 0000000000..5fabf8b89d --- /dev/null +++ b/tests/vpp/perf/ip4/regenerate_testcases.py @@ -0,0 +1,18 @@ +#!/usr/bin/env python + +# Copyright (c) 2018 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from resources.libraries.python.autogen.Regenerator import Regenerator + +Regenerator().regenerate_glob("*.robot") |