aboutsummaryrefslogtreecommitdiffstats
path: root/test/sanity_run_vpp.py
AgeCommit message (Expand)AuthorFilesLines
2022-05-10tests: replace pycodestyle with blackKlement Sekera1-4/+5
2022-02-17tests: make tests less make dependentKlement Sekera1-2/+5
2021-04-16tests: cpus awarenessKlement Sekera1-1/+1
2021-04-12tests: support attaching to existing vppKlement Sekera1-2/+2
2019-12-20tests: don't prompt to launch gdb for sanity test casePaul Vinciguerra1-7/+18
2019-11-05misc: Fix python scripts shebang lineRenato Botelho do Couto1-1/+1
2019-11-05tests: support setting random seedKlement Sekera1-0/+3
2019-06-24tests: refactor VppDiedError.Paul Vinciguerra1-2/+1
2018-12-11Fix typo in sanity test descriptionjuraj.linkes1-1/+1
2018-11-26Split non-parallel testsuitejuraj.linkes1-1/+6
2018-11-08make test: rename dummy test nameKlement Sekera1-1/+1
2017-08-10make test: detect hung testsKlement Sekera1-2/+8
2017-04-10make test: automatic "vpp finishes startup" checkKlement Sekera1-0/+26
href='#n25'>25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 
# Copyright (c) 2023 Cisco and/or its affiliates.
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at:
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

*** Settings ***
| Resource | resources/libraries/robot/shared/default.robot
|
| Force Tags | 2_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR
| ... | NIC_Intel-X710 | ETH | IP4FWD | FEATURE | NAT44 | UDP | UDP_UDIR
| ... | NAT44_ENDPOINT_DEPENDENT | SCALE | HOSTS_1024 | DRV_VFIO_PCI
| ... | RXQ_SIZE_0 | TXQ_SIZE_0
| ... | ethip4udp-nat44ed-h1024-p63-s64512-udir
|
| Suite Setup | Setup suite topology interfaces | performance
| Suite Teardown | Tear down suite | performance
| Test Setup | Setup test | performance
| Test Teardown | Tear down test | performance | nat-ed
|
| Test Template | Local Template
|
| Documentation | **RFC2544: Pkt throughput NAT44 endpoint-dependent mode
| ... | performance test cases**
| ... |
| ... | - **[Top] Network Topologies:** TG-DUT1-TG 2-node circular topology \
| ... | with single links between nodes.
| ... |
| ... | - **[Enc] Packet Encapsulations:** Eth-IPv4-UDP for IPv4 routing.
| ... |
| ... | - **[Cfg] DUT configuration:** DUT1 is configured with IPv4 routing \
| ... | and one static IPv4 /${dest_mask} route entries. \
| ... | DUT1 is tested with ${nic_name}.
| ... |
| ... | - **[Ver] TG verification:** TG finds and reports throughput NDR (Non \
| ... | Drop Rate) with zero packet loss tolerance and throughput PDR \
| ... | (Partial Drop Rate) with non-zero packet loss tolerance (LT) \
| ... | expressed in percentage of packets transmitted. NDR and PDR are \
| ... | discovered for different Ethernet L2 frame sizes using MLRsearch \
| ... | library.
| ... | Test packets are generated by TG on links to DUTs. TG traffic profile \
| ... | contain L4 flows (uni-directional UDP flows) with all packets \
| ... | containing Ethernet header, IPv4 header with UDP header and static \
| ... | payload. MAC addresses are matching MAC addresses of the TG node \
| ... | interfaces.
| ... |
| ... | - **[Ref] Applicable standard specifications:** Benchmarking \
| ... | Methodology for Network Security Device Performance \
| ... | (draft-ietf-bmwg-ngfw-performance-03).

*** Variables ***
| @{plugins_to_enable}= | dpdk_plugin.so | perfmon_plugin.so | nat_plugin.so
| ${crypto_type}= | ${None}
| ${nic_name}= | Intel-X710
| ${nic_driver}= | vfio-pci
| ${nic_rxq_size}= | 0
| ${nic_txq_size}= | 0
| ${nic_pfs}= | 2
| ${nic_vfs}= | 0
| ${osi_layer}= | L3
| ${overhead}= | ${0}
# IP settings
| ${tg_if1_ip4}= | 10.0.0.2
| ${tg_if1_mask}= | ${20}
| ${tg_if2_ip4}= | 12.0.0.2
| ${tg_if2_mask}= | ${20}
| ${dut1_if1_ip4}= | 10.0.0.1
| ${dut1_if1_mask}= | ${24}
| ${dut1_if2_ip4}= | 12.0.0.1
| ${dut1_if2_mask}= | ${24}
| ${dest_net}= | 20.0.0.0
| ${dest_mask}= | ${8}
# NAT settings
| ${nat_mode}= | endpoint-dependent
| ${in_net}= | 192.168.0.0
| ${in_mask}= | ${22}
| ${out_net}= | 68.142.68.0
| ${out_net_end}= | 68.142.68.0
| ${out_mask}= | ${32}
# Scale settings
| ${n_hosts}= | ${1024}
| ${n_ports}= | ${63}
| ${n_sessions}= | ${${n_hosts} * ${n_ports}}
# Ramp-up settings
| ${ramp_up_rate}= | ${500000}
| ${ramp_up_duration}= | ${1}
# Traffic profile
| ${traffic_profile}= | trex-stl-ethip4udp-${n_hosts}u${n_ports}p-udir
| ${traffic_directions}= | ${1}

*** Keywords ***
| Local Template
| |
| | [Documentation]
| | ... | - **[Cfg]** DUT runs NAT44 ${nat_mode} configuration. \
| | ... | Each DUT uses ${phy_cores} physical core(s) for worker threads.
| | ... | - **[Ver]** Measure NDR and PDR values using MLRsearch algorithm.
| |
| | ... | *Arguments:*
| | ... | - frame_size - Framesize in Bytes in integer or string (IMIX_v4_1).
| | ... | Type: integer, string
| | ... | - phy_cores - Number of physical cores. Type: integer
| | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer
| |
| | [Arguments] | ${frame_size} | ${phy_cores} | ${rxq}=${None}
| |
| | Set Test Variable | \${frame_size}
| |
| | Given Set Max Rate And Jumbo
| | And Add worker threads to all DUTs | ${phy_cores} | ${rxq}
| | And Pre-initialize layer driver | ${nic_driver}
| | And Apply startup configuration on all VPP DUTs
| | When Initialize layer driver | ${nic_driver}
| | And Initialize layer interface
| | And Initialize IPv4 forwarding for NAT44 in circular topology
| | And Initialize NAT44 endpoint-dependent mode in circular topology
| | Then Send ramp-up traffic
| | And Verify NAT44 UDP sessions number on DUT1 node
| | And Find NDR and PDR intervals using optimized search

*** Test Cases ***
| 64B-1c-ethip4udp-nat44ed-h1024-p63-s64512-udir-ndrpdr
| | [Tags] | 64B | 1C
| | frame_size=${64} | phy_cores=${1}

| 64B-2c-ethip4udp-nat44ed-h1024-p63-s64512-udir-ndrpdr
| | [Tags] | 64B | 2C
| | frame_size=${64} | phy_cores=${2}

| 64B-4c-ethip4udp-nat44ed-h1024-p63-s64512-udir-ndrpdr
| | [Tags] | 64B | 4C
| | frame_size=${64} | phy_cores=${4}

| 1518B-1c-ethip4udp-nat44ed-h1024-p63-s64512-udir-ndrpdr
| | [Tags] | 1518B | 1C
| | frame_size=${1518} | phy_cores=${1}

| 1518B-2c-ethip4udp-nat44ed-h1024-p63-s64512-udir-ndrpdr
| | [Tags] | 1518B | 2C
| | frame_size=${1518} | phy_cores=${2}

| 1518B-4c-ethip4udp-nat44ed-h1024-p63-s64512-udir-ndrpdr
| | [Tags] | 1518B | 4C
| | frame_size=${1518} | phy_cores=${4}

| 9000B-1c-ethip4udp-nat44ed-h1024-p63-s64512-udir-ndrpdr
| | [Tags] | 9000B | 1C
| | frame_size=${9000} | phy_cores=${1}

| 9000B-2c-ethip4udp-nat44ed-h1024-p63-s64512-udir-ndrpdr
| | [Tags] | 9000B | 2C
| | frame_size=${9000} | phy_cores=${2}

| 9000B-4c-ethip4udp-nat44ed-h1024-p63-s64512-udir-ndrpdr
| | [Tags] | 9000B | 4C
| | frame_size=${9000} | phy_cores=${4}

| IMIX-1c-ethip4udp-nat44ed-h1024-p63-s64512-udir-ndrpdr
| | [Tags] | IMIX | 1C
| | frame_size=IMIX_v4_1 | phy_cores=${1}

| IMIX-2c-ethip4udp-nat44ed-h1024-p63-s64512-udir-ndrpdr
| | [Tags] | IMIX | 2C
| | frame_size=IMIX_v4_1 | phy_cores=${2}

| IMIX-4c-ethip4udp-nat44ed-h1024-p63-s64512-udir-ndrpdr
| | [Tags] | IMIX | 4C
| | frame_size=IMIX_v4_1 | phy_cores=${4}