aboutsummaryrefslogtreecommitdiffstats
path: root/tests/vpp/perf/ip4/2n1l-10ge2p1x710-ethip4-ip4scale2m-ndrpdr.robot
blob: 8a25d47202ce0aa688d42747ffcf9c7cf9018aaa (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
# Copyright (c) 2019 Cisco and/or its affiliates.
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at:
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

*** Settings ***
| Resource | resources/libraries/robot/shared/default.robot
| ...
| Force Tags | 2_NODE_SINGLE_LINK_TOPO | PERFTEST | HW_ENV | NDRPDR
| ... | NIC_Intel-X710 | ETH | IP4FWD | SCALE | FIB_2M
| ...
| Suite Setup | Setup suite single link | performance
| Suite Teardown | Tear down suite | performance
| Test Setup | Setup test
| Test Teardown | Tear down test | performance
| ...
| Test Template | Local Template
| ...
| Documentation | *RFC2544: Pkt throughput IPv4 routing test cases*
| ...
| ... | *[Top] Network Topologies:* TG-DUT1-TG 2-node circular topology\
| ... | with single links between nodes.
| ... | *[Enc] Packet Encapsulations:* Eth-IPv4 for IPv4 routing.
| ... | *[Cfg] DUT configuration:* DUT1 is configured with IPv4 routing and\
| ... | 2x1M static IPv4 /32 route entries. DUT1 is tested with ${nic_name}.\
| ... | *[Ver] TG verification:* TG finds and reports throughput NDR (Non Drop\
| ... | Rate) with zero packet loss tolerance and throughput PDR (Partial Drop\
| ... | Rate) with non-zero packet loss tolerance (LT) expressed in percentage\
| ... | of packets transmitted. NDR and PDR are discovered for different\
| ... | Ethernet L2 frame sizes using MLRsearch library.\
| ... | Test packets are generated by TG on links to DUT1. TG traffic profile\
| ... | contains two L3 flow-groups (flow-group per direction, 1M flows per\
| ... | flow-group) with all packets containing Ethernet header, IPv4 header\
| ... | with IP protocol=61 and static payload. MAC addresses are matching MAC\
| ... | addresses of the TG node interfaces.
| ... | *[Ref] Applicable standard specifications:* RFC2544.

*** Variables ***
| @{plugins_to_enable}= | dpdk_plugin.so
| ${osi_layer}= | L3
| ${nic_name}= | Intel-X710
| ${nic_driver}= | vfio-pci
| ${overhead}= | ${0}
| ${rts_per_flow}= | ${1000000}
# Traffic profile:
| ${traffic_profile}= | trex-sl-2n-ethip4-ip4dst${rts_per_flow}

*** Keywords ***
| Local Template
| | ...
| | [Documentation]
| | ... | [Cfg] DUT runs IPv4 routing config.
| | ... | Each DUT uses ${phy_cores} physical core(s) for worker threads.
| | ... | [Ver] Measure NDR and PDR values using MLRsearch algorithm.\
| | ...
| | ... | *Arguments:*
| | ... | - frame_size - Framesize in Bytes in integer or string (IMIX_v4_1).
| | ... | Type: integer, string
| | ... | - phy_cores - Number of physical cores. Type: integer
| | ... | - rxq - Number of RX queues, default value: ${None}. Type: integer
| | ...
| | [Arguments] | ${frame_size} | ${phy_cores} | ${rxq}=${None}
| | ...
| | Set Test Variable | \${frame_size}
| | ...
| | Given Add worker threads and rxqueues to all DUTs | ${phy_cores} | ${rxq}
| | And Add PCI devices to all DUTs
| | And Set Max Rate And Jumbo And Handle Multi Seg
| | And Apply startup configuration on all VPP DUTs
| | When Initialize layer driver | ${nic_driver}
| | And Initialize IPv4 forwarding with scaling in circular topology
| | ... | ${rts_per_flow}
| | Then Find NDR and PDR intervals using optimized search

*** Test Cases ***
| tc01-64B-1c-ethip4-ip4scale2m-ndrpdr
| | [Tags] | 64B | 1C
| | frame_size=${64} | phy_cores=${1}

| tc02-64B-2c-ethip4-ip4scale2m-ndrpdr
| | [Tags] | 64B | 2C
| | frame_size=${64} | phy_cores=${2}

| tc03-64B-4c-ethip4-ip4scale2m-ndrpdr
| | [Tags] | 64B | 4C
| | frame_size=${64} | phy_cores=${4}

| tc04-1518B-1c-ethip4-ip4scale2m-ndrpdr
| | [Tags] | 1518B | 1C
| | frame_size=${1518} | phy_cores=${1}

| tc05-1518B-2c-ethip4-ip4scale2m-ndrpdr
| | [Tags] | 1518B | 2C
| | frame_size=${1518} | phy_cores=${2}

| tc06-1518B-4c-ethip4-ip4scale2m-ndrpdr
| | [Tags] | 1518B | 4C
| | frame_size=${1518} | phy_cores=${4}

| tc07-9000B-1c-ethip4-ip4scale2m-ndrpdr
| | [Tags] | 9000B | 1C
| | frame_size=${9000} | phy_cores=${1}

| tc08-9000B-2c-ethip4-ip4scale2m-ndrpdr
| | [Tags] | 9000B | 2C
| | frame_size=${9000} | phy_cores=${2}

| tc09-9000B-4c-ethip4-ip4scale2m-ndrpdr
| | [Tags] | 9000B | 4C
| | frame_size=${9000} | phy_cores=${4}

| tc10-IMIX-1c-ethip4-ip4scale2m-ndrpdr
| | [Tags] | IMIX | 1C
| | frame_size=IMIX_v4_1 | phy_cores=${1}

| tc11-IMIX-2c-ethip4-ip4scale2m-ndrpdr
| | [Tags] | IMIX | 2C
| | frame_size=IMIX_v4_1 | phy_cores=${2}

| tc12-IMIX-4c-ethip4-ip4scale2m-ndrpdr
| | [Tags] | IMIX | 4C
| | frame_size=IMIX_v4_1 | phy_cores=${4}
s). #. When CSIT committer is satisfied, the activating change is merged to CSIT master branch and cherry-picked to the latest oper branch. This enters a "critical section" of the process. Merges of other activating changes are not allowed from now on. The targeted API change becomes the active API change. This does not break any jobs. #. VPP developer (or CSIT committer) issues a recheck on the VPP patch. #. On failure, VPP and CSIT committers analyze what went wrong. Typically, the active CRC collection is matching only an older patch set, but a newer patch set needs different CRC values. Either due to improvements on the VPP change in question, or due to a rebase over previously merged (unrelated) API change. VPP perhaps needs to rebase, and CSIT definitely needs to merge edits to the active collection. Then issue a recheck again, and iterate until success. #. On success, VPP Committer merges the active API change patch. (This is also a delayed verification of the current active CRC collection.) #. VPP committer sends an e-mail to vpp-api-dev stating the support for the previous CRC values will soon be removed, implying other changes (whether API or not) should be rebased soon. #. VPP merge jobs create and upload new VPP packages. This breaks trending jobs, but both VPP and CSIT verify jobs still work. #. CSIT developer makes the deactivating change mergeable: The stable VPP build indicator is bumped to the build that contains the active API change. The active CRC collection (added by the activating change) is renamed to the new stable collection. (The previous stable collection has already been deleted.) At this time, the deactivating change should be uploaded to Gerrit and csit verify jobs should be triggered. #. CSIT committer reviews the code, perhaps triggering any additional jobs needed to verify the tests using the edited APIs are still working. #. When satisfied, CSIT committer merges the mergeable deactivating change (to both master and oper). The merge fixes trending jobs. VPP and CSIT verify jobs continue to work. The merge also breaks some verify jobs for old changes in VPP, as announced when the active API change was merged. The merge is the point where the process leaves the "critical section", thus allowing merges of activating changes for other API changes. #. CSIT committer sends an e-mail to vpp-api-dev stating the support for the previous CRC values has been removed, and rebase is needed for all affected VPP changes. #. Recheck of existing VPP patches in gerrit may cause the "VPP API Incompatible Change Test" to send an email to the patch submitter to rebase the patch to pick up the compatible VPP API version files. ### Real life examples Simple API change: https://gerrit.fd.io/r/c/vpp/+/23829 Activating change: https://gerrit.fd.io/r/c/csit/+/23956 Mergeable deactivating change: https://gerrit.fd.io/r/c/csit/+/24280 Less straightforward mergeable deactivating change: https://gerrit.fd.io/r/c/csit/+/22526 It shows: + Crc edits: supported_crcs.yaml + Version bump: VPP_STABLE_VER_UBUNTU_BIONIC + And even a way to work around failing tests: eth2p-ethicmpv4-ip4base-eth-1tap-dev.robot Simple change that is both deactivating and activating: https://gerrit.fd.io/r/c/csit/+/23969