CSIT-678: Re-organize L2 libraries - performance

Change-Id: Ic4e37df31fbc10f15351e3671b541e6f047405bc
Signed-off-by: Tibor Frank <tifrank@cisco.com>

1 files changed, 501 insertions, 0 deletions

diff --git a/resources/libraries/robot/performance_setup.robot b/resources/libraries/robot/performance_setup.robot
new file mode 100644
index 0000000000..781c4588e4
--- /dev/null
+++ b/resources/libraries/robot/performance_setup.robot
@@ -0,0 +1,501 @@
+# Copyright (c) 2017 Cisco and/or its affiliates.
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+*** Settings ***
+| Resource | resources/libraries/robot/performance_configuration.robot
+| Resource | resources/libraries/robot/performance_utils.robot
+| Documentation | Performance suite keywords - Suite and test setups and
+| ... | teardowns.
+
+*** Keywords ***
+
+# Keywords used in setups and teardowns
+
+| Set variables in 2-node circular topology
+| | [Documentation]
+| | ... | Compute path for testing on two given nodes in circular
+| | ... | topology and set corresponding suite variables.
+| | ...
+| | ... | _NOTE:_ This KW sets following suite variables:
+| | ... | - tg - TG node
+| | ... | - tg_if1 - 1st TG interface towards DUT.
+| | ... | - tg_if2 - 2nd TG interface towards DUT.
+| | ... | - dut1 - DUT1 node
+| | ... | - dut1_if1 - 1st DUT interface towards TG.
+| | ... | - dut1_if2 - 2nd DUT interface towards TG.
+| | ...
+| | Append Nodes | ${nodes['TG']} | ${nodes['DUT1']} | ${nodes['TG']}
+| | Compute Path | always_same_link=${FALSE}
+| | ${tg_if1} | ${tg}= | First Interface
+| | ${dut1_if1} | ${dut1}= | First Ingress Interface
+| | ${dut1_if2} | ${dut1}= | Last Egress Interface
+| | ${tg_if2} | ${tg}= | Last Interface
+| | Set Suite Variable | ${tg}
+| | Set Suite Variable | ${tg_if1}
+| | Set Suite Variable | ${tg_if2}
+| | Set Suite Variable | ${dut1}
+| | Set Suite Variable | ${dut1_if1}
+| | Set Suite Variable | ${dut1_if2}
+
+| Set variables in 3-node circular topology
+| | [Documentation]
+| | ... | Compute path for testing on three given nodes in circular
+| | ... | topology and set corresponding suite variables.
+| | ...
+| | ... | _NOTE:_ This KW sets following suite variables:
+| | ... | - tg - TG node
+| | ... | - tg_if1 - TG interface towards DUT1.
+| | ... | - tg_if2 - TG interface towards DUT2.
+| | ... | - dut1 - DUT1 node
+| | ... | - dut1_if1 - DUT1 interface towards TG.
+| | ... | - dut1_if2 - DUT1 interface towards DUT2.
+| | ... | - dut2 - DUT2 node
+| | ... | - dut2_if1 - DUT2 interface towards DUT1.
+| | ... | - dut2_if2 - DUT2 interface towards TG.
+| | ...
+| | Append Nodes | ${nodes['TG']} | ${nodes['DUT1']} | ${nodes['DUT2']}
+| | ... | ${nodes['TG']}
+| | Compute Path
+| | ${tg_if1} | ${tg}= | Next Interface
+| | ${dut1_if1} | ${dut1}= | Next Interface
+| | ${dut1_if2} | ${dut1}= | Next Interface
+| | ${dut2_if1} | ${dut2}= | Next Interface
+| | ${dut2_if2} | ${dut2}= | Next Interface
+| | ${tg_if2} | ${tg}= | Next Interface
+| | Set Suite Variable | ${tg}
+| | Set Suite Variable | ${tg_if1}
+| | Set Suite Variable | ${tg_if2}
+| | Set Suite Variable | ${dut1}
+| | Set Suite Variable | ${dut1_if1}
+| | Set Suite Variable | ${dut1_if2}
+| | Set Suite Variable | ${dut2}
+| | Set Suite Variable | ${dut2_if1}
+| | Set Suite Variable | ${dut2_if2}
+
+| Set variables in 2-node circular topology with DUT interface model
+| | [Documentation]
+| | ... | Compute path for testing on two given nodes in circular topology
+| | ... | based on interface model provided as an argument and set
+| | ... | corresponding suite variables.
+| | ...
+| | ... | *Arguments:*
+| | ... | - iface_model - Interface model. Type: string
+| | ...
+| | ... | _NOTE:_ This KW sets following suite variables:
+| | ... | - tg - TG node
+| | ... | - tg_if1 - 1st TG interface towards DUT.
+| | ... | - tg_if2 - 2nd TG interface towards DUT.
+| | ... | - dut1 - DUT1 node
+| | ... | - dut1_if1 - 1st DUT interface towards TG.
+| | ... | - dut1_if2 - 2nd DUT interface towards TG.
+| | ...
+| | ... | *Example:*
+| | ...
+| | ... | \| Set variables in 2-node circular topology with DUT interface model\
+| | ... | \| Intel-X520-DA2 \|
+| | ...
+| | [Arguments] | ${iface_model}
+| | ...
+| | ${iface_model_list}= | Create list | ${iface_model}
+| | Append Node | ${nodes['TG']}
+| | Append Node | ${nodes['DUT1']} | filter_list=${iface_model_list}
+| | Append Node | ${nodes['TG']}
+| | Compute Path | always_same_link=${FALSE}
+| | ${tg_if1} | ${tg}= | First Interface
+| | ${dut1_if1} | ${dut1}= | First Ingress Interface
+| | ${dut1_if2} | ${dut1}= | Last Egress Interface
+| | ${tg_if2} | ${tg}= | Last Interface
+| | Set Suite Variable | ${tg}
+| | Set Suite Variable | ${tg_if1}
+| | Set Suite Variable | ${tg_if2}
+| | Set Suite Variable | ${dut1}
+| | Set Suite Variable | ${dut1_if1}
+| | Set Suite Variable | ${dut1_if2}
+
+| Set variables in 3-node circular topology with DUT interface model
+| | [Documentation]
+| | ... | Compute path for testing on three given nodes in circular topology
+| | ... | based on interface model provided as an argument and set
+| | ... | corresponding suite variables.
+| | ...
+| | ... | *Arguments:*
+| | ... | - iface_model - Interface model. Type: string
+| | ...
+| | ... | _NOTE:_ This KW sets following suite variables:
+| | ... | - tg - TG node
+| | ... | - tg_if1 - TG interface towards DUT1.
+| | ... | - tg_if2 - TG interface towards DUT2.
+| | ... | - dut1 - DUT1 node
+| | ... | - dut1_if1 - DUT1 interface towards TG.
+| | ... | - dut1_if2 - DUT1 interface towards DUT2.
+| | ... | - dut2 - DUT2 node
+| | ... | - dut2_if1 - DUT2 interface towards TG.
+| | ... | - dut2_if2 - DUT2 interface towards DUT1.
+| | ...
+| | ... | *Example:*
+| | ...
+| | ... | \| Set variables in 3-node circular topology with DUT interface model\
+| | ... | \| Intel-X520-DA2 \|
+| | ...
+| | [Arguments] | ${iface_model}
+| | ...
+| | ${iface_model_list}= | Create list | ${iface_model}
+| | Append Node | ${nodes['TG']}
+| | Append Node | ${nodes['DUT1']} | filter_list=${iface_model_list}
+| | Append Node | ${nodes['DUT2']} | filter_list=${iface_model_list}
+| | Append Node | ${nodes['TG']}
+| | Compute Path
+| | ${tg_if1} | ${tg}= | Next Interface
+| | ${dut1_if1} | ${dut1}= | Next Interface
+| | ${dut1_if2} | ${dut1}= | Next Interface
+| | ${dut2_if1} | ${dut2}= | Next Interface
+| | ${dut2_if2} | ${dut2}= | Next Interface
+| | ${tg_if2} | ${tg}= | Next Interface
+| | Set Suite Variable | ${tg}
+| | Set Suite Variable | ${tg_if1}
+| | Set Suite Variable | ${tg_if2}
+| | Set Suite Variable | ${dut1}
+| | Set Suite Variable | ${dut1_if1}
+| | Set Suite Variable | ${dut1_if2}
+| | Set Suite Variable | ${dut2}
+| | Set Suite Variable | ${dut2_if1}
+| | Set Suite Variable | ${dut2_if2}
+
+| Tear down guest VM with dpdk-testpmd
+| | [Documentation]
+| | ... | Stop all qemu processes with dpdk-testpmd running on ${dut_node}.
+| | ... | Argument is dictionary of all qemu nodes running with its names.
+| | ... | Dpdk-testpmd is stopped gracefully with printing stats.
+| | ...
+| | ... | *Arguments:*
+| | ... | - dut_node - Node where to clean qemu. Type: dictionary
+| | ... | - dut_vm_refs - VM references on node. Type: dictionary
+| | ...
+| | ... | *Example:*
+| | ...
+| | ... | \| Tear down guest VM with dpdk-testpmd \| ${node['DUT1']} \
+| | ... | \| ${dut_vm_refs} \|
+| | ...
+| | [Arguments] | ${dut_node} | ${dut_vm_refs}
+| | ${vms_number}= | Get Length | ${dut_vm_refs}
+| | ${index}= | Set Variable | ${0}
+| | :FOR | ${vm_name} | IN | @{dut_vm_refs}
+| | | ${vm}= | Get From Dictionary | ${dut_vm_refs} | ${vm_name}
+| | | ${index}= | Evaluate | ${index} + 1
+| | | Dpdk Testpmd Stop | ${vm}
+| | | Run Keyword | ${vm_name}.Qemu Set Node | ${dut_node}
+| | | Run Keyword | ${vm_name}.Qemu Clear Socks
+| | | Run Keyword If | '${index}' == '${vms_number}' | ${vm_name}.Qemu Kill All
+
+| Tear down guest VM
+| | [Documentation]
+| | ... | Stop all qemu processes running on ${dut_node}.
+| | ... | Argument is dictionary of all qemu nodes running with its names.
+| | ...
+| | ... | *Arguments:*
+| | ... | - dut_node - Node where to clean qemu. Type: dictionary
+| | ... | - dut_vm_refs - VM references on node. Type: dictionary
+| | ...
+| | ... | *Example:*
+| | ...
+| | ... | \| Tear down guest VM \| ${node['DUT1']} \
+| | ... | \| ${dut_vm_refs} \|
+| | ...
+| | [Arguments] | ${dut_node} | ${dut_vm_refs}
+| | ${vms_number}= | Get Length | ${dut_vm_refs}
+| | ${index}= | Set Variable | ${0}
+| | :FOR | ${vm_name} | IN | @{dut_vm_refs}
+| | | ${vm}= | Get From Dictionary | ${dut_vm_refs} | ${vm_name}
+| | | ${index}= | Evaluate | ${index} + 1
+| | | Run Keyword | ${vm_name}.Qemu Set Node | ${dut_node}
+| | | Run Keyword | ${vm_name}.Qemu Clear Socks
+| | | Run Keyword If | '${index}' == '${vms_number}' | ${vm_name}.Qemu Kill All
+
+# Suite setups
+
+| Set up 2-node performance topology with DUT's NIC model
+| | [Documentation]
+| | ... | Suite preparation phase that setup default startup configuration of
+| | ... | VPP on all DUTs. Updates interfaces on all nodes and setup global
+| | ... | variables used in test cases based on interface model provided as an
+| | ... | argument. Initializes traffic generator.
+| | ...
+| | ... | *Arguments:*
+| | ... | - topology_type - Topology type. Type: string
+| | ... | - nic_model - Interface model. Type: string
+| | ...
+| | ... | *Example:*
+| | ...
+| | ... | \| 2-node Performance Suite Setup \| L2 \| Intel-X520-DA2 \|
+| | ...
+| | [Arguments] | ${topology_type} | ${nic_model}
+| | ...
+| | Show vpp version on all DUTs
+| | Set variables in 2-node circular topology with DUT interface model
+| | ... | ${nic_model}
+| | Initialize traffic generator | ${tg} | ${tg_if1} | ${tg_if2}
+| | ... | ${dut1} | ${dut1_if1} | ${dut1} | ${dut1_if2} | ${topology_type}
+
+| Set up 2-node-switched performance topology with DUT's NIC model
+| | [Documentation]
+| | ... | Suite preparation phase that setup default startup configuration of
+| | ... | VPP on all DUTs. Updates interfaces on all nodes and setup global
+| | ... | variables used in test cases based on interface model provided as an
+| | ... | argument. Initializes traffic generator.
+| | ...
+| | ... | *Arguments:*
+| | ... | - topology_type - Topology type. Type: string
+| | ... | - nic_model - Interface model. Type: string
+| | ... | - tg_if1_dest_mac - Interface 1 destination MAC address. Type: string
+| | ... | - tg_if2_dest_mac - Interface 2 destination MAC address. Type: string
+| | ...
+| | ... | *Example:*
+| | ...
+| | ... | \| 2-node Performance Suite Setup \| L2 \| Intel-X520-DA2 \
+| | ... | \| 22:22:33:44:55:66 \| 22:22:33:44:55:55 \|
+| | ...
+| | [Arguments] | ${topology_type} | ${nic_model} | ${tg_if1_dest_mac}
+| | ... | ${tg_if2_dest_mac}
+| | ...
+| | Show vpp version on all DUTs
+| | Set variables in 2-node circular topology with DUT interface model
+| | ... | ${nic_model}
+| | Initialize traffic generator | ${tg} | ${tg_if1} | ${tg_if2}
+| | ... | ${dut1} | ${dut1_if1} | ${dut1} | ${dut1_if2} | ${topology_type}
+| | ... | ${tg_if1_dest_mac} | ${tg_if2_dest_mac}
+
+| Set up 3-node performance topology with DUT's NIC model
+| | [Documentation]
+| | ... | Suite preparation phase that setup default startup configuration of
+| | ... | VPP on all DUTs. Updates interfaces on all nodes and setup global
+| | ... | variables used in test cases based on interface model provided as an
+| | ... | argument. Initializes traffic generator.
+| | ...
+| | ... | *Arguments:*
+| | ... | - topology_type - Topology type. Type: string
+| | ... | - nic_model - Interface model. Type: string
+| | ...
+| | ... | *Example:*
+| | ...
+| | ... | \| Set up 3-node performance topology with DUT's NIC model \| L2 \
+| | ... | \| Intel-X520-DA2 \|
+| | ...
+| | [Arguments] | ${topology_type} | ${nic_model}
+| | ...
+| | Show vpp version on all DUTs
+| | Set variables in 3-node circular topology with DUT interface model
+| | ... | ${nic_model}
+| | Initialize traffic generator | ${tg} | ${tg_if1} | ${tg_if2}
+| | ... | ${dut1} | ${dut1_if1} | ${dut2} | ${dut2_if2} | ${topology_type}
+
+| Set up DPDK 2-node performance topology with DUT's NIC model
+| | [Documentation]
+| | ... | Updates interfaces on all nodes and setup global
+| | ... | variables used in test cases based on interface model provided as an
+| | ... | argument. Initializes traffic generator. Initializes DPDK test
+| | ... | environment.
+| | ...
+| | ... | *Arguments:*
+| | ... | - topology_type - Topology type. Type: string
+| | ... | - nic_model - Interface model. Type: string
+| | ...
+| | ... | *Example:*
+| | ...
+| | ... | \| Set up DPDK 2-node performance topology with DUT's NIC model \
+| | ... | \| L2 \| Intel-X520-DA2 \|
+| | ...
+| | [Arguments] | ${topology_type} | ${nic_model}
+| | ...
+| | Set variables in 2-node circular topology with DUT interface model
+| | ... | ${nic_model}
+| | Initialize traffic generator | ${tg} | ${tg_if1} | ${tg_if2}
+| | ... | ${dut1} | ${dut1_if1} | ${dut1} | ${dut1_if2} | ${topology_type}
+| | Initialize DPDK Environment | ${dut1} | ${dut1_if1} | ${dut1_if2}
+
+| Set up DPDK 3-node performance topology with DUT's NIC model
+| | [Documentation]
+| | ... | Updates interfaces on all nodes and setup global
+| | ... | variables used in test cases based on interface model provided as an
+| | ... | argument. Initializes traffic generator. Initializes DPDK test
+| | ... | environment.
+| | ...
+| | ... | *Arguments:*
+| | ... | - topology_type - Topology type. Type: string
+| | ... | - nic_model - Interface model. Type: string
+| | ...
+| | ... | *Example:*
+| | ...
+| | ... | \| 3-node Performance Suite Setup \| L2 \| Intel-X520-DA2 \|
+| | ...
+| | [Arguments] | ${topology_type} | ${nic_model}
+| | ...
+| | Set variables in 3-node circular topology with DUT interface model
+| | ... | ${nic_model}
+| | Initialize traffic generator | ${tg} | ${tg_if1} | ${tg_if2}
+| | ... | ${dut1} | ${dut1_if1} | ${dut2} | ${dut2_if2} | ${topology_type}
+| | Initialize DPDK Environment | ${dut1} | ${dut1_if1} | ${dut1_if2}
+| | Initialize DPDK Environment | ${dut2} | ${dut2_if1} | ${dut2_if2}
+
+| Set up IPSec performance test suite
+| | [Documentation]
+| | ... | Suite preparation phase that sets default startup configuration of
+| | ... | VPP on all DUTs. Updates interfaces on all nodes and sets global
+| | ... | variables used in test cases based on interface model provided as an
+| | ... | argument. Initializes traffic generator.
+| | ... | Then it configures crypto device and kernel module on all DUTs.
+| | ...
+| | ... | *Arguments:*
+| | ... | - topology_type - Topology type. Type: string
+| | ... | - nic_model - Interface model. Type: string
+| | ...
+| | ... | *Example:*
+| | ...
+| | ... | \| Set up IPSec performance test suite \| L2 \
+| | ... | \| Intel-X520-DA2 \|
+| | ...
+| | [Arguments] | ${topology_type} | ${nic_model}
+| | ...
+| | Set up 3-node performance topology with DUT's NIC model
+| | ... | ${topology_type} | ${nic_model}
+| | Configure crypto device on all DUTs | force_init=${True}
+| | Configure kernel module on all DUTs | igb_uio | force_load=${True}
+
+# Suite teardowns
+
+| Tear down 3-node performance topology
+| | [Documentation]
+| | ... | Suite teardown phase with traffic generator teardown.
+| | ...
+| | Teardown traffic generator | ${tg}
+
+| Tear down 2-node performance topology
+| | [Documentation]
+| | ... | Suite teardown phase with traffic generator teardown.
+| | ...
+| | Teardown traffic generator | ${tg}
+
+# Tests setups
+
+| Set up performance test
+| | [Documentation] | Common test setup for performance tests.
+| | ...
+| | Reset VAT History On All DUTs | ${nodes}
+| | Create base startup configuration of VPP on all DUTs
+
+# Tests teardowns
+
+| Tear down performance discovery test
+| | [Documentation] | Common test teardown for ndrdisc and pdrdisc performance \
+| | ... | tests.
+| | ...
+| | ... | *Arguments:*
+| | ... | - rate - Rate for sending packets. Type: string
+| | ... | - framesize - L2 Frame Size [B]. Type: integer
+| | ... | - topology_type - Topology type. Type: string
+| | ...
+| | ... | *Example:*
+| | ...
+| | ... | \| Tear down performance discovery test \| 4.0mpps \| 64 \
+| | ... | \| 3-node-IPv4 \|
+| | ...
+| | [Arguments] | ${rate} | ${framesize} | ${topology_type}
+| | ...
+| | Show VAT History On All DUTs | ${nodes}
+| | Show statistics on all DUTs
+| | Run Keyword If Test Failed
+| | ... | Traffic should pass with no loss | ${perf_trial_duration} | ${rate}
+| | ... | ${framesize} | ${topology_type} | fail_on_loss=${False}
+
+| Tear down performance ndrchk test
+| | [Documentation] | Common test teardown for ndrchk performance tests.
+| | ...
+| | Show VAT History On All DUTs | ${nodes}
+| | Show statistics on all DUTs
+
+| Tear down performance pdrchk test
+| | [Documentation] | Common test teardown for pdrchk performance tests.
+| | ...
+| | Show VAT History On All DUTs | ${nodes}
+| | Show statistics on all DUTs
+
+| Tear down performance test with vhost and VM with dpdk-testpmd
+| | [Documentation] | Common test teardown for performance tests which use
+| | ... | vhost(s) and VM(s) with dpdk-testpmd.
+| | ...
+| | ... | *Arguments:*
+| | ... | - rate - Rate for sending packets. Type: string
+| | ... | - framesize - L2 Frame Size [B]. Type: integer
+| | ... | - topology_type - Topology type. Type: string
+| | ... | - dut1_node - Node where to clean qemu. Type: dictionary
+| | ... | - dut1_vm_refs - VM references on node. Type: dictionary
+| | ... | - dut2_node - Node where to clean qemu. Type: dictionary
+| | ... | - dut2_vm_refs - VM references on node. Type: dictionary
+| | ...
+| | ... | *Example:*
+| | ...
+| | ... | \| Tear down performance test with vhost and VM with dpdk-testpmd \
+| | ... | \| 4.0mpps \| 64 \| 3-node-IPv4 \| ${node['DUT1']} \| ${dut_vm_refs} \
+| | ... | \| ${node['DUT2']} \| ${dut_vm_refs} \|
+| | ...
+| | [Arguments] | ${rate} | ${framesize} | ${topology_type}
+| | ... | ${dut1_node}=${None} | ${dut1_vm_refs}=${None}
+| | ... | ${dut2_node}=${None} | ${dut2_vm_refs}=${None}
+| | ...
+| | Show VAT History On All DUTs | ${nodes}
+| | Show VPP vhost on all DUTs
+| | Show statistics on all DUTs
+| | Run Keyword If Test Failed
+| | ... | Traffic should pass with no loss | ${perf_trial_duration} | ${rate}
+| | ... | ${framesize} | ${topology_type} | fail_on_loss=${False}
+| | Run keyword unless | ${dut1_node}==${None}
+| | ... | Tear down guest VM with dpdk-testpmd | ${dut1} | ${dut1_vm_refs}
+| | Run keyword unless | ${dut2_node}==${None}
+| | ... | Tear down guest VM with dpdk-testpmd | ${dut2} | ${dut2_vm_refs}
+
+| Tear down DPDK 3-node performance topology
+| | [Documentation]
+| | ... | Suite teardown phase with traffic generator teardown.
+| | ... | Cleanup DPDK test environment.
+| | ...
+| | Teardown traffic generator | ${tg}
+| | Cleanup DPDK Environment | ${dut1} | ${dut1_if1} | ${dut1_if2}
+| | Cleanup DPDK Environment | ${dut2} | ${dut2_if1} | ${dut2_if2}
+
+| Tear down DPDK 2-node performance topology
+| | [Documentation]
+| | ... | Suite teardown phase with traffic generator teardown.
+| | ... | Cleanup DPDK test environment.
+| | ...
+| | Teardown traffic generator | ${tg}
+| | Cleanup DPDK Environment | ${dut1} | ${dut1_if1} | ${dut1_if2}
+
+| Tear down performance discovery test with SNAT
+| | [Documentation] | Common test teardown for ndrdisc and pdrdisc performance \
+| | ... | tests with SNAT feature used.
+| | ...
+| | ... | *Arguments:*
+| | ... | - rate - Rate for sending packets. Type: string
+| | ... | - framesize - L2 Frame Size [B]. Type: integer
+| | ... | - topology_type - Topology type. Type: string
+| | ...
+| | ... | *Example:*
+| | ...
+| | ... | \| Tear down performance discovery test with SNAT \| 4.0mpps \| 64 \
+| | ... | \| ${traffic_profile} \|
+| | ...
+| | [Arguments] | ${rate} | ${framesize} | ${traffic_profile}
+| | ...
+| | Tear down performance discovery test | ${rate}pps | ${framesize}
+| | ... | ${traffic_profile}
+| | Show SNAT verbose | ${dut1}
+| | Show SNAT verbose | ${dut2}