Align suite/test teardown/setup

+ Phase I - aligning FUNC/PERF

Change-Id: I46b1e43687ddb29f5ed3c6335fe1baf21ec02822
Signed-off-by: Peter Mikus <pmikus@cisco.com>

4 files changed, 230 insertions, 34 deletions

diff --git a/resources/libraries/robot/shared/default.robot b/resources/libraries/robot/shared/default.robot
index 262dc02f55..b7e4a75773 100644
--- a/resources/libraries/robot/shared/default.robot
+++ b/resources/libraries/robot/shared/default.robot
@@ -34,7 +34,10 @@
 | Library | resources.libraries.python.topology.Topology
 | ...
 | Resource | resources/libraries/robot/shared/container.robot
-| Resource | resources/libraries/robot/vm/qemu.robot
+| Resource | resources/libraries/robot/shared/qemu.robot
+| Resource | resources/libraries/robot/shared/suite_teardown.robot
+| Resource | resources/libraries/robot/shared/test_teardown.robot
+
 
 *** Keywords ***
 | Configure all TGs for traffic script
@@ -421,6 +424,7 @@
 | | ${duts}= | Get Matches | ${nodes} | DUT*
 | | :FOR | ${dut} | IN | @{duts}
 | | | Run keyword | ${dut}.Apply Config
+| | Save VPP PIDs
 | | Enable Coredump Limit VPP on All DUTs | ${nodes}
 | | Update All Interface Data On All Nodes | ${nodes} | skip_tg=${True}
 
@@ -434,8 +438,6 @@
 | | :FOR | ${key} | IN | @{keys}
 | | | ${pid}= | Get From Dictionary | ${setup_vpp_pids} | ${key}
 | | | Run Keyword If | $pid is None | FAIL | No VPP PID found on node ${key}
-| | | Run Keyword If | ',' in '${pid}'
-| | | ... | FAIL | More then one VPP PID found on node ${key}: ${pid}
 | | Set Test Variable | ${setup_vpp_pids}
 
 | Verify VPP PID in Teardown
@@ -486,17 +488,6 @@
 | | Update All Interface Data On All Nodes | ${nodes} | skip_tg_udev=${True}
 | | Reset PAPI History On All DUTs | ${nodes}
 
-| Tear down VPP device test
-# TODO: Generalize this KW if it will not diverge from Functional derivate too
-# much
-| | [Documentation] | Common test teardown for vpp-device tests.
-| | ...
-| | Remove All Added Ports On All DUTs From Topology | ${nodes}
-| | Show Packet Trace on All DUTs | ${nodes}
-| | Show PAPI History On All DUTs | ${nodes}
-| | Vpp Show Errors On All DUTs | ${nodes}
-| | Verify VPP PID in Teardown
-
 | Tear down LISP functional test
 | | [Documentation] | Common test teardown for functional tests with LISP.
 | | ...
@@ -508,19 +499,6 @@
 | | Vpp Show Errors On All DUTs | ${nodes}
 | | Verify VPP PID in Teardown
 
-| Tear down LISP functional test with QEMU
-| | [Documentation] | Common test teardown for functional tests with LISP and\
-| | ... | QEMU.
-| | ...
-| | Remove All Added Ports On All DUTs From Topology | ${nodes}
-| | Show Packet Trace on All DUTs | ${nodes}
-| | Show PAPI History On All DUTs | ${nodes}
-| | Show Vpp Settings | ${nodes['DUT1']}
-| | Show Vpp Settings | ${nodes['DUT2']}
-| | Vpp Show Errors On All DUTs | ${nodes}
-| | Tear down QEMU
-| | Verify VPP PID in Teardown
-
 | Set up functional test with containers
 | | [Documentation]
 | | ... | Common test setup for functional tests with containers.
@@ -560,13 +538,6 @@
 | | Start VPP in all '${container_group}' containers
 | | Append To List | ${container_groups} | ${container_group}
 
-| Tear down functional test with container
-| | [Documentation]
-| | ... | Common test teardown for functional tests which uses containers.
-| | ...
-| | :FOR | ${container_group} | IN | @{container_groups}
-| | | Destroy all '${container_group}' containers
-
 | Stop VPP Service on DUT
 | | [Documentation] | Stop the VPP service on the specified node.
 | | ...
diff --git a/resources/libraries/robot/shared/qemu.robot b/resources/libraries/robot/shared/qemu.robot
new file mode 100644
index 0000000000..1178fa93b4
--- /dev/null
+++ b/resources/libraries/robot/shared/qemu.robot
@@ -0,0 +1,51 @@
+# Copyright (c) 2019 Cisco and/or its affiliates.
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+*** Settings ***
+| Library | resources.libraries.python.L2Util
+| Library | resources.libraries.python.InterfaceUtil
+
+*** Keywords ***
+| Configure VM for vhost L2BD forwarding
+| | [Documentation] | Setup QEMU and start VM with two vhost interfaces.
+| | ...
+| | ... | *Arguments:*
+| | ... | - ${dut_node} - DUT node to start VM on. Type: dictionary
+| | ... | - ${sock1} - Socket path for first Vhost-User interface. Type: string
+| | ... | - ${sock2} - Socket path for second Vhost-User interface. Type: string
+| | ... | - ${qemu_name} - Qemu instance name by which the object will be
+| | ... | accessed (Optional). Type: string
+| | ...
+| | ... | _NOTE:_ This KW sets following test case variable:
+| | ... | - ${${qemu_name}} - VM node info. Type: dictionary
+| | ...
+| | ... | *Example:*
+| | ...
+| | ... | \| Configure VM for vhost L2BD forwarding \| ${nodes['DUT1']} \
+| | ... | \| /tmp/sock1 \| /tmp/sock2 \|
+| | ... | \| Configure VM for vhost L2BD forwarding \| ${nodes['DUT2']} \
+| | ... | \| /tmp/sock1 \| /tmp/sock2 \| qemu_instance_2 \|
+| | [Arguments] | ${dut_node} | ${sock1} | ${sock2} | ${qemu_name}=vm_node
+| | Import Library | resources.libraries.python.QemuUtils | node=${dut_node} |
+| | ... | WITH NAME | ${qemu_name}
+| | Set Test Variable | ${${qemu_name}} | ${None}
+| | Run Keyword  | ${qemu_name}.Qemu Add Vhost User If | ${sock1}
+| | Run Keyword  | ${qemu_name}.Qemu Add Vhost User If | ${sock2}
+| | ${vm}= | Run keyword | ${qemu_name}.Qemu Start
+| | ${br}= | Set Variable | br0
+| | ${vhost1}= | Get Vhost User If Name By Sock | ${vm} | ${sock1}
+| | ${vhost2}= | Get Vhost User If Name By Sock | ${vm} | ${sock2}
+| | Linux Add Bridge | ${vm} | ${br} | ${vhost1} | ${vhost2}
+| | Set Interface State | ${vm} | ${vhost1} | up | if_type=name
+| | Set Interface State | ${vm} | ${vhost2} | up | if_type=name
+| | Set Interface State | ${vm} | ${br} | up | if_type=name
+| | Set Test Variable | ${${qemu_name}} | ${vm}
diff --git a/resources/libraries/robot/shared/suite_teardown.robot b/resources/libraries/robot/shared/suite_teardown.robot
new file mode 100644
index 0000000000..1b738eaeef
--- /dev/null
+++ b/resources/libraries/robot/shared/suite_teardown.robot
@@ -0,0 +1,47 @@
+# Copyright (c) 2019 Cisco and/or its affiliates.
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Keywords used in suite teardowns."""
+
+*** Settings ***
+| Library | resources.libraries.python.DPDK.DPDKTools
+| ...
+| Documentation | Suite teardown keywords.
+
+*** Keywords ***
+| Tear down suite
+| | [Documentation]
+| | ... | Common suite teardown for tests.
+| | ...
+| | ... | *Arguments:*
+| | ... | - ${actions} - Additional teardown action. Type: list
+| | ...
+| | [Arguments] | @{actions}
+| | ...
+| | :FOR | ${action} | IN | @{actions}
+| | | Run Keyword | Additional Suite Tear Down Action For ${action}
+
+| Additional Suite Tear Down Action For performance
+| | [Documentation]
+| | ... | Additional teardown for suites which uses performance measurement.
+| | ...
+| | Teardown traffic generator | ${tg}
+
+| Additional Suite Tear Down Action For dpdk
+| | [Documentation]
+| | ... | Additional teardown for suites which uses dpdk.
+| | ...
+| | ${duts}= | Get Matches | ${nodes} | DUT*
+| | :FOR | ${dut} | IN | @{duts}
+| | | Cleanup DPDK Environment
+| | | ... | ${nodes['${dut}']} | ${${dut}_if1} | ${${dut}_if2}
diff --git a/resources/libraries/robot/shared/test_teardown.robot b/resources/libraries/robot/shared/test_teardown.robot
new file mode 100644
index 0000000000..e30aa9f41e
--- /dev/null
+++ b/resources/libraries/robot/shared/test_teardown.robot
@@ -0,0 +1,127 @@
+# Copyright (c) 2019 Cisco and/or its affiliates.
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Keywords used in test teardowns."""
+
+*** Settings ***
+| Documentation | Test teardown keywords.
+
+*** Keywords ***
+| Tear down test
+| | [Documentation]
+| | ... | Common test teardown for tests.
+| | ...
+| | ... | *Arguments:*
+| | ... | - ${actions} - Additional teardown action. Type: list
+| | ...
+| | [Arguments] | @{actions}
+| | ...
+| | Remove All Added Ports On All DUTs From Topology | ${nodes}
+| | Show PAPI History On All DUTs | ${nodes}
+| | Get Core Files on All Nodes | ${nodes}
+| | Verify VPP PID in Teardown
+| | :FOR | ${action} | IN | @{actions}
+| | | Run Keyword | Additional Test Tear Down Action For ${action}
+
+| Additional Test Tear Down Action For performance
+| | [Documentation]
+| | ... | Additional teardown for tests which uses performance measurement.
+| | ...
+| | Set Test Variable | ${pkt_trace} | ${True}
+| | Run Keyword If Test Failed
+| | ... | Send traffic at specified rate | ${perf_trial_duration} | 10000pps
+| | ... | ${frame_size} | ${traffic_profile}
+
+| Additional Test Tear Down Action For packet_trace
+| | [Documentation]
+| | ... | Additional teardown for tests which uses packet trace.
+| | ...
+| | Show Packet Trace on All DUTs | ${nodes}
+
+| Additional Test Tear Down Action For container
+| | [Documentation]
+| | ... | Additional teardown for tests which uses containers.
+| | ...
+| | :FOR | ${container_group} | IN | @{container_groups}
+| | | Destroy all '${container_group}' containers
+
+| Additional Test Tear Down Action For vhost
+| | [Documentation]
+| | ... | Additional teardown for tests which uses vhost(s) and VM(s).
+| | ...
+| | # TODO: Remove IF condition once devicetest is running KernelVM.
+| | Show VPP vhost on all DUTs | ${nodes}
+| | Run Keyword If | "PERFTEST" in @{TEST TAGS} | vnf_manager.Kill All VMs
+| | Run Keyword If | "DEVICETEST" in @{TEST TAGS} | vm_node.Qemu Kill
+
+| Additional Test Tear Down Action For nat
+| | [Documentation]
+| | ... | Additional teardown for tests which uses NAT feature.
+| | ...
+| | ${duts}= | Get Matches | ${nodes} | DUT*
+| | :FOR | ${dut} | IN | @{duts}
+| | | Show NAT verbose | ${nodes['${dut}']}
+
+| Additional Test Tear Down Action For namespace
+| | [Documentation]
+| | ... | Additional teardown for tests which uses namespace.
+| | ...
+| | ${duts}= | Get Matches | ${nodes} | DUT*
+| | :FOR | ${dut} | IN | @{duts}
+| | | Clean Up Namespaces | ${nodes['${dut}']}
+
+| Additional Test Tear Down Action For linux_bridge
+| | [Documentation]
+| | ... | Additional teardown for tests which uses linux_bridge.
+| | ...
+| | ${duts}= | Get Matches | ${nodes} | DUT*
+| | :FOR | ${dut} | IN | @{duts}
+| | | Linux Del Bridge | ${nodes['${dut}']} | ${bid_TAP}
+
+| Additional Test Tear Down Action For acl
+| | [Documentation]
+| | ... | Additional teardown for tests which uses ACL feature.
+| | ...
+| | Run Keyword If Test Failed
+| | ... | Vpp Log Plugin Acl Settings | ${dut1}
+| | Run Keyword If Test Failed | Run Keyword And Ignore Error
+| | ... | Vpp Log Plugin Acl Interface Assignment | ${dut1}
+
+| Additional Test Tear Down Action For macipacl
+| | [Documentation]
+| | ... | Additional teardown for tests which uses MACIP ACL feature.
+| | ...
+| | Run Keyword If Test Failed | Run Keyword And Ignore Error
+| | ... | Vpp Log Macip Acl Settings | ${dut1}
+| | Run Keyword And Ignore Error
+| | ... | Vpp Log Macip Acl Interface Assignment | ${dut1}
+
+| Additional Test Tear Down Action For srv6
+| | [Documentation]
+| | ... | Additional teardown for tests which uses SRv6.
+| | ...
+| | Run Keyword If Test Failed
+| | ... | Show SR Policies on all DUTs | ${nodes}
+| | Run Keyword If Test Failed
+| | ... | Show SR Steering Policies on all DUTs | ${nodes}
+| | Run Keyword If Test Failed | Show SR LocalSIDs on all DUTs | ${nodes}
+
+| Additional Test Tear Down Action For ligato
+| | [Documentation]
+| | ... | Additional teardown for performance tests with Ligato.
+| | ...
+| | Run Keyword If Test Failed
+| | ... | Get Kubernetes logs on all DUTs | ${nodes} | csit
+| | Run Keyword If Test Failed
+| | ... | Describe Kubernetes resource on all DUTs | ${nodes} | csit
+| | Delete Kubernetes resource on all DUTs | ${nodes} | csit