From 6871d4e7f6ed43ce0bb003cb08e88b2581a474eb Mon Sep 17 00:00:00 2001 From: Peter Mikus Date: Tue, 18 Jun 2019 20:31:48 +0000 Subject: Align suite/test teardown/setup + Phase I - aligning FUNC/PERF Change-Id: I46b1e43687ddb29f5ed3c6335fe1baf21ec02822 Signed-off-by: Peter Mikus --- resources/libraries/robot/shared/default.robot | 39 +------ resources/libraries/robot/shared/qemu.robot | 51 +++++++++ .../libraries/robot/shared/suite_teardown.robot | 47 ++++++++ .../libraries/robot/shared/test_teardown.robot | 127 +++++++++++++++++++++ 4 files changed, 230 insertions(+), 34 deletions(-) create mode 100644 resources/libraries/robot/shared/qemu.robot create mode 100644 resources/libraries/robot/shared/suite_teardown.robot create mode 100644 resources/libraries/robot/shared/test_teardown.robot (limited to 'resources/libraries/robot/shared') diff --git a/resources/libraries/robot/shared/default.robot b/resources/libraries/robot/shared/default.robot index 262dc02f55..b7e4a75773 100644 --- a/resources/libraries/robot/shared/default.robot +++ b/resources/libraries/robot/shared/default.robot @@ -34,7 +34,10 @@ | Library | resources.libraries.python.topology.Topology | ... | Resource | resources/libraries/robot/shared/container.robot -| Resource | resources/libraries/robot/vm/qemu.robot +| Resource | resources/libraries/robot/shared/qemu.robot +| Resource | resources/libraries/robot/shared/suite_teardown.robot +| Resource | resources/libraries/robot/shared/test_teardown.robot + *** Keywords *** | Configure all TGs for traffic script @@ -421,6 +424,7 @@ | | ${duts}= | Get Matches | ${nodes} | DUT* | | :FOR | ${dut} | IN | @{duts} | | | Run keyword | ${dut}.Apply Config +| | Save VPP PIDs | | Enable Coredump Limit VPP on All DUTs | ${nodes} | | Update All Interface Data On All Nodes | ${nodes} | skip_tg=${True} @@ -434,8 +438,6 @@ | | :FOR | ${key} | IN | @{keys} | | | ${pid}= | Get From Dictionary | ${setup_vpp_pids} | ${key} | | | Run Keyword If | $pid is None | FAIL | No VPP PID found on node ${key} -| | | Run Keyword If | ',' in '${pid}' -| | | ... | FAIL | More then one VPP PID found on node ${key}: ${pid} | | Set Test Variable | ${setup_vpp_pids} | Verify VPP PID in Teardown @@ -486,17 +488,6 @@ | | Update All Interface Data On All Nodes | ${nodes} | skip_tg_udev=${True} | | Reset PAPI History On All DUTs | ${nodes} -| Tear down VPP device test -# TODO: Generalize this KW if it will not diverge from Functional derivate too -# much -| | [Documentation] | Common test teardown for vpp-device tests. -| | ... -| | Remove All Added Ports On All DUTs From Topology | ${nodes} -| | Show Packet Trace on All DUTs | ${nodes} -| | Show PAPI History On All DUTs | ${nodes} -| | Vpp Show Errors On All DUTs | ${nodes} -| | Verify VPP PID in Teardown - | Tear down LISP functional test | | [Documentation] | Common test teardown for functional tests with LISP. | | ... @@ -508,19 +499,6 @@ | | Vpp Show Errors On All DUTs | ${nodes} | | Verify VPP PID in Teardown -| Tear down LISP functional test with QEMU -| | [Documentation] | Common test teardown for functional tests with LISP and\ -| | ... | QEMU. -| | ... -| | Remove All Added Ports On All DUTs From Topology | ${nodes} -| | Show Packet Trace on All DUTs | ${nodes} -| | Show PAPI History On All DUTs | ${nodes} -| | Show Vpp Settings | ${nodes['DUT1']} -| | Show Vpp Settings | ${nodes['DUT2']} -| | Vpp Show Errors On All DUTs | ${nodes} -| | Tear down QEMU -| | Verify VPP PID in Teardown - | Set up functional test with containers | | [Documentation] | | ... | Common test setup for functional tests with containers. @@ -560,13 +538,6 @@ | | Start VPP in all '${container_group}' containers | | Append To List | ${container_groups} | ${container_group} -| Tear down functional test with container -| | [Documentation] -| | ... | Common test teardown for functional tests which uses containers. -| | ... -| | :FOR | ${container_group} | IN | @{container_groups} -| | | Destroy all '${container_group}' containers - | Stop VPP Service on DUT | | [Documentation] | Stop the VPP service on the specified node. | | ... diff --git a/resources/libraries/robot/shared/qemu.robot b/resources/libraries/robot/shared/qemu.robot new file mode 100644 index 0000000000..1178fa93b4 --- /dev/null +++ b/resources/libraries/robot/shared/qemu.robot @@ -0,0 +1,51 @@ +# Copyright (c) 2019 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +*** Settings *** +| Library | resources.libraries.python.L2Util +| Library | resources.libraries.python.InterfaceUtil + +*** Keywords *** +| Configure VM for vhost L2BD forwarding +| | [Documentation] | Setup QEMU and start VM with two vhost interfaces. +| | ... +| | ... | *Arguments:* +| | ... | - ${dut_node} - DUT node to start VM on. Type: dictionary +| | ... | - ${sock1} - Socket path for first Vhost-User interface. Type: string +| | ... | - ${sock2} - Socket path for second Vhost-User interface. Type: string +| | ... | - ${qemu_name} - Qemu instance name by which the object will be +| | ... | accessed (Optional). Type: string +| | ... +| | ... | _NOTE:_ This KW sets following test case variable: +| | ... | - ${${qemu_name}} - VM node info. Type: dictionary +| | ... +| | ... | *Example:* +| | ... +| | ... | \| Configure VM for vhost L2BD forwarding \| ${nodes['DUT1']} \ +| | ... | \| /tmp/sock1 \| /tmp/sock2 \| +| | ... | \| Configure VM for vhost L2BD forwarding \| ${nodes['DUT2']} \ +| | ... | \| /tmp/sock1 \| /tmp/sock2 \| qemu_instance_2 \| +| | [Arguments] | ${dut_node} | ${sock1} | ${sock2} | ${qemu_name}=vm_node +| | Import Library | resources.libraries.python.QemuUtils | node=${dut_node} | +| | ... | WITH NAME | ${qemu_name} +| | Set Test Variable | ${${qemu_name}} | ${None} +| | Run Keyword | ${qemu_name}.Qemu Add Vhost User If | ${sock1} +| | Run Keyword | ${qemu_name}.Qemu Add Vhost User If | ${sock2} +| | ${vm}= | Run keyword | ${qemu_name}.Qemu Start +| | ${br}= | Set Variable | br0 +| | ${vhost1}= | Get Vhost User If Name By Sock | ${vm} | ${sock1} +| | ${vhost2}= | Get Vhost User If Name By Sock | ${vm} | ${sock2} +| | Linux Add Bridge | ${vm} | ${br} | ${vhost1} | ${vhost2} +| | Set Interface State | ${vm} | ${vhost1} | up | if_type=name +| | Set Interface State | ${vm} | ${vhost2} | up | if_type=name +| | Set Interface State | ${vm} | ${br} | up | if_type=name +| | Set Test Variable | ${${qemu_name}} | ${vm} diff --git a/resources/libraries/robot/shared/suite_teardown.robot b/resources/libraries/robot/shared/suite_teardown.robot new file mode 100644 index 0000000000..1b738eaeef --- /dev/null +++ b/resources/libraries/robot/shared/suite_teardown.robot @@ -0,0 +1,47 @@ +# Copyright (c) 2019 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Keywords used in suite teardowns.""" + +*** Settings *** +| Library | resources.libraries.python.DPDK.DPDKTools +| ... +| Documentation | Suite teardown keywords. + +*** Keywords *** +| Tear down suite +| | [Documentation] +| | ... | Common suite teardown for tests. +| | ... +| | ... | *Arguments:* +| | ... | - ${actions} - Additional teardown action. Type: list +| | ... +| | [Arguments] | @{actions} +| | ... +| | :FOR | ${action} | IN | @{actions} +| | | Run Keyword | Additional Suite Tear Down Action For ${action} + +| Additional Suite Tear Down Action For performance +| | [Documentation] +| | ... | Additional teardown for suites which uses performance measurement. +| | ... +| | Teardown traffic generator | ${tg} + +| Additional Suite Tear Down Action For dpdk +| | [Documentation] +| | ... | Additional teardown for suites which uses dpdk. +| | ... +| | ${duts}= | Get Matches | ${nodes} | DUT* +| | :FOR | ${dut} | IN | @{duts} +| | | Cleanup DPDK Environment +| | | ... | ${nodes['${dut}']} | ${${dut}_if1} | ${${dut}_if2} diff --git a/resources/libraries/robot/shared/test_teardown.robot b/resources/libraries/robot/shared/test_teardown.robot new file mode 100644 index 0000000000..e30aa9f41e --- /dev/null +++ b/resources/libraries/robot/shared/test_teardown.robot @@ -0,0 +1,127 @@ +# Copyright (c) 2019 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Keywords used in test teardowns.""" + +*** Settings *** +| Documentation | Test teardown keywords. + +*** Keywords *** +| Tear down test +| | [Documentation] +| | ... | Common test teardown for tests. +| | ... +| | ... | *Arguments:* +| | ... | - ${actions} - Additional teardown action. Type: list +| | ... +| | [Arguments] | @{actions} +| | ... +| | Remove All Added Ports On All DUTs From Topology | ${nodes} +| | Show PAPI History On All DUTs | ${nodes} +| | Get Core Files on All Nodes | ${nodes} +| | Verify VPP PID in Teardown +| | :FOR | ${action} | IN | @{actions} +| | | Run Keyword | Additional Test Tear Down Action For ${action} + +| Additional Test Tear Down Action For performance +| | [Documentation] +| | ... | Additional teardown for tests which uses performance measurement. +| | ... +| | Set Test Variable | ${pkt_trace} | ${True} +| | Run Keyword If Test Failed +| | ... | Send traffic at specified rate | ${perf_trial_duration} | 10000pps +| | ... | ${frame_size} | ${traffic_profile} + +| Additional Test Tear Down Action For packet_trace +| | [Documentation] +| | ... | Additional teardown for tests which uses packet trace. +| | ... +| | Show Packet Trace on All DUTs | ${nodes} + +| Additional Test Tear Down Action For container +| | [Documentation] +| | ... | Additional teardown for tests which uses containers. +| | ... +| | :FOR | ${container_group} | IN | @{container_groups} +| | | Destroy all '${container_group}' containers + +| Additional Test Tear Down Action For vhost +| | [Documentation] +| | ... | Additional teardown for tests which uses vhost(s) and VM(s). +| | ... +| | # TODO: Remove IF condition once devicetest is running KernelVM. +| | Show VPP vhost on all DUTs | ${nodes} +| | Run Keyword If | "PERFTEST" in @{TEST TAGS} | vnf_manager.Kill All VMs +| | Run Keyword If | "DEVICETEST" in @{TEST TAGS} | vm_node.Qemu Kill + +| Additional Test Tear Down Action For nat +| | [Documentation] +| | ... | Additional teardown for tests which uses NAT feature. +| | ... +| | ${duts}= | Get Matches | ${nodes} | DUT* +| | :FOR | ${dut} | IN | @{duts} +| | | Show NAT verbose | ${nodes['${dut}']} + +| Additional Test Tear Down Action For namespace +| | [Documentation] +| | ... | Additional teardown for tests which uses namespace. +| | ... +| | ${duts}= | Get Matches | ${nodes} | DUT* +| | :FOR | ${dut} | IN | @{duts} +| | | Clean Up Namespaces | ${nodes['${dut}']} + +| Additional Test Tear Down Action For linux_bridge +| | [Documentation] +| | ... | Additional teardown for tests which uses linux_bridge. +| | ... +| | ${duts}= | Get Matches | ${nodes} | DUT* +| | :FOR | ${dut} | IN | @{duts} +| | | Linux Del Bridge | ${nodes['${dut}']} | ${bid_TAP} + +| Additional Test Tear Down Action For acl +| | [Documentation] +| | ... | Additional teardown for tests which uses ACL feature. +| | ... +| | Run Keyword If Test Failed +| | ... | Vpp Log Plugin Acl Settings | ${dut1} +| | Run Keyword If Test Failed | Run Keyword And Ignore Error +| | ... | Vpp Log Plugin Acl Interface Assignment | ${dut1} + +| Additional Test Tear Down Action For macipacl +| | [Documentation] +| | ... | Additional teardown for tests which uses MACIP ACL feature. +| | ... +| | Run Keyword If Test Failed | Run Keyword And Ignore Error +| | ... | Vpp Log Macip Acl Settings | ${dut1} +| | Run Keyword And Ignore Error +| | ... | Vpp Log Macip Acl Interface Assignment | ${dut1} + +| Additional Test Tear Down Action For srv6 +| | [Documentation] +| | ... | Additional teardown for tests which uses SRv6. +| | ... +| | Run Keyword If Test Failed +| | ... | Show SR Policies on all DUTs | ${nodes} +| | Run Keyword If Test Failed +| | ... | Show SR Steering Policies on all DUTs | ${nodes} +| | Run Keyword If Test Failed | Show SR LocalSIDs on all DUTs | ${nodes} + +| Additional Test Tear Down Action For ligato +| | [Documentation] +| | ... | Additional teardown for performance tests with Ligato. +| | ... +| | Run Keyword If Test Failed +| | ... | Get Kubernetes logs on all DUTs | ${nodes} | csit +| | Run Keyword If Test Failed +| | ... | Describe Kubernetes resource on all DUTs | ${nodes} | csit +| | Delete Kubernetes resource on all DUTs | ${nodes} | csit -- cgit 1.2.3-korg