Modify sweep ping test cases

- Write separate sweep ping test cases for jumbo frames
- Compute sweep ping end size from reported MTU on DUT interface
- Set MTU on TG according to MTU on DUT interface
- Log VPP packet traces on IPv4 and IPv6 tests failure
- Remove VM_ENV tag from sweep ping test cases for jumbo frames

Change-Id: I47aa7977bcff9c4366c67578aef542924a1d055b
Signed-off-by: Juraj Sloboda <jsloboda@cisco.com>

3 files changed, 61 insertions, 20 deletions

diff --git a/tests/suites/ipv4/ipv4_untagged.robot b/tests/suites/ipv4/ipv4_untagged.robot
index 4f1904e3ba..47fe34acdf 100644
--- a/tests/suites/ipv4/ipv4_untagged.robot
+++ b/tests/suites/ipv4/ipv4_untagged.robot
@@ -14,19 +14,22 @@
 *** Settings ***
 | Library | resources.libraries.python.topology.Topology
 | Library | resources.libraries.python.NodePath
+| Library | resources.libraries.python.Trace
 | Resource | resources/libraries/robot/default.robot
+| Resource | resources/libraries/robot/interfaces.robot
 | Resource | resources/libraries/robot/ipv4.robot
-| Force Tags | HW_ENV | VM_ENV
+| Force Tags | HW_ENV
 | Suite Setup | Run Keywords | Setup all DUTs before test
 | ...         | AND          | Setup all TGs before traffic script
 | ...         | AND          | Update All Interface Data On All Nodes | ${nodes}
 | ...         | AND          | Setup DUT nodes for IPv4 testing
 | Test Setup | Clear interface counters on all vpp nodes in topology | ${nodes}
+| Test Teardown | Run Keyword If Test Failed | Show packet trace on all DUTs | ${nodes}
 
 *** Test Cases ***
 
 | VPP replies to ICMPv4 echo request
-| | [Tags] | 3_NODE_SINGLE_LINK_TOPO
+| | [Tags] | 3_NODE_SINGLE_LINK_TOPO | VM_ENV
 | | Append Nodes | ${nodes['TG']} | ${nodes['DUT1']}
 | | Compute Path
 | | ${src_port} | ${src_node}= | First Interface
@@ -35,6 +38,7 @@
 | | Node "${src_node}" interface "${src_port}" can route to node "${dst_node}" interface "${dst_port}" ${hops} hops away using IPv4
 
 | TG can route to DUT egress interface
+| | [Tags] | 3_NODE_SINGLE_LINK_TOPO | VM_ENV
 | | Append Nodes | ${nodes['TG']} | ${nodes['DUT1']} | ${nodes['DUT2']}
 | | Compute Path
 | | ${src_port} | ${src_node}= | First Interface
@@ -43,7 +47,7 @@
 | | Node "${src_node}" interface "${src_port}" can route to node "${dst_node}" interface "${dst_port}" ${hops} hops away using IPv4
 
 | TG can route to DUT2 through DUT1
-| | [Tags] | 3_NODE_SINGLE_LINK_TOPO
+| | [Tags] | 3_NODE_SINGLE_LINK_TOPO | VM_ENV
 | | Append Nodes | ${nodes['TG']} | ${nodes['DUT1']} | ${nodes['DUT2']}
 | | Compute Path
 | | ${src_port} | ${src_node}= | First Interface
@@ -52,7 +56,7 @@
 | | Node "${src_node}" interface "${src_port}" can route to node "${dst_node}" interface "${dst_port}" ${hops} hops away using IPv4
 
 | TG can route to DUT2 egress interface through DUT1
-| | [Tags] | 3_NODE_SINGLE_LINK_TOPO
+| | [Tags] | 3_NODE_SINGLE_LINK_TOPO | VM_ENV
 | | Append Nodes | ${nodes['TG']} | ${nodes['DUT1']} | ${nodes['DUT2']} | ${nodes['TG']}
 | | Compute Path
 | | ${src_port} | ${src_node}= | First Interface
@@ -61,7 +65,7 @@
 | | Node "${src_node}" interface "${src_port}" can route to node "${dst_node}" interface "${dst_port}" ${hops} hops away using IPv4
 
 | TG can route to TG through DUT1 and DUT2
-| | [Tags] | 3_NODE_SINGLE_LINK_TOPO
+| | [Tags] | 3_NODE_SINGLE_LINK_TOPO | VM_ENV
 | | Append Nodes | ${nodes['TG']} | ${nodes['DUT1']} | ${nodes['DUT2']} | ${nodes['TG']}
 | | Compute Path
 | | ${src_port} | ${src_node}= | First Interface
@@ -81,12 +85,28 @@
 | | ${port} | ${node}= | Next Interface
 | | Check ipv4 interface counter | ${node} | ${port} | ${exp_counter_val}
 
-| VPP can process ICMP echo request from min to max packet size with 1B increment
+| VPP can process ICMP echo request from min to 1500B packet size with 1B increment
+| | [Tags] | 3_NODE_SINGLE_LINK_TOPO | VM_ENV
+| | Ipv4 icmp echo sweep | ${nodes['TG']} | ${nodes['DUT1']} | 0 | 1452 | 1
+
+| VPP can process ICMP echo request from 1500B to max packet size with 10B increment
 | | [Tags] | 3_NODE_SINGLE_LINK_TOPO
-| | Ipv4 icmp echo sweep | ${nodes['TG']} | ${nodes['DUT1']}
-| | ...                  | ${nodes['TG']['interfaces']['port3']['name']}
-| | ...                  | ${nodes['DUT1']['interfaces']['port1']['name']}
+| | [Documentation] | This test case cannot be run reliably on VM_ENV because
+| | ...             | the virtual hosts can be connected using a bridge which
+| | ...             | has its own MTU
+| | [Setup] | Setup MTU on TG based on MTU on DUT | ${nodes['TG']} | ${nodes['DUT1']}
+| | [Teardown] | Set default Ethernet MTU on all interfaces on node | ${nodes['TG']}
+| | Append Nodes | ${nodes['TG']} | ${nodes['DUT1']}
+| | Compute Path
+| | ${dut_port} | ${dut_node}= | Last Interface
+| | ${mtu}= | Get Interface MTU | ${dut_node} | ${dut_port}
+| | # ICMP payload size is frame size minus size of Ehternet header, FCS,
+| | # IPv4 header and ICMP header
+| | ${end_size}= | Evaluate | ${mtu} - 14 - 4 - 20 - 8
+| | Run Keyword If | ${mtu} > 1518
+| | ...            | Ipv4 icmp echo sweep | ${nodes['TG']} | ${nodes['DUT1']}
+| | ...            | 1452 | ${end_size} | 10
 
 | VPP responds to ARP request
-| | [Tags] | 3_NODE_SINGLE_LINK_TOPO
+| | [Tags] | 3_NODE_SINGLE_LINK_TOPO | VM_ENV
 | | Send ARP request and validate response | ${nodes['TG']} | ${nodes['DUT1']}
diff --git a/tests/suites/ipv6/ipv6_untagged.robot b/tests/suites/ipv6/ipv6_untagged.robot
index c9304a23c5..f41e1b5767 100644
--- a/tests/suites/ipv6/ipv6_untagged.robot
+++ b/tests/suites/ipv6/ipv6_untagged.robot
@@ -15,46 +15,67 @@
 
 *** Settings ***
 | Documentation | IPv6 untagged test suite
+| Library | resources.libraries.python.Trace
+| Resource | resources/libraries/robot/interfaces.robot
 | Resource | resources/libraries/robot/ipv6.robot
 | Resource | resources/libraries/robot/counters.robot
 | Resource | resources/libraries/robot/default.robot
 | Variables | resources/libraries/python/IPv6NodesAddr.py | ${nodes}
-| Force Tags | HW_ENV | VM_ENV
+| Force Tags | HW_ENV
 | Suite Setup | Run Keywords | Setup ipv6 to all dut in topology | ${nodes} | ${nodes_ipv6_addr}
 | ...         | AND          | Vpp nodes ra supress link layer | ${nodes}
 | ...         | AND          | Vpp nodes setup ipv6 routing | ${nodes} | ${nodes_ipv6_addr}
 | ...         | AND          | Setup all TGs before traffic script
 | Test Setup | Clear interface counters on all vpp nodes in topology | ${nodes}
+| Test Teardown | Run Keyword If Test Failed | Show packet trace on all DUTs | ${nodes}
 
 *** Test Cases ***
 | VPP replies to ICMPv6 echo request
-| | [Tags] | 3_NODE_SINGLE_LINK_TOPO
+| | [Tags] | 3_NODE_SINGLE_LINK_TOPO | VM_ENV
 | | Ipv6 icmp echo | ${nodes['TG']} | ${nodes['DUT1']} | ${nodes_ipv6_addr}
 
-| VPP can process ICMPv6 echo request from min to max packet size with 1B increment
+| VPP can process ICMPv6 echo request from min to 1500B packet size with 1B increment
+| | [Tags] | 3_NODE_SINGLE_LINK_TOPO | VM_ENV
+| | Ipv6 icmp echo sweep | ${nodes['TG']} | ${nodes['DUT1']} | 0 | 1452 | 1 | ${nodes_ipv6_addr}
+
+| VPP can process ICMPv6 echo request from 1500B to max packet size with 10B increment
 | | [Tags] | 3_NODE_SINGLE_LINK_TOPO
-| | Ipv6 icmp echo sweep | ${nodes['TG']} | ${nodes['DUT1']} | ${nodes_ipv6_addr}
+| | [Documentation] | This test case cannot be run reliably on VM_ENV because
+| | ...             | the virtual hosts can be connected using a bridge which
+| | ...             | has its own MTU
+| | [Setup] | Setup MTU on TG based on MTU on DUT | ${nodes['TG']} | ${nodes['DUT1']}
+| | [Teardown] | Set default Ethernet MTU on all interfaces on node | ${nodes['TG']}
+| | Append Nodes | ${nodes['TG']} | ${nodes['DUT1']}
+| | Compute Path
+| | ${dut_port} | ${dut_node}= | Last Interface
+| | ${mtu}= | Get Interface MTU | ${dut_node} | ${dut_port}
+| | # ICMPv6 payload size is frame size minus size of Ehternet header, FCS,
+| | # IPv6 header and ICMPv6 header
+| | ${end_size}= | Evaluate | ${mtu} - 14 - 4 - 40 - 8
+| | Run Keyword If | ${mtu} > 1518
+| | ...            | Ipv6 icmp echo sweep | ${nodes['TG']} | ${nodes['DUT1']}
+| | ...            | 1452 | ${end_size} | 10 | ${nodes_ipv6_addr}
 
 | TG can route to first DUT egress interface
-| | [Tags] | 3_NODE_SINGLE_LINK_TOPO
+| | [Tags] | 3_NODE_SINGLE_LINK_TOPO | VM_ENV
 | | Ipv6 tg to dut1 egress | ${nodes['TG']} | ${nodes['DUT1']} |
 | | ...                    | ${nodes['DUT2']} | ${nodes_ipv6_addr}
 
 | TG can route to second DUT through first DUT
-| | [Tags] | 3_NODE_SINGLE_LINK_TOPO
+| | [Tags] | 3_NODE_SINGLE_LINK_TOPO | VM_ENV
 | | Ipv6 tg to dut2 via dut1 | ${nodes['TG']} | ${nodes['DUT1']}
 | | ...                      | ${nodes['DUT2']} | ${nodes_ipv6_addr}
 
 | TG can route to second DUT egress interface through first DUT
-| | [Tags] | 3_NODE_SINGLE_LINK_TOPO
+| | [Tags] | 3_NODE_SINGLE_LINK_TOPO | VM_ENV
 | | Ipv6 tg to dut2 egress via dut1 | ${nodes['TG']} | ${nodes['DUT1']}
 | | ...                             | ${nodes['DUT2']} | ${nodes_ipv6_addr}
 
 | TG can route to TG through first and second DUT
-| | [Tags] | 3_NODE_SINGLE_LINK_TOPO
+| | [Tags] | 3_NODE_SINGLE_LINK_TOPO | VM_ENV
 | | Ipv6 tg to tg routed | ${nodes['TG']} | ${nodes['DUT1']} | ${nodes['DUT2']}
 | | ...                  | ${nodes_ipv6_addr}
 
 | VPP replies to IPv6 Neighbor Solicitation
-| | [Tags] | 3_NODE_SINGLE_LINK_TOPO
+| | [Tags] | 3_NODE_SINGLE_LINK_TOPO | VM_ENV
 | | Ipv6 neighbor solicitation | ${nodes['TG']} | ${nodes['DUT1']} | ${nodes_ipv6_addr}
diff --git a/tests/suites/vxlan/vxlan_untagged.robot b/tests/suites/vxlan/vxlan_untagged.robot
index 45d57a902d..0ec977052f 100644
--- a/tests/suites/vxlan/vxlan_untagged.robot
+++ b/tests/suites/vxlan/vxlan_untagged.robot
@@ -16,7 +16,7 @@
 | Resource | resources/libraries/robot/default.robot
 | Resource | resources/libraries/robot/vxlan.robot
 | Resource | resources/libraries/robot/l2_traffic.robot
-| Force Tags | 3_NODE_SINGLE_LINK_TOPO | EXPECTED_FAILING
+| Force Tags | 3_NODE_SINGLE_LINK_TOPO | EXPECTED_FAILING | VM_ENV | HW_ENV
 | Suite Setup | Run Keywords | Setup all DUTs before test
 | ...         | AND          | Setup all TGs before traffic script
 | ...         | AND          | Setup VXLAN tunnel on nodes | ${nodes['TG']}