diff options
| author |   selias <samelias@cisco.com> | 2017-06-21 18:52:10 +0200 |
|---|---|---|
| committer |   Tibor Frank <tifrank@cisco.com> | 2017-06-26 12:39:32 +0000 |
| commit | b7f0c133f672b39b49e0480f71dd0629986bcf72 (patch) | |
| tree | 5a8504f5cb2c3b2bddbbf3a8289ea5ce120bc1f4 /tests/func/honeycomb | |
| parent | 41bded220766aad6f805a9864ef677abe9ff9004 (diff) | |
HC Test: improve functional suite setup and teardown
Restart the whole stack (VPP+Honeycomb+-ODL) in between test suites.
Add generic suite setup and suite teardown keywords for all func suites
Re-enable NSH_SFC suite, cleanup test failures
Change-Id: I71025b4ed3a7c4912dcd7b62aedef6082357990c
Signed-off-by: selias <samelias@cisco.com>
Diffstat (limited to 'tests/func/honeycomb')
25 files changed, 186 insertions, 151 deletions
diff --git a/tests/func/honeycomb/__init__.robot b/tests/func/honeycomb/__init__.robot index 3f79a6a1f8..fff6097d08 100644 --- a/tests/func/honeycomb/__init__.robot +++ b/tests/func/honeycomb/__init__.robot @@ -21,29 +21,7 @@ | Resource | resources/libraries/robot/honeycomb/honeycomb.robot | ... | Suite Setup | Run Keywords | Configure all DUTs before test | AND -| ... | Configure Honeycomb for testing | ${node} | AND -| ... | Configure ODL Client for testing | ${node} | AND -| ... | Set Global Variable | ${node} +| ... | Set Global Variable | ${node} | AND +| ... | Stop Honeycomb service on DUTs | ${node} | ... | Suite Teardown | Archive Honeycomb log file | ${node} - -*** Keywords *** -| Configure Honeycomb for testing -| | [Arguments] | ${node} -| | Configure Restconf binding address | ${node} -| | Enable Module Features | ${node} -| | Configure Log Level | ${node} | TRACE -| | Configure Persistence | ${node} | disable -| | Configure jVPP timeout | ${node} | ${14} -| | Clear Persisted Honeycomb Configuration | ${node} -| | Configure Honeycomb service on DUTs | ${node} - -| Configure ODL Client for testing -| | [Arguments] | ${node} -| | ${use_odl_client}= | Get Variable Value | ${HC_ODL} -| | Run Keyword If | '${use_odl_client}' != '${NONE}' -| | ... | Run Keywords -| | ... | Set Global Variable | ${use_odl_client} | AND -| | ... | Configure ODL Client Service On DUT | ${node} | ${use_odl_client} -| | ... | ELSE | Log | Variable HC_ODL is not present. Not using ODL. -| | ... | level=INFO diff --git a/tests/func/honeycomb/mgmt-cfg-acl-apihc-apivat-func.robot b/tests/func/honeycomb/mgmt-cfg-acl-apihc-apivat-func.robot index b18b111981..f66eef88c4 100644 --- a/tests/func/honeycomb/mgmt-cfg-acl-apihc-apivat-func.robot +++ b/tests/func/honeycomb/mgmt-cfg-acl-apihc-apivat-func.robot @@ -21,7 +21,9 @@ | Resource | resources/libraries/robot/honeycomb/access_control_lists.robot | Variables | resources/test_data/honeycomb/acl.py | ... -| Suite Teardown | Restart Honeycomb and VPP | ${node} +| Suite Setup | Set Up Honeycomb Functional Test Suite | ${node} +| ... +| Suite Teardown | Tear Down Honeycomb Functional Test Suite | ${node} | ... | Documentation | *Honeycomb access control lists test suite.* | ... diff --git a/tests/func/honeycomb/mgmt-cfg-dhcp-apihc-apivat-func.robot b/tests/func/honeycomb/mgmt-cfg-dhcp-apihc-apivat-func.robot index fba37e6318..d73334eabd 100644 --- a/tests/func/honeycomb/mgmt-cfg-dhcp-apihc-apivat-func.robot +++ b/tests/func/honeycomb/mgmt-cfg-dhcp-apihc-apivat-func.robot @@ -28,9 +28,9 @@ | ... | Documentation | *Honeycomb DHCP relay test suite.* | ... -| Test Setup | Clear Packet Trace on All DUTs | ${nodes} +| Suite Setup | Set Up Honeycomb Functional Test Suite | ${node} | ... -| Suite Teardown | Restart Honeycomb and VPP | ${node} +| Suite Teardown | Tear Down Honeycomb Functional Test Suite | ${node} | ... | Force Tags | HC_FUNC diff --git a/tests/func/honeycomb/mgmt-cfg-int-apihcnc-func.robot b/tests/func/honeycomb/mgmt-cfg-int-apihcnc-func.robot index 3e3c64fd1f..6989d356c7 100644 --- a/tests/func/honeycomb/mgmt-cfg-int-apihcnc-func.robot +++ b/tests/func/honeycomb/mgmt-cfg-int-apihcnc-func.robot @@ -25,7 +25,10 @@ | ... | Force Tags | HC_FUNC | HC_REST_ONLY | ... -| Suite Teardown | Restart Honeycomb and VPP | ${node} +| Suite Setup | Set Up Honeycomb Functional Test Suite | ${node} +| ... +| Suite Teardown | Tear Down Honeycomb Functional Test Suite | ${node} +| ... *** Variables *** | ${interface}= | ${node['interfaces']['port1']['name']} diff --git a/tests/func/honeycomb/mgmt-cfg-int-subint-apihc-apivat-func.robot b/tests/func/honeycomb/mgmt-cfg-int-subint-apihc-apivat-func.robot index f766eb296c..45ffd93af6 100644 --- a/tests/func/honeycomb/mgmt-cfg-int-subint-apihc-apivat-func.robot +++ b/tests/func/honeycomb/mgmt-cfg-int-subint-apihc-apivat-func.robot @@ -19,9 +19,11 @@ | Resource | resources/libraries/robot/honeycomb/interfaces.robot | Variables | resources/test_data/honeycomb/sub_interfaces.py | ... -| Suite Setup | Add Interface local0 To Topology | ${node} +| Suite Setup | Run Keywords +| ... | Set Up Honeycomb Functional Test Suite | ${node} | AND +| ... | Add Interface local0 To Topology | ${node} | ... -| Suite Teardown | Restart Honeycomb and VPP | ${node} +| Suite Teardown | Tear Down Honeycomb Functional Test Suite | ${node} | ... | Force Tags | HC_FUNC | ... diff --git a/tests/func/honeycomb/mgmt-cfg-intip4-intip6-apihc-apivat-func.robot b/tests/func/honeycomb/mgmt-cfg-intip4-intip6-apihc-apivat-func.robot index 03160e0a65..804c067a8a 100644 --- a/tests/func/honeycomb/mgmt-cfg-intip4-intip6-apihc-apivat-func.robot +++ b/tests/func/honeycomb/mgmt-cfg-intip4-intip6-apihc-apivat-func.robot @@ -25,7 +25,9 @@ | ... | Force Tags | HC_FUNC | ... -| Suite Teardown | Restart Honeycomb and VPP | ${node} +| Suite Setup | Set Up Honeycomb Functional Test Suite | ${node} +| ... +| Suite Teardown | Tear Down Honeycomb Functional Test Suite | ${node} | ... | Documentation | *Honeycomb interface management test suite.* @@ -214,40 +216,7 @@ | | ... | ${tg_to_dut_if1} | ${tg_to_dut_if1_mac} | | ... | ${tg_to_dut_if1} | ${dut_to_tg_if1_mac} -| TC11: Honeycomb fails to configure two IPv4 addresses from the same subnet -| | [Documentation] | Check if Honeycomb can configure two IPv4 addresses in\ -| | ... | the same subnet onto a single interface. It should not be possible. -| | ... -| | [Teardown] | Honeycomb removes interface IPv4 addresses | ${node} -| | ... | ${interface} -| | ... -| | [Tags] | EXPECTED_FAILING -| | ... -# VPP API does not configure the second address, but returns success. VPP-649 -| | When Honeycomb sets interface IPv4 address with prefix -| | ... | ${node} | ${interface} | 192.168.0.1 | ${9} -| | Then Honeycomb fails to add interface IPv4 address -| | ... | ${node} | ${interface} | 192.168.0.2 | ${9} -| | And Honeycomb fails to add interface IPv4 address -| | ... | ${node} | ${interface} | 192.232.0.2 | ${9} - -| TC12: Honeycomb fails to configure two IPv6 addresses from the same subnet -| | [Documentation] | Check if Honeycomb can configure two IPv6 addresses in\ -| | ... | the same subnet onto a single interface. It should not be possible. -| | ... -| | [Tags] | EXPECTED_FAILING -| | ... -# VPP API does not configure the second address, but returns success. VPP-649 -| | [Teardown] | Honeycomb removes interface IPv6 addresses | ${node} -| | ... | ${interface} -| | When Honeycomb sets interface IPv6 address -| | ... | ${node} | ${interface} | 10::FF10 | ${64} -| | Then Honeycomb fails to add interface IPv6 address -| | ... | ${node} | ${interface} | 10::FF11 | ${64} -| | And Honeycomb fails to add interface IPv6 address -| | ... | ${node} | ${interface} | 10::FFFF | ${64} - -| TC13: Honeycomb can configure unnumbered interface +| TC11: Honeycomb can configure unnumbered interface | | [Documentation] | Check if Honeycomb can configure an unnumbered interface\ | | ... | on a physical interface, borrowing the IP address of 'local0'. | | ... @@ -266,7 +235,7 @@ | | ... | ${node} | ${interface} | ${ipv4_address} | | ... | ${ipv4_prefix} | ${ipv4_mask} -| TC14: Honeycomb removes interface unnumbered configuration +| TC12: Honeycomb removes interface unnumbered configuration | | [Documentation] | Check if Honeycomb can remove unnumbered configuration\ | | ... | from an interface. | | ... @@ -289,3 +258,36 @@ | | ... | ${ipv4_prefix} | ${ipv4_mask} | | And IPv4 address from Honeycomb should be empty | ${node} | ${interface} | | And ipv4 address from VAT should be empty | ${node} | ${interface} + +| TC13: Honeycomb fails to configure two IPv4 addresses from the same subnet +| | [Documentation] | Check if Honeycomb can configure two IPv4 addresses in\ +| | ... | the same subnet onto a single interface. It should not be possible. +| | ... +| | [Teardown] | Honeycomb removes interface IPv4 addresses | ${node} +| | ... | ${interface} +| | ... +| | [Tags] | EXPECTED_FAILING +| | ... +# VPP API does not configure the second address, but returns success. VPP-649 +| | When Honeycomb sets interface IPv4 address with prefix +| | ... | ${node} | ${interface} | 192.168.0.1 | ${9} +| | Then Honeycomb fails to add interface IPv4 address +| | ... | ${node} | ${interface} | 192.168.0.2 | ${9} +| | And Honeycomb fails to add interface IPv4 address +| | ... | ${node} | ${interface} | 192.232.0.2 | ${9} + +| TC14: Honeycomb fails to configure two IPv6 addresses from the same subnet +| | [Documentation] | Check if Honeycomb can configure two IPv6 addresses in\ +| | ... | the same subnet onto a single interface. It should not be possible. +| | ... +| | [Tags] | EXPECTED_FAILING +| | ... +# VPP API does not configure the second address, but returns success. VPP-649 +| | [Teardown] | Honeycomb removes interface IPv6 addresses | ${node} +| | ... | ${interface} +| | When Honeycomb sets interface IPv6 address +| | ... | ${node} | ${interface} | 10::FF10 | ${64} +| | Then Honeycomb fails to add interface IPv6 address +| | ... | ${node} | ${interface} | 10::FF11 | ${64} +| | And Honeycomb fails to add interface IPv6 address +| | ... | ${node} | ${interface} | 10::FFFF | ${64} diff --git a/tests/func/honeycomb/mgmt-cfg-inttap-apihc-apivat-func.robot b/tests/func/honeycomb/mgmt-cfg-inttap-apihc-apivat-func.robot index 74d99a9684..f35384c473 100644 --- a/tests/func/honeycomb/mgmt-cfg-inttap-apihc-apivat-func.robot +++ b/tests/func/honeycomb/mgmt-cfg-inttap-apihc-apivat-func.robot @@ -29,8 +29,9 @@ | ... | Force Tags | HC_FUNC | ... -| Suite Teardown | Run Keyword If Any Tests Failed -| ... | Restart Honeycomb and VPP | ${node} +| Suite Setup | Set Up Honeycomb Functional Test Suite | ${node} +| ... +| Suite Teardown | Tear Down Honeycomb Functional Test Suite | ${node} | ... | Documentation | *Honeycomb TAP management test suite.* diff --git a/tests/func/honeycomb/mgmt-cfg-intvhost-apihc-apivat-func.robot b/tests/func/honeycomb/mgmt-cfg-intvhost-apihc-apivat-func.robot index 64136bcfd6..4a2a763523 100644 --- a/tests/func/honeycomb/mgmt-cfg-intvhost-apihc-apivat-func.robot +++ b/tests/func/honeycomb/mgmt-cfg-intvhost-apihc-apivat-func.robot @@ -29,8 +29,9 @@ | ... | Force Tags | HC_FUNC | ... -| Suite Teardown | Run Keyword If Any Tests Failed -| ... | Restart Honeycomb and VPP | ${node} +| Suite Setup | Set Up Honeycomb Functional Test Suite | ${node} +| ... +| Suite Teardown | Tear Down Honeycomb Functional Test Suite | ${node} | ... | Documentation | *Honeycomb vhost-user interface management test suite.* diff --git a/tests/func/honeycomb/mgmt-cfg-l2bd-apihc-apivat-func.robot b/tests/func/honeycomb/mgmt-cfg-l2bd-apihc-apivat-func.robot index e6287e79ae..6955167cb4 100644 --- a/tests/func/honeycomb/mgmt-cfg-l2bd-apihc-apivat-func.robot +++ b/tests/func/honeycomb/mgmt-cfg-l2bd-apihc-apivat-func.robot @@ -29,10 +29,9 @@ | Resource | resources/libraries/robot/honeycomb/interfaces.robot | Resource | resources/libraries/robot/honeycomb/bridge_domain.robot | ... -| Suite Teardown | Run keywords -| ... | Run Keyword If Any Tests Failed -| ... | Restart Honeycomb and VPP | ${node} -| ... | AND | Honeycomb removes all bridge domains | ${node} | @{interfaces} +| Suite Setup | Set Up Honeycomb Functional Test Suite | ${node} +| ... +| Suite Teardown | Tear Down Honeycomb Functional Test Suite | ${node} | ... | Force Tags | HC_FUNC | ... diff --git a/tests/func/honeycomb/mgmt-cfg-l2fib-apihc-apivat-func.robot b/tests/func/honeycomb/mgmt-cfg-l2fib-apihc-apivat-func.robot index 184f361947..2073b9b24b 100644 --- a/tests/func/honeycomb/mgmt-cfg-l2fib-apihc-apivat-func.robot +++ b/tests/func/honeycomb/mgmt-cfg-l2fib-apihc-apivat-func.robot @@ -22,14 +22,9 @@ | ... | Documentation | *Honeycomb L2 FIB management test suite.* | ... -| Suite Setup | Run keywords | Set test interface down +| Suite Setup | Set Up Honeycomb Functional Test Suite | ${node} | ... -| Suite Teardown | Run keywords -| ... | Run Keyword If Any Tests Failed -| ... | Restart Honeycomb and VPP | ${node} -| ... | AND -| ... | Honeycomb removes all bridge domains -| ... | ${node} | ${interface} | ${interface2} +| Suite Teardown | Tear Down Honeycomb Functional Test Suite | ${node} | ... | Force tags | HC_FUNC diff --git a/tests/func/honeycomb/mgmt-cfg-lisp-apihc-apivat-func.robot b/tests/func/honeycomb/mgmt-cfg-lisp-apihc-apivat-func.robot index 42f5a84b8f..83634bf79b 100644 --- a/tests/func/honeycomb/mgmt-cfg-lisp-apihc-apivat-func.robot +++ b/tests/func/honeycomb/mgmt-cfg-lisp-apihc-apivat-func.robot @@ -30,8 +30,9 @@ | ... | Documentation | *Honeycomb Lisp test suite.* | ... -| Suite Teardown | Run Keyword If Any Tests Failed -| ... | Restart Honeycomb and VPP | ${node} +| Suite Setup | Set Up Honeycomb Functional Test Suite | ${node} +| ... +| Suite Teardown | Tear Down Honeycomb Functional Test Suite | ${node} | ... | Force Tags | HC_FUNC diff --git a/tests/func/honeycomb/mgmt-cfg-nsh-apihc-apivat-func.robot b/tests/func/honeycomb/mgmt-cfg-nsh-apihc-apivat-func.robot index 03ccc320fc..9270b98156 100644 --- a/tests/func/honeycomb/mgmt-cfg-nsh-apihc-apivat-func.robot +++ b/tests/func/honeycomb/mgmt-cfg-nsh-apihc-apivat-func.robot @@ -24,10 +24,15 @@ | ... | Documentation | *Honeycomb NSH test suite.* | ... -| Suite Teardown | Run Keyword If Any Tests Failed -| ... | Restart Honeycomb and VPP | ${node} -# disabled pending NSH version 17.04 -#| Force Tags | honeycomb_sanity | honeycomb_odl +| Suite Setup | Run Keywords +| ... | Enable Honeycomb Feature | ${node} | NSH | AND +| ... | Set Up Honeycomb Functional Test Suite | ${node} +| ... +| Suite Teardown | Run Keywords +| ... | Tear Down Honeycomb Functional Test Suite | ${node} | AND +| ... | Disable Honeycomb Feature | ${node} | NSH +| ... +| Force Tags | honeycomb_sanity | honeycomb_odl *** Test Cases *** | TC01: Honeycomb can configure NSH entry diff --git a/tests/func/honeycomb/mgmt-cfg-pbb-apihc-apivat-func.robot b/tests/func/honeycomb/mgmt-cfg-pbb-apihc-apivat-func.robot index a04932556a..fb9e49e059 100644 --- a/tests/func/honeycomb/mgmt-cfg-pbb-apihc-apivat-func.robot +++ b/tests/func/honeycomb/mgmt-cfg-pbb-apihc-apivat-func.robot @@ -22,8 +22,9 @@ | ... | Documentation | *Honeycomb provider backbone bridge test suite.* | ... -| Suite Teardown | Run Keyword If Any Tests Failed -| ... | Restart Honeycomb and VPP | ${node} +| Suite Setup | Set Up Honeycomb Functional Test Suite | ${node} +| ... +| Suite Teardown | Tear Down Honeycomb Functional Test Suite | ${node} | ... | Force Tags | HC_FUNC diff --git a/tests/func/honeycomb/mgmt-cfg-pluginacl-apihc-apivat-func.robot b/tests/func/honeycomb/mgmt-cfg-pluginacl-apihc-apivat-func.robot index a365f54348..d9a3933578 100644 --- a/tests/func/honeycomb/mgmt-cfg-pluginacl-apihc-apivat-func.robot +++ b/tests/func/honeycomb/mgmt-cfg-pluginacl-apihc-apivat-func.robot @@ -45,11 +45,14 @@ | ... | Test Setup | Clear Packet Trace on All DUTs | ${nodes} | ... -| Suite Teardown | Restart Honeycomb and VPP | ${node} +| Suite Setup | Set Up Honeycomb Functional Test Suite | ${node} +| ... +| Suite Teardown | Tear Down Honeycomb Functional Test Suite | ${node} | ... | Documentation | *Honeycomb access control lists test suite for ACL plugin.* | ... -| Force Tags | HC_FUNC +# Failing due to HC2VPP-173: cannot clean up ACLs in test teardown +| Force Tags | HC_FUNC | EXPECTED_FAILING *** Test Cases *** | TC01: ACL MAC filtering through plugin-acl node - bridged @@ -71,17 +74,20 @@ | | And Honeycomb Assigns plugin-acl Chain To Interface | | ... | ${dut_node} | ${dut_to_tg_if1} | ${acl_name_macip} | | ... | ingress | macip=${True} -| | Then Send TCP or UDP packet and verify received packet | ${tg_node} | ${src_ip} | ${dst_ip} +| | Then Send TCP or UDP packet and verify received packet | ${tg_node} +| | ... | ${src_ip} | ${dst_ip} | | ... | ${tg_to_dut_if1} | ${tg_to_dut_if1_mac} | | ... | ${tg_to_dut_if2} | ${dut_to_tg_if1_mac} | | ... | TCP | ${src_port} | ${dst_port} | | And Run Keyword And Expect Error | TCP/UDP Rx timeout -| | ... | Send TCP or UDP packet and verify received packet | ${tg_node} | ${src_ip} | ${dst_ip} +| | ... | Send TCP or UDP packet and verify received packet | ${tg_node} +| | ... | ${src_ip} | ${dst_ip} | | ... | ${tg_to_dut_if1} | ${classify_src} | | ... | ${tg_to_dut_if2} | ${dut_to_tg_if1_mac} | | ... | TCP | ${src_port} | ${dst_port} | | And Run Keyword And Expect Error | TCP/UDP Rx timeout -| | ... | Send TCP or UDP packet and verify received packet | ${tg_node} | ${src_ip} | ${dst_ip} +| | ... | Send TCP or UDP packet and verify received packet | ${tg_node} +| | ... | ${src_ip} | ${dst_ip} | | ... | ${tg_to_dut_if1} | ${classify_src2} | | ... | ${tg_to_dut_if2} | ${dut_to_tg_if1_mac} | | ... | TCP | ${src_port} | ${dst_port} @@ -214,7 +220,8 @@ | | ... | ${dut_node} | ${acl_name_mixed} | ${acl_settings} | | And Honeycomb Assigns plugin-acl Chain To Interface | | ... | ${dut_node} | ${dut_to_tg_if1} | ${acl_name_mixed} | ingress -| | Then Send TCP or UDP packet and verify received packet | ${tg_node} | ${src_ip} | ${dst_ip} +| | Then Send TCP or UDP packet and verify received packet | ${tg_node} +| | ... | ${src_ip} | ${dst_ip} | | ... | ${tg_to_dut_if1} | ${tg_to_dut_if1_mac} | | ... | ${tg_to_dut_if2} | ${dut_to_tg_if1_mac} | | ... | TCP | ${src_port} | ${dst_port} @@ -248,18 +255,21 @@ | | ... | ${dut_node} | ${acl_name_icmp} | ${acl_settings} | | And Honeycomb Assigns plugin-acl Chain To Interface | | ... | ${dut_node} | ${dut_to_tg_if1} | ${acl_name_icmp} | ingress -| | Then Send ICMP packet with type and code and verify received packet | ${tg_node} +| | Then Send ICMP packet with type and code and verify received packet +| | ... | ${tg_node} | | ... | ${src_ip} | ${dst_ip} | | ... | ${tg_to_dut_if1} | ${tg_to_dut_if1_mac} | | ... | ${tg_to_dut_if2} | ${dut_to_tg_if1_mac} | | ... | ${icmp_type} | ${icmp_code} -| | Then Send ICMP packet with type and code and verify received packet | ${tg_node} +| | Then Send ICMP packet with type and code and verify received packet +| | ... | ${tg_node} | | ... | ${src_ip} | ${dst_ip} | | ... | ${tg_to_dut_if1} | ${tg_to_dut_if1_mac} | | ... | ${tg_to_dut_if2} | ${dut_to_tg_if1_mac} | | ... | ${classify_type} | ${icmp_code} | | And Run Keyword And Expect Error | ICMP echo Rx timeout -| | ... | Send ICMP packet with type and code and verify received packet | ${tg_node} +| | ... | Send ICMP packet with type and code and verify received packet +| | ... | ${tg_node} | | ... | ${src_ip} | ${dst_ip} | | ... | ${tg_to_dut_if1} | ${tg_to_dut_if1_mac} | | ... | ${tg_to_dut_if2} | ${dut_to_tg_if1_mac} @@ -283,18 +293,21 @@ | | ... | ${dut_node} | ${acl_name_icmpv6} | ${acl_settings} | | And Honeycomb Assigns plugin-acl Chain To Interface | | ... | ${dut_node} | ${dut_to_tg_if1} | ${acl_name_icmpv6} | ingress -| | Then Send ICMP packet with type and code and verify received packet | ${tg_node} +| | Then Send ICMP packet with type and code and verify received packet +| | ... | ${tg_node} | | ... | ${src_ip} | ${dst_ip} | | ... | ${tg_to_dut_if1} | ${tg_to_dut_if1_mac} | | ... | ${tg_to_dut_if2} | ${dut_to_tg_if1_mac} | | ... | ${icmp_type} | ${icmp_code} -| | Then Send ICMP packet with type and code and verify received packet | ${tg_node} +| | Then Send ICMP packet with type and code and verify received packet +| | ... | ${tg_node} | | ... | ${src_ip} | ${dst_ip} | | ... | ${tg_to_dut_if1} | ${tg_to_dut_if1_mac} | | ... | ${tg_to_dut_if2} | ${dut_to_tg_if1_mac} | | ... | ${classify_type} | ${icmp_code} | | And Run Keyword And Expect Error | ICMP echo Rx timeout -| | ... | Send ICMP packet with type and code and verify received packet | ${tg_node} +| | ... | Send ICMP packet with type and code and verify received packet +| | ... | ${tg_node} | | ... | ${src_ip} | ${dst_ip} | | ... | ${tg_to_dut_if1} | ${tg_to_dut_if1_mac} | | ... | ${tg_to_dut_if2} | ${dut_to_tg_if1_mac} @@ -399,8 +412,10 @@ | | ... | ${nodes['TG']} | ${nodes['DUT1']} | ${nodes['TG']} | | And Import Variables | resources/test_data/honeycomb/plugin_acl.py | | ... | L3_IP6 | ${acl_name_l3_ip6} -| | And Honeycomb configures interface state | ${dut_node} | ${dut_to_tg_if1} | up -| | And Honeycomb configures interface state | ${dut_node} | ${dut_to_tg_if2} | up +| | And Honeycomb configures interface state +| | ... | ${dut_node} | ${dut_to_tg_if1} | up +| | And Honeycomb configures interface state +| | ... | ${dut_node} | ${dut_to_tg_if2} | up | | And Honeycomb sets interface IPv6 address | ${dut_node} | | ... | ${dut_to_tg_if1} | ${dut_to_tg_if1_ip} | ${prefix_length} | | And Honeycomb sets interface IPv6 address | ${dut_node} @@ -489,7 +504,8 @@ | | ... | ${dut_node} | ${acl_name_mixed} | ${acl_settings} | | And Honeycomb Assigns plugin-acl Chain To Interface | | ... | ${dut_node} | ${dut_to_tg_if1} | ${acl_name_mixed} | ingress -| | Then Send TCP or UDP packet and verify received packet | ${tg_node} | ${src_ip} | ${dst_ip} +| | Then Send TCP or UDP packet and verify received packet | ${tg_node} +| | ... | ${src_ip} | ${dst_ip} | | ... | ${tg_to_dut_if1} | ${tg_to_dut_if1_mac} | | ... | ${tg_to_dut_if2} | ${dut_to_tg_if1_mac} | | ... | TCP | ${src_port} | ${dst_port} @@ -524,18 +540,21 @@ | | ... | ${dut_node} | ${acl_name_icmp} | ${acl_settings} | | And Honeycomb Assigns plugin-acl Chain To Interface | | ... | ${dut_node} | ${dut_to_tg_if1} | ${acl_name_icmp} | ingress -| | Then Send ICMP packet with type and code and verify received packet | ${tg_node} +| | Then Send ICMP packet with type and code and verify received packet +| | ... | ${tg_node} | | ... | ${src_ip} | ${dst_ip} | | ... | ${tg_to_dut_if1} | ${tg_to_dut_if1_mac} | | ... | ${tg_to_dut_if2} | ${dut_to_tg_if1_mac} | | ... | ${icmp_type} | ${icmp_code} -| | Then Send ICMP packet with type and code and verify received packet | ${tg_node} +| | Then Send ICMP packet with type and code and verify received packet +| | ... | ${tg_node} | | ... | ${src_ip} | ${dst_ip} | | ... | ${tg_to_dut_if1} | ${tg_to_dut_if1_mac} | | ... | ${tg_to_dut_if2} | ${dut_to_tg_if1_mac} | | ... | ${classify_type} | ${icmp_code} | | And Run Keyword And Expect Error | ICMP echo Rx timeout -| | ... | Send ICMP packet with type and code and verify received packet | ${tg_node} +| | ... | Send ICMP packet with type and code and verify received packet +| | ... | ${tg_node} | | ... | ${src_ip} | ${dst_ip} | | ... | ${tg_to_dut_if1} | ${tg_to_dut_if1_mac} | | ... | ${tg_to_dut_if2} | ${dut_to_tg_if1_mac} @@ -558,8 +577,10 @@ | | ... | ${nodes['TG']} | ${nodes['DUT1']} | ${nodes['TG']} | | And Import Variables | resources/test_data/honeycomb/plugin_acl.py | | ... | icmpv6 | ${acl_name_icmpv6} -| | And Honeycomb configures interface state | ${dut_node} | ${dut_to_tg_if1} | up -| | And Honeycomb configures interface state | ${dut_node} | ${dut_to_tg_if2} | up +| | And Honeycomb configures interface state +| | ... | ${dut_node} | ${dut_to_tg_if1} | up +| | And Honeycomb configures interface state +| | ... | ${dut_node} | ${dut_to_tg_if2} | up | | And Honeycomb sets interface IPv6 address | ${dut_node} | | ... | ${dut_to_tg_if1} | ${dut_to_tg_if1_ip} | ${prefix_length} | | And Honeycomb sets interface IPv6 address | ${dut_node} @@ -575,18 +596,21 @@ | | ... | ${dut_node} | ${acl_name_icmpv6} | ${acl_settings} | | And Honeycomb Assigns plugin-acl Chain To Interface | | ... | ${dut_node} | ${dut_to_tg_if1} | ${acl_name_icmpv6} | ingress -| | Then Send ICMP packet with type and code and verify received packet | ${tg_node} +| | Then Send ICMP packet with type and code and verify received packet +| | ... | ${tg_node} | | ... | ${src_ip} | ${dst_ip} | | ... | ${tg_to_dut_if1} | ${tg_to_dut_if1_mac} | | ... | ${tg_to_dut_if2} | ${dut_to_tg_if1_mac} | | ... | ${icmp_type} | ${icmp_code} -| | Then Send ICMP packet with type and code and verify received packet | ${tg_node} +| | Then Send ICMP packet with type and code and verify received packet +| | ... | ${tg_node} | | ... | ${src_ip} | ${dst_ip} | | ... | ${tg_to_dut_if1} | ${tg_to_dut_if1_mac} | | ... | ${tg_to_dut_if2} | ${dut_to_tg_if1_mac} | | ... | ${classify_type} | ${icmp_code} | | And Run Keyword And Expect Error | ICMP echo Rx timeout -| | ... | Send ICMP packet with type and code and verify received packet | ${tg_node} +| | ... | Send ICMP packet with type and code and verify received packet +| | ... | ${tg_node} | | ... | ${src_ip} | ${dst_ip} | | ... | ${tg_to_dut_if1} | ${tg_to_dut_if1_mac} | | ... | ${tg_to_dut_if2} | ${dut_to_tg_if1_mac} diff --git a/tests/func/honeycomb/mgmt-cfg-policer-apihc-func.robot b/tests/func/honeycomb/mgmt-cfg-policer-apihc-func.robot index 1fe4847f23..7afffad4d8 100644 --- a/tests/func/honeycomb/mgmt-cfg-policer-apihc-func.robot +++ b/tests/func/honeycomb/mgmt-cfg-policer-apihc-func.robot @@ -30,9 +30,9 @@ | Library | resources.libraries.python.Trace | Variables | resources/test_data/honeycomb/policer_variables.py | ... -| Suite Teardown -| ... | Run Keyword If Any Tests Failed -| ... | Restart Honeycomb and VPP | ${node} +| Suite Setup | Set Up Honeycomb Functional Test Suite | ${node} +| ... +| Suite Teardown | Tear Down Honeycomb Functional Test Suite | ${node} | ... | Force Tags | HC_FUNC | ... @@ -101,6 +101,8 @@ | | ... | ${node} | ${interface} | ${acl_tables['hc_acl_table']['name']} | TC06: VPP policer 2R3C Color-aware marks packet +# Pending rework +| | [Tags] | EXPECTED_FAILING | | [Documentation] | | ... | [Top] TG=DUT1. | | ... | [Ref] RFC2474, RFC2698. @@ -135,7 +137,7 @@ | | And Honeycomb adds interface IPv4 neighbor | | ... | ${dut_node} | ${dut_to_tg_if2} | ${tg_to_dut_if2_ip} | | ... | ${tg_to_dut_if2_mac} -| | And interfaceCLI.VPP Node Interfaces Ready Wait | ${dut_node} +| | And VPP Node Interfaces Ready Wait | ${dut_node} | | Then Honeycomb Send packet and verify marking | ${tg_node} | | ... | ${tg_to_dut_if1} | | ... | ${tg_to_dut_if2} | ${tg_to_dut_if1_mac} | ${dut_to_tg_if1_mac} diff --git a/tests/func/honeycomb/mgmt-cfg-proxyarp-apihc-func.robot b/tests/func/honeycomb/mgmt-cfg-proxyarp-apihc-func.robot index 26f73b054c..5e162a3e4e 100644 --- a/tests/func/honeycomb/mgmt-cfg-proxyarp-apihc-func.robot +++ b/tests/func/honeycomb/mgmt-cfg-proxyarp-apihc-func.robot @@ -31,9 +31,9 @@ | Resource | resources/libraries/robot/traffic.robot | Library | resources.libraries.python.Trace | ... -| Suite Teardown -| ... | Run Keyword If Any Tests Failed -| ... | Restart Honeycomb And VPP | ${node} +| Suite Setup | Set Up Honeycomb Functional Test Suite | ${node} +| ... +| Suite Teardown | Tear Down Honeycomb Functional Test Suite | ${node} | ... | Force Tags | HC_FUNC | ... diff --git a/tests/func/honeycomb/mgmt-cfg-proxynd6-apihc-func.robot b/tests/func/honeycomb/mgmt-cfg-proxynd6-apihc-func.robot index 6c718e705a..6b69f5b170 100644 --- a/tests/func/honeycomb/mgmt-cfg-proxynd6-apihc-func.robot +++ b/tests/func/honeycomb/mgmt-cfg-proxynd6-apihc-func.robot @@ -35,9 +35,9 @@ | Resource | resources/libraries/robot/dhcp_proxy.robot | Library | resources.libraries.python.Trace | ... -| Test Setup | Clear Packet Trace on All DUTs | ${nodes} +| Suite Setup | Set Up Honeycomb Functional Test Suite | ${node} | ... -| Suite Teardown | Restart Honeycomb And VPP | ${node} +| Suite Teardown | Tear Down Honeycomb Functional Test Suite | ${node} | ... | Force Tags | HC_FUNC | ... diff --git a/tests/func/honeycomb/mgmt-cfg-routing-apihc-apivat-func.robot b/tests/func/honeycomb/mgmt-cfg-routing-apihc-apivat-func.robot index be151c347a..acf203b796 100644 --- a/tests/func/honeycomb/mgmt-cfg-routing-apihc-apivat-func.robot +++ b/tests/func/honeycomb/mgmt-cfg-routing-apihc-apivat-func.robot @@ -24,7 +24,9 @@ | ... | Test Setup | Clear Packet Trace on All DUTs | ${nodes} | ... -| Suite Teardown | Restart Honeycomb And VPP | ${node} +| Suite Setup | Set Up Honeycomb Functional Test Suite | ${node} +| ... +| Suite Teardown | Tear Down Honeycomb Functional Test Suite | ${node} | ... | Test Teardown | Honeycomb routing test teardown | ${node} | ${table} | ... diff --git a/tests/func/honeycomb/mgmt-cfg-slaac-apihc-func.robot b/tests/func/honeycomb/mgmt-cfg-slaac-apihc-func.robot index c5533c27d0..b621e17cb1 100644 --- a/tests/func/honeycomb/mgmt-cfg-slaac-apihc-func.robot +++ b/tests/func/honeycomb/mgmt-cfg-slaac-apihc-func.robot @@ -23,9 +23,9 @@ | Resource | resources/libraries/robot/testing_path.robot | Resource | resources/libraries/robot/traffic.robot | ... -| Suite Teardown -| ... | Run Keyword If Any Tests Failed -| ... | Restart Honeycomb and VPP | ${node} +| Suite Setup | Set Up Honeycomb Functional Test Suite | ${node} +| ... +| Suite Teardown | Tear Down Honeycomb Functional Test Suite | ${node} | ... | Force Tags | HC_FUNC | ... @@ -39,7 +39,7 @@ | | ... | | Given SLAAC Operational Data From Honeycomb Should Be empty | ${node} | | ... | ${interface} -| | And InterfaceAPI.Set Interface State | ${node} | ${interface} | up +| | And Honeycomb Configures Interface State | ${node} | ${interface} | up | | And Honeycomb sets interface IPv6 address | ${node} | ${interface} | | ... | ${address} | ${prefix} | | When Honeycomb configures SLAAC | ${node} | ${interface} | ${slaac_data} @@ -62,7 +62,7 @@ | | ... | | Given SLAAC Operational Data From Honeycomb Should Be empty | ${node} | | ... | ${interface} -| | And InterfaceAPI.Set Interface State | ${node} | ${interface} | up +| | And Honeycomb Configures Interface State | ${node} | ${interface} | up | | And Honeycomb sets interface IPv6 address | ${node} | ${interface} | | ... | ${address} | ${prefix} | | When Honeycomb configures SLAAC | ${node} | ${interface} | ${slaac_data} @@ -77,7 +77,7 @@ | | ... | | Given SLAAC Operational Data From Honeycomb Should Be empty | ${node} | | ... | ${interface} -| | And InterfaceAPI.Set Interface State | ${node} | ${interface} | up +| | And Honeycomb Configures Interface State | ${node} | ${interface} | up | | And Honeycomb sets interface IPv6 address | ${node} | ${interface} | | ... | ${address} | ${prefix} | | When Honeycomb configures SLAAC | ${node} | ${interface} | ${slaac_data_01} @@ -92,7 +92,7 @@ | | ... | | Given SLAAC Operational Data From Honeycomb Should Be empty | ${node} | | ... | ${interface} -| | And InterfaceAPI.Set Interface State | ${node} | ${interface} | up +| | And Honeycomb Configures Interface State | ${node} | ${interface} | up | | And Honeycomb sets interface IPv6 address | ${node} | ${interface} | | ... | ${address} | ${prefix} | | When Honeycomb configures SLAAC | ${node} | ${interface} | ${slaac_data_02} @@ -107,7 +107,7 @@ | | ... | | Given SLAAC Operational Data From Honeycomb Should Be empty | ${node} | | ... | ${interface} -| | And InterfaceAPI.Set Interface State | ${node} | ${interface} | up +| | And Honeycomb Configures Interface State | ${node} | ${interface} | up | | And Honeycomb sets interface IPv6 address | ${node} | ${interface} | | ... | ${address} | ${prefix} | | When Honeycomb configures SLAAC | ${node} | ${interface} | ${slaac_data_03} @@ -115,6 +115,8 @@ | | ... | ${interface} | ${slaac_data_03} | TC07: DUT retransmits RA on IPv6 enabled interface after a set interval +# Traffic test failing in VIRL +| | [Tags] | EXPECTED_FAILING | | [Documentation] | | ... | [Top] TG-DUT1-DUT2-TG. | | ... | [Cfg] On DUT1 configure IPv6 interface on the link to TG. diff --git a/tests/func/honeycomb/mgmt-cfg-snat44-apihc-apivat-func.robot b/tests/func/honeycomb/mgmt-cfg-snat44-apihc-apivat-func.robot index 3c6be394c3..c281d95819 100644 --- a/tests/func/honeycomb/mgmt-cfg-snat44-apihc-apivat-func.robot +++ b/tests/func/honeycomb/mgmt-cfg-snat44-apihc-apivat-func.robot @@ -22,8 +22,9 @@ | ... | Documentation | *Honeycomb NAT test suite.* | ... -| Suite Teardown | Run Keyword If Any Tests Failed -| ... | Restart Honeycomb and VPP | ${node} +| Suite Setup | Set Up Honeycomb Functional Test Suite | ${node} +| ... +| Suite Teardown | Tear Down Honeycomb Functional Test Suite | ${node} | ... | Force Tags | HC_FUNC diff --git a/tests/func/honeycomb/mgmt-cfg-spanrx-apihc-apivat-func.robot b/tests/func/honeycomb/mgmt-cfg-spanrx-apihc-apivat-func.robot index e7fe2eb94e..a24d355382 100644 --- a/tests/func/honeycomb/mgmt-cfg-spanrx-apihc-apivat-func.robot +++ b/tests/func/honeycomb/mgmt-cfg-spanrx-apihc-apivat-func.robot @@ -27,9 +27,11 @@ | ... | Force Tags | HC_FUNC | ... -| Suite Setup | Add Interface local0 To Topology | ${node} +| Suite Setup | Run Keywords +| ... | Set Up Honeycomb Functional Test Suite | ${node} | AND +| ... | Add Interface local0 To Topology | ${node} | ... -| Suite Teardown | Restart Honeycomb and VPP | ${node} +| Suite Teardown | Tear Down Honeycomb Functional Test Suite | ${node} | ... | Documentation | *Honeycomb port mirroring test suite.* @@ -101,6 +103,8 @@ | | ... | ${node} | ${interface2} | ${settings_if2} | TC07: DUT mirrors IPv4 packets from one interface to another +# Pending rework +| | [Tags] | EXPECTED_FAILING | | [Documentation] | | ... | [Top] TG=DUT1 | | ... | [Cfg] (using Honeycomb) On DUT1 configure IPv4 address and set SPAN\ @@ -130,7 +134,7 @@ | | ... | ${tg_to_dut_if1_ip} | ${tg_to_dut_if1_mac} | | ${settings_5}= | create dictionary | state=both | | ... | iface-ref=${dut_to_tg_if1} -| | And InterfaceCLI.All Vpp Interfaces Ready Wait | ${nodes} +| | And All Vpp Interfaces Ready Wait | ${nodes} | | When Honeycomb configures SPAN on interface | | ... | ${node} | ${dut_to_tg_if2} | ${settings_5} | | Then Send Packet And Check Received Copies | ${tg_node} @@ -216,7 +220,9 @@ | | Then Sub-Interface SPAN Operational Data from Honeycomb should be | | ... | ${node} | ${interface2} | ${1} | ${settings_if2} -| TC14: DUT mirrors IPv4 packets from one interface to another +| TC14: DUT mirrors IPv4 packets from an interface to a sub-interface +# Pending rework +| | [Tags] | EXPECTED_FAILING | | [Documentation] | | ... | [Top] TG=DUT1 | | ... | [Cfg] (using Honeycomb) On DUT1 configure IPv4 address and set SPAN\ @@ -245,7 +251,7 @@ | | ... | ${tg_to_dut_if2_ip} | ${tg_to_dut_if2_mac} | | ${settings_5}= | create dictionary | state=both | | ... | iface-ref=${dut_to_tg_if2} -| | And InterfaceCLI.All Vpp Interfaces Ready Wait | ${nodes} +| | And All Vpp Interfaces Ready Wait | ${nodes} | | When Honeycomb Configures SPAN on sub-interface | | ... | ${node} | ${dut_to_tg_if1} | ${1} | ${settings_5} | | Then Send Packet And Check Received Copies | ${tg_node} diff --git a/tests/func/honeycomb/mgmt-cfg-vxlan-apihc-apivat-func.robot b/tests/func/honeycomb/mgmt-cfg-vxlan-apihc-apivat-func.robot index 7d0f5af665..74c18109d1 100644 --- a/tests/func/honeycomb/mgmt-cfg-vxlan-apihc-apivat-func.robot +++ b/tests/func/honeycomb/mgmt-cfg-vxlan-apihc-apivat-func.robot @@ -35,8 +35,9 @@ | ... | Force Tags | HC_FUNC | ... -| Suite Teardown | Run Keyword If Any Tests Failed -| ... | Restart Honeycomb and VPP | ${node} +| Suite Setup | Set Up Honeycomb Functional Test Suite | ${node} +| ... +| Suite Teardown | Tear Down Honeycomb Functional Test Suite | ${node} | ... | Documentation | *Honeycomb VxLAN management test suite.* diff --git a/tests/func/honeycomb/mgmt-cfg-vxlangpe-apihc-apivat-func.robot b/tests/func/honeycomb/mgmt-cfg-vxlangpe-apihc-apivat-func.robot index 8dd884af71..9f1cd86548 100644 --- a/tests/func/honeycomb/mgmt-cfg-vxlangpe-apihc-apivat-func.robot +++ b/tests/func/honeycomb/mgmt-cfg-vxlangpe-apihc-apivat-func.robot @@ -41,9 +41,12 @@ | ... | Documentation | *Honeycomb VxLAN-GPE management test suite.* | ... -#| Force Tags | HC_FUNC +| Suite Setup | Set Up Honeycomb Functional Test Suite | ${node} +| ... +| Suite Teardown | Tear Down Honeycomb Functional Test Suite | ${node} | ... -| Suite Setup | Restart Honeycomb and VPP | ${node} +# Disabled due to VPP-875: Configuring VxLAN GPE tunnel crashes VPP +#| Force Tags | HC_FUNC *** Test Cases *** | TC01: Honeycomb creates VxLAN GPE tunnel diff --git a/tests/func/honeycomb/mgmt-notif-apihcnc-func.robot b/tests/func/honeycomb/mgmt-notif-apihcnc-func.robot index 2f834ac4b6..b943d78ad7 100644 --- a/tests/func/honeycomb/mgmt-notif-apihcnc-func.robot +++ b/tests/func/honeycomb/mgmt-notif-apihcnc-func.robot @@ -25,18 +25,13 @@ | Resource | resources/libraries/robot/honeycomb/tap.robot | Resource | resources/libraries/robot/honeycomb/notifications.robot | ... -| Suite Setup | Run keywords -| ... | Honeycomb configures interface state -| ... | ${node} | ${interface} | down | AND -| ... | Honeycomb creates TAP interface -| ... | ${node} | ${tap_interface} | ${tap_settings} -| ... | Documentation | *Honeycomb notifications test suite.* | ... -| Force Tags | HC_FUNC +| Suite Setup | Set Up Honeycomb Notifications Functional Test Suite +| ... +| Suite Teardown | Tear Down Honeycomb Functional Test Suite | ${node} | ... -| Suite Teardown | Run Keyword If Any Tests Failed -| ... | Restart Honeycomb and VPP | ${node} +| Force Tags | HC_FUNC *** Test Cases *** | TC01: Honeycomb sends notification on interface state change @@ -63,3 +58,11 @@ | | And Notification listener should be established | ${node} | | When Honeycomb removes TAP interface | ${node} | ${tap_interface} | | Then Honeycomb should send interface deleted notification | ${tap_interface} + +*** Keywords *** +| Set Up Honeycomb Notifications Functional Test Suite +| | Set Up Honeycomb Functional Test Suite | ${node} +| | Honeycomb configures interface state +| | ... | ${node} | ${interface} | down +| | Honeycomb creates TAP interface +| | ... | ${node} | ${tap_interface} | ${tap_settings}
\ No newline at end of file diff --git a/tests/func/honeycomb/mgmt-statepersist-apihc-func.robot b/tests/func/honeycomb/mgmt-statepersist-apihc-func.robot index 90ed077533..4277c91a4e 100644 --- a/tests/func/honeycomb/mgmt-statepersist-apihc-func.robot +++ b/tests/func/honeycomb/mgmt-statepersist-apihc-func.robot @@ -39,7 +39,8 @@ | | [Documentation] | Checks if Honeycomb maintains configuration after both\ | | ... | Restart Honeycomb and VPP. | | ... -| | [Tags] | HC_FUNC +# Failing due to HC2VPP-47 +| | [Tags] | HC_FUNC | EXPECTED_FAILING | | ... | | [Teardown] | | ... | Restart Honeycomb And VPP And Clear Persisted Configuration | ${node} |