diff options
author | selias <samelias@cisco.com> | 2017-01-17 16:59:03 +0100 |
---|---|---|
committer | Peter Mikus <pmikus@cisco.com> | 2017-01-27 14:54:19 +0000 |
commit | fc42ade3d5c7ed279e867cb55ca42a44db5e706d (patch) | |
tree | efe592ce21ea62d3c6723a994a10eb707966fa11 | |
parent | c53c0cbad736e937a4a41bf13eff4c5d8b5dcb55 (diff) |
CSIT-506: HC Test- Update and cleanup all suites
- remove EXPECTED_FAILING tag from IPv4 neighbor, IPv6 address, MTU
- fix bridge domain removal with interfaces assigned, it should fail
- fix teardown of L2 FIB suite (bridge domain removal, see above)
- disable vhost-user "server" test cases (VPP bug)
- fix keyword verifying sub-interface state
- update ACL test data (yang model changes)
- remove EXPECTED_FAILING tag from ACL table removal test
- update Jira IDs and comments in failing Lisp test case
- remove EXPECTED_FAILING tag from Lisp removal test case
- use vhost-user "client" instead of "server" in persistence tests
Change-Id: I32eafb6013b4512090c0d9365e10c61029179d49
Signed-off-by: selias <samelias@cisco.com>
25 files changed, 155 insertions, 124 deletions
diff --git a/resources/libraries/python/NAT.py b/resources/libraries/python/NAT.py index d0e01c7f84..e30381682c 100644 --- a/resources/libraries/python/NAT.py +++ b/resources/libraries/python/NAT.py @@ -64,7 +64,7 @@ class NATUtil(object): "vrf": items[4] }) else: - raise RuntimeError("Unexpected output from span_mapping_dump.") + raise RuntimeError("Unexpected output from snat_mapping_dump.") return data @@ -90,8 +90,9 @@ class NATUtil(object): data = [] for line in lines: items = line.split(" ") - while "" in items: - items.remove("") + for trash in ("", "vat#"): + while trash in items: + items.remove(trash) if len(items) == 0: continue elif len(items) == 3: @@ -102,6 +103,6 @@ class NATUtil(object): }) else: raise RuntimeError( - "Unexpected output from span_interface_dump.") + "Unexpected output from snat_interface_dump.") return data diff --git a/resources/libraries/python/honeycomb/HcAPIKwInterfaces.py b/resources/libraries/python/honeycomb/HcAPIKwInterfaces.py index 1e0883db27..a62cee85c7 100644 --- a/resources/libraries/python/honeycomb/HcAPIKwInterfaces.py +++ b/resources/libraries/python/honeycomb/HcAPIKwInterfaces.py @@ -1143,7 +1143,7 @@ class InterfaceKeywords(object): path = ("interfaces", ("interface", "name", super_interface), "vpp-vlan:sub-interfaces", - ("sub-interface", "identifier", identifier), + ("sub-interface", "identifier", int(identifier)), "enabled") return InterfaceKeywords._set_interface_properties( @@ -1375,10 +1375,11 @@ class InterfaceKeywords(object): else: raise HoneycombError( - "Unexpected data type {data_type}, reference type is" - " {ref_type}. Must be list or dictionary.".format( + "Unexpected data type {data_type} in path {path}, reference" + " type is {ref_type}. Must be list or dictionary.".format( data_type=type(data), - ref_type=type(ref))) + ref_type=type(ref), + path=_path)) @staticmethod def compare_interface_lists(list1, list2): diff --git a/resources/libraries/robot/honeycomb/interfaces.robot b/resources/libraries/robot/honeycomb/interfaces.robot index 765aaef7c0..3c91c77eb7 100644 --- a/resources/libraries/robot/honeycomb/interfaces.robot +++ b/resources/libraries/robot/honeycomb/interfaces.robot @@ -220,25 +220,23 @@ | IPv4 neighbor from Honeycomb should be | | [Documentation] | Retrieves ipv4 neighbor list through Honeycomb\ -| | ... | and compares with neighbor list supplied in argument. +| | ... | and compares the first entry with addresses supplied in arguments. | | ... | | ... | *Arguments:* | | ... | - node - information about a DUT node. Type: dictionary | | ... | - interface - name of an interface on the specified node. Type: string -| | ... | - neighbors - list of ipv4 neighbor dictionaries. Type: list +| | ... | - ip_address - ipv4 address of expected neighbor entry. Type: string +| | ... | - mac_address - MAC address of expected neighbor entry. Type: string | | ... | | ... | *Example:* | | ... | | ... | \| IPv4 neighbor from Honeycomb should be \| ${nodes['DUT1']} \ | | ... | \| GigabitEthernet0/8/0 \| 192.168.0.4 \| 08:00:27:60:26:ab \| -| | [Arguments] | ${node} | ${interface} | @{neighbors} +| | [Arguments] | ${node} | ${interface} | ${ip_address} | ${mac_address} | | ${api_data}= | interfaceAPI.Get interface oper data | ${node} | ${interface} -| | ${data_neighbors}= | Set Variable | ${api_data['ietf-ip:ipv4']['neighbor']} -| | Compare data structures -| | ... | ${data_neighbors} | ${neighbors} -| | Should be equal | ${neighbor['fib_address']} +| | Should be equal | ${ip_address} | | ... | ${api_data['ietf-ip:ipv4']['neighbor'][0]['ip']} -| | Should be equal | ${neighbor['fib_mac']} +| | Should be equal | ${mac_address} | | ... | ${api_data['ietf-ip:ipv4']['neighbor'][0]['link-layer-address']} | Honeycomb clears all interface ipv4 neighbors @@ -290,11 +288,9 @@ | | ... | \| GigabitEthernet0/8/0 \| 10::10 \| 64 \| | | [Arguments] | ${node} | ${interface} | ${address} | ${prefix} | | ${api_data}= | interfaceAPI.Get interface oper data | ${node} | ${interface} -| | Should be equal | ${address} -| | ... | ${api_data['ietf-ip:ipv6']['address'][0]['ip']} -| | Should be equal | ${prefix} -| | ... | ${api_data['ietf-ip:ipv6']['address'][0]['prefix-length']} -| | Should be equal | ${fib_address} +| | ${settings}= | Create Dictionary +| | ... | ip=${address} | prefix-length=${prefix} +| | Should contain | ${api_data['ietf-ip:ipv6']['address']} | ${settings} | IPv6 address from VAT should be | | [Documentation] | Retrieves interface ipv6 address through VAT and\ @@ -313,8 +309,9 @@ | | [Arguments] | ${node} | ${interface} | ${address} | ${prefix} | | ${vpp_data}= | interfaceCLI.VPP get interface ip addresses | | ... | ${node} | ${interface} | ipv6 -| | Should be equal | ${vpp_data[0]['ip']} | ${address} -| | Should be equal | ${vpp_data[0]['prefix-length']} | ${prefix} +| | ${settings}= | Create Dictionary +| | ... | ip=${address} | prefix_length=${prefix} +| | Should contain | ${vpp_data} | ${settings} | Honeycomb sets interface ethernet configuration | | [Documentation] | Uses Honeycomb API to change interface ethernet\ diff --git a/resources/libraries/robot/honeycomb/nat.robot b/resources/libraries/robot/honeycomb/nat.robot index af24d56e37..17cda4763e 100644 --- a/resources/libraries/robot/honeycomb/nat.robot +++ b/resources/libraries/robot/honeycomb/nat.robot @@ -171,4 +171,4 @@ | | ... | \| ${nodes['DUT1']} \| ${settings} \| | | [Arguments] | ${node} | ${settings} | | ${data}= | VPP get NAT interfaces | ${node} -| | Compare data structures | ${data[0]} | ${settings} +| | Compare data structures | ${data} | ${settings} diff --git a/resources/libraries/robot/honeycomb/persistence.robot b/resources/libraries/robot/honeycomb/persistence.robot index 84607a8991..aac920914c 100644 --- a/resources/libraries/robot/honeycomb/persistence.robot +++ b/resources/libraries/robot/honeycomb/persistence.robot @@ -103,7 +103,7 @@ | | Honeycomb creates TAP interface | | ... | ${node} | ${tap_interface} | ${tap_settings} | | Honeycomb creates vhost-user interface -| | ... | ${node} | ${vhost_interface} | ${vhost_user_server} +| | ... | ${node} | ${vhost_interface} | ${vhost_user_client} | | Honeycomb creates sub-interface | ${node} | ${interface} | | ... | ${sub_if_1_match} | ${sub_if_1_tags} | ${sub_if_1_settings} | | Honeycomb sets interface state | ${node} | ${interface} | up @@ -140,9 +140,9 @@ | | TAP configuration from VAT should be | | ... | ${node} | ${tap_interface} | ${tap_settings} | | Vhost-user configuration from Honeycomb should be -| | ... | ${node} | ${vhost_interface} | ${vhost_user_server} +| | ... | ${node} | ${vhost_interface} | ${vhost_user_client} | | Vhost-user configuration from VAT should be -| | ... | ${node} | ${vhost_user_server} +| | ... | ${node} | ${vhost_user_client} | | Sub-interface configuration from Honeycomb should be | | ... | ${node} | ${interface} | ${sub_if_id} | ${sub_if_1_oper} | | Sub-interface configuration from VAT should be diff --git a/resources/test_data/honeycomb/acl.py b/resources/test_data/honeycomb/acl.py index ee7f9525b8..46b88dad83 100644 --- a/resources/test_data/honeycomb/acl.py +++ b/resources/test_data/honeycomb/acl.py @@ -17,6 +17,7 @@ hc_acl_table = { "name": "acl_table_test", "nbuckets": 1, + "memory_size": 1048576, "skip_n_vectors": 0, "miss_next": "permit", "mask": "00:00:00:00:00:00:ff:ff:ff:ff:ff:ff:00:00:00:00" @@ -25,11 +26,31 @@ hc_acl_table = { hc_acl_table2 = { "name": "acl_table_test2", "nbuckets": 2, + "memory_size": 1048576, "skip_n_vectors": 1, "next_table": "acl_table_test", "miss_next": "deny", "mask": "ff:ff:ff:00:00:00:ff:ff:ff:ff:ff:ff:00:00:00:00" } + +#TODO: remove once memory_size is visible in oper data(HC2VPP-10) +hc_acl_table_oper = { + "name": "acl_table_test", + "nbuckets": 1, + "skip_n_vectors": 0, + "miss_next": "permit", + "mask": "00:00:00:00:00:00:ff:ff:ff:ff:ff:ff:00:00:00:00" +} + +hc_acl_table2_oper = { + "name": "acl_table_test2", + "nbuckets": 2, + "skip_n_vectors": 1, + "next_table": "acl_table_test", + "miss_next": "deny", + "mask": "ff:ff:ff:00:00:00:ff:ff:ff:ff:ff:ff:00:00:00:00" +} + # representation of table settings in VAT table_index = 0 vat_acl_table = { diff --git a/resources/test_data/honeycomb/l2_fib.py b/resources/test_data/honeycomb/l2_fib.py index d1600fc46d..c8fabdc943 100644 --- a/resources/test_data/honeycomb/l2_fib.py +++ b/resources/test_data/honeycomb/l2_fib.py @@ -119,7 +119,6 @@ def get_variables(node, interface, interface2): # Configuration data: 'l2_fib_filter_cfg': { "phys-address": filtered, - "outgoing-interface": interface, "static-config": True, "action": "l2-fib-filter" }, @@ -127,7 +126,6 @@ def get_variables(node, interface, interface2): # Expected operational data: 'l2_fib_filter_oper': { "phys-address": filtered, - "outgoing-interface": interface, "bridged-virtual-interface": False, "action": "v3po:l2-fib-filter", "static-config": True @@ -136,7 +134,6 @@ def get_variables(node, interface, interface2): # Expected VAT data: 'l2_fib_filter_vat': { "mac": int("".join(filtered.split(':')), 16), - "sw_if_index": sw_if_index, "static_mac": 1, "filter_mac": 1, "bvi_mac": 0 diff --git a/resources/test_data/honeycomb/lisp.py b/resources/test_data/honeycomb/lisp.py index 049d222b7b..12a614698c 100644 --- a/resources/test_data/honeycomb/lisp.py +++ b/resources/test_data/honeycomb/lisp.py @@ -108,25 +108,61 @@ lisp_settings_enable = { } } -remote_vrf_adjacency = { - "adjacency": { +prepare_vrf_adjacency = { + "virtual-network-identifier": 7, + "vrf-subtable": { + "table-id": 3, + "local-mappings": { + "local-mapping": [{ + "id": "local_map_vrf", + "eid": { + "virtual-network-id": 7, + "address-type": "ietf-lisp-address-types:ipv4-afi", + "ipv4": "192.168.1.1" + }, + "locator-set": locator_set + }] + }, + "remote-mappings": { + "remote-mapping": [{ + "id": "remote_map_vrf", + "eid": { + "virtual-network-id": 7, + "address-type": "ietf-lisp-address-types:ipv4-afi", + "ipv4": "192.168.0.2" + }, + "rlocs": { + "locator": [{ + "address": "192.168.0.3", + "priority": 1, + "weight": 1 + }] + }, + + }] + }, + } +} + +vrf_adjacency = { + "adjacency": [{ "id": "adj01", "local-eid": { - "virtual-network-id": 4, + "virtual-network-id": 7, "address-type": "ietf-lisp-address-types:ipv4-afi", "ipv4": "192.168.1.1" }, "remote-eid": { - "virtual-network-id": 4, + "virtual-network-id": 7, "address-type": "ietf-lisp-address-types:ipv4-afi", "ipv4": "192.168.0.2" }, - } + }] } -remote_adj_subtable = deepcopy(remote_vrf_subtable) -remote_adj_subtable["vrf-subtable"]["remote-mappings"]\ - ["remote-mapping"][0]["adjacencies"] = {}.update(remote_vrf_adjacency) +adj_subtable = deepcopy(prepare_vrf_adjacency) +adj_subtable["vrf-subtable"]["remote-mappings"]\ + ["remote-mapping"][0]["adjacencies"] = deepcopy(vrf_adjacency) def create_settings_dict(subtable): @@ -142,6 +178,7 @@ lisp_settings_remote_bd = create_settings_dict(remote_bd_subtable) lisp_settings_remote_vrf = create_settings_dict(remote_vrf_subtable) lisp_settings_local_bd = create_settings_dict(local_bd_subtable) lisp_settings_local_vrf = create_settings_dict(local_vrf_subtable) +lisp_settings_both_vrf = create_settings_dict(prepare_vrf_adjacency) vat_remote_bd = { "is_local": 0, diff --git a/resources/test_data/honeycomb/nat.py b/resources/test_data/honeycomb/nat.py index d56fcfff56..fe2894b469 100644 --- a/resources/test_data/honeycomb/nat.py +++ b/resources/test_data/honeycomb/nat.py @@ -87,11 +87,11 @@ def get_variables(node, interface): } ], "nat_interface_vat_in": [ - {"sw_if_index": sw_if_index, + {"sw_if_index": str(sw_if_index), "direction": "in"} ], "nat_interface_vat_out": [ - {"sw_if_index": sw_if_index, + {"sw_if_index": str(sw_if_index), "direction": "out"} ] } diff --git a/resources/test_data/honeycomb/persistence.py b/resources/test_data/honeycomb/persistence.py index f46cfba597..216d4facef 100644 --- a/resources/test_data/honeycomb/persistence.py +++ b/resources/test_data/honeycomb/persistence.py @@ -50,8 +50,8 @@ def get_variables(interface): }, # vhost-user interface settings 'vhost_interface': 'test_vhost', - 'vhost_user_server': {'socket': 'soc1', - 'role': 'server' + 'vhost_user_client': {'socket': 'soc1', + 'role': 'client' }, # Vlan subinterface settings 'sub_if_id': sub_if_id, diff --git a/tests/func/honeycomb/mgmt-cfg-acl-apihc-apivat-func.robot b/tests/func/honeycomb/mgmt-cfg-acl-apihc-apivat-func.robot index 98ff3bda13..fdfc281d82 100644 --- a/tests/func/honeycomb/mgmt-cfg-acl-apihc-apivat-func.robot +++ b/tests/func/honeycomb/mgmt-cfg-acl-apihc-apivat-func.robot @@ -36,13 +36,13 @@ | | ... | ${node} | ${table_index} | | When Honeycomb creates ACL table | | ... | ${node} | ${hc_acl_table} -| | Then ACL table from Honeycomb should be | ${node} | ${hc_acl_table} +| | Then ACL table from Honeycomb should be | ${node} | ${hc_acl_table_oper} | | And ACL table from VAT should be | | ... | ${node} | ${table_index} | ${vat_acl_table} | TC02: Honeycomb can remove ACL table | | [Documentation] | Check if Honeycomb API can delete an ACL table. -| | Given ACL table from Honeycomb should be | ${node} | ${hc_acl_table} +| | Given ACL table from Honeycomb should be | ${node} | ${hc_acl_table_oper} | | And ACL table from VAT should be | | ... | ${node} | ${table_index} | ${vat_acl_table} | | When Honeycomb removes ACL table | ${node} | ${hc_acl_table['name']} @@ -59,16 +59,16 @@ | | ... | ${node} | ${table_index} | | When Honeycomb creates ACL table | ${node} | ${hc_acl_table} | | And Honeycomb creates ACL table | ${node} | ${hc_acl_table2} -| | Then ACL table from Honeycomb should be | ${node} | ${hc_acl_table} +| | Then ACL table from Honeycomb should be | ${node} | ${hc_acl_table_oper} | | And ACL table from VAT should be | | ... | ${node} | ${table_index} | ${vat_acl_table} -| | And ACL table from Honeycomb should be | ${node} | ${hc_acl_table2} +| | And ACL table from Honeycomb should be | ${node} | ${hc_acl_table2_oper} | | And ACL table from VAT should be | | ... | ${node} | ${table_index2} | ${vat_acl_table2} | TC04: Honeycomb can add ACL session to table | | [Documentation] | Check if Honeycomb API can add an ACL session to a table. -| | Given ACL table from Honeycomb should be | ${node} | ${hc_acl_table} +| | Given ACL table from Honeycomb should be | ${node} | ${hc_acl_table_oper} | | And ACL table from VAT should be | | ... | ${node} | ${table_index} | ${vat_acl_table} | | When Honeycomb adds ACL session @@ -113,7 +113,7 @@ | TC07: Honeycomb enables ACL on interface | | [Documentation] | Check if Honeycomb API can enable ACL on an interface. -| | Given ACL table from Honeycomb should be | ${node} | ${hc_acl_table} +| | Given ACL table from Honeycomb should be | ${node} | ${hc_acl_table_oper} | | And ACL table from VAT should be | | ... | ${node} | ${table_index} | ${vat_acl_table} | | And ACL session from Honeycomb should be @@ -142,16 +142,14 @@ | TC09: Honeycomb can remove one out of multiple ACL tables | | [Documentation] | Check if Honeycomb API can delete an ACL table if more\ | | ... | than one table exists. -# Attempting to remove one ACL table removes all of them (VPP-206) -| | [Tags] | EXPECTED_FAILING -| | Given ACL table from Honeycomb should be | ${node} | ${hc_acl_table} +| | Given ACL table from Honeycomb should be | ${node} | ${hc_acl_table_oper} | | And ACL table from VAT should be | | ... | ${node} | ${table_index} | ${vat_acl_table} -| | And ACL table from Honeycomb should be | ${node} | ${hc_acl_table2} +| | And ACL table from Honeycomb should be | ${node} | ${hc_acl_table2_oper} | | And ACL table from VAT should be | | ... | ${node} | ${table_index2} | ${vat_acl_table2} | | When Honeycomb removes ACL table | ${node} | ${hc_acl_table2['name']} -| | Then ACL table from Honeycomb should be | ${node} | ${hc_acl_table} +| | Then ACL table from Honeycomb should be | ${node} | ${hc_acl_table_oper} | | And ACL table from VAT should be | | ... | ${node} | ${table_index} | ${vat_acl_table} | | And ACL table from Honeycomb should not exist diff --git a/tests/func/honeycomb/mgmt-cfg-ietfacl-apihc-apivat-func.robot b/tests/func/honeycomb/mgmt-cfg-ietfacl-apihc-apivat-func.robot index a0b967980a..4d4785a6df 100644 --- a/tests/func/honeycomb/mgmt-cfg-ietfacl-apihc-apivat-func.robot +++ b/tests/func/honeycomb/mgmt-cfg-ietfacl-apihc-apivat-func.robot @@ -47,6 +47,7 @@ | ... | Restart Honeycomb And VPP And Clear Persisted Configuration | ${node} | Documentation | *Honeycomb access control lists test suite for IETF-ACL node.* # Test suite out of date since https://gerrit.fd.io/r/#/c/4331/ +# Ietf-ACL fucntionality will be removed # | Force Tags | Honeycomb_sanity *** Test Cases *** diff --git a/tests/func/honeycomb/mgmt-cfg-int-subint-apihc-apivat-func.robot b/tests/func/honeycomb/mgmt-cfg-int-subint-apihc-apivat-func.robot index 9e2216b6a4..9ee12e1a18 100644 --- a/tests/func/honeycomb/mgmt-cfg-int-subint-apihc-apivat-func.robot +++ b/tests/func/honeycomb/mgmt-cfg-int-subint-apihc-apivat-func.robot @@ -24,9 +24,6 @@ | ... | AND | Honeycomb removes all bridge domains | ${node} | Force Tags | honeycomb_sanity | Documentation | *Honeycomb sub-interface management test suite.* -| ... -| ... | This test suite tests if it is posible to create, modify and \ -| ... | delete a sub-interface. *** Variables *** # Test interface 1 and its sub-interface parameters: diff --git a/tests/func/honeycomb/mgmt-cfg-intip4-intip6-apihc-apivat-func.robot b/tests/func/honeycomb/mgmt-cfg-intip4-intip6-apihc-apivat-func.robot index 4ffac6d438..6675d18d9c 100644 --- a/tests/func/honeycomb/mgmt-cfg-intip4-intip6-apihc-apivat-func.robot +++ b/tests/func/honeycomb/mgmt-cfg-intip4-intip6-apihc-apivat-func.robot @@ -38,8 +38,6 @@ | Suite Teardown | Run Keyword If Any Tests Failed | | ... | Restart Honeycomb And VPP And Clear Persisted Configuration | ${node} | Documentation | *Honeycomb interface management test suite.* -| ... -| ... | Test suite uses the first interface of the first DUT node. *** Test Cases *** | TC01: Honeycomb configures and reads interface state @@ -96,8 +94,6 @@ | TC05: Honeycomb modifies IPv4 neighbor table | | [Documentation] | Check if Honeycomb API can add and remove ARP entries. -# Operational data and VAT dump not available (HONEYCOMB-111) -| | [Tags] | EXPECTED_FAILING | | [Teardown] | Honeycomb clears all interface ipv4 neighbors | | ... | ${node} | ${interface} | | When Honeycomb adds interface ipv4 neighbor @@ -107,8 +103,6 @@ | TC06: Honeycomb modifies interface configuration - IPv6 | | [Documentation] | Check if Honeycomb API can configure interfaces for ipv6. -# Configuring IPv6 not implemented (HONEYCOMB-102) -| | [Tags] | EXPECTED_FAILING | | When Honeycomb sets interface ipv6 address | | ... | ${node} | ${interface} | @{ipv6_address} | | Then IPv6 address from Honeycomb should be @@ -121,8 +115,6 @@ | TC07: Honeycomb modifies interface configuration - MTU | | [Documentation] | Check if Honeycomb API can configure interface\ | | ... | MTU value. -# Configuring MTU not implemented (HONEYCOMB-126) -| | [Tags] | EXPECTED_FAILING | | When Honeycomb sets interface ethernet configuration | | ... | ${node} | ${interface} | ${ethernet} | | Then Interface ethernet configuration from Honeycomb should be diff --git a/tests/func/honeycomb/mgmt-cfg-inttap-apihc-apivat-func.robot b/tests/func/honeycomb/mgmt-cfg-inttap-apihc-apivat-func.robot index 04f94b81c0..05ca9dc28a 100644 --- a/tests/func/honeycomb/mgmt-cfg-inttap-apihc-apivat-func.robot +++ b/tests/func/honeycomb/mgmt-cfg-inttap-apihc-apivat-func.robot @@ -30,8 +30,6 @@ | Suite Teardown | Run Keyword If Any Tests Failed | ... | Restart Honeycomb And VPP And Clear Persisted Configuration | ${node} | Documentation | *Honeycomb TAP management test suite.* -| ... -| ... | Test suite uses the first interface of the first DUT node. *** Test Cases *** | TC01: Honeycomb configures TAP interface diff --git a/tests/func/honeycomb/mgmt-cfg-intvhost-apihc-apivat-func.robot b/tests/func/honeycomb/mgmt-cfg-intvhost-apihc-apivat-func.robot index f96187278e..cbc9b3fb6f 100644 --- a/tests/func/honeycomb/mgmt-cfg-intvhost-apihc-apivat-func.robot +++ b/tests/func/honeycomb/mgmt-cfg-intvhost-apihc-apivat-func.robot @@ -14,29 +14,26 @@ *** Variables *** | ${interface}= | ${node['interfaces']['port1']['name']} | ${vhost_interface}= | test_vhost -| &{vhost_user_server}= | socket=soc1 | role=server -| &{vhost_user_server_edit_1}= | socket=soc12 | role=server -| &{vhost_user_server_edit_2}= | socket=soc12 | role=client -| &{vhost_user_client}= | socket=soc2 | role=client -| &{vhost_user_client_edit_1}= | socket=soc22 | role=client -| &{vhost_user_client_edit_2}= | socket=soc22 | role=server -| &{vhost_user_wrong}= | socket=soc2 | role=wrong +| &{vhost_user_server}= | socket=/tmp/soc1 | role=server +| &{vhost_user_server_edit_1}= | socket=/tmp/soc12 | role=server +| &{vhost_user_server_edit_2}= | socket=/tmp/soc12 | role=client +| &{vhost_user_client}= | socket=/tmp/soc2 | role=client +| &{vhost_user_client_edit_1}= | socket=/tmp/soc22 | role=client +| &{vhost_user_client_edit_2}= | socket=/tmp/soc22 | role=server +| &{vhost_user_wrong}= | socket=/tmp/soc2 | role=wrong *** Settings *** | Resource | resources/libraries/robot/default.robot | Resource | resources/libraries/robot/honeycomb/honeycomb.robot | Resource | resources/libraries/robot/honeycomb/vhost_user.robot -# Whole suite failing due to bug https://jira.fd.io/browse/VPP-562 +# vhost-user as server failing due to VPP bug (VPP-562) | Force Tags | honeycomb_sanity | EXPECTED_FAILING | Suite Teardown | Run Keyword If Any Tests Failed | ... | Restart Honeycomb And VPP And Clear Persisted Configuration | ${node} | Documentation | *Honeycomb vhost-user interface management test suite.* -| ... -| ... | This test suite tests if it is posible to create, modify and\ -| ... | delete a vhost-user interface. *** Test Cases *** -| TC01: Honycomb creates vhost-user interface - server +| TC01: Honeycomb creates vhost-user interface - server | | [Documentation] | Check if Honeycomb creates a vhost-user interface, role:\ | | ... | server. | | ... @@ -49,7 +46,7 @@ | | And vhost-user configuration from VAT should be | | ... | ${node} | ${vhost_user_server} -| TC02: Honycomb modifies vhost-user interface - server +| TC02: Honeycomb modifies vhost-user interface - server | | [Documentation] | Check if Honeycomb can modify properties of existing\ | | ... | vhost-user interface, role: server. | | ... @@ -74,7 +71,7 @@ | | And vhost-user configuration from VAT should be | | ... | ${node} | ${vhost_user_server} -| TC03: Honycomb deletes vhost-user interface - server +| TC03: Honeycomb deletes vhost-user interface - server | | [Documentation] | Check if Honeycomb can delete an existing vhost-user\ | | ... | interface, role: server. | | ... @@ -87,7 +84,7 @@ | | And vhost-user configuration from VAT should be empty | | ... | ${node} -| TC04: Honycomb creates vhost-user interface - client +| TC04: Honeycomb creates vhost-user interface - client | | [Documentation] | Check if Honeycomb creates a vhost-user interface, role:\ | | ... | client. | | ... @@ -100,7 +97,7 @@ | | And vhost-user configuration from VAT should be | | ... | ${node} | ${vhost_user_client} -| TC05: Honycomb modifies vhost-user interface - client +| TC05: Honeycomb modifies vhost-user interface - client | | [Documentation] | Check if Honeycomb can modify properties of existing\ | | ... | vhost-user interface, role: client. | | ... @@ -125,7 +122,7 @@ | | And vhost-user configuration from VAT should be | | ... | ${node} | ${vhost_user_client} -| TC06: Honycomb deletes vhost-user interface - client +| TC06: Honeycomb deletes vhost-user interface - client | | [Documentation] | Check if Honeycomb can delete an existing vhost-user\ | | ... | interface, role: client. | | ... diff --git a/tests/func/honeycomb/mgmt-cfg-l2bd-apihc-apivat-func.robot b/tests/func/honeycomb/mgmt-cfg-l2bd-apihc-apivat-func.robot index 3a32f02136..05440ba9dc 100644 --- a/tests/func/honeycomb/mgmt-cfg-l2bd-apihc-apivat-func.robot +++ b/tests/func/honeycomb/mgmt-cfg-l2bd-apihc-apivat-func.robot @@ -34,8 +34,6 @@ | ... | AND | Honeycomb removes all bridge domains | ${node} | @{interfaces} | Force Tags | honeycomb_sanity | Documentation | *Honeycomb bridge domain management test suite.* -| ... -| ... | Test suite uses the first two interfaces on the first DUT node. *** Test Cases *** | TC01: Honeycomb sets up l2 bridge domain @@ -79,22 +77,33 @@ | | ... | ${node} | ${bd1_name} | ${bd_settings} | | When Honeycomb adds interfaces to bridge domain | | ... | ${node} | @{interfaces} | ${bd1_name} | ${if_settings} -| | Then Honeycomb should show interfaces assigned to bridge domain +| | Then Bridge domain configuration from Honeycomb should be +| | ... | ${node} | ${bd1_name} | ${bd_settings} +| | And Bridge domain configuration from VAT should be +| | ... | ${node} | ${0} | ${bd_settings} +| | And Honeycomb should show interfaces assigned to bridge domain | | ... | ${node} | @{interfaces} | ${bd1_name} | ${if_settings} | | And VAT should show interfaces assigned to bridge domain | | ... | ${node} | ${0} | @{interfaces} | ${if_settings} -| TC05: Honeycomb removes bridge domain with an interface assigned +| TC05: Honeycomb cannot remove bridge domain with an interface assigned | | [Documentation] | Check if Honeycomb can remove a bridge domain that has an\ -| | ... | interface assigned to it. -# Bridge domain references not cleaned up on delete (HONEYCOMB-267) -| | [Tags] | EXPECTED_FAILING -| | Given Honeycomb should show interfaces assigned to bridge domain +| | ... | interface assigned to it. Expect to fail with code 500. +| | Given Bridge domain configuration from Honeycomb should be +| | ... | ${node} | ${bd1_name} | ${bd_settings} +| | And Bridge domain configuration from VAT should be +| | ... | ${node} | ${0} | ${bd_settings} +| | And Honeycomb should show interfaces assigned to bridge domain +| | ... | ${node} | @{interfaces} | ${bd1_name} | ${if_settings} +| | And VAT should show interfaces assigned to bridge domain +| | ... | ${node} | ${0} | @{interfaces} | ${if_settings} +| | When Run keyword and expect error | HoneycombError* Status code: 500. +| | ... | Honeycomb removes all bridge domains | ${node} +| | Then Bridge domain configuration from Honeycomb should be +| | ... | ${node} | ${bd1_name} | ${bd_settings} +| | And Bridge domain configuration from VAT should be +| | ... | ${node} | ${0} | ${bd_settings} +| | And Honeycomb should show interfaces assigned to bridge domain | | ... | ${node} | @{interfaces} | ${bd1_name} | ${if_settings} | | And VAT should show interfaces assigned to bridge domain | | ... | ${node} | ${0} | @{interfaces} | ${if_settings} -| | When Honeycomb removes all bridge domains | ${node} -| | Then Honeycomb should show no bridge domains | ${node} -| | And VAT should show no bridge domains | ${node} -| | And Honeycomb should not show interfaces assigned to bridge domain -| | ... | ${node} | @{interfaces} | ${bd1_name} diff --git a/tests/func/honeycomb/mgmt-cfg-l2fib-apihc-apivat-func.robot b/tests/func/honeycomb/mgmt-cfg-l2fib-apihc-apivat-func.robot index 9a299df231..25891dd470 100644 --- a/tests/func/honeycomb/mgmt-cfg-l2fib-apihc-apivat-func.robot +++ b/tests/func/honeycomb/mgmt-cfg-l2fib-apihc-apivat-func.robot @@ -22,12 +22,12 @@ | Documentation | *Honeycomb L2 FIB management test suite.* | Suite Setup | Run keywords | ... | Set test interface down -| ... | AND -| ... | Honeycomb removes all bridge domains | ${node} | ${interface} | Suite Teardown | Run keywords | ... | Run Keyword If Any Tests Failed | ... | Restart Honeycomb And VPP And Clear Persisted Configuration | ${node} -| ... | AND | Honeycomb removes all bridge domains | ${node} +| ... | AND +| ... | Honeycomb removes all bridge domains +| ... | ${node} | ${interface} | ${interface2} | Force tags | honeycomb_sanity *** Variables *** diff --git a/tests/func/honeycomb/mgmt-cfg-lisp-apihc-apivat-func.robot b/tests/func/honeycomb/mgmt-cfg-lisp-apihc-apivat-func.robot index 5f602c5780..2356e8b00a 100644 --- a/tests/func/honeycomb/mgmt-cfg-lisp-apihc-apivat-func.robot +++ b/tests/func/honeycomb/mgmt-cfg-lisp-apihc-apivat-func.robot @@ -114,8 +114,6 @@ | TC08: Honeycomb configures Lisp mapping with adjacency | | [Documentation] | Check if Honeycomb can configure local and remote Lisp\ | | ... | mappings with VRF, and configure adjacency. -| | [Tags] | EXPECTED_FAILING -# Requests below "vrf-subtable" level fail on table-id lookup (HONEYCOMB-290) | | [Teardown] | Honeycomb removes all lisp mappings | ${node} | | Given Locator Set From Honeycomb Should Be | | ... | ${node} | ${interface} | ${locator_set} @@ -125,12 +123,11 @@ | | ... | ${node} | | And Lisp mappings from VAT should not exist | | ... | ${node} -| | And Honeycomb adds Lisp mapping | ${node} | ${lisp_settings_local_vrf} -| | And Honeycomb adds Lisp mapping | ${node} | ${lisp_settings_remote_vrf} -| | When Honeycomb adds Lisp adjacency | ${node} | ${4} | remote_map_vrf -| | ... | adj01 | ${remote_vrf_adjacency} +| | And Honeycomb adds Lisp mapping | ${node} | ${lisp_settings_both_vrf} +| | When Honeycomb adds Lisp adjacency | ${node} | ${7} | remote_map_vrf +| | ... | adj01 | ${vrf_adjacency} | | Then Lisp mapping from Honeycomb should be -| | ... | ${node} | ${remote_adj_subtable} +| | ... | ${node} | ${adj_subtable} | TC09: Honeycomb configures Lisp map resolver | | [Documentation] | Check if Honeycomb can configure a Lisp map resolver. @@ -150,9 +147,6 @@ | TC11: Honeycomb can remove configuration of Lisp features | | [Documentation] | Check if Honeycomb can disable all Lisp features. -| | [Tags] | EXPECTED_FAILING -# Delete operation fails due to incorrect write order(HONEYCOMB-296), -# but returns code 200: OK (HONEYCOMB-297) | | Given Map resolver from Honeycomb should be | ${node} | 192.168.0.4 | | And PITR config from Honeycomb should be | ${node} | ${locator_set} | | When Honeycomb disables all Lisp features | ${node} diff --git a/tests/func/honeycomb/mgmt-cfg-pbb-apihc-apivat-func.robot b/tests/func/honeycomb/mgmt-cfg-pbb-apihc-apivat-func.robot index 99396028f6..6ae76e0ec9 100644 --- a/tests/func/honeycomb/mgmt-cfg-pbb-apihc-apivat-func.robot +++ b/tests/func/honeycomb/mgmt-cfg-pbb-apihc-apivat-func.robot @@ -61,7 +61,7 @@ | TC06: Honeycomb fails to set wrong b-vlan-tag-vlan-id for new PBB sub-interface | | [Documentation] | Honeycomb fails to create a new PBB sub-interface with\ | | ... | wrong value of parameter b-vlan-tag-vlan-id, type uint16, 12 bit\ -| | ... | range, range "1..4095". +| | ... | range, range 1..4095. | | ... | | Honeycomb fails to create PBB sub interface | ${node} | ${super_if} | | ... | ${cfg_pbb_sub_if_wrong_vlan_tag} @@ -69,7 +69,7 @@ | TC07: Honeycomb fails to set wrong i-tag-isid for new PBB sub-interface | | [Documentation] | Honeycomb fails to create a new PBB sub-interface with\ | | ... | wrong value of parameter i-tag-isid, type uint32, 24 bit range,\ -| | ... | range "1..16777215". +| | ... | range 1..16777215. | | ... | | Honeycomb fails to create PBB sub interface | ${node} | ${super_if} | | ... | ${cfg_pbb_sub_if_wrong_i_tag} diff --git a/tests/func/honeycomb/mgmt-cfg-snat44-apihc-apivat-func.robot b/tests/func/honeycomb/mgmt-cfg-snat44-apihc-apivat-func.robot index d761cabe52..0b45c55573 100644 --- a/tests/func/honeycomb/mgmt-cfg-snat44-apihc-apivat-func.robot +++ b/tests/func/honeycomb/mgmt-cfg-snat44-apihc-apivat-func.robot @@ -55,8 +55,6 @@ | TC04: Honeycomb enables NAT on interface - inbound | | [Documentation] | Honeycomb configures NAT on an interface\ | | ... | in inbound direction. -# Interface config not visible in VAT - https://jira.fd.io/browse/HC2VPP-8 -| | [Tags] | EXPECTED_FAILING | | Given NAT Interface Configuration From Honeycomb Should Be Empty | | ... | ${node} | ${interface} | inbound | | And NAT Interface Configuration From Honeycomb Should Be Empty @@ -86,8 +84,6 @@ | TC06: Honeycomb enables NAT on interface - outbound | | [Documentation] | Honeycomb configures NAT on an interface\ | | ... | in outbound direction. -# Interface config not visible in VAT - https://jira.fd.io/browse/HC2VPP-8 -| | [Tags] | EXPECTED_FAILING | | [Teardown] | Honeycomb removes NAT interface configuration | | ... | ${node} | ${interface} | outbound | | Given NAT Interface Configuration From Honeycomb Should Be empty diff --git a/tests/func/honeycomb/mgmt-cfg-spanrx-apihc-apivat-func.robot b/tests/func/honeycomb/mgmt-cfg-spanrx-apihc-apivat-func.robot index ff72f956af..8beea9a42a 100644 --- a/tests/func/honeycomb/mgmt-cfg-spanrx-apihc-apivat-func.robot +++ b/tests/func/honeycomb/mgmt-cfg-spanrx-apihc-apivat-func.robot @@ -22,8 +22,6 @@ | Suite Teardown | Run Keyword If Any Tests Failed | | ... | Restart Honeycomb And VPP And Clear Persisted Configuration | ${node} | Documentation | *Honeycomb port mirroring test suite.* -| ... -| ... | Test suite uses the first interface of the first DUT node. *** Variables *** | ${interface1}= | ${node['interfaces']['port1']['name']} diff --git a/tests/func/honeycomb/mgmt-cfg-vxlan-apihc-apivat-func.robot b/tests/func/honeycomb/mgmt-cfg-vxlan-apihc-apivat-func.robot index 55131e713c..1838f469a3 100644 --- a/tests/func/honeycomb/mgmt-cfg-vxlan-apihc-apivat-func.robot +++ b/tests/func/honeycomb/mgmt-cfg-vxlan-apihc-apivat-func.robot @@ -36,8 +36,6 @@ | Suite Teardown | Run Keyword If Any Tests Failed | ... | Restart Honeycomb And VPP And Clear Persisted Configuration | ${node} | Documentation | *Honeycomb VxLAN management test suite.* -| ... -| ... | Test suite uses the first interface of the first DUT node. *** Test Cases *** | TC01: Honeycomb configures VxLAN tunnel diff --git a/tests/func/honeycomb/mgmt-cfg-vxlangpe-apihc-apivat-func.robot b/tests/func/honeycomb/mgmt-cfg-vxlangpe-apihc-apivat-func.robot index bb95b20f04..78635ac48d 100644 --- a/tests/func/honeycomb/mgmt-cfg-vxlangpe-apihc-apivat-func.robot +++ b/tests/func/honeycomb/mgmt-cfg-vxlangpe-apihc-apivat-func.robot @@ -40,7 +40,7 @@ | Variables | resources/test_data/honeycomb/vxlan_gpe.py | Documentation | *Honeycomb VxLAN-GPE management test suite.* | Force Tags | honeycomb_sanity -| Suite Teardown | Run Keyword If Any Tests Failed +| Suite Setup | ... | Restart Honeycomb And VPP And Clear Persisted Configuration | ${node} *** Test Cases *** diff --git a/tests/func/honeycomb/mgmt-statepersist-apihc-func.robot b/tests/func/honeycomb/mgmt-statepersist-apihc-func.robot index 84dad452b5..2c0067e14e 100644 --- a/tests/func/honeycomb/mgmt-statepersist-apihc-func.robot +++ b/tests/func/honeycomb/mgmt-statepersist-apihc-func.robot @@ -27,7 +27,7 @@ | TC01: Honeycomb persists configuration through restart of both Honeycomb and VPP | | [Documentation] | Checks if Honeycomb maintains configuration after both\ | | ... | Honeycomb and VPP are restarted. -# Vxlan tunnel name is not properly restored (HONEYCOMB-301) +# Vxlan tunnel name is sometimes not properly restored (HONEYCOMB-301) | | [Tags] | EXPECTED_FAILING | | Given Honeycomb configures every setting | ${node} | ${interface} | | And Honeycomb and VPP should verify every setting | ${node} | ${interface} @@ -38,7 +38,7 @@ | TC02: Honeycomb persists configuration through restart of Honeycomb | | [Documentation] | Checks if Honeycomb maintains configuration after it\ | | ... | is restarted. -# Vxlan tunnel name is not properly restored (HONEYCOMB-301) +# Vxlan tunnel name is sometimes not properly restored (HONEYCOMB-301) | | [Tags] | EXPECTED_FAILING | | Given Honeycomb and VPP should verify every setting | ${node} | ${interface} | | When Honeycomb is restarted | ${node} @@ -48,7 +48,7 @@ | TC03: Honeycomb persists configuration through restart of VPP | | [Documentation] | Checks if Honeycomb updates VPP settings after VPP is\ | | ... | restarted. -# Vxlan tunnel name is not properly restored (HONEYCOMB-301) +# Vxlan tunnel name is sometimes not properly restored (HONEYCOMB-301) | | [Tags] | EXPECTED_FAILING | | Given Honeycomb and VPP should verify every setting | ${node} | ${interface} | | When VPP is restarted | ${node} @@ -60,6 +60,5 @@ | | ... | persistence files are damaged or invalid. | | [Teardown] | Run keyword if test failed | | ... | Restart Honeycomb And VPP And Clear Persisted Configuration | ${node} -| | Given Honeycomb and VPP should not have default configuration | ${node} | | When Persistence file is damaged during restart | ${node} | | Then Honeycomb and VPP should have default configuration | ${node} |