Update Honeycomb tests

 - add new negative test for bridged-virtual-interface setting
 - modify variables in original bd tests to use bvi=False
 - separate interface ipv4 test case into netmask and prefix
 - replace per-testcase tag definitions with per-suite forced tags
 - add separate tag for persistence suite, due to it's long execution time

Change-Id: I79ae699125d4838ed8b394dc80a028e08b4de684
Signed-off-by: selias <samelias@cisco.com>

2 files changed, 153 insertions, 14 deletions

diff --git a/resources/libraries/robot/honeycomb/bridge_domain.robot b/resources/libraries/robot/honeycomb/bridge_domain.robot
index 4c455ff50d..76122fa8f9 100644
--- a/resources/libraries/robot/honeycomb/bridge_domain.robot
+++ b/resources/libraries/robot/honeycomb/bridge_domain.robot
@@ -114,6 +114,29 @@
 | | interfaceAPI.Add bridge domain to interface
 | | ... | ${node} | ${interface2} | ${bd_name} | &{settings}
 
+| Honeycomb fails to add interfaces to bridge domain
+| | [Documentation] | Uses Honeycomb API to assign interfaces to a bridge\
+| | ... | domain.
+| | ...
+| | ... | *Arguments:*
+| | ... | - node - information about a DUT node. Type: dictionary
+| | ... | - interface1, interface2 - names of interfaces to assign to bridge\
+| | ... | domain. Type: string
+| | ... | - bd_name - name of the bridge domain. Type: string
+| | ... | - settings - bridge domain specific interface settings.\
+| | ... | Type: dictionary
+| | ...
+| | ... | *Example:*
+| | ...
+| | ... | \| Honeycomb fails to add interfaces to bridge domain \
+| | ... | \| ${nodes['DUT1']} \| GigabitEthernet0/8/0 \| GigabitEthernet0/9/0 \
+| | ... | \| bd-04 \| ${{split_horizon_group:2, bvi:True}} \|
+| | [Arguments] | ${node} | ${interface1} | ${interface2} | ${bd_name}
+| | ... | ${settings}
+| | Run keyword and expect error | *Failed to validate*
+| | ... | Honeycomb adds interfaces to bridge domain | ${node} | ${interface1}
+| | ... | ${interface2} | ${bd_name} | ${settings}
+
 | Honeycomb should show interfaces assigned to bridge domain
 | | [Documentation] | Uses Honeycomb API to verify interface assignment to\
 | | ... | bridge domain.
@@ -146,6 +169,29 @@
 | | ... | ${if2_data['v3po:l2']['bridged-virtual-interface']}
 | | ... | ${settings['bvi']}
 
+| Honeycomb should not show interfaces assigned to bridge domain
+| | [Documentation] | Uses Honeycomb API to verify interface assignment to\
+| | ... | bridge domain.
+| | ...
+| | ... | *Arguments:*
+| | ... | - node - information about a DUT node. Type: dictionary
+| | ... | - interface1, interface2 - names of interfaces to check bridge domain\
+| | ... | assignment on. Type: string
+| | ... | - bd_name - name of the bridge domain. Type: string
+| | ... | - settings - bridge domain specific interface settings.\
+| | ... | Type: dictionary
+| | ...
+| | ... | *Example:*
+| | ...
+| | ... | \| Honeycomb should not show interfaces assigned to bridge domain \
+| | ... | \| ${nodes['DUT1']} \| GigabitEthernet0/8/0 \| GigabitEthernet0/9/0 \
+| | ... | \| bd-04 \| ${{split_horizon_group:2, bvi:False}} \|
+| | [Arguments] | ${node} | ${interface1} | ${interface2} | ${bd_name}
+| | ... | ${settings}
+| | Run keyword and expect error | *ValueError*
+| | ... | Honeycomb should show interfaces assigned to bridge domain
+| | ... | ${node} | ${interface1} | ${interface2} | ${bd_name} | ${settings}
+
 | VAT should show interfaces assigned to bridge domain
 | | [Documentation] | Uses VAT to verify interface assignment to\
 | | ... | bridge domain.
@@ -154,9 +200,9 @@
 | | ... | - node - Information about a DUT node. Type: dictionary
 | | ... | - index - Index of bridge domains on VPP node. Starts from 0,\
 | | ... | new BDs reuse numbers after a bridge domain is removed. Type: int
-| | ... | - interface1, interface2 - Names of interfaces to assign to bridge\
-| | ... | domain. Type: string
-| | ... | - settings - Bridge domain specific interface settings.\
+| | ... | - interface1, interface2 - names of interfaces to check bridge domain\
+| | ... | assignment on. Type: string
+| | ... | - settings - bridge domain specific interface settings.\
 | | ... | Type: dictionary
 | | ...
 | | ... | *Example:*
@@ -176,6 +222,30 @@
 | | | Should contain | ${if_indices} | ${interface['sw_if_index']}
 | | | Should be equal | ${interface['shg']} | ${settings['split_horizon_group']}
 
+| VAT should not show interfaces assigned to bridge domain
+| | [Documentation] | Uses VAT to verify interface assignment to\
+| | ... | bridge domain, and expects to fail.
+| | ...
+| | ... | *Arguments:*
+| | ... | - node - information about a DUT node. Type: dictionary
+| | ... | - index - index of bridge domains on VPP node. Starts from 0,\
+| | ... | new BDs reuse numbers after a bridge domain is removed. Type: integer
+| | ... | - interface1, interface2 - names of interfaces to check bridge domain\
+| | ... | assignment on. Type: string
+| | ... | - settings - bridge domain specific interface settings.\
+| | ... | Type: dictionary
+| | ...
+| | ... | *Example:*
+| | ...
+| | ... | \| VAT should not show interfaces assigned to bridge domain \
+| | ... | \| ${nodes['DUT1']} \| ${4} \| GigabitEthernet0/8/0 \
+| | ... | \| GigabitEthernet0/9/0 \| ${{split_horizon_group:2, bvi:False}} \|
+| | [Arguments] | ${node} | ${index} | ${interface1} | ${interface2}
+| | ... | ${settings}
+| | Run keyword and expect error | *No JSON object could be decoded*
+| | ... | VAT should show interfaces assigned to bridge domain
+| | ... | ${node} | ${index} | ${interface1} | ${interface2} | ${settings}
+
 | Honeycomb removes all bridge domains
 | | [Documentation] | Uses Honeycomb API to remove all bridge domains from the \
 | | ... | VPP node.
diff --git a/resources/libraries/robot/honeycomb/interfaces.robot b/resources/libraries/robot/honeycomb/interfaces.robot
index 02d0a49cb8..ea1d2b090b 100644
--- a/resources/libraries/robot/honeycomb/interfaces.robot
+++ b/resources/libraries/robot/honeycomb/interfaces.robot
@@ -90,7 +90,7 @@
 | | ... | ${vat_data['admin_up_down']} == 1 | up | down
 | | Should be equal | ${vat_state} | ${state}
 
-| Honeycomb sets interface ipv4 configuration
+| Honeycomb sets interface ipv4 address
 | | [Documentation] | Uses Honeycomb API to change ipv4 configuration\
 | | ... | of the specified interface.
 | | ...
@@ -99,25 +99,21 @@
 | | ... | - interface - name of an interface on the specified node. Type: string
 | | ... | - address - IP address to set. Type: string
 | | ... | - netmask - subnet mask to set. Type: string
-| | ... | - fib_address - IP address to add to fib table. Type: string
-| | ... | - fib_mac - MAC address to add to fib table. Type: string
 | | ... | - settings - ipv4 interface settings. Type: dictionary
 | | ...
 | | ... | *Example:*
 | | ...
 | | ... | \| Honeycomb sets interface ipv4 configuration \| ${nodes['DUT1']} \
 | | ... | \| GigabitEthernet0/8/0 \| 192.168.0.2 \| 255.255.255.0 \
-| | ... | \| 192.168.0.3 \| 08:00:27:c0:5d:37 \
 | | ... | \| ${{'enabled': True, 'mtu': 1500}} \|
 | | [Arguments] | ${node} | ${interface} | ${address} | ${netmask}
-| | ... | ${fib_address} | ${fib_mac} | ${settings}
+| | ... | ${settings}
 | | interfaceAPI.Add first ipv4 address
 | | ... | ${node} | ${interface} | ${address} | ${netmask}
-| | interfaceAPI.Add first ipv4 neighbor
-| | ... | ${node} | ${interface} | ${fib_address} | ${fib_mac}
 | | :FOR | ${key} | IN | @{settings.keys()}
 | | | interfaceAPI.Configure interface ipv4
 | | | ... | ${node} | ${interface} | ${key} | ${settings['${key}']}
+| | | ${api_data}= | interfaceAPI.Get interface oper data | ${node} | ${interface}
 
 | Honeycomb sets interface ipv4 address with prefix
 | | [Documentation] | Uses Honeycomb API to assign an ipv4 address to the\
@@ -128,14 +124,38 @@
 | | ... | - interface - name of an interface on the specified node. Type: string
 | | ... | - address - IP address to set. Type: string
 | | ... | - prefix - length of address network prefix. Type: int
+| | ... | - settings - ipv4 interface settings. Type: dictionary
 | | ...
 | | ... | *Example:*
 | | ...
 | | ... | \| Honeycomb sets interface ipv4 address with prefix \
 | | ... | \| ${nodes['DUT1']} \| GigabitEthernet0/8/0 \| 192.168.0.2 \| 24 \|
 | | [Arguments] | ${node} | ${interface} | ${address} | ${prefix}
+| | ... | ${settings}
 | | interfaceAPI.Add first ipv4 address
 | | ... | ${node} | ${interface} | ${address} | ${prefix}
+| | :FOR | ${key} | IN | @{settings.keys()}
+| | | interfaceAPI.Configure interface ipv4
+| | | ... | ${node} | ${interface} | ${key} | ${settings['${key}']}
+
+| Honeycomb adds interface ipv4 neighbor
+| | [Documentation] | Uses Honeycomb API to assign an ipv4 neighbor to the\
+| | ... | specified interface.
+| | ...
+| | ... | *Arguments:*
+| | ... | - node - information about a DUT node. Type: dictionary
+| | ... | - interface - name of an interface on the specified node. Type: string
+| | ... | - fib_address - IP address to add to fib table. Type: string
+| | ... | - fib_mac - MAC address to add to fib table. Type: string
+| | ...
+| | ... | *Example:*
+| | ...
+| | ... | \| Honeycomb adds interface ipv4 neighbor \| ${nodes['DUT1']} \
+| | ... | \| GigabitEthernet0/8/0 \| 192.168.0.3 \| 08:00:27:c0:5d:37 \
+| | ... | \| ${{'enabled': True, 'mtu': 1500}} \|
+| | [Arguments] | ${node} | ${interface} | ${fib_address} | ${fib_mac}
+| | interfaceAPI.Add ipv4 neighbor
+| | ... | ${node} | ${interface} | ${fib_address} | ${fib_mac}
 
 | IPv4 config from Honeycomb should be
 | | [Documentation] | Retrieves interface ipv4 configuration through Honeycomb\
@@ -145,7 +165,7 @@
 | | ... | - node - information about a DUT node. Type: dictionary
 | | ... | - interface - name of an interface on the specified node. Type: string
 | | ... | - address - IP address to expect. Type: string
-| | ... | - netmask - subnet mask to expect. Type: string
+| | ... | - prefix - prefix length to expect. Type: string
 | | ... | - fib_address - IP address to expect in fib table. Type: string
 | | ... | - fib_mac - MAC address to expect in fib table. Type: string
 | | ... | - settings - ipv4 interface settings to expect. Type: dictionary
@@ -156,13 +176,13 @@
 | | ... | \| GigabitEthernet0/8/0 \| 192.168.0.2 \| 255.255.255.0 \
 | | ... | \| 192.168.0.3 \| 08:00:27:c0:5d:37 \
 | | ... | \| ${{'enabled': True, 'mtu': 1500}} \|
-| | [Arguments] | ${node} | ${interface} | ${address} | ${netmask}
+| | [Arguments] | ${node} | ${interface} | ${address} | ${prefix}
 | | ... | ${fib_address} | ${fib_mac} | ${settings}
 | | ${api_data}= | interfaceAPI.Get interface oper data | ${node} | ${interface}
 | | Should be equal | ${address}
 | | ... | ${api_data['ietf-ip:ipv4']['address'][0]['ip']}
-| | Should be equal | ${netmask}
-| | ... | ${api_data['ietf-ip:ipv4']['address'][0]['netmask']}
+| | Should be equal | ${prefix}
+| | ... | ${api_data['ietf-ip:ipv4']['address'][0]['prefix-length']}
 | | Should be equal | ${fib_address}
 | | ... | ${api_data['ietf-ip:ipv4']['neighbor'][0]['ip']
 | | Should be equal | ${fib_mac}
@@ -188,9 +208,58 @@
 | | [Arguments] | ${node} | ${interface} | ${address} | ${netmask}
 | | ${vpp_data}= | interfaceCLI.VPP get interface ip addresses
 | | ... | ${node} | ${interface} | ipv4
+#TODO: update based on resolution of bug https://jira.fd.io/browse/VPP-132
 | | Should be equal | ${vpp_data[0]['ip']} | ${address}
 | | Should be equal | ${vpp_data[0]['netmask']} | ${netmask}
 
+| Honeycomb removes interface ipv4 addresses
+| | [Documentation] | Removes all configured ipv4 addresses from the specified\
+| | ... | interface.
+| | ...
+| | ... | *Arguments:*
+| | ... | - node - information about a DUT node. Type: dictionary
+| | ... | - interface - name of an interface on the specified node. Type: string
+| | ...
+| | ... | *Example:*
+| | ...
+| | ... | \| Honeycomb removes interface ipv4 addresses \| ${nodes['DUT1']} \
+| | ... | \| GigabitEthernet0/8/0 \|
+| | [Arguments] | ${node} | ${interface}
+| | Remove all ipv4 addresses | ${node} | ${interface}
+
+| IPv4 address from Honeycomb should be empty
+| | [Documentation] | Retrieves interface ipv4 configuration through Honeycomb\
+| | ... | and expects to find no IPv4 addresses.
+| | ...
+| | ... | *Arguments:*
+| | ... | - node - information about a DUT node. Type: dictionary
+| | ... | - interface - name of an interface on the specified node. Type: string
+| | ...
+| | ... | *Example:*
+| | ...
+| | ... | \| IPv4 address from Honeycomb should be empty\| ${nodes['DUT1']} \
+| | ... | \| GigabitEthernet0/8/0 \|
+| | [Arguments] | ${node} | ${interface}
+| | ${api_data}= | interfaceAPI.Get interface oper data | ${node} | ${interface}
+| | Should be empty | ${api_data['ietf-ip:ipv4']['address']
+
+| IPv4 address from VAT should be empty
+| | [Documentation] | Retrieves interface ipv4 configuration through VAT and\
+| | ... | and expects to find no ipv4 addresses.
+| | ...
+| | ... | *Arguments:*
+| | ... | - node - information about a DUT node. Type: dictionary
+| | ... | - interface - name of an interface on the specified node. Type: string
+| | ...
+| | ... | *Example:*
+| | ...
+| | ... | \| IPv4 config from VAT should be empty \| ${nodes['DUT1']} \
+| | ... | \| GigabitEthernet0/8/0 \|
+| | [Arguments] | ${node} | ${interface}
+| | Run keyword and expect error | *No JSON object could be decoded.*
+| | ... | InterfaceCLI.VPP get interface ip addresses
+| | ... | ${node} | ${interface} | ipv4
+
 | Honeycomb sets interface ipv6 configuration
 | | [Documentation] | Uses Honeycomb API to change ipv6 configuration\
 | | ... | of the specified interface.