Update Honeycomb tests

 - add new negative test for bridged-virtual-interface setting
 - modify variables in original bd tests to use bvi=False
 - separate interface ipv4 test case into netmask and prefix
 - replace per-testcase tag definitions with per-suite forced tags
 - add separate tag for persistence suite, due to it's long execution time

Change-Id: I79ae699125d4838ed8b394dc80a028e08b4de684
Signed-off-by: selias <samelias@cisco.com>

7 files changed, 60 insertions, 41 deletions

diff --git a/tests/suites/honeycomb/1 - interface_management.robot b/tests/suites/honeycomb/1 - interface_management.robot
index ce5fa0492f..01be01a6b0 100644
--- a/tests/suites/honeycomb/1 - interface_management.robot
+++ b/tests/suites/honeycomb/1 - interface_management.robot
@@ -16,8 +16,10 @@
 | ${node}= | ${nodes['DUT1']}
 | ${interface}= | ${node['interfaces']['port1']['name']}
 # Configuration which will be set and verified during tests.
-| @{ipv4_address_mask}= | 192.168.0.2 | 255.255.255.0
-| @{ipv4_address_prefix}= | 192.168.0.3 | ${16}
+| ${ipv4_address}= | 192.168.0.2
+| ${ipv4_address2}= | 192.168.0.3
+| ${ipv4_mask}= | 255.255.255.0
+| ${ipv4_prefix}= | ${24}
 | @{ipv4_neighbor}= | 192.168.0.4 | 08:00:27:c0:5d:37
 | &{ipv4_settings}= | enabled=${True} | forwarding=${True} | mtu=${9000}
 | @{ipv6_address}= | 10::10 | ${64}
@@ -32,6 +34,7 @@
 *** Settings ***
 | Resource | resources/libraries/robot/default.robot
 | Resource | resources/libraries/robot/honeycomb/interfaces.robot
+| Force Tags | honeycomb_sanity
 | Documentation | *Honeycomb interface management test suite.*
 | ...
 | ... | Test suite uses the first interface of the first DUT node.
@@ -40,7 +43,6 @@
 | Honeycomb configures and reads interface state
 | | [Documentation] | Check if Honeycomb API can modify the admin state of\
 | | ... | VPP interfaces.
-| | [Tags] | honeycomb_sanity
 | | Given Interface state is | ${node} | ${interface} | down
 | | When Honeycomb sets interface state | ${node} | ${interface} | up
 | | Then Interface state from Honeycomb should be
@@ -51,28 +53,50 @@
 | | ... | ${node} | ${interface} | down
 | | And Interface state from VAT should be | ${node} | ${interface} | down
 
-| Honeycomb modifies interface configuration - ipv4
-| | [Documentation] | Check if Honeycomb API can configure interfaces for ipv4.
-| | [Tags] | honeycomb_sanity
-| | When Honeycomb sets interface ipv4 configuration
-| | ... | ${node} | ${interface} | @{ipv4_address_mask} | @{ipv4_neighbor}
-| | ... | ${ipv4_settings}
+| Honeycomb modifies interface configuration - ipv4 (netmask)
+| | [Documentation] | Check if Honeycomb API can configure interfaces for ipv4\
+| | ... | with address and netmask provided.
+| | When Honeycomb sets interface ipv4 address | ${node} | ${interface}
+| | ... | ${ipv4_address} | ${ipv4_mask} | ${ipv4_settings}
+| | And Honeycomb adds interface ipv4 neighbor
+| | ... | ${node} | ${interface} | @{ipv4_neighbor}
 | | Then IPv4 config from Honeycomb should be
-| | ... | ${node} | ${interface} | @{ipv4_address_mask} | @{ipv4_neighbor}
-| | ... | ${ipv4_settings}
+| | ... | ${node} | ${interface} | ${ipv4_address} | ${ipv4_prefix}
+| | ... | @{ipv4_neighbor} | ${ipv4_settings}
+| | And IPv4 config from VAT should be
+| | ... | ${node} | ${interface} | ${ipv4_address} | ${ipv4_prefix}
+
+| Honeycomb removes ipv4 address from interface
+| | [Documentation] | Check if Honeycomb API can remove configured ipv4\
+| | ... | addresses from interface.
+| | Given IPv4 config from Honeycomb should be
+| | ... | ${node} | ${interface} | ${ipv4_address} | ${ipv4_prefix}
+| | ... | @{ipv4_neighbor} | ${ipv4_settings}
 | | And IPv4 config from VAT should be
-| | ... | ${node} | ${interface} | @{ipv4_address_mask}
+| | ... | ${node} | ${interface} | ${ipv4_address} | ${ipv4_prefix}
+| | When Honeycomb removes interface ipv4 addresses | ${node} | ${interface}
+| | Then IPv4 address from Honeycomb should be empty | ${node} |${interface}
+| | And ipv4 address from VAT should be empty | ${node} | ${interface}
+
+| Honeycomb modifies interface configuration - ipv4 (prefix)
+| | [Documentation] | Check if Honeycomb API can configure interfaces for ipv4\
+| | ... | with address and prefix provided.
+| | [Teardown] | Honeycomb removes interface ipv4 addresses | ${node}
+| | ... | ${interface}
 | | When Honeycomb sets interface ipv4 address with prefix
-| | ... | ${node} | ${interface} | @{ipv4_address_prefix}
+| | ... | ${node} | ${interface} | ${ipv4_address2} | ${ipv4_prefix}
+| | ... | ${ipv4_settings}
+| | And Honeycomb adds interface ipv4 neighbor
+| | ... | ${node} | ${interface} | @{ipv4_neighbor}
 | | Then IPv4 config from Honeycomb should be
-| | ... | ${node} | ${interface} | @{ipv4_address_prefix} | @{ipv4_neighbor}
+| | ... | ${node} | ${interface} | ${ipv4_address2} | ${ipv4_prefix}
+| | ... | @{ipv4_neighbor}
 | | ... | ${ipv4_settings}
 | | And IPv4 config from VAT should be
-| | ... | ${node} | ${interface} | @{ipv4_address_prefix}
+| | ... | ${node} | ${interface} | ${ipv4_address2} | ${ipv4_prefix}
 
 | Honeycomb modifies interface configuration - ipv6
 | | [Documentation] | Check if Honeycomb API can configure interfaces for ipv6.
-| | [Tags] | honeycomb_sanity
 | | When Honeycomb sets interface ipv6 configuration
 | | ... | ${node} | ${interface} | @{ipv6_address} | @{ipv6_neighbor}
 | | ... | ${ipv6_settings}
@@ -85,7 +109,6 @@
 | Honeycomb modifies interface configuration - ethernet,routing
 | | [Documentation] | Check if Honeycomb API can configure interface ethernet\
 | | ... | and routing settings.
-| | [Tags] | honeycomb_sanity
 | | When Honeycomb sets interface ethernet and routing configuration
 | | ... | ${node} | ${interface} | ${ethernet} | ${routing}
 | | Then Interface ethernet and routing configuration from Honeycomb should be
diff --git a/tests/suites/honeycomb/2 - vxlan.robot b/tests/suites/honeycomb/2 - vxlan.robot
index 64bfb3814c..c49c14c9f8 100644
--- a/tests/suites/honeycomb/2 - vxlan.robot
+++ b/tests/suites/honeycomb/2 - vxlan.robot
@@ -32,6 +32,7 @@
 | Resource | resources/libraries/robot/honeycomb/vxlan.robot
 # import additional VxLAN settings from resource file
 | Variables | tests/suites/honeycomb/resources/vxlan.py
+| Force Tags | honeycomb_sanity
 | Documentation | *Honeycomb VxLAN management test suite.*
 | ...
 | ... | Test suite uses the first interface of the first DUT node.
@@ -39,7 +40,6 @@
 *** Test Cases ***
 | Honeycomb configures VxLAN tunnel
 | | [Documentation] | Check if Honeycomb API can configure VxLAN settings.
-| | [Tags] | honeycomb_sanity
 | | Given VxLAN configuration from Honeycomb should be empty
 | | ... | ${node} | ${vx_interface}
 | | And VxLAN configuration from VAT should be empty | ${node}
@@ -52,7 +52,6 @@
 
 | Honeycomb disables VxLAN tunnel
 | | [Documentation] | Check if Honeycomb API can reset VxLAN configuration.
-| | [Tags] | honeycomb_sanity
 | | Given VxLAN configuration from Honeycomb should be
 | | ... | ${node} | ${vx_interface} | ${vxlan_settings}
 | | And VxLAN configuration from VAT should be
@@ -65,7 +64,6 @@
 Honeycomb can configure VXLAN tunnel after one has been disabled
 | | [Documentation] | Check if Honeycomb API can configure VxLAN settings again\
 | | ... | after previous settings have been removed.
-| | [Tags] | honeycomb_sanity
 | | [Teardown] | Honeycomb removes VxLAN tunnel settings
 | | ... | ${node} | ${vx_interface}
 | | Given VxLAN configuration from Honeycomb should be empty
@@ -81,7 +79,6 @@ Honeycomb can configure VXLAN tunnel after one has been disabled
 | Honeycomb does not set VxLAN configuration on another interface type
 | | [Documentation] | Check if Honeycomb API prevents setting VxLAN\
 | | ... | on incorrect interface.
-| | [Tags] | honeycomb_sanity
 | | Given VxLAN configuration from Honeycomb should be empty
 | | ... | ${node} | ${interface}
 | | And VxLAN configuration from VAT should be empty | ${node}
@@ -95,7 +92,6 @@ Honeycomb can configure VXLAN tunnel after one has been disabled
 | Honeycomb does not set invalid VxLAN configuration
 | | [Documentation] | Check if Honeycomb API prevents setting incorrect VxLAN\
 | | ... | settings.
-| | [Tags] | honeycomb_sanity
 | | Given VxLAN configuration from Honeycomb should be empty
 | | ... | ${node} | ${vx_interface}
 | | And VxLAN configuration from VAT should be empty | ${node}
@@ -107,7 +103,6 @@ Honeycomb can configure VXLAN tunnel after one has been disabled
 | Honeycomb configures VxLAN tunnel with ipv6
 | | [Documentation] | Check if Honeycomb API can configure VxLAN with\
 | | ... | ipv6 settings.
-| | [Tags] | honeycomb_sanity
 | | [Teardown] | Honeycomb removes VxLAN tunnel settings
 | | ... | ${node} | ${vx_interface}
 | | Given VxLAN configuration from Honeycomb should be empty
diff --git a/tests/suites/honeycomb/3 - bridge_domain.robot b/tests/suites/honeycomb/3 - bridge_domain.robot
index 8eea9fb359..c2986b65b7 100644
--- a/tests/suites/honeycomb/3 - bridge_domain.robot
+++ b/tests/suites/honeycomb/3 - bridge_domain.robot
@@ -22,12 +22,14 @@
 | &{bd_settings}= | flood=${True} | forward=${True} | learn=${True}
 | ... | unknown-unicast-flood=${True} | arp-termination=${True}
 | &{if_settings}= | split_horizon_group=${1} | bvi=${False}
+| &{if_settings2}= | split_horizon_group=${2} | bvi=${True}
 
 *** Settings ***
 | Resource | resources/libraries/robot/default.robot
 | Resource | resources/libraries/robot/honeycomb/interfaces.robot
 | Resource | resources/libraries/robot/honeycomb/bridge_domain.robot
 | Suite Teardown | Honeycomb removes all bridge domains | ${node}
+| Force Tags | honeycomb_sanity
 | Documentation | *Honeycomb bridge domain management test suite.*
 | ...
 | ... | Test suite uses the first two interfaces on the first DUT node.
@@ -35,7 +37,6 @@
 *** Test Cases ***
 | Honeycomb sets up l2 bridge domain
 | | [Documentation] | Check if Honeycomb can create bridge domains on VPP node.
-| | [Tags] | honeycomb_sanity
 | | When Honeycomb creates first l2 bridge domain
 | | ... | ${node} | ${bd1_name} | ${bd_settings}
 | | Then Bridge domain configuration from Honeycomb should be
@@ -46,7 +47,6 @@
 | Honeycomb manages multiple bridge domains on node
 | | [Documentation] | Check if Honeycomb can manage multiple bridge domains on\
 | | ... | a single node.
-| | [Tags] | honeycomb_sanity
 | | Given Bridge domain configuration from Honeycomb should be
 | | ... | ${node} | ${bd1_name} | ${bd_settings}
 | | When Honeycomb creates l2 bridge domain
@@ -63,7 +63,6 @@
 | Honeycomb removes bridge domains
 | | [Documentation] | Check if Honeycomb can remove bridge domains from a VPP\
 | | ... | node.
-| | [Tags] | honeycomb_sanity
 | | Given Bridge domain configuration from Honeycomb should be
 | | ... | ${node} | ${bd1_name} | ${bd_settings}
 | | When Honeycomb removes all bridge domains | ${node}
@@ -73,7 +72,6 @@
 | Honeycomb assigns interfaces to bridge domain
 | | [Documentation] | Check if Honeycomb can assign VPP interfaces to an\
 | | ... | existing bridge domain.
-| | [Tags] | honeycomb_sanity
 | | Given Honeycomb creates first l2 bridge domain
 | | ... | ${node} | ${bd1_name} | ${bd_settings}
 | | When Honeycomb adds interfaces to bridge domain
@@ -86,7 +84,6 @@
 | Honeycomb removes bridge domain with an interface assigned
 | | [Documentation] | Check if Honeycomb can remove a bridge domain that has an\
 | | ... | interface assigned to it.
-| | [Tags] | honeycomb_sanity
 | | Given Honeycomb should show interfaces assigned to bridge domain
 | | ... | ${node} | @{interfaces} | ${bd1_name} | ${if_settings}
 | | And VAT should show interfaces assigned to bridge domain
@@ -94,3 +91,16 @@
 | | When Honeycomb removes all bridge domains | ${node}
 | | Then Honeycomb should show no bridge domains | ${node}
 | | And VAT should show no bridge domains | ${node}
+
+| Honeycomb does not assign two bridged virtual interfaces to one bridge domain.
+| | [Documentation] | Check if Honeycomb can assign two bridged virtual\
+| | ... | interfaces to a single bridge domain, and expect to fail.
+| | [Teardown] | Honeycomb removes all bridge domains | ${node}
+| | Given Honeycomb creates first l2 bridge domain
+| | ... | ${node} | ${bd1_name} | ${bd_settings}
+| | When Honeycomb fails to add interfaces to bridge domain
+| | ... | ${node} | @{interfaces} | ${bd1_name} | ${if_settings2}
+| | Then Honeycomb should not show interfaces assigned to bridge domain
+| | ... | ${node} | @{interfaces} | ${bd1_name} | ${if_settings2}
+| | And VAT should not show interfaces assigned to bridge domain
+| | ... | ${node} | ${0} | @{interfaces} | ${if_settings2}
diff --git a/tests/suites/honeycomb/4 - tap.robot b/tests/suites/honeycomb/4 - tap.robot
index e3613c0e48..4bce990ee8 100644
--- a/tests/suites/honeycomb/4 - tap.robot
+++ b/tests/suites/honeycomb/4 - tap.robot
@@ -26,6 +26,7 @@
 | Resource | resources/libraries/robot/default.robot
 | Resource | resources/libraries/robot/honeycomb/interfaces.robot
 | Resource | resources/libraries/robot/honeycomb/tap.robot
+| Force Tags | honeycomb_sanity
 | Documentation | *Honeycomb TAP management test suite.*
 | ...
 | ... | Test suite uses the first interface of the first DUT node.
@@ -33,7 +34,6 @@
 *** Test Cases ***
 | Honeycomb configures TAP interface
 | | [Documentation] | Check if Honeycomb API can configure a TAP interface.
-| | [Tags] | honeycomb_sanity
 | | Given TAP configuration from Honeycomb should be empty
 | | ... | ${node} | ${tap_interface}
 | | And TAP configuration from VAT should be empty
@@ -48,7 +48,6 @@
 | Honeycomb modifies existing TAP interface configuration
 | | [Documentation] | Check if Honeycomb API can re-configure and existing TAP\
 | | ... | interface with new settings.
-| | [Tags] | honeycomb_sanity
 | | Given TAP configuration from Honeycomb should be
 | | ... | ${node} | ${tap_interface} | ${tap_settings}
 | | And TAP configuration from VAT should be
@@ -62,7 +61,6 @@
 
 | Honeycomb removes TAP interface
 | | [Documentation] | Check if Honeycomb API can remove TAP interface.
-| | [Tags] | honeycomb_sanity
 | | Given TAP configuration from Honeycomb should be
 | | ... | ${node} | ${tap_interface} | ${tap_settings2}
 | | And TAP configuration from VAT should be
diff --git a/tests/suites/honeycomb/5 - interface_vhost_user.robot b/tests/suites/honeycomb/5 - interface_vhost_user.robot
index 505eb8929b..bba026ebc4 100644
--- a/tests/suites/honeycomb/5 - interface_vhost_user.robot
+++ b/tests/suites/honeycomb/5 - interface_vhost_user.robot
@@ -26,6 +26,7 @@
 *** Settings ***
 | Resource | resources/libraries/robot/default.robot
 | Resource | resources/libraries/robot/honeycomb/vhost_user.robot
+| Force Tags | honeycomb_sanity
 | Documentation | *Honeycomb vhost-user interface management test suite.*
 | ...
 | ...           | This test suite tests if it is posible to create, modify and\
@@ -35,7 +36,6 @@
 | Honycomb creates vhost-user interface - server
 | | [Documentation] | Check if Honeycomb creates a vhost-user interface, role:\
 | | ... | server.
-| | [Tags] | honeycomb_sanity
 | | ...
 | | Given vhost-user configuration from Honeycomb should be empty
 | | ... | ${node} | ${vhost_interface}
@@ -49,7 +49,6 @@
 | Honycomb modifies vhost-user interface - server
 | | [Documentation] | Check if Honeycomb can modify properties of existing\
 | | ... | vhost-user interface, role: server.
-| | [Tags] | honeycomb_sanity
 | | ...
 | | Given vhost-user configuration from Honeycomb should be
 | | ... | ${node} | ${vhost_interface} | ${vhost_user_server}
@@ -75,7 +74,6 @@
 | Honycomb deletes vhost-user interface - server
 | | [Documentation] | Check if Honeycomb can delete an existing vhost-user\
 | | ... | interface, role: server.
-| | [Tags] | honeycomb_sanity
 | | ...
 | | Given vhost-user configuration from Honeycomb should be
 | | ... | ${node} | ${vhost_interface} | ${vhost_user_server}
@@ -89,7 +87,6 @@
 | Honycomb creates vhost-user interface - client
 | | [Documentation] | Check if Honeycomb creates a vhost-user interface, role:\
 | | ... | client.
-| | [Tags] | honeycomb_sanity
 | | ...
 | | Given vhost-user configuration from Honeycomb should be empty
 | | ... | ${node} | ${vhost_interface}
@@ -103,7 +100,6 @@
 | Honycomb modifies vhost-user interface - client
 | | [Documentation] | Check if Honeycomb can modify properties of existing\
 | | ... | vhost-user interface, role: client.
-| | [Tags] | honeycomb_sanity
 | | ...
 | | Given vhost-user configuration from Honeycomb should be
 | | ... | ${node} | ${vhost_interface} | ${vhost_user_client}
@@ -129,7 +125,6 @@
 | Honycomb deletes vhost-user interface - client
 | | [Documentation] | Check if Honeycomb can delete an existing vhost-user\
 | | ... | interface, role: client.
-| | [Tags] | honeycomb_sanity
 | | ...
 | | Given vhost-user configuration from Honeycomb should be
 | | ... | ${node} | ${vhost_interface} | ${vhost_user_client}
@@ -143,7 +138,6 @@
 | Honeycomb does not set vhost-user configuration on another interface type
 | | [Documentation] | Check if Honeycomb refuses to set vhost-user\
 | | ... | configuration for interface which is not v3po:vhost-user type.
-| | [Tags] | honeycomb_sanity
 | | ...
 | | When Honeycomb fails setting vhost-user on different interface type
 | | ... | ${node} | ${interface} | ${vhost_user_server}
@@ -155,7 +149,6 @@
 | Honeycomb does not set invalid vhost-user configuration
 | | [Documentation] | Check if Honeycomb refuses to set invalid parameters to\
 | | ... | vhost-user interface.
-| | [Tags] | honeycomb_sanity
 | | ...
 | | Given vhost-user configuration from Honeycomb should be empty
 | | ... | ${node} | ${vhost_interface}
diff --git a/tests/suites/honeycomb/6 - sub_interface.robot b/tests/suites/honeycomb/6 - sub_interface.robot
index 49f56914be..0ad6deb0d6 100644
--- a/tests/suites/honeycomb/6 - sub_interface.robot
+++ b/tests/suites/honeycomb/6 - sub_interface.robot
@@ -18,11 +18,11 @@
 | Resource | resources/libraries/robot/honeycomb/interfaces.robot
 | Variables | tests/suites/honeycomb/resources/sub_interfaces.py
 | Suite Teardown | Honeycomb removes all bridge domains | ${node}
+| Force Tags | honeycomb_sanity
 | Documentation | *Honeycomb sub-interface management test suite.*
 | ...
 | ...           | This test suite tests if it is posible to create, modify and \
 | ...           | delete a sub-interface.
-| Force Tags | honeycomb_sanity
 
 *** Variables ***
 | ${node}= | ${nodes['DUT1']}
diff --git a/tests/suites/honeycomb/7 - persistence.robot b/tests/suites/honeycomb/7 - persistence.robot
index f992b64010..ad0aeced8d 100644
--- a/tests/suites/honeycomb/7 - persistence.robot
+++ b/tests/suites/honeycomb/7 - persistence.robot
@@ -21,8 +21,8 @@
 | Resource | resources/libraries/robot/honeycomb/persistence.robot
 | Suite Setup | Restart Honeycomb and VPP and clear persisted configuration
 | ... | ${node}
+| Force Tags | honeycomb_persistence
 | Documentation | *Honeycomb configuration persistence test suite.*
-| Force Tags | honeycomb_sanity
 
 *** Test Cases ***
 | Honeycomb persists configuration through restart of both Honeycomb and VPP