HC Tests: move honeycomb tests out of vpp directory

https://gerrit.fd.io/r/#/c/9257/ moved VPP instalation
to vpp/func/__init__.robot,
which is run before Honeycomb suite.

Instalation process starts with removing all vpp packages,
which fails because of honeycomb dependency installed by bootstrap script.

This patch fixes HC func jobs by moving them to separate dir.
The honeycomb/func/__init__.robot was updated
to include previous content of vpp/func/__init__.robot.

HC perf jobs were also moved,
but they may require additional care (CSIT-1006).

Change-Id: I99d94272c80a4c57c85ec5cf99cddfbeab7de663
Signed-off-by: Marek Gradzki <mgradzki@cisco.com>

4 files changed, 332 insertions, 0 deletions

diff --git a/tests/honeycomb/perf/__init__.robot b/tests/honeycomb/perf/__init__.robot
new file mode 100644
index 0000000000..b984993139
--- /dev/null
+++ b/tests/honeycomb/perf/__init__.robot
@@ -0,0 +1,61 @@
+# Copyright (c) 2017 Cisco and/or its affiliates.
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+*** Variables***
+# Honeycomb node to run tests on.
+| ${node}= | ${nodes['DUT1']}
+
+*** Settings ***
+| Resource | resources/libraries/robot/shared/default.robot
+| Resource | resources/libraries/robot/shared/interfaces.robot
+| Resource | resources/libraries/robot/honeycomb/honeycomb.robot
+| Resource | resources/libraries/robot/honeycomb/performance.robot
+| Library | resources.libraries.python.SetupFramework
+| Library | resources.libraries.python.CpuUtils
+| Library | resources.libraries.python.honeycomb.Performance
+| Suite Setup | Setup suite for Honeycomb performance tests
+| Suite Teardown | Run Keywords
+| ... | Stop VPP Service on DUT | ${node}
+| ... | AND | Archive Honeycomb Log File | ${node} | perf=${True}
+| ... | AND | Stop honeycomb service on DUTs | ${node}
+
+*** Keywords ***
+| Setup suite for Honeycomb performance tests
+| | [Documentation] | Setup variables and configure VPP and Honeycomb
+| | ... | for performance testing.
+| | ...
+| | Set Global Variable | ${node}
+| | ${cores}= | Get Length | ${node['cpuinfo']}
+| | Set Global Variable | ${cores}
+| | Stop VPP Service on DUT | ${node}
+| | Stop Honeycomb service on DUTs | ${node}
+| | Generate VPP Startup Configuration for Honeycomb Test on DUT | ${node}
+| | Configure Restconf binding address | ${node}
+| | Configure Log Level | ${node} | INFO
+| | Configure Persistence | ${node} | disable
+| | Configure jVPP timeout | ${node} | ${14}
+| | Generate Honeycomb startup configuration for performance test
+| | ... | ${node} | ${cores}
+| | Clear Persisted Honeycomb Configuration | ${node}
+
+| Configure ODL Client for performance tests
+| | [Documentation] | Setup ODL client for performance testing.
+| | ...
+| | ${use_odl_client}= | Get Variable Value | ${HC_ODL}
+| | Run Keyword If | '${use_odl_client}' != '${NONE}'
+| | ... | Run Keywords
+| | ... | Set Global Variable | ${use_odl_client}
+| | ... | AND | Copy ODL client | ${node} | ${HC_ODL} | ~ | ${install_dir}
+| | ... | AND | Configure ODL Client Service On DUT | ${node} | ${install_dir}
+| | ... | ELSE | Log | Variable HC_ODL is not present. Not using ODL.
+| | ... | level=INFO
diff --git a/tests/honeycomb/perf/localhostp1-tcp-netconf16t-crud-read.robot b/tests/honeycomb/perf/localhostp1-tcp-netconf16t-crud-read.robot
new file mode 100644
index 0000000000..c13851cc62
--- /dev/null
+++ b/tests/honeycomb/perf/localhostp1-tcp-netconf16t-crud-read.robot
@@ -0,0 +1,87 @@
+# Copyright (c) 2017 Cisco and/or its affiliates.
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+*** Settings ***
+| Resource | resources/libraries/robot/honeycomb/performance.robot
+| ...
+| Suite Setup | Run Keywords
+| ... | Configure Honeycomb Netconf threads | ${node} | ${16}
+| ... | AND | Honeycomb Performance Suite Setup Generic | ${node}
+| ...
+| Suite Teardown | Honeycomb Performance Suite Teardown Generic | ${node}
+| ...
+| Force Tags | HC_PERF
+| ...
+| Documentation | *Base Netconf operational read test suite with sixteen Netconf
+| ... | threads.*
+| ...
+| ... | *[Top] Network Topologies:* DUT1 single-node topology.
+| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-TCP.
+| ... | *[Cfg] DUT configuration:* Default
+| ... | *[Ver] verification:* DUT verifies Honeycomb's response times with zero
+| ... | error or loss tolerance. Test packets are generated by DUT1 on localhost
+| ... | loopback interface.
+| ... | Traffic profile contains flow-groups containing REST requests
+| ... | for operational reads.
+| ... | *[Ref] Applicable standard specifications:* RFC6241.
+
+*** Test Cases ***
+| TC01: Base operational read with 1 TG thread
+| | [Documentation]
+| | ... | [Ver] Measure response time using single trial throughput test
+| | ... | with 100K requests generated in a single thread.
+| | ${cycles}= | Set Variable | ${1}
+| | ${threads}= | Set Variable | ${1}
+| | ${entries}= | Set Variable | ${100000}
+| | Run base operational read performance trial
+| | ... | ${node} | ${cores} | ${cycles} | ${threads} | ${entries}
+
+| TC02: Base operational read with 2 TG threads
+| | [Documentation]
+| | ... | [Ver] Measure response time using single trial throughput test
+| | ... | with 100K requests generated in two threads.
+| | ${cycles}= | Set Variable | ${1}
+| | ${threads}= | Set Variable | ${2}
+| | ${entries}= | Set Variable | ${50000}
+| | Run base operational read performance trial
+| | ... | ${node} | ${cores} | ${cycles} | ${threads} | ${entries}
+
+| TC03: Base operational read with 4 TG threads
+| | [Documentation]
+| | ... | [Ver] Measure response time using single trial throughput test
+| | ... | with 100K requests generated in four threads.
+| | ${cycles}= | Set Variable | ${1}
+| | ${threads}= | Set Variable | ${4}
+| | ${entries}= | Set Variable | ${25000}
+| | Run base operational read performance trial
+| | ... | ${node} | ${cores} | ${cycles} | ${threads} | ${entries}
+
+| TC04: Base operational read with 8 TG threads
+| | [Documentation]
+| | ... | [Ver] Measure response time using single trial throughput test
+| | ... | with 100K requests generated in eigth threads.
+| | ${cycles}= | Set Variable | ${1}
+| | ${threads}= | Set Variable | ${8}
+| | ${entries}= | Set Variable | ${12500}
+| | Run base operational read performance trial
+| | ... | ${node} | ${cores} | ${cycles} | ${threads} | ${entries}
+
+| TC05: Base operational read with 16 TG threads
+| | [Documentation]
+| | ... | [Ver] Measure response time using single trial throughput test
+| | ... | with 100K requests generated in sixteen threads.
+| | ${cycles}= | Set Variable | ${1}
+| | ${threads}= | Set Variable | ${16}
+| | ${entries}= | Set Variable | ${6250}
+| | Run base operational read performance trial
+| | ... | ${node} | ${cores} | ${cycles} | ${threads} | ${entries}
diff --git a/tests/honeycomb/perf/localhostp1-tcp-netconf1t-crud-read.robot b/tests/honeycomb/perf/localhostp1-tcp-netconf1t-crud-read.robot
new file mode 100644
index 0000000000..d46fdf795b
--- /dev/null
+++ b/tests/honeycomb/perf/localhostp1-tcp-netconf1t-crud-read.robot
@@ -0,0 +1,87 @@
+# Copyright (c) 2017 Cisco and/or its affiliates.
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+*** Settings ***
+| Resource | resources/libraries/robot/honeycomb/performance.robot
+| ...
+| Suite Setup | Run Keywords
+| ... | Configure Honeycomb Netconf threads | ${node} | ${1}
+| ... | AND | Honeycomb Performance Suite Setup Generic | ${node}
+| ...
+| Suite Teardown | Honeycomb Performance Suite Teardown Generic | ${node}
+| ...
+| Force Tags | HC_PERF
+| ...
+| Documentation | *Base Netconf operational read test suite with one Netconf
+| ... | thread.*
+| ...
+| ... | *[Top] Network Topologies:* DUT1 single-node topology.
+| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-TCP.
+| ... | *[Cfg] DUT configuration:* Default
+| ... | *[Ver] verification:* DUT verifies Honeycomb's response times with zero
+| ... | error or loss tolerance. Test packets are generated by DUT1 on localhost
+| ... | loopback interface.
+| ... | Traffic profile contains flow-groups containing REST requests
+| ... | for operational reads.
+| ... | *[Ref] Applicable standard specifications:* RFC6241.
+
+*** Test Cases ***
+| TC01: Base operational read with 1 TG thread
+| | [Documentation]
+| | ... | [Ver] Measure response time using single trial throughput test
+| | ... | with 100K requests generated in a single thread.
+| | ${cycles}= | Set Variable | ${1}
+| | ${threads}= | Set Variable | ${1}
+| | ${entries}= | Set Variable | ${100000}
+| | Run base operational read performance trial
+| | ... | ${node} | ${cores} | ${cycles} | ${threads} | ${entries}
+
+| TC02: Base operational read with 2 TG threads
+| | [Documentation]
+| | ... | [Ver] Measure response time using single trial throughput test
+| | ... | with 100K requests generated in two threads.
+| | ${cycles}= | Set Variable | ${1}
+| | ${threads}= | Set Variable | ${2}
+| | ${entries}= | Set Variable | ${50000}
+| | Run base operational read performance trial
+| | ... | ${node} | ${cores} | ${cycles} | ${threads} | ${entries}
+
+| TC03: Base operational read with 4 TG threads
+| | [Documentation]
+| | ... | [Ver] Measure response time using single trial throughput test
+| | ... | with 100K requests generated in four threads.
+| | ${cycles}= | Set Variable | ${1}
+| | ${threads}= | Set Variable | ${4}
+| | ${entries}= | Set Variable | ${25000}
+| | Run base operational read performance trial
+| | ... | ${node} | ${cores} | ${cycles} | ${threads} | ${entries}
+
+| TC04: Base operational read with 8 TG threads
+| | [Documentation]
+| | ... | [Ver] Measure response time using single trial throughput test
+| | ... | with 100K requests generated in eigth threads.
+| | ${cycles}= | Set Variable | ${1}
+| | ${threads}= | Set Variable | ${8}
+| | ${entries}= | Set Variable | ${12500}
+| | Run base operational read performance trial
+| | ... | ${node} | ${cores} | ${cycles} | ${threads} | ${entries}
+
+| TC05: Base operational read with 16 TG threads
+| | [Documentation]
+| | ... | [Ver] Measure response time using single trial throughput test
+| | ... | with 100K requests generated in sixteen threads.
+| | ${cycles}= | Set Variable | ${1}
+| | ${threads}= | Set Variable | ${16}
+| | ${entries}= | Set Variable | ${6250}
+| | Run base operational read performance trial
+| | ... | ${node} | ${cores} | ${cycles} | ${threads} | ${entries}
diff --git a/tests/honeycomb/perf/localhostp1-tcp-netconf8t-crud-read.robot b/tests/honeycomb/perf/localhostp1-tcp-netconf8t-crud-read.robot
new file mode 100644
index 0000000000..9928f15f1d
--- /dev/null
+++ b/tests/honeycomb/perf/localhostp1-tcp-netconf8t-crud-read.robot
@@ -0,0 +1,97 @@
+# Copyright (c) 2017 Cisco and/or its affiliates.
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+*** Settings ***
+| Resource | resources/libraries/robot/honeycomb/performance.robot
+| ...
+| Suite Setup | Run Keywords
+| ... | Configure Honeycomb Netconf threads | ${node} | ${8}
+| ... | AND | Honeycomb Performance Suite Setup Generic | ${node}
+| ...
+| Suite Teardown | Honeycomb Performance Suite Teardown Generic | ${node}
+| ...
+| Force Tags | HC_PERF
+| ...
+| Documentation | *Base Netconf operational read test suite with eight Netconf
+| ... | threads.*
+| ...
+| ... | *[Top] Network Topologies:* DUT1 single-node topology.
+| ... | *[Enc] Packet Encapsulations:* Eth-IPv4-TCP.
+| ... | *[Cfg] DUT configuration:* Default
+| ... | *[Ver] verification:* DUT verifies Honeycomb's response times with zero
+| ... | error or loss tolerance. Test packets are generated by DUT1 on localhost
+| ... | loopback interface.
+| ... | Traffic profile contains flow-groups containing REST requests
+| ... | for operational reads.
+| ... | *[Ref] Applicable standard specifications:* RFC6241.
+
+*** Test Cases ***
+| TC01: Base operational read with 1 TG thread
+| | [Documentation]
+| | ... | [Ver] Measure response time using single trial throughput test
+| | ... | with 100K requests generated in a single thread.
+| | ${cycles}= | Set Variable | ${1}
+| | ${threads}= | Set Variable | ${1}
+| | ${entries}= | Set Variable | ${100000}
+| | Run base operational read performance trial
+| | ... | ${node} | ${cores} | ${cycles} | ${threads} | ${entries}
+
+| TC02: Base operational read with 2 TG threads
+| | [Documentation]
+| | ... | [Ver] Measure response time using single trial throughput test
+| | ... | with 100K requests generated in two threads.
+| | ${cycles}= | Set Variable | ${1}
+| | ${threads}= | Set Variable | ${2}
+| | ${entries}= | Set Variable | ${50000}
+| | Run base operational read performance trial
+| | ... | ${node} | ${cores} | ${cycles} | ${threads} | ${entries}
+
+| TC03: Base operational read with 4 TG threads
+| | [Documentation]
+| | ... | [Ver] Measure response time using single trial throughput test
+| | ... | with 100K requests generated in four threads.
+| | ${cycles}= | Set Variable | ${1}
+| | ${threads}= | Set Variable | ${4}
+| | ${entries}= | Set Variable | ${25000}
+| | Run base operational read performance trial
+| | ... | ${node} | ${cores} | ${cycles} | ${threads} | ${entries}
+
+| TC04: Base operational read with 8 TG threads
+| | [Documentation]
+| | ... | [Ver] Measure response time using single trial throughput test
+| | ... | with 100K requests generated in eigth threads.
+| | ${cycles}= | Set Variable | ${1}
+| | ${threads}= | Set Variable | ${8}
+| | ${entries}= | Set Variable | ${12500}
+| | Run base operational read performance trial
+| | ... | ${node} | ${cores} | ${cycles} | ${threads} | ${entries}
+
+| TC05: Base operational read with 16 TG threads
+| | [Documentation]
+| | ... | [Ver] Measure response time using single trial throughput test
+| | ... | with 100K requests generated in sixteen threads.
+| | ${cycles}= | Set Variable | ${1}
+| | ${threads}= | Set Variable | ${16}
+| | ${entries}= | Set Variable | ${6250}
+| | Run base operational read performance trial
+| | ... | ${node} | ${cores} | ${cycles} | ${threads} | ${entries}
+
+| TC05: Base operational read with 24 TG threads
+| | [Documentation]
+| | ... | [Ver] Measure response time using single trial throughput test
+| | ... | with 100K requests generated in sixteen threads.
+| | ${cycles}= | Set Variable | ${1}
+| | ${threads}= | Set Variable | ${24}
+| | ${entries}= | Set Variable | ${6250}
+| | Run base operational read performance trial
+| | ... | ${node} | ${cores} | ${cycles} | ${threads} | ${entries}