diff options
Diffstat (limited to 'resources/libraries/robot/honeycomb')
| -rw-r--r-- | resources/libraries/robot/honeycomb/honeycomb.robot | 86 | ||||
| -rw-r--r-- | resources/libraries/robot/honeycomb/performance.robot | 120 |
2 files changed, 202 insertions, 4 deletions
diff --git a/resources/libraries/robot/honeycomb/honeycomb.robot b/resources/libraries/robot/honeycomb/honeycomb.robot index 9017584f5a..d0f9f0fbf6 100644 --- a/resources/libraries/robot/honeycomb/honeycomb.robot +++ b/resources/libraries/robot/honeycomb/honeycomb.robot @@ -73,6 +73,7 @@ | | ... | *Example:* | | ... | | ... | \| Clear persisted Honeycomb configuration \| ${nodes['DUT1']} \| +| | ... | | [Arguments] | @{duts} | | Clear persisted Honeycomb config | @{duts} @@ -86,14 +87,34 @@ | | ... | | ... | \| Restart Honeycomb and VPP and clear persisted configuration \ | | ... | \| ${nodes['DUT1']} \| +| | ... | | [Arguments] | ${node} | | Stop Honeycomb service on DUTs | ${node} | | Clear persisted Honeycomb configuration | ${node} | | Setup DUT | ${node} +| | Sleep | 10s | Wait 10sec so VPP is up for sure. | | Configure Honeycomb service on DUTs | ${node} | Restart Honeycomb and VPP -| | [Documentation] | Restarts Honeycomb service and wait until it starts up. +| | [Documentation] | Stops the Honeycomb service and verifies it is stopped. +| | ... | Then restarts VPP, starts Honeycomb again and verifies it is running. +| | ... +| | ... | *Arguments:* +| | ... | - node - information about a DUT node. Type: dictionary +| | ... +| | ... | *Example:* +| | ... +| | ... | \| Restart Honeycomb and VPP \| ${nodes['DUT1']} \| +| | ... +| | [Arguments] | ${node} +| | Stop Honeycomb service on DUTs | ${node} +| | Setup DUT | ${node} +| | Sleep | 10s | Wait 10sec so VPP is up for sure. +| | Configure Honeycomb service on DUTs | ${node} + +| Restart Honeycomb and VPP in performance test +| | [Documentation] | Stops Honeycomb and VPP and verifies HC is stopped. +| | ... | Then restarts VPP, starts Honeycomb again and verifies it is running. | | ... | | ... | *Arguments:* | | ... | - node - information about a DUT node. Type: dictionary @@ -101,10 +122,15 @@ | | ... | *Example:* | | ... | | ... | \| Restart Honeycomb and VPP \| ${nodes['DUT1']} \| +| | ... | | [Arguments] | ${node} | | Stop Honeycomb service on DUTs | ${node} +| | Stop VPP service on DUT | ${node} | | Setup DUT | ${node} +| | Sleep | 10s | Wait 10sec so VPP is up for sure. | | Configure Honeycomb service on DUTs | ${node} +| | Wait until keyword succeeds | 2min | 16sec +| | ... | Check honeycomb startup state | ${node} | Archive Honeycomb log file | | [Documentation] | Copy honeycomb.log file from Honeycomb node\ @@ -112,12 +138,14 @@ | | ... | | ... | *Arguments:* | | ... | - node - information about a DUT node. Type: dictionary +| | ... | - perf - Running on performance testbed? Yes/no Type: boolean | | ... | | ... | *Example:* | | ... | | ... | \| Archive Honeycomb log file \| ${nudes['DUT1']} \| -| | [Arguments] | ${node} -| | Archive Honeycomb log | ${node} +| | ... +| | [Arguments] | ${node} | ${perf}=${False} +| | Archive Honeycomb log | ${node} | ${perf} | Configure ODL Client Service On DUT | | [Documentation] | Configure and start ODL client, then repeatedly check if @@ -131,6 +159,7 @@ | | ... | | ... | \| Configure ODL Client Service on DUT \| ${nodes['DUT1']} \ | | ... | \| carbon-SR1 \| +| | ... | | [Arguments] | ${node} | ${odl_name} | | Copy ODL Client | ${node} | ${odl_name} | /mnt/common | /tmp | | Setup ODL Client | ${node} | /tmp @@ -153,6 +182,7 @@ | | ... | *Example:* | | ... | | ... | \| Configure Honeycomb for functional testing \| ${nodes['DUT1']} \| +| | ... | | [Arguments] | ${node} | | Configure Restconf binding address | ${node} | | Configure Log Level | ${node} | TRACE @@ -172,6 +202,7 @@ | | ... | *Example:* | | ... | | ... | \| Configure ODL Client for functional testing \| ${nodes['DUT1']} \| +| | ... | | [Arguments] | ${node} | | ${use_odl_client}= | Get Variable Value | ${HC_ODL} | | Run Keyword If | '${use_odl_client}' != '${NONE}' @@ -192,6 +223,7 @@ | | ... | *Example:* | | ... | | ... | \| Set Up Honeycomb Functional Test Suite \| ${nodes['DUT1']} \| +| | ... | | [Arguments] | ${node} | | Setup DUT | ${node} | | Configure all TGs for traffic script @@ -209,6 +241,7 @@ | | ... | *Example:* | | ... | | ... | \| Tear Down Honeycomb Functional Test Suite \| ${nodes['DUT1']} \| +| | ... | | [Arguments] | ${node} | | ${use_odl_client}= | Get Variable Value | ${HC_ODL} | | Run Keyword If | '${use_odl_client}' != '${NONE}' @@ -227,6 +260,7 @@ | | ... | *Example:* | | ... | | ... | \| Enable Honeycomb Feature \| ${nodes['DUT1']} \| NSH \| +| | ... | | [arguments] | ${node} | ${feature} | | Manage Honeycomb Features | ${node} | ${feature} @@ -240,5 +274,49 @@ | | ... | *Example:* | | ... | | ... | \| Disable Honeycomb Feature \| ${nodes['DUT1']} \| NSH \| +| | ... | | [arguments] | ${node} | ${feature} -| | Manage Honeycomb Features | ${node} | ${feature} | disable=${True}
\ No newline at end of file +| | Manage Honeycomb Features | ${node} | ${feature} | disable=${True} + +| Stop VPP Service on DUT +| | [Documentation] | Stop the VPP service on the specified node. +| | ... +| | ... | *Arguments:* +| | ... | - node - information about a DUT node. Type: dictionary +| | ... +| | ... | *Example:* +| | ... +| | ... | \| Stop VPP Service on DUT \| ${nodes['DUT1']} \| +| | ... +| | [Arguments] | ${node} +| | Stop VPP Service | ${node} + +| Honeycomb Performance Suite Setup Generic +| | [Documentation] | Generic test suite setup for Honeycomb performance tests. +| | ... | Performs multiple attempts to start Honeycomb+VPP stack. +| | ... +| | ... | *Arguments:* +| | ... | - node - information about a DUT node. Type: dictionary +| | ... +| | ... | *Example:* +| | ... +| | ... | \| Honeycomb Performance Suite Setup Generic \| ${nodes['DUT1']} \| +| | ... +| | [Arguments] | ${node} +| | Wait until keyword succeeds | 8min | 2min +| | ... | Restart Honeycomb and VPP in Performance test | ${node} + +| Honeycomb Performance Suite Teardown Generic +| | [Documentation] | Generic test suite teardown for Honeycomb performance +| | ... | tests. Logs CPU usage before stopping Honeycomb. +| | ... +| | ... | *Arguments:* +| | ... | - node - information about a DUT node. Type: dictionary +| | ... +| | ... | *Example:* +| | ... +| | ... | \| Honeycomb Performance Suite Teardown Generic \| ${nodes['DUT1']} \| +| | ... +| | [Arguments] | ${node} +| | Log Honeycomb and VPP process distribution on cores | ${node} +| | Stop Honeycomb service on DUTs | ${node} diff --git a/resources/libraries/robot/honeycomb/performance.robot b/resources/libraries/robot/honeycomb/performance.robot new file mode 100644 index 0000000000..bd906e1742 --- /dev/null +++ b/resources/libraries/robot/honeycomb/performance.robot @@ -0,0 +1,120 @@ +# Copyright (c) 2017 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +| Library | resources.libraries.python.honeycomb.Performance +| Library | resources.libraries.python.InterfaceUtil +| Resource | resources/libraries/robot/honeycomb/honeycomb.robot +| Documentation | Keywords used in Honeycomb performance testing. + +*** Keywords *** +| Configure Honeycomb Netconf threads +| | [Documentation] | Modify thread configuration of Honeycomb's Netconf server, +| | ... | Requires a restart of Honeycomb to take effect. +| | ... +| | ... | *Arguments:* +| | ... | - node - Node to change configuration on. Type: dictionary +| | ... | - threads - Number of threads to configure. Type: integer +| | ... +| | ... | *Example:* +| | ... +| | ... | \| Configure Honeycomb Netconf Threads \| ${nodes[DUT1]} \| ${2} \| +| | ... +| | [Arguments] | ${node} | ${threads} +| | Configure Netconf Threads | ${node} | ${threads} + +| Run base operational read performance trial +| | [Documentation] | Send Netconf requests over plain TCP to obtain VPP version +| | ... | from Honeycomb operational data. +| | ... +| | ... | *Arguments:* +| | ... | - node - Node to run test on. Type: dictionary +| | ... | - cores - Number of available processor cores. Type: integer +| | ... | - cycles - Number of test cycles to run. Final results will\ +| | ... | be averaged across all runs. Type: integer +| | ... | - threads - Number of threads to use for generating traffic.\ +| | ... | Type: integer +| | ... | - requests - Number of requests to send in each thread and cycle.\ +| | ... | Type: integer +| | ... +| | ... | *Example:* +| | ... +| | ... | \| Run base operational read performance trial \| ${nodes[DUT1]} \ +| | ... | \| ${36} \| ${1} \| ${4} \| ${10000} \| +| | ... +| | [Arguments] | ${node} | ${cores} | ${cycles} | ${threads} | ${requests} +| | ${result}= | Run traffic script on DUT | ${node} | read_vpp_version.py +| | ... | ${cores} | cycles=${cycles} | threads=${threads} +| | ... | requests=${requests} +| | Set Test Message | ${result} + +| Generate VPP Startup Configuration for Honeycomb Test on DUT +| | [Arguments] | ${node} +| | [Documentation] | Create VPP base startup configuration on DUT, then restart +| | ... | VPP to apply the configuration. +| | ... +| | ... | *Arguments:* +| | ... | - node - VPP node to configure. Type: dictionary +| | ... +| | ... | *Example:* +| | ... +| | ... | \| Generate VPP Startup Configuration for Honeycomb Test on DUT \ +| | ... | \| ${nodes[DUT1]} \| +| | ... +| | Import Library | resources.libraries.python.VppConfigGenerator +| | ... | WITH NAME | VPP_config +| | Run keyword | VPP_config.Set Node | ${node} +| | Run keyword | VPP_config.Add Unix Log +| | Run keyword | VPP_config.Add Unix CLI Listen +| | Run keyword | VPP_config.Add Unix Nodaemon +| | Run keyword | VPP_config.Add CPU Main Core | ${1} +| | Run keyword | VPP_config.Apply Config + +| Log Honeycomb and VPP process distribution on cores +| | [Documentation] | Log the distribution of VPP and Honeycomb child processes +| | ... | over the CPU cores. +| | ... +| | ... | *Arguments:* +| | ... | - node - Honeycomb node. Type: dictionary +| | ... +| | ... | *Example:* +| | ... +| | ... | \| Log Honeycomb and VPP process distribution on cores \ +| | ... | \| ${nodes[DUT1]} \| +| | ... +| | [Arguments] | ${node} +| | Log Core Schedule | ${node} | vpp +| | Log Core Schedule | ${node} | java + +| Generate Honeycomb startup configuration for performance test +| | [Documentation] | Create HC startup configuration and apply to config +| | ... | file on DUT. Requires Honeycomb restart to take effect. +| | ... +| | ... | *Arguments:* +| | ... | - node - Honeycomb node. Type: dictionary +| | ... +| | ... | *Example:* +| | ... +| | ... | \| Generate Honeycomb startup configuration for performance test \ +| | ... | \| ${nodes[DUT1]} \| +| | ... +| | [Arguments] | ${node} | ${cores} +| | Import Library | resources.libraries.python.honeycomb.HoneycombSetup.HoneycombStartupConfig +| | ... | WITH NAME | HC_config +| | Run Keyword | HC_config.Set CPU Scheduler | FIFO +| | Run Keyword | HC_config.Set CPU Core Affinity | ${2} | ${cores} +| | Run Keyword | HC_config.Set JIT Compiler Mode | server +| | Run Keyword | HC_config.Set Memory Size | ${512} | ${2048} +| | Run Keyword | HC_config.Set Metaspace Size | ${128} | ${512} +| | Run Keyword | HC_config.Set NUMA Optimization +| | Run Keyword | HC_config.apply config | ${node} |