diff options
author | Vratko Polak <vrpolak@cisco.com> | 2021-12-15 17:14:36 +0100 |
---|---|---|
committer | Vratko Polak <vrpolak@cisco.com> | 2021-12-15 17:14:36 +0100 |
commit | 01d8f262afc567c3d49a23c3cb2cdeaced8a6887 (patch) | |
tree | 0449c972d8201be16d648dd749e0a7d116aa8b71 /resources/libraries/robot/performance/performance_utils.robot | |
parent | cca05a55f3434d8a031b98f4a496adb8df20c122 (diff) |
UTI: Export results
+ Model version 1.0.0.
- Only some result types are exported.
+ MRR, NDRPDR and SOAK.
- Other result types to be added later.
+ In contrast, all test types are detected.
+ Convert custom classes to JSON-serializable equivalents.
+ Sort dict keys before converting to JSON.
+ Override the order for some known keys.
+ Export sets as sorted arrays.
+ Convert to info content from serialized raw content.
+ Also export outputs for suite setups and teardowns.
+ Info files for setup/teardown exist only temporarily.
+ The data is merged into suite.info.json file.
+ This simplifies presentation of total suite duration.
+ Define model via JSON schema:
- Just test case, suite setup/teardown/suite to be added later.
- Just info, raw to be added later.
+ Proper descriptions.
+ Json is generated from yaml.
+ This is a convenience for maintainers.
+ The officially used schema is the .json one.
+ TODOs written into a separate .txt file.
+ Validate exported instance against the schema.
+ Include format checking.
+ Update CSIT requirements for validation dependencies.
+ This needs python-dateutil==2.8.2, only a patch bump.
+ Compute bandwidth also for soak tests.
+ This unifies with NDRPDR to simplify schema definition.
- PAL may need an update for parsing soak test message.
+ Include SSH log items, raw output only.
+ Generate all outputs in a single filesystem tree.
+ Move raw outputs into test_output_raw.tar.xz.
+ Rename existing tar with suites to generated_robot_files.tar.xz.
Change-Id: I69ff7b330ed1a14dc435fd0ef008e753c0d7f78c
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
Diffstat (limited to 'resources/libraries/robot/performance/performance_utils.robot')
-rw-r--r-- | resources/libraries/robot/performance/performance_utils.robot | 19 |
1 files changed, 15 insertions, 4 deletions
diff --git a/resources/libraries/robot/performance/performance_utils.robot b/resources/libraries/robot/performance/performance_utils.robot index 8de74c5707..f2cb873d1a 100644 --- a/resources/libraries/robot/performance/performance_utils.robot +++ b/resources/libraries/robot/performance/performance_utils.robot @@ -13,6 +13,7 @@ *** Settings *** | Library | Collections +| Library | resources.libraries.python.model.ExportResult | Library | resources.libraries.python.topology.Topology | Library | resources.libraries.python.NodePath | Library | resources.libraries.python.InterfaceUtil @@ -301,6 +302,7 @@ | | ... | ramp_up_rate=${ramp_up_rate} | | ${latency} = | Get Latency Int | | Set Test Message | ${\n}${message_prefix} ${latency} | append=${True} +| | Export Ndrpdr Latency | ${message_prefix} | ${latency} | Send ramp-up traffic | | [Documentation] @@ -371,14 +373,17 @@ | | ... | Type: boolean | | ... | - duration_limit - Hard limit for trial duration, overriding duration | | ... | computed from transaction_scale. Default 0.0 means no limit. +| | ... | - export_mrr_unit - Use this unit when exporting MRR values, +| | ... | or empty string for no export. | | | | ... | *Example:* | | | | ... | \| Send traffic at specified rate \| \${1.0} \| ${4000000.0} \ -| | ... | \| \${10} \| ${False} \| ${1.0} \| +| | ... | \| \${10} \| ${False} \| ${1.0} \| pps \| | | | | [Arguments] | ${trial_duration} | ${rate} | ${trial_multiplicity} | | ... | ${use_latency}=${False} | ${duration_limit}=${0.0} +| | ... | ${export_mrr_unit}=${Empty} | | | | ${ppta} = | Get Packets Per Transaction Aggregated | | ${ramp_up_duration} = | Get Ramp Up Duration @@ -415,7 +420,10 @@ | | | ... | ramp_up_rate=${ramp_up_rate} | | | # Out of several quantities for aborted traffic (duration stretching), | | | # the approximated receive rate is the best estimate we have. -| | | Append To List | ${results} | ${result.approximated_receive_rate} +| | | ${value} = | Set Variable | ${result.approximated_receive_rate} +| | | # TODO: Add correct bandwidth computation. +| | | Append Mrr Value | ${value} | ${export_mrr_unit} +| | | Append To List | ${results} | ${value} | | END | | FOR | ${action} | IN | @{stat_post_trial} | | | Run Keyword | Additional Statistics Action For ${action} @@ -637,6 +645,8 @@ | | ${trial_duration} = | Get Mrr Trial Duration | | ${trial_multiplicity} = | Get Mrr Trial Multiplicity | | ${use_latency} = | Get Use Latency +| | ${unit} = | Set Variable If | """_cps""" in """${transaction_type}""" +| | ... | cps | pps | | # The following also sets \${rate_for_teardown} | | ${results} = | Send traffic at specified rate | | ... | rate=${max_rate} @@ -644,9 +654,10 @@ | | ... | trial_multiplicity=${trial_multiplicity} | | ... | use_latency=${use_latency} | | ... | duration_limit=${0.0} -| | ${unit} = | Set Variable If | """_cps""" in """${transaction_type}""" +| | ... | export_mrr_unit=${unit} +| | ${unit_text} = | Set Variable If | """_cps""" in """${transaction_type}""" | | ... | estimated connections per second | packets per second | | Set Test Message | ${\n}Maximum Receive Rate trial results -| | Set Test Message | in ${unit}: ${results} +| | Set Test Message | in ${unit_text}: ${results} | | ... | append=yes | | Fail if no traffic forwarded |