UTI: Export results

+ Model version 1.0.0.
- Only some result types are exported.
 + MRR, NDRPDR and SOAK.
 - Other result types to be added later.
 + In contrast, all test types are detected.
+ Convert custom classes to JSON-serializable equivalents.
 + Sort dict keys before converting to JSON.
  + Override the order for some known keys.
 + Export sets as sorted arrays.
+ Convert to info content from serialized raw content.
+ Also export outputs for suite setups and teardowns.
 + Info files for setup/teardown exist only temporarily.
 + The data is merged into suite.info.json file.
  + This simplifies presentation of total suite duration.
+ Define model via JSON schema:
 - Just test case, suite setup/teardown/suite to be added later.
 - Just info, raw to be added later.
 + Proper descriptions.
 + Json is generated from yaml.
  + This is a convenience for maintainers.
  + The officially used schema is the .json one.
 + TODOs written into a separate .txt file.
+ Validate exported instance against the schema.
 + Include format checking.
 + Update CSIT requirements for validation dependencies.
  + This needs python-dateutil==2.8.2, only a patch bump.
+ Compute bandwidth also for soak tests.
 + This unifies with NDRPDR to simplify schema definition.
 - PAL may need an update for parsing soak test message.
+ Include SSH log items, raw output only.
+ Generate all outputs in a single filesystem tree.
 + Move raw outputs into test_output_raw.tar.xz.
 + Rename existing tar with suites to generated_robot_files.tar.xz.

Change-Id: I69ff7b330ed1a14dc435fd0ef008e753c0d7f78c
Signed-off-by: Vratko Polak <vrpolak@cisco.com>

1 files changed, 15 insertions, 4 deletions

diff --git a/resources/libraries/robot/performance/performance_utils.robot b/resources/libraries/robot/performance/performance_utils.robot
index 8de74c5707..f2cb873d1a 100644
--- a/resources/libraries/robot/performance/performance_utils.robot
+++ b/resources/libraries/robot/performance/performance_utils.robot
@@ -13,6 +13,7 @@
 
 *** Settings ***
 | Library | Collections
+| Library | resources.libraries.python.model.ExportResult
 | Library | resources.libraries.python.topology.Topology
 | Library | resources.libraries.python.NodePath
 | Library | resources.libraries.python.InterfaceUtil
@@ -301,6 +302,7 @@
 | | ... | ramp_up_rate=${ramp_up_rate}
 | | ${latency} = | Get Latency Int
 | | Set Test Message | ${\n}${message_prefix} ${latency} | append=${True}
+| | Export Ndrpdr Latency | ${message_prefix} | ${latency}
 
 | Send ramp-up traffic
 | | [Documentation]
@@ -371,14 +373,17 @@
 | | ... | Type: boolean
 | | ... | - duration_limit - Hard limit for trial duration, overriding duration
 | | ... | computed from transaction_scale. Default 0.0 means no limit.
+| | ... | - export_mrr_unit - Use this unit when exporting MRR values,
+| | ... | or empty string for no export.
 | |
 | | ... | *Example:*
 | |
 | | ... | \| Send traffic at specified rate \| \${1.0} \| ${4000000.0} \
-| | ... | \| \${10} \| ${False} \| ${1.0} \|
+| | ... | \| \${10} \| ${False} \| ${1.0} \| pps \|
 | |
 | | [Arguments] | ${trial_duration} | ${rate} | ${trial_multiplicity}
 | | ... | ${use_latency}=${False} | ${duration_limit}=${0.0}
+| | ... | ${export_mrr_unit}=${Empty}
 | |
 | | ${ppta} = | Get Packets Per Transaction Aggregated
 | | ${ramp_up_duration} = | Get Ramp Up Duration
@@ -415,7 +420,10 @@
 | | | ... | ramp_up_rate=${ramp_up_rate}
 | | | # Out of several quantities for aborted traffic (duration stretching),
 | | | # the approximated receive rate is the best estimate we have.
-| | | Append To List | ${results} | ${result.approximated_receive_rate}
+| | | ${value} = | Set Variable | ${result.approximated_receive_rate}
+| | | # TODO: Add correct bandwidth computation.
+| | | Append Mrr Value | ${value} | ${export_mrr_unit}
+| | | Append To List | ${results} | ${value}
 | | END
 | | FOR | ${action} | IN | @{stat_post_trial}
 | | | Run Keyword | Additional Statistics Action For ${action}
@@ -637,6 +645,8 @@
 | | ${trial_duration} = | Get Mrr Trial Duration
 | | ${trial_multiplicity} = | Get Mrr Trial Multiplicity
 | | ${use_latency} = | Get Use Latency
+| | ${unit} = | Set Variable If | """_cps""" in """${transaction_type}"""
+| | ... | cps | pps
 | | # The following also sets \${rate_for_teardown}
 | | ${results} = | Send traffic at specified rate
 | | ... | rate=${max_rate}
@@ -644,9 +654,10 @@
 | | ... | trial_multiplicity=${trial_multiplicity}
 | | ... | use_latency=${use_latency}
 | | ... | duration_limit=${0.0}
-| | ${unit} = | Set Variable If | """_cps""" in """${transaction_type}"""
+| | ... | export_mrr_unit=${unit}
+| | ${unit_text} = | Set Variable If | """_cps""" in """${transaction_type}"""
 | | ... | estimated connections per second | packets per second
 | | Set Test Message | ${\n}Maximum Receive Rate trial results
-| | Set Test Message | in ${unit}: ${results}
+| | Set Test Message | in ${unit_text}: ${results}
 | | ... | append=yes
 | | Fail if no traffic forwarded