aboutsummaryrefslogtreecommitdiffstats
path: root/resources/libraries/robot/performance
diff options
context:
space:
mode:
authorVratko Polak <vrpolak@cisco.com>2021-12-15 17:14:36 +0100
committerVratko Polak <vrpolak@cisco.com>2021-12-15 17:14:36 +0100
commit01d8f262afc567c3d49a23c3cb2cdeaced8a6887 (patch)
tree0449c972d8201be16d648dd749e0a7d116aa8b71 /resources/libraries/robot/performance
parentcca05a55f3434d8a031b98f4a496adb8df20c122 (diff)
UTI: Export results
+ Model version 1.0.0. - Only some result types are exported. + MRR, NDRPDR and SOAK. - Other result types to be added later. + In contrast, all test types are detected. + Convert custom classes to JSON-serializable equivalents. + Sort dict keys before converting to JSON. + Override the order for some known keys. + Export sets as sorted arrays. + Convert to info content from serialized raw content. + Also export outputs for suite setups and teardowns. + Info files for setup/teardown exist only temporarily. + The data is merged into suite.info.json file. + This simplifies presentation of total suite duration. + Define model via JSON schema: - Just test case, suite setup/teardown/suite to be added later. - Just info, raw to be added later. + Proper descriptions. + Json is generated from yaml. + This is a convenience for maintainers. + The officially used schema is the .json one. + TODOs written into a separate .txt file. + Validate exported instance against the schema. + Include format checking. + Update CSIT requirements for validation dependencies. + This needs python-dateutil==2.8.2, only a patch bump. + Compute bandwidth also for soak tests. + This unifies with NDRPDR to simplify schema definition. - PAL may need an update for parsing soak test message. + Include SSH log items, raw output only. + Generate all outputs in a single filesystem tree. + Move raw outputs into test_output_raw.tar.xz. + Rename existing tar with suites to generated_robot_files.tar.xz. Change-Id: I69ff7b330ed1a14dc435fd0ef008e753c0d7f78c Signed-off-by: Vratko Polak <vrpolak@cisco.com>
Diffstat (limited to 'resources/libraries/robot/performance')
-rw-r--r--resources/libraries/robot/performance/performance_display.robot32
-rw-r--r--resources/libraries/robot/performance/performance_utils.robot19
2 files changed, 44 insertions, 7 deletions
diff --git a/resources/libraries/robot/performance/performance_display.robot b/resources/libraries/robot/performance/performance_display.robot
index de515412a5..db2b522091 100644
--- a/resources/libraries/robot/performance/performance_display.robot
+++ b/resources/libraries/robot/performance/performance_display.robot
@@ -46,6 +46,31 @@
| | ... | ${message}${\n}${message_zero} | ${message}${\n}${message_other}
| | Fail | ${message}
+| Compute bandwidth
+| | [Documentation]
+| | ... | Compute (bidir) bandwidth from given (unidir) transaction rate.
+| | ...
+| | ... | This keyword reads "ppta" and "avg_frame_size" set elsewhere.
+| | ... | The implementation should work for both pps and cps rates.
+| | ... |
+| | ... | *Arguments:*
+| | ... | - tps - Transaction rate (unidirectional) [tps]. Type: float
+| | ...
+| | ... | *Returns:*
+| | ... | - Computed bandwidth in Gbps.
+| | ... | - Computed aggregate packet rate in pps.
+| |
+| | ... | *Example:*
+| |
+| | ... | |\ \${bandwidth} \| \${pps} = \| Compute Bandwidth \| \${12345.67} \|
+| |
+| | [Arguments] | ${tps}
+| |
+| | ${ppta} = | Get Packets Per Transaction Aggregated
+| | ${pps} = | Evaluate | ${tps} * ${ppta}
+| | ${bandwidth} = | Evaluate | ${pps} * (${avg_frame_size}+20)*8 / 1e9
+| | Return From Keyword | ${bandwidth} | ${pps}
+
| Display Reconfig Test Message
| | [Documentation]
| | ... | Display the number of packets lost (bidirectionally)
@@ -198,6 +223,8 @@
| | [Arguments] | ${text} | ${tps} | ${latency}=${EMPTY}
| |
| | Set Test Message | ${\n}${text}: ${tps} CPS | append=yes
+| | ${bandwidth} | ${pps} = | Compute Bandwidth | ${tps}
+| | Export Search Bound | ${text} | ${tps} | cps | ${bandwidth}
| | Return From Keyword If | not """${latency}"""
| | Set Test Message | ${\n}LATENCY [min/avg/max/hdrh] per stream: ${latency}
| | ... | append=yes
@@ -230,11 +257,10 @@
| |
| | [Arguments] | ${text} | ${tps} | ${latency}=${EMPTY}
| |
-| | ${ppta} = | Get Packets Per Transaction Aggregated
-| | ${pps} = | Evaluate | ${tps} * ${ppta}
-| | ${bandwidth} = | Evaluate | ${pps} * (${avg_frame_size}+20)*8 / 1e9
+| | ${bandwidth} | ${pps} = | Compute Bandwidth | ${tps}
| | Set Test Message | ${\n}${text}: ${pps} pps, | append=yes
| | Set Test Message | ${bandwidth} Gbps (initial) | append=yes
+| | Export Search Bound | ${text} | ${pps} | pps | ${bandwidth * 1e9}
| | Return From Keyword If | not """${latency}"""
| | Set Test Message | ${\n}LATENCY [min/avg/max/hdrh] per stream: ${latency}
| | ... | append=yes
diff --git a/resources/libraries/robot/performance/performance_utils.robot b/resources/libraries/robot/performance/performance_utils.robot
index 8de74c5707..f2cb873d1a 100644
--- a/resources/libraries/robot/performance/performance_utils.robot
+++ b/resources/libraries/robot/performance/performance_utils.robot
@@ -13,6 +13,7 @@
*** Settings ***
| Library | Collections
+| Library | resources.libraries.python.model.ExportResult
| Library | resources.libraries.python.topology.Topology
| Library | resources.libraries.python.NodePath
| Library | resources.libraries.python.InterfaceUtil
@@ -301,6 +302,7 @@
| | ... | ramp_up_rate=${ramp_up_rate}
| | ${latency} = | Get Latency Int
| | Set Test Message | ${\n}${message_prefix} ${latency} | append=${True}
+| | Export Ndrpdr Latency | ${message_prefix} | ${latency}
| Send ramp-up traffic
| | [Documentation]
@@ -371,14 +373,17 @@
| | ... | Type: boolean
| | ... | - duration_limit - Hard limit for trial duration, overriding duration
| | ... | computed from transaction_scale. Default 0.0 means no limit.
+| | ... | - export_mrr_unit - Use this unit when exporting MRR values,
+| | ... | or empty string for no export.
| |
| | ... | *Example:*
| |
| | ... | \| Send traffic at specified rate \| \${1.0} \| ${4000000.0} \
-| | ... | \| \${10} \| ${False} \| ${1.0} \|
+| | ... | \| \${10} \| ${False} \| ${1.0} \| pps \|
| |
| | [Arguments] | ${trial_duration} | ${rate} | ${trial_multiplicity}
| | ... | ${use_latency}=${False} | ${duration_limit}=${0.0}
+| | ... | ${export_mrr_unit}=${Empty}
| |
| | ${ppta} = | Get Packets Per Transaction Aggregated
| | ${ramp_up_duration} = | Get Ramp Up Duration
@@ -415,7 +420,10 @@
| | | ... | ramp_up_rate=${ramp_up_rate}
| | | # Out of several quantities for aborted traffic (duration stretching),
| | | # the approximated receive rate is the best estimate we have.
-| | | Append To List | ${results} | ${result.approximated_receive_rate}
+| | | ${value} = | Set Variable | ${result.approximated_receive_rate}
+| | | # TODO: Add correct bandwidth computation.
+| | | Append Mrr Value | ${value} | ${export_mrr_unit}
+| | | Append To List | ${results} | ${value}
| | END
| | FOR | ${action} | IN | @{stat_post_trial}
| | | Run Keyword | Additional Statistics Action For ${action}
@@ -637,6 +645,8 @@
| | ${trial_duration} = | Get Mrr Trial Duration
| | ${trial_multiplicity} = | Get Mrr Trial Multiplicity
| | ${use_latency} = | Get Use Latency
+| | ${unit} = | Set Variable If | """_cps""" in """${transaction_type}"""
+| | ... | cps | pps
| | # The following also sets \${rate_for_teardown}
| | ${results} = | Send traffic at specified rate
| | ... | rate=${max_rate}
@@ -644,9 +654,10 @@
| | ... | trial_multiplicity=${trial_multiplicity}
| | ... | use_latency=${use_latency}
| | ... | duration_limit=${0.0}
-| | ${unit} = | Set Variable If | """_cps""" in """${transaction_type}"""
+| | ... | export_mrr_unit=${unit}
+| | ${unit_text} = | Set Variable If | """_cps""" in """${transaction_type}"""
| | ... | estimated connections per second | packets per second
| | Set Test Message | ${\n}Maximum Receive Rate trial results
-| | Set Test Message | in ${unit}: ${results}
+| | Set Test Message | in ${unit_text}: ${results}
| | ... | append=yes
| | Fail if no traffic forwarded