UTI: Export results

+ Model version 1.0.0.
- Only some result types are exported.
 + MRR, NDRPDR and SOAK.
 - Other result types to be added later.
 + In contrast, all test types are detected.
+ Convert custom classes to JSON-serializable equivalents.
 + Sort dict keys before converting to JSON.
  + Override the order for some known keys.
 + Export sets as sorted arrays.
+ Convert to info content from serialized raw content.
+ Also export outputs for suite setups and teardowns.
 + Info files for setup/teardown exist only temporarily.
 + The data is merged into suite.info.json file.
  + This simplifies presentation of total suite duration.
+ Define model via JSON schema:
 - Just test case, suite setup/teardown/suite to be added later.
 - Just info, raw to be added later.
 + Proper descriptions.
 + Json is generated from yaml.
  + This is a convenience for maintainers.
  + The officially used schema is the .json one.
 + TODOs written into a separate .txt file.
+ Validate exported instance against the schema.
 + Include format checking.
 + Update CSIT requirements for validation dependencies.
  + This needs python-dateutil==2.8.2, only a patch bump.
+ Compute bandwidth also for soak tests.
 + This unifies with NDRPDR to simplify schema definition.
 - PAL may need an update for parsing soak test message.
+ Include SSH log items, raw output only.
+ Generate all outputs in a single filesystem tree.
 + Move raw outputs into test_output_raw.tar.xz.
 + Rename existing tar with suites to generated_robot_files.tar.xz.

Change-Id: I69ff7b330ed1a14dc435fd0ef008e753c0d7f78c
Signed-off-by: Vratko Polak <vrpolak@cisco.com>

1 files changed, 16 insertions, 0 deletions

diff --git a/docs/model/current/schema/todos.txt b/docs/model/current/schema/todos.txt
new file mode 100644
index 0000000000..8e63b21a19
--- /dev/null
+++ b/docs/model/current/schema/todos.txt
@@ -0,0 +1,16 @@
+
+Export also tg_type and tg_version properties.
+
+Add description with link to methodology for MRR, NDRPDR and SOAK.
+
+Add multiplicity field to MRR result, so PAL can detect incomplete samples.
+
+Add link explaining our L1 bandwidth calculation.
+
+Add a link to URL explaining how to decode the hdrh data.
+
+Do we want to require more structure for dut_version, e.g. at least two dots?
+
+Should we parse version strings?
+E.g.: Turn dut_version from "21.11.0"
+into {"major": 21, "minor": 11, "patch": 0}.