UTI: Export results

+ Model version 1.0.0.
- Only some result types are exported.
 + MRR, NDRPDR and SOAK.
 - Other result types to be added later.
 + In contrast, all test types are detected.
+ Convert custom classes to JSON-serializable equivalents.
 + Sort dict keys before converting to JSON.
  + Override the order for some known keys.
 + Export sets as sorted arrays.
+ Convert to info content from serialized raw content.
+ Also export outputs for suite setups and teardowns.
 + Info files for setup/teardown exist only temporarily.
 + The data is merged into suite.info.json file.
  + This simplifies presentation of total suite duration.
+ Define model via JSON schema:
 - Just test case, suite setup/teardown/suite to be added later.
 - Just info, raw to be added later.
 + Proper descriptions.
 + Json is generated from yaml.
  + This is a convenience for maintainers.
  + The officially used schema is the .json one.
 + TODOs written into a separate .txt file.
+ Validate exported instance against the schema.
 + Include format checking.
 + Update CSIT requirements for validation dependencies.
  + This needs python-dateutil==2.8.2, only a patch bump.
+ Compute bandwidth also for soak tests.
 + This unifies with NDRPDR to simplify schema definition.
 - PAL may need an update for parsing soak test message.
+ Include SSH log items, raw output only.
+ Generate all outputs in a single filesystem tree.
 + Move raw outputs into test_output_raw.tar.xz.
 + Rename existing tar with suites to generated_robot_files.tar.xz.

Change-Id: I69ff7b330ed1a14dc435fd0ef008e753c0d7f78c
Signed-off-by: Vratko Polak <vrpolak@cisco.com>

1 files changed, 39 insertions, 4 deletions

diff --git a/docs/model/current/top.rst b/docs/model/current/top.rst
index 640abe2343..d86e3fde4c 100644
--- a/docs/model/current/top.rst
+++ b/docs/model/current/top.rst
@@ -22,7 +22,7 @@ especially the export side (UTI), not import side (PAL).
 Version
 ~~~~~~~
 
-This document is valid for CSIT model version 0.1.0.
+This document is valid for CSIT model version 1.0.0.
 
 It is recommended to use semantic versioning: https://semver.org/
 That means, if the new model misses a field present in the old model,
@@ -42,7 +42,42 @@ UTI stands for Unified Test Interface.
 It mainly focuses on exporting information gathered during test run
 into JSON output files.
 
-Files
------
+Output Structure
+-----------------
 
-No files are exported yet in this version.
+UTI outputs come in filesystem tree structure (single tree), where directories
+correspond to suite levels and files correspond to suite setup, suite teardown
+or any test case at this level of suite.
+The directory name comes from SUITE_NAME Robot variable (the last part
+as the previous parts are higher level suites), converted to lowercase.
+If the suite name contains spaces (Robot converts underscores to spaces),
+they are replaced with underscores.
+
+The filesystem tree is rooted under tests/ (as suites in git are there),
+and for each component (test case, suite setup, suite teardown)
+two files are generated.
+The "raw" variant is suitable for debugging (can contain lower level logging),
+while the "info" variant is suitable for processing by PAL
+(can contain derivative values so PAL does not need to compute them
+on every download).
+Their structure and content is mostly identical, model definition mentions
+if a particular subschema is not identical in the two variants.
+It is possible to convert from raw to info, but not the other way.
+
+Although we expect only ASCII text in the exported files,
+we manipulate files using UTF-8 encoding,
+so if Robot Framework uses a non-ascii character, it will be handled.
+
+JSON schemas
+------------
+
+CSIT model is formally defined as a collection of JSON schema documents,
+one for each output file type.
+
+The current version specifies only one output file type:
+Info output for test case.
+
+The authoritative JSON schema documents are in JSON format.
+Git repository also contains YAML formatted document and conversion utility,
+which simplifies maintaining of the JSON document
+(no need to track brackets and commas), but are not authoritative.