| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
- Due to divergence from original design path the RAW was never
consumed. It adds too much code complexity and requires processing
on both storage and compute. Removing entirely to make modeling
efficient.
- log (apparently SSH) section will never be consumed in the way it is
coded in model. This section is also not part of model schema itself
due to the point above.
- Introducing telemetry section that is going to carry telemetry
items required for CDash.
Signed-off-by: pmikus <peter.mikus@protonmail.ch>
Change-Id: I7e0256c6c9715de8ee559eed29dce96329aac97d
|
|
+ Model version 1.0.0.
- Only some result types are exported.
+ MRR, NDRPDR and SOAK.
- Other result types to be added later.
+ In contrast, all test types are detected.
+ Convert custom classes to JSON-serializable equivalents.
+ Sort dict keys before converting to JSON.
+ Override the order for some known keys.
+ Export sets as sorted arrays.
+ Convert to info content from serialized raw content.
+ Also export outputs for suite setups and teardowns.
+ Info files for setup/teardown exist only temporarily.
+ The data is merged into suite.info.json file.
+ This simplifies presentation of total suite duration.
+ Define model via JSON schema:
- Just test case, suite setup/teardown/suite to be added later.
- Just info, raw to be added later.
+ Proper descriptions.
+ Json is generated from yaml.
+ This is a convenience for maintainers.
+ The officially used schema is the .json one.
+ TODOs written into a separate .txt file.
+ Validate exported instance against the schema.
+ Include format checking.
+ Update CSIT requirements for validation dependencies.
+ This needs python-dateutil==2.8.2, only a patch bump.
+ Compute bandwidth also for soak tests.
+ This unifies with NDRPDR to simplify schema definition.
- PAL may need an update for parsing soak test message.
+ Include SSH log items, raw output only.
+ Generate all outputs in a single filesystem tree.
+ Move raw outputs into test_output_raw.tar.xz.
+ Rename existing tar with suites to generated_robot_files.tar.xz.
Change-Id: I69ff7b330ed1a14dc435fd0ef008e753c0d7f78c
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
Instead of using EnsureGlobalVariable,
which is clunky to use from Python.
As a consequence, all caps variables from Constants.py are used directly
and tests/__init__.robot and robot_enhancements.robot are deleted.
+ Rename the CRC global kill switch based on ci-man review.
Change-Id: I10723792475bc83352bf8c3b7f7946ecf885a194
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
The plan is to override the default in some (but not all) jobs.
The csit-vpp jobs shall do the checking, but vpp-csit jobs shall not
(except api-crc job of course) in order to confirm benign changes.
Change-Id: If59d17991de3fb0e847113a87e0533a8ee62af7f
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
Change-Id: Ibd3e99390bf004173d3d126946b95ffe9ba8f196
Signed-off-by: Peter Mikus <pmikus@cisco.com>
|
|
But keep the detection in PAL using only average, not size nor stdev.
Change-Id: I07092bdeb892c871550265757fd2846a36fbb352
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
+ All suites and tests still refer to MRR instead of BMRR (or FRMOBL).
+ API change: Traffic should pass with maximum rate
first argument got a default value,
thus it needs to be moved towards the end of argument list.
+ Consequently, the value for it has been deleted from suites.
+ Test message is now compatible only with BMRR parsing.
- Reporting of transmit rate is left as TODO.
+ Global test __init__ with new way of giving defaults to global variables.
+ The new keyword is put into robot_enhancements.robot resource.
+ Number of trials and trial duration are now configurable.
- Bootstrap scripts have no support for overriding the defaults yet.
Change-Id: I4b108cc9c256730e62c03fbacba0a1c314f8ae88
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
pmikus
Vratko Polak
Peter Mikus