Age | Commit message (Collapse) | Author | Files | Lines |
|
Signed-off-by: Viliam Luc <vluc@cisco.com>
Change-Id: I935ee5cf5bd7e760ef29fdc338f981e4e7cc333f
|
|
+ Model version 1.0.0.
- Only some result types are exported.
+ MRR, NDRPDR and SOAK.
- Other result types to be added later.
+ In contrast, all test types are detected.
+ Convert custom classes to JSON-serializable equivalents.
+ Sort dict keys before converting to JSON.
+ Override the order for some known keys.
+ Export sets as sorted arrays.
+ Convert to info content from serialized raw content.
+ Also export outputs for suite setups and teardowns.
+ Info files for setup/teardown exist only temporarily.
+ The data is merged into suite.info.json file.
+ This simplifies presentation of total suite duration.
+ Define model via JSON schema:
- Just test case, suite setup/teardown/suite to be added later.
- Just info, raw to be added later.
+ Proper descriptions.
+ Json is generated from yaml.
+ This is a convenience for maintainers.
+ The officially used schema is the .json one.
+ TODOs written into a separate .txt file.
+ Validate exported instance against the schema.
+ Include format checking.
+ Update CSIT requirements for validation dependencies.
+ This needs python-dateutil==2.8.2, only a patch bump.
+ Compute bandwidth also for soak tests.
+ This unifies with NDRPDR to simplify schema definition.
- PAL may need an update for parsing soak test message.
+ Include SSH log items, raw output only.
+ Generate all outputs in a single filesystem tree.
+ Move raw outputs into test_output_raw.tar.xz.
+ Rename existing tar with suites to generated_robot_files.tar.xz.
Change-Id: I69ff7b330ed1a14dc435fd0ef008e753c0d7f78c
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
When building documentation using sphinx we see ~1200 similar warnings [0]
[0] - https://s3-logs.fd.io/vex-yul-rot-jenkins-1/csit-verify-tox-master-ubuntu2004-x86_64/3289/doc_verify.log.gz
These warning are harmless and can be fixed later
Signed-off-by: Viliam Luc <vluc@cisco.com>
Change-Id: I1ac1099d38935971d47491dde905715345d3935c
|
|
Signed-off-by: pmikus <pmikus@cisco.com>
Change-Id: I710af90aa5474381a82095c8a11ecf6c01c2483b
|
|
Change-Id: I74641cc89d2f25d50b67d51bf2567082b420aabb
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
The test names are unique without it and the information doesn't add
anything extra.
Change-Id: Idc7d6d1d21c8c05691e1757227a0a3787406d370
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
|
|
Signed-off-by: Peter Mikus <pmikus@cisco.com>
Change-Id: I84b3c07e22a313e96ac59fc7818960c502507651
|
|
+ Rework BASH scripts (more code to python).
+ Move BASH into libraries.
+ Allows RDMA usage.
+ Fix 9000B tests.
+ Rename confusing l2fwd -> testpmd.
+ Fix suite setup.
+ Fix PCI whitelist to not accidentally pickup wrong interface.
+ Fix deprecated DPDK cli arguments.
- MLX5 jumbo are disabled on NIC (i will increase separately).
https://jenkins.fd.io/job/csit-dpdk-perf-verify-master-2n-clx/6/console (l3fwd still broken)
- MLX5 IMIX seems to be some TRex issue with IMIX for mlx5 (i will handle separately)
Signed-off-by: Peter Mikus <pmikus@cisco.com>
Change-Id: I31d1b67305fa247cb5e1f57e739d3ef30dc1a04b
|
|
Change-Id: Ib30bc4697fcba93a6723ee492a59a0523425f623
Signed-off-by: Peter Mikus <pmikus@cisco.com>
|
|
+ Include a script to add suite tags to many suites at once.
+ Add suite tags also to device tests (not covered by autogen).
Change-Id: I514ee6178e22999b43460028fe2696738b012f04
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
Change-Id: I1392c06b1d64f62b141d24c0d42a8e36913b15e2
Signed-off-by: Jan Gelety <jgelety@cisco.com>
|
|
+ Phase II - aligning FUNC/PERF setup.
+ 2n/3n compatibility of suite setup.
+ Same structure of testcase for DEV/PERF.
+ DEVICE tests are now running VPP startup config in same
way as PERF.
+ DEVICE is using COREKEEPER!
- DEVICE tests are running topology compute twice for backward
compatibility. TODO to remove.
- Still phase III needed to align TAGS, LIBRARIES and remove OLD
setup phase.
Change-Id: Ib6e1b0ebf4abebf3cbe73b4fda5cac953c2a7270
Signed-off-by: Peter Mikus <pmikus@cisco.com>
|
|
+ Phase I - aligning FUNC/PERF
Change-Id: I46b1e43687ddb29f5ed3c6335fe1baf21ec02822
Signed-off-by: Peter Mikus <pmikus@cisco.com>
|
|
+ All perf suites updated, as autogen change is backward incompatible.
+ Only x710 -ndrpdr suites remain in git repository.
+ Removed retry feature of run_tests.
+ Renamed topology_type and traffic_type to traffic_profile.
+ Renamed framesize to frame_size; nic_model to nic_name.
+ Reordered Variables table to start with nic_name and overhead.
+ Fixed wrong overhead value in some cbc-sha1 suites.
+ Fixed some suite tags.
++ Introduced tags to distinguish suites, such as IPSECINT.
+ Bound crypto hardware to NIC name.
+ Implemented NIC limit hiding.
+ Also search and teardown related arguments are now hidden.
+ Main measurement keywords updated to new arg handling.
+ Max rate related keywords moved to performance_limits.robot
+ Teardowns unified.
+ Generated tests are archived.
++ Generated directory is .gitignore-d.
+ Regenerator raises an exception on seeing non-compatible suite.
++ Relatively helpful message should be seen in exception.
+ Suite and template Documentation has generated parts.
+- With short NIC names only.
+ Autogen checker also upgraded.
+ Tag expressions replaced with long files to ensure analogous tests.
Change-Id: I60e9a999187e7da1f60d0eb4fb02afa14682aa46
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
Change-Id: If2fceb47639dc1add32184271c1e911f7920a51a
Signed-off-by: Vratko Polak <vrpolak@cisco.com>
|
|
+ RXQ refactor
Change-Id: Ic03a2e208b9fe5d324a5ed75a603af4cff1854a9
Signed-off-by: Peter Mikus <pmikus@cisco.com>
|