aboutsummaryrefslogtreecommitdiffstats
path: root/resources/libraries/bash/function/per_patch.sh
AgeCommit message (Collapse)AuthorFilesLines
2023-09-08feat(perpatch): parse results from jsonVratko Polak1-238/+0
+ Use test names in output. - Methodology updated in subsequent change. Change-Id: I6a62f87249ea79262778f68d00f9bb81134f0b02 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2023-08-17Add ability to parse more kinds of test resultsVratko Polak1-16/+203
Previously, only BMRR results were recongnized. Now also lower bounds for PDR (optionally NDR) and soak are recongnized. This code expects all tests are of the same type, e.g. when both MRR and NDRPDR tests are run, only MRR result will get parsed. If test or parsing fails, generate fake data based on overall pass or fail, so at least passrate of unknown tests can be compared in theory. Currently affects only per-patch job (vpp-csit-verify-perf-*), but is useful mainly for the upcoming bisect job. + Do not force MRR test type in vpp-csit jobs. - Some test results are still not recognized (e.g. hoststack). + Do not exit per-patch job early on robot failure. + Only changes that introduce a failure (not present in parent) get -1. + The same is true also for introducing unrecognized test results. - The fake values from passrate can be misleading. + Add default nic tag only if NIC tag is missing. + In all jobs, not only in vpp-csit ones. + Do not add NIC tags for device jobs. - No job supports NDR parsing yet. + Can be enabled in future from ci-management side. Change-Id: Iee904116d1ffed69aec7e31821c67d8447f49ebe Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2023-01-12fix(perpatch): do not expect output_rawVratko Polak1-4/+3
Fixes: bfbdfaedb044b7643b81f47e76285baedfee9e25 Change-Id: Iaefff972d6d47ccdd34b2ccb150375d084df6d93 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2022-02-17Per-patch: Allow parsing CPS MRR resultsVratko Polak1-1/+1
The previous grep was too strict about test message. Change-Id: Ib0c120a32d205b905f257e5b3c948a309ed8abb2 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2022-01-17fix(perpatch): Correct the autogen archive nameVratko Polak1-3/+3
Change-Id: I82bc5cd69de82a2d6b6689cf3380f667ea905886 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-12-22fix(uti): Fixing broken code part IIpmikus1-1/+1
Part of fixing chain after early merge. Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I983b3036a388656bbad91aba71cf14b503f875bd
2021-12-15UTI: Export resultsVratko Polak1-1/+3
+ Model version 1.0.0. - Only some result types are exported. + MRR, NDRPDR and SOAK. - Other result types to be added later. + In contrast, all test types are detected. + Convert custom classes to JSON-serializable equivalents. + Sort dict keys before converting to JSON. + Override the order for some known keys. + Export sets as sorted arrays. + Convert to info content from serialized raw content. + Also export outputs for suite setups and teardowns. + Info files for setup/teardown exist only temporarily. + The data is merged into suite.info.json file. + This simplifies presentation of total suite duration. + Define model via JSON schema: - Just test case, suite setup/teardown/suite to be added later. - Just info, raw to be added later. + Proper descriptions. + Json is generated from yaml. + This is a convenience for maintainers. + The officially used schema is the .json one. + TODOs written into a separate .txt file. + Validate exported instance against the schema. + Include format checking. + Update CSIT requirements for validation dependencies. + This needs python-dateutil==2.8.2, only a patch bump. + Compute bandwidth also for soak tests. + This unifies with NDRPDR to simplify schema definition. - PAL may need an update for parsing soak test message. + Include SSH log items, raw output only. + Generate all outputs in a single filesystem tree. + Move raw outputs into test_output_raw.tar.xz. + Rename existing tar with suites to generated_robot_files.tar.xz. Change-Id: I69ff7b330ed1a14dc435fd0ef008e753c0d7f78c Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-06-16Line length: Fix recent mergesVratko Polak1-1/+1
Not fixing .rst, .md, .yaml, conf.py, .vat, and so on. Change-Id: Icc585d6dbebc8eb5c483b10326302571e94c614d Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-06-10FIX: Pylint reducepmikus1-2/+2
Signed-off-by: pmikus <pmikus@cisco.com> Change-Id: I909942dbb920df7f0fe15c0c92cda92c3cd8d8ad
2020-12-02Avoid ci-management archiving files twiceVratko Polak1-30/+1
This is a follow-up to https://gerrit.fd.io/r/c/csit/+/20394 https://gerrit.fd.io/r/20119 has changed the way archival works, everything should now go to logs.fd.io (instead of appearing on run page in jenkins.fd.io). The glob pattern for archiving is quite eager, doing recursive search. That is good, as it can find also misplaced useful outputs. But it also means our usage of copy_archives function creates two copies of archived directtories, usually archives/ and archives/archive/. This change renames copy_archives to move_archives, with few workarounds to support multiple calls. I also renamed ARCHIVE_DIR value from $CSIT_DIR/archive to $CSIT_DIR/archives to make "move" operation look natural. Finally, download_builds function is being removed, as after recent improvements to VPP compilation speed nobody seems to be using it. Change-Id: I19a429e1dfdfaab7fcf32a9609963b1aebd33c6c Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-07-23perpatch: Echo MAKE_PARALLEL_* var before buildJuraj Linkeš1-1/+15
VPP uses MAKE_PARALLEL_FLAGS or MAKE_PARALLEL_JOBS to limit the number of cpus to use during build, so emit a line on stdout if it's used. Change-Id: I669398d474d172abb6c848a45f24f1bdd56990d8 Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2019-12-16FIX PIP requirement packagesPeter Mikus1-4/+0
+ CSIT will only use packages installed from master PIP requirement.txt file + All CSIT subsequent code must use single function and PIP list for virtualenv - Report/PAL/doc/trending will follow in next patch Change-Id: Ia9150fb05a144182a5a163178d553370ede55cdd Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-12-02Refactor jumpavg to be more readable and usableVratko Polak1-3/+4
This is the python3 change, the python2 is still used for testing bisect. + New version is 0.2.0 due to amount o API changes. + Jumpavg is now part of CSIT resource package tree. + Perpatch migrated to new jumpavg. - PAL NOT updated (the update moved to a different Change). Change-Id: I7d7a8bf8a411196c20c2a40a8c64478d6709bc07 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-12-02FIX: Perpatch PY3Peter Mikus1-2/+2
Signed-off-by: Peter Mikus <pmikus@cisco.com> Change-Id: Iae97cf4490fea38f5f03bdaa120ee7ae48b941ab
2019-07-23vppdevice: fix install-dep failure when component install is requiredDave Wallace1-1/+1
Change-Id: Ibc4379ef25c1bf536da68eb6d0b035993e907c7a Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2019-07-17FIX: Move per-patch building mechanicsPeter Mikus1-5/+2
- Use make pkg-verify Change-Id: Id0f95c9261f1a1b5fa017b0823f87aeb5d6bb6d1 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-07-10Bash functions style cleanupVratko Polak1-25/+24
+ Update rst documentation for bash style + Command substitution: + Clarify when to use backticks. + Recommend avoiding nested command substitution. + Do not recommend putting command substitution results into quotes. + Function definition content: + Move "set -exuo pipefail" after comment only blocks. + Other set flags allowed for functions with good reasons. + Apply the new recommendations. - Blank lines unified in code but no written recommendation in rst. + Add missing references to functions called, variables read or set. + Add TODOs to where lists would be long. + Minor improvements to function descriptions. + Make "if" expressions more python-like. + Add missing "|| die" (or "|| true") where spotted. + Downgrade DEFAULT_NIC to a local variable. + Add TODO to list reasons for blacklisted tags. Change-Id: I05dce030a8c2cb1b3a242d8b977e8fe150d8ee20 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2019-02-20FIX: Static variables all over the place for per patchPeter Mikus1-10/+25
- Incredible why is it so hard to write scripts not hardcoding anything. Change-Id: I48cf2cc3d25f9665a9ff7ca26756d3b70cdd2f9c Signed-off-by: Peter Mikus <pmikus@cisco.com>
2019-02-19CSIT-1419 Convert vpp-virl job to vpp-devicePeter Mikus1-65/+46
Change-Id: I5af6de4b96775a72855e1644ca4d8d9b4b2f9e25 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-10-24Remove usage of vpp-ext-deps packageJan Gelety1-7/+2
This package is not needed for csit enymore. Change-Id: Ie32a9c9322873e66554bb69112d4428697c866db Signed-off-by: Jan Gelety <jgelety@cisco.com>
2018-10-22Per patch: multiple BMRR callsVratko Polak1-66/+71
This is to reduce sensitivity on testbed unstable performance. Also add topo_cleanupbefore every pybot run to avoid issues with VPP uninstall. TRACE prints are left there to simplify investigation of false positives if/when they happen. Change-Id: I9b0cdcfbbe4aa0735a0596746ac32c9e93af0897 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2018-10-01CSIT-1327 Migrate from vpp-dkms-dpdk to vpp-ext-depsPeter Mikus1-10/+11
Change-Id: I6fad3303c5c331ad36d163ba735907e35931dea0 Signed-off-by: Peter Mikus <pmikus@cisco.com>
2018-08-30CSIT-1135: Scripts for VPP per-patch testingVratko Polak1-0/+314
+ Scripts do not rely on (other) bootstraps. + Perf verify bootstrap is also the new style sript now. + Scripts are divided to functions for better re-use. + Functions are sourced from small number of large "library" files. - Still using jumpavg from pipy. - Perpatch has specific simplified parsing (instead of the PAL one). - Bash style document is in a separate Change. Change-Id: If88fa528ce155ea86b614e3d77c0550b91bbdf11 Signed-off-by: Vratko Polak <vrpolak@cisco.com>