aboutsummaryrefslogtreecommitdiffstats
path: root/resources/libraries
diff options
context:
space:
mode:
authorVratko Polak <vrpolak@cisco.com>2019-04-10 10:33:17 +0200
committerVratko Polak <vrpolak@cisco.com>2019-04-17 14:50:25 +0000
commit661035ac4ce6e51649f302fe2b7a8218257c0587 (patch)
tree6e7f0a0643fca10c8d4d7da0ab9dff40ba18e626 /resources/libraries
parent5aa2e158983f61df6d26129ad566b906051fc5b5 (diff)
Increase reliability of per patch perf job
CSIT-1485 - Measurement results are inconsistent, creating false positives. - To avoid false positives, sensitivity has to be lowered. + Measurements are no more interleaved. + Measurement result are no longer filtered. + The job now runs faster. - Ip6base regressions smaller than 4% are tolerated. + Ip6base false positives can happen, but less than 10% of the time. - Part of job parameters is set on ci-management level. + Voter script output is improved. Change-Id: I0ec7d5df8b397daf7ff0277a1137ee9f36d8d866 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
Diffstat (limited to 'resources/libraries')
-rw-r--r--resources/libraries/bash/entry/per_patch_perf.sh12
1 files changed, 8 insertions, 4 deletions
diff --git a/resources/libraries/bash/entry/per_patch_perf.sh b/resources/libraries/bash/entry/per_patch_perf.sh
index d3aabfae17..82dadc2ed6 100644
--- a/resources/libraries/bash/entry/per_patch_perf.sh
+++ b/resources/libraries/bash/entry/per_patch_perf.sh
@@ -58,17 +58,21 @@ select_tags || die
compose_pybot_arguments || die
generate_tests || die
archive_tests || die
-iterations=8
+# Support for interleaved measurements is kept for future.
+iterations=1 # 8
for ((iter=0; iter<iterations; iter++)); do
- # TODO: Use less heavy way to avoid apt remove falilures.
- # Also, reserve_testbed has already called cleanup once.
- cleanup_topo
+ if ((iter)); then
+ # Reserve testbed has already cleaned it once,
+ # but we need to clean it explicitly on subsequent iterations.
+ cleanup_topo
+ fi
select_build "build_parent" || die
check_download_dir || die
run_pybot || die
copy_archives || die
archive_parse_test_results "csit_parent/${iter}" || die
die_on_pybot_error || die
+ # TODO: Use less heavy way to avoid apt remove failures.
cleanup_topo
select_build "build_current" || die
check_download_dir || die