aboutsummaryrefslogtreecommitdiffstats
path: root/docs/cpta/methodology/jenkins_jobs.rst
diff options
context:
space:
mode:
authorVratko Polak <vrpolak@cisco.com>2019-04-16 18:59:33 +0200
committerTibor Frank <tifrank@cisco.com>2019-04-18 08:29:21 +0000
commit4f2d0c379b50b66e70d9615fc8425cd4772f7738 (patch)
treea74671303729c6eb50475b179463e64ef8cdfc64 /docs/cpta/methodology/jenkins_jobs.rst
parent07111b63caa5f162c0921b5a91a4de2d13505a7d (diff)
Add perpatch info to cpta methodology
Also, split methodology file into multiple, per section. Change-Id: I973b93d1a99205d7adb80996a3657215e05b8985 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
Diffstat (limited to 'docs/cpta/methodology/jenkins_jobs.rst')
-rw-r--r--docs/cpta/methodology/jenkins_jobs.rst62
1 files changed, 62 insertions, 0 deletions
diff --git a/docs/cpta/methodology/jenkins_jobs.rst b/docs/cpta/methodology/jenkins_jobs.rst
new file mode 100644
index 0000000000..677e0bc748
--- /dev/null
+++ b/docs/cpta/methodology/jenkins_jobs.rst
@@ -0,0 +1,62 @@
+Jenkins Jobs
+------------
+
+Performance Trending (PT)
+`````````````````````````
+
+CSIT PT runs regular performance test jobs measuring and collecting MRR
+data per test case. PT is designed as follows:
+
+1. PT job triggers:
+
+ a) Periodic e.g. twice a day.
+ b) On-demand gerrit triggered.
+
+2. Measurements and data calculations per test case:
+
+ a) Max Received Rate (MRR) - for each trial measurement,
+ send packets at link rate for trial duration,
+ count total received packets, divide by trial duration.
+
+3. Archive MRR values per test case.
+4. Archive all counters collected at MRR.
+
+Performance Analysis (PA)
+`````````````````````````
+
+CSIT PA runs performance analysis
+including anomaly detection as described above.
+PA is defined as follows:
+
+1. PA job triggers:
+
+ a) By PT jobs at their completion.
+ b) On-demand gerrit triggered.
+
+2. Download and parse archived historical data and the new data:
+
+ a) Download RF output.xml files from latest PT job and compressed
+ archived data from nexus.
+ b) Parse out the data filtering test cases listed in PA specification
+ (part of CSIT PAL specification file).
+
+3. Re-calculate new groups and their averages.
+
+4. Evaluate new test data:
+
+ a) If the existing group is prolonged => Result = Pass,
+ Reason = Normal.
+ b) If a new group is detected with lower average =>
+ Result = Fail, Reason = Regression.
+ c) If a new group is detected with higher average =>
+ Result = Pass, Reason = Progression.
+
+5. Generate and publish results
+
+ a) Relay evaluation result to job result.
+ b) Generate a new set of trend summary dashboard, list of failed
+ tests and graphs.
+ c) Publish trend dashboard and graphs in html format on
+ https://docs.fd.io/.
+ d) Generate an alerting email. This email is sent by Jenkins to
+ csit-report@lists.fd.io