From 4f2d0c379b50b66e70d9615fc8425cd4772f7738 Mon Sep 17 00:00:00 2001 From: Vratko Polak Date: Tue, 16 Apr 2019 18:59:33 +0200 Subject: Add perpatch info to cpta methodology Also, split methodology file into multiple, per section. Change-Id: I973b93d1a99205d7adb80996a3657215e05b8985 Signed-off-by: Vratko Polak --- docs/cpta/methodology/trend_presentation.rst | 42 ++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) create mode 100644 docs/cpta/methodology/trend_presentation.rst (limited to 'docs/cpta/methodology/trend_presentation.rst') diff --git a/docs/cpta/methodology/trend_presentation.rst b/docs/cpta/methodology/trend_presentation.rst new file mode 100644 index 0000000000..e9918020c5 --- /dev/null +++ b/docs/cpta/methodology/trend_presentation.rst @@ -0,0 +1,42 @@ +Trend Presentation +------------------ + +Performance Dashboard +````````````````````` + +Dashboard tables list a summary of per test-case VPP MRR performance +trend and trend compliance metrics and detected number of anomalies. + +Separate tables are generated for each testbed and each tested number of +physical cores for VPP workers (1c, 2c, 4c). Test case names are linked to +respective trending graphs for ease of navigation through the test data. + +Failed tests +```````````` + +The Failed tests tables list the tests which failed over the specified seven- +day period together with the number of fails over the period and last failure +details - Time, VPP-Build-Id and CSIT-Job-Build-Id. + +Separate tables are generated for each testbed. Test case names are linked to +respective trending graphs for ease of navigation through the test data. + +Trendline Graphs +```````````````` + +Trendline graphs show measured per run averages of MRR values, +group average values, and detected anomalies. +The graphs are constructed as follows: + +- X-axis represents the date in the format MMDD. +- Y-axis represents run-average MRR value in Mpps. +- Markers to indicate anomaly classification: + + - Regression - red circle. + - Progression - green circle. + +- The line shows average MRR value of each group. + +In addition the graphs show dynamic labels while hovering over graph +data points, presenting the CSIT build date, measured MRR value, VPP +reference, trend job build ID and the LF testbed ID. -- cgit 1.2.3-korg