Trending: CSIT-1301: Display stdev in hover info

+ make possible to use dashboard for ndrpdr tests
+ fixes in ndrpdr weekly comparison

Change-Id: I846dc207ea620fad78424a445b32095160d4431d
Signed-off-by: Tibor Frank <tifrank@cisco.com>
(cherry picked from commit 68079ecfc6265a621d8e91b43d08fff5801f75d7)
(cherry picked from commit 60d0062d8cdf93271548d06b06b54eb20280907a)

7 files changed, 356 insertions, 121 deletions

diff --git a/docs/cpta/data/index.rst b/docs/cpta/data/index.rst
index 7d0451495b..3f7d313256 100644
--- a/docs/cpta/data/index.rst
+++ b/docs/cpta/data/index.rst
@@ -2,63 +2,52 @@ Performance Data
 ================
 
 Performance test data used to generate the trending graphs is available
-in the ASCII and CSV formats:
+in the CSV formats:
 
 **VPP**
 
     - 3n-hsw
 
-        - `ASCII format <../_static/vpp/csit-vpp-perf-mrr-daily-master-trending.txt>`_
         - `CSV format <../_static/vpp/csit-vpp-perf-mrr-daily-master-trending.csv>`_
 
     - 3n-skx
 
-        - `ASCII format <../_static/vpp/csit-vpp-perf-mrr-daily-master-3n-skx-trending.txt>`_
         - `CSV format <../_static/vpp/csit-vpp-perf-mrr-daily-master-3n-skx-trending.csv>`_
 
     - 2n-skx
 
-        - `ASCII format <../_static/vpp/csit-vpp-perf-mrr-daily-master-2n-skx-trending.txt>`_
         - `CSV format <../_static/vpp/csit-vpp-perf-mrr-daily-master-2n-skx-trending.csv>`_
 
     - 2n-clx
 
-        - `ASCII format <../_static/vpp/csit-vpp-perf-mrr-daily-master-2n-clx-trending.txt>`_
         - `CSV format <../_static/vpp/csit-vpp-perf-mrr-daily-master-2n-clx-trending.csv>`_
 
     - 3n-tsh
 
-        - `ASCII format <../_static/vpp/csit-vpp-perf-mrr-daily-master-3n-tsh-trending.txt>`_
         - `CSV format <../_static/vpp/csit-vpp-perf-mrr-daily-master-3n-tsh-trending.csv>`_
 
     - 3n-dnv
 
-        - `ASCII format <../_static/vpp/csit-vpp-perf-mrr-daily-master-3n-dnv-trending.txt>`_
         - `CSV format <../_static/vpp/csit-vpp-perf-mrr-daily-master-3n-dnv-trending.csv>`_
 
     - 2n-dnv
 
-        - `ASCII format <../_static/vpp/csit-vpp-perf-mrr-daily-master-2n-dnv-trending.txt>`_
         - `CSV format <../_static/vpp/csit-vpp-perf-mrr-daily-master-2n-dnv-trending.csv>`_
 
 **DPDK**
 
     - 3n-hsw
 
-        - `ASCII format <../_static/vpp/csit-dpdk-perf-mrr-weekly-master-trending.txt>`_
         - `CSV format <../_static/vpp/csit-dpdk-perf-mrr-weekly-master-trending.csv>`_
 
     - 3n-skx
 
-        - `ASCII format <../_static/vpp/csit-dpdk-perf-mrr-weekly-master-3n-skx-trending.txt>`_
         - `CSV format <../_static/vpp/csit-dpdk-perf-mrr-weekly-master-3n-skx-trending.csv>`_
 
     - 2n-skx
 
-        - `ASCII format <../_static/vpp/csit-dpdk-perf-mrr-weekly-master-2n-skx-trending.txt>`_
         - `CSV format <../_static/vpp/csit-dpdk-perf-mrr-weekly-master-2n-skx-trending.csv>`_
 
     - 2n-clx
 
-        - `ASCII format <../_static/vpp/csit-dpdk-perf-mrr-weekly-master-2n-clx-trending.txt>`_
         - `CSV format <../_static/vpp/csit-dpdk-perf-mrr-weekly-master-2n-clx-trending.csv>`_
diff --git a/docs/cpta/introduction/dashboard.rst b/docs/cpta/introduction/dashboard.rst
index 38754b5e0b..8c1403cade 100644
--- a/docs/cpta/introduction/dashboard.rst
+++ b/docs/cpta/introduction/dashboard.rst
@@ -31,6 +31,16 @@ VPP worker on 8t4c
 
 .. include:: ../../../_build/_static/vpp/performance-trending-dashboard-8t4c-2n-skx.rst
 
+NDR Weekly VPP worker on 2t1c
+`````````````````````````````
+
+.. include:: ../../../_build/_static/vpp/performance-trending-dashboard-2t1c-2n-skx-ndr.rst
+
+PDR Weekly VPP worker on 2t1c
+`````````````````````````````
+
+.. include:: ../../../_build/_static/vpp/performance-trending-dashboard-2t1c-2n-skx-pdr.rst
+
 3n-skx
 ------
 
@@ -49,6 +59,16 @@ VPP worker on 8t4c
 
 .. include:: ../../../_build/_static/vpp/performance-trending-dashboard-8t4c-3n-skx.rst
 
+NDR Weekly VPP worker on 2t1c
+`````````````````````````````
+
+.. include:: ../../../_build/_static/vpp/performance-trending-dashboard-2t1c-3n-skx-ndr.rst
+
+PDR Weekly VPP worker on 2t1c
+`````````````````````````````
+
+.. include:: ../../../_build/_static/vpp/performance-trending-dashboard-2t1c-3n-skx-pdr.rst
+
 3n-hsw
 ------
 
@@ -67,6 +87,17 @@ VPP worker on 4t4c
 
 .. include:: ../../../_build/_static/vpp/performance-trending-dashboard-4t4c-3n-hsw.rst
 
+
+NDR Weekly VPP worker on 2t1c
+`````````````````````````````
+
+.. include:: ../../../_build/_static/vpp/performance-trending-dashboard-1t1c-3n-hsw-ndr.rst
+
+PDR Weekly VPP worker on 2t1c
+`````````````````````````````
+
+.. include:: ../../../_build/_static/vpp/performance-trending-dashboard-1t1c-3n-hsw-pdr.rst
+
 3n-tsh
 ------
 
@@ -138,3 +169,13 @@ VPP worker on 8t4c
 ``````````````````
 
 .. include:: ../../../_build/_static/vpp/performance-trending-dashboard-8t4c-2n-clx.rst
+
+NDR Weekly VPP worker on 2t1c
+`````````````````````````````
+
+.. include:: ../../../_build/_static/vpp/performance-trending-dashboard-2t1c-2n-clx-ndr.rst
+
+PDR Weekly VPP worker on 2t1c
+`````````````````````````````
+
+.. include:: ../../../_build/_static/vpp/performance-trending-dashboard-2t1c-2n-clx-pdr.rst
diff --git a/docs/cpta/introduction/weekly.rst b/docs/cpta/introduction/weekly.rst
index 0118f44148..bf73a585ce 100644
--- a/docs/cpta/introduction/weekly.rst
+++ b/docs/cpta/introduction/weekly.rst
@@ -4,56 +4,65 @@ NDRPDR Weekly Comparison
 2n-skx
 ------
 
+NDR Comparison
+``````````````
+
 Comparison tables in HTML, ASCII and CSV formats:
 
-  - `HTML 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-skx-2t1c-ndr_in.html>`_
-  - `ASCII 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-skx-2t1c-ndr.txt>`_
-  - `CSV 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-skx-2t1c-ndr.csv>`_
+  - `HTML 2t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-skx-2t1c-ndr_in.html>`_
+  - `ASCII 2t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-skx-2t1c-ndr.txt>`_
+  - `CSV 2t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-skx-2t1c-ndr.csv>`_
 
 PDR Comparison
 ``````````````
 
 Comparison tables in HTML, ASCII and CSV formats:
 
-  - `HTML 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-skx-2t1c-pdr_in.html>`_
-  - `ASCII 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-skx-2t1c-pdr.txt>`_
-  - `CSV 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-skx-2t1c-pdr.csv>`_
+  - `HTML 2t1c PDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-skx-2t1c-pdr_in.html>`_
+  - `ASCII 2t1c PDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-skx-2t1c-pdr.txt>`_
+  - `CSV 2t1c PDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-skx-2t1c-pdr.csv>`_
 
 3n-skx
 ------
 
+NDR Comparison
+``````````````
+
 Comparison tables in HTML, ASCII and CSV formats:
 
-  - `HTML 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-3n-skx-2t1c-ndr_in.html>`_
-  - `ASCII 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-3n-skx-2t1c-ndr.txt>`_
-  - `CSV 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-3n-skx-2t1c-ndr.csv>`_
+  - `HTML 2t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-3n-skx-2t1c-ndr_in.html>`_
+  - `ASCII 2t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-3n-skx-2t1c-ndr.txt>`_
+  - `CSV 2t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-3n-skx-2t1c-ndr.csv>`_
 
 PDR Comparison
 ``````````````
 
 Comparison tables in HTML, ASCII and CSV formats:
 
-  - `HTML 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-3n-skx-2t1c-pdr_in.html>`_
-  - `ASCII 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-3n-skx-2t1c-pdr.txt>`_
-  - `CSV 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-3n-skx-2t1c-pdr.csv>`_
+  - `HTML 2t1c PDR comparison <../_static/vpp/ndrpdr-weekly-comp-3n-skx-2t1c-pdr_in.html>`_
+  - `ASCII 2t1c PDR comparison <../_static/vpp/ndrpdr-weekly-comp-3n-skx-2t1c-pdr.txt>`_
+  - `CSV 2t1c PDR comparison <../_static/vpp/ndrpdr-weekly-comp-3n-skx-2t1c-pdr.csv>`_
 
 2n-clx
 ------
 
+NDR Comparison
+``````````````
+
 Comparison tables in HTML, ASCII and CSV formats:
 
-  - `HTML 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-clx-2t1c-ndr_in.html>`_
-  - `ASCII 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-clx-2t1c-ndr.txt>`_
-  - `CSV 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-clx-2t1c-ndr.csv>`_
+  - `HTML 2t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-clx-2t1c-ndr_in.html>`_
+  - `ASCII 2t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-clx-2t1c-ndr.txt>`_
+  - `CSV 2t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-clx-2t1c-ndr.csv>`_
 
 PDR Comparison
 ``````````````
 
 Comparison tables in HTML, ASCII and CSV formats:
 
-  - `HTML 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-clx-2t1c-pdr_in.html>`_
-  - `ASCII 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-clx-2t1c-pdr.txt>`_
-  - `CSV 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-clx-2t1c-pdr.csv>`_
+  - `HTML 2t1c PDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-clx-2t1c-pdr_in.html>`_
+  - `ASCII 2t1c PDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-clx-2t1c-pdr.txt>`_
+  - `CSV 2t1c PDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-clx-2t1c-pdr.csv>`_
 
 3n-hsw
 ------
@@ -72,6 +81,6 @@ PDR Comparison
 
 Comparison tables in HTML, ASCII and CSV formats:
 
-  - `HTML 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-3n-hsw-1t1c-pdr_in.html>`_
-  - `ASCII 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-3n-hsw-1t1c-pdr.txt>`_
-  - `CSV 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-3n-hsw-1t1c-pdr.csv>`_
+  - `HTML 1t1c PDR comparison <../_static/vpp/ndrpdr-weekly-comp-3n-hsw-1t1c-pdr_in.html>`_
+  - `ASCII 1t1c PDR comparison <../_static/vpp/ndrpdr-weekly-comp-3n-hsw-1t1c-pdr.txt>`_
+  - `CSV 1t1c PDR comparison <../_static/vpp/ndrpdr-weekly-comp-3n-hsw-1t1c-pdr.csv>`_
diff --git a/resources/tools/presentation/generator_cpta.py b/resources/tools/presentation/generator_cpta.py
index ac0a5c60e1..511800a988 100644
--- a/resources/tools/presentation/generator_cpta.py
+++ b/resources/tools/presentation/generator_cpta.py
@@ -169,8 +169,13 @@ def _generate_trending_traces(in_data, job_name, build_info,
     """
 
     data_x = list(in_data.keys())
-    data_y_pps = list(in_data.values())
-    data_y_mpps = [float(item) / 1e6 for item in data_y_pps]
+    data_y_pps = list()
+    data_y_mpps = list()
+    data_y_stdev = list()
+    for item in in_data.values():
+        data_y_pps.append(float(item[u"receive-rate"]))
+        data_y_stdev.append(float(item[u"receive-stdev"]) / 1e6)
+        data_y_mpps.append(float(item[u"receive-rate"]) / 1e6)
 
     hover_text = list()
     xaxis = list()
@@ -179,6 +184,7 @@ def _generate_trending_traces(in_data, job_name, build_info,
         date = build_info[job_name][str_key][0]
         hover_str = (u"date: {date}<br>"
                      u"value [Mpps]: {value:.3f}<br>"
+                     u"stdev [Mpps]: {stdev:.3f}<br>"
                      u"{sut}-ref: {build}<br>"
                      u"csit-ref: mrr-{period}-build-{build_nr}<br>"
                      u"testbed: {testbed}")
@@ -186,6 +192,7 @@ def _generate_trending_traces(in_data, job_name, build_info,
             hover_text.append(hover_str.format(
                 date=date,
                 value=data_y_mpps[index],
+                stdev=data_y_stdev[index],
                 sut=u"dpdk",
                 build=build_info[job_name][str_key][1].rsplit(u'~', 1)[0],
                 period=u"weekly",
@@ -195,6 +202,7 @@ def _generate_trending_traces(in_data, job_name, build_info,
             hover_text.append(hover_str.format(
                 date=date,
                 value=data_y_mpps[index],
+                stdev=data_y_stdev[index],
                 sut=u"vpp",
                 build=build_info[job_name][str_key][1].rsplit(u'~', 1)[0],
                 period=u"daily",
@@ -221,8 +229,7 @@ def _generate_trending_traces(in_data, job_name, build_info,
     }
     if anomaly_classification:
         for index, (key, value) in enumerate(data_pd.items()):
-            if anomaly_classification[index] in \
-                    (u"outlier", u"regression", u"progression"):
+            if anomaly_classification[index] in (u"regression", u"progression"):
                 anomalies[key] = value / 1e6
                 anomalies_colors.append(
                     anomaly_color[anomaly_classification[index]])
@@ -377,8 +384,10 @@ def _generate_all_charts(spec, input_data):
                     if chart_data.get(test_name, None) is None:
                         chart_data[test_name] = OrderedDict()
                     try:
-                        chart_data[test_name][int(index)] = \
-                            test[u"result"][u"receive-rate"]
+                        chart_data[test_name][int(index)] = {
+                            u"receive-rate": test[u"result"][u"receive-rate"],
+                            u"receive-stdev": test[u"result"][u"receive-stdev"]
+                        }
                         chart_tags[test_name] = test.get(u"tags", None)
                     except (KeyError, TypeError):
                         pass
@@ -387,9 +396,12 @@ def _generate_all_charts(spec, input_data):
         for tst_name, tst_data in chart_data.items():
             tst_lst = list()
             for bld in builds_dict[job_name]:
-                itm = tst_data.get(int(bld), u'')
+                itm = tst_data.get(int(bld), dict())
                 # CSIT-1180: Itm will be list, compute stats.
-                tst_lst.append(str(itm))
+                try:
+                    tst_lst.append(str(itm.get(u"receive-rate", u"")))
+                except AttributeError:
+                    tst_lst.append(u"")
             csv_tbl.append(f"{tst_name}," + u",".join(tst_lst) + u'\n')
 
         # Generate traces:
diff --git a/resources/tools/presentation/generator_tables.py b/resources/tools/presentation/generator_tables.py
index 021744dd37..4a4ccceaa0 100644
--- a/resources/tools/presentation/generator_tables.py
+++ b/resources/tools/presentation/generator_tables.py
@@ -829,6 +829,8 @@ def table_perf_trending_dash(table, input_data):
     ]
     header_str = u",".join(header) + u"\n"
 
+    incl_tests = table.get(u"include-tests", u"MRR")
+
     # Prepare data to the table:
     tbl_dict = dict()
     for job, builds in table[u"data"].items():
@@ -846,8 +848,15 @@ def table_perf_trending_dash(table, input_data):
                         u"data": OrderedDict()
                     }
                 try:
-                    tbl_dict[tst_name][u"data"][str(build)] = \
-                        tst_data[u"result"][u"receive-rate"]
+                    if incl_tests == u"MRR":
+                        tbl_dict[tst_name][u"data"][str(build)] = \
+                            tst_data[u"result"][u"receive-rate"]
+                    elif incl_tests == u"NDR":
+                        tbl_dict[tst_name][u"data"][str(build)] = \
+                            tst_data[u"throughput"][u"NDR"][u"LOWER"]
+                    elif incl_tests == u"PDR":
+                        tbl_dict[tst_name][u"data"][str(build)] = \
+                            tst_data[u"throughput"][u"PDR"][u"LOWER"]
                 except (TypeError, KeyError):
                     pass  # No data in output.xml for this test
 
@@ -1790,10 +1799,10 @@ def table_weekly_comparison(table, in_data):
     )
 
     header = [
-        [u"Version"],
-        [u"Date", ],
-        [u"Build", ],
-        [u"Testbed", ]
+        [u"VPP Version", ],
+        [u"Start Timestamp", ],
+        [u"CSIT Build", ],
+        [u"CSIT Testbed", ]
     ]
     tbl_dict = dict()
     idx = 0
@@ -1843,10 +1852,13 @@ def table_weekly_comparison(table, in_data):
         idx_cmp = cmp.get(u"compare", None)
         if idx_ref is None or idx_cmp is None:
             continue
-        header[0].append(f"Diff{idx + 1}")
-        header[1].append(header[0][idx_ref - idx - 1])
-        header[2].append(u"vs")
-        header[3].append(header[0][idx_cmp - idx - 1])
+        header[0].append(
+            f"Diff({header[0][idx_ref - idx].split(u'~')[-1]} vs "
+            f"{header[0][idx_cmp - idx].split(u'~')[-1]})"
+        )
+        header[1].append(u"")
+        header[2].append(u"")
+        header[3].append(u"")
         for tst_name, tst_data in tbl_dict.items():
             if not cmp_dict.get(tst_name, None):
                 cmp_dict[tst_name] = list()
diff --git a/resources/tools/presentation/pal_utils.py b/resources/tools/presentation/pal_utils.py
index 1b9131a30e..98d5837989 100644
--- a/resources/tools/presentation/pal_utils.py
+++ b/resources/tools/presentation/pal_utils.py
@@ -318,14 +318,13 @@ def convert_csv_to_pretty_txt(csv_file_name, txt_file_name, delimiter=u","):
                 txt_table.add_row(
                     [str(itm.replace(u"\u00B1", u"+-")) for itm in row]
                 )
-    txt_table.align = u"r"
-    txt_table.align[u"Test Case"] = u"l"
-    txt_table.align[u"Build"] = u"l"
-    txt_table.align[u"Version"] = u"l"
-
     if not txt_table:
         return
 
+    txt_table.align = u"r"
+    for itm in (u"Test Case", u"Build", u"Version", u"VPP Version"):
+        txt_table.align[itm] = u"l"
+
     if txt_file_name.endswith(u".txt"):
         with open(txt_file_name, u"wt", encoding='utf-8') as txt_file:
             txt_file.write(str(txt_table))
diff --git a/resources/tools/presentation/specification_CPTA.yaml b/resources/tools/presentation/specification_CPTA.yaml
index 2068499339..3a4857855e 100644
--- a/resources/tools/presentation/specification_CPTA.yaml
+++ b/resources/tools/presentation/specification_CPTA.yaml
@@ -889,32 +889,32 @@
 
     table-ndrpdr-comparison-2n-skx:
       csit-vpp-perf-ndrpdr-weekly-master-2n-skx:
-        start: 84
+        start: 76
         end: "lastCompletedBuild"
-        max-builds: 4
+        max-builds: 15
         reverse: True  # Must be reversed
 
     table-ndrpdr-comparison-3n-skx:
       csit-vpp-perf-ndrpdr-weekly-master-3n-skx:
-        start: 84
+        start: 76
         end: "lastCompletedBuild"
-        max-builds: 4
+        max-builds: 15
         reverse: True  # Must be reversed
         skip:
           - 87
 
     table-ndrpdr-comparison-2n-clx:
       csit-vpp-perf-ndrpdr-weekly-master-2n-clx:
-        start: 30
+        start: 21
         end: "lastCompletedBuild"
-        max-builds: 4
+        max-builds: 15
         reverse: True  # Must be reversed
 
     table-ndrpdr-comparison-3n-hsw:
       csit-vpp-perf-ndrpdr-weekly-master-3n-hsw:
-        start: 84
+        start: 76
         end: "lastCompletedBuild"
-        max-builds: 4
+        max-builds: 15
         reverse: True  # Must be reversed
 
   plot-layouts:
@@ -1013,29 +1013,29 @@
 
     # 2n-skx ndrpdr
     csit-vpp-perf-ndrpdr-weekly-master-2n-skx:
-      start: 84
+      start: 76
       end: "lastCompletedBuild"
-      max-builds: 4
+      max-builds: 15
 
     # 3n-skx ndrpdr
     csit-vpp-perf-ndrpdr-weekly-master-3n-skx:
-      start: 84
+      start: 76
       end: "lastCompletedBuild"
-      max-builds: 4
+      max-builds: 15
       skip:
         - 87
 
     # 2n-clx ndrpdr
     csit-vpp-perf-ndrpdr-weekly-master-2n-clx:
-      start: 30
+      start: 21
       end: "lastCompletedBuild"
-      max-builds: 4
+      max-builds: 15
 
     # 3n-hsw ndrpdr
     csit-vpp-perf-ndrpdr-weekly-master-3n-hsw:
-      start: 84
+      start: 76
       end: "lastCompletedBuild"
-      max-builds: 4
+      max-builds: 15
 
     # 3n-hsw
     csit-vpp-perf-mrr-daily-master:
@@ -1092,7 +1092,11 @@
 =======
 >>>>>>> CHANGE (51210b Trending: Update graphs - dnv)
         - 1102
+<<<<<<< HEAD   (8189a4 Report: Add data)
 >>>>>>> CHANGE (b51708 Trending: Plots)
+=======
+        - 1159
+>>>>>>> CHANGE (60d006 Trending: CSIT-1301: Display stdev in hover info)
     csit-dpdk-perf-mrr-weekly-master:
       start: 113
       end: "lastCompletedBuild"
@@ -1174,7 +1178,11 @@
 =======
         - 865
         - 866
+<<<<<<< HEAD   (8189a4 Report: Add data)
 >>>>>>> CHANGE (12d5c9 Trending: Fixes in plots, data)
+=======
+        - 867
+>>>>>>> CHANGE (60d006 Trending: CSIT-1301: Display stdev in hover info)
     csit-dpdk-perf-mrr-weekly-master-3n-skx:
       start: 78
       end: "lastCompletedBuild"
@@ -1239,7 +1247,11 @@
         - 868
         - 870
         - 874
+<<<<<<< HEAD   (8189a4 Report: Add data)
 >>>>>>> CHANGE (01597c Trending: Update graphs)
+=======
+        - 889
+>>>>>>> CHANGE (60d006 Trending: CSIT-1301: Display stdev in hover info)
     csit-vpp-perf-mrr-weekly-master-2n-skx:
       start: 50
       end: "lastCompletedBuild"
@@ -1289,6 +1301,7 @@
         - 251
         - 252
         - 253
+        - 308
 
     csit-dpdk-perf-mrr-weekly-master-2n-clx:
       start: 23
@@ -1336,7 +1349,11 @@
         - 155
         - 156
         - 157
+<<<<<<< HEAD   (8189a4 Report: Add data)
 >>>>>>> CHANGE (01597c Trending: Update graphs)
+=======
+        - 192
+>>>>>>> CHANGE (60d006 Trending: CSIT-1301: Display stdev in hover info)
 
     # 3n-dnv
     csit-vpp-perf-mrr-daily-master-3n-dnv:
@@ -1410,7 +1427,11 @@
         - 390
         - 391
         - 392
+<<<<<<< HEAD   (8189a4 Report: Add data)
 >>>>>>> CHANGE (12d5c9 Trending: Fixes in plots, data)
+=======
+        - 396
+>>>>>>> CHANGE (60d006 Trending: CSIT-1301: Display stdev in hover info)
 
     # 2n-dnv
     csit-vpp-perf-mrr-daily-master-2n-dnv:
@@ -1461,7 +1482,7 @@
         - 350
         - 351
         - 354
-        - 360
+        - 36
         - 364
         - 365
         - 366
@@ -1475,7 +1496,11 @@
         - 395
         - 396
         - 397
+<<<<<<< HEAD   (8189a4 Report: Add data)
 >>>>>>> CHANGE (12d5c9 Trending: Fixes in plots, data)
+=======
+        - 402
+>>>>>>> CHANGE (60d006 Trending: CSIT-1301: Display stdev in hover info)
 
 -
   type: "output"
@@ -1715,10 +1740,6 @@
   - "name"
   - "parent"
   - "result"
-  ignore-list:
-    # This test is "ndrdisc" test and was improperly tagged. It was fixed
-    # but it remains in the old output.xml files.
-  - "tests.vpp.perf.l2.10ge2p1x520-eth-l2bdscale1mmaclrn-mrr.tc01-64b-1t1c-eth-l2bdscale1mmaclrn-ndrdisc"
   window: 14
   long-trend-window: 180
 
@@ -1734,10 +1755,6 @@
   - "name"
   - "parent"
   - "result"
-  ignore-list:
-    # This test is "ndrdisc" test and was improperly tagged. It was fixed
-    # but it remains in the old output.xml files.
-  - "tests.vpp.perf.l2.10ge2p1x520-eth-l2bdscale1mmaclrn-mrr.tc05-64b-2t2c-eth-l2bdscale1mmaclrn-ndrdisc"
   window: 14
   long-trend-window: 180
 
@@ -1753,10 +1770,6 @@
   - "name"
   - "parent"
   - "result"
-  ignore-list:
-    # This test is "ndrdisc" test and was improperly tagged. It was fixed
-    # but it remains in the old output.xml files.
-  - "tests.vpp.perf.l2.10ge2p1x520-eth-l2bdscale1mmaclrn-mrr.tc09-64b-4t4c-eth-l2bdscale1mmaclrn-ndrdisc"
   window: 14
   long-trend-window: 180
 
@@ -1784,6 +1797,202 @@
   output-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-4t4c-3n-hsw.rst"
   testbed: "3n-hsw"
 
+# 3n-hsw
+-
+  type: "table"
+  title: "Performance trending dashboard 1t1c-3n-hsw ndr"
+  algorithm: "table_perf_trending_dash"
+  output-file-ext: ".csv"
+  output-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-1t1c-3n-hsw-ndr"
+  data: "table-ndrpdr-comparison-3n-hsw"
+  filter: "'NDRPDR' and '1T1C'"
+  include-tests: "NDR"
+  parameters:
+    - "name"
+    - "parent"
+    - "throughput"
+  window: 4
+  long-trend-window: 12
+
+-
+  type: "table"
+  title: "Performance trending dashboard 1t1c-3n-hsw pdr"
+  algorithm: "table_perf_trending_dash"
+  output-file-ext: ".csv"
+  output-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-1t1c-3n-hsw-pdr"
+  data: "table-ndrpdr-comparison-3n-hsw"
+  filter: "'NDRPDR' and '1T1C'"
+  include-tests: "PDR"
+  parameters:
+    - "name"
+    - "parent"
+    - "throughput"
+  window: 4
+  long-trend-window: 12
+
+-
+  type: "table"
+  title: "HTML performance trending dashboard 1t1c-3n-hsw ndr"
+  algorithm: "table_perf_trending_dash_html"
+  input-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-1t1c-3n-hsw-ndr.csv"
+  output-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-1t1c-3n-hsw-ndr.rst"
+  testbed: "3n-hsw"
+
+-
+  type: "table"
+  title: "HTML performance trending dashboard 1t1c-3n-hsw pdr"
+  algorithm: "table_perf_trending_dash_html"
+  input-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-1t1c-3n-hsw-pdr.csv"
+  output-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-1t1c-3n-hsw-pdr.rst"
+  testbed: "3n-hsw"
+
+# 3n-skx
+-
+  type: "table"
+  title: "Performance trending dashboard 2t1c-3n-skx ndr"
+  algorithm: "table_perf_trending_dash"
+  output-file-ext: ".csv"
+  output-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-2t1c-3n-skx-ndr"
+  data: "table-ndrpdr-comparison-3n-skx"
+  filter: "'NDRPDR' and '2T1C'"
+  include-tests: "NDR"
+  parameters:
+    - "name"
+    - "parent"
+    - "throughput"
+  window: 4
+  long-trend-window: 12
+
+-
+  type: "table"
+  title: "Performance trending dashboard 2t1c-3n-skx pdr"
+  algorithm: "table_perf_trending_dash"
+  output-file-ext: ".csv"
+  output-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-2t1c-3n-skx-pdr"
+  data: "table-ndrpdr-comparison-3n-skx"
+  filter: "'NDRPDR' and '2T1C'"
+  include-tests: "PDR"
+  parameters:
+    - "name"
+    - "parent"
+    - "throughput"
+  window: 4
+  long-trend-window: 12
+
+-
+  type: "table"
+  title: "HTML performance trending dashboard 2t1c-3n-skx ndr"
+  algorithm: "table_perf_trending_dash_html"
+  input-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-2t1c-3n-skx-ndr.csv"
+  output-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-2t1c-3n-skx-ndr.rst"
+  testbed: "3n-skx"
+
+-
+  type: "table"
+  title: "HTML performance trending dashboard 2t1c-3n-skx pdr"
+  algorithm: "table_perf_trending_dash_html"
+  input-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-2t1c-3n-skx-pdr.csv"
+  output-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-2t1c-3n-skx-pdr.rst"
+  testbed: "3n-skx"
+
+# 2n-skx
+-
+  type: "table"
+  title: "Performance trending dashboard 2t1c-2n-skx ndr"
+  algorithm: "table_perf_trending_dash"
+  output-file-ext: ".csv"
+  output-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-2t1c-2n-skx-ndr"
+  data: "table-ndrpdr-comparison-2n-skx"
+  filter: "'NDRPDR' and '2T1C'"
+  include-tests: "NDR"
+  parameters:
+    - "name"
+    - "parent"
+    - "throughput"
+  window: 4
+  long-trend-window: 12
+
+-
+  type: "table"
+  title: "Performance trending dashboard 2t1c-2n-skx pdr"
+  algorithm: "table_perf_trending_dash"
+  output-file-ext: ".csv"
+  output-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-2t1c-2n-skx-pdr"
+  data: "table-ndrpdr-comparison-2n-skx"
+  filter: "'NDRPDR' and '2T1C'"
+  include-tests: "PDR"
+  parameters:
+    - "name"
+    - "parent"
+    - "throughput"
+  window: 4
+  long-trend-window: 12
+
+-
+  type: "table"
+  title: "HTML performance trending dashboard 2t1c-2n-skx ndr"
+  algorithm: "table_perf_trending_dash_html"
+  input-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-2t1c-2n-skx-ndr.csv"
+  output-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-2t1c-2n-skx-ndr.rst"
+  testbed: "2n-skx"
+
+-
+  type: "table"
+  title: "HTML performance trending dashboard 2t1c-2n-skx pdr"
+  algorithm: "table_perf_trending_dash_html"
+  input-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-2t1c-2n-skx-pdr.csv"
+  output-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-2t1c-2n-skx-pdr.rst"
+  testbed: "2n-skx"
+
+# 2n-clx
+-
+  type: "table"
+  title: "Performance trending dashboard 2t1c-2n-clx ndr"
+  algorithm: "table_perf_trending_dash"
+  output-file-ext: ".csv"
+  output-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-2t1c-2n-clx-ndr"
+  data: "table-ndrpdr-comparison-2n-clx"
+  filter: "'NDRPDR' and '2T1C'"
+  include-tests: "NDR"
+  parameters:
+    - "name"
+    - "parent"
+    - "throughput"
+  window: 14
+  long-trend-window: 180
+
+-
+  type: "table"
+  title: "Performance trending dashboard 2t1c-2n-clx pdr"
+  algorithm: "table_perf_trending_dash"
+  output-file-ext: ".csv"
+  output-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-2t1c-2n-clx-pdr"
+  data: "table-ndrpdr-comparison-2n-clx"
+  filter: "'NDRPDR' and '2T1C'"
+  include-tests: "PDR"
+  parameters:
+    - "name"
+    - "parent"
+    - "throughput"
+  window: 14
+  long-trend-window: 180
+
+-
+  type: "table"
+  title: "HTML performance trending dashboard 2t1c-2n-clx ndr"
+  algorithm: "table_perf_trending_dash_html"
+  input-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-2t1c-2n-clx-ndr.csv"
+  output-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-2t1c-2n-clx-ndr.rst"
+  testbed: "2n-clx"
+
+-
+  type: "table"
+  title: "HTML performance trending dashboard 2t1c-2n-clx pdr"
+  algorithm: "table_perf_trending_dash_html"
+  input-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-2t1c-2n-clx-pdr.csv"
+  output-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-2t1c-2n-clx-pdr.rst"
+  testbed: "2n-clx"
+
 -
   type: "table"
   title: "Failed MRR tests 3n-hsw"
@@ -1796,10 +2005,6 @@
   - "name"
   - "parent"
   - "status"
-  ignore-list:
-    # This test is "ndrdisc" test and was improperly tagged. It was fixed
-    # but it remains in the old output.xml files.
-  - "tests.vpp.perf.l2.10ge2p1x520-eth-l2bdscale1mmaclrn-mrr.tc01-64b-1t1c-eth-l2bdscale1mmaclrn-ndrdisc"
   window: 7  # days
 
 -
@@ -1824,10 +2029,6 @@
   - "name"
   - "parent"
   - "result"
-  ignore-list:
-    # This test is "ndrdisc" test and was improperly tagged. It was fixed
-    # but it remains in the old output.xml files.
-  - "tests.vpp.perf.l2.10ge2p1x520-eth-l2bdscale1mmaclrn-mrr.tc01-64b-2t1c-eth-l2bdscale1mmaclrn-ndrdisc"
   window: 14
   long-trend-window: 180
 
@@ -1843,10 +2044,6 @@
   - "name"
   - "parent"
   - "result"
-  ignore-list:
-    # This test is "ndrdisc" test and was improperly tagged. It was fixed
-    # but it remains in the old output.xml files.
-  - "tests.vpp.perf.l2.10ge2p1x520-eth-l2bdscale1mmaclrn-mrr.tc05-64b-4t2c-eth-l2bdscale1mmaclrn-ndrdisc"
   window: 14
   long-trend-window: 180
 
@@ -1862,10 +2059,6 @@
   - "name"
   - "parent"
   - "result"
-  ignore-list:
-    # This test is "ndrdisc" test and was improperly tagged. It was fixed
-    # but it remains in the old output.xml files.
-  - "tests.vpp.perf.l2.10ge2p1x520-eth-l2bdscale1mmaclrn-mrr.tc09-64b-8t4c-eth-l2bdscale1mmaclrn-ndrdisc"
   window: 14
   long-trend-window: 180
 
@@ -1905,10 +2098,6 @@
   - "name"
   - "parent"
   - "status"
-  ignore-list:
-    # This test is "ndrdisc" test and was improperly tagged. It was fixed
-    # but it remains in the old output.xml files.
-  - "tests.vpp.perf.l2.10ge2p1x520-eth-l2bdscale1mmaclrn-mrr.tc01-64b-2t1c-eth-l2bdscale1mmaclrn-ndrdisc"
   window: 7  # days
 
 -
@@ -1933,10 +2122,6 @@
   - "name"
   - "parent"
   - "result"
-  ignore-list:
-    # This test is "ndrdisc" test and was improperly tagged. It was fixed
-    # but it remains in the old output.xml files.
-  - "tests.vpp.perf.l2.10ge2p1x520-eth-l2bdscale1mmaclrn-mrr.tc01-64b-2t1c-eth-l2bdscale1mmaclrn-ndrdisc"
   window: 14
   long-trend-window: 180
 
@@ -1952,10 +2137,6 @@
   - "name"
   - "parent"
   - "result"
-  ignore-list:
-    # This test is "ndrdisc" test and was improperly tagged. It was fixed
-    # but it remains in the old output.xml files.
-  - "tests.vpp.perf.l2.10ge2p1x520-eth-l2bdscale1mmaclrn-mrr.tc05-64b-4t2c-eth-l2bdscale1mmaclrn-ndrdisc"
   window: 14
   long-trend-window: 180
 
@@ -1971,10 +2152,6 @@
   - "name"
   - "parent"
   - "result"
-  ignore-list:
-    # This test is "ndrdisc" test and was improperly tagged. It was fixed
-    # but it remains in the old output.xml files.
-  - "tests.vpp.perf.l2.10ge2p1x520-eth-l2bdscale1mmaclrn-mrr.tc09-64b-8t4c-eth-l2bdscale1mmaclrn-ndrdisc"
   window: 14
   long-trend-window: 180
 
@@ -2014,10 +2191,6 @@
   - "name"
   - "parent"
   - "status"
-  ignore-list:
-    # This test is "ndrdisc" test and was improperly tagged. It was fixed
-    # but it remains in the old output.xml files.
-  - "tests.vpp.perf.l2.10ge2p1x520-eth-l2bdscale1mmaclrn-mrr.tc01-64b-2t1c-eth-l2bdscale1mmaclrn-ndrdisc"
   window: 7  # days
 
 -