aboutsummaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorTibor Frank <tifrank@cisco.com>2020-05-04 09:13:01 +0200
committerTibor Frank <tifrank@cisco.com>2020-05-05 04:49:09 +0000
commitd8137cbff06412747d100c2a641eb815cd9d2a79 (patch)
tree616fdbe23f95dda12ff42000b289683a3a9a3450
parent8189a403bddf456f95639c6b997a3aeb2af7b76c (diff)
Trending: CSIT-1301: Display stdev in hover info
+ make possible to use dashboard for ndrpdr tests + fixes in ndrpdr weekly comparison Change-Id: I846dc207ea620fad78424a445b32095160d4431d Signed-off-by: Tibor Frank <tifrank@cisco.com> (cherry picked from commit 68079ecfc6265a621d8e91b43d08fff5801f75d7) (cherry picked from commit 60d0062d8cdf93271548d06b06b54eb20280907a)
-rw-r--r--docs/cpta/data/index.rst13
-rw-r--r--docs/cpta/introduction/dashboard.rst41
-rw-r--r--docs/cpta/introduction/weekly.rst51
-rw-r--r--resources/tools/presentation/generator_cpta.py28
-rw-r--r--resources/tools/presentation/generator_tables.py32
-rw-r--r--resources/tools/presentation/pal_utils.py9
-rw-r--r--resources/tools/presentation/specification_CPTA.yaml303
7 files changed, 356 insertions, 121 deletions
diff --git a/docs/cpta/data/index.rst b/docs/cpta/data/index.rst
index 7d04514..3f7d313 100644
--- a/docs/cpta/data/index.rst
+++ b/docs/cpta/data/index.rst
@@ -2,63 +2,52 @@ Performance Data
================
Performance test data used to generate the trending graphs is available
-in the ASCII and CSV formats:
+in the CSV formats:
**VPP**
- 3n-hsw
- - `ASCII format <../_static/vpp/csit-vpp-perf-mrr-daily-master-trending.txt>`_
- `CSV format <../_static/vpp/csit-vpp-perf-mrr-daily-master-trending.csv>`_
- 3n-skx
- - `ASCII format <../_static/vpp/csit-vpp-perf-mrr-daily-master-3n-skx-trending.txt>`_
- `CSV format <../_static/vpp/csit-vpp-perf-mrr-daily-master-3n-skx-trending.csv>`_
- 2n-skx
- - `ASCII format <../_static/vpp/csit-vpp-perf-mrr-daily-master-2n-skx-trending.txt>`_
- `CSV format <../_static/vpp/csit-vpp-perf-mrr-daily-master-2n-skx-trending.csv>`_
- 2n-clx
- - `ASCII format <../_static/vpp/csit-vpp-perf-mrr-daily-master-2n-clx-trending.txt>`_
- `CSV format <../_static/vpp/csit-vpp-perf-mrr-daily-master-2n-clx-trending.csv>`_
- 3n-tsh
- - `ASCII format <../_static/vpp/csit-vpp-perf-mrr-daily-master-3n-tsh-trending.txt>`_
- `CSV format <../_static/vpp/csit-vpp-perf-mrr-daily-master-3n-tsh-trending.csv>`_
- 3n-dnv
- - `ASCII format <../_static/vpp/csit-vpp-perf-mrr-daily-master-3n-dnv-trending.txt>`_
- `CSV format <../_static/vpp/csit-vpp-perf-mrr-daily-master-3n-dnv-trending.csv>`_
- 2n-dnv
- - `ASCII format <../_static/vpp/csit-vpp-perf-mrr-daily-master-2n-dnv-trending.txt>`_
- `CSV format <../_static/vpp/csit-vpp-perf-mrr-daily-master-2n-dnv-trending.csv>`_
**DPDK**
- 3n-hsw
- - `ASCII format <../_static/vpp/csit-dpdk-perf-mrr-weekly-master-trending.txt>`_
- `CSV format <../_static/vpp/csit-dpdk-perf-mrr-weekly-master-trending.csv>`_
- 3n-skx
- - `ASCII format <../_static/vpp/csit-dpdk-perf-mrr-weekly-master-3n-skx-trending.txt>`_
- `CSV format <../_static/vpp/csit-dpdk-perf-mrr-weekly-master-3n-skx-trending.csv>`_
- 2n-skx
- - `ASCII format <../_static/vpp/csit-dpdk-perf-mrr-weekly-master-2n-skx-trending.txt>`_
- `CSV format <../_static/vpp/csit-dpdk-perf-mrr-weekly-master-2n-skx-trending.csv>`_
- 2n-clx
- - `ASCII format <../_static/vpp/csit-dpdk-perf-mrr-weekly-master-2n-clx-trending.txt>`_
- `CSV format <../_static/vpp/csit-dpdk-perf-mrr-weekly-master-2n-clx-trending.csv>`_
diff --git a/docs/cpta/introduction/dashboard.rst b/docs/cpta/introduction/dashboard.rst
index 38754b5..8c1403c 100644
--- a/docs/cpta/introduction/dashboard.rst
+++ b/docs/cpta/introduction/dashboard.rst
@@ -31,6 +31,16 @@ VPP worker on 8t4c
.. include:: ../../../_build/_static/vpp/performance-trending-dashboard-8t4c-2n-skx.rst
+NDR Weekly VPP worker on 2t1c
+`````````````````````````````
+
+.. include:: ../../../_build/_static/vpp/performance-trending-dashboard-2t1c-2n-skx-ndr.rst
+
+PDR Weekly VPP worker on 2t1c
+`````````````````````````````
+
+.. include:: ../../../_build/_static/vpp/performance-trending-dashboard-2t1c-2n-skx-pdr.rst
+
3n-skx
------
@@ -49,6 +59,16 @@ VPP worker on 8t4c
.. include:: ../../../_build/_static/vpp/performance-trending-dashboard-8t4c-3n-skx.rst
+NDR Weekly VPP worker on 2t1c
+`````````````````````````````
+
+.. include:: ../../../_build/_static/vpp/performance-trending-dashboard-2t1c-3n-skx-ndr.rst
+
+PDR Weekly VPP worker on 2t1c
+`````````````````````````````
+
+.. include:: ../../../_build/_static/vpp/performance-trending-dashboard-2t1c-3n-skx-pdr.rst
+
3n-hsw
------
@@ -67,6 +87,17 @@ VPP worker on 4t4c
.. include:: ../../../_build/_static/vpp/performance-trending-dashboard-4t4c-3n-hsw.rst
+
+NDR Weekly VPP worker on 2t1c
+`````````````````````````````
+
+.. include:: ../../../_build/_static/vpp/performance-trending-dashboard-1t1c-3n-hsw-ndr.rst
+
+PDR Weekly VPP worker on 2t1c
+`````````````````````````````
+
+.. include:: ../../../_build/_static/vpp/performance-trending-dashboard-1t1c-3n-hsw-pdr.rst
+
3n-tsh
------
@@ -138,3 +169,13 @@ VPP worker on 8t4c
``````````````````
.. include:: ../../../_build/_static/vpp/performance-trending-dashboard-8t4c-2n-clx.rst
+
+NDR Weekly VPP worker on 2t1c
+`````````````````````````````
+
+.. include:: ../../../_build/_static/vpp/performance-trending-dashboard-2t1c-2n-clx-ndr.rst
+
+PDR Weekly VPP worker on 2t1c
+`````````````````````````````
+
+.. include:: ../../../_build/_static/vpp/performance-trending-dashboard-2t1c-2n-clx-pdr.rst
diff --git a/docs/cpta/introduction/weekly.rst b/docs/cpta/introduction/weekly.rst
index 0118f44..bf73a58 100644
--- a/docs/cpta/introduction/weekly.rst
+++ b/docs/cpta/introduction/weekly.rst
@@ -4,56 +4,65 @@ NDRPDR Weekly Comparison
2n-skx
------
+NDR Comparison
+``````````````
+
Comparison tables in HTML, ASCII and CSV formats:
- - `HTML 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-skx-2t1c-ndr_in.html>`_
- - `ASCII 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-skx-2t1c-ndr.txt>`_
- - `CSV 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-skx-2t1c-ndr.csv>`_
+ - `HTML 2t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-skx-2t1c-ndr_in.html>`_
+ - `ASCII 2t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-skx-2t1c-ndr.txt>`_
+ - `CSV 2t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-skx-2t1c-ndr.csv>`_
PDR Comparison
``````````````
Comparison tables in HTML, ASCII and CSV formats:
- - `HTML 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-skx-2t1c-pdr_in.html>`_
- - `ASCII 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-skx-2t1c-pdr.txt>`_
- - `CSV 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-skx-2t1c-pdr.csv>`_
+ - `HTML 2t1c PDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-skx-2t1c-pdr_in.html>`_
+ - `ASCII 2t1c PDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-skx-2t1c-pdr.txt>`_
+ - `CSV 2t1c PDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-skx-2t1c-pdr.csv>`_
3n-skx
------
+NDR Comparison
+``````````````
+
Comparison tables in HTML, ASCII and CSV formats:
- - `HTML 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-3n-skx-2t1c-ndr_in.html>`_
- - `ASCII 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-3n-skx-2t1c-ndr.txt>`_
- - `CSV 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-3n-skx-2t1c-ndr.csv>`_
+ - `HTML 2t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-3n-skx-2t1c-ndr_in.html>`_
+ - `ASCII 2t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-3n-skx-2t1c-ndr.txt>`_
+ - `CSV 2t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-3n-skx-2t1c-ndr.csv>`_
PDR Comparison
``````````````
Comparison tables in HTML, ASCII and CSV formats:
- - `HTML 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-3n-skx-2t1c-pdr_in.html>`_
- - `ASCII 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-3n-skx-2t1c-pdr.txt>`_
- - `CSV 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-3n-skx-2t1c-pdr.csv>`_
+ - `HTML 2t1c PDR comparison <../_static/vpp/ndrpdr-weekly-comp-3n-skx-2t1c-pdr_in.html>`_
+ - `ASCII 2t1c PDR comparison <../_static/vpp/ndrpdr-weekly-comp-3n-skx-2t1c-pdr.txt>`_
+ - `CSV 2t1c PDR comparison <../_static/vpp/ndrpdr-weekly-comp-3n-skx-2t1c-pdr.csv>`_
2n-clx
------
+NDR Comparison
+``````````````
+
Comparison tables in HTML, ASCII and CSV formats:
- - `HTML 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-clx-2t1c-ndr_in.html>`_
- - `ASCII 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-clx-2t1c-ndr.txt>`_
- - `CSV 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-clx-2t1c-ndr.csv>`_
+ - `HTML 2t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-clx-2t1c-ndr_in.html>`_
+ - `ASCII 2t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-clx-2t1c-ndr.txt>`_
+ - `CSV 2t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-clx-2t1c-ndr.csv>`_
PDR Comparison
``````````````
Comparison tables in HTML, ASCII and CSV formats:
- - `HTML 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-clx-2t1c-pdr_in.html>`_
- - `ASCII 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-clx-2t1c-pdr.txt>`_
- - `CSV 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-clx-2t1c-pdr.csv>`_
+ - `HTML 2t1c PDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-clx-2t1c-pdr_in.html>`_
+ - `ASCII 2t1c PDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-clx-2t1c-pdr.txt>`_
+ - `CSV 2t1c PDR comparison <../_static/vpp/ndrpdr-weekly-comp-2n-clx-2t1c-pdr.csv>`_
3n-hsw
------
@@ -72,6 +81,6 @@ PDR Comparison
Comparison tables in HTML, ASCII and CSV formats:
- - `HTML 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-3n-hsw-1t1c-pdr_in.html>`_
- - `ASCII 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-3n-hsw-1t1c-pdr.txt>`_
- - `CSV 1t1c NDR comparison <../_static/vpp/ndrpdr-weekly-comp-3n-hsw-1t1c-pdr.csv>`_
+ - `HTML 1t1c PDR comparison <../_static/vpp/ndrpdr-weekly-comp-3n-hsw-1t1c-pdr_in.html>`_
+ - `ASCII 1t1c PDR comparison <../_static/vpp/ndrpdr-weekly-comp-3n-hsw-1t1c-pdr.txt>`_
+ - `CSV 1t1c PDR comparison <../_static/vpp/ndrpdr-weekly-comp-3n-hsw-1t1c-pdr.csv>`_
diff --git a/resources/tools/presentation/generator_cpta.py b/resources/tools/presentation/generator_cpta.py
index ac0a5c6..511800a 100644
--- a/resources/tools/presentation/generator_cpta.py
+++ b/resources/tools/presentation/generator_cpta.py
@@ -169,8 +169,13 @@ def _generate_trending_traces(in_data, job_name, build_info,
"""
data_x = list(in_data.keys())
- data_y_pps = list(in_data.values())
- data_y_mpps = [float(item) / 1e6 for item in data_y_pps]
+ data_y_pps = list()
+ data_y_mpps = list()
+ data_y_stdev = list()
+ for item in in_data.values():
+ data_y_pps.append(float(item[u"receive-rate"]))
+ data_y_stdev.append(float(item[u"receive-stdev"]) / 1e6)
+ data_y_mpps.append(float(item[u"receive-rate"]) / 1e6)
hover_text = list()
xaxis = list()
@@ -179,6 +184,7 @@ def _generate_trending_traces(in_data, job_name, build_info,
date = build_info[job_name][str_key][0]
hover_str = (u"date: {date}<br>"
u"value [Mpps]: {value:.3f}<br>"
+ u"stdev [Mpps]: {stdev:.3f}<br>"
u"{sut}-ref: {build}<br>"
u"csit-ref: mrr-{period}-build-{build_nr}<br>"
u"testbed: {testbed}")
@@ -186,6 +192,7 @@ def _generate_trending_traces(in_data, job_name, build_info,
hover_text.append(hover_str.format(
date=date,
value=data_y_mpps[index],
+ stdev=data_y_stdev[index],
sut=u"dpdk",
build=build_info[job_name][str_key][1].rsplit(u'~', 1)[0],
period=u"weekly",
@@ -195,6 +202,7 @@ def _generate_trending_traces(in_data, job_name, build_info,
hover_text.append(hover_str.format(
date=date,
value=data_y_mpps[index],
+ stdev=data_y_stdev[index],
sut=u"vpp",
build=build_info[job_name][str_key][1].rsplit(u'~', 1)[0],
period=u"daily",
@@ -221,8 +229,7 @@ def _generate_trending_traces(in_data, job_name, build_info,
}
if anomaly_classification:
for index, (key, value) in enumerate(data_pd.items()):
- if anomaly_classification[index] in \
- (u"outlier", u"regression", u"progression"):
+ if anomaly_classification[index] in (u"regression", u"progression"):
anomalies[key] = value / 1e6
anomalies_colors.append(
anomaly_color[anomaly_classification[index]])
@@ -377,8 +384,10 @@ def _generate_all_charts(spec, input_data):
if chart_data.get(test_name, None) is None:
chart_data[test_name] = OrderedDict()
try:
- chart_data[test_name][int(index)] = \
- test[u"result"][u"receive-rate"]
+ chart_data[test_name][int(index)] = {
+ u"receive-rate": test[u"result"][u"receive-rate"],
+ u"receive-stdev": test[u"result"][u"receive-stdev"]
+ }
chart_tags[test_name] = test.get(u"tags", None)
except (KeyError, TypeError):
pass
@@ -387,9 +396,12 @@ def _generate_all_charts(spec, input_data):
for tst_name, tst_data in chart_data.items():
tst_lst = list()
for bld in builds_dict[job_name]:
- itm = tst_data.get(int(bld), u'')
+ itm = tst_data.get(int(bld), dict())
# CSIT-1180: Itm will be list, compute stats.
- tst_lst.append(str(itm))
+ try:
+ tst_lst.append(str(itm.get(u"receive-rate", u"")))
+ except AttributeError:
+ tst_lst.append(u"")
csv_tbl.append(f"{tst_name}," + u",".join(tst_lst) + u'\n')
# Generate traces:
diff --git a/resources/tools/presentation/generator_tables.py b/resources/tools/presentation/generator_tables.py
index 021744d..4a4ccce 100644
--- a/resources/tools/presentation/generator_tables.py
+++ b/resources/tools/presentation/generator_tables.py
@@ -829,6 +829,8 @@ def table_perf_trending_dash(table, input_data):
]
header_str = u",".join(header) + u"\n"
+ incl_tests = table.get(u"include-tests", u"MRR")
+
# Prepare data to the table:
tbl_dict = dict()
for job, builds in table[u"data"].items():
@@ -846,8 +848,15 @@ def table_perf_trending_dash(table, input_data):
u"data": OrderedDict()
}
try:
- tbl_dict[tst_name][u"data"][str(build)] = \
- tst_data[u"result"][u"receive-rate"]
+ if incl_tests == u"MRR":
+ tbl_dict[tst_name][u"data"][str(build)] = \
+ tst_data[u"result"][u"receive-rate"]
+ elif incl_tests == u"NDR":
+ tbl_dict[tst_name][u"data"][str(build)] = \
+ tst_data[u"throughput"][u"NDR"][u"LOWER"]
+ elif incl_tests == u"PDR":
+ tbl_dict[tst_name][u"data"][str(build)] = \
+ tst_data[u"throughput"][u"PDR"][u"LOWER"]
except (TypeError, KeyError):
pass # No data in output.xml for this test
@@ -1790,10 +1799,10 @@ def table_weekly_comparison(table, in_data):
)
header = [
- [u"Version"],
- [u"Date", ],
- [u"Build", ],
- [u"Testbed", ]
+ [u"VPP Version", ],
+ [u"Start Timestamp", ],
+ [u"CSIT Build", ],
+ [u"CSIT Testbed", ]
]
tbl_dict = dict()
idx = 0
@@ -1843,10 +1852,13 @@ def table_weekly_comparison(table, in_data):
idx_cmp = cmp.get(u"compare", None)
if idx_ref is None or idx_cmp is None:
continue
- header[0].append(f"Diff{idx + 1}")
- header[1].append(header[0][idx_ref - idx - 1])
- header[2].append(u"vs")
- header[3].append(header[0][idx_cmp - idx - 1])
+ header[0].append(
+ f"Diff({header[0][idx_ref - idx].split(u'~')[-1]} vs "
+ f"{header[0][idx_cmp - idx].split(u'~')[-1]})"
+ )
+ header[1].append(u"")
+ header[2].append(u"")
+ header[3].append(u"")
for tst_name, tst_data in tbl_dict.items():
if not cmp_dict.get(tst_name, None):
cmp_dict[tst_name] = list()
diff --git a/resources/tools/presentation/pal_utils.py b/resources/tools/presentation/pal_utils.py
index 1b9131a..98d5837 100644
--- a/resources/tools/presentation/pal_utils.py
+++ b/resources/tools/presentation/pal_utils.py
@@ -318,14 +318,13 @@ def convert_csv_to_pretty_txt(csv_file_name, txt_file_name, delimiter=u","):
txt_table.add_row(
[str(itm.replace(u"\u00B1", u"+-")) for itm in row]
)
- txt_table.align = u"r"
- txt_table.align[u"Test Case"] = u"l"
- txt_table.align[u"Build"] = u"l"
- txt_table.align[u"Version"] = u"l"
-
if not txt_table:
return
+ txt_table.align = u"r"
+ for itm in (u"Test Case", u"Build", u"Version", u"VPP Version"):
+ txt_table.align[itm] = u"l"
+
if txt_file_name.endswith(u".txt"):
with open(txt_file_name, u"wt", encoding='utf-8') as txt_file:
txt_file.write(str(txt_table))
diff --git a/resources/tools/presentation/specification_CPTA.yaml b/resources/tools/presentation/specification_CPTA.yaml
index 2068499..3a48578 100644
--- a/resources/tools/presentation/specification_CPTA.yaml
+++ b/resources/tools/presentation/specification_CPTA.yaml
@@ -889,32 +889,32 @@
table-ndrpdr-comparison-2n-skx:
csit-vpp-perf-ndrpdr-weekly-master-2n-skx:
- start: 84
+ start: 76
end: "lastCompletedBuild"
- max-builds: 4
+ max-builds: 15
reverse: True # Must be reversed
table-ndrpdr-comparison-3n-skx:
csit-vpp-perf-ndrpdr-weekly-master-3n-skx:
- start: 84
+ start: 76
end: "lastCompletedBuild"
- max-builds: 4
+ max-builds: 15
reverse: True # Must be reversed
skip:
- 87
table-ndrpdr-comparison-2n-clx:
csit-vpp-perf-ndrpdr-weekly-master-2n-clx:
- start: 30
+ start: 21
end: "lastCompletedBuild"
- max-builds: 4
+ max-builds: 15
reverse: True # Must be reversed
table-ndrpdr-comparison-3n-hsw:
csit-vpp-perf-ndrpdr-weekly-master-3n-hsw:
- start: 84
+ start: 76
end: "lastCompletedBuild"
- max-builds: 4
+ max-builds: 15
reverse: True # Must be reversed
plot-layouts:
@@ -1013,29 +1013,29 @@
# 2n-skx ndrpdr
csit-vpp-perf-ndrpdr-weekly-master-2n-skx:
- start: 84
+ start: 76
end: "lastCompletedBuild"
- max-builds: 4
+ max-builds: 15
# 3n-skx ndrpdr
csit-vpp-perf-ndrpdr-weekly-master-3n-skx:
- start: 84
+ start: 76
end: "lastCompletedBuild"
- max-builds: 4
+ max-builds: 15
skip:
- 87
# 2n-clx ndrpdr
csit-vpp-perf-ndrpdr-weekly-master-2n-clx:
- start: 30
+ start: 21
end: "lastCompletedBuild"
- max-builds: 4
+ max-builds: 15
# 3n-hsw ndrpdr
csit-vpp-perf-ndrpdr-weekly-master-3n-hsw:
- start: 84
+ start: 76
end: "lastCompletedBuild"
- max-builds: 4
+ max-builds: 15
# 3n-hsw
csit-vpp-perf-mrr-daily-master:
@@ -1092,7 +1092,11 @@
=======
>>>>>>> CHANGE (51210b Trending: Update graphs - dnv)
- 1102
+<<<<<<< HEAD (8189a4 Report: Add data)
>>>>>>> CHANGE (b51708 Trending: Plots)
+=======
+ - 1159
+>>>>>>> CHANGE (60d006 Trending: CSIT-1301: Display stdev in hover info)
csit-dpdk-perf-mrr-weekly-master:
start: 113
end: "lastCompletedBuild"
@@ -1174,7 +1178,11 @@
=======
- 865
- 866
+<<<<<<< HEAD (8189a4 Report: Add data)
>>>>>>> CHANGE (12d5c9 Trending: Fixes in plots, data)
+=======
+ - 867
+>>>>>>> CHANGE (60d006 Trending: CSIT-1301: Display stdev in hover info)
csit-dpdk-perf-mrr-weekly-master-3n-skx:
start: 78
end: "lastCompletedBuild"
@@ -1239,7 +1247,11 @@
- 868
- 870
- 874
+<<<<<<< HEAD (8189a4 Report: Add data)
>>>>>>> CHANGE (01597c Trending: Update graphs)
+=======
+ - 889
+>>>>>>> CHANGE (60d006 Trending: CSIT-1301: Display stdev in hover info)
csit-vpp-perf-mrr-weekly-master-2n-skx:
start: 50
end: "lastCompletedBuild"
@@ -1289,6 +1301,7 @@
- 251
- 252
- 253
+ - 308
csit-dpdk-perf-mrr-weekly-master-2n-clx:
start: 23
@@ -1336,7 +1349,11 @@
- 155
- 156
- 157
+<<<<<<< HEAD (8189a4 Report: Add data)
>>>>>>> CHANGE (01597c Trending: Update graphs)
+=======
+ - 192
+>>>>>>> CHANGE (60d006 Trending: CSIT-1301: Display stdev in hover info)
# 3n-dnv
csit-vpp-perf-mrr-daily-master-3n-dnv:
@@ -1410,7 +1427,11 @@
- 390
- 391
- 392
+<<<<<<< HEAD (8189a4 Report: Add data)
>>>>>>> CHANGE (12d5c9 Trending: Fixes in plots, data)
+=======
+ - 396
+>>>>>>> CHANGE (60d006 Trending: CSIT-1301: Display stdev in hover info)
# 2n-dnv
csit-vpp-perf-mrr-daily-master-2n-dnv:
@@ -1461,7 +1482,7 @@
- 350
- 351
- 354
- - 360
+ - 36
- 364
- 365
- 366
@@ -1475,7 +1496,11 @@
- 395
- 396
- 397
+<<<<<<< HEAD (8189a4 Report: Add data)
>>>>>>> CHANGE (12d5c9 Trending: Fixes in plots, data)
+=======
+ - 402
+>>>>>>> CHANGE (60d006 Trending: CSIT-1301: Display stdev in hover info)
-
type: "output"
@@ -1715,10 +1740,6 @@
- "name"
- "parent"
- "result"
- ignore-list:
- # This test is "ndrdisc" test and was improperly tagged. It was fixed
- # but it remains in the old output.xml files.
- - "tests.vpp.perf.l2.10ge2p1x520-eth-l2bdscale1mmaclrn-mrr.tc01-64b-1t1c-eth-l2bdscale1mmaclrn-ndrdisc"
window: 14
long-trend-window: 180
@@ -1734,10 +1755,6 @@
- "name"
- "parent"
- "result"
- ignore-list:
- # This test is "ndrdisc" test and was improperly tagged. It was fixed
- # but it remains in the old output.xml files.
- - "tests.vpp.perf.l2.10ge2p1x520-eth-l2bdscale1mmaclrn-mrr.tc05-64b-2t2c-eth-l2bdscale1mmaclrn-ndrdisc"
window: 14
long-trend-window: 180
@@ -1753,10 +1770,6 @@
- "name"
- "parent"
- "result"
- ignore-list:
- # This test is "ndrdisc" test and was improperly tagged. It was fixed
- # but it remains in the old output.xml files.
- - "tests.vpp.perf.l2.10ge2p1x520-eth-l2bdscale1mmaclrn-mrr.tc09-64b-4t4c-eth-l2bdscale1mmaclrn-ndrdisc"
window: 14
long-trend-window: 180
@@ -1784,6 +1797,202 @@
output-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-4t4c-3n-hsw.rst"
testbed: "3n-hsw"
+# 3n-hsw
+-
+ type: "table"
+ title: "Performance trending dashboard 1t1c-3n-hsw ndr"
+ algorithm: "table_perf_trending_dash"
+ output-file-ext: ".csv"
+ output-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-1t1c-3n-hsw-ndr"
+ data: "table-ndrpdr-comparison-3n-hsw"
+ filter: "'NDRPDR' and '1T1C'"
+ include-tests: "NDR"
+ parameters:
+ - "name"
+ - "parent"
+ - "throughput"
+ window: 4
+ long-trend-window: 12
+
+-
+ type: "table"
+ title: "Performance trending dashboard 1t1c-3n-hsw pdr"
+ algorithm: "table_perf_trending_dash"
+ output-file-ext: ".csv"
+ output-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-1t1c-3n-hsw-pdr"
+ data: "table-ndrpdr-comparison-3n-hsw"
+ filter: "'NDRPDR' and '1T1C'"
+ include-tests: "PDR"
+ parameters:
+ - "name"
+ - "parent"
+ - "throughput"
+ window: 4
+ long-trend-window: 12
+
+-
+ type: "table"
+ title: "HTML performance trending dashboard 1t1c-3n-hsw ndr"
+ algorithm: "table_perf_trending_dash_html"
+ input-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-1t1c-3n-hsw-ndr.csv"
+ output-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-1t1c-3n-hsw-ndr.rst"
+ testbed: "3n-hsw"
+
+-
+ type: "table"
+ title: "HTML performance trending dashboard 1t1c-3n-hsw pdr"
+ algorithm: "table_perf_trending_dash_html"
+ input-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-1t1c-3n-hsw-pdr.csv"
+ output-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-1t1c-3n-hsw-pdr.rst"
+ testbed: "3n-hsw"
+
+# 3n-skx
+-
+ type: "table"
+ title: "Performance trending dashboard 2t1c-3n-skx ndr"
+ algorithm: "table_perf_trending_dash"
+ output-file-ext: ".csv"
+ output-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-2t1c-3n-skx-ndr"
+ data: "table-ndrpdr-comparison-3n-skx"
+ filter: "'NDRPDR' and '2T1C'"
+ include-tests: "NDR"
+ parameters:
+ - "name"
+ - "parent"
+ - "throughput"
+ window: 4
+ long-trend-window: 12
+
+-
+ type: "table"
+ title: "Performance trending dashboard 2t1c-3n-skx pdr"
+ algorithm: "table_perf_trending_dash"
+ output-file-ext: ".csv"
+ output-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-2t1c-3n-skx-pdr"
+ data: "table-ndrpdr-comparison-3n-skx"
+ filter: "'NDRPDR' and '2T1C'"
+ include-tests: "PDR"
+ parameters:
+ - "name"
+ - "parent"
+ - "throughput"
+ window: 4
+ long-trend-window: 12
+
+-
+ type: "table"
+ title: "HTML performance trending dashboard 2t1c-3n-skx ndr"
+ algorithm: "table_perf_trending_dash_html"
+ input-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-2t1c-3n-skx-ndr.csv"
+ output-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-2t1c-3n-skx-ndr.rst"
+ testbed: "3n-skx"
+
+-
+ type: "table"
+ title: "HTML performance trending dashboard 2t1c-3n-skx pdr"
+ algorithm: "table_perf_trending_dash_html"
+ input-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-2t1c-3n-skx-pdr.csv"
+ output-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-2t1c-3n-skx-pdr.rst"
+ testbed: "3n-skx"
+
+# 2n-skx
+-
+ type: "table"
+ title: "Performance trending dashboard 2t1c-2n-skx ndr"
+ algorithm: "table_perf_trending_dash"
+ output-file-ext: ".csv"
+ output-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-2t1c-2n-skx-ndr"
+ data: "table-ndrpdr-comparison-2n-skx"
+ filter: "'NDRPDR' and '2T1C'"
+ include-tests: "NDR"
+ parameters:
+ - "name"
+ - "parent"
+ - "throughput"
+ window: 4
+ long-trend-window: 12
+
+-
+ type: "table"
+ title: "Performance trending dashboard 2t1c-2n-skx pdr"
+ algorithm: "table_perf_trending_dash"
+ output-file-ext: ".csv"
+ output-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-2t1c-2n-skx-pdr"
+ data: "table-ndrpdr-comparison-2n-skx"
+ filter: "'NDRPDR' and '2T1C'"
+ include-tests: "PDR"
+ parameters:
+ - "name"
+ - "parent"
+ - "throughput"
+ window: 4
+ long-trend-window: 12
+
+-
+ type: "table"
+ title: "HTML performance trending dashboard 2t1c-2n-skx ndr"
+ algorithm: "table_perf_trending_dash_html"
+ input-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-2t1c-2n-skx-ndr.csv"
+ output-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-2t1c-2n-skx-ndr.rst"
+ testbed: "2n-skx"
+
+-
+ type: "table"
+ title: "HTML performance trending dashboard 2t1c-2n-skx pdr"
+ algorithm: "table_perf_trending_dash_html"
+ input-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-2t1c-2n-skx-pdr.csv"
+ output-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-2t1c-2n-skx-pdr.rst"
+ testbed: "2n-skx"
+
+# 2n-clx
+-
+ type: "table"
+ title: "Performance trending dashboard 2t1c-2n-clx ndr"
+ algorithm: "table_perf_trending_dash"
+ output-file-ext: ".csv"
+ output-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-2t1c-2n-clx-ndr"
+ data: "table-ndrpdr-comparison-2n-clx"
+ filter: "'NDRPDR' and '2T1C'"
+ include-tests: "NDR"
+ parameters:
+ - "name"
+ - "parent"
+ - "throughput"
+ window: 14
+ long-trend-window: 180
+
+-
+ type: "table"
+ title: "Performance trending dashboard 2t1c-2n-clx pdr"
+ algorithm: "table_perf_trending_dash"
+ output-file-ext: ".csv"
+ output-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-2t1c-2n-clx-pdr"
+ data: "table-ndrpdr-comparison-2n-clx"
+ filter: "'NDRPDR' and '2T1C'"
+ include-tests: "PDR"
+ parameters:
+ - "name"
+ - "parent"
+ - "throughput"
+ window: 14
+ long-trend-window: 180
+
+-
+ type: "table"
+ title: "HTML performance trending dashboard 2t1c-2n-clx ndr"
+ algorithm: "table_perf_trending_dash_html"
+ input-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-2t1c-2n-clx-ndr.csv"
+ output-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-2t1c-2n-clx-ndr.rst"
+ testbed: "2n-clx"
+
+-
+ type: "table"
+ title: "HTML performance trending dashboard 2t1c-2n-clx pdr"
+ algorithm: "table_perf_trending_dash_html"
+ input-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-2t1c-2n-clx-pdr.csv"
+ output-file: "{DIR[STATIC,VPP]}/performance-trending-dashboard-2t1c-2n-clx-pdr.rst"
+ testbed: "2n-clx"
+
-
type: "table"
title: "Failed MRR tests 3n-hsw"
@@ -1796,10 +2005,6 @@
- "name"
- "parent"
- "status"
- ignore-list:
- # This test is "ndrdisc" test and was improperly tagged. It was fixed
- # but it remains in the old output.xml files.
- - "tests.vpp.perf.l2.10ge2p1x520-eth-l2bdscale1mmaclrn-mrr.tc01-64b-1t1c-eth-l2bdscale1mmaclrn-ndrdisc"
window: 7 # days
-
@@ -1824,10 +2029,6 @@
- "name"
- "parent"
- "result"
- ignore-list:
- # This test is "ndrdisc" test and was improperly tagged. It was fixed
- # but it remains in the old output.xml files.
- - "tests.vpp.perf.l2.10ge2p1x520-eth-l2bdscale1mmaclrn-mrr.tc01-64b-2t1c-eth-l2bdscale1mmaclrn-ndrdisc"
window: 14
long-trend-window: 180
@@ -1843,10 +2044,6 @@
- "name"
- "parent"
- "result"
- ignore-list:
- # This test is "ndrdisc" test and was improperly tagged. It was fixed
- # but it remains in the old output.xml files.
- - "tests.vpp.perf.l2.10ge2p1x520-eth-l2bdscale1mmaclrn-mrr.tc05-64b-4t2c-eth-l2bdscale1mmaclrn-ndrdisc"
window: 14
long-trend-window: 180
@@ -1862,10 +2059,6 @@
- "name"
- "parent"
- "result"
- ignore-list:
- # This test is "ndrdisc" test and was improperly tagged. It was fixed
- # but it remains in the old output.xml files.
- - "tests.vpp.perf.l2.10ge2p1x520-eth-l2bdscale1mmaclrn-mrr.tc09-64b-8t4c-eth-l2bdscale1mmaclrn-ndrdisc"
window: 14
long-trend-window: 180
@@ -1905,10 +2098,6 @@
- "name"
- "parent"
- "status"
- ignore-list:
- # This test is "ndrdisc" test and was improperly tagged. It was fixed
- # but it remains in the old output.xml files.
- - "tests.vpp.perf.l2.10ge2p1x520-eth-l2bdscale1mmaclrn-mrr.tc01-64b-2t1c-eth-l2bdscale1mmaclrn-ndrdisc"
window: 7 # days
-
@@ -1933,10 +2122,6 @@
- "name"
- "parent"
- "result"
- ignore-list:
- # This test is "ndrdisc" test and was improperly tagged. It was fixed
- # but it remains in the old output.xml files.
- - "tests.vpp.perf.l2.10ge2p1x520-eth-l2bdscale1mmaclrn-mrr.tc01-64b-2t1c-eth-l2bdscale1mmaclrn-ndrdisc"
window: 14
long-trend-window: 180
@@ -1952,10 +2137,6 @@
- "name"
- "parent"
- "result"
- ignore-list:
- # This test is "ndrdisc" test and was improperly tagged. It was fixed
- # but it remains in the old output.xml files.
- - "tests.vpp.perf.l2.10ge2p1x520-eth-l2bdscale1mmaclrn-mrr.tc05-64b-4t2c-eth-l2bdscale1mmaclrn-ndrdisc"
window: 14
long-trend-window: 180
@@ -1971,10 +2152,6 @@
- "name"
- "parent"
- "result"
- ignore-list:
- # This test is "ndrdisc" test and was improperly tagged. It was fixed
- # but it remains in the old output.xml files.
- - "tests.vpp.perf.l2.10ge2p1x520-eth-l2bdscale1mmaclrn-mrr.tc09-64b-8t4c-eth-l2bdscale1mmaclrn-ndrdisc"
window: 14
long-trend-window: 180
@@ -2014,10 +2191,6 @@
- "name"
- "parent"
- "status"
- ignore-list:
- # This test is "ndrdisc" test and was improperly tagged. It was fixed
- # but it remains in the old output.xml files.
- - "tests.vpp.perf.l2.10ge2p1x520-eth-l2bdscale1mmaclrn-mrr.tc01-64b-2t1c-eth-l2bdscale1mmaclrn-ndrdisc"
window: 7 # days
-