CSIT-901: Throughput Speedup Analysis - Static content

 - Add robustnes to the function which generates the plots
 - Add "Model for TSA" to LLD.

Change-Id: I78dffe3622b4566b30ecac411f8ff285689d1427
Signed-off-by: Tibor Frank <tifrank@cisco.com>
Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>

4 files changed, 206 insertions, 30 deletions

diff --git a/resources/tools/presentation/doc/pal_lld.rst b/resources/tools/presentation/doc/pal_lld.rst
index 12318ccfe9..027d6b326a 100644
--- a/resources/tools/presentation/doc/pal_lld.rst
+++ b/resources/tools/presentation/doc/pal_lld.rst
@@ -1139,43 +1139,62 @@ Data analytics part implements:
 
  - methods to compute statistical data from the filtered input data.
  - trending.
- - etc.
 
+Throughput Speedup Analysis - Multi-Core with Multi-Threading
+'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
 
-Throughput Speedup Analysis - Multi-Core Speedup Ratio
-''''''''''''''''''''''''''''''''''''''''''''''''''''''
-
-Throughput Speedup Analysis (TSA) calculates a speedup factor for 1, 2, 4 cores
-which is defined as:
+Throughput Speedup Analysis (TSA) calculates throughput speedup ratios
+for tested 1-, 2- and 4-core multi-threaded VPP configurations using the
+following formula:
 
 ::
 
-                        throughput
-    speedup factor = -----------------
-                     1-core-throughput
+                                N_core_throughput
+    N_core_throughput_speedup = -----------------
+                                1_core_throughput
 
-A bar plot displays the speedup factor (normalized throughput for 64B/78B on 1
-core). The plot displays number of cores on the X-axis and the speedup factor on
-the Y-axis.
+Multi-core throughput speedup ratios are plotted in grouped bar graphs
+for throughput tests with 64B/78B frame size, with number of cores on
+X-axis and speedup ratio on Y-axis.
 
-For better comparision, there can be displayed more than one set of data in a
-plot. So, in general:
+For better comparison multiple test results' data sets are plotted per
+each graph:
 
     - graph type: grouped bars;
     - graph X-axis: (testcase index, number of cores);
     - graph Y-axis: speedup factor.
 
-The data displayed is a subset of existing performance tests with 1core, 2core,
-4core.
+Subset of existing performance tests is covered by TSA graphs.
+
+**Model for TSA:**
+
+::
 
-:TODO: Specify the data model for TSA.
+    -
+      type: "plot"
+      title: "TSA: 64B-*-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc"
+      algorithm: "plot_throughput_speedup_analysis"
+      output-file-type: ".html"
+      output-file: "{DIR[STATIC,VPP]}/10ge2p1x520-64B-l2-tsa-ndrdisc"
+      data:
+        "plot-throughput-speedup-analysis"
+      filter: "'NIC_Intel-X520-DA2' and '64B' and 'BASE' and 'NDRDISC' and ('L2BDMACSTAT' or 'L2BDMACLRN' or 'L2XCFWD') and not 'VHOST'"
+      parameters:
+      - "throughput"
+      - "parent"
+      - "tags"
+      layout:
+        title: "64B-*-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc"
+        layout:
+          "plot-throughput-speedup-analysis"
 
 
 Advanced data analytics
 ```````````````````````
 
-As the next steps, advanced data analytics (ADA) will be implemented using
-machine learning (ML) and artificial intelligence (AI).
+In the future advanced data analytics (ADA) will be added to analyze the
+telemetry data collected from SUT telemetry sources and correlate it to
+performance test results.
 
 :TODO:
 
diff --git a/resources/tools/presentation/generator_plots.py b/resources/tools/presentation/generator_plots.py
index 5580cbdf1c..66656679d0 100644
--- a/resources/tools/presentation/generator_plots.py
+++ b/resources/tools/presentation/generator_plots.py
@@ -261,13 +261,16 @@ def plot_throughput_speedup_analysis(plot, input_data):
                 except (KeyError, TypeError):
                     pass
 
+    if not throughput:
+        logging.warning("No data for the plot '{}'".
+                        format(plot.get("title", "")))
+        return
+
     for test_name, test_vals in throughput.items():
         for key, test_val in test_vals.items():
             if test_val:
                 throughput[test_name][key] = sum(test_val) / len(test_val)
 
-    print(throughput)
-
     names = ['1 core', '2 cores', '4 cores']
     x_vals = list()
     y_vals_1 = list()
@@ -276,7 +279,7 @@ def plot_throughput_speedup_analysis(plot, input_data):
 
     for test_name, test_vals in throughput.items():
         if test_vals["1"]:
-            x_vals.append("-".join(test_name.split('-')[:-1]))
+            x_vals.append("-".join(test_name.split('-')[1:-1]))
             y_vals_1.append(1)
             if test_vals["2"]:
                 y_vals_2.append(
diff --git a/resources/tools/presentation/input_data_parser.py b/resources/tools/presentation/input_data_parser.py
index 8c44dc45fa..2351942377 100644
--- a/resources/tools/presentation/input_data_parser.py
+++ b/resources/tools/presentation/input_data_parser.py
@@ -854,6 +854,8 @@ class InputData(object):
             logging.error("   Missing mandatory parameter in the element "
                           "specification.", err)
             return None
+        except AttributeError:
+            return None
         except SyntaxError:
             logging.error("   The filter '{0}' is not correct. Check if all "
                           "tags are enclosed by apostrophes.".format(cond))
diff --git a/resources/tools/presentation/specification.yaml b/resources/tools/presentation/specification.yaml
index 63f7605119..8a105fe974 100644
--- a/resources/tools/presentation/specification.yaml
+++ b/resources/tools/presentation/specification.yaml
@@ -88,6 +88,7 @@
   type: "configuration"
   data-sets:
     plot-throughput-speedup-analysis:
+# TODO: Add the data sources
 #      csit-vpp-perf-1801-all:
 #      - 1
 #      - 2
@@ -303,7 +304,7 @@
       autosize: False
       margin:
           't': 50
-          'b': 250
+          'b': 200
           'l': 50
           'r': 20
       showlegend: True
@@ -873,23 +874,174 @@
 
 # Plot Throughput Speedup Analysis
 
+# L2 - 10ge2p1x520 - NDR
 -
   type: "plot"
-  title: "TODO Throughput Speedup Analysis"
+  title: "TSA: 64B-*-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc"
   algorithm: "plot_throughput_speedup_analysis"
   output-file-type: ".html"
-  output-file: "{DIR[STATIC,VPP]}/tsa-64B-l2"
+  output-file: "{DIR[STATIC,VPP]}/10ge2p1x520-64B-l2-tsa-ndrdisc"
   data:
-    "plot-throughput-speedup-analysis" # CHANGE
-  # Keep this formatting, the filter is enclosed with " (quotation mark) and
-  # each tag is enclosed with ' (apostrophe).
-  filter: "('64B' or '78B') and 'BASE' and 'NDRDISC' and ('L2BDMACSTAT' or 'L2BDMACLRN' or 'L2XCFWD' or 'IP4FWD' or 'IP6FWD') and not 'VHOST'"
+    "plot-throughput-speedup-analysis"
+  filter: "'NIC_Intel-X520-DA2' and '64B' and 'BASE' and 'NDRDISC' and ('L2BDMACSTAT' or 'L2BDMACLRN' or 'L2XCFWD') and not 'VHOST'"
+  parameters:
+  - "throughput"
+  - "parent"
+  - "tags"
+  layout:
+    title: "64B-*-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc"
+    layout:
+      "plot-throughput-speedup-analysis"
+
+# L2 - 40ge2p1xl710 - NDR
+-
+  type: "plot"
+  title: "TSA: 64B-*-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc"
+  algorithm: "plot_throughput_speedup_analysis"
+  output-file-type: ".html"
+  output-file: "{DIR[STATIC,VPP]}/40ge2p1xl710-64B-l2-tsa-ndrdisc"
+  data:
+    "plot-throughput-speedup-analysis"
+  filter: "'NIC_Intel-XL710' and '64B' and 'BASE' and 'NDRDISC' and ('L2BDMACSTAT' or 'L2BDMACLRN' or 'L2XCFWD') and not 'VHOST'"
+  parameters:
+  - "throughput"
+  - "parent"
+  - "tags"
+  layout:
+    title: "64B-*-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-ndrdisc"
+    layout:
+      "plot-throughput-speedup-analysis"
+
+# L2 - 10ge2p1x520 - PDR
+-
+  type: "plot"
+  title: "TSA: 64B-*-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-pdrdisc"
+  algorithm: "plot_throughput_speedup_analysis"
+  output-file-type: ".html"
+  output-file: "{DIR[STATIC,VPP]}/10ge2p1x520-64B-l2-tsa-pdrdisc"
+  data:
+    "plot-throughput-speedup-analysis"
+  filter: "'NIC_Intel-X520-DA2' and '64B' and 'BASE' and 'PDRDISC' and not 'NDRDISC' and ('L2BDMACSTAT' or 'L2BDMACLRN' or 'L2XCFWD') and not 'VHOST'"
+  parameters:
+  - "throughput"
+  - "parent"
+  - "tags"
+  layout:
+    title: "64B-*-(eth|dot1q|dot1ad)-(l2xcbase|l2bdbasemaclrn)-pdrdisc"
+    layout:
+      "plot-throughput-speedup-analysis"
+
+# IPv4 - 10ge2p1x520 - NDR
+-
+  type: "plot"
+  title: "TSA: 64B-*-ethip4-ip4(base|scale)*ndrdisc"
+  algorithm: "plot_throughput_speedup_analysis"
+  output-file-type: ".html"
+  output-file: "{DIR[STATIC,VPP]}/10ge2p1x520-64B-ip4-tsa-ndrdisc"
+  data:
+    "plot-throughput-speedup-analysis"
+  filter: "'NIC_Intel-X520-DA2' and '64B' and ('BASE' or 'SCALE') and 'NDRDISC' and not 'VHOST'"
+  parameters:
+  - "throughput"
+  - "parent"
+  - "tags"
+  layout:
+    title: "64B-*-ethip4-ip4(base|scale)*ndrdisc"
+    layout:
+      "plot-throughput-speedup-analysis"
+
+# IPv4 - 40ge2p1xl710 - NDR
+-
+  type: "plot"
+  title: "TSA: 64B-*-ethip4-ip4(base|scale)*ndrdisc"
+  algorithm: "plot_throughput_speedup_analysis"
+  output-file-type: ".html"
+  output-file: "{DIR[STATIC,VPP]}/40ge2p1xl710-64B-ip4-tsa-ndrdisc"
+  data:
+    "plot-throughput-speedup-analysis"
+  filter: "'NIC_Intel-XL710' and '64B' and ('BASE' or 'SCALE') and 'NDRDISC' and not 'VHOST'"
+  parameters:
+  - "throughput"
+  - "parent"
+  - "tags"
+  layout:
+    title: "64B-*-ethip4-ip4(base|scale)*ndrdisc"
+    layout:
+      "plot-throughput-speedup-analysis"
+
+# IPv4 - 10ge2p1x520 - PDR
+-
+  type: "plot"
+  title: "TSA: 64B-*-ethip4-ip4(base|scale)*pdrdisc"
+  algorithm: "plot_throughput_speedup_analysis"
+  output-file-type: ".html"
+  output-file: "{DIR[STATIC,VPP]}/10ge2p1x520-64B-ip4-tsa-pdrdisc"
+  data:
+    "plot-throughput-speedup-analysis"
+  filter: "'NIC_Intel-X520-DA2' and '64B' and ('BASE' or 'SCALE') and 'PDRDISC' and not 'NDRDISC' and not 'VHOST'"
+  parameters:
+  - "throughput"
+  - "parent"
+  - "tags"
+  layout:
+    title: "64B-*-ethip4-ip4(base|scale)*pdrdisc"
+    layout:
+      "plot-throughput-speedup-analysis"
+
+# IPv6 - 10ge2p1x520 - NDR
+-
+  type: "plot"
+  title: "TSA: 78B-*-ethip6-ip6(base|scale)*ndrdisc"
+  algorithm: "plot_throughput_speedup_analysis"
+  output-file-type: ".html"
+  output-file: "{DIR[STATIC,VPP]}/10ge2p1x520-78B-ip6-tsa-ndrdisc"
+  data:
+    "plot-throughput-speedup-analysis"
+  filter: "'NIC_Intel-X520-DA2' and '78B' and ('BASE' or 'SCALE') and 'NDRDISC' and not 'VHOST'"
+  parameters:
+  - "throughput"
+  - "parent"
+  - "tags"
+  layout:
+    title: "78B-*-ethip6-ip6(base|scale)*ndrdisc"
+    layout:
+      "plot-throughput-speedup-analysis"
+
+# IPv6 - 40ge2p1xl710 - NDR
+-
+  type: "plot"
+  title: "TSA: 78B-*-ethip6-ip6(base|scale)*ndrdisc"
+  algorithm: "plot_throughput_speedup_analysis"
+  output-file-type: ".html"
+  output-file: "{DIR[STATIC,VPP]}/40ge2p1xl710-78B-ip6-tsa-ndrdisc"
+  data:
+    "plot-throughput-speedup-analysis"
+  filter: "'NIC_Intel-XL710' and '78B' and ('BASE' or 'SCALE') and 'NDRDISC' and not 'VHOST'"
+  parameters:
+  - "throughput"
+  - "parent"
+  - "tags"
+  layout:
+    title: "78B-*-ethip6-ip6(base|scale)*ndrdisc"
+    layout:
+      "plot-throughput-speedup-analysis"
+
+# IPv6 - 10ge2p1x520 - PDR
+-
+  type: "plot"
+  title: "TSA: 78B-*-ethip6-ip6(base|scale)*pdrdisc"
+  algorithm: "plot_throughput_speedup_analysis"
+  output-file-type: ".html"
+  output-file: "{DIR[STATIC,VPP]}/10ge2p1x520-78B-ip6-tsa-pdrdisc"
+  data:
+    "plot-throughput-speedup-analysis"
+  filter: "'NIC_Intel-X520-DA2' and '78B' and ('BASE' or 'SCALE') and 'PDRDISC' and not 'NDRDISC' and not 'VHOST'"
   parameters:
   - "throughput"
   - "parent"
   - "tags"
   layout:
-    title: "TODO Throughput Speedup Analysis ..."
+    title: "78B-*-ethip6-ip6(base|scale)*pdrdisc"
     layout:
       "plot-throughput-speedup-analysis"