CSIT-902: Compare performance results before/after meltdown/spectre

 - CSIT-903: LLD
 - CSIT-904: Data model
 - CSIT-905: Algorithm
 - CSIT-906: Static content

Change-Id: Ia7b77fc35ab852110c2f50efb7756ac15576749a
Signed-off-by: Tibor Frank <tifrank@cisco.com>
Signed-off-by: Maciek Konstantynowicz <mkonstan@cisco.com>

3 files changed, 327 insertions, 1 deletions

diff --git a/docs/report/vpp_performance_tests/index.rst b/docs/report/vpp_performance_tests/index.rst
index 2d234ce921..0cdea485e9 100644
--- a/docs/report/vpp_performance_tests/index.rst
+++ b/docs/report/vpp_performance_tests/index.rst
@@ -7,6 +7,8 @@ VPP Performance Tests
     csit_release_notes
     packet_throughput_graphs/index
     packet_latency_graphs/index
-    throughput_speedup_multi_core
+    throughput_speedup_multi_core/index
+    performance_impact_meltdown/index
+    performance_impact_spectre/index
     test_environment
     documentation/index
diff --git a/docs/report/vpp_performance_tests/performance_impact_meltdown/index.rst b/docs/report/vpp_performance_tests/performance_impact_meltdown/index.rst
new file mode 100644
index 0000000000..bd3a377e80
--- /dev/null
+++ b/docs/report/vpp_performance_tests/performance_impact_meltdown/index.rst
@@ -0,0 +1,160 @@
+Performance Impact of Meltdown Patches
+======================================
+
+The following tables present performance impact on VPP after
+applying patches addressing Meltdown (Variant3: Rogue Data Cache Load) security
+ ulnerabilities. Incremental kernel patches are applied for Ubuntu 16.04LTS as
+documented on `Ubuntu SpectreAndMeltdown page <https://wiki.ubuntu.com/SecurityTeam/KnowledgeBase/SpectreAndMeltdown>`_.
+For detailed listing of used software versions and patches please refer
+to :ref:`test_environment`.
+
+NDR and PDR packet throughput results are compared for 1-core/1-thread,
+2-cores/2-threads and 4-cores/4-threads VPP configurations, with
+reference performance numbers coming from tests without the Meltdown
+patches.
+
+NDR throughput: Best 20 changes
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. only:: html
+
+   .. csv-table::
+      :align: center
+      :file: performance_impact_meltdown/meltdown-impact-ndr-1t1c-top.csv
+
+.. only:: latex
+
+   .. raw:: latex
+
+      \makeatletter
+      \csvset{
+        perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead},
+      }
+      \makeatother
+
+      {\tiny
+      \csvautobooklongtable[separator=comma,
+        respect all,
+        no check column count,
+        perfimprovements column width=1cm,
+        late after line={\\\hline},
+        late after last line={\end{longtable}}
+        ]{../_tmp/src/vpp_performance_tests/performance_improvements/meltdown-impact-ndr-1t1c-top.csv}
+      }
+
+NDR throughput: Worst 20 changes
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. only:: html
+
+   .. csv-table::
+      :align: center
+      :file: performance_impact_meltdown/meltdown-impact-ndr-1t1c-bottom.csv
+
+.. only:: latex
+
+   .. raw:: latex
+
+      \makeatletter
+      \csvset{
+        perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead},
+      }
+      \makeatother
+
+      {\tiny
+      \csvautobooklongtable[separator=comma,
+        respect all,
+        no check column count,
+        perfimprovements column width=1cm,
+        late after line={\\\hline},
+        late after last line={\end{longtable}}
+        ]{../_tmp/src/vpp_performance_tests/performance_improvements/meltdown-impact-ndr-1t1c-bottom.csv}
+      }
+
+.. only:: html
+
+NDR throughput: All changes
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Complete results for all NDR tests are available in a CSV and pretty
+ASCII formats:
+
+  - `csv format for 1t1c <meltdown-impact-ndr-1t1c-full.csv>`_,
+  - `csv format for 2t2c <meltdown-impact-ndr-2t2c-full.csv>`_,
+  - `csv format for 4t4c <meltdown-impact-ndr-4t4c-full.csv>`_,
+  - `pretty ASCII format for 1t1c <meltdown-impact-ndr-1t1c-full.txt>`_,
+  - `pretty ASCII format for 2t2c <meltdown-impact-ndr-2t2c-full.txt>`_,
+  - `pretty ASCII format for 4t4c <meltdown-impact-ndr-4t4c-full.txt>`_.
+
+PDR throughput: Best 20 changes
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. only:: html
+
+   .. csv-table::
+      :align: center
+      :file: performance_impact_meltdown/meltdown-impact-pdr-1t1c-top.csv
+
+.. only:: latex
+
+   .. raw:: latex
+
+      \makeatletter
+      \csvset{
+        perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead},
+      }
+      \makeatother
+
+      {\tiny
+      \csvautobooklongtable[separator=comma,
+        respect all,
+        no check column count,
+        perfimprovements column width=1cm,
+        late after line={\\\hline},
+        late after last line={\end{longtable}}
+        ]{../_tmp/src/vpp_performance_tests/performance_improvements/meltdown-impact-pdr-1t1c-top.csv}
+      }
+
+PDR throughput: Worst 20 changes
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. only:: html
+
+   .. csv-table::
+      :align: center
+      :file: performance_impact_meltdown/meltdown-impact-pdr-1t1c-bottom.csv
+
+.. only:: latex
+
+   .. raw:: latex
+
+      \makeatletter
+      \csvset{
+        perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead},
+      }
+      \makeatother
+
+      {\tiny
+      \csvautobooklongtable[separator=comma,
+        respect all,
+        no check column count,
+        perfimprovements column width=1cm,
+        late after line={\\\hline},
+        late after last line={\end{longtable}}
+        ]{../_tmp/src/vpp_performance_tests/performance_improvements/meltdown-impact-pdr-1t1c-bottom.csv}
+      }
+
+.. only:: html
+
+PDR throughput: All changes
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Complete results for all PDR tests are available in a CSV and pretty
+ASCII formats:
+
+  - `csv format for 1t1c <meltdown-impact-pdr-1t1c-full.csv>`_,
+  - `csv format for 2t2c <meltdown-impact-pdr-2t2c-full.csv>`_,
+  - `csv format for 4t4c <meltdown-impact-pdr-4t4c-full.csv>`_,
+  - `pretty ASCII format for 1t1c <meltdown-impact-pdr-1t1c-full.txt>`_,
+  - `pretty ASCII format for 2t2c <meltdown-impact-pdr-2t2c-full.txt>`_,
+  - `pretty ASCII format for 4t4c <meltdown-impact-pdr-4t4c-full.txt>`_.
diff --git a/docs/report/vpp_performance_tests/performance_impact_spectre/index.rst b/docs/report/vpp_performance_tests/performance_impact_spectre/index.rst
new file mode 100644
index 0000000000..cb3b03043e
--- /dev/null
+++ b/docs/report/vpp_performance_tests/performance_impact_spectre/index.rst
@@ -0,0 +1,164 @@
+Performance Impact of Meltdown and Spectre Patches
+==================================================
+
+The following tables present performance impact on VPP after applying
+patches addressing Meltdown (Variant3: Rogue Data Cache Load) and
+Spectre (Variant1: Bounds Check Bypass; Variant2: Branch Target
+Injection) security vulnerabilities. Incremental kernel patches for
+Ubuntu 16.04 LTS as documented on
+`Ubuntu SpectreAndMeltdown page <https://wiki.ubuntu.com/SecurityTeam/KnowledgeBase/SpectreAndMeltdown>`_.
+For Spectre additional Processor microcode and BIOS firmware changes are
+applied. For detailed listing of used software versions and patches
+please refer to :ref:`test_environment`.
+
+NDR and PDR packet throughput results are compared for 1-core/1-thread,
+2-cores/2-threads and 4-cores/4-threads VPP configurations, with
+reference performance numbers coming from tests without the Meltdown and
+Spectre patches.
+
+NDR throughput: Best 20 changes
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. only:: html
+
+   .. csv-table::
+      :align: center
+      :file: performance_impact_meltdown/spectre-impact-ndr-1t1c-top.csv
+
+.. only:: latex
+
+   .. raw:: latex
+
+      \makeatletter
+      \csvset{
+        perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead},
+      }
+      \makeatother
+
+      {\tiny
+      \csvautobooklongtable[separator=comma,
+        respect all,
+        no check column count,
+        perfimprovements column width=1cm,
+        late after line={\\\hline},
+        late after last line={\end{longtable}}
+        ]{../_tmp/src/vpp_performance_tests/performance_improvements/spectre-impact-ndr-1t1c-top.csv}
+      }
+
+NDR throughput: Worst 20 changes
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. only:: html
+
+   .. csv-table::
+      :align: center
+      :file: performance_impact_meltdown/spectre-impact-ndr-1t1c-bottom.csv
+
+.. only:: latex
+
+   .. raw:: latex
+
+      \makeatletter
+      \csvset{
+        perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead},
+      }
+      \makeatother
+
+      {\tiny
+      \csvautobooklongtable[separator=comma,
+        respect all,
+        no check column count,
+        perfimprovements column width=1cm,
+        late after line={\\\hline},
+        late after last line={\end{longtable}}
+        ]{../_tmp/src/vpp_performance_tests/performance_improvements/spectre-impact-ndr-1t1c-bottom.csv}
+      }
+
+.. only:: html
+
+
+NDR throughput: All changes
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Complete results for all NDR tests are available in a CSV and pretty
+ASCII formats:
+
+  - `csv format for 1t1c <meltdown-spectre-impact-ndr-1t1c-full.csv>`_,
+  - `csv format for 2t2c <meltdown-spectre-impact-ndr-2t2c-full.csv>`_,
+  - `csv format for 4t4c <meltdown-spectre-impact-ndr-4t4c-full.csv>`_,
+  - `pretty ASCII format for 1t1c <meltdown-spectre-impact-ndr-1t1c-full.txt>`_,
+  - `pretty ASCII format for 2t2c <meltdown-spectre-impact-ndr-2t2c-full.txt>`_,
+  - `pretty ASCII format for 4t4c <meltdown-spectre-impact-ndr-4t4c-full.txt>`_.
+
+PDR throughput: Best 20 changes
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. only:: html
+
+   .. csv-table::
+      :align: center
+      :file: performance_impact_meltdown/spectre-impact-pdr-1t1c-top.csv
+
+.. only:: latex
+
+   .. raw:: latex
+
+      \makeatletter
+      \csvset{
+        perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead},
+      }
+      \makeatother
+
+      {\tiny
+      \csvautobooklongtable[separator=comma,
+        respect all,
+        no check column count,
+        perfimprovements column width=1cm,
+        late after line={\\\hline},
+        late after last line={\end{longtable}}
+        ]{../_tmp/src/vpp_performance_tests/performance_improvements/spectre-impact-pdr-1t1c-top.csv}
+      }
+
+PDR throughput: Worst 20 changes
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. only:: html
+
+   .. csv-table::
+      :align: center
+      :file: performance_impact_meltdown/spectre-impact-pdr-1t1c-bottom.csv
+
+.. only:: latex
+
+   .. raw:: latex
+
+      \makeatletter
+      \csvset{
+        perfimprovements column width/.style={after head=\csv@pretable\begin{longtable}{m{4cm} m{#1} m{#1} m{#1} m{#1} m{#1}}\csv@tablehead},
+      }
+      \makeatother
+
+      {\tiny
+      \csvautobooklongtable[separator=comma,
+        respect all,
+        no check column count,
+        perfimprovements column width=1cm,
+        late after line={\\\hline},
+        late after last line={\end{longtable}}
+        ]{../_tmp/src/vpp_performance_tests/performance_improvements/spectre-impact-pdr-1t1c-bottom.csv}
+      }
+
+.. only:: html
+
+PDR throughput: All changes
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Complete results for all PDR tests are available in a CSV and pretty
+ASCII formats:
+
+  - `csv format for 1t1c <meltdown-spectre-impact-pdr-1t1c-full.csv>`_,
+  - `csv format for 2t2c <meltdown-spectre-impact-pdr-2t2c-full.csv>`_,
+  - `csv format for 4t4c <meltdown-spectre-impact-pdr-4t4c-full.csv>`_,
+  - `pretty ASCII format for 1t1c <meltdown-spectre-impact-pdr-1t1c-full.txt>`_,
+  - `pretty ASCII format for 2t2c <meltdown-spectre-impact-pdr-2t2c-full.txt>`_,
+  - `pretty ASCII format for 4t4c <meltdown-spectre-impact-pdr-4t4c-full.txt>`_.