Report: Add TRex tests

Change-Id: Ic221a1e5b6706ea6cca49cc0a3e7637169500d9c
Signed-off-by: Tibor Frank <tifrank@cisco.com>

3 files changed, 573 insertions, 0 deletions

diff --git a/docs/report/trex_performance_tests/packet_throughput_graphs/2n-skx-x710.rst b/docs/report/trex_performance_tests/packet_throughput_graphs/2n-skx-x710.rst
new file mode 100644
index 0000000000..6de5475d24
--- /dev/null
+++ b/docs/report/trex_performance_tests/packet_throughput_graphs/2n-skx-x710.rst
@@ -0,0 +1,258 @@
+
+.. raw:: latex
+
+    \clearpage
+
+.. raw:: html
+
+    <script type="text/javascript">
+
+        function getDocHeight(doc) {
+            doc = doc || document;
+            var body = doc.body, html = doc.documentElement;
+            var height = Math.max( body.scrollHeight, body.offsetHeight,
+                html.clientHeight, html.scrollHeight, html.offsetHeight );
+            return height;
+        }
+
+        function setIframeHeight(id) {
+            var ifrm = document.getElementById(id);
+            var doc = ifrm.contentDocument? ifrm.contentDocument:
+                ifrm.contentWindow.document;
+            ifrm.style.visibility = 'hidden';
+            ifrm.style.height = "10px"; // reset to minimal height ...
+            // IE opt. for bing/msn needs a bit added or scrollbar appears
+            ifrm.style.height = getDocHeight( doc ) + 4 + "px";
+            ifrm.style.visibility = 'visible';
+        }
+
+    </script>
+
+2n-skx-x710
+~~~~~~~~~~~
+
+Following sections include summary graphs of Phy-to-Phy performance with
+packet routed forwarding, including NDR throughput (zero packet loss)
+and PDR throughput (<0.5% packet loss).
+
+CSIT source code for the test cases used for plots can be found in
+`CSIT git repository <https://git.fd.io/csit/tree/tests/trex/perf?h=rls2110>`_.
+
+.. raw:: latex
+
+    \clearpage
+
+64b-ip4routing-base-scale
+-------------------------
+
+.. raw:: html
+
+    <center>
+    <iframe id="01" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/2n-skx-x710-64b--ip4-base-scale-ndr.html"></iframe>
+    <p><br></p>
+    </center>
+
+.. raw:: latex
+
+    \begin{figure}[H]
+        \centering
+            \graphicspath{{../_build/_static/trex/}}
+            \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{2n-skx-x710-64b--ip4-base-scale-ndr}
+            \label{fig:2n-skx-x710-64b--ip4-base-scale-ndr}
+    \end{figure}
+
+.. raw:: latex
+
+    \clearpage
+
+.. raw:: html
+
+    <center>
+    <iframe id="02" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/2n-skx-x710-64b--ip4-base-scale-pdr.html"></iframe>
+    <p><br></p>
+    </center>
+
+.. raw:: latex
+
+    \begin{figure}[H]
+        \centering
+            \graphicspath{{../_build/_static/trex/}}
+            \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{2n-skx-x710-64b--ip4-base-scale-pdr}
+            \label{fig:2n-skx-x710-64b--ip4-base-scale-pdr}
+    \end{figure}
+
+.. raw:: latex
+
+    \clearpage
+
+64b-ip4routing-[udp|tcp]-stf-cps
+--------------------------------
+
+.. raw:: html
+
+    <center>
+    <iframe id="03" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/2n-skx-x710-64b--ip4routing-stf-cps-ndr.html"></iframe>
+    <p><br></p>
+    </center>
+
+.. raw:: latex
+
+    \begin{figure}[H]
+        \centering
+            \graphicspath{{../_build/_static/trex/}}
+            \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{2n-skx-x710-64b--ip4routing-stf-cps-ndr}
+            \label{fig:2n-skx-x710-64b--ip4routing-stf-cps-ndr}
+    \end{figure}
+
+.. raw:: latex
+
+    \clearpage
+
+.. raw:: html
+
+    <center>
+    <iframe id="04" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/2n-skx-x710-64b--ip4routing-stf-cps-pdr.html"></iframe>
+    <p><br></p>
+    </center>
+
+.. raw:: latex
+
+    \begin{figure}[H]
+        \centering
+            \graphicspath{{../_build/_static/trex/}}
+            \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{2n-skx-x710-64b--ip4routing-stf-cps-pdr}
+            \label{fig:2n-skx-x710-64b--ip4routing-stf-cps-pdr}
+    \end{figure}
+
+.. raw:: latex
+
+    \clearpage
+
+64b-ip4routing-[udp|tcp]-stf-pps
+--------------------------------
+
+.. raw:: html
+
+    <center>
+    <iframe id="05" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/2n-skx-x710-64b--ip4routing-stf-pps-ndr.html"></iframe>
+    <p><br></p>
+    </center>
+
+.. raw:: latex
+
+    \begin{figure}[H]
+        \centering
+            \graphicspath{{../_build/_static/trex/}}
+            \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{2n-skx-x710-64b--ip4routing-stf-pps-ndr}
+            \label{fig:2n-skx-x710-64b--ip4routing-stf-pps-ndr}
+    \end{figure}
+
+.. raw:: latex
+
+    \clearpage
+
+.. raw:: html
+
+    <center>
+    <iframe id="06" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/2n-skx-x710-64b--ip4routing-stf-pps-pdr.html"></iframe>
+    <p><br></p>
+    </center>
+
+.. raw:: latex
+
+    \begin{figure}[H]
+        \centering
+            \graphicspath{{../_build/_static/trex/}}
+            \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{2n-skx-x710-64b--ip4routing-stf-pps-pdr}
+            \label{fig:2n-skx-x710-64b--ip4routing-stf-pps-pdr}
+    \end{figure}
+
+.. raw:: latex
+
+    \clearpage
+
+78b-ip6routing-base-scale
+-------------------------
+
+.. raw:: html
+
+    <center>
+    <iframe id="07" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/2n-skx-x710-78b--ip6-base-scale-ndr.html"></iframe>
+    <p><br></p>
+    </center>
+
+.. raw:: latex
+
+    \begin{figure}[H]
+        \centering
+            \graphicspath{{../_build/_static/trex/}}
+            \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{2n-skx-x710-78b--ip6-base-scale-ndr}
+            \label{fig:2n-skx-x710-78b--ip6-base-scale-ndr}
+    \end{figure}
+
+.. raw:: latex
+
+    \clearpage
+
+.. raw:: html
+
+    <center>
+    <iframe id="08" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/2n-skx-x710-78b--ip6-base-scale-pdr.html"></iframe>
+    <p><br></p>
+    </center>
+
+.. raw:: latex
+
+    \begin{figure}[H]
+        \centering
+            \graphicspath{{../_build/_static/trex/}}
+            \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{2n-skx-x710-78b--ip6-base-scale-pdr}
+            \label{fig:2n-skx-x710-78b--ip6-base-scale-pdr}
+    \end{figure}
+
+.. raw:: latex
+
+    \clearpage
+
+64b-l2switching-scale
+---------------------
+
+.. raw:: html
+
+    <center>
+    <iframe id="09" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/2n-skx-x710-64b--l2-scale-ndr.html"></iframe>
+    <p><br></p>
+    </center>
+
+.. raw:: latex
+
+    \begin{figure}[H]
+        \centering
+            \graphicspath{{../_build/_static/trex/}}
+            \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{2n-skx-x710-64b--l2-scale-ndr}
+            \label{fig:2n-skx-x710-64b--l2-scale-ndr}
+    \end{figure}
+
+.. raw:: latex
+
+    \clearpage
+
+.. raw:: html
+
+    <center>
+    <iframe id="10" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/2n-skx-x710-64b--l2-scale-pdr.html"></iframe>
+    <p><br></p>
+    </center>
+
+.. raw:: latex
+
+    \begin{figure}[H]
+        \centering
+            \graphicspath{{../_build/_static/trex/}}
+            \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{2n-skx-x710-64b--l2-scale-pdr}
+            \label{fig:2n-skx-x710-64b--l2-scale-pdr}
+    \end{figure}
+
+.. raw:: latex
+
+    \clearpage
diff --git a/docs/report/trex_performance_tests/packet_throughput_graphs/3n-skx-x710.rst b/docs/report/trex_performance_tests/packet_throughput_graphs/3n-skx-x710.rst
new file mode 100644
index 0000000000..9c4a1ec769
--- /dev/null
+++ b/docs/report/trex_performance_tests/packet_throughput_graphs/3n-skx-x710.rst
@@ -0,0 +1,258 @@
+
+.. raw:: latex
+
+    \clearpage
+
+.. raw:: html
+
+    <script type="text/javascript">
+
+        function getDocHeight(doc) {
+            doc = doc || document;
+            var body = doc.body, html = doc.documentElement;
+            var height = Math.max( body.scrollHeight, body.offsetHeight,
+                html.clientHeight, html.scrollHeight, html.offsetHeight );
+            return height;
+        }
+
+        function setIframeHeight(id) {
+            var ifrm = document.getElementById(id);
+            var doc = ifrm.contentDocument? ifrm.contentDocument:
+                ifrm.contentWindow.document;
+            ifrm.style.visibility = 'hidden';
+            ifrm.style.height = "10px"; // reset to minimal height ...
+            // IE opt. for bing/msn needs a bit added or scrollbar appears
+            ifrm.style.height = getDocHeight( doc ) + 4 + "px";
+            ifrm.style.visibility = 'visible';
+        }
+
+    </script>
+
+3n-skx-x710
+~~~~~~~~~~~
+
+Following sections include summary graphs of Phy-to-Phy performance with
+packet routed forwarding, including NDR throughput (zero packet loss)
+and PDR throughput (<0.5% packet loss).
+
+CSIT source code for the test cases used for plots can be found in
+`CSIT git repository <https://git.fd.io/csit/tree/tests/trex/perf?h=rls2110>`_.
+
+.. raw:: latex
+
+    \clearpage
+
+64b-ip4routing-base-scale
+-------------------------
+
+.. raw:: html
+
+    <center>
+    <iframe id="01" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/3n-skx-x710-64b--ip4-base-scale-ndr.html"></iframe>
+    <p><br></p>
+    </center>
+
+.. raw:: latex
+
+    \begin{figure}[H]
+        \centering
+            \graphicspath{{../_build/_static/trex/}}
+            \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{3n-skx-x710-64b--ip4-base-scale-ndr}
+            \label{fig:3n-skx-x710-64b--ip4-base-scale-ndr}
+    \end{figure}
+
+.. raw:: latex
+
+    \clearpage
+
+.. raw:: html
+
+    <center>
+    <iframe id="02" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/3n-skx-x710-64b--ip4-base-scale-pdr.html"></iframe>
+    <p><br></p>
+    </center>
+
+.. raw:: latex
+
+    \begin{figure}[H]
+        \centering
+            \graphicspath{{../_build/_static/trex/}}
+            \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{3n-skx-x710-64b--ip4-base-scale-pdr}
+            \label{fig:3n-skx-x710-64b--ip4-base-scale-pdr}
+    \end{figure}
+
+.. raw:: latex
+
+    \clearpage
+
+64b-ip4routing-[udp|tcp]-stf-cps
+--------------------------------
+
+.. raw:: html
+
+    <center>
+    <iframe id="03" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/3n-skx-x710-64b--ip4routing-stf-cps-ndr.html"></iframe>
+    <p><br></p>
+    </center>
+
+.. raw:: latex
+
+    \begin{figure}[H]
+        \centering
+            \graphicspath{{../_build/_static/trex/}}
+            \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{3n-skx-x710-64b--ip4routing-stf-cps-ndr}
+            \label{fig:3n-skx-x710-64b--ip4routing-stf-cps-ndr}
+    \end{figure}
+
+.. raw:: latex
+
+    \clearpage
+
+.. raw:: html
+
+    <center>
+    <iframe id="04" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/3n-skx-x710-64b--ip4routing-stf-cps-pdr.html"></iframe>
+    <p><br></p>
+    </center>
+
+.. raw:: latex
+
+    \begin{figure}[H]
+        \centering
+            \graphicspath{{../_build/_static/trex/}}
+            \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{3n-skx-x710-64b--ip4routing-stf-cps-pdr}
+            \label{fig:3n-skx-x710-64b--ip4routing-stf-cps-pdr}
+    \end{figure}
+
+.. raw:: latex
+
+    \clearpage
+
+64b-ip4routing-[udp|tcp]-stf-pps
+--------------------------------
+
+.. raw:: html
+
+    <center>
+    <iframe id="05" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/3n-skx-x710-64b--ip4routing-stf-pps-ndr.html"></iframe>
+    <p><br></p>
+    </center>
+
+.. raw:: latex
+
+    \begin{figure}[H]
+        \centering
+            \graphicspath{{../_build/_static/trex/}}
+            \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{3n-skx-x710-64b--ip4routing-stf-pps-ndr}
+            \label{fig:3n-skx-x710-64b--ip4routing-stf-pps-ndr}
+    \end{figure}
+
+.. raw:: latex
+
+    \clearpage
+
+.. raw:: html
+
+    <center>
+    <iframe id="06" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/3n-skx-x710-64b--ip4routing-stf-pps-pdr.html"></iframe>
+    <p><br></p>
+    </center>
+
+.. raw:: latex
+
+    \begin{figure}[H]
+        \centering
+            \graphicspath{{../_build/_static/trex/}}
+            \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{3n-skx-x710-64b--ip4routing-stf-pps-pdr}
+            \label{fig:3n-skx-x710-64b--ip4routing-stf-pps-pdr}
+    \end{figure}
+
+.. raw:: latex
+
+    \clearpage
+
+78b-ip6routing-base-scale
+-------------------------
+
+.. raw:: html
+
+    <center>
+    <iframe id="07" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/3n-skx-x710-78b--ip6-base-scale-ndr.html"></iframe>
+    <p><br></p>
+    </center>
+
+.. raw:: latex
+
+    \begin{figure}[H]
+        \centering
+            \graphicspath{{../_build/_static/trex/}}
+            \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{3n-skx-x710-78b--ip6-base-scale-ndr}
+            \label{fig:3n-skx-x710-78b--ip6-base-scale-ndr}
+    \end{figure}
+
+.. raw:: latex
+
+    \clearpage
+
+.. raw:: html
+
+    <center>
+    <iframe id="08" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/3n-skx-x710-78b--ip6-base-scale-pdr.html"></iframe>
+    <p><br></p>
+    </center>
+
+.. raw:: latex
+
+    \begin{figure}[H]
+        \centering
+            \graphicspath{{../_build/_static/trex/}}
+            \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{3n-skx-x710-78b--ip6-base-scale-pdr}
+            \label{fig:3n-skx-x710-78b--ip6-base-scale-pdr}
+    \end{figure}
+
+.. raw:: latex
+
+    \clearpage
+
+64b-l2switching-scale
+---------------------
+
+.. raw:: html
+
+    <center>
+    <iframe id="09" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/3n-skx-x710-64b--l2-scale-ndr.html"></iframe>
+    <p><br></p>
+    </center>
+
+.. raw:: latex
+
+    \begin{figure}[H]
+        \centering
+            \graphicspath{{../_build/_static/trex/}}
+            \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{3n-skx-x710-64b--l2-scale-ndr}
+            \label{fig:3n-skx-x710-64b--l2-scale-ndr}
+    \end{figure}
+
+.. raw:: latex
+
+    \clearpage
+
+.. raw:: html
+
+    <center>
+    <iframe id="10" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/3n-skx-x710-64b--l2-scale-pdr.html"></iframe>
+    <p><br></p>
+    </center>
+
+.. raw:: latex
+
+    \begin{figure}[H]
+        \centering
+            \graphicspath{{../_build/_static/trex/}}
+            \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{3n-skx-x710-64b--l2-scale-pdr}
+            \label{fig:3n-skx-x710-64b--l2-scale-pdr}
+    \end{figure}
+
+.. raw:: latex
+
+    \clearpage
diff --git a/docs/report/trex_performance_tests/packet_throughput_graphs/index.rst b/docs/report/trex_performance_tests/packet_throughput_graphs/index.rst
new file mode 100644
index 0000000000..eafc4868b3
--- /dev/null
+++ b/docs/report/trex_performance_tests/packet_throughput_graphs/index.rst
@@ -0,0 +1,57 @@
+
+.. raw:: latex
+
+    \clearpage
+
+Packet Throughput
+=================
+
+Throughput graphs are generated by multiple executions of the same
+performance tests across physical testbeds hosted LF FD.io labs:
+2n-skx, 3n-skx. Box-and-Whisker plots are used to display variations in
+measured throughput values, without making any assumptions of the
+underlying statistical distribution.
+
+For each test case, Box-and-Whisker plots show the quartiles (Min, 1st
+quartile / 25th percentile, 2nd quartile / 50th percentile / mean, 3rd
+quartile / 75th percentile, Max) across collected data set. Outliers are
+plotted as individual points.
+
+Additional information about graph data:
+
+#. **Graph Title**: describes tested packet path, testbed topology,
+   processor model, NIC model, packet size, number of cores and
+   threads used by data plane workers and indication of DPDK DUT
+   configuration.
+
+#. **X-axis Labels**: indices of individual test suites as listed in
+   Graph Legend.
+
+#. **Y-axis Labels**: measured Packets Per Second [pps] throughput
+   values.
+
+#. **Graph Legend**: lists X-axis indices with associated CSIT test
+   suites executed to generate graphed test results.
+
+#. **Hover Information**: lists minimum, first quartile, median,
+   third quartile, and maximum. If either type of outlier is present the
+   whisker on the appropriate side is taken to 1.5×IQR from the quartile
+   (the "inner fence") rather than the max or min, and individual outlying
+   data points are displayed as unfilled circles (for suspected outliers)
+   or filled circles (for outliers). (The "outer fence" is 3×IQR from the
+   quartile.)
+
+.. note::
+
+    Test results are stored in
+    `build logs from FD.io trex performance job 2n-skx`_ and
+    `build logs from FD.io trex performance job 3n-skx`_.
+    Required per test case data set size is **10** and for TRex tests
+    this is the actual size, as all scheduled test executions completed
+    successfully.
+
+.. toctree::
+    :maxdepth: 3
+
+    2n-skx-x710
+    3n-skx-x710