diff options
author | Tibor Frank <tifrank@cisco.com> | 2021-10-08 10:35:56 +0200 |
---|---|---|
committer | Tibor Frank <tifrank@cisco.com> | 2021-10-08 11:46:56 +0000 |
commit | 75119f759eb2020bb6fb6543e11877f9c256dc5f (patch) | |
tree | b8c90741c99604f7e43ba2ae2144197d59cc7f70 /docs | |
parent | bd15bb418bf618ff3055fb25c339778d12c1eaef (diff) |
Report: Add TRex tests
Change-Id: Ic221a1e5b6706ea6cca49cc0a3e7637169500d9c
Signed-off-by: Tibor Frank <tifrank@cisco.com>
Diffstat (limited to 'docs')
17 files changed, 986 insertions, 0 deletions
diff --git a/docs/report/detailed_test_results/trex_performance_results_2n_skx/index.rst b/docs/report/detailed_test_results/trex_performance_results_2n_skx/index.rst new file mode 100644 index 0000000000..8607595606 --- /dev/null +++ b/docs/report/detailed_test_results/trex_performance_results_2n_skx/index.rst @@ -0,0 +1,12 @@ +2n-skx +------ + +.. note:: + + Data sources for reported test results: + i) `build logs from FD.io trex coverage job 2n-skx`_, + ii) archived FD.io jobs test result `output files + <../../_static/archive/>`_. + +.. toctree:: + diff --git a/docs/report/detailed_test_results/trex_performance_results_3n_skx/index.rst b/docs/report/detailed_test_results/trex_performance_results_3n_skx/index.rst new file mode 100644 index 0000000000..d67848a42d --- /dev/null +++ b/docs/report/detailed_test_results/trex_performance_results_3n_skx/index.rst @@ -0,0 +1,12 @@ +3n-skx +------ + +.. note:: + + Data sources for reported test results: + i) `build logs from FD.io trex coverage job 3n-skx`_, + ii) archived FD.io jobs test result `output files + <../../_static/archive/>`_. + +.. toctree:: + diff --git a/docs/report/index.html.template b/docs/report/index.html.template index 1eb188893d..1502e7514a 100644 --- a/docs/report/index.html.template +++ b/docs/report/index.html.template @@ -45,6 +45,17 @@ CSIT-2110 .. toctree:: :maxdepth: 2 + :caption: TRex Performance + + trex_performance_tests/overview + trex_performance_tests/csit_release_notes + trex_performance_tests/packet_throughput_graphs/index + trex_performance_tests/packet_latency/index + trex_performance_tests/throughput_trending + trex_performance_tests/test_environment + +.. toctree:: + :maxdepth: 2 :caption: VPP Device vpp_device_tests/overview diff --git a/docs/report/index.pdf.template b/docs/report/index.pdf.template index 0099942f69..2c93a1f075 100644 --- a/docs/report/index.pdf.template +++ b/docs/report/index.pdf.template @@ -7,5 +7,6 @@ CSIT-2110 introduction/index vpp_performance_tests/index dpdk_performance_tests/index + trex_performance_tests/index vpp_device_tests/index csit_framework_documentation/index diff --git a/docs/report/trex_performance_tests/csit_release_notes.rst b/docs/report/trex_performance_tests/csit_release_notes.rst new file mode 100644 index 0000000000..ec0175a81b --- /dev/null +++ b/docs/report/trex_performance_tests/csit_release_notes.rst @@ -0,0 +1,22 @@ +Release Notes +============= + +Changes in |csit-release| +------------------------- + +#. TEST FRAMEWORK + +#. TRex RELEASE VERSION CHANGE + +.. _trex_known_issues: + +Known Issues +------------ + +List of known issues in |csit-release| for TRex performance tests: + ++----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+ +| # | JiraID | Issue Description | ++====+=========================================+===========================================================================================================+ +| 1 | | | ++----+-----------------------------------------+-----------------------------------------------------------------------------------------------------------+ diff --git a/docs/report/trex_performance_tests/hdrh_packet_latency_by_percentile_graphs/2n_skx.rst b/docs/report/trex_performance_tests/hdrh_packet_latency_by_percentile_graphs/2n_skx.rst new file mode 100644 index 0000000000..61baf8fcbf --- /dev/null +++ b/docs/report/trex_performance_tests/hdrh_packet_latency_by_percentile_graphs/2n_skx.rst @@ -0,0 +1,3 @@ +2n-skx +====== + diff --git a/docs/report/trex_performance_tests/hdrh_packet_latency_by_percentile_graphs/3n_skx.rst b/docs/report/trex_performance_tests/hdrh_packet_latency_by_percentile_graphs/3n_skx.rst new file mode 100644 index 0000000000..60840bbb1e --- /dev/null +++ b/docs/report/trex_performance_tests/hdrh_packet_latency_by_percentile_graphs/3n_skx.rst @@ -0,0 +1,3 @@ +3n-skx +====== + diff --git a/docs/report/trex_performance_tests/index.rst b/docs/report/trex_performance_tests/index.rst new file mode 100644 index 0000000000..8596f71479 --- /dev/null +++ b/docs/report/trex_performance_tests/index.rst @@ -0,0 +1,11 @@ +TRex Performance +================ + +.. toctree:: + + overview + csit_release_notes + packet_throughput_graphs/index + packet_latency/index + throughput_trending + test_environment diff --git a/docs/report/trex_performance_tests/overview.rst b/docs/report/trex_performance_tests/overview.rst new file mode 100644 index 0000000000..802d341477 --- /dev/null +++ b/docs/report/trex_performance_tests/overview.rst @@ -0,0 +1,2 @@ +Overview +======== diff --git a/docs/report/trex_performance_tests/packet_latency/2n-skx-x710.rst b/docs/report/trex_performance_tests/packet_latency/2n-skx-x710.rst new file mode 100644 index 0000000000..7100d1cafe --- /dev/null +++ b/docs/report/trex_performance_tests/packet_latency/2n-skx-x710.rst @@ -0,0 +1,144 @@ + +.. raw:: latex + + \clearpage + +.. raw:: html + + <script type="text/javascript"> + + function getDocHeight(doc) { + doc = doc || document; + var body = doc.body, html = doc.documentElement; + var height = Math.max( body.scrollHeight, body.offsetHeight, + html.clientHeight, html.scrollHeight, html.offsetHeight ); + return height; + } + + function setIframeHeight(id) { + var ifrm = document.getElementById(id); + var doc = ifrm.contentDocument? ifrm.contentDocument: + ifrm.contentWindow.document; + ifrm.style.visibility = 'hidden'; + ifrm.style.height = "10px"; // reset to minimal height ... + // IE opt. for bing/msn needs a bit added or scrollbar appears + ifrm.style.height = getDocHeight( doc ) + 4 + "px"; + ifrm.style.visibility = 'visible'; + } + + </script> + +2n-skx-x710 +~~~~~~~~~~~ + +CSIT source code for the test cases used for plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/trex/perf?h=rls2110>`_. + +.. raw:: latex + + \clearpage + +64b-ip4routing-base-scale +------------------------- + +.. raw:: html + + <center> + <iframe id="hdrh-lat-percentile-2n-skx-10ge2p1x710-64b--ethip4-ip4base-tg" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/hdrh-lat-percentile-2n-skx-10ge2p1xxv710-64b--ethip4-ip4base-tg.html"></iframe> + <p><br></p> + </center> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/trex/}} + \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{hdrh-lat-percentile-2n-skx-10ge2p1x710-64b--ethip4-ip4base-tg} + \label{fig:hdrh-lat-percentile-2n-skx-10ge2p1x710-64b--ethip4-ip4base-tg} + \end{figure} + +.. raw:: latex + + \clearpage + +.. raw:: html + + <center> + <iframe id="hdrh-lat-percentile-2n-skx-10ge2p1x710-64b--ethip4-ip4scale2m-tg" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/hdrh-lat-percentile-2n-skx-10ge2p1xxv710-64b--ethip4-ip4scale2m-tg.html"></iframe> + <p><br></p> + </center> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/trex/}} + \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{hdrh-lat-percentile-2n-skx-10ge2p1x710-64b--ethip4-ip4scale2m-tg} + \label{fig:hdrh-lat-percentile-2n-skx-10ge2p1x710-64b--ethip4-ip4scale2m-tg} + \end{figure} + +.. raw:: latex + + \clearpage + +78b-ip6routing-base-scale +------------------------- + +.. raw:: html + + <center> + <iframe id="hdrh-lat-percentile-2n-skx-10ge2p1x710-78b--ethip6-ip6base-tg" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/hdrh-lat-percentile-2n-skx-10ge2p1xxv710-78b--ethip6-ip6base-tg.html"></iframe> + <p><br></p> + </center> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/trex/}} + \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{hdrh-lat-percentile-2n-skx-10ge2p1x710-78b--ethip6-ip6base-tg} + \label{fig:hdrh-lat-percentile-2n-skx-10ge2p1x710-78b--ethip6-ip6base-tg} + \end{figure} + +.. raw:: latex + + \clearpage + +.. raw:: html + + <center> + <iframe id="hdrh-lat-percentile-2n-skx-10ge2p1x710-78b--ethip6-ip6scale2m-tg" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/hdrh-lat-percentile-2n-skx-10ge2p1xxv710-78b--ethip6-ip6scale2m-tg.html"></iframe> + <p><br></p> + </center> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/trex/}} + \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{hdrh-lat-percentile-2n-skx-10ge2p1x710-78b--ethip6-ip6scale2m-tg} + \label{fig:hdrh-lat-percentile-2n-skx-10ge2p1x710-78b--ethip6-ip6scale2m-tg} + \end{figure} + +.. raw:: latex + + \clearpage + +64b-l2switching-scale +--------------------- + +.. raw:: html + + <center> + <iframe id="hdrh-lat-percentile-2n-skx-10ge2p1x710-64b--eth-l2bdscale1mmaclrn-tg" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/hdrh-lat-percentile-2n-skx-10ge2p1xxv710-64b--eth-l2bdscale1mmaclrn-tg.html"></iframe> + <p><br></p> + </center> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/trex/}} + \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{hdrh-lat-percentile-2n-skx-10ge2p1x710-64b--eth-l2bdscale1mmaclrn-tg} + \label{fig:hdrh-lat-percentile-2n-skx-10ge2p1x710-64b--eth-l2bdscale1mmaclrn-tg} + \end{figure} diff --git a/docs/report/trex_performance_tests/packet_latency/3n-skx-x710.rst b/docs/report/trex_performance_tests/packet_latency/3n-skx-x710.rst new file mode 100644 index 0000000000..325000ffb2 --- /dev/null +++ b/docs/report/trex_performance_tests/packet_latency/3n-skx-x710.rst @@ -0,0 +1,144 @@ + +.. raw:: latex + + \clearpage + +.. raw:: html + + <script type="text/javascript"> + + function getDocHeight(doc) { + doc = doc || document; + var body = doc.body, html = doc.documentElement; + var height = Math.max( body.scrollHeight, body.offsetHeight, + html.clientHeight, html.scrollHeight, html.offsetHeight ); + return height; + } + + function setIframeHeight(id) { + var ifrm = document.getElementById(id); + var doc = ifrm.contentDocument? ifrm.contentDocument: + ifrm.contentWindow.document; + ifrm.style.visibility = 'hidden'; + ifrm.style.height = "10px"; // reset to minimal height ... + // IE opt. for bing/msn needs a bit added or scrollbar appears + ifrm.style.height = getDocHeight( doc ) + 4 + "px"; + ifrm.style.visibility = 'visible'; + } + + </script> + +3n-skx-x710 +~~~~~~~~~~~ + +CSIT source code for the test cases used for plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/trex/perf?h=rls2110>`_. + +.. raw:: latex + + \clearpage + +64b-ip4routing-base-scale +------------------------- + +.. raw:: html + + <center> + <iframe id="hdrh-lat-percentile-3n-skx-10ge2p1x710-64b--ethip4-ip4base-tg" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/hdrh-lat-percentile-3n-skx-10ge2p1xxv710-64b--ethip4-ip4base-tg.html"></iframe> + <p><br></p> + </center> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/trex/}} + \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{hdrh-lat-percentile-3n-skx-10ge2p1x710-64b--ethip4-ip4base-tg} + \label{fig:hdrh-lat-percentile-3n-skx-10ge2p1x710-64b--ethip4-ip4base-tg} + \end{figure} + +.. raw:: latex + + \clearpage + +.. raw:: html + + <center> + <iframe id="hdrh-lat-percentile-3n-skx-10ge2p1x710-64b--ethip4-ip4scale2m-tg" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/hdrh-lat-percentile-3n-skx-10ge2p1xxv710-64b--ethip4-ip4scale2m-tg.html"></iframe> + <p><br></p> + </center> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/trex/}} + \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{hdrh-lat-percentile-3n-skx-10ge2p1x710-64b--ethip4-ip4scale2m-tg} + \label{fig:hdrh-lat-percentile-3n-skx-10ge2p1x710-64b--ethip4-ip4scale2m-tg} + \end{figure} + +.. raw:: latex + + \clearpage + +78b-ip6routing-base-scale +------------------------- + +.. raw:: html + + <center> + <iframe id="hdrh-lat-percentile-3n-skx-10ge2p1x710-78b--ethip6-ip6base-tg" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/hdrh-lat-percentile-3n-skx-10ge2p1xxv710-78b--ethip6-ip6base-tg.html"></iframe> + <p><br></p> + </center> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/trex/}} + \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{hdrh-lat-percentile-3n-skx-10ge2p1x710-78b--ethip6-ip6base-tg} + \label{fig:hdrh-lat-percentile-3n-skx-10ge2p1x710-78b--ethip6-ip6base-tg} + \end{figure} + +.. raw:: latex + + \clearpage + +.. raw:: html + + <center> + <iframe id="hdrh-lat-percentile-3n-skx-10ge2p1x710-78b--ethip6-ip6scale2m-tg" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/hdrh-lat-percentile-3n-skx-10ge2p1xxv710-78b--ethip6-ip6scale2m-tg.html"></iframe> + <p><br></p> + </center> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/trex/}} + \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{hdrh-lat-percentile-3n-skx-10ge2p1x710-78b--ethip6-ip6scale2m-tg} + \label{fig:hdrh-lat-percentile-3n-skx-10ge2p1x710-78b--ethip6-ip6scale2m-tg} + \end{figure} + +.. raw:: latex + + \clearpage + +64b-l2switching-scale +--------------------- + +.. raw:: html + + <center> + <iframe id="hdrh-lat-percentile-3n-skx-10ge2p1x710-64b--eth-l2bdscale1mmaclrn-tg" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/hdrh-lat-percentile-3n-skx-10ge2p1xxv710-64b--eth-l2bdscale1mmaclrn-tg.html"></iframe> + <p><br></p> + </center> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/trex/}} + \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{hdrh-lat-percentile-3n-skx-10ge2p1x710-64b--eth-l2bdscale1mmaclrn-tg} + \label{fig:hdrh-lat-percentile-3n-skx-10ge2p1x710-64b--eth-l2bdscale1mmaclrn-tg} + \end{figure} diff --git a/docs/report/trex_performance_tests/packet_latency/index.rst b/docs/report/trex_performance_tests/packet_latency/index.rst new file mode 100644 index 0000000000..1ebf39400d --- /dev/null +++ b/docs/report/trex_performance_tests/packet_latency/index.rst @@ -0,0 +1,44 @@ + +.. raw:: latex + + \clearpage + +Packet Latency +============== + +TRex latency results are generated based on the test +data obtained from |csit-release| NDR-PDR throughput tests executed +across physical testbeds hosted in LF FD.io labs: 3n-skx, 2n-skx. + +Latency by percentile distribution plots are used to show packet latency +percentiles at different packet rate load levels: i) No-Load latency +streams only, ii) Low-Load at 10% PDR, iii) Mid-Load at 50% PDR and iv) +High-Load at 90% PDR. + +For more details, see :ref:`latency_methodology`. + +Additional information about graph data: + +#. **Graph Title**: describes tested DUT packet path. + +#. **X-axis Labels**: percentile of packets. + +#. **Y-axis Labels**: measured one-way packet latency values in [uSec]. + +#. **Graph Legend**: list of latency tests at different packet rate load + level. + +#. **Hover Information**: packet rate load level, stream direction + (East-West, West-East), percentile, one-way latency. + +.. note:: + + Test results are stored in + `build logs from FD.io trex performance job 2n-skx`_ and + `build logs from FD.io trex performance job 3n-skx`_. + +.. toctree:: + :maxdepth: 3 + + 2n-skx-x710 + 3n-skx-x710 diff --git a/docs/report/trex_performance_tests/packet_throughput_graphs/2n-skx-x710.rst b/docs/report/trex_performance_tests/packet_throughput_graphs/2n-skx-x710.rst new file mode 100644 index 0000000000..6de5475d24 --- /dev/null +++ b/docs/report/trex_performance_tests/packet_throughput_graphs/2n-skx-x710.rst @@ -0,0 +1,258 @@ + +.. raw:: latex + + \clearpage + +.. raw:: html + + <script type="text/javascript"> + + function getDocHeight(doc) { + doc = doc || document; + var body = doc.body, html = doc.documentElement; + var height = Math.max( body.scrollHeight, body.offsetHeight, + html.clientHeight, html.scrollHeight, html.offsetHeight ); + return height; + } + + function setIframeHeight(id) { + var ifrm = document.getElementById(id); + var doc = ifrm.contentDocument? ifrm.contentDocument: + ifrm.contentWindow.document; + ifrm.style.visibility = 'hidden'; + ifrm.style.height = "10px"; // reset to minimal height ... + // IE opt. for bing/msn needs a bit added or scrollbar appears + ifrm.style.height = getDocHeight( doc ) + 4 + "px"; + ifrm.style.visibility = 'visible'; + } + + </script> + +2n-skx-x710 +~~~~~~~~~~~ + +Following sections include summary graphs of Phy-to-Phy performance with +packet routed forwarding, including NDR throughput (zero packet loss) +and PDR throughput (<0.5% packet loss). + +CSIT source code for the test cases used for plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/trex/perf?h=rls2110>`_. + +.. raw:: latex + + \clearpage + +64b-ip4routing-base-scale +------------------------- + +.. raw:: html + + <center> + <iframe id="01" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/2n-skx-x710-64b--ip4-base-scale-ndr.html"></iframe> + <p><br></p> + </center> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/trex/}} + \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{2n-skx-x710-64b--ip4-base-scale-ndr} + \label{fig:2n-skx-x710-64b--ip4-base-scale-ndr} + \end{figure} + +.. raw:: latex + + \clearpage + +.. raw:: html + + <center> + <iframe id="02" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/2n-skx-x710-64b--ip4-base-scale-pdr.html"></iframe> + <p><br></p> + </center> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/trex/}} + \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{2n-skx-x710-64b--ip4-base-scale-pdr} + \label{fig:2n-skx-x710-64b--ip4-base-scale-pdr} + \end{figure} + +.. raw:: latex + + \clearpage + +64b-ip4routing-[udp|tcp]-stf-cps +-------------------------------- + +.. raw:: html + + <center> + <iframe id="03" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/2n-skx-x710-64b--ip4routing-stf-cps-ndr.html"></iframe> + <p><br></p> + </center> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/trex/}} + \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{2n-skx-x710-64b--ip4routing-stf-cps-ndr} + \label{fig:2n-skx-x710-64b--ip4routing-stf-cps-ndr} + \end{figure} + +.. raw:: latex + + \clearpage + +.. raw:: html + + <center> + <iframe id="04" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/2n-skx-x710-64b--ip4routing-stf-cps-pdr.html"></iframe> + <p><br></p> + </center> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/trex/}} + \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{2n-skx-x710-64b--ip4routing-stf-cps-pdr} + \label{fig:2n-skx-x710-64b--ip4routing-stf-cps-pdr} + \end{figure} + +.. raw:: latex + + \clearpage + +64b-ip4routing-[udp|tcp]-stf-pps +-------------------------------- + +.. raw:: html + + <center> + <iframe id="05" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/2n-skx-x710-64b--ip4routing-stf-pps-ndr.html"></iframe> + <p><br></p> + </center> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/trex/}} + \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{2n-skx-x710-64b--ip4routing-stf-pps-ndr} + \label{fig:2n-skx-x710-64b--ip4routing-stf-pps-ndr} + \end{figure} + +.. raw:: latex + + \clearpage + +.. raw:: html + + <center> + <iframe id="06" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/2n-skx-x710-64b--ip4routing-stf-pps-pdr.html"></iframe> + <p><br></p> + </center> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/trex/}} + \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{2n-skx-x710-64b--ip4routing-stf-pps-pdr} + \label{fig:2n-skx-x710-64b--ip4routing-stf-pps-pdr} + \end{figure} + +.. raw:: latex + + \clearpage + +78b-ip6routing-base-scale +------------------------- + +.. raw:: html + + <center> + <iframe id="07" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/2n-skx-x710-78b--ip6-base-scale-ndr.html"></iframe> + <p><br></p> + </center> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/trex/}} + \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{2n-skx-x710-78b--ip6-base-scale-ndr} + \label{fig:2n-skx-x710-78b--ip6-base-scale-ndr} + \end{figure} + +.. raw:: latex + + \clearpage + +.. raw:: html + + <center> + <iframe id="08" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/2n-skx-x710-78b--ip6-base-scale-pdr.html"></iframe> + <p><br></p> + </center> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/trex/}} + \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{2n-skx-x710-78b--ip6-base-scale-pdr} + \label{fig:2n-skx-x710-78b--ip6-base-scale-pdr} + \end{figure} + +.. raw:: latex + + \clearpage + +64b-l2switching-scale +--------------------- + +.. raw:: html + + <center> + <iframe id="09" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/2n-skx-x710-64b--l2-scale-ndr.html"></iframe> + <p><br></p> + </center> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/trex/}} + \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{2n-skx-x710-64b--l2-scale-ndr} + \label{fig:2n-skx-x710-64b--l2-scale-ndr} + \end{figure} + +.. raw:: latex + + \clearpage + +.. raw:: html + + <center> + <iframe id="10" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/2n-skx-x710-64b--l2-scale-pdr.html"></iframe> + <p><br></p> + </center> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/trex/}} + \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{2n-skx-x710-64b--l2-scale-pdr} + \label{fig:2n-skx-x710-64b--l2-scale-pdr} + \end{figure} + +.. raw:: latex + + \clearpage diff --git a/docs/report/trex_performance_tests/packet_throughput_graphs/3n-skx-x710.rst b/docs/report/trex_performance_tests/packet_throughput_graphs/3n-skx-x710.rst new file mode 100644 index 0000000000..9c4a1ec769 --- /dev/null +++ b/docs/report/trex_performance_tests/packet_throughput_graphs/3n-skx-x710.rst @@ -0,0 +1,258 @@ + +.. raw:: latex + + \clearpage + +.. raw:: html + + <script type="text/javascript"> + + function getDocHeight(doc) { + doc = doc || document; + var body = doc.body, html = doc.documentElement; + var height = Math.max( body.scrollHeight, body.offsetHeight, + html.clientHeight, html.scrollHeight, html.offsetHeight ); + return height; + } + + function setIframeHeight(id) { + var ifrm = document.getElementById(id); + var doc = ifrm.contentDocument? ifrm.contentDocument: + ifrm.contentWindow.document; + ifrm.style.visibility = 'hidden'; + ifrm.style.height = "10px"; // reset to minimal height ... + // IE opt. for bing/msn needs a bit added or scrollbar appears + ifrm.style.height = getDocHeight( doc ) + 4 + "px"; + ifrm.style.visibility = 'visible'; + } + + </script> + +3n-skx-x710 +~~~~~~~~~~~ + +Following sections include summary graphs of Phy-to-Phy performance with +packet routed forwarding, including NDR throughput (zero packet loss) +and PDR throughput (<0.5% packet loss). + +CSIT source code for the test cases used for plots can be found in +`CSIT git repository <https://git.fd.io/csit/tree/tests/trex/perf?h=rls2110>`_. + +.. raw:: latex + + \clearpage + +64b-ip4routing-base-scale +------------------------- + +.. raw:: html + + <center> + <iframe id="01" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/3n-skx-x710-64b--ip4-base-scale-ndr.html"></iframe> + <p><br></p> + </center> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/trex/}} + \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{3n-skx-x710-64b--ip4-base-scale-ndr} + \label{fig:3n-skx-x710-64b--ip4-base-scale-ndr} + \end{figure} + +.. raw:: latex + + \clearpage + +.. raw:: html + + <center> + <iframe id="02" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/3n-skx-x710-64b--ip4-base-scale-pdr.html"></iframe> + <p><br></p> + </center> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/trex/}} + \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{3n-skx-x710-64b--ip4-base-scale-pdr} + \label{fig:3n-skx-x710-64b--ip4-base-scale-pdr} + \end{figure} + +.. raw:: latex + + \clearpage + +64b-ip4routing-[udp|tcp]-stf-cps +-------------------------------- + +.. raw:: html + + <center> + <iframe id="03" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/3n-skx-x710-64b--ip4routing-stf-cps-ndr.html"></iframe> + <p><br></p> + </center> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/trex/}} + \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{3n-skx-x710-64b--ip4routing-stf-cps-ndr} + \label{fig:3n-skx-x710-64b--ip4routing-stf-cps-ndr} + \end{figure} + +.. raw:: latex + + \clearpage + +.. raw:: html + + <center> + <iframe id="04" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/3n-skx-x710-64b--ip4routing-stf-cps-pdr.html"></iframe> + <p><br></p> + </center> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/trex/}} + \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{3n-skx-x710-64b--ip4routing-stf-cps-pdr} + \label{fig:3n-skx-x710-64b--ip4routing-stf-cps-pdr} + \end{figure} + +.. raw:: latex + + \clearpage + +64b-ip4routing-[udp|tcp]-stf-pps +-------------------------------- + +.. raw:: html + + <center> + <iframe id="05" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/3n-skx-x710-64b--ip4routing-stf-pps-ndr.html"></iframe> + <p><br></p> + </center> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/trex/}} + \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{3n-skx-x710-64b--ip4routing-stf-pps-ndr} + \label{fig:3n-skx-x710-64b--ip4routing-stf-pps-ndr} + \end{figure} + +.. raw:: latex + + \clearpage + +.. raw:: html + + <center> + <iframe id="06" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/3n-skx-x710-64b--ip4routing-stf-pps-pdr.html"></iframe> + <p><br></p> + </center> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/trex/}} + \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{3n-skx-x710-64b--ip4routing-stf-pps-pdr} + \label{fig:3n-skx-x710-64b--ip4routing-stf-pps-pdr} + \end{figure} + +.. raw:: latex + + \clearpage + +78b-ip6routing-base-scale +------------------------- + +.. raw:: html + + <center> + <iframe id="07" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/3n-skx-x710-78b--ip6-base-scale-ndr.html"></iframe> + <p><br></p> + </center> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/trex/}} + \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{3n-skx-x710-78b--ip6-base-scale-ndr} + \label{fig:3n-skx-x710-78b--ip6-base-scale-ndr} + \end{figure} + +.. raw:: latex + + \clearpage + +.. raw:: html + + <center> + <iframe id="08" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/3n-skx-x710-78b--ip6-base-scale-pdr.html"></iframe> + <p><br></p> + </center> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/trex/}} + \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{3n-skx-x710-78b--ip6-base-scale-pdr} + \label{fig:3n-skx-x710-78b--ip6-base-scale-pdr} + \end{figure} + +.. raw:: latex + + \clearpage + +64b-l2switching-scale +--------------------- + +.. raw:: html + + <center> + <iframe id="09" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/3n-skx-x710-64b--l2-scale-ndr.html"></iframe> + <p><br></p> + </center> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/trex/}} + \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{3n-skx-x710-64b--l2-scale-ndr} + \label{fig:3n-skx-x710-64b--l2-scale-ndr} + \end{figure} + +.. raw:: latex + + \clearpage + +.. raw:: html + + <center> + <iframe id="10" onload="setIframeHeight(this.id)" width="700" frameborder="0" scrolling="no" src="../../_static/trex/3n-skx-x710-64b--l2-scale-pdr.html"></iframe> + <p><br></p> + </center> + +.. raw:: latex + + \begin{figure}[H] + \centering + \graphicspath{{../_build/_static/trex/}} + \includegraphics[clip, trim=0cm 0cm 5cm 0cm, width=0.70\textwidth]{3n-skx-x710-64b--l2-scale-pdr} + \label{fig:3n-skx-x710-64b--l2-scale-pdr} + \end{figure} + +.. raw:: latex + + \clearpage diff --git a/docs/report/trex_performance_tests/packet_throughput_graphs/index.rst b/docs/report/trex_performance_tests/packet_throughput_graphs/index.rst new file mode 100644 index 0000000000..eafc4868b3 --- /dev/null +++ b/docs/report/trex_performance_tests/packet_throughput_graphs/index.rst @@ -0,0 +1,57 @@ + +.. raw:: latex + + \clearpage + +Packet Throughput +================= + +Throughput graphs are generated by multiple executions of the same +performance tests across physical testbeds hosted LF FD.io labs: +2n-skx, 3n-skx. Box-and-Whisker plots are used to display variations in +measured throughput values, without making any assumptions of the +underlying statistical distribution. + +For each test case, Box-and-Whisker plots show the quartiles (Min, 1st +quartile / 25th percentile, 2nd quartile / 50th percentile / mean, 3rd +quartile / 75th percentile, Max) across collected data set. Outliers are +plotted as individual points. + +Additional information about graph data: + +#. **Graph Title**: describes tested packet path, testbed topology, + processor model, NIC model, packet size, number of cores and + threads used by data plane workers and indication of DPDK DUT + configuration. + +#. **X-axis Labels**: indices of individual test suites as listed in + Graph Legend. + +#. **Y-axis Labels**: measured Packets Per Second [pps] throughput + values. + +#. **Graph Legend**: lists X-axis indices with associated CSIT test + suites executed to generate graphed test results. + +#. **Hover Information**: lists minimum, first quartile, median, + third quartile, and maximum. If either type of outlier is present the + whisker on the appropriate side is taken to 1.5×IQR from the quartile + (the "inner fence") rather than the max or min, and individual outlying + data points are displayed as unfilled circles (for suspected outliers) + or filled circles (for outliers). (The "outer fence" is 3×IQR from the + quartile.) + +.. note:: + + Test results are stored in + `build logs from FD.io trex performance job 2n-skx`_ and + `build logs from FD.io trex performance job 3n-skx`_. + Required per test case data set size is **10** and for TRex tests + this is the actual size, as all scheduled test executions completed + successfully. + +.. toctree:: + :maxdepth: 3 + + 2n-skx-x710 + 3n-skx-x710 diff --git a/docs/report/trex_performance_tests/test_environment.rst b/docs/report/trex_performance_tests/test_environment.rst new file mode 100644 index 0000000000..cc422ff713 --- /dev/null +++ b/docs/report/trex_performance_tests/test_environment.rst @@ -0,0 +1,2 @@ +DUT Settings - TRex +------------------- diff --git a/docs/report/trex_performance_tests/throughput_trending.rst b/docs/report/trex_performance_tests/throughput_trending.rst new file mode 100644 index 0000000000..3d56443f92 --- /dev/null +++ b/docs/report/trex_performance_tests/throughput_trending.rst @@ -0,0 +1,2 @@ +Throughput Trending +------------------- |