diff options
Diffstat (limited to 'docs/report/introduction/methodology_data_plane_throughput.rst')
| -rw-r--r-- | docs/report/introduction/methodology_data_plane_throughput.rst | 97 |
1 files changed, 75 insertions, 22 deletions
diff --git a/docs/report/introduction/methodology_data_plane_throughput.rst b/docs/report/introduction/methodology_data_plane_throughput.rst index 10d85e991c..2b5b085452 100644 --- a/docs/report/introduction/methodology_data_plane_throughput.rst +++ b/docs/report/introduction/methodology_data_plane_throughput.rst @@ -1,33 +1,86 @@ Data Plane Throughput --------------------- -Network data plane packet and bandwidth throughput are measured in -accordance with :rfc:`2544`, using FD.io CSIT Multiple Loss Ratio search -(MLRsearch), an optimized throughput search algorithm, that measures -SUT/DUT packet throughput rates at different Packet Loss Ratio (PLR) -values. +Network data plane packet and bandwidth throughput are measured using +multiple methods in order to obtain representative and repeatable +results across the large set of performance test cases implemented and +executed within CSIT. Following throughput test methods are used: -Following MLRsearch values are measured across a range of L2 frame sizes -and reported: +#. MLRsearch: Multiple Loss Ratio search -- NON DROP RATE (NDR): packet and bandwidth throughput at PLR=0%. + - **Description**: MLRsearch discovers multiple packet throughput + rates in a single search, reducing the overall test execution + time compared to a binary search. Each rate associated with a + distinct Packet Loss Ratio (PLR) criteria. In FD.io CSIT two + throughput rates are discovered: Non-Drop Rate (NDR, with zero + packet loss, PLR=0) and Partial Drop Rate (PDR, with PLR<0.5%). + MLRsearch is compliant with :rfc:`2544`. + - **Usage**: MLRsearch tests are run to discover NDR and PDR rates + for each VPP and DPDK release covered by CSIT report. Results for + small frame sizes (64b/78B, IMIX) are presented in packet + throughput graphs (Box-and-Whisker Plots) with NDR and PDR rates + plotted against the test cases covering popular VPP packet paths. + Each test is executed at least 10 times to verify measurements + repeatability and results are compared between releases and test + environments. NDR and PDR packet and bandwidth throughput results + for all frame sizes and for all tests are presented in detailed + results tables. + - **References**: See :ref:`mlrsearch_algorithm` for more detailed + description of MLRsearch tests. MLRsearch is being standardized + in IETF with `draft-vpolak-mkonstan-mlrsearch + <https://tools.ietf.org/html/draft-vpolak-mkonstan-bmwg-mlrsearch>`_. - - **Aggregate packet rate**: NDR_LOWER <bi-directional packet rate> - pps. - - **Aggregate bandwidth rate**: NDR_LOWER <bi-directional bandwidth - rate> Gbps. +#. MRR Measurements: Maximum Receive Rate -- PARTIAL DROP RATE (PDR): packet and bandwidth throughput at PLR=0.5%. + - **Description**: MRR tests are complementary to MLRsearch tests, + as they provide a maximum “raw” throughput benchmark for + development and testing community. MRR tests measure the packet + forwarding rate under the maximum load offered by traffic + generator (dependent on link type and NIC model) over a set trial + duration, regardless of packet loss. Maximum load for specified + Ethernet frame size is set to the bi-directional link rate. + - **Usage**: MRR tests are much faster than MLRsearch as they rely + on a single trial or a small set of trials with very short + duration. It is this property that makes them suitable for + continuous execution in daily performance trending jobs enabling + detection of performance anomalies (regressions, progressions) + resulting from data plane code changes. MRR tests are also used + for VPP per patch performance jobs verifying patch performance + vs. parent. CSIT reports include MRR throughput comparisons + between releases and test environments. Small frame sizes only + (64b/78B, IMIX). + - **References**: See :ref:`mrr_throughput` for more detailed + description of MRR tests configuration used for daily performance + trending jobs. VPP per patch test methodology is available on + `FD.io CSIT trending pages + <https://docs.fd.io/csit/master/trending/methodology/perpatch_performance_tests.html>`_. - - **Aggregate packet rate**: PDR_LOWER <bi-directional packet rate> - pps. - - **Aggregate bandwidth rate**: PDR_LOWER <bi-directional bandwidth - rate> Gbps. +#. PLRsearch: Probabilistic Loss Ratio search -NDR and PDR are measured for the following L2 frame sizes (untagged -Ethernet): + - **Description**: PLRsearch discovers a packet throughput rate + associated with configured Packet Loss Ratio (PLR) criteria for + tests run over an extended period of time a.k.a. soak testing. + PLRsearch assumes that system under test is probabilistic in + nature, and not deterministic. + - **Usage**: PLRsearch are run to discover a sustained throughput + for PLR=10^-7 (close to NDR) for VPP release covered by CSIT + report. Results for small frame sizes (64b/78B) are presented in + packet throughput graphs (Box Plots) for a small subset of + baseline tests. Each soak test lasts 2hrs and is executed at + least twice. Results are compared against NDR and PDR rates + discovered with MLRsearch. + - **References**: See :ref:`plrsearch_algorithm` for more detailed + description of PLRsearch tests. PLRsearch is being standardized + in IETF with `draft-vpolak-bmwg-plrsearch + <https://tools.ietf.org/html/draft-vpolak-bmwg-plrsearch>`_. -- IPv4 payload: 64B, IMIX (28x64B, 16x570B, 4x1518B), 1518B, 9000B. -- IPv6 payload: 78B, IMIX (28x78B, 16x570B, 4x1518B), 1518B, 9000B. +All of the listed data plane throughput test methodologies share +following properties: -All rates are reported from external Traffic Generator perspective. +- Tested L2 frame sizes (untagged Ethernet): + + - IPv4 payload: 64B, IMIX (28x64B, 16x570B, 4x1518B), 1518B, 9000B. + - IPv6 payload: 78B, IMIX (28x78B, 16x570B, 4x1518B), 1518B, 9000B. + +- All measured rates are aggregate bi-directional rates reported from + external Traffic Generator perspective.
\ No newline at end of file |