diff options
Diffstat (limited to 'docs/report')
18 files changed, 1242 insertions, 916 deletions
diff --git a/docs/report/introduction/methodology.rst b/docs/report/introduction/methodology.rst index 16d3edacdb..1f9fcfe7fb 100644 --- a/docs/report/introduction/methodology.rst +++ b/docs/report/introduction/methodology.rst @@ -4,919 +4,21 @@ Test Methodology ================ -VPP Forwarding Modes --------------------- - -VPP is tested in a number of L2 and IP packet lookup and forwarding -modes. Within each mode baseline and scale tests are executed, the -latter with varying number of lookup entries. - -L2 Ethernet Switching -~~~~~~~~~~~~~~~~~~~~~ - -VPP is tested in three L2 forwarding modes: - -- *l2patch*: L2 patch, the fastest point-to-point L2 path that loops - packets between two interfaces without any Ethernet frame checks or - lookups. -- *l2xc*: L2 cross-connect, point-to-point L2 path with all Ethernet - frame checks, but no MAC learning and no MAC lookup. -- *l2bd*: L2 bridge-domain, multipoint-to-multipoint L2 path with all - Ethernet frame checks, with MAC learning (unless static MACs are used) - and MAC lookup. - -l2bd tests are executed in baseline and scale configurations: - -- *l2bdbase*: low number of L2 flows (253 per direction) is switched by - VPP. They drive the content of MAC FIB size (506 total MAC entries). - Both source and destination MAC addresses are incremented on a packet - by packet basis. - -- *l2bdscale*: high number of L2 flows is switched by VPP. Tested MAC - FIB sizes include: i) 10k (5k unique flows per direction), ii) 100k - (2x 50k flows) and iii) 1M (2x 500k). Both source and destination MAC - addresses are incremented on a packet by packet basis, ensuring new - entries are learn refreshed and looked up at every packet, making it - the worst case scenario. - -Ethernet wire encapsulations tested include: untagged, dot1q, dot1ad. - -IPv4 Routing -~~~~~~~~~~~~ - -IPv4 routing tests are executed in baseline and scale configurations: - -- *ip4base*: low number of IPv4 flows (253 per direction) is routed by - VPP. They drive the content of IPv4 FIB size (506 total /32 prefixes). - Destination IPv4 addresses are incremented on a packet by packet - basis. - -- *ip4scale*: high number of IPv4 flows is routed by VPP. Tested IPv4 - FIB sizes of /32 prefixes include: i) 20k (10k unique flows per - direction), ii) 200k (2x 100k flows) and iii) 2M (2x 1M). Destination - IPv4 addresses are incremented on a packet by packet basis, ensuring - new FIB entries are looked up at every packet, making it the worst - case scenario. - -IPv6 Routing -~~~~~~~~~~~~ - -IPv6 routing tests are executed in baseline and scale configurations: - -- *ip6base*: low number of IPv6 flows (253 per direction) is routed by - VPP. They drive the content of IPv6 FIB size (506 total /128 prefixes). - Destination IPv6 addresses are incremented on a packet by packet - basis. - -- *ip6scale*: high number of IPv6 flows is routed by VPP. Tested IPv6 - FIB sizes of /128 prefixes include: i) 20k (10k unique flows per - direction), ii) 200k (2x 100k flows) and iii) 2M (2x 1M). Destination - IPv6 addresses are incremented on a packet by packet basis, ensuring - new FIB entries are looked up at every packet, making it the worst - case scenario. - -SRv6 Routing -~~~~~~~~~~~~ - -SRv6 routing tests are executed in a number of baseline configurations, -in each case SR policy and steering policy are configured for one -direction and one (or two) SR behaviours (functions) in the other -directions: - -- *srv6enc1sid*: One SID (no SRH present), one SR function - End. -- *srv6enc2sids*: Two SIDs (SRH present), two SR functions - End and - End.DX6. -- *srv6enc2sids-nodecaps*: Two SIDs (SRH present) without decapsulation, - one SR function - End. -- *srv6proxy-dyn*: Dynamic SRv6 proxy, one SR function - End.AD. -- *srv6proxy-masq*: Masquerading SRv6 proxy, one SR function - End.AM. -- *srv6proxy-stat*: Static SRv6 proxy, one SR function - End.AS. - -In all listed cases low number of IPv6 flows (253 per direction) is -routed by VPP. - -Tunnel Encapsulations ---------------------- - -Tunnel encapsulations testing is grouped based on the type of outer -header: IPv4 or IPv6. - -IPv4 Tunnels -~~~~~~~~~~~~ - -VPP is tested in the following IPv4 tunnel baseline configurations: - -- *ip4vxlan-l2bdbase*: VXLAN over IPv4 tunnels with L2 bridge-domain MAC - switching. -- *ip4vxlan-l2xcbase*: VXLAN over IPv4 tunnels with L2 cross-connect. -- *ip4lispip4-ip4base*: LISP over IPv4 tunnels with IPv4 routing. -- *ip4lispip6-ip6base*: LISP over IPv4 tunnels with IPv6 routing. - -In all cases listed above low number of MAC, IPv4, IPv6 flows (253 per -direction) is switched or routed by VPP. - -In addition selected IPv4 tunnels are tested at scale: - -- *dot1q--ip4vxlanscale-l2bd*: VXLAN over IPv4 tunnels with L2 bridge- - domain MAC switching, with scaled up dot1q VLANs (10, 100, 1k), - mapped to scaled up L2 bridge-domains (10, 100, 1k), that are in turn - mapped to (10, 100, 1k) VXLAN tunnels. 64.5k flows are transmitted per - direction. - -IPv6 Tunnels -~~~~~~~~~~~~ - -VPP is tested in the following IPv6 tunnel baseline configurations: - -- *ip6lispip4-ip4base*: LISP over IPv4 tunnels with IPv4 routing. -- *ip6lispip6-ip6base*: LISP over IPv4 tunnels with IPv6 routing. - -In all cases listed above low number of IPv4, IPv6 flows (253 per -direction) is routed by VPP. - -VPP Features ------------- - -VPP is tested in a number of data plane feature configurations across -different forwarding modes. Following sections list features tested. - -ACL Security-Groups -~~~~~~~~~~~~~~~~~~~ - -Both stateless and stateful access control lists (ACL), also known as -security-groups, are supported by VPP. - -Following ACL configurations are tested for MAC switching with L2 -bridge-domains: - -- *l2bdbasemaclrn-iacl{E}sl-{F}flows*: Input stateless ACL, with {E} - entries and {F} flows. -- *l2bdbasemaclrn-oacl{E}sl-{F}flows*: Output stateless ACL, with {E} - entries and {F} flows. -- *l2bdbasemaclrn-iacl{E}sf-{F}flows*: Input stateful ACL, with {E} - entries and {F} flows. -- *l2bdbasemaclrn-oacl{E}sf-{F}flows*: Output stateful ACL, with {E} - entries and {F} flows. - -Following ACL configurations are tested with IPv4 routing: - -- *ip4base-iacl{E}sl-{F}flows*: Input stateless ACL, with {E} entries - and {F} flows. -- *ip4base-oacl{E}sl-{F}flows*: Output stateless ACL, with {E} entries - and {F} flows. -- *ip4base-iacl{E}sf-{F}flows*: Input stateful ACL, with {E} entries and - {F} flows. -- *ip4base-oacl{E}sf-{F}flows*: Output stateful ACL, with {E} entries - and {F} flows. - -ACL tests are executed with the following combinations of ACL entries -and number of flows: - -- ACL entry definitions - - - flow non-matching deny entry: (src-ip4, dst-ip4, src-port, dst-port). - - flow matching permit ACL entry: (src-ip4, dst-ip4). - -- {E} - number of non-matching deny ACL entries, {E} = [1, 10, 50]. -- {F} - number of UDP flows with different tuple (src-ip4, dst-ip4, - src-port, dst-port), {F} = [100, 10k, 100k]. -- All {E}x{F} combinations are tested per ACL type, total of 9. - -ACL MAC-IP -~~~~~~~~~~ - -MAC-IP binding ACLs are tested for MAC switching with L2 bridge-domains: - -- *l2bdbasemaclrn-macip-iacl{E}sl-{F}flows*: Input stateless ACL, with - {E} entries and {F} flows. - -MAC-IP ACL tests are executed with the following combinations of ACL -entries and number of flows: - -- ACL entry definitions - - - flow non-matching deny entry: (dst-ip4, dst-mac, bit-mask) - - flow matching permit ACL entry: (dst-ip4, dst-mac, bit-mask) - -- {E} - number of non-matching deny ACL entries, {E} = [1, 10, 50] -- {F} - number of UDP flows with different tuple (dst-ip4, dst-mac), - {F} = [100, 10k, 100k] -- All {E}x{F} combinations are tested per ACL type, total of 9. - -NAT44 -~~~~~ - -NAT44 is tested in baseline and scale configurations with IPv4 routing: - -- *ip4base-nat44*: baseline test with single NAT entry (addr, port), - single UDP flow. -- *ip4base-udpsrcscale{U}-nat44*: baseline test with {U} NAT entries - (addr, {U}ports), {U}=15. -- *ip4scale{R}-udpsrcscale{U}-nat44*: scale tests with {R}*{U} NAT - entries ({R}addr, {U}ports), {R}=[100, 1k, 2k, 4k], {U}=15. - -Data Plane Throughput ---------------------- - -Network data plane packet and bandwidth throughput are measured in -accordance with :rfc:`2544`, using FD.io CSIT Multiple Loss Ratio search -(MLRsearch), an optimized throughput search algorithm, that measures -SUT/DUT packet throughput rates at different Packet Loss Ratio (PLR) -values. - -Following MLRsearch values are measured across a range of L2 frame sizes -and reported: - -- NON DROP RATE (NDR): packet and bandwidth throughput at PLR=0%. - - - **Aggregate packet rate**: NDR_LOWER <bi-directional packet rate> - pps. - - **Aggregate bandwidth rate**: NDR_LOWER <bi-directional bandwidth - rate> Gbps. - -- PARTIAL DROP RATE (PDR): packet and bandwidth throughput at PLR=0.5%. - - - **Aggregate packet rate**: PDR_LOWER <bi-directional packet rate> - pps. - - **Aggregate bandwidth rate**: PDR_LOWER <bi-directional bandwidth - rate> Gbps. - -NDR and PDR are measured for the following L2 frame sizes (untagged -Ethernet): - -- IPv4 payload: 64B, IMIX (28x64B, 16x570B, 4x1518B), 1518B, 9000B. -- IPv6 payload: 78B, IMIX (28x78B, 16x570B, 4x1518B), 1518B, 9000B. - -All rates are reported from external Traffic Generator perspective. - -.. _mlrsearch_algorithm: - -MLRsearch Tests ---------------- - -Multiple Loss Rate search (MLRsearch) tests use new search algorithm -implemented in FD.io CSIT project. MLRsearch discovers multiple packet -throughput rates in a single search, with each rate associated with a -distinct Packet Loss Ratio (PLR) criteria. MLRsearch is being -standardized in IETF with `draft-vpolak-mkonstan-mlrsearch-XX -<https://tools.ietf.org/html/draft-vpolak-mkonstan-mlrsearch-00>`_. - -Two throughput measurements used in FD.io CSIT are Non-Drop Rate (NDR, -with zero packet loss, PLR=0) and Partial Drop Rate (PDR, with packet -loss rate not greater than the configured non-zero PLR). MLRsearch -discovers NDR and PDR in a single pass reducing required execution time -compared to separate binary searches for NDR and PDR. MLRsearch reduces -execution time even further by relying on shorter trial durations -of intermediate steps, with only the final measurements -conducted at the specified final trial duration. -This results in the shorter overall search -execution time when compared to a standard NDR/PDR binary search, -while guaranteeing the same or similar results. - -If needed, MLRsearch can be easily adopted to discover more throughput rates -with different pre-defined PLRs. - -.. Note:: All throughput rates are *always* bi-directional - aggregates of two equal (symmetric) uni-directional packet rates - received and reported by an external traffic generator. - -Overview -~~~~~~~~ - -The main properties of MLRsearch: - -- MLRsearch is a duration aware multi-phase multi-rate search algorithm. - - - Initial phase determines promising starting interval for the search. - - Intermediate phases progress towards defined final search criteria. - - Final phase executes measurements according to the final search - criteria. - -- *Initial phase*: - - - Uses link rate as a starting transmit rate and discovers the Maximum - Receive Rate (MRR) used as an input to the first intermediate phase. - -- *Intermediate phases*: - - - Start with initial trial duration (in the first phase) and converge - geometrically towards the final trial duration (in the final phase). - - Track two values for NDR and two for PDR. - - - The values are called (NDR or PDR) lower_bound and upper_bound. - - Each value comes from a specific trial measurement - (most recent for that transmit rate), - and as such the value is associated with that measurement's duration and loss. - - A bound can be invalid, for example if NDR lower_bound - has been measured with nonzero loss. - - Invalid bounds are not real boundaries for the searched value, - but are needed to track interval widths. - - Valid bounds are real boundaries for the searched value. - - Each non-initial phase ends with all bounds valid. - - - Start with a large (lower_bound, upper_bound) interval width and - geometrically converge towards the width goal (measurement resolution) - of the phase. Each phase halves the previous width goal. - - Use internal and external searches: - - - External search - measures at transmit rates outside the (lower_bound, - upper_bound) interval. Activated when a bound is invalid, - to search for a new valid bound by doubling the interval width. - It is a variant of `exponential search`_. - - Internal search - `binary search`_, measures at transmit rates within the - (lower_bound, upper_bound) valid interval, halving the interval width. - -- *Final phase* is executed with the final test trial duration, and the final - width goal that determines resolution of the overall search. - Intermediate phases together with the final phase are called non-initial phases. - -The main benefits of MLRsearch vs. binary search include: - -- In general MLRsearch is likely to execute more search trials overall, but - less trials at a set final duration. -- In well behaving cases it greatly reduces (>50%) the overall duration - compared to a single PDR (or NDR) binary search duration, - while finding multiple drop rates. -- In all cases MLRsearch yields the same or similar results to binary search. -- Note: both binary search and MLRsearch are susceptible to reporting - non-repeatable results across multiple runs for very bad behaving - cases. - -Caveats: - -- Worst case MLRsearch can take longer than a binary search e.g. in case of - drastic changes in behaviour for trials at varying durations. - -Search Implementation -~~~~~~~~~~~~~~~~~~~~~ - -Following is a brief description of the current MLRsearch -implementation in FD.io CSIT. - -Input Parameters -```````````````` - -#. *maximum_transmit_rate* - maximum packet transmit rate to be used by - external traffic generator, limited by either the actual Ethernet - link rate or traffic generator NIC model capabilities. Sample - defaults: 2 * 14.88 Mpps for 64B 10GE link rate, - 2 * 18.75 Mpps for 64B 40GE NIC maximum rate. -#. *minimum_transmit_rate* - minimum packet transmit rate to be used for - measurements. MLRsearch fails if lower transmit rate needs to be - used to meet search criteria. Default: 2 * 10 kpps (could be higher). -#. *final_trial_duration* - required trial duration for final rate - measurements. Default: 30 sec. -#. *initial_trial_duration* - trial duration for initial MLRsearch phase. - Default: 1 sec. -#. *final_relative_width* - required measurement resolution expressed as - (lower_bound, upper_bound) interval width relative to upper_bound. - Default: 0.5%. -#. *packet_loss_ratio* - maximum acceptable PLR search criteria for - PDR measurements. Default: 0.5%. -#. *number_of_intermediate_phases* - number of phases between the initial - phase and the final phase. Impacts the overall MLRsearch duration. - Less phases are required for well behaving cases, more phases - may be needed to reduce the overall search duration for worse behaving cases. - Default (2). (Value chosen based on limited experimentation to date. - More experimentation needed to arrive to clearer guidelines.) - -Initial Phase -````````````` - -1. First trial measures at maximum rate and discovers MRR. - - a. *in*: trial_duration = initial_trial_duration. - b. *in*: offered_transmit_rate = maximum_transmit_rate. - c. *do*: single trial. - d. *out*: measured loss ratio. - e. *out*: mrr = measured receive rate. - -2. Second trial measures at MRR and discovers MRR2. - - a. *in*: trial_duration = initial_trial_duration. - b. *in*: offered_transmit_rate = MRR. - c. *do*: single trial. - d. *out*: measured loss ratio. - e. *out*: mrr2 = measured receive rate. - -3. Third trial measures at MRR2. - - a. *in*: trial_duration = initial_trial_duration. - b. *in*: offered_transmit_rate = MRR2. - c. *do*: single trial. - d. *out*: measured loss ratio. - -Non-initial Phases -`````````````````` - -1. Main loop: - - a. *in*: trial_duration for the current phase. - Set to initial_trial_duration for the first intermediate phase; - to final_trial_duration for the final phase; - or to the element of interpolating geometric sequence - for other intermediate phases. - For example with two intermediate phases, trial_duration - of the second intermediate phase is the geometric average - of initial_strial_duration and final_trial_duration. - b. *in*: relative_width_goal for the current phase. - Set to final_relative_width for the final phase; - doubled for each preceding phase. - For example with two intermediate phases, - the first intermediate phase uses quadruple of final_relative_width - and the second intermediate phase uses double of final_relative_width. - c. *in*: ndr_interval, pdr_interval from the previous main loop iteration - or the previous phase. - If the previous phase is the initial phase, both intervals have - lower_bound = MRR2, uper_bound = MRR. - Note that the initial phase is likely to create intervals with invalid bounds. - d. *do*: According to the procedure described in point 2, - either exit the phase (by jumping to 1.g.), - or prepare new transmit rate to measure with. - e. *do*: Perform the trial measurement at the new transmit rate - and trial_duration, compute its loss ratio. - f. *do*: Update the bounds of both intervals, based on the new measurement. - The actual update rules are numerous, as NDR external search - can affect PDR interval and vice versa, but the result - agrees with rules of both internal and external search. - For example, any new measurement below an invalid lower_bound - becomes the new lower_bound, while the old measurement - (previously acting as the invalid lower_bound) - becomes a new and valid upper_bound. - Go to next iteration (1.c.), taking the updated intervals as new input. - g. *out*: current ndr_interval and pdr_interval. - In the final phase this is also considered - to be the result of the whole search. - For other phases, the next phase loop is started - with the current results as an input. - -2. New transmit rate (or exit) calculation (for 1.d.): - - - If there is an invalid bound then prepare for external search: - - - *If* the most recent measurement at NDR lower_bound transmit rate - had the loss higher than zero, then - the new transmit rate is NDR lower_bound - decreased by two NDR interval widths. - - Else, *if* the most recent measurement at PDR lower_bound - transmit rate had the loss higher than PLR, then - the new transmit rate is PDR lower_bound - decreased by two PDR interval widths. - - Else, *if* the most recent measurement at NDR upper_bound - transmit rate had no loss, then - the new transmit rate is NDR upper_bound - increased by two NDR interval widths. - - Else, *if* the most recent measurement at PDR upper_bound - transmit rate had the loss lower or equal to PLR, then - the new transmit rate is PDR upper_bound - increased by two PDR interval widths. - - If interval width is higher than the current phase goal: - - - Else, *if* NDR interval does not meet the current phase width goal, - prepare for internal search. The new transmit rate is - (NDR lower bound + NDR upper bound) / 2. - - Else, *if* PDR interval does not meet the current phase width goal, - prepare for internal search. The new transmit rate is - (PDR lower bound + PDR upper bound) / 2. - - Else, *if* some bound has still only been measured at a lower duration, - prepare to re-measure at the current duration (and the same transmit rate). - The order of priorities is: - - - NDR lower_bound, - - PDR lower_bound, - - NDR upper_bound, - - PDR upper_bound. - - *Else*, do not prepare any new rate, to exit the phase. - This ensures that at the end of each non-initial phase - all intervals are valid, narrow enough, and measured - at current phase trial duration. - -Implementation Deviations -~~~~~~~~~~~~~~~~~~~~~~~~~ - -This document so far has been describing a simplified version of MLRsearch algorithm. -The full algorithm as implemented contains additional logic, -which makes some of the details (but not general ideas) above incorrect. -Here is a short description of the additional logic as a list of principles, -explaining their main differences from (or additions to) the simplified description, -but without detailing their mutual interaction. - -1. *Logarithmic transmit rate.* - In order to better fit the relative width goal, - the interval doubling and halving is done differently. - For example, the middle of 2 and 8 is 4, not 5. -2. *Optimistic maximum rate.* - The increased rate is never higher than the maximum rate. - Upper bound at that rate is always considered valid. -3. *Pessimistic minimum rate.* - The decreased rate is never lower than the minimum rate. - If a lower bound at that rate is invalid, - a phase stops refining the interval further (until it gets re-measured). -4. *Conservative interval updates.* - Measurements above current upper bound never update a valid upper bound, - even if drop ratio is low. - Measurements below current lower bound always update any lower bound - if drop ratio is high. -5. *Ensure sufficient interval width.* - Narrow intervals make external search take more time to find a valid bound. - If the new transmit increased or decreased rate would result in width - less than the current goal, increase/decrease more. - This can happen if the measurement for the other interval - makes the current interval too narrow. - Similarly, take care the measurements in the initial phase - create wide enough interval. -6. *Timeout for bad cases.* - The worst case for MLRsearch is when each phase converges to intervals - way different than the results of the previous phase. - Rather than suffer total search time several times larger - than pure binary search, the implemented tests fail themselves - when the search takes too long (given by argument *timeout*). - -(B)MRR Throughput ------------------ - -Maximum Receive Rate (MRR) tests are complementary to MLRsearch tests, -as they provide a maximum "raw" throughput benchmark for development and -testing community. MRR tests measure the packet forwarding rate under -the maximum load offered by traffic generator over a set trial duration, -regardless of packet loss. Maximum load for specified Ethernet frame -size is set to the bi-directional link rate. - -In |csit-release| MRR test code has been updated with a configurable -burst MRR parameters: trial duration and number of trials in a single -burst. This enabled a new Burst MRR (BMRR) methodology for more precise -performance trending. - -Current parameters for BMRR tests: - -- Ethernet frame sizes: 64B (78B for IPv6), IMIX, 1518B, 9000B; all - quoted sizes include frame CRC, but exclude per frame transmission - overhead of 20B (preamble, inter frame gap). - -- Maximum load offered: 10GE and 40GE link (sub-)rates depending on NIC - tested, with the actual packet rate depending on frame size, - transmission overhead and traffic generator NIC forwarding capacity. - - - For 10GE NICs the maximum packet rate load is 2* 14.88 Mpps for 64B, - a 10GE bi-directional link rate. - - For 25GE NICs the maximum packet rate load is 2* 18.75 Mpps for 64B, - a 25GE bi-directional link sub-rate limited by TG 25GE NIC used, - XXV710. - - For 40GE NICs the maximum packet rate load is 2* 18.75 Mpps for 64B, - a 40GE bi-directional link sub-rate limited by TG 40GE NIC used, - XL710. Packet rate for other tested frame sizes is limited by PCIe - Gen3 x8 bandwidth limitation of ~50Gbps. - -- Trial duration: 1 sec. - -- Number of trials per burst: 10. - -Similarly to NDR/PDR throughput tests, MRR test should be reporting bi- -directional link rate (or NIC rate, if lower) if tested VPP -configuration can handle the packet rate higher than bi-directional link -rate, e.g. large packet tests and/or multi-core tests. - -MRR tests are currently used for FD.io CSIT continuous performance -trending and for comparison between releases. Daily trending job tests -subset of frame sizes, focusing on 64B (78B for IPv6) for all tests and -IMIX for selected tests (vhost, memif). - -MRR-like measurements are being used to establish starting conditions -for experimental Probabilistic Loss Ratio Search (PLRsearch) used for -soak testing, aimed at verifying continuous system performance over an -extended period of time, hours, days, weeks, months. PLRsearch code is -currently in experimental phase in FD.io CSIT project. - -Packet Latency --------------- - -TRex Traffic Generator (TG) is used for measuring latency of VPP DUTs. -Reported latency values are measured using following methodology: - -- Latency tests are performed at 100% of discovered NDR and PDR rates - for each throughput test and packet size (except IMIX). -- TG sends dedicated latency streams, one per direction, each at the - rate of 9 kpps at the prescribed packet size; these are sent in - addition to the main load streams. -- TG reports min/avg/max latency values per stream direction, hence two - sets of latency values are reported per test case; future release of - TRex is expected to report latency percentiles. -- Reported latency values are aggregate across two SUTs due to three - node topology used for all performance tests; for per SUT latency, - reported value should be divided by two. -- 1usec is the measurement accuracy advertised by TRex TG for the setup - used in FD.io labs used by CSIT project. -- TRex setup introduces an always-on error of about 2*2usec per latency - flow additonal Tx/Rx interface latency induced by TRex SW writing and - reading packet timestamps on CPU cores without HW acceleration on NICs - closer to the interface line. - -Multi-Core Speedup ------------------- - -All performance tests are executed with single processor core and with -multiple cores scenarios. - -Intel Hyper-Threading (HT) -~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Intel Xeon processors used in FD.io CSIT can operate either in HT -Disabled mode (single logical core per each physical core) or in HT -Enabled mode (two logical cores per each physical core). HT setting is -applied in BIOS and requires server SUT reload for it to take effect, -making it impractical for continuous changes of HT mode of operation. - -|csit-release| performance tests are executed with server SUTs' Intel -XEON processors configured with Intel Hyper-Threading Disabled for all -Xeon Haswell testbeds (3n-hsw) and with Intel Hyper-Threading Enabled -for all Xeon Skylake testbeds. - -More information about physical testbeds is provided in -:ref:`tested_physical_topologies`. - -Multi-core Tests -~~~~~~~~~~~~~~~~ - -|csit-release| multi-core tests are executed in the following VPP worker -thread and physical core configurations: - -#. Intel Xeon Haswell testbeds (3n-hsw) with Intel HT disabled - (1 logical CPU core per each physical core): - - #. 1t1c - 1 VPP worker thread on 1 physical core. - #. 2t2c - 2 VPP worker threads on 2 physical cores. - #. 4t4c - 4 VPP worker threads on 4 physical cores. - -#. Intel Xeon Skylake testbeds (2n-skx, 3n-skx) with Intel HT enabled - (2 logical CPU cores per each physical core): - - #. 2t1c - 2 VPP worker threads on 1 physical core. - #. 4t2c - 4 VPP worker threads on 2 physical cores. - #. 8t4c - 8 VPP worker threads on 4 physical cores. - -VPP worker threads are the data plane threads running on isolated -logical cores. With Intel HT enabled VPP workers are placed as sibling -threads on each used physical core. VPP control threads (main, stats) -are running on a separate non-isolated core together with other Linux -processes. - -In all CSIT tests care is taken to ensure that each VPP worker handles -the same amount of received packet load and does the same amount of -packet processing work. This is achieved by evenly distributing per -interface type (e.g. physical, virtual) receive queues over VPP workers -using default VPP round- robin mapping and by loading these queues with -the same amount of packet flows. - -If number of VPP workers is higher than number of physical or virtual -interfaces, multiple receive queues are configured on each interface. -NIC Receive Side Scaling (RSS) for physical interfaces and multi-queue -for virtual interfaces are used for this purpose. - -Section :ref:`throughput_speedup_multi_core` includes a set of graphs -illustrating packet throughout speedup when running VPP worker threads -on multiple cores. Note that in quite a few test cases running VPP -workers on 2 or 4 physical cores hits the I/O bandwidth or packets-per- -second limit of tested NIC. - -VPP Startup Settings --------------------- - -CSIT code manipulates a number of VPP settings in startup.conf for optimized -performance. List of common settings applied to all tests and test -dependent settings follows. - -See `VPP startup.conf <https://git.fd.io/vpp/tree/src/vpp/conf/startup.conf?h=stable/1807>`_ -for a complete set and description of listed settings. - -Common Settings -~~~~~~~~~~~~~~~ - -List of vpp startup.conf settings applied to all tests: - -#. heap-size <value> - set separately for ip4, ip6, stats, main - depending on scale tested. -#. no-tx-checksum-offload - disables UDP / TCP TX checksum offload in DPDK. - Typically needed for use faster vector PMDs (together with - no-multi-seg). -#. socket-mem <value>,<value> - memory per numa. (Not required anymore - due to VPP code changes, should be removed in CSIT-18.10.) - -Per Test Settings -~~~~~~~~~~~~~~~~~ - -List of vpp startup.conf settings applied dynamically per test: - -#. corelist-workers <list_of_cores> - list of logical cores to run VPP - worker data plane threads. Depends on HyperThreading and core per - test configuration. -#. num-rx-queues <value> - depends on a number of VPP threads and NIC - interfaces. -#. num-rx-desc/num-tx-desc - number of rx/tx descriptors for specific - NICs, incl. xl710, x710, xxv710. -#. num-mbufs <value> - increases number of buffers allocated, needed - only in scenarios with large number of interfaces and worker threads. - Value is per CPU socket. Default is 16384. -#. no-multi-seg - disables multi-segment buffers in DPDK, improves - packet throughput, but disables Jumbo MTU support. Disabled for all - tests apart from the ones that require Jumbo 9000B frame support. -#. UIO driver - depends on topology file definition. -#. QAT VFs - depends on NRThreads, each thread = 1QAT VFs. - -KVM VMs vhost-user ------------------- - -FD.io CSIT performance lab is testing VPP vhost with KVM VMs using -following environment settings: - -- Tests with varying Qemu virtio queue (a.k.a. vring) sizes: [vr256] - default 256 descriptors, [vr1024] 1024 descriptors to optimize for - packet throughput. -- Tests with varying Linux :abbr:`CFS (Completely Fair Scheduler)` - settings: [cfs] default settings, [cfsrr1] CFS RoundRobin(1) policy - applied to all data plane threads handling test packet path including - all VPP worker threads and all Qemu testpmd poll-mode threads. -- Resulting test cases are all combinations with [vr256,vr1024] and - [cfs,cfsrr1] settings. -- Adjusted Linux kernel :abbr:`CFS (Completely Fair Scheduler)` - scheduler policy for data plane threads used in CSIT is documented in - `CSIT Performance Environment Tuning wiki - <https://wiki.fd.io/view/CSIT/csit-perf-env-tuning-ubuntu1604>`_. -- The purpose is to verify performance impact (MRR and NDR/PDR - throughput) and same test measurements repeatability, by making VPP - and VM data plane threads less susceptible to other Linux OS system - tasks hijacking CPU cores running those data plane threads. - -LXC/DRC Container Memif ------------------------ - -|csit-release| includes tests taking advantage of VPP memif virtual -interface (shared memory interface) to interconnect VPP running in -Containers. VPP vswitch instance runs in bare-metal user-mode handling -NIC interfaces and connecting over memif (Slave side) to VPPs running in -:abbr:`Linux Container (LXC)` or in Docker Container (DRC) configured -with memif (Master side). LXCs and DRCs run in a priviliged mode with -VPP data plane worker threads pinned to dedicated physical CPU cores per -usual CSIT practice. All VPP instances run the same version of software. -This test topology is equivalent to existing tests with vhost-user and -VMs as described earlier in :ref:`tested_logical_topologies`. - -In addition to above vswitch tests, a single memif interface test is -executed. It runs in a simple topology of two VPP container instances -connected over memif interface in order to verify standalone memif -interface performance. - -More information about CSIT LXC and DRC setup and control is available -in :ref:`container_orchestration_in_csit`. - -K8s Container Memif -------------------- - -|csit-release| includes tests of VPP topologies running in K8s -orchestrated Pods/Containers and connected over memif virtual -interfaces. In order to provide simple topology coding flexibility and -extensibility container orchestration is done with `Kubernetes -<https://github.com/kubernetes>`_ using `Docker -<https://github.com/docker>`_ images for all container applications -including VPP. `Ligato <https://github.com/ligato>`_ is used for the -Pod/Container networking orchestration that is integrated with K8s, -including memif support. - -In these tests VPP vswitch runs in a K8s Pod with Docker Container (DRC) -handling NIC interfaces and connecting over memif to more instances of -VPP running in Pods/DRCs. All DRCs run in a priviliged mode with VPP -data plane worker threads pinned to dedicated physical CPU cores per -usual CSIT practice. All VPP instances run the same version of software. -This test topology is equivalent to existing tests with vhost-user and -VMs as described earlier in :ref:`tested_physical_topologies`. - -Further documentation is available in -:ref:`container_orchestration_in_csit`. - -VPP_Device Functional ---------------------- - -|csit-release| added new VPP_Device test environment for functional VPP -device tests integrated into LFN CI/CD infrastructure. VPP_Device tests -run on 1-Node testbeds (1n-skx, 1n-arm) and rely on Linux SRIOV Virtual -Function (VF), dot1q VLAN tagging and external loopback cables to -facilitate packet passing over exernal physical links. Initial focus is -on few baseline tests. Existing CSIT VIRL tests can be moved to -VPP_Device framework by changing L1 and L2 KW(s). RF test definition -code stays unchanged with the exception of requiring adjustments from -3-Node to 2-Node logical topologies. CSIT VIRL to VPP_Device migration -is expected in the next CSIT release. - -IPSec on Intel QAT ------------------- - -VPP IPSec performance tests are using DPDK cryptodev device driver in -combination with HW cryptodev devices - Intel QAT 8950 50G - present in -LF FD.io physical testbeds. DPDK cryptodev can be used for all IPSec -data plane functions supported by VPP. - -Currently |csit-release| implements following IPSec test cases: - -- AES-GCM, CBC-SHA1 ciphers, in combination with IPv4 routed-forwarding - with Intel xl710 NIC. -- CBC-SHA1 ciphers, in combination with LISP-GPE overlay tunneling for - IPv4-over-IPv4 with Intel xl710 NIC. - -TRex Traffic Generator ----------------------- - -Usage -~~~~~ - -`TRex traffic generator <https://wiki.fd.io/view/TRex>`_ is used for all -CSIT performance tests. TRex stateless mode is used to measure NDR and -PDR throughputs using binary search (NDR and PDR discovery tests) and -for quick checks of DUT performance against the reference NDRs (NDR -check tests) for specific configuration. - -TRex is installed and run on the TG compute node. The typical procedure -is: - -- If the TRex is not already installed on TG, it is installed in the - suite setup phase - see `TRex intallation`_. -- TRex configuration is set in its configuration file - :: - - /etc/trex_cfg.yaml - -- TRex is started in the background mode - :: - - $ sh -c 'cd <t-rex-install-dir>/scripts/ && sudo nohup ./t-rex-64 -i -c 7 --iom 0 > /tmp/trex.log 2>&1 &' > /dev/null - -- There are traffic streams dynamically prepared for each test, based on traffic - profiles. The traffic is sent and the statistics obtained using - :command:`trex_stl_lib.api.STLClient`. - -Measuring Packet Loss -~~~~~~~~~~~~~~~~~~~~~ - -Following sequence is followed to measure packet loss: - -- Create an instance of STLClient. -- Connect to the client. -- Add all streams. -- Clear statistics. -- Send the traffic for defined time. -- Get the statistics. - -If there is a warm-up phase required, the traffic is sent also before -test and the statistics are ignored. - -Measuring Latency -~~~~~~~~~~~~~~~~~ - -If measurement of latency is requested, two more packet streams are -created (one for each direction) with TRex flow_stats parameter set to -STLFlowLatencyStats. In that case, returned statistics will also include -min/avg/max latency values. - -HTTP/TCP with WRK Tool ----------------------- - -`WRK HTTP benchmarking tool <https://github.com/wg/wrk>`_ is used for -experimental TCP/IP and HTTP tests of VPP TCP/IP stack and built-in -static HTTP server. WRK has been chosen as it is capable of generating -significant TCP/IP and HTTP loads by scaling number of threads across -multi-core processors. - -This in turn enables quite high scale benchmarking of the main TCP/IP -and HTTP service including HTTP TCP/IP Connections-Per-Second (CPS), -HTTP Requests-Per-Second and HTTP Bandwidth Throughput. - -The initial tests are designed as follows: - -- HTTP and TCP/IP Connections-Per-Second (CPS) - - - WRK configured to use 8 threads across 8 cores, 1 thread per core. - - Maximum of 50 concurrent connections across all WRK threads. - - Timeout for server responses set to 5 seconds. - - Test duration is 30 seconds. - - Expected HTTP test sequence: - - - Single HTTP GET Request sent per open connection. - - Connection close after valid HTTP reply. - - Resulting flow sequence - 8 packets: >Syn, <Syn-Ack, >Ack, >Req, - <Rep, >Fin, <Fin, >Ack. - -- HTTP Requests-Per-Second - - - WRK configured to use 8 threads across 8 cores, 1 thread per core. - - Maximum of 50 concurrent connections across all WRK threads. - - Timeout for server responses set to 5 seconds. - - Test duration is 30 seconds. - - Expected HTTP test sequence: - - - Multiple HTTP GET Requests sent in sequence per open connection. - - Connection close after set test duration time. - - Resulting flow sequence: >Syn, <Syn-Ack, >Ack, >Req[1], <Rep[1], - .., >Req[n], <Rep[n], >Fin, <Fin, >Ack. - -.. _binary search: https://en.wikipedia.org/wiki/Binary_search -.. _exponential search: https://en.wikipedia.org/wiki/Exponential_search -.. _estimation of standard deviation: https://en.wikipedia.org/wiki/Unbiased_estimation_of_standard_deviation -.. _simplified error propagation formula: https://en.wikipedia.org/wiki/Propagation_of_uncertainty#Simplification +.. toctree:: + + methodology_vpp_forwarding_modes + methodology_tunnel_encapsulations + methodology_vpp_features + methodology_data_plane_throughput + methodology_mlrsearch_tests + methodology_bmrr_throughput + methodology_packet_latency + methodology_multi_core_speedup + methodology_vpp_startup_settings + methodology_kvm_vms_vhost_user + methodology_lxc_drc_container_memif + methodology_k8s_container_memif + methodology_vpp_device_functional + methodology_ipsec_on_intel_qat + methodology_trex_traffic_generator + methodology_http_tcp_with_wrk_tool diff --git a/docs/report/introduction/methodology_bmrr_throughput.rst b/docs/report/introduction/methodology_bmrr_throughput.rst new file mode 100644 index 0000000000..ac3c54e907 --- /dev/null +++ b/docs/report/introduction/methodology_bmrr_throughput.rst @@ -0,0 +1,54 @@ +(B)MRR Throughput +----------------- + +Maximum Receive Rate (MRR) tests are complementary to MLRsearch tests, +as they provide a maximum "raw" throughput benchmark for development and +testing community. MRR tests measure the packet forwarding rate under +the maximum load offered by traffic generator over a set trial duration, +regardless of packet loss. Maximum load for specified Ethernet frame +size is set to the bi-directional link rate. + +In |csit-release| MRR test code has been updated with a configurable +burst MRR parameters: trial duration and number of trials in a single +burst. This enabled a new Burst MRR (BMRR) methodology for more precise +performance trending. + +Current parameters for BMRR tests: + +- Ethernet frame sizes: 64B (78B for IPv6), IMIX, 1518B, 9000B; all + quoted sizes include frame CRC, but exclude per frame transmission + overhead of 20B (preamble, inter frame gap). + +- Maximum load offered: 10GE and 40GE link (sub-)rates depending on NIC + tested, with the actual packet rate depending on frame size, + transmission overhead and traffic generator NIC forwarding capacity. + + - For 10GE NICs the maximum packet rate load is 2* 14.88 Mpps for 64B, + a 10GE bi-directional link rate. + - For 25GE NICs the maximum packet rate load is 2* 18.75 Mpps for 64B, + a 25GE bi-directional link sub-rate limited by TG 25GE NIC used, + XXV710. + - For 40GE NICs the maximum packet rate load is 2* 18.75 Mpps for 64B, + a 40GE bi-directional link sub-rate limited by TG 40GE NIC used, + XL710. Packet rate for other tested frame sizes is limited by PCIe + Gen3 x8 bandwidth limitation of ~50Gbps. + +- Trial duration: 1 sec. + +- Number of trials per burst: 10. + +Similarly to NDR/PDR throughput tests, MRR test should be reporting bi- +directional link rate (or NIC rate, if lower) if tested VPP +configuration can handle the packet rate higher than bi-directional link +rate, e.g. large packet tests and/or multi-core tests. + +MRR tests are currently used for FD.io CSIT continuous performance +trending and for comparison between releases. Daily trending job tests +subset of frame sizes, focusing on 64B (78B for IPv6) for all tests and +IMIX for selected tests (vhost, memif). + +MRR-like measurements are being used to establish starting conditions +for experimental Probabilistic Loss Ratio Search (PLRsearch) used for +soak testing, aimed at verifying continuous system performance over an +extended period of time, hours, days, weeks, months. PLRsearch code is +currently in experimental phase in FD.io CSIT project. diff --git a/docs/report/introduction/methodology_data_plane_throughput.rst b/docs/report/introduction/methodology_data_plane_throughput.rst new file mode 100644 index 0000000000..10d85e991c --- /dev/null +++ b/docs/report/introduction/methodology_data_plane_throughput.rst @@ -0,0 +1,33 @@ +Data Plane Throughput +--------------------- + +Network data plane packet and bandwidth throughput are measured in +accordance with :rfc:`2544`, using FD.io CSIT Multiple Loss Ratio search +(MLRsearch), an optimized throughput search algorithm, that measures +SUT/DUT packet throughput rates at different Packet Loss Ratio (PLR) +values. + +Following MLRsearch values are measured across a range of L2 frame sizes +and reported: + +- NON DROP RATE (NDR): packet and bandwidth throughput at PLR=0%. + + - **Aggregate packet rate**: NDR_LOWER <bi-directional packet rate> + pps. + - **Aggregate bandwidth rate**: NDR_LOWER <bi-directional bandwidth + rate> Gbps. + +- PARTIAL DROP RATE (PDR): packet and bandwidth throughput at PLR=0.5%. + + - **Aggregate packet rate**: PDR_LOWER <bi-directional packet rate> + pps. + - **Aggregate bandwidth rate**: PDR_LOWER <bi-directional bandwidth + rate> Gbps. + +NDR and PDR are measured for the following L2 frame sizes (untagged +Ethernet): + +- IPv4 payload: 64B, IMIX (28x64B, 16x570B, 4x1518B), 1518B, 9000B. +- IPv6 payload: 78B, IMIX (28x78B, 16x570B, 4x1518B), 1518B, 9000B. + +All rates are reported from external Traffic Generator perspective. diff --git a/docs/report/introduction/methodology_http_tcp_with_wrk_tool.rst b/docs/report/introduction/methodology_http_tcp_with_wrk_tool.rst new file mode 100644 index 0000000000..28f3fc6bbb --- /dev/null +++ b/docs/report/introduction/methodology_http_tcp_with_wrk_tool.rst @@ -0,0 +1,40 @@ +HTTP/TCP with WRK Tool +---------------------- + +`WRK HTTP benchmarking tool <https://github.com/wg/wrk>`_ is used for +experimental TCP/IP and HTTP tests of VPP TCP/IP stack and built-in +static HTTP server. WRK has been chosen as it is capable of generating +significant TCP/IP and HTTP loads by scaling number of threads across +multi-core processors. + +This in turn enables quite high scale benchmarking of the main TCP/IP +and HTTP service including HTTP TCP/IP Connections-Per-Second (CPS), +HTTP Requests-Per-Second and HTTP Bandwidth Throughput. + +The initial tests are designed as follows: + +- HTTP and TCP/IP Connections-Per-Second (CPS) + + - WRK configured to use 8 threads across 8 cores, 1 thread per core. + - Maximum of 50 concurrent connections across all WRK threads. + - Timeout for server responses set to 5 seconds. + - Test duration is 30 seconds. + - Expected HTTP test sequence: + + - Single HTTP GET Request sent per open connection. + - Connection close after valid HTTP reply. + - Resulting flow sequence - 8 packets: >Syn, <Syn-Ack, >Ack, >Req, + <Rep, >Fin, <Fin, >Ack. + +- HTTP Requests-Per-Second + + - WRK configured to use 8 threads across 8 cores, 1 thread per core. + - Maximum of 50 concurrent connections across all WRK threads. + - Timeout for server responses set to 5 seconds. + - Test duration is 30 seconds. + - Expected HTTP test sequence: + + - Multiple HTTP GET Requests sent in sequence per open connection. + - Connection close after set test duration time. + - Resulting flow sequence: >Syn, <Syn-Ack, >Ack, >Req[1], <Rep[1], + .., >Req[n], <Rep[n], >Fin, <Fin, >Ack. diff --git a/docs/report/introduction/methodology_ipsec_on_intel_qat.rst b/docs/report/introduction/methodology_ipsec_on_intel_qat.rst new file mode 100644 index 0000000000..54fb1b0ef2 --- /dev/null +++ b/docs/report/introduction/methodology_ipsec_on_intel_qat.rst @@ -0,0 +1,14 @@ +IPSec on Intel QAT +------------------ + +VPP IPSec performance tests are using DPDK cryptodev device driver in +combination with HW cryptodev devices - Intel QAT 8950 50G - present in +LF FD.io physical testbeds. DPDK cryptodev can be used for all IPSec +data plane functions supported by VPP. + +Currently |csit-release| implements following IPSec test cases: + +- AES-GCM, CBC-SHA1 ciphers, in combination with IPv4 routed-forwarding + with Intel xl710 NIC. +- CBC-SHA1 ciphers, in combination with LISP-GPE overlay tunneling for + IPv4-over-IPv4 with Intel xl710 NIC. diff --git a/docs/report/introduction/methodology_k8s_container_memif.rst b/docs/report/introduction/methodology_k8s_container_memif.rst new file mode 100644 index 0000000000..2e5ce0e017 --- /dev/null +++ b/docs/report/introduction/methodology_k8s_container_memif.rst @@ -0,0 +1,23 @@ +K8s Container Memif +------------------- + +|csit-release| includes tests of VPP topologies running in K8s +orchestrated Pods/Containers and connected over memif virtual +interfaces. In order to provide simple topology coding flexibility and +extensibility container orchestration is done with `Kubernetes +<https://github.com/kubernetes>`_ using `Docker +<https://github.com/docker>`_ images for all container applications +including VPP. `Ligato <https://github.com/ligato>`_ is used for the +Pod/Container networking orchestration that is integrated with K8s, +including memif support. + +In these tests VPP vswitch runs in a K8s Pod with Docker Container (DRC) +handling NIC interfaces and connecting over memif to more instances of +VPP running in Pods/DRCs. All DRCs run in a priviliged mode with VPP +data plane worker threads pinned to dedicated physical CPU cores per +usual CSIT practice. All VPP instances run the same version of software. +This test topology is equivalent to existing tests with vhost-user and +VMs as described earlier in :ref:`tested_physical_topologies`. + +Further documentation is available in +:ref:`container_orchestration_in_csit`. diff --git a/docs/report/introduction/methodology_kvm_vms_vhost_user.rst b/docs/report/introduction/methodology_kvm_vms_vhost_user.rst new file mode 100644 index 0000000000..34e3bc0447 --- /dev/null +++ b/docs/report/introduction/methodology_kvm_vms_vhost_user.rst @@ -0,0 +1,23 @@ +KVM VMs vhost-user +------------------ + +FD.io CSIT performance lab is testing VPP vhost with KVM VMs using +following environment settings: + +- Tests with varying Qemu virtio queue (a.k.a. vring) sizes: [vr256] + default 256 descriptors, [vr1024] 1024 descriptors to optimize for + packet throughput. +- Tests with varying Linux :abbr:`CFS (Completely Fair Scheduler)` + settings: [cfs] default settings, [cfsrr1] CFS RoundRobin(1) policy + applied to all data plane threads handling test packet path including + all VPP worker threads and all Qemu testpmd poll-mode threads. +- Resulting test cases are all combinations with [vr256,vr1024] and + [cfs,cfsrr1] settings. +- Adjusted Linux kernel :abbr:`CFS (Completely Fair Scheduler)` + scheduler policy for data plane threads used in CSIT is documented in + `CSIT Performance Environment Tuning wiki + <https://wiki.fd.io/view/CSIT/csit-perf-env-tuning-ubuntu1604>`_. +- The purpose is to verify performance impact (MRR and NDR/PDR + throughput) and same test measurements repeatability, by making VPP + and VM data plane threads less susceptible to other Linux OS system + tasks hijacking CPU cores running those data plane threads. diff --git a/docs/report/introduction/methodology_lxc_drc_container_memif.rst b/docs/report/introduction/methodology_lxc_drc_container_memif.rst new file mode 100644 index 0000000000..56f052550c --- /dev/null +++ b/docs/report/introduction/methodology_lxc_drc_container_memif.rst @@ -0,0 +1,21 @@ +LXC/DRC Container Memif +----------------------- + +|csit-release| includes tests taking advantage of VPP memif virtual +interface (shared memory interface) to interconnect VPP running in +Containers. VPP vswitch instance runs in bare-metal user-mode handling +NIC interfaces and connecting over memif (Slave side) to VPPs running in +:abbr:`Linux Container (LXC)` or in Docker Container (DRC) configured +with memif (Master side). LXCs and DRCs run in a priviliged mode with +VPP data plane worker threads pinned to dedicated physical CPU cores per +usual CSIT practice. All VPP instances run the same version of software. +This test topology is equivalent to existing tests with vhost-user and +VMs as described earlier in :ref:`tested_logical_topologies`. + +In addition to above vswitch tests, a single memif interface test is +executed. It runs in a simple topology of two VPP container instances +connected over memif interface in order to verify standalone memif +interface performance. + +More information about CSIT LXC and DRC setup and control is available +in :ref:`container_orchestration_in_csit`. diff --git a/docs/report/introduction/methodology_mlrsearch_tests.rst b/docs/report/introduction/methodology_mlrsearch_tests.rst new file mode 100644 index 0000000000..15b9ee0290 --- /dev/null +++ b/docs/report/introduction/methodology_mlrsearch_tests.rst @@ -0,0 +1,292 @@ + +.. _mlrsearch_algorithm: + +MLRsearch Tests +--------------- + +Multiple Loss Rate search (MLRsearch) tests use new search algorithm +implemented in FD.io CSIT project. MLRsearch discovers multiple packet +throughput rates in a single search, with each rate associated with a +distinct Packet Loss Ratio (PLR) criteria. MLRsearch is being +standardized in IETF with `draft-vpolak-mkonstan-mlrsearch-XX +<https://tools.ietf.org/html/draft-vpolak-mkonstan-mlrsearch-00>`_. + +Two throughput measurements used in FD.io CSIT are Non-Drop Rate (NDR, +with zero packet loss, PLR=0) and Partial Drop Rate (PDR, with packet +loss rate not greater than the configured non-zero PLR). MLRsearch +discovers NDR and PDR in a single pass reducing required execution time +compared to separate binary searches for NDR and PDR. MLRsearch reduces +execution time even further by relying on shorter trial durations +of intermediate steps, with only the final measurements +conducted at the specified final trial duration. +This results in the shorter overall search +execution time when compared to a standard NDR/PDR binary search, +while guaranteeing the same or similar results. + +If needed, MLRsearch can be easily adopted to discover more throughput rates +with different pre-defined PLRs. + +.. Note:: All throughput rates are *always* bi-directional + aggregates of two equal (symmetric) uni-directional packet rates + received and reported by an external traffic generator. + +Overview +~~~~~~~~ + +The main properties of MLRsearch: + +- MLRsearch is a duration aware multi-phase multi-rate search algorithm. + + - Initial phase determines promising starting interval for the search. + - Intermediate phases progress towards defined final search criteria. + - Final phase executes measurements according to the final search + criteria. + +- *Initial phase*: + + - Uses link rate as a starting transmit rate and discovers the Maximum + Receive Rate (MRR) used as an input to the first intermediate phase. + +- *Intermediate phases*: + + - Start with initial trial duration (in the first phase) and converge + geometrically towards the final trial duration (in the final phase). + - Track two values for NDR and two for PDR. + + - The values are called (NDR or PDR) lower_bound and upper_bound. + - Each value comes from a specific trial measurement + (most recent for that transmit rate), + and as such the value is associated with that measurement's duration and + loss. + - A bound can be invalid, for example if NDR lower_bound + has been measured with nonzero loss. + - Invalid bounds are not real boundaries for the searched value, + but are needed to track interval widths. + - Valid bounds are real boundaries for the searched value. + - Each non-initial phase ends with all bounds valid. + + - Start with a large (lower_bound, upper_bound) interval width and + geometrically converge towards the width goal (measurement resolution) + of the phase. Each phase halves the previous width goal. + - Use internal and external searches: + + - External search - measures at transmit rates outside the (lower_bound, + upper_bound) interval. Activated when a bound is invalid, + to search for a new valid bound by doubling the interval width. + It is a variant of `exponential search`_. + - Internal search - `binary search`_, measures at transmit rates within the + (lower_bound, upper_bound) valid interval, halving the interval width. + +- *Final phase* is executed with the final test trial duration, and the final + width goal that determines resolution of the overall search. + Intermediate phases together with the final phase are called non-initial + phases. + +The main benefits of MLRsearch vs. binary search include: + +- In general MLRsearch is likely to execute more search trials overall, but + less trials at a set final duration. +- In well behaving cases it greatly reduces (>50%) the overall duration + compared to a single PDR (or NDR) binary search duration, + while finding multiple drop rates. +- In all cases MLRsearch yields the same or similar results to binary search. +- Note: both binary search and MLRsearch are susceptible to reporting + non-repeatable results across multiple runs for very bad behaving + cases. + +Caveats: + +- Worst case MLRsearch can take longer than a binary search e.g. in case of + drastic changes in behaviour for trials at varying durations. + +Search Implementation +~~~~~~~~~~~~~~~~~~~~~ + +Following is a brief description of the current MLRsearch +implementation in FD.io CSIT. + +Input Parameters +```````````````` + +#. *maximum_transmit_rate* - maximum packet transmit rate to be used by + external traffic generator, limited by either the actual Ethernet + link rate or traffic generator NIC model capabilities. Sample + defaults: 2 * 14.88 Mpps for 64B 10GE link rate, + 2 * 18.75 Mpps for 64B 40GE NIC maximum rate. +#. *minimum_transmit_rate* - minimum packet transmit rate to be used for + measurements. MLRsearch fails if lower transmit rate needs to be + used to meet search criteria. Default: 2 * 10 kpps (could be higher). +#. *final_trial_duration* - required trial duration for final rate + measurements. Default: 30 sec. +#. *initial_trial_duration* - trial duration for initial MLRsearch phase. + Default: 1 sec. +#. *final_relative_width* - required measurement resolution expressed as + (lower_bound, upper_bound) interval width relative to upper_bound. + Default: 0.5%. +#. *packet_loss_ratio* - maximum acceptable PLR search criteria for + PDR measurements. Default: 0.5%. +#. *number_of_intermediate_phases* - number of phases between the initial + phase and the final phase. Impacts the overall MLRsearch duration. + Less phases are required for well behaving cases, more phases + may be needed to reduce the overall search duration for worse behaving + cases. + Default (2). (Value chosen based on limited experimentation to date. + More experimentation needed to arrive to clearer guidelines.) + +Initial Phase +````````````` + +1. First trial measures at maximum rate and discovers MRR. + + a. *in*: trial_duration = initial_trial_duration. + b. *in*: offered_transmit_rate = maximum_transmit_rate. + c. *do*: single trial. + d. *out*: measured loss ratio. + e. *out*: mrr = measured receive rate. + +2. Second trial measures at MRR and discovers MRR2. + + a. *in*: trial_duration = initial_trial_duration. + b. *in*: offered_transmit_rate = MRR. + c. *do*: single trial. + d. *out*: measured loss ratio. + e. *out*: mrr2 = measured receive rate. + +3. Third trial measures at MRR2. + + a. *in*: trial_duration = initial_trial_duration. + b. *in*: offered_transmit_rate = MRR2. + c. *do*: single trial. + d. *out*: measured loss ratio. + +Non-initial Phases +`````````````````` + +1. Main loop: + + a. *in*: trial_duration for the current phase. + Set to initial_trial_duration for the first intermediate phase; + to final_trial_duration for the final phase; + or to the element of interpolating geometric sequence + for other intermediate phases. + For example with two intermediate phases, trial_duration + of the second intermediate phase is the geometric average + of initial_strial_duration and final_trial_duration. + b. *in*: relative_width_goal for the current phase. + Set to final_relative_width for the final phase; + doubled for each preceding phase. + For example with two intermediate phases, + the first intermediate phase uses quadruple of final_relative_width + and the second intermediate phase uses double of final_relative_width. + c. *in*: ndr_interval, pdr_interval from the previous main loop iteration + or the previous phase. + If the previous phase is the initial phase, both intervals have + lower_bound = MRR2, uper_bound = MRR. + Note that the initial phase is likely to create intervals with invalid + bounds. + d. *do*: According to the procedure described in point 2, + either exit the phase (by jumping to 1.g.), + or prepare new transmit rate to measure with. + e. *do*: Perform the trial measurement at the new transmit rate + and trial_duration, compute its loss ratio. + f. *do*: Update the bounds of both intervals, based on the new measurement. + The actual update rules are numerous, as NDR external search + can affect PDR interval and vice versa, but the result + agrees with rules of both internal and external search. + For example, any new measurement below an invalid lower_bound + becomes the new lower_bound, while the old measurement + (previously acting as the invalid lower_bound) + becomes a new and valid upper_bound. + Go to next iteration (1.c.), taking the updated intervals as new input. + g. *out*: current ndr_interval and pdr_interval. + In the final phase this is also considered + to be the result of the whole search. + For other phases, the next phase loop is started + with the current results as an input. + +2. New transmit rate (or exit) calculation (for 1.d.): + + - If there is an invalid bound then prepare for external search: + + - *If* the most recent measurement at NDR lower_bound transmit rate + had the loss higher than zero, then + the new transmit rate is NDR lower_bound + decreased by two NDR interval widths. + - Else, *if* the most recent measurement at PDR lower_bound + transmit rate had the loss higher than PLR, then + the new transmit rate is PDR lower_bound + decreased by two PDR interval widths. + - Else, *if* the most recent measurement at NDR upper_bound + transmit rate had no loss, then + the new transmit rate is NDR upper_bound + increased by two NDR interval widths. + - Else, *if* the most recent measurement at PDR upper_bound + transmit rate had the loss lower or equal to PLR, then + the new transmit rate is PDR upper_bound + increased by two PDR interval widths. + - If interval width is higher than the current phase goal: + + - Else, *if* NDR interval does not meet the current phase width goal, + prepare for internal search. The new transmit rate is + (NDR lower bound + NDR upper bound) / 2. + - Else, *if* PDR interval does not meet the current phase width goal, + prepare for internal search. The new transmit rate is + (PDR lower bound + PDR upper bound) / 2. + - Else, *if* some bound has still only been measured at a lower duration, + prepare to re-measure at the current duration (and the same transmit + rate). The order of priorities is: + + - NDR lower_bound, + - PDR lower_bound, + - NDR upper_bound, + - PDR upper_bound. + - *Else*, do not prepare any new rate, to exit the phase. + This ensures that at the end of each non-initial phase + all intervals are valid, narrow enough, and measured + at current phase trial duration. + +Implementation Deviations +~~~~~~~~~~~~~~~~~~~~~~~~~ + +This document so far has been describing a simplified version of MLRsearch +algorithm. The full algorithm as implemented contains additional logic, +which makes some of the details (but not general ideas) above incorrect. +Here is a short description of the additional logic as a list of principles, +explaining their main differences from (or additions to) the simplified +description,but without detailing their mutual interaction. + +1. *Logarithmic transmit rate.* + In order to better fit the relative width goal, + the interval doubling and halving is done differently. + For example, the middle of 2 and 8 is 4, not 5. +2. *Optimistic maximum rate.* + The increased rate is never higher than the maximum rate. + Upper bound at that rate is always considered valid. +3. *Pessimistic minimum rate.* + The decreased rate is never lower than the minimum rate. + If a lower bound at that rate is invalid, + a phase stops refining the interval further (until it gets re-measured). +4. *Conservative interval updates.* + Measurements above current upper bound never update a valid upper bound, + even if drop ratio is low. + Measurements below current lower bound always update any lower bound + if drop ratio is high. +5. *Ensure sufficient interval width.* + Narrow intervals make external search take more time to find a valid bound. + If the new transmit increased or decreased rate would result in width + less than the current goal, increase/decrease more. + This can happen if the measurement for the other interval + makes the current interval too narrow. + Similarly, take care the measurements in the initial phase + create wide enough interval. +6. *Timeout for bad cases.* + The worst case for MLRsearch is when each phase converges to intervals + way different than the results of the previous phase. + Rather than suffer total search time several times larger + than pure binary search, the implemented tests fail themselves + when the search takes too long (given by argument *timeout*). + +.. _binary search: https://en.wikipedia.org/wiki/Binary_search +.. _exponential search: https://en.wikipedia.org/wiki/Exponential_search +.. _estimation of standard deviation: https://en.wikipedia.org/wiki/Unbiased_estimation_of_standard_deviation +.. _simplified error propagation formula: https://en.wikipedia.org/wiki/Propagation_of_uncertainty#Simplification diff --git a/docs/report/introduction/methodology_multi_core_speedup.rst b/docs/report/introduction/methodology_multi_core_speedup.rst new file mode 100644 index 0000000000..94840406a1 --- /dev/null +++ b/docs/report/introduction/methodology_multi_core_speedup.rst @@ -0,0 +1,66 @@ +Multi-Core Speedup +------------------ + +All performance tests are executed with single processor core and with +multiple cores scenarios. + +Intel Hyper-Threading (HT) +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Intel Xeon processors used in FD.io CSIT can operate either in HT +Disabled mode (single logical core per each physical core) or in HT +Enabled mode (two logical cores per each physical core). HT setting is +applied in BIOS and requires server SUT reload for it to take effect, +making it impractical for continuous changes of HT mode of operation. + +|csit-release| performance tests are executed with server SUTs' Intel +XEON processors configured with Intel Hyper-Threading Disabled for all +Xeon Haswell testbeds (3n-hsw) and with Intel Hyper-Threading Enabled +for all Xeon Skylake testbeds. + +More information about physical testbeds is provided in +:ref:`tested_physical_topologies`. + +Multi-core Tests +~~~~~~~~~~~~~~~~ + +|csit-release| multi-core tests are executed in the following VPP worker +thread and physical core configurations: + +#. Intel Xeon Haswell testbeds (3n-hsw) with Intel HT disabled + (1 logical CPU core per each physical core): + + #. 1t1c - 1 VPP worker thread on 1 physical core. + #. 2t2c - 2 VPP worker threads on 2 physical cores. + #. 4t4c - 4 VPP worker threads on 4 physical cores. + +#. Intel Xeon Skylake testbeds (2n-skx, 3n-skx) with Intel HT enabled + (2 logical CPU cores per each physical core): + + #. 2t1c - 2 VPP worker threads on 1 physical core. + #. 4t2c - 4 VPP worker threads on 2 physical cores. + #. 8t4c - 8 VPP worker threads on 4 physical cores. + +VPP worker threads are the data plane threads running on isolated +logical cores. With Intel HT enabled VPP workers are placed as sibling +threads on each used physical core. VPP control threads (main, stats) +are running on a separate non-isolated core together with other Linux +processes. + +In all CSIT tests care is taken to ensure that each VPP worker handles +the same amount of received packet load and does the same amount of +packet processing work. This is achieved by evenly distributing per +interface type (e.g. physical, virtual) receive queues over VPP workers +using default VPP round- robin mapping and by loading these queues with +the same amount of packet flows. + +If number of VPP workers is higher than number of physical or virtual +interfaces, multiple receive queues are configured on each interface. +NIC Receive Side Scaling (RSS) for physical interfaces and multi-queue +for virtual interfaces are used for this purpose. + +Section :ref:`throughput_speedup_multi_core` includes a set of graphs +illustrating packet throughout speedup when running VPP worker threads +on multiple cores. Note that in quite a few test cases running VPP +workers on 2 or 4 physical cores hits the I/O bandwidth or packets-per- +second limit of tested NIC. diff --git a/docs/report/introduction/methodology_packet_latency.rst b/docs/report/introduction/methodology_packet_latency.rst new file mode 100644 index 0000000000..550d12f688 --- /dev/null +++ b/docs/report/introduction/methodology_packet_latency.rst @@ -0,0 +1,23 @@ +Packet Latency +-------------- + +TRex Traffic Generator (TG) is used for measuring latency of VPP DUTs. +Reported latency values are measured using following methodology: + +- Latency tests are performed at 100% of discovered NDR and PDR rates + for each throughput test and packet size (except IMIX). +- TG sends dedicated latency streams, one per direction, each at the + rate of 9 kpps at the prescribed packet size; these are sent in + addition to the main load streams. +- TG reports min/avg/max latency values per stream direction, hence two + sets of latency values are reported per test case; future release of + TRex is expected to report latency percentiles. +- Reported latency values are aggregate across two SUTs due to three + node topology used for all performance tests; for per SUT latency, + reported value should be divided by two. +- 1usec is the measurement accuracy advertised by TRex TG for the setup + used in FD.io labs used by CSIT project. +- TRex setup introduces an always-on error of about 2*2usec per latency + flow additonal Tx/Rx interface latency induced by TRex SW writing and + reading packet timestamps on CPU cores without HW acceleration on NICs + closer to the interface line. diff --git a/docs/report/introduction/methodology_trex_traffic_generator.rst b/docs/report/introduction/methodology_trex_traffic_generator.rst new file mode 100644 index 0000000000..4d4de96fb0 --- /dev/null +++ b/docs/report/introduction/methodology_trex_traffic_generator.rst @@ -0,0 +1,53 @@ +TRex Traffic Generator +---------------------- + +Usage +~~~~~ + +`TRex traffic generator <https://wiki.fd.io/view/TRex>`_ is used for all +CSIT performance tests. TRex stateless mode is used to measure NDR and +PDR throughputs using binary search (NDR and PDR discovery tests) and +for quick checks of DUT performance against the reference NDRs (NDR +check tests) for specific configuration. + +TRex is installed and run on the TG compute node. The typical procedure +is: + +- If the TRex is not already installed on TG, it is installed in the + suite setup phase - see `TRex intallation`_. +- TRex configuration is set in its configuration file + :: + + /etc/trex_cfg.yaml + +- TRex is started in the background mode + :: + + $ sh -c 'cd <t-rex-install-dir>/scripts/ && sudo nohup ./t-rex-64 -i -c 7 --iom 0 > /tmp/trex.log 2>&1 &' > /dev/null + +- There are traffic streams dynamically prepared for each test, based on traffic + profiles. The traffic is sent and the statistics obtained using + :command:`trex_stl_lib.api.STLClient`. + +Measuring Packet Loss +~~~~~~~~~~~~~~~~~~~~~ + +Following sequence is followed to measure packet loss: + +- Create an instance of STLClient. +- Connect to the client. +- Add all streams. +- Clear statistics. +- Send the traffic for defined time. +- Get the statistics. + +If there is a warm-up phase required, the traffic is sent also before +test and the statistics are ignored. + +Measuring Latency +~~~~~~~~~~~~~~~~~ + +If measurement of latency is requested, two more packet streams are +created (one for each direction) with TRex flow_stats parameter set to +STLFlowLatencyStats. In that case, returned statistics will also include +min/avg/max latency values. diff --git a/docs/report/introduction/methodology_tunnel_encapsulations.rst b/docs/report/introduction/methodology_tunnel_encapsulations.rst new file mode 100644 index 0000000000..6c47d1bd33 --- /dev/null +++ b/docs/report/introduction/methodology_tunnel_encapsulations.rst @@ -0,0 +1,38 @@ +Tunnel Encapsulations +--------------------- + +Tunnel encapsulations testing is grouped based on the type of outer +header: IPv4 or IPv6. + +IPv4 Tunnels +~~~~~~~~~~~~ + +VPP is tested in the following IPv4 tunnel baseline configurations: + +- *ip4vxlan-l2bdbase*: VXLAN over IPv4 tunnels with L2 bridge-domain MAC + switching. +- *ip4vxlan-l2xcbase*: VXLAN over IPv4 tunnels with L2 cross-connect. +- *ip4lispip4-ip4base*: LISP over IPv4 tunnels with IPv4 routing. +- *ip4lispip6-ip6base*: LISP over IPv4 tunnels with IPv6 routing. + +In all cases listed above low number of MAC, IPv4, IPv6 flows (253 per +direction) is switched or routed by VPP. + +In addition selected IPv4 tunnels are tested at scale: + +- *dot1q--ip4vxlanscale-l2bd*: VXLAN over IPv4 tunnels with L2 bridge- + domain MAC switching, with scaled up dot1q VLANs (10, 100, 1k), + mapped to scaled up L2 bridge-domains (10, 100, 1k), that are in turn + mapped to (10, 100, 1k) VXLAN tunnels. 64.5k flows are transmitted per + direction. + +IPv6 Tunnels +~~~~~~~~~~~~ + +VPP is tested in the following IPv6 tunnel baseline configurations: + +- *ip6lispip4-ip4base*: LISP over IPv4 tunnels with IPv4 routing. +- *ip6lispip6-ip6base*: LISP over IPv4 tunnels with IPv6 routing. + +In all cases listed above low number of IPv4, IPv6 flows (253 per +direction) is routed by VPP. diff --git a/docs/report/introduction/methodology_vpp_device_functional.rst b/docs/report/introduction/methodology_vpp_device_functional.rst new file mode 100644 index 0000000000..41a8040ef6 --- /dev/null +++ b/docs/report/introduction/methodology_vpp_device_functional.rst @@ -0,0 +1,13 @@ +VPP_Device Functional +--------------------- + +|csit-release| added new VPP_Device test environment for functional VPP +device tests integrated into LFN CI/CD infrastructure. VPP_Device tests +run on 1-Node testbeds (1n-skx, 1n-arm) and rely on Linux SRIOV Virtual +Function (VF), dot1q VLAN tagging and external loopback cables to +facilitate packet passing over exernal physical links. Initial focus is +on few baseline tests. Existing CSIT VIRL tests can be moved to +VPP_Device framework by changing L1 and L2 KW(s). RF test definition +code stays unchanged with the exception of requiring adjustments from +3-Node to 2-Node logical topologies. CSIT VIRL to VPP_Device migration +is expected in the next CSIT release. diff --git a/docs/report/introduction/methodology_vpp_features.rst b/docs/report/introduction/methodology_vpp_features.rst new file mode 100644 index 0000000000..a19caa4428 --- /dev/null +++ b/docs/report/introduction/methodology_vpp_features.rst @@ -0,0 +1,80 @@ +VPP Features +------------ + +VPP is tested in a number of data plane feature configurations across +different forwarding modes. Following sections list features tested. + +ACL Security-Groups +~~~~~~~~~~~~~~~~~~~ + +Both stateless and stateful access control lists (ACL), also known as +security-groups, are supported by VPP. + +Following ACL configurations are tested for MAC switching with L2 +bridge-domains: + +- *l2bdbasemaclrn-iacl{E}sl-{F}flows*: Input stateless ACL, with {E} + entries and {F} flows. +- *l2bdbasemaclrn-oacl{E}sl-{F}flows*: Output stateless ACL, with {E} + entries and {F} flows. +- *l2bdbasemaclrn-iacl{E}sf-{F}flows*: Input stateful ACL, with {E} + entries and {F} flows. +- *l2bdbasemaclrn-oacl{E}sf-{F}flows*: Output stateful ACL, with {E} + entries and {F} flows. + +Following ACL configurations are tested with IPv4 routing: + +- *ip4base-iacl{E}sl-{F}flows*: Input stateless ACL, with {E} entries + and {F} flows. +- *ip4base-oacl{E}sl-{F}flows*: Output stateless ACL, with {E} entries + and {F} flows. +- *ip4base-iacl{E}sf-{F}flows*: Input stateful ACL, with {E} entries and + {F} flows. +- *ip4base-oacl{E}sf-{F}flows*: Output stateful ACL, with {E} entries + and {F} flows. + +ACL tests are executed with the following combinations of ACL entries +and number of flows: + +- ACL entry definitions + + - flow non-matching deny entry: (src-ip4, dst-ip4, src-port, dst-port). + - flow matching permit ACL entry: (src-ip4, dst-ip4). + +- {E} - number of non-matching deny ACL entries, {E} = [1, 10, 50]. +- {F} - number of UDP flows with different tuple (src-ip4, dst-ip4, + src-port, dst-port), {F} = [100, 10k, 100k]. +- All {E}x{F} combinations are tested per ACL type, total of 9. + +ACL MAC-IP +~~~~~~~~~~ + +MAC-IP binding ACLs are tested for MAC switching with L2 bridge-domains: + +- *l2bdbasemaclrn-macip-iacl{E}sl-{F}flows*: Input stateless ACL, with + {E} entries and {F} flows. + +MAC-IP ACL tests are executed with the following combinations of ACL +entries and number of flows: + +- ACL entry definitions + + - flow non-matching deny entry: (dst-ip4, dst-mac, bit-mask) + - flow matching permit ACL entry: (dst-ip4, dst-mac, bit-mask) + +- {E} - number of non-matching deny ACL entries, {E} = [1, 10, 50] +- {F} - number of UDP flows with different tuple (dst-ip4, dst-mac), + {F} = [100, 10k, 100k] +- All {E}x{F} combinations are tested per ACL type, total of 9. + +NAT44 +~~~~~ + +NAT44 is tested in baseline and scale configurations with IPv4 routing: + +- *ip4base-nat44*: baseline test with single NAT entry (addr, port), + single UDP flow. +- *ip4base-udpsrcscale{U}-nat44*: baseline test with {U} NAT entries + (addr, {U}ports), {U}=15. +- *ip4scale{R}-udpsrcscale{U}-nat44*: scale tests with {R}*{U} NAT + entries ({R}addr, {U}ports), {R}=[100, 1k, 2k, 4k], {U}=15. diff --git a/docs/report/introduction/methodology_vpp_forwarding_modes.rst b/docs/report/introduction/methodology_vpp_forwarding_modes.rst new file mode 100644 index 0000000000..6cf206f2f6 --- /dev/null +++ b/docs/report/introduction/methodology_vpp_forwarding_modes.rst @@ -0,0 +1,90 @@ +VPP Forwarding Modes +-------------------- + +VPP is tested in a number of L2 and IP packet lookup and forwarding +modes. Within each mode baseline and scale tests are executed, the +latter with varying number of lookup entries. + +L2 Ethernet Switching +~~~~~~~~~~~~~~~~~~~~~ + +VPP is tested in three L2 forwarding modes: + +- *l2patch*: L2 patch, the fastest point-to-point L2 path that loops + packets between two interfaces without any Ethernet frame checks or + lookups. +- *l2xc*: L2 cross-connect, point-to-point L2 path with all Ethernet + frame checks, but no MAC learning and no MAC lookup. +- *l2bd*: L2 bridge-domain, multipoint-to-multipoint L2 path with all + Ethernet frame checks, with MAC learning (unless static MACs are used) + and MAC lookup. + +l2bd tests are executed in baseline and scale configurations: + +- *l2bdbase*: low number of L2 flows (253 per direction) is switched by + VPP. They drive the content of MAC FIB size (506 total MAC entries). + Both source and destination MAC addresses are incremented on a packet + by packet basis. + +- *l2bdscale*: high number of L2 flows is switched by VPP. Tested MAC + FIB sizes include: i) 10k (5k unique flows per direction), ii) 100k + (2x 50k flows) and iii) 1M (2x 500k). Both source and destination MAC + addresses are incremented on a packet by packet basis, ensuring new + entries are learn refreshed and looked up at every packet, making it + the worst case scenario. + +Ethernet wire encapsulations tested include: untagged, dot1q, dot1ad. + +IPv4 Routing +~~~~~~~~~~~~ + +IPv4 routing tests are executed in baseline and scale configurations: + +- *ip4base*: low number of IPv4 flows (253 per direction) is routed by + VPP. They drive the content of IPv4 FIB size (506 total /32 prefixes). + Destination IPv4 addresses are incremented on a packet by packet + basis. + +- *ip4scale*: high number of IPv4 flows is routed by VPP. Tested IPv4 + FIB sizes of /32 prefixes include: i) 20k (10k unique flows per + direction), ii) 200k (2x 100k flows) and iii) 2M (2x 1M). Destination + IPv4 addresses are incremented on a packet by packet basis, ensuring + new FIB entries are looked up at every packet, making it the worst + case scenario. + +IPv6 Routing +~~~~~~~~~~~~ + +IPv6 routing tests are executed in baseline and scale configurations: + +- *ip6base*: low number of IPv6 flows (253 per direction) is routed by + VPP. They drive the content of IPv6 FIB size (506 total /128 prefixes). + Destination IPv6 addresses are incremented on a packet by packet + basis. + +- *ip6scale*: high number of IPv6 flows is routed by VPP. Tested IPv6 + FIB sizes of /128 prefixes include: i) 20k (10k unique flows per + direction), ii) 200k (2x 100k flows) and iii) 2M (2x 1M). Destination + IPv6 addresses are incremented on a packet by packet basis, ensuring + new FIB entries are looked up at every packet, making it the worst + case scenario. + +SRv6 Routing +~~~~~~~~~~~~ + +SRv6 routing tests are executed in a number of baseline configurations, +in each case SR policy and steering policy are configured for one +direction and one (or two) SR behaviours (functions) in the other +directions: + +- *srv6enc1sid*: One SID (no SRH present), one SR function - End. +- *srv6enc2sids*: Two SIDs (SRH present), two SR functions - End and + End.DX6. +- *srv6enc2sids-nodecaps*: Two SIDs (SRH present) without decapsulation, + one SR function - End. +- *srv6proxy-dyn*: Dynamic SRv6 proxy, one SR function - End.AD. +- *srv6proxy-masq*: Masquerading SRv6 proxy, one SR function - End.AM. +- *srv6proxy-stat*: Static SRv6 proxy, one SR function - End.AS. + +In all listed cases low number of IPv6 flows (253 per direction) is +routed by VPP. diff --git a/docs/report/introduction/methodology_vpp_startup_settings.rst b/docs/report/introduction/methodology_vpp_startup_settings.rst new file mode 100644 index 0000000000..16185b4c05 --- /dev/null +++ b/docs/report/introduction/methodology_vpp_startup_settings.rst @@ -0,0 +1,43 @@ +VPP Startup Settings +-------------------- + +CSIT code manipulates a number of VPP settings in startup.conf for optimized +performance. List of common settings applied to all tests and test +dependent settings follows. + +See `VPP startup.conf`_ +for a complete set and description of listed settings. + +Common Settings +~~~~~~~~~~~~~~~ + +List of vpp startup.conf settings applied to all tests: + +#. heap-size <value> - set separately for ip4, ip6, stats, main + depending on scale tested. +#. no-tx-checksum-offload - disables UDP / TCP TX checksum offload in DPDK. + Typically needed for use faster vector PMDs (together with + no-multi-seg). +#. socket-mem <value>,<value> - memory per numa. (Not required anymore + due to VPP code changes, should be removed in CSIT-18.10.) + +Per Test Settings +~~~~~~~~~~~~~~~~~ + +List of vpp startup.conf settings applied dynamically per test: + +#. corelist-workers <list_of_cores> - list of logical cores to run VPP + worker data plane threads. Depends on HyperThreading and core per + test configuration. +#. num-rx-queues <value> - depends on a number of VPP threads and NIC + interfaces. +#. num-rx-desc/num-tx-desc - number of rx/tx descriptors for specific + NICs, incl. xl710, x710, xxv710. +#. num-mbufs <value> - increases number of buffers allocated, needed + only in scenarios with large number of interfaces and worker threads. + Value is per CPU socket. Default is 16384. +#. no-multi-seg - disables multi-segment buffers in DPDK, improves + packet throughput, but disables Jumbo MTU support. Disabled for all + tests apart from the ones that require Jumbo 9000B frame support. +#. UIO driver - depends on topology file definition. +#. QAT VFs - depends on NRThreads, each thread = 1QAT VFs. diff --git a/docs/report/vpp_device_tests/vpp_device.svg b/docs/report/vpp_device_tests/vpp_device.svg new file mode 100644 index 0000000000..177d49af89 --- /dev/null +++ b/docs/report/vpp_device_tests/vpp_device.svg @@ -0,0 +1,318 @@ +<?xml version="1.0" encoding="UTF-8"?> +<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"> +<svg version="1.2" width="163mm" height="127mm" viewBox="0 0 16300 12700" preserveAspectRatio="xMidYMid" fill-rule="evenodd" stroke-width="28.222" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" xmlns:ooo="http://xml.openoffice.org/svg/export" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:presentation="http://sun.com/xmlns/staroffice/presentation" xmlns:smil="http://www.w3.org/2001/SMIL20/" xmlns:anim="urn:oasis:names:tc:opendocument:xmlns:animation:1.0" xml:space="preserve"> + <defs class="ClipPathGroup"> + <clipPath id="presentation_clip_path" clipPathUnits="userSpaceOnUse"> + <rect x="0" y="0" width="16300" height="12700"/> + </clipPath> + <clipPath id="presentation_clip_path_shrink" clipPathUnits="userSpaceOnUse"> + <rect x="16" y="12" width="16268" height="12675"/> + </clipPath> + </defs> + <defs> + <font id="EmbeddedFont_1" horiz-adv-x="2048"> + <font-face font-family="Liberation Sans embedded" units-per-em="2048" font-weight="normal" font-style="normal" ascent="1852" descent="423"/> + <missing-glyph horiz-adv-x="2048" d="M 0,0 L 2047,0 2047,2047 0,2047 0,0 Z"/> + <glyph unicode="x" horiz-adv-x="1006" d="M 801,0 L 510,444 217,0 23,0 408,556 41,1082 240,1082 510,661 778,1082 979,1082 612,558 1002,0 801,0 Z"/> + <glyph unicode="w" horiz-adv-x="1509" d="M 1174,0 L 965,0 776,765 740,934 C 734,904 725,861 712,805 699,748 631,480 508,0 L 300,0 -3,1082 175,1082 358,347 C 363,331 377,265 401,149 L 418,223 644,1082 837,1082 1026,339 1072,149 1103,288 1308,1082 1484,1082 1174,0 Z"/> + <glyph unicode="u" horiz-adv-x="874" d="M 314,1082 L 314,396 C 314,325 321,269 335,230 349,191 371,162 402,145 433,128 478,119 537,119 624,119 692,149 742,208 792,267 817,350 817,455 L 817,1082 997,1082 997,231 C 997,105 999,28 1003,0 L 833,0 C 832,3 832,12 831,27 830,42 830,59 829,78 828,97 826,132 825,185 L 822,185 C 781,110 733,58 679,27 624,-4 557,-20 476,-20 357,-20 271,10 216,69 161,128 133,225 133,361 L 133,1082 314,1082 Z"/> + <glyph unicode="t" horiz-adv-x="531" d="M 554,8 C 495,-8 434,-16 372,-16 228,-16 156,66 156,229 L 156,951 31,951 31,1082 163,1082 216,1324 336,1324 336,1082 536,1082 536,951 336,951 336,268 C 336,216 345,180 362,159 379,138 408,127 450,127 474,127 509,132 554,141 L 554,8 Z"/> + <glyph unicode="r" horiz-adv-x="530" d="M 142,0 L 142,830 C 142,906 140,990 136,1082 L 306,1082 C 311,959 314,886 314,861 L 318,861 C 347,954 380,1017 417,1051 454,1085 507,1102 575,1102 599,1102 623,1099 648,1092 L 648,927 C 624,934 592,937 552,937 477,937 420,905 381,841 342,776 322,684 322,564 L 322,0 142,0 Z"/> + <glyph unicode="q" horiz-adv-x="927" d="M 484,-20 C 347,-20 246,26 182,119 118,212 86,351 86,536 86,913 219,1102 484,1102 566,1102 634,1088 687,1059 740,1030 785,981 821,914 L 823,914 C 823,934 824,969 827,1018 830,1067 832,1093 835,1096 L 1008,1096 C 1003,1057 1001,958 1001,801 L 1001,-425 821,-425 821,14 825,178 823,178 C 787,107 743,56 690,26 637,-5 569,-20 484,-20 Z M 821,554 C 821,695 798,799 752,867 706,935 633,969 532,969 441,969 375,935 335,867 295,799 275,691 275,542 275,391 295,282 336,217 376,152 441,119 530,119 632,119 706,155 752,228 798,301 821,409 821,554 Z"/> + <glyph unicode="p" horiz-adv-x="953" d="M 1053,546 C 1053,169 920,-20 655,-20 488,-20 376,43 319,168 L 314,168 C 317,163 318,106 318,-2 L 318,-425 138,-425 138,861 C 138,972 136,1046 132,1082 L 306,1082 C 307,1079 308,1070 309,1054 310,1037 312,1012 314,978 315,944 316,921 316,908 L 320,908 C 352,975 394,1024 447,1055 500,1086 569,1101 655,1101 788,1101 888,1056 954,967 1020,878 1053,737 1053,546 Z M 864,542 C 864,693 844,800 803,865 762,930 698,962 609,962 538,962 482,947 442,917 401,887 371,840 350,777 329,713 318,630 318,528 318,386 341,281 386,214 431,147 505,113 607,113 696,113 762,146 803,212 844,277 864,387 864,542 Z"/> + <glyph unicode="o" horiz-adv-x="980" d="M 1053,542 C 1053,353 1011,212 928,119 845,26 724,-20 565,-20 407,-20 288,28 207,125 126,221 86,360 86,542 86,915 248,1102 571,1102 736,1102 858,1057 936,966 1014,875 1053,733 1053,542 Z M 864,542 C 864,691 842,800 798,868 753,935 679,969 574,969 469,969 393,935 346,866 299,797 275,689 275,542 275,399 298,292 345,221 391,149 464,113 563,113 671,113 748,148 795,217 841,286 864,395 864,542 Z"/> + <glyph unicode="n" horiz-adv-x="874" d="M 825,0 L 825,686 C 825,757 818,813 804,852 790,891 768,920 737,937 706,954 661,963 602,963 515,963 447,933 397,874 347,815 322,732 322,627 L 322,0 142,0 142,851 C 142,977 140,1054 136,1082 L 306,1082 C 307,1079 307,1070 308,1055 309,1040 310,1024 311,1005 312,986 313,950 314,897 L 317,897 C 358,972 406,1025 461,1056 515,1087 582,1102 663,1102 782,1102 869,1073 924,1014 979,955 1006,857 1006,721 L 1006,0 825,0 Z"/> + <glyph unicode="m" horiz-adv-x="1457" d="M 768,0 L 768,686 C 768,791 754,863 725,903 696,943 645,963 570,963 493,963 433,934 388,875 343,816 321,734 321,627 L 321,0 142,0 142,851 C 142,977 140,1054 136,1082 L 306,1082 C 307,1079 307,1070 308,1055 309,1040 310,1024 311,1005 312,986 313,950 314,897 L 317,897 C 356,974 400,1027 450,1057 500,1087 561,1102 633,1102 715,1102 780,1086 828,1053 875,1020 908,968 927,897 L 930,897 C 967,970 1013,1022 1066,1054 1119,1086 1183,1102 1258,1102 1367,1102 1447,1072 1497,1013 1546,954 1571,856 1571,721 L 1571,0 1393,0 1393,686 C 1393,791 1379,863 1350,903 1321,943 1270,963 1195,963 1116,963 1055,934 1012,876 968,817 946,734 946,627 L 946,0 768,0 Z"/> + <glyph unicode="k" horiz-adv-x="901" d="M 816,0 L 450,494 318,385 318,0 138,0 138,1484 318,1484 318,557 793,1082 1004,1082 565,617 1027,0 816,0 Z"/> + <glyph unicode="i" horiz-adv-x="187" d="M 137,1312 L 137,1484 317,1484 317,1312 137,1312 Z M 137,0 L 137,1082 317,1082 317,0 137,0 Z"/> + <glyph unicode="g" horiz-adv-x="927" d="M 548,-425 C 430,-425 336,-402 266,-356 196,-309 151,-243 131,-158 L 312,-132 C 324,-182 351,-220 392,-248 433,-274 486,-288 553,-288 732,-288 822,-183 822,27 L 822,201 820,201 C 786,132 739,80 680,45 621,10 551,-8 472,-8 339,-8 242,36 180,124 117,212 86,350 86,539 86,730 120,872 187,963 254,1054 355,1099 492,1099 569,1099 635,1082 692,1047 748,1012 791,962 822,897 L 824,897 C 824,917 825,952 828,1001 831,1050 833,1077 836,1082 L 1007,1082 C 1003,1046 1001,971 1001,858 L 1001,31 C 1001,-273 850,-425 548,-425 Z M 822,541 C 822,629 810,705 786,769 762,832 728,881 685,915 641,948 591,965 536,965 444,965 377,932 335,865 293,798 272,690 272,541 272,393 292,287 331,222 370,157 438,125 533,125 590,125 640,142 684,175 728,208 762,256 786,319 810,381 822,455 822,541 Z"/> + <glyph unicode="e" horiz-adv-x="980" d="M 276,503 C 276,379 302,283 353,216 404,149 479,115 578,115 656,115 719,131 766,162 813,193 844,233 861,281 L 1019,236 C 954,65 807,-20 578,-20 418,-20 296,28 213,123 129,218 87,360 87,548 87,727 129,864 213,959 296,1054 416,1102 571,1102 889,1102 1048,910 1048,527 L 1048,503 276,503 Z M 862,641 C 852,755 823,838 775,891 727,943 658,969 568,969 481,969 412,940 361,882 310,823 282,743 278,641 L 862,641 Z"/> + <glyph unicode="d" horiz-adv-x="927" d="M 821,174 C 788,105 744,55 689,25 634,-5 565,-20 484,-20 347,-20 247,26 183,118 118,210 86,349 86,536 86,913 219,1102 484,1102 566,1102 634,1087 689,1057 744,1027 788,979 821,914 L 823,914 821,1035 821,1484 1001,1484 1001,223 C 1001,110 1003,36 1007,0 L 835,0 C 833,11 831,35 829,74 826,113 825,146 825,174 L 821,174 Z M 275,542 C 275,391 295,282 335,217 375,152 440,119 530,119 632,119 706,154 752,225 798,296 821,405 821,554 821,697 798,802 752,869 706,936 633,969 532,969 441,969 376,936 336,869 295,802 275,693 275,542 Z"/> + <glyph unicode="c" horiz-adv-x="901" d="M 275,546 C 275,402 298,295 343,226 388,157 457,122 548,122 612,122 666,139 709,174 752,209 778,262 788,334 L 970,322 C 956,218 912,135 837,73 762,11 668,-20 553,-20 402,-20 286,28 207,124 127,219 87,359 87,542 87,724 127,863 207,959 287,1054 402,1102 551,1102 662,1102 754,1073 827,1016 900,959 945,880 964,779 L 779,765 C 770,825 746,873 708,908 670,943 616,961 546,961 451,961 382,929 339,866 296,803 275,696 275,546 Z"/> + <glyph unicode="b" horiz-adv-x="953" d="M 1053,546 C 1053,169 920,-20 655,-20 573,-20 505,-5 451,25 396,54 352,102 318,168 L 316,168 C 316,147 315,116 312,74 309,31 307,7 306,0 L 132,0 C 136,36 138,110 138,223 L 138,1484 318,1484 318,1061 C 318,1018 317,967 314,908 L 318,908 C 351,977 396,1027 451,1057 506,1087 574,1102 655,1102 792,1102 892,1056 957,964 1021,872 1053,733 1053,546 Z M 864,540 C 864,691 844,800 804,865 764,930 699,963 609,963 508,963 434,928 388,859 341,790 318,680 318,529 318,387 341,282 386,215 431,147 505,113 607,113 698,113 763,147 804,214 844,281 864,389 864,540 Z"/> + <glyph unicode="a" horiz-adv-x="1060" d="M 414,-20 C 305,-20 224,9 169,66 114,123 87,202 87,302 87,414 124,500 198,560 271,620 390,652 554,656 L 797,660 797,719 C 797,807 778,870 741,908 704,946 645,965 565,965 484,965 426,951 389,924 352,897 330,853 323,793 L 135,810 C 166,1005 310,1102 569,1102 705,1102 807,1071 876,1009 945,946 979,856 979,738 L 979,272 C 979,219 986,179 1000,152 1014,125 1041,111 1080,111 1097,111 1117,113 1139,118 L 1139,6 C 1094,-5 1047,-10 1000,-10 933,-10 885,8 855,43 824,78 807,132 803,207 L 797,207 C 751,124 698,66 637,32 576,-3 501,-20 414,-20 Z M 455,115 C 521,115 580,130 631,160 682,190 723,231 753,284 782,336 797,390 797,445 L 797,534 600,530 C 515,529 451,520 408,504 364,488 330,463 307,430 284,397 272,353 272,299 272,240 288,195 320,163 351,131 396,115 455,115 Z"/> + <glyph unicode="U" horiz-adv-x="1192" d="M 731,-20 C 616,-20 515,1 429,43 343,85 276,146 229,226 182,306 158,401 158,512 L 158,1409 349,1409 349,528 C 349,399 382,302 447,235 512,168 607,135 730,135 857,135 955,170 1026,239 1096,308 1131,408 1131,541 L 1131,1409 1321,1409 1321,530 C 1321,416 1297,318 1249,235 1200,152 1132,89 1044,46 955,2 851,-20 731,-20 Z"/> + <glyph unicode="S" horiz-adv-x="1192" d="M 1272,389 C 1272,259 1221,158 1120,87 1018,16 875,-20 690,-20 347,-20 148,99 93,338 L 278,375 C 299,290 345,228 414,189 483,149 578,129 697,129 820,129 916,150 983,193 1050,235 1083,297 1083,379 1083,425 1073,462 1052,491 1031,520 1001,543 963,562 925,581 880,596 827,609 774,622 716,635 652,650 541,675 456,699 399,724 341,749 295,776 262,807 229,837 203,872 186,913 168,954 159,1000 159,1053 159,1174 205,1267 298,1332 390,1397 522,1430 694,1430 854,1430 976,1406 1061,1357 1146,1308 1205,1224 1239,1106 L 1051,1073 C 1030,1148 991,1202 933,1236 875,1269 795,1286 692,1286 579,1286 493,1267 434,1230 375,1193 345,1137 345,1063 345,1020 357,984 380,956 403,927 436,903 479,884 522,864 609,840 738,811 781,801 825,791 868,781 911,770 952,758 991,744 1030,729 1067,712 1102,693 1136,674 1166,650 1191,622 1216,594 1236,561 1251,523 1265,485 1272,440 1272,389 Z"/> + <glyph unicode="N" horiz-adv-x="1165" d="M 1082,0 L 328,1200 333,1103 338,936 338,0 168,0 168,1409 390,1409 1152,201 C 1144,332 1140,426 1140,485 L 1140,1409 1312,1409 1312,0 1082,0 Z"/> + <glyph unicode="H" horiz-adv-x="1165" d="M 1121,0 L 1121,653 359,653 359,0 168,0 168,1409 359,1409 359,813 1121,813 1121,1409 1312,1409 1312,0 1121,0 Z"/> + <glyph unicode="1" horiz-adv-x="927" d="M 156,0 L 156,153 515,153 515,1237 197,1010 197,1180 530,1409 696,1409 696,153 1039,153 1039,0 156,0 Z"/> + <glyph unicode=" " horiz-adv-x="556"/> + </font> + </defs> + <defs> + <font id="EmbeddedFont_2" horiz-adv-x="2048"> + <font-face font-family="Liberation Sans embedded" units-per-em="2048" font-weight="bold" font-style="normal" ascent="1852" descent="423"/> + <missing-glyph horiz-adv-x="2048" d="M 0,0 L 2047,0 2047,2047 0,2047 0,0 Z"/> + <glyph unicode="y" horiz-adv-x="1139" d="M 283,-425 C 216,-425 157,-421 106,-412 L 106,-212 C 141,-217 174,-220 203,-220 243,-220 276,-214 303,-201 329,-188 353,-167 374,-138 395,-109 418,-59 444,11 L 16,1082 313,1082 483,575 C 510,502 543,391 584,241 L 609,336 674,571 834,1082 1128,1082 700,-57 C 643,-196 583,-292 522,-345 460,-398 380,-425 283,-425 Z"/> + <glyph unicode="v" horiz-adv-x="1139" d="M 731,0 L 395,0 8,1082 305,1082 494,477 C 504,444 528,360 565,227 572,254 585,302 606,371 627,440 703,677 836,1082 L 1130,1082 731,0 Z"/> + <glyph unicode="t" horiz-adv-x="662" d="M 420,-18 C 337,-18 274,5 229,50 184,95 162,163 162,254 L 162,892 25,892 25,1082 176,1082 264,1336 440,1336 440,1082 645,1082 645,892 440,892 440,330 C 440,277 450,239 470,214 490,189 521,176 563,176 585,176 616,181 657,190 L 657,16 C 588,-7 509,-18 420,-18 Z"/> + <glyph unicode="s" horiz-adv-x="1006" d="M 1055,316 C 1055,211 1012,129 927,70 841,10 722,-20 571,-20 422,-20 309,4 230,51 151,98 98,171 72,270 L 319,307 C 333,256 357,219 392,198 426,177 486,166 571,166 650,166 707,176 743,196 779,216 797,247 797,290 797,325 783,352 754,373 725,393 675,410 606,424 447,455 340,485 285,512 230,539 188,574 159,617 130,660 115,712 115,775 115,878 155,959 235,1017 314,1074 427,1103 573,1103 702,1103 805,1078 884,1028 962,978 1011,906 1030,811 L 781,785 C 773,829 753,862 722,884 691,905 641,916 573,916 506,916 456,908 423,891 390,874 373,845 373,805 373,774 386,749 412,731 437,712 480,697 541,685 626,668 701,650 767,632 832,613 885,591 925,566 964,541 996,508 1020,469 1043,429 1055,378 1055,316 Z"/> + <glyph unicode="r" horiz-adv-x="636" d="M 143,0 L 143,828 C 143,887 142,937 141,977 139,1016 137,1051 135,1082 L 403,1082 C 405,1070 408,1034 411,973 414,912 416,871 416,851 L 420,851 C 447,927 472,981 493,1012 514,1043 540,1066 569,1081 598,1096 635,1103 679,1103 715,1103 744,1098 766,1088 L 766,853 C 721,863 681,868 646,868 576,868 522,840 483,783 444,726 424,642 424,531 L 424,0 143,0 Z"/> + <glyph unicode="o" horiz-adv-x="1113" d="M 1171,542 C 1171,367 1122,229 1025,130 928,30 793,-20 621,-20 452,-20 320,30 224,130 128,230 80,367 80,542 80,716 128,853 224,953 320,1052 454,1102 627,1102 804,1102 939,1054 1032,958 1125,861 1171,723 1171,542 Z M 877,542 C 877,671 856,764 814,822 772,880 711,909 631,909 460,909 375,787 375,542 375,421 396,330 438,267 479,204 539,172 618,172 791,172 877,295 877,542 Z"/> + <glyph unicode="n" horiz-adv-x="1007" d="M 844,0 L 844,607 C 844,797 780,892 651,892 583,892 528,863 487,805 445,746 424,671 424,580 L 424,0 143,0 143,840 C 143,898 142,946 141,983 139,1020 137,1053 135,1082 L 403,1082 C 405,1069 408,1036 411,981 414,926 416,888 416,867 L 420,867 C 458,950 506,1010 563,1047 620,1084 689,1103 768,1103 883,1103 971,1068 1032,997 1093,926 1124,823 1124,687 L 1124,0 844,0 Z"/> + <glyph unicode="m" horiz-adv-x="1562" d="M 780,0 L 780,607 C 780,797 725,892 616,892 559,892 513,863 478,805 442,747 424,672 424,580 L 424,0 143,0 143,840 C 143,898 142,946 141,983 139,1020 137,1053 135,1082 L 403,1082 C 405,1069 408,1036 411,981 414,926 416,888 416,867 L 420,867 C 455,950 498,1010 550,1047 601,1084 663,1103 735,1103 900,1103 1001,1024 1036,867 L 1042,867 C 1079,951 1123,1011 1174,1048 1225,1085 1291,1103 1370,1103 1475,1103 1556,1067 1611,996 1666,924 1694,821 1694,687 L 1694,0 1415,0 1415,607 C 1415,797 1360,892 1251,892 1196,892 1152,866 1117,813 1082,760 1062,686 1059,593 L 1059,0 780,0 Z"/> + <glyph unicode="l" horiz-adv-x="292" d="M 143,0 L 143,1484 424,1484 424,0 143,0 Z"/> + <glyph unicode="k" horiz-adv-x="1007" d="M 834,0 L 545,490 424,406 424,0 143,0 143,1484 424,1484 424,634 810,1082 1112,1082 732,660 1141,0 834,0 Z"/> + <glyph unicode="i" horiz-adv-x="292" d="M 143,1277 L 143,1484 424,1484 424,1277 143,1277 Z M 143,0 L 143,1082 424,1082 424,0 143,0 Z"/> + <glyph unicode="h" horiz-adv-x="1007" d="M 420,866 C 458,949 506,1009 563,1046 620,1083 689,1102 768,1102 883,1102 971,1067 1032,996 1093,925 1124,822 1124,686 L 1124,0 844,0 844,606 C 844,796 780,891 651,891 583,891 528,862 487,804 445,745 424,670 424,579 L 424,0 143,0 143,1484 424,1484 424,1079 C 424,1006 421,935 416,866 L 420,866 Z"/> + <glyph unicode="e" horiz-adv-x="1007" d="M 586,-20 C 423,-20 298,28 211,125 124,221 80,361 80,546 80,725 124,862 213,958 302,1054 427,1102 590,1102 745,1102 864,1051 946,948 1028,845 1069,694 1069,495 L 1069,487 375,487 C 375,382 395,302 434,249 473,195 528,168 600,168 699,168 762,211 788,297 L 1053,274 C 976,78 821,-20 586,-20 Z M 586,925 C 520,925 469,902 434,856 398,810 379,746 377,663 L 797,663 C 792,750 771,816 734,860 697,903 648,925 586,925 Z"/> + <glyph unicode="d" horiz-adv-x="1033" d="M 844,0 C 841,10 838,35 835,76 831,116 829,149 829,176 L 825,176 C 764,45 649,-20 479,-20 353,-20 256,29 187,128 118,226 84,363 84,540 84,719 120,858 193,956 265,1053 367,1102 500,1102 577,1102 643,1086 699,1054 754,1022 797,974 827,911 L 829,911 827,1089 827,1484 1108,1484 1108,236 C 1108,169 1111,91 1116,0 L 844,0 Z M 831,547 C 831,664 812,754 773,817 734,880 676,911 600,911 525,911 469,881 432,820 395,759 377,665 377,540 377,295 451,172 598,172 672,172 729,205 770,270 811,335 831,427 831,547 Z"/> + <glyph unicode="a" horiz-adv-x="1112" d="M 393,-20 C 288,-20 207,9 148,66 89,123 60,203 60,306 60,418 97,503 170,562 243,621 348,651 487,652 L 720,656 720,711 C 720,782 708,834 683,869 658,903 618,920 562,920 510,920 472,908 448,885 423,861 408,822 402,767 L 109,781 C 127,886 175,966 254,1021 332,1075 439,1102 574,1102 711,1102 816,1068 890,1001 964,934 1001,838 1001,714 L 1001,320 C 1001,259 1008,218 1022,195 1035,172 1058,160 1090,160 1111,160 1132,162 1152,166 L 1152,14 C 1135,10 1120,6 1107,3 1094,0 1080,-3 1067,-5 1054,-7 1040,-9 1025,-10 1010,-11 992,-12 972,-12 901,-12 849,5 816,40 782,75 762,126 755,193 L 749,193 C 670,51 552,-20 393,-20 Z M 720,501 L 576,499 C 511,496 464,489 437,478 410,466 389,448 375,424 360,400 353,368 353,328 353,277 365,239 389,214 412,189 444,176 483,176 527,176 567,188 604,212 640,236 668,269 689,312 710,354 720,399 720,446 L 720,501 Z"/> + <glyph unicode="U" horiz-adv-x="1244" d="M 723,-20 C 529,-20 381,27 278,122 175,217 123,352 123,528 L 123,1409 418,1409 418,551 C 418,440 445,355 498,298 551,240 628,211 731,211 836,211 917,241 974,302 1031,362 1059,448 1059,561 L 1059,1409 1354,1409 1354,543 C 1354,364 1299,226 1189,128 1078,29 923,-20 723,-20 Z"/> + <glyph unicode="T" horiz-adv-x="1245" d="M 773,1181 L 773,0 478,0 478,1181 23,1181 23,1409 1229,1409 1229,1181 773,1181 Z"/> + <glyph unicode="S" horiz-adv-x="1244" d="M 1286,406 C 1286,268 1235,163 1133,90 1030,17 880,-20 682,-20 501,-20 360,12 257,76 154,140 88,237 59,367 L 344,414 C 363,339 401,285 457,252 513,218 591,201 690,201 896,201 999,264 999,389 999,429 987,462 964,488 940,514 907,536 864,553 821,570 738,591 616,616 511,641 437,661 396,676 355,691 317,708 284,729 251,749 222,773 199,802 176,831 158,864 145,903 132,942 125,986 125,1036 125,1163 173,1261 269,1329 364,1396 503,1430 686,1430 861,1430 992,1403 1080,1348 1167,1293 1224,1203 1249,1077 L 963,1038 C 948,1099 919,1144 874,1175 829,1206 764,1221 680,1221 501,1221 412,1165 412,1053 412,1016 422,986 441,963 460,940 488,920 525,904 562,887 638,867 752,842 887,813 984,787 1043,763 1101,738 1147,710 1181,678 1215,645 1241,607 1259,562 1277,517 1286,465 1286,406 Z"/> + <glyph unicode="N" horiz-adv-x="1218" d="M 995,0 L 381,1085 C 393,980 399,895 399,831 L 399,0 137,0 137,1409 474,1409 1097,315 C 1085,416 1079,507 1079,590 L 1079,1409 1341,1409 1341,0 995,0 Z"/> + <glyph unicode="J" horiz-adv-x="980" d="M 524,-20 C 378,-20 266,12 188,75 109,138 57,241 31,382 L 324,425 C 336,352 358,299 391,264 424,229 469,211 526,211 585,211 629,231 660,270 690,309 705,366 705,439 L 705,1178 424,1178 424,1409 999,1409 999,446 C 999,299 957,185 874,103 791,21 674,-20 524,-20 Z"/> + <glyph unicode="I" horiz-adv-x="319" d="M 137,0 L 137,1409 432,1409 432,0 137,0 Z"/> + <glyph unicode="G" horiz-adv-x="1404" d="M 806,211 C 883,211 957,222 1029,245 1101,267 1157,295 1196,330 L 1196,525 852,525 852,743 1466,743 1466,225 C 1391,148 1294,88 1175,45 1055,2 929,-20 798,-20 569,-20 392,44 269,171 146,298 84,478 84,711 84,943 146,1121 270,1245 394,1368 572,1430 805,1430 1136,1430 1346,1308 1436,1063 L 1164,981 C 1135,1052 1089,1106 1026,1143 963,1180 890,1198 805,1198 666,1198 561,1156 489,1072 417,988 381,868 381,711 381,552 418,429 493,342 567,255 671,211 806,211 Z"/> + <glyph unicode="C" horiz-adv-x="1351" d="M 795,212 C 973,212 1097,301 1166,480 L 1423,383 C 1368,247 1287,146 1180,80 1073,13 944,-20 795,-20 568,-20 393,44 270,173 146,301 84,480 84,711 84,942 144,1120 263,1244 382,1368 555,1430 782,1430 947,1430 1082,1397 1186,1331 1290,1264 1363,1167 1405,1038 L 1145,967 C 1123,1038 1080,1094 1016,1136 951,1177 875,1198 788,1198 655,1198 554,1157 485,1074 416,991 381,870 381,711 381,549 417,425 488,340 559,255 661,212 795,212 Z"/> + <glyph unicode="1" horiz-adv-x="980" d="M 129,0 L 129,209 478,209 478,1170 140,959 140,1180 493,1409 759,1409 759,209 1082,209 1082,0 129,0 Z"/> + <glyph unicode=" " horiz-adv-x="556"/> + </font> + </defs> + <defs class="TextShapeIndex"> + <g ooo:slide="id1" ooo:id-list="id3 id4 id5 id6 id7 id8 id9 id10 id11 id12 id13 id14 id15 id16 id17 id18 id19 id20 id21 id22 id23 id24 id25 id26"/> + </defs> + <defs class="EmbeddedBulletChars"> + <g id="bullet-char-template-57356" transform="scale(0.00048828125,-0.00048828125)"> + <path d="M 580,1141 L 1163,571 580,0 -4,571 580,1141 Z"/> + </g> + <g id="bullet-char-template-57354" transform="scale(0.00048828125,-0.00048828125)"> + <path d="M 8,1128 L 1137,1128 1137,0 8,0 8,1128 Z"/> + </g> + <g id="bullet-char-template-10146" transform="scale(0.00048828125,-0.00048828125)"> + <path d="M 174,0 L 602,739 174,1481 1456,739 174,0 Z M 1358,739 L 309,1346 659,739 1358,739 Z"/> + </g> + <g id="bullet-char-template-10132" transform="scale(0.00048828125,-0.00048828125)"> + <path d="M 2015,739 L 1276,0 717,0 1260,543 174,543 174,936 1260,936 717,1481 1274,1481 2015,739 Z"/> + </g> + <g id="bullet-char-template-10007" transform="scale(0.00048828125,-0.00048828125)"> + <path d="M 0,-2 C -7,14 -16,27 -25,37 L 356,567 C 262,823 215,952 215,954 215,979 228,992 255,992 264,992 276,990 289,987 310,991 331,999 354,1012 L 381,999 492,748 772,1049 836,1024 860,1049 C 881,1039 901,1025 922,1006 886,937 835,863 770,784 769,783 710,716 594,584 L 774,223 C 774,196 753,168 711,139 L 727,119 C 717,90 699,76 672,76 641,76 570,178 457,381 L 164,-76 C 142,-110 111,-127 72,-127 30,-127 9,-110 8,-76 1,-67 -2,-52 -2,-32 -2,-23 -1,-13 0,-2 Z"/> + </g> + <g id="bullet-char-template-10004" transform="scale(0.00048828125,-0.00048828125)"> + <path d="M 285,-33 C 182,-33 111,30 74,156 52,228 41,333 41,471 41,549 55,616 82,672 116,743 169,778 240,778 293,778 328,747 346,684 L 369,508 C 377,444 397,411 428,410 L 1163,1116 C 1174,1127 1196,1133 1229,1133 1271,1133 1292,1118 1292,1087 L 1292,965 C 1292,929 1282,901 1262,881 L 442,47 C 390,-6 338,-33 285,-33 Z"/> + </g> + <g id="bullet-char-template-9679" transform="scale(0.00048828125,-0.00048828125)"> + <path d="M 813,0 C 632,0 489,54 383,161 276,268 223,411 223,592 223,773 276,916 383,1023 489,1130 632,1184 813,1184 992,1184 1136,1130 1245,1023 1353,916 1407,772 1407,592 1407,412 1353,268 1245,161 1136,54 992,0 813,0 Z"/> + </g> + <g id="bullet-char-template-8226" transform="scale(0.00048828125,-0.00048828125)"> + <path d="M 346,457 C 273,457 209,483 155,535 101,586 74,649 74,723 74,796 101,859 155,911 209,963 273,989 346,989 419,989 480,963 531,910 582,859 608,796 608,723 608,648 583,586 532,535 482,483 420,457 346,457 Z"/> + </g> + <g id="bullet-char-template-8211" transform="scale(0.00048828125,-0.00048828125)"> + <path d="M -4,459 L 1135,459 1135,606 -4,606 -4,459 Z"/> + </g> + <g id="bullet-char-template-61548" transform="scale(0.00048828125,-0.00048828125)"> + <path d="M 173,740 C 173,903 231,1043 346,1159 462,1274 601,1332 765,1332 928,1332 1067,1274 1183,1159 1299,1043 1357,903 1357,740 1357,577 1299,437 1183,322 1067,206 928,148 765,148 601,148 462,206 346,322 231,437 173,577 173,740 Z"/> + </g> + </defs> + <defs class="TextEmbeddedBitmaps"/> + <g> + <g id="id2" class="Master_Slide"> + <g id="bg-id2" class="Background"/> + <g id="bo-id2" class="BackgroundObjects"/> + </g> + </g> + <g class="SlideGroup"> + <g> + <g id="container-id1"> + <g id="id1" class="Slide" clip-path="url(#presentation_clip_path)"> + <g class="Page"> + <g class="Graphic"> + <g id="id3"> + <rect class="BoundingBox" stroke="none" fill="none" x="9188" y="4640" width="2066" height="3618"/> + <path fill="rgb(62,62,63)" stroke="none" d="M 10221,8249 L 10994,8249 C 11134,8249 11247,8131 11247,7985 L 11247,4913 C 11247,4767 11134,4650 10994,4650 L 9447,4650 C 9307,4650 9194,4767 9194,4913 L 9194,7985 C 9194,8131 9307,8249 9447,8249 L 10221,8249 Z"/> + <path fill="rgb(127,128,130)" stroke="none" d="M 10221,7596 L 9194,7596 9194,7353 11247,7353 11247,7596 10221,7596 Z"/> + <path fill="rgb(127,128,130)" stroke="none" d="M 10221,7044 L 9194,7044 9194,6802 11247,6802 11247,7044 10221,7044 Z"/> + <path fill="rgb(127,128,130)" stroke="none" d="M 10221,6492 L 9194,6492 9194,6250 11247,6250 11247,6492 10221,6492 Z"/> + <path fill="rgb(127,128,130)" stroke="none" d="M 10221,5941 L 9194,5941 9194,5698 11247,5698 11247,5941 10221,5941 Z"/> + <path fill="rgb(250,203,27)" stroke="none" d="M 9801,5177 C 9801,5212 9793,5242 9777,5272 9761,5303 9740,5325 9712,5342 9683,5360 9656,5368 9623,5368 9590,5368 9562,5360 9534,5342 9505,5325 9485,5303 9469,5272 9452,5242 9445,5212 9445,5177 9445,5141 9452,5111 9469,5081 9485,5050 9505,5029 9534,5011 9562,4993 9590,4985 9623,4985 9656,4985 9683,4993 9712,5011 9740,5029 9761,5050 9777,5081 9793,5111 9801,5141 9801,5177 L 9801,5177 Z"/> + <path fill="rgb(249,145,52)" stroke="none" d="M 10391,5177 C 10391,5212 10384,5242 10368,5272 10351,5303 10331,5325 10302,5342 10274,5360 10246,5368 10213,5368 10181,5368 10153,5360 10124,5342 10096,5325 10075,5303 10059,5272 10043,5242 10035,5212 10035,5177 10035,5141 10043,5111 10059,5081 10075,5050 10096,5029 10124,5011 10153,4993 10181,4985 10213,4985 10246,4985 10274,4993 10302,5011 10331,5029 10351,5050 10368,5081 10384,5111 10391,5141 10391,5177 L 10391,5177 Z"/> + <path fill="rgb(250,85,85)" stroke="none" d="M 10982,5177 C 10982,5212 10974,5242 10958,5272 10942,5303 10921,5325 10893,5342 10864,5360 10837,5368 10804,5368 10771,5368 10743,5360 10715,5342 10686,5325 10666,5303 10650,5272 10633,5242 10626,5212 10626,5177 10626,5141 10633,5111 10650,5081 10666,5050 10686,5029 10715,5011 10743,4993 10771,4985 10804,4985 10837,4985 10864,4993 10893,5011 10921,5029 10942,5050 10958,5081 10974,5111 10982,5141 10982,5177 L 10982,5177 Z"/> + </g> + </g> + <g class="Graphic"> + <g id="id4"> + <rect class="BoundingBox" stroke="none" fill="none" x="514" y="506" width="2066" height="3618"/> + <path fill="rgb(62,62,63)" stroke="none" d="M 1547,4115 L 2320,4115 C 2460,4115 2573,3997 2573,3851 L 2573,779 C 2573,633 2460,516 2320,516 L 773,516 C 633,516 520,633 520,779 L 520,3851 C 520,3997 633,4115 773,4115 L 1547,4115 Z"/> + <path fill="rgb(127,128,130)" stroke="none" d="M 1547,3462 L 520,3462 520,3219 2573,3219 2573,3462 1547,3462 Z"/> + <path fill="rgb(127,128,130)" stroke="none" d="M 1547,2910 L 520,2910 520,2668 2573,2668 2573,2910 1547,2910 Z"/> + <path fill="rgb(127,128,130)" stroke="none" d="M 1547,2358 L 520,2358 520,2116 2573,2116 2573,2358 1547,2358 Z"/> + <path fill="rgb(127,128,130)" stroke="none" d="M 1547,1807 L 520,1807 520,1564 2573,1564 2573,1807 1547,1807 Z"/> + <path fill="rgb(250,203,27)" stroke="none" d="M 1127,1043 C 1127,1078 1119,1108 1103,1138 1087,1169 1066,1191 1038,1208 1009,1226 982,1234 949,1234 916,1234 888,1226 860,1208 831,1191 811,1169 795,1138 778,1108 771,1078 771,1043 771,1007 778,977 795,947 811,916 831,895 860,877 888,859 916,851 949,851 982,851 1009,859 1038,877 1066,895 1087,916 1103,947 1119,977 1127,1007 1127,1043 L 1127,1043 Z"/> + <path fill="rgb(249,145,52)" stroke="none" d="M 1717,1043 C 1717,1078 1710,1108 1694,1138 1677,1169 1657,1191 1628,1208 1600,1226 1572,1234 1539,1234 1507,1234 1479,1226 1450,1208 1422,1191 1401,1169 1385,1138 1369,1108 1361,1078 1361,1043 1361,1007 1369,977 1385,947 1401,916 1422,895 1450,877 1479,859 1507,851 1539,851 1572,851 1600,859 1628,877 1657,895 1677,916 1694,947 1710,977 1717,1007 1717,1043 L 1717,1043 Z"/> + <path fill="rgb(250,85,85)" stroke="none" d="M 2308,1043 C 2308,1078 2300,1108 2284,1138 2268,1169 2247,1191 2219,1208 2190,1226 2163,1234 2130,1234 2097,1234 2069,1226 2041,1208 2012,1191 1992,1169 1976,1138 1959,1108 1952,1078 1952,1043 1952,1007 1959,977 1976,947 1992,916 2012,895 2041,877 2069,859 2097,851 2130,851 2163,851 2190,859 2219,877 2247,895 2268,916 2284,947 2300,977 2308,1007 2308,1043 L 2308,1043 Z"/> + </g> + </g> + <g class="Graphic"> + <g id="id5"> + <rect class="BoundingBox" stroke="none" fill="none" x="514" y="4640" width="2066" height="3618"/> + <path fill="rgb(62,62,63)" stroke="none" d="M 1547,8249 L 2320,8249 C 2460,8249 2573,8131 2573,7985 L 2573,4913 C 2573,4767 2460,4650 2320,4650 L 773,4650 C 633,4650 520,4767 520,4913 L 520,7985 C 520,8131 633,8249 773,8249 L 1547,8249 Z"/> + <path fill="rgb(127,128,130)" stroke="none" d="M 1547,7596 L 520,7596 520,7353 2573,7353 2573,7596 1547,7596 Z"/> + <path fill="rgb(127,128,130)" stroke="none" d="M 1547,7044 L 520,7044 520,6802 2573,6802 2573,7044 1547,7044 Z"/> + <path fill="rgb(127,128,130)" stroke="none" d="M 1547,6492 L 520,6492 520,6250 2573,6250 2573,6492 1547,6492 Z"/> + <path fill="rgb(127,128,130)" stroke="none" d="M 1547,5941 L 520,5941 520,5698 2573,5698 2573,5941 1547,5941 Z"/> + <path fill="rgb(250,203,27)" stroke="none" d="M 1127,5177 C 1127,5212 1119,5242 1103,5272 1087,5303 1066,5325 1038,5342 1009,5360 982,5368 949,5368 916,5368 888,5360 860,5342 831,5325 811,5303 795,5272 778,5242 771,5212 771,5177 771,5141 778,5111 795,5081 811,5050 831,5029 860,5011 888,4993 916,4985 949,4985 982,4985 1009,4993 1038,5011 1066,5029 1087,5050 1103,5081 1119,5111 1127,5141 1127,5177 L 1127,5177 Z"/> + <path fill="rgb(249,145,52)" stroke="none" d="M 1717,5177 C 1717,5212 1710,5242 1694,5272 1677,5303 1657,5325 1628,5342 1600,5360 1572,5368 1539,5368 1507,5368 1479,5360 1450,5342 1422,5325 1401,5303 1385,5272 1369,5242 1361,5212 1361,5177 1361,5141 1369,5111 1385,5081 1401,5050 1422,5029 1450,5011 1479,4993 1507,4985 1539,4985 1572,4985 1600,4993 1628,5011 1657,5029 1677,5050 1694,5081 1710,5111 1717,5141 1717,5177 L 1717,5177 Z"/> + <path fill="rgb(250,85,85)" stroke="none" d="M 2308,5177 C 2308,5212 2300,5242 2284,5272 2268,5303 2247,5325 2219,5342 2190,5360 2163,5368 2130,5368 2097,5368 2069,5360 2041,5342 2012,5325 1992,5303 1976,5272 1959,5242 1952,5212 1952,5177 1952,5141 1959,5111 1976,5081 1992,5050 2012,5029 2041,5011 2069,4993 2097,4985 2130,4985 2163,4985 2190,4993 2219,5011 2247,5029 2268,5050 2284,5081 2300,5111 2308,5141 2308,5177 L 2308,5177 Z"/> + </g> + </g> + <g class="Graphic"> + <g id="id6"> + <rect class="BoundingBox" stroke="none" fill="none" x="13747" y="506" width="2066" height="3618"/> + <path fill="rgb(62,62,63)" stroke="none" d="M 14780,4115 L 15553,4115 C 15693,4115 15806,3997 15806,3851 L 15806,779 C 15806,633 15693,516 15553,516 L 14006,516 C 13866,516 13753,633 13753,779 L 13753,3851 C 13753,3997 13866,4115 14006,4115 L 14780,4115 Z"/> + <path fill="rgb(127,128,130)" stroke="none" d="M 14780,3462 L 13753,3462 13753,3219 15806,3219 15806,3462 14780,3462 Z"/> + <path fill="rgb(127,128,130)" stroke="none" d="M 14780,2910 L 13753,2910 13753,2668 15806,2668 15806,2910 14780,2910 Z"/> + <path fill="rgb(127,128,130)" stroke="none" d="M 14780,2358 L 13753,2358 13753,2116 15806,2116 15806,2358 14780,2358 Z"/> + <path fill="rgb(127,128,130)" stroke="none" d="M 14780,1807 L 13753,1807 13753,1564 15806,1564 15806,1807 14780,1807 Z"/> + <path fill="rgb(250,203,27)" stroke="none" d="M 14360,1043 C 14360,1078 14352,1108 14336,1138 14320,1169 14299,1191 14271,1208 14242,1226 14215,1234 14182,1234 14149,1234 14121,1226 14093,1208 14064,1191 14044,1169 14028,1138 14011,1108 14004,1078 14004,1043 14004,1007 14011,977 14028,947 14044,916 14064,895 14093,877 14121,859 14149,851 14182,851 14215,851 14242,859 14271,877 14299,895 14320,916 14336,947 14352,977 14360,1007 14360,1043 L 14360,1043 Z"/> + <path fill="rgb(249,145,52)" stroke="none" d="M 14950,1043 C 14950,1078 14943,1108 14927,1138 14910,1169 14890,1191 14861,1208 14833,1226 14805,1234 14772,1234 14740,1234 14712,1226 14683,1208 14655,1191 14634,1169 14618,1138 14602,1108 14594,1078 14594,1043 14594,1007 14602,977 14618,947 14634,916 14655,895 14683,877 14712,859 14740,851 14772,851 14805,851 14833,859 14861,877 14890,895 14910,916 14927,947 14943,977 14950,1007 14950,1043 L 14950,1043 Z"/> + <path fill="rgb(250,85,85)" stroke="none" d="M 15541,1043 C 15541,1078 15533,1108 15517,1138 15501,1169 15480,1191 15452,1208 15423,1226 15396,1234 15363,1234 15330,1234 15302,1226 15274,1208 15245,1191 15225,1169 15209,1138 15192,1108 15185,1078 15185,1043 15185,1007 15192,977 15209,947 15225,916 15245,895 15274,877 15302,859 15330,851 15363,851 15396,851 15423,859 15452,877 15480,895 15501,916 15517,947 15533,977 15541,1007 15541,1043 L 15541,1043 Z"/> + </g> + </g> + <g class="Graphic"> + <g id="id7"> + <rect class="BoundingBox" stroke="none" fill="none" x="13741" y="8598" width="2066" height="3618"/> + <path fill="rgb(62,62,63)" stroke="none" d="M 14774,12207 L 15547,12207 C 15687,12207 15800,12089 15800,11943 L 15800,8871 C 15800,8725 15687,8608 15547,8608 L 14000,8608 C 13860,8608 13747,8725 13747,8871 L 13747,11943 C 13747,12089 13860,12207 14000,12207 L 14774,12207 Z"/> + <path fill="rgb(127,128,130)" stroke="none" d="M 14774,11554 L 13747,11554 13747,11311 15800,11311 15800,11554 14774,11554 Z"/> + <path fill="rgb(127,128,130)" stroke="none" d="M 14774,11002 L 13747,11002 13747,10760 15800,10760 15800,11002 14774,11002 Z"/> + <path fill="rgb(127,128,130)" stroke="none" d="M 14774,10450 L 13747,10450 13747,10208 15800,10208 15800,10450 14774,10450 Z"/> + <path fill="rgb(127,128,130)" stroke="none" d="M 14774,9899 L 13747,9899 13747,9656 15800,9656 15800,9899 14774,9899 Z"/> + <path fill="rgb(250,203,27)" stroke="none" d="M 14354,9135 C 14354,9170 14346,9200 14330,9230 14314,9261 14293,9283 14265,9300 14236,9318 14209,9326 14176,9326 14143,9326 14115,9318 14087,9300 14058,9283 14038,9261 14022,9230 14005,9200 13998,9170 13998,9135 13998,9099 14005,9069 14022,9039 14038,9008 14058,8987 14087,8969 14115,8951 14143,8943 14176,8943 14209,8943 14236,8951 14265,8969 14293,8987 14314,9008 14330,9039 14346,9069 14354,9099 14354,9135 L 14354,9135 Z"/> + <path fill="rgb(249,145,52)" stroke="none" d="M 14944,9135 C 14944,9170 14937,9200 14921,9230 14904,9261 14884,9283 14855,9300 14827,9318 14799,9326 14766,9326 14734,9326 14706,9318 14677,9300 14649,9283 14628,9261 14612,9230 14596,9200 14588,9170 14588,9135 14588,9099 14596,9069 14612,9039 14628,9008 14649,8987 14677,8969 14706,8951 14734,8943 14766,8943 14799,8943 14827,8951 14855,8969 14884,8987 14904,9008 14921,9039 14937,9069 14944,9099 14944,9135 L 14944,9135 Z"/> + <path fill="rgb(250,85,85)" stroke="none" d="M 15535,9135 C 15535,9170 15527,9200 15511,9230 15495,9261 15474,9283 15446,9300 15417,9318 15390,9326 15357,9326 15324,9326 15296,9318 15268,9300 15239,9283 15219,9261 15203,9230 15186,9200 15179,9170 15179,9135 15179,9099 15186,9069 15203,9039 15219,9008 15239,8987 15268,8969 15296,8951 15324,8943 15357,8943 15390,8943 15417,8951 15446,8969 15474,8987 15495,9008 15511,9039 15527,9069 15535,9099 15535,9135 L 15535,9135 Z"/> + </g> + </g> + <g class="com.sun.star.drawing.ConnectorShape"> + <g id="id8"> + <rect class="BoundingBox" stroke="none" fill="none" x="14732" y="4082" width="89" height="4558"/> + <path fill="none" stroke="rgb(128,128,128)" stroke-width="81" stroke-linejoin="round" d="M 14779,4123 L 14779,6361 14773,6361 14773,8598"/> + </g> + </g> + <g class="com.sun.star.drawing.ConnectorShape"> + <g id="id9"> + <rect class="BoundingBox" stroke="none" fill="none" x="1505" y="4082" width="83" height="600"/> + <path fill="none" stroke="rgb(128,128,128)" stroke-width="81" stroke-linejoin="round" d="M 1546,4123 L 1546,4640"/> + </g> + </g> + <g class="com.sun.star.drawing.ConnectorShape"> + <g id="id10"> + <rect class="BoundingBox" stroke="none" fill="none" x="2538" y="6407" width="6692" height="83"/> + <path fill="none" stroke="rgb(188,49,46)" stroke-width="81" stroke-linejoin="round" d="M 2579,6448 L 9188,6448"/> + </g> + </g> + <g class="com.sun.star.drawing.ConnectorShape"> + <g id="id11"> + <rect class="BoundingBox" stroke="none" fill="none" x="11212" y="6407" width="2571" height="4041"/> + <path fill="none" stroke="rgb(188,49,46)" stroke-width="81" stroke-linejoin="round" d="M 11253,6448 L 11973,6448 11973,10406 13741,10406"/> + </g> + </g> + <g class="com.sun.star.drawing.ConnectorShape"> + <g id="id12"> + <rect class="BoundingBox" stroke="none" fill="none" x="11212" y="2273" width="2577" height="4217"/> + <path fill="none" stroke="rgb(188,49,46)" stroke-width="81" stroke-linejoin="round" d="M 11253,6448 L 11973,6448 11973,2314 13747,2314"/> + </g> + </g> + <g class="com.sun.star.drawing.ConnectorShape"> + <g id="id13"> + <rect class="BoundingBox" stroke="none" fill="none" x="1505" y="8216" width="2426" height="2231"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="81" stroke-linejoin="round" d="M 1546,8257 L 1546,10405 3889,10405"/> + </g> + </g> + <g class="com.sun.star.drawing.ConnectorShape"> + <g id="id14"> + <rect class="BoundingBox" stroke="none" fill="none" x="7913" y="8216" width="2349" height="2231"/> + <path fill="none" stroke="rgb(52,101,164)" stroke-width="81" stroke-linejoin="round" d="M 7954,10405 L 10220,10405 10220,8257"/> + </g> + </g> + <g class="com.sun.star.drawing.TextShape"> + <g id="id15"> + <rect class="BoundingBox" stroke="none" fill="none" x="488" y="3366" width="2101" height="1200"/> + <text class="TextShape"><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="423px" font-weight="700"><tspan class="TextPosition" x="738" y="3876"><tspan fill="rgb(255,255,255)" stroke="none">Jenkins</tspan></tspan></tspan></text> + </g> + </g> + <g class="com.sun.star.drawing.TextShape"> + <g id="id16"> + <rect class="BoundingBox" stroke="none" fill="none" x="488" y="7045" width="2101" height="1674"/> + <text class="TextShape"><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="423px" font-weight="700"><tspan class="TextPosition" x="747" y="7555"><tspan fill="rgb(255,255,255)" stroke="none">Jenkins</tspan></tspan></tspan><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="423px" font-weight="700"><tspan class="TextPosition" x="1005" y="8029"><tspan fill="rgb(255,255,255)" stroke="none">slave</tspan></tspan></tspan></text> + </g> + </g> + <g class="com.sun.star.drawing.TextShape"> + <g id="id17"> + <rect class="BoundingBox" stroke="none" fill="none" x="13806" y="3424" width="2001" height="726"/> + <text class="TextShape"><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="423px" font-weight="700"><tspan class="TextPosition" x="14512" y="3934"><tspan fill="rgb(255,255,255)" stroke="none">TG</tspan></tspan></tspan></text> + </g> + </g> + <g class="com.sun.star.drawing.TextShape"> + <g id="id18"> + <rect class="BoundingBox" stroke="none" fill="none" x="13806" y="11490" width="2001" height="726"/> + <text class="TextShape"><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="423px" font-weight="700"><tspan class="TextPosition" x="14383" y="12000"><tspan fill="rgb(255,255,255)" stroke="none">SUT</tspan></tspan></tspan></text> + </g> + </g> + <g class="com.sun.star.drawing.TextShape"> + <g id="id19"> + <rect class="BoundingBox" stroke="none" fill="none" x="9188" y="5546" width="2101" height="2622"/> + <text class="TextShape"><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="423px" font-weight="700"><tspan class="TextPosition" x="9756" y="6056"><tspan fill="rgb(255,255,255)" stroke="none">CSIT</tspan></tspan></tspan><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="423px" font-weight="700"><tspan class="TextPosition" x="9743" y="6530"><tspan fill="rgb(255,255,255)" stroke="none">shim </tspan></tspan><tspan class="TextPosition" x="9980" y="7004"><tspan fill="rgb(255,255,255)" stroke="none">on</tspan></tspan></tspan><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="423px" font-weight="700"><tspan class="TextPosition" x="9682" y="7478"><tspan fill="rgb(255,255,255)" stroke="none">every</tspan></tspan></tspan><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="423px" font-weight="700"><tspan class="TextPosition" x="9792" y="7952"><tspan fill="rgb(255,255,255)" stroke="none">host</tspan></tspan></tspan></text> + </g> + </g> + <g class="Graphic"> + <g id="id20"> + <rect class="BoundingBox" stroke="none" fill="none" x="3889" y="8597" width="2066" height="3618"/> + <path fill="rgb(62,62,63)" stroke="none" d="M 4922,12206 L 5695,12206 C 5835,12206 5948,12088 5948,11942 L 5948,8870 C 5948,8724 5835,8607 5695,8607 L 4148,8607 C 4008,8607 3895,8724 3895,8870 L 3895,11942 C 3895,12088 4008,12206 4148,12206 L 4922,12206 Z"/> + <path fill="rgb(127,128,130)" stroke="none" d="M 4922,11553 L 3895,11553 3895,11310 5948,11310 5948,11553 4922,11553 Z"/> + <path fill="rgb(127,128,130)" stroke="none" d="M 4922,11001 L 3895,11001 3895,10759 5948,10759 5948,11001 4922,11001 Z"/> + <path fill="rgb(127,128,130)" stroke="none" d="M 4922,10449 L 3895,10449 3895,10207 5948,10207 5948,10449 4922,10449 Z"/> + <path fill="rgb(127,128,130)" stroke="none" d="M 4922,9898 L 3895,9898 3895,9655 5948,9655 5948,9898 4922,9898 Z"/> + <path fill="rgb(250,203,27)" stroke="none" d="M 4502,9134 C 4502,9169 4494,9199 4478,9229 4462,9260 4441,9282 4413,9299 4384,9317 4357,9325 4324,9325 4291,9325 4263,9317 4235,9299 4206,9282 4186,9260 4170,9229 4153,9199 4146,9169 4146,9134 4146,9098 4153,9068 4170,9038 4186,9007 4206,8986 4235,8968 4263,8950 4291,8942 4324,8942 4357,8942 4384,8950 4413,8968 4441,8986 4462,9007 4478,9038 4494,9068 4502,9098 4502,9134 L 4502,9134 Z"/> + <path fill="rgb(249,145,52)" stroke="none" d="M 5092,9134 C 5092,9169 5085,9199 5069,9229 5052,9260 5032,9282 5003,9299 4975,9317 4947,9325 4914,9325 4882,9325 4854,9317 4825,9299 4797,9282 4776,9260 4760,9229 4744,9199 4736,9169 4736,9134 4736,9098 4744,9068 4760,9038 4776,9007 4797,8986 4825,8968 4854,8950 4882,8942 4914,8942 4947,8942 4975,8950 5003,8968 5032,8986 5052,9007 5069,9038 5085,9068 5092,9098 5092,9134 L 5092,9134 Z"/> + <path fill="rgb(250,85,85)" stroke="none" d="M 5683,9134 C 5683,9169 5675,9199 5659,9229 5643,9260 5622,9282 5594,9299 5565,9317 5538,9325 5505,9325 5472,9325 5444,9317 5416,9299 5387,9282 5367,9260 5351,9229 5334,9199 5327,9169 5327,9134 5327,9098 5334,9068 5351,9038 5367,9007 5387,8986 5416,8968 5444,8950 5472,8942 5505,8942 5538,8942 5565,8950 5594,8968 5622,8986 5643,9007 5659,9038 5675,9068 5683,9098 5683,9134 L 5683,9134 Z"/> + </g> + </g> + <g class="Graphic"> + <g id="id21"> + <rect class="BoundingBox" stroke="none" fill="none" x="5889" y="8597" width="2066" height="3618"/> + <path fill="rgb(62,62,63)" stroke="none" d="M 6922,12206 L 7695,12206 C 7835,12206 7948,12088 7948,11942 L 7948,8870 C 7948,8724 7835,8607 7695,8607 L 6148,8607 C 6008,8607 5895,8724 5895,8870 L 5895,11942 C 5895,12088 6008,12206 6148,12206 L 6922,12206 Z"/> + <path fill="rgb(127,128,130)" stroke="none" d="M 6922,11553 L 5895,11553 5895,11310 7948,11310 7948,11553 6922,11553 Z"/> + <path fill="rgb(127,128,130)" stroke="none" d="M 6922,11001 L 5895,11001 5895,10759 7948,10759 7948,11001 6922,11001 Z"/> + <path fill="rgb(127,128,130)" stroke="none" d="M 6922,10449 L 5895,10449 5895,10207 7948,10207 7948,10449 6922,10449 Z"/> + <path fill="rgb(127,128,130)" stroke="none" d="M 6922,9898 L 5895,9898 5895,9655 7948,9655 7948,9898 6922,9898 Z"/> + <path fill="rgb(250,203,27)" stroke="none" d="M 6502,9134 C 6502,9169 6494,9199 6478,9229 6462,9260 6441,9282 6413,9299 6384,9317 6357,9325 6324,9325 6291,9325 6263,9317 6235,9299 6206,9282 6186,9260 6170,9229 6153,9199 6146,9169 6146,9134 6146,9098 6153,9068 6170,9038 6186,9007 6206,8986 6235,8968 6263,8950 6291,8942 6324,8942 6357,8942 6384,8950 6413,8968 6441,8986 6462,9007 6478,9038 6494,9068 6502,9098 6502,9134 L 6502,9134 Z"/> + <path fill="rgb(249,145,52)" stroke="none" d="M 7092,9134 C 7092,9169 7085,9199 7069,9229 7052,9260 7032,9282 7003,9299 6975,9317 6947,9325 6914,9325 6882,9325 6854,9317 6825,9299 6797,9282 6776,9260 6760,9229 6744,9199 6736,9169 6736,9134 6736,9098 6744,9068 6760,9038 6776,9007 6797,8986 6825,8968 6854,8950 6882,8942 6914,8942 6947,8942 6975,8950 7003,8968 7032,8986 7052,9007 7069,9038 7085,9068 7092,9098 7092,9134 L 7092,9134 Z"/> + <path fill="rgb(250,85,85)" stroke="none" d="M 7683,9134 C 7683,9169 7675,9199 7659,9229 7643,9260 7622,9282 7594,9299 7565,9317 7538,9325 7505,9325 7472,9325 7444,9317 7416,9299 7387,9282 7367,9260 7351,9229 7334,9199 7327,9169 7327,9134 7327,9098 7334,9068 7351,9038 7367,9007 7387,8986 7416,8968 7444,8950 7472,8942 7505,8942 7538,8942 7565,8950 7594,8968 7622,8986 7643,9007 7659,9038 7675,9068 7683,9098 7683,9134 L 7683,9134 Z"/> + </g> + </g> + <g class="com.sun.star.drawing.TextShape"> + <g id="id22"> + <rect class="BoundingBox" stroke="none" fill="none" x="4288" y="11471" width="3153" height="726"/> + <text class="TextShape"><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="423px" font-weight="700"><tspan class="TextPosition" x="5028" y="11981"><tspan fill="rgb(255,255,255)" stroke="none">Nomad1</tspan></tspan></tspan></text> + </g> + </g> + <g class="com.sun.star.drawing.TextShape"> + <g id="id23"> + <rect class="BoundingBox" stroke="none" fill="none" x="2540" y="5825" width="6601" height="726"/> + <text class="TextShape"><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="423px" font-weight="400"><tspan class="TextPosition" x="4069" y="6335"><tspan fill="rgb(0,0,0)" stroke="none">SSH to known port</tspan></tspan></tspan></text> + </g> + </g> + <g class="com.sun.star.drawing.TextShape"> + <g id="id24"> + <rect class="BoundingBox" stroke="none" fill="none" x="11774" y="4040" width="1901" height="2401"/> + <text class="TextShape"><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="423px" font-weight="400"><tspan class="TextPosition" x="12024" y="4550"><tspan fill="rgb(0,0,0)" stroke="none">SSH or </tspan></tspan><tspan class="TextPosition" x="12024" y="5024"><tspan fill="rgb(0,0,0)" stroke="none">docker </tspan></tspan><tspan class="TextPosition" x="12024" y="5498"><tspan fill="rgb(0,0,0)" stroke="none">exec</tspan></tspan></tspan></text> + </g> + </g> + <g class="com.sun.star.drawing.TextShape"> + <g id="id25"> + <rect class="BoundingBox" stroke="none" fill="none" x="12800" y="7300" width="2101" height="1401"/> + <text class="TextShape"><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="423px" font-weight="400"><tspan class="TextPosition" x="13305" y="7810"><tspan fill="rgb(0,0,0)" stroke="none">Unique </tspan></tspan><tspan class="TextPosition" x="13165" y="8284"><tspan fill="rgb(0,0,0)" stroke="none">network </tspan></tspan></tspan></text> + </g> + </g> + <g class="com.sun.star.drawing.TextShape"> + <g id="id26"> + <rect class="BoundingBox" stroke="none" fill="none" x="8340" y="9840" width="2201" height="1301"/> + <text class="TextShape"><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="423px" font-weight="400"><tspan class="TextPosition" x="8590" y="10350"><tspan fill="rgb(0,0,0)" stroke="none">Nomad1 </tspan></tspan><tspan class="TextPosition" x="8590" y="10824"><tspan fill="rgb(0,0,0)" stroke="none">bridge</tspan></tspan></tspan></text> + </g> + </g> + </g> + </g> + </g> + </g> + </g> +</svg>
\ No newline at end of file |
