diff options
Diffstat (limited to 'doc/guides')
148 files changed, 11103 insertions, 1778 deletions
diff --git a/doc/guides/conf.py b/doc/guides/conf.py index 39880752..31f914a8 100644 --- a/doc/guides/conf.py +++ b/doc/guides/conf.py @@ -340,7 +340,12 @@ def print_table_css(outfile, table_id): font-size: 80%; white-space: pre-wrap; vertical-align: top; - padding: 2px; + padding: 0.5em 0; + min-width: 0.9em; + width: 2em; + } + table#idx col:first-child { + width: 0; } table#idx th:first-child { vertical-align: bottom; diff --git a/doc/guides/contributing/documentation.rst b/doc/guides/contributing/documentation.rst index cddbd7bb..170dacdb 100644 --- a/doc/guides/contributing/documentation.rst +++ b/doc/guides/contributing/documentation.rst @@ -34,7 +34,6 @@ The main directories that contain files related to documentation are shown below |-- testpmd_app_ug |-- rel_notes |-- nics - |-- xen |-- ... diff --git a/doc/guides/contributing/patches.rst b/doc/guides/contributing/patches.rst index 27e218b2..40983c15 100644 --- a/doc/guides/contributing/patches.rst +++ b/doc/guides/contributing/patches.rst @@ -367,7 +367,6 @@ The builds can be modified via the following environmental variables: * ``DPDK_BUILD_TEST_CONFIGS`` (target1+option1+option2 target2) * ``DPDK_DEP_CFLAGS`` * ``DPDK_DEP_LDFLAGS`` -* ``DPDK_DEP_MOFED`` (y/[n]) * ``DPDK_DEP_PCAP`` (y/[n]) * ``DPDK_NOTIFY`` (notify-send) @@ -404,6 +403,10 @@ The appropriate maintainer can be found in the ``MAINTAINERS`` file:: git send-email --to maintainer@some.org --cc dev@dpdk.org 000*.patch +Script ``get-maintainer.sh`` can be used to select maintainers automatically:: + + git send-email --to-cmd ./devtools/get-maintainer.sh --cc dev@dpdk.org 000*.patch + New additions can be sent without a maintainer:: git send-email --to dev@dpdk.org 000*.patch diff --git a/doc/guides/contributing/versioning.rst b/doc/guides/contributing/versioning.rst index 8aaf370d..8d0fdb77 100644 --- a/doc/guides/contributing/versioning.rst +++ b/doc/guides/contributing/versioning.rst @@ -18,6 +18,8 @@ General Guidelines #. The modification of symbols can generally be managed with versioning #. The removal of symbols generally is an ABI break and requires bumping of the LIBABIVER macro +#. Updates to the minimum hardware requirements, which drop support for hardware which + was previously supported, should be treated as an ABI change. What is an ABI -------------- @@ -77,6 +79,13 @@ for significant reasons, such as performance enhancements. ABI breakage due to changes such as reorganizing public structure fields for aesthetic or readability purposes should be avoided. +.. note:: + + Updates to the minimum hardware requirements, which drop support for hardware + which was previously supported, should be treated as an ABI change, and + follow the relevant deprecation policy procedures as above: 3 acks and + announcement at least one release in advance. + Examples of Deprecation Notices ------------------------------- diff --git a/doc/guides/cryptodevs/aesni_gcm.rst b/doc/guides/cryptodevs/aesni_gcm.rst index 6e9160aa..a1f58483 100644 --- a/doc/guides/cryptodevs/aesni_gcm.rst +++ b/doc/guides/cryptodevs/aesni_gcm.rst @@ -62,8 +62,8 @@ Installation To build DPDK with the AESNI_GCM_PMD the user is required to download the multi-buffer library from `here <https://github.com/01org/intel-ipsec-mb>`_ and compile it on their user system before building DPDK. -The latest version of the library supported by this PMD is v0.46, which -can be downloaded in `<https://github.com/01org/intel-ipsec-mb/archive/v0.46.zip>`_. +The latest version of the library supported by this PMD is v0.47, which +can be downloaded in `<https://github.com/01org/intel-ipsec-mb/archive/v0.47.zip>`_. .. code-block:: console diff --git a/doc/guides/cryptodevs/aesni_mb.rst b/doc/guides/cryptodevs/aesni_mb.rst index b3b937fe..79e722c6 100644 --- a/doc/guides/cryptodevs/aesni_mb.rst +++ b/doc/guides/cryptodevs/aesni_mb.rst @@ -52,6 +52,8 @@ Cipher algorithms: * RTE_CRYPTO_CIPHER_AES192_CTR * RTE_CRYPTO_CIPHER_AES256_CTR * RTE_CRYPTO_CIPHER_AES_DOCSISBPI +* RTE_CRYPTO_CIPHER_DES_CBC +* RTE_CRYPTO_CIPHER_DES_DOCSISBPI Hash algorithms: @@ -76,8 +78,8 @@ Installation To build DPDK with the AESNI_MB_PMD the user is required to download the multi-buffer library from `here <https://github.com/01org/intel-ipsec-mb>`_ and compile it on their user system before building DPDK. -The latest version of the library supported by this PMD is v0.46, which -can be downloaded in `<https://github.com/01org/intel-ipsec-mb/archive/v0.46.zip>`_. +The latest version of the library supported by this PMD is v0.47, which +can be downloaded from `<https://github.com/01org/intel-ipsec-mb/archive/v0.47.zip>`_. .. code-block:: console @@ -90,13 +92,14 @@ and the Multi-Buffer library version supported by them: .. table:: DPDK and Multi-Buffer library version compatibility - ============= ============================ - DPDK version Multi-buffer library version - ============= ============================ - 2.2 - 16.11 0.43 - 0.44 - 17.02 0.44 - 17.05+ 0.45+ - ============= ============================ + ============== ============================ + DPDK version Multi-buffer library version + ============== ============================ + 2.2 - 16.11 0.43 - 0.44 + 17.02 0.44 + 17.05 - 17.08 0.45 - 0.47 + 17.11+ 0.47 + ============== ============================ Initialization diff --git a/doc/guides/cryptodevs/dpaa2_sec.rst b/doc/guides/cryptodevs/dpaa2_sec.rst index 18d980eb..13474c18 100644 --- a/doc/guides/cryptodevs/dpaa2_sec.rst +++ b/doc/guides/cryptodevs/dpaa2_sec.rst @@ -183,17 +183,17 @@ The following dependencies are not part of DPDK and must be installed separately SDK and related information can be obtained from: `NXP QorIQ SDK <http://www.nxp.com/products/software-and-tools/run-time-software/linux-sdk/linux-sdk-for-qoriq-processors:SDKLINUX>`_. -* **DPDK Helper Scripts** +* **DPDK Extra Scripts** DPAA2 based resources can be configured easily with the help of ready scripts as provided in the DPDK helper repository. - `DPDK Helper Scripts <https://github.com/qoriq-open-source/dpdk-helper>`_. + `DPDK Extra Scripts <https://github.com/qoriq-open-source/dpdk-extras>`_. Currently supported by DPDK: -* NXP SDK **2.0+**. -* MC Firmware version **10.0.0** and higher. +* NXP SDK **17.08+**. +* MC Firmware version **10.3.1** and higher. * Supported architectures: **arm64 LE**. * Follow the DPDK :ref:`Getting Started Guide for Linux <linux_gsg>` to setup the basic DPDK environment. diff --git a/doc/guides/cryptodevs/dpaa_sec.rst b/doc/guides/cryptodevs/dpaa_sec.rst new file mode 100644 index 00000000..d3438cc3 --- /dev/null +++ b/doc/guides/cryptodevs/dpaa_sec.rst @@ -0,0 +1,182 @@ +.. BSD LICENSE + Copyright 2017 NXP. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of NXP nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + +NXP DPAA CAAM (DPAA_SEC) +======================== + +The DPAA_SEC PMD provides poll mode crypto driver support for NXP DPAA CAAM +hardware accelerator. + +Architecture +------------ + +SEC is the SOC's security engine, which serves as NXP's latest cryptographic +acceleration and offloading hardware. It combines functions previously +implemented in separate modules to create a modular and scalable acceleration +and assurance engine. It also implements block encryption algorithms, stream +cipher algorithms, hashing algorithms, public key algorithms, run-time +integrity checking, and a hardware random number generator. SEC performs +higher-level cryptographic operations than previous NXP cryptographic +accelerators. This provides significant improvement to system level performance. + +DPAA_SEC is one of the hardware resource in DPAA Architecture. More information +on DPAA Architecture is described in :ref:`dpaa_overview`. + +DPAA_SEC PMD is one of DPAA drivers which interacts with QBMAN to create, +configure and destroy the device instance using queue pair with CAAM portal. + +DPAA_SEC PMD also uses some of the other hardware resources like buffer pools, +queues, queue portals to store and to enqueue/dequeue data to the hardware SEC. + +Implementation +-------------- + +SEC provides platform assurance by working with SecMon, which is a companion +logic block that tracks the security state of the SOC. SEC is programmed by +means of descriptors (not to be confused with frame descriptors (FDs)) that +indicate the operations to be performed and link to the message and +associated data. SEC incorporates two DMA engines to fetch the descriptors, +read the message data, and write the results of the operations. The DMA +engine provides a scatter/gather capability so that SEC can read and write +data scattered in memory. SEC may be configured by means of software for +dynamic changes in byte ordering. The default configuration for this version +of SEC is little-endian mode. + +Features +-------- + +The DPAA PMD has support for: + +Cipher algorithms: + +* ``RTE_CRYPTO_CIPHER_3DES_CBC`` +* ``RTE_CRYPTO_CIPHER_AES128_CBC`` +* ``RTE_CRYPTO_CIPHER_AES192_CBC`` +* ``RTE_CRYPTO_CIPHER_AES256_CBC`` +* ``RTE_CRYPTO_CIPHER_AES128_CTR`` +* ``RTE_CRYPTO_CIPHER_AES192_CTR`` +* ``RTE_CRYPTO_CIPHER_AES256_CTR`` + +Hash algorithms: + +* ``RTE_CRYPTO_AUTH_SHA1_HMAC`` +* ``RTE_CRYPTO_AUTH_SHA224_HMAC`` +* ``RTE_CRYPTO_AUTH_SHA256_HMAC`` +* ``RTE_CRYPTO_AUTH_SHA384_HMAC`` +* ``RTE_CRYPTO_AUTH_SHA512_HMAC`` +* ``RTE_CRYPTO_AUTH_MD5_HMAC`` + +AEAD algorithms: + +* ``RTE_CRYPTO_AEAD_AES_GCM`` + +Supported DPAA SoCs +-------------------- + +* LS1046A/LS1026A +* LS1043A/LS1023A + +Limitations +----------- + +* Chained mbufs are not supported. +* Hash followed by Cipher mode is not supported +* Only supports the session-oriented API implementation (session-less APIs are not supported). + +Prerequisites +------------- + +DPAA_SEC driver has similar pre-requisites as described in :ref:`dpaa_overview`. +The following dependencies are not part of DPDK and must be installed separately: + +* **NXP Linux SDK** + + NXP Linux software development kit (SDK) includes support for the family + of QorIQ® ARM-Architecture-based system on chip (SoC) processors + and corresponding boards. + + It includes the Linux board support packages (BSPs) for NXP SoCs, + a fully operational tool chain, kernel and board specific modules. + + SDK and related information can be obtained from: `NXP QorIQ SDK <http://www.nxp.com/products/software-and-tools/run-time-software/linux-sdk/linux-sdk-for-qoriq-processors:SDKLINUX>`_. + +* **DPDK Extras Scripts** + + DPAA based resources can be configured easily with the help of ready scripts + as provided in the DPDK Extras repository. + + `DPDK Extras Scripts <https://github.com/qoriq-open-source/dpdk-extras>`_. + +Currently supported by DPDK: + +* NXP SDK **2.0+**. +* Supported architectures: **arm64 LE**. + +* Follow the DPDK :ref:`Getting Started Guide for Linux <linux_gsg>` to setup the basic DPDK environment. + +Pre-Installation Configuration +------------------------------ + +Config File Options +~~~~~~~~~~~~~~~~~~~ + +Basic DPAA config file options are described in :ref:`dpaa_overview`. +In addition to those, the following options can be modified in the ``config`` file +to enable DPAA_SEC PMD. + +Please note that enabling debugging options may affect system performance. + +* ``CONFIG_RTE_LIBRTE_PMD_DPAA_SEC`` (default ``n``) + By default it is only enabled in defconfig_arm64-dpaa-* config. + Toggle compilation of the ``librte_pmd_dpaa_sec`` driver. + +* ``CONFIG_RTE_LIBRTE_DPAA_SEC_DEBUG_INIT`` (default ``n``) + Toggle display of initialization related driver messages + +* ``CONFIG_RTE_LIBRTE_DPAA_SEC_DEBUG_DRIVER`` (default ``n``) + Toggle display of driver runtime messages + +* ``CONFIG_RTE_LIBRTE_DPAA_SEC_DEBUG_RX`` (default ``n``) + Toggle display of receive fast path run-time message + +* ``CONFIG_RTE_DPAA_SEC_PMD_MAX_NB_SESSIONS`` + By default it is set as 2048 in defconfig_arm64-dpaa-* config. + It indicates Number of sessions to create in the session memory pool + on a single DPAA SEC device. + +Installations +------------- +To compile the DPAA_SEC PMD for Linux arm64 gcc target, run the +following ``make`` command: + +.. code-block:: console + + cd <DPDK-source-directory> + make config T=arm64-dpaa-linuxapp-gcc install diff --git a/doc/guides/cryptodevs/features/aesni_mb.ini b/doc/guides/cryptodevs/features/aesni_mb.ini index 03d8485d..fab07cb2 100644 --- a/doc/guides/cryptodevs/features/aesni_mb.ini +++ b/doc/guides/cryptodevs/features/aesni_mb.ini @@ -23,6 +23,9 @@ AES CTR (128) = Y AES CTR (192) = Y AES CTR (256) = Y AES DOCSIS BPI = Y +DES CBC = Y +DES DOCSIS BPI = Y + ; ; Supported authentication algorithms of the 'aesni_mb' crypto driver. ; diff --git a/doc/guides/cryptodevs/features/default.ini b/doc/guides/cryptodevs/features/default.ini index 0926887b..18d66cb9 100644 --- a/doc/guides/cryptodevs/features/default.ini +++ b/doc/guides/cryptodevs/features/default.ini @@ -10,6 +10,7 @@ Symmetric crypto = Asymmetric crypto = Sym operation chaining = HW Accelerated = +Protocol offload = CPU SSE = CPU AVX = CPU AVX2 = @@ -68,3 +69,6 @@ ZUC EIA3 = AES GCM (128) = AES GCM (192) = AES GCM (256) = +AES CCM (128) = +AES CCM (192) = +AES CCM (256) = diff --git a/doc/guides/cryptodevs/features/dpaa2_sec.ini b/doc/guides/cryptodevs/features/dpaa2_sec.ini index c3bb3ddc..8fd07d6a 100644 --- a/doc/guides/cryptodevs/features/dpaa2_sec.ini +++ b/doc/guides/cryptodevs/features/dpaa2_sec.ini @@ -7,6 +7,7 @@ Symmetric crypto = Y Sym operation chaining = Y HW Accelerated = Y +Protocol offload = Y ; ; Supported crypto algorithms of the 'dpaa2_sec' crypto driver. diff --git a/doc/guides/cryptodevs/features/dpaa_sec.ini b/doc/guides/cryptodevs/features/dpaa_sec.ini new file mode 100644 index 00000000..0e8f5b2a --- /dev/null +++ b/doc/guides/cryptodevs/features/dpaa_sec.ini @@ -0,0 +1,40 @@ +; +; Supported features of the 'dpaa_sec' crypto driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +Symmetric crypto = Y +Sym operation chaining = Y +HW Accelerated = Y + +; +; Supported crypto algorithms of the 'dpaa_sec' crypto driver. +; +[Cipher] +AES CBC (128) = Y +AES CBC (192) = Y +AES CBC (256) = Y +AES CTR (128) = Y +AES CTR (192) = Y +AES CTR (256) = Y +3DES CBC = Y + +; +; Supported authentication algorithms of the 'dpaa_sec' crypto driver. +; +[Auth] +MD5 HMAC = Y +SHA1 HMAC = Y +SHA224 HMAC = Y +SHA256 HMAC = Y +SHA384 HMAC = Y +SHA512 HMAC = Y + +; +; Supported AEAD algorithms of the 'dpaa_sec' crypto driver. +; +[AEAD] +AES GCM (128) = Y +AES GCM (192) = Y +AES GCM (256) = Y diff --git a/doc/guides/cryptodevs/features/mrvl.ini b/doc/guides/cryptodevs/features/mrvl.ini new file mode 100644 index 00000000..6d2fe6aa --- /dev/null +++ b/doc/guides/cryptodevs/features/mrvl.ini @@ -0,0 +1,42 @@ +; Supported features of the 'mrvl' crypto driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +Symmetric crypto = Y +Sym operation chaining = Y + +; +; Supported crypto algorithms of a default crypto driver. +; +[Cipher] +AES CBC (128) = Y +AES CBC (192) = Y +AES CBC (256) = Y +AES CTR (128) = Y +AES CTR (192) = Y +AES CTR (256) = Y +3DES CBC = Y +3DES CTR = Y + +; +; Supported authentication algorithms of a default crypto driver. +; +[Auth] +MD5 = Y +MD5 HMAC = Y +SHA1 = Y +SHA1 HMAC = Y +SHA256 = Y +SHA256 HMAC = Y +SHA384 = Y +SHA384 HMAC = Y +SHA512 = Y +SHA512 HMAC = Y +AES GMAC = Y + +; +; Supported AEAD algorithms of a default crypto driver. +; +[AEAD] +AES GCM (128) = Y diff --git a/doc/guides/cryptodevs/features/openssl.ini b/doc/guides/cryptodevs/features/openssl.ini index aeb2a500..385ec4e0 100644 --- a/doc/guides/cryptodevs/features/openssl.ini +++ b/doc/guides/cryptodevs/features/openssl.ini @@ -45,3 +45,6 @@ AES GMAC = Y AES GCM (128) = Y AES GCM (192) = Y AES GCM (256) = Y +AES CCM (128) = Y +AES CCM (192) = Y +AES CCM (256) = Y diff --git a/doc/guides/cryptodevs/index.rst b/doc/guides/cryptodevs/index.rst index 361b82dd..6d4e15b9 100644 --- a/doc/guides/cryptodevs/index.rst +++ b/doc/guides/cryptodevs/index.rst @@ -40,8 +40,10 @@ Crypto Device Drivers aesni_gcm armv8 dpaa2_sec + dpaa_sec kasumi openssl + mrvl null scheduler snow3g diff --git a/doc/guides/cryptodevs/mrvl.rst b/doc/guides/cryptodevs/mrvl.rst new file mode 100644 index 00000000..4e992fbb --- /dev/null +++ b/doc/guides/cryptodevs/mrvl.rst @@ -0,0 +1,206 @@ +.. BSD LICENSE + Copyright(c) 2017 Marvell International Ltd. + Copyright(c) 2017 Semihalf. + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of the copyright holder nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +MRVL Crypto Poll Mode Driver +============================ + +The MRVL CRYPTO PMD (**librte_crypto_mrvl_pmd**) provides poll mode crypto driver +support by utilizing MUSDK library, which provides cryptographic operations +acceleration by using Security Acceleration Engine (EIP197) directly from +user-space with minimum overhead and high performance. + +Features +-------- + +MRVL CRYPTO PMD has support for: + +* Symmetric crypto +* Sym operation chaining +* AES CBC (128) +* AES CBC (192) +* AES CBC (256) +* AES CTR (128) +* AES CTR (192) +* AES CTR (256) +* 3DES CBC +* 3DES CTR +* MD5 +* MD5 HMAC +* SHA1 +* SHA1 HMAC +* SHA256 +* SHA256 HMAC +* SHA384 +* SHA384 HMAC +* SHA512 +* SHA512 HMAC +* AES GCM (128) + +Limitations +----------- + +* Hardware only supports scenarios where ICV (digest buffer) is placed just + after the authenticated data. Other placement will result in error. + +* Before running crypto test suite it is advised to increase limit of + opened files: + + .. code-block:: console + + ulimit -n 20000 + +Installation +------------ + +MRVL CRYPTO PMD driver compilation is disabled by default due to external dependencies. +Currently there are two driver specific compilation options in +``config/common_base`` available: + +- ``CONFIG_RTE_LIBRTE_MRVL_CRYPTO`` (default ``n``) + + Toggle compilation of the librte_pmd_mrvl driver. + +- ``CONFIG_RTE_LIBRTE_MRVL_CRYPTO_DEBUG`` (default ``n``) + + Toggle display of debugging messages. + +During compilation external MUSDK library, which provides direct access +to Marvell's EIP197 cryptographic engine, is necessary. Library sources are +available `here <https://github.com/MarvellEmbeddedProcessors/musdk-marvell/tree/musdk-armada-17.08>`__. + +Alternatively, prebuilt library can be downloaded from +`Marvell Extranet <https://extranet.marvell.com>`_. Once approval has been +granted, library can be found by typing ``musdk`` in the search box. + +For MUSDK library build instructions please refer to ``doc/musdk_get_started.txt`` +in library sources directory. + +MUSDK requires out of tree kernel modules to work. Kernel tree needed to build +them is available +`here <https://github.com/MarvellEmbeddedProcessors/linux-marvell/tree/linux-4.4.52-armada-17.08>`__. + +Initialization +-------------- + +After successfully building MRVL CRYPTO PMD, the following modules need to be +loaded: + +.. code-block:: console + + insmod musdk_uio.ko + insmod mvpp2x_sysfs.ko + insmod mv_pp_uio.ko + insmod mv_sam_uio.ko + insmod crypto_safexcel.ko + +- `musdk_uio.ko`, `mv_pp2_uio.ko` and `mv_sam_uio.ko` are distributed together with MUSDK library. +- `crypto_safexcel.ko` is an in-kernel module. +- `mvpp2x_sysfs.ko` can be build from sources available `here <https://github.com/MarvellEmbeddedProcessors/mvpp2x-marvell/tree/mvpp2x-armada-17.08>`__. + +The following parameters (all optional) are exported by the driver: + +* max_nb_queue_pairs: maximum number of queue pairs in the device (8 by default). +* max_nb_sessions: maximum number of sessions that can be created (2048 by default). +* socket_id: socket on which to allocate the device resources on. + +l2fwd-crypto example application can be used to verify MRVL CRYPTO PMD +operation: + +.. code-block:: console + + ./l2fwd-crypto --vdev=net_mrvl,iface=eth0 --vdev=crypto_mrvl -- \ + --cipher_op ENCRYPT --cipher_algo aes-cbc \ + --cipher_key 00:01:02:03:04:05:06:07:08:09:0a:0b:0c:0d:0e:0f \ + --auth_op GENERATE --auth_algo sha1-hmac \ + --auth_key 10:11:12:13:14:15:16:17:18:19:1a:1b:1c:1d:1e:1f + +Example output: + +.. code-block:: console + + [...] + AAD: at [0x7f253ceb80], len= + P ID 0 configuration ---- + Port mode : KR + MAC status : disabled + Link status : link up + Port speed : 10G + Port duplex : full + Port: Egress enable tx_port_num=16 qmap=0x1 + PORT: Port0 - link + P ID 0 configuration ---- + Port mode : KR + MAC status : disabled + Link status : link down + Port speed : 10G + Port duplex : full + Port: Egress enable tx_port_num=16 qmap=0x1 + Port 0, MAC address: 00:50:43:02:21:20 + + + Checking link statusdone + Port 0 Link Up - speed 0 Mbps - full-duplex + Lcore 0: RX port 0 + Allocated session pool on socket 0 + eip197: 0:0 registers: paddr: 0xf2880000, vaddr: 0x0x7f56a80000 + DMA buffer (131136 bytes) for CDR #0 allocated: paddr = 0xb0585e00, vaddr = 0x7f09384e00 + DMA buffer (131136 bytes) for RDR #0 allocated: paddr = 0xb05a5f00, vaddr = 0x7f093a4f00 + DMA buffers allocated for 2049 operations. Tokens - 256 bytes + Lcore 0: cryptodev 0 + L2FWD: lcore 1 has nothing to do + L2FWD: lcore 2 has nothing to do + L2FWD: lcore 3 has nothing to do + L2FWD: entering main loop on lcore 0 + L2FWD: -- lcoreid=0 portid=0 + L2FWD: -- lcoreid=0 cryptoid=0 + Options:- + nportmask: ffffffff + ports per lcore: 1 + refresh period : 10000 + single lcore mode: disabled + stats_printing: enabled + sessionless crypto: disabled + + Crypto chain: Input --> Encrypt --> Auth generate --> Output + + ---- Cipher information --- + Algorithm: aes-cbc + Cipher key: at [0x7f56db4e80], len=16 + 00000000: 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F | ................ + IV: at [0x7f56db4b80], len=16 + 00000000: 20 F0 63 0E 45 EB 2D 84 72 D4 13 6E 36 B5 AF FE | .c.E.-.r..n6... + + ---- Authentication information --- + Algorithm: sha1-hmac + Auth key: at [0x7f56db4d80], len=16 + 00000000: 10 11 12 13 14 15 16 17 18 19 1A 1B 1C 1D 1E 1F | ................ + IV: at [0x7f56db4a80], len=0 + AAD: at [0x7f253ceb80], len= diff --git a/doc/guides/cryptodevs/openssl.rst b/doc/guides/cryptodevs/openssl.rst index f18a4567..243ea36a 100644 --- a/doc/guides/cryptodevs/openssl.rst +++ b/doc/guides/cryptodevs/openssl.rst @@ -67,6 +67,7 @@ Supported authentication algorithms: Supported AEAD algorithms: * ``RTE_CRYPTO_AEAD_AES_GCM`` +* ``RTE_CRYPTO_AEAD_AES_CCM`` Installation diff --git a/doc/guides/cryptodevs/qat.rst b/doc/guides/cryptodevs/qat.rst index a3fce7b8..cb17b6b3 100644 --- a/doc/guides/cryptodevs/qat.rst +++ b/doc/guides/cryptodevs/qat.rst @@ -90,6 +90,7 @@ Limitations * No BSD support as BSD QAT kernel driver not available. * ZUC EEA3/EIA3 is not supported by dh895xcc devices * Maximum additional authenticated data (AAD) for GCM is 240 bytes long. +* Queue pairs are not thread-safe (that is, within a single queue pair, RX and TX from different lcores is not supported). Installation diff --git a/doc/guides/eventdevs/octeontx.rst b/doc/guides/eventdevs/octeontx.rst index b43d5155..7e601a07 100644 --- a/doc/guides/eventdevs/octeontx.rst +++ b/doc/guides/eventdevs/octeontx.rst @@ -149,3 +149,8 @@ Burst mode support Burst mode is not supported. Dequeue and Enqueue functions accepts only single event at a time. +Rx adapter support +~~~~~~~~~~~~~~~~~~ + +When eth_octeontx is used as Rx adapter event schedule type +``RTE_SCHED_TYPE_PARALLEL`` is not supported. diff --git a/doc/guides/eventdevs/sw.rst b/doc/guides/eventdevs/sw.rst index a3e66243..ec49b3b5 100644 --- a/doc/guides/eventdevs/sw.rst +++ b/doc/guides/eventdevs/sw.rst @@ -78,9 +78,9 @@ Scheduling Quanta ~~~~~~~~~~~~~~~~~ The scheduling quanta sets the number of events that the device attempts to -schedule before returning to the application from the ``rte_event_schedule()`` -function. Note that is a *hint* only, and that fewer or more events may be -scheduled in a given iteration. +schedule in a single schedule call performed by the service core. Note that +is a *hint* only, and that fewer or more events may be scheduled in a given +iteration. The scheduling quanta can be set using a string argument to the vdev create call: @@ -140,10 +140,9 @@ eventdev. Distributed Scheduler ~~~~~~~~~~~~~~~~~~~~~ -The software eventdev is a centralized scheduler, requiring the -``rte_event_schedule()`` function to be called by a CPU core to perform the -required event distribution. This is not really a limitation but rather a -design decision. +The software eventdev is a centralized scheduler, requiring a service core to +perform the required event distribution. This is not really a limitation but +rather a design decision. The ``RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED`` flag is not set in the ``event_dev_cap`` field of the ``rte_event_dev_info`` struct for the software diff --git a/doc/guides/freebsd_gsg/build_sample_apps.rst b/doc/guides/freebsd_gsg/build_sample_apps.rst index 9faa0e6e..d84f15b8 100644 --- a/doc/guides/freebsd_gsg/build_sample_apps.rst +++ b/doc/guides/freebsd_gsg/build_sample_apps.rst @@ -163,6 +163,9 @@ Other options, specific to Linux and are not supported under FreeBSD are as foll * ``--huge-dir``: The directory where hugetlbfs is mounted. +* ``mbuf-pool-ops-name``: + Pool ops name for mbuf to use. + * ``--file-prefix``: The prefix text used for hugepage filenames. diff --git a/doc/guides/howto/index.rst b/doc/guides/howto/index.rst index 468939b6..b5e8ac68 100644 --- a/doc/guides/howto/index.rst +++ b/doc/guides/howto/index.rst @@ -38,6 +38,7 @@ HowTo Guides lm_bond_virtio_sriov lm_virtio_vhost_user flow_bifurcation + rte_flow pvp_reference_benchmark vfd virtio_user_for_container_networking diff --git a/doc/guides/howto/rte_flow.rst b/doc/guides/howto/rte_flow.rst new file mode 100644 index 00000000..4a27c995 --- /dev/null +++ b/doc/guides/howto/rte_flow.rst @@ -0,0 +1,333 @@ +.. BSD LICENSE + Copyright(c) 2017 Mellanox Corporation. All rights reserved. + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of Mellanox Corporation nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + +Generic flow API - examples +=========================== + +This document demonstrates some concrete examples for programming flow rules +with the ``rte_flow`` APIs. + +* Detail of the rte_flow APIs can be found in the following link: + :ref:`Generic flow API <Generic_flow_API>` . + +* Details of the TestPMD commands to set the flow rules can be found in the + following link: :ref:`TestPMD Flow rules <testpmd_rte_flow>` + +Simple IPv4 drop +---------------- + +Description +~~~~~~~~~~~ + +In this example we will create a simple rule that drops packets whose IPv4 +destination equals 192.168.3.2. This code is equivalent to the following +testpmd command (wrapped for clarity):: + + tpmd> flow create 0 ingress pattern eth / vlan / + ipv4 dst is 192.168.3.2 / end actions drop / end + +Code +~~~~ + +.. code-block:: c + + /* create the attribute structure */ + struct rte_flow_attr attr = {.ingress = 1}; + struct rte_flow_item pattern[MAX_PATTERN_IN_FLOW]; + struct rte_flow_action actions[MAX_ACTIONS_IN_FLOW]; + struct rte_flow_item_etc eth; + struct rte_flow_item_vlan vlan; + struct rte_flow_item_ipv4 ipv4; + struct rte_flow *flow; + struct rte_flow_error error; + + /* setting the eth to pass all packets */ + pattern[0].type = RTE_FLOW_ITEM_TYPE_ETH; + pattern[0].spec = ð + + /* set the vlan to pas all packets */ + pattern[1] = RTE_FLOW_ITEM_TYPE_VLAN; + pattern[1].spec = &vlan; + + /* set the dst ipv4 packet to the required value */ + ipv4.hdr.dst_addr = htonl(0xc0a80302); + pattern[2].type = RTE_FLOW_ITEM_TYPE_IPV4; + pattern[2].spec = &ipv4; + + /* end the pattern array */ + pattern[3].type = RTE_FLOW_ITEM)TYPE_END; + + /* create the drop action */ + actions[0].type = RTE_FLOW_ACTION_TYPE_DROP; + actions[1].type = RTE_FLOW_ACTION_TYPE_END; + + /* validate and create the flow rule */ + if (!rte_flow_validate(port_id, &attr, pattern, actions, &error) + flow = rte_flow_create(port_id, &attr, pattern, actions, &error) + +Output +~~~~~~ + +Terminal 1: running sample app with the flow rule disabled:: + + ./filter-program disable + [waiting for packets] + +Terminal 2: running scapy:: + + $scapy + welcome to Scapy + >> sendp(Ether()/Dot1Q()/IP(src='176.80.50.4', dst='192.168.3.1'), \ + iface='some interface', count=1) + >> sendp(Ether()/Dot1Q()/IP(src='176.80.50.5', dst='192.168.3.2'), \ + iface='some interface', count=1) + +Terminal 1: output log:: + + received packet with src ip = 176.80.50.4 + received packet with src ip = 176.80.50.5 + +Terminal 1: running sample the app flow rule enabled:: + + ./filter-program enabled + [waiting for packets] + +Terminal 2: running scapy:: + + $scapy + welcome to Scapy + >> sendp(Ether()/Dot1Q()/IP(src='176.80.50.4', dst='192.168.3.1'), \ + iface='some interface', count=1) + >> sendp(Ether()/Dot1Q()/IP(src='176.80.50.5', dst ='192.168.3.2'), \ + iface='some interface', count=1) + +Terminal 1: output log:: + + received packet with src ip = 176.80.50.4 + +Range IPv4 drop +---------------- + +Description +~~~~~~~~~~~ + +In this example we will create a simple rule that drops packets whose IPv4 +destination is in the range 192.168.3.0 to 192.168.3.255. This is done using +a mask. + +This code is equivalent to the following testpmd command (wrapped for +clarity):: + + tpmd> flow create 0 ingress pattern eth / vlan / + ipv4 dst spec 192.168.3.0 dst mask 255.255.255.0 / + end actions drop / end + +Code +~~~~ + +.. code-block:: c + + struct rte_flow_attr attr = {.ingress = 1}; + struct rte_flow_item pattern[MAX_PATTERN_IN_FLOW]; + struct rte_flow_action actions[MAX_ACTIONS_IN_FLOW]; + struct rte_flow_item_etc eth; + struct rte_flow_item_vlan vlan; + struct rte_flow_item_ipv4 ipv4; + struct rte_flow_item_ipv4 ipv4_mask; + struct rte_flow *flow; + struct rte_flow_error error; + + /* setting the eth to pass all packets */ + pattern[0].type = RTE_FLOW_ITEM_TYPE_ETH; + pattern[0].spec = ð + + /* set the vlan to pas all packets */ + pattern[1] = RTE_FLOW_ITEM_TYPE_VLAN; + pattern[1].spec = &vlan; + + /* set the dst ipv4 packet to the required value */ + ipv4.hdr.dst_addr = htonl(0xc0a80300); + ipv4_mask.hdr.dst_addr = htonl(0xffffff00); + pattern[2].type = RTE_FLOW_ITEM_TYPE_IPV4; + pattern[2].spec = &ipv4; + pattern[2].mask = &ipv4_mask; + + /* end the pattern array */ + pattern[3].type = RTE_FLOW_ITEM)TYPE_END; + + /* create the drop action */ + actions[0].type = RTE_FLOW_ACTION_TYPE_DROP; + actions[1].type = RTE_FLOW_ACTION_TYPE_END; + + /* validate and create the flow rule */ + if (!rte_flow_validate(port_id, &attr, pattern, actions, &error) + flow = rte_flow_create(port_id, &attr, pattern, actions, &error) + +Output +~~~~~~ + +Terminal 1: running sample app flow rule disabled:: + + ./filter-program disable + [waiting for packets] + +Terminal 2: running scapy:: + + $scapy + welcome to Scapy + >> sendp(Ether()/Dot1Q()/IP(src='176.80.50.4', dst='192.168.3.1'), \ + iface='some interface', count=1) + >> sendp(Ether()/Dot1Q()/IP(src='176.80.50.5', dst='192.168.3.2'), \ + iface='some interface', count=1) + >> sendp(Ether()/Dot1Q()/IP(src='176.80.50.6', dst='192.168.5.2'), \ + iface='some interface', count=1) + +Terminal 1: output log:: + + received packet with src ip = 176.80.50.4 + received packet with src ip = 176.80.50.5 + received packet with src ip = 176.80.50.6 + +Terminal 1: running sample app flow rule enabled:: + + ./filter-program enabled + [waiting for packets] + +Terminal 2: running scapy:: + + $scapy + welcome to Scapy + >> sendp(Ether()/Dot1Q()/IP(src='176.80.50.4', dst='192.168.3.1'), \ + iface='some interface', count=1) + >> sendp(Ether()/Dot1Q()/IP(src='176.80.50.5', dst='192.168.3.2'), \ + iface='some interface', count=1) + >> sendp(Ether()/Dot1Q()/IP(src='176.80.50.6', dst='192.168.5.2'), \ + iface='some interface', count=1) + +Terminal 1: output log:: + + received packet with src ip = 176.80.50.6 + +Send vlan to queue +------------------ + +Description +~~~~~~~~~~~ + +In this example we will create a rule that routes all vlan id 123 to queue 3. + +This code is equivalent to the following testpmd command (wrapped for +clarity):: + + tpmd> flow create 0 ingress pattern eth / vlan vid spec 123 / + end actions queue index 3 / end + +Code +~~~~ + +.. code-block:: c + + struct rte_flow_attr attr = {.ingress = 1}; + struct rte_flow_item pattern[MAX_PATTERN_IN_FLOW]; + struct rte_flow_action actions[MAX_ACTIONS_IN_FLOW]; + struct rte_flow_item_etc eth; + struct rte_flow_item_vlan vlan; + struct rte_flow_action_queue queue = { .index = 3 }; + struct rte_flow *flow; + struct rte_flow_error error; + + /* setting the eth to pass all packets */ + pattern[0].type = RTE_FLOW_ITEM_TYPE_ETH; + pattern[0].spec = ð + + /* set the vlan to pas all packets */ + vlan.vid = 123; + pattern[1] = RTE_FLOW_ITEM_TYPE_VLAN; + pattern[1].spec = &vlan; + + /* end the pattern array */ + pattern[2].type = RTE_FLOW_ITEM)TYPE_END; + + /* create the drop action */ + actions[0].type = RTE_FLOW_ACTION_TYPE_QUEUE; + actions[0].conf = &queue + actions[1].type = RTE_FLOW_ACTION_TYPE_END; + + /* validate and create the flow rule */ + if (!rte_flow_validate(port_id, &attr, pattern, actions, &error) + flow = rte_flow_create(port_id, &attr, pattern, actions, &error) + +Output +~~~~~~ + +Terminal 1: running sample app flow rule disabled:: + + ./filter-program disable + [waiting for packets] + +Terminal 2: running scapy:: + + $scapy + welcome to Scapy + >> sendp(Ether()/Dot1Q(vlan=123)/IP(src='176.80.50.4', dst='192.168.3.1'), \ + iface='some interface', count=1) + >> sendp(Ether()/Dot1Q(vlan=50)/IP(src='176.80.50.5', dst='192.168.3.2'), \ + iface='some interface', count=1) + >> sendp(Ether()/Dot1Q(vlan=123)/IP(src='176.80.50.6', dst='192.168.5.2'), \ + iface='some interface', count=1) + +Terminal 1: output log:: + + received packet with src ip = 176.80.50.4 sent to queue 2 + received packet with src ip = 176.80.50.5 sent to queue 1 + received packet with src ip = 176.80.50.6 sent to queue 0 + +Terminal 1: running sample app flow rule enabled:: + + ./filter-program enabled + [waiting for packets] + +Terminal 2: running scapy:: + + $scapy + welcome to Scapy + >> sendp(Ether()/Dot1Q(vlan=123)/IP(src='176.80.50.4', dst='192.168.3.1'), \ + iface='some interface', count=1) + >> sendp(Ether()/Dot1Q(vlan=50)/IP(src='176.80.50.5', dst='192.168.3.2'), \ + iface='some interface', count=1) + >> sendp(Ether()/Dot1Q(vlan=123)/IP(src='176.80.50.6', dst='192.168.5.2'), \ + iface='some interface', count=1) + +Terminal 1: output log:: + + received packet with src ip = 176.80.50.4 sent to queue 3 + received packet with src ip = 176.80.50.5 sent to queue 1 + received packet with src ip = 176.80.50.6 sent to queue 3 diff --git a/doc/guides/index.rst b/doc/guides/index.rst index 63716b09..5b6eb7ec 100644 --- a/doc/guides/index.rst +++ b/doc/guides/index.rst @@ -44,7 +44,6 @@ DPDK documentation nics/index cryptodevs/index eventdevs/index - xen/index contributing/index rel_notes/index faq/index diff --git a/doc/guides/linux_gsg/build_dpdk.rst b/doc/guides/linux_gsg/build_dpdk.rst index e32afd5f..5452041f 100644 --- a/doc/guides/linux_gsg/build_dpdk.rst +++ b/doc/guides/linux_gsg/build_dpdk.rst @@ -69,9 +69,9 @@ The format of a DPDK target is:: where: -* ``ARCH`` can be: ``i686``, ``x86_64``, ``ppc_64`` +* ``ARCH`` can be: ``i686``, ``x86_64``, ``ppc_64``, ``arm64`` -* ``MACHINE`` can be: ``native``, ``power8`` +* ``MACHINE`` can be: ``native``, ``power8``, ``armv8a`` * ``EXECENV`` can be: ``linuxapp``, ``bsdapp`` diff --git a/doc/guides/linux_gsg/build_sample_apps.rst b/doc/guides/linux_gsg/build_sample_apps.rst index 0cc5fd17..594cbc59 100644 --- a/doc/guides/linux_gsg/build_sample_apps.rst +++ b/doc/guides/linux_gsg/build_sample_apps.rst @@ -116,7 +116,7 @@ The following is the list of options that can be given to the EAL: ./rte-app [-c COREMASK | -l CORELIST] [-n NUM] [-b <domain:bus:devid.func>] \ [--socket-mem=MB,...] [-d LIB.so|DIR] [-m MB] [-r NUM] [-v] [--file-prefix] \ - [--proc-type <primary|secondary|auto>] [-- xen-dom0] + [--proc-type <primary|secondary|auto>] The EAL options are as follows: @@ -157,15 +157,15 @@ The EAL options are as follows: * ``--huge-dir``: The directory where hugetlbfs is mounted. +* ``mbuf-pool-ops-name``: + Pool ops name for mbuf to use. + * ``--file-prefix``: The prefix text used for hugepage filenames. * ``--proc-type``: The type of process instance. -* ``--xen-dom0``: - Support application running on Xen Domain0 without hugetlbfs. - * ``--vmware-tsc-map``: Use VMware TSC map instead of native RDTSC. diff --git a/doc/guides/linux_gsg/sys_reqs.rst b/doc/guides/linux_gsg/sys_reqs.rst index eb8442c6..3e7fe637 100644 --- a/doc/guides/linux_gsg/sys_reqs.rst +++ b/doc/guides/linux_gsg/sys_reqs.rst @@ -228,56 +228,3 @@ The mount point can be made permanent across reboots, by adding the following li For 1GB pages, the page size must be specified as a mount option:: nodev /mnt/huge_1GB hugetlbfs pagesize=1GB 0 0 - -Xen Domain0 Support in the Linux Environment -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The existing memory management implementation is based on the Linux kernel hugepage mechanism. -On the Xen hypervisor, hugepage support for DomainU (DomU) Guests means that DPDK applications work as normal for guests. - -However, Domain0 (Dom0) does not support hugepages. -To work around this limitation, a new kernel module rte_dom0_mm is added to facilitate the allocation and mapping of memory via -**IOCTL** (allocation) and **MMAP** (mapping). - -Enabling Xen Dom0 Mode in the DPDK -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -By default, Xen Dom0 mode is disabled in the DPDK build configuration files. -To support Xen Dom0, the CONFIG_RTE_LIBRTE_XEN_DOM0 setting should be changed to “y”, which enables the Xen Dom0 mode at compile time. - -Furthermore, the CONFIG_RTE_EAL_ALLOW_INV_SOCKET_ID setting should also be changed to “y” in the case of the wrong socket ID being received. - -Loading the DPDK rte_dom0_mm Module -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -To run any DPDK application on Xen Dom0, the ``rte_dom0_mm`` module must be loaded into the running kernel with rsv_memsize option. -The module is found in the kmod sub-directory of the DPDK target directory. -This module should be loaded using the insmod command as shown below (assuming that the current directory is the DPDK target directory):: - - sudo insmod kmod/rte_dom0_mm.ko rsv_memsize=X - -The value X cannot be greater than 4096(MB). - -Configuring Memory for DPDK Use -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -After the rte_dom0_mm.ko kernel module has been loaded, the user must configure the memory size for DPDK usage. -This is done by echoing the memory size to a memsize file in the /sys/devices/ directory. -Use the following command (assuming that 2048 MB is required):: - - echo 2048 > /sys/kernel/mm/dom0-mm/memsize-mB/memsize - -The user can also check how much memory has already been used:: - - cat /sys/kernel/mm/dom0-mm/memsize-mB/memsize_rsvd - -Xen Domain0 does not support NUMA configuration, as a result the ``--socket-mem`` command line option is invalid for Xen Domain0. - -.. note:: - - The memsize value cannot be greater than the rsv_memsize value. - -Running the DPDK Application on Xen Domain0 -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -To run the DPDK application on Xen Domain0, an extra command line option ``--xen-dom0`` is required. diff --git a/doc/guides/nics/ark.rst b/doc/guides/nics/ark.rst index a7c2590b..6c135eeb 100644 --- a/doc/guides/nics/ark.rst +++ b/doc/guides/nics/ark.rst @@ -146,9 +146,6 @@ Data Path Interface Ingress RX and Egress TX operation is by the nominal DPDK API . The driver supports single-port, multi-queue for both RX and TX. -Refer to ``ark_ethdev.h`` for the list of supported methods to -act upon RX and TX Queues. - Configuration Information ------------------------- diff --git a/doc/guides/nics/dpaa.rst b/doc/guides/nics/dpaa.rst new file mode 100644 index 00000000..d331c053 --- /dev/null +++ b/doc/guides/nics/dpaa.rst @@ -0,0 +1,378 @@ +.. BSD LICENSE + Copyright 2017 NXP. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of NXP nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +DPAA Poll Mode Driver +===================== + +The DPAA NIC PMD (**librte_pmd_dpaa**) provides poll mode driver +support for the inbuilt NIC found in the **NXP DPAA** SoC family. + +More information can be found at `NXP Official Website +<http://www.nxp.com/products/microcontrollers-and-processors/arm-processors/qoriq-arm-processors:QORIQ-ARM>`_. + +NXP DPAA (Data Path Acceleration Architecture - Gen 1) +------------------------------------------------------ + +This section provides an overview of the NXP DPAA architecture +and how it is integrated into the DPDK. + +Contents summary + +- DPAA overview +- DPAA driver architecture overview + +.. _dpaa_overview: + +DPAA Overview +~~~~~~~~~~~~~ + +Reference: `FSL DPAA Architecture <http://www.nxp.com/assets/documents/data/en/white-papers/QORIQDPAAWP.pdf>`_. + +The QorIQ Data Path Acceleration Architecture (DPAA) is a set of hardware +components on specific QorIQ series multicore processors. This architecture +provides the infrastructure to support simplified sharing of networking +interfaces and accelerators by multiple CPU cores, and the accelerators +themselves. + +DPAA includes: + +- Cores +- Network and packet I/O +- Hardware offload accelerators +- Infrastructure required to facilitate flow of packets between the components above + +Infrastructure components are: + +- The Queue Manager (QMan) is a hardware accelerator that manages frame queues. + It allows CPUs and other accelerators connected to the SoC datapath to + enqueue and dequeue ethernet frames, thus providing the infrastructure for + data exchange among CPUs and datapath accelerators. +- The Buffer Manager (BMan) is a hardware buffer pool management block that + allows software and accelerators on the datapath to acquire and release + buffers in order to build frames. + +Hardware accelerators are: + +- SEC - Cryptographic accelerator +- PME - Pattern matching engine + +The Network and packet I/O component: + +- The Frame Manager (FMan) is a key component in the DPAA and makes use of the + DPAA infrastructure (QMan and BMan). FMan is responsible for packet + distribution and policing. Each frame can be parsed, classified and results + may be attached to the frame. This meta data can be used to select + particular QMan queue, which the packet is forwarded to. + + +DPAA DPDK - Poll Mode Driver Overview +------------------------------------- + +This section provides an overview of the drivers for DPAA: + +* Bus driver and associated "DPAA infrastructure" drivers +* Functional object drivers (such as Ethernet). + +Brief description of each driver is provided in layout below as well as +in the following sections. + +.. code-block:: console + + +------------+ + | DPDK DPAA | + | PMD | + +-----+------+ + | + +-----+------+ +---------------+ + : Ethernet :.......| DPDK DPAA | + . . . . . . . . . : (FMAN) : | Mempool driver| + . +---+---+----+ | (BMAN) | + . ^ | +-----+---------+ + . | |<enqueue, . + . | | dequeue> . + . | | . + . +---+---V----+ . + . . . . . . . . . . .: Portal drv : . + . . : : . + . . +-----+------+ . + . . : QMAN : . + . . : Driver : . + +----+------+-------+ +-----+------+ . + | DPDK DPAA Bus | | . + | driver |....................|..................... + | /bus/dpaa | | + +-------------------+ | + | + ========================== HARDWARE =====|======================== + PHY + =========================================|======================== + +In the above representation, solid lines represent components which interface +with DPDK RTE Framework and dotted lines represent DPAA internal components. + +DPAA Bus driver +~~~~~~~~~~~~~~~ + +The DPAA bus driver is a ``rte_bus`` driver which scans the platform like bus. +Key functions include: + +- Scanning and parsing the various objects and adding them to their respective + device list. +- Performing probe for available drivers against each scanned device +- Creating necessary ethernet instance before passing control to the PMD + +DPAA NIC Driver (PMD) +~~~~~~~~~~~~~~~~~~~~~ + +DPAA PMD is traditional DPDK PMD which provides necessary interface between +RTE framework and DPAA internal components/drivers. + +- Once devices have been identified by DPAA Bus, each device is associated + with the PMD +- PMD is responsible for implementing necessary glue layer between RTE APIs + and lower level QMan and FMan blocks. + The Ethernet driver is bound to a FMAN port and implements the interfaces + needed to connect the DPAA network interface to the network stack. + Each FMAN Port corresponds to a DPDK network interface. + + +Features +^^^^^^^^ + + Features of the DPAA PMD are: + + - Multiple queues for TX and RX + - Receive Side Scaling (RSS) + - Packet type information + - Checksum offload + - Promiscuous mode + +DPAA Mempool Driver +~~~~~~~~~~~~~~~~~~~ + +DPAA has a hardware offloaded buffer pool manager, called BMan, or Buffer +Manager. + +- Using standard Mempools operations RTE API, the mempool driver interfaces + with RTE to service each mempool creation, deletion, buffer allocation and + deallocation requests. +- Each FMAN instance has a BMan pool attached to it during initialization. + Each Tx frame can be automatically released by hardware, if allocated from + this pool. + + +Supported DPAA SoCs +------------------- + +- LS1043A/LS1023A +- LS1046A/LS1026A + +Prerequisites +------------- + +There are three main pre-requisities for executing DPAA PMD on a DPAA +compatible board: + +1. **ARM 64 Tool Chain** + + For example, the `*aarch64* Linaro Toolchain <https://releases.linaro.org/components/toolchain/binaries/6.4-2017.08/aarch64-linux-gnu/>`_. + +2. **Linux Kernel** + + It can be obtained from `NXP's Github hosting <https://github.com/qoriq-open-source/linux>`_. + +3. **Rootfile system** + + Any *aarch64* supporting filesystem can be used. For example, + Ubuntu 15.10 (Wily) or 16.04 LTS (Xenial) userland which can be obtained + from `here <http://cdimage.ubuntu.com/ubuntu-base/releases/16.04/release/ubuntu-base-16.04.1-base-arm64.tar.gz>`_. + +4. **FMC Tool** + + Before any DPDK application can be executed, the Frame Manager Configuration + Tool (FMC) need to be executed to set the configurations of the queues. This + includes the queue state, RSS and other policies. + This tool can be obtained from `NXP (Freescale) Public Git Repository <https://github.com/qoriq-open-source/fmc>`_. + + This tool needs configuration files which are available in the + :ref:`DPDK Extra Scripts <extra_scripts>`, described below for DPDK usages. + +As an alternative method, DPAA PMD can also be executed using images provided +as part of SDK from NXP. The SDK includes all the above prerequisites necessary +to bring up a DPAA board. + +The following dependencies are not part of DPDK and must be installed +separately: + +- **NXP Linux SDK** + + NXP Linux software development kit (SDK) includes support for family + of QorIQ® ARM-Architecture-based system on chip (SoC) processors + and corresponding boards. + + It includes the Linux board support packages (BSPs) for NXP SoCs, + a fully operational tool chain, kernel and board specific modules. + + SDK and related information can be obtained from: `NXP QorIQ SDK <http://www.nxp.com/products/software-and-tools/run-time-software/linux-sdk/linux-sdk-for-qoriq-processors:SDKLINUX>`_. + + +.. _extra_scripts: + +- **DPDK Extra Scripts** + + DPAA based resources can be configured easily with the help of ready scripts + as provided in the DPDK Extra repository. + + `DPDK Extras Scripts <https://github.com/qoriq-open-source/dpdk-extras>`_. + +Currently supported by DPDK: + +- NXP SDK **2.0+**. +- Supported architectures: **arm64 LE**. + +- Follow the DPDK :ref:`Getting Started Guide for Linux <linux_gsg>` + to setup the basic DPDK environment. + +.. note:: + + Some part of dpaa bus code (qbman and fman - library) routines are + dual licensed (BSD & GPLv2), however they are used as BSD in DPDK in userspace. + +Pre-Installation Configuration +------------------------------ + +Config File Options +~~~~~~~~~~~~~~~~~~~ + +The following options can be modified in the ``config`` file. +Please note that enabling debugging options may affect system performance. + +- ``CONFIG_RTE_LIBRTE_DPAA_BUS`` (default ``n``) + + By default it is enabled only for defconfig_arm64-dpaa-* config. + Toggle compilation of the ``librte_bus_dpaa`` driver. + +- ``CONFIG_RTE_LIBRTE_DPAA_PMD`` (default ``n``) + + By default it is enabled only for defconfig_arm64-dpaa-* config. + Toggle compilation of the ``librte_pmd_dpaa`` driver. + +- ``CONFIG_RTE_LIBRTE_DPAA_DEBUG_DRIVER`` (default ``n``) + + Toggles display of bus configurations and enables a debugging queue + to fetch error (Rx/Tx) packets to driver. By default, packets with errors + (like wrong checksum) are dropped by the hardware. + +- ``CONFIG_RTE_LIBRTE_DPAA_HWDEBUG`` (default ``n``) + + Enables debugging of the Queue and Buffer Manager layer which interacts + with the DPAA hardware. + +- ``CONFIG_RTE_MBUF_DEFAULT_MEMPOOL_OPS`` (default ``dpaa``) + + This is not a DPAA specific configuration - it is a generic RTE config. + For optimal performance and hardware utilization, it is expected that DPAA + Mempool driver is used for mempools. For that, this configuration needs to + enabled. + +Environment Variables +~~~~~~~~~~~~~~~~~~~~~ + +DPAA drivers uses the following environment variables to configure its +state during application initialization: + +- ``DPAA_NUM_RX_QUEUES`` (default 1) + + This defines the number of Rx queues configured for an application, per + port. Hardware would distribute across these many number of queues on Rx + of packets. + In case the application is configured to use lesser number of queues than + configured above, it might result in packet loss (because of distribution). + + +Driver compilation and testing +------------------------------ + +Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>` +for details. + +#. Running testpmd: + + Follow instructions available in the document + :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>` + to run testpmd. + + Example output: + + .. code-block:: console + + ./arm64-dpaa-linuxapp-gcc/testpmd -c 0xff -n 1 \ + -- -i --portmask=0x3 --nb-cores=1 --no-flush-rx + + ..... + EAL: Registered [pci] bus. + EAL: Registered [dpaa] bus. + EAL: Detected 4 lcore(s) + ..... + EAL: dpaa: Bus scan completed + ..... + Configuring Port 0 (socket 0) + Port 0: 00:00:00:00:00:01 + Configuring Port 1 (socket 0) + Port 1: 00:00:00:00:00:02 + ..... + Checking link statuses... + Port 0 Link Up - speed 10000 Mbps - full-duplex + Port 1 Link Up - speed 10000 Mbps - full-duplex + Done + testpmd> + +Limitations +----------- + +Platform Requirement +~~~~~~~~~~~~~~~~~~~~ + +DPAA drivers for DPDK can only work on NXP SoCs as listed in the +``Supported DPAA SoCs``. + +Maximum packet length +~~~~~~~~~~~~~~~~~~~~~ + +The DPAA SoC family support a maximum of a 10240 jumbo frame. The value +is fixed and cannot be changed. So, even when the ``rxmode.max_rx_pkt_len`` +member of ``struct rte_eth_conf`` is set to a value lower than 10240, frames +up to 10240 bytes can still reach the host interface. + +Multiprocess Support +~~~~~~~~~~~~~~~~~~~~ + +Current version of DPAA driver doesn't support multi-process applications +where I/O is performed using secondary processes. This feature would be +implemented in subsequent versions. diff --git a/doc/guides/nics/dpaa2.rst b/doc/guides/nics/dpaa2.rst index d6b1f21e..85bc0488 100644 --- a/doc/guides/nics/dpaa2.rst +++ b/doc/guides/nics/dpaa2.rst @@ -448,7 +448,7 @@ compatible board: 1. **ARM 64 Tool Chain** - For example, the `*aarch64* Linaro Toolchain <https://releases.linaro.org/components/toolchain/binaries/4.9-2017.01/aarch64-linux-gnu>`_. + For example, the `*aarch64* Linaro Toolchain <https://releases.linaro.org/components/toolchain/binaries/6.4-2017.08/aarch64-linux-gnu/>`_. 2. **Linux Kernel** @@ -487,8 +487,8 @@ separately: Currently supported by DPDK: -- NXP SDK **2.0+**. -- MC Firmware version **10.0.0** and higher. +- NXP SDK **17.08+**. +- MC Firmware version **10.3.1** and higher. - Supported architectures: **arm64 LE**. - Follow the DPDK :ref:`Getting Started Guide for Linux <linux_gsg>` to setup the basic DPDK environment. @@ -496,7 +496,7 @@ Currently supported by DPDK: .. note:: Some part of fslmc bus code (mc flib - object library) routines are - dual licensed (BSD & GPLv2). + dual licensed (BSD & GPLv2), however they are used as BSD in DPDK in userspace. Pre-Installation Configuration ------------------------------ diff --git a/doc/guides/nics/fail_safe.rst b/doc/guides/nics/fail_safe.rst index 34ecae2d..c4e3d2e8 100644 --- a/doc/guides/nics/fail_safe.rst +++ b/doc/guides/nics/fail_safe.rst @@ -136,7 +136,7 @@ This section shows some example of using **testpmd** with a fail-safe PMD. .. code-block:: console $RTE_TARGET/build/app/testpmd -c 0xff -n 4 \ - --vdev 'net_failsafe0,mac=de:ad:be:ef:01:02,dev(84:00.0),dev(net_ring0)' + --vdev 'net_failsafe0,mac=de:ad:be:ef:01:02,dev(84:00.0),dev(net_ring0)' \ -b 84:00.0 -b 00:04.0 -- -i If the slave device being used is not blacklisted, it will be probed by the @@ -150,7 +150,7 @@ This section shows some example of using **testpmd** with a fail-safe PMD. .. code-block:: console $RTE_TARGET/build/app/testpmd -c 0xff -n 4 \ - --vdev 'net_failsafe0,mac=de:ad:be:ef:01:02,dev(84:00.0),dev(net_ring0)' + --vdev 'net_failsafe0,mac=de:ad:be:ef:01:02,dev(84:00.0),dev(net_ring0)' \ -w 81:00.0 -- -i #. Start testpmd using a flexible device definition diff --git a/doc/guides/nics/features.rst b/doc/guides/nics/features.rst index 37ffbc68..bfeae80e 100644 --- a/doc/guides/nics/features.rst +++ b/doc/guides/nics/features.rst @@ -131,7 +131,8 @@ Lock-free Tx queue If a PMD advertises DEV_TX_OFFLOAD_MT_LOCKFREE capable, multiple threads can invoke rte_eth_tx_burst() concurrently on the same Tx queue without SW lock. -* **[provides] rte_eth_dev_info**: ``tx_offload_capa:DEV_TX_OFFLOAD_MT_LOCKFREE``. +* **[uses] rte_eth_txconf,rte_eth_txmode**: ``offloads:DEV_TX_OFFLOAD_MT_LOCKFREE``. +* **[provides] rte_eth_dev_info**: ``tx_offload_capa,tx_queue_offload_capa:DEV_TX_OFFLOAD_MT_LOCKFREE``. * **[related] API**: ``rte_eth_tx_burst()``. @@ -179,7 +180,7 @@ Jumbo frame Supports Rx jumbo frames. -* **[uses] user config**: ``dev_conf.rxmode.jumbo_frame``, +* **[uses] rte_eth_rxconf,rte_eth_rxmode**: ``offloads:DEV_RX_OFFLOAD_JUMBO_FRAME``. ``dev_conf.rxmode.max_rx_pkt_len``. * **[related] rte_eth_dev_info**: ``max_rx_pktlen``. * **[related] API**: ``rte_eth_dev_set_mtu()``. @@ -192,7 +193,7 @@ Scattered Rx Supports receiving segmented mbufs. -* **[uses] user config**: ``dev_conf.rxmode.enable_scatter``. +* **[uses] rte_eth_rxconf,rte_eth_rxmode**: ``offloads:DEV_RX_OFFLOAD_SCATTER``. * **[implements] datapath**: ``Scattered Rx function``. * **[implements] rte_eth_dev_data**: ``scattered_rx``. * **[provides] eth_dev_ops**: ``rxq_info_get:scattered_rx``. @@ -206,11 +207,11 @@ LRO Supports Large Receive Offload. -* **[uses] user config**: ``dev_conf.rxmode.enable_lro``. +* **[uses] rte_eth_rxconf,rte_eth_rxmode**: ``offloads:DEV_RX_OFFLOAD_TCP_LRO``. * **[implements] datapath**: ``LRO functionality``. * **[implements] rte_eth_dev_data**: ``lro``. * **[provides] mbuf**: ``mbuf.ol_flags:PKT_RX_LRO``, ``mbuf.tso_segsz``. -* **[provides] rte_eth_dev_info**: ``rx_offload_capa:DEV_RX_OFFLOAD_TCP_LRO``. +* **[provides] rte_eth_dev_info**: ``rx_offload_capa,rx_queue_offload_capa:DEV_RX_OFFLOAD_TCP_LRO``. .. _nic_features_tso: @@ -220,11 +221,12 @@ TSO Supports TCP Segmentation Offloading. +* **[uses] rte_eth_txconf,rte_eth_txmode**: ``offloads:DEV_TX_OFFLOAD_TCP_TSO``. * **[uses] rte_eth_desc_lim**: ``nb_seg_max``, ``nb_mtu_seg_max``. * **[uses] mbuf**: ``mbuf.ol_flags:PKT_TX_TCP_SEG``. * **[uses] mbuf**: ``mbuf.tso_segsz``, ``mbuf.l2_len``, ``mbuf.l3_len``, ``mbuf.l4_len``. * **[implements] datapath**: ``TSO functionality``. -* **[provides] rte_eth_dev_info**: ``tx_offload_capa:DEV_TX_OFFLOAD_TCP_TSO,DEV_TX_OFFLOAD_UDP_TSO``. +* **[provides] rte_eth_dev_info**: ``tx_offload_capa,tx_queue_offload_capa:DEV_TX_OFFLOAD_TCP_TSO,DEV_TX_OFFLOAD_UDP_TSO``. .. _nic_features_promiscuous_mode: @@ -363,7 +365,7 @@ VLAN filter Supports filtering of a VLAN Tag identifier. -* **[uses] user config**: ``dev_conf.rxmode.hw_vlan_filter``. +* **[uses] rte_eth_rxconf,rte_eth_rxmode**: ``offloads:DEV_RX_OFFLOAD_VLAN_FILTER``. * **[implements] eth_dev_ops**: ``vlan_filter_set``. * **[related] API**: ``rte_eth_dev_vlan_filter()``. @@ -499,7 +501,7 @@ CRC offload Supports CRC stripping by hardware. -* **[uses] user config**: ``dev_conf.rxmode.hw_strip_crc``. +* **[uses] rte_eth_rxconf,rte_eth_rxmode**: ``offloads:DEV_RX_OFFLOAD_CRC_STRIP``. .. _nic_features_vlan_offload: @@ -509,12 +511,12 @@ VLAN offload Supports VLAN offload to hardware. -* **[uses] user config**: ``dev_conf.rxmode.hw_vlan_strip``, - ``dev_conf.rxmode.hw_vlan_filter``, ``dev_conf.rxmode.hw_vlan_extend``. +* **[uses] rte_eth_rxconf,rte_eth_rxmode**: ``offloads:DEV_RX_OFFLOAD_VLAN_STRIP,DEV_RX_OFFLOAD_VLAN_FILTER,DEV_RX_OFFLOAD_VLAN_EXTEND``. +* **[uses] rte_eth_txconf,rte_eth_txmode**: ``offloads:DEV_TX_OFFLOAD_VLAN_INSERT``. * **[implements] eth_dev_ops**: ``vlan_offload_set``. * **[provides] mbuf**: ``mbuf.ol_flags:PKT_RX_VLAN_STRIPPED``, ``mbuf.vlan_tci``. -* **[provides] rte_eth_dev_info**: ``rx_offload_capa:DEV_RX_OFFLOAD_VLAN_STRIP``, - ``tx_offload_capa:DEV_TX_OFFLOAD_VLAN_INSERT``. +* **[provides] rte_eth_dev_info**: ``rx_offload_capa,rx_queue_offload_capa:DEV_RX_OFFLOAD_VLAN_STRIP``, + ``tx_offload_capa,tx_queue_offload_capa:DEV_TX_OFFLOAD_VLAN_INSERT``. * **[related] API**: ``rte_eth_dev_set_vlan_offload()``, ``rte_eth_dev_get_vlan_offload()``. @@ -526,11 +528,13 @@ QinQ offload Supports QinQ (queue in queue) offload. +* **[uses] rte_eth_rxconf,rte_eth_rxmode**: ``offloads:DEV_RX_OFFLOAD_QINQ_STRIP``. +* **[uses] rte_eth_txconf,rte_eth_txmode**: ``offloads:DEV_TX_OFFLOAD_QINQ_INSERT``. * **[uses] mbuf**: ``mbuf.ol_flags:PKT_TX_QINQ_PKT``. * **[provides] mbuf**: ``mbuf.ol_flags:PKT_RX_QINQ_STRIPPED``, ``mbuf.vlan_tci``, ``mbuf.vlan_tci_outer``. -* **[provides] rte_eth_dev_info**: ``rx_offload_capa:DEV_RX_OFFLOAD_QINQ_STRIP``, - ``tx_offload_capa:DEV_TX_OFFLOAD_QINQ_INSERT``. +* **[provides] rte_eth_dev_info**: ``rx_offload_capa,rx_queue_offload_capa:DEV_RX_OFFLOAD_QINQ_STRIP``, + ``tx_offload_capa,tx_queue_offload_capa:DEV_TX_OFFLOAD_QINQ_INSERT``. .. _nic_features_l3_checksum_offload: @@ -540,14 +544,15 @@ L3 checksum offload Supports L3 checksum offload. -* **[uses] user config**: ``dev_conf.rxmode.hw_ip_checksum``. +* **[uses] rte_eth_rxconf,rte_eth_rxmode**: ``offloads:DEV_RX_OFFLOAD_IPV4_CKSUM``. +* **[uses] rte_eth_txconf,rte_eth_txmode**: ``offloads:DEV_TX_OFFLOAD_IPV4_CKSUM``. * **[uses] mbuf**: ``mbuf.ol_flags:PKT_TX_IP_CKSUM``, ``mbuf.ol_flags:PKT_TX_IPV4`` | ``PKT_TX_IPV6``. * **[provides] mbuf**: ``mbuf.ol_flags:PKT_RX_IP_CKSUM_UNKNOWN`` | ``PKT_RX_IP_CKSUM_BAD`` | ``PKT_RX_IP_CKSUM_GOOD`` | ``PKT_RX_IP_CKSUM_NONE``. -* **[provides] rte_eth_dev_info**: ``rx_offload_capa:DEV_RX_OFFLOAD_IPV4_CKSUM``, - ``tx_offload_capa:DEV_TX_OFFLOAD_IPV4_CKSUM``. +* **[provides] rte_eth_dev_info**: ``rx_offload_capa,rx_queue_offload_capa:DEV_RX_OFFLOAD_IPV4_CKSUM``, + ``tx_offload_capa,tx_queue_offload_capa:DEV_TX_OFFLOAD_IPV4_CKSUM``. .. _nic_features_l4_checksum_offload: @@ -557,15 +562,29 @@ L4 checksum offload Supports L4 checksum offload. +* **[uses] rte_eth_rxconf,rte_eth_rxmode**: ``offloads:DEV_RX_OFFLOAD_UDP_CKSUM,DEV_RX_OFFLOAD_TCP_CKSUM``. +* **[uses] rte_eth_txconf,rte_eth_txmode**: ``offloads:DEV_TX_OFFLOAD_UDP_CKSUM,DEV_TX_OFFLOAD_TCP_CKSUM,DEV_TX_OFFLOAD_SCTP_CKSUM``. +* **[uses] user config**: ``dev_conf.rxmode.hw_ip_checksum``. * **[uses] mbuf**: ``mbuf.ol_flags:PKT_TX_IPV4`` | ``PKT_TX_IPV6``, ``mbuf.ol_flags:PKT_TX_L4_NO_CKSUM`` | ``PKT_TX_TCP_CKSUM`` | ``PKT_TX_SCTP_CKSUM`` | ``PKT_TX_UDP_CKSUM``. * **[provides] mbuf**: ``mbuf.ol_flags:PKT_RX_L4_CKSUM_UNKNOWN`` | ``PKT_RX_L4_CKSUM_BAD`` | ``PKT_RX_L4_CKSUM_GOOD`` | ``PKT_RX_L4_CKSUM_NONE``. -* **[provides] rte_eth_dev_info**: ``rx_offload_capa:DEV_RX_OFFLOAD_UDP_CKSUM,DEV_RX_OFFLOAD_TCP_CKSUM``, - ``tx_offload_capa:DEV_TX_OFFLOAD_UDP_CKSUM,DEV_TX_OFFLOAD_TCP_CKSUM,DEV_TX_OFFLOAD_SCTP_CKSUM``. +* **[provides] rte_eth_dev_info**: ``rx_offload_capa,rx_queue_offload_capa:DEV_RX_OFFLOAD_UDP_CKSUM,DEV_RX_OFFLOAD_TCP_CKSUM``, + ``tx_offload_capa,tx_queue_offload_capa:DEV_TX_OFFLOAD_UDP_CKSUM,DEV_TX_OFFLOAD_TCP_CKSUM,DEV_TX_OFFLOAD_SCTP_CKSUM``. + +.. _nic_features_hw_timestamp: + +Timestamp offload +----------------- +Supports Timestamp. + +* **[uses] rte_eth_rxconf,rte_eth_rxmode**: ``offloads:DEV_RX_OFFLOAD_TIMESTAMP``. +* **[provides] mbuf**: ``mbuf.ol_flags:PKT_RX_TIMESTAMP``. +* **[provides] mbuf**: ``mbuf.timestamp``. +* **[provides] rte_eth_dev_info**: ``rx_offload_capa,rx_queue_offload_capa: DEV_RX_OFFLOAD_TIMESTAMP``. .. _nic_features_macsec_offload: @@ -574,9 +593,11 @@ MACsec offload Supports MACsec. +* **[uses] rte_eth_rxconf,rte_eth_rxmode**: ``offloads:DEV_RX_OFFLOAD_MACSEC_STRIP``. +* **[uses] rte_eth_txconf,rte_eth_txmode**: ``offloads:DEV_TX_OFFLOAD_MACSEC_INSERT``. * **[uses] mbuf**: ``mbuf.ol_flags:PKT_TX_MACSEC``. -* **[provides] rte_eth_dev_info**: ``rx_offload_capa:DEV_RX_OFFLOAD_MACSEC_STRIP``, - ``tx_offload_capa:DEV_TX_OFFLOAD_MACSEC_INSERT``. +* **[provides] rte_eth_dev_info**: ``rx_offload_capa,rx_queue_offload_capa:DEV_RX_OFFLOAD_MACSEC_STRIP``, + ``tx_offload_capa,tx_queue_offload_capa:DEV_TX_OFFLOAD_MACSEC_INSERT``. .. _nic_features_inner_l3_checksum: @@ -586,14 +607,16 @@ Inner L3 checksum Supports inner packet L3 checksum. +* **[uses] rte_eth_rxconf,rte_eth_rxmode**: ``offloads:DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM``. +* **[uses] rte_eth_txconf,rte_eth_txmode**: ``offloads:DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM``. * **[uses] mbuf**: ``mbuf.ol_flags:PKT_TX_IP_CKSUM``, ``mbuf.ol_flags:PKT_TX_IPV4`` | ``PKT_TX_IPV6``, ``mbuf.ol_flags:PKT_TX_OUTER_IP_CKSUM``, ``mbuf.ol_flags:PKT_TX_OUTER_IPV4`` | ``PKT_TX_OUTER_IPV6``. * **[uses] mbuf**: ``mbuf.outer_l2_len``, ``mbuf.outer_l3_len``. * **[provides] mbuf**: ``mbuf.ol_flags:PKT_RX_EIP_CKSUM_BAD``. -* **[provides] rte_eth_dev_info**: ``rx_offload_capa:DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM``, - ``tx_offload_capa:DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM``. +* **[provides] rte_eth_dev_info**: ``rx_offload_capa,rx_queue_offload_capa:DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM``, + ``tx_offload_capa,tx_queue_offload_capa:DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM``. .. _nic_features_inner_l4_checksum: @@ -617,6 +640,15 @@ Supports packet type parsing and returns a list of supported types. .. _nic_features_timesync: +Mbuf fast free +-------------- + +Supports optimization for fast release of mbufs following successful Tx. +Requires that per queue, all mbufs come from the same mempool and has refcnt = 1. + +* **[uses] rte_eth_txconf,rte_eth_txmode**: ``offloads:DEV_TX_OFFLOAD_MBUF_FAST_FREE``. +* **[provides] rte_eth_dev_info**: ``tx_offload_capa,tx_queue_offload_capa:DEV_TX_OFFLOAD_MBUF_FAST_FREE``. + Timesync -------- diff --git a/doc/guides/nics/features/bnxt.ini b/doc/guides/nics/features/bnxt.ini index 119132e1..ef45dd76 100644 --- a/doc/guides/nics/features/bnxt.ini +++ b/doc/guides/nics/features/bnxt.ini @@ -5,21 +5,38 @@ ; [Features] Link status = Y +Rx interrupt = Y Queue start/stop = Y MTU update = Y Jumbo frame = Y LRO = Y +TSO = Y Promiscuous mode = Y Allmulticast mode = Y Unicast MAC filter = Y Multicast MAC filter = Y +RSS hash = Y +RSS key update = Y RSS reta update = Y +VMDq = Y SR-IOV = Y VLAN filter = Y +Ethertype filter = Y +N-tuple filter = Y +Flow control = Y +Flow API = Y +L3 checksum offload = Y +L4 checksum offload = Y +Inner L3 checksum = Y +Inner L4 checksum = Y +Packet type parsing = Y VLAN offload = Y +Rx descriptor status = Y +Tx descriptor status = Y Basic stats = Y Extended stats = Y FW version = Y +EEPROM dump = Y LED = Y Linux UIO = Y Linux VFIO = Y diff --git a/doc/guides/nics/features/default.ini b/doc/guides/nics/features/default.ini index 54243069..dc527ddf 100644 --- a/doc/guides/nics/features/default.ini +++ b/doc/guides/nics/features/default.ini @@ -48,6 +48,7 @@ VLAN offload = QinQ offload = L3 checksum offload = L4 checksum offload = +Timestamp offload = MACsec offload = Inner L3 checksum = Inner L4 checksum = @@ -75,3 +76,4 @@ x86-64 = Usage doc = Design doc = Perf doc = +Mbuf fast free = diff --git a/doc/guides/nics/features/dpaa.ini b/doc/guides/nics/features/dpaa.ini new file mode 100644 index 00000000..24cfd856 --- /dev/null +++ b/doc/guides/nics/features/dpaa.ini @@ -0,0 +1,24 @@ +; +; Supported features of the 'dpaa' network poll mode driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +Speed capabilities = P +Link status = Y +Jumbo frame = Y +MTU update = Y +Scattered Rx = Y +Promiscuous mode = Y +Allmulticast mode = Y +Unicast MAC filter = Y +RSS hash = Y +Flow control = Y +L3 checksum offload = Y +L4 checksum offload = Y +Packet type parsing = Y +Basic stats = Y +Extended stats = Y +FW version = Y +ARMv8 = Y +Usage doc = Y diff --git a/doc/guides/nics/features/dpaa2.ini b/doc/guides/nics/features/dpaa2.ini index 146e087f..6ebbab4b 100644 --- a/doc/guides/nics/features/dpaa2.ini +++ b/doc/guides/nics/features/dpaa2.ini @@ -6,6 +6,7 @@ [Features] Speed capabilities = P Link status = Y +Link status event = Y Queue start/stop = Y Jumbo frame = Y MTU update = Y @@ -20,6 +21,7 @@ L3 checksum offload = Y L4 checksum offload = Y Packet type parsing = Y Basic stats = Y +Extended stats = Y FW version = Y Linux VFIO = Y ARMv8 = Y diff --git a/doc/guides/nics/features/enic.ini b/doc/guides/nics/features/enic.ini index 0de3ef53..498341f0 100644 --- a/doc/guides/nics/features/enic.ini +++ b/doc/guides/nics/features/enic.ini @@ -25,6 +25,7 @@ L3 checksum offload = Y L4 checksum offload = Y Packet type parsing = Y Basic stats = Y +Multiprocess aware = Y BSD nic_uio = Y Linux UIO = Y Linux VFIO = Y diff --git a/doc/guides/nics/features/fm10k_vec.ini b/doc/guides/nics/features/fm10k_vec.ini deleted file mode 100644 index 4917e820..00000000 --- a/doc/guides/nics/features/fm10k_vec.ini +++ /dev/null @@ -1,35 +0,0 @@ -; -; Supported features of the 'fm10k_vec' network poll mode driver. -; -; Refer to default.ini for the full list of available PMD features. -; -[Features] -Speed capabilities = P -Rx interrupt = Y -Queue start/stop = Y -Jumbo frame = Y -Scattered Rx = Y -TSO = Y -Promiscuous mode = Y -Allmulticast mode = Y -Unicast MAC filter = Y -Multicast MAC filter = Y -RSS hash = Y -RSS key update = Y -RSS reta update = Y -VMDq = Y -VLAN filter = Y -CRC offload = Y -VLAN offload = Y -L3 checksum offload = Y -L4 checksum offload = Y -Packet type parsing = Y -Basic stats = Y -Extended stats = Y -Stats per queue = Y -Multiprocess aware = Y -BSD nic_uio = Y -Linux UIO = Y -Linux VFIO = Y -x86-32 = Y -x86-64 = Y diff --git a/doc/guides/nics/features/liquidio.ini b/doc/guides/nics/features/liquidio.ini index 3bea03a3..f628b764 100644 --- a/doc/guides/nics/features/liquidio.ini +++ b/doc/guides/nics/features/liquidio.ini @@ -7,8 +7,10 @@ Speed capabilities = Y Link status = Y Link status event = Y +MTU update = Y Jumbo frame = Y Scattered Rx = Y +Promiscuous mode = Y Allmulticast mode = Y RSS hash = Y RSS key update = Y diff --git a/doc/guides/nics/features/mlx4.ini b/doc/guides/nics/features/mlx4.ini index 1d5f2668..f6efd21d 100644 --- a/doc/guides/nics/features/mlx4.ini +++ b/doc/guides/nics/features/mlx4.ini @@ -27,7 +27,6 @@ Inner L4 checksum = Y Packet type parsing = Y Basic stats = Y Stats per queue = Y -Multiprocess aware = Y Other kdrv = Y Power8 = Y x86-32 = Y diff --git a/doc/guides/nics/features/mlx5.ini b/doc/guides/nics/features/mlx5.ini index 2913591c..c3636391 100644 --- a/doc/guides/nics/features/mlx5.ini +++ b/doc/guides/nics/features/mlx5.ini @@ -7,6 +7,7 @@ Speed capabilities = Y Link status = Y Link status event = Y +Removal event = Y Rx interrupt = Y Queue start/stop = Y MTU update = Y @@ -38,6 +39,5 @@ Multiprocess aware = Y Other kdrv = Y ARMv8 = Y Power8 = Y -x86-32 = Y x86-64 = Y Usage doc = Y diff --git a/doc/guides/nics/features/mrvl.ini b/doc/guides/nics/features/mrvl.ini new file mode 100644 index 00000000..00d96218 --- /dev/null +++ b/doc/guides/nics/features/mrvl.ini @@ -0,0 +1,23 @@ +; +; Supported features of the 'mrvl' network poll mode driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +Speed capabilities = Y +Link status = Y +MTU update = Y +Jumbo frame = Y +Promiscuous mode = Y +Allmulticast mode = Y +Unicast MAC filter = Y +Multicast MAC filter = Y +RSS hash = Y +VLAN filter = Y +CRC offload = Y +L3 checksum offload = Y +L4 checksum offload = Y +Packet type parsing = Y +Basic stats = Y +ARMv8 = Y +Usage doc = Y diff --git a/doc/guides/nics/features/nfp.ini b/doc/guides/nics/features/nfp_pf.ini index a1281d2a..d2899e7f 100644 --- a/doc/guides/nics/features/nfp.ini +++ b/doc/guides/nics/features/nfp_pf.ini @@ -16,7 +16,6 @@ TSO = Y RSS hash = Y RSS key update = Y RSS reta update = Y -SR-IOV = Y Flow control = Y VLAN offload = Y L3 checksum offload = Y diff --git a/doc/guides/nics/features/fm10k_vf_vec.ini b/doc/guides/nics/features/nfp_vf.ini index 6f4a639a..d2899e7f 100644 --- a/doc/guides/nics/features/fm10k_vf_vec.ini +++ b/doc/guides/nics/features/nfp_vf.ini @@ -1,29 +1,28 @@ ; -; Supported features of the 'fm10kvf_vec' network poll mode driver. +; Supported features of the 'nfp' network poll mode driver. ; ; Refer to default.ini for the full list of available PMD features. ; [Features] -Speed capabilities = P +Speed capabilities = Y +Link status = Y +Link status event = Y Rx interrupt = Y Queue start/stop = Y +MTU update = Y Jumbo frame = Y -Scattered Rx = Y +Promiscuous mode = Y TSO = Y RSS hash = Y RSS key update = Y RSS reta update = Y -CRC offload = Y +Flow control = Y VLAN offload = Y L3 checksum offload = Y L4 checksum offload = Y -Packet type parsing = Y Basic stats = Y -Extended stats = Y Stats per queue = Y -Multiprocess aware = Y -BSD nic_uio = Y Linux UIO = Y Linux VFIO = Y -x86-32 = Y x86-64 = Y +Usage doc = Y diff --git a/doc/guides/nics/features/octeontx.ini b/doc/guides/nics/features/octeontx.ini new file mode 100644 index 00000000..c92d5aa0 --- /dev/null +++ b/doc/guides/nics/features/octeontx.ini @@ -0,0 +1,19 @@ +; +; Supported features of the 'octeontx' network poll mode driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +Speed capabilities = Y +Link status = Y +Lock-free Tx queue = Y +Queue start/stop = P +Jumbo frame = Y +Promiscuous mode = Y +Unicast MAC filter = P +CRC offload = Y +Packet type parsing = Y +Basic stats = Y +Linux VFIO = Y +ARMv8 = Y +Usage doc = Y diff --git a/doc/guides/nics/features/sfc_efx.ini b/doc/guides/nics/features/sfc_efx.ini index 1db7f67b..03890f30 100644 --- a/doc/guides/nics/features/sfc_efx.ini +++ b/doc/guides/nics/features/sfc_efx.ini @@ -25,6 +25,8 @@ VLAN offload = P L3 checksum offload = Y L4 checksum offload = Y Packet type parsing = Y +Rx descriptor status = Y +Tx descriptor status = Y Basic stats = Y Extended stats = Y FW version = Y diff --git a/doc/guides/nics/features/thunderx.ini b/doc/guides/nics/features/thunderx.ini index b9720be6..62685897 100644 --- a/doc/guides/nics/features/thunderx.ini +++ b/doc/guides/nics/features/thunderx.ini @@ -4,6 +4,7 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Speed capabilities = Y Link status = Y Link status event = Y Queue start/stop = Y diff --git a/doc/guides/nics/features/xenvirt.ini b/doc/guides/nics/features/xenvirt.ini deleted file mode 100644 index 8ab5f465..00000000 --- a/doc/guides/nics/features/xenvirt.ini +++ /dev/null @@ -1,6 +0,0 @@ -; -; Supported features of the 'xenvirt' network poll mode driver. -; -; Refer to default.ini for the full list of available PMD features. -; -[Features] diff --git a/doc/guides/nics/fm10k.rst b/doc/guides/nics/fm10k.rst index 7fc48624..b47fc0db 100644 --- a/doc/guides/nics/fm10k.rst +++ b/doc/guides/nics/fm10k.rst @@ -161,6 +161,17 @@ FM10000 PMD driver. The switch driver can be acquired from Intel support. Only Testpoint is validated with DPDK, the latest version that has been validated with DPDK is 4.1.6. +Support for Switch Restart +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +For FM10000 multi host based design a DPDK app running in the VM or host needs +to be aware of the switch's state since it may undergo a quit-restart. When +the switch goes down the DPDK app will receive a LSC event indicating link +status down, and the app should stop the worker threads that are polling on +the Rx/Tx queues. When switch comes up, a LSC event indicating ``LINK_UP`` is +sent to the app, which can then restart the FM10000 port to resume network +processing. + CRC striping ~~~~~~~~~~~~ diff --git a/doc/guides/nics/i40e.rst b/doc/guides/nics/i40e.rst index bc200d39..cd468748 100644 --- a/doc/guides/nics/i40e.rst +++ b/doc/guides/nics/i40e.rst @@ -78,6 +78,8 @@ Prerequisites - To get better performance on Intel platforms, please follow the "How to get best performance with NICs on Intel platforms" section of the :ref:`Getting Started Guide for Linux <linux_gsg>`. +- Upgrade the NVM/FW version following the `Intel® Ethernet NVM Update Tool Quick Usage Guide for Linux + <https://www-ssl.intel.com/content/www/us/en/embedded/products/networking/nvm-update-tool-quick-linux-usage-guide.html>`_ if needed. Pre-Installation Configuration ------------------------------ @@ -396,13 +398,24 @@ used to classify MPLS packet by using a command in testpmd like: testpmd> ethertype_filter 0 add mac_ignr 00:00:00:00:00:00 ethertype \ 0x8847 fwd queue <M> -16 Byte Descriptor cannot be used on DPDK VF -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +16 Byte RX Descriptor setting on DPDK VF +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -If the Linux i40e kernel driver is used as host driver, while DPDK i40e PMD -is used as the VF driver, DPDK cannot choose 16 byte receive descriptor. That -is to say, user should keep ``CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC=n`` in -config file. +Currently the VF's RX descriptor mode is decided by PF. There's no PF-VF +interface for VF to request the RX descriptor mode, also no interface to notify +VF its own RX descriptor mode. +For all available versions of the i40e driver, these drivers don't support 16 +byte RX descriptor. If the Linux i40e kernel driver is used as host driver, +while DPDK i40e PMD is used as the VF driver, DPDK cannot choose 16 byte receive +descriptor. The reason is that the RX descriptor is already set to 32 byte by +the i40e kernel driver. That is to say, user should keep +``CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC=n`` in config file. +In the future, if the Linux i40e driver supports 16 byte RX descriptor, user +should make sure the DPDK VF uses the same RX descriptor mode, 16 byte or 32 +byte, as the PF driver. + +The same rule for DPDK PF + DPDK VF. The PF and VF should use the same RX +descriptor mode. Or the VF RX will not work. Receive packets with Ethertype 0x88A8 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -465,6 +478,11 @@ enabled using the steps below. setpci -s <XX:XX.X> a8.w=<value> +Vlan strip of VF +~~~~~~~~~~~~~~~~ + +The VF vlan strip function is only supported in the i40e kernel driver >= 2.1.26. + High Performance of Small Packets on 40G NIC -------------------------------------------- diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst index 36f4f3ff..23babe93 100644 --- a/doc/guides/nics/index.rst +++ b/doc/guides/nics/index.rst @@ -43,6 +43,7 @@ Network Interface Controller Drivers bnx2x bnxt cxgbe + dpaa dpaa2 e1000em ena @@ -55,7 +56,9 @@ Network Interface Controller Drivers liquidio mlx4 mlx5 + mrvl nfp + octeontx qede sfc_efx szedata2 diff --git a/doc/guides/nics/ixgbe.rst b/doc/guides/nics/ixgbe.rst index 696ff693..c687c63f 100644 --- a/doc/guides/nics/ixgbe.rst +++ b/doc/guides/nics/ixgbe.rst @@ -227,6 +227,17 @@ So when the user sets different MTUs on PF and VF ports in one physical port, the real MTU for all these PF and VF ports is the largest value set. This behavior is based on the kernel driver behavior. +VF MAC address setting +~~~~~~~~~~~~~~~~~~~~~~ + +On ixgbe, the concept of "pool" can be used for different things depending on +the mode. In VMDq mode, "pool" means a VMDq pool. In IOV mode, "pool" means a +VF. + +There is no RTE API to add a VF's MAC address from the PF. On ixgbe, the +``rte_eth_dev_mac_addr_add()`` function can be used to add a VF's MAC address, +as a workaround. + Supported Chipsets and NICs --------------------------- diff --git a/doc/guides/nics/liquidio.rst b/doc/guides/nics/liquidio.rst index f04cb16d..7bc16040 100644 --- a/doc/guides/nics/liquidio.rst +++ b/doc/guides/nics/liquidio.rst @@ -42,7 +42,9 @@ Supported LiquidIO Adapters ----------------------------- - LiquidIO II CN2350 210SV/225SV +- LiquidIO II CN2350 210SVPT - LiquidIO II CN2360 210SV/225SV +- LiquidIO II CN2360 210SVPT Pre-Installation Configuration @@ -195,6 +197,14 @@ This section provides instructions to configure SR-IOV with Linux OS. Done testpmd> +#. Enabling VF promiscuous mode + + One VF per PF can be marked as trusted for promiscuous mode. + + .. code-block:: console + + ip link set dev <PF iface> vf <VF id> trust on + Limitations ----------- diff --git a/doc/guides/nics/mlx4.rst b/doc/guides/nics/mlx4.rst index f8885b23..5c3fb764 100644 --- a/doc/guides/nics/mlx4.rst +++ b/doc/guides/nics/mlx4.rst @@ -1,5 +1,5 @@ .. BSD LICENSE - Copyright 2012-2015 6WIND S.A. + Copyright 2012 6WIND S.A. Copyright 2015 Mellanox Redistribution and use in source and binary forms, with or without @@ -78,29 +78,9 @@ Features -------- - Multi arch support: x86_64 and POWER8. -- RSS, also known as RCA, is supported. In this mode the number of - configured RX queues must be a power of two. -- VLAN filtering is supported. - Link state information is provided. -- Promiscuous mode is supported. -- All multicast mode is supported. -- Multiple MAC addresses (unicast, multicast) can be configured. -- Scattered packets are supported for TX and RX. -- Inner L3/L4 (IP, TCP and UDP) TX/RX checksum offloading and validation. -- Outer L3 (IP) TX/RX checksum offloading and validation for VXLAN frames. -- Secondary process TX is supported. - RX interrupts. -Limitations ------------ - -- RSS hash key cannot be modified. -- RSS RETA cannot be configured -- RSS always includes L3 (IPv4/IPv6) and L4 (UDP/TCP). They cannot be - dissociated. -- Hardware counters are not implemented (they are software counters). -- Secondary process RX is not supported. - Configuration ------------- @@ -127,13 +107,6 @@ These options can be modified in the ``.config`` file. to abort with harmless debugging messages as a workaround. Relevant only when CONFIG_RTE_LIBRTE_MLX4_DEBUG is enabled. -- ``CONFIG_RTE_LIBRTE_MLX4_SGE_WR_N`` (default **4**) - - Number of scatter/gather elements (SGEs) per work request (WR). Lowering - this number improves performance but also limits the ability to receive - scattered packets (packets that do not fit a single mbuf). The default - value is a safe tradeoff. - - ``CONFIG_RTE_LIBRTE_MLX4_MAX_INLINE`` (default **0**) Amount of data to be inlined during TX operations. Improves latency but @@ -147,11 +120,6 @@ These options can be modified in the ``.config`` file. This value is always 1 for RX queues since they use a single MP. -- ``CONFIG_RTE_LIBRTE_MLX4_SOFT_COUNTERS`` (default **1**) - - Toggle software counters. No counters are available if this option is - disabled since hardware counters are not supported. - Environment variables ~~~~~~~~~~~~~~~~~~~~~ @@ -164,9 +132,6 @@ Environment variables Run-time configuration ~~~~~~~~~~~~~~~~~~~~~~ -- The only constraint when RSS mode is requested is to make sure the number - of RX queues is a power of two. This is a hardware requirement. - - librte_pmd_mlx4 brings kernel network interfaces up during initialization because it is affected by their state. Forcing them down prevents packets reception. diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index f4cb18bc..f9558da8 100644 --- a/doc/guides/nics/mlx5.rst +++ b/doc/guides/nics/mlx5.rst @@ -92,21 +92,23 @@ Features - Flow director (RTE_FDIR_MODE_PERFECT, RTE_FDIR_MODE_PERFECT_MAC_VLAN and RTE_ETH_FDIR_REJECT). - Flow API. -- Secondary process TX is supported. +- Multiple process. - KVM and VMware ESX SR-IOV modes are supported. - RSS hash result is supported. - Hardware TSO. - Hardware checksum TX offload for VXLAN and GRE. - RX interrupts. - Statistics query including Basic, Extended and per queue. +- Rx HW timestamp. Limitations ----------- - Inner RSS for VXLAN frames is not supported yet. -- Port statistics through software counters only. +- Port statistics through software counters only. Flow statistics are + supported by hardware counters. - Hardware checksum RX offloads for VXLAN inner header are not supported yet. -- Secondary process RX is not supported. +- Forked secondary process not supported. - Flow pattern without any specific vlan will match for vlan packets as well: When VLAN spec is not specified in the pattern, the matching rule will be created with VLAN as a wild card. @@ -124,6 +126,14 @@ Limitations Will match any ipv4 packet (VLAN included). +- A multi segment packet must have less than 6 segments in case the Tx burst function + is set to multi-packet send or Enhanced multi-packet send. Otherwise it must have + less than 50 segments. +- Count action for RTE flow is only supported in Mellanox OFED 4.2. +- Flows with a VXLAN Network Identifier equal (or ends to be equal) + to 0 are not supported. +- VXLAN TSO and checksum offloads are not supported on VM. + Configuration ------------- @@ -167,6 +177,23 @@ Environment variables This is disabled by default since this can also decrease performance for unaligned packet sizes. +- ``MLX5_SHUT_UP_BF`` + + Configures HW Tx doorbell register as IO-mapped. + + By default, the HW Tx doorbell is configured as a write-combining register. + The register would be flushed to HW usually when the write-combining buffer + becomes full, but it depends on CPU design. + + Except for vectorized Tx burst routines, a write memory barrier is enforced + after updating the register so that the update can be immediately visible to + HW. + + When vectorized Tx burst is called, the barrier is set only if the burst size + is not aligned to MLX5_VPMD_TX_MAX_BURST. However, setting this environmental + variable will bring better latency even though the maximum throughput can + slightly decline. + Run-time configuration ~~~~~~~~~~~~~~~~~~~~~~ @@ -293,7 +320,7 @@ DPDK and must be installed separately: This library basically implements send/receive calls to the hardware queues. -- **Kernel modules** (mlnx-ofed-kernel) +- **Kernel modules** They provide the kernel-side Verbs API and low level device drivers that manage actual hardware initialization and resources sharing with user @@ -320,18 +347,33 @@ DPDK and must be installed separately: Both libraries are BSD and GPL licensed. Linux kernel modules are GPL licensed. -Currently supported by DPDK: +Installation +~~~~~~~~~~~~ -- Mellanox OFED version: **4.1**. -- firmware version: +Either RDMA Core library with a recent enough Linux kernel release +(recommended) or Mellanox OFED, which provides compatibility with older +releases. - - ConnectX-4: **12.20.1010** and above. - - ConnectX-4 Lx: **14.20.1010** and above. - - ConnectX-5: **16.20.1010** and above. - - ConnectX-5 Ex: **16.20.1010** and above. +RMDA Core with Linux Kernel +^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Getting Mellanox OFED -~~~~~~~~~~~~~~~~~~~~~ +- Minimal kernel version : v4.14 or the most recent 4.14-rc (see `Linux installation documentation`_) +- Minimal rdma-core version: v15+ commit 0c5f5765213a ("Merge pull request #227 from yishaih/tm") + (see `RDMA Core installation documentation`_) + +.. _`Linux installation documentation`: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/plain/Documentation/admin-guide/README.rst +.. _`RDMA Core installation documentation`: https://raw.githubusercontent.com/linux-rdma/rdma-core/master/README.md + +Mellanox OFED +^^^^^^^^^^^^^ + +- Mellanox OFED version: **4.2**. +- firmware version: + + - ConnectX-4: **12.21.1000** and above. + - ConnectX-4 Lx: **14.21.1000** and above. + - ConnectX-5: **16.21.1000** and above. + - ConnectX-5 Ex: **16.21.1000** and above. While these libraries and kernel modules are available on OpenFabrics Alliance's `website <https://www.openfabrics.org/>`__ and provided by package @@ -373,8 +415,8 @@ Supported NICs * Mellanox(R) ConnectX(R)-5 100G MCX556A-ECAT (2x100G) * Mellanox(R) ConnectX(R)-5 Ex EN 100G MCX516A-CDAT (2x100G) -Quick Start Guide ------------------ +Quick Start Guide on OFED +------------------------- 1. Download latest Mellanox OFED. For more info check the `prerequisites`_. @@ -384,7 +426,7 @@ Quick Start Guide .. code-block:: console - ./mlnxofedinstall + ./mlnxofedinstall --upstream-libs --dpdk 3. Verify the firmware is the correct one: diff --git a/doc/guides/nics/mrvl.rst b/doc/guides/nics/mrvl.rst new file mode 100644 index 00000000..fbfdf478 --- /dev/null +++ b/doc/guides/nics/mrvl.rst @@ -0,0 +1,253 @@ +.. BSD LICENSE + Copyright(c) 2017 Marvell International Ltd. + Copyright(c) 2017 Semihalf. + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of the copyright holder nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +MRVL Poll Mode Driver +====================== + +The MRVL PMD (librte_pmd_mrvl) provides poll mode driver support +for the Marvell PPv2 (Packet Processor v2) 1/10 Gbps adapter. + +Detailed information about SoCs that use PPv2 can be obtained here: + +* https://www.marvell.com/embedded-processors/armada-70xx/ +* https://www.marvell.com/embedded-processors/armada-80xx/ + +.. Note:: + + Due to external dependencies, this driver is disabled by default. It must + be enabled manually by setting relevant configuration option manually. + Please refer to `Config File Options`_ section for further details. + + +Features +-------- + +Features of the MRVL PMD are: + +- Speed capabilities +- Link status +- Queue start/stop +- MTU update +- Jumbo frame +- Promiscuous mode +- Allmulticast mode +- Unicast MAC filter +- Multicast MAC filter +- RSS hash +- VLAN filter +- CRC offload +- L3 checksum offload +- L4 checksum offload +- Packet type parsing +- Basic stats +- QoS + + +Limitations +----------- + +- Number of lcores is limited to 9 by MUSDK internal design. If more lcores + need to be allocated, locking will have to be considered. Number of available + lcores can be changed via ``MRVL_MUSDK_HIFS_RESERVED`` define in + ``mrvl_ethdev.c`` source file. + +- Flushing vlans added for filtering is not possible due to MUSDK missing + functionality. Current workaround is to reset board so that PPv2 has a + chance to start in a sane state. + + +Prerequisites +------------- + +- Custom Linux Kernel sources available + `here <https://github.com/MarvellEmbeddedProcessors/linux-marvell/tree/linux-4.4.52-armada-17.08>`__. + +- Out of tree `mvpp2x_sysfs` kernel module sources available + `here <https://github.com/MarvellEmbeddedProcessors/mvpp2x-marvell/tree/mvpp2x-armada-17.08>`__. + +- MUSDK (Marvell User-Space SDK) sources available + `here <https://github.com/MarvellEmbeddedProcessors/musdk-marvell/tree/musdk-armada-17.08>`__. + + MUSDK is a light-weight library that provides direct access to Marvell's + PPv2 (Packet Processor v2). Alternatively prebuilt MUSDK library can be + requested from `Marvell Extranet <https://extranet.marvell.com>`_. Once + approval has been granted, library can be found by typing ``musdk`` in + the search box. + + MUSDK must be configured with the following features: + + .. code-block:: console + + --enable-bpool-dma=64 + +- DPDK environment + + Follow the DPDK :ref:`Getting Started Guide for Linux <linux_gsg>` to setup + DPDK environment. + + +Config File Options +------------------- + +The following options can be modified in the ``config`` file. + +- ``CONFIG_RTE_LIBRTE_MRVL_PMD`` (default ``n``) + + Toggle compilation of the librte_pmd_mrvl driver. + +- ``CONFIG_RTE_MRVL_MUSDK_DMA_MEMSIZE`` (default ``41943040``) + + Size in bytes of the contiguous memory region that MUSDK will allocate + for run-time DMA-able data buffers. + + +QoS Configuration +----------------- + +QoS configuration is done through external configuration file. Path to the +file must be given as `cfg` in driver's vdev parameter list. + +Configuration syntax +~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: console + + [port <portnum> default] + default_tc = <default_tc> + qos_mode = <qos_priority> + + [port <portnum> tc <traffic_class>] + rxq = <rx_queue_list> + pcp = <pcp_list> + dscp = <dscp_list> + + [port <portnum> tc <traffic_class>] + rxq = <rx_queue_list> + pcp = <pcp_list> + dscp = <dscp_list> + +Where: + +- ``<portnum>``: DPDK Port number (0..n). + +- ``<default_tc>``: Default traffic class (e.g. 0) + +- ``<qos_priority>``: QoS priority for mapping (`ip`, `vlan`, `ip/vlan` or `vlan/ip`). + +- ``<traffic_class>``: Traffic Class to be configured. + +- ``<rx_queue_list>``: List of DPDK RX queues (e.g. 0 1 3-4) + +- ``<pcp_list>``: List of PCP values to handle in particular TC (e.g. 0 1 3-4 7). + +- ``<dscp_list>``: List of DSCP values to handle in particular TC (e.g. 0-12 32-48 63). + +Setting PCP/DSCP values for the default TC is not required. All PCP/DSCP +values not assigned explicitly to particular TC will be handled by the +default TC. + +Configuration file example +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: console + + [port 0 default] + default_tc = 0 + qos_mode = ip + + [port 0 tc 0] + rxq = 0 1 + + [port 0 tc 1] + rxq = 2 + pcp = 5 6 7 + dscp = 26-38 + + [port 1 default] + default_tc = 0 + qos_mode = vlan/ip + + [port 1 tc 0] + rxq = 0 + + [port 1 tc 1] + rxq = 1 2 + pcp = 5 6 7 + dscp = 26-38 + +Usage example +^^^^^^^^^^^^^ + +.. code-block:: console + + ./testpmd --vdev=eth_mrvl,iface=eth0,iface=eth2,cfg=/home/user/mrvl.conf \ + -c 7 -- -i -a --disable-hw-vlan-strip --rxq=2 + + +Building DPDK +------------- + +Driver needs precompiled MUSDK library during compilation. Please consult +``doc/musdk_get_started.txt`` for the detailed build instructions. + +Before the DPDK build process the environmental variable ``LIBMUSDK_PATH`` with +the path to the MUSDK installation directory needs to be exported. + + +Usage Example +------------- + +MRVL PMD requires extra out of tree kernel modules to function properly. +`musdk_uio` and `mv_pp_uio` sources are part of the MUSDK. Please consult +``doc/musdk_get_started.txt`` for the detailed build instructions. +For `mvpp2x_sysfs` please consult ``Documentation/pp22_sysfs.txt`` for the +detailed build instructions. + +.. code-block:: console + + insmod musdk_uio.ko + insmod mv_pp_uio.ko + insmod mvpp2x_sysfs.ko + +Additionally interfaces used by DPDK application need to be put up: + +.. code-block:: console + + ip link set eth0 up + ip link set eth1 up + +In order to run testpmd example application following command can be used: + +.. code-block:: console + + ./testpmd --vdev=eth_mrvl,iface=eth0,iface=eth2 -c 7 -- \ + --burst=128 --txd=2048 --rxd=1024 --rxq=2 --txq=2 --nb-cores=2 \ + -i -a --disable-hw-vlan-strip --rss-udp diff --git a/doc/guides/nics/nfp.rst b/doc/guides/nics/nfp.rst index c732fb1f..99a3b76e 100644 --- a/doc/guides/nics/nfp.rst +++ b/doc/guides/nics/nfp.rst @@ -1,5 +1,5 @@ .. BSD LICENSE - Copyright(c) 2015 Netronome Systems, Inc. All rights reserved. + Copyright(c) 2015-2017 Netronome Systems, Inc. All rights reserved. All rights reserved. Redistribution and use in source and binary forms, with or without @@ -38,31 +38,32 @@ up to 400 Gbps. This document explains how to use DPDK with the Netronome Poll Mode Driver (PMD) supporting Netronome's Network Flow Processor 6xxx -(NFP-6xxx). +(NFP-6xxx) and Netronome's Flow Processor 4xxx (NFP-4xxx). -Currently the driver supports virtual functions (VFs) only. +NFP is a SRIOV capable device and the PMD driver supports the physical +function (PF) and virtual functions (VFs). Dependencies ------------ -Before using the Netronome's DPDK PMD some NFP-6xxx configuration, +Before using the Netronome's DPDK PMD some NFP configuration, which is not related to DPDK, is required. The system requires -installation of **Netronome's BSP (Board Support Package)** which includes -Linux drivers, programs and libraries. +installation of **Netronome's BSP (Board Support Package)** along +with some specific NFP firmware application. Netronome's NSP ABI +version should be 0.20 or higher. -If you have a NFP-6xxx device you should already have the code and -documentation for doing this configuration. Contact +If you have a NFP device you should already have the code and +documentation for doing all this configuration. Contact **support@netronome.com** to obtain the latest available firmware. -The NFP Linux kernel drivers (including the required PF driver for the -NFP) are available on Github at +The NFP Linux netdev kernel driver for VFs is part of vanilla kernel +since kernel version 4.5, and support for the PF since kernel version +4.11. Support for older kernels can be obtained on Github at **https://github.com/Netronome/nfp-drv-kmods** along with build instructions. -DPDK runs in userspace and PMDs uses the Linux kernel UIO interface to -allow access to physical devices from userspace. The NFP PMD requires -the **igb_uio** UIO driver, available with DPDK, to perform correct -initialization. +NFP PMD needs to be used along with UIO ``igb_uio`` or VFIO (``vfio-pci``) +Linux kernel driver. Building the software --------------------- @@ -71,7 +72,7 @@ Netronome's PMD code is provided in the **drivers/net/nfp** directory. Although NFP PMD has Netronome´s BSP dependencies, it is possible to compile it along with other DPDK PMDs even if no BSP was installed before. Of course, a DPDK app will require such a BSP installed for using the -NFP PMD. +NFP PMD, along with a specific NFP firmware application. Default PMD configuration is at **common_linuxapp configuration** file: @@ -87,14 +88,53 @@ Driver compilation and testing Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>` for details. +Using the PF +------------ + +NFP PMD has support for using the NFP PF as another DPDK port, but it does not +have any functionality for controlling VFs. In fact, it is not possible to use +the PMD with the VFs if the PF is being used by DPDK, that is, with the NFP PF +bound to ``igb_uio`` or ``vfio-pci`` kernel drivers. Future DPDK version will +have a PMD able to work with the PF and VFs at the same time and with the PF +implementing VF management along with other PF-only functionalities/offloads. + +The PMD PF has extra work to do which will delay the DPDK app initialization +like checking if a firmware is already available in the device, uploading the +firmware if necessary, and configure the Link state properly when starting or +stopping a PF port. Note that firmware upload is not always necessary which is +the main delay for NFP PF PMD initialization. + +Depending on the Netronome product installed in the system, firmware files +should be available under ``/lib/firmware/netronome``. DPDK PMD supporting the +PF requires a specific link, ``/lib/firmware/netronome/nic_dpdk_default.nffw``, +which should be created automatically with Netronome's Agilio products +installation. + +PF multiport support +-------------------- + +Some NFP cards support several physical ports with just one single PCI device. +DPDK core is designed with the 1:1 relationship between PCI devices and DPDK +ports, so NFP PMD PF support requires handling the multiport case specifically. +During NFP PF initialization, the PMD will extract the information about the +number of PF ports from the firmware and will create as many DPDK ports as +needed. + +Because the unusual relationship between a single PCI device and several DPDK +ports, there are some limitations when using more than one PF DPDK ports: there +is no support for RX interrupts and it is not possible either to use those PF +ports with the device hotplug functionality. + System configuration -------------------- -#. **Enable SR-IOV on the NFP-6xxx device:** The current NFP PMD works with - Virtual Functions (VFs) on a NFP device. Make sure that one of the Physical - Function (PF) drivers from the above Github repository is installed and - loaded. +#. **Enable SR-IOV on the NFP device:** The current NFP PMD supports the PF and + the VFs on a NFP device. However, it is not possible to work with both at the + same time because the VFs require the PF being bound to the NFP PF Linux + netdev driver. Make sure you are working with a kernel with NFP PF support or + get the drivers from the above Github repository and follow the instructions + for building and installing it. Virtual Functions need to be enabled before they can be used with the PMD. Before enabling the VFs it is useful to obtain information about the diff --git a/doc/guides/nics/octeontx.rst b/doc/guides/nics/octeontx.rst new file mode 100644 index 00000000..a6631cd0 --- /dev/null +++ b/doc/guides/nics/octeontx.rst @@ -0,0 +1,252 @@ +.. BSD LICENSE + Copyright (C) Cavium, Inc. 2017. + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of Cavium, Inc nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +OCTEONTX Poll Mode driver +========================= + +The OCTEONTX ETHDEV PMD (**librte_pmd_octeontx**) provides poll mode ethdev +driver support for the inbuilt network device found in the **Cavium OCTEONTX** +SoC family as well as their virtual functions (VF) in SR-IOV context. + +More information can be found at `Cavium, Inc Official Website +<http://www.cavium.com/OCTEON-TX_ARM_Processors.html>`_. + +Features +-------- + +Features of the OCTEONTX Ethdev PMD are: + +- Packet type information +- Promiscuous mode +- Port hardware statistics +- Jumbo frames +- Link state information +- SR-IOV VF +- Multiple queues for TX +- Lock-free Tx queue +- HW offloaded `ethdev Rx queue` to `eventdev event queue` packet injection + +Supported OCTEONTX SoCs +----------------------- + +- CN83xx + +Unsupported features +-------------------- + +The features supported by the device and not yet supported by this PMD include: + +- Receive Side Scaling (RSS) +- Scattered and gather for TX and RX +- Ingress classification support +- Egress hierarchical scheduling, traffic shaping, and marking + +Prerequisites +------------- + +There are three main pre-perquisites for executing OCTEONTX PMD on a OCTEONTX +compatible board: + +1. **OCTEONTX Linux kernel PF driver for Network acceleration HW blocks** + + The OCTEONTX Linux kernel drivers (including the required PF driver for the + all network acceleration blocks) are available on GitHub at + `octeontx-kmod <https://github.com/caviumnetworks/octeontx-kmod>`_ + along with build, install and dpdk usage instructions. + +2. **ARM64 Tool Chain** + + For example, the *aarch64* Linaro Toolchain, which can be obtained from + `here <https://releases.linaro.org/components/toolchain/binaries/4.9-2017.01/aarch64-linux-gnu>`_. + +3. **Rootfile system** + + Any *aarch64* supporting filesystem can be used. For example, + Ubuntu 15.10 (Wily) or 16.04 LTS (Xenial) userland which can be obtained + from `<http://cdimage.ubuntu.com/ubuntu-base/releases/16.04/release/ubuntu-base-16.04.1-base-arm64.tar.gz>`_. + + As an alternative method, OCTEONTX PMD can also be executed using images provided + as part of SDK from Cavium. The SDK includes all the above prerequisites necessary + to bring up a OCTEONTX board. + + SDK and related information can be obtained from: `Cavium support site <https://support.cavium.com/>`_. + +Follow the DPDK :ref:`Getting Started Guide for Linux <linux_gsg>` to setup the basic DPDK environment. + +Pre-Installation Configuration +------------------------------ + +Config File Options +~~~~~~~~~~~~~~~~~~~ + +The following options can be modified in the ``config`` file. +Please note that enabling debugging options may affect system performance. + +- ``CONFIG_RTE_LIBRTE_OCTEONTX_PMD`` (default ``y``) + + Toggle compilation of the ``librte_pmd_octeontx`` driver. + +- ``CONFIG_RTE_LIBRTE_OCTEONTX_DEBUG_DRIVER`` (default ``n``) + + Toggle display of generic debugging messages + +- ``CONFIG_RTE_LIBRTE_OCTEONTX_DEBUG_INIT`` (default ``n``) + + Toggle display of initialization related messages. + +- ``CONFIG_RTE_LIBRTE_OCTEONTX_DEBUG_RX`` (default ``n``) + + Toggle display of receive path message + +- ``CONFIG_RTE_LIBRTE_OCTEONTX_DEBUG_TX`` (default ``n``) + + Toggle display of transmit path message + +- ``CONFIG_RTE_LIBRTE_OCTEONTX_DEBUG_MBOX`` (default ``n``) + + Toggle display of mbox related message + + +Driver compilation and testing +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>` +for details. + +To compile the OCTEONTX PMD for Linux arm64 gcc target, run the +following ``make`` command: + +.. code-block:: console + + cd <DPDK-source-directory> + make config T=arm64-thunderx-linuxapp-gcc install + +#. Running testpmd: + + Follow instructions available in the document + :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>` + to run testpmd. + + Example output: + + .. code-block:: console + + ./arm64-thunderx-linuxapp-gcc/app/testpmd -c 700 \ + --base-virtaddr=0x100000000000 \ + --mbuf-pool-ops-name="octeontx_fpavf" \ + --vdev='event_octeontx' \ + --vdev='eth_octeontx,nr_port=2' \ + -- --rxq=1 --txq=1 --nb-core=2 --total-num-mbufs=16384 \ + --disable-hw-vlan-filter -i + ..... + EAL: Detected 24 lcore(s) + EAL: Probing VFIO support... + EAL: VFIO support initialized + ..... + EAL: PCI device 0000:07:00.1 on NUMA socket 0 + EAL: probe driver: 177d:a04b octeontx_ssovf + ..... + EAL: PCI device 0001:02:00.7 on NUMA socket 0 + EAL: probe driver: 177d:a0dd octeontx_pkivf + ..... + EAL: PCI device 0001:03:01.0 on NUMA socket 0 + EAL: probe driver: 177d:a049 octeontx_pkovf + ..... + PMD: octeontx_probe(): created ethdev eth_octeontx for port 0 + PMD: octeontx_probe(): created ethdev eth_octeontx for port 1 + ..... + Configuring Port 0 (socket 0) + Port 0: 00:0F:B7:11:94:46 + Configuring Port 1 (socket 0) + Port 1: 00:0F:B7:11:94:47 + ..... + Checking link statuses... + Port 0 Link Up - speed 40000 Mbps - full-duplex + Port 1 Link Up - speed 40000 Mbps - full-duplex + Done + testpmd> + + +Initialization +-------------- + +The octeontx ethdev pmd is exposed as a vdev device which consists of a set +of PKI and PKO PCIe VF devices. On EAL initialization, +PKI/PKO PCIe VF devices will be probed and then the vdev device can be created +from the application code, or from the EAL command line based on +the number of probed/bound PKI/PKO PCIe VF device to DPDK by + +* Invoking ``rte_vdev_init("eth_octeontx")`` from the application + +* Using ``--vdev="eth_octeontx"`` in the EAL options, which will call + rte_vdev_init() internally + +Device arguments +~~~~~~~~~~~~~~~~ +Each ethdev port is mapped to a physical port(LMAC), Application can specify +the number of interesting ports with ``nr_ports`` argument. + +Dependency +~~~~~~~~~~ +``eth_octeontx`` pmd is depend on ``event_octeontx`` eventdev device and +``octeontx_fpavf`` external mempool handler. + +Example: + +.. code-block:: console + + ./your_dpdk_application --mbuf-pool-ops="octeontx_fpavf" \ + --vdev='event_octeontx' \ + --vdev="eth_octeontx,nr_port=2" + +Limitations +----------- + +``octeontx_fpavf`` external mempool handler dependency +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The OCTEONTX SoC family NIC has inbuilt HW assisted external mempool manager. +This driver will only work with ``octeontx_fpavf`` external mempool handler +as it is the most performance effective way for packet allocation and Tx buffer +recycling on OCTEONTX SoC platform. + +CRC striping +~~~~~~~~~~~~ + +The OCTEONTX SoC family NICs strip the CRC for every packets coming into the +host interface. So, CRC will be stripped even when the +``rxmode.hw_strip_crc`` member is set to 0 in ``struct rte_eth_conf``. + +Maximum packet length +~~~~~~~~~~~~~~~~~~~~~ + +The OCTEONTX SoC family NICs support a maximum of a 32K jumbo frame. The value +is fixed and cannot be changed. So, even when the ``rxmode.max_rx_pkt_len`` +member of ``struct rte_eth_conf`` is set to a value lower than 32k, frames +up to 32k bytes can still reach the host interface. diff --git a/doc/guides/nics/sfc_efx.rst b/doc/guides/nics/sfc_efx.rst index 7761989c..ae2b54a2 100644 --- a/doc/guides/nics/sfc_efx.rst +++ b/doc/guides/nics/sfc_efx.rst @@ -148,6 +148,8 @@ Supported actions: - QUEUE +- RSS + Validating flow rules depends on the firmware variant. Ethernet destinaton individual/group match @@ -243,12 +245,16 @@ boolean parameters value. features available and required by the datapath implementation. **efx** chooses libefx-based datapath which supports VLAN insertion (full-feature firmware variant only), TSO and multi-segment mbufs. + Mbuf segments may come from different mempools, and mbuf reference + counters are treated responsibly. **ef10** chooses EF10 (SFN7xxx, SFN8xxx) native datapath which is more efficient than libefx-based but has no VLAN insertion and TSO support yet. + Mbuf segments may come from different mempools, and mbuf reference + counters are treated responsibly. **ef10_simple** chooses EF10 (SFN7xxx, SFN8xxx) native datapath which is even more faster then **ef10** but does not support multi-segment - mbufs. + mbufs, disallows multiple mempools and neglects mbuf reference counters. - ``perf_profile`` [auto|throughput|low-latency] (default **throughput**) diff --git a/doc/guides/nics/tap.rst b/doc/guides/nics/tap.rst index f3ee95d2..04086b11 100644 --- a/doc/guides/nics/tap.rst +++ b/doc/guides/nics/tap.rst @@ -113,7 +113,14 @@ Flow API support ---------------- The tap PMD supports major flow API pattern items and actions, when running on -linux kernels above 4.2 ("Flower" classifier required). Supported items: +linux kernels above 4.2 ("Flower" classifier required). +The kernel support can be checked with this command:: + + zcat /proc/config.gz | ( grep 'CLS_FLOWER=' || echo 'not supported' ) | + tee -a /dev/stderr | grep -q '=m' && + lsmod | ( grep cls_flower || echo 'try modprobe cls_flower' ) + +Supported items: - eth: src and dst (with variable masks), and eth_type (0xffff mask). - vlan: vid, pcp, tpid, but not eid. (requires kernel 4.9) diff --git a/doc/guides/nics/vhost.rst b/doc/guides/nics/vhost.rst index e651a166..4f7ae899 100644 --- a/doc/guides/nics/vhost.rst +++ b/doc/guides/nics/vhost.rst @@ -66,6 +66,11 @@ The user can specify below arguments in `--vdev` option. It is used to specify the number of queues virtio-net device has. (Default: 1) +#. ``iommu-support``: + + It is used to enable iommu support in vhost library. + (Default: 0 (disabled)) + Vhost PMD event handling ------------------------ diff --git a/doc/guides/nics/virtio.rst b/doc/guides/nics/virtio.rst index 4d6a8376..af82f86e 100644 --- a/doc/guides/nics/virtio.rst +++ b/doc/guides/nics/virtio.rst @@ -291,6 +291,13 @@ interrupt, Rx interrupts, and Tx interrupts. Config interrupt is used for notification of device configuration changes, especially link status (lsc). Interrupt mode is translated into Rx interrupts in the context of DPDK. +.. Note:: + + Virtio PMD already has support for receiving lsc from qemu when the link + status changes, especially when vhost user disconnects. However, it fails + to do that if the VM is created by qemu 2.6.2 or below, since the + capability to detect vhost user disconnection is introduced in qemu 2.7.0. + Prerequisites for Rx interrupts ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/doc/guides/prog_guide/cryptodev_lib.rst b/doc/guides/prog_guide/cryptodev_lib.rst index 75ae085f..2b338b92 100644 --- a/doc/guides/prog_guide/cryptodev_lib.rst +++ b/doc/guides/prog_guide/cryptodev_lib.rst @@ -539,12 +539,12 @@ chain. struct { uint8_t *data; - phys_addr_t phys_addr; + rte_iova_t phys_addr; } digest; /**< Digest parameters */ struct { uint8_t *data; - phys_addr_t phys_addr; + rte_iova_t phys_addr; } aad; /**< Additional authentication parameters */ } aead; @@ -566,7 +566,7 @@ chain. struct { uint8_t *data; - phys_addr_t phys_addr; + rte_iova_t phys_addr; } digest; /**< Digest parameters */ } auth; }; diff --git a/doc/guides/prog_guide/env_abstraction_layer.rst b/doc/guides/prog_guide/env_abstraction_layer.rst index fff1c063..9e834fc5 100644 --- a/doc/guides/prog_guide/env_abstraction_layer.rst +++ b/doc/guides/prog_guide/env_abstraction_layer.rst @@ -117,17 +117,6 @@ The physical address of the reserved memory for that memory zone is also returne Memory reservations done using the APIs provided by rte_malloc are also backed by pages from the hugetlbfs filesystem. -Xen Dom0 support without hugetbls -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The existing memory management implementation is based on the Linux kernel hugepage mechanism. -However, Xen Dom0 does not support hugepages, so a new Linux kernel module rte_dom0_mm is added to workaround this limitation. - -The EAL uses IOCTL interface to notify the Linux kernel module rte_dom0_mm to allocate memory of specified size, -and get all memory segments information from the module, -and the EAL uses MMAP interface to map the allocated memory. -For each memory segment, the physical addresses are contiguous within it but actual hardware addresses are contiguous within 2MB. - PCI Access ~~~~~~~~~~ @@ -164,7 +153,7 @@ which can trigger the generation of a core file, readable by gdb. CPU Feature Identification ~~~~~~~~~~~~~~~~~~~~~~~~~~ -The EAL can query the CPU at runtime (using the rte_cpu_get_feature() function) to determine which CPU features are available. +The EAL can query the CPU at runtime (using the rte_cpu_get_features() function) to determine which CPU features are available. User Space Interrupt Event ~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/doc/guides/prog_guide/event_ethernet_rx_adapter.rst b/doc/guides/prog_guide/event_ethernet_rx_adapter.rst new file mode 100644 index 00000000..4e722219 --- /dev/null +++ b/doc/guides/prog_guide/event_ethernet_rx_adapter.rst @@ -0,0 +1,168 @@ +.. BSD LICENSE + Copyright(c) 2017 Intel Corporation. All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of Intel Corporation nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +Event Ethernet Rx Adapter Library +================================= + +The DPDK Eventdev API allows the application to use an event driven programming +model for packet processing. In this model, the application polls an event +device port for receiving events that reference packets instead of polling Rx +queues of ethdev ports. Packet transfer between ethdev and the event device can +be supported in hardware or require a software thread to receive packets from +the ethdev port using ethdev poll mode APIs and enqueue these as events to the +event device using the eventdev API. Both transfer mechanisms may be present on +the same platform depending on the particular combination of the ethdev and +the event device. + +The Event Ethernet Rx Adapter library is intended for the application code to +configure both transfer mechanisms using a common API. A capability API allows +the eventdev PMD to advertise features supported for a given ethdev and allows +the application to perform configuration as per supported features. + +API Walk-through +---------------- + +This section will introduce the reader to the adapter API. The +application has to first instantiate an adapter which is associated with +a single eventdev, next the adapter instance is configured with Rx queues +that are either polled by a SW thread or linked using hardware support. Finally +the adapter is started. + +For SW based packet transfers from ethdev to eventdev, the adapter uses a +DPDK service function and the application is also required to assign a core to +the service function. + +Creating an Adapter Instance +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +An adapter instance is created using ``rte_event_eth_rx_adapter_create()``. This +function is passed the event device to be associated with the adapter and port +configuration for the adapter to setup an event port if the adapter needs to use +a service function. + +.. code-block:: c + + int err; + uint8_t dev_id; + struct rte_event_dev_info dev_info; + struct rte_event_port_conf rx_p_conf; + + err = rte_event_dev_info_get(id, &dev_info); + + rx_p_conf.new_event_threshold = dev_info.max_num_events; + rx_p_conf.dequeue_depth = dev_info.max_event_port_dequeue_depth; + rx_p_conf.enqueue_depth = dev_info.max_event_port_enqueue_depth; + err = rte_event_eth_rx_adapter_create(id, dev_id, &rx_p_conf); + +If the application desires to have finer control of eventdev port allocation +and setup, it can use the ``rte_event_eth_rx_adapter_create_ext()`` function. +The ``rte_event_eth_rx_adapter_create_ext()`` function is passed a callback +function. The callback function is invoked if the adapter needs to use a +service function and needs to create an event port for it. The callback is +expected to fill the ``struct rte_event_eth_rx_adapter_conf structure`` +passed to it. + +Adding Rx Queues to the Adapter Instance +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Ethdev Rx queues are added to the instance using the +``rte_event_eth_rx_adapter_queue_add()`` function. Configuration for the Rx +queue is passed in using a ``struct rte_event_eth_rx_adapter_queue_conf`` +parameter. Event information for packets from this Rx queue is encoded in the +``ev`` field of ``struct rte_event_eth_rx_adapter_queue_conf``. The +servicing_weight member of the struct rte_event_eth_rx_adapter_queue_conf +is the relative polling frequency of the Rx queue and is applicable when the +adapter uses a service core function. + +.. code-block:: c + + ev.queue_id = 0; + ev.sched_type = RTE_SCHED_TYPE_ATOMIC; + ev.priority = 0; + + queue_config.rx_queue_flags = 0; + queue_config.ev = ev; + queue_config.servicing_weight = 1; + + err = rte_event_eth_rx_adapter_queue_add(id, + eth_dev_id, + 0, &queue_config); + +Querying Adapter Capabilities +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The ``rte_event_eth_rx_adapter_caps_get()`` function allows +the application to query the adapter capabilities for an eventdev and ethdev +combination. For e.g, if the ``RTE_EVENT_ETH_RX_ADAPTER_CAP_OVERRIDE_FLOW_ID`` +is set, the application can override the adapter generated flow ID in the event +using ``rx_queue_flags`` field in ``struct rte_event_eth_rx_adapter_queue_conf`` +which is passed as a parameter to the ``rte_event_eth_rx_adapter_queue_add()`` +function. + +.. code-block:: c + + err = rte_event_eth_rx_adapter_caps_get(dev_id, eth_dev_id, &cap); + + queue_config.rx_queue_flags = 0; + if (cap & RTE_EVENT_ETH_RX_ADAPTER_CAP_OVERRIDE_FLOW_ID) { + ev.flow_id = 1; + queue_config.rx_queue_flags = + RTE_EVENT_ETH_RX_ADAPTER_QUEUE_FLOW_ID_VALID; + } + +Configuring the Service Function +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If the adapter uses a service function, the application is required to assign +a service core to the service function as show below. + +.. code-block:: c + + uint32_t service_id; + + if (rte_event_eth_rx_adapter_service_id_get(0, &service_id) == 0) + rte_service_map_lcore_set(service_id, RX_CORE_ID); + +Starting the Adapter Instance +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The application calls ``rte_event_eth_rx_adapter_start()`` to start the adapter. +This function calls the start callbacks of the eventdev PMDs for hardware based +eventdev-ethdev connections and ``rte_service_run_state_set()`` to enable the +service function if one exists. + +Getting Adapter Statistics +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The ``rte_event_eth_rx_adapter_stats_get()`` function reports counters defined +in struct ``rte_event_eth_rx_adapter_stats``. The received packet and +enqueued event counts are a sum of the counts from the eventdev PMD callbacks +if the callback is supported, and the counts maintained by the service function, +if one exists. The service function also maintains a count of cycles for which +it was not able to enqueue to the event device. diff --git a/doc/guides/prog_guide/eventdev.rst b/doc/guides/prog_guide/eventdev.rst index 908d123a..be9fccdd 100644 --- a/doc/guides/prog_guide/eventdev.rst +++ b/doc/guides/prog_guide/eventdev.rst @@ -217,7 +217,7 @@ calling the setup function. Repeat this step for each queue, starting from .. code-block:: c struct rte_event_queue_conf atomic_conf = { - .event_queue_cfg = RTE_EVENT_QUEUE_CFG_ATOMIC_ONLY, + .schedule_type = RTE_SCHED_TYPE_ATOMIC, .priority = RTE_EVENT_DEV_PRIORITY_NORMAL, .nb_atomic_flows = 1024, .nb_atomic_order_sequences = 1024, @@ -320,7 +320,7 @@ The following code shows how those packets can be enqueued into the eventdev: for (i = 0; i < nb_rx; i++) { ev[i].flow_id = mbufs[i]->hash.rss; ev[i].op = RTE_EVENT_OP_NEW; - ev[i].sched_type = RTE_EVENT_QUEUE_CFG_ATOMIC_ONLY; + ev[i].sched_type = RTE_SCHED_TYPE_ATOMIC; ev[i].queue_id = 0; ev[i].event_type = RTE_EVENT_TYPE_ETHDEV; ev[i].sub_event_type = 0; diff --git a/doc/guides/prog_guide/flow_classify_lib.rst b/doc/guides/prog_guide/flow_classify_lib.rst new file mode 100644 index 00000000..820dc72a --- /dev/null +++ b/doc/guides/prog_guide/flow_classify_lib.rst @@ -0,0 +1,427 @@ +.. BSD LICENSE + Copyright(c) 2017 Intel Corporation. All rights reserved. + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of Intel Corporation nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +Flow Classification Library +=========================== + +DPDK provides a Flow Classification library that provides the ability +to classify an input packet by matching it against a set of Flow rules. + +The initial implementation supports counting of IPv4 5-tuple packets which match +a particular Flow rule only. + +Please refer to the +:doc:`./rte_flow` +for more information. + +The Flow Classification library uses the ``librte_table`` API for managing Flow +rules and matching packets against the Flow rules. +The library is table agnostic and can use the following tables: +``Access Control List``, ``Hash`` and ``Longest Prefix Match(LPM)``. +The ``Access Control List`` table is used in the initial implementation. + +Please refer to the +:doc:`./packet_framework` +for more information.on ``librte_table``. + +DPDK provides an Access Control List library that provides the ability to +classify an input packet based on a set of classification rules. + +Please refer to the +:doc:`./packet_classif_access_ctrl` +library for more information on ``librte_acl``. + +There is also a Flow Classify sample application which demonstrates the use of +the Flow Classification Library API's. + +Please refer to the +:doc:`../sample_app_ug/flow_classify` +for more information on the ``flow_classify`` sample application. + +Overview +-------- + +The library has the following API's + +.. code-block:: c + + /** + * Flow classifier create + * + * @param params + * Parameters for flow classifier creation + * @return + * Handle to flow classifier instance on success or NULL otherwise + */ + struct rte_flow_classifier * + rte_flow_classifier_create(struct rte_flow_classifier_params *params); + + /** + * Flow classifier free + * + * @param cls + * Handle to flow classifier instance + * @return + * 0 on success, error code otherwise + */ + int + rte_flow_classifier_free(struct rte_flow_classifier *cls); + + /** + * Flow classify table create + * + * @param cls + * Handle to flow classifier instance + * @param params + * Parameters for flow_classify table creation + * @param table_id + * Table ID. Valid only within the scope of table IDs of the current + * classifier. Only returned after a successful invocation. + * @return + * 0 on success, error code otherwise + */ + int + rte_flow_classify_table_create(struct rte_flow_classifier *cls, + struct rte_flow_classify_table_params *params, + uint32_t *table_id); + + /** + * Add a flow classify rule to the flow_classifier table. + * + * @param[in] cls + * Flow classifier handle + * @param[in] table_id + * id of table + * @param[in] attr + * Flow rule attributes + * @param[in] pattern + * Pattern specification (list terminated by the END pattern item). + * @param[in] actions + * Associated actions (list terminated by the END pattern item). + * @param[out] error + * Perform verbose error reporting if not NULL. Structure + * initialised in case of error only. + * @return + * A valid handle in case of success, NULL otherwise. + */ + struct rte_flow_classify_rule * + rte_flow_classify_table_entry_add(struct rte_flow_classifier *cls, + uint32_t table_id, + const struct rte_flow_attr *attr, + const struct rte_flow_item pattern[], + const struct rte_flow_action actions[], + struct rte_flow_error *error); + + /** + * Delete a flow classify rule from the flow_classifier table. + * + * @param[in] cls + * Flow classifier handle + * @param[in] table_id + * id of table + * @param[in] rule + * Flow classify rule + * @return + * 0 on success, error code otherwise. + */ + int + rte_flow_classify_table_entry_delete(struct rte_flow_classifier *cls, + uint32_t table_id, + struct rte_flow_classify_rule *rule); + + /** + * Query flow classifier for given rule. + * + * @param[in] cls + * Flow classifier handle + * @param[in] table_id + * id of table + * @param[in] pkts + * Pointer to packets to process + * @param[in] nb_pkts + * Number of packets to process + * @param[in] rule + * Flow classify rule + * @param[in] stats + * Flow classify stats + * + * @return + * 0 on success, error code otherwise. + */ + int + rte_flow_classifier_query(struct rte_flow_classifier *cls, + uint32_t table_id, + struct rte_mbuf **pkts, + const uint16_t nb_pkts, + struct rte_flow_classify_rule *rule, + struct rte_flow_classify_stats *stats); + +Classifier creation +~~~~~~~~~~~~~~~~~~~ + +The application creates the ``Classifier`` using the +``rte_flow_classifier_create`` API. +The ``rte_flow_classify_params`` structure must be initialised by the +application before calling the API. + +.. code-block:: c + + struct rte_flow_classifier_params { + /** flow classifier name */ + const char *name; + + /** CPU socket ID where memory for the flow classifier and its */ + /** elements (tables) should be allocated */ + int socket_id; + + /** Table type */ + enum rte_flow_classify_table_type type; + }; + +The ``Classifier`` has the following internal structures: + +.. code-block:: c + + struct rte_table { + /* Input parameters */ + struct rte_table_ops ops; + uint32_t entry_size; + enum rte_flow_classify_table_type type; + + /* Handle to the low-level table object */ + void *h_table; + }; + + #define RTE_FLOW_CLASSIFIER_MAX_NAME_SZ 256 + + struct rte_flow_classifier { + /* Input parameters */ + char name[RTE_FLOW_CLASSIFIER_MAX_NAME_SZ]; + int socket_id; + enum rte_flow_classify_table_type type; + + /* Internal tables */ + struct rte_table tables[RTE_FLOW_CLASSIFY_TABLE_MAX]; + uint32_t num_tables; + uint16_t nb_pkts; + struct rte_flow_classify_table_entry + *entries[RTE_PORT_IN_BURST_SIZE_MAX]; + } __rte_cache_aligned; + +Adding a table to the Classifier +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The application adds a table to the ``Classifier`` using the +``rte_flow_classify_table_create`` API. +The ``rte_flow_classify_table_params`` structure must be initialised by the +application before calling the API. + +.. code-block:: c + + struct rte_flow_classify_table_params { + /** Table operations (specific to each table type) */ + struct rte_table_ops *ops; + + /** Opaque param to be passed to the table create operation */ + void *arg_create; + + /** Memory size to be reserved per classifier object entry for */ + /** storing meta data */ + uint32_t table_metadata_size; + }; + +To create an ACL table the ``rte_table_acl_params`` structure must be +initialised and assigned to ``arg_create`` in the +``rte_flow_classify_table_params`` structure. + +.. code-block:: c + + struct rte_table_acl_params { + /** Name */ + const char *name; + + /** Maximum number of ACL rules in the table */ + uint32_t n_rules; + + /** Number of fields in the ACL rule specification */ + uint32_t n_rule_fields; + + /** Format specification of the fields of the ACL rule */ + struct rte_acl_field_def field_format[RTE_ACL_MAX_FIELDS]; + }; + +The fields for the ACL rule must also be initialised by the application. + +An ACL table can be added to the ``Classifier`` for each ACL rule, for example +another table could be added for the IPv6 5-tuple rule. + +Flow Parsing +~~~~~~~~~~~~ + +The library currently supports three IPv4 5-tuple flow patterns, for UDP, TCP +and SCTP. + +.. code-block:: c + + /* Pattern for IPv4 5-tuple UDP filter */ + static enum rte_flow_item_type pattern_ntuple_1[] = { + RTE_FLOW_ITEM_TYPE_ETH, + RTE_FLOW_ITEM_TYPE_IPV4, + RTE_FLOW_ITEM_TYPE_UDP, + RTE_FLOW_ITEM_TYPE_END, + }; + + /* Pattern for IPv4 5-tuple TCP filter */ + static enum rte_flow_item_type pattern_ntuple_2[] = { + RTE_FLOW_ITEM_TYPE_ETH, + RTE_FLOW_ITEM_TYPE_IPV4, + RTE_FLOW_ITEM_TYPE_TCP, + RTE_FLOW_ITEM_TYPE_END, + }; + + /* Pattern for IPv4 5-tuple SCTP filter */ + static enum rte_flow_item_type pattern_ntuple_3[] = { + RTE_FLOW_ITEM_TYPE_ETH, + RTE_FLOW_ITEM_TYPE_IPV4, + RTE_FLOW_ITEM_TYPE_SCTP, + RTE_FLOW_ITEM_TYPE_END, + }; + +The internal function ``flow_classify_parse_flow`` parses the +IPv4 5-tuple pattern, attributes and actions and returns the 5-tuple data in the +``rte_eth_ntuple_filter`` structure. + +.. code-block:: c + + static int + flow_classify_parse_flow( + const struct rte_flow_attr *attr, + const struct rte_flow_item pattern[], + const struct rte_flow_action actions[], + struct rte_flow_error *error) + +Adding Flow Rules +~~~~~~~~~~~~~~~~~ + +The ``rte_flow_classify_table_entry_add`` API creates an +``rte_flow_classify`` object which contains the flow_classify id and type, the +action, a union of add and delete keys and a union of rules. +It uses the ``flow_classify_parse_flow`` internal function for parsing the +flow parameters. +The 5-tuple ACL key data is obtained from the ``rte_eth_ntuple_filter`` +structure populated by the ``classify_parse_ntuple_filter`` function which +parses the Flow rule. + +.. code-block:: c + + struct acl_keys { + struct rte_table_acl_rule_add_params key_add; /* add key */ + struct rte_table_acl_rule_delete_params key_del; /* delete key */ + }; + + struct classify_rules { + enum rte_flow_classify_rule_type type; + union { + struct rte_flow_classify_ipv4_5tuple ipv4_5tuple; + } u; + }; + + struct rte_flow_classify { + uint32_t id; /* unique ID of classify object */ + struct rte_flow_action action; /* action when match found */ + struct classify_rules rules; /* union of rules */ + union { + struct acl_keys key; + } u; + int key_found; /* rule key found in table */ + void *entry; /* pointer to buffer to hold rule meta data */ + void *entry_ptr; /* handle to the table entry for rule meta data */ + }; + +It then calls the ``table[table_id].ops.f_add`` API to add the rule to the ACL +table. + +Deleting Flow Rules +~~~~~~~~~~~~~~~~~~~ + +The ``rte_flow_classify_table_entry_delete`` API calls the +``table[table_id].ops.f_delete`` API to delete a rule from the ACL table. + +Packet Matching +~~~~~~~~~~~~~~~ + +The ``rte_flow_classifier_query`` API is used to find packets which match a +given flow Flow rule in the table. +This API calls the flow_classify_run internal function which calls the +``table[table_id].ops.f_lookup`` API to see if any packets in a burst match any +of the Flow rules in the table. +The meta data for the highest priority rule matched for each packet is returned +in the entries array in the ``rte_flow_classify`` object. +The internal function ``action_apply`` implements the ``Count`` action which is +used to return data which matches a particular Flow rule. + +The rte_flow_classifier_query API uses the following structures to return data +to the application. + +.. code-block:: c + + /** IPv4 5-tuple data */ + struct rte_flow_classify_ipv4_5tuple { + uint32_t dst_ip; /**< Destination IP address in big endian. */ + uint32_t dst_ip_mask; /**< Mask of destination IP address. */ + uint32_t src_ip; /**< Source IP address in big endian. */ + uint32_t src_ip_mask; /**< Mask of destination IP address. */ + uint16_t dst_port; /**< Destination port in big endian. */ + uint16_t dst_port_mask; /**< Mask of destination port. */ + uint16_t src_port; /**< Source Port in big endian. */ + uint16_t src_port_mask; /**< Mask of source port. */ + uint8_t proto; /**< L4 protocol. */ + uint8_t proto_mask; /**< Mask of L4 protocol. */ + }; + + /** + * Flow stats + * + * For the count action, stats can be returned by the query API. + * + * Storage for stats is provided by the application. + * + * + */ + struct rte_flow_classify_stats { + void *stats; + }; + + struct rte_flow_classify_5tuple_stats { + /** count of packets that match IPv4 5tuple pattern */ + uint64_t counter1; + /** IPv4 5tuple data */ + struct rte_flow_classify_ipv4_5tuple ipv4_5tuple; + }; diff --git a/doc/guides/prog_guide/generic_segmentation_offload_lib.rst b/doc/guides/prog_guide/generic_segmentation_offload_lib.rst new file mode 100644 index 00000000..4bbf01a4 --- /dev/null +++ b/doc/guides/prog_guide/generic_segmentation_offload_lib.rst @@ -0,0 +1,257 @@ +.. BSD LICENSE + Copyright(c) 2017 Intel Corporation. All rights reserved. + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of Intel Corporation nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +Generic Segmentation Offload Library +==================================== + +Overview +-------- +Generic Segmentation Offload (GSO) is a widely used software implementation of +TCP Segmentation Offload (TSO), which reduces per-packet processing overhead. +Much like TSO, GSO gains performance by enabling upper layer applications to +process a smaller number of large packets (e.g. MTU size of 64KB), instead of +processing higher numbers of small packets (e.g. MTU size of 1500B), thus +reducing per-packet overhead. + +For example, GSO allows guest kernel stacks to transmit over-sized TCP segments +that far exceed the kernel interface's MTU; this eliminates the need to segment +packets within the guest, and improves the data-to-overhead ratio of both the +guest-host link, and PCI bus. The expectation of the guest network stack in this +scenario is that segmentation of egress frames will take place either in the NIC +HW, or where that hardware capability is unavailable, either in the host +application, or network stack. + +Bearing that in mind, the GSO library enables DPDK applications to segment +packets in software. Note however, that GSO is implemented as a standalone +library, and not via a 'fallback' mechanism (i.e. for when TSO is unsupported +in the underlying hardware); that is, applications must explicitly invoke the +GSO library to segment packets. The size of GSO segments ``(segsz)`` is +configurable by the application. + +Limitations +----------- + +#. The GSO library doesn't check if input packets have correct checksums. + +#. In addition, the GSO library doesn't re-calculate checksums for segmented + packets (that task is left to the application). + +#. IP fragments are unsupported by the GSO library. + +#. The egress interface's driver must support multi-segment packets. + +#. Currently, the GSO library supports the following IPv4 packet types: + + - TCP + - VxLAN + - GRE + + See `Supported GSO Packet Types`_ for further details. + +Packet Segmentation +------------------- + +The ``rte_gso_segment()`` function is the GSO library's primary +segmentation API. + +Before performing segmentation, an application must create a GSO context object +``(struct rte_gso_ctx)``, which provides the library with some of the +information required to understand how the packet should be segmented. Refer to +`How to Segment a Packet`_ for additional details on same. Once the GSO context +has been created, and populated, the application can then use the +``rte_gso_segment()`` function to segment packets. + +The GSO library typically stores each segment that it creates in two parts: the +first part contains a copy of the original packet's headers, while the second +part contains a pointer to an offset within the original packet. This mechanism +is explained in more detail in `GSO Output Segment Format`_. + +The GSO library supports both single- and multi-segment input mbufs. + +GSO Output Segment Format +~~~~~~~~~~~~~~~~~~~~~~~~~ +To reduce the number of expensive memcpy operations required when segmenting a +packet, the GSO library typically stores each segment that it creates as a +two-part mbuf (technically, this is termed a 'two-segment' mbuf; however, since +the elements produced by the API are also called 'segments', for clarity the +term 'part' is used here instead). + +The first part of each output segment is a direct mbuf and contains a copy of +the original packet's headers, which must be prepended to each output segment. +These headers are copied from the original packet into each output segment. + +The second part of each output segment, represents a section of data from the +original packet, i.e. a data segment. Rather than copy the data directly from +the original packet into the output segment (which would impact performance +considerably), the second part of each output segment is an indirect mbuf, +which contains no actual data, but simply points to an offset within the +original packet. + +The combination of the 'header' segment and the 'data' segment constitutes a +single logical output GSO segment of the original packet. This is illustrated +in :numref:`figure_gso-output-segment-format`. + +.. _figure_gso-output-segment-format: + +.. figure:: img/gso-output-segment-format.svg + :align: center + + Two-part GSO output segment + +In one situation, the output segment may contain additional 'data' segments. +This only occurs when: + +- the input packet on which GSO is to be performed is represented by a + multi-segment mbuf. + +- the output segment is required to contain data that spans the boundaries + between segments of the input multi-segment mbuf. + +The GSO library traverses each segment of the input packet, and produces +numerous output segments; for optimal performance, the number of output +segments is kept to a minimum. Consequently, the GSO library maximizes the +amount of data contained within each output segment; i.e. each output segment +``segsz`` bytes of data. The only exception to this is in the case of the very +final output segment; if ``pkt_len`` % ``segsz``, then the final segment is +smaller than the rest. + +In order for an output segment to meet its MSS, it may need to include data from +multiple input segments. Due to the nature of indirect mbufs (each indirect mbuf +can point to only one direct mbuf), the solution here is to add another indirect +mbuf to the output segment; this additional segment then points to the next +input segment. If necessary, this chaining process is repeated, until the sum of +all of the data 'contained' in the output segment reaches ``segsz``. This +ensures that the amount of data contained within each output segment is uniform, +with the possible exception of the last segment, as previously described. + +:numref:`figure_gso-three-seg-mbuf` illustrates an example of a three-part +output segment. In this example, the output segment needs to include data from +the end of one input segment, and the beginning of another. To achieve this, +an additional indirect mbuf is chained to the second part of the output segment, +and is attached to the next input segment (i.e. it points to the data in the +next input segment). + +.. _figure_gso-three-seg-mbuf: + +.. figure:: img/gso-three-seg-mbuf.svg + :align: center + + Three-part GSO output segment + +Supported GSO Packet Types +-------------------------- + +TCP/IPv4 GSO +~~~~~~~~~~~~ +TCP/IPv4 GSO supports segmentation of suitably large TCP/IPv4 packets, which +may also contain an optional VLAN tag. + +VxLAN GSO +~~~~~~~~~ +VxLAN packets GSO supports segmentation of suitably large VxLAN packets, +which contain an outer IPv4 header, inner TCP/IPv4 headers, and optional +inner and/or outer VLAN tag(s). + +GRE GSO +~~~~~~~ +GRE GSO supports segmentation of suitably large GRE packets, which contain +an outer IPv4 header, inner TCP/IPv4 headers, and an optional VLAN tag. + +How to Segment a Packet +----------------------- + +To segment an outgoing packet, an application must: + +#. First create a GSO context ``(struct rte_gso_ctx)``; this contains: + + - a pointer to the mbuf pool for allocating the direct buffers, which are + used to store the GSO segments' packet headers. + + - a pointer to the mbuf pool for allocating indirect buffers, which are + used to locate GSO segments' packet payloads. + + .. note:: + + An application may use the same pool for both direct and indirect + buffers. However, since indirect mbufs simply store a pointer, the + application may reduce its memory consumption by creating a separate memory + pool, containing smaller elements, for the indirect pool. + + + - the size of each output segment, including packet headers and payload, + measured in bytes. + + - the bit mask of required GSO types. The GSO library uses the same macros as + those that describe a physical device's TX offloading capabilities (i.e. + ``DEV_TX_OFFLOAD_*_TSO``) for gso_types. For example, if an application + wants to segment TCP/IPv4 packets, it should set gso_types to + ``DEV_TX_OFFLOAD_TCP_TSO``. The only other supported values currently + supported for gso_types are ``DEV_TX_OFFLOAD_VXLAN_TNL_TSO``, and + ``DEV_TX_OFFLOAD_GRE_TNL_TSO``; a combination of these macros is also + allowed. + + - a flag, that indicates whether the IPv4 headers of output segments should + contain fixed or incremental ID values. + +2. Set the appropriate ol_flags in the mbuf. + + - The GSO library use the value of an mbuf's ``ol_flags`` attribute to + to determine how a packet should be segmented. It is the application's + responsibility to ensure that these flags are set. + + - For example, in order to segment TCP/IPv4 packets, the application should + add the ``PKT_TX_IPV4`` and ``PKT_TX_TCP_SEG`` flags to the mbuf's + ol_flags. + + - If checksum calculation in hardware is required, the application should + also add the ``PKT_TX_TCP_CKSUM`` and ``PKT_TX_IP_CKSUM`` flags. + +#. Check if the packet should be processed. Packets with one of the + following properties are not processed and are returned immediately: + + - Packet length is less than ``segsz`` (i.e. GSO is not required). + + - Packet type is not supported by GSO library (see + `Supported GSO Packet Types`_). + + - Application has not enabled GSO support for the packet type. + + - Packet's ol_flags have been incorrectly set. + +#. Allocate space in which to store the output GSO segments. If the amount of + space allocated by the application is insufficient, segmentation will fail. + +#. Invoke the GSO segmentation API, ``rte_gso_segment()``. + +#. If required, update the L3 and L4 checksums of the newly-created segments. + For tunneled packets, the outer IPv4 headers' checksums should also be + updated. Alternatively, the application may offload checksum calculation + to HW. + diff --git a/doc/guides/prog_guide/img/gso-output-segment-format.svg b/doc/guides/prog_guide/img/gso-output-segment-format.svg new file mode 100644 index 00000000..bdb5ec33 --- /dev/null +++ b/doc/guides/prog_guide/img/gso-output-segment-format.svg @@ -0,0 +1,313 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"> +<!-- Generated by Microsoft Visio, SVG Export gso-output-segment-format.svg Page-1 --> +<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events" + xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" width="19.3975in" height="8.21796in" + viewBox="0 0 1396.62 591.693" xml:space="preserve" color-interpolation-filters="sRGB" class="st21"> + <v:documentProperties v:langID="1033" v:metric="true" v:viewMarkup="false"/> + + <style type="text/css"> + <![CDATA[ + .st1 {fill:#006fc5;stroke:#006fc5;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.0552552} + .st2 {fill:#ffffff;stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75} + .st3 {stroke:#c3d600;stroke-linecap:round;stroke-linejoin:round;stroke-width:3.68828} + .st4 {fill:#c3d600;stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75} + .st5 {stroke:#8f9d00;stroke-linecap:round;stroke-linejoin:round;stroke-width:3.75735} + .st6 {fill:#00aeef;stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75} + .st7 {stroke:#007fb0;stroke-linecap:round;stroke-linejoin:round;stroke-width:3.75735} + .st8 {stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75} + .st9 {fill:#ffffff;font-family:Intel Clear;font-size:1.99999em;font-weight:bold} + .st10 {fill:#000000;stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.0552552} + .st11 {fill:#ffffff;font-family:Intel Clear;font-size:2.44732em;font-weight:bold} + .st12 {fill:none;stroke:#ffffff;stroke-linecap:round;stroke-linejoin:round;stroke-width:5.52552} + .st13 {fill:#000000;font-family:Intel Clear;font-size:2.15291em} + .st14 {fill:#000000;font-family:Intel Clear;font-size:1.8401em} + .st15 {fill:#006fc5;stroke:#006fc5;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.0276276} + .st16 {fill:#c3d600;font-family:Intel Clear;font-size:2.44732em} + .st17 {fill:#ffc000;font-family:Intel Clear;font-size:2.44732em} + .st18 {fill:#ffc000;stroke:#ffc000;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.0276276} + .st19 {fill:#0070c0;font-family:Intel Clear;font-size:1.8401em} + .st20 {fill:#006fc5;font-family:Intel Clear;font-size:1.61927em} + .st21 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3} + ]]> + </style> + + <g v:mID="0" v:index="1" v:groupContext="foregroundPage"> + <title>Page-1</title> + <v:pageProperties v:drawingScale="0.0393701" v:pageScale="0.0393701" v:drawingUnits="24" v:shadowOffsetX="8.50394" + v:shadowOffsetY="-8.50394"/> + <g id="shape3-1" v:mID="3" v:groupContext="shape" transform="translate(577.244,-560.42)"> + <title>Sheet.3</title> + <path d="M9.24 585.29 L16.32 585.29 L16.32 587.06 L9.24 587.06 L9.24 585.29 L9.24 585.29 ZM21.63 585.29 L23.4 585.29 + L23.4 587.06 L21.63 587.06 L21.63 585.29 L21.63 585.29 ZM28.7 585.29 L35.78 585.29 L35.78 587.06 L28.7 587.06 + L28.7 585.29 L28.7 585.29 ZM41.09 585.29 L42.86 585.29 L42.86 587.06 L41.09 587.06 L41.09 585.29 L41.09 + 585.29 ZM48.17 585.29 L55.25 585.29 L55.25 587.06 L48.17 587.06 L48.17 585.29 L48.17 585.29 ZM60.56 585.29 + L62.33 585.29 L62.33 587.06 L60.56 587.06 L60.56 585.29 L60.56 585.29 ZM67.64 585.29 L74.72 585.29 L74.72 + 587.06 L67.64 587.06 L67.64 585.29 L67.64 585.29 ZM80.03 585.29 L81.8 585.29 L81.8 587.06 L80.03 587.06 + L80.03 585.29 L80.03 585.29 ZM87.11 585.29 L94.19 585.29 L94.19 587.06 L87.11 587.06 L87.11 585.29 L87.11 + 585.29 ZM99.5 585.29 L101.27 585.29 L101.27 587.06 L99.5 587.06 L99.5 585.29 L99.5 585.29 ZM106.58 585.29 + L113.66 585.29 L113.66 587.06 L106.58 587.06 L106.58 585.29 L106.58 585.29 ZM118.97 585.29 L120.74 585.29 + L120.74 587.06 L118.97 587.06 L118.97 585.29 L118.97 585.29 ZM126.05 585.29 L133.13 585.29 L133.13 587.06 + L126.05 587.06 L126.05 585.29 L126.05 585.29 ZM138.43 585.29 L140.2 585.29 L140.2 587.06 L138.43 587.06 + L138.43 585.29 L138.43 585.29 ZM145.51 585.29 L152.59 585.29 L152.59 587.06 L145.51 587.06 L145.51 585.29 + L145.51 585.29 ZM157.9 585.29 L159.67 585.29 L159.67 587.06 L157.9 587.06 L157.9 585.29 L157.9 585.29 ZM164.98 + 585.29 L172.06 585.29 L172.06 587.06 L164.98 587.06 L164.98 585.29 L164.98 585.29 ZM177.37 585.29 L179.14 + 585.29 L179.14 587.06 L177.37 587.06 L177.37 585.29 L177.37 585.29 ZM184.45 585.29 L191.53 585.29 L191.53 + 587.06 L184.45 587.06 L184.45 585.29 L184.45 585.29 ZM196.84 585.29 L198.61 585.29 L198.61 587.06 L196.84 + 587.06 L196.84 585.29 L196.84 585.29 ZM203.92 585.29 L211 585.29 L211 587.06 L203.92 587.06 L203.92 585.29 + L203.92 585.29 ZM216.31 585.29 L218.08 585.29 L218.08 587.06 L216.31 587.06 L216.31 585.29 L216.31 585.29 + ZM223.39 585.29 L230.47 585.29 L230.47 587.06 L223.39 587.06 L223.39 585.29 L223.39 585.29 ZM235.78 585.29 + L237.55 585.29 L237.55 587.06 L235.78 587.06 L235.78 585.29 L235.78 585.29 ZM242.86 585.29 L249.93 585.29 + L249.93 587.06 L242.86 587.06 L242.86 585.29 L242.86 585.29 ZM255.24 585.29 L257.01 585.29 L257.01 587.06 + L255.24 587.06 L255.24 585.29 L255.24 585.29 ZM262.32 585.29 L269.4 585.29 L269.4 587.06 L262.32 587.06 + L262.32 585.29 L262.32 585.29 ZM274.71 585.29 L276.48 585.29 L276.48 587.06 L274.71 587.06 L274.71 585.29 + L274.71 585.29 ZM281.79 585.29 L288.87 585.29 L288.87 587.06 L281.79 587.06 L281.79 585.29 L281.79 585.29 + ZM294.18 585.29 L295.95 585.29 L295.95 587.06 L294.18 587.06 L294.18 585.29 L294.18 585.29 ZM301.26 585.29 + L308.34 585.29 L308.34 587.06 L301.26 587.06 L301.26 585.29 L301.26 585.29 ZM313.65 585.29 L315.42 585.29 + L315.42 587.06 L313.65 587.06 L313.65 585.29 L313.65 585.29 ZM320.73 585.29 L324.99 585.29 L324.99 587.06 + L320.73 587.06 L320.73 585.29 L320.73 585.29 ZM11.06 591.69 L0 586.17 L11.06 580.65 L11.06 591.69 L11.06 + 591.69 ZM323.16 580.65 L334.22 586.17 L323.16 591.69 L323.16 580.65 L323.16 580.65 Z" class="st1"/> + </g> + <g id="shape4-3" v:mID="4" v:groupContext="shape" transform="translate(184.298,-201.906)"> + <title>Sheet.4</title> + <path d="M94.04 570.43 L117.87 557.26 L0 344.58 L47.68 318.26 L165.55 530.94 L189.39 517.79 L168.08 591.69 L94.04 570.43 + Z" class="st2"/> + </g> + <g id="shape5-5" v:mID="5" v:groupContext="shape" transform="translate(184.298,-201.906)"> + <title>Sheet.5</title> + <path d="M94.04 570.43 L117.87 557.26 L0 344.58 L47.68 318.26 L165.55 530.94 L189.39 517.79 L168.08 591.69 L94.04 570.43" + class="st3"/> + </g> + <g id="shape6-8" v:mID="6" v:groupContext="shape" transform="translate(119.408,-447.917)"> + <title>Sheet.6</title> + <path d="M0 510.21 L0 591.69 L129.86 591.69 L129.86 510.21 L0 510.21 L0 510.21 Z" class="st4"/> + </g> + <g id="shape7-10" v:mID="7" v:groupContext="shape" transform="translate(119.408,-447.917)"> + <title>Sheet.7</title> + <path d="M0 510.21 L129.86 510.21 L129.86 591.69 L0 591.69 L0 510.21" class="st5"/> + </g> + <g id="shape10-13" v:mID="10" v:groupContext="shape" transform="translate(250.819,-447.917)"> + <title>Sheet.10</title> + <path d="M0 510.21 L0 591.69 L822.53 591.69 L822.53 510.21 L0 510.21 L0 510.21 Z" class="st6"/> + </g> + <g id="shape11-15" v:mID="11" v:groupContext="shape" transform="translate(250.819,-447.917)"> + <title>Sheet.11</title> + <path d="M0 510.21 L822.53 510.21 L822.53 591.69 L0 591.69 L0 510.21" class="st7"/> + </g> + <g id="shape12-18" v:mID="12" v:groupContext="shape" transform="translate(255.478,-470.123)"> + <title>Sheet.12</title> + <desc>Payload 0</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="157.315" cy="574.07" width="314.63" height="35.245"/> + <path d="M314.63 556.45 L0 556.45 L0 591.69 L314.63 591.69 L314.63 556.45" class="st8"/> + <text x="102.08" y="581.27" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Payload 0</text> </g> + <g id="shape13-22" v:mID="13" v:groupContext="shape" transform="translate(577.354,-470.123)"> + <title>Sheet.13</title> + <desc>Payload 1</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="167.112" cy="574.07" width="334.23" height="35.245"/> + <path d="M334.22 556.45 L0 556.45 L0 591.69 L334.22 591.69 L334.22 556.45" class="st8"/> + <text x="111.88" y="581.27" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Payload 1</text> </g> + <g id="shape14-26" v:mID="14" v:groupContext="shape" transform="translate(910.635,-470.956)"> + <title>Sheet.14</title> + <desc>Payload 2</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="81.8509" cy="574.07" width="163.71" height="35.245"/> + <path d="M163.7 556.45 L0 556.45 L0 591.69 L163.7 591.69 L163.7 556.45" class="st8"/> + <text x="26.61" y="581.27" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Payload 2</text> </g> + <g id="shape15-30" v:mID="15" v:groupContext="shape" transform="translate(909.144,-453.824)"> + <title>Sheet.15</title> + <path d="M1.16 453.85 L1.05 465.33 L3.93 465.39 L4.04 453.91 L1.16 453.85 L1.16 453.85 ZM1 473.95 L0.94 476.82 L3.82 + 476.87 L3.87 474 L1 473.95 L1 473.95 ZM0.88 485.43 L0.77 496.91 L3.65 496.96 L3.76 485.48 L0.88 485.43 L0.88 + 485.43 ZM0.72 505.52 L0.72 508.39 L3.59 508.45 L3.59 505.58 L0.72 505.52 L0.72 505.52 ZM0.61 517 L0.55 528.49 + L3.43 528.54 L3.48 517.06 L0.61 517 L0.61 517 ZM0.44 537.1 L0.44 539.97 L3.32 540.02 L3.32 537.15 L0.44 + 537.1 L0.44 537.1 ZM0.39 548.58 L0.28 560.06 L3.15 560.12 L3.26 548.63 L0.39 548.58 L0.39 548.58 ZM0.22 + 568.67 L0.17 571.54 L3.04 571.6 L3.1 568.73 L0.22 568.67 L0.22 568.67 ZM0.11 580.16 L0 591.64 L2.88 591.69 + L2.99 580.21 L0.11 580.16 L0.11 580.16 Z" class="st10"/> + </g> + <g id="shape16-32" v:mID="16" v:groupContext="shape" transform="translate(119.187,-447.917)"> + <title>Sheet.16</title> + <path d="M0 510.21 L0 591.69 L129.86 591.69 L129.86 510.21 L0 510.21 L0 510.21 Z" class="st4"/> + </g> + <g id="shape17-34" v:mID="17" v:groupContext="shape" transform="translate(119.187,-447.917)"> + <title>Sheet.17</title> + <path d="M0 510.21 L129.86 510.21 L129.86 591.69 L0 591.69 L0 510.21" class="st5"/> + </g> + <g id="shape18-37" v:mID="18" v:groupContext="shape" transform="translate(121.944,-471.034)"> + <title>Sheet.18</title> + <desc>Header</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="61.0973" cy="574.07" width="122.2" height="35.245"/> + <path d="M122.19 556.45 L0 556.45 L0 591.69 L122.19 591.69 L122.19 556.45" class="st8"/> + <text x="20.61" y="581.27" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Header</text> </g> + <g id="shape19-41" v:mID="19" v:groupContext="shape" transform="translate(329.798,-1.87868)"> + <title>Sheet.19</title> + <path d="M0 510.43 L0 591.69 L289.81 591.69 L289.81 510.43 L0 510.43 L0 510.43 Z" class="st4"/> + </g> + <g id="shape20-43" v:mID="20" v:groupContext="shape" transform="translate(329.798,-1.87868)"> + <title>Sheet.20</title> + <path d="M0 510.43 L289.81 510.43 L289.81 591.69 L0 591.69 L0 510.43" class="st5"/> + </g> + <g id="shape21-46" v:mID="21" v:groupContext="shape" transform="translate(424.908,-21.567)"> + <title>Sheet.21</title> + <desc>Header</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="61.0973" cy="574.07" width="122.2" height="35.245"/> + <path d="M122.19 556.45 L0 556.45 L0 591.69 L122.19 591.69 L122.19 556.45" class="st8"/> + <text x="11.55" y="582.88" class="st11" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Header</text> </g> + <g id="shape22-50" v:mID="22" v:groupContext="shape" transform="translate(619.609,-1.87868)"> + <title>Sheet.22</title> + <path d="M0 510.43 L0 591.69 L453.74 591.69 L453.74 510.43 L0 510.43 L0 510.43 Z" class="st6"/> + </g> + <g id="shape23-52" v:mID="23" v:groupContext="shape" transform="translate(619.609,-1.87868)"> + <title>Sheet.23</title> + <path d="M0 510.43 L453.74 510.43 L453.74 591.69 L0 591.69 L0 510.43" class="st7"/> + </g> + <g id="shape24-55" v:mID="24" v:groupContext="shape" transform="translate(778.624,-21.5672)"> + <title>Sheet.24</title> + <desc>Payload 1</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="81.8509" cy="574.07" width="163.71" height="35.245"/> + <path d="M163.7 556.45 L0 556.45 L0 591.69 L163.7 591.69 L163.7 556.45" class="st8"/> + <text x="14.26" y="582.88" class="st11" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Payload 1</text> </g> + <g id="shape25-59" v:mID="25" v:groupContext="shape" transform="translate(710.092,-113.83)"> + <title>Sheet.25</title> + <path d="M0 522.69 C0 515.07 6.19 508.89 13.83 508.89 L349.43 508.89 C357.12 508.89 363.26 515.07 363.26 522.69 L363.26 + 577.89 C363.26 585.57 357.12 591.69 349.43 591.69 L13.83 591.69 C6.19 591.69 0 585.57 0 577.89 L0 522.69 + Z" class="st6"/> + </g> + <g id="shape26-61" v:mID="26" v:groupContext="shape" transform="translate(710.092,-113.83)"> + <title>Sheet.26</title> + <path d="M0 522.69 C0 515.07 6.19 508.89 13.83 508.89 L349.43 508.89 C357.12 508.89 363.26 515.07 363.26 522.69 L363.26 + 577.89 C363.26 585.57 357.12 591.69 349.43 591.69 L13.83 591.69 C6.19 591.69 0 585.57 0 577.89 L0 522.69 + Z" class="st12"/> + </g> + <g id="shape27-63" v:mID="27" v:groupContext="shape" transform="translate(813.057,-150.108)"> + <title>Sheet.27</title> + <desc>Indirect mbuf</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="94.1386" cy="576.19" width="188.28" height="31.0055"/> + <path d="M188.28 560.69 L0 560.69 L0 591.69 L188.28 591.69 L188.28 560.69" class="st8"/> + <text x="15.43" y="583.94" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Indirect mbuf</text> </g> + <g id="shape28-67" v:mID="28" v:groupContext="shape" transform="translate(810.845,-123.854)"> + <title>Sheet.28</title> + <desc>(pointer to data)</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="95.5065" cy="578.442" width="191.02" height="26.501"/> + <path d="M191.01 565.19 L0 565.19 L0 591.69 L191.01 591.69 L191.01 565.19" class="st8"/> + <text x="15.15" y="585.07" class="st14" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>(pointer to data)</text> </g> + <g id="shape29-71" v:mID="29" v:groupContext="shape" transform="translate(573.151,-149.601)"> + <title>Sheet.29</title> + <path d="M0 584.74 L127.76 584.74 L127.76 587.61 L0 587.61 L0 584.74 L0 584.74 ZM125.91 580.65 L136.97 586.17 L125.91 + 591.69 L125.91 580.65 L125.91 580.65 Z" class="st15"/> + </g> + <g id="shape30-73" v:mID="30" v:groupContext="shape" transform="translate(0,-309.671)"> + <title>Sheet.30</title> + <desc>Memory copy</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="108.076" cy="574.07" width="216.16" height="35.245"/> + <path d="M216.15 556.45 L0 556.45 L0 591.69 L216.15 591.69 L216.15 556.45" class="st8"/> + <text x="17.68" y="582.88" class="st16" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Memory copy</text> </g> + <g id="shape31-77" v:mID="31" v:groupContext="shape" transform="translate(680.77,-305.707)"> + <title>Sheet.31</title> + <desc>No Memory Copy</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="136.547" cy="574.07" width="273.1" height="35.245"/> + <path d="M273.09 556.45 L0 556.45 L0 591.69 L273.09 591.69 L273.09 556.45" class="st8"/> + <text x="21.4" y="582.88" class="st17" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>No Memory Copy</text> </g> + <g id="shape32-81" v:mID="32" v:groupContext="shape" transform="translate(1102.72,-26.7532)"> + <title>Sheet.32</title> + <desc>Logical output segment</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="138.243" cy="578.442" width="276.49" height="26.501"/> + <path d="M276.49 565.19 L0 565.19 L0 591.69 L276.49 591.69 L276.49 565.19" class="st8"/> + <text x="20.73" y="585.07" class="st14" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Logical output segment</text> </g> + <g id="shape36-85" v:mID="36" v:groupContext="shape" transform="translate(1106.81,-138.647)"> + <title>Sheet.36</title> + <desc>Two-part output segment</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="144.906" cy="578.442" width="289.82" height="26.501"/> + <path d="M289.81 565.19 L0 565.19 L0 591.69 L289.81 591.69 L289.81 565.19" class="st8"/> + <text x="16.56" y="585.07" class="st14" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Two-part output segment</text> </g> + <g id="shape37-89" v:mID="37" v:groupContext="shape" transform="translate(575.916,-453.879)"> + <title>Sheet.37</title> + <path d="M2.88 453.91 L2.9 465.39 L0.03 465.39 L0 453.91 L2.88 453.91 L2.88 453.91 ZM2.9 474 L2.9 476.87 L0.03 476.87 + L0.03 474 L2.9 474 L2.9 474 ZM2.9 485.48 L2.9 496.96 L0.03 496.96 L0.03 485.48 L2.9 485.48 L2.9 485.48 ZM2.9 + 505.58 L2.9 508.45 L0.03 508.45 L0.03 505.58 L2.9 505.58 L2.9 505.58 ZM2.9 517.06 L2.9 528.54 L0.03 528.54 + L0.03 517.06 L2.9 517.06 L2.9 517.06 ZM2.9 537.15 L2.9 540.02 L0.03 540.02 L0.03 537.15 L2.9 537.15 L2.9 + 537.15 ZM2.9 548.63 L2.9 560.12 L0.03 560.12 L0.03 548.63 L2.9 548.63 L2.9 548.63 ZM2.9 568.73 L2.9 571.6 + L0.03 571.6 L0.03 568.73 L2.9 568.73 L2.9 568.73 ZM2.9 580.21 L2.9 591.69 L0.03 591.69 L0.03 580.21 L2.9 + 580.21 L2.9 580.21 Z" class="st18"/> + </g> + <g id="shape38-91" v:mID="38" v:groupContext="shape" transform="translate(577.354,-193.764)"> + <title>Sheet.38</title> + <path d="M5.59 347.01 L10.92 357.16 L8.38 358.52 L3.04 348.36 L5.59 347.01 L5.59 347.01 ZM14.96 364.78 L16.29 367.32 + L13.74 368.67 L12.42 366.13 L14.96 364.78 L14.96 364.78 ZM20.33 374.97 L25.66 385.12 L23.12 386.45 L17.78 + 376.29 L20.33 374.97 L20.33 374.97 ZM29.7 392.74 L31.03 395.28 L28.48 396.61 L27.16 394.07 L29.7 392.74 + L29.7 392.74 ZM35.04 402.9 L40.4 413.06 L37.86 414.38 L32.49 404.22 L35.04 402.9 L35.04 402.9 ZM44.41 420.67 + L45.77 423.21 L43.22 424.57 L41.87 422.03 L44.41 420.67 L44.41 420.67 ZM49.78 430.83 L55.14 440.99 L52.6 + 442.34 L47.23 432.18 L49.78 430.83 L49.78 430.83 ZM59.15 448.61 L60.51 451.15 L57.96 452.5 L56.61 449.96 + L59.15 448.61 L59.15 448.61 ZM64.52 458.79 L69.88 468.95 L67.34 470.27 L61.97 460.12 L64.52 458.79 L64.52 + 458.79 ZM73.89 476.57 L75.25 479.11 L72.7 480.43 L71.35 477.89 L73.89 476.57 L73.89 476.57 ZM79.26 486.72 + L84.62 496.88 L82.08 498.21 L76.71 488.05 L79.26 486.72 L79.26 486.72 ZM88.63 504.5 L89.96 507.04 L87.41 + 508.39 L86.09 505.85 L88.63 504.5 L88.63 504.5 ZM94 514.66 L99.33 524.81 L96.79 526.17 L91.45 516.01 L94 + 514.66 L94 514.66 ZM103.37 532.43 L104.7 534.97 L102.15 536.32 L100.83 533.79 L103.37 532.43 L103.37 532.43 + ZM108.73 542.62 L114.07 552.77 L111.53 554.1 L106.19 543.94 L108.73 542.62 L108.73 542.62 ZM118.11 560.39 + L119.44 562.93 L116.89 564.26 L115.57 561.72 L118.11 560.39 L118.11 560.39 ZM123.45 570.55 L128.81 580.71 + L126.27 582.03 L120.9 571.87 L123.45 570.55 L123.45 570.55 ZM132.82 588.33 L133.9 590.37 L131.36 591.69 + L130.28 589.68 L132.82 588.33 L132.82 588.33 ZM0.28 351.89 L0 339.53 L10.07 346.73 L0.28 351.89 L0.28 351.89 + Z" class="st18"/> + </g> + <g id="shape39-93" v:mID="39" v:groupContext="shape" transform="translate(329.798,-113.83)"> + <title>Sheet.39</title> + <path d="M0 522.69 C0 515.07 6.19 508.89 13.83 508.89 L229.53 508.89 C237.19 508.89 243.35 515.07 243.35 522.69 L243.35 + 577.89 C243.35 585.54 237.19 591.69 229.53 591.69 L13.83 591.69 C6.19 591.69 0 585.54 0 577.89 L0 522.69 + Z" class="st4"/> + </g> + <g id="shape40-95" v:mID="40" v:groupContext="shape" transform="translate(329.798,-113.83)"> + <title>Sheet.40</title> + <path d="M0 522.69 C0 515.07 6.19 508.89 13.83 508.89 L229.53 508.89 C237.19 508.89 243.35 515.07 243.35 522.69 L243.35 + 577.89 C243.35 585.54 237.19 591.69 229.53 591.69 L13.83 591.69 C6.19 591.69 0 585.54 0 577.89 L0 522.69 + Z" class="st12"/> + </g> + <g id="shape41-97" v:mID="41" v:groupContext="shape" transform="translate(368.774,-150.453)"> + <title>Sheet.41</title> + <desc>Direct mbuf</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="82.7002" cy="576.19" width="165.41" height="31.0055"/> + <path d="M165.4 560.69 L0 560.69 L0 591.69 L165.4 591.69 L165.4 560.69" class="st8"/> + <text x="13.94" y="583.94" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Direct mbuf</text> </g> + <g id="shape42-101" v:mID="42" v:groupContext="shape" transform="translate(351.856,-123.854)"> + <title>Sheet.42</title> + <desc>(copy of headers)</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="102.121" cy="578.442" width="204.25" height="26.501"/> + <path d="M204.24 565.19 L0 565.19 L0 591.69 L204.24 591.69 L204.24 565.19" class="st8"/> + <text x="16.02" y="585.07" class="st14" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>(copy of headers)</text> </g> + <g id="shape43-105" v:mID="43" v:groupContext="shape" transform="translate(619.797,-155.563)"> + <title>Sheet.43</title> + <desc>next</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="28.011" cy="578.442" width="56.03" height="26.501"/> + <path d="M56.02 565.19 L0 565.19 L0 591.69 L56.02 591.69 L56.02 565.19" class="st8"/> + <text x="6.35" y="585.07" class="st19" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>next</text> </g> + <g id="shape44-109" v:mID="44" v:groupContext="shape" transform="translate(700.911,-551.367)"> + <title>Sheet.44</title> + <path d="M0 559.23 L0 591.69 L84.29 591.69 L84.29 559.23 L0 559.23 L0 559.23 Z" class="st2"/> + </g> + <g id="shape45-111" v:mID="45" v:groupContext="shape" transform="translate(709.883,-555.163)"> + <title>Sheet.45</title> + <desc>segsz</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="30.7501" cy="580.032" width="61.51" height="23.3211"/> + <path d="M61.5 568.37 L0 568.37 L0 591.69 L61.5 591.69 L61.5 568.37" class="st8"/> + <text x="6.38" y="585.86" class="st20" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>segsz</text> </g> + <g id="shape46-115" v:mID="46" v:groupContext="shape" transform="translate(1111.54,-477.36)"> + <title>Sheet.46</title> + <desc>Input packet</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="74.9" cy="578.442" width="149.8" height="26.501"/> + <path d="M149.8 565.19 L0 565.19 L0 591.69 L149.8 591.69 L149.8 565.19" class="st8"/> + <text x="12.47" y="585.07" class="st14" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Input packet</text> </g> + </g> +</svg> diff --git a/doc/guides/prog_guide/img/gso-three-seg-mbuf.svg b/doc/guides/prog_guide/img/gso-three-seg-mbuf.svg new file mode 100644 index 00000000..0431012d --- /dev/null +++ b/doc/guides/prog_guide/img/gso-three-seg-mbuf.svg @@ -0,0 +1,477 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"> +<!-- Generated by Microsoft Visio, SVG Export gso-three-seg-mbuf_new.svg Page-1 --> +<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events" + xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" width="25.5895in" height="9.63966in" + viewBox="0 0 1842.44 694.055" xml:space="preserve" color-interpolation-filters="sRGB" class="st23"> + <title>GSO three-part output segment</title> + <v:documentProperties v:langID="1033" v:metric="true" v:viewMarkup="false"/> + + <style type="text/css"> + <![CDATA[ + .st1 {fill:#ffc000;stroke:#ffc000;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.0329073} + .st2 {fill:#006fc5;stroke:#006fc5;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.0329073} + .st3 {fill:none;stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:3.42236} + .st4 {fill:#c3d600;stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75} + .st5 {stroke:#8f9d00;stroke-linecap:round;stroke-linejoin:round;stroke-width:4.47539} + .st6 {fill:#00aeef;stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75} + .st7 {stroke:#007fb0;stroke-linecap:round;stroke-linejoin:round;stroke-width:4.47539} + .st8 {stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75} + .st9 {fill:#ffffff;font-family:Calibri;font-size:2.08333em;font-weight:bold} + .st10 {fill:#ffffff;font-family:Intel Clear;font-size:2.91502em;font-weight:bold} + .st11 {fill:#000000;font-family:Intel Clear;font-size:2.19175em} + .st12 {fill:none;stroke:#ffffff;stroke-linecap:round;stroke-linejoin:round;stroke-width:6.58146} + .st13 {fill:#000000;font-family:Intel Clear;font-size:2.50001em} + .st14 {fill:#000000;font-family:Intel Clear;font-size:1.99999em} + .st15 {fill:#0070c0;font-family:Intel Clear;font-size:2.19175em} + .st16 {fill:#ffffff;stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75} + .st17 {fill:#006fc5;font-family:Intel Clear;font-size:1.92874em} + .st18 {fill:#000000;stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.0329073} + .st19 {fill:#0070c0;font-family:Intel Clear;font-size:1.5em} + .st20 {fill:#000000;stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.0658146} + .st21 {fill:#000000;font-family:Intel Clear;font-size:1.81915em} + .st22 {fill:#000000;font-family:Intel Clear;font-size:1.49785em} + .st23 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3} + ]]> + </style> + + <g v:mID="0" v:index="1" v:groupContext="foregroundPage"> + <title>Page-1</title> + <v:pageProperties v:drawingScale="0.0393701" v:pageScale="0.0393701" v:drawingUnits="24" v:shadowOffsetX="8.50394" + v:shadowOffsetY="-8.50394"/> + <v:layer v:name="top" v:index="0"/> + <v:layer v:name="middle" v:index="1"/> + <g id="shape111-1" v:mID="111" v:groupContext="shape" v:layerMember="0" transform="translate(787.208,-220.973)"> + <title>Sheet.111</title> + <path d="M6.65 402.61 L13.01 414.71 L9.98 416.32 L3.62 404.22 L6.65 402.61 L6.65 402.61 ZM17.82 423.78 L19.4 426.81 L16.37 + 428.42 L14.79 425.39 L17.82 423.78 L17.82 423.78 ZM24.21 435.91 L30.57 448.01 L27.54 449.59 L21.18 437.49 + L24.21 435.91 L24.21 435.91 ZM35.38 457.08 L36.96 460.11 L33.93 461.69 L32.35 458.66 L35.38 457.08 L35.38 + 457.08 ZM41.73 469.18 L48.12 481.28 L45.09 482.86 L38.7 470.76 L41.73 469.18 L41.73 469.18 ZM52.9 490.36 + L54.51 493.38 L51.48 494.99 L49.87 491.97 L52.9 490.36 L52.9 490.36 ZM59.29 502.45 L65.68 514.55 L62.65 + 516.16 L56.26 504.06 L59.29 502.45 L59.29 502.45 ZM70.46 523.63 L72.07 526.65 L69.04 528.26 L67.43 525.24 + L70.46 523.63 L70.46 523.63 ZM76.85 535.76 L83.24 547.86 L80.21 549.43 L73.82 537.34 L76.85 535.76 L76.85 + 535.76 ZM88.01 556.93 L89.63 559.95 L86.6 561.53 L84.98 558.51 L88.01 556.93 L88.01 556.93 ZM94.4 569.03 + L100.79 581.13 L97.76 582.7 L91.37 570.61 L94.4 569.03 L94.4 569.03 ZM105.57 590.2 L107.15 593.22 L104.12 + 594.84 L102.54 591.81 L105.57 590.2 L105.57 590.2 ZM111.96 602.3 L118.32 614.4 L115.28 616.01 L108.93 603.91 + L111.96 602.3 L111.96 602.3 ZM123.12 623.47 L124.71 626.5 L121.67 628.11 L120.09 625.08 L123.12 623.47 L123.12 + 623.47 ZM129.51 635.6 L135.87 647.7 L132.84 649.28 L126.48 637.18 L129.51 635.6 L129.51 635.6 ZM140.68 656.77 + L142.26 659.8 L139.23 661.38 L137.65 658.35 L140.68 656.77 L140.68 656.77 ZM147.04 668.87 L153.43 680.97 + L150.4 682.55 L144.01 670.45 L147.04 668.87 L147.04 668.87 ZM158.2 690.04 L159.49 692.48 L156.46 694.06 + L155.17 691.66 L158.2 690.04 L158.2 690.04 ZM0.33 408.43 L0 393.7 L11.99 402.28 L0.33 408.43 L0.33 408.43 + Z" class="st1"/> + </g> + <g id="shape110-3" v:mID="110" v:groupContext="shape" v:layerMember="0" transform="translate(685.078,-560.166)"> + <title>Sheet.110</title> + <path d="M0 685.77 L90.67 685.77 L90.67 689.19 L0 689.19 L0 685.77 L0 685.77 ZM89.36 680.9 L97.21 687.48 L89.36 694.06 + L89.36 680.9 L89.36 680.9 Z" class="st2"/> + </g> + <g id="shape4-5" v:mID="4" v:groupContext="shape" transform="translate(718.715,-469.955)"> + <title>Sheet.4</title> + <path d="M0 655.13 L0 678.22 C0 686.97 11.69 694.06 26.05 694.06 C40.45 694.06 52.11 686.97 52.11 678.22 L52.11 673.91 + L59.55 673.91 L44.66 664.86 L29.78 673.91 L37.22 673.91 L37.22 678.22 C37.22 681.98 32.25 685 26.05 685 + C19.89 685 14.89 681.98 14.89 678.22 L14.89 655.13 L0 655.13 Z" class="st3"/> + </g> + <g id="shape5-7" v:mID="5" v:groupContext="shape" transform="translate(547.831,-656.823)"> + <title>Sheet.5</title> + <path d="M11 686.43 L19.43 686.43 L19.43 688.53 L11 688.53 L11 686.43 L11 686.43 ZM25.76 686.43 L27.87 686.43 L27.87 + 688.53 L25.76 688.53 L25.76 686.43 L25.76 686.43 ZM34.19 686.43 L42.62 686.43 L42.62 688.53 L34.19 688.53 + L34.19 686.43 L34.19 686.43 ZM48.95 686.43 L51.05 686.43 L51.05 688.53 L48.95 688.53 L48.95 686.43 L48.95 + 686.43 ZM57.38 686.43 L65.81 686.43 L65.81 688.53 L57.38 688.53 L57.38 686.43 L57.38 686.43 ZM72.14 686.43 + L74.24 686.43 L74.24 688.53 L72.14 688.53 L72.14 686.43 L72.14 686.43 ZM80.57 686.43 L89 686.43 L89 688.53 + L80.57 688.53 L80.57 686.43 L80.57 686.43 ZM95.32 686.43 L97.43 686.43 L97.43 688.53 L95.32 688.53 L95.32 + 686.43 L95.32 686.43 ZM103.76 686.43 L112.19 686.43 L112.19 688.53 L103.76 688.53 L103.76 686.43 L103.76 + 686.43 ZM118.51 686.43 L120.62 686.43 L120.62 688.53 L118.51 688.53 L118.51 686.43 L118.51 686.43 ZM126.94 + 686.43 L135.38 686.43 L135.38 688.53 L126.94 688.53 L126.94 686.43 L126.94 686.43 ZM141.7 686.43 L143.81 + 686.43 L143.81 688.53 L141.7 688.53 L141.7 686.43 L141.7 686.43 ZM150.13 686.43 L158.57 686.43 L158.57 688.53 + L150.13 688.53 L150.13 686.43 L150.13 686.43 ZM164.89 686.43 L167 686.43 L167 688.53 L164.89 688.53 L164.89 + 686.43 L164.89 686.43 ZM173.32 686.43 L181.75 686.43 L181.75 688.53 L173.32 688.53 L173.32 686.43 L173.32 + 686.43 ZM188.08 686.43 L190.19 686.43 L190.19 688.53 L188.08 688.53 L188.08 686.43 L188.08 686.43 ZM196.51 + 686.43 L204.94 686.43 L204.94 688.53 L196.51 688.53 L196.51 686.43 L196.51 686.43 ZM211.27 686.43 L213.38 + 686.43 L213.38 688.53 L211.27 688.53 L211.27 686.43 L211.27 686.43 ZM219.7 686.43 L228.13 686.43 L228.13 + 688.53 L219.7 688.53 L219.7 686.43 L219.7 686.43 ZM234.46 686.43 L236.56 686.43 L236.56 688.53 L234.46 688.53 + L234.46 686.43 L234.46 686.43 ZM242.89 686.43 L251.32 686.43 L251.32 688.53 L242.89 688.53 L242.89 686.43 + L242.89 686.43 ZM257.64 686.43 L259.75 686.43 L259.75 688.53 L257.64 688.53 L257.64 686.43 L257.64 686.43 + ZM266.08 686.43 L274.51 686.43 L274.51 688.53 L266.08 688.53 L266.08 686.43 L266.08 686.43 ZM280.83 686.43 + L282.94 686.43 L282.94 688.53 L280.83 688.53 L280.83 686.43 L280.83 686.43 ZM289.27 686.43 L297.7 686.43 + L297.7 688.53 L289.27 688.53 L289.27 686.43 L289.27 686.43 ZM304.02 686.43 L306.13 686.43 L306.13 688.53 + L304.02 688.53 L304.02 686.43 L304.02 686.43 ZM312.45 686.43 L320.89 686.43 L320.89 688.53 L312.45 688.53 + L312.45 686.43 L312.45 686.43 ZM327.21 686.43 L329.32 686.43 L329.32 688.53 L327.21 688.53 L327.21 686.43 + L327.21 686.43 ZM335.64 686.43 L344.08 686.43 L344.08 688.53 L335.64 688.53 L335.64 686.43 L335.64 686.43 + ZM350.4 686.43 L352.51 686.43 L352.51 688.53 L350.4 688.53 L350.4 686.43 L350.4 686.43 ZM358.83 686.43 L367.26 + 686.43 L367.26 688.53 L358.83 688.53 L358.83 686.43 L358.83 686.43 ZM373.59 686.43 L375.7 686.43 L375.7 + 688.53 L373.59 688.53 L373.59 686.43 L373.59 686.43 ZM382.02 686.43 L387.06 686.43 L387.06 688.53 L382.02 + 688.53 L382.02 686.43 L382.02 686.43 ZM13.18 694.06 L0 687.48 L13.18 680.9 L13.18 694.06 L13.18 694.06 ZM384.89 + 680.9 L398.06 687.48 L384.89 694.06 L384.89 680.9 L384.89 680.9 Z" class="st2"/> + </g> + <g id="shape6-9" v:mID="6" v:groupContext="shape" transform="translate(2.5012,-522.82)"> + <title>Sheet.6</title> + <path d="M0 597.01 L0 694.06 L154.68 694.06 L154.68 597.01 L0 597.01 L0 597.01 Z" class="st4"/> + </g> + <g id="shape7-11" v:mID="7" v:groupContext="shape" transform="translate(2.5012,-522.82)"> + <title>Sheet.7</title> + <path d="M0 597.01 L154.68 597.01 L154.68 694.06 L0 694.06 L0 597.01" class="st5"/> + </g> + <g id="shape10-14" v:mID="10" v:groupContext="shape" transform="translate(159.025,-522.82)"> + <title>Sheet.10</title> + <path d="M0 597.01 L0 694.06 L563.73 694.06 L563.73 597.01 L0 597.01 L0 597.01 Z" class="st6"/> + </g> + <g id="shape11-16" v:mID="11" v:groupContext="shape" transform="translate(159.025,-522.82)"> + <title>Sheet.11</title> + <path d="M0 597.01 L563.73 597.01 L563.73 694.06 L0 694.06 L0 597.01" class="st7"/> + </g> + <g id="shape12-19" v:mID="12" v:groupContext="shape" transform="translate(262.039,-549.269)"> + <title>Sheet.12</title> + <desc>Payload 0</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="97.4929" cy="673.065" width="194.99" height="41.9798"/> + <path d="M194.99 652.08 L0 652.08 L0 694.06 L194.99 694.06 L194.99 652.08" class="st8"/> + <text x="46.92" y="680.57" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Payload 0</text> </g> + <g id="shape13-23" v:mID="13" v:groupContext="shape" transform="translate(547.615,-549.269)"> + <title>Sheet.13</title> + <desc>Payload 1</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="87.5716" cy="673.065" width="175.15" height="41.9798"/> + <path d="M175.14 652.08 L0 652.08 L0 694.06 L175.14 694.06 L175.14 652.08" class="st8"/> + <text x="22.61" y="680.57" class="st9" v:langID="1033"><v:paragraph/><v:tabList/>Payload 1</text> </g> + <g id="shape15-27" v:mID="15" v:groupContext="shape" transform="translate(2.2377,-522.82)"> + <title>Sheet.15</title> + <path d="M0 597.01 L0 694.06 L154.68 694.06 L154.68 597.01 L0 597.01 L0 597.01 Z" class="st4"/> + </g> + <g id="shape16-29" v:mID="16" v:groupContext="shape" transform="translate(2.2377,-522.82)"> + <title>Sheet.16</title> + <path d="M0 597.01 L154.68 597.01 L154.68 694.06 L0 694.06 L0 597.01" class="st5"/> + </g> + <g id="shape17-32" v:mID="17" v:groupContext="shape" transform="translate(6.52106,-546.331)"> + <title>Sheet.17</title> + <desc>Header</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="72.773" cy="673.065" width="145.55" height="41.9798"/> + <path d="M145.55 652.08 L0 652.08 L0 694.06 L145.55 694.06 L145.55 652.08" class="st8"/> + <text x="34.98" y="680.57" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Header</text> </g> + <g id="shape23-36" v:mID="23" v:groupContext="shape" transform="translate(286.548,-2.2377)"> + <title>Sheet.23</title> + <path d="M0 597.27 L0 694.06 L345.2 694.06 L345.2 597.27 L0 597.27 L0 597.27 Z" class="st4"/> + </g> + <g id="shape24-38" v:mID="24" v:groupContext="shape" transform="translate(286.548,-2.2377)"> + <title>Sheet.24</title> + <path d="M0 597.27 L345.2 597.27 L345.2 694.06 L0 694.06 L0 597.27" class="st5"/> + </g> + <g id="shape25-41" v:mID="25" v:groupContext="shape" transform="translate(399.834,-25.6887)"> + <title>Sheet.25</title> + <desc>Header</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="72.773" cy="673.065" width="145.55" height="41.9798"/> + <path d="M145.55 652.08 L0 652.08 L0 694.06 L145.55 694.06 L145.55 652.08" class="st8"/> + <text x="13.76" y="683.56" class="st10" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Header</text> </g> + <g id="shape31-45" v:mID="31" v:groupContext="shape" transform="translate(631.744,-2.2377)"> + <title>Sheet.31</title> + <path d="M0 597.27 L0 694.06 L516.21 694.06 L516.21 597.27 L0 597.27 L0 597.27 Z" class="st6"/> + </g> + <g id="shape32-47" v:mID="32" v:groupContext="shape" transform="translate(631.744,-2.2377)"> + <title>Sheet.32</title> + <path d="M0 597.27 L516.21 597.27 L516.21 694.06 L0 694.06 L0 597.27" class="st7"/> + </g> + <g id="shape33-50" v:mID="33" v:groupContext="shape" transform="translate(809.035,-25.6889)"> + <title>Sheet.33</title> + <desc>Payload 1</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="97.4929" cy="673.065" width="194.99" height="41.9798"/> + <path d="M194.99 652.08 L0 652.08 L0 694.06 L194.99 694.06 L194.99 652.08" class="st8"/> + <text x="16.99" y="683.56" class="st10" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Payload 1</text> </g> + <g id="shape35-54" v:mID="35" v:groupContext="shape" transform="translate(1190.48,-32.2189)"> + <title>Sheet.35</title> + <desc>Logical output segment</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="164.662" cy="678.273" width="329.33" height="31.5648"/> + <path d="M329.32 662.49 L0 662.49 L0 694.06 L329.32 694.06 L329.32 662.49" class="st8"/> + <text x="0" y="686.16" class="st11" v:langID="1033"><v:paragraph/><v:tabList/>Logical output segment</text> </g> + <g id="shape39-58" v:mID="39" v:groupContext="shape" transform="translate(546.25,-529.921)"> + <title>Sheet.39</title> + <path d="M3.43 529.94 L3.46 543.61 L0.03 543.61 L0 529.94 L3.43 529.94 L3.43 529.94 ZM3.46 553.87 L3.46 557.29 L0.03 + 557.29 L0.03 553.87 L3.46 553.87 L3.46 553.87 ZM3.46 567.55 L3.46 581.22 L0.03 581.22 L0.03 567.55 L3.46 + 567.55 L3.46 567.55 ZM3.46 591.48 L3.46 594.9 L0.03 594.9 L0.03 591.48 L3.46 591.48 L3.46 591.48 ZM3.46 + 605.16 L3.46 618.83 L0.03 618.83 L0.03 605.16 L3.46 605.16 L3.46 605.16 ZM3.46 629.09 L3.46 632.51 L0.03 + 632.51 L0.03 629.09 L3.46 629.09 L3.46 629.09 ZM3.46 642.77 L3.46 656.45 L0.03 656.45 L0.03 642.77 L3.46 + 642.77 L3.46 642.77 ZM3.46 666.7 L3.46 670.12 L0.03 670.12 L0.03 666.7 L3.46 666.7 L3.46 666.7 ZM3.46 680.38 + L3.46 694.06 L0.03 694.06 L0.03 680.38 L3.46 680.38 L3.46 680.38 Z" class="st1"/> + </g> + <g id="shape40-60" v:mID="40" v:groupContext="shape" transform="translate(549.097,-223.749)"> + <title>Sheet.40</title> + <path d="M6.65 402.61 L13.01 414.71 L9.98 416.32 L3.62 404.22 L6.65 402.61 L6.65 402.61 ZM17.82 423.78 L19.4 426.81 L16.37 + 428.42 L14.79 425.39 L17.82 423.78 L17.82 423.78 ZM24.21 435.91 L30.57 448.01 L27.54 449.59 L21.18 437.49 + L24.21 435.91 L24.21 435.91 ZM35.38 457.08 L36.96 460.11 L33.93 461.69 L32.35 458.66 L35.38 457.08 L35.38 + 457.08 ZM41.73 469.18 L48.12 481.28 L45.09 482.86 L38.7 470.76 L41.73 469.18 L41.73 469.18 ZM52.9 490.36 + L54.51 493.38 L51.48 494.99 L49.87 491.97 L52.9 490.36 L52.9 490.36 ZM59.29 502.45 L65.68 514.55 L62.65 + 516.16 L56.26 504.06 L59.29 502.45 L59.29 502.45 ZM70.46 523.63 L72.07 526.65 L69.04 528.26 L67.43 525.24 + L70.46 523.63 L70.46 523.63 ZM76.85 535.76 L83.24 547.86 L80.21 549.43 L73.82 537.34 L76.85 535.76 L76.85 + 535.76 ZM88.01 556.93 L89.63 559.95 L86.6 561.53 L84.98 558.51 L88.01 556.93 L88.01 556.93 ZM94.4 569.03 + L100.79 581.13 L97.76 582.7 L91.37 570.61 L94.4 569.03 L94.4 569.03 ZM105.57 590.2 L107.15 593.22 L104.12 + 594.84 L102.54 591.81 L105.57 590.2 L105.57 590.2 ZM111.96 602.3 L118.32 614.4 L115.28 616.01 L108.93 603.91 + L111.96 602.3 L111.96 602.3 ZM123.12 623.47 L124.71 626.5 L121.67 628.11 L120.09 625.08 L123.12 623.47 L123.12 + 623.47 ZM129.51 635.6 L135.87 647.7 L132.84 649.28 L126.48 637.18 L129.51 635.6 L129.51 635.6 ZM140.68 656.77 + L142.26 659.8 L139.23 661.38 L137.65 658.35 L140.68 656.77 L140.68 656.77 ZM147.04 668.87 L153.43 680.97 + L150.4 682.55 L144.01 670.45 L147.04 668.87 L147.04 668.87 ZM158.2 690.04 L159.49 692.48 L156.46 694.06 + L155.17 691.66 L158.2 690.04 L158.2 690.04 ZM0.33 408.43 L0 393.7 L11.99 402.28 L0.33 408.43 L0.33 408.43 + Z" class="st1"/> + </g> + <g id="shape46-62" v:mID="46" v:groupContext="shape" transform="translate(66.8445,-221.499)"> + <title>Sheet.46</title> + <path d="M0 611.87 C0 602.79 7.38 595.43 16.47 595.43 L273.39 595.43 C282.51 595.43 289.86 602.79 289.86 611.87 L289.86 + 677.62 C289.86 686.72 282.51 694.06 273.39 694.06 L16.47 694.06 C7.38 694.06 -0 686.72 0 677.62 L0 611.87 + Z" class="st4"/> + </g> + <g id="shape47-64" v:mID="47" v:groupContext="shape" transform="translate(66.8445,-221.499)"> + <title>Sheet.47</title> + <path d="M0 611.87 C0 602.79 7.38 595.43 16.47 595.43 L273.39 595.43 C282.51 595.43 289.86 602.79 289.86 611.87 L289.86 + 677.62 C289.86 686.72 282.51 694.06 273.39 694.06 L16.47 694.06 C7.38 694.06 -0 686.72 0 677.62 L0 611.87 + Z" class="st12"/> + </g> + <g id="shape48-66" v:mID="48" v:groupContext="shape" transform="translate(113.27,-263.667)"> + <title>Sheet.48</title> + <desc>Direct mbuf</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="98.5041" cy="675.59" width="197.01" height="36.9302"/> + <path d="M197.01 657.13 L0 657.13 L0 694.06 L197.01 694.06 L197.01 657.13" class="st8"/> + <text x="18.66" y="684.59" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Direct mbuf</text> </g> + <g id="shape51-70" v:mID="51" v:groupContext="shape" transform="translate(85.817,-233.439)"> + <title>Sheet.51</title> + <desc>(copy of headers)</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="127.916" cy="678.273" width="255.84" height="31.5648"/> + <path d="M255.83 662.49 L0 662.49 L0 694.06 L255.83 694.06 L255.83 662.49" class="st8"/> + <text x="34.33" y="685.47" class="st14" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>(copy of headers)</text> </g> + <g id="shape53-74" v:mID="53" v:groupContext="shape" transform="translate(371.944,-275.998)"> + <title>Sheet.53</title> + <desc>next</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="33.3635" cy="678.273" width="66.73" height="31.5648"/> + <path d="M66.73 662.49 L0 662.49 L0 694.06 L66.73 694.06 L66.73 662.49" class="st8"/> + <text x="7.56" y="686.16" class="st15" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>next</text> </g> + <g id="shape54-78" v:mID="54" v:groupContext="shape" transform="translate(695.132,-646.04)"> + <title>Sheet.54</title> + <path d="M0 655.39 L0 694.06 L100.4 694.06 L100.4 655.39 L0 655.39 L0 655.39 Z" class="st16"/> + </g> + <g id="shape55-80" v:mID="55" v:groupContext="shape" transform="translate(709.033,-648.946)"> + <title>Sheet.55</title> + <desc>segsz</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="36.6265" cy="680.167" width="73.26" height="27.7775"/> + <path d="M73.25 666.28 L0 666.28 L0 694.06 L73.25 694.06 L73.25 666.28" class="st8"/> + <text x="7.6" y="687.11" class="st17" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>segsz</text> </g> + <g id="shape56-84" v:mID="56" v:groupContext="shape" transform="translate(785.874,-521.182)"> + <title>Sheet.56</title> + <path d="M0 597.27 L0 694.06 L363.41 694.06 L363.41 597.27 L0 597.27 L0 597.27 Z" class="st6"/> + </g> + <g id="shape57-86" v:mID="57" v:groupContext="shape" transform="translate(785.874,-521.182)"> + <title>Sheet.57</title> + <path d="M0 597.27 L363.41 597.27 L363.41 694.06 L0 694.06 L0 597.27" class="st7"/> + </g> + <g id="shape58-89" v:mID="58" v:groupContext="shape" v:layerMember="0" transform="translate(943.158,-529.889)"> + <title>Sheet.58</title> + <path d="M1.35 529.91 L1.25 543.58 L4.68 543.61 L4.78 529.94 L1.35 529.91 L1.35 529.91 ZM1.15 553.84 L1.12 557.26 L4.55 + 557.29 L4.58 553.87 L1.15 553.84 L1.15 553.84 ZM1.05 567.52 L0.92 581.19 L4.35 581.22 L4.48 567.55 L1.05 + 567.52 L1.05 567.52 ZM0.86 591.45 L0.82 594.87 L4.25 594.9 L4.28 591.48 L0.86 591.45 L0.86 591.45 ZM0.72 + 605.13 L0.63 618.8 L4.05 618.83 L4.15 605.16 L0.72 605.13 L0.72 605.13 ZM0.53 629.06 L0.53 632.48 L3.95 + 632.51 L3.95 629.09 L0.53 629.06 L0.53 629.06 ZM0.43 642.74 L0.33 656.41 L3.75 656.45 L3.85 642.77 L0.43 + 642.74 L0.43 642.74 ZM0.23 666.67 L0.2 670.09 L3.62 670.12 L3.66 666.7 L0.23 666.67 L0.23 666.67 ZM0.13 + 680.35 L0 694.02 L3.43 694.06 L3.56 680.38 L0.13 680.35 L0.13 680.35 Z" class="st18"/> + </g> + <g id="shape59-91" v:mID="59" v:groupContext="shape" transform="translate(785.874,-549.473)"> + <title>Sheet.59</title> + <desc>Payload 1</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="77.3395" cy="673.065" width="154.68" height="41.9798"/> + <path d="M154.68 652.08 L0 652.08 L0 694.06 L154.68 694.06 L154.68 652.08" class="st8"/> + <text x="16.96" y="680.57" class="st9" v:langID="1033"><v:paragraph/><v:tabList/>Payload 1</text> </g> + <g id="shape60-95" v:mID="60" v:groupContext="shape" transform="translate(952.97,-548.822)"> + <title>Sheet.60</title> + <desc>Payload 2</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="97.4929" cy="673.065" width="194.99" height="41.9798"/> + <path d="M194.99 652.08 L0 652.08 L0 694.06 L194.99 694.06 L194.99 652.08" class="st8"/> + <text x="46.92" y="680.57" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Payload 2</text> </g> + <g id="shape63-99" v:mID="63" v:groupContext="shape" transform="translate(1190.48,-552.568)"> + <title>Sheet.63</title> + <desc>Multi-segment input packet</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="181.707" cy="678.273" width="363.42" height="31.5648"/> + <path d="M363.41 662.49 L0 662.49 L0 694.06 L363.41 694.06 L363.41 662.49" class="st8"/> + <text x="0" y="686.16" class="st11" v:langID="1033"><v:paragraph/><v:tabList/>Multi-segment input packet</text> </g> + <g id="shape70-103" v:mID="70" v:groupContext="shape" v:layerMember="1" transform="translate(455.049,-221.499)"> + <title>Sheet.70</title> + <path d="M0 611.87 C0 602.79 5.33 595.43 11.89 595.43 L282.92 595.43 C289.53 595.43 294.8 602.79 294.8 611.87 L294.8 + 677.62 C294.8 686.76 289.53 694.06 282.92 694.06 L11.89 694.06 C5.33 694.06 0 686.76 0 677.62 L0 611.87 + Z" class="st6"/> + </g> + <g id="shape71-105" v:mID="71" v:groupContext="shape" transform="translate(455.049,-221.499)"> + <title>Sheet.71</title> + <path d="M0 611.87 C0 602.79 5.33 595.43 11.89 595.43 L282.92 595.43 C289.53 595.43 294.8 602.79 294.8 611.87 L294.8 + 677.62 C294.8 686.76 289.53 694.06 282.92 694.06 L11.89 694.06 C5.33 694.06 0 686.76 0 677.62 L0 611.87 + Z" class="st12"/> + </g> + <g id="shape72-107" v:mID="72" v:groupContext="shape" transform="translate(489.065,-263.434)"> + <title>Sheet.72</title> + <desc>Indirect mbuf</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="112.128" cy="675.59" width="224.26" height="36.9302"/> + <path d="M224.26 657.13 L0 657.13 L0 694.06 L224.26 694.06 L224.26 657.13" class="st8"/> + <text x="20.73" y="684.59" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Indirect mbuf</text> </g> + <g id="shape75-111" v:mID="75" v:groupContext="shape" transform="translate(849.065,-281.435)"> + <title>Sheet.75</title> + <desc>(pointer to data)</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="100.199" cy="678.273" width="200.4" height="31.5648"/> + <path d="M200.4 662.49 L0 662.49 L0 694.06 L200.4 694.06 L200.4 662.49" class="st8"/> + <text x="4.49" y="686.16" class="st11" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>(pointer to data)</text> </g> + <g id="shape77-115" v:mID="77" v:groupContext="shape" transform="translate(717.742,-563.523)"> + <title>Sheet.77</title> + <desc>next</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="33.3635" cy="678.273" width="66.73" height="31.5648"/> + <path d="M66.73 662.49 L0 662.49 L0 694.06 L66.73 694.06 L66.73 662.49" class="st8"/> + <text x="15.71" y="683.67" class="st19" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>next</text> </g> + <g id="shape78-119" v:mID="78" v:groupContext="shape" transform="translate(1148.17,-529.067)"> + <title>Sheet.78</title> + <path d="M1.38 529.87 L1.25 543.55 L4.68 543.61 L4.81 529.94 L1.38 529.87 L1.38 529.87 ZM1.19 553.81 L1.12 557.23 L4.55 + 557.29 L4.61 553.87 L1.19 553.81 L1.19 553.81 ZM1.05 567.48 L0.92 581.16 L4.35 581.22 L4.48 567.55 L1.05 + 567.48 L1.05 567.48 ZM0.86 591.42 L0.86 594.84 L4.28 594.9 L4.28 591.48 L0.86 591.42 L0.86 591.42 ZM0.72 + 605.09 L0.66 618.77 L4.08 618.83 L4.15 605.16 L0.72 605.09 L0.72 605.09 ZM0.53 629.03 L0.53 632.45 L3.95 + 632.51 L3.95 629.09 L0.53 629.03 L0.53 629.03 ZM0.46 642.7 L0.33 656.38 L3.75 656.45 L3.89 642.77 L0.46 + 642.7 L0.46 642.7 ZM0.26 666.64 L0.2 670.06 L3.62 670.12 L3.69 666.7 L0.26 666.64 L0.26 666.64 ZM0.13 680.31 + L0 693.99 L3.43 694.06 L3.56 680.38 L0.13 680.31 L0.13 680.31 Z" class="st20"/> + </g> + <g id="shape79-121" v:mID="79" v:groupContext="shape" transform="translate(946.254,-657.81)"> + <title>Sheet.79</title> + <path d="M11 686.69 L17.33 686.69 L17.33 688.27 L11 688.27 L11 686.69 L11 686.69 ZM22.07 686.69 L23.65 686.69 L23.65 + 688.27 L22.07 688.27 L22.07 686.69 L22.07 686.69 ZM28.39 686.69 L34.72 686.69 L34.72 688.27 L28.39 688.27 + L28.39 686.69 L28.39 686.69 ZM39.46 686.69 L41.04 686.69 L41.04 688.27 L39.46 688.27 L39.46 686.69 L39.46 + 686.69 ZM45.78 686.69 L52.11 686.69 L52.11 688.27 L45.78 688.27 L45.78 686.69 L45.78 686.69 ZM56.85 686.69 + L58.43 686.69 L58.43 688.27 L56.85 688.27 L56.85 686.69 L56.85 686.69 ZM63.18 686.69 L69.5 686.69 L69.5 + 688.27 L63.18 688.27 L63.18 686.69 L63.18 686.69 ZM74.24 686.69 L75.82 686.69 L75.82 688.27 L74.24 688.27 + L74.24 686.69 L74.24 686.69 ZM80.57 686.69 L86.89 686.69 L86.89 688.27 L80.57 688.27 L80.57 686.69 L80.57 + 686.69 ZM91.63 686.69 L93.22 686.69 L93.22 688.27 L91.63 688.27 L91.63 686.69 L91.63 686.69 ZM97.96 686.69 + L104.28 686.69 L104.28 688.27 L97.96 688.27 L97.96 686.69 L97.96 686.69 ZM109.03 686.69 L110.61 686.69 L110.61 + 688.27 L109.03 688.27 L109.03 686.69 L109.03 686.69 ZM115.35 686.69 L121.67 686.69 L121.67 688.27 L115.35 + 688.27 L115.35 686.69 L115.35 686.69 ZM126.42 686.69 L128 686.69 L128 688.27 L126.42 688.27 L126.42 686.69 + L126.42 686.69 ZM132.74 686.69 L139.07 686.69 L139.07 688.27 L132.74 688.27 L132.74 686.69 L132.74 686.69 + ZM143.81 686.69 L145.39 686.69 L145.39 688.27 L143.81 688.27 L143.81 686.69 L143.81 686.69 ZM150.13 686.69 + L156.46 686.69 L156.46 688.27 L150.13 688.27 L150.13 686.69 L150.13 686.69 ZM161.2 686.69 L162.78 686.69 + L162.78 688.27 L161.2 688.27 L161.2 686.69 L161.2 686.69 ZM167.53 686.69 L173.85 686.69 L173.85 688.27 L167.53 + 688.27 L167.53 686.69 L167.53 686.69 ZM178.59 686.69 L180.17 686.69 L180.17 688.27 L178.59 688.27 L178.59 + 686.69 L178.59 686.69 ZM184.92 686.69 L189.4 686.69 L189.4 688.27 L184.92 688.27 L184.92 686.69 L184.92 + 686.69 ZM13.18 694.06 L0 687.41 L13.18 680.9 L13.18 694.06 L13.18 694.06 ZM187.22 680.9 L200.4 687.48 L187.22 + 694.06 L187.22 680.9 L187.22 680.9 Z" class="st20"/> + </g> + <g id="shape80-123" v:mID="80" v:groupContext="shape" transform="translate(982.882,-643.673)"> + <title>Sheet.80</title> + <path d="M0 655.13 L0 694.06 L127.01 694.06 L127.01 655.13 L0 655.13 L0 655.13 Z" class="st16"/> + </g> + <g id="shape81-125" v:mID="81" v:groupContext="shape" transform="translate(1003.39,-660.621)"> + <title>Sheet.81</title> + <desc>pkt_len</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="48.6041" cy="680.956" width="97.21" height="26.1994"/> + <path d="M97.21 667.86 L0 667.86 L0 694.06 L97.21 694.06 L97.21 667.86" class="st8"/> + <text x="11.67" y="687.5" class="st21" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>pkt_len </text> </g> + <g id="shape82-129" v:mID="82" v:groupContext="shape" transform="translate(1001.18,-634.321)"> + <title>Sheet.82</title> + <desc>% segsz</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="49.2945" cy="680.956" width="98.59" height="26.1994"/> + <path d="M98.59 667.86 L0 667.86 L0 694.06 L98.59 694.06 L98.59 667.86" class="st8"/> + <text x="9.09" y="687.5" class="st21" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>% segsz</text> </g> + <g id="shape34-133" v:mID="34" v:groupContext="shape" v:layerMember="0" transform="translate(356.703,-264.106)"> + <title>Sheet.34</title> + <path d="M0 685.77 L90.67 685.77 L90.67 689.19 L0 689.19 L0 685.77 L0 685.77 ZM89.36 680.9 L97.21 687.48 L89.36 694.06 + L89.36 680.9 L89.36 680.9 Z" class="st2"/> + </g> + <g id="shape85-135" v:mID="85" v:groupContext="shape" v:layerMember="0" transform="translate(78.5359,-282.66)"> + <title>Sheet.85</title> + <path d="M0 680.87 C-0 673.59 6.88 667.69 15.37 667.69 C23.86 667.69 30.73 673.59 30.73 680.87 C30.73 688.15 23.86 694.06 + 15.37 694.06 C6.88 694.06 0 688.15 0 680.87 Z" class="st16"/> + </g> + <g id="shape87-137" v:mID="87" v:groupContext="shape" v:layerMember="0" transform="translate(85.4791,-284.062)"> + <title>Sheet.87</title> + <desc>1</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="8.66303" cy="683.269" width="17.33" height="21.5726"/> + <path d="M17.33 672.48 L0 672.48 L0 694.06 L17.33 694.06 L17.33 672.48" class="st8"/> + <text x="3.32" y="688.66" class="st22" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>1</text> </g> + <g id="shape88-141" v:mID="88" v:groupContext="shape" v:layerMember="0" transform="translate(468.906,-282.66)"> + <title>Sheet.88</title> + <path d="M0 680.87 C-0 673.59 6.89 667.69 15.37 667.69 C23.86 667.69 30.73 673.59 30.73 680.87 C30.73 688.15 23.86 694.06 + 15.37 694.06 C6.89 694.06 -0 688.15 0 680.87 Z" class="st16"/> + </g> + <g id="shape90-143" v:mID="90" v:groupContext="shape" v:layerMember="0" transform="translate(474.575,-284.062)"> + <title>Sheet.90</title> + <desc>2</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="8.66303" cy="683.269" width="17.33" height="21.5726"/> + <path d="M17.33 672.48 L0 672.48 L0 694.06 L17.33 694.06 L17.33 672.48" class="st8"/> + <text x="3.32" y="688.66" class="st22" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>2</text> </g> + <g id="shape95-147" v:mID="95" v:groupContext="shape" v:layerMember="0" transform="translate(764.026,-275.998)"> + <title>Sheet.95</title> + <desc>next</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="33.3635" cy="678.273" width="66.73" height="31.5648"/> + <path d="M66.73 662.49 L0 662.49 L0 694.06 L66.73 694.06 L66.73 662.49" class="st8"/> + <text x="7.56" y="686.16" class="st15" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>next</text> </g> + <g id="shape97-151" v:mID="97" v:groupContext="shape" v:layerMember="0" transform="translate(844.524,-219.79)"> + <title>Sheet.97</title> + <path d="M0 611.87 C0 602.79 5.45 595.43 12.17 595.43 L289.54 595.43 C296.31 595.43 301.71 602.79 301.71 611.87 L301.71 + 677.62 C301.71 686.76 296.31 694.06 289.54 694.06 L12.17 694.06 C5.45 694.06 0 686.76 0 677.62 L0 611.87 + Z" class="st12"/> + </g> + <g id="shape100-153" v:mID="100" v:groupContext="shape" v:layerMember="0" transform="translate(751.857,-262.528)"> + <title>Sheet.100</title> + <path d="M0 685.77 L90.67 685.77 L90.67 689.19 L0 689.19 L0 685.77 L0 685.77 ZM89.36 680.9 L97.21 687.48 L89.36 694.06 + L89.36 680.9 L89.36 680.9 Z" class="st2"/> + </g> + <g id="shape104-155" v:mID="104" v:groupContext="shape" v:layerMember="1" transform="translate(851.429,-218.08)"> + <title>Sheet.104</title> + <path d="M0 611.87 C0 602.79 5.33 595.43 11.89 595.43 L282.92 595.43 C289.53 595.43 294.8 602.79 294.8 611.87 L294.8 + 677.62 C294.8 686.76 289.53 694.06 282.92 694.06 L11.89 694.06 C5.33 694.06 0 686.76 0 677.62 L0 611.87 + Z" class="st6"/> + </g> + <g id="shape105-157" v:mID="105" v:groupContext="shape" v:layerMember="0" transform="translate(885.444,-260.015)"> + <title>Sheet.105</title> + <desc>Indirect mbuf</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="112.128" cy="675.59" width="224.26" height="36.9302"/> + <path d="M224.26 657.13 L0 657.13 L0 694.06 L224.26 694.06 L224.26 657.13" class="st8"/> + <text x="20.73" y="684.59" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Indirect mbuf</text> </g> + <g id="shape106-161" v:mID="106" v:groupContext="shape" v:layerMember="0" transform="translate(895.672,-229.419)"> + <title>Sheet.106</title> + <desc>(pointer to data)</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="100.199" cy="678.273" width="200.4" height="31.5648"/> + <path d="M200.4 662.49 L0 662.49 L0 694.06 L200.4 694.06 L200.4 662.49" class="st8"/> + <text x="12.86" y="685.47" class="st14" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>(pointer to data)</text> </g> + <g id="shape107-165" v:mID="107" v:groupContext="shape" v:layerMember="0" transform="translate(863.297,-280.442)"> + <title>Sheet.107</title> + <path d="M0 680.87 C-0 673.59 6.89 667.69 15.37 667.69 C23.86 667.69 30.73 673.59 30.73 680.87 C30.73 688.15 23.86 694.06 + 15.37 694.06 C6.89 694.06 -0 688.15 0 680.87 Z" class="st16"/> + </g> + <g id="shape108-167" v:mID="108" v:groupContext="shape" v:layerMember="0" transform="translate(870.001,-281.547)"> + <title>Sheet.108</title> + <desc>3</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="8.66303" cy="683.269" width="17.33" height="21.5726"/> + <path d="M17.33 672.48 L0 672.48 L0 694.06 L17.33 694.06 L17.33 672.48" class="st8"/> + <text x="3.32" y="688.66" class="st22" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>3</text> </g> + <g id="shape109-171" v:mID="109" v:groupContext="shape" v:layerMember="0" transform="translate(500.959,-231.87)"> + <title>Sheet.109</title> + <desc>(pointer to data)</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="100.199" cy="678.273" width="200.4" height="31.5648"/> + <path d="M200.4 662.49 L0 662.49 L0 694.06 L200.4 694.06 L200.4 662.49" class="st8"/> + <text x="12.86" y="685.47" class="st14" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>(pointer to data)</text> </g> + <g id="shape113-175" v:mID="113" v:groupContext="shape" v:layerMember="0" transform="translate(1187.64,-251.712)"> + <title>Sheet.113</title> + <desc>Three-part output segment</desc> + <v:textBlock v:margins="rect(0,0,0,0)" v:tabSpace="42.5197"/> + <v:textRect cx="327.402" cy="678.273" width="654.81" height="31.5648"/> + <path d="M654.8 662.49 L0 662.49 L0 694.06 L654.8 694.06 L654.8 662.49" class="st8"/> + <text x="0" y="686.16" class="st11" v:langID="1033"><v:paragraph/><v:tabList/>Three-part output segment</text> </g> + </g> +</svg> diff --git a/doc/guides/prog_guide/img/mbuf1.svg b/doc/guides/prog_guide/img/mbuf1.svg index 5bd84d1b..8750424e 100644 --- a/doc/guides/prog_guide/img/mbuf1.svg +++ b/doc/guides/prog_guide/img/mbuf1.svg @@ -482,7 +482,7 @@ sodipodi:role="line" x="187.85715" y="347.7193" - id="tspan5240">(m->buf_physaddr is the</tspan><tspan + id="tspan5240">(m->buf_iova is the</tspan><tspan sodipodi:role="line" x="187.85715" y="360.2193" diff --git a/doc/guides/prog_guide/img/member_i1.svg b/doc/guides/prog_guide/img/member_i1.svg new file mode 100644 index 00000000..fc5f56ac --- /dev/null +++ b/doc/guides/prog_guide/img/member_i1.svg @@ -0,0 +1,1613 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.0//EN" "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd"> +<!-- Generated by Microsoft Visio 11.0, SVG Export, v1.0 memship_i1.svg Page-1 --> +<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events" + xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" width="7.18709in" height="4.75757in" + viewBox="0 0 517.471 342.545" xml:space="preserve" color-interpolation-filters="sRGB" class="st61"> + <v:documentProperties v:langID="1033" v:viewMarkup="false"> + <v:userDefs> + <v:ud v:nameU="msvSubprocessMaster" v:prompt="" v:val="VT4(Rectangle)"/> + <v:ud v:nameU="msvNoAutoConnect" v:val="VT0(1):26"/> + </v:userDefs> + </v:documentProperties> + + <style type="text/css"> + <![CDATA[ + .st1 {visibility:visible} + .st2 {fill:none;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:3} + .st3 {fill:#5b9bd5;font-family:Calibri;font-size:0.666664em;opacity:0.219608} + .st4 {font-size:1em} + .st5 {fill:none;stroke:#41719c;stroke-width:3} + .st6 {fill:#5b9bd5;font-family:Calibri;font-size:0.666664em} + .st7 {fill:#5b9bd5;font-family:Calibri;font-size:0.75em;opacity:0.219608} + .st8 {fill:#5b9bd5;font-family:Calibri;font-size:0.75em} + .st9 {fill:#5b9bd5;fill-opacity:0.22;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:0.25} + .st10 {fill:#5b9bd5;stroke:#c8c8c8;stroke-width:0.25} + .st11 {fill:none;stroke:none;stroke-width:0.25} + .st12 {fill:#ffffff;font-family:Calibri;font-size:0.499992em;font-weight:bold} + .st13 {fill:#ffffff;font-family:Calibri;font-size:0.75em;font-weight:bold} + .st14 {marker-end:url(#mrkr5-63);stroke:#5b9bd5;stroke-linecap:round;stroke-linejoin:round;stroke-width:1} + .st15 {fill:#5b9bd5;fill-opacity:1;stroke:#5b9bd5;stroke-opacity:1;stroke-width:0.28409090909091} + .st16 {fill:#5b9bd5;font-family:Calibri;font-size:0.499992em;font-weight:bold} + .st17 {stroke:#5b9bd5;stroke-linecap:round;stroke-linejoin:round;stroke-width:1} + .st18 {fill:#feffff;font-family:Calibri;font-size:0.499992em} + .st19 {fill:#deebf6;stroke:#c8c8c8;stroke-width:0.25} + .st20 {fill:#000000;font-family:Calibri;font-size:0.499992em} + .st21 {marker-end:url(#mrkr5-178);stroke:#ff0000;stroke-linecap:round;stroke-linejoin:round;stroke-width:1} + .st22 {fill:#ff0000;fill-opacity:1;stroke:#ff0000;stroke-opacity:1;stroke-width:0.28409090909091} + .st23 {fill:#ff0000;font-family:Calibri;font-size:0.666664em} + .st24 {fill:#5b9bd5;fill-opacity:0.22} + .st25 {stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:0.25} + .st26 {fill:#ffffff} + .st27 {stroke:#0070c0;stroke-width:0.25} + .st28 {fill:#5b9bd5;stroke:#0070c0;stroke-width:0.25} + .st29 {fill:#5b9bd5;stroke:#ffffff;stroke-width:0.25} + .st30 {fill:#5b9bd5} + .st31 {stroke:#c8c8c8;stroke-width:0.25} + .st32 {fill:#acccea;stroke:#c8c8c8;stroke-width:0.25} + .st33 {fill:#5b9bd5;fill-opacity:0.22;stroke:none;stroke-linecap:butt;stroke-width:0.75} + .st34 {fill:#000000;fill-opacity:0;stroke:none;stroke-linecap:butt;stroke-width:0.75} + .st35 {fill:url(#grad30-309);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75} + .st36 {fill:url(#grad25-313);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75} + .st37 {fill:url(#grad35-317);stroke:#308dda;stroke-linecap:butt;stroke-width:0.130208} + .st38 {fill:url(#grad36-325);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75} + .st39 {fill:url(#grad40-335);stroke:#000000;stroke-linecap:butt;stroke-width:0.130208} + .st40 {fill:url(#grad39-342);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75} + .st41 {fill:url(#grad40-355);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75} + .st42 {fill:none} + .st43 {stroke:#308dda;stroke-linecap:butt;stroke-width:0.130208} + .st44 {stroke:#ffffff;stroke-linecap:butt;stroke-width:0.130208} + .st45 {fill:url(#grad30-383);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75} + .st46 {fill:url(#grad36-396);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75} + .st47 {fill:none;stroke:#c8c8c8;stroke-width:0.75} + .st48 {fill:#9a9a9a;stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.0833333} + .st49 {fill:url(#grad40-415);stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.0833333} + .st50 {fill:url(#grad40-419);stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.0833333} + .st51 {stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.25} + .st52 {fill:url(#grad35-430);stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.0833333} + .st53 {stroke:#c8c8c8;stroke-width:0.75} + .st54 {stroke:#4f88bb;stroke-width:0.75} + .st55 {fill:#feffff;font-family:Calibri;font-size:0.416656em} + .st56 {fill:#5b9bd5;fill-opacity:0.25;stroke:#5b9bd5;stroke-opacity:0.25;stroke-width:0.75} + .st57 {fill:#4f88bb;stroke:#41719c;stroke-width:0.75} + .st58 {fill:none;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:2.25} + .st59 {fill:none;stroke:#0070c0;stroke-width:2.25} + .st60 {fill:#595959;font-family:Arial;font-size:0.666664em} + .st61 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3} + ]]> + </style> + + <defs id="Patterns_And_Gradients"> + <linearGradient id="grad30-309" v:fillPattern="30" v:foreground="#97c2e6" v:background="#4274a2" x1="0" y1="1" x2="0" + y2="0"> + <stop offset="0" style="stop-color:#97c2e6;stop-opacity:1"/> + <stop offset="1" style="stop-color:#4274a2;stop-opacity:1"/> + </linearGradient> + <linearGradient id="grad25-313" v:fillPattern="25" v:foreground="#5491d3" v:background="#246ba6" x1="0" y1="0" x2="1" + y2="0"> + <stop offset="0" style="stop-color:#5491d3;stop-opacity:1"/> + <stop offset="1" style="stop-color:#246ba6;stop-opacity:1"/> + </linearGradient> + <pattern id="grad35-317" v:fillPattern="35" v:foreground="#569bd3" v:background="#aed0ec" x="0" y="0" width="1" height="1" + patternContentUnits="objectBoundingBox"> + <path d="M 0.5 0.5 L 0 0 L 0 1 z" style="fill:url(#grad27-318)"/> + <path d="M 0.5 0.5 L 1 0 L 1 1 z" style="fill:url(#grad25-319)"/> + <path d="M 0.5 0.5 L 0 0 L 1 0 z" style="fill:url(#grad30-320)"/> + <path d="M 0.5 0.5 L 0 1 L 1 1 z" style="fill:url(#grad28-321)"/> + </pattern> + <linearGradient id="grad27-318" v:fillPattern="35" v:foreground="#569bd3" v:background="#aed0ec" x1="1" y1="0" x2="0" + y2="0"> + <stop offset="0" style="stop-color:#569bd3;stop-opacity:1"/> + <stop offset="1" style="stop-color:#aed0ec;stop-opacity:1"/> + </linearGradient> + <linearGradient id="grad25-319" v:fillPattern="35" v:foreground="#569bd3" v:background="#aed0ec" x1="0" y1="0" x2="1" + y2="0"> + <stop offset="0" style="stop-color:#569bd3;stop-opacity:1"/> + <stop offset="1" style="stop-color:#aed0ec;stop-opacity:1"/> + </linearGradient> + <linearGradient id="grad30-320" v:fillPattern="35" v:foreground="#569bd3" v:background="#aed0ec" x1="0" y1="1" x2="0" + y2="0"> + <stop offset="0" style="stop-color:#569bd3;stop-opacity:1"/> + <stop offset="1" style="stop-color:#aed0ec;stop-opacity:1"/> + </linearGradient> + <linearGradient id="grad28-321" v:fillPattern="35" v:foreground="#569bd3" v:background="#aed0ec" x1="0" y1="0" x2="0" + y2="1"> + <stop offset="0" style="stop-color:#569bd3;stop-opacity:1"/> + <stop offset="1" style="stop-color:#aed0ec;stop-opacity:1"/> + </linearGradient> + <radialGradient id="grad36-325" v:fillPattern="36" v:foreground="#c0dff1" v:background="#246ba6" cx="0" cy="0" r="1"> + <stop offset="0" style="stop-color:#c0dff1;stop-opacity:1"/> + <stop offset="1" style="stop-color:#246ba6;stop-opacity:1"/> + </radialGradient> + <radialGradient id="grad40-335" v:fillPattern="40" v:foreground="#c8e5c8" v:background="#19bf19" cx="0.5" cy="0.5" r="0.5"> + <stop offset="0" style="stop-color:#c8e5c8;stop-opacity:1"/> + <stop offset="1" style="stop-color:#19bf19;stop-opacity:1"/> + </radialGradient> + <radialGradient id="grad39-342" v:fillPattern="39" v:foreground="#5599d7" v:background="#b9daf2" cx="1" cy="1" r="1"> + <stop offset="0" style="stop-color:#5599d7;stop-opacity:1"/> + <stop offset="1" style="stop-color:#b9daf2;stop-opacity:1"/> + </radialGradient> + <radialGradient id="grad40-355" v:fillPattern="40" v:foreground="#5599d7" v:background="#214383" cx="0.5" cy="0.5" r="0.5"> + <stop offset="0" style="stop-color:#5599d7;stop-opacity:1"/> + <stop offset="1" style="stop-color:#214383;stop-opacity:1"/> + </radialGradient> + <linearGradient id="grad30-383" v:fillPattern="30" v:foreground="#97c2e6" v:background="#6ba4dc" x1="0" y1="1" x2="0" + y2="0"> + <stop offset="0" style="stop-color:#97c2e6;stop-opacity:1"/> + <stop offset="1" style="stop-color:#6ba4dc;stop-opacity:1"/> + </linearGradient> + <radialGradient id="grad36-396" v:fillPattern="36" v:foreground="#89bee9" v:background="#b9daf2" cx="0" cy="0" r="1"> + <stop offset="0" style="stop-color:#89bee9;stop-opacity:1"/> + <stop offset="1" style="stop-color:#b9daf2;stop-opacity:1"/> + </radialGradient> + <radialGradient id="grad40-415" v:fillPattern="40" v:foreground="#000000" v:background="#ffffff" cx="0.5" cy="0.5" r="0.5"> + <stop offset="0" style="stop-color:#000000;stop-opacity:1"/> + <stop offset="1" style="stop-color:#ffffff;stop-opacity:1"/> + </radialGradient> + <radialGradient id="grad40-419" v:fillPattern="40" v:foreground="#ffffff" v:background="#9a9a9a" cx="0.5" cy="0.5" r="0.5"> + <stop offset="0" style="stop-color:#ffffff;stop-opacity:1"/> + <stop offset="1" style="stop-color:#9a9a9a;stop-opacity:1"/> + </radialGradient> + <pattern id="grad35-430" v:fillPattern="35" v:foreground="#ffffff" v:background="#ffcc00" x="0" y="0" width="1" height="1" + patternContentUnits="objectBoundingBox"> + <path d="M 0.5 0.5 L 0 0 L 0 1 z" style="fill:url(#grad27-431)"/> + <path d="M 0.5 0.5 L 1 0 L 1 1 z" style="fill:url(#grad25-432)"/> + <path d="M 0.5 0.5 L 0 0 L 1 0 z" style="fill:url(#grad30-433)"/> + <path d="M 0.5 0.5 L 0 1 L 1 1 z" style="fill:url(#grad28-434)"/> + </pattern> + <linearGradient id="grad27-431" v:fillPattern="35" v:foreground="#ffffff" v:background="#ffcc00" x1="1" y1="0" x2="0" + y2="0"> + <stop offset="0" style="stop-color:#ffffff;stop-opacity:1"/> + <stop offset="1" style="stop-color:#ffcc00;stop-opacity:1"/> + </linearGradient> + <linearGradient id="grad25-432" v:fillPattern="35" v:foreground="#ffffff" v:background="#ffcc00" x1="0" y1="0" x2="1" + y2="0"> + <stop offset="0" style="stop-color:#ffffff;stop-opacity:1"/> + <stop offset="1" style="stop-color:#ffcc00;stop-opacity:1"/> + </linearGradient> + <linearGradient id="grad30-433" v:fillPattern="35" v:foreground="#ffffff" v:background="#ffcc00" x1="0" y1="1" x2="0" + y2="0"> + <stop offset="0" style="stop-color:#ffffff;stop-opacity:1"/> + <stop offset="1" style="stop-color:#ffcc00;stop-opacity:1"/> + </linearGradient> + <linearGradient id="grad28-434" v:fillPattern="35" v:foreground="#ffffff" v:background="#ffcc00" x1="0" y1="0" x2="0" + y2="1"> + <stop offset="0" style="stop-color:#ffffff;stop-opacity:1"/> + <stop offset="1" style="stop-color:#ffcc00;stop-opacity:1"/> + </linearGradient> + </defs> + <defs id="Markers"> + <g id="lend5"> + <path d="M 2 1 L 0 0 L 1.98117 -0.993387 C 1.67173 -0.364515 1.67301 0.372641 1.98465 1.00043 " style="stroke:none"/> + </g> + <marker id="mrkr5-63" class="st15" v:arrowType="5" v:arrowSize="2" v:setback="6.16" refX="-6.16" orient="auto" + markerUnits="strokeWidth" overflow="visible"> + <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/> + </marker> + <marker id="mrkr5-178" class="st22" v:arrowType="5" v:arrowSize="2" v:setback="5.8" refX="-5.8" orient="auto" + markerUnits="strokeWidth" overflow="visible"> + <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/> + </marker> + </defs> + <g v:mID="0" v:index="1" v:groupContext="foregroundPage"> + <v:userDefs> + <v:ud v:nameU="msvThemeOrder" v:val="VT0(0):26"/> + </v:userDefs> + <title>Page-1</title> + <v:pageProperties v:drawingScale="1" v:pageScale="1" v:drawingUnits="0" v:shadowOffsetX="9" v:shadowOffsetY="-9"/> + <v:layer v:name="Flowchart" v:index="0"/> + <g id="group165-1" transform="translate(21.7794,-24.0978)" v:mID="165" v:groupContext="group"> + <title>Sheet.165</title> + <g id="group1-2" transform="translate(308.647,-25.7109)" v:mID="1" v:groupContext="group"> + <title>Sheet.1</title> + <g id="shape2-3" v:mID="2" v:groupContext="shape" transform="translate(11.5732,-58.1913)"> + <title>Circle</title> + <desc>List 1 matching Criteria 1</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="27.2233" cy="315.322" width="47.65" height="40.835"/> + <g id="shadow2-4" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st2"/> + <text x="18.79" y="308.12" class="st3" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 1 <tspan + x="12.08" dy="1.2em" class="st4">matching </tspan><tspan x="12.29" dy="1.2em" class="st4">Criteria </tspan>1</text> </g> + <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st5"/> + <text x="18.79" y="308.12" class="st6" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 1 <tspan + x="12.08" dy="1.2em" class="st4">matching </tspan><tspan x="12.29" dy="1.2em" class="st4">Criteria </tspan>1</text> </g> + <g id="shape3-13" v:mID="3" v:groupContext="shape" transform="translate(58.9839,-58.9839)"> + <title>Circle.23</title> + <desc>List 2</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="27.2233" cy="315.322" width="47.65" height="40.835"/> + <g id="shadow3-14" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st2"/> + <text x="17.73" y="318.02" class="st7" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 2</text> </g> + <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st5"/> + <text x="17.73" y="318.02" class="st8" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 2</text> </g> + <g id="shape4-19" v:mID="4" v:groupContext="shape"> + <title>Circle.24</title> + <desc>List 1 matching Criteria 1</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="27.2233" cy="315.322" width="47.65" height="40.835"/> + <g id="shadow4-20" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st2"/> + <text x="18.79" y="308.12" class="st3" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 1 <tspan + x="12.08" dy="1.2em" class="st4">matching </tspan><tspan x="12.29" dy="1.2em" class="st4">Criteria </tspan>1</text> </g> + <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st5"/> + <text x="18.79" y="308.12" class="st6" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 1 <tspan + x="12.08" dy="1.2em" class="st4">matching </tspan><tspan x="12.29" dy="1.2em" class="st4">Criteria </tspan>1</text> </g> + <g id="group5-29" transform="translate(50.7413,-4.53722)" v:mID="5" v:groupContext="group"> + <title>Sheet.5</title> + <g id="shape6-30" v:mID="6" v:groupContext="shape" transform="translate(344.2,300.5) rotate(90)"> + <title>Triangle</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow6-31" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,1.9728,-0.3456)" class="st1"> + <path d="M42.04 342.55 L21.02 318.64 L0 342.55 L42.04 342.55 Z" class="st9"/> + </g> + <path d="M42.04 342.55 L21.02 318.64 L0 342.55 L42.04 342.55 Z" class="st10"/> + </g> + <g id="shape7-34" v:mID="7" v:groupContext="shape" transform="translate(-0.884982,-14.7157)"> + <title>Sheet.7</title> + <desc>setsum</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="12.9268" cy="336.238" width="25.86" height="12.6135"/> + <rect x="0" y="329.932" width="25.8535" height="12.6135" class="st11"/> + <text x="6.37" y="334.44" class="st12" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>setsu<tspan + x="10.49" dy="1.2em" class="st4">m</tspan></text> </g> + </g> + <g id="shape8-38" v:mID="8" v:groupContext="shape" transform="translate(72.5955,0)"> + <title>Circle.29</title> + <desc>List 2 matching Criteria 2</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="27.2233" cy="315.322" width="47.65" height="40.835"/> + <g id="shadow8-39" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st2"/> + <text x="18.79" y="308.12" class="st3" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 2 <tspan + x="12.08" dy="1.2em" class="st4">matching </tspan><tspan x="12.29" dy="1.2em" class="st4">Criteria </tspan>2</text> </g> + <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st5"/> + <text x="18.79" y="308.12" class="st6" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 2 <tspan + x="12.08" dy="1.2em" class="st4">matching </tspan><tspan x="12.29" dy="1.2em" class="st4">Criteria </tspan>2</text> </g> + </g> + <g id="group9-48" transform="translate(31.6515,-49.9094)" v:mID="9" v:groupContext="group"> + <title>Sheet.9</title> + <g id="group10-49" transform="translate(99.5691,0)" v:mID="10" v:groupContext="group"> + <title>Sheet.10</title> + <g id="shape11-50" v:mID="11" v:groupContext="shape" transform="translate(346.175,275.999) rotate(90)"> + <title>Triangle</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow11-51" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,1.9728,-0.3456)" class="st1"> + <path d="M66.55 342.55 L33.27 290.12 L0 342.55 L66.55 342.55 Z" class="st9"/> + </g> + <path d="M66.55 342.55 L33.27 290.12 L0 342.55 L66.55 342.55 Z" class="st10"/> + </g> + <g id="shape12-54" v:mID="12" v:groupContext="shape" transform="translate(355.063,285.074) rotate(90)"> + <title>Sheet.12</title> + <desc>Set Summary</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="24.1985" cy="332.563" width="48.4" height="19.9638"/> + <rect x="0" y="322.581" width="48.397" height="19.9638" class="st11"/> + <text x="18.25" y="329.86" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Set <tspan + x="6.38" dy="1.2em" class="st4">Summary</tspan></text> </g> + </g> + <g id="shape13-58" v:mID="13" v:groupContext="shape" transform="translate(57.5835,-54.4467)"> + <title>Sheet.13</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 342.55 L38.9 342.55" class="st14"/> + </g> + <g id="shape14-64" v:mID="14" v:groupContext="shape" transform="translate(20.2363,-51.8439)"> + <title>Sheet.14</title> + <desc>Flow Key</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="25.3328" cy="333.471" width="50.67" height="18.1489"/> + <rect x="0" y="324.396" width="50.6656" height="18.1489" class="st11"/> + <text x="14.12" y="335.27" class="st16" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Key</text> </g> + <g id="shape15-67" v:mID="15" v:groupContext="shape" transform="translate(5.02911,1.60865) rotate(-26.0815)"> + <title>Sheet.15</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 342.55 L39.25 342.55" class="st14"/> + </g> + <g id="shape16-72" v:mID="16" v:groupContext="shape" transform="translate(155.629,-33.273)"> + <title>Sheet.16</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 342.55 L38.34 342.55" class="st14"/> + </g> + <g id="shape17-77" v:mID="17" v:groupContext="shape" transform="translate(304.141,0.595416) rotate(25.6934)"> + <title>Sheet.17</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 342.55 L42.68 342.55" class="st14"/> + </g> + <g id="shape18-82" v:mID="18" v:groupContext="shape" transform="translate(102.642,654.842) rotate(180)"> + <title>Sheet.18</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 342.55 L30.14 342.55" class="st14"/> + </g> + <g id="shape19-87" v:mID="19" v:groupContext="shape" transform="translate(-15.1809,-33.9928)"> + <title>Sheet.19</title> + <desc>New Flow => New Assignment</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="42.75" cy="338.045" width="85.5" height="9"/> + <rect x="0" y="333.545" width="85.5" height="9" class="st11"/> + <text x="5.06" y="339.85" class="st16" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>New Flow => New Assignment</text> </g> + <g id="shape20-90" v:mID="20" v:groupContext="shape" transform="translate(102.844,679.041) rotate(180)"> + <title>Sheet.20</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 342.55 L30.14 342.55" class="st14"/> + </g> + <g id="shape21-95" v:mID="21" v:groupContext="shape" transform="translate(-35.4309,-11.4928)"> + <title>Sheet.21</title> + <desc>Old Flow => forward to specific thread</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="54" cy="337.971" width="108" height="9.14889"/> + <rect x="0" y="333.396" width="108" height="9.14889" class="st11"/> + <text x="6.36" y="339.77" class="st16" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Old Flow => forward to specific thread</text> </g> + <g id="shape22-98" v:mID="22" v:groupContext="shape" transform="translate(541.496,275.999) rotate(90)"> + <title>Sheet.22</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 335.81 C2.14 344.21 5.09 343.6 7.56 340.31 C10.62 336.25 12.94 328.1 18.15 335.81" class="st17"/> + </g> + <g id="shape23-101" v:mID="23" v:groupContext="shape" transform="translate(541.496,300.198) rotate(90)"> + <title>Sheet.23</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 335.81 C2.14 344.21 5.09 343.6 7.56 340.31 C10.62 336.25 12.94 328.1 18.15 335.81" class="st17"/> + </g> + <g id="shape24-104" v:mID="24" v:groupContext="shape" transform="translate(541.496,324.396) rotate(90)"> + <title>Sheet.24</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 335.81 C2.14 344.21 5.09 343.6 7.56 340.31 C10.62 336.25 12.94 328.1 18.15 335.81" class="st17"/> + </g> + </g> + <g id="group25-107" transform="translate(285.961,-178.628)" v:mID="25" v:groupContext="group"> + <title>Sheet.25</title> + <g id="shape26-108" v:mID="26" v:groupContext="shape" transform="translate(51.2583,-51.2583)"> + <title>Circle</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow26-109" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st9"/> + </g> + <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st10"/> + </g> + <g id="shape27-112" v:mID="27" v:groupContext="shape" transform="translate(107.177,-55.9182)"> + <title>Circle.156</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow27-113" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st9"/> + </g> + <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st10"/> + </g> + <g id="shape28-116" v:mID="28" v:groupContext="shape" transform="translate(79.2174,-83.8773)"> + <title>Circle.157</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow28-117" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st9"/> + </g> + <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st10"/> + </g> + <g id="shape29-120" v:mID="29" v:groupContext="shape" transform="translate(153.775,-51.2583)"> + <title>Circle.158</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow29-121" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st9"/> + </g> + <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st10"/> + </g> + <g id="shape30-124" v:mID="30" v:groupContext="shape" transform="translate(93.197,-18.6394)"> + <title>Circle.159</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow30-125" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st9"/> + </g> + <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st10"/> + </g> + <g id="shape31-128" v:mID="31" v:groupContext="shape" transform="translate(27.4102,-57.9329) rotate(-7.12502)"> + <title>Sheet.31</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 342.55 L31.41 342.55" class="st14"/> + </g> + <g id="shape32-133" v:mID="32" v:groupContext="shape" transform="translate(182.13,-60.5772) rotate(9.46232)"> + <title>Sheet.32</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 342.55 L22.18 342.55" class="st14"/> + </g> + <g id="shape33-138" v:mID="33" v:groupContext="shape" transform="translate(47.8843,595.237) rotate(-160.346)"> + <title>Sheet.33</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 342.55 L63.11 342.55" class="st14"/> + </g> + <g id="shape34-143" v:mID="34" v:groupContext="shape" transform="translate(292.945,525.785) rotate(141.977)"> + <title>Sheet.34</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 342.55 L20.97 342.55" class="st14"/> + </g> + <g id="shape35-148" v:mID="35" v:groupContext="shape" transform="translate(-95.8971,591.793) rotate(-145.945)"> + <title>Sheet.35</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 342.55 L28.55 342.55" class="st14"/> + </g> + <g id="shape36-153" v:mID="36" v:groupContext="shape" transform="translate(37.2788,2.27374E-013)"> + <title>Rectangle.167</title> + <desc>SUM</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="10.8652" cy="335.555" width="21.74" height="13.9795"/> + <g id="shadow36-154" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="328.566" width="21.7305" height="13.9795" class="st9"/> + </g> + <rect x="0" y="328.566" width="21.7305" height="13.9795" class="st10"/> + <text x="5" y="337.36" class="st18" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>SUM</text> </g> + <g id="shape37-158" v:mID="37" v:groupContext="shape" transform="translate(55.9182,2.27374E-013)"> + <title>Rectangle.168</title> + <desc>Packet</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="23.2992" cy="335.555" width="46.6" height="13.9795"/> + <g id="shadow37-159" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="328.566" width="46.5985" height="13.9795" class="st9"/> + </g> + <rect x="0" y="328.566" width="46.5985" height="13.9795" class="st19"/> + <text x="15.18" y="337.36" class="st20" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Packet</text> </g> + <g id="shape38-163" v:mID="38" v:groupContext="shape" transform="translate(-1.65867E-013,-32.6189)"> + <title>Rectangle.169</title> + <desc>SUM</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="10.3796" cy="335.555" width="20.76" height="13.9795"/> + <g id="shadow38-164" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="328.566" width="20.7593" height="13.9795" class="st9"/> + </g> + <rect x="0" y="328.566" width="20.7593" height="13.9795" class="st10"/> + <text x="4.51" y="337.36" class="st18" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>SUM</text> </g> + <g id="shape39-168" v:mID="39" v:groupContext="shape" transform="translate(18.6394,-32.6189)"> + <title>Rectangle.170</title> + <desc>Packet</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="23.2992" cy="335.555" width="46.6" height="13.9795"/> + <g id="shadow39-169" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="328.566" width="46.5985" height="13.9795" class="st9"/> + </g> + <rect x="0" y="328.566" width="46.5985" height="13.9795" class="st19"/> + <text x="15.18" y="337.36" class="st20" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Packet</text> </g> + <g id="shape40-173" v:mID="40" v:groupContext="shape" transform="translate(197.019,626.053) rotate(161.565)"> + <title>Sheet.40</title> + <path d="M0 328.31 A55.7483 27.2427 -124.2 0 0 42.37 334.19 L42.47 333.85" class="st21"/> + </g> + <g id="shape41-179" v:mID="41" v:groupContext="shape" transform="translate(154.607,584.177) rotate(161.121)"> + <title>Sheet.41</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 319.39 A80.5593 29.9756 -101.99 0 0 41.7 325.37 L41.79 325.02" class="st21"/> + </g> + <g id="shape42-184" v:mID="42" v:groupContext="shape" transform="translate(3.02481,-66.7025)"> + <title>Sheet.42</title> + <desc>Encode ID</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="19.4569" cy="335.555" width="38.92" height="13.9795"/> + <rect x="0" y="328.566" width="38.9138" height="13.9795" class="st11"/> + <text x="7.51" y="333.16" class="st23" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Encode <tspan + x="15.99" dy="1.2em" class="st4">ID</tspan></text> </g> + </g> + <g id="group43-188" transform="translate(12.0993,-165.858)" v:mID="43" v:groupContext="group"> + <title>Sheet.43</title> + <g id="group44-189" transform="translate(7.21495,-75.757)" v:mID="44" v:groupContext="group" v:layerMember="0"> + <v:userDefs> + <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/> + <v:ud v:nameU="ConnGap" v:prompt="" v:val="VT0(0.083333333333333):0"/> + </v:userDefs> + <title>User</title> + <g id="shape45-190" v:mID="45" v:groupContext="shape" v:layerMember="0" + transform="translate(13.3353,-1.13687E-013)"> + <title>Sheet.45</title> + <g id="shadow45-191" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42 ZM20.96 + 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 + 13.66 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 + L21.12 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 + 329.46 C22.2 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77 Z" + class="st24"/> + <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42" + class="st25"/> + <path d="M20.96 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 + 13.66 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 + L21.12 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 + 329.46 C22.2 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77" + class="st25"/> + <path d="M18.1 342.55 L18.1 338.37" class="st25"/> + <path d="M15.03 337.32 L15.03 333.71" class="st25"/> + <path d="M21.12 337.32 L21.12 333.71" class="st25"/> + <path d="M0 337.32 L13.47 337.32" class="st25"/> + </g> + <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42 ZM20.96 + 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 13.66 + 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 L21.12 + 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 329.46 C22.2 + 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77 Z" class="st26"/> + <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42" + class="st27"/> + <path d="M20.96 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 13.66 + 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 L21.12 + 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 329.46 C22.2 + 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77" class="st27"/> + <path d="M18.1 342.55 L18.1 338.37" class="st27"/> + <path d="M15.03 337.32 L15.03 333.71" class="st27"/> + <path d="M21.12 337.32 L21.12 333.71" class="st27"/> + <path d="M0 337.32 L13.47 337.32" class="st27"/> + </g> + <g id="shape46-206" v:mID="46" v:groupContext="shape" v:layerMember="0" transform="translate(0,-8.39743)"> + <title>Sheet.46</title> + <g id="shadow46-207" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <path d="M21.09 325.52 C21.09 325.13 20.96 324.79 20.74 324.51 C20.59 324.32 20.4 324.16 20.19 324.06 + C19.97 323.95 19.72 323.89 19.46 323.89 L3.55 323.89 C3.16 323.89 2.82 324.02 2.54 324.23 + C2.35 324.38 2.19 324.57 2.09 324.79 C1.98 325.01 1.92 325.25 1.92 325.52 L1.92 336.04 L21.09 + 336.04 L21.09 325.52 ZM21.18 337.33 L1.77 337.33 L0 340.51 L0 342.55 L23.06 342.55 L23.06 + 340.51 L21.18 337.33 Z" class="st9"/> + </g> + <path d="M21.09 325.52 C21.09 325.13 20.96 324.79 20.74 324.51 C20.59 324.32 20.4 324.16 20.19 324.06 C19.97 + 323.95 19.72 323.89 19.46 323.89 L3.55 323.89 C3.16 323.89 2.82 324.02 2.54 324.23 C2.35 324.38 + 2.19 324.57 2.09 324.79 C1.98 325.01 1.92 325.25 1.92 325.52 L1.92 336.04 L21.09 336.04 L21.09 + 325.52 ZM21.18 337.33 L1.77 337.33 L0 340.51 L0 342.55 L23.06 342.55 L23.06 340.51 L21.18 337.33 + Z" class="st28"/> + </g> + <g id="shape47-210" v:mID="47" v:groupContext="shape" v:layerMember="0" transform="translate(3.19243,-16.175)"> + <title>Sheet.47</title> + <v:userDefs> + <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/> + </v:userDefs> + <path d="M16.62 342.55 L16.62 333.29 C16.62 333.23 16.61 333.18 16.58 333.13 C16.55 333.07 16.5 333.02 16.44 + 332.98 C16.39 332.95 16.33 332.94 16.27 332.94 L0.35 332.94 C0.29 332.94 0.24 332.95 0.19 332.98 + C0.13 333.01 0.08 333.07 0.04 333.12 C0.02 333.17 0 333.23 0 333.29 L0 342.55 L16.62 342.55 + Z" class="st29"/> + </g> + <g id="shape48-212" v:mID="48" v:groupContext="shape" v:layerMember="0" transform="translate(1.97942,-10.81)"> + <title>Sheet.48</title> + <v:userDefs> + <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/> + </v:userDefs> + <path d="M0.96 340.83 L0 342.55 L19.06 342.55 L18.1 340.83 L0.96 340.83 Z" class="st26"/> + </g> + </g> + <g id="group49-215" transform="translate(7.21495,-47.1858)" v:mID="49" v:groupContext="group" v:layerMember="0"> + <v:userDefs> + <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/> + <v:ud v:nameU="ConnGap" v:prompt="" v:val="VT0(0.083333333333333):0"/> + </v:userDefs> + <title>User.7</title> + <g id="shape50-216" v:mID="50" v:groupContext="shape" v:layerMember="0" + transform="translate(13.3353,-1.13687E-013)"> + <title>Sheet.50</title> + <g id="shadow50-217" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42 ZM20.96 + 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 + 13.66 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 + L21.12 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 + 329.46 C22.2 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77 Z" + class="st24"/> + <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42" + class="st25"/> + <path d="M20.96 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 + 13.66 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 + L21.12 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 + 329.46 C22.2 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77" + class="st25"/> + <path d="M18.1 342.55 L18.1 338.37" class="st25"/> + <path d="M15.03 337.32 L15.03 333.71" class="st25"/> + <path d="M21.12 337.32 L21.12 333.71" class="st25"/> + <path d="M0 337.32 L13.47 337.32" class="st25"/> + </g> + <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42 ZM20.96 + 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 13.66 + 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 L21.12 + 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 329.46 C22.2 + 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77 Z" class="st26"/> + <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42" + class="st27"/> + <path d="M20.96 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 13.66 + 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 L21.12 + 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 329.46 C22.2 + 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77" class="st27"/> + <path d="M18.1 342.55 L18.1 338.37" class="st27"/> + <path d="M15.03 337.32 L15.03 333.71" class="st27"/> + <path d="M21.12 337.32 L21.12 333.71" class="st27"/> + <path d="M0 337.32 L13.47 337.32" class="st27"/> + </g> + <g id="shape51-232" v:mID="51" v:groupContext="shape" v:layerMember="0" transform="translate(0,-8.39743)"> + <title>Sheet.51</title> + <g id="shadow51-233" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <path d="M21.09 325.52 C21.09 325.13 20.96 324.79 20.74 324.51 C20.59 324.32 20.4 324.16 20.19 324.06 + C19.97 323.95 19.72 323.89 19.46 323.89 L3.55 323.89 C3.16 323.89 2.82 324.02 2.54 324.23 + C2.35 324.38 2.19 324.57 2.09 324.79 C1.98 325.01 1.92 325.25 1.92 325.52 L1.92 336.04 L21.09 + 336.04 L21.09 325.52 ZM21.18 337.33 L1.77 337.33 L0 340.51 L0 342.55 L23.06 342.55 L23.06 + 340.51 L21.18 337.33 Z" class="st9"/> + </g> + <path d="M21.09 325.52 C21.09 325.13 20.96 324.79 20.74 324.51 C20.59 324.32 20.4 324.16 20.19 324.06 C19.97 + 323.95 19.72 323.89 19.46 323.89 L3.55 323.89 C3.16 323.89 2.82 324.02 2.54 324.23 C2.35 324.38 + 2.19 324.57 2.09 324.79 C1.98 325.01 1.92 325.25 1.92 325.52 L1.92 336.04 L21.09 336.04 L21.09 + 325.52 ZM21.18 337.33 L1.77 337.33 L0 340.51 L0 342.55 L23.06 342.55 L23.06 340.51 L21.18 337.33 + Z" class="st28"/> + </g> + <g id="shape52-236" v:mID="52" v:groupContext="shape" v:layerMember="0" transform="translate(3.19243,-16.175)"> + <title>Sheet.52</title> + <v:userDefs> + <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/> + </v:userDefs> + <path d="M16.62 342.55 L16.62 333.29 C16.62 333.23 16.61 333.18 16.58 333.13 C16.55 333.07 16.5 333.02 16.44 + 332.98 C16.39 332.95 16.33 332.94 16.27 332.94 L0.35 332.94 C0.29 332.94 0.24 332.95 0.19 332.98 + C0.13 333.01 0.08 333.07 0.04 333.12 C0.02 333.17 0 333.23 0 333.29 L0 342.55 L16.62 342.55 + Z" class="st29"/> + </g> + <g id="shape53-238" v:mID="53" v:groupContext="shape" v:layerMember="0" transform="translate(1.97942,-10.81)"> + <title>Sheet.53</title> + <v:userDefs> + <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/> + </v:userDefs> + <path d="M0.96 340.83 L0 342.55 L19.06 342.55 L18.1 340.83 L0.96 340.83 Z" class="st26"/> + </g> + </g> + <g id="group54-241" transform="translate(7.21495,-18.6146)" v:mID="54" v:groupContext="group" v:layerMember="0"> + <v:userDefs> + <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/> + <v:ud v:nameU="ConnGap" v:prompt="" v:val="VT0(0.083333333333333):0"/> + </v:userDefs> + <title>User.12</title> + <g id="shape55-242" v:mID="55" v:groupContext="shape" v:layerMember="0" + transform="translate(13.3353,-1.13687E-013)"> + <title>Sheet.55</title> + <g id="shadow55-243" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42 ZM20.96 + 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 + 13.66 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 + L21.12 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 + 329.46 C22.2 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77 Z" + class="st24"/> + <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42" + class="st25"/> + <path d="M20.96 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 + 13.66 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 + L21.12 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 + 329.46 C22.2 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77" + class="st25"/> + <path d="M18.1 342.55 L18.1 338.37" class="st25"/> + <path d="M15.03 337.32 L15.03 333.71" class="st25"/> + <path d="M21.12 337.32 L21.12 333.71" class="st25"/> + <path d="M0 337.32 L13.47 337.32" class="st25"/> + </g> + <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42 ZM20.96 + 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 13.66 + 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 L21.12 + 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 329.46 C22.2 + 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77 Z" class="st26"/> + <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42" + class="st27"/> + <path d="M20.96 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 13.66 + 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 L21.12 + 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 329.46 C22.2 + 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77" class="st27"/> + <path d="M18.1 342.55 L18.1 338.37" class="st27"/> + <path d="M15.03 337.32 L15.03 333.71" class="st27"/> + <path d="M21.12 337.32 L21.12 333.71" class="st27"/> + <path d="M0 337.32 L13.47 337.32" class="st27"/> + </g> + <g id="shape56-258" v:mID="56" v:groupContext="shape" v:layerMember="0" transform="translate(0,-8.39743)"> + <title>Sheet.56</title> + <g id="shadow56-259" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <path d="M21.09 325.52 C21.09 325.13 20.96 324.79 20.74 324.51 C20.59 324.32 20.4 324.16 20.19 324.06 + C19.97 323.95 19.72 323.89 19.46 323.89 L3.55 323.89 C3.16 323.89 2.82 324.02 2.54 324.23 + C2.35 324.38 2.19 324.57 2.09 324.79 C1.98 325.01 1.92 325.25 1.92 325.52 L1.92 336.04 L21.09 + 336.04 L21.09 325.52 ZM21.18 337.33 L1.77 337.33 L0 340.51 L0 342.55 L23.06 342.55 L23.06 + 340.51 L21.18 337.33 Z" class="st9"/> + </g> + <path d="M21.09 325.52 C21.09 325.13 20.96 324.79 20.74 324.51 C20.59 324.32 20.4 324.16 20.19 324.06 C19.97 + 323.95 19.72 323.89 19.46 323.89 L3.55 323.89 C3.16 323.89 2.82 324.02 2.54 324.23 C2.35 324.38 + 2.19 324.57 2.09 324.79 C1.98 325.01 1.92 325.25 1.92 325.52 L1.92 336.04 L21.09 336.04 L21.09 + 325.52 ZM21.18 337.33 L1.77 337.33 L0 340.51 L0 342.55 L23.06 342.55 L23.06 340.51 L21.18 337.33 + Z" class="st28"/> + </g> + <g id="shape57-262" v:mID="57" v:groupContext="shape" v:layerMember="0" transform="translate(3.19243,-16.175)"> + <title>Sheet.57</title> + <v:userDefs> + <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/> + </v:userDefs> + <path d="M16.62 342.55 L16.62 333.29 C16.62 333.23 16.61 333.18 16.58 333.13 C16.55 333.07 16.5 333.02 16.44 + 332.98 C16.39 332.95 16.33 332.94 16.27 332.94 L0.35 332.94 C0.29 332.94 0.24 332.95 0.19 332.98 + C0.13 333.01 0.08 333.07 0.04 333.12 C0.02 333.17 0 333.23 0 333.29 L0 342.55 L16.62 342.55 + Z" class="st29"/> + </g> + <g id="shape58-264" v:mID="58" v:groupContext="shape" v:layerMember="0" transform="translate(1.97942,-10.81)"> + <title>Sheet.58</title> + <v:userDefs> + <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/> + </v:userDefs> + <path d="M0.96 340.83 L0 342.55 L19.06 342.55 L18.1 340.83 L0.96 340.83 Z" class="st26"/> + </g> + </g> + <g id="group59-267" transform="translate(171.161,-45.6707)" v:mID="59" v:groupContext="group" v:layerMember="0"> + <v:userDefs> + <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/> + <v:ud v:nameU="ConnGap" v:prompt="" v:val="VT0(0.083333333333333):0"/> + </v:userDefs> + <title>Data Center</title> + <g id="shape60-268" v:mID="60" v:groupContext="shape" v:layerMember="0"> + <title>Sheet.60</title> + <g id="shadow60-269" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <ellipse cx="37.8785" cy="331.299" rx="37.8785" ry="11.246" class="st9"/> + </g> + <ellipse cx="37.8785" cy="331.299" rx="37.8785" ry="11.246" class="st10"/> + </g> + <g id="shape61-272" v:mID="61" v:groupContext="shape" v:layerMember="0" transform="translate(6.86487,-7.30475)"> + <title>Sheet.61</title> + <g id="shadow61-273" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <path d="M54.1 311.79 L43.28 311.79 L43.28 342.55 L62.03 342.55 L62.03 311.79 L54.1 311.79 ZM43.28 332.44 + L43.28 311.79 L51.21 311.79 L51.21 301.69 L32.33 301.69 L32.33 311.79 L40.39 311.79 L40.39 + 332.44 L43.28 332.44 ZM40.39 301.69 L40.39 293.03 L21.64 293.03 L21.64 301.69 L29.57 301.69 + L29.57 311.79 L32.46 311.79 L32.46 301.69 L40.39 301.69 ZM32.46 311.79 L21.64 311.79 L21.64 + 342.55 L40.39 342.55 L40.39 311.79 L32.46 311.79 ZM10.82 311.79 L0 311.79 L0 342.55 L18.75 + 342.55 L18.75 311.79 L10.82 311.79 ZM21.64 311.79 L29.57 311.79 L29.57 301.69 L10.82 301.69 + L10.82 311.79 L18.75 311.79 L18.75 332.44 L21.64 332.44 L21.64 311.79 Z" class="st9"/> + </g> + <path d="M54.1 311.79 L43.28 311.79 L43.28 342.55 L62.03 342.55 L62.03 311.79 L54.1 311.79 ZM43.28 332.44 + L43.28 311.79 L51.21 311.79 L51.21 301.69 L32.33 301.69 L32.33 311.79 L40.39 311.79 L40.39 332.44 + L43.28 332.44 ZM40.39 301.69 L40.39 293.03 L21.64 293.03 L21.64 301.69 L29.57 301.69 L29.57 + 311.79 L32.46 311.79 L32.46 301.69 L40.39 301.69 ZM32.46 311.79 L21.64 311.79 L21.64 342.55 + L40.39 342.55 L40.39 311.79 L32.46 311.79 ZM10.82 311.79 L0 311.79 L0 342.55 L18.75 342.55 L18.75 + 311.79 L10.82 311.79 ZM21.64 311.79 L29.57 311.79 L29.57 301.69 L10.82 301.69 L10.82 311.79 + L18.75 311.79 L18.75 332.44 L21.64 332.44 L21.64 311.79 Z" class="st10"/> + </g> + <g id="shape62-276" v:mID="62" v:groupContext="shape" v:layerMember="0" transform="translate(20.0835,-20.5174)"> + <title>Sheet.62</title> + <g id="shadow62-277" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <path d="M45.36 341.36 A1.13296 1.18615 -180 1 0 43.09 341.36 A1.13296 1.18615 -180 1 0 45.36 341.36 + ZM23.46 341.36 A1.13296 1.18615 -180 1 0 21.2 341.36 A1.13296 1.18615 -180 1 0 23.46 341.36 + ZM2.27 341.36 A1.13296 1.18615 -180 1 0 0 341.36 A1.13296 1.18615 -180 1 0 2.27 341.36 Z" + class="st24"/> + </g> + <path d="M45.36 341.36 A1.13296 1.18615 -180 1 0 43.09 341.36 A1.13296 1.18615 -180 1 0 45.36 341.36 ZM23.46 + 341.36 A1.13296 1.18615 -180 1 0 21.2 341.36 A1.13296 1.18615 -180 1 0 23.46 341.36 ZM2.27 341.36 + A1.13296 1.18615 -180 1 0 0 341.36 A1.13296 1.18615 -180 1 0 2.27 341.36 Z" class="st30"/> + </g> + <g id="shape63-282" v:mID="63" v:groupContext="shape" v:layerMember="0" transform="translate(14.2717,-12.5134)"> + <title>Sheet.63</title> + <v:userDefs> + <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/> + </v:userDefs> + <g id="shadow63-283" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <path d="M43.09 342.55 L51.17 342.55 L51.17 341.74 L43.09 341.74 L43.09 342.55 ZM43.09 340.12 L51.17 + 340.12 L51.17 339.32 L43.09 339.32 L43.09 340.12 ZM43.09 337.69 L51.17 337.69 L51.17 336.89 + L43.09 336.89 L43.09 337.69 ZM21.2 342.55 L29.27 342.55 L29.27 341.74 L21.2 341.74 L21.2 + 342.55 ZM21.2 340.12 L29.27 340.12 L29.27 339.32 L21.2 339.32 L21.2 340.12 ZM21.2 337.69 + L29.27 337.69 L29.27 336.89 L21.2 336.89 L21.2 337.69 ZM-0 342.55 L8.08 342.55 L8.08 341.74 + L-0 341.74 L-0 342.55 ZM-0 340.12 L8.08 340.12 L8.08 339.32 L-0 339.32 L-0 340.12 ZM-0 337.69 + L8.08 337.69 L8.08 336.89 L-0 336.89 L-0 337.69 Z" class="st24"/> + </g> + <path d="M43.09 342.55 L51.17 342.55 L51.17 341.74 L43.09 341.74 L43.09 342.55 ZM43.09 340.12 L51.17 340.12 + L51.17 339.32 L43.09 339.32 L43.09 340.12 ZM43.09 337.69 L51.17 337.69 L51.17 336.89 L43.09 + 336.89 L43.09 337.69 ZM21.2 342.55 L29.27 342.55 L29.27 341.74 L21.2 341.74 L21.2 342.55 ZM21.2 + 340.12 L29.27 340.12 L29.27 339.32 L21.2 339.32 L21.2 340.12 ZM21.2 337.69 L29.27 337.69 L29.27 + 336.89 L21.2 336.89 L21.2 337.69 ZM-0 342.55 L8.08 342.55 L8.08 341.74 L-0 341.74 L-0 342.55 + ZM-0 340.12 L8.08 340.12 L8.08 339.32 L-0 339.32 L-0 340.12 ZM-0 337.69 L8.08 337.69 L8.08 336.89 + L-0 336.89 L-0 337.69 Z" class="st26"/> + </g> + </g> + <g id="group64-288" transform="translate(59.5234,-47.1858)" v:mID="64" v:groupContext="group"> + <v:custProps> + <v:cp v:nameU="AssetNumber" v:lbl="Asset Number" v:prompt="" v:type="0" v:format="" v:sortKey="Asset" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="SerialNumber" v:lbl="Serial Number" v:prompt="" v:type="0" v:format="" v:sortKey="Asset" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="Location" v:lbl="Location" v:prompt="" v:type="0" v:format="" v:sortKey="Asset" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="Building" v:lbl="Building" v:prompt="" v:type="0" v:format="" v:sortKey="Asset" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="Room" v:lbl="Room" v:prompt="" v:type="0" v:format="" v:sortKey="Asset" v:invis="false" + v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="Manufacturer" v:lbl="Manufacturer" v:prompt="" v:type="0" v:format="" v:sortKey="Equipment" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="ProductNumber" v:lbl="Product Number" v:prompt="" v:type="0" v:format="" + v:sortKey="Equipment" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="PartNumber" v:lbl="Part Number" v:prompt="" v:type="0" v:format="" v:sortKey="Equipment" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="ProductDescription" v:lbl="Product Description" v:prompt="" v:type="0" v:format="" + v:sortKey="Equipment" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="NetworkName" v:lbl="Network Name" v:prompt="" v:type="0" v:format="" v:sortKey="Network" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="IPAddress" v:lbl="IP Address" v:prompt="" v:type="0" v:format="" v:sortKey="Network" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="SubnetMask" v:lbl="Subnet Mask" v:prompt="" v:type="0" v:format="" v:sortKey="Network" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="AdminInterface" v:lbl="Administrative Interface" v:prompt="" v:type="0" v:format="" + v:sortKey="Network" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="NumberofPorts" v:lbl="Number of Ports" v:prompt="" v:type="0" v:format="" + v:sortKey="Network" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="CommunityString" v:lbl="Community String" v:prompt="" v:type="0" v:format="" + v:sortKey="Network" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="NetworkDescription" v:lbl="Network Description" v:prompt="" v:type="0" v:format="" + v:sortKey="Network" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="MACAddress" v:lbl="MAC Address" v:prompt="" v:type="0" v:format="" v:sortKey="Network" + v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/> + <v:cp v:nameU="ShapeClass" v:lbl="ShapeClass" v:prompt="" v:type="0" v:format="" v:sortKey="" + v:invis="true" v:ask="false" v:langID="1033" v:cal="0" v:val="VT4(Equipment)"/> + <v:cp v:nameU="ShapeType" v:lbl="ShapeType" v:prompt="" v:type="0" v:format="" v:sortKey="" v:invis="true" + v:ask="false" v:langID="1033" v:cal="0" v:val="VT4(Device)"/> + <v:cp v:nameU="SubShapeType" v:lbl="SubShapeType" v:prompt="" v:type="0" v:format="" v:sortKey="" + v:invis="true" v:ask="false" v:langID="1033" v:cal="0" v:val="VT4(Load balancer)"/> + </v:custProps> + <v:userDefs> + <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/> + <v:ud v:nameU="ShapeClass" v:prompt="" v:val="VT0(5):26"/> + <v:ud v:nameU="SolSH" v:prompt="" v:val="VT14({BF0433D9-CD73-4EB5-8390-8653BE590246}):41"/> + <v:ud v:nameU="visLegendShape" v:prompt="" v:val="VT0(2):26"/> + </v:userDefs> + <title>Load balancer</title> + <g id="shape65-289" v:mID="65" v:groupContext="shape" transform="translate(0,-1.653)"> + <title>Sheet.65</title> + <g id="shadow65-290" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <path d="M12.18 329.62 L4.06 329.62 L0 332.02 L0 342.55 L16.23 342.55 L16.23 332.02 L12.18 329.62 Z" + class="st24"/> + <path d="M0 332.02 L16.23 332.02" class="st25"/> + <path d="M12.18 329.62 L4.06 329.62 L0 332.02 L0 342.55 L16.23 342.55 L16.23 332.02 L12.18 329.62" + class="st25"/> + </g> + <path d="M12.18 329.62 L4.06 329.62 L0 332.02 L0 342.55 L16.23 342.55 L16.23 332.02 L12.18 329.62 Z" + class="st30"/> + <path d="M0 332.02 L16.23 332.02" class="st31"/> + <path d="M12.18 329.62 L4.06 329.62 L0 332.02 L0 342.55 L16.23 342.55 L16.23 332.02 L12.18 329.62" + class="st31"/> + </g> + <g id="shape66-297" v:mID="66" v:groupContext="shape" transform="translate(1.81062,-2.91583)"> + <title>Sheet.66</title> + <g id="shadow66-298" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <path d="M10.22 341.92 L9.29 342.12 L9.95 342.55 L11.2 342.23 L10.99 340.96 L10.33 340.52 L10.53 341.44 + L8.34 340.01 L8.03 340.49 L10.22 341.92 ZM11.46 338.22 L8.84 338.22 L8.84 338.78 L11.45 + 338.78 L10.78 339.45 L11.57 339.45 L12.45 338.5 L11.57 337.55 L10.78 337.55 L11.46 338.22 + ZM10.48 335.2 L8.29 336.64 L8.6 337.12 L10.79 335.68 L10.59 336.61 L11.25 336.17 L11.46 + 334.9 L10.21 334.58 L9.55 335.01 L10.48 335.2 ZM6.25 336.37 C5.11 336.37 4.19 337.29 4.19 + 338.43 C4.19 339.56 5.11 340.48 6.25 340.48 C7.38 340.48 8.31 339.56 8.31 338.43 C8.31 337.29 + 7.38 336.37 6.25 336.37 ZM6.25 337.02 C7.02 337.02 7.66 337.65 7.66 338.43 C7.66 339.2 7.02 + 339.83 6.25 339.83 C5.47 339.83 4.84 339.2 4.84 338.43 C4.84 337.65 5.47 337.02 6.25 337.02 + ZM2.62 338.14 L0 338.14 L0 338.71 L2.62 338.71 L1.94 339.38 L2.74 339.38 L3.61 338.43 L2.73 + 337.47 L1.95 337.47 L2.62 338.14 Z" class="st9"/> + </g> + <path d="M10.22 341.92 L9.29 342.12 L9.95 342.55 L11.2 342.23 L10.99 340.96 L10.33 340.52 L10.53 341.44 L8.34 + 340.01 L8.03 340.49 L10.22 341.92 ZM11.46 338.22 L8.84 338.22 L8.84 338.78 L11.45 338.78 L10.78 + 339.45 L11.57 339.45 L12.45 338.5 L11.57 337.55 L10.78 337.55 L11.46 338.22 ZM10.48 335.2 L8.29 + 336.64 L8.6 337.12 L10.79 335.68 L10.59 336.61 L11.25 336.17 L11.46 334.9 L10.21 334.58 L9.55 + 335.01 L10.48 335.2 ZM6.25 336.37 C5.11 336.37 4.19 337.29 4.19 338.43 C4.19 339.56 5.11 340.48 + 6.25 340.48 C7.38 340.48 8.31 339.56 8.31 338.43 C8.31 337.29 7.38 336.37 6.25 336.37 ZM6.25 + 337.02 C7.02 337.02 7.66 337.65 7.66 338.43 C7.66 339.2 7.02 339.83 6.25 339.83 C5.47 339.83 + 4.84 339.2 4.84 338.43 C4.84 337.65 5.47 337.02 6.25 337.02 ZM2.62 338.14 L0 338.14 L0 338.71 + L2.62 338.71 L1.94 339.38 L2.74 339.38 L3.61 338.43 L2.73 337.47 L1.95 337.47 L2.62 338.14 Z" + class="st32"/> + </g> + </g> + <g id="group67-301" transform="translate(104.617,-86.5795)" v:mID="67" v:groupContext="group"> + <v:userDefs> + <v:ud v:nameU="SkinColor" v:prompt="" v:val="VT5(#da8c36)"/> + <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/> + </v:userDefs> + <title>Directory server</title> + <g id="shape68-302" v:mID="68" v:groupContext="shape" transform="translate(0,-0.451005)"> + <title>Sheet.68</title> + <g id="shadow68-303" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <path d="M0.47 329.86 L0.47 331.94 L1.46 332.57 L3.33 331.52 L15.43 338.57 L15.43 340.61 L16.42 341.24 + L18.24 340.22 L22.24 342.55 L22.24 339.27 L36.07 331.28 L36.07 321.27 L19.64 311.85 L3.16 + 321.13 L3.16 321.5 L0 319.68 L0 329.58 L0.47 329.86 Z" class="st33"/> + </g> + <path d="M0.47 329.86 L0.47 331.94 L1.46 332.57 L3.33 331.52 L15.43 338.57 L15.43 340.61 L16.42 341.24 L18.24 + 340.22 L22.24 342.55 L22.24 339.27 L36.07 331.28 L36.07 321.27 L19.64 311.85 L3.16 321.13 L3.16 + 321.5 L0 319.68 L0 329.58 L0.47 329.86 Z" class="st34"/> + </g> + <g id="shape69-306" v:mID="69" v:groupContext="shape" transform="translate(3.1636,-11.8063)"> + <title>Sheet.69</title> + <path d="M16.48 323.24 L32.91 332.66 L16.31 342.55 L0 333.26 L0 332.52 L16.48 323.24 Z" class="st35"/> + </g> + <g id="shape70-310" v:mID="70" v:groupContext="shape" transform="translate(19.06,-3.68954)"> + <title>Sheet.70</title> + <path d="M17.01 324.55 L0 334.19 L3.18 342.55 L17.01 334.56 L17.01 324.55 Z" class="st36"/> + </g> + <g id="shape71-314" v:mID="71" v:groupContext="shape" transform="translate(0,-0.415652)"> + <title>Sheet.71</title> + <path d="M22.24 342.55 L0 329.58 L0 319.68 L22.24 332.43 L22.24 342.55 Z" class="st37"/> + </g> + <g id="shape72-322" v:mID="72" v:groupContext="shape" transform="translate(0.82443,-19.8334)"> + <title>Sheet.72</title> + <path d="M1.13 341.58 a0.653986 0.653986 -180 0 0 -0.73971 -0.492434 a0.656072 0.656072 -180 0 0 -0.253101 + 0.865731 a0.653066 0.653066 -180 0 0 0.740769 0.491375 a0.655459 0.655459 -180 0 0 0.252042 + -0.864672 Z" class="st38"/> + </g> + <g id="shape73-326" v:mID="73" v:groupContext="shape" transform="translate(3.62283,-15.1638)"> + <title>Sheet.73</title> + <path d="M3.22 339.78 A1.86495 1.86495 -180 0 0 1.11 338.38 A1.8709 1.8709 -180 0 0 0.38 340.85 A1.86532 + 1.86532 -180 0 0 2.5 342.25 A1.87264 1.87264 -180 0 0 3.22 339.78 Z" class="st38"/> + </g> + <g id="shape74-329" v:mID="74" v:groupContext="shape" transform="translate(3.62283,-10.4867)"> + <title>Sheet.74</title> + <path d="M3.22 339.78 A1.86495 1.86495 -180 0 0 1.11 338.38 A1.8709 1.8709 -180 0 0 0.38 340.85 A1.86532 + 1.86532 -180 0 0 2.5 342.25 A1.87264 1.87264 -180 0 0 3.22 339.78 Z" class="st38"/> + </g> + <g id="shape75-332" v:mID="75" v:groupContext="shape" transform="translate(4.52404,-16.3668)"> + <title>Sheet.75</title> + <path d="M1.61 341.16 a0.931952 0.931952 -180 0 0 -1.05741 -0.702645 a0.935408 0.935408 -180 0 0 -0.361118 + 1.23585 a0.932139 0.932139 -180 0 0 1.05794 0.702645 a0.935822 0.935822 -180 0 0 0.360589 -1.23585 + Z" class="st39"/> + </g> + <g id="shape76-336" v:mID="76" v:groupContext="shape" transform="translate(4.52404,-11.6897)"> + <title>Sheet.76</title> + <path d="M1.61 341.16 a0.931952 0.931952 -180 0 0 -1.05741 -0.702645 a0.935875 0.935875 -180 0 0 -0.361118 + 1.23585 a0.932139 0.932139 -180 0 0 1.05794 0.702645 a0.935822 0.935822 -180 0 0 0.360589 -1.23585 + Z" class="st39"/> + </g> + <g id="shape77-339" v:mID="77" v:groupContext="shape" transform="translate(7.78787,-8.83469)"> + <title>Sheet.77</title> + <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/> + </g> + <g id="shape78-343" v:mID="78" v:groupContext="shape" transform="translate(10.204,-7.4008)"> + <title>Sheet.78</title> + <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/> + </g> + <g id="shape79-346" v:mID="79" v:groupContext="shape" transform="translate(12.6196,-5.96639)"> + <title>Sheet.79</title> + <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/> + </g> + <g id="shape80-349" v:mID="80" v:groupContext="shape" transform="translate(15.0357,-4.53251)"> + <title>Sheet.80</title> + <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/> + </g> + <g id="shape81-352" v:mID="81" v:groupContext="shape" transform="translate(8.24006,-10.0631)"> + <title>Sheet.81</title> + <path d="M0.85 342.24 a0.388199 0.388199 0 0 1 -0.425188 0.308698 a0.638045 0.638045 0 0 1 -0.424658 -0.573447 + L0 336.5 a0.387575 0.387575 0 0 1 0.424658 -0.308698 a0.637725 0.637725 0 0 1 0.425188 0.573447 + L0.85 342.24 Z" class="st41"/> + </g> + <g id="shape82-356" v:mID="82" v:groupContext="shape" transform="translate(10.6556,-8.62924)"> + <title>Sheet.82</title> + <path d="M0.85 342.24 a0.388199 0.388199 0 0 1 -0.425188 0.308698 a0.638045 0.638045 0 0 1 -0.424658 -0.573447 + L0 336.5 a0.387575 0.387575 0 0 1 0.424658 -0.308698 a0.637725 0.637725 0 0 1 0.425188 0.573447 + L0.85 342.24 Z" class="st41"/> + </g> + <g id="shape83-359" v:mID="83" v:groupContext="shape" transform="translate(13.0717,-7.19483)"> + <title>Sheet.83</title> + <path d="M0.85 342.24 a0.388199 0.388199 0 0 1 -0.425188 0.308698 a0.638045 0.638045 0 0 1 -0.424658 -0.573447 + L0 336.5 a0.387575 0.387575 0 0 1 0.424658 -0.308698 a0.637725 0.637725 0 0 1 0.425188 0.573447 + L0.85 342.24 Z" class="st41"/> + </g> + <g id="shape84-362" v:mID="84" v:groupContext="shape" transform="translate(15.4873,-5.76095)"> + <title>Sheet.84</title> + <path d="M0.85 342.24 a0.388502 0.388502 0 0 1 -0.425717 0.308698 a0.638367 0.638367 0 0 1 -0.424129 -0.573447 + L0 336.5 a0.387272 0.387272 0 0 1 0.424129 -0.308698 a0.638235 0.638235 0 0 1 0.425717 0.573447 + L0.85 342.24 Z" class="st41"/> + </g> + <g id="shape85-365" v:mID="85" v:groupContext="shape" transform="translate(7.78787,-9.81214)"> + <title>Sheet.85</title> + <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/> + <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/> + </g> + <g id="shape86-368" v:mID="86" v:groupContext="shape" transform="translate(10.204,-8.37826)"> + <title>Sheet.86</title> + <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/> + <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/> + </g> + <g id="shape87-371" v:mID="87" v:groupContext="shape" transform="translate(12.6196,-6.94385)"> + <title>Sheet.87</title> + <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/> + <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/> + </g> + <g id="shape88-374" v:mID="88" v:groupContext="shape" transform="translate(15.0357,-5.50996)"> + <title>Sheet.88</title> + <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/> + <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/> + </g> + <g id="shape89-377" v:mID="89" v:groupContext="shape" transform="translate(7.78787,-4.53251)"> + <title>Sheet.89</title> + <path d="M9.08 334.57 L9.03 342.55 L7.25 341.57 L9.08 334.57 ZM6.66 333.14 L6.61 341.11 L4.83 340.13 L6.66 + 333.14 ZM4.25 331.7 L4.2 339.68 L2.42 338.7 L4.25 331.7 ZM1.83 330.27 L1.78 338.24 L0 337.27 + L1.83 330.27 Z" class="st42"/> + <path d="M9.08 334.57 L9.03 342.55 L7.25 341.57M6.66 333.14 L6.61 341.11 L4.83 340.13M4.25 331.7 L4.2 339.68 + L2.42 338.7M1.83 330.27 L1.78 338.24 L0 337.27" class="st44"/> + </g> + <g id="shape90-380" v:mID="90" v:groupContext="shape" transform="translate(2.22125,-11.8454)"> + <title>Sheet.90</title> + <path d="M0 341.85 L0.63 341.42 L1.42 341.78 L0.03 342.55 L0 341.85 Z" class="st45"/> + </g> + <g id="shape91-384" v:mID="91" v:groupContext="shape" transform="translate(17.1796,-3.17487)"> + <title>Sheet.91</title> + <path d="M0 341.85 L0.63 341.42 L1.42 341.78 L0.03 342.55 L0 341.85 Z" class="st45"/> + </g> + <g id="shape92-387" v:mID="92" v:groupContext="shape" transform="translate(1.46036,-10.3893)"> + <title>Sheet.92</title> + <path d="M2.12 341.35 L0 342.55 L0 333.54 L2.12 332.29 L2.12 333.41 L0.79 334.15 L0.79 341.09 L2.18 340.33 + L2.12 341.35 Z" class="st36"/> + </g> + <g id="shape93-390" v:mID="93" v:groupContext="shape" transform="translate(16.4187,-1.71875)"> + <title>Sheet.93</title> + <path d="M2.12 341.35 L0 342.55 L0 333.54 L2.12 332.29 L2.12 333.41 L0.79 334.15 L0.79 341.09 L2.18 340.33 + L2.12 341.35 Z" class="st36"/> + </g> + <g id="shape94-393" v:mID="94" v:groupContext="shape" transform="translate(0.467548,-10.3893)"> + <title>Sheet.94</title> + <path d="M0.99 333.54 L3.11 332.29 L2.12 331.66 L0 332.91 L0 341.92 L0.99 342.55 L0.99 333.54 Z" + class="st46"/> + </g> + <g id="shape95-397" v:mID="95" v:groupContext="shape" transform="translate(15.4259,-1.71875)"> + <title>Sheet.95</title> + <path d="M0.99 333.54 L3.11 332.29 L2.12 331.66 L0 332.91 L0 341.92 L0.99 342.55 L0.99 333.54 Z" + class="st46"/> + </g> + <g id="shape96-400" v:mID="96" v:groupContext="shape" transform="translate(0.467548,-1.71928)"> + <title>Sheet.96</title> + <path d="M17.34 339.96 L16.75 340.37 L16.75 334.15 L18.07 333.41 L18.07 332.29 L17.08 331.66 L14.96 332.91 + L14.96 341.92 L15.95 342.55 L18.07 341.35 L18.14 340.33 L17.34 339.96 ZM2.38 331.29 L1.79 331.7 + L1.79 325.48 L3.11 324.74 L3.11 323.62 L2.12 322.99 L0 324.24 L0 333.25 L0.99 333.87 L3.11 332.68 + L3.18 331.66 L2.38 331.29 Z" class="st47"/> + </g> + <g id="shape97-402" v:mID="97" v:groupContext="shape" transform="translate(19.9526,-8.71396)"> + <title>Sheet.97</title> + <path d="M1.13 341.58 a0.653986 0.653986 -180 0 0 -0.73971 -0.492434 a0.656072 0.656072 -180 0 0 -0.253101 + 0.865731 a0.653066 0.653066 -180 0 0 0.740769 0.491375 a0.655459 0.655459 -180 0 0 0.252042 + -0.864672 Z" class="st38"/> + </g> + <g id="shape98-405" v:mID="98" v:groupContext="shape" transform="translate(19.9526,-2.35997)"> + <title>Sheet.98</title> + <path d="M1.13 341.58 a0.653986 0.653986 -180 0 0 -0.73971 -0.492434 a0.656072 0.656072 -180 0 0 -0.253101 + 0.865731 a0.653066 0.653066 -180 0 0 0.740769 0.491375 a0.655459 0.655459 -180 0 0 0.252042 + -0.864672 Z" class="st38"/> + </g> + <g id="shape99-408" v:mID="99" v:groupContext="shape" transform="translate(0,-0.415652)"> + <title>Sheet.99</title> + <path d="M36.07 331.28 L36.07 321.27 L19.64 311.85 L3.16 321.13 L3.16 321.52 L0 319.68 L0 329.58 L0.47 329.86 + L0.47 331.94 L1.46 332.57 L3.33 331.52 L15.43 338.57 L15.43 340.61 L16.42 341.24 L18.24 340.22 + L22.24 342.55 L22.24 339.27 L36.07 331.28 Z" class="st47"/> + </g> + <g id="shape100-410" v:mID="100" v:groupContext="shape" transform="translate(27.8077,-2.86477)"> + <title>Sheet.100</title> + <path d="M0.29 342.55 L6.62 338.89 A1.82805 1.82805 0 0 1 6.62 336.9 L0.29 340.55 A1.82805 1.82805 -180 0 + 0 0.29 342.55 Z" class="st48"/> + </g> + <g id="shape101-412" v:mID="101" v:groupContext="shape" transform="translate(23.5035,-4.85627)"> + <title>Sheet.101</title> + <path d="M4.6 342.55 L10.92 338.89 L6.32 336.24 L0 339.89 L4.6 342.55 Z" class="st49"/> + </g> + <g id="shape102-416" v:mID="102" v:groupContext="shape" transform="translate(23.3588,-2.86477)"> + <title>Sheet.102</title> + <path d="M0.14 339.89 L4.74 342.55 A1.82805 1.82805 0 0 1 4.74 340.55 L0.14 337.9 A3.49826 3.49826 -180 0 + 0 0.14 339.89 Z" class="st50"/> + </g> + <g id="shape103-420" v:mID="103" v:groupContext="shape" transform="translate(25.8933,-5.98478)"> + <title>Sheet.103</title> + <path d="M2.87 342.55 L0 340.89" class="st51"/> + <path d="M0.94 340.34 L3.82 342" class="st51"/> + <path d="M1.88 339.8 L4.76 341.46" class="st51"/> + <path d="M2.82 339.26 L5.7 340.92" class="st51"/> + <path d="M3.76 338.71 L6.64 340.37" class="st51"/> + </g> + <g id="shape104-427" v:mID="104" v:groupContext="shape" transform="translate(23.5035,-7.51159)"> + <title>Sheet.104</title> + <path d="M5.13 341.17 L11.45 337.52 A11.9345 11.9345 0 0 1 6.32 338.89 L0 342.55 A11.9345 11.9345 -180 0 + 0 5.13 341.17 Z" class="st52"/> + </g> + <g id="shape105-435" v:mID="105" v:groupContext="shape" transform="translate(30.2106,-4.74563)"> + <title>Sheet.105</title> + <path d="M0.98 341.98 L0 342.55" class="st51"/> + <path d="M1.26 341.48 L2.24 340.92" class="st51"/> + <path d="M2.53 340.42 L3.51 339.86" class="st51"/> + </g> + <g id="shape106-440" v:mID="106" v:groupContext="shape" transform="translate(23.3588,-2.86477)"> + <title>Sheet.106</title> + <path d="M0.14 339.89 L4.74 342.55 L11.07 338.89 A1.82805 1.82805 0 0 1 11.07 336.9 L7.85 335.04 L11.6 332.87 + A11.9345 11.9345 0 0 1 6.47 334.25 L0.14 337.9 A3.49826 3.49826 -180 0 0 0.14 339.89" + class="st53"/> + </g> + </g> + <g id="group107-443" transform="translate(104.617,-33.8201)" v:mID="107" v:groupContext="group"> + <v:userDefs> + <v:ud v:nameU="SkinColor" v:prompt="" v:val="VT5(#da8c36)"/> + <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/> + </v:userDefs> + <title>Directory server.104</title> + <g id="shape108-444" v:mID="108" v:groupContext="shape" transform="translate(0,-0.451005)"> + <title>Sheet.108</title> + <g id="shadow108-445" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <path d="M0.47 329.86 L0.47 331.94 L1.46 332.57 L3.33 331.52 L15.43 338.57 L15.43 340.61 L16.42 341.24 + L18.24 340.22 L22.24 342.55 L22.24 339.27 L36.07 331.28 L36.07 321.27 L19.64 311.85 L3.16 + 321.13 L3.16 321.5 L0 319.68 L0 329.58 L0.47 329.86 Z" class="st33"/> + </g> + <path d="M0.47 329.86 L0.47 331.94 L1.46 332.57 L3.33 331.52 L15.43 338.57 L15.43 340.61 L16.42 341.24 L18.24 + 340.22 L22.24 342.55 L22.24 339.27 L36.07 331.28 L36.07 321.27 L19.64 311.85 L3.16 321.13 L3.16 + 321.5 L0 319.68 L0 329.58 L0.47 329.86 Z" class="st34"/> + </g> + <g id="shape109-448" v:mID="109" v:groupContext="shape" transform="translate(3.1636,-11.8063)"> + <title>Sheet.109</title> + <path d="M16.48 323.24 L32.91 332.66 L16.31 342.55 L0 333.26 L0 332.52 L16.48 323.24 Z" class="st35"/> + </g> + <g id="shape110-451" v:mID="110" v:groupContext="shape" transform="translate(19.06,-3.68954)"> + <title>Sheet.110</title> + <path d="M17.01 324.55 L0 334.19 L3.18 342.55 L17.01 334.56 L17.01 324.55 Z" class="st36"/> + </g> + <g id="shape111-454" v:mID="111" v:groupContext="shape" transform="translate(0,-0.415652)"> + <title>Sheet.111</title> + <path d="M22.24 342.55 L0 329.58 L0 319.68 L22.24 332.43 L22.24 342.55 Z" class="st37"/> + </g> + <g id="shape112-457" v:mID="112" v:groupContext="shape" transform="translate(0.82443,-19.8334)"> + <title>Sheet.112</title> + <path d="M1.13 341.58 a0.653986 0.653986 -180 0 0 -0.73971 -0.492434 a0.656072 0.656072 -180 0 0 -0.253101 + 0.865731 a0.653066 0.653066 -180 0 0 0.740769 0.491375 a0.655459 0.655459 -180 0 0 0.252042 + -0.864672 Z" class="st38"/> + </g> + <g id="shape113-460" v:mID="113" v:groupContext="shape" transform="translate(3.62283,-15.1638)"> + <title>Sheet.113</title> + <path d="M3.22 339.78 A1.86495 1.86495 -180 0 0 1.11 338.38 A1.8709 1.8709 -180 0 0 0.38 340.85 A1.86532 + 1.86532 -180 0 0 2.5 342.25 A1.87264 1.87264 -180 0 0 3.22 339.78 Z" class="st38"/> + </g> + <g id="shape114-463" v:mID="114" v:groupContext="shape" transform="translate(3.62283,-10.4867)"> + <title>Sheet.114</title> + <path d="M3.22 339.78 A1.86495 1.86495 -180 0 0 1.11 338.38 A1.8709 1.8709 -180 0 0 0.38 340.85 A1.86532 + 1.86532 -180 0 0 2.5 342.25 A1.87264 1.87264 -180 0 0 3.22 339.78 Z" class="st38"/> + </g> + <g id="shape115-466" v:mID="115" v:groupContext="shape" transform="translate(4.52404,-16.3668)"> + <title>Sheet.115</title> + <path d="M1.61 341.16 a0.931952 0.931952 -180 0 0 -1.05741 -0.702645 a0.935408 0.935408 -180 0 0 -0.361118 + 1.23585 a0.932139 0.932139 -180 0 0 1.05794 0.702645 a0.935822 0.935822 -180 0 0 0.360589 -1.23585 + Z" class="st39"/> + </g> + <g id="shape116-469" v:mID="116" v:groupContext="shape" transform="translate(4.52404,-11.6897)"> + <title>Sheet.116</title> + <path d="M1.61 341.16 a0.931952 0.931952 -180 0 0 -1.05741 -0.702645 a0.935875 0.935875 -180 0 0 -0.361118 + 1.23585 a0.932139 0.932139 -180 0 0 1.05794 0.702645 a0.935822 0.935822 -180 0 0 0.360589 -1.23585 + Z" class="st39"/> + </g> + <g id="shape117-472" v:mID="117" v:groupContext="shape" transform="translate(7.78787,-8.83469)"> + <title>Sheet.117</title> + <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/> + </g> + <g id="shape118-475" v:mID="118" v:groupContext="shape" transform="translate(10.204,-7.4008)"> + <title>Sheet.118</title> + <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/> + </g> + <g id="shape119-478" v:mID="119" v:groupContext="shape" transform="translate(12.6196,-5.96639)"> + <title>Sheet.119</title> + <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/> + </g> + <g id="shape120-481" v:mID="120" v:groupContext="shape" transform="translate(15.0357,-4.53251)"> + <title>Sheet.120</title> + <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/> + </g> + <g id="shape121-484" v:mID="121" v:groupContext="shape" transform="translate(8.24006,-10.0631)"> + <title>Sheet.121</title> + <path d="M0.85 342.24 a0.388199 0.388199 0 0 1 -0.425188 0.308698 a0.638045 0.638045 0 0 1 -0.424658 -0.573447 + L0 336.5 a0.387575 0.387575 0 0 1 0.424658 -0.308698 a0.637725 0.637725 0 0 1 0.425188 0.573447 + L0.85 342.24 Z" class="st41"/> + </g> + <g id="shape122-487" v:mID="122" v:groupContext="shape" transform="translate(10.6556,-8.62924)"> + <title>Sheet.122</title> + <path d="M0.85 342.24 a0.388199 0.388199 0 0 1 -0.425188 0.308698 a0.638045 0.638045 0 0 1 -0.424658 -0.573447 + L0 336.5 a0.387575 0.387575 0 0 1 0.424658 -0.308698 a0.637725 0.637725 0 0 1 0.425188 0.573447 + L0.85 342.24 Z" class="st41"/> + </g> + <g id="shape123-490" v:mID="123" v:groupContext="shape" transform="translate(13.0717,-7.19483)"> + <title>Sheet.123</title> + <path d="M0.85 342.24 a0.388199 0.388199 0 0 1 -0.425188 0.308698 a0.638045 0.638045 0 0 1 -0.424658 -0.573447 + L0 336.5 a0.387575 0.387575 0 0 1 0.424658 -0.308698 a0.637725 0.637725 0 0 1 0.425188 0.573447 + L0.85 342.24 Z" class="st41"/> + </g> + <g id="shape124-493" v:mID="124" v:groupContext="shape" transform="translate(15.4873,-5.76095)"> + <title>Sheet.124</title> + <path d="M0.85 342.24 a0.388502 0.388502 0 0 1 -0.425717 0.308698 a0.638367 0.638367 0 0 1 -0.424129 -0.573447 + L0 336.5 a0.387272 0.387272 0 0 1 0.424129 -0.308698 a0.638235 0.638235 0 0 1 0.425717 0.573447 + L0.85 342.24 Z" class="st41"/> + </g> + <g id="shape125-496" v:mID="125" v:groupContext="shape" transform="translate(7.78787,-9.81214)"> + <title>Sheet.125</title> + <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/> + <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/> + </g> + <g id="shape126-499" v:mID="126" v:groupContext="shape" transform="translate(10.204,-8.37826)"> + <title>Sheet.126</title> + <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/> + <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/> + </g> + <g id="shape127-502" v:mID="127" v:groupContext="shape" transform="translate(12.6196,-6.94385)"> + <title>Sheet.127</title> + <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/> + <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/> + </g> + <g id="shape128-505" v:mID="128" v:groupContext="shape" transform="translate(15.0357,-5.50996)"> + <title>Sheet.128</title> + <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/> + <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/> + </g> + <g id="shape129-508" v:mID="129" v:groupContext="shape" transform="translate(7.78787,-4.53251)"> + <title>Sheet.129</title> + <path d="M9.08 334.57 L9.03 342.55 L7.25 341.57 L9.08 334.57 ZM6.66 333.14 L6.61 341.11 L4.83 340.13 L6.66 + 333.14 ZM4.25 331.7 L4.2 339.68 L2.42 338.7 L4.25 331.7 ZM1.83 330.27 L1.78 338.24 L0 337.27 + L1.83 330.27 Z" class="st42"/> + <path d="M9.08 334.57 L9.03 342.55 L7.25 341.57M6.66 333.14 L6.61 341.11 L4.83 340.13M4.25 331.7 L4.2 339.68 + L2.42 338.7M1.83 330.27 L1.78 338.24 L0 337.27" class="st44"/> + </g> + <g id="shape130-511" v:mID="130" v:groupContext="shape" transform="translate(2.22125,-11.8454)"> + <title>Sheet.130</title> + <path d="M0 341.85 L0.63 341.42 L1.42 341.78 L0.03 342.55 L0 341.85 Z" class="st45"/> + </g> + <g id="shape131-514" v:mID="131" v:groupContext="shape" transform="translate(17.1796,-3.17487)"> + <title>Sheet.131</title> + <path d="M0 341.85 L0.63 341.42 L1.42 341.78 L0.03 342.55 L0 341.85 Z" class="st45"/> + </g> + <g id="shape132-517" v:mID="132" v:groupContext="shape" transform="translate(1.46036,-10.3893)"> + <title>Sheet.132</title> + <path d="M2.12 341.35 L0 342.55 L0 333.54 L2.12 332.29 L2.12 333.41 L0.79 334.15 L0.79 341.09 L2.18 340.33 + L2.12 341.35 Z" class="st36"/> + </g> + <g id="shape133-520" v:mID="133" v:groupContext="shape" transform="translate(16.4187,-1.71875)"> + <title>Sheet.133</title> + <path d="M2.12 341.35 L0 342.55 L0 333.54 L2.12 332.29 L2.12 333.41 L0.79 334.15 L0.79 341.09 L2.18 340.33 + L2.12 341.35 Z" class="st36"/> + </g> + <g id="shape134-523" v:mID="134" v:groupContext="shape" transform="translate(0.467548,-10.3893)"> + <title>Sheet.134</title> + <path d="M0.99 333.54 L3.11 332.29 L2.12 331.66 L0 332.91 L0 341.92 L0.99 342.55 L0.99 333.54 Z" + class="st46"/> + </g> + <g id="shape135-526" v:mID="135" v:groupContext="shape" transform="translate(15.4259,-1.71875)"> + <title>Sheet.135</title> + <path d="M0.99 333.54 L3.11 332.29 L2.12 331.66 L0 332.91 L0 341.92 L0.99 342.55 L0.99 333.54 Z" + class="st46"/> + </g> + <g id="shape136-529" v:mID="136" v:groupContext="shape" transform="translate(0.467548,-1.71928)"> + <title>Sheet.136</title> + <path d="M17.34 339.96 L16.75 340.37 L16.75 334.15 L18.07 333.41 L18.07 332.29 L17.08 331.66 L14.96 332.91 + L14.96 341.92 L15.95 342.55 L18.07 341.35 L18.14 340.33 L17.34 339.96 ZM2.38 331.29 L1.79 331.7 + L1.79 325.48 L3.11 324.74 L3.11 323.62 L2.12 322.99 L0 324.24 L0 333.25 L0.99 333.87 L3.11 332.68 + L3.18 331.66 L2.38 331.29 Z" class="st47"/> + </g> + <g id="shape137-531" v:mID="137" v:groupContext="shape" transform="translate(19.9526,-8.71396)"> + <title>Sheet.137</title> + <path d="M1.13 341.58 a0.653986 0.653986 -180 0 0 -0.73971 -0.492434 a0.656072 0.656072 -180 0 0 -0.253101 + 0.865731 a0.653066 0.653066 -180 0 0 0.740769 0.491375 a0.655459 0.655459 -180 0 0 0.252042 + -0.864672 Z" class="st38"/> + </g> + <g id="shape138-534" v:mID="138" v:groupContext="shape" transform="translate(19.9526,-2.35997)"> + <title>Sheet.138</title> + <path d="M1.13 341.58 a0.653986 0.653986 -180 0 0 -0.73971 -0.492434 a0.656072 0.656072 -180 0 0 -0.253101 + 0.865731 a0.653066 0.653066 -180 0 0 0.740769 0.491375 a0.655459 0.655459 -180 0 0 0.252042 + -0.864672 Z" class="st38"/> + </g> + <g id="shape139-537" v:mID="139" v:groupContext="shape" transform="translate(0,-0.415652)"> + <title>Sheet.139</title> + <path d="M36.07 331.28 L36.07 321.27 L19.64 311.85 L3.16 321.13 L3.16 321.52 L0 319.68 L0 329.58 L0.47 329.86 + L0.47 331.94 L1.46 332.57 L3.33 331.52 L15.43 338.57 L15.43 340.61 L16.42 341.24 L18.24 340.22 + L22.24 342.55 L22.24 339.27 L36.07 331.28 Z" class="st47"/> + </g> + <g id="shape140-539" v:mID="140" v:groupContext="shape" transform="translate(27.8077,-2.86477)"> + <title>Sheet.140</title> + <path d="M0.29 342.55 L6.62 338.89 A1.82805 1.82805 0 0 1 6.62 336.9 L0.29 340.55 A1.82805 1.82805 -180 0 + 0 0.29 342.55 Z" class="st48"/> + </g> + <g id="shape141-541" v:mID="141" v:groupContext="shape" transform="translate(23.5035,-4.85627)"> + <title>Sheet.141</title> + <path d="M4.6 342.55 L10.92 338.89 L6.32 336.24 L0 339.89 L4.6 342.55 Z" class="st49"/> + </g> + <g id="shape142-544" v:mID="142" v:groupContext="shape" transform="translate(23.3588,-2.86477)"> + <title>Sheet.142</title> + <path d="M0.14 339.89 L4.74 342.55 A1.82805 1.82805 0 0 1 4.74 340.55 L0.14 337.9 A3.49826 3.49826 -180 0 + 0 0.14 339.89 Z" class="st50"/> + </g> + <g id="shape143-547" v:mID="143" v:groupContext="shape" transform="translate(25.8933,-5.98478)"> + <title>Sheet.143</title> + <path d="M2.87 342.55 L0 340.89" class="st51"/> + <path d="M0.94 340.34 L3.82 342" class="st51"/> + <path d="M1.88 339.8 L4.76 341.46" class="st51"/> + <path d="M2.82 339.26 L5.7 340.92" class="st51"/> + <path d="M3.76 338.71 L6.64 340.37" class="st51"/> + </g> + <g id="shape144-554" v:mID="144" v:groupContext="shape" transform="translate(23.5035,-7.51159)"> + <title>Sheet.144</title> + <path d="M5.13 341.17 L11.45 337.52 A11.9345 11.9345 0 0 1 6.32 338.89 L0 342.55 A11.9345 11.9345 -180 0 + 0 5.13 341.17 Z" class="st52"/> + </g> + <g id="shape145-557" v:mID="145" v:groupContext="shape" transform="translate(30.2106,-4.74563)"> + <title>Sheet.145</title> + <path d="M0.98 341.98 L0 342.55" class="st51"/> + <path d="M1.26 341.48 L2.24 340.92" class="st51"/> + <path d="M2.53 340.42 L3.51 339.86" class="st51"/> + </g> + <g id="shape146-562" v:mID="146" v:groupContext="shape" transform="translate(23.3588,-2.86477)"> + <title>Sheet.146</title> + <path d="M0.14 339.89 L4.74 342.55 L11.07 338.89 A1.82805 1.82805 0 0 1 11.07 336.9 L7.85 335.04 L11.6 332.87 + A11.9345 11.9345 0 0 1 6.47 334.25 L0.14 337.9 A3.49826 3.49826 -180 0 0 0.14 339.89" + class="st53"/> + </g> + </g> + <g id="shape147-565" v:mID="147" v:groupContext="shape" transform="translate(427.321,214.49) rotate(90)"> + <title>Cloud</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/> + </v:userDefs> + <path d="M5.37 311.54 A8.61618 10.0654 0 0 1 9.5 292.2 A17.4727 20.4114 0 0 1 34.86 275.89 A20.0634 23.4379 0 + 0 1 56.58 272.26 A12.5816 14.6977 0 0 1 75.21 271.05 A14.3244 16.7336 0 0 1 97.98 277.09 A10.2423 + 11.9646 0 0 1 106.25 294.02 A12.6864 14.8197 0 0 1 95.9 318.19 A16.0049 18.6962 0 0 1 73.14 330.27 + A18.8712 22.0444 0 0 1 42.1 335.11 A23.9217 27.9441 0 0 1 15.2 330.27 A9.43759 11.0249 0 0 1 5.37 + 311.54 Z" class="st42"/> + <path d="M5.37 311.54 A8.61618 10.0654 0 0 1 9.5 292.2 A17.4727 20.4114 0 0 1 34.86 275.89 A20.0634 23.4379 0 + 0 1 56.58 272.26 A12.5816 14.6977 0 0 1 75.21 271.05 A14.3244 16.7336 0 0 1 97.98 277.09 A10.2423 + 11.9646 0 0 1 106.25 294.02 A12.6864 14.8197 0 0 1 95.9 318.19 A16.0049 18.6962 0 0 1 73.14 330.27 + A18.8712 22.0444 0 0 1 42.1 335.11 A23.9217 27.9441 0 0 1 15.2 330.27 A9.43759 11.0249 0 0 1 5.37 + 311.54" class="st54"/> + <path d="M11.05 312.14 A8.59237 10.0375 0 0 1 5.37 311.54" class="st54"/> + <path d="M40.54 332.09 A8.62978 10.0812 -180 0 0 42.1 335.11" class="st54"/> + <path d="M73.92 326.65 A6.96633 8.13801 0 0 1 73.14 330.27" class="st54"/> + <path d="M89.7 308.52 A7.30994 8.5394 0 0 1 95.9 318.19" class="st54"/> + <path d="M103.15 297.64 A6.67364 7.79609 -180 0 0 106.25 294.02" class="st54"/> + <path d="M37.96 278.3 A10.2914 12.0219 -180 0 0 34.86 275.89" class="st54"/> + </g> + <g id="shape148-574" v:mID="148" v:groupContext="shape" transform="translate(110.222,-64.9346)"> + <title>Triangle</title> + <desc>setsum</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="15.6995" cy="336.449" width="31.4" height="12.1933"/> + <g id="shadow148-575" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <path d="M31.4 342.55 L14.67 324.26 L0 342.55 L31.4 342.55 Z" class="st9"/> + </g> + <path d="M31.4 342.55 L14.67 324.26 L0 342.55 L31.4 342.55 Z" class="st10"/> + <text x="8.35" y="337.95" class="st55" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>setsum</text> </g> + <g id="shape149-579" v:mID="149" v:groupContext="shape" transform="translate(292.639,20.8827) rotate(45)"> + <title>Sheet.149</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 342.55 L14.71 342.55" class="st14"/> + </g> + <g id="shape150-584" v:mID="150" v:groupContext="shape" transform="translate(43.2897,-54.1122)"> + <title>Sheet.150</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 342.55 L10.07 342.55" class="st14"/> + </g> + <g id="shape151-589" v:mID="151" v:groupContext="shape" transform="translate(-112.261,8.34531) rotate(-28.1394)"> + <title>Sheet.151</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 342.55 L18 342.55" class="st14"/> + </g> + <g id="shape152-594" v:mID="152" v:groupContext="shape"> + <title>Sheet.152</title> + <desc>Clients</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="32.4673" cy="337.134" width="64.94" height="10.8224"/> + <rect x="0" y="331.723" width="64.9346" height="10.8224" class="st11"/> + <text x="21.5" y="339.53" class="st6" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Clients</text> </g> + <g id="shape153-597" v:mID="153" v:groupContext="shape" transform="translate(83.578,-9.58078)"> + <title>Sheet.153</title> + <desc>Distributed Cache</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="42.0677" cy="337.134" width="84.14" height="10.8224"/> + <rect x="0" y="331.723" width="84.1355" height="10.8224" class="st11"/> + <text x="13.1" y="339.53" class="st6" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Distributed Cache</text> </g> + <g id="shape154-600" v:mID="154" v:groupContext="shape" transform="translate(181.983,-18.6146)"> + <title>Sheet.154</title> + <desc>Web Servers</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="32.4673" cy="337.134" width="64.94" height="10.8224"/> + <rect x="0" y="331.723" width="64.9346" height="10.8224" class="st11"/> + <text x="11.93" y="339.53" class="st6" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Web Servers</text> </g> + <g id="shape155-603" v:mID="155" v:groupContext="shape" transform="translate(96.6068,630.978) rotate(180)"> + <title>Simple Arrow</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + <v:ud v:nameU="ArrowType" v:prompt="" v:val="VT0(2):26"/> + </v:userDefs> + <g id="shadow155-604" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,-0.3456,-1.9728)" class="st1"> + <path d="M0 342.55 L12 330.55 L12 336.55 L16.23 336.55 L16.23 342.55 L16.23 348.54 L12 348.54 L12 354.54 + L0 342.55 Z" class="st56"/> + </g> + <path d="M0 342.55 L12 330.55 L12 336.55 L16.23 336.55 L16.23 342.55 L16.23 348.54 L12 348.54 L12 354.54 L0 342.55 + Z" class="st57"/> + </g> + <g id="shape156-607" v:mID="156" v:groupContext="shape" transform="translate(173.159,625.567) rotate(180)"> + <title>Simple Arrow.153</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + <v:ud v:nameU="ArrowType" v:prompt="" v:val="VT0(2):26"/> + </v:userDefs> + <g id="shadow156-608" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,-0.3456,-1.9728)" class="st1"> + <path d="M0 342.55 L12 330.55 L12 336.55 L16.23 336.55 L16.23 342.55 L16.23 348.54 L12 348.54 L12 354.54 + L0 342.55 Z" class="st56"/> + </g> + <path d="M0 342.55 L12 330.55 L12 336.55 L16.23 336.55 L16.23 342.55 L16.23 348.54 L12 348.54 L12 354.54 L0 342.55 + Z" class="st57"/> + </g> + </g> + <g id="shape157-611" v:mID="157" v:groupContext="shape" transform="translate(0,-149.475)"> + <title>Rectangle</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow157-612" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="193.823" width="271.116" height="148.722" class="st58"/> + </g> + <rect x="0" y="193.823" width="271.116" height="148.722" class="st59"/> + </g> + <g id="shape158-615" v:mID="158" v:groupContext="shape" transform="translate(271.116,-149.475)"> + <title>Rectangle.158</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow158-616" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="193.823" width="202.104" height="148.722" class="st58"/> + </g> + <rect x="0" y="193.823" width="202.104" height="148.722" class="st59"/> + </g> + <g id="shape159-619" v:mID="159" v:groupContext="shape"> + <title>Rectangle.159</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow159-620" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="193.823" width="271.116" height="148.722" class="st58"/> + </g> + <rect x="0" y="193.823" width="271.116" height="148.722" class="st59"/> + </g> + <g id="shape160-623" v:mID="160" v:groupContext="shape" transform="translate(271.116,0)"> + <title>Rectangle.160</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow160-624" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="193.823" width="202.104" height="148.722" class="st58"/> + </g> + <rect x="0" y="193.823" width="202.104" height="148.722" class="st59"/> + </g> + <g id="shape161-627" v:mID="161" v:groupContext="shape" transform="translate(83.578,-151.241)"> + <title>Sheet.161</title> + <desc>(a) Distributed Web Cache</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="54.3546" cy="333.806" width="108.71" height="17.4792"/> + <g id="shadow161-628" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="325.066" width="108.709" height="17.4792" class="st9"/> + </g> + <rect x="0" y="325.066" width="108.709" height="17.4792" class="st10"/> + <text x="4" y="336.81" class="st60" v:langID="1033"><v:paragraph v:spLine="-1.5" v:spBefore="8" v:spAfter="16" + v:bulletSize="0.166667"/><v:tabList/>(a) Distributed Web Cache</text> </g> + <g id="shape162-632" v:mID="162" v:groupContext="shape" transform="translate(319.513,-151.241)"> + <title>Sheet.162</title> + <desc>(b) Detecting Routing Loops</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="54.3546" cy="333.806" width="108.71" height="17.4792"/> + <g id="shadow162-633" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="325.066" width="108.709" height="17.4792" class="st9"/> + </g> + <rect x="0" y="325.066" width="108.709" height="17.4792" class="st10"/> + <text x="4" y="336.81" class="st60" v:langID="1033"><v:paragraph v:spLine="-1.5" v:spBefore="8" v:spAfter="16" + v:bulletSize="0.166667"/><v:tabList/>(b) Detecting Routing Loops</text> </g> + <g id="shape163-637" v:mID="163" v:groupContext="shape" transform="translate(77.5283,-3.35965)"> + <title>Sheet.163</title> + <desc>(c) In-order Workload Scheduler</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="63.5211" cy="333.806" width="127.05" height="17.4792"/> + <g id="shadow163-638" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="325.066" width="127.042" height="17.4792" class="st9"/> + </g> + <rect x="0" y="325.066" width="127.042" height="17.4792" class="st10"/> + <text x="4" y="336.81" class="st60" v:langID="1033"><v:paragraph v:spLine="-1.5" v:spBefore="8" v:spAfter="16" + v:bulletSize="0.166667"/><v:tabList/>(c) In-order Workload Scheduler</text> </g> + <g id="shape164-642" v:mID="164" v:groupContext="shape" transform="translate(307.414,-3.35965)"> + <title>Sheet.164</title> + <desc>(d) Database Semi-join Operations</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="66.2253" cy="333.806" width="132.46" height="17.4792"/> + <g id="shadow164-643" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="325.066" width="132.451" height="17.4792" class="st9"/> + </g> + <rect x="0" y="325.066" width="132.451" height="17.4792" class="st10"/> + <text x="4" y="336.81" class="st60" v:langID="1033"><v:paragraph v:spLine="-1.5" v:spBefore="8" v:spAfter="16" + v:bulletSize="0.166667"/><v:tabList/>(d) Database Semi-join Operations</text> </g> + </g> + </g> +</svg> diff --git a/doc/guides/prog_guide/img/member_i2.svg b/doc/guides/prog_guide/img/member_i2.svg new file mode 100644 index 00000000..759c6545 --- /dev/null +++ b/doc/guides/prog_guide/img/member_i2.svg @@ -0,0 +1,36 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"> +<!-- Generated by Microsoft Visio, SVG Export memship_i2.svg Page-1 --> +<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events" + xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" width="4.38194in" height="1.25694in" + viewBox="0 0 315.5 90.5" xml:space="preserve" color-interpolation-filters="sRGB" class="st6"> + <v:documentProperties v:langID="1033" v:viewMarkup="false"/> + + <style type="text/css"> + <![CDATA[ + .st1 {fill:none;stroke:none;stroke-width:0.25} + .st2 {fill:#5b9bd5;font-family:Calibri;font-size:1.16666em} + .st3 {baseline-shift:32.4943%;font-size:0.649886em} + .st4 {font-size:1em} + .st5 {font-family:Cambria Math;font-size:1em} + .st6 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3} + ]]> + </style> + + <g v:mID="0" v:index="1" v:groupContext="foregroundPage"> + <title>Page-1</title> + <v:pageProperties v:drawingScale="1" v:pageScale="1" v:drawingUnits="0" v:shadowOffsetX="9" v:shadowOffsetY="-9"/> + <g id="shape3-1" v:mID="3" v:groupContext="shape" transform="translate(0.25,-0.25)"> + <title>Sheet.3</title> + <desc>False Positive Probability = (1-(1-1/m)kn)k ≃ (1-ekn/m)k</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="157.5" cy="45.5" width="315" height="90"/> + <rect x="0" y="0.5" width="315" height="90" class="st1"/> + <text x="8.28" y="49.82" class="st2" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>False Positive Probability = (1-(1-1/m)<tspan + dy="-0.234em" class="st3" v:baseFontSize="14">kn</tspan><tspan dy="0.152em" class="st4">)</tspan><tspan + dy="-0.234em" class="st3" v:baseFontSize="14">k</tspan><tspan dy="0.152em" class="st4"> </tspan><tspan + class="st5">≃</tspan> (1-e<tspan dy="-0.234em" class="st3" v:baseFontSize="14">kn</tspan><tspan class="st3" + v:baseFontSize="14">/</tspan><tspan class="st3" v:baseFontSize="14">m</tspan><tspan dy="0.152em" + class="st4">)</tspan><tspan dy="-0.234em" class="st3" v:baseFontSize="14">k</tspan></text> </g> + </g> +</svg> diff --git a/doc/guides/prog_guide/img/member_i3.svg b/doc/guides/prog_guide/img/member_i3.svg new file mode 100644 index 00000000..41e92cb8 --- /dev/null +++ b/doc/guides/prog_guide/img/member_i3.svg @@ -0,0 +1,148 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"> +<!-- Generated by Microsoft Visio, SVG Export memship_i3.svg Page-1 --> +<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events" + width="4.71875in" height="2.84375in" viewBox="0 0 339.75 204.75" xml:space="preserve" color-interpolation-filters="sRGB" + class="st14"> + <style type="text/css"> + <![CDATA[ + .st1 {visibility:visible} + .st2 {fill:#5b9bd5;fill-opacity:0.22;filter:url(#filter_2);stroke:#5b9bd5;stroke-opacity:0.22} + .st3 {fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25} + .st4 {marker-end:url(#mrkr5-32);stroke:#5b9bd5;stroke-linecap:round;stroke-linejoin:round;stroke-width:1} + .st5 {fill:#5b9bd5;fill-opacity:1;stroke:#5b9bd5;stroke-opacity:1;stroke-width:0.28409090909091} + .st6 {fill:#feffff;font-family:Calibri;font-size:0.833336em} + .st7 {font-size:1em} + .st8 {fill:#deebf6;stroke:#c7c8c8;stroke-width:0.25} + .st9 {fill:#000000;font-family:Calibri;font-size:0.833336em} + .st10 {marker-end:url(#mrkr5-84);stroke:#ff0000;stroke-linecap:round;stroke-linejoin:round;stroke-width:1} + .st11 {fill:#ff0000;fill-opacity:1;stroke:#ff0000;stroke-opacity:1;stroke-width:0.28409090909091} + .st12 {fill:none;stroke:none;stroke-width:0.25} + .st13 {fill:#ff0000;font-family:Calibri;font-size:1.00001em} + .st14 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3} + ]]> + </style> + + <defs id="Markers"> + <g id="lend5"> + <path d="M 2 1 L 0 0 L 1.98117 -0.993387 C 1.67173 -0.364515 1.67301 0.372641 1.98465 1.00043 " style="stroke:none"/> + </g> + <marker id="mrkr5-32" class="st5" refX="-6.16" orient="auto" markerUnits="strokeWidth" overflow="visible"> + <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/> + </marker> + <marker id="mrkr5-84" class="st11" refX="-5.8" orient="auto" markerUnits="strokeWidth" overflow="visible"> + <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/> + </marker> + </defs> + <defs id="Filters"> + <filter id="filter_2"> + <feGaussianBlur stdDeviation="2"/> + </filter> + </defs> + <g> + <title>Page-1</title> + <g id="group174-1" transform="translate(3.0294,-5.3478)"> + <title>Sheet.174</title> + <g id="shape155-2" transform="translate(99,-99)"> + <title>Circle</title> + <g id="shadow155-3" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st2"/> + </g> + <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st3"/> + </g> + <g id="shape156-7" transform="translate(207,-108)"> + <title>Circle.156</title> + <g id="shadow156-8" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st2"/> + </g> + <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st3"/> + </g> + <g id="shape157-12" transform="translate(153,-162)"> + <title>Circle.157</title> + <g id="shadow157-13" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st2"/> + </g> + <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st3"/> + </g> + <g id="shape158-17" transform="translate(297,-99)"> + <title>Circle.158</title> + <g id="shadow158-18" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st2"/> + </g> + <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st3"/> + </g> + <g id="shape159-22" transform="translate(180,-36)"> + <title>Circle.159</title> + <g id="shadow159-23" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st2"/> + </g> + <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st3"/> + </g> + <g id="shape160-27" transform="translate(109.604,-115.419) rotate(-7.12502)"> + <title>Sheet.160</title> + <path d="M0 204.75 L66.4 204.75" class="st4"/> + </g> + <g id="shape161-33" transform="translate(276.661,-123.214) rotate(9.46232)"> + <title>Sheet.161</title> + <path d="M0 204.75 L48.58 204.75" class="st4"/> + </g> + <g id="shape162-38" transform="translate(246.135,262.572) rotate(-160.346)"> + <title>Sheet.162</title> + <path d="M0 204.75 L127.63 204.75" class="st4"/> + </g> + <g id="shape163-43" transform="translate(284.391,198.775) rotate(141.977)"> + <title>Sheet.163</title> + <path d="M0 204.75 L46.23 204.75" class="st4"/> + </g> + <g id="shape164-48" transform="translate(70.6118,307.655) rotate(-145.945)"> + <title>Sheet.164</title> + <path d="M0 204.75 L60.88 204.75" class="st4"/> + </g> + <g id="shape167-53" transform="translate(72,0)"> + <title>Rectangle.167</title> + <desc>BF of IDs</desc> + <g id="shadow167-54" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="177.75" width="36" height="27" class="st2"/> + </g> + <rect x="0" y="177.75" width="36" height="27" class="st3"/> + <text x="7.69" y="188.25" class="st6">BF of <tspan x="11.71" dy="1.2em" class="st7">IDs</tspan></text> </g> + <g id="shape168-60" transform="translate(108,0)"> + <title>Rectangle.168</title> + <desc>Packet</desc> + <g id="shadow168-61" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="177.75" width="90" height="27" class="st2"/> + </g> + <rect x="0" y="177.75" width="90" height="27" class="st8"/> + <text x="31.47" y="194.25" class="st9">Packet</text> </g> + <g id="shape169-66" transform="translate(0,-63)"> + <title>Rectangle.169</title> + <desc>BF of IDs</desc> + <g id="shadow169-67" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="177.75" width="36" height="27" class="st2"/> + </g> + <rect x="0" y="177.75" width="36" height="27" class="st3"/> + <text x="7.69" y="188.25" class="st6">BF of <tspan x="11.71" dy="1.2em" class="st7">IDs</tspan></text> </g> + <g id="shape170-73" transform="translate(36,-63)"> + <title>Rectangle.170</title> + <desc>Packet</desc> + <g id="shadow170-74" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="177.75" width="90" height="27" class="st2"/> + </g> + <rect x="0" y="177.75" width="90" height="27" class="st8"/> + <text x="31.47" y="194.25" class="st9">Packet</text> </g> + <g id="shape171-79" transform="translate(240.248,331.493) rotate(161.565)"> + <title>Sheet.171</title> + <path d="M-0 190.52 A81.3416 36.0611 -153.48 0 0 82.31 195.86 L82.49 195.55" class="st10"/> + </g> + <g id="shape172-85" transform="translate(156.426,260.029) rotate(161.565)"> + <title>Sheet.172</title> + <path d="M-0 181.6 A88.1422 54.1439 -124.1 0 0 82.68 187.13 L82.83 186.81" class="st10"/> + </g> + <g id="shape173-90" transform="translate(18,-121.5)"> + <title>Sheet.173</title> + <desc>Encode ID</desc> + <rect x="0" y="177.75" width="63" height="27" class="st12"/> + <text x="7.02" y="194.85" class="st13">Encode ID</text> </g> + </g> + </g> +</svg> diff --git a/doc/guides/prog_guide/img/member_i4.svg b/doc/guides/prog_guide/img/member_i4.svg new file mode 100644 index 00000000..a2b6f2f5 --- /dev/null +++ b/doc/guides/prog_guide/img/member_i4.svg @@ -0,0 +1,450 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.0//EN" "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd"> +<!-- Generated by Microsoft Visio 11.0, SVG Export, v1.0 memship_i4.svg Page-1 --> +<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events" + xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" width="7.625in" height="3.125in" viewBox="0 0 549 225" + xml:space="preserve" color-interpolation-filters="sRGB" class="st18"> + <v:documentProperties v:langID="1033" v:viewMarkup="false"/> + + <style type="text/css"> + <![CDATA[ + .st1 {fill:none;stroke:#ff0000;stroke-width:0.25} + .st2 {fill:#5b9bd5;font-family:Calibri;font-size:1.00001em} + .st3 {marker-end:url(#mrkr5-10);stroke:#ff0000;stroke-linecap:round;stroke-linejoin:round;stroke-width:2.25} + .st4 {fill:#ff0000;fill-opacity:1;stroke:#ff0000;stroke-opacity:1;stroke-width:0.47169811320755} + .st5 {visibility:visible} + .st6 {fill:#5b9bd5;fill-opacity:0.22;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:0.25} + .st7 {fill:#5b9bd5;stroke:#c8c8c8;stroke-width:0.25} + .st8 {fill:none;stroke:none;stroke-width:0.25} + .st9 {fill:#5b9bd5;font-family:Calibri;font-size:1.16666em} + .st10 {stroke:#5b9bd5;stroke-dasharray:1.5,3;stroke-linecap:round;stroke-linejoin:round;stroke-width:1.5} + .st11 {fill:#5b9bd5;fill-opacity:0.25;stroke:#5b9bd5;stroke-opacity:0.25;stroke-width:0.75} + .st12 {fill:#4f88bb;stroke:#41719c;stroke-width:0.75} + .st13 {fill:#ffffff;font-family:Calibri;font-size:1.00001em;font-weight:bold} + .st14 {fill:#000000;font-family:Calibri;font-size:1.00001em} + .st15 {font-size:1em} + .st16 {marker-end:url(#mrkr5-162);stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:1} + .st17 {fill:#000000;fill-opacity:1;stroke:#000000;stroke-opacity:1;stroke-width:0.28409090909091} + .st18 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3} + ]]> + </style> + + <defs id="Markers"> + <g id="lend5"> + <path d="M 2 1 L 0 0 L 1.98117 -0.993387 C 1.67173 -0.364515 1.67301 0.372641 1.98465 1.00043 " style="stroke:none"/> + </g> + <marker id="mrkr5-10" class="st4" v:arrowType="5" v:arrowSize="2" v:setback="3.71" refX="-3.71" orient="auto" + markerUnits="strokeWidth" overflow="visible"> + <use xlink:href="#lend5" transform="scale(-2.12,-2.12) "/> + </marker> + <marker id="mrkr5-162" class="st17" v:arrowType="5" v:arrowSize="2" v:setback="5.8" refX="-5.8" orient="auto" + markerUnits="strokeWidth" overflow="visible"> + <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/> + </marker> + </defs> + <g v:mID="0" v:index="1" v:groupContext="foregroundPage"> + <title>Page-1</title> + <v:pageProperties v:drawingScale="1" v:pageScale="1" v:drawingUnits="0" v:shadowOffsetX="9" v:shadowOffsetY="-9"/> + <g id="group47-1" transform="translate(3.0294,-0.25)" v:mID="47" v:groupContext="group"> + <title>Sheet.47</title> + <g id="shape1-2" v:mID="1" v:groupContext="shape" transform="translate(177.75,-191.922)"> + <title>Sheet.1</title> + <desc>Element</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="54" cy="216" width="108" height="18"/> + <rect x="0" y="207" width="108" height="18" class="st1"/> + <text x="33.77" y="219.6" class="st2" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Element</text> </g> + <g id="shape2-5" v:mID="2" v:groupContext="shape" transform="translate(456.75,33.0781) rotate(90)"> + <title>Sheet.2</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 225 L18.65 225" class="st3"/> + </g> + <g id="shape3-11" v:mID="3" v:groupContext="shape" transform="translate(0,-67.0469)"> + <title>Rectangle.54</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow3-12" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5"> + <rect x="0" y="211.5" width="27" height="13.5" class="st6"/> + </g> + <rect x="0" y="211.5" width="27" height="13.5" class="st7"/> + </g> + <g id="shape4-15" v:mID="4" v:groupContext="shape" transform="translate(27,-67.0469)"> + <title>Rectangle.55</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow4-16" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5"> + <rect x="0" y="211.5" width="27" height="13.5" class="st6"/> + </g> + <rect x="0" y="211.5" width="27" height="13.5" class="st7"/> + </g> + <g id="shape5-19" v:mID="5" v:groupContext="shape" transform="translate(54,-67.0469)"> + <title>Rectangle.56</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow5-20" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5"> + <rect x="0" y="211.5" width="27" height="13.5" class="st6"/> + </g> + <rect x="0" y="211.5" width="27" height="13.5" class="st7"/> + </g> + <g id="shape6-23" v:mID="6" v:groupContext="shape" transform="translate(0,-53.5469)"> + <title>Rectangle.57</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow6-24" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5"> + <rect x="0" y="211.5" width="27" height="13.5" class="st6"/> + </g> + <rect x="0" y="211.5" width="27" height="13.5" class="st7"/> + </g> + <g id="shape7-27" v:mID="7" v:groupContext="shape" transform="translate(27,-53.5469)"> + <title>Rectangle.58</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow7-28" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5"> + <rect x="0" y="211.5" width="27" height="13.5" class="st6"/> + </g> + <rect x="0" y="211.5" width="27" height="13.5" class="st7"/> + </g> + <g id="shape8-31" v:mID="8" v:groupContext="shape" transform="translate(54,-53.5469)"> + <title>Rectangle.59</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow8-32" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5"> + <rect x="0" y="211.5" width="27" height="13.5" class="st6"/> + </g> + <rect x="0" y="211.5" width="27" height="13.5" class="st7"/> + </g> + <g id="shape9-35" v:mID="9" v:groupContext="shape" transform="translate(5.625,-72.6719)"> + <title>Sheet.9</title> + <desc>BF-1</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="36" cy="211.5" width="72" height="27"/> + <rect x="0" y="198" width="72" height="27" class="st8"/> + <text x="23.29" y="215.7" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>BF-1</text> </g> + <g id="shape10-38" v:mID="10" v:groupContext="shape" transform="translate(128.25,-65.0781)"> + <title>Rectangle.74</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow10-39" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5"> + <rect x="0" y="211.5" width="27" height="13.5" class="st6"/> + </g> + <rect x="0" y="211.5" width="27" height="13.5" class="st7"/> + </g> + <g id="shape11-42" v:mID="11" v:groupContext="shape" transform="translate(155.25,-65.0781)"> + <title>Rectangle.75</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow11-43" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5"> + <rect x="0" y="211.5" width="27" height="13.5" class="st6"/> + </g> + <rect x="0" y="211.5" width="27" height="13.5" class="st7"/> + </g> + <g id="shape12-46" v:mID="12" v:groupContext="shape" transform="translate(182.25,-65.0781)"> + <title>Rectangle.76</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow12-47" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5"> + <rect x="0" y="211.5" width="27" height="13.5" class="st6"/> + </g> + <rect x="0" y="211.5" width="27" height="13.5" class="st7"/> + </g> + <g id="shape13-50" v:mID="13" v:groupContext="shape" transform="translate(128.25,-51.5781)"> + <title>Rectangle.77</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow13-51" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5"> + <rect x="0" y="211.5" width="27" height="13.5" class="st6"/> + </g> + <rect x="0" y="211.5" width="27" height="13.5" class="st7"/> + </g> + <g id="shape14-54" v:mID="14" v:groupContext="shape" transform="translate(155.25,-51.5781)"> + <title>Rectangle.78</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow14-55" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5"> + <rect x="0" y="211.5" width="27" height="13.5" class="st6"/> + </g> + <rect x="0" y="211.5" width="27" height="13.5" class="st7"/> + </g> + <g id="shape15-58" v:mID="15" v:groupContext="shape" transform="translate(182.25,-51.5781)"> + <title>Rectangle.79</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow15-59" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5"> + <rect x="0" y="211.5" width="27" height="13.5" class="st6"/> + </g> + <rect x="0" y="211.5" width="27" height="13.5" class="st7"/> + </g> + <g id="shape16-62" v:mID="16" v:groupContext="shape" transform="translate(301.5,-65.0781)"> + <title>Rectangle.81</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow16-63" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5"> + <rect x="0" y="211.5" width="27" height="13.5" class="st6"/> + </g> + <rect x="0" y="211.5" width="27" height="13.5" class="st7"/> + </g> + <g id="shape17-66" v:mID="17" v:groupContext="shape" transform="translate(328.5,-65.0781)"> + <title>Rectangle.82</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow17-67" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5"> + <rect x="0" y="211.5" width="27" height="13.5" class="st6"/> + </g> + <rect x="0" y="211.5" width="27" height="13.5" class="st7"/> + </g> + <g id="shape18-70" v:mID="18" v:groupContext="shape" transform="translate(355.5,-65.0781)"> + <title>Rectangle.83</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow18-71" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5"> + <rect x="0" y="211.5" width="27" height="13.5" class="st6"/> + </g> + <rect x="0" y="211.5" width="27" height="13.5" class="st7"/> + </g> + <g id="shape19-74" v:mID="19" v:groupContext="shape" transform="translate(301.5,-51.5781)"> + <title>Rectangle.84</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow19-75" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5"> + <rect x="0" y="211.5" width="27" height="13.5" class="st6"/> + </g> + <rect x="0" y="211.5" width="27" height="13.5" class="st7"/> + </g> + <g id="shape20-78" v:mID="20" v:groupContext="shape" transform="translate(328.5,-51.5781)"> + <title>Rectangle.85</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow20-79" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5"> + <rect x="0" y="211.5" width="27" height="13.5" class="st6"/> + </g> + <rect x="0" y="211.5" width="27" height="13.5" class="st7"/> + </g> + <g id="shape21-82" v:mID="21" v:groupContext="shape" transform="translate(355.5,-51.5781)"> + <title>Rectangle.86</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow21-83" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5"> + <rect x="0" y="211.5" width="27" height="13.5" class="st6"/> + </g> + <rect x="0" y="211.5" width="27" height="13.5" class="st7"/> + </g> + <g id="shape22-86" v:mID="22" v:groupContext="shape" transform="translate(447.75,-65.6406)"> + <title>Rectangle.88</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow22-87" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5"> + <rect x="0" y="211.5" width="27" height="13.5" class="st6"/> + </g> + <rect x="0" y="211.5" width="27" height="13.5" class="st7"/> + </g> + <g id="shape23-90" v:mID="23" v:groupContext="shape" transform="translate(474.75,-65.6406)"> + <title>Rectangle.89</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow23-91" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5"> + <rect x="0" y="211.5" width="27" height="13.5" class="st6"/> + </g> + <rect x="0" y="211.5" width="27" height="13.5" class="st7"/> + </g> + <g id="shape24-94" v:mID="24" v:groupContext="shape" transform="translate(501.75,-65.6406)"> + <title>Rectangle.90</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow24-95" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5"> + <rect x="0" y="211.5" width="27" height="13.5" class="st6"/> + </g> + <rect x="0" y="211.5" width="27" height="13.5" class="st7"/> + </g> + <g id="shape25-98" v:mID="25" v:groupContext="shape" transform="translate(447.75,-52.1406)"> + <title>Rectangle.91</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow25-99" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5"> + <rect x="0" y="211.5" width="27" height="13.5" class="st6"/> + </g> + <rect x="0" y="211.5" width="27" height="13.5" class="st7"/> + </g> + <g id="shape26-102" v:mID="26" v:groupContext="shape" transform="translate(474.75,-52.1406)"> + <title>Rectangle.92</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow26-103" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5"> + <rect x="0" y="211.5" width="27" height="13.5" class="st6"/> + </g> + <rect x="0" y="211.5" width="27" height="13.5" class="st7"/> + </g> + <g id="shape27-106" v:mID="27" v:groupContext="shape" transform="translate(501.75,-52.1406)"> + <title>Rectangle.93</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow27-107" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5"> + <rect x="0" y="211.5" width="27" height="13.5" class="st6"/> + </g> + <rect x="0" y="211.5" width="27" height="13.5" class="st7"/> + </g> + <g id="shape28-110" v:mID="28" v:groupContext="shape" transform="translate(213.75,-63.9531)"> + <title>Sheet.28</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 225 L83.25 225" class="st10"/> + </g> + <g id="shape29-113" v:mID="29" v:groupContext="shape" transform="translate(387,-63.9531)"> + <title>Sheet.29</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 225 L54 225" class="st10"/> + </g> + <g id="group31-116" transform="translate(184.5,-113.172)" v:mID="31" v:groupContext="group"> + <title>Sheet.31</title> + <g id="shape32-117" v:mID="32" v:groupContext="shape" transform="translate(225,173.25) rotate(90)"> + <title>Block Arrow</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow32-118" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,1.9728,-0.3456)" class="st5"> + <path d="M0 225 L25.87 225 L51.75 177.75 L25.87 130.5 L0 130.5 L0 225 Z" class="st11"/> + </g> + <path d="M0 225 L25.87 225 L51.75 177.75 L25.87 130.5 L0 130.5 L0 225 Z" class="st12"/> + </g> + <g id="shape33-121" v:mID="33" v:groupContext="shape" transform="translate(2.25,-24.3529)"> + <title>Sheet.33</title> + <desc>h1, h2 .. hk</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="45" cy="215.868" width="90" height="18.2647"/> + <rect x="0" y="206.735" width="90" height="18.2647" class="st8"/> + <text x="17.56" y="219.47" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>h1, h2 .. hk</text> </g> + </g> + <g id="shape34-124" v:mID="34" v:groupContext="shape" transform="translate(307.011,286.73) rotate(152.323)"> + <title>Sheet.34</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 225 L128.85 225" class="st3"/> + </g> + <g id="shape35-129" v:mID="35" v:groupContext="shape" transform="translate(433.272,125.452) rotate(99.7172)"> + <title>Sheet.35</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 225 L58.31 225" class="st3"/> + </g> + <g id="shape36-134" v:mID="36" v:groupContext="shape" transform="translate(407.724,-64.1459) rotate(45)"> + <title>Sheet.36</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 225 L79.16 225" class="st3"/> + </g> + <g id="shape37-139" v:mID="37" v:groupContext="shape" transform="translate(320.441,-127.12) rotate(15.6155)"> + <title>Sheet.37</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 225 L200.75 225" class="st3"/> + </g> + <g id="shape38-144" v:mID="38" v:groupContext="shape" transform="translate(132.75,-75.2588)"> + <title>Sheet.38</title> + <desc>BF-2</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="36" cy="211.5" width="72" height="27"/> + <rect x="0" y="198" width="72" height="27" class="st8"/> + <text x="23.29" y="215.7" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>BF-2</text> </g> + <g id="shape39-147" v:mID="39" v:groupContext="shape" transform="translate(303.75,-70.7588)"> + <title>Sheet.39</title> + <desc>BF-X</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="36" cy="211.5" width="72" height="27"/> + <rect x="0" y="198" width="72" height="27" class="st8"/> + <text x="23.2" y="215.7" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>BF-X</text> </g> + <g id="shape40-150" v:mID="40" v:groupContext="shape" transform="translate(447.75,-75.2588)"> + <title>Sheet.40</title> + <desc>BF-L</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="36" cy="211.5" width="72" height="27"/> + <rect x="0" y="198" width="72" height="27" class="st8"/> + <text x="23.89" y="215.7" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>BF-L</text> </g> + <g id="shape41-153" v:mID="41" v:groupContext="shape" transform="translate(300.375,-117)"> + <title>Sheet.41</title> + <desc>Hashing for lookup/Insertion into a vector of BFs happens once</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="90" cy="202.5" width="180" height="45"/> + <rect x="0" y="180" width="180" height="45" class="st8"/> + <text x="4.6" y="198.9" class="st14" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Hashing for lookup/Insertion into a <tspan + x="23.06" dy="1.2em" class="st15">vector of BFs happens once</tspan></text> </g> + <g id="shape44-157" v:mID="44" v:groupContext="shape" transform="translate(249.698,-151.505) rotate(-3.74012)"> + <title>Sheet.44</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M-0 225 A93.4958 45.6256 42.23 0 1 79.38 221.66 L79.68 221.85" class="st16"/> + </g> + <g id="shape45-163" v:mID="45" v:groupContext="shape" transform="translate(30.375,0.25)"> + <title>Sheet.45</title> + <desc>Lookup/Insertion is done in the series of BFs, one by one or ...</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="233.048" cy="202.5" width="466.1" height="45"/> + <rect x="0" y="180" width="466.096" height="45" class="st8"/> + <text x="4.34" y="206.1" class="st14" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Lookup/Insertion is done in the series of BFs, one by one or can be optimized to do in parallel. </text> </g> + <g id="shape46-166" v:mID="46" v:groupContext="shape" transform="translate(123.252,-43.6868) rotate(17.0249)"> + <title>Sheet.46</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M-0 225 A88.2185 43.0621 47.63 0 1 70.31 221.39 L70.6 221.6" class="st16"/> + </g> + </g> + </g> +</svg> diff --git a/doc/guides/prog_guide/img/member_i5.svg b/doc/guides/prog_guide/img/member_i5.svg new file mode 100644 index 00000000..c1728cf2 --- /dev/null +++ b/doc/guides/prog_guide/img/member_i5.svg @@ -0,0 +1,163 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.0//EN" "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd"> +<!-- Generated by Microsoft Visio 11.0, SVG Export, v1.0 memship_i5.svg Page-1 --> +<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events" + xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" width="5.30481in" height="1.96146in" + viewBox="0 0 381.946 141.225" xml:space="preserve" color-interpolation-filters="sRGB" class="st15"> + <v:documentProperties v:langID="1033" v:viewMarkup="false"/> + + <style type="text/css"> + <![CDATA[ + .st1 {visibility:visible} + .st2 {fill:#5b9bd5;fill-opacity:0.22;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:0.25} + .st3 {fill:#5b9bd5;stroke:#c8c8c8;stroke-width:0.25} + .st4 {fill:none;stroke:none;stroke-width:0.25} + .st5 {fill:#ffffff;font-family:Calibri;font-size:1.16666em;font-weight:bold} + .st6 {marker-end:url(#mrkr5-14);stroke:#5b9bd5;stroke-linecap:round;stroke-linejoin:round;stroke-width:1} + .st7 {fill:#5b9bd5;fill-opacity:1;stroke:#5b9bd5;stroke-opacity:1;stroke-width:0.28409090909091} + .st8 {fill:#5b9bd5;font-family:Calibri;font-size:0.833336em} + .st9 {fill:#5b9bd5;font-family:Calibri;font-size:0.75em} + .st10 {stroke:#5b9bd5;stroke-linecap:round;stroke-linejoin:round;stroke-width:1} + .st11 {marker-end:url(#mrkr5-63);stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75} + .st12 {fill:#000000;fill-opacity:1;stroke:#000000;stroke-opacity:1;stroke-width:0.22935779816514} + .st13 {fill:#000000;font-family:Calibri;font-size:1.00001em} + .st14 {font-size:1em} + .st15 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3} + ]]> + </style> + + <defs id="Markers"> + <g id="lend5"> + <path d="M 2 1 L 0 0 L 1.98117 -0.993387 C 1.67173 -0.364515 1.67301 0.372641 1.98465 1.00043 " style="stroke:none"/> + </g> + <marker id="mrkr5-14" class="st7" v:arrowType="5" v:arrowSize="2" v:setback="6.16" refX="-6.16" orient="auto" + markerUnits="strokeWidth" overflow="visible"> + <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/> + </marker> + <marker id="mrkr5-63" class="st12" v:arrowType="5" v:arrowSize="2" v:setback="7.15" refX="-7.15" orient="auto" + markerUnits="strokeWidth" overflow="visible"> + <use xlink:href="#lend5" transform="scale(-4.36,-4.36) "/> + </marker> + </defs> + <g v:mID="0" v:index="1" v:groupContext="foregroundPage"> + <title>Page-1</title> + <v:pageProperties v:drawingScale="1" v:pageScale="1" v:drawingUnits="0" v:shadowOffsetX="9" v:shadowOffsetY="-9"/> + <g id="group1-1" transform="translate(191.995,-19.4751)" v:mID="1" v:groupContext="group"> + <title>Sheet.1</title> + <g id="shape2-2" v:mID="2" v:groupContext="shape" transform="translate(146.944,42.2251) rotate(90)"> + <title>Triangle</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow2-3" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,1.9728,-0.3456)" class="st1"> + <path d="M99 141.23 L49.5 58.62 L0 141.23 L99 141.23 Z" class="st2"/> + </g> + <path d="M99 141.23 L49.5 58.62 L0 141.23 L99 141.23 Z" class="st3"/> + </g> + <g id="shape3-6" v:mID="3" v:groupContext="shape" transform="translate(0,-34.65)"> + <title>Sheet.3</title> + <desc>vBF</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="38.1251" cy="126.375" width="76.26" height="29.7"/> + <rect x="0" y="111.525" width="76.2502" height="29.7" class="st4"/> + <text x="27.68" y="130.58" class="st5" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>vBF </text> </g> + </g> + <g id="shape4-9" v:mID="4" v:groupContext="shape" transform="translate(126.724,-100.475)"> + <title>Sheet.4</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 141.23 L64.83 141.23" class="st6"/> + </g> + <g id="shape5-15" v:mID="5" v:groupContext="shape" transform="translate(103.5,-101.775)"> + <title>Sheet.5</title> + <desc>Flow Key</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="39.9122" cy="135.863" width="79.83" height="10.7251"/> + <rect x="0" y="130.5" width="79.8244" height="10.7251" class="st4"/> + <text x="21.78" y="138.86" class="st8" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Key</text> </g> + <g id="shape6-18" v:mID="6" v:groupContext="shape" transform="translate(221.726,-56.2468) rotate(-24.5123)"> + <title>Sheet.6</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 141.23 L65.42 141.23" class="st6"/> + </g> + <g id="shape7-23" v:mID="7" v:groupContext="shape" transform="translate(280.318,-68.9751)"> + <title>Sheet.7</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 141.23 L64.83 141.23" class="st6"/> + </g> + <g id="shape8-28" v:mID="8" v:groupContext="shape" transform="translate(338.125,-56.6022) rotate(24.1625)"> + <title>Sheet.8</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 141.23 L70.8 141.23" class="st6"/> + </g> + <g id="shape9-33" v:mID="9" v:groupContext="shape" transform="translate(197.714,217.975) rotate(180)"> + <title>Sheet.9</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 141.23 L51.03 141.23" class="st6"/> + </g> + <g id="shape10-38" v:mID="10" v:groupContext="shape" transform="translate(18,-67.5)"> + <title>Sheet.10</title> + <desc>New Flow => New Assignment</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="80.4201" cy="134.475" width="160.85" height="13.5"/> + <rect x="0" y="127.725" width="160.84" height="13.5" class="st4"/> + <text x="25.11" y="137.18" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>New Flow => New Assignment</text> </g> + <g id="shape11-41" v:mID="11" v:groupContext="shape" transform="translate(198.032,253.975) rotate(180)"> + <title>Sheet.11</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 141.23 L51.03 141.23" class="st6"/> + </g> + <g id="shape12-46" v:mID="12" v:groupContext="shape" transform="translate(0,-31.5)"> + <title>Sheet.12</title> + <desc>Old Flow => forward to specific thread</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="81" cy="136.725" width="162.01" height="9"/> + <rect x="0" y="132.225" width="162" height="9" class="st4"/> + <text x="11.04" y="139.43" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Old Flow => forward to specific thread</text> </g> + <g id="shape13-49" v:mID="13" v:groupContext="shape" transform="translate(494.552,22.75) rotate(90)"> + <title>Sheet.13</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 134.49 C3.18 142.89 7.57 142.28 11.25 138.99 C15.79 134.93 19.26 126.78 27 134.49" class="st10"/> + </g> + <g id="shape14-52" v:mID="14" v:groupContext="shape" transform="translate(494.552,58.75) rotate(90)"> + <title>Sheet.14</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 134.49 C3.18 142.89 7.57 142.28 11.25 138.99 C15.79 134.93 19.26 126.78 27 134.49" class="st10"/> + </g> + <g id="shape15-55" v:mID="15" v:groupContext="shape" transform="translate(494.552,94.75) rotate(90)"> + <title>Sheet.15</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 134.49 C3.18 142.89 7.57 142.28 11.25 138.99 C15.79 134.93 19.26 126.78 27 134.49" class="st10"/> + </g> + <g id="shape17-58" v:mID="17" v:groupContext="shape" transform="translate(348.769,-25.0593) rotate(44.5185)"> + <title>Sheet.17</title> + <path d="M-0 141.23 A35.1884 19.2595 167.75 0 1 42.43 138.27 L42.74 138.46" class="st11"/> + </g> + <g id="shape18-64" v:mID="18" v:groupContext="shape" transform="translate(222.188,-5.40005)"> + <title>Sheet.18</title> + <desc>A BF corresponding to each worker thread</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="59.0625" cy="127.725" width="118.13" height="27"/> + <rect x="0" y="114.225" width="118.125" height="27" class="st4"/> + <text x="5.14" y="124.13" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>A BF corresponding to <tspan + x="11.19" dy="1.2em" class="st14">each worker thread</tspan></text> </g> + </g> +</svg> diff --git a/doc/guides/prog_guide/img/member_i6.svg b/doc/guides/prog_guide/img/member_i6.svg new file mode 100644 index 00000000..265179f6 --- /dev/null +++ b/doc/guides/prog_guide/img/member_i6.svg @@ -0,0 +1,332 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.0//EN" "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd"> +<!-- Generated by Microsoft Visio 11.0, SVG Export, v1.0 memship_i6.svg Page-1 --> +<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events" + xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" width="8in" height="3.625in" viewBox="0 0 576 261" + xml:space="preserve" color-interpolation-filters="sRGB" class="st16"> + <v:documentProperties v:langID="1033" v:viewMarkup="false"/> + + <style type="text/css"> + <![CDATA[ + .st1 {visibility:visible} + .st2 {fill:#5b9bd5;fill-opacity:0.22;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:0.25} + .st3 {fill:#5b9bd5;stroke:#c8c8c8;stroke-width:0.25} + .st4 {fill:#feffff;font-family:Calibri;font-size:0.666664em;font-weight:bold} + .st5 {font-size:1em} + .st6 {fill:#70ad47;fill-opacity:0.5;stroke:#00b050;stroke-width:1.5} + .st7 {fill:none;stroke:none;stroke-width:0.25} + .st8 {fill:#00b050;font-family:Calibri;font-size:1.00001em} + .st9 {fill:none;stroke:#ff0000;stroke-width:0.25} + .st10 {fill:#5b9bd5;font-family:Calibri;font-size:0.833336em} + .st11 {marker-end:url(#mrkr5-29);stroke:#ff0000;stroke-linecap:round;stroke-linejoin:round;stroke-width:2.25} + .st12 {fill:#ff0000;fill-opacity:1;stroke:#ff0000;stroke-opacity:1;stroke-width:0.47169811320755} + .st13 {fill:#5b9bd5;font-family:Calibri;font-size:0.75em} + .st14 {fill:#92d050;stroke:#c8c8c8;stroke-width:0.25} + .st15 {fill:#5b9bd5;font-family:Calibri;font-size:1.00001em;font-weight:bold} + .st16 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3} + ]]> + </style> + + <defs id="Markers"> + <g id="lend5"> + <path d="M 2 1 L 0 0 L 1.98117 -0.993387 C 1.67173 -0.364515 1.67301 0.372641 1.98465 1.00043 " style="stroke:none"/> + </g> + <marker id="mrkr5-29" class="st12" v:arrowType="5" v:arrowSize="2" v:setback="3.71" refX="-3.71" orient="auto" + markerUnits="strokeWidth" overflow="visible"> + <use xlink:href="#lend5" transform="scale(-2.12,-2.12) "/> + </marker> + </defs> + <g v:mID="0" v:index="1" v:groupContext="foregroundPage"> + <title>Page-1</title> + <v:pageProperties v:drawingScale="1" v:pageScale="1" v:drawingUnits="0" v:shadowOffsetX="9" v:shadowOffsetY="-9"/> + <g id="group121-1" transform="translate(21.0294,-9.8478)" v:mID="121" v:groupContext="group"> + <title>Sheet.121</title> + <g id="shape49-2" v:mID="49" v:groupContext="shape" transform="translate(396.989,-54.9268)"> + <title>Rectangle.2</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow49-3" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="219.549" width="99.4817" height="41.4507" class="st2"/> + </g> + <rect x="0" y="219.549" width="99.4817" height="41.4507" class="st3"/> + </g> + <g id="shape50-6" v:mID="50" v:groupContext="shape" transform="translate(248.261,-12.1936)"> + <title>Rectangle.4</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow50-7" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="178.099" width="99.4817" height="82.9014" class="st2"/> + </g> + <rect x="0" y="178.099" width="99.4817" height="82.9014" class="st3"/> + </g> + <g id="shape52-10" v:mID="52" v:groupContext="shape" transform="translate(6.07514E-013,-29.0155)"> + <title>Rectangle.10</title> + <desc>Signatures for target 1</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="49.7409" cy="225.767" width="99.49" height="70.4662"/> + <g id="shadow52-11" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="190.534" width="99.4817" height="70.4662" class="st2"/> + </g> + <rect x="0" y="190.534" width="99.4817" height="70.4662" class="st3"/> + <text x="26.54" y="223.37" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Signatures for<v:newlineChar/><tspan + x="36.73" dy="1.2em" class="st5">target </tspan>1</text> </g> + <g id="shape53-16" v:mID="53" v:groupContext="shape" transform="translate(239.971,-20.4837)"> + <title>Sheet.53</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <rect x="0" y="248.565" width="116.062" height="12.4352" class="st6"/> + </g> + <g id="shape54-18" v:mID="54" v:groupContext="shape" transform="translate(353.649,-19.9346)"> + <title>Sheet.54</title> + <desc>Match 1</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="33.1606" cy="254.782" width="66.33" height="12.4352"/> + <rect x="0" y="248.565" width="66.3211" height="12.4352" class="st7"/> + <text x="13.06" y="258.38" class="st8" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Match 1</text> </g> + <g id="shape55-21" v:mID="55" v:groupContext="shape" transform="translate(216.989,-210.652)"> + <title>Sheet.55</title> + <desc>Packet Payload</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="49.7409" cy="252.71" width="99.49" height="16.5803"/> + <rect x="0" y="244.42" width="99.4817" height="16.5803" class="st9"/> + <text x="19.04" y="255.71" class="st10" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Packet Payload</text> </g> + <g id="shape56-24" v:mID="56" v:groupContext="shape" transform="translate(526.665,52.2365) rotate(90)"> + <title>Sheet.56</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 261 L16.52 261" class="st11"/> + </g> + <g id="shape96-30" v:mID="96" v:groupContext="shape" transform="translate(-3.0294,-95.7818)"> + <title>Sheet.96</title> + <desc>Attack Signature Length 1</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="51.75" cy="248.565" width="103.5" height="24.8704"/> + <rect x="0" y="236.13" width="103.5" height="24.8704" class="st7"/> + <text x="4.79" y="251.26" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Attack Signature Length 1</text> </g> + <g id="group114-33" transform="translate(228.359,-134.152)" v:mID="114" v:groupContext="group"> + <title>Sheet.114</title> + <g id="group106-34" transform="translate(0,-24.8704)" v:mID="106" v:groupContext="group"> + <title>Sheet.106</title> + <g id="shape100-35" v:mID="100" v:groupContext="shape" transform="translate(3.65707E-013,-12.4352)"> + <title>Rectangle.100</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow100-36" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/> + </g> + <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/> + </g> + <g id="shape101-39" v:mID="101" v:groupContext="shape" transform="translate(24.8704,-12.4352)"> + <title>Rectangle.101</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow101-40" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/> + </g> + <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/> + </g> + <g id="shape102-43" v:mID="102" v:groupContext="shape" transform="translate(49.7409,-12.4352)"> + <title>Rectangle.102</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow102-44" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/> + </g> + <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/> + </g> + <g id="shape103-47" v:mID="103" v:groupContext="shape"> + <title>Rectangle.103</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow103-48" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/> + </g> + <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/> + </g> + <g id="shape104-51" v:mID="104" v:groupContext="shape" transform="translate(24.8704,1.13687E-013)"> + <title>Rectangle.104</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow104-52" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/> + </g> + <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/> + </g> + <g id="shape105-55" v:mID="105" v:groupContext="shape" transform="translate(49.7409,1.13687E-013)"> + <title>Rectangle.105</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow105-56" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/> + </g> + <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/> + </g> + </g> + <g id="group107-59" v:mID="107" v:groupContext="group"> + <title>Sheet.107</title> + <g id="shape108-60" v:mID="108" v:groupContext="shape" transform="translate(3.65707E-013,-12.4352)"> + <title>Rectangle.100</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow108-61" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/> + </g> + <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/> + </g> + <g id="shape109-64" v:mID="109" v:groupContext="shape" transform="translate(24.8704,-12.4352)"> + <title>Rectangle.101</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow109-65" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/> + </g> + <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/> + </g> + <g id="shape110-68" v:mID="110" v:groupContext="shape" transform="translate(49.7409,-12.4352)"> + <title>Rectangle.102</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow110-69" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/> + </g> + <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/> + </g> + <g id="shape111-72" v:mID="111" v:groupContext="shape"> + <title>Rectangle.103</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <g id="shadow111-73" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/> + </g> + <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st14"/> + </g> + <g id="shape112-76" v:mID="112" v:groupContext="shape" transform="translate(24.8704,1.13687E-013)"> + <title>Rectangle.104</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow112-77" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/> + </g> + <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/> + </g> + <g id="shape113-80" v:mID="113" v:groupContext="shape" transform="translate(49.7409,1.13687E-013)"> + <title>Rectangle.105</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <g id="shadow113-81" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/> + </g> + <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st14"/> + </g> + </g> + </g> + <g id="shape89-84" v:mID="89" v:groupContext="shape" transform="translate(398.644,-116.927) rotate(24.4696)"> + <title>Sheet.89</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 261 L143.75 261" class="st11"/> + </g> + <g id="shape115-89" v:mID="115" v:groupContext="shape" transform="translate(116.062,-1.19371E-012)"> + <title>Rectangle.115</title> + <desc>Signatures for target 2</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="49.7409" cy="211.259" width="99.49" height="99.4817"/> + <g id="shadow115-90" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="161.518" width="99.4817" height="99.4817" class="st2"/> + </g> + <rect x="0" y="161.518" width="99.4817" height="99.4817" class="st3"/> + <text x="26.54" y="208.86" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Signatures for<v:newlineChar/><tspan + x="36.73" dy="1.2em" class="st5">target </tspan>2</text> </g> + <g id="shape116-95" v:mID="116" v:groupContext="shape" transform="translate(117.989,-95.7818)"> + <title>Sheet.116</title> + <desc>Attack Signature Length 2</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="51.9909" cy="248.565" width="103.99" height="24.8704"/> + <rect x="0" y="236.13" width="103.982" height="24.8704" class="st7"/> + <text x="5.03" y="251.26" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Attack Signature Length 2</text> </g> + <g id="shape118-98" v:mID="118" v:groupContext="shape" transform="translate(392.971,-90.217)"> + <title>Sheet.118</title> + <desc>Attack Signature Length L</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="54" cy="248.565" width="108" height="24.8704"/> + <rect x="0" y="236.13" width="108" height="24.8704" class="st7"/> + <text x="7.43" y="251.26" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Attack Signature Length L</text> </g> + <g id="shape119-101" v:mID="119" v:groupContext="shape" transform="translate(384.909,-64.9346)"> + <title>Sheet.119</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <rect x="0" y="248.565" width="116.062" height="12.4352" class="st6"/> + </g> + <g id="shape120-103" v:mID="120" v:groupContext="shape" transform="translate(491.971,-64.9346)"> + <title>Sheet.120</title> + <desc>Match 2</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="33.1606" cy="254.782" width="66.33" height="12.4352"/> + <rect x="0" y="248.565" width="66.3211" height="12.4352" class="st7"/> + <text x="13.06" y="258.38" class="st8" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Match 2</text> </g> + <g id="shape85-106" v:mID="85" v:groupContext="shape" transform="translate(478.538,12.9307) rotate(65.6291)"> + <title>Sheet.85</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 261 L109.61 261" class="st11"/> + </g> + <g id="shape117-111" v:mID="117" v:groupContext="shape" transform="translate(247.054,-91.2818)"> + <title>Sheet.117</title> + <desc>Attack Signature Length X</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="52.7082" cy="248.565" width="105.42" height="24.8704"/> + <rect x="0" y="236.13" width="105.416" height="24.8704" class="st7"/> + <text x="5.7" y="251.26" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Attack Signature Length X</text> </g> + </g> + <g id="shape122-114" v:mID="122" v:groupContext="shape" transform="translate(315.114,-164.13)"> + <title>Sheet.122</title> + <desc>HTSS</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="26.943" cy="248.565" width="53.89" height="24.8704"/> + <rect x="0" y="236.13" width="53.8859" height="24.8704" class="st7"/> + <text x="14.52" y="252.16" class="st15" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>HTSS</text> </g> + </g> +</svg> diff --git a/doc/guides/prog_guide/img/member_i7.svg b/doc/guides/prog_guide/img/member_i7.svg new file mode 100644 index 00000000..e23ae26c --- /dev/null +++ b/doc/guides/prog_guide/img/member_i7.svg @@ -0,0 +1,399 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.0//EN" "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd"> +<!-- Generated by Microsoft Visio 11.0, SVG Export, v1.0 memship_i7.svg Page-1 --> +<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events" + xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" width="8.5in" height="4.5in" viewBox="0 0 612 324" + xml:space="preserve" color-interpolation-filters="sRGB" class="st23"> + <v:documentProperties v:langID="1033" v:viewMarkup="false"/> + + <style type="text/css"> + <![CDATA[ + .st1 {visibility:visible} + .st2 {fill:#5b9bd5;fill-opacity:0.22;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:0.25} + .st3 {fill:#5b9bd5;stroke:#c8c8c8;stroke-width:0.25} + .st4 {fill:#feffff;font-family:Calibri;font-size:0.833336em;font-weight:bold} + .st5 {font-size:1em} + .st6 {fill:#70ad47;fill-opacity:0.5;stroke:#00b050;stroke-width:1.5} + .st7 {fill:none;stroke:none;stroke-width:0.25} + .st8 {fill:#00b050;font-family:Calibri;font-size:1.16666em} + .st9 {fill:none;stroke:#00b050;stroke-width:2.25} + .st10 {fill:#5b9bd5;font-family:Calibri;font-size:0.833336em} + .st11 {fill:#5b9bd5;font-family:Calibri;font-size:1.16666em} + .st12 {fill:#a8d08d;stroke:#c8c8c8;stroke-width:0.25} + .st13 {marker-end:url(#mrkr5-83);stroke:#ff0000;stroke-linecap:round;stroke-linejoin:round;stroke-width:2.25} + .st14 {fill:#ff0000;fill-opacity:1;stroke:#ff0000;stroke-opacity:1;stroke-width:0.47169811320755} + .st15 {marker-end:url(#mrkr5-95);stroke:#92d050;stroke-linecap:round;stroke-linejoin:round;stroke-width:2.25} + .st16 {fill:#92d050;fill-opacity:1;stroke:#92d050;stroke-opacity:1;stroke-width:0.47169811320755} + .st17 {fill:#00b050;font-family:Calibri;font-size:1.00001em;font-weight:bold} + .st18 {fill:#5b9bd5;font-family:Calibri;font-size:1.00001em} + .st19 {fill:none;stroke:#ff0000;stroke-width:2.25} + .st20 {fill:#ff0000;font-family:Calibri;font-size:1.00001em;font-weight:bold} + .st21 {marker-end:url(#mrkr5-123);stroke:#ff0000;stroke-linecap:round;stroke-linejoin:round;stroke-width:1} + .st22 {fill:#ff0000;fill-opacity:1;stroke:#ff0000;stroke-opacity:1;stroke-width:0.28409090909091} + .st23 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3} + ]]> + </style> + + <defs id="Markers"> + <g id="lend5"> + <path d="M 2 1 L 0 0 L 1.98117 -0.993387 C 1.67173 -0.364515 1.67301 0.372641 1.98465 1.00043 " style="stroke:none"/> + </g> + <marker id="mrkr5-83" class="st14" v:arrowType="5" v:arrowSize="2" v:setback="3.71" refX="-3.71" orient="auto" + markerUnits="strokeWidth" overflow="visible"> + <use xlink:href="#lend5" transform="scale(-2.12,-2.12) "/> + </marker> + <marker id="mrkr5-95" class="st16" v:arrowType="5" v:arrowSize="2" v:setback="3.71" refX="-3.71" orient="auto" + markerUnits="strokeWidth" overflow="visible"> + <use xlink:href="#lend5" transform="scale(-2.12,-2.12) "/> + </marker> + <marker id="mrkr5-123" class="st22" v:arrowType="5" v:arrowSize="2" v:setback="5.8" refX="-5.8" orient="auto" + markerUnits="strokeWidth" overflow="visible"> + <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/> + </marker> + </defs> + <g v:mID="0" v:index="1" v:groupContext="foregroundPage"> + <title>Page-1</title> + <v:pageProperties v:drawingScale="1" v:pageScale="1" v:drawingUnits="0" v:shadowOffsetX="9" v:shadowOffsetY="-9"/> + <g id="group121-1" transform="translate(21.0294,-32.2733)" v:mID="121" v:groupContext="group"> + <title>Sheet.121</title> + <g id="shape49-2" v:mID="49" v:groupContext="shape" transform="translate(460.471,-62.2267)"> + <title>Rectangle.2</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow49-3" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="279" width="108" height="45" class="st2"/> + </g> + <rect x="0" y="279" width="108" height="45" class="st3"/> + </g> + <g id="shape50-6" v:mID="50" v:groupContext="shape" transform="translate(320.452,-18.123)"> + <title>Rectangle.4</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow50-7" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="234" width="108" height="90" class="st2"/> + </g> + <rect x="0" y="234" width="108" height="90" class="st3"/> + </g> + <g id="shape52-10" v:mID="52" v:groupContext="shape" transform="translate(0,-31.5)"> + <title>Rectangle.10</title> + <desc>Flow Keys Matching Mask 1</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="54" cy="285.75" width="108" height="76.5"/> + <g id="shadow52-11" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="247.5" width="108" height="76.5" class="st2"/> + </g> + <rect x="0" y="247.5" width="108" height="76.5" class="st3"/> + <text x="12.56" y="282.75" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Keys Matching <tspan + x="39.1" dy="1.2em" class="st5">Mask </tspan>1</text> </g> + <g id="shape53-16" v:mID="53" v:groupContext="shape" transform="translate(311.452,-27.123)"> + <title>Sheet.53</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <rect x="0" y="310.5" width="126" height="13.5" class="st6"/> + </g> + <g id="shape54-18" v:mID="54" v:groupContext="shape" transform="translate(424.471,-26.2267)"> + <title>Sheet.54</title> + <desc>Match</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="36" cy="317.25" width="72" height="13.5"/> + <rect x="0" y="310.5" width="72" height="13.5" class="st7"/> + <text x="17.68" y="321.45" class="st8" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Match</text> </g> + <g id="shape55-21" v:mID="55" v:groupContext="shape" transform="translate(261,-247.163)"> + <title>Sheet.55</title> + <desc>Flow ID1</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="27.1728" cy="315" width="54.35" height="18"/> + <rect x="0" y="306" width="54.3456" height="18" class="st9"/> + <text x="9.52" y="318" class="st10" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow ID1</text> </g> + <g id="shape96-24" v:mID="96" v:groupContext="shape" transform="translate(0,-109.783)"> + <title>Sheet.96</title> + <desc>Flow Mask 1</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="54" cy="319.5" width="108" height="9"/> + <rect x="0" y="315" width="108" height="9" class="st7"/> + <text x="18.51" y="323.7" class="st11" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Mask 1</text> </g> + <g id="group114-27" transform="translate(247.5,-163.783)" v:mID="114" v:groupContext="group"> + <title>Sheet.114</title> + <g id="group106-28" transform="translate(0,-27)" v:mID="106" v:groupContext="group"> + <title>Sheet.106</title> + <g id="shape100-29" v:mID="100" v:groupContext="shape" transform="translate(0,-13.5)"> + <title>Rectangle.100</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow100-30" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="310.5" width="27" height="13.5" class="st2"/> + </g> + <rect x="0" y="310.5" width="27" height="13.5" class="st3"/> + </g> + <g id="shape101-33" v:mID="101" v:groupContext="shape" transform="translate(27,-13.5)"> + <title>Rectangle.101</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow101-34" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="310.5" width="27" height="13.5" class="st2"/> + </g> + <rect x="0" y="310.5" width="27" height="13.5" class="st3"/> + </g> + <g id="shape102-37" v:mID="102" v:groupContext="shape" transform="translate(54,-13.5)"> + <title>Rectangle.102</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow102-38" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="310.5" width="27" height="13.5" class="st2"/> + </g> + <rect x="0" y="310.5" width="27" height="13.5" class="st3"/> + </g> + <g id="shape103-41" v:mID="103" v:groupContext="shape"> + <title>Rectangle.103</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow103-42" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="310.5" width="27" height="13.5" class="st2"/> + </g> + <rect x="0" y="310.5" width="27" height="13.5" class="st3"/> + </g> + <g id="shape104-45" v:mID="104" v:groupContext="shape" transform="translate(27,0)"> + <title>Rectangle.104</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow104-46" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="310.5" width="27" height="13.5" class="st2"/> + </g> + <rect x="0" y="310.5" width="27" height="13.5" class="st3"/> + </g> + <g id="shape105-49" v:mID="105" v:groupContext="shape" transform="translate(54,0)"> + <title>Rectangle.105</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow105-50" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="310.5" width="27" height="13.5" class="st2"/> + </g> + <rect x="0" y="310.5" width="27" height="13.5" class="st3"/> + </g> + </g> + <g id="group107-53" v:mID="107" v:groupContext="group"> + <title>Sheet.107</title> + <g id="shape108-54" v:mID="108" v:groupContext="shape" transform="translate(0,-13.5)"> + <title>Rectangle.100</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow108-55" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="310.5" width="27" height="13.5" class="st2"/> + </g> + <rect x="0" y="310.5" width="27" height="13.5" class="st3"/> + </g> + <g id="shape109-58" v:mID="109" v:groupContext="shape" transform="translate(27,-13.5)"> + <title>Rectangle.101</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow109-59" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="310.5" width="27" height="13.5" class="st2"/> + </g> + <rect x="0" y="310.5" width="27" height="13.5" class="st3"/> + </g> + <g id="shape110-62" v:mID="110" v:groupContext="shape" transform="translate(54,-13.5)"> + <title>Rectangle.102</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <g id="shadow110-63" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="310.5" width="27" height="13.5" class="st2"/> + </g> + <rect x="0" y="310.5" width="27" height="13.5" class="st12"/> + </g> + <g id="shape111-66" v:mID="111" v:groupContext="shape"> + <title>Rectangle.103</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <g id="shadow111-67" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="310.5" width="27" height="13.5" class="st2"/> + </g> + <rect x="0" y="310.5" width="27" height="13.5" class="st3"/> + </g> + <g id="shape112-70" v:mID="112" v:groupContext="shape" transform="translate(27,0)"> + <title>Rectangle.104</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <g id="shadow112-71" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="310.5" width="27" height="13.5" class="st2"/> + </g> + <rect x="0" y="310.5" width="27" height="13.5" class="st3"/> + </g> + <g id="shape113-74" v:mID="113" v:groupContext="shape" transform="translate(54,0)"> + <title>Rectangle.105</title> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <g id="shadow113-75" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" + v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="310.5" width="27" height="13.5" class="st2"/> + </g> + <rect x="0" y="310.5" width="27" height="13.5" class="st3"/> + </g> + </g> + </g> + <g id="shape89-78" v:mID="89" v:groupContext="shape" transform="translate(413.723,393.802) rotate(146.31)"> + <title>Sheet.89</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 324 L153.9 324" class="st13"/> + </g> + <g id="shape115-84" v:mID="115" v:groupContext="shape" transform="translate(126,0)"> + <title>Rectangle.115</title> + <desc>Flow Keys Matching Mask 2</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="54" cy="270" width="108" height="108"/> + <g id="shadow115-85" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1" + transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1"> + <rect x="0" y="216" width="108" height="108" class="st2"/> + </g> + <rect x="0" y="216" width="108" height="108" class="st3"/> + <text x="12.56" y="267" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Keys Matching <tspan + x="39.1" dy="1.2em" class="st5">Mask </tspan>2</text> </g> + <g id="shape85-90" v:mID="85" v:groupContext="shape" transform="translate(635.321,91.2793) rotate(81.3573)"> + <title>Sheet.85</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 324 L143.93 324" class="st15"/> + </g> + <g id="shape56-96" v:mID="56" v:groupContext="shape" transform="translate(579.175,-64.556) rotate(64.1257)"> + <title>Sheet.56</title> + <path d="M0 324 L54.31 324" class="st15"/> + </g> + </g> + <g id="shape122-101" v:mID="122" v:groupContext="shape" transform="translate(351,-213.444)"> + <title>Sheet.122</title> + <desc>HTSS with False Negative (Cache)</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="45" cy="304.722" width="90" height="38.556"/> + <rect x="0" y="285.444" width="90" height="38.556" class="st7"/> + <text x="13.29" y="301.72" class="st10" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>HTSS with False <tspan + x="10.52" dy="1.2em" class="st5">Negative </tspan>(Cache)</text> </g> + <g id="shape123-105" v:mID="123" v:groupContext="shape" transform="translate(287.654,-290.556)"> + <title>Sheet.123</title> + <desc>Active</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="24.1875" cy="310.5" width="48.38" height="27"/> + <rect x="0" y="297" width="48.375" height="27" class="st7"/> + <text x="8.63" y="314.1" class="st17" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Active</text> </g> + <g id="shape124-108" v:mID="124" v:groupContext="shape" transform="translate(278.827,-153)"> + <title>Sheet.124</title> + <desc>Target for Flow ID 1</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="36.0864" cy="310.5" width="72.18" height="27"/> + <rect x="0" y="297" width="72.1728" height="27" class="st9"/> + <text x="11.93" y="306.9" class="st18" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Target for <tspan + x="13.54" dy="1.2em" class="st5">Flow ID </tspan>1</text> </g> + <g id="shape125-112" v:mID="125" v:groupContext="shape" transform="translate(155.857,-254.556)"> + <title>Sheet.125</title> + <desc>Flow ID2</desc> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="27.1728" cy="315" width="54.35" height="18"/> + <rect x="0" y="306" width="54.3456" height="18" class="st19"/> + <text x="9.52" y="318" class="st10" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow ID2</text> </g> + <g id="shape126-115" v:mID="126" v:groupContext="shape" transform="translate(153,-270)"> + <title>Sheet.126</title> + <desc>New/Inactive</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="40.5" cy="310.5" width="81" height="27"/> + <rect x="0" y="297" width="81" height="27" class="st7"/> + <text x="6.77" y="314.1" class="st20" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>New/Inactive</text> </g> + <g id="shape127-118" v:mID="127" v:groupContext="shape" transform="translate(251.739,-239.709) rotate(14.0795)"> + <title>Sheet.127</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 318.73 A39.2404 18 -180 0 0 49.73 320.91 L50.07 320.78" class="st21"/> + </g> + <g id="shape128-124" v:mID="128" v:groupContext="shape" transform="translate(219.24,-229.5)"> + <title>Sheet.128</title> + <desc>Miss</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="20.88" cy="310.5" width="41.76" height="27"/> + <rect x="0" y="297" width="41.76" height="27" class="st7"/> + <text x="7.81" y="314.7" class="st11" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Miss</text> </g> + <g id="shape129-127" v:mID="129" v:groupContext="shape" transform="translate(147.029,-142.056)"> + <title>Sheet.129</title> + <desc>Flow Mask 2</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="54" cy="319.5" width="108" height="9"/> + <rect x="0" y="315" width="108" height="9" class="st7"/> + <text x="18.51" y="323.7" class="st11" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Mask 2</text> </g> + <g id="shape130-130" v:mID="130" v:groupContext="shape" transform="translate(166.845,-18.5004) rotate(18.2325)"> + <title>Sheet.130</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 293.46 A71.1913 104.269 -180 0 0 97.04 298.43 L97.25 298.14" class="st21"/> + </g> + <g id="shape131-135" v:mID="131" v:groupContext="shape" transform="translate(184.406,-3.04505) rotate(-3.24734)"> + <title>Sheet.131</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 293.46 A112.345 104.269 -180 0 0 154.25 297.52 L154.52 297.28" class="st21"/> + </g> + <g id="shape132-140" v:mID="132" v:groupContext="shape" transform="translate(301.368,16.888) rotate(-25.868)"> + <title>Sheet.132</title> + <v:userDefs> + <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/> + </v:userDefs> + <path d="M0 293.46 A83.375 104.269 -180 0 0 113.91 298.14 L114.14 297.87" class="st21"/> + </g> + <g id="shape133-145" v:mID="133" v:groupContext="shape" transform="translate(345.029,-142.056)"> + <title>Sheet.133</title> + <desc>Flow Mask X</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="54" cy="319.5" width="108" height="9"/> + <rect x="0" y="315" width="108" height="9" class="st7"/> + <text x="18.43" y="323.7" class="st11" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Mask X</text> </g> + <g id="shape134-148" v:mID="134" v:groupContext="shape" transform="translate(481.5,-139.5)"> + <title>Sheet.134</title> + <desc>Flow Mask L</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="54" cy="319.5" width="108" height="9"/> + <rect x="0" y="315" width="108" height="9" class="st7"/> + <text x="19.12" y="323.7" class="st11" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Mask L</text> </g> + </g> +</svg> diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index 40f04a10..c4beb346 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -44,23 +44,29 @@ Programmer's Guide mbuf_lib poll_mode_drv rte_flow + traffic_metering_and_policing traffic_management cryptodev_lib + rte_security link_bonding_poll_mode_drv_lib timer_lib hash_lib efd_lib + member_lib lpm_lib lpm6_lib + flow_classify_lib packet_distrib_lib reorder_lib ip_fragment_reassembly_lib generic_receive_offload_lib + generic_segmentation_offload_lib pdump_lib multi_proc_support kernel_nic_interface thread_safety_dpdk_functions eventdev + event_ethernet_rx_adapter qos_framework power_man packet_classif_access_ctrl @@ -191,6 +197,19 @@ Programmer's Guide :numref:`figure_efd11` :ref:`figure_efd11` +:numref:`figure_membership1` :ref:`figure_membership1` + +:numref:`figure_membership2` :ref:`figure_membership2` + +:numref:`figure_membership3` :ref:`figure_membership3` + +:numref:`figure_membership4` :ref:`figure_membership4` + +:numref:`figure_membership5` :ref:`figure_membership5` + +:numref:`figure_membership6` :ref:`figure_membership6` + +:numref:`figure_membership7` :ref:`figure_membership7` **Tables** diff --git a/doc/guides/prog_guide/member_lib.rst b/doc/guides/prog_guide/member_lib.rst new file mode 100644 index 00000000..caca8dc2 --- /dev/null +++ b/doc/guides/prog_guide/member_lib.rst @@ -0,0 +1,420 @@ +.. BSD LICENSE + Copyright(c) 2017 Intel Corporation. All rights reserved. + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of Intel Corporation nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + +.. _member_library: + +Membership Library +================== + +Introduction +------------ + +The DPDK Membership Library provides an API for DPDK applications to insert a +new member, delete an existing member, or query the existence of a member in a +given set, or a group of sets. For the case of a group of sets, the library +will return not only whether the element has been inserted before in one of +the sets but also which set it belongs to. The Membership Library is an +extension and generalization of a traditional filter structure (for example +Bloom Filter [Member-bloom]) that has multiple usages in a wide variety of +workloads and applications. In general, the Membership Library is a data +structure that provides a "set-summary" on whether a member belongs to a set, +and as discussed in detail later, there are two advantages of using such a +set-summary rather than operating on a "full-blown" complete list of elements: +first, it has a much smaller storage requirement than storing the whole list of +elements themselves, and secondly checking an element membership (or other +operations) in this set-summary is much faster than checking it for the +original full-blown complete list of elements. + +We use the term "Set-Summary" in this guide to refer to the space-efficient, +probabilistic membership data structure that is provided by the library. A +membership test for an element will return the set this element belongs to or +that the element is "not-found" with very high probability of accuracy. Set-summary +is a fundamental data aggregation component that can be used in many network +(and other) applications. It is a crucial structure to address performance and +scalability issues of diverse network applications including overlay networks, +data-centric networks, flow table summaries, network statistics and +traffic monitoring. A set-summary is useful for applications who need to +include a list of elements while a complete list requires too much space +and/or too much processing cost. In these situations, the set-summary works as +a lossy hash-based representation of a set of members. It can dramatically +reduce space requirement and significantly improve the performance of set +membership queries at the cost of introducing a very small membership test error +probability. + +.. _figure_membership1: +.. figure:: img/member_i1.* + + Example Usages of Membership Library + +There are various usages for a Membership Library in a very +large set of applications and workloads. Interested readers can refer to +[Member-survey] for a survey of possible networking usages. The above figure +provide a small set of examples of using the Membership Library: + +* Sub-figure (a) + depicts a distributed web cache architecture where a collection of proxies + attempt to share their web caches (cached from a set of back-end web servers) to + provide faster responses to clients, and the proxies use the Membership + Library to share summaries of what web pages/objects they are caching. With the + Membership Library, a proxy receiving an http request will inquire the + set-summary to find its location and quickly determine whether to retrieve the + requested web page from a nearby proxy or from a back-end web server. + +* Sub-figure (b) depicts another example for using the Membership Library to + prevent routing loops which is typically done using slow TTL countdown and + dropping packets when TTL expires. As shown in Sub-figure (b), an embedded + set-summary in the packet header itself can be used to summarize the set of + nodes a packet has gone through, and each node upon receiving a packet can check + whether its id is a member of the set of visited nodes, and if it is, then a + routing loop is detected. + +* Sub-Figure (c) presents another usage of the Membership + Library to load-balance flows to worker threads with in-order guarantee where a + set-summary is used to query if a packet belongs to an existing flow or a new + flow. Packets belonging to a new flow are forwarded to the current least loaded + worker thread, while those belonging to an existing flow are forwarded to the + pre-assigned thread to guarantee in-order processing. + +* Sub-figure (d) highlights + yet another usage example in the database domain where a set-summary is used to + determine joins between sets instead of creating a join by comparing each + element of a set against the other elements in a different set, a join is done + on the summaries since they can efficiently encode members of a given set. + +Membership Library is a configurable library that is optimized to cover set +membership functionality for both a single set and multi-set scenarios. Two set-summary +schemes are presented including (a) vector of Bloom Filters and (b) Hash-Table based +set-summary schemes with and without false negative probability. +This guide first briefly describes these different types of set-summaries, usage examples for each, +and then it highlights the Membership Library API. + +Vector of Bloom Filters +----------------------- + +Bloom Filter (BF) [Member-bloom] is a well-known space-efficient +probabilistic data structure that answers set membership queries (test whether +an element is a member of a set) with some probability of false positives and +zero false negatives; a query for an element returns either it is "possibly in +a set" (with very high probability) or "definitely not in a set". + +The BF is a method for representing a set of ``n`` elements (for example flow keys +in network applications domain) to support membership queries. The idea of BF is +to allocate a bit-vector ``v`` with ``m`` bits, which are initially all set to 0. Then +it chooses ``k`` independent hash functions ``h1``, ``h2``, ... ``hk`` with hash values range from +``0`` to ``m-1`` to perform hashing calculations on each element to be inserted. Every time when an +element ``X`` being inserted into the set, the bits at positions ``h1(X)``, ``h2(X)``, ... +``hk(X)`` in ``v`` are set to 1 (any particular bit might be set to 1 multiple times +for multiple different inserted elements). Given a query for any element ``Y``, the +bits at positions ``h1(Y)``, ``h2(Y)``, ... ``hk(Y)`` are checked. If any of them is 0, +then Y is definitely not in the set. Otherwise there is a high probability that +Y is a member of the set with certain false positive probability. As shown in +the next equation, the false positive probability can be made arbitrarily small +by changing the number of hash functions (``k``) and the vector length (``m``). + +.. _figure_membership2: +.. figure:: img/member_i2.* + + Bloom Filter False Positive Probability + +Without BF, an accurate membership testing could involve a costly hash table +lookup and full element comparison. The advantage of using a BF is to simplify +the membership test into a series of hash calculations and memory accesses for a +small bit-vector, which can be easily optimized. Hence the lookup throughput +(set membership test) can be significantly faster than a normal hash table +lookup with element comparison. + +.. _figure_membership3: +.. figure:: img/member_i3.* + + Detecting Routing Loops Using BF + +BF is used for applications that need only one set, and the +membership of elements is checked against the BF. The example discussed +in the above figure is one example of potential applications that uses only one +set to capture the node IDs that have been visited so far by the packet. Each +node will then check this embedded BF in the packet header for its own id, and +if the BF indicates that the current node is definitely not in the set then a +loop-free route is guaranteed. + + +.. _figure_membership4: +.. figure:: img/member_i4.* + + Vector Bloom Filter (vBF) Overview + +To support membership test for both multiple sets and a single set, +the library implements a Vector Bloom Filter (vBF) scheme. +vBF basically composes multiple bloom filters into a vector of bloom filers. +The membership test is conducted on all of the +bloom filters concurrently to determine which set(s) it belongs to or none of +them. The basic idea of vBF is shown in the above figure where an element is +used to address multiple bloom filters concurrently and the bloom filter +index(es) with a hit is returned. + +.. _figure_membership5: +.. figure:: img/member_i5.* + + vBF for Flow Scheduling to Worker Thread + +As previously mentioned, there are many usages of such structures. vBF is used +for applications that need to check membership against multiple sets +simultaneously. The example shown in the above figure uses a set to capture +all flows being assigned for processing at a given worker thread. Upon receiving +a packet the vBF is used to quickly figure out if this packet belongs to a new flow +so as to be forwarded to the current least loaded worker thread, or otherwise it +should be queued for an existing thread to guarantee in-order processing (i.e. +the property of vBF to indicate right away that a given flow is a new one or +not is critical to minimize response time latency). + +It should be noted that vBF can be implemented using a set of single bloom +filters with sequential lookup of each BF. However, being able to concurrently +search all set-summaries is a big throughput advantage. In the library, certain +parallelism is realized by the implementation of checking all bloom filters +together. + + +Hash-Table based Set-Summaries +------------------------------ + +Hash-table based set-summary (HTSS) is another scheme in the membership library. +Cuckoo filter [Member-cfilter] is an example of HTSS. +HTSS supports multi-set membership testing like +vBF does. However, while vBF is better for a small number of targets, HTSS is more suitable +and can easily outperform vBF when the number of sets is +large, since HTSS uses a single hash table for membership testing while vBF +requires testing a series of Bloom Filters each corresponding to one set. +As a result, generally speaking vBF is more adequate for the case of a small limited number of sets +while HTSS should be used with a larger number of sets. + +.. _figure_membership6: +.. figure:: img/member_i6.* + + Using HTSS for Attack Signature Matching + +As shown in the above figure, attack signature matching where each set +represents a certain signature length (for correctness of this example, an +attack signature should not be a subset of another one) in the payload is a good +example for using HTSS with 0% false negative (i.e., when an element returns not +found, it has a 100% certainty that it is not a member of any set). The packet +inspection application benefits from knowing right away that the current payload +does not match any attack signatures in the database to establish its +legitimacy, otherwise a deep inspection of the packet is needed. + +HTSS employs a similar but simpler data structure to a traditional hash table, +and the major difference is that HTSS stores only the signatures but not the +full keys/elements which can significantly reduce the footprint of the table. +Along with the signature, HTSS also stores a value to indicate the target set. +When looking up an element, the element is hashed and the HTSS is addressed +to retrieve the signature stored. If the signature matches then the value is +retrieved corresponding to the index of the target set which the element belongs +to. Because signatures can collide, HTSS can still has false positive +probability. Furthermore, if elements are allowed to be +overwritten or evicted when the hash table becomes full, it will also have a +false negative probability. We discuss this case in the next section. + +Set-Summaries with False Negative Probability +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +As previously mentioned, traditional set-summaries (e.g. Bloom Filters) do not +have a false negative probability, i.e., it is 100% certain when an element +returns "not to be present" for a given set. However, the Membership Library +also supports a set-summary probabilistic data structure based on HTSS which +allows for false negative probability. + +In HTSS, when the hash table becomes full, keys/elements will fail to be added +into the table and the hash table has to be resized to accommodate for these new +elements, which can be expensive. However, if we allow new elements to overwrite +or evict existing elements (as a cache typically does), then the resulting +set-summary will begin to have false negative probability. This is because the +element that was evicted from the set-summary may still be present in the target +set. For subsequent inquiries the set-summary will falsely report the element +not being in the set, hence having a false negative probability. + +The major usage of HTSS with false negative is to use it as a cache for +distributing elements to different target sets. By allowing HTSS to evict old +elements, the set-summary can keep track of the most recent elements +(i.e. active) as a cache typically does. Old inactive elements (infrequently +used elements) will automatically and eventually get evicted from the +set-summary. It is worth noting that the set-summary still has false positive +probability, which means the application either can tolerate certain false positive +or it has fall-back path when false positive happens. + +.. _figure_membership7: +.. figure:: img/member_i7.* + + Using HTSS with False Negatives for Wild Card Classification + +HTSS with false negative (i.e. a cache) also has its wide set of applications. +For example wild card flow classification (e.g. ACL rules) highlighted in the +above figure is an example of such application. In that case each target set +represents a sub-table with rules defined by a certain flow mask. The flow masks +are non-overlapping, and for flows matching more than one rule only the highest +priority one is inserted in the corresponding sub-table (interested readers can +refer to the Open vSwitch (OvS) design of Mega Flow Cache (MFC) [Member-OvS] +for further details). Typically the rules will have a large number of distinct +unique masks and hence, a large number of target sets each corresponding to one +mask. Because the active set of flows varies widely based on the network +traffic, HTSS with false negative will act as a cache for <flowid, target ACL +sub-table> pair for the current active set of flows. When a miss occurs (as +shown in red in the above figure) the sub-tables will be searched sequentially +one by one for a possible match, and when found the flow key and target +sub-table will be inserted into the set-summary (i.e. cache insertion) so +subsequent packets from the same flow don’t incur the overhead of the +sequential search of sub-tables. + +Library API Overview +-------------------- + +The design goal of the Membership Library API is to be as generic as possible to +support all the different types of set-summaries we discussed in previous +sections and beyond. Fundamentally, the APIs need to include creation, +insertion, deletion, and lookup. + + +Set-summary Create +~~~~~~~~~~~~~~~~~~ + +The ``rte_member_create()`` function is used to create a set-summary structure, the input parameter +is a struct to pass in parameters that needed to initialize the set-summary, while the function returns the +pointer to the created set-summary or ``NULL`` if the creation failed. + +The general input arguments used when creating the set-summary should include ``name`` +which is the name of the created set-summary, *type* which is one of the types +supported by the library (e.g. ``RTE_MEMBER_TYPE_HT`` for HTSS or ``RTE_MEMBER_TYPE_VBF`` for vBF), and ``key_len`` +which is the length of the element/key. There are other parameters +are only used for certain type of set-summary, or which have a slightly different meaning for different types of set-summary. +For example, ``num_keys`` parameter means the maximum number of entries for Hash table based set-summary. +However, for bloom filter, this value means the expected number of keys that could be +inserted into the bloom filter(s). The value is used to calculate the size of each +bloom filter. + +We also pass two seeds: ``prim_hash_seed`` and +``sec_hash_seed`` for the primary and secondary hash functions to calculate two independent hash values. +``socket_id`` parameter is the NUMA socket ID for the memory used to create the +set-summary. For HTSS, another parameter ``is_cache`` is used to indicate +if this set-summary is a cache (i.e. with false negative probability) or not. +For vBF, extra parameters are needed. For example, ``num_set`` is the number of +sets needed to initialize the vector bloom filters. This number is equal to the +number of bloom filters will be created. +``false_pos_rate`` is the false positive rate. num_keys and false_pos_rate will be used to determine +the number of hash functions and the bloom filter size. + + +Set-summary Element Insertion +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The ``rte_member_add()`` function is used to insert an element/key into a set-summary structure. If it fails an +error is returned. For success the returned value is dependent on the +set-summary mode to provide extra information for the users. For vBF +mode, a return value of 0 means a successful insert. For HTSS mode without false negative, the insert +could fail with ``-ENOSPC`` if the table is full. With false negative (i.e. cache mode), +for insert that does not cause any eviction (i.e. no overwriting happens to an +existing entry) the return value is 0. For insertion that causes eviction, the return +value is 1 to indicate such situation, but it is not an error. + +The input arguments for the function should include the ``key`` which is a pointer to the element/key that needs to +be added to the set-summary, and ``set_id`` which is the set id associated +with the key that needs to be added. + + +Set-summary Element Lookup +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The ``rte_member_lookup()`` function looks up a single key/element in the set-summary structure. It +returns as soon as the first match is found. The return value is 1 if a +match is found and 0 otherwise. The arguments for the function include ``key`` which is a pointer to the +element/key that needs to be looked up, and ``set_id`` which is used to return the +first target set id where the key has matched, if any. + +The ``rte_member_lookup_bulk()`` function is used to look up a bulk of keys/elements in the +set-summary structure for their first match. Each key lookup returns as soon as the first match is found. The +return value is the number of keys that find a match. The arguments of the function include ``keys`` +which is a pointer to a bulk of keys that are to be looked up, +``num_keys`` is the number +of keys that will be looked up, and ``set_ids`` are the return target set +ids for the first match found for each of the input keys. ``set_ids`` is an array +needs to be sized according to the ``num_keys``. If there is no match, the set id +for that key will be set to RTE_MEMBER_NO_MATCH. + +The ``rte_member_lookup_multi()`` function looks up a single key/element in the +set-summary structure for multiple matches. It +returns ALL the matches (possibly more than one) found for this key when it +is matched against all target sets (it is worth noting that for cache mode HTSS, +the current implementation matches at most one target set). The return value is +the number of matches +that was found for this key (for cache mode HTSS the return value +should be at most 1). The arguments for the function include ``key`` which is a pointer to the +element/key that needs to be looked up, ``max_match_per_key`` which is to indicate the maximum number of matches +the user expects to find for each key, and ``set_id`` which is used to return all +target set ids where the key has matched, if any. The ``set_id`` array should be sized +according to ``max_match_per_key``. For vBF, the maximum number of matches per key is equal +to the number of sets. For HTSS, the maximum number of matches per key is equal to two time +entry count per bucket. ``max_match_per_key`` should be equal or smaller than the maximum number of +possible matches. + +The ``rte_membership_lookup_multi_bulk()`` function looks up a bulk of keys/elements in the +set-summary structure for multiple matches, each key lookup returns ALL the matches (possibly more +than one) found for this key when it is matched against all target sets (cache mode HTSS +matches at most one target set). The +return value is the number of keys that find one or more matches in the +set-summary structure. The arguments of the +function include ``keys`` which is +a pointer to a bulk of keys that are to be looked up, ``num_keys`` is the number +of keys that will be looked up, ``max_match_per_key`` is the possible +maximum number of matches for each key, ``match_count`` which is the returned number +of matches for each key, and ``set_ids`` are the returned target set +ids for all matches found for each keys. ``set_ids`` is 2-D array +containing a 1-D array for each key (the size of 1-D array per key should be set by the user according to ``max_match_per_key``). +``max_match_per_key`` should be equal or smaller than the maximum number of +possible matches, similar to ``rte_member_lookup_multi``. + + +Set-summary Element Delete +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The ``rte_membership_delete()`` function deletes an element/key from a set-summary structure, if it fails +an error is returned. The input arguments should include ``key`` which is a pointer to the +element/key that needs to be deleted from the set-summary, and ``set_id`` +which is the set id associated with the key to delete. It is worth noting that current +implementation of vBF does not support deletion [1]_. An error code ``-EINVAL`` will be returned. + +.. [1] Traditional bloom filter does not support proactive deletion. Supporting proactive deletion require additional implementation and performance overhead. + +References +----------- + +[Member-bloom] B H Bloom, "Space/Time Trade-offs in Hash Coding with Allowable Errors," Communications of the ACM, 1970. + +[Member-survey] A Broder and M Mitzenmacher, "Network Applications of Bloom Filters: A Survey," in Internet Mathematics, 2005. + +[Member-cfilter] B Fan, D G Andersen and M Kaminsky, "Cuckoo Filter: Practically Better Than Bloom," in Conference on emerging Networking Experiments and Technologies, 2014. + +[Member-OvS] B Pfaff, "The Design and Implementation of Open vSwitch," in NSDI, 2015. diff --git a/doc/guides/prog_guide/poll_mode_drv.rst b/doc/guides/prog_guide/poll_mode_drv.rst index 1ac8f7eb..6a0c9f99 100644 --- a/doc/guides/prog_guide/poll_mode_drv.rst +++ b/doc/guides/prog_guide/poll_mode_drv.rst @@ -310,6 +310,26 @@ exported by each PMD. The list of flags and their precise meaning is described in the mbuf API documentation and in the in :ref:`Mbuf Library <Mbuf_Library>`, section "Meta Information". +Per-Port and Per-Queue Offloads +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +In the DPDK offload API, offloads are divided into per-port and per-queue offloads. +The different offloads capabilities can be queried using ``rte_eth_dev_info_get()``. +Supported offloads can be either per-port or per-queue. + +Offloads are enabled using the existing ``DEV_TX_OFFLOAD_*`` or ``DEV_RX_OFFLOAD_*`` flags. +Per-port offload configuration is set using ``rte_eth_dev_configure``. +Per-queue offload configuration is set using ``rte_eth_rx_queue_setup`` and ``rte_eth_tx_queue_setup``. +To enable per-port offload, the offload should be set on both device configuration and queue setup. +In case of a mixed configuration the queue setup shall return with an error. +To enable per-queue offload, the offload can be set only on the queue setup. +Offloads which are not enabled are disabled by default. + +For an application to use the Tx offloads API it should set the ``ETH_TXQ_FLAGS_IGNORE`` flag in the ``txq_flags`` field located in ``rte_eth_txconf`` struct. +In such cases it is not required to set other flags in ``txq_flags``. +For an application to use the Rx offloads API it should set the ``ignore_offload_bitfield`` bit in the ``rte_eth_rxmode`` struct. +In such cases it is not required to set other bitfield offloads in the ``rxmode`` struct. + Poll Mode Driver API -------------------- @@ -536,3 +556,43 @@ call. As an end result, the application is able to achieve its goal of monitoring a single statistic ("rx_errors" in this case), and if that shows packets being dropped, it can easily retrieve a "set" of statistics using the IDs array parameter to ``rte_eth_xstats_get_by_id`` function. + +NIC Reset API +~~~~~~~~~~~~~ + +.. code-block:: c + + int rte_eth_dev_reset(uint16_t port_id); + +Sometimes a port has to be reset passively. For example when a PF is +reset, all its VFs should also be reset by the application to make them +consistent with the PF. A DPDK application also can call this function +to trigger a port reset. Normally, a DPDK application would invokes this +function when an RTE_ETH_EVENT_INTR_RESET event is detected. + +It is the duty of the PMD to trigger RTE_ETH_EVENT_INTR_RESET events and +the application should register a callback function to handle these +events. When a PMD needs to trigger a reset, it can trigger an +RTE_ETH_EVENT_INTR_RESET event. On receiving an RTE_ETH_EVENT_INTR_RESET +event, applications can handle it as follows: Stop working queues, stop +calling Rx and Tx functions, and then call rte_eth_dev_reset(). For +thread safety all these operations should be called from the same thread. + +For example when PF is reset, the PF sends a message to notify VFs of +this event and also trigger an interrupt to VFs. Then in the interrupt +service routine the VFs detects this notification message and calls +_rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_INTR_RESET, NULL, +NULL). This means that a PF reset triggers an RTE_ETH_EVENT_INTR_RESET +event within VFs. The function _rte_eth_dev_callback_process() will +call the registered callback function. The callback function can trigger +the application to handle all operations the VF reset requires including +stopping Rx/Tx queues and calling rte_eth_dev_reset(). + +The rte_eth_dev_reset() itself is a generic function which only does +some hardware reset operations through calling dev_unint() and +dev_init(), and itself does not handle synchronization, which is handled +by application. + +The PMD itself should not call rte_eth_dev_reset(). The PMD can trigger +the application to handle reset event. It is duty of application to +handle all synchronization before it calls rte_eth_dev_reset(). diff --git a/doc/guides/prog_guide/port_hotplug_framework.rst b/doc/guides/prog_guide/port_hotplug_framework.rst index ee765773..fb0efc18 100644 --- a/doc/guides/prog_guide/port_hotplug_framework.rst +++ b/doc/guides/prog_guide/port_hotplug_framework.rst @@ -101,10 +101,6 @@ Limitations * The framework can only be enabled with Linux. BSD is not supported. -* To detach a port, the port should be backed by a device that igb_uio - or VFIO manages. - * Not all PMDs support detaching feature. - To know whether a PMD can support detaching, search for the - "RTE_ETH_DEV_DETACHABLE" flag in rte_eth_dev::data::dev_flags. If the flag is - defined in the PMD, detaching is supported. + The underlying bus must support hot-unplug. If not supported, + the function ``rte_eth_dev_detach()`` will return negative ENOTSUP. diff --git a/doc/guides/prog_guide/power_man.rst b/doc/guides/prog_guide/power_man.rst index 114d0b1c..c5d62a39 100644 --- a/doc/guides/prog_guide/power_man.rst +++ b/doc/guides/prog_guide/power_man.rst @@ -89,6 +89,14 @@ Core state can be altered by speculative sleeps whenever the specified lcore has In the DPDK, if no packet is received after polling, speculative sleeps can be triggered according the strategies defined by the user space application. +Per-core Turbo Boost +-------------------- + +Individual cores can be allowed to enter a Turbo Boost state on a per-core +basis. This is achieved by enabling Turbo Boost Technology in the BIOS, then +looping through the relevant cores and enabling/disabling Turbo Boost on each +core. + API Overview of the Power Library --------------------------------- @@ -108,6 +116,10 @@ The main methods exported by power library are for CPU frequency scaling and inc * **Freq set**: Prompt the kernel to set the frequency for the specific lcore. +* **Enable turbo**: Prompt the kernel to enable Turbo Boost for the specific lcore. + +* **Disable turbo**: Prompt the kernel to disable Turbo Boost for the specific lcore. + User Cases ---------- diff --git a/doc/guides/prog_guide/profile_app.rst b/doc/guides/prog_guide/profile_app.rst index 54b546ac..ca1c91f6 100644 --- a/doc/guides/prog_guide/profile_app.rst +++ b/doc/guides/prog_guide/profile_app.rst @@ -39,7 +39,8 @@ Profiling on x86 ---------------- Intel processors provide performance counters to monitor events. -Some tools provided by Intel, such as VTune, can be used to profile and benchmark an application. +Some tools provided by Intel, such as Intel® VTune™ Amplifier, can be used +to profile and benchmark an application. See the *VTune Performance Analyzer Essentials* publication from Intel Press for more information. For a DPDK application, this can be done in a Linux* application environment only. @@ -59,6 +60,40 @@ Refer to the for details about application profiling. +Empty cycles tracing +~~~~~~~~~~~~~~~~~~~~ + +Iterations that yielded no RX packets (empty cycles, wasted iterations) can +be analyzed using VTune Amplifier. This profiling employs the +`Instrumentation and Tracing Technology (ITT) API +<https://software.intel.com/en-us/node/544195>`_ +feature of VTune Amplifier and requires only reconfiguring the DPDK library, +no changes in a DPDK application are needed. + +To trace wasted iterations on RX queues, first reconfigure DPDK with +``CONFIG_RTE_ETHDEV_RXTX_CALLBACKS`` and +``CONFIG_RTE_ETHDEV_PROFILE_ITT_WASTED_RX_ITERATIONS`` enabled. + +Then rebuild DPDK, specifying paths to the ITT header and library, which can +be found in any VTune Amplifier distribution in the *include* and *lib* +directories respectively: + +.. code-block:: console + + make EXTRA_CFLAGS=-I<path to ittnotify.h> \ + EXTRA_LDLIBS="-L<path to libittnotify.a> -littnotify" + +Finally, to see wasted iterations in your performance analysis results, +select the *"Analyze user tasks, events, and counters"* checkbox in the +*"Analysis Type"* tab when configuring analysis via VTune Amplifier GUI. +Alternatively, when running VTune Amplifier via command line, specify +``-knob enable-user-tasks=true`` option. + +Collected regions of wasted iterations will be marked on VTune Amplifier's +timeline as ITT tasks. These ITT tasks have predefined names, containing +Ethernet device and RX queue identifiers. + + Profiling on ARM64 ------------------ diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst index 662a9123..d158be5e 100644 --- a/doc/guides/prog_guide/rte_flow.rst +++ b/doc/guides/prog_guide/rte_flow.rst @@ -187,7 +187,7 @@ Pattern item Pattern items fall in two categories: - Matching protocol headers and packet data (ANY, RAW, ETH, VLAN, IPV4, - IPV6, ICMP, UDP, TCP, SCTP, VXLAN, MPLS, GRE and so on), usually + IPV6, ICMP, UDP, TCP, SCTP, VXLAN, MPLS, GRE, ESP and so on), usually associated with a specification structure. - Matching meta-data or affecting pattern processing (END, VOID, INVERT, PF, @@ -955,6 +955,31 @@ Usage example, fuzzy match a TCPv4 packets: | 4 | END | +-------+----------+ +Item: ``GTP``, ``GTPC``, ``GTPU`` +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Matches a GTPv1 header. + +Note: GTP, GTPC and GTPU use the same structure. GTPC and GTPU item +are defined for a user-friendly API when creating GTP-C and GTP-U +flow rules. + +- ``v_pt_rsv_flags``: version (3b), protocol type (1b), reserved (1b), + extension header flag (1b), sequence number flag (1b), N-PDU number + flag (1b). +- ``msg_type``: message type. +- ``msg_len``: message length. +- ``teid``: tunnel endpoint identifier. +- Default ``mask`` matches teid only. + +Item: ``ESP`` +^^^^^^^^^^^^^ + +Matches an ESP header. + +- ``hdr``: ESP header definition (``rte_esp.h``). +- Default ``mask`` matches SPI only. + Actions ~~~~~~~ @@ -972,7 +997,7 @@ They fall in three categories: additional processing by subsequent flow rules. - Other non-terminating meta actions that do not affect the fate of packets - (END, VOID, MARK, FLAG, COUNT). + (END, VOID, MARK, FLAG, COUNT, SECURITY). When several actions are combined in a flow rule, they should all have different types (e.g. dropping a packet twice is not possible). @@ -1354,6 +1379,101 @@ rule or if packets are not addressed to a VF in the first place. | ``vf`` | VF ID to redirect packets to | +--------------+--------------------------------+ +Action: ``METER`` +^^^^^^^^^^^^^^^^^ + +Applies a stage of metering and policing. + +The metering and policing (MTR) object has to be first created using the +rte_mtr_create() API function. The ID of the MTR object is specified as +action parameter. More than one flow can use the same MTR object through +the meter action. The MTR object can be further updated or queried using +the rte_mtr* API. + +- Non-terminating by default. + +.. _table_rte_flow_action_meter: + +.. table:: METER + + +--------------+---------------+ + | Field | Value | + +==============+===============+ + | ``mtr_id`` | MTR object ID | + +--------------+---------------+ + +Action: ``SECURITY`` +^^^^^^^^^^^^^^^^^^^^ + +Perform the security action on flows matched by the pattern items +according to the configuration of the security session. + +This action modifies the payload of matched flows. For INLINE_CRYPTO, the +security protocol headers and IV are fully provided by the application as +specified in the flow pattern. The payload of matching packets is +encrypted on egress, and decrypted and authenticated on ingress. +For INLINE_PROTOCOL, the security protocol is fully offloaded to HW, +providing full encapsulation and decapsulation of packets in security +protocols. The flow pattern specifies both the outer security header fields +and the inner packet fields. The security session specified in the action +must match the pattern parameters. + +The security session specified in the action must be created on the same +port as the flow action that is being specified. + +The ingress/egress flow attribute should match that specified in the +security session if the security session supports the definition of the +direction. + +Multiple flows can be configured to use the same security session. + +- Non-terminating by default. + +.. _table_rte_flow_action_security: + +.. table:: SECURITY + + +----------------------+--------------------------------------+ + | Field | Value | + +======================+======================================+ + | ``security_session`` | security session to apply | + +----------------------+--------------------------------------+ + +The following is an example of configuring IPsec inline using the +INLINE_CRYPTO security session: + +The encryption algorithm, keys and salt are part of the opaque +``rte_security_session``. The SA is identified according to the IP and ESP +fields in the pattern items. + +.. _table_rte_flow_item_esp_inline_example: + +.. table:: IPsec inline crypto flow pattern items. + + +-------+----------+ + | Index | Item | + +=======+==========+ + | 0 | Ethernet | + +-------+----------+ + | 1 | IPv4 | + +-------+----------+ + | 2 | ESP | + +-------+----------+ + | 3 | END | + +-------+----------+ + +.. _table_rte_flow_action_esp_inline_example: + +.. table:: IPsec inline flow actions. + + +-------+----------+ + | Index | Action | + +=======+==========+ + | 0 | SECURITY | + +-------+----------+ + | 1 | END | + +-------+----------+ + Negative types ~~~~~~~~~~~~~~ @@ -1401,7 +1521,7 @@ supported and can be created. .. code-block:: c int - rte_flow_validate(uint8_t port_id, + rte_flow_validate(uint16_t port_id, const struct rte_flow_attr *attr, const struct rte_flow_item pattern[], const struct rte_flow_action actions[], @@ -1456,7 +1576,7 @@ actually created and a handle returned. .. code-block:: c struct rte_flow * - rte_flow_create(uint8_t port_id, + rte_flow_create(uint16_t port_id, const struct rte_flow_attr *attr, const struct rte_flow_item pattern[], const struct rte_flow_action *actions[], @@ -1488,7 +1608,7 @@ performing this step before releasing resources. .. code-block:: c int - rte_flow_destroy(uint8_t port_id, + rte_flow_destroy(uint16_t port_id, struct rte_flow *flow, struct rte_flow_error *error); @@ -1519,7 +1639,7 @@ port. They are released as with successive calls to ``rte_flow_destroy()``. .. code-block:: c int - rte_flow_flush(uint8_t port_id, + rte_flow_flush(uint16_t port_id, struct rte_flow_error *error); In the unlikely event of failure, handles are still considered destroyed and @@ -1547,7 +1667,7 @@ definition. .. code-block:: c int - rte_flow_query(uint8_t port_id, + rte_flow_query(uint16_t port_id, struct rte_flow *flow, enum rte_flow_action_type action, void *data, @@ -1620,7 +1740,7 @@ port and may return errors such as ``ENOTSUP`` ("not supported"): .. code-block:: c int - rte_flow_isolate(uint8_t port_id, int set, struct rte_flow_error *error); + rte_flow_isolate(uint16_t port_id, int set, struct rte_flow_error *error); Arguments: @@ -1678,6 +1798,25 @@ freed by the application, however its pointer can be considered valid only as long as its associated DPDK port remains configured. Closing the underlying device or unloading the PMD invalidates it. +Helpers +------- + +Error initializer +~~~~~~~~~~~~~~~~~ + +.. code-block:: c + + static inline int + rte_flow_error_set(struct rte_flow_error *error, + int code, + enum rte_flow_error_type type, + const void *cause, + const char *message); + +This function initializes ``error`` (if non-NULL) with the provided +parameters and sets ``rte_errno`` to ``code``. A negative error ``code`` is +then returned. + Caveats ------- @@ -1743,13 +1882,11 @@ the legacy filtering framework, which should eventually disappear. whatsoever). They only make sure these callbacks are non-NULL or return the ``ENOSYS`` (function not supported) error. -This interface additionally defines the following helper functions: +This interface additionally defines the following helper function: - ``rte_flow_ops_get()``: get generic flow operations structure from a port. -- ``rte_flow_error_set()``: initialize generic flow error structure. - More will be added over time. Device compatibility diff --git a/doc/guides/prog_guide/rte_security.rst b/doc/guides/prog_guide/rte_security.rst new file mode 100644 index 00000000..71be036c --- /dev/null +++ b/doc/guides/prog_guide/rte_security.rst @@ -0,0 +1,564 @@ +.. BSD LICENSE + Copyright 2017 NXP. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of NXP nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + +Security Library +================ + +The security library provides a framework for management and provisioning +of security protocol operations offloaded to hardware based devices. The +library defines generic APIs to create and free security sessions which can +support full protocol offload as well as inline crypto operation with +NIC or crypto devices. The framework currently only supports the IPSec protocol +and associated operations, other protocols will be added in future. + +Design Principles +----------------- + +The security library provides an additional offload capability to an existing +crypto device and/or ethernet device. + +.. code-block:: console + + +---------------+ + | rte_security | + +---------------+ + \ / + +-----------+ +--------------+ + | NIC PMD | | CRYPTO PMD | + +-----------+ +--------------+ + +.. note:: + + Currently, the security library does not support the case of multi-process. + It will be updated in the future releases. + +The supported offload types are explained in the sections below. + +Inline Crypto +~~~~~~~~~~~~~ + +RTE_SECURITY_ACTION_TYPE_INLINE_CRYPTO: +The crypto processing for security protocol (e.g. IPSec) is processed +inline during receive and transmission on NIC port. The flow based +security action should be configured on the port. + +Ingress Data path - The packet is decrypted in RX path and relevant +crypto status is set in Rx descriptors. After the successful inline +crypto processing the packet is presented to host as a regular Rx packet +however all security protocol related headers are still attached to the +packet. e.g. In case of IPSec, the IPSec tunnel headers (if any), +ESP/AH headers will remain in the packet but the received packet +contains the decrypted data where the encrypted data was when the packet +arrived. The driver Rx path check the descriptors and and based on the +crypto status sets additional flags in the rte_mbuf.ol_flags field. + +.. note:: + + The underlying device may not support crypto processing for all ingress packet + matching to a particular flow (e.g. fragmented packets), such packets will + be passed as encrypted packets. It is the responsibility of application to + process such encrypted packets using other crypto driver instance. + +Egress Data path - The software prepares the egress packet by adding +relevant security protocol headers. Only the data will not be +encrypted by the software. The driver will accordingly configure the +tx descriptors. The hardware device will encrypt the data before sending the +the packet out. + +.. note:: + + The underlying device may support post encryption TSO. + +.. code-block:: console + + Egress Data Path + | + +--------|--------+ + | egress IPsec | + | | | + | +------V------+ | + | | SADB lookup | | + | +------|------+ | + | +------V------+ | + | | Tunnel | | <------ Add tunnel header to packet + | +------|------+ | + | +------V------+ | + | | ESP | | <------ Add ESP header without trailer to packet + | | | | <------ Mark packet to be offloaded, add trailer + | +------|------+ | meta-data to mbuf + +--------V--------+ + | + +--------V--------+ + | L2 Stack | + +--------|--------+ + | + +--------V--------+ + | | + | NIC PMD | <------ Set hw context for inline crypto offload + | | + +--------|--------+ + | + +--------|--------+ + | HW ACCELERATED | <------ Packet Encryption and + | NIC | Authentication happens inline + | | + +-----------------+ + + +Inline protocol offload +~~~~~~~~~~~~~~~~~~~~~~~ + +RTE_SECURITY_ACTION_TYPE_INLINE_PROTOCOL: +The crypto and protocol processing for security protocol (e.g. IPSec) +is processed inline during receive and transmission. The flow based +security action should be configured on the port. + +Ingress Data path - The packet is decrypted in the RX path and relevant +crypto status is set in the Rx descriptors. After the successful inline +crypto processing the packet is presented to the host as a regular Rx packet +but all security protocol related headers are optionally removed from the +packet. e.g. in the case of IPSec, the IPSec tunnel headers (if any), +ESP/AH headers will be removed from the packet and the received packet +will contains the decrypted packet only. The driver Rx path checks the +descriptors and based on the crypto status sets additional flags in +``rte_mbuf.ol_flags`` field. + +.. note:: + + The underlying device in this case is stateful. It is expected that + the device shall support crypto processing for all kind of packets matching + to a given flow, this includes fragmented packets (post reassembly). + E.g. in case of IPSec the device may internally manage anti-replay etc. + It will provide a configuration option for anti-replay behavior i.e. to drop + the packets or pass them to driver with error flags set in the descriptor. + +Egress Data path - The software will send the plain packet without any +security protocol headers added to the packet. The driver will configure +the security index and other requirement in tx descriptors. +The hardware device will do security processing on the packet that includes +adding the relevant protocol headers and encrypting the data before sending +the packet out. The software should make sure that the buffer +has required head room and tail room for any protocol header addition. The +software may also do early fragmentation if the resultant packet is expected +to cross the MTU size. + + +.. note:: + + The underlying device will manage state information required for egress + processing. E.g. in case of IPSec, the seq number will be added to the + packet, however the device shall provide indication when the sequence number + is about to overflow. The underlying device may support post encryption TSO. + +.. code-block:: console + + Egress Data Path + | + +--------|--------+ + | egress IPsec | + | | | + | +------V------+ | + | | SADB lookup | | + | +------|------+ | + | +------V------+ | + | | Desc | | <------ Mark packet to be offloaded + | +------|------+ | + +--------V--------+ + | + +--------V--------+ + | L2 Stack | + +--------|--------+ + | + +--------V--------+ + | | + | NIC PMD | <------ Set hw context for inline crypto offload + | | + +--------|--------+ + | + +--------|--------+ + | HW ACCELERATED | <------ Add tunnel, ESP header etc header to + | NIC | packet. Packet Encryption and + | | Authentication happens inline. + +-----------------+ + + +Lookaside protocol offload +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL: +This extends librte_cryptodev to support the programming of IPsec +Security Association (SA) as part of a crypto session creation including +the definition. In addition to standard crypto processing, as defined by +the cryptodev, the security protocol processing is also offloaded to the +crypto device. + +Decryption: The packet is sent to the crypto device for security +protocol processing. The device will decrypt the packet and it will also +optionally remove additional security headers from the packet. +E.g. in case of IPSec, IPSec tunnel headers (if any), ESP/AH headers +will be removed from the packet and the decrypted packet may contain +plain data only. + +.. note:: + + In case of IPSec the device may internally manage anti-replay etc. + It will provide a configuration option for anti-replay behavior i.e. to drop + the packets or pass them to driver with error flags set in descriptor. + +Encryption: The software will submit the packet to cryptodev as usual +for encryption, the hardware device in this case will also add the relevant +security protocol header along with encrypting the packet. The software +should make sure that the buffer has required head room and tail room +for any protocol header addition. + +.. note:: + + In the case of IPSec, the seq number will be added to the packet, + It shall provide an indication when the sequence number is about to + overflow. + +.. code-block:: console + + Egress Data Path + | + +--------|--------+ + | egress IPsec | + | | | + | +------V------+ | + | | SADB lookup | | <------ SA maps to cryptodev session + | +------|------+ | + | +------|------+ | + | | \--------------------\ + | | Crypto | | | <- Crypto processing through + | | /----------------\ | inline crypto PMD + | +------|------+ | | | + +--------V--------+ | | + | | | + +--------V--------+ | | create <-- SA is added to hw + | L2 Stack | | | inline using existing create + +--------|--------+ | | session sym session APIs + | | | | + +--------V--------+ +---|---|----V---+ + | | | \---/ | | <--- Add tunnel, ESP header etc + | NIC PMD | | INLINE | | header to packet.Packet + | | | CRYPTO PMD | | Encryption/Decryption and + +--------|--------+ +----------------+ Authentication happens + | inline. + +--------|--------+ + | NIC | + +--------|--------+ + V + +Device Features and Capabilities +--------------------------------- + +Device Capabilities For Security Operations +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The device (crypto or ethernet) capabilities which support security operations, +are defined by the security action type, security protocol, protocol +capabilities and corresponding crypto capabilities for security. For the full +scope of the Security capability see definition of rte_security_capability +structure in the *DPDK API Reference*. + +.. code-block:: c + + struct rte_security_capability; + +Each driver (crypto or ethernet) defines its own private array of capabilities +for the operations it supports. Below is an example of the capabilities for a +PMD which supports the IPSec protocol. + +.. code-block:: c + + static const struct rte_security_capability pmd_security_capabilities[] = { + { /* IPsec Lookaside Protocol offload ESP Tunnel Egress */ + .action = RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL, + .protocol = RTE_SECURITY_PROTOCOL_IPSEC, + .ipsec = { + .proto = RTE_SECURITY_IPSEC_SA_PROTO_ESP, + .mode = RTE_SECURITY_IPSEC_SA_MODE_TUNNEL, + .direction = RTE_SECURITY_IPSEC_SA_DIR_EGRESS, + .options = { 0 } + }, + .crypto_capabilities = pmd_capabilities + }, + { /* IPsec Lookaside Protocol offload ESP Tunnel Ingress */ + .action = RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL, + .protocol = RTE_SECURITY_PROTOCOL_IPSEC, + .ipsec = { + .proto = RTE_SECURITY_IPSEC_SA_PROTO_ESP, + .mode = RTE_SECURITY_IPSEC_SA_MODE_TUNNEL, + .direction = RTE_SECURITY_IPSEC_SA_DIR_INGRESS, + .options = { 0 } + }, + .crypto_capabilities = pmd_capabilities + }, + { + .action = RTE_SECURITY_ACTION_TYPE_NONE + } + }; + static const struct rte_cryptodev_capabilities pmd_capabilities[] = { + { /* SHA1 HMAC */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + .sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_AUTH, + .auth = { + .algo = RTE_CRYPTO_AUTH_SHA1_HMAC, + .block_size = 64, + .key_size = { + .min = 64, + .max = 64, + .increment = 0 + }, + .digest_size = { + .min = 12, + .max = 12, + .increment = 0 + }, + .aad_size = { 0 }, + .iv_size = { 0 } + } + } + }, + { /* AES CBC */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + .sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_CIPHER, + .cipher = { + .algo = RTE_CRYPTO_CIPHER_AES_CBC, + .block_size = 16, + .key_size = { + .min = 16, + .max = 32, + .increment = 8 + }, + .iv_size = { + .min = 16, + .max = 16, + .increment = 0 + } + } + } + } + } + + +Capabilities Discovery +~~~~~~~~~~~~~~~~~~~~~~ + +Discovering the features and capabilities of a driver (crypto/ethernet) +is achieved through the ``rte_security_capabilities_get()`` function. + +.. code-block:: c + + const struct rte_security_capability *rte_security_capabilities_get(uint16_t id); + +This allows the user to query a specific driver and get all device +security capabilities. It returns an array of ``rte_security_capability`` structures +which contains all the capabilities for that device. + +Security Session Create/Free +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Security Sessions are created to store the immutable fields of a particular Security +Association for a particular protocol which is defined by a security session +configuration structure which is used in the operation processing of a packet flow. +Sessions are used to manage protocol specific information as well as crypto parameters. +Security sessions cache this immutable data in a optimal way for the underlying PMD +and this allows further acceleration of the offload of Crypto workloads. + +The Security framework provides APIs to create and free sessions for crypto/ethernet +devices, where sessions are mempool objects. It is the application's responsibility +to create and manage the session mempools. The mempool object size should be able to +accommodate the driver's private data of security session. + +Once the session mempools have been created, ``rte_security_session_create()`` +is used to allocate and initialize a session for the required crypto/ethernet device. + +Session APIs need a parameter ``rte_security_ctx`` to identify the crypto/ethernet +security ops. This parameter can be retrieved using the APIs +``rte_cryptodev_get_sec_ctx()`` (for crypto device) or ``rte_eth_dev_get_sec_ctx`` +(for ethernet port). + +Sessions already created can be updated with ``rte_security_session_update()``. + +When a session is no longer used, the user must call ``rte_security_session_destroy()`` +to free the driver private session data and return the memory back to the mempool. + +For look aside protocol offload to hardware crypto device, the ``rte_crypto_op`` +created by the application is attached to the security session by the API +``rte_security_attach_session()``. + +For Inline Crypto and Inline protocol offload, device specific defined metadata is +updated in the mbuf using ``rte_security_set_pkt_metadata()`` if +``DEV_TX_OFFLOAD_SEC_NEED_MDATA`` is set. + +Security session configuration +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Security Session configuration structure is defined as ``rte_security_session_conf`` + +.. code-block:: c + + struct rte_security_session_conf { + enum rte_security_session_action_type action_type; + /**< Type of action to be performed on the session */ + enum rte_security_session_protocol protocol; + /**< Security protocol to be configured */ + union { + struct rte_security_ipsec_xform ipsec; + struct rte_security_macsec_xform macsec; + }; + /**< Configuration parameters for security session */ + struct rte_crypto_sym_xform *crypto_xform; + /**< Security Session Crypto Transformations */ + }; + +The configuration structure reuses the ``rte_crypto_sym_xform`` struct for crypto related +configuration. The ``rte_security_session_action_type`` struct is used to specify whether the +session is configured for Lookaside Protocol offload or Inline Crypto or Inline Protocol +Offload. + +.. code-block:: c + + enum rte_security_session_action_type { + RTE_SECURITY_ACTION_TYPE_NONE, + /**< No security actions */ + RTE_SECURITY_ACTION_TYPE_INLINE_CRYPTO, + /**< Crypto processing for security protocol is processed inline + * during transmission */ + RTE_SECURITY_ACTION_TYPE_INLINE_PROTOCOL, + /**< All security protocol processing is performed inline during + * transmission */ + RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL + /**< All security protocol processing including crypto is performed + * on a lookaside accelerator */ + }; + +The ``rte_security_session_protocol`` is defined as + +.. code-block:: c + + enum rte_security_session_protocol { + RTE_SECURITY_PROTOCOL_IPSEC, + /**< IPsec Protocol */ + RTE_SECURITY_PROTOCOL_MACSEC, + /**< MACSec Protocol */ + }; + +Currently the library defines configuration parameters for IPSec only. For other +protocols like MACSec, structures and enums are defined as place holders which +will be updated in the future. + +IPsec related configuration parameters are defined in ``rte_security_ipsec_xform`` + +.. code-block:: c + + struct rte_security_ipsec_xform { + uint32_t spi; + /**< SA security parameter index */ + uint32_t salt; + /**< SA salt */ + struct rte_security_ipsec_sa_options options; + /**< various SA options */ + enum rte_security_ipsec_sa_direction direction; + /**< IPSec SA Direction - Egress/Ingress */ + enum rte_security_ipsec_sa_protocol proto; + /**< IPsec SA Protocol - AH/ESP */ + enum rte_security_ipsec_sa_mode mode; + /**< IPsec SA Mode - transport/tunnel */ + struct rte_security_ipsec_tunnel_param tunnel; + /**< Tunnel parameters, NULL for transport mode */ + }; + + +Security API +~~~~~~~~~~~~ + +The rte_security Library API is described in the *DPDK API Reference* document. + +Flow based Security Session +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In the case of NIC based offloads, the security session specified in the +'rte_flow_action_security' must be created on the same port as the +flow action that is being specified. + +The ingress/egress flow attribute should match that specified in the security +session if the security session supports the definition of the direction. + +Multiple flows can be configured to use the same security session. For +example if the security session specifies an egress IPsec SA, then multiple +flows can be specified to that SA. In the case of an ingress IPsec SA then +it is only valid to have a single flow to map to that security session. + +.. code-block:: console + + Configuration Path + | + +--------|--------+ + | Add/Remove | + | IPsec SA | <------ Build security flow action of + | | | ipsec transform + |--------|--------| + | + +--------V--------+ + | Flow API | + +--------|--------+ + | + +--------V--------+ + | | + | NIC PMD | <------ Add/Remove SA to/from hw context + | | + +--------|--------+ + | + +--------|--------+ + | HW ACCELERATED | + | NIC | + | | + +--------|--------+ + +* Add/Delete SA flow: + To add a new inline SA construct a rte_flow_item for Ethernet + IP + ESP + using the SA selectors and the ``rte_crypto_ipsec_xform`` as the ``rte_flow_action``. + Note that any rte_flow_items may be empty, which means it is not checked. + +.. code-block:: console + + In its most basic form, IPsec flow specification is as follows: + +-------+ +----------+ +--------+ +-----+ + | Eth | -> | IP4/6 | -> | ESP | -> | END | + +-------+ +----------+ +--------+ +-----+ + + However, the API can represent, IPsec crypto offload with any encapsulation: + +-------+ +--------+ +-----+ + | Eth | -> ... -> | ESP | -> | END | + +-------+ +--------+ +-----+ diff --git a/doc/guides/prog_guide/source_org.rst b/doc/guides/prog_guide/source_org.rst index d5d01f38..7aab4b45 100644 --- a/doc/guides/prog_guide/source_org.rst +++ b/doc/guides/prog_guide/source_org.rst @@ -108,7 +108,6 @@ The drivers directory has a *net* subdirectory which contains:: +-- szedata2 # SZEDATA2 poll mode driver +-- virtio # Virtio poll mode driver +-- vmxnet3 # VMXNET3 poll mode driver - +-- xenvirt # Xen virtio poll mode driver .. note:: diff --git a/doc/guides/prog_guide/traffic_metering_and_policing.rst b/doc/guides/prog_guide/traffic_metering_and_policing.rst new file mode 100644 index 00000000..89f0e68b --- /dev/null +++ b/doc/guides/prog_guide/traffic_metering_and_policing.rst @@ -0,0 +1,102 @@ +.. BSD LICENSE + Copyright(c) 2017 Intel Corporation. All rights reserved. + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of Intel Corporation nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + +Traffic Metering and Policing API +================================= + + +Overview +-------- + +This is the generic API for the Quality of Service (QoS) Traffic Metering and +Policing (MTR) of Ethernet devices. This API is agnostic of the underlying HW, +SW or mixed HW-SW implementation. + +The main features are: + +* Part of DPDK rte_ethdev API +* Capability query API +* Metering algorithms: RFC 2697 Single Rate Three Color Marker (srTCM), RFC 2698 + and RFC 4115 Two Rate Three Color Marker (trTCM) +* Policer actions (per meter output color): recolor, drop +* Statistics (per policer output color) + +Configuration steps +------------------- + +The metering and policing stage typically sits on top of flow classification, +which is why the MTR objects are enabled through a special "meter" action. + +The MTR objects are created and updated in their own name space (``rte_mtr``) +within the ``librte_ether`` library. Whether an MTR object is private to a +flow or potentially shared by several flows has to be specified at its +creation time. + +Once successfully created, an MTR object is hooked into the RX processing path +of the Ethernet device by linking it to one or several flows through the +dedicated "meter" flow action. One or several "meter" actions can be registered +for the same flow. An MTR object can only be destroyed if there are no flows +using it. + +Run-time processing +------------------- + +Traffic metering determines the color for the current packet (green, yellow, +red) based on the previous history for this flow as maintained by the MTR +object. The policer can do nothing, override the color the packet or drop the +packet. Statistics counters are maintained for MTR object, as configured. + +The processing done for each input packet hitting an MTR object is: + +* Traffic metering: The packet is assigned a color (the meter output color) + based on the previous traffic history reflected in the current state of the + MTR object, according to the specific traffic metering algorithm. The + traffic metering algorithm can typically work in color aware mode, in which + case the input packet already has an initial color (the input color), or in + color blind mode, which is equivalent to considering all input packets + initially colored as green. + +* Policing: There is a separate policer action configured for each meter + output color, which can: + + * Drop the packet. + + * Keep the same packet color: the policer output color matches the meter + output color (essentially a no-op action). + + * Recolor the packet: the policer output color is set to a different color + than the meter output color. The policer output color is the output color + of the packet, which is set in the packet meta-data (i.e. struct + ``rte_mbuf::sched::color``). + +* Statistics: The set of counters maintained for each MTR object is + configurable and subject to the implementation support. This set includes + the number of packets and bytes dropped or passed for each output color. diff --git a/doc/guides/prog_guide/vhost_lib.rst b/doc/guides/prog_guide/vhost_lib.rst index 59792907..e71bdbd5 100644 --- a/doc/guides/prog_guide/vhost_lib.rst +++ b/doc/guides/prog_guide/vhost_lib.rst @@ -110,6 +110,20 @@ The following is an overview of some key Vhost API functions: of those segments, thus the fewer the segments, the quicker we will get the mapping. NOTE: we may speed it by using tree searching in future. + - ``RTE_VHOST_USER_IOMMU_SUPPORT`` + + IOMMU support will be enabled when this flag is set. It is disabled by + default. + + Enabling this flag makes possible to use guest vIOMMU to protect vhost + from accessing memory the virtio device isn't allowed to, when the feature + is negotiated and an IOMMU device is declared. + + However, this feature enables vhost-user's reply-ack protocol feature, + which implementation is buggy in Qemu v2.7.0-v2.9.0 when doing multiqueue. + Enabling this flag with these Qemu version results in Qemu being blocked + when multiple queue pairs are declared. + * ``rte_vhost_driver_set_features(path, features)`` This function sets the feature bits the vhost-user driver supports. The @@ -129,8 +143,7 @@ The following is an overview of some key Vhost API functions: * ``destroy_device(int vid)`` - This callback is invoked when a virtio device shuts down (or when the - vhost connection is broken). + This callback is invoked when a virtio device is paused or shut down. * ``vring_state_changed(int vid, uint16_t queue_id, int enable)`` @@ -143,6 +156,18 @@ The following is an overview of some key Vhost API functions: ``VHOST_F_LOG_ALL`` will be set/cleared at the start/end of live migration, respectively. + * ``new_connection(int vid)`` + + This callback is invoked on new vhost-user socket connection. If DPDK + acts as the server the device should not be deleted before + ``destroy_connection`` callback is received. + + * ``destroy_connection(int vid)`` + + This callback is invoked when vhost-user socket connection is closed. + It indicates that device with id ``vid`` is no longer in use and can be + safely deleted. + * ``rte_vhost_driver_disable/enable_features(path, features))`` This function disables/enables some features. For example, it can be used to diff --git a/doc/guides/prog_guide/writing_efficient_code.rst b/doc/guides/prog_guide/writing_efficient_code.rst index 8223acee..d7ac6778 100644 --- a/doc/guides/prog_guide/writing_efficient_code.rst +++ b/doc/guides/prog_guide/writing_efficient_code.rst @@ -105,6 +105,21 @@ meaning that if all memory access operations are done on the first channel only, By default, the :ref:`Mempool Library <Mempool_Library>` spreads the addresses of objects among memory channels. +Locking memory pages +~~~~~~~~~~~~~~~~~~~~ + +The underlying operating system is allowed to load/unload memory pages at its own discretion. +These page loads could impact the performance, as the process is on hold when the kernel fetches them. + +To avoid these you could pre-load, and lock them into memory with the ``mlockall()`` call. + +.. code-block:: c + + if (mlockall(MCL_CURRENT | MCL_FUTURE)) { + RTE_LOG(NOTICE, USER1, "mlockall() failed with error \"%s\"\n", + strerror(errno)); + } + Communication Between lcores ---------------------------- diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst index 3362f335..7fcebf13 100644 --- a/doc/guides/rel_notes/deprecation.rst +++ b/doc/guides/rel_notes/deprecation.rst @@ -8,14 +8,6 @@ API and ABI deprecation notices are to be posted here. Deprecation Notices ------------------- -* eal: the following functions are deprecated starting from 17.05 and will - be removed in 17.11: - - - ``rte_set_log_level``, replaced by ``rte_log_set_global_level`` - - ``rte_get_log_level``, replaced by ``rte_log_get_global_level`` - - ``rte_set_log_type``, replaced by ``rte_log_set_level`` - - ``rte_get_log_type``, replaced by ``rte_log_get_level`` - * eal: several API and ABI changes are planned for ``rte_devargs`` in v17.11. The format of device command line parameters will change. The bus will need to be explicitly stated in the device declaration. The enum ``rte_devtype`` @@ -29,29 +21,12 @@ Deprecation Notices - ``rte_eal_devargs_type_count`` - ``rte_eal_parse_devargs_str``, replaced by ``rte_eal_devargs_parse`` -* eal: the support of Xen dom0 will be removed from EAL in 17.11; and with - that, drivers/net/xenvirt and examples/vhost_xen will also be removed. - -* eal: An ABI change is planned for 17.11 to make DPDK aware of IOVA address - translation scheme. - Reference to phys address in EAL data-structure or functions may change to - IOVA address or more appropriate name. - The change will be only for the name. - Functional aspects of the API or data-structure will remain same. - -* The mbuf flags PKT_RX_VLAN_PKT and PKT_RX_QINQ_PKT are deprecated and - are respectively replaced by PKT_RX_VLAN_STRIPPED and - PKT_RX_QINQ_STRIPPED, that are better described. The old flags and - their behavior will be kept until 17.08 and will be removed in 17.11. - -* mempool: The following will be modified in 17.11: +* pci: Several exposed functions are misnamed. + The following functions are deprecated starting from v17.11 and are replaced: - - ``rte_mempool_xmem_size`` and ``rte_mempool_xmem_usage`` need to know - the mempool flag status so adding new param rte_mempool in those API. - - Removing __rte_unused int flag param from ``rte_mempool_generic_put`` - and ``rte_mempool_generic_get`` API. - - ``rte_mempool`` flags data type will changed from int to - unsigned int. + - ``eal_parse_pci_BDF`` replaced by ``rte_pci_addr_parse`` + - ``eal_parse_pci_DomBDF`` replaced by ``rte_pci_addr_parse`` + - ``rte_eal_compare_pci_addr`` replaced by ``rte_pci_addr_cmp`` * ethdev: Tx offloads will no longer be enabled by default in 17.11. Instead, the ``rte_eth_txmode`` structure will be extended with @@ -68,55 +43,5 @@ Deprecation Notices Target release for removal of the legacy API will be defined once most PMDs have switched to rte_flow. -* ethdev: The device flag advertizing hotplug capability - ``RTE_ETH_DEV_DETACHABLE`` is not needed anymore and will be removed in - v17.11. - This capability is verified upon calling the relevant hotplug functions in EAL - by checking that the ``unplug`` ops is set in the bus. This verification is - done by the EAL and not by the ``ethdev`` layer anymore. Users relying on this - flag being present only have to remove their checks to follow the change. - -* ABI/API changes are planned for 17.11 in all structures which include port_id - definition such as "rte_eth_dev_data", "rte_port_ethdev_reader_params", - "rte_port_ethdev_writer_params", and so on. The definition of port_id will be - changed from 8 bits to 16 bits in order to support more than 256 ports in - DPDK. All APIs which have port_id parameter will be changed at the same time. - -* ethdev: An ABI change is planned for 17.11 for the structure rte_eth_dev_data. - The size of the unique name will increase RTE_ETH_NAME_MAX_LEN from 32 to - 64 characters to allow using a globally unique identifier (GUID) in this field. - -* ethdev: new parameters - ``rte_security_capabilities`` and - ``rte_security_ops`` will be added to ``rte_eth_dev_info`` and - ``rte_eth_dev`` respectively to support security operations like - ipsec inline. - -* cryptodev: new parameters - ``rte_security_capabilities`` and - ``rte_security_ops`` will be added to ``rte_cryptodev_info`` and - ``rte_cryptodev`` respectively to support security protocol offloaded - operations. - -* cryptodev: the following function is deprecated starting from 17.08 and will - be removed in 17.11: - - - ``rte_cryptodev_create_vdev`` - -* cryptodev: the following function will be static in 17.11 and included - by all crypto drivers, therefore, will not be public: - - - ``rte_cryptodev_vdev_pmd_init`` - -* cryptodev: the following function will have an extra parameter, passing a - statically allocated crypto driver structure, instead of calling malloc, - in 17.11: - - - ``rte_cryptodev_allocate_driver`` - * librte_meter: The API will change to accommodate configuration profiles. Most of the API functions will have an additional opaque parameter. - -* librte_table: The ``key_mask`` parameter will be added to all the hash tables - that currently do not have it, as well as to the hash compute function prototype. - The non-"do-sig" versions of the hash tables will be removed - (including the ``signature_offset`` parameter) - and the "do-sig" versions renamed accordingly. diff --git a/doc/guides/rel_notes/index.rst b/doc/guides/rel_notes/index.rst index b3c8090a..35659d8a 100644 --- a/doc/guides/rel_notes/index.rst +++ b/doc/guides/rel_notes/index.rst @@ -36,6 +36,7 @@ Release Notes :numbered: rel_description + release_17_11 release_17_08 release_17_05 release_17_02 diff --git a/doc/guides/rel_notes/release_17_11.rst b/doc/guides/rel_notes/release_17_11.rst new file mode 100644 index 00000000..e6e4407c --- /dev/null +++ b/doc/guides/rel_notes/release_17_11.rst @@ -0,0 +1,600 @@ +DPDK Release 17.11 +================== + +.. **Read this first.** + + The text in the sections below explains how to update the release notes. + + Use proper spelling, capitalization and punctuation in all sections. + + Variable and config names should be quoted as fixed width text: + ``LIKE_THIS``. + + Build the docs and view the output file to ensure the changes are correct:: + + make doc-guides-html + + xdg-open build/doc/html/guides/rel_notes/release_17_11.html + + +New Features +------------ + +.. This section should contain new features added in this release. Sample + format: + + * **Add a title in the past tense with a full stop.** + + Add a short 1-2 sentence description in the past tense. The description + should be enough to allow someone scanning the release notes to + understand the new feature. + + If the feature adds a lot of sub-features you can use a bullet list like + this: + + * Added feature foo to do something. + * Enhanced feature bar to do something else. + + Refer to the previous release notes for examples. + + This section is a comment. do not overwrite or remove it. + Also, make sure to start the actual text at the margin. + ========================================================= + +* **Extended port_id range from uint8_t to uint16_t.** + + Increased port_id range from 8 bits to 16 bits in order to support more than + 256 ports in dpdk. All ethdev APIs which have port_id as parameter are changed + in the meantime. + +* **Modified the return type of rte_eth_stats_reset.** + + Changed return type of ``rte_eth_stats_reset`` from ``void`` to ``int`` + so the caller may know whether a device supports the operation or not + and if the operation was carried out. + +* **Added a new driver for Marvell Armada 7k/8k devices.** + + Added the new mrvl net driver for Marvell Armada 7k/8k devices. See the + "Network Interface Controller Drivers" document for more details on this new + driver. + +* **Updated mlx5 driver.** + + Updated the mlx5 driver including the following changes: + + * Enabled PMD to run on top of upstream linux kernel and rdma-core libs. + By that removed the dependency on specific Mellanox OFED libraries. + * Improved PMD latency performance. + * Improved PMD memory footprint. + * Supported vectorized Rx/Tx burst for ARMv8. + * Supported secondary process. + * Supported flow counters. + * Supported Rx hardware timestamp offload. + * Supported device removal event. + +* **Added SoftNIC PMD.** + + Added new SoftNIC PMD. This virtual device offers applications a software + fallback support for traffic management. + +* **Added support for NXP DPAA Devices.** + + Added support for NXP's DPAA devices - LS104x series. This includes: + + * DPAA Bus driver + * DPAA Mempool driver for supporting offloaded packet memory pool + * DPAA PMD for DPAA devices + + See the "Network Interface Controller Drivers" document for more details of + this new driver. + +* **Updated support for Cavium OCTEONTX Device.** + + Updated support for Cavium's OCTEONTX device(CN83xx). This includes: + + * OCTEONTX Mempool driver for supporting offloaded packet memory pool + * OCTEONTX Ethdev PMD + * OCTEONTX Eventdev-Ethdev Rx adapter + + See the "Network Interface Controller Drivers" document for more details of + this new driver. + +* **nfp: Added PF support.** + + Previously Netronome's NFP PMD had just support for VFs. PF support is + just as a basic DPDK port and has no VF management yet. + + PF support comes with firmware upload support which allows the PMD to + independently work from kernel netdev NFP drivers. + + NFP 4000 devices are also now supported along with previous 6000 devices. + +* **Updated bnxt PMD.** + + Major enhancements include: + + * Support for Flow API + * Support for Tx and Rx descriptor status functions + +* **Add bus agnostic functions to cryptodev for PMD initialisation** + + Adds new PMD assist functions ``rte_cryptodev_pmd_parse_input_args()``, + ``rte_cryptodev_pmd_create()`` and ``rte_cryptodev_pmd_destroy()`` which + are bus independent for driver to manage creation and destruction of new + device instances. + +* **Updated QAT crypto PMD.** + + Performance enhancements: + + * Removed atomics from the internal queue pair structure. + * Coalesce writes to HEAD CSR on response processing. + * Coalesce writes to TAIL CSR on request processing. + + Additional support for: + + * AES CCM algorithm. + +* **Updated the AESNI MB PMD.** + + The AESNI MB PMD has been updated with additional support for: + + * DES CBC algorithm. + * DES DOCSIS BPI algorithm. + + This requires the IPSec Multi-buffer library 0.47. For more details, + check out the AESNI MB PMD documenation. + +* **Updated the OpenSSL PMD.** + + The OpenSSL PMD has been updated with additional support for: + + * DES CBC algorithm. + * AES CCM algorithm. + +* **Added NXP DPAA SEC crypto PMD.** + + A new "dpaa_sec" hardware based crypto PMD for NXP DPAA devices has been + added. See the "Crypto Device Drivers" document for more details on this + driver. + +* **Added MRVL crypto PMD.** + + A new crypto PMD has been added, which provides several ciphering and hashing + algorithms. All cryptography operations use the MUSDK library crypto API. + +* **Add new benchmarking mode to dpdk-test-crypto-perf application.** + + Added new "PMD cyclecount" benchmark mode to dpdk-test-crypto-perf application + that displays more detailed breakdown of CPU cycles used by hardware + acceleration. + +* **Added the Security Offload Library.** + + Added an experimental library - rte_security. It provide security APIs for + protocols like IPSec using inline ipsec offload to ethernet device or full + protocol offload with lookaside crypto device. + + See the "Security_Library" section of the DPDK Programmers Guide document, + for more information. + +* **Updated DPAA2_SEC crypto driver.** + + Updated dpaa2_sec crypto PMD to support rte_security lookaside protocol + offload for IPSec. + +* **Updated IXGBE ethernet driver.** + + Updated ixgbe ethernet PMD to support rte_security inline IPSec offload. + +* **Updated ipsec-secgw application** + + Updated ipsec-secgw sample application to support rte_security actions for + ipsec inline and full protocol offload using lookaside crypto offload. + +* **Added IOMMU support to libvhost-user** + + Implemented device IOTLB in Vhost-user backend, and enabled Virtio's IOMMU + feature. The feature is disabled by default, and can be enabled by setting + RTE_VHOST_USER_IOMMU_SUPPORT flag at vhost device registration time. + +* **Added the Event Ethernet Adapter Library.** + + Added the Event Ethernet Adapter library. It provices APIs for + eventdev applications to configure the ethdev to eventdev packet flow. + +* **Updated DPAA2 Event PMD.** + + Added support for eventdev ethernet adapter for DPAA2. + +* **Added Membership library (rte_member).** + + Added membership library. It provides an API for DPDK applications to insert a + new member, delete an existing member, or query the existence of a member in a + given set, or a group of sets. For the case of a group of sets the library + will return not only whether the element has been inserted before in one of + the sets but also which set it belongs to. + + The Membership Library is an extension and generalization of a traditional + filter (for example Bloom Filter) structure that has multiple usages in a wide + variety of workloads and applications. In general, the Membership Library is a + data structure that provides a “set-summary” and responds to set-membership + queries whether a certain member belongs to a set(s). + + See the :ref:`Membership Library <Member_Library>` documentation in + the Programmers Guide document, for more information. + +* **Added the Generic Segmentation Offload Library.** + + Added the Generic Segmentation Offload (GSO) library to enable + applications to split large packets (e.g. MTU is 64KB) into small + ones (e.g. MTU is 1500B). Supported packet types are: + + * TCP/IPv4 packets. + * VxLAN packets, which must have an outer IPv4 header, and contain + an inner TCP/IPv4 packet. + * GRE packets, which must contain an outer IPv4 header, and inner + TCP/IPv4 headers. + + The GSO library doesn't check if the input packets have correct + checksums, and doesn't update checksums for output packets. + Additionally, the GSO library doesn't process IP fragmented packets. + +* **Added the Flow Classification Library.** + + Added the Flow Classification library, it provides an API for DPDK + applications to classify an input packet by matching it against a set of flow + rules. It uses the librte_table API to manage the flow rules. + + +Resolved Issues +--------------- + +.. This section should contain bug fixes added to the relevant + sections. Sample format: + + * **code/section Fixed issue in the past tense with a full stop.** + + Add a short 1-2 sentence description of the resolved issue in the past + tense. + + The title should contain the code/lib section like a commit message. + + Add the entries in alphabetic order in the relevant sections below. + + This section is a comment. do not overwrite or remove it. + Also, make sure to start the actual text at the margin. + ========================================================= + + +EAL +~~~ + +* **Service core fails to call service callback due to atomic lock** + + In a specific configuration of multi-thread unsafe services and service + cores, a service core previously did not correctly release the atomic lock + on the service. This would result in the cores polling the service, but it + looked like another thread was executing the service callback. The logic for + atomic locking of the services has been fixed and refactored for readability. + +Drivers +~~~~~~~ + + +Libraries +~~~~~~~~~ + + +Examples +~~~~~~~~ + + +Other +~~~~~ + + +Known Issues +------------ + +.. This section should contain new known issues in this release. Sample format: + + * **Add title in present tense with full stop.** + + Add a short 1-2 sentence description of the known issue in the present + tense. Add information on any known workarounds. + + This section is a comment. do not overwrite or remove it. + Also, make sure to start the actual text at the margin. + ========================================================= + + +API Changes +----------- + +.. This section should contain API changes. Sample format: + + * Add a short 1-2 sentence description of the API change. Use fixed width + quotes for ``rte_function_names`` or ``rte_struct_names``. Use the past + tense. + + This section is a comment. do not overwrite or remove it. + Also, make sure to start the actual text at the margin. + ========================================================= + +* **Ethdev device name length increased** + + The size of internal device name is increased to 64 characters + to allow for storing longer bus specific name. + +* **Ethdev flag RTE_ETH_DEV_DETACHABLE was removed** + + This flag is not necessary anymore, with the new hotplug implementation. + It is now removed from the ether library. Its semantic is expressed at the bus + and PMD level. + +* **Service cores API updated for usability** + + The service cores API has been changed, removing pointers from the API + where possible, instead using integer IDs to identify each service. This + simplifed application code, aids debugging, and provides better + encapsulation. A summary of the main changes made is as follows: + + * Services identified by ID not by ``rte_service_spec`` pointer + * Reduced API surface by using ``set`` functions instead of enable/disable + * Reworked ``rte_service_register`` to provide the service ID to registrar + * Rework start and stop APIs into ``rte_service_runstate_set`` + * Added API to set runstate of service implementation to indicate readyness + +* **The following changes made in mempool library** + + * Moved ``flags`` datatype from int to unsigned int for ``rte_mempool``. + * Removed ``__rte_unused int flag`` param from ``rte_mempool_generic_put`` + and ``rte_mempool_generic_get`` API. + * Added ``flags`` param in ``rte_mempool_xmem_size`` and + ``rte_mempool_xmem_usage``. + +* ``rte_mem_phy2mch`` was used in Xen dom0 to obtain the physical address; + remove this API as Xen dom0 support was removed. + +* **Some data type, structure members and functions related to physical address + are deprecated and have new alias with IOVA wording.** + + * ``phys_addr_t`` can be often replaced by ``rte_iova_t`` of same size. + * ``RTE_BAD_PHYS_ADDR`` is often replaced by ``RTE_BAD_IOVA`` of same value. + * ``rte_memseg.phys_addr`` is aliased with ``rte_memseg.iova_addr``. + * ``rte_mem_virt2phy()`` can often be replaced by ``rte_mem_virt2iova``. + * ``rte_malloc_virt2phy`` is aliased with ``rte_malloc_virt2iova``. + * ``rte_memzone.phys_addr`` is aliased with ``rte_memzone.iova``. + * ``rte_mempool_objhdr.physaddr`` is aliased with ``rte_mempool_objhdr.iova``. + * ``rte_mempool_memhdr.phys_addr`` is aliased with ``rte_mempool_memhdr.iova``. + * ``rte_mempool_virt2phy()`` can be replaced by ``rte_mempool_virt2iova()``. + * ``rte_mempool_populate_phys*()`` are aliased with ``rte_mempool_populate_iova*()`` + * ``rte_mbuf.buf_physaddr`` is aliased with ``rte_mbuf.buf_iova``. + * ``rte_mbuf_data_dma_addr*()`` are aliased with ``rte_mbuf_data_iova*()``. + * ``rte_pktmbuf_mtophys*`` are aliased with ``rte_pktmbuf_iova*()``. + +* **PCI bus API moved outside of the EAL** + + The PCI bus previously implemented within the EAL has been moved. + A first part has been added as an RTE library providing PCI helpers to + parse device locations or other such utilities. + A second part consisting in the actual bus driver has been moved to its + proper subdirectory, without changing its functionalities. + + As such, several PCI-related functions are not proposed by the EAL anymore: + + * rte_pci_detach + * rte_pci_dump + * rte_pci_ioport_map + * rte_pci_ioport_read + * rte_pci_ioport_unmap + * rte_pci_ioport_write + * rte_pci_map_device + * rte_pci_probe + * rte_pci_probe_one + * rte_pci_read_config + * rte_pci_register + * rte_pci_scan + * rte_pci_unmap_device + * rte_pci_unregister + * rte_pci_write_config + + These functions are made available either as part of ``librte_pci`` or + ``librte_bus_pci``. + +* **Moved vdev bus APIs outside of the EAL** + + Moved the following APIs from ``librte_eal`` to ``librte_bus_vdev``: + * ``rte_vdev_init`` + * ``rte_vdev_register`` + * ``rte_vdev_uninit`` + * ``rte_vdev_unregister`` + +* **Add return value to stats_get dev op API** + + The ``stats_get`` dev op API return value has been changed to be int. + By this way PMDs can return an error value in case of failure at stats + getting process time. + +* **Modified the rte_cryptodev_allocate_driver function in the cryptodev library.** + + The function ``rte_cryptodev_allocate_driver()`` has been modified. + An extra parameter ``struct cryptodev_driver *crypto_drv`` has been added. + +* **Removed virtual device bus specific functions from librte_cryptodev.** + + The functions ``rte_cryptodev_vdev_parse_init_params()`` and + ``rte_cryptodev_vdev_pmd_init()`` have been removed from librte_cryptodev + and have been replaced by non bus specific functions + ``rte_cryptodev_pmd_parse_input_args()`` and ``rte_cryptodev_pmd_create()``. + +* ``rte_cryptodev_create_vdev`` was removed to avoid the dependency on vdev + in librte_cryptodev; instead, users can call rte_vdev_init() directly. + +* **Removed PCI device bus specific functions from librte_cryptodev.** + + The functions ``rte_cryptodev_pci_generic_probe()`` and + ``rte_cryptodev_pci_generic_remove()`` have been removed from librte_cryptodev + and have been replaced by non bus specific functions + ``rte_cryptodev_pmd_create()`` and ``rte_cryptodev_pmd_destroy()``. + +* **Removed deprecated functions to manage log level or type.** + + The functions ``rte_set_log_level()``, ``rte_get_log_level()``, + ``rte_set_log_type()`` and ``rte_get_log_type()`` have been removed. + They are respectively replaced by ``rte_log_set_global_level()``, + ``rte_log_get_global_level()``, ``rte_log_set_level()`` and + ``rte_log_get_level()``. + +* **Removed ``mbuf`` flags ``PKT_RX_VLAN_PKT`` and ``PKT_RX_QINQ_PKT``.** + + The ``mbuf`` flags ``PKT_RX_VLAN_PKT`` and ``PKT_RX_QINQ_PKT`` have + been removed since their behavior were not properly described. + +* **Added ``mbuf`` flags ``PKT_RX_VLAN`` and ``PKT_RX_QINQ``.** + + Two ``mbuf`` flags have been added to indicate that the VLAN + identifier has been saved in in the ``mbuf`` structure. For instance: + + - if VLAN is not stripped and TCI is saved: ``PKT_RX_VLAN`` + - if VLAN is stripped and TCI is saved: ``PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED`` + +* **Modified the vlan_offload_set_t function prototype in the ethdev library.** + + Changed the function prototype of ``vlan_offload_set_t``. The return value + has been changed from ``void`` to ``int`` so the caller to knows whether + the backing device supports the operation or if the operation was + successfully performed. + + +ABI Changes +----------- + +.. This section should contain ABI changes. Sample format: + + * Add a short 1-2 sentence description of the ABI change that was announced + in the previous releases and made in this release. Use fixed width quotes + for ``rte_function_names`` or ``rte_struct_names``. Use the past tense. + + This section is a comment. do not overwrite or remove it. + Also, make sure to start the actual text at the margin. + ========================================================= + +* **Extended port_id range.** + + The size of the field ``port_id`` in the ``rte_eth_dev_data`` structure + changed, as described in the `New Features` section. + +* **New parameter added to rte_eth_dev.** + + New parameter ``security_ctx`` added to ``rte_eth_dev`` to support security + operations like IPSec inline. + +* **New parameter added to rte_cryptodev.** + + New parameter ``security_ctx`` added to ``rte_cryptodev`` to support security + operations like lookaside crypto. + + +Removed Items +------------- + +.. This section should contain removed items in this release. Sample format: + + * Add a short 1-2 sentence description of the removed item in the past + tense. + + This section is a comment. do not overwrite or remove it. + Also, make sure to start the actual text at the margin. + ========================================================= + +* Xen dom0 in EAL was removed, as well as xenvirt PMD and vhost_xen. + +* The crypto performance unit tests have been removed, + replaced by the dpdk-test-crypto-perf application. + + +Shared Library Versions +----------------------- + +.. Update any library version updated in this release and prepend with a ``+`` + sign, like this: + + librte_acl.so.2 + + librte_cfgfile.so.2 + librte_cmdline.so.2 + + This section is a comment. do not overwrite or remove it. + ========================================================= + + +The libraries prepended with a plus sign were incremented in this version. + +.. code-block:: diff + + librte_acl.so.2 + + librte_bitratestats.so.2 + + librte_bus_vdev.so.1 + librte_cfgfile.so.2 + librte_cmdline.so.2 + + librte_cryptodev.so.4 + librte_distributor.so.1 + + librte_eal.so.6 + + librte_ethdev.so.8 + + librte_eventdev.so.3 + + librte_flow_classify.so.1 + librte_gro.so.1 + + librte_gso.so.1 + librte_hash.so.2 + librte_ip_frag.so.1 + librte_jobstats.so.1 + librte_kni.so.2 + librte_kvargs.so.1 + librte_latencystats.so.1 + librte_lpm.so.2 + librte_mbuf.so.3 + + librte_mempool.so.3 + librte_meter.so.1 + librte_metrics.so.1 + librte_net.so.1 + + librte_pci.so.1 + + librte_pdump.so.2 + librte_pipeline.so.3 + + librte_pmd_bnxt.so.2 + + librte_pmd_bond.so.2 + + librte_pmd_i40e.so.2 + + librte_pmd_ixgbe.so.2 + librte_pmd_ring.so.2 + + librte_pmd_softnic.so.1 + + librte_pmd_vhost.so.2 + librte_port.so.3 + librte_power.so.1 + librte_reorder.so.1 + librte_ring.so.1 + librte_sched.so.1 + + librte_security.so.1 + + librte_table.so.3 + librte_timer.so.1 + librte_vhost.so.3 + + +Tested Platforms +---------------- + +.. This section should contain a list of platforms that were tested with this + release. + + The format is: + + * <vendor> platform with <vendor> <type of devices> combinations + + * List of CPU + * List of OS + * List of devices + * Other relevant details... + + This section is a comment. do not overwrite or remove it. + Also, make sure to start the actual text at the margin. + ========================================================= diff --git a/doc/guides/sample_app_ug/cmd_line.rst b/doc/guides/sample_app_ug/cmd_line.rst index 36c7971c..eabbac15 100644 --- a/doc/guides/sample_app_ug/cmd_line.rst +++ b/doc/guides/sample_app_ug/cmd_line.rst @@ -68,26 +68,9 @@ There are three simple commands: Compiling the Application ------------------------- -#. Go to example directory: +To compile the sample application see :doc:`compiling` - .. code-block:: console - - export RTE_SDK=/path/to/rte_sdk - cd ${RTE_SDK}/examples/cmdline - -#. Set the target (a default target is used if not specified). For example: - - .. code-block:: console - - export RTE_TARGET=x86_64-native-linuxapp-gcc - - Refer to the *DPDK Getting Started Guide* for possible RTE_TARGET values. - -#. Build the application: - - .. code-block:: console - - make +The application is located in the ``cmd_line`` sub-directory. Running the Application ----------------------- diff --git a/doc/guides/sample_app_ug/compiling.rst b/doc/guides/sample_app_ug/compiling.rst new file mode 100644 index 00000000..8bedaa79 --- /dev/null +++ b/doc/guides/sample_app_ug/compiling.rst @@ -0,0 +1,122 @@ + .. BSD LICENSE + Copyright(c) 2015 Intel Corporation. All rights reserved. + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of Intel Corporation nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +Compiling the Sample Applications +================================= + +This section explains how to compile the DPDK sample applications. + +To compile all the sample applications +-------------------------------------- + + +Set the path to DPDK source code if its not set: + + .. code-block:: console + + export RTE_SDK=/path/to/rte_sdk + +Go to DPDK source: + + .. code-block:: console + + cd $RTE_SDK + +Build DPDK: + + .. code-block:: console + + make defconfig + make + +Build the sample applications: + + .. code-block:: console + + export RTE_TARGET=build + make -C examples + +For other possible ``RTE_TARGET`` values and additional information on +compiling see +:ref:`Compiling DPDK on Linux <linux_gsg_compiling_dpdk>` or +:ref:`Compiling DPDK on FreeBSD <building_from_source>`. +Applications are output to: ``$RTE_SDK/examples/app-dir/build`` or +``$RTE_SDK/examples/app-dir/$RTE_TARGET``. + + +In the example above the compiled application is written to the ``build`` subdirectory. +To have the applications written to a different location, +the ``O=/path/to/build/directory`` option may be specified in the make command. + + .. code-block:: console + + make O=/tmp + +To build the applications for debugging use the ``DEBUG`` option. +This option adds some extra flags, disables compiler optimizations and +sets verbose output. + + .. code-block:: console + + make DEBUG=1 + + +To compile a single application +------------------------------- + +Set the path to DPDK source code: + + .. code-block:: console + + export RTE_SDK=/path/to/rte_sdk + +Go to DPDK source: + + .. code-block:: console + + cd $RTE_SDK + +Build DPDK: + + .. code-block:: console + + make defconfig + make + +Go to the sample application directory. Unless otherwise specified the sample +applications are located in ``$RTE_SDK/examples/``. + + +Build the application: + + .. code-block:: console + + export RTE_TARGET=build + make diff --git a/doc/guides/sample_app_ug/dist_app.rst b/doc/guides/sample_app_ug/dist_app.rst index 1cae4739..45a51313 100644 --- a/doc/guides/sample_app_ug/dist_app.rst +++ b/doc/guides/sample_app_ug/dist_app.rst @@ -53,30 +53,12 @@ generator as shown in the figure below. Performance Benchmarking Setup (Basic Environment) - Compiling the Application ------------------------- -#. Go to the sample application directory: - - .. code-block:: console - - export RTE_SDK=/path/to/rte_sdk - cd ${RTE_SDK}/examples/distributor - -#. Set the target (a default target is used if not specified). For example: - - .. code-block:: console - - export RTE_TARGET=x86_64-native-linuxapp-gcc - - See the DPDK Getting Started Guide for possible RTE_TARGET values. - -#. Build the application: - - .. code-block:: console +To compile the sample application see :doc:`compiling`. - make +The application is located in the ``distributor`` sub-directory. Running the Application ----------------------- diff --git a/doc/guides/sample_app_ug/ethtool.rst b/doc/guides/sample_app_ug/ethtool.rst index 67797954..6dd11dc9 100644 --- a/doc/guides/sample_app_ug/ethtool.rst +++ b/doc/guides/sample_app_ug/ethtool.rst @@ -40,28 +40,9 @@ is based upon a simple L2 frame reflector. Compiling the Application ------------------------- -To compile the application: +To compile the sample application see :doc:`compiling`. -#. Go to the sample application directory: - - .. code-block:: console - - export RTE_SDK=/path/to/rte_sdk - cd ${RTE_SDK}/examples/ethtool - -#. Set the target (a default target is used if not specified). For example: - - .. code-block:: console - - export RTE_TARGET=x86_64-native-linuxapp-gcc - - See the *DPDK Getting Started Guide* for possible RTE_TARGET values. - -#. Build the application: - - .. code-block:: console - - make +The application is located in the ``ethtool`` sub-directory. Running the Application ----------------------- diff --git a/doc/guides/sample_app_ug/eventdev_pipeline_sw_pmd.rst b/doc/guides/sample_app_ug/eventdev_pipeline_sw_pmd.rst index b1b18dd0..01a5f9b2 100644 --- a/doc/guides/sample_app_ug/eventdev_pipeline_sw_pmd.rst +++ b/doc/guides/sample_app_ug/eventdev_pipeline_sw_pmd.rst @@ -46,28 +46,11 @@ a particular pipeline configuration. Compiling the Application ------------------------- -To compile the application: +To compile the sample application see :doc:`compiling`. -#. Go to the sample application directory: +The application is located in the ``examples`` sub-directory. - .. code-block:: console - export RTE_SDK=/path/to/rte_sdk - cd ${RTE_SDK}/examples/eventdev_pipeline_sw_pmd - -#. Set the target (a default target is used if not specified). For example: - - .. code-block:: console - - export RTE_TARGET=x86_64-native-linuxapp-gcc - - See the *DPDK Getting Started Guide* for possible RTE_TARGET values. - -#. Build the application: - - .. code-block:: console - - make Running the Application ----------------------- diff --git a/doc/guides/sample_app_ug/exception_path.rst b/doc/guides/sample_app_ug/exception_path.rst index e505fb32..d7f45e6d 100644 --- a/doc/guides/sample_app_ug/exception_path.rst +++ b/doc/guides/sample_app_ug/exception_path.rst @@ -58,28 +58,9 @@ To make throughput measurements, kernel bridges must be setup to forward data be Compiling the Application ------------------------- -#. Go to example directory: +To compile the sample application see :doc:`compiling`. - .. code-block:: console - - export RTE_SDK=/path/to/rte_sdk - cd ${RTE_SDK}/examples/exception_path - -#. Set the target (a default target will be used if not specified). - For example: - - .. code-block:: console - - export RTE_TARGET=x86_64-native-linuxapp-gcc - -This application is intended as a linuxapp only. -See the *DPDK Getting Started Guide* for possible RTE_TARGET values. - -#. Build the application: - - .. code-block:: console - - make +The application is located in the ``exception_path`` sub-directory. Running the Application ----------------------- diff --git a/doc/guides/sample_app_ug/flow_classify.rst b/doc/guides/sample_app_ug/flow_classify.rst new file mode 100644 index 00000000..bc12b87d --- /dev/null +++ b/doc/guides/sample_app_ug/flow_classify.rst @@ -0,0 +1,575 @@ +.. BSD LICENSE + Copyright(c) 2017 Intel Corporation. All rights reserved. + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of Intel Corporation nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +Flow Classify Sample Application +================================ + +The Flow Classify sample application is based on the simple *skeleton* example +of a forwarding application. + +It is intended as a demonstration of the basic components of a DPDK forwarding +application which uses the Flow Classify library API's. + +Please refer to the +:doc:`../prog_guide/flow_classify_lib` +for more information. + +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``flow_classify`` sub-directory. + +Running the Application +----------------------- + +To run the example in a ``linuxapp`` environment: + +.. code-block:: console + + cd ~/dpdk/examples/flow_classify + ./build/flow_classify -c 4 -n 4 -- --rule_ipv4="../ipv4_rules_file.txt" + +Please refer to the *DPDK Getting Started Guide*, section +:doc:`../linux_gsg/build_sample_apps` +for general information on running applications and the Environment Abstraction +Layer (EAL) options. + + +Sample ipv4_rules_file.txt +-------------------------- + +.. code-block:: console + + #file format: + #src_ip/masklen dst_ip/masklen src_port : mask dst_port : mask proto/mask priority + # + 2.2.2.3/24 2.2.2.7/24 32 : 0xffff 33 : 0xffff 17/0xff 0 + 9.9.9.3/24 9.9.9.7/24 32 : 0xffff 33 : 0xffff 17/0xff 1 + 9.9.9.3/24 9.9.9.7/24 32 : 0xffff 33 : 0xffff 6/0xff 2 + 9.9.8.3/24 9.9.8.7/24 32 : 0xffff 33 : 0xffff 6/0xff 3 + 6.7.8.9/24 2.3.4.5/24 32 : 0x0000 33 : 0x0000 132/0xff 4 + +Explanation +----------- + +The following sections provide an explanation of the main components of the +code. + +All DPDK library functions used in the sample code are prefixed with ``rte_`` +and are explained in detail in the *DPDK API Documentation*. + +ACL field definitions for the IPv4 5 tuple rule +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The following field definitions are used when creating the ACL table during +initialisation of the ``Flow Classify`` application.. + +.. code-block:: c + + enum { + PROTO_FIELD_IPV4, + SRC_FIELD_IPV4, + DST_FIELD_IPV4, + SRCP_FIELD_IPV4, + DSTP_FIELD_IPV4, + NUM_FIELDS_IPV4 + }; + + enum { + PROTO_INPUT_IPV4, + SRC_INPUT_IPV4, + DST_INPUT_IPV4, + SRCP_DESTP_INPUT_IPV4 + }; + + static struct rte_acl_field_def ipv4_defs[NUM_FIELDS_IPV4] = { + /* first input field - always one byte long. */ + { + .type = RTE_ACL_FIELD_TYPE_BITMASK, + .size = sizeof(uint8_t), + .field_index = PROTO_FIELD_IPV4, + .input_index = PROTO_INPUT_IPV4, + .offset = sizeof(struct ether_hdr) + + offsetof(struct ipv4_hdr, next_proto_id), + }, + /* next input field (IPv4 source address) - 4 consecutive bytes. */ + { + /* rte_flow uses a bit mask for IPv4 addresses */ + .type = RTE_ACL_FIELD_TYPE_BITMASK, + .size = sizeof(uint32_t), + .field_index = SRC_FIELD_IPV4, + .input_index = SRC_INPUT_IPV4, + .offset = sizeof(struct ether_hdr) + + offsetof(struct ipv4_hdr, src_addr), + }, + /* next input field (IPv4 destination address) - 4 consecutive bytes. */ + { + /* rte_flow uses a bit mask for IPv4 addresses */ + .type = RTE_ACL_FIELD_TYPE_BITMASK, + .size = sizeof(uint32_t), + .field_index = DST_FIELD_IPV4, + .input_index = DST_INPUT_IPV4, + .offset = sizeof(struct ether_hdr) + + offsetof(struct ipv4_hdr, dst_addr), + }, + /* + * Next 2 fields (src & dst ports) form 4 consecutive bytes. + * They share the same input index. + */ + { + /* rte_flow uses a bit mask for protocol ports */ + .type = RTE_ACL_FIELD_TYPE_BITMASK, + .size = sizeof(uint16_t), + .field_index = SRCP_FIELD_IPV4, + .input_index = SRCP_DESTP_INPUT_IPV4, + .offset = sizeof(struct ether_hdr) + + sizeof(struct ipv4_hdr) + + offsetof(struct tcp_hdr, src_port), + }, + { + /* rte_flow uses a bit mask for protocol ports */ + .type = RTE_ACL_FIELD_TYPE_BITMASK, + .size = sizeof(uint16_t), + .field_index = DSTP_FIELD_IPV4, + .input_index = SRCP_DESTP_INPUT_IPV4, + .offset = sizeof(struct ether_hdr) + + sizeof(struct ipv4_hdr) + + offsetof(struct tcp_hdr, dst_port), + }, + }; + +The Main Function +~~~~~~~~~~~~~~~~~ + +The ``main()`` function performs the initialization and calls the execution +threads for each lcore. + +The first task is to initialize the Environment Abstraction Layer (EAL). +The ``argc`` and ``argv`` arguments are provided to the ``rte_eal_init()`` +function. The value returned is the number of parsed arguments: + +.. code-block:: c + + int ret = rte_eal_init(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "Error with EAL initialization\n"); + +It then parses the flow_classify application arguments + +.. code-block:: c + + ret = parse_args(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "Invalid flow_classify parameters\n"); + +The ``main()`` function also allocates a mempool to hold the mbufs +(Message Buffers) used by the application: + +.. code-block:: c + + mbuf_pool = rte_mempool_create("MBUF_POOL", + NUM_MBUFS * nb_ports, + MBUF_SIZE, + MBUF_CACHE_SIZE, + sizeof(struct rte_pktmbuf_pool_private), + rte_pktmbuf_pool_init, NULL, + rte_pktmbuf_init, NULL, + rte_socket_id(), + 0); + +mbufs are the packet buffer structure used by DPDK. They are explained in +detail in the "Mbuf Library" section of the *DPDK Programmer's Guide*. + +The ``main()`` function also initializes all the ports using the user defined +``port_init()`` function which is explained in the next section: + +.. code-block:: c + + for (portid = 0; portid < nb_ports; portid++) { + if (port_init(portid, mbuf_pool) != 0) { + rte_exit(EXIT_FAILURE, + "Cannot init port %" PRIu8 "\n", portid); + } + } + +The ``main()`` function creates the ``flow classifier object`` and adds an ``ACL +table`` to the flow classifier. + +.. code-block:: c + + struct flow_classifier { + struct rte_flow_classifier *cls; + uint32_t table_id[RTE_FLOW_CLASSIFY_TABLE_MAX]; + }; + + struct flow_classifier_acl { + struct flow_classifier cls; + } __rte_cache_aligned; + + /* Memory allocation */ + size = RTE_CACHE_LINE_ROUNDUP(sizeof(struct flow_classifier_acl)); + cls_app = rte_zmalloc(NULL, size, RTE_CACHE_LINE_SIZE); + if (cls_app == NULL) + rte_exit(EXIT_FAILURE, "Cannot allocate classifier memory\n"); + + cls_params.name = "flow_classifier"; + cls_params.socket_id = socket_id; + cls_params.type = RTE_FLOW_CLASSIFY_TABLE_TYPE_ACL; + + cls_app->cls = rte_flow_classifier_create(&cls_params); + if (cls_app->cls == NULL) { + rte_free(cls_app); + rte_exit(EXIT_FAILURE, "Cannot create classifier\n"); + } + + /* initialise ACL table params */ + table_acl_params.name = "table_acl_ipv4_5tuple"; + table_acl_params.n_rule_fields = RTE_DIM(ipv4_defs); + table_acl_params.n_rules = FLOW_CLASSIFY_MAX_RULE_NUM; + memcpy(table_acl_params.field_format, ipv4_defs, sizeof(ipv4_defs)); + + /* initialise table create params */ + cls_table_params.ops = &rte_table_acl_ops, + cls_table_params.arg_create = &table_acl_params, + cls_table_params.table_metadata_size = 0; + + ret = rte_flow_classify_table_create(cls_app->cls, &cls_table_params, + &cls->table_id[0]); + if (ret) { + rte_flow_classifier_free(cls_app->cls); + rte_free(cls); + rte_exit(EXIT_FAILURE, "Failed to create classifier table\n"); + } + +It then reads the ipv4_rules_file.txt file and initialises the parameters for +the ``rte_flow_classify_table_entry_add`` API. +This API adds a rule to the ACL table. + +.. code-block:: c + + if (add_rules(parm_config.rule_ipv4_name)) { + rte_flow_classifier_free(cls_app->cls); + rte_free(cls_app); + rte_exit(EXIT_FAILURE, "Failed to add rules\n"); + } + +Once the initialization is complete, the application is ready to launch a +function on an lcore. In this example ``lcore_main()`` is called on a single +lcore. + +.. code-block:: c + + lcore_main(cls_app); + +The ``lcore_main()`` function is explained below. + +The Port Initialization Function +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The main functional part of the port initialization used in the Basic +Forwarding application is shown below: + +.. code-block:: c + + static inline int + port_init(uint8_t port, struct rte_mempool *mbuf_pool) + { + struct rte_eth_conf port_conf = port_conf_default; + const uint16_t rx_rings = 1, tx_rings = 1; + struct ether_addr addr; + int retval; + uint16_t q; + + if (port >= rte_eth_dev_count()) + return -1; + + /* Configure the Ethernet device. */ + retval = rte_eth_dev_configure(port, rx_rings, tx_rings, &port_conf); + if (retval != 0) + return retval; + + /* Allocate and set up 1 RX queue per Ethernet port. */ + for (q = 0; q < rx_rings; q++) { + retval = rte_eth_rx_queue_setup(port, q, RX_RING_SIZE, + rte_eth_dev_socket_id(port), NULL, mbuf_pool); + if (retval < 0) + return retval; + } + + /* Allocate and set up 1 TX queue per Ethernet port. */ + for (q = 0; q < tx_rings; q++) { + retval = rte_eth_tx_queue_setup(port, q, TX_RING_SIZE, + rte_eth_dev_socket_id(port), NULL); + if (retval < 0) + return retval; + } + + /* Start the Ethernet port. */ + retval = rte_eth_dev_start(port); + if (retval < 0) + return retval; + + /* Display the port MAC address. */ + rte_eth_macaddr_get(port, &addr); + printf("Port %u MAC: %02" PRIx8 " %02" PRIx8 " %02" PRIx8 + " %02" PRIx8 " %02" PRIx8 " %02" PRIx8 "\n", + port, + addr.addr_bytes[0], addr.addr_bytes[1], + addr.addr_bytes[2], addr.addr_bytes[3], + addr.addr_bytes[4], addr.addr_bytes[5]); + + /* Enable RX in promiscuous mode for the Ethernet device. */ + rte_eth_promiscuous_enable(port); + + return 0; + } + +The Ethernet ports are configured with default settings using the +``rte_eth_dev_configure()`` function and the ``port_conf_default`` struct. + +.. code-block:: c + + static const struct rte_eth_conf port_conf_default = { + .rxmode = { .max_rx_pkt_len = ETHER_MAX_LEN } + }; + +For this example the ports are set up with 1 RX and 1 TX queue using the +``rte_eth_rx_queue_setup()`` and ``rte_eth_tx_queue_setup()`` functions. + +The Ethernet port is then started: + +.. code-block:: c + + retval = rte_eth_dev_start(port); + + +Finally the RX port is set in promiscuous mode: + +.. code-block:: c + + rte_eth_promiscuous_enable(port); + +The Add Rules function +~~~~~~~~~~~~~~~~~~~~~~ + +The ``add_rules`` function reads the ``ipv4_rules_file.txt`` file and calls the +``add_classify_rule`` function which calls the +``rte_flow_classify_table_entry_add`` API. + +.. code-block:: c + + static int + add_rules(const char *rule_path) + { + FILE *fh; + char buff[LINE_MAX]; + unsigned int i = 0; + unsigned int total_num = 0; + struct rte_eth_ntuple_filter ntuple_filter; + + fh = fopen(rule_path, "rb"); + if (fh == NULL) + rte_exit(EXIT_FAILURE, "%s: Open %s failed\n", __func__, + rule_path); + + fseek(fh, 0, SEEK_SET); + + i = 0; + while (fgets(buff, LINE_MAX, fh) != NULL) { + i++; + + if (is_bypass_line(buff)) + continue; + + if (total_num >= FLOW_CLASSIFY_MAX_RULE_NUM - 1) { + printf("\nINFO: classify rule capacity %d reached\n", + total_num); + break; + } + + if (parse_ipv4_5tuple_rule(buff, &ntuple_filter) != 0) + rte_exit(EXIT_FAILURE, + "%s Line %u: parse rules error\n", + rule_path, i); + + if (add_classify_rule(&ntuple_filter) != 0) + rte_exit(EXIT_FAILURE, "add rule error\n"); + + total_num++; + } + + fclose(fh); + return 0; + } + + +The Lcore Main function +~~~~~~~~~~~~~~~~~~~~~~~ + +As we saw above the ``main()`` function calls an application function on the +available lcores. +The ``lcore_main`` function calls the ``rte_flow_classifier_query`` API. +For the Basic Forwarding application the ``lcore_main`` function looks like the +following: + +.. code-block:: c + + /* flow classify data */ + static int num_classify_rules; + static struct rte_flow_classify_rule *rules[MAX_NUM_CLASSIFY]; + static struct rte_flow_classify_ipv4_5tuple_stats ntuple_stats; + static struct rte_flow_classify_stats classify_stats = { + .stats = (void *)&ntuple_stats + }; + + static __attribute__((noreturn)) void + lcore_main(cls_app) + { + const uint8_t nb_ports = rte_eth_dev_count(); + uint8_t port; + + /* + * Check that the port is on the same NUMA node as the polling thread + * for best performance. + */ + for (port = 0; port < nb_ports; port++) + if (rte_eth_dev_socket_id(port) > 0 && + rte_eth_dev_socket_id(port) != (int)rte_socket_id()) { + printf("\n\n"); + printf("WARNING: port %u is on remote NUMA node\n", + port); + printf("to polling thread.\n"); + printf("Performance will not be optimal.\n"); + + printf("\nCore %u forwarding packets. \n", + rte_lcore_id()); + printf("[Ctrl+C to quit]\n + } + + /* Run until the application is quit or killed. */ + for (;;) { + /* + * Receive packets on a port and forward them on the paired + * port. The mapping is 0 -> 1, 1 -> 0, 2 -> 3, 3 -> 2, etc. + */ + for (port = 0; port < nb_ports; port++) { + + /* Get burst of RX packets, from first port of pair. */ + struct rte_mbuf *bufs[BURST_SIZE]; + const uint16_t nb_rx = rte_eth_rx_burst(port, 0, + bufs, BURST_SIZE); + + if (unlikely(nb_rx == 0)) + continue; + + for (i = 0; i < MAX_NUM_CLASSIFY; i++) { + if (rules[i]) { + ret = rte_flow_classifier_query( + cls_app->cls, + cls_app->table_id[0], + bufs, nb_rx, rules[i], + &classify_stats); + if (ret) + printf( + "rule [%d] query failed ret [%d]\n\n", + i, ret); + else { + printf( + "rule[%d] count=%"PRIu64"\n", + i, ntuple_stats.counter1); + + printf("proto = %d\n", + ntuple_stats.ipv4_5tuple.proto); + } + } + } + + /* Send burst of TX packets, to second port of pair. */ + const uint16_t nb_tx = rte_eth_tx_burst(port ^ 1, 0, + bufs, nb_rx); + + /* Free any unsent packets. */ + if (unlikely(nb_tx < nb_rx)) { + uint16_t buf; + for (buf = nb_tx; buf < nb_rx; buf++) + rte_pktmbuf_free(bufs[buf]); + } + } + } + } + +The main work of the application is done within the loop: + +.. code-block:: c + + for (;;) { + for (port = 0; port < nb_ports; port++) { + + /* Get burst of RX packets, from first port of pair. */ + struct rte_mbuf *bufs[BURST_SIZE]; + const uint16_t nb_rx = rte_eth_rx_burst(port, 0, + bufs, BURST_SIZE); + + if (unlikely(nb_rx == 0)) + continue; + + /* Send burst of TX packets, to second port of pair. */ + const uint16_t nb_tx = rte_eth_tx_burst(port ^ 1, 0, + bufs, nb_rx); + + /* Free any unsent packets. */ + if (unlikely(nb_tx < nb_rx)) { + uint16_t buf; + for (buf = nb_tx; buf < nb_rx; buf++) + rte_pktmbuf_free(bufs[buf]); + } + } + } + +Packets are received in bursts on the RX ports and transmitted in bursts on +the TX ports. The ports are grouped in pairs with a simple mapping scheme +using the an XOR on the port number:: + + 0 -> 1 + 1 -> 0 + + 2 -> 3 + 3 -> 2 + + etc. + +The ``rte_eth_tx_burst()`` function frees the memory buffers of packets that +are transmitted. If packets fail to transmit, ``(nb_tx < nb_rx)``, then they +must be freed explicitly using ``rte_pktmbuf_free()``. + +The forwarding loop can be interrupted and the application closed using +``Ctrl-C``. diff --git a/doc/guides/sample_app_ug/flow_filtering.rst b/doc/guides/sample_app_ug/flow_filtering.rst new file mode 100644 index 00000000..725dcb44 --- /dev/null +++ b/doc/guides/sample_app_ug/flow_filtering.rst @@ -0,0 +1,545 @@ +.. BSD LICENSE + Copyright(c) 2017 Mellanox Corporation. All rights reserved. + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of Mellanox Corporation nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + +Basic RTE Flow Filtering Sample Application +=========================================== + +The Basic RTE flow filtering sample application is a simple example of a +creating a RTE flow rule. + +It is intended as a demonstration of the basic components RTE flow rules. + + +Compiling the Application +------------------------- + +To compile the application export the path to the DPDK source tree and go to +the example directory: + +.. code-block:: console + + export RTE_SDK=/path/to/rte_sdk + + cd ${RTE_SDK}/examples/flow_filtering + +Set the target, for example: + +.. code-block:: console + + export RTE_TARGET=x86_64-native-linuxapp-gcc + +See the *DPDK Getting Started* Guide for possible ``RTE_TARGET`` values. + +Build the application as follows: + +.. code-block:: console + + make + + +Running the Application +----------------------- + +To run the example in a ``linuxapp`` environment: + +.. code-block:: console + + ./build/flow -l 1 -n 1 + +Refer to *DPDK Getting Started Guide* for general information on running +applications and the Environment Abstraction Layer (EAL) options. + + +Explanation +----------- + +The example is build from 2 main files, +``main.c`` which holds the example logic and ``flow_blocks.c`` that holds the +implementation for building the flow rule. + +The following sections provide an explanation of the main components of the +code. + +All DPDK library functions used in the sample code are prefixed with ``rte_`` +and are explained in detail in the *DPDK API Documentation*. + + +The Main Function +~~~~~~~~~~~~~~~~~ + +The ``main()`` function located in ``main.c`` file performs the initialization +and runs the main loop function. + +The first task is to initialize the Environment Abstraction Layer (EAL). The +``argc`` and ``argv`` arguments are provided to the ``rte_eal_init()`` +function. The value returned is the number of parsed arguments: + +.. code-block:: c + + int ret = rte_eal_init(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "Error with EAL initialization\n"); + + +The ``main()`` also allocates a mempool to hold the mbufs (Message Buffers) +used by the application: + +.. code-block:: c + + mbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", 4096, 128, 0, + RTE_MBUF_DEFAULT_BUF_SIZE, + rte_socket_id()); + +Mbufs are the packet buffer structure used by DPDK. They are explained in +detail in the "Mbuf Library" section of the *DPDK Programmer's Guide*. + +The ``main()`` function also initializes all the ports using the user defined +``init_port()`` function which is explained in the next section: + +.. code-block:: c + + init_port(); + +Once the initialization is complete, we set the flow rule using the +following code: + +.. code-block:: c + + /* create flow for send packet with */ + flow = generate_ipv4_flow(port_id, selected_queue, + SRC_IP, EMPTY_MASK, + DEST_IP, FULL_MASK, &error); + if (!flow) { + printf("Flow can't be created %d message: %s\n", + error.type, + error.message ? error.message : "(no stated reason)"); + rte_exit(EXIT_FAILURE, "error in creating flow"); + } + +In the last part the application is ready to launch the +``main_loop()`` function. Which is explained below. + + +.. code-block:: c + + main_loop(); + +The Port Initialization Function +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The main functional part of the port initialization used in the flow filtering +application is shown below: + +.. code-block:: c + + init_port(void) + { + int ret; + uint16_t i; + struct rte_eth_conf port_conf = { + .rxmode = { + .split_hdr_size = 0, + /**< Header Split disabled */ + .header_split = 0, + /**< IP checksum offload disabled */ + .hw_ip_checksum = 0, + /**< VLAN filtering disabled */ + .hw_vlan_filter = 0, + /**< Jumbo Frame Support disabled */ + .jumbo_frame = 0, + /**< CRC stripped by hardware */ + .hw_strip_crc = 1, + }, + }; + + printf(":: initializing port: %d\n", port_id); + ret = rte_eth_dev_configure(port_id, + nr_queues, nr_queues, &port_conf); + if (ret < 0) { + rte_exit(EXIT_FAILURE, + ":: cannot configure device: err=%d, port=%u\n", + ret, port_id); + } + + /* only set Rx queues: something we care only so far */ + for (i = 0; i < nr_queues; i++) { + ret = rte_eth_rx_queue_setup(port_id, i, 512, + rte_eth_dev_socket_id(port_id), + NULL, + mbuf_pool); + if (ret < 0) { + rte_exit(EXIT_FAILURE, + ":: Rx queue setup failed: err=%d, port=%u\n", + ret, port_id); + } + } + + + rte_eth_promiscuous_enable(port_id); + + ret = rte_eth_dev_start(port_id); + if (ret < 0) { + rte_exit(EXIT_FAILURE, + "rte_eth_dev_start:err=%d, port=%u\n", + ret, port_id); + } + + assert_link_status(); + + printf(":: initializing port: %d done\n", port_id); + } + +The Ethernet port is configured with default settings using the +``rte_eth_dev_configure()`` function and the ``port_conf_default`` struct: + +.. code-block:: c + + struct rte_eth_conf port_conf = { + .rxmode = { + .split_hdr_size = 0, + /**< Header Split disabled */ + .header_split = 0, + /**< IP checksum offload disabled */ + .hw_ip_checksum = 0, + /**< VLAN filtering disabled */ + .hw_vlan_filter = 0, + /**< Jumbo Frame Support disabled */ + .jumbo_frame = 0, + /**< CRC stripped by hardware */ + .hw_strip_crc = 1, + }, + }; + + ret = rte_eth_dev_configure(port_id, nr_queues, nr_queues, &port_conf); + if (ret < 0) { + rte_exit(EXIT_FAILURE, + ":: cannot configure device: err=%d, port=%u\n", + ret, port_id); + } + +For this example we are configuring number of rx queues that are connected to +a single port. + +.. code-block:: c + + for (i = 0; i < nr_queues; i++) { + ret = rte_eth_rx_queue_setup(port_id, i, 512, + rte_eth_dev_socket_id(port_id), + NULL, + mbuf_pool); + if (ret < 0) { + rte_exit(EXIT_FAILURE, + ":: Rx queue setup failed: err=%d, port=%u\n", + ret, port_id); + } + } + +In the next step we create and apply the flow rule. which is to send packets +with destination ip equals to 192.168.1.1 to queue number 1. The detail +explanation of the ``generate_ipv4_flow()`` appears later in this document: + +.. code-block:: c + + flow = generate_ipv4_flow(port_id, selected_queue, + SRC_IP, EMPTY_MASK, + DEST_IP, FULL_MASK, &error); + +We are setting the RX port to promiscuous mode: + +.. code-block:: c + + rte_eth_promiscuous_enable(port_id); + +The last step is to start the port. + +.. code-block:: c + + ret = rte_eth_dev_start(port_id); + if (ret < 0) { + rte_exit(EXIT_FAILURE, "rte_eth_dev_start:err%d, port=%u\n", + ret, port_id); + } + + +The main_loop function +~~~~~~~~~~~~~~~~~~~~~~ + +As we saw above the ``main()`` function calls an application function to handle +the main loop. For the flow filtering application the main_loop function +looks like the following: + +.. code-block:: c + + static void + main_loop(void) + { + struct rte_mbuf *mbufs[32]; + struct ether_hdr *eth_hdr; + uint16_t nb_rx; + uint16_t i; + uint16_t j; + + while (!force_quit) { + for (i = 0; i < nr_queues; i++) { + nb_rx = rte_eth_rx_burst(port_id, + i, mbufs, 32); + if (nb_rx) { + for (j = 0; j < nb_rx; j++) { + struct rte_mbuf *m = mbufs[j]; + + eth_hdr = rte_pktmbuf_mtod(m, + struct ether_hdr *); + print_ether_addr("src=", + ð_hdr->s_addr); + print_ether_addr(" - dst=", + ð_hdr->d_addr); + printf(" - queue=0x%x", + (unsigned int)i); + printf("\n"); + rte_pktmbuf_free(m); + } + } + } + } + /* closing and releasing resources */ + rte_flow_flush(port_id, &error); + rte_eth_dev_stop(port_id); + rte_eth_dev_close(port_id); + } + +The main work of the application is reading the packets from all +queues and printing for each packet the destination queue: + +.. code-block:: c + + while (!force_quit) { + for (i = 0; i < nr_queues; i++) { + nb_rx = rte_eth_rx_burst(port_id, i, mbufs, 32); + if (nb_rx) { + for (j = 0; j < nb_rx; j++) { + struct rte_mbuf *m = mbufs[j]; + eth_hdr = rte_pktmbuf_mtod(m, struct ether_hdr *); + print_ether_addr("src=", ð_hdr->s_addr); + print_ether_addr(" - dst=", ð_hdr->d_addr); + printf(" - queue=0x%x", (unsigned int)i); + printf("\n"); + rte_pktmbuf_free(m); + } + } + } + } + + +The forwarding loop can be interrupted and the application closed using +``Ctrl-C``. Which results in closing the port and the device using +``rte_eth_dev_stop`` and ``rte_eth_dev_close`` + +The generate_ipv4_flow function +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The generate_ipv4_rule function is responsible for creating the flow rule. +This function is located in the ``flow_blocks.c`` file. + +.. code-block:: c + + static struct rte_flow * + generate_ipv4_flow(uint8_t port_id, uint16_t rx_q, + uint32_t src_ip, uint32_t src_mask, + uint32_t dest_ip, uint32_t dest_mask, + struct rte_flow_error *error) + { + struct rte_flow_attr attr; + struct rte_flow_item pattern[MAX_PATTERN_NUM]; + struct rte_flow_action action[MAX_PATTERN_NUM]; + struct rte_flow *flow = NULL; + struct rte_flow_action_queue queue = { .index = rx_q }; + struct rte_flow_item_eth eth_spec; + struct rte_flow_item_eth eth_mask; + struct rte_flow_item_vlan vlan_spec; + struct rte_flow_item_vlan vlan_mask; + struct rte_flow_item_ipv4 ip_spec; + struct rte_flow_item_ipv4 ip_mask; + + memset(pattern, 0, sizeof(pattern)); + memset(action, 0, sizeof(action)); + + /* + * set the rule attribute. + * in this case only ingress packets will be checked. + */ + memset(&attr, 0, sizeof(struct rte_flow_attr)); + attr.ingress = 1; + + /* + * create the action sequence. + * one action only, move packet to queue + */ + + action[0].type = RTE_FLOW_ACTION_TYPE_QUEUE; + action[0].conf = &queue; + action[1].type = RTE_FLOW_ACTION_TYPE_END; + + /* + * set the first level of the pattern (eth). + * since in this example we just want to get the + * ipv4 we set this level to allow all. + */ + memset(ð_spec, 0, sizeof(struct rte_flow_item_eth)); + memset(ð_mask, 0, sizeof(struct rte_flow_item_eth)); + eth_spec.type = 0; + eth_mask.type = 0; + pattern[0].type = RTE_FLOW_ITEM_TYPE_ETH; + pattern[0].spec = ð_spec; + pattern[0].mask = ð_mask; + + /* + * setting the second level of the pattern (vlan). + * since in this example we just want to get the + * ipv4 we also set this level to allow all. + */ + memset(&vlan_spec, 0, sizeof(struct rte_flow_item_vlan)); + memset(&vlan_mask, 0, sizeof(struct rte_flow_item_vlan)); + pattern[1].type = RTE_FLOW_ITEM_TYPE_VLAN; + pattern[1].spec = &vlan_spec; + pattern[1].mask = &vlan_mask; + + /* + * setting the third level of the pattern (ip). + * in this example this is the level we care about + * so we set it according to the parameters. + */ + memset(&ip_spec, 0, sizeof(struct rte_flow_item_ipv4)); + memset(&ip_mask, 0, sizeof(struct rte_flow_item_ipv4)); + ip_spec.hdr.dst_addr = htonl(dest_ip); + ip_mask.hdr.dst_addr = dest_mask; + ip_spec.hdr.src_addr = htonl(src_ip); + ip_mask.hdr.src_addr = src_mask; + pattern[2].type = RTE_FLOW_ITEM_TYPE_IPV4; + pattern[2].spec = &ip_spec; + pattern[2].mask = &ip_mask; + + /* the final level must be always type end */ + pattern[3].type = RTE_FLOW_ITEM_TYPE_END; + + int res = rte_flow_validate(port_id, &attr, pattern, action, error); + if(!res) + flow = rte_flow_create(port_id, &attr, pattern, action, error); + + return flow; + } + +The first part of the function is declaring the structures that will be used. + +.. code-block:: c + + struct rte_flow_attr attr; + struct rte_flow_item pattern[MAX_PATTERN_NUM]; + struct rte_flow_action action[MAX_PATTERN_NUM]; + struct rte_flow *flow; + struct rte_flow_error error; + struct rte_flow_action_queue queue = { .index = rx_q }; + struct rte_flow_item_eth eth_spec; + struct rte_flow_item_eth eth_mask; + struct rte_flow_item_vlan vlan_spec; + struct rte_flow_item_vlan vlan_mask; + struct rte_flow_item_ipv4 ip_spec; + struct rte_flow_item_ipv4 ip_mask; + +The following part create the flow attributes, in our case ingress. + +.. code-block:: c + + memset(&attr, 0, sizeof(struct rte_flow_attr)); + attr.ingress = 1; + +The third part defines the action to be taken when a packet matches +the rule. In this case send the packet to queue. + +.. code-block:: c + + action[0].type = RTE_FLOW_ACTION_TYPE_QUEUE; + action[0].conf = &queue; + action[1].type = RTE_FLOW_ACTION_TYPE_END; + +The forth part is responsible for creating the pattern and is build from +number of step. In each step we build one level of the pattern starting with +the lowest one. + +Setting the first level of the pattern ETH: + +.. code-block:: c + + memset(ð_spec, 0, sizeof(struct rte_flow_item_eth)); + memset(ð_mask, 0, sizeof(struct rte_flow_item_eth)); + eth_spec.type = 0; + eth_mask.type = 0; + pattern[0].type = RTE_FLOW_ITEM_TYPE_ETH; + pattern[0].spec = ð_spec; + pattern[0].mask = ð_mask; + +Setting the second level of the pattern VLAN: + +.. code-block:: c + + memset(&vlan_spec, 0, sizeof(struct rte_flow_item_vlan)); + memset(&vlan_mask, 0, sizeof(struct rte_flow_item_vlan)); + pattern[1].type = RTE_FLOW_ITEM_TYPE_VLAN; + pattern[1].spec = &vlan_spec; + pattern[1].mask = &vlan_mask; + +Setting the third level ip: + +.. code-block:: c + + memset(&ip_spec, 0, sizeof(struct rte_flow_item_ipv4)); + memset(&ip_mask, 0, sizeof(struct rte_flow_item_ipv4)); + ip_spec.hdr.dst_addr = htonl(dest_ip); + ip_mask.hdr.dst_addr = dest_mask; + ip_spec.hdr.src_addr = htonl(src_ip); + ip_mask.hdr.src_addr = src_mask; + pattern[2].type = RTE_FLOW_ITEM_TYPE_IPV4; + pattern[2].spec = &ip_spec; + pattern[2].mask = &ip_mask; + +Closing the pattern part. + +.. code-block:: c + + pattern[3].type = RTE_FLOW_ITEM_TYPE_END; + +The last part of the function is to validate the rule and create it. + +.. code-block:: c + + int res = rte_flow_validate(port_id, &attr, pattern, action, &error); + if (!res) + flow = rte_flow_create(port_id, &attr, pattern, action, &error); + diff --git a/doc/guides/sample_app_ug/hello_world.rst b/doc/guides/sample_app_ug/hello_world.rst index f4753af0..3018d459 100644 --- a/doc/guides/sample_app_ug/hello_world.rst +++ b/doc/guides/sample_app_ug/hello_world.rst @@ -37,26 +37,9 @@ The application simply prints an "helloworld" message on every enabled lcore. Compiling the Application ------------------------- -#. Go to the example directory: +To compile the sample application see :doc:`compiling`. - .. code-block:: console - - export RTE_SDK=/path/to/rte_sdk - cd ${RTE_SDK}/examples/helloworld - -#. Set the target (a default target is used if not specified). For example: - - .. code-block:: console - - export RTE_TARGET=x86_64-native-linuxapp-gcc - - See the *DPDK Getting Started* Guide for possible RTE_TARGET values. - -#. Build the application: - - .. code-block:: console - - make +The application is located in the ``helloworld`` sub-directory. Running the Application ----------------------- diff --git a/doc/guides/sample_app_ug/index.rst b/doc/guides/sample_app_ug/index.rst index 069d4f12..db68ef76 100644 --- a/doc/guides/sample_app_ug/index.rst +++ b/doc/guides/sample_app_ug/index.rst @@ -36,12 +36,15 @@ Sample Applications User Guides :numbered: intro + compiling cmd_line ethtool exception_path hello_world skeleton rxtx_callbacks + flow_classify + flow_filtering ip_frag ipv4_multicast ip_reassembly @@ -58,6 +61,7 @@ Sample Applications User Guides link_status_intr load_balancer server_node_efd + service_cores multi_process qos_metering qos_scheduler diff --git a/doc/guides/sample_app_ug/intro.rst b/doc/guides/sample_app_ug/intro.rst index d3f261b3..ae12503b 100644 --- a/doc/guides/sample_app_ug/intro.rst +++ b/doc/guides/sample_app_ug/intro.rst @@ -1,5 +1,5 @@ .. BSD LICENSE - Copyright(c) 2010-2014 Intel Corporation. All rights reserved. + Copyright(c) 2010-2017 Intel Corporation. All rights reserved. All rights reserved. Redistribution and use in source and binary forms, with or without @@ -28,42 +28,115 @@ (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. -Introduction -============ - -This document describes the sample applications that are included in the Data Plane Development Kit (DPDK). -Each chapter describes a sample application that showcases specific functionality and -provides instructions on how to compile, run and use the sample application. - -Documentation Roadmap ---------------------- - -The following is a list of DPDK documents in suggested reading order: - -* **Release Notes** : Provides release-specific information, including supported features, - limitations, fixed issues, known issues and so on. - Also, provides the answers to frequently asked questions in FAQ format. - -* **Getting Started Guides** : Describes how to install and - configure the DPDK software for your operating system; - designed to get users up and running quickly with the software. - -* **Programmer's Guide:** Describes: - - * The software architecture and how to use it (through examples), - specifically in a Linux* application (linuxapp) environment. - - * The content of the DPDK, the build system - (including the commands that can be used in the root DPDK Makefile to build the development kit and an application) - and guidelines for porting an application. - - * Optimizations used in the software and those that should be considered for new development - -A glossary of terms is also provided. - -* **API Reference** : Provides detailed information about DPDK functions, - data structures and other programming constructs. - -* **Sample Applications User Guide** : Describes a set of sample applications. - Each chapter describes a sample application that showcases specific functionality and - provides instructions on how to compile, run and use the sample application. +Introduction to the DPDK Sample Applications +============================================ + +The DPDK Sample Applications are small standalone applications which +demonstrate various features of DPDK. They can be considered as a cookbook of +DPDK features. Users interested in getting started with DPDK can take the +applications, try out the features, and then extend them to fit their needs. + + +The DPDK Sample Applications +---------------------------- + +Table :numref:`table_sample_apps` shows a list of some of the main sample +applications that are available in the examples directory of DPDK: + + .. _table_sample_apps: + + .. table:: **Some of the DPDK Sample applications** + + +---------------------------------------+--------------------------------------+ + | Bonding | Netmap Compatibility | + +---------------------------------------+--------------------------------------+ + | Command Line | Packet Ordering | + +---------------------------------------+--------------------------------------+ + | Distributor | Performance Thread | + +---------------------------------------+--------------------------------------+ + | Ethtool | Precision Time Protocol (PTP) Client | + +---------------------------------------+--------------------------------------+ + | Exception Path | Quality of Service (QoS) Metering | + +---------------------------------------+--------------------------------------+ + | Hello World | QoS Scheduler | + +---------------------------------------+--------------------------------------+ + | Internet Protocol (IP) Fragmentation | Quota and Watermark | + +---------------------------------------+--------------------------------------+ + | IP Pipeline | RX/TX Callbacks | + +---------------------------------------+--------------------------------------+ + | IP Reassembly | Server node EFD | + +---------------------------------------+--------------------------------------+ + | IPsec Security Gateway | Basic Forwarding/Skeleton App | + +---------------------------------------+--------------------------------------+ + | IPv4 multicast | Tunnel End Point (TEP) termination | + +---------------------------------------+--------------------------------------+ + | Kernel NIC Interface | Timer | + +---------------------------------------+--------------------------------------+ + | Network Layer 2 Forwarding + variants | Vhost | + +---------------------------------------+--------------------------------------+ + | Network Layer 3 Forwarding + variants | Vhost Xen | + +---------------------------------------+--------------------------------------+ + | Link Status Interrupt | VMDQ Forwarding | + +---------------------------------------+--------------------------------------+ + | Load Balancer | VMDQ and DCB Forwarding | + +---------------------------------------+--------------------------------------+ + | Multi-process | VM Power Management | + +---------------------------------------+--------------------------------------+ + +These examples range from simple to reasonably complex but most are designed +to demonstrate one particular feature of DPDK. Some of the more interesting +examples are highlighted below. + + +* :doc:`Hello World<hello_world>`: As with most introductions to a + programming framework a good place to start is with the Hello World + application. The Hello World example sets up the DPDK Environment Abstraction + Layer (EAL), and prints a simple "Hello World" message to each of the DPDK + enabled cores. This application doesn't do any packet forwarding but it is a + good way to test if the DPDK environment is compiled and set up properly. + +* :doc:`Basic Forwarding/Skeleton Application<skeleton>`: The Basic + Forwarding/Skeleton contains the minimum amount of code required to enable + basic packet forwarding with DPDK. This allows you to test if your network + interfaces are working with DPDK. + +* :doc:`Network Layer 2 forwarding<l2_forward_real_virtual>`: The Network Layer 2 + forwarding, or ``l2fwd`` application does forwarding based on Ethernet MAC + addresses like a simple switch. + +* :doc:`Network Layer 3 forwarding<l3_forward>`: The Network Layer3 + forwarding, or ``l3fwd`` application does forwarding based on Internet + Protocol, IPv4 or IPv6 like a simple router. + +* :doc:`Packet Distributor<dist_app>`: The Packet Distributor + demonstrates how to distribute packets arriving on an Rx port to different + cores for processing and transmission. + +* :doc:`Multi-Process Application<multi_process>`: The + multi-process application shows how two DPDK processes can work together using + queues and memory pools to share information. + +* :doc:`RX/TX callbacks Application<rxtx_callbacks>`: The RX/TX + callbacks sample application is a packet forwarding application that + demonstrates the use of user defined callbacks on received and transmitted + packets. The application calculates the latency of a packet between RX + (packet arrival) and TX (packet transmission) by adding callbacks to the RX + and TX packet processing functions. + +* :doc:`IPSec Security Gateway<ipsec_secgw>`: The IPSec Security + Gateway application is minimal example of something closer to a real world + example. This is also a good example of an application using the DPDK + Cryptodev framework. + +* :doc:`Precision Time Protocol (PTP) client<ptpclient>`: The PTP + client is another minimal implementation of a real world application. + In this case the application is a PTP client that communicates with a PTP + master clock to synchronize time on a Network Interface Card (NIC) using the + IEEE1588 protocol. + +* :doc:`Quality of Service (QoS) Scheduler<qos_scheduler>`: The QoS + Scheduler application demonstrates the use of DPDK to provide QoS scheduling. + +There are many more examples shown in the following chapters. Each of the +documented sample applications show how to compile, configure and run the +application as well as explaining the main functionality of the code. diff --git a/doc/guides/sample_app_ug/ip_frag.rst b/doc/guides/sample_app_ug/ip_frag.rst index bd1fe2dc..f45adfe7 100644 --- a/doc/guides/sample_app_ug/ip_frag.rst +++ b/doc/guides/sample_app_ug/ip_frag.rst @@ -59,31 +59,12 @@ Any unmatched packets are forwarded to the originating port. By default, input frame sizes up to 9.5 KB are supported. Before forwarding, the input IP packet is fragmented to fit into the "standard" Ethernet* v2 MTU (1500 bytes). -Building the Application ------------------------- +Compiling the Application +------------------------- -To build the application: +To compile the sample application see :doc:`compiling`. -#. Go to the sample application directory: - - .. code-block:: console - - export RTE_SDK=/path/to/rte_sdk - cd ${RTE_SDK}/examples/ip_fragmentation - -#. Set the target (a default target is used if not specified). For example: - - .. code-block:: console - - export RTE_TARGET=x86_64-native-linuxapp-gcc - -See the *DPDK Getting Started Guide* for possible RTE_TARGET values. - -#. Build the application: - - .. code-block:: console - - make +The application is located in the ``ip_fragmentation`` sub-directory. Running the Application ----------------------- diff --git a/doc/guides/sample_app_ug/ip_reassembly.rst b/doc/guides/sample_app_ug/ip_reassembly.rst index cc9e5911..7dc80d0b 100644 --- a/doc/guides/sample_app_ug/ip_reassembly.rst +++ b/doc/guides/sample_app_ug/ip_reassembly.rst @@ -53,28 +53,14 @@ There are two key differences from the L2 Forwarding sample application: The Longest Prefix Match (LPM for IPv4, LPM6 for IPv6) table is used to store/lookup an outgoing port number, associated with that IPv4 address. Any unmatched packets are forwarded to the originating port.Compiling the Application -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- -To compile the application: -#. Go to the sample application directory: +Compiling the Application +------------------------- - .. code-block:: console +To compile the sample application see :doc:`compiling`. - export RTE_SDK=/path/to/rte_sdk - cd ${RTE_SDK}/examples/ip_reassembly +The application is located in the ``ip_reassembly`` sub-directory. -#. Set the target (a default target is used if not specified). For example: - - .. code-block:: console - - export RTE_TARGET=x86_64-native-linuxapp-gcc - -See the *DPDK Getting Started Guide* for possible RTE_TARGET values. - -#. Build the application: - - .. code-block:: console - - make Running the Application ----------------------- diff --git a/doc/guides/sample_app_ug/ipsec_secgw.rst b/doc/guides/sample_app_ug/ipsec_secgw.rst index b675cbae..d6cfdbf7 100644 --- a/doc/guides/sample_app_ug/ipsec_secgw.rst +++ b/doc/guides/sample_app_ug/ipsec_secgw.rst @@ -52,13 +52,22 @@ The application classifies the ports as *Protected* and *Unprotected*. Thus, traffic received on an Unprotected or Protected port is consider Inbound or Outbound respectively. +The application also supports complete IPSec protocol offload to hardware +(Look aside crypto accelarator or using ethernet device). It also support +inline ipsec processing by the supported ethernet device during transmission. +These modes can be selected during the SA creation configuration. + +In case of complete protocol offload, the processing of headers(ESP and outer +IP header) is done by the hardware and the application does not need to +add/remove them during outbound/inbound processing. + The Path for IPsec Inbound traffic is: * Read packets from the port. * Classify packets between IPv4 and ESP. * Perform Inbound SA lookup for ESP packets based on their SPI. -* Perform Verification/Decryption. -* Remove ESP and outer IP header +* Perform Verification/Decryption (Not needed in case of inline ipsec). +* Remove ESP and outer IP header (Not needed in case of protocol offload). * Inbound SP check using ACL of decrypted packets and any other IPv4 packets. * Routing. * Write packet to port. @@ -68,8 +77,8 @@ The Path for the IPsec Outbound traffic is: * Read packets from the port. * Perform Outbound SP check using ACL of all IPv4 traffic. * Perform Outbound SA lookup for packets that need IPsec protection. -* Add ESP and outer IP header. -* Perform Encryption/Digest. +* Add ESP and outer IP header (Not needed in case protocol offload). +* Perform Encryption/Digest (Not needed in case of inline ipsec). * Routing. * Write packet to port. @@ -83,27 +92,12 @@ Constraints * Each SA must be handle by a unique lcore (*1 RX queue per port*). * No chained mbufs. - Compiling the Application ------------------------- -To compile the application: - -#. Go to the sample application directory:: - - export RTE_SDK=/path/to/rte_sdk - cd ${RTE_SDK}/examples/ipsec-secgw - -#. Set the target (a default target is used if not specified). For example:: - +To compile the sample application see :doc:`compiling`. - export RTE_TARGET=x86_64-native-linuxapp-gcc - - See the *DPDK Getting Started Guide* for possible RTE_TARGET values. - -#. Build the application:: - - make +The application is located in the ``rpsec-secgw`` sub-directory. #. [Optional] Build the application for debugging: This option adds some extra flags, disables compiler optimizations and @@ -119,7 +113,7 @@ The application has a number of command line options:: ./build/ipsec-secgw [EAL options] -- - -p PORTMASK -P -u PORTMASK + -p PORTMASK -P -u PORTMASK -j FRAMESIZE --config (port,queue,lcore)[,(port,queue,lcore] --single-sa SAIDX -f CONFIG_FILE_PATH @@ -135,6 +129,10 @@ Where: * ``-u PORTMASK``: hexadecimal bitmask of unprotected ports +* ``-j FRAMESIZE``: *optional*. Enables jumbo frames with the maximum size + specified as FRAMESIZE. If an invalid value is provided as FRAMESIZE + then the default value 9000 is used. + * ``--config (port,queue,lcore)[,(port,queue,lcore)]``: determines which queues from which ports are mapped to which cores. @@ -385,7 +383,7 @@ The SA rule syntax is shown as follows: .. code-block:: console sa <dir> <spi> <cipher_algo> <cipher_key> <auth_algo> <auth_key> - <mode> <src_ip> <dst_ip> + <mode> <src_ip> <dst_ip> <action_type> <port_id> where each options means: @@ -526,6 +524,34 @@ where each options means: * *dst X.X.X.X* for IPv4 * *dst XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX* for IPv6 +``<type>`` + + * Action type to specify the security action. This option specify + the SA to be performed with look aside protocol offload to HW + accelerator or protocol offload on ethernet device or inline + crypto processing on the ethernet device during transmission. + + * Optional: Yes, default type *no-offload* + + * Available options: + + * *lookaside-protocol-offload*: look aside protocol offload to HW accelerator + * *inline-protocol-offload*: inline protocol offload on ethernet device + * *inline-crypto-offload*: inline crypto processing on ethernet device + * *no-offload*: no offloading to hardware + + ``<port_id>`` + + * Port/device ID of the ethernet/crypto accelerator for which the SA is + configured. This option is used when *type* is NOT *no-offload* + + * Optional: No, if *type* is not *no-offload* + + * Syntax: + + * *port_id X* X is a valid device number in decimal + + Example SA rules: .. code-block:: console @@ -545,6 +571,11 @@ Example SA rules: aead_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \ mode ipv4-tunnel src 172.16.2.5 dst 172.16.1.5 + sa out 5 cipher_algo aes-128-cbc cipher_key 0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0 \ + auth_algo sha1-hmac auth_key 0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0 \ + mode ipv4-tunnel src 172.16.1.5 dst 172.16.2.5 \ + type lookaside-protocol-offload port_id 4 + Routing rule syntax ^^^^^^^^^^^^^^^^^^^ diff --git a/doc/guides/sample_app_ug/ipv4_multicast.rst b/doc/guides/sample_app_ug/ipv4_multicast.rst index 49712a0f..fd1af006 100644 --- a/doc/guides/sample_app_ug/ipv4_multicast.rst +++ b/doc/guides/sample_app_ug/ipv4_multicast.rst @@ -63,37 +63,12 @@ with the mask of ports to multicast packets to. Also, the application does not consider the Ethernet addresses; it looks only at the IPv4 destination address for any given packet. -Building the Application ------------------------- +Compiling the Application +------------------------- -To compile the application: +To compile the sample application see :doc:`compiling`. -#. Go to the sample application directory: - - .. code-block:: console - - export RTE_SDK=/path/to/rte_sdk - cd ${RTE_SDK}/examples/ipv4_multicast - -#. Set the target (a default target is used if not specified). For example: - - .. code-block:: console - - export RTE_TARGET=x86_64-native-linuxapp-gcc - -See the *DPDK Getting Started Guide* for possible RTE_TARGET values. - -#. Build the application: - - .. code-block:: console - - make - -.. note:: - - The compiled application is written to the build subdirectory. - To have the application written to a different location, - the O=/path/to/build/directory option may be specified in the make command. +The application is located in the ``ipv4_multicast`` sub-directory. Running the Application ----------------------- @@ -268,7 +243,7 @@ The actual packet transmission is done in the mcast_send_pkt() function: .. code-block:: c - static inline void mcast_send_pkt(struct rte_mbuf *pkt, struct ether_addr *dest_addr, struct lcore_queue_conf *qconf, uint8_t port) + static inline void mcast_send_pkt(struct rte_mbuf *pkt, struct ether_addr *dest_addr, struct lcore_queue_conf *qconf, uint16_t port) { struct ether_hdr *ethdr; uint16_t len; diff --git a/doc/guides/sample_app_ug/keep_alive.rst b/doc/guides/sample_app_ug/keep_alive.rst index fe908206..9b8be489 100644 --- a/doc/guides/sample_app_ug/keep_alive.rst +++ b/doc/guides/sample_app_ug/keep_alive.rst @@ -66,27 +66,9 @@ of the L2 forwarding application. Compiling the Application ------------------------- -To compile the application: +To compile the sample application see :doc:`compiling`. -#. Go to the sample application directory: - - .. code-block:: console - - export RTE_SDK=/path/to/rte_sdk cd ${RTE_SDK}/examples/keep_alive - -#. Set the target (a default target is used if not specified). For example: - - .. code-block:: console - - export RTE_TARGET=x86_64-native-linuxapp-gcc - - See the *DPDK Getting Started Guide* for possible RTE_TARGET values. - -#. Build the application: - - .. code-block:: console - - make +The application is located in the ``l2fwd_keep_alive`` sub-directory. Running the Application ----------------------- diff --git a/doc/guides/sample_app_ug/kernel_nic_interface.rst b/doc/guides/sample_app_ug/kernel_nic_interface.rst index 619a7b52..e1ac4153 100644 --- a/doc/guides/sample_app_ug/kernel_nic_interface.rst +++ b/doc/guides/sample_app_ug/kernel_nic_interface.rst @@ -77,35 +77,17 @@ The packet flow through the Kernel NIC Interface application is as shown in the Kernel NIC Application Packet Flow - Compiling the Application ------------------------- -Compile the application as follows: - -#. Go to the example directory: - - .. code-block:: console +To compile the sample application see :doc:`compiling`. - export RTE_SDK=/path/to/rte_sdk - cd ${RTE_SDK}/examples/kni +The application is located in the ``kni`` sub-directory. -#. Set the target (a default target is used if not specified) - - .. note:: +.. note:: This application is intended as a linuxapp only. - .. code-block:: console - - export RTE_TARGET=x86_64-native-linuxapp-gcc - -#. Build the application: - - .. code-block:: console - - make - Loading the Kernel Module ------------------------- @@ -246,7 +228,7 @@ The code for allocating the kernel NIC interfaces for a specific port is as foll .. code-block:: c static int - kni_alloc(uint8_t port_id) + kni_alloc(uint16_t port_id) { uint8_t i; struct rte_kni *kni; @@ -335,7 +317,7 @@ The code is as follows: int i, j, nb_token; char *str_fld[_NUM_FLD]; unsigned long int_fld[_NUM_FLD]; - uint8_t port_id, nb_kni_port_params = 0; + uint16_t port_id, nb_kni_port_params = 0; memset(&kni_port_params_array, 0, sizeof(kni_port_params_array)); @@ -532,7 +514,7 @@ Currently, setting a new MTU and configuring the network interface (up/ down) ar /* Callback for request of changing MTU */ static int - kni_change_mtu(uint8_t port_id, unsigned new_mtu) + kni_change_mtu(uint16_t port_id, unsigned new_mtu) { int ret; struct rte_eth_conf conf; @@ -581,7 +563,7 @@ Currently, setting a new MTU and configuring the network interface (up/ down) ar /* Callback for request of configuring network interface up/down */ static int - kni_config_network_interface(uint8_t port_id, uint8_t if_up) + kni_config_network_interface(uint16_t port_id, uint8_t if_up) { int ret = 0; diff --git a/doc/guides/sample_app_ug/l2_forward_cat.rst b/doc/guides/sample_app_ug/l2_forward_cat.rst index b0c2e109..a10f23a1 100644 --- a/doc/guides/sample_app_ug/l2_forward_cat.rst +++ b/doc/guides/sample_app_ug/l2_forward_cat.rst @@ -67,38 +67,28 @@ White paper demonstrating example use case: Compiling the Application ------------------------- +.. note:: -Requires ``libpqos`` from Intel's -`intel-cmt-cat software package <https://github.com/01org/intel-cmt-cat>`_ -hosted on GitHub repository. For installation notes, please see ``README`` file. + Requires ``libpqos`` from Intel's + `intel-cmt-cat software package <https://github.com/01org/intel-cmt-cat>`_ + hosted on GitHub repository. For installation notes, please see ``README`` file. -GIT: + GIT: -* https://github.com/01org/intel-cmt-cat - -To compile the application export the path to PQoS lib -and the DPDK source tree and go to the example directory: - -.. code-block:: console + * https://github.com/01org/intel-cmt-cat - export PQOS_INSTALL_PATH=/path/to/libpqos - export RTE_SDK=/path/to/rte_sdk - cd ${RTE_SDK}/examples/l2fwd-cat +#. To compile the application export the path to PQoS lib + and the DPDK source tree and go to the example directory: -Set the target, for example: + .. code-block:: console -.. code-block:: console - - export RTE_TARGET=x86_64-native-linuxapp-gcc + export PQOS_INSTALL_PATH=/path/to/libpqos -See the *DPDK Getting Started* Guide for possible ``RTE_TARGET`` values. -Build the application as follows: - -.. code-block:: console +To compile the sample application see :doc:`compiling`. - make +The application is located in the ``l2fwd-cat`` sub-directory. Running the Application diff --git a/doc/guides/sample_app_ug/l2_forward_crypto.rst b/doc/guides/sample_app_ug/l2_forward_crypto.rst index 158f4b44..1e85b4a3 100644 --- a/doc/guides/sample_app_ug/l2_forward_crypto.rst +++ b/doc/guides/sample_app_ug/l2_forward_crypto.rst @@ -55,26 +55,9 @@ Also, if MAC addresses updating is enabled, the MAC addresses are affected as fo Compiling the Application ------------------------- -#. Go to the example directory: +To compile the sample application see :doc:`compiling`. - .. code-block:: console - - export RTE_SDK=/path/to/rte_sdk - cd ${RTE_SDK}/examples/l2fwd-crypto - -#. Set the target (a default target is used if not specified). For example: - - .. code-block:: console - - export RTE_TARGET=x86_64-native-linuxapp-gcc - - *See the DPDK Getting Started Guide* for possible RTE_TARGET values. - -#. Build the application: - - .. code-block:: console - - make +The application is located in the ``l2fwd-crypt`` sub-directory. Running the Application ----------------------- diff --git a/doc/guides/sample_app_ug/l2_forward_job_stats.rst b/doc/guides/sample_app_ug/l2_forward_job_stats.rst index 1029cc8f..54b970f8 100644 --- a/doc/guides/sample_app_ug/l2_forward_job_stats.rst +++ b/doc/guides/sample_app_ug/l2_forward_job_stats.rst @@ -97,26 +97,9 @@ in this case enabling a total of four Virtual Functions. Compiling the Application ------------------------- -#. Go to the example directory: +To compile the sample application see :doc:`compiling`. - .. code-block:: console - - export RTE_SDK=/path/to/rte_sdk - cd ${RTE_SDK}/examples/l2fwd-jobstats - -#. Set the target (a default target is used if not specified). For example: - - .. code-block:: console - - export RTE_TARGET=x86_64-native-linuxapp-gcc - - *See the DPDK Getting Started Guide* for possible RTE_TARGET values. - -#. Build the application: - - .. code-block:: console - - make +The application is located in the ``l2fwd-jobstats`` sub-directory. Running the Application ----------------------- @@ -558,7 +541,7 @@ If the table is full, the whole packets table is transmitted using the l2fwd_sen /* Send the packet on an output interface */ static int - l2fwd_send_packet(struct rte_mbuf *m, uint8_t port) + l2fwd_send_packet(struct rte_mbuf *m, uint16_t port) { unsigned lcore_id, len; struct lcore_queue_conf *qconf; @@ -593,7 +576,7 @@ however it improves performance: unsigned lcore_id; struct lcore_queue_conf *qconf; struct mbuf_table *m_table; - uint8_t portid; + uint16_t portid; lcore_id = rte_lcore_id(); qconf = &lcore_queue_conf[lcore_id]; diff --git a/doc/guides/sample_app_ug/l2_forward_real_virtual.rst b/doc/guides/sample_app_ug/l2_forward_real_virtual.rst index f579c8fe..4d9cae43 100644 --- a/doc/guides/sample_app_ug/l2_forward_real_virtual.rst +++ b/doc/guides/sample_app_ug/l2_forward_real_virtual.rst @@ -107,26 +107,9 @@ in this case enabling a total of four Virtual Functions. Compiling the Application ------------------------- -#. Go to the example directory: +To compile the sample application see :doc:`compiling`. - .. code-block:: console - - export RTE_SDK=/path/to/rte_sdk - cd ${RTE_SDK}/examples/l2fwd - -#. Set the target (a default target is used if not specified). For example: - - .. code-block:: console - - export RTE_TARGET=x86_64-native-linuxapp-gcc - - *See the DPDK Getting Started Guide* for possible RTE_TARGET values. - -#. Build the application: - - .. code-block:: console - - make +The application is located in the ``l2fwd`` sub-directory. Running the Application ----------------------- @@ -474,7 +457,7 @@ If the table is full, the whole packets table is transmitted using the l2fwd_sen /* Send the packet on an output interface */ static int - l2fwd_send_packet(struct rte_mbuf *m, uint8_t port) + l2fwd_send_packet(struct rte_mbuf *m, uint16_t port) { unsigned lcore_id, len; struct lcore_queue_conf *qconf; diff --git a/doc/guides/sample_app_ug/l3_forward.rst b/doc/guides/sample_app_ug/l3_forward.rst index 6a6b8fbe..19b91e27 100644 --- a/doc/guides/sample_app_ug/l3_forward.rst +++ b/doc/guides/sample_app_ug/l3_forward.rst @@ -62,28 +62,9 @@ In the sample application, hash-based forwarding supports IPv4 and IPv6. LPM-bas Compiling the Application ------------------------- -To compile the application: +To compile the sample application see :doc:`compiling`. -#. Go to the sample application directory: - - .. code-block:: console - - export RTE_SDK=/path/to/rte_sdk - cd ${RTE_SDK}/examples/l3fwd - -#. Set the target (a default target is used if not specified). For example: - - .. code-block:: console - - export RTE_TARGET=x86_64-native-linuxapp-gcc - - See the *DPDK Getting Started Guide* for possible RTE_TARGET values. - -#. Build the application: - - .. code-block:: console - - make +The application is located in the ``l3fwd`` sub-directory. Running the Application ----------------------- @@ -283,7 +264,7 @@ The get_ipv4_dst_port() function is shown below: .. code-block:: c static inline uint8_t - get_ipv4_dst_port(void *ipv4_hdr, uint8_t portid, lookup_struct_t *ipv4_l3fwd_lookup_struct) + get_ipv4_dst_port(void *ipv4_hdr, uint16_t portid, lookup_struct_t *ipv4_l3fwd_lookup_struct) { int ret = 0; union ipv4_5tuple_host key; @@ -312,7 +293,7 @@ The key code snippet of simple_ipv4_fwd_4pkts() is shown below: .. code-block:: c static inline void - simple_ipv4_fwd_4pkts(struct rte_mbuf* m[4], uint8_t portid, struct lcore_conf *qconf) + simple_ipv4_fwd_4pkts(struct rte_mbuf* m[4], uint16_t portid, struct lcore_conf *qconf) { // ... @@ -351,10 +332,10 @@ for LPM-based lookups is done by the get_ipv4_dst_port() function below: .. code-block:: c - static inline uint8_t - get_ipv4_dst_port(struct ipv4_hdr *ipv4_hdr, uint8_t portid, lookup_struct_t *ipv4_l3fwd_lookup_struct) + static inline uint16_t + get_ipv4_dst_port(struct ipv4_hdr *ipv4_hdr, uint16_t portid, lookup_struct_t *ipv4_l3fwd_lookup_struct) { uint8_t next_hop; - return (uint8_t) ((rte_lpm_lookup(ipv4_l3fwd_lookup_struct, rte_be_to_cpu_32(ipv4_hdr->dst_addr), &next_hop) == 0)? next_hop : portid); + return ((rte_lpm_lookup(ipv4_l3fwd_lookup_struct, rte_be_to_cpu_32(ipv4_hdr->dst_addr), &next_hop) == 0)? next_hop : portid); } diff --git a/doc/guides/sample_app_ug/l3_forward_access_ctrl.rst b/doc/guides/sample_app_ug/l3_forward_access_ctrl.rst index a6aa4fb1..dfd372fb 100644 --- a/doc/guides/sample_app_ug/l3_forward_access_ctrl.rst +++ b/doc/guides/sample_app_ug/l3_forward_access_ctrl.rst @@ -252,28 +252,9 @@ Once the application starts, it transitions through three phases: Compiling the Application ------------------------- -To compile the application: +To compile the sample application see :doc:`compiling`. -#. Go to the sample application directory: - - .. code-block:: console - - export RTE_SDK=/path/to/rte_sdk - cd ${RTE_SDK}/examples/l3fwd-acl - -#. Set the target (a default target is used if not specified). For example: - - .. code-block:: console - - export RTE_TARGET=x86_64-native-linuxapp-gcc - - See the *DPDK IPL Getting Started Guide* for possible RTE_TARGET values. - -#. Build the application: - - .. code-block:: console - - make +The application is located in the ``l3fwd-acl`` sub-directory. Running the Application ----------------------- diff --git a/doc/guides/sample_app_ug/l3_forward_power_man.rst b/doc/guides/sample_app_ug/l3_forward_power_man.rst index ea9c404d..20fcc3c6 100644 --- a/doc/guides/sample_app_ug/l3_forward_power_man.rst +++ b/doc/guides/sample_app_ug/l3_forward_power_man.rst @@ -104,28 +104,9 @@ instead of always running to the C0 state waiting for packets. Compiling the Application ------------------------- -To compile the application: +To compile the sample application see :doc:`compiling`. -#. Go to the sample application directory: - - .. code-block:: console - - export RTE_SDK=/path/to/rte_sdk - cd ${RTE_SDK}/examples/l3fwd-power - -#. Set the target (a default target is used if not specified). For example: - - .. code-block:: console - - export RTE_TARGET=x86_64-native-linuxapp-gcc - - See the *DPDK Getting Started Guide* for possible RTE_TARGET values. - -#. Build the application: - - .. code-block:: console - - make +The application is located in the ``l3fwd-power`` sub-directory. Running the Application ----------------------- @@ -181,11 +162,11 @@ responsible for checking if it needs to scale down frequency at run time by chec struct lcore_conf *qconf; int ret; unsigned nb_ports; - uint16_t queueid; + uint16_t queueid, portid; unsigned lcore_id; uint64_t hz; uint32_t n_tx_queue, nb_lcores; - uint8_t portid, nb_rx_queue, queue, socketid; + uint8_t nb_rx_queue, queue, socketid; // ... diff --git a/doc/guides/sample_app_ug/l3_forward_virtual.rst b/doc/guides/sample_app_ug/l3_forward_virtual.rst index 5f9d8948..95e89651 100644 --- a/doc/guides/sample_app_ug/l3_forward_virtual.rst +++ b/doc/guides/sample_app_ug/l3_forward_virtual.rst @@ -64,34 +64,9 @@ The set of LPM rules used by the application is statically configured and loaded Compiling the Application ------------------------- -To compile the application: +To compile the sample application see :doc:`compiling`. -#. Go to the sample application directory: - - .. code-block:: console - - export RTE_SDK=/path/to/rte_sdk - cd ${RTE_SDK}/examples/l3fwd-vf - -#. Set the target (a default target is used if not specified). For example: - - .. code-block:: console - - export RTE_TARGET=x86_64-native-linuxapp-gcc - - See the *DPDK Getting Started Guide* for possible RTE_TARGET values. - -#. Build the application: - - .. code-block:: console - - make - -.. note:: - - The compiled application is written to the build subdirectory. - To have the application written to a different location, - the O=/path/to/build/directory option may be specified in the make command. +The application is located in the ``l3fwd-vf`` sub-directory. Running the Application ----------------------- diff --git a/doc/guides/sample_app_ug/link_status_intr.rst b/doc/guides/sample_app_ug/link_status_intr.rst index f9af4749..358524c3 100644 --- a/doc/guides/sample_app_ug/link_status_intr.rst +++ b/doc/guides/sample_app_ug/link_status_intr.rst @@ -55,32 +55,9 @@ and the behavior of L2 forwarding each time the link status changes. Compiling the Application ------------------------- -#. Go to the example directory: +To compile the sample application see :doc:`compiling`. - .. code-block:: console - - export RTE_SDK=/path/to/rte_sdk - cd ${RTE_SDK}/examples/link_status_interrupt - -#. Set the target (a default target is used if not specified). For example: - - .. code-block:: console - - export RTE_TARGET=x86_64-native-linuxapp-gcc - - See the *DPDK Getting Started Guide* for possible RTE_TARGET values. - -#. Build the application: - - .. code-block:: console - - make - -.. note:: - - The compiled application is written to the build subdirectory. - To have the application written to a different location, - the O=/path/to/build/directory option may be specified on the make command line. +The application is located in the ``link_status_interrupt`` sub-directory. Running the Application ----------------------- @@ -219,7 +196,7 @@ An example callback function that has been written as indicated below. .. code-block:: c static void - lsi_event_callback(uint8_t port_id, enum rte_eth_event_type type, void *param) + lsi_event_callback(uint16_t port_id, enum rte_eth_event_type type, void *param) { struct rte_eth_link link; @@ -405,7 +382,7 @@ If the table is full, the whole packets table is transmitted using the lsi_send_ /* Send the packet on an output interface */ static int - lsi_send_packet(struct rte_mbuf *m, uint8_t port) + lsi_send_packet(struct rte_mbuf *m, uint16_t port) { unsigned lcore_id, len; struct lcore_queue_conf *qconf; diff --git a/doc/guides/sample_app_ug/load_balancer.rst b/doc/guides/sample_app_ug/load_balancer.rst index e101a5f2..18b29b24 100644 --- a/doc/guides/sample_app_ug/load_balancer.rst +++ b/doc/guides/sample_app_ug/load_balancer.rst @@ -86,24 +86,9 @@ The routing logic is LPM based, with all the worker threads sharing the same LPM Compiling the Application ------------------------- -The sequence of steps used to build the application is: +To compile the sample application see :doc:`compiling`. -#. Export the required environment variables: - - .. code-block:: console - - export RTE_SDK=<Path to the DPDK installation folder> - export RTE_TARGET=x86_64-native-linuxapp-gcc - -#. Build the application executable file: - - .. code-block:: console - - cd ${RTE_SDK}/examples/load_balancer - make - - For more details on how to build the DPDK libraries and sample applications, - please refer to the *DPDK Getting Started Guide.* +The application is located in the ``load_balancer`` sub-directory. Running the Application ----------------------- diff --git a/doc/guides/sample_app_ug/multi_process.rst b/doc/guides/sample_app_ug/multi_process.rst index d4df2fa7..95a32432 100644 --- a/doc/guides/sample_app_ug/multi_process.rst +++ b/doc/guides/sample_app_ug/multi_process.rst @@ -40,31 +40,13 @@ Example Applications Building the Sample Applications ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - The multi-process example applications are built in the same way as other sample applications, and as documented in the *DPDK Getting Started Guide*. -To build all the example applications: - -#. Set RTE_SDK and go to the example directory: - - .. code-block:: console - - export RTE_SDK=/path/to/rte_sdk - cd ${RTE_SDK}/examples/multi_process - -#. Set the target (a default target will be used if not specified). For example: - - .. code-block:: console - - export RTE_TARGET=x86_64-native-linuxapp-gcc - - See the *DPDK Getting Started Guide* for possible RTE_TARGET values. -#. Build the applications: - .. code-block:: console +To compile the sample application see :doc:`compiling`. - make +The applications are located in the ``multi_process`` sub-directory. .. note:: diff --git a/doc/guides/sample_app_ug/netmap_compatibility.rst b/doc/guides/sample_app_ug/netmap_compatibility.rst index 030fd980..955f86b9 100644 --- a/doc/guides/sample_app_ug/netmap_compatibility.rst +++ b/doc/guides/sample_app_ug/netmap_compatibility.rst @@ -121,29 +121,12 @@ The bridge application is an example largely based on the bridge example shipped It shows how a minimal Netmap application with minimal and straightforward source code changes can be run on top of the DPDK. Please refer to ``$RTE_SDK/examples/netmap_compat/bridge/bridge.c`` for an example of a ported application. -Compiling the "bridge" Sample Application ------------------------------------------ +Compiling the Application +------------------------- -#. Go to the example directory: +To compile the sample application see :doc:`compiling`. - .. code-block:: console - - export RTE_SDK=/path/to/rte_sdk - cd ${RTE_SDK}/examples/netmap_compat - -#. Set the target (a default target is used if not specified). For example: - - .. code-block:: console - - export RTE_TARGET=x86_64-native-linuxapp-gcc - - See the *DPDK Getting Started Guide for Linux* for possible ``RTE_TARGET`` values. - -#. Build the application: - - .. code-block:: console - - make +The application is located in the ``netmap_compat`` sub-directory. Running the "bridge" Sample Application --------------------------------------- diff --git a/doc/guides/sample_app_ug/packet_ordering.rst b/doc/guides/sample_app_ug/packet_ordering.rst index ef851500..098cc89f 100644 --- a/doc/guides/sample_app_ug/packet_ordering.rst +++ b/doc/guides/sample_app_ug/packet_ordering.rst @@ -51,28 +51,11 @@ The application uses at least three CPU cores: from the reorder buffer and sends them to the NIC ports for transmission. Compiling the Application --------------------------- +------------------------- -#. Go to the example directory: +To compile the sample application see :doc:`compiling`. - .. code-block:: console - - export RTE_SDK=/path/to/rte_sdk - cd ${RTE_SDK}/examples/helloworld - -#. Set the target (a default target is used if not specified). For example: - - .. code-block:: console - - export RTE_TARGET=x86_64-native-linuxapp-gcc - - See the *DPDK Getting Started* Guide for possible RTE_TARGET values. - -#. Build the application: - - .. code-block:: console - - make +The application is located in the ``packet_ordering`` sub-directory. Running the Application ----------------------- diff --git a/doc/guides/sample_app_ug/performance_thread.rst b/doc/guides/sample_app_ug/performance_thread.rst index a55a6246..57391caf 100644 --- a/doc/guides/sample_app_ug/performance_thread.rst +++ b/doc/guides/sample_app_ug/performance_thread.rst @@ -73,28 +73,10 @@ invalid. Compiling the Application ------------------------- -The application is located in the sample application folder in the -``performance-thread`` folder. -#. Go to the example applications folder - - .. code-block:: console - - export RTE_SDK=/path/to/rte_sdk - cd ${RTE_SDK}/examples/performance-thread/l3fwd-thread - -#. Set the target (a default target is used if not specified). For example: - - .. code-block:: console - - export RTE_TARGET=x86_64-native-linuxapp-gcc - - See the *DPDK Linux Getting Started Guide* for possible RTE_TARGET values. - -#. Build the application: - - make +To compile the sample application see :doc:`compiling`. +The application is located in the `performance-thread/l3fwd-thread` sub-directory. Running the Application ----------------------- diff --git a/doc/guides/sample_app_ug/ptpclient.rst b/doc/guides/sample_app_ug/ptpclient.rst index d3ef1d11..9cbb6c2a 100644 --- a/doc/guides/sample_app_ug/ptpclient.rst +++ b/doc/guides/sample_app_ug/ptpclient.rst @@ -78,39 +78,20 @@ The adjustment for slave can be represented as: If the command line parameter ``-T 1`` is used the application also synchronizes the PTP PHC clock with the Linux kernel clock. - Compiling the Application ------------------------- -To compile the application, export the path to the DPDK source tree and edit -the ``config/common_linuxapp`` configuration file to enable IEEE1588: - -.. code-block:: console - - export RTE_SDK=/path/to/rte_sdk - - # Edit common_linuxapp and set the following options: - CONFIG_RTE_LIBRTE_IEEE1588=y - -Set the target, for example: +To compile the sample application see :doc:`compiling`. -.. code-block:: console - - export RTE_TARGET=x86_64-native-linuxapp-gcc - -See the *DPDK Getting Started* Guide for possible ``RTE_TARGET`` values. - -Build the application as follows: - -.. code-block:: console +The application is located in the ``ptpclient`` sub-directory. - # Recompile DPDK. - make install T=$RTE_TARGET +.. note:: + To compile the application edit the ``config/common_linuxapp`` configuration file to enable IEEE1588 + and then recompile DPDK: - # Compile the application. - cd ${RTE_SDK}/examples/ptpclient - make + .. code-block:: console + CONFIG_RTE_LIBRTE_IEEE1588=y Running the Application ----------------------- @@ -257,7 +238,7 @@ PTP IEEE1588 L2 functionality. .. code-block:: c void - parse_ptp_frames(uint8_t portid, struct rte_mbuf *m) { + parse_ptp_frames(uint16_t portid, struct rte_mbuf *m) { struct ptp_header *ptp_hdr; struct ether_hdr *eth_hdr; uint16_t eth_type; diff --git a/doc/guides/sample_app_ug/qos_metering.rst b/doc/guides/sample_app_ug/qos_metering.rst index e1a6ac7a..067de9e1 100644 --- a/doc/guides/sample_app_ug/qos_metering.rst +++ b/doc/guides/sample_app_ug/qos_metering.rst @@ -64,29 +64,9 @@ all the incoming packets are colored as green. Compiling the Application ------------------------- -#. Go to the example directory: +To compile the sample application see :doc:`compiling`. - .. code-block:: console - - export RTE_SDK=/path/to/rte_sdk - cd ${RTE_SDK}/examples/qos_meter - -#. Set the target - (a default target is used if not specified): - - .. note:: - - This application is intended as a linuxapp only. - - .. code-block:: console - - export RTE_TARGET=x86_64-native-linuxapp-gcc - -#. Build the application: - - .. code-block:: console - - make +The application is located in the ``qos_meter`` sub-directory. Running the Application ----------------------- diff --git a/doc/guides/sample_app_ug/qos_scheduler.rst b/doc/guides/sample_app_ug/qos_scheduler.rst index a6654cb5..1c38d191 100644 --- a/doc/guides/sample_app_ug/qos_scheduler.rst +++ b/doc/guides/sample_app_ug/qos_scheduler.rst @@ -58,31 +58,14 @@ The TX thread, if present, reads from the TX ring and write the packets to the T Compiling the Application ------------------------- -To compile the application: +To compile the sample application see :doc:`compiling`. -#. Go to the sample application directory: - - .. code-block:: console - - export RTE_SDK=/path/to/rte_sdk - cd ${RTE_SDK}/examples/qos_sched - -#. Set the target (a default target is used if not specified). For example: +The application is located in the ``qos_sched`` sub-directory. .. note:: This application is intended as a linuxapp only. - .. code-block:: console - - export RTE_TARGET=x86_64-native-linuxapp-gcc - -#. Build the application: - - .. code-block:: console - - make - .. note:: To get statistics on the sample app using the command line interface as described in the next section, diff --git a/doc/guides/sample_app_ug/quota_watermark.rst b/doc/guides/sample_app_ug/quota_watermark.rst index 2c3a4320..32bd78d7 100644 --- a/doc/guides/sample_app_ug/quota_watermark.rst +++ b/doc/guides/sample_app_ug/quota_watermark.rst @@ -88,30 +88,12 @@ as shown in :numref:`figure_ring_pipeline_perf_setup`. Ring-based Processing Pipeline Performance Setup - Compiling the Application ------------------------- -#. Go to the example directory: - - .. code-block:: console - - export RTE_SDK=/path/to/rte_sdk - cd ${RTE_SDK}/examples/quota_watermark - -#. Set the target (a default target is used if not specified). For example: - - .. code-block:: console - - export RTE_TARGET=x86_64-native-linuxapp-gcc - - See the *DPDK Getting Started Guide* for possible RTE_TARGET values. - -#. Build the application: - - .. code-block:: console +To compile the sample application see :doc:`compiling`. - make +The application is located in the ``quota_watermark`` sub-directory. Running the Application ----------------------- diff --git a/doc/guides/sample_app_ug/rxtx_callbacks.rst b/doc/guides/sample_app_ug/rxtx_callbacks.rst index 8d1bb86f..928c9635 100644 --- a/doc/guides/sample_app_ug/rxtx_callbacks.rst +++ b/doc/guides/sample_app_ug/rxtx_callbacks.rst @@ -45,23 +45,9 @@ prior to transmission to calculate the elapsed time, in CPU cycles. Compiling the Application ------------------------- -To compile the application export the path to the DPDK source tree and go to -the example directory: +To compile the sample application see :doc:`compiling`. -.. code-block:: console - - export RTE_SDK=/path/to/rte_sdk - - cd ${RTE_SDK}/examples/rxtx_callbacks - - -Set the target, for example: - -.. code-block:: console - - export RTE_TARGET=x86_64-native-linuxapp-gcc - -See the *DPDK Getting Started* Guide for possible ``RTE_TARGET`` values. +The application is located in the ``rxtx_callbacks`` sub-directory. The callbacks feature requires that the ``CONFIG_RTE_ETHDEV_RXTX_CALLBACKS`` setting is on in the ``config/common_`` config file that applies to the @@ -71,13 +57,6 @@ target. This is generally on by default: CONFIG_RTE_ETHDEV_RXTX_CALLBACKS=y -Build the application as follows: - -.. code-block:: console - - make - - Running the Application ----------------------- @@ -124,7 +103,7 @@ comments: .. code-block:: c static inline int - port_init(uint8_t port, struct rte_mempool *mbuf_pool) + port_init(uint16_t port, struct rte_mempool *mbuf_pool) { struct rte_eth_conf port_conf = port_conf_default; const uint16_t rx_rings = 1, tx_rings = 1; @@ -196,7 +175,7 @@ all packets received: .. code-block:: c static uint16_t - add_timestamps(uint8_t port __rte_unused, uint16_t qidx __rte_unused, + add_timestamps(uint16_t port __rte_unused, uint16_t qidx __rte_unused, struct rte_mbuf **pkts, uint16_t nb_pkts, void *_ __rte_unused) { unsigned i; @@ -222,7 +201,7 @@ packets prior to transmission: .. code-block:: c static uint16_t - calc_latency(uint8_t port __rte_unused, uint16_t qidx __rte_unused, + calc_latency(uint16_t port __rte_unused, uint16_t qidx __rte_unused, struct rte_mbuf **pkts, uint16_t nb_pkts, void *_ __rte_unused) { uint64_t cycles = 0; diff --git a/doc/guides/sample_app_ug/server_node_efd.rst b/doc/guides/sample_app_ug/server_node_efd.rst index c2a5f20a..8891f3b3 100644 --- a/doc/guides/sample_app_ug/server_node_efd.rst +++ b/doc/guides/sample_app_ug/server_node_efd.rst @@ -108,26 +108,9 @@ that this is a new flow, which is dropped. Compiling the Application ------------------------- -The sequence of steps used to build the application is: - -#. Export the required environment variables: - - .. code-block:: console - - export RTE_SDK=/path/to/rte_sdk - export RTE_TARGET=x86_64-native-linuxapp-gcc - -#. Build the application executable file: - - .. code-block:: console - - cd ${RTE_SDK}/examples/server_node_efd/ - make - - For more details on how to build the DPDK libraries and sample - applications, - please refer to the *DPDK Getting Started Guide.* +To compile the sample application see :doc:`compiling`. +The application is located in the ``server_node_efd`` sub-directory. Running the Application ----------------------- diff --git a/doc/guides/sample_app_ug/service_cores.rst b/doc/guides/sample_app_ug/service_cores.rst new file mode 100644 index 00000000..36c1b3c5 --- /dev/null +++ b/doc/guides/sample_app_ug/service_cores.rst @@ -0,0 +1,172 @@ +.. BSD LICENSE + Copyright(c) 2017 Intel Corporation. All rights reserved. + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of Intel Corporation nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +Service Cores Sample Application +================================ + +The service cores sample application demonstrates the service cores capabilities +of DPDK. The service cores infrastructure is part of the DPDK EAL, and allows +any DPDK component to register a service. A service is a work item or task, that +requires CPU time to perform its duty. + +This sample application registers 5 dummy services. These 5 services are used +to show how the service_cores API can be used to orchestrate these services to +run on different service lcores. This orchestration is done by calling the +service cores APIs, however the sample application introduces a "profile" +concept to contain the service mapping details. Note that the profile concept +is application specific, and not a part of the service cores API. + + +Compiling the Application +------------------------- + +#. Go to the example directory: + + .. code-block:: console + + export RTE_SDK=/path/to/rte_sdk + cd ${RTE_SDK}/examples/service_cores + +#. Set the target (a default target is used if not specified). For example: + + .. code-block:: console + + export RTE_TARGET=x86_64-native-linuxapp-gcc + + See the *DPDK Getting Started* Guide for possible RTE_TARGET values. + +#. Build the application: + + .. code-block:: console + + make + +Running the Application +----------------------- + +To run the example, just execute the binary. Since the application dynamically +adds service cores in the application code itself, there is no requirement to +pass a service core-mask as an EAL argument at startup time. + +.. code-block:: console + + $ ./build/service_cores + + +Explanation +----------- + +The following sections provide some explanation of code focusing on +registering applications from an applications point of view, and modifying the +service core counts and mappings at runtime. + + +Registering a Service +~~~~~~~~~~~~~~~~~~~~~ + +The following code section shows how to register a service as an application. +Note that the service component header must be included by the application in +order to register services: ``rte_service_component.h``, in addition +to the ordinary service cores header ``rte_service.h`` which provides +the runtime functions to add, remove and remap service cores. + +.. code-block:: c + + struct rte_service_spec service = { + .name = "service_name", + }; + int ret = rte_service_component_register(services, &id); + if (ret) + return -1; + + /* set the service itself to be ready to run. In the case of + * ethdev, eventdev etc PMDs, this will be set when the + * appropriate configure or setup function is called. + */ + rte_service_component_runstate_set(id, 1); + + /* Collect statistics for the service */ + rte_service_set_stats_enable(id, 1); + + /* The application sets the service to running state. Note that this + * function enables the service to run - while the 'component' version + * of this function (as above) marks the service itself as ready */ + ret = rte_service_runstate_set(id, 1); + + +Controlling A Service Core +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This section demonstrates how to add a service core. The ``rte_service.h`` +header file provides the functions for dynamically adding and removing cores. +The APIs to add and remove cores use lcore IDs similar to existing DPDK +functions. + +These are the functions to start a service core, and have it run a service: + +.. code-block:: c + + /* the lcore ID to use as a service core */ + uint32_t service_core_id = 7; + ret = rte_service_lcore_add(service_core_id); + if(ret) + return -1; + + /* service cores are in "stopped" state when added, so start it */ + ret = rte_service_lcore_start(service_core_id); + if(ret) + return -1; + + /* map a service to the service core, causing it to run the service */ + uint32_t service_id; /* ID of a registered service */ + uint32_t enable = 1; /* 1 maps the service, 0 unmaps */ + ret = rte_service_map_lcore_set(service_id, service_core_id, enable); + if(ret) + return -1; + + +Removing A Service Core +~~~~~~~~~~~~~~~~~~~~~~~ + +To remove a service core, the steps are similar to adding but in reverse order. +Note that it is not allowed to remove a service core if the service is running, +and the service-core is the only core running that service (see documentation +for ``rte_service_lcore_stop`` function for details). + + +Conclusion +~~~~~~~~~~ + +The service cores infrastructure provides DPDK with two main features. The first +is to abstract away hardware differences: the service core can CPU cycles to +a software fallback implementation, allowing the application to be abstracted +from the difference in HW / SW availability. The second feature is a flexible +method of registering functions to be run, allowing the running of the +functions to be scaled across multiple CPUs. diff --git a/doc/guides/sample_app_ug/skeleton.rst b/doc/guides/sample_app_ug/skeleton.rst index c39b0332..9c78f494 100644 --- a/doc/guides/sample_app_ug/skeleton.rst +++ b/doc/guides/sample_app_ug/skeleton.rst @@ -39,33 +39,12 @@ It is intended as a demonstration of the basic components of a DPDK forwarding application. For more detailed implementations see the L2 and L3 forwarding sample applications. - Compiling the Application ------------------------- -To compile the application export the path to the DPDK source tree and go to -the example directory: - -.. code-block:: console - - export RTE_SDK=/path/to/rte_sdk - - cd ${RTE_SDK}/examples/skeleton - -Set the target, for example: - -.. code-block:: console - - export RTE_TARGET=x86_64-native-linuxapp-gcc - -See the *DPDK Getting Started* Guide for possible ``RTE_TARGET`` values. - -Build the application as follows: - -.. code-block:: console - - make +To compile the sample application see :doc:`compiling`. +The application is located in the ``skeleton`` sub-directory. Running the Application ----------------------- @@ -160,7 +139,7 @@ Forwarding application is shown below: .. code-block:: c static inline int - port_init(uint8_t port, struct rte_mempool *mbuf_pool) + port_init(uint16_t port, struct rte_mempool *mbuf_pool) { struct rte_eth_conf port_conf = port_conf_default; const uint16_t rx_rings = 1, tx_rings = 1; @@ -241,8 +220,8 @@ looks like the following: static __attribute__((noreturn)) void lcore_main(void) { - const uint8_t nb_ports = rte_eth_dev_count(); - uint8_t port; + const uint16_t nb_ports = rte_eth_dev_count(); + uint16_t port; /* * Check that the port is on the same NUMA node as the polling thread diff --git a/doc/guides/sample_app_ug/tep_termination.rst b/doc/guides/sample_app_ug/tep_termination.rst index 087823b1..04bf8a2c 100644 --- a/doc/guides/sample_app_ug/tep_termination.rst +++ b/doc/guides/sample_app_ug/tep_termination.rst @@ -121,39 +121,16 @@ The example in this section have been validated with the following distributions Compiling the Sample Code ------------------------- -#. Compile vhost lib: - To enable vhost, turn on vhost library in the configure file config/common_linuxapp. +To enable vhost, turn on vhost library in the configure file +``config/common_linuxapp``. .. code-block:: console CONFIG_RTE_LIBRTE_VHOST=y - -#. Go to the examples directory: - - .. code-block:: console - - export RTE_SDK=/path/to/rte_sdk - cd ${RTE_SDK}/examples/tep_termination - -#. Set the target (a default target is used if not specified). For example: - - .. code-block:: console - - export RTE_TARGET=x86_64-native-linuxapp-gcc - - See the DPDK Getting Started Guide for possible RTE_TARGET values. - -#. Build the application: - - .. code-block:: console - - cd ${RTE_SDK} - make config ${RTE_TARGET} - make install ${RTE_TARGET} - cd ${RTE_SDK}/examples/tep_termination - make +Then following the to compile the sample application shown in +:doc:`compiling`. Running the Sample Code ----------------------- diff --git a/doc/guides/sample_app_ug/test_pipeline.rst b/doc/guides/sample_app_ug/test_pipeline.rst index 95c7e0fc..f538cd05 100644 --- a/doc/guides/sample_app_ug/test_pipeline.rst +++ b/doc/guides/sample_app_ug/test_pipeline.rst @@ -55,28 +55,12 @@ The application uses three CPU cores: Test Pipeline Application - Compiling the Application ------------------------- +To compile the sample application see :doc:`compiling` -#. Go to the app/test directory: - - .. code-block:: console - - export RTE_SDK=/path/to/rte_sdk - cd ${RTE_SDK}/app/test/test-pipeline - -#. Set the target (a default target is used if not specified): - - .. code-block:: console - - export RTE_TARGET=x86_64-native-linuxapp-gcc - -#. Build the application: - - .. code-block:: console +The application is located in the ``$RTE_SDK/test/test-pipline`` directory. - make Running the Application ----------------------- diff --git a/doc/guides/sample_app_ug/timer.rst b/doc/guides/sample_app_ug/timer.rst index 00b69532..46a3a2fa 100644 --- a/doc/guides/sample_app_ug/timer.rst +++ b/doc/guides/sample_app_ug/timer.rst @@ -37,26 +37,9 @@ This application prints some messages from different lcores regularly, demonstra Compiling the Application ------------------------- -#. Go to the example directory: +To compile the sample application see :doc:`compiling`. - .. code-block:: console - - export RTE_SDK=/path/to/rte_sdk - cd ${RTE_SDK}/examples/timer - -#. Set the target (a default target is used if not specified). For example: - - .. code-block:: console - - export RTE_TARGET=x86_64-native-linuxapp-gcc - - See the *DPDK Getting Started Guide* for possible *RTE_TARGET* values. - -#. Build the application: - - .. code-block:: console - - make +The application is located in the ``timer`` sub-directory. Running the Application ----------------------- diff --git a/doc/guides/sample_app_ug/vhost.rst b/doc/guides/sample_app_ug/vhost.rst index a2a3909e..5735adbb 100644 --- a/doc/guides/sample_app_ug/vhost.rst +++ b/doc/guides/sample_app_ug/vhost.rst @@ -55,20 +55,12 @@ puts back to the same physical NIC port. Build ~~~~~ -Follow the *Getting Started Guide for Linux* on generic info about -environment setup and building DPDK from source. +To compile the sample application see :doc:`compiling`. -In this example, you need build DPDK both on the host and inside guest. -Also, you need build this example. - -.. code-block:: console - - export RTE_SDK=/path/to/dpdk_source - export RTE_TARGET=x86_64-native-linuxapp-gcc - - cd ${RTE_SDK}/examples/vhost - make +The application is located in the ``vhost`` sub-directory. +.. note:: + In this example, you need build DPDK both on the host and inside guest. Start the vswitch example ~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/doc/guides/sample_app_ug/vhost_scsi.rst b/doc/guides/sample_app_ug/vhost_scsi.rst index 8be069e0..1b4dab32 100644 --- a/doc/guides/sample_app_ug/vhost_scsi.rst +++ b/doc/guides/sample_app_ug/vhost_scsi.rst @@ -51,23 +51,14 @@ Testing steps This section shows the steps how to start a VM with the block device as fast data path for critical application. -Build -~~~~~ +Compiling the Application +------------------------- -Follow the *Getting Started Guide for Linux* on generic info about -environment setup and building DPDK from source. +To compile the sample application see :doc:`compiling`. -In this example, you need build DPDK both on the host and inside guest. -Also, you need build this example. - -.. code-block:: console - - export RTE_SDK=/path/to/dpdk_source - export RTE_TARGET=x86_64-native-linuxapp-gcc - - cd ${RTE_SDK}/examples/vhost_scsi - make +The application is located in the ``examples`` sub-directory. +You will also need to build DPDK both on the host and inside the guest Start the vhost_scsi example ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/doc/guides/sample_app_ug/vm_power_management.rst b/doc/guides/sample_app_ug/vm_power_management.rst index 05c26b03..fe53706e 100644 --- a/doc/guides/sample_app_ug/vm_power_management.rst +++ b/doc/guides/sample_app_ug/vm_power_management.rst @@ -201,9 +201,12 @@ Compiling and Running the Host Application Compiling ~~~~~~~~~ -#. export RTE_SDK=/path/to/rte_sdk -#. cd ${RTE_SDK}/examples/vm_power_manager -#. make +Compiling the Application +------------------------- + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``vm_power_manager`` sub-directory. Running ~~~~~~~ diff --git a/doc/guides/sample_app_ug/vmdq_dcb_forwarding.rst b/doc/guides/sample_app_ug/vmdq_dcb_forwarding.rst index 70e1d19e..4a6c70d2 100644 --- a/doc/guides/sample_app_ug/vmdq_dcb_forwarding.rst +++ b/doc/guides/sample_app_ug/vmdq_dcb_forwarding.rst @@ -86,26 +86,11 @@ No command-line options are taken by this application apart from the standard EA Compiling the Application ------------------------- -#. Go to the examples directory: - .. code-block:: console - export RTE_SDK=/path/to/rte_sdk - cd ${RTE_SDK}/examples/vmdq_dcb +To compile the sample application see :doc:`compiling`. -#. Set the target (a default target is used if not specified). For example: - - .. code-block:: console - - export RTE_TARGET=x86_64-native-linuxapp-gcc - - See the *DPDK Getting Started Guide* for possible RTE_TARGET values. - -#. Build the application: - - .. code-block:: console - - make +The application is located in the ``vmdq_dcb`` sub-directory. Running the Application ----------------------- diff --git a/doc/guides/testpmd_app_ug/run_app.rst b/doc/guides/testpmd_app_ug/run_app.rst index e8303f3b..4c0d2ced 100644 --- a/doc/guides/testpmd_app_ug/run_app.rst +++ b/doc/guides/testpmd_app_ug/run_app.rst @@ -94,10 +94,6 @@ See the DPDK Getting Started Guides for more information on these options. Display the version information on startup. -* ``--xen-dom0`` - - Support application running on Xen Domain0 without hugetlbfs. - * ``--syslog`` Set the syslog facility. @@ -110,6 +106,10 @@ See the DPDK Getting Started Guides for more information on these options. Specify the directory where the hugetlbfs is mounted. +* ``mbuf-pool-ops-name``: + + Pool ops name for mbuf to use. + * ``--proc-type`` Set the type of the current process. @@ -362,6 +362,7 @@ The commandline options are: csum icmpecho ieee1588 + tm * ``--rss-ip`` diff --git a/doc/guides/testpmd_app_ug/testpmd_funcs.rst b/doc/guides/testpmd_app_ug/testpmd_funcs.rst index 2ed62f5b..9789139a 100644 --- a/doc/guides/testpmd_app_ug/testpmd_funcs.rst +++ b/doc/guides/testpmd_app_ug/testpmd_funcs.rst @@ -347,6 +347,13 @@ The available information categories are: * ``ieee1588``: Demonstrate L2 IEEE1588 V2 PTP timestamping for RX and TX. Requires ``CONFIG_RTE_LIBRTE_IEEE1588=y``. +* ``tm``: Traffic Management forwarding mode + Demonstrates the use of ethdev traffic management APIs and softnic PMD for + QoS traffic management. In this mode, 5-level hierarchical QoS scheduler is + available as an default option that can be enabled through CLI. The user can + also modify the default hierarchy or specify the new hierarchy through CLI for + implementing QoS scheduler. Requires ``CONFIG_RTE_LIBRTE_PMD_SOFTNIC=y`` ``CONFIG_RTE_LIBRTE_SCHED=y``. + Note: TX timestamping is only available in the "Full Featured" TX path. To force ``testpmd`` into this mode set ``--txqflags=0``. Example:: @@ -392,7 +399,7 @@ ddp get info Display information about dynamic device personalization (DDP) profile:: - testpmd> ddp get info (profile_patch) + testpmd> ddp get info (profile_path) show vf stats ~~~~~~~~~~~~~ @@ -408,6 +415,14 @@ Reset VF statistics:: testpmd> clear vf stats (port_id) (vf_id) +show port pctype mapping +~~~~~~~~~~~~~~~~~~~~~~~~ + +List all items from the pctype mapping table:: + + testpmd> show port (port_id) pctype mapping + + Configuration Functions ----------------------- @@ -842,6 +857,49 @@ Where: Check the NIC Datasheet for hardware limits. +RSS queue region +~~~~~~~~~~~~~~~~ + +Set RSS queue region span on a port:: + + testpmd> set port (port_id) queue-region region_id (value) \ + queue_start_index (value) queue_num (value) + +Set flowtype mapping on a RSS queue region on a port:: + + testpmd> set port (port_id) queue-region region_id (value) flowtype (value) + +where: + +* For the flowtype(pctype) of packet,the specific index for each type has + been defined in file i40e_type.h as enum i40e_filter_pctype. + +Set user priority mapping on a RSS queue region on a port:: + + testpmd> set port (port_id) queue-region UP (value) region_id (value) + +Flush all queue region related configuration on a port:: + + testpmd> set port (port_id) queue-region flush (on|off) + +where: + +* "on"is just an enable function which server for other configuration, + it is for all configuration about queue region from up layer, + at first will only keep in DPDK softwarestored in driver, + only after "flush on", it commit all configuration to HW. + "off" is just clean all configuration about queue region just now, + and restore all to DPDK i40e driver default config when start up. + +Show all queue region related configuration info on a port:: + + testpmd> show port (port_id) queue-region + +.. note:: + + Queue region only support on PF by now, so these command is + only for configuration of queue region on PF port. + csum parse-tunnel ~~~~~~~~~~~~~~~~~ @@ -898,12 +956,12 @@ Display the status of TCP Segmentation Offload:: testpmd> tso show (port_id) -gro -~~~ +set port - gro +~~~~~~~~~~~~~~ Enable or disable GRO in ``csum`` forwarding engine:: - testpmd> gro (on|off) (port_id) + testpmd> set port <port_id> gro on|off If enabled, the csum forwarding engine will perform GRO on the TCP/IPv4 packets received from the given port. @@ -914,23 +972,89 @@ GRO. By default, GRO is disabled for all ports. .. note:: When enable GRO for a port, TCP/IPv4 packets received from the port - will be performed GRO. After GRO, the merged packets are multi-segments. - But csum forwarding engine doesn't support to calculate TCP checksum - for multi-segment packets in SW. So please select TCP HW checksum - calculation for the port which GROed packets are transmitted to. + will be performed GRO. After GRO, all merged packets have bad + checksums, since the GRO library doesn't re-calculate checksums for + the merged packets. Therefore, if users want the merged packets to + have correct checksums, please select HW IP checksum calculation and + HW TCP checksum calculation for the port which the merged packets are + transmitted to. + +show port - gro +~~~~~~~~~~~~~~~ -gro set -~~~~~~~ +Display GRO configuration for a given port:: -Set max flow number and max packet number per-flow for GRO:: + testpmd> show port <port_id> gro + +set gro flush +~~~~~~~~~~~~~ - testpmd> gro set (max_flow_num) (max_item_num_per_flow) (port_id) +Set the cycle to flush the GROed packets from reassembly tables:: -The product of ``max_flow_num`` and ``max_item_num_per_flow`` is the max -number of packets a GRO table can store. + testpmd> set gro flush <cycles> -If current packet number is greater than or equal to the max value, GRO -will stop processing incoming packets. +When enable GRO, the csum forwarding engine performs GRO on received +packets, and the GROed packets are stored in reassembly tables. Users +can use this command to determine when the GROed packets are flushed +from the reassembly tables. + +The ``cycles`` is measured in GRO operation times. The csum forwarding +engine flushes the GROed packets from the tables every ``cycles`` GRO +operations. + +By default, the value of ``cycles`` is 1, which means flush GROed packets +from the reassembly tables as soon as one GRO operation finishes. The value +of ``cycles`` should be in the range of 1 to ``GRO_MAX_FLUSH_CYCLES``. + +Please note that the large value of ``cycles`` may cause the poor TCP/IP +stack performance. Because the GROed packets are delayed to arrive the +stack, thus causing more duplicated ACKs and TCP retransmissions. + +set port - gso +~~~~~~~~~~~~~~ + +Toggle per-port GSO support in ``csum`` forwarding engine:: + + testpmd> set port <port_id> gso on|off + +If enabled, the csum forwarding engine will perform GSO on supported IPv4 +packets, transmitted on the given port. + +If disabled, packets transmitted on the given port will not undergo GSO. +By default, GSO is disabled for all ports. + +.. note:: + + When GSO is enabled on a port, supported IPv4 packets transmitted on that + port undergo GSO. Afterwards, the segmented packets are represented by + multi-segment mbufs; however, the csum forwarding engine doesn't calculation + of checksums for GSO'd segments in SW. As a result, if users want correct + checksums in GSO segments, they should enable HW checksum calculation for + GSO-enabled ports. + + For example, HW checksum calculation for VxLAN GSO'd packets may be enabled + by setting the following options in the csum forwarding engine: + + testpmd> csum set outer_ip hw <port_id> + + testpmd> csum set ip hw <port_id> + + testpmd> csum set tcp hw <port_id> + +set gso segsz +~~~~~~~~~~~~~ + +Set the maximum GSO segment size (measured in bytes), which includes the +packet header and the packet payload for GSO-enabled ports (global):: + + testpmd> set gso segsz <length> + +show port - gso +~~~~~~~~~~~~~~~ + +Display the status of Generic Segmentation Offload for a given port:: + + testpmd> show port <port_id> gso mac_addr add ~~~~~~~~~~~~ @@ -1096,6 +1220,17 @@ For example, to set rx queue 2 on port 0 to mapping 5:: testpmd>set stat_qmap rx 0 2 5 +set xstats-hide-zero +~~~~~~~~~~~~~~~~~~~~ + +Set the option to hide zero values for xstats display:: + + testpmd> set xstats-hide-zero on|off + +.. note:: + + By default, the zero values are displayed for xstats. + set port - rx/tx (for VF) ~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -1697,6 +1832,23 @@ Enable/disable the E-tag support:: testpmd> port config (port_id|all) l2-tunnel E-tag (enable|disable) +port config pctype mapping +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Reset pctype mapping table:: + + testpmd> port config (port_id) pctype mapping reset + +Update hardware defined pctype to software defined flow type mapping table:: + + testpmd> port config (port_id) pctype mapping update (pctype_id_0[,pctype_id_1]*) (flow_type_id) + +where: + +* ``pctype_id_x``: hardware pctype id as index of bit in bitmask value of the pctype mapping table. + +* ``flow_type_id``: software flow type id as the index of the pctype mapping table. + Link Bonding Functions ---------------------- @@ -1917,6 +2069,261 @@ For example, to set the high bit in the register from the example above:: port 0 PCI register at offset 0xEE00: 0x8000000A (2147483658) +Traffic Management +------------------ + +The following section shows functions for configuring traffic management on +on the ethernet device through the use of generic TM API. + +show port traffic management capability +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Show traffic management capability of the port:: + + testpmd> show port tm cap (port_id) + +show port traffic management capability (hierarchy level) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Show traffic management hierarchy level capability of the port:: + + testpmd> show port tm cap (port_id) (level_id) + +show port traffic management capability (hierarchy node level) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Show the traffic management hierarchy node capability of the port:: + + testpmd> show port tm cap (port_id) (node_id) + +show port traffic management hierarchy node type +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Show the port traffic management hierarchy node type:: + + testpmd> show port tm node type (port_id) (node_id) + +show port traffic management hierarchy node stats +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Show the port traffic management hierarchy node statistics:: + + testpmd> show port tm node stats (port_id) (node_id) (clear) + +where: + +* ``clear``: When this parameter has a non-zero value, the statistics counters + are cleared (i.e. set to zero) immediately after they have been read, + otherwise the statistics counters are left untouched. + +Add port traffic management private shaper profile +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Add the port traffic management private shaper profile:: + + testpmd> add port tm node shaper profile (port_id) (shaper_profile_id) \ + (tb_rate) (tb_size) (packet_length_adjust) + +where: + +* ``shaper_profile id``: Shaper profile ID for the new profile. +* ``tb_rate``: Token bucket rate (bytes per second). +* ``tb_size``: Token bucket size (bytes). +* ``packet_length_adjust``: The value (bytes) to be added to the length of + each packet for the purpose of shaping. This parameter value can be used to + correct the packet length with the framing overhead bytes that are consumed + on the wire. + +Delete port traffic management private shaper profile +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Delete the port traffic management private shaper:: + + testpmd> del port tm node shaper profile (port_id) (shaper_profile_id) + +where: + +* ``shaper_profile id``: Shaper profile ID that needs to be deleted. + +Add port traffic management shared shaper +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Create the port traffic management shared shaper:: + + testpmd> add port tm node shared shaper (port_id) (shared_shaper_id) \ + (shaper_profile_id) + +where: + +* ``shared_shaper_id``: Shared shaper ID to be created. +* ``shaper_profile id``: Shaper profile ID for shared shaper. + +Set port traffic management shared shaper +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Update the port traffic management shared shaper:: + + testpmd> set port tm node shared shaper (port_id) (shared_shaper_id) \ + (shaper_profile_id) + +where: + +* ``shared_shaper_id``: Shared shaper ID to be update. +* ``shaper_profile id``: Shaper profile ID for shared shaper. + +Delete port traffic management shared shaper +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Delete the port traffic management shared shaper:: + + testpmd> del port tm node shared shaper (port_id) (shared_shaper_id) + +where: + +* ``shared_shaper_id``: Shared shaper ID to be deleted. + +Set port traffic management hiearchy node private shaper +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +set the port traffic management hierarchy node private shaper:: + + testpmd> set port tm node shaper profile (port_id) (node_id) \ + (shaper_profile_id) + +where: + +* ``shaper_profile id``: Private shaper profile ID to be enabled on the + hierarchy node. + +Add port traffic management WRED profile +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Create a new WRED profile:: + + testpmd> add port tm node wred profile (port_id) (wred_profile_id) \ + (color_g) (min_th_g) (max_th_g) (maxp_inv_g) (wq_log2_g) \ + (color_y) (min_th_y) (max_th_y) (maxp_inv_y) (wq_log2_y) \ + (color_r) (min_th_r) (max_th_r) (maxp_inv_r) (wq_log2_r) + +where: + +* ``wred_profile id``: Identifier for the newly create WRED profile +* ``color_g``: Packet color (green) +* ``min_th_g``: Minimum queue threshold for packet with green color +* ``max_th_g``: Minimum queue threshold for packet with green color +* ``maxp_inv_g``: Inverse of packet marking probability maximum value (maxp) +* ``wq_log2_g``: Negated log2 of queue weight (wq) +* ``color_y``: Packet color (yellow) +* ``min_th_y``: Minimum queue threshold for packet with yellow color +* ``max_th_y``: Minimum queue threshold for packet with yellow color +* ``maxp_inv_y``: Inverse of packet marking probability maximum value (maxp) +* ``wq_log2_y``: Negated log2 of queue weight (wq) +* ``color_r``: Packet color (red) +* ``min_th_r``: Minimum queue threshold for packet with yellow color +* ``max_th_r``: Minimum queue threshold for packet with yellow color +* ``maxp_inv_r``: Inverse of packet marking probability maximum value (maxp) +* ``wq_log2_r``: Negated log2 of queue weight (wq) + +Delete port traffic management WRED profile +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Delete the WRED profile:: + + testpmd> del port tm node wred profile (port_id) (wred_profile_id) + +Add port traffic management hierarchy nonleaf node +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Add nonleaf node to port traffic management hiearchy:: + + testpmd> add port tm nonleaf node (port_id) (node_id) (parent_node_id) \ + (priority) (weight) (level_id) (shaper_profile_id) \ + (n_sp_priorities) (stats_mask) (n_shared_shapers) \ + [(shared_shaper_0) (shared_shaper_1) ...] \ + +where: + +* ``parent_node_id``: Node ID of the parent. +* ``priority``: Node priority (highest node priority is zero). This is used by + the SP algorithm running on the parent node for scheduling this node. +* ``weight``: Node weight (lowest weight is one). The node weight is relative + to the weight sum of all siblings that have the same priority. It is used by + the WFQ algorithm running on the parent node for scheduling this node. +* ``level_id``: Hiearchy level of the node. +* ``shaper_profile_id``: Shaper profile ID of the private shaper to be used by + the node. +* ``n_sp_priorities``: Number of strict priorities. +* ``stats_mask``: Mask of statistics counter types to be enabled for this node. +* ``n_shared_shapers``: Number of shared shapers. +* ``shared_shaper_id``: Shared shaper id. + +Add port traffic management hierarchy leaf node +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Add leaf node to port traffic management hiearchy:: + + testpmd> add port tm leaf node (port_id) (node_id) (parent_node_id) \ + (priority) (weight) (level_id) (shaper_profile_id) \ + (cman_mode) (wred_profile_id) (stats_mask) (n_shared_shapers) \ + [(shared_shaper_id) (shared_shaper_id) ...] \ + +where: + +* ``parent_node_id``: Node ID of the parent. +* ``priority``: Node priority (highest node priority is zero). This is used by + the SP algorithm running on the parent node for scheduling this node. +* ``weight``: Node weight (lowest weight is one). The node weight is relative + to the weight sum of all siblings that have the same priority. It is used by + the WFQ algorithm running on the parent node for scheduling this node. +* ``level_id``: Hiearchy level of the node. +* ``shaper_profile_id``: Shaper profile ID of the private shaper to be used by + the node. +* ``cman_mode``: Congestion management mode to be enabled for this node. +* ``wred_profile_id``: WRED profile id to be enabled for this node. +* ``stats_mask``: Mask of statistics counter types to be enabled for this node. +* ``n_shared_shapers``: Number of shared shapers. +* ``shared_shaper_id``: Shared shaper id. + +Delete port traffic management hierarchy node +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Delete node from port traffic management hiearchy:: + + testpmd> del port tm node (port_id) (node_id) + +Update port traffic management hierarchy parent node +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Update port traffic management hierarchy parent node:: + + testpmd> set port tm node parent (port_id) (node_id) (parent_node_id) \ + (priority) (weight) + +This function can only be called after the hierarchy commit invocation. Its +success depends on the port support for this operation, as advertised through +the port capability set. This function is valid for all nodes of the traffic +management hierarchy except root node. + +Commit port traffic management hierarchy +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Commit the traffic management hierarchy on the port:: + + testpmd> port tm hierarchy commit (port_id) (clean_on_fail) + +where: + +* ``clean_on_fail``: When set to non-zero, hierarchy is cleared on function + call failure. On the other hand, hierarchy is preserved when this parameter + is equal to zero. + +Set port traffic management default hierarchy (tm forwarding mode) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +set the traffic management default hierarchy on the port:: + + testpmd> set port tm hierarchy default (port_id) + Filter Functions ---------------- @@ -2696,6 +3103,10 @@ This section lists supported pattern items and their attributes, if any. - ``thresh {unsigned}``: accuracy threshold. +- ``gtp``, ``gtpc``, ``gtpu``: match GTPv1 header. + + - ``teid {unsigned}``: tunnel endpoint identifier. + Actions list ^^^^^^^^^^^^ diff --git a/doc/guides/tools/cryptoperf.rst b/doc/guides/tools/cryptoperf.rst index 457f817c..7e126770 100644 --- a/doc/guides/tools/cryptoperf.rst +++ b/doc/guides/tools/cryptoperf.rst @@ -50,7 +50,8 @@ offload are still consumed by the test tool and included in the cycle-count. These cycles are consumed by retries and inefficient API calls enqueuing and dequeuing smaller bursts than specified by the cmdline parameter. This results in a larger cycle-count measurement and should not be interpreted as an offload -cost measurement. +cost measurement. Using "pmd-cyclecount" mode will give a better idea of +actual costs of hardware acceleration. On hardware devices the throughput measurement is not necessarily the maximum possible for the device, e.g. it may be necessary to use multiple cores to keep @@ -134,6 +135,7 @@ The following are the appication command-line options: throughput latency verify + pmd-cyclecount * ``--silent`` @@ -170,9 +172,11 @@ The following are the appication command-line options: * List of values, up to 32 values, separated in commas (i.e. ``--buffer-sz 32,64,128``) -* ``--segments-nb <n>`` +* ``--segment-sz <n>`` - Set the number of segments per packet. + Set the size of the segment to use, for Scatter Gather List testing. + By default, it is set to the size of the maximum buffer size, including the digest size, + so a single segment is created. * ``--devtype <name>`` @@ -186,9 +190,11 @@ The following are the appication command-line options: crypto_snow3g crypto_kasumi crypto_zuc + crypto_dpaa_sec crypto_dpaa2_sec crypto_armv8 crypto_scheduler + crypto_mrvl * ``--optype <name>`` @@ -325,6 +331,16 @@ The following are the appication command-line options: Set the size of digest. +* ``--desc-nb <n>`` + + Set number of descriptors for each crypto device. + +* ``--pmd-cyclecount-delay-ms <n>`` + + Add a delay (in milliseconds) between enqueue and dequeue in + pmd-cyclecount benchmarking mode (useful when benchmarking + hardware acceleration). + * ``--csv-friendly`` Enable test result output CSV friendly rather than human friendly. diff --git a/doc/guides/tools/testeventdev.rst b/doc/guides/tools/testeventdev.rst index 34b1c318..5aa2237b 100644 --- a/doc/guides/tools/testeventdev.rst +++ b/doc/guides/tools/testeventdev.rst @@ -106,10 +106,6 @@ The following are the application command-line options: Set the number of mbufs to be allocated from the mempool. -* ``--slcore <n>`` - - Set the scheduler lcore id.(Valid when eventdev is not RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED capable) - * ``--plcores <CORELIST>`` Set the list of cores to be used as producers. @@ -362,7 +358,6 @@ Supported application command line options are following:: --test --socket_id --pool_sz - --slcore (Valid when eventdev is not RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED capable) --plcores --wlcores --stlist @@ -379,8 +374,8 @@ Example command to run perf queue test: .. code-block:: console - sudo build/app/dpdk-test-eventdev --vdev=event_sw0 -- \ - --test=perf_queue --slcore=1 --plcores=2 --wlcore=3 --stlist=p --nb_pkts=0 + sudo build/app/dpdk-test-eventdev -c 0xf -s 0x1 --vdev=event_sw0 -- \ + --test=perf_queue --plcores=2 --wlcore=3 --stlist=p --nb_pkts=0 PERF_ATQ Test @@ -441,7 +436,6 @@ Supported application command line options are following:: --test --socket_id --pool_sz - --slcore (Valid when eventdev is not RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED capable) --plcores --wlcores --stlist diff --git a/doc/guides/xen/img/dpdk_xen_pkt_switch.png b/doc/guides/xen/img/dpdk_xen_pkt_switch.png Binary files differdeleted file mode 100644 index 32a6d161..00000000 --- a/doc/guides/xen/img/dpdk_xen_pkt_switch.png +++ /dev/null diff --git a/doc/guides/xen/img/grant_refs.png b/doc/guides/xen/img/grant_refs.png Binary files differdeleted file mode 100644 index baa34e1e..00000000 --- a/doc/guides/xen/img/grant_refs.png +++ /dev/null diff --git a/doc/guides/xen/img/grant_table.png b/doc/guides/xen/img/grant_table.png Binary files differdeleted file mode 100644 index c23e5fa7..00000000 --- a/doc/guides/xen/img/grant_table.png +++ /dev/null diff --git a/doc/guides/xen/index.rst b/doc/guides/xen/index.rst deleted file mode 100644 index cb43cd2f..00000000 --- a/doc/guides/xen/index.rst +++ /dev/null @@ -1,38 +0,0 @@ -.. BSD LICENSE - Copyright(c) 2010-2014 Intel Corporation. All rights reserved. - All rights reserved. - - Redistribution and use in source and binary forms, with or without - modification, are permitted provided that the following conditions - are met: - - * Redistributions of source code must retain the above copyright - notice, this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright - notice, this list of conditions and the following disclaimer in - the documentation and/or other materials provided with the - distribution. - * Neither the name of Intel Corporation nor the names of its - contributors may be used to endorse or promote products derived - from this software without specific prior written permission. - - THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS - "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT - LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR - A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT - OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, - SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT - LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, - DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY - THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -Xen Guide -========= - -.. toctree:: - :maxdepth: 2 - :numbered: - - pkt_switch diff --git a/doc/guides/xen/pkt_switch.rst b/doc/guides/xen/pkt_switch.rst deleted file mode 100644 index 717a04b2..00000000 --- a/doc/guides/xen/pkt_switch.rst +++ /dev/null @@ -1,470 +0,0 @@ -.. BSD LICENSE - Copyright(c) 2010-2014 Intel Corporation. All rights reserved. - All rights reserved. - - Redistribution and use in source and binary forms, with or without - modification, are permitted provided that the following conditions - are met: - - * Redistributions of source code must retain the above copyright - notice, this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright - notice, this list of conditions and the following disclaimer in - the documentation and/or other materials provided with the - distribution. - * Neither the name of Intel Corporation nor the names of its - contributors may be used to endorse or promote products derived - from this software without specific prior written permission. - - THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS - "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT - LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR - A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT - OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, - SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT - LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, - DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY - THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -DPDK Xen Based Packet-Switching Solution -======================================== - -Introduction ------------- - -DPDK provides a para-virtualization packet switching solution, based on the Xen hypervisor's Grant Table, Note 1, -which provides simple and fast packet switching capability between guest domains and host domain based on MAC address or VLAN tag. - -This solution is comprised of two components; -a Poll Mode Driver (PMD) as the front end in the guest domain and a switching back end in the host domain. -XenStore is used to exchange configure information between the PMD front end and switching back end, -including grant reference IDs for shared Virtio RX/TX rings, -MAC address, device state, and so on. XenStore is an information storage space shared between domains, -see further information on XenStore below. - -The front end PMD can be found in the DPDK directory lib/ librte_pmd_xenvirt and back end example in examples/vhost_xen. - -The PMD front end and switching back end use shared Virtio RX/TX rings as para- virtualized interface. -The Virtio ring is created by the front end, and Grant table references for the ring are passed to host. -The switching back end maps those grant table references and creates shared rings in a mapped address space. - -The following diagram describes the functionality of the DPDK Xen Packet- Switching Solution. - - -.. _figure_dpdk_xen_pkt_switch: - -.. figure:: img/dpdk_xen_pkt_switch.* - - Functionality of the DPDK Xen Packet Switching Solution. - - -Note 1 The Xen hypervisor uses a mechanism called a Grant Table to share memory between domains -(`http://wiki.xen.org/wiki/Grant Table <http://wiki.xen.org/wiki/Grant%20Table>`_). - -A diagram of the design is shown below, where "gva" is the Guest Virtual Address, -which is the data pointer of the mbuf, and "hva" is the Host Virtual Address: - - -.. _figure_grant_table: - -.. figure:: img/grant_table.* - - DPDK Xen Layout - - -In this design, a Virtio ring is used as a para-virtualized interface for better performance over a Xen private ring -when packet switching to and from a VM. -The additional performance is gained by avoiding a system call and memory map in each memory copy with a XEN private ring. - -Device Creation ---------------- - -Poll Mode Driver Front End -~~~~~~~~~~~~~~~~~~~~~~~~~~ - -* Mbuf pool allocation: - - To use a Xen switching solution, the DPDK application should use rte_mempool_gntalloc_create() - to reserve mbuf pools during initialization. - rte_mempool_gntalloc_create() creates a mempool with objects from memory allocated and managed via gntalloc/gntdev. - - The DPDK now supports construction of mempools from allocated virtual memory through the rte_mempool_xmem_create() API. - - This front end constructs mempools based on memory allocated through the xen_gntalloc driver. - rte_mempool_gntalloc_create() allocates Grant pages, maps them to continuous virtual address space, - and calls rte_mempool_xmem_create() to build mempools. - The Grant IDs for all Grant pages are passed to the host through XenStore. - -* Virtio Ring Creation: - - The Virtio queue size is defined as 256 by default in the VQ_DESC_NUM macro. - Using the queue setup function, - Grant pages are allocated based on ring size and are mapped to continuous virtual address space to form the Virtio ring. - Normally, one ring is comprised of several pages. - Their Grant IDs are passed to the host through XenStore. - - There is no requirement that this memory be physically continuous. - -* Interrupt and Kick: - - There are no interrupts in DPDK Xen Switching as both front and back ends work in polling mode. - There is no requirement for notification. - -* Feature Negotiation: - - Currently, feature negotiation through XenStore is not supported. - -* Packet Reception & Transmission: - - With mempools and Virtio rings created, the front end can operate Virtio devices, - as it does in Virtio PMD for KVM Virtio devices with the exception that the host - does not require notifications or deal with interrupts. - -XenStore is a database that stores guest and host information in the form of (key, value) pairs. -The following is an example of the information generated during the startup of the front end PMD in a guest VM (domain ID 1): - -.. code-block:: console - - xenstore -ls /local/domain/1/control/dpdk - 0_mempool_gref="3042,3043,3044,3045" - 0_mempool_va="0x7fcbc6881000" - 0_tx_vring_gref="3049" - 0_rx_vring_gref="3053" - 0_ether_addr="4e:0b:d0:4e:aa:f1" - 0_vring_flag="3054" - ... - -Multiple mempools and multiple Virtios may exist in the guest domain, the first number is the index, starting from zero. - -The idx#_mempool_va stores the guest virtual address for mempool idx#. - -The idx#_ether_adder stores the MAC address of the guest Virtio device. - -For idx#_rx_ring_gref, idx#_tx_ring_gref, and idx#_mempool_gref, the value is a list of Grant references. -Take idx#_mempool_gref node for example, the host maps those Grant references to a continuous virtual address space. -The real Grant reference information is stored in this virtual address space, -where (gref, pfn) pairs follow each other with -1 as the terminator. - - -.. _figure_grant_refs: - -.. figure:: img/grant_refs.* - - Mapping Grant references to a continuous virtual address space - - -After all gref# IDs are retrieved, the host maps them to a continuous virtual address space. -With the guest mempool virtual address, the host establishes 1:1 address mapping. -With multiple guest mempools, the host establishes multiple address translation regions. - -Switching Back End -~~~~~~~~~~~~~~~~~~ - -The switching back end monitors changes in XenStore. -When the back end detects that a new Virtio device has been created in a guest domain, it will: - -#. Retrieve Grant and configuration information from XenStore. - -#. Map and create a Virtio ring. - -#. Map mempools in the host and establish address translation between the guest address and host address. - -#. Select a free VMDQ pool, set its affinity with the Virtio device, and set the MAC/ VLAN filter. - -Packet Reception -~~~~~~~~~~~~~~~~ - -When packets arrive from an external network, the MAC?VLAN filter classifies packets into queues in one VMDQ pool. -As each pool is bonded to a Virtio device in some guest domain, the switching back end will: - -#. Fetch an available entry from the Virtio RX ring. - -#. Get gva, and translate it to hva. - -#. Copy the contents of the packet to the memory buffer pointed to by gva. - -The DPDK application in the guest domain, based on the PMD front end, -is polling the shared Virtio RX ring for available packets and receives them on arrival. - -Packet Transmission -~~~~~~~~~~~~~~~~~~~ - -When a Virtio device in one guest domain is to transmit a packet, -it puts the virtual address of the packet's data area into the shared Virtio TX ring. - -The packet switching back end is continuously polling the Virtio TX ring. -When new packets are available for transmission from a guest, it will: - -#. Fetch an available entry from the Virtio TX ring. - -#. Get gva, and translate it to hva. - -#. Copy the packet from hva to the host mbuf's data area. - -#. Compare the destination MAC address with all the MAC addresses of the Virtio devices it manages. - If a match exists, it directly copies the packet to the matched VIrtio RX ring. - Otherwise, it sends the packet out through hardware. - -.. note:: - - The packet switching back end is for demonstration purposes only. - The user could implement their switching logic based on this example. - In this example, only one physical port on the host is supported. - Multiple segments are not supported. The biggest mbuf supported is 4KB. - When the back end is restarted, all front ends must also be restarted. - -Running the Application ------------------------ - -The following describes the steps required to run the application. - -Validated Environment -~~~~~~~~~~~~~~~~~~~~~ - -Host: - - Xen-hypervisor: 4.2.2 - - Distribution: Fedora release 18 - - Kernel: 3.10.0 - - Xen development package (including Xen, Xen-libs, xen-devel): 4.2.3 - -Guest: - - Distribution: Fedora 16 and 18 - - Kernel: 3.6.11 - -Xen Host Prerequisites -~~~~~~~~~~~~~~~~~~~~~~ - -Note that the following commands might not be the same on different Linux* distributions. - -* Install xen-devel package: - - .. code-block:: console - - yum install xen-devel.x86_64 - -* Start xend if not already started: - - .. code-block:: console - - /etc/init.d/xend start - -* Mount xenfs if not already mounted: - - .. code-block:: console - - mount -t xenfs none /proc/xen - -* Enlarge the limit for xen_gntdev driver: - - .. code-block:: console - - modprobe -r xen_gntdev - modprobe xen_gntdev limit=1000000 - -.. note:: - - The default limit for earlier versions of the xen_gntdev driver is 1024. - That is insufficient to support the mapping of multiple Virtio devices into multiple VMs, - so it is necessary to enlarge the limit by reloading this module. - The default limit of recent versions of xen_gntdev is 1048576. - The rough calculation of this limit is: - - limit=nb_mbuf# * VM#. - - In DPDK examples, nb_mbuf# is normally 8192. - -Building and Running the Switching Backend -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -#. Edit config/common_linuxapp, and change the default configuration value for the following two items: - - .. code-block:: console - - CONFIG_RTE_LIBRTE_XEN_DOM0=y - CONFIG RTE_LIBRTE_PMD_XENVIRT=n - -#. Build the target: - - .. code-block:: console - - make install T=x86_64-native-linuxapp-gcc - -#. Ensure that RTE_SDK and RTE_TARGET are correctly set. Build the switching example: - - .. code-block:: console - - make -C examples/vhost_xen/ - -#. Load the Xen DPDK memory management module and preallocate memory: - - .. code-block:: console - - insmod ./x86_64-native-linuxapp-gcc/build/lib/librte_eal/linuxapp/xen_dom0/rte_dom0_mm.ko - echo 2048> /sys/kernel/mm/dom0-mm/memsize-mB/memsize - - .. note:: - - On Xen Dom0, there is no hugepage support. - Under Xen Dom0, the DPDK uses a special memory management kernel module - to allocate chunks of physically continuous memory. - Refer to the *DPDK Getting Started Guide* for more information on memory management in the DPDK. - In the above command, 4 GB memory is reserved (2048 of 2 MB pages) for DPDK. - -#. Load uio_pci_generic and bind one Intel NIC controller to it: - - .. code-block:: console - - modprobe uio_pci_generic - python usertools/dpdk-devbind.py -b uio_pci_generic 0000:09:00:00.0 - - In this case, 0000:09:00.0 is the PCI address for the NIC controller. - -#. Run the switching back end example: - - .. code-block:: console - - examples/vhost_xen/build/vhost-switch -l 0-3 -n 3 --xen-dom0 -- -p1 - -.. note:: - - The -xen-dom0 option instructs the DPDK to use the Xen kernel module to allocate memory. - -Other Parameters: - -* -vm2vm - - The vm2vm parameter enables/disables packet switching in software. - Disabling vm2vm implies that on a VM packet transmission will always go to the Ethernet port - and will not be switched to another VM - -* -Stats - - The Stats parameter controls the printing of Virtio-net device statistics. - The parameter specifies the interval (in seconds) at which to print statistics, - an interval of 0 seconds will disable printing statistics. - -Xen PMD Frontend Prerequisites -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -#. Install xen-devel package for accessing XenStore: - - .. code-block:: console - - yum install xen-devel.x86_64 - -#. Mount xenfs, if it is not already mounted: - - .. code-block:: console - - mount -t xenfs none /proc/xen - -#. Enlarge the default limit for xen_gntalloc driver: - - .. code-block:: console - - modprobe -r xen_gntalloc - modprobe xen_gntalloc limit=6000 - -.. note:: - - Before the Linux kernel version 3.8-rc5, Jan 15th 2013, - a critical defect occurs when a guest is heavily allocating Grant pages. - The Grant driver allocates fewer pages than expected which causes kernel memory corruption. - This happens, for example, when a guest uses the v1 format of a Grant table entry and allocates - more than 8192 Grant pages (this number might be different on different hypervisor versions). - To work around this issue, set the limit for gntalloc driver to 6000. - (The kernel normally allocates hundreds of Grant pages with one Xen front end per virtualized device). - If the kernel allocates a lot of Grant pages, for example, if the user uses multiple net front devices, - it is best to upgrade the Grant alloc driver. - This defect has been fixed in kernel version 3.8-rc5 and later. - -Building and Running the Front End -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -#. Edit config/common_linuxapp, and change the default configuration value: - - .. code-block:: console - - CONFIG_RTE_LIBRTE_XEN_DOM0=n - CONFIG_RTE_LIBRTE_PMD_XENVIRT=y - -#. Build the package: - - .. code-block:: console - - make install T=x86_64-native-linuxapp-gcc - -#. Enable hugepages. Refer to the *DPDK Getting Started Guide* for instructions on - how to use hugepages in the DPDK. - -#. Run TestPMD. Refer to *DPDK TestPMD Application User Guide* for detailed parameter usage. - - .. code-block:: console - - ./x86_64-native-linuxapp-gcc/app/testpmd -l 0-3 -n 4 --vdev="net_xenvirt0,mac=00:00:00:00:00:11" - testpmd>set fwd mac - testpmd>start - - As an example to run two TestPMD instances over 2 Xen Virtio devices: - - .. code-block:: console - - --vdev="net_xenvirt0,mac=00:00:00:00:00:11" --vdev="net_xenvirt1;mac=00:00:00:00:00:22" - - -Usage Examples: Injecting a Packet Stream Using a Packet Generator -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Loopback Mode -^^^^^^^^^^^^^ - -Run TestPMD in a guest VM: - -.. code-block:: console - - ./x86_64-native-linuxapp-gcc/app/testpmd -l 0-3 -n 4 --vdev="net_xenvirt0,mac=00:00:00:00:00:11" -- -i --eth-peer=0,00:00:00:00:00:22 - testpmd> set fwd mac - testpmd> start - -Example output of the vhost_switch would be: - -.. code-block:: console - - DATA:(0) MAC_ADDRESS 00:00:00:00:00:11 and VLAN_TAG 1000 registered. - -The above message indicates that device 0 has been registered with MAC address 00:00:00:00:00:11 and VLAN tag 1000. -Any packets received on the NIC with these values is placed on the device's receive queue. - -Configure a packet stream in the packet generator, set the destination MAC address to 00:00:00:00:00:11, and VLAN to 1000, -the guest Virtio receives these packets and sends them out with destination MAC address 00:00:00:00:00:22. - -Inter-VM Mode -^^^^^^^^^^^^^ - -Run TestPMD in guest VM1: - -.. code-block:: console - - ./x86_64-native-linuxapp-gcc/app/testpmd -l 0-3 -n 4 --vdev="net_xenvirt0,mac=00:00:00:00:00:11" -- -i --eth-peer=0,00:00:00:00:00:22 -- -i - -Run TestPMD in guest VM2: - -.. code-block:: console - - ./x86_64-native-linuxapp-gcc/app/testpmd -l 0-3 -n 4 --vdev="net_xenvirt0,mac=00:00:00:00:00:22" -- -i --eth-peer=0,00:00:00:00:00:33 - -Configure a packet stream in the packet generator, and set the destination MAC address to 00:00:00:00:00:11 and VLAN to 1000. -The packets received in Virtio in guest VM1 will be forwarded to Virtio in guest VM2 and -then sent out through hardware with destination MAC address 00:00:00:00:00:33. - -The packet flow is: - -packet generator->Virtio in guest VM1->switching backend->Virtio in guest VM2->switching backend->wire |
