diff options
Diffstat (limited to 'doc/guides')
115 files changed, 16585 insertions, 781 deletions
diff --git a/doc/guides/conf.py b/doc/guides/conf.py index c3cd0bd6..39880752 100644 --- a/doc/guides/conf.py +++ b/doc/guides/conf.py @@ -47,11 +47,17 @@ except: # Python 3. import configparser +try: + import sphinx_rtd_theme -project = 'Data Plane Development Kit' - -if LooseVersion(sphinx_version) >= LooseVersion('1.3.1'): html_theme = "sphinx_rtd_theme" + html_theme_path = [sphinx_rtd_theme.get_html_theme_path()] +except: + print('Install the sphinx ReadTheDocs theme for improved html documentation ' + 'layout: pip install sphinx_rtd_theme') + pass + +project = 'Data Plane Development Kit' html_logo = '../logo/DPDK_logo_vertical_rev_small.png' latex_logo = '../logo/DPDK_logo_horizontal_tag.png' html_add_permalinks = "" diff --git a/doc/guides/contributing/documentation.rst b/doc/guides/contributing/documentation.rst index 4c85da7b..cddbd7bb 100644 --- a/doc/guides/contributing/documentation.rst +++ b/doc/guides/contributing/documentation.rst @@ -282,33 +282,21 @@ The additional guidelines below reiterate or expand upon those guidelines. Line Length ~~~~~~~~~~~ -* The recommended style for the DPDK documentation is to put sentences on separate lines. - This allows for easier reviewing of patches. - Multiple sentences which are not separated by a blank line are joined automatically into paragraphs, for example:: +* Lines in sentences should be less than 80 characters and wrapped at + words. Multiple sentences which are not separated by a blank line are joined + automatically into paragraphs. - Here is an example sentence. - Long sentences over the limit shown below can be wrapped onto - a new line. - These three sentences will be joined into the same paragraph. +* Lines in literal blocks **must** be less than 80 characters since + they are not wrapped by the document formatters and can exceed the page width + in PDF documents. - This is a new paragraph, since it is separated from the - previous paragraph by a blank line. + Long literal command lines can be shown wrapped with backslashes. For + example:: - This would be rendered as follows: - - *Here is an example sentence. - Long sentences over the limit shown below can be wrapped onto - a new line. - These three sentences will be joined into the same paragraph.* - - *This is a new paragraph, since it is separated from the - previous paragraph by a blank line.* - - -* Long sentences should be wrapped at 120 characters +/- 10 characters. They should be wrapped at words. - -* Lines in literal blocks must by less than 80 characters since they aren't wrapped by the document formatters - and can exceed the page width in PDF documents. + testpmd -l 2-3 -n 4 \ + --vdev=virtio_user0,path=/dev/vhost-net,queues=2,queue_size=1024 \ + -- -i --txqflags=0x0 --disable-hw-vlan --enable-lro \ + --enable-rx-cksum --txq=2 --rxq=2 --rxd=1024 --txd=1024 Whitespace diff --git a/doc/guides/contributing/patches.rst b/doc/guides/contributing/patches.rst index 84a5dab8..27e218b2 100644 --- a/doc/guides/contributing/patches.rst +++ b/doc/guides/contributing/patches.rst @@ -207,18 +207,21 @@ Here are some guidelines for the body of a commit message: * The text of the commit message should be wrapped at 72 characters. -* When fixing a regression, it is a good idea to reference the id of the commit which introduced the bug. - You can generate the required text using the following git alias:: +* When fixing a regression, it is required to reference the id of the commit + which introduced the bug, and put the original author of that commit on CC. + You can generate the required lines using the following git alias, which prints + the commit SHA and the author of the original code:: - git config alias.fixline "log -1 --abbrev=12 --format='Fixes: %h (\"%s\")'" + git config alias.fixline "log -1 --abbrev=12 --format='Fixes: %h (\"%s\")%nCc: %ae'" - The ``Fixes:`` line can then be added to the commit message:: + The output of ``git fixline <SHA>`` must then be added to the commit message:: - doc: fix vhost sample parameter + doc: fix some parameter description - Update the docs to reflect removed dev-index. + Update the docs, fixing description of some parameter. - Fixes: 17b8320a3e11 ("vhost: remove index parameter") + Fixes: abcdefgh1234 ("doc: add some parameter") + Cc: author@example.com Signed-off-by: Alex Smith <alex.smith@example.com> @@ -359,7 +362,7 @@ Examples of configs are:: x86_64-native-linuxapp-gcc+next+shared x86_64-native-linuxapp-clang+shared -The builds can be modifies via the following environmental variables: +The builds can be modified via the following environmental variables: * ``DPDK_BUILD_TEST_CONFIGS`` (target1+option1+option2 target2) * ``DPDK_DEP_CFLAGS`` diff --git a/doc/guides/cryptodevs/aesni_gcm.rst b/doc/guides/cryptodevs/aesni_gcm.rst index 84cdc52a..6e9160aa 100644 --- a/doc/guides/cryptodevs/aesni_gcm.rst +++ b/doc/guides/cryptodevs/aesni_gcm.rst @@ -1,5 +1,5 @@ .. BSD LICENSE - Copyright(c) 2016 Intel Corporation. All rights reserved. + Copyright(c) 2016-2017 Intel Corporation. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -32,36 +32,68 @@ AES-NI GCM Crypto Poll Mode Driver The AES-NI GCM PMD (**librte_pmd_aesni_gcm**) provides poll mode crypto driver -support for utilizing Intel ISA-L crypto library, which provides operation acceleration -through the AES-NI instruction sets for AES-GCM authenticated cipher algorithm. +support for utilizing Intel multi buffer library (see AES-NI Multi-buffer PMD documentation +to learn more about it, including installation). Features -------- AESNI GCM PMD has support for: -Cipher algorithms: - -* RTE_CRYPTO_CIPHER_AES_GCM - Authentication algorithms: -* RTE_CRYPTO_AUTH_AES_GCM * RTE_CRYPTO_AUTH_AES_GMAC +AEAD algorithms: + +* RTE_CRYPTO_AEAD_AES_GCM + + +Limitations +----------- + +* Chained mbufs are supported but only out-of-place (destination mbuf must be contiguous). +* Cipher only is not supported. + + Installation ------------ -To build DPDK with the AESNI_GCM_PMD the user is required to install -the ``libisal_crypto`` library in the build environment. -For download and more details please visit `<https://github.com/01org/isa-l_crypto>`_. +To build DPDK with the AESNI_GCM_PMD the user is required to download the multi-buffer +library from `here <https://github.com/01org/intel-ipsec-mb>`_ +and compile it on their user system before building DPDK. +The latest version of the library supported by this PMD is v0.46, which +can be downloaded in `<https://github.com/01org/intel-ipsec-mb/archive/v0.46.zip>`_. + +.. code-block:: console + + make + +As a reference, the following table shows a mapping between the past DPDK versions +and the external crypto libraries supported by them: + +.. _table_aesni_gcm_versions: + +.. table:: DPDK and external crypto library version compatibility + + ============= ================================ + DPDK version Crypto library version + ============= ================================ + 16.04 - 16.11 Multi-buffer library 0.43 - 0.44 + 17.02 - 17.05 ISA-L Crypto v2.18 + 17.08+ Multi-buffer library 0.46+ + ============= ================================ + Initialization -------------- In order to enable this virtual crypto PMD, user must: -* Install the ISA-L crypto library (explained in Installation section). +* Export the environmental variable AESNI_MULTI_BUFFER_LIB_PATH with the path where + the library was extracted. + +* Build the multi buffer library (explained in Installation section). * Set CONFIG_RTE_LIBRTE_PMD_AESNI_GCM=y in config/common_base. @@ -84,11 +116,5 @@ Example: .. code-block:: console - ./l2fwd-crypto -l 6 -n 4 --vdev="crypto_aesni_gcm,socket_id=1,max_nb_sessions=128" - -Limitations ------------ - -* Chained mbufs are supported but only out-of-place (destination mbuf must be contiguous). -* Hash only is not supported. -* Cipher only is not supported. + ./l2fwd-crypto -l 1 -n 4 --vdev="crypto_aesni_gcm,socket_id=0,max_nb_sessions=128" \ + -- -p 1 --cdev SW --chain AEAD --aead_algo "aes-gcm" diff --git a/doc/guides/cryptodevs/aesni_mb.rst b/doc/guides/cryptodevs/aesni_mb.rst index ecb52a10..b3b937fe 100644 --- a/doc/guides/cryptodevs/aesni_mb.rst +++ b/doc/guides/cryptodevs/aesni_mb.rst @@ -1,5 +1,5 @@ .. BSD LICENSE - Copyright(c) 2015 Intel Corporation. All rights reserved. + Copyright(c) 2015-2017 Intel Corporation. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -68,7 +68,7 @@ Limitations * Chained mbufs are not supported. * Only in-place is currently supported (destination address is the same as source address). -* Only supports session-oriented API implementation (session-less APIs are not supported). + Installation ------------ @@ -76,8 +76,8 @@ Installation To build DPDK with the AESNI_MB_PMD the user is required to download the multi-buffer library from `here <https://github.com/01org/intel-ipsec-mb>`_ and compile it on their user system before building DPDK. -The latest version of the library supported by this PMD is v0.45, which -can be downloaded in `<https://github.com/01org/intel-ipsec-mb/archive/v0.45.zip>`_. +The latest version of the library supported by this PMD is v0.46, which +can be downloaded in `<https://github.com/01org/intel-ipsec-mb/archive/v0.46.zip>`_. .. code-block:: console @@ -95,7 +95,7 @@ and the Multi-Buffer library version supported by them: ============= ============================ 2.2 - 16.11 0.43 - 0.44 17.02 0.44 - 17.05 0.45 + 17.05+ 0.45+ ============= ============================ @@ -130,4 +130,18 @@ Example: .. code-block:: console - ./l2fwd-crypto -l 6 -n 4 --vdev="crypto_aesni_mb,socket_id=1,max_nb_sessions=128" + ./l2fwd-crypto -l 1 -n 4 --vdev="crypto_aesni_mb,socket_id=0,max_nb_sessions=128" \ + -- -p 1 --cdev SW --chain CIPHER_HASH --cipher_algo "aes-cbc" --auth_algo "sha1-hmac" + +Extra notes +----------- + +For AES Counter mode (AES-CTR), the library supports two different sizes for Initialization +Vector (IV): + +* 12 bytes: used mainly for IPSec, as it requires 12 bytes from the user, which internally + are appended the counter block (4 bytes), which is set to 1 for the first block + (no padding required from the user) + +* 16 bytes: when passing 16 bytes, the library will take them and use the last 4 bytes + as the initial counter block for the first block. diff --git a/doc/guides/cryptodevs/armv8.rst b/doc/guides/cryptodevs/armv8.rst index de63793f..b1107114 100644 --- a/doc/guides/cryptodevs/armv8.rst +++ b/doc/guides/cryptodevs/armv8.rst @@ -1,5 +1,5 @@ .. BSD LICENSE - Copyright (C) Cavium networks Ltd. 2017. + Copyright (C) Cavium, Inc. 2017. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -11,7 +11,7 @@ notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. - * Neither the name of Cavium networks nor the names of its + * Neither the name of Cavium, Inc nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. diff --git a/doc/guides/cryptodevs/dpaa2_sec.rst b/doc/guides/cryptodevs/dpaa2_sec.rst index becb910e..18d980eb 100644 --- a/doc/guides/cryptodevs/dpaa2_sec.rst +++ b/doc/guides/cryptodevs/dpaa2_sec.rst @@ -1,5 +1,5 @@ .. BSD LICENSE - Copyright(c) 2016 NXP. All rights reserved. + Copyright 2016 NXP. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -126,7 +126,7 @@ fits in the DPAA2 Bus model Features -------- -The DPAA2 PMD has support for: +The DPAA2_SEC PMD has support for: Cipher algorithms: @@ -134,6 +134,9 @@ Cipher algorithms: * ``RTE_CRYPTO_CIPHER_AES128_CBC`` * ``RTE_CRYPTO_CIPHER_AES192_CBC`` * ``RTE_CRYPTO_CIPHER_AES256_CBC`` +* ``RTE_CRYPTO_CIPHER_AES128_CTR`` +* ``RTE_CRYPTO_CIPHER_AES192_CTR`` +* ``RTE_CRYPTO_CIPHER_AES256_CTR`` Hash algorithms: @@ -144,6 +147,10 @@ Hash algorithms: * ``RTE_CRYPTO_AUTH_SHA512_HMAC`` * ``RTE_CRYPTO_AUTH_MD5_HMAC`` +AEAD algorithms: + +* ``RTE_CRYPTO_AEAD_AES_GCM`` + Supported DPAA2 SoCs -------------------- diff --git a/doc/guides/cryptodevs/features/aesni_gcm.ini b/doc/guides/cryptodevs/features/aesni_gcm.ini index 5d9e119d..bacd94e3 100644 --- a/doc/guides/cryptodevs/features/aesni_gcm.ini +++ b/doc/guides/cryptodevs/features/aesni_gcm.ini @@ -7,7 +7,9 @@ Symmetric crypto = Y Sym operation chaining = Y CPU AESNI = Y - +CPU SSE = Y +CPU AVX = Y +CPU AVX2 = Y ; ; Supported crypto algorithms of the 'aesni_gcm' crypto driver. ; @@ -24,4 +26,5 @@ AES GMAC = Y ; [AEAD] AES GCM (128) = Y +AES GCM (192) = Y AES GCM (256) = Y diff --git a/doc/guides/cryptodevs/features/dpaa2_sec.ini b/doc/guides/cryptodevs/features/dpaa2_sec.ini index db0ea4f9..c3bb3ddc 100644 --- a/doc/guides/cryptodevs/features/dpaa2_sec.ini +++ b/doc/guides/cryptodevs/features/dpaa2_sec.ini @@ -15,6 +15,9 @@ HW Accelerated = Y AES CBC (128) = Y AES CBC (192) = Y AES CBC (256) = Y +AES CTR (128) = Y +AES CTR (192) = Y +AES CTR (256) = Y 3DES CBC = Y ; @@ -29,6 +32,9 @@ SHA384 HMAC = Y SHA512 HMAC = Y ; -; Supported AEAD algorithms of the 'openssl' crypto driver. +; Supported AEAD algorithms of the 'dpaa2_sec' crypto driver. ; [AEAD] +AES GCM (128) = Y +AES GCM (192) = Y +AES GCM (256) = Y diff --git a/doc/guides/cryptodevs/kasumi.rst b/doc/guides/cryptodevs/kasumi.rst index bff9321e..573312b4 100644 --- a/doc/guides/cryptodevs/kasumi.rst +++ b/doc/guides/cryptodevs/kasumi.rst @@ -51,7 +51,7 @@ Limitations ----------- * Chained mbufs are not supported. -* KASUMI(F9) supported only if hash offset field is byte-aligned. +* KASUMI(F9) supported only if hash offset and length field is byte-aligned. * In-place bit-level operations for KASUMI(F8) are not supported (if length and/or offset of data to be ciphered is not byte-aligned). @@ -70,6 +70,18 @@ on their system before building DPDK:: make +**Note**: When encrypting with KASUMI F8, by default the library +encrypts full blocks of 8 bytes, regardless the number of bytes to +be encrypted provided (which leads to a possible buffer overflow). +To avoid this situation, it is necessary not to pass +3GPP_SAFE_BUFFERS as a compilation flag. +Also, this is required when using chained operations +(cipher-then-auth/auth-then-cipher). +For this, in the Makefile of the library, make sure that this flag +is commented out:: + + #EXTRA_CFLAGS += -D_3GPP_SAFE_BUFFERS + **Note**: To build the PMD as a shared library, the libsso_kasumi library must be built as follows:: @@ -107,4 +119,22 @@ Example: .. code-block:: console - ./l2fwd-crypto -l 6 -n 4 --vdev="crypto_kasumi,socket_id=1,max_nb_sessions=128" + ./l2fwd-crypto -l 1 -n 4 --vdev="crypto_kasumi,socket_id=0,max_nb_sessions=128" \ + -- -p 1 --cdev SW --chain CIPHER_ONLY --cipher_algo "kasumi-f8" + +Extra notes on KASUMI F9 +------------------------ + +When using KASUMI F9 authentication algorithm, the input buffer must be +constructed according to the 3GPP KASUMI specifications (section 4.4, page 13): +`<http://cryptome.org/3gpp/35201-900.pdf>`_. +Input buffer has to have COUNT (4 bytes), FRESH (4 bytes), MESSAGE and DIRECTION (1 bit) +concatenated. After the DIRECTION bit, a single '1' bit is appended, followed by +between 0 and 7 '0' bits, so that the total length of the buffer is multiple of 8 bits. +Note that the actual message can be any length, specified in bits. + +Once this buffer is passed this way, when creating the crypto operation, +length of data to authenticate (op.sym.auth.data.length) must be the length +of all the items described above, including the padding at the end. +Also, offset of data to authenticate (op.sym.auth.data.offset) +must be such that points at the start of the COUNT bytes. diff --git a/doc/guides/cryptodevs/null.rst b/doc/guides/cryptodevs/null.rst index 4a3bfdfd..03a3ddc0 100644 --- a/doc/guides/cryptodevs/null.rst +++ b/doc/guides/cryptodevs/null.rst @@ -93,4 +93,5 @@ Example: .. code-block:: console - ./l2fwd-crypto -l 6 -n 4 --vdev="crypto_null,socket_id=1,max_nb_sessions=128" + ./l2fwd-crypto -l 1 -n 4 --vdev="crypto_null,socket_id=0,max_nb_sessions=128" \ + -- -p 1 --cdev SW --chain CIPHER_ONLY --cipher_algo "null" diff --git a/doc/guides/cryptodevs/openssl.rst b/doc/guides/cryptodevs/openssl.rst index e3419151..f18a4567 100644 --- a/doc/guides/cryptodevs/openssl.rst +++ b/doc/guides/cryptodevs/openssl.rst @@ -48,7 +48,6 @@ Supported cipher algorithms: * ``RTE_CRYPTO_CIPHER_AES_CBC`` * ``RTE_CRYPTO_CIPHER_AES_CTR`` * ``RTE_CRYPTO_CIPHER_3DES_CTR`` -* ``RTE_CRYPTO_CIPHER_AES_GCM`` * ``RTE_CRYPTO_CIPHER_DES_DOCSISBPI`` Supported authentication algorithms: @@ -66,6 +65,9 @@ Supported authentication algorithms: * ``RTE_CRYPTO_AUTH_SHA384_HMAC`` * ``RTE_CRYPTO_AUTH_SHA512_HMAC`` +Supported AEAD algorithms: +* ``RTE_CRYPTO_AEAD_AES_GCM`` + Installation ------------ diff --git a/doc/guides/cryptodevs/qat.rst b/doc/guides/cryptodevs/qat.rst index 21b56fc6..a3fce7b8 100644 --- a/doc/guides/cryptodevs/qat.rst +++ b/doc/guides/cryptodevs/qat.rst @@ -55,7 +55,6 @@ Cipher algorithms: * ``RTE_CRYPTO_CIPHER_AES192_CTR`` * ``RTE_CRYPTO_CIPHER_AES256_CTR`` * ``RTE_CRYPTO_CIPHER_SNOW3G_UEA2`` -* ``RTE_CRYPTO_CIPHER_AES_GCM`` * ``RTE_CRYPTO_CIPHER_NULL`` * ``RTE_CRYPTO_CIPHER_KASUMI_F8`` * ``RTE_CRYPTO_CIPHER_DES_CBC`` @@ -78,14 +77,16 @@ Hash algorithms: * ``RTE_CRYPTO_AUTH_AES_GMAC`` * ``RTE_CRYPTO_AUTH_ZUC_EIA3`` +Supported AEAD algorithms: +* ``RTE_CRYPTO_AEAD_AES_GCM`` + Limitations ----------- -* Hash only is not supported except SNOW 3G UIA2 and KASUMI F9. * Only supports the session-oriented API implementation (session-less APIs are not supported). -* SNOW 3G (UEA2) and KASUMI (F8) supported only if cipher length, cipher offset fields are byte-aligned. -* SNOW 3G (UIA2) and KASUMI (F9) supported only if hash length, hash offset fields are byte-aligned. +* SNOW 3G (UEA2), KASUMI (F8) and ZUC (EEA3) supported only if cipher length and offset fields are byte-multiple. +* SNOW 3G (UIA2) and ZUC (EIA3) supported only if hash length and offset fields are byte-multiple. * No BSD support as BSD QAT kernel driver not available. * ZUC EEA3/EIA3 is not supported by dh895xcc devices * Maximum additional authenticated data (AAD) for GCM is 240 bytes long. @@ -112,21 +113,21 @@ available kernel drivers and device ids are : .. _table_qat_pmds_drivers: -.. table:: QAT devices and drivers +.. table:: QAT device generations, devices and drivers - +----------+--------+---------------+------------+--------+---------+--------+------------+ - | Device | Driver | Kernel Module | Pci Driver | PF Did | Num PFs | Vf Did | VFs per PF | - +==========+========+===============+============+========+=========+========+============+ - | DH895xCC | 01.org | icp_qa_al | n/a | 435 | 1 | 443 | 32 | - +----------+--------+---------------+------------+--------+---------+--------+------------+ - | DH895xCC | 4.4+ | qat_dh895xcc | dh895xcc | 435 | 1 | 443 | 32 | - +----------+--------+---------------+------------+--------+---------+--------+------------+ - | C62x | 4.5+ | qat_c62x | c6xx | 37c8 | 3 | 37c9 | 16 | - +----------+--------+---------------+------------+--------+---------+--------+------------+ - | C3xxx | 4.5+ | qat_c3xxx | c3xxx | 19e2 | 1 | 19e3 | 16 | - +----------+--------+---------------+------------+--------+---------+--------+------------+ - | D15xx | p | qat_d15xx | d15xx | 6f54 | 1 | 6f55 | 16 | - +----------+--------+---------------+------------+--------+---------+--------+------------+ + +-----+----------+--------+---------------+------------+--------+------+--------+--------+ + | Gen | Device | Driver | Kernel Module | Pci Driver | PF Did | #PFs | Vf Did | VFs/PF | + +=====+==========+========+===============+============+========+======+========+========+ + | 1 | DH895xCC | 01.org | icp_qa_al | n/a | 435 | 1 | 443 | 32 | + +-----+----------+--------+---------------+------------+--------+------+--------+--------+ + | 1 | DH895xCC | 4.4+ | qat_dh895xcc | dh895xcc | 435 | 1 | 443 | 32 | + +-----+----------+--------+---------------+------------+--------+------+--------+--------+ + | 2 | C62x | 4.5+ | qat_c62x | c6xx | 37c8 | 3 | 37c9 | 16 | + +-----+----------+--------+---------------+------------+--------+------+--------+--------+ + | 2 | C3xxx | 4.5+ | qat_c3xxx | c3xxx | 19e2 | 1 | 19e3 | 16 | + +-----+----------+--------+---------------+------------+--------+------+--------+--------+ + | 2 | D15xx | p | qat_d15xx | d15xx | 6f54 | 1 | 6f55 | 16 | + +-----+----------+--------+---------------+------------+--------+------+--------+--------+ The ``Driver`` column indicates either the Linux kernel version in which @@ -364,3 +365,21 @@ Another way to bind the VFs to the DPDK UIO driver is by using the cd to the top-level DPDK directory ./usertools/dpdk-devbind.py -b igb_uio 0000:03:01.1 + + +Extra notes on KASUMI F9 +------------------------ + +When using KASUMI F9 authentication algorithm, the input buffer must be +constructed according to the 3GPP KASUMI specifications (section 4.4, page 13): +`<http://cryptome.org/3gpp/35201-900.pdf>`_. +Input buffer has to have COUNT (4 bytes), FRESH (4 bytes), MESSAGE and DIRECTION (1 bit) +concatenated. After the DIRECTION bit, a single '1' bit is appended, followed by +between 0 and 7 '0' bits, so that the total length of the buffer is multiple of 8 bits. +Note that the actual message can be any length, specified in bits. + +Once this buffer is passed this way, when creating the crypto operation, +length of data to authenticate (op.sym.auth.data.length) must be the length +of all the items described above, including the padding at the end. +Also, offset of data to authenticate (op.sym.auth.data.offset) +must be such that points at the start of the COUNT bytes. diff --git a/doc/guides/cryptodevs/scheduler.rst b/doc/guides/cryptodevs/scheduler.rst index 32e56537..e7e0247b 100644 --- a/doc/guides/cryptodevs/scheduler.rst +++ b/doc/guides/cryptodevs/scheduler.rst @@ -72,9 +72,9 @@ Initialization To use the PMD in an application, user must: -* Call rte_vdev_init("crpyto_scheduler") within the application. +* Call rte_vdev_init("crypto_scheduler") within the application. -* Use --vdev="crpyto_scheduler" in the EAL options, which will call +* Use --vdev="crypto_scheduler" in the EAL options, which will call rte_vdev_init() internally. @@ -106,7 +106,7 @@ Example: .. code-block:: console - ... --vdev "crypto_aesni_mb_pmd,name=aesni_mb_1" --vdev "crypto_aesni_mb_pmd,name=aesni_mb_2" --vdev "crypto_scheduler_pmd,slave=aesni_mb_1,slave=aesni_mb_2" ... + ... --vdev "crypto_aesni_mb0,name=aesni_mb_1" --vdev "crypto_aesni_mb1,name=aesni_mb_2" --vdev "crypto_scheduler,slave=aesni_mb_1,slave=aesni_mb_2" ... .. note:: @@ -170,3 +170,30 @@ operation: crypto operation burst to the primary slave. When one or more crypto operations fail to be enqueued, then they will be enqueued to the secondary slave. + +* **CDEV_SCHED_MODE_MULTICORE:** + + *Initialization mode parameter*: **multi-core** + + Multi-core mode, which distributes the workload with several (up to eight) + worker cores. The enqueued bursts are distributed among the worker cores in a + round-robin manner. If scheduler cannot enqueue entire burst to the same worker, + it will enqueue the remaining operations to the next available worker. + For pure small packet size (64 bytes) traffic however the multi-core mode is not + an optimal solution, as it doesn't give significant per-core performance improvement. + For mixed traffic (IMIX) the optimal number of worker cores is around 2-3. + For large packets (1.5 Kbytes) scheduler shows linear scaling in performance + up to eight cores. + Each worker uses its own slave cryptodev. Only software cryptodevs + are supported. Only the same type of cryptodevs should be used concurrently. + + The multi-core mode uses one extra parameter: + + * corelist: Semicolon-separated list of logical cores to be used as workers. + The number of worker cores should be equal to the number of slave cryptodevs. + These cores should be present in EAL core list parameter and + should not be used by the application or any other process. + + Example: + ... --vdev "crypto_aesni_mb1,name=aesni_mb_1" --vdev "crypto_aesni_mb_pmd2,name=aesni_mb_2" \ + --vdev "crypto_scheduler,slave=aesni_mb_1,slave=aesni_mb_2,mode=multi-core,corelist=23;24" ... diff --git a/doc/guides/cryptodevs/snow3g.rst b/doc/guides/cryptodevs/snow3g.rst index 12b6c4af..8914e29c 100644 --- a/doc/guides/cryptodevs/snow3g.rst +++ b/doc/guides/cryptodevs/snow3g.rst @@ -69,6 +69,17 @@ on their system before building DPDK:: make snow3G +**Note**: When encrypting with SNOW3G UEA2, by default the library +encrypts blocks of 4 bytes, regardless the number of bytes to +be encrypted provided (which leads to a possible buffer overflow). +To avoid this situation, it is necessary not to pass +3GPP_SAFE_BUFFERS as a compilation flag. +For this, in the Makefile of the library, make sure that this flag +is commented out.:: + + #EXTRA_CFLAGS += -D_3GPP_SAFE_BUFFERS + + Initialization -------------- @@ -100,4 +111,5 @@ Example: .. code-block:: console - ./l2fwd-crypto -l 6 -n 4 --vdev="crypto_snow3g,socket_id=1,max_nb_sessions=128" + ./l2fwd-crypto -l 1 -n 4 --vdev="crypto_snow3g,socket_id=0,max_nb_sessions=128" \ + -- -p 1 --cdev SW --chain CIPHER_ONLY --cipher_algo "snow3g-uea2" diff --git a/doc/guides/cryptodevs/zuc.rst b/doc/guides/cryptodevs/zuc.rst index 6deb11ab..7fcfc077 100644 --- a/doc/guides/cryptodevs/zuc.rst +++ b/doc/guides/cryptodevs/zuc.rst @@ -79,6 +79,9 @@ In order to enable this virtual crypto PMD, user must: * Export the environmental variable LIBSSO_ZUC_PATH with the path where the library was extracted (zuc folder). +* Export the environmental variable LD_LIBRARY_PATH with the path + where the built libsso library is (LIBSSO_ZUC_PATH/build). + * Build the LIBSSO_ZUC library (explained in Installation section). * Build DPDK as follows: @@ -108,4 +111,5 @@ Example: .. code-block:: console - ./l2fwd-crypto -l 6 -n 4 --vdev="crypto_zuc,socket_id=1,max_nb_sessions=128" + ./l2fwd-crypto -l 1 -n 4 --vdev="crypto_zuc,socket_id=0,max_nb_sessions=128" \ + -- -p 1 --cdev SW --chain CIPHER_ONLY --cipher_algo "zuc-eea3" diff --git a/doc/guides/eventdevs/dpaa2.rst b/doc/guides/eventdevs/dpaa2.rst new file mode 100644 index 00000000..0970b266 --- /dev/null +++ b/doc/guides/eventdevs/dpaa2.rst @@ -0,0 +1,175 @@ +.. BSD LICENSE + Copyright 2017 NXP. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of NXP nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +NXP DPAA2 Eventdev Driver +========================= + +The dpaa2 eventdev is an implementation of the eventdev API, that provides a +wide range of the eventdev features. The eventdev relies on a dpaa2 hw to +perform event scheduling. + +More information can be found at `NXP Official Website +<http://www.nxp.com/products/microcontrollers-and-processors/arm-processors/qoriq-arm-processors:QORIQ-ARM>`_. + +Features +-------- + +The DPAA2 EVENTDEV implements many features in the eventdev API; + +- Hardware based event scheduler +- 8 event ports +- 8 event queues +- Parallel flows +- Atomic flows + +Supported DPAA2 SoCs +-------------------- + +- LS2080A/LS2040A +- LS2084A/LS2044A +- LS2088A/LS2048A +- LS1088A/LS1048A + +Prerequisites +------------- + +There are three main pre-requisities for executing DPAA2 EVENTDEV on a DPAA2 +compatible board: + +1. **ARM 64 Tool Chain** + + For example, the `*aarch64* Linaro Toolchain <https://releases.linaro.org/components/toolchain/binaries/4.9-2017.01/aarch64-linux-gnu>`_. + +2. **Linux Kernel** + + It can be obtained from `NXP's Github hosting <https://github.com/qoriq-open-source/linux>`_. + +3. **Rootfile system** + + Any *aarch64* supporting filesystem can be used. For example, + Ubuntu 15.10 (Wily) or 16.04 LTS (Xenial) userland which can be obtained + from `here <http://cdimage.ubuntu.com/ubuntu-base/releases/16.04/release/ubuntu-base-16.04.1-base-arm64.tar.gz>`_. + +As an alternative method, DPAA2 EVENTDEV can also be executed using images provided +as part of SDK from NXP. The SDK includes all the above prerequisites necessary +to bring up a DPAA2 board. + +The following dependencies are not part of DPDK and must be installed +separately: + +- **NXP Linux SDK** + + NXP Linux software development kit (SDK) includes support for family + of QorIQ® ARM-Architecture-based system on chip (SoC) processors + and corresponding boards. + + It includes the Linux board support packages (BSPs) for NXP SoCs, + a fully operational tool chain, kernel and board specific modules. + + SDK and related information can be obtained from: `NXP QorIQ SDK <http://www.nxp.com/products/software-and-tools/run-time-software/linux-sdk/linux-sdk-for-qoriq-processors:SDKLINUX>`_. + +- **DPDK Extra Scripts** + + DPAA2 based resources can be configured easily with the help of ready scripts + as provided in the DPDK Extra repository. + + `DPDK Extras Scripts <https://github.com/qoriq-open-source/dpdk-extras>`_. + +Currently supported by DPDK: + +- NXP SDK **2.0+**. +- MC Firmware version **10.0.0** and higher. +- Supported architectures: **arm64 LE**. + +- Follow the DPDK :ref:`Getting Started Guide for Linux <linux_gsg>` to setup the basic DPDK environment. + +.. note:: + + Some part of fslmc bus code (mc flib - object library) routines are + dual licensed (BSD & GPLv2). + +Pre-Installation Configuration +------------------------------ + +Config File Options +~~~~~~~~~~~~~~~~~~~ + +The following options can be modified in the ``config`` file. +Please note that enabling debugging options may affect system performance. + +- ``CONFIG_RTE_LIBRTE_PMD_DPAA2_EVENTDEV`` (default ``y``) + + Toggle compilation of the ``lrte_pmd_dpaa2_event`` driver. + +- ``CONFIG_RTE_LIBRTE_PMD_DPAA2_EVENTDEV_DEBUG`` (default ``n``) + + Toggle display of generic debugging messages + +Driver Compilation +~~~~~~~~~~~~~~~~~~ + +To compile the DPAA2 EVENTDEV PMD for Linux arm64 gcc target, run the +following ``make`` command: + +.. code-block:: console + + cd <DPDK-source-directory> + make config T=arm64-dpaa2-linuxapp-gcc install + +Initialization +-------------- + +The dpaa2 eventdev is exposed as a vdev device which consists of a set of dpcon +devices and dpci devices. On EAL initialization, dpcon and dpci devices will be +probed and then vdev device can be created from the application code by + +* Invoking ``rte_vdev_init("event_dpaa2")`` from the application + +* Using ``--vdev="event_dpaa2"`` in the EAL options, which will call + rte_vdev_init() internally + +Example: + +.. code-block:: console + + ./your_eventdev_application --vdev="event_dpaa2" + +Limitations +----------- + +Platform Requirement +~~~~~~~~~~~~~~~~~~~~ + +DPAA2 drivers for DPDK can only work on NXP SoCs as listed in the +``Supported DPAA2 SoCs``. + +Port-core binding +~~~~~~~~~~~~~~~~~ + +DPAA2 EVENTDEV driver requires event port 'x' to be used on core 'x'. diff --git a/doc/guides/eventdevs/index.rst b/doc/guides/eventdevs/index.rst index fad869d6..ba2048c7 100644 --- a/doc/guides/eventdevs/index.rst +++ b/doc/guides/eventdevs/index.rst @@ -37,5 +37,6 @@ application trough the eventdev API. :maxdepth: 2 :numbered: + dpaa2 sw octeontx diff --git a/doc/guides/eventdevs/octeontx.rst b/doc/guides/eventdevs/octeontx.rst index 6697c544..b43d5155 100644 --- a/doc/guides/eventdevs/octeontx.rst +++ b/doc/guides/eventdevs/octeontx.rst @@ -1,5 +1,5 @@ .. BSD LICENSE - Copyright (C) Cavium networks Ltd. 2017. + Copyright (C) Cavium, Inc. 2017. All rights reserved. Redistribution and use in source and binary forms, with or without @@ -12,7 +12,7 @@ notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. - * Neither the name of Cavium networks nor the names of its + * Neither the name of Cavium, Inc nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. @@ -35,7 +35,7 @@ The OCTEONTX SSOVF PMD (**librte_pmd_octeontx_ssovf**) provides poll mode eventdev driver support for the inbuilt event device found in the **Cavium OCTEONTX** SoC family as well as their virtual functions (VF) in SR-IOV context. -More information can be found at `Cavium Networks Official Website +More information can be found at `Cavium, Inc Official Website <http://www.cavium.com/OCTEON-TX_ARM_Processors.html>`_. Features diff --git a/doc/guides/eventdevs/sw.rst b/doc/guides/eventdevs/sw.rst index fb63c844..a3e66243 100644 --- a/doc/guides/eventdevs/sw.rst +++ b/doc/guides/eventdevs/sw.rst @@ -32,7 +32,9 @@ Software Eventdev Poll Mode Driver The software eventdev is an implementation of the eventdev API, that provides a wide range of the eventdev features. The eventdev relies on a CPU core to -perform event scheduling. +perform event scheduling. This PMD can use the service core library to run the +scheduling function, allowing an application to utilize the power of service +cores to multiplex other work on the same core if required. Features diff --git a/doc/guides/faq/faq.rst b/doc/guides/faq/faq.rst index 308a2873..dac80509 100644 --- a/doc/guides/faq/faq.rst +++ b/doc/guides/faq/faq.rst @@ -37,8 +37,8 @@ Alternatively, applications can also be run as regular user. For more information, please refer to :ref:`DPDK Getting Started Guide <linux_gsg>`. -If I want to change the number of TLB Hugepages allocated, how do I remove the original pages allocated? --------------------------------------------------------------------------------------------------------- +If I want to change the number of hugepages allocated, how do I remove the original pages allocated? +---------------------------------------------------------------------------------------------------- The number of pages allocated can be seen by executing the following command:: @@ -115,16 +115,16 @@ but the end-to-end latency of an average packet typically increases as a result. Similarly, the application can be tuned to have, on average, a low end-to-end latency at the cost of lower throughput. To achieve higher throughput, the DPDK attempts to aggregate the cost of processing each packet individually by processing packets in bursts. -Using the testpmd application as an example, the "burst" size can be set on the command line to a value of 16 (also the default value). -This allows the application to request 16 packets at a time from the PMD. -The testpmd application then immediately attempts to transmit all the packets that were received, in this case, all 16 packets. +Using the testpmd application as an example, the "burst" size can be set on the command line to a value of 32 (also the default value). +This allows the application to request 32 packets at a time from the PMD. +The testpmd application then immediately attempts to transmit all the packets that were received, in this case, all 32 packets. The packets are not transmitted until the tail pointer is updated on the corresponding TX queue of the network port. This behavior is desirable when tuning for high throughput because the cost of tail pointer updates to both the RX and TX queues -can be spread across 16 packets, effectively hiding the relatively slow MMIO cost of writing to the PCIe* device. +can be spread across 32 packets, effectively hiding the relatively slow MMIO cost of writing to the PCIe* device. -However, this is not very desirable when tuning for low latency, because the first packet that was received must also wait for the other 15 packets to be received. -It cannot be transmitted until the other 15 packets have also been processed because the NIC will not know to transmit the packets until the TX tail pointer has been updated, -which is not done until all 16 packets have been processed for transmission. +However, this is not very desirable when tuning for low latency, because the first packet that was received must also wait for the other 31 packets to be received. +It cannot be transmitted until the other 31 packets have also been processed because the NIC will not know to transmit the packets until the TX tail pointer has been updated, +which is not done until all 32 packets have been processed for transmission. To consistently achieve low latency even under heavy system load, the application developer should avoid processing packets in bunches. The testpmd application can be configured from the command line to use a burst value of 1. @@ -222,3 +222,10 @@ How can hugepage-backed memory be shared among multiple processes? ------------------------------------------------------------------ See the Primary and Secondary examples in the :ref:`multi-process sample application <multi_process_app>`. + + +Why can't my application receive packets on my system with UEFI Secure Boot enabled? +------------------------------------------------------------------------------------ + +If UEFI secure boot is enabled, the Linux kernel may disallow the use of UIO on the system. +Therefore, devices for use by DPDK should be bound to the ``vfio-pci`` kernel module rather than ``igb_uio`` or ``uio_pci_generic``. diff --git a/doc/guides/howto/img/packet_capture_framework.svg b/doc/guides/howto/img/packet_capture_framework.svg new file mode 100644 index 00000000..a76baf71 --- /dev/null +++ b/doc/guides/howto/img/packet_capture_framework.svg @@ -0,0 +1,471 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<!-- Created with Inkscape (http://www.inkscape.org/) --> + +<svg + xmlns:osb="http://www.openswatchbook.org/uri/2009/osb" + xmlns:dc="http://purl.org/dc/elements/1.1/" + xmlns:cc="http://creativecommons.org/ns#" + xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" + xmlns:svg="http://www.w3.org/2000/svg" + xmlns="http://www.w3.org/2000/svg" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" + xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" + width="120mm" + height="80mm" + viewBox="0 0 425.19685 283.46457" + id="svg2" + version="1.1" + inkscape:version="0.91 r13725" + sodipodi:docname="drawing-pcap.svg"> + <defs + id="defs4"> + <marker + inkscape:stockid="Arrow2Lend" + orient="auto" + refY="0" + refX="0" + id="marker7773" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path7775" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" + d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" + transform="matrix(-1.1,0,0,-1.1,-1.1,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow2Lstart" + orient="auto" + refY="0" + refX="0" + id="marker7679" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path7681" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" + d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" + transform="matrix(1.1,0,0,1.1,1.1,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow2Lend" + orient="auto" + refY="0" + refX="0" + id="marker7583" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path7585" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" + d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" + transform="matrix(-1.1,0,0,-1.1,-1.1,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow2Lstart" + orient="auto" + refY="0" + refX="0" + id="marker7501" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path7503" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" + d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" + transform="matrix(1.1,0,0,1.1,1.1,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow2Lstart" + orient="auto" + refY="0" + refX="0" + id="marker7421" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path7423" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" + d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" + transform="matrix(1.1,0,0,1.1,1.1,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow2Lend" + orient="auto" + refY="0" + refX="0" + id="marker7331" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path7333" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" + d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" + transform="matrix(-1.1,0,0,-1.1,-1.1,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow2Lend" + orient="auto" + refY="0" + refX="0" + id="marker7265" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path7267" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" + d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" + transform="matrix(-1.1,0,0,-1.1,-1.1,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker7199" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="Arrow2Lstart"> + <path + transform="matrix(1.1,0,0,1.1,1.1,0)" + d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" + id="path7201" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow2Lend" + orient="auto" + refY="0" + refX="0" + id="marker7111" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path7113" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" + d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" + transform="matrix(-1.1,0,0,-1.1,-1.1,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow2Lstart" + orient="auto" + refY="0" + refX="0" + id="Arrow2Lstart" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path5820" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" + d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" + transform="matrix(1.1,0,0,1.1,1.1,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow2Lend" + orient="auto" + refY="0" + refX="0" + id="Arrow2Lend" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path5823" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" + d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" + transform="matrix(-1.1,0,0,-1.1,-1.1,0)" + inkscape:connector-curvature="0" /> + </marker> + <linearGradient + id="linearGradient5784" + osb:paint="solid"> + <stop + style="stop-color:#1e5dae;stop-opacity:1;" + offset="0" + id="stop5786" /> + </linearGradient> + <linearGradient + id="linearGradient5741" + osb:paint="solid"> + <stop + style="stop-color:#ffffff;stop-opacity:1;" + offset="0" + id="stop5743" /> + </linearGradient> + <linearGradient + id="linearGradient5735" + osb:paint="solid"> + <stop + style="stop-color:#ffffff;stop-opacity:1;" + offset="0" + id="stop5737" /> + </linearGradient> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient5741" + id="linearGradient5745" + x1="167.94293" + y1="226.05743" + x2="263.39221" + y2="226.05743" + gradientUnits="userSpaceOnUse" + gradientTransform="translate(-77.340273,715.61336)" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient5784" + id="linearGradient5788" + x1="392.19681" + y1="258.38232" + x2="487.64606" + y2="258.38232" + gradientUnits="userSpaceOnUse" + gradientTransform="translate(-84.916417,744.90779)" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient5784" + id="linearGradient5788-1" + x1="392.19681" + y1="258.38232" + x2="487.64606" + y2="258.38232" + gradientUnits="userSpaceOnUse" + gradientTransform="matrix(1.2713979,0,0,0.99644866,-421.24046,743.3)" /> + <marker + inkscape:stockid="Arrow2Lstart" + orient="auto" + refY="0" + refX="0" + id="marker6152-5" + style="overflow:visible" + inkscape:isstock="true"> + <path + inkscape:connector-curvature="0" + id="path6154-8" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" + d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" + transform="matrix(1.1,0,0,1.1,1.1,0)" /> + </marker> + <marker + inkscape:stockid="Arrow2Lstart" + orient="auto" + refY="0" + refX="0" + id="marker6152-6" + style="overflow:visible" + inkscape:isstock="true"> + <path + inkscape:connector-curvature="0" + id="path6154-2" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" + d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" + transform="matrix(1.1,0,0,1.1,1.1,0)" /> + </marker> + </defs> + <sodipodi:namedview + id="base" + pagecolor="#ffffff" + bordercolor="#666666" + borderopacity="1.0" + inkscape:pageopacity="0.0" + inkscape:pageshadow="2" + inkscape:zoom="0.57434918" + inkscape:cx="215.17857" + inkscape:cy="285.26445" + inkscape:document-units="px" + inkscape:current-layer="layer1" + showgrid="false" + inkscape:window-width="1874" + inkscape:window-height="971" + inkscape:window-x="2" + inkscape:window-y="24" + inkscape:window-maximized="0" /> + <metadata + id="metadata7"> + <rdf:RDF> + <cc:Work + rdf:about=""> + <dc:format>image/svg+xml</dc:format> + <dc:type + rdf:resource="http://purl.org/dc/dcmitype/StillImage" /> + <dc:title></dc:title> + </cc:Work> + </rdf:RDF> + </metadata> + <g + inkscape:label="Layer 1" + inkscape:groupmode="layer" + id="layer1" + transform="translate(0,-768.89764)"> + <rect + style="fill:#000000;fill-opacity:0" + id="rect3336" + width="527.29962" + height="395.97977" + x="98.994949" + y="57.361946" /> + <rect + style="fill:#000000;fill-opacity:0;stroke:#257cdc;stroke-width:4;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" + id="rect4148" + width="222.39552" + height="150.6747" + x="29.402397" + y="790.82452" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;font-size:12.5px;line-height:125%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + x="61.050636" + y="807.3205" + id="text4152" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + id="tspan4154" + x="61.050636" + y="807.3205">DPDK Primary Application</tspan></text> + <rect + style="fill:#000000;fill-opacity:0;stroke:#257cdc;stroke-width:2;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" + id="rect4156-6" + width="94.449265" + height="35.355339" + x="305.76007" + y="827.01843" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;font-size:12.5px;line-height:125%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + x="350.68585" + y="841.16058" + id="text4189" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + id="tspan4191" + x="350.68585" + y="841.16058">dpdk-pdump</tspan><tspan + sodipodi:role="line" + x="350.68585" + y="856.78558" + id="tspan4193">tool</tspan></text> + <rect + style="fill:#000000;fill-opacity:0;stroke:#257cdc;stroke-width:2;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" + id="rect4156-6-4" + width="94.449265" + height="35.355339" + x="307.78033" + y="891.16315" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;font-size:12.5px;line-height:125%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + x="352.70612" + y="905.3053" + id="text4189-1" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + x="352.70612" + y="905.3053" + id="tspan4193-3">PCAP PMD</tspan></text> + <rect + style="fill:url(#linearGradient5745);fill-opacity:1;stroke:#257cdc;stroke-width:2;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" + id="rect4156-6-6" + width="94.449265" + height="35.355339" + x="91.102669" + y="923.9931" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;font-size:12.5px;line-height:125%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + x="136.02846" + y="938.13525" + id="text4189-0" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + x="136.02846" + y="938.13525" + id="tspan4193-6">dpdk_port0</tspan></text> + <rect + style="fill:#000000;fill-opacity:0;stroke:#257cdc;stroke-width:2;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" + id="rect4156-6-5" + width="94.449265" + height="35.355339" + x="92.617897" + y="824.99817" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;font-size:12.5px;line-height:125%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + x="137.54369" + y="839.14026" + id="text4189-4" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + x="137.54369" + y="839.14026" + id="tspan4193-2">librte_pdump</tspan></text> + <rect + style="fill:url(#linearGradient5788);fill-opacity:1;stroke:#257cdc;stroke-width:1;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" + id="rect4156-6-4-5" + width="94.449265" + height="35.355339" + x="307.7804" + y="985.61243" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;font-size:12.5px;line-height:125%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#ffffff;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + x="352.70618" + y="999.75458" + id="text4189-1-8" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + x="352.70618" + y="999.75458" + id="tspan4193-3-2">capture.pcap</tspan></text> + <rect + style="fill:url(#linearGradient5788-1);fill-opacity:1;stroke:#257cdc;stroke-width:1.12555885;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" + id="rect4156-6-4-5-1" + width="120.0826" + height="35.229782" + x="78.03347" + y="983.14984" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;font-size:12.5px;line-height:125%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#ffffff;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + x="136.53352" + y="1002.785" + id="text4189-1-8-4" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + x="136.53352" + y="1002.785" + id="tspan4193-3-2-7">Traffic Generator</tspan></text> + <path + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#marker7331)" + d="m 351.46948,927.02357 c 0,57.5787 0,57.5787 0,57.5787" + id="path7329" + inkscape:connector-curvature="0" /> + <path + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow2Lend)" + d="m 349.44918,862.37381 c 0,28.28427 0,28.28427 0,28.28427" + id="path7405" + inkscape:connector-curvature="0" /> + <path + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-start:url(#marker7421)" + d="m 134.79176,960.86368 c 0,22.72844 0,22.22336 0,22.22336" + id="path7419" + inkscape:connector-curvature="0" /> + <path + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-start:url(#marker7501);marker-end:url(#marker7583)" + d="m 136.30295,923.75113 c 0,-63.57143 0,-63.57143 0,-63.57143" + id="path7499" + inkscape:connector-curvature="0" /> + <path + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-start:url(#marker7679);marker-end:url(#marker7773)" + d="m 187.01723,841.96541 c 118.21429,0 118.21429,0 118.21429,0" + id="path7677" + inkscape:connector-curvature="0" /> + </g> +</svg> diff --git a/doc/guides/howto/index.rst b/doc/guides/howto/index.rst index a483444d..468939b6 100644 --- a/doc/guides/howto/index.rst +++ b/doc/guides/howto/index.rst @@ -42,3 +42,4 @@ HowTo Guides vfd virtio_user_for_container_networking virtio_user_as_exceptional_path + packet_capture_framework diff --git a/doc/guides/howto/packet_capture_framework.rst b/doc/guides/howto/packet_capture_framework.rst new file mode 100644 index 00000000..4467442e --- /dev/null +++ b/doc/guides/howto/packet_capture_framework.rst @@ -0,0 +1,140 @@ +.. BSD LICENSE + Copyright(c) 2017 Intel Corporation. All rights reserved. + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of Intel Corporation nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + + +DPDK pdump Library and pdump Tool +================================= + +This document describes how the Data Plane Development Kit (DPDK) Packet +Capture Framework is used for capturing packets on DPDK ports. It is intended +for users of DPDK who want to know more about the Packet Capture feature and +for those who want to monitor traffic on DPDK-controlled devices. + +The DPDK packet capture framework was introduced in DPDK v16.07. The DPDK +packet capture framework consists of the DPDK pdump library and DPDK pdump +tool. + + +Introduction +------------ + +The :ref:`librte_pdump <pdump_library>` library provides the APIs required to +allow users to initialize the packet capture framework and to enable or +disable packet capture. The library works on a client/server model and its +usage is recommended for debugging purposes. + +The :ref:`dpdk-pdump <pdump_tool>` tool is developed based on the +``librte_pdump`` library. It runs as a DPDK secondary process and is capable +of enabling or disabling packet capture on DPDK ports. The ``dpdk-pdump`` tool +provides command-line options with which users can request enabling or +disabling of the packet capture on DPDK ports. + +The application which initializes the packet capture framework will act as a +server and the application that enables or disables the packet capture will +act as a client. The server sends the Rx and Tx packets from the DPDK ports +to the client. + +In DPDK the ``testpmd`` application can be used to initialize the packet +capture framework and act as a server, and the ``dpdk-pdump`` tool acts as a +client. To view Rx or Tx packets of ``testpmd``, the application should be +launched first, and then the ``dpdk-pdump`` tool. Packets from ``testpmd`` +will be sent to the tool, which then sends them on to the Pcap PMD device and +that device writes them to the Pcap file or to an external interface depending +on the command-line option used. + +Some things to note: + +* The ``dpdk-pdump`` tool can only be used in conjunction with a primary + application which has the packet capture framework initialized already. In + dpdk, only ``testpmd`` is modified to initialize packet capture framework, + other applications remain untouched. So, if the ``dpdk-pdump`` tool has to + be used with any application other than the testpmd, the user needs to + explicitly modify that application to call the packet capture framework + initialization code. Refer to the ``app/test-pmd/testpmd.c`` code and look + for ``pdump`` keyword to see how this is done. + +* The ``dpdk-pdump`` tool depends on the libpcap based PMD which is disabled + by default in the build configuration files, owing to an external dependency + on the libpcap development files. Once the libpcap development files are + installed, the libpcap based PMD can be enabled by setting + ``CONFIG_RTE_LIBRTE_PMD_PCAP=y`` and recompiling the DPDK. + + +Test Environment +---------------- + +The overview of using the Packet Capture Framework and the ``dpdk-pdump`` tool +for packet capturing on the DPDK port in +:numref:`figure_packet_capture_framework`. + +.. _figure_packet_capture_framework: + +.. figure:: img/packet_capture_framework.* + + Packet capturing on a DPDK port using the dpdk-pdump tool. + + +Configuration +------------- + +Modify the DPDK primary application to initialize the packet capture framework +as mentioned in the above notes and enable the following config options and +build DPDK:: + + CONFIG_RTE_LIBRTE_PMD_PCAP=y + CONFIG_RTE_LIBRTE_PDUMP=y + + +Running the Application +----------------------- + +The following steps demonstrate how to run the ``dpdk-pdump`` tool to capture +Rx side packets on dpdk_port0 in :numref:`figure_packet_capture_framework` and +inspect them using ``tcpdump``. + +#. Launch testpmd as the primary application:: + + sudo ./app/testpmd -c 0xf0 -n 4 -- -i --port-topology=chained + +#. Launch the pdump tool as follows:: + + sudo ./build/app/dpdk-pdump -- \ + --pdump 'port=0,queue=*,rx-dev=/tmp/capture.pcap' + +#. Send traffic to dpdk_port0 from traffic generator. + Inspect packets captured in the file capture.pcap using a tool + that can interpret Pcap files, for example tcpdump:: + + $tcpdump -nr /tmp/capture.pcap + reading from file /tmp/capture.pcap, link-type EN10MB (Ethernet) + 11:11:36.891404 IP 4.4.4.4.whois++ > 3.3.3.3.whois++: UDP, length 18 + 11:11:36.891442 IP 4.4.4.4.whois++ > 3.3.3.3.whois++: UDP, length 18 + 11:11:36.891445 IP 4.4.4.4.whois++ > 3.3.3.3.whois++: UDP, length 18 diff --git a/doc/guides/howto/virtio_user_as_exceptional_path.rst b/doc/guides/howto/virtio_user_as_exceptional_path.rst index 0bbcd3fd..3f99fe82 100644 --- a/doc/guides/howto/virtio_user_as_exceptional_path.rst +++ b/doc/guides/howto/virtio_user_as_exceptional_path.rst @@ -54,7 +54,7 @@ solution is very promising in: * Performance similar to KNI, this solution would use one or more kthreads to - send/receive packets from user space DPDK applications, which has little + send/receive packets to/from user space DPDK applications, which has little impact on user space polling thread (except that it might enter into kernel space to wake up those kthreads if necessary). @@ -94,7 +94,7 @@ compiling the kernel and those kernel modules should be inserted. This is used to negotiate VIRTIO_NET_F_GUEST_TSO4 and VIRTIO_NET_F_GUEST_TSO6 feature so that large packets from kernel can be - transmitted DPDK application and further TSOed by physical NIC. + transmitted to DPDK application and further TSOed by physical NIC. * ``--enable-rx-cksum`` diff --git a/doc/guides/linux_gsg/build_dpdk.rst b/doc/guides/linux_gsg/build_dpdk.rst index cf6c06d6..e32afd5f 100644 --- a/doc/guides/linux_gsg/build_dpdk.rst +++ b/doc/guides/linux_gsg/build_dpdk.rst @@ -143,138 +143,3 @@ Browsing the Installed DPDK Environment Target Once a target is created it contains all libraries, including poll-mode drivers, and header files for the DPDK environment that are required to build customer applications. In addition, the test and testpmd applications are built under the build/app directory, which may be used for testing. A kmod directory is also present that contains kernel modules which may be loaded if needed. - -Loading Modules to Enable Userspace IO for DPDK ------------------------------------------------ - -To run any DPDK application, a suitable uio module can be loaded into the running kernel. -In many cases, the standard ``uio_pci_generic`` module included in the Linux kernel -can provide the uio capability. This module can be loaded using the command - -.. code-block:: console - - sudo modprobe uio_pci_generic - -.. note:: - - ``uio_pci_generic`` module doesn't support the creation of virtual functions. - -As an alternative to the ``uio_pci_generic``, the DPDK also includes the igb_uio -module which can be found in the kmod subdirectory referred to above. It can -be loaded as shown below: - -.. code-block:: console - - sudo modprobe uio - sudo insmod kmod/igb_uio.ko - -.. note:: - - For some devices which lack support for legacy interrupts, e.g. virtual function - (VF) devices, the ``igb_uio`` module may be needed in place of ``uio_pci_generic``. - -Since DPDK release 1.7 onward provides VFIO support, use of UIO is optional -for platforms that support using VFIO. - -Loading VFIO Module -------------------- - -To run an DPDK application and make use of VFIO, the ``vfio-pci`` module must be loaded: - -.. code-block:: console - - sudo modprobe vfio-pci - -Note that in order to use VFIO, your kernel must support it. -VFIO kernel modules have been included in the Linux kernel since version 3.6.0 and are usually present by default, -however please consult your distributions documentation to make sure that is the case. - -Also, to use VFIO, both kernel and BIOS must support and be configured to use IO virtualization (such as Intel® VT-d). - -.. note:: - - ``vfio-pci`` module doesn't support the creation of virtual functions. - -For proper operation of VFIO when running DPDK applications as a non-privileged user, correct permissions should also be set up. -This can be done by using the DPDK setup script (called dpdk-setup.sh and located in the usertools directory). - -.. _linux_gsg_binding_kernel: - -Binding and Unbinding Network Ports to/from the Kernel Modules --------------------------------------------------------------- - -As of release 1.4, DPDK applications no longer automatically unbind all supported network ports from the kernel driver in use. -Instead, all ports that are to be used by an DPDK application must be bound to the -``uio_pci_generic``, ``igb_uio`` or ``vfio-pci`` module before the application is run. -Any network ports under Linux* control will be ignored by the DPDK poll-mode drivers and cannot be used by the application. - -.. warning:: - - The DPDK will, by default, no longer automatically unbind network ports from the kernel driver at startup. - Any ports to be used by an DPDK application must be unbound from Linux* control and - bound to the ``uio_pci_generic``, ``igb_uio`` or ``vfio-pci`` module before the application is run. - -To bind ports to the ``uio_pci_generic``, ``igb_uio`` or ``vfio-pci`` module for DPDK use, -and then subsequently return ports to Linux* control, -a utility script called dpdk-devbind.py is provided in the usertools subdirectory. -This utility can be used to provide a view of the current state of the network ports on the system, -and to bind and unbind those ports from the different kernel modules, including the uio and vfio modules. -The following are some examples of how the script can be used. -A full description of the script and its parameters can be obtained by calling the script with the ``--help`` or ``--usage`` options. -Note that the uio or vfio kernel modules to be used, should be loaded into the kernel before -running the ``dpdk-devbind.py`` script. - -.. warning:: - - Due to the way VFIO works, there are certain limitations to which devices can be used with VFIO. - Mainly it comes down to how IOMMU groups work. - Any Virtual Function device can be used with VFIO on its own, but physical devices will require either all ports bound to VFIO, - or some of them bound to VFIO while others not being bound to anything at all. - - If your device is behind a PCI-to-PCI bridge, the bridge will then be part of the IOMMU group in which your device is in. - Therefore, the bridge driver should also be unbound from the bridge PCI device for VFIO to work with devices behind the bridge. - -.. warning:: - - While any user can run the dpdk-devbind.py script to view the status of the network ports, - binding or unbinding network ports requires root privileges. - -To see the status of all network ports on the system: - -.. code-block:: console - - ./usertools/dpdk-devbind.py --status - - Network devices using DPDK-compatible driver - ============================================ - 0000:82:00.0 '82599EB 10-GbE NIC' drv=uio_pci_generic unused=ixgbe - 0000:82:00.1 '82599EB 10-GbE NIC' drv=uio_pci_generic unused=ixgbe - - Network devices using kernel driver - =================================== - 0000:04:00.0 'I350 1-GbE NIC' if=em0 drv=igb unused=uio_pci_generic *Active* - 0000:04:00.1 'I350 1-GbE NIC' if=eth1 drv=igb unused=uio_pci_generic - 0000:04:00.2 'I350 1-GbE NIC' if=eth2 drv=igb unused=uio_pci_generic - 0000:04:00.3 'I350 1-GbE NIC' if=eth3 drv=igb unused=uio_pci_generic - - Other network devices - ===================== - <none> - -To bind device ``eth1``,``04:00.1``, to the ``uio_pci_generic`` driver: - -.. code-block:: console - - ./usertools/dpdk-devbind.py --bind=uio_pci_generic 04:00.1 - -or, alternatively, - -.. code-block:: console - - ./usertools/dpdk-devbind.py --bind=uio_pci_generic eth1 - -To restore device ``82:00.0`` to its original kernel binding: - -.. code-block:: console - - ./usertools/dpdk-devbind.py --bind=ixgbe 82:00.0 diff --git a/doc/guides/linux_gsg/build_sample_apps.rst b/doc/guides/linux_gsg/build_sample_apps.rst index 12fefffd..0cc5fd17 100644 --- a/doc/guides/linux_gsg/build_sample_apps.rst +++ b/doc/guides/linux_gsg/build_sample_apps.rst @@ -98,12 +98,14 @@ Running a Sample Application .. warning:: - The UIO drivers and hugepages must be setup prior to running an application. + Before running the application make sure: -.. warning:: + - Hugepages setup is done. + - Any kernel driver being used is loaded. + - In case needed, ports being used by the application should be + bound to the corresponding kernel driver. - Any ports to be used by the application must be already bound to an appropriate kernel - module, as described in :ref:`linux_gsg_binding_kernel`, prior to running the application. + refer to :ref:`linux_gsg_linux_drivers` for more details. The application is linked with the DPDK target environment's Environmental Abstraction Layer (EAL) library, which provides some options that are generic to every DPDK application. diff --git a/doc/guides/linux_gsg/enable_func.rst b/doc/guides/linux_gsg/enable_func.rst index 04e066c9..5ac0184a 100644 --- a/doc/guides/linux_gsg/enable_func.rst +++ b/doc/guides/linux_gsg/enable_func.rst @@ -35,8 +35,8 @@ Enabling Additional Functionality .. _High_Precision_Event_Timer: -High Precision Event Timer HPET) Functionality ----------------------------------------------- +High Precision Event Timer (HPET) Functionality +----------------------------------------------- BIOS Support ~~~~~~~~~~~~ @@ -87,6 +87,14 @@ The application can then determine what action to take, if any, if the HPET is n Running DPDK Applications Without Root Privileges -------------------------------------------------------- +.. note:: + + The instructions below will allow running DPDK as non-root with older + Linux kernel versions. However, since version 4.0, the kernel does not allow + unprivileged processes to read the physical address information from + the pagemaps file, making it impossible for those processes to use HW + devices which require physical addresses + Although applications using the DPDK use network ports and other hardware resources directly, with a number of small permission adjustments it is possible to run these applications as a user other than "root". To do so, the ownership, or permissions, on the following Linux file system objects should be adjusted to ensure that @@ -176,28 +184,3 @@ Also, if ``INTEL_IOMMU_DEFAULT_ON`` is not set in the kernel, the ``intel_iommu= This ensures that the Intel IOMMU is being initialized as expected. Please note that while using ``iommu=pt`` is compulsory for ``igb_uio driver``, the ``vfio-pci`` driver can actually work with both ``iommu=pt`` and ``iommu=on``. - -High Performance of Small Packets on 40G NIC --------------------------------------------- - -As there might be firmware fixes for performance enhancement in latest version -of firmware image, the firmware update might be needed for getting high performance. -Check with the local Intel's Network Division application engineers for firmware updates. -Users should consult the release notes specific to a DPDK release to identify -the validated firmware version for a NIC using the i40e driver. - -Use 16 Bytes RX Descriptor Size -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -As i40e PMD supports both 16 and 32 bytes RX descriptor sizes, and 16 bytes size can provide helps to high performance of small packets. -Configuration of ``CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC`` in config files can be changed to use 16 bytes size RX descriptors. - -High Performance and per Packet Latency Tradeoff -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Due to the hardware design, the interrupt signal inside NIC is needed for per -packet descriptor write-back. The minimum interval of interrupts could be set -at compile time by ``CONFIG_RTE_LIBRTE_I40E_ITR_INTERVAL`` in configuration files. -Though there is a default configuration, the interval could be tuned by the -users with that configuration item depends on what the user cares about more, -performance or per packet latency. diff --git a/doc/guides/linux_gsg/index.rst b/doc/guides/linux_gsg/index.rst index 3d3ada15..799559c2 100644 --- a/doc/guides/linux_gsg/index.rst +++ b/doc/guides/linux_gsg/index.rst @@ -40,6 +40,7 @@ Getting Started Guide for Linux intro sys_reqs build_dpdk + linux_drivers build_sample_apps enable_func quick_start diff --git a/doc/guides/linux_gsg/linux_drivers.rst b/doc/guides/linux_gsg/linux_drivers.rst new file mode 100644 index 00000000..08f7c9ba --- /dev/null +++ b/doc/guides/linux_gsg/linux_drivers.rst @@ -0,0 +1,204 @@ +.. BSD LICENSE + Copyright(c) 2010-2015 Intel Corporation. + Copyright(c) 2017 Mellanox Corporation. + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of Intel Corporation nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +.. _linux_gsg_linux_drivers: + +Linux Drivers +============= + +Different PMDs may require different kernel drivers in order to work properly. +Depends on the PMD being used, a corresponding kernel driver should be load +and bind to the network ports. + +UIO +--- + +A small kernel module to set up the device, map device memory to user-space and register interrupts. +In many cases, the standard ``uio_pci_generic`` module included in the Linux kernel +can provide the uio capability. This module can be loaded using the command: + +.. code-block:: console + + sudo modprobe uio_pci_generic + +.. note:: + + ``uio_pci_generic`` module doesn't support the creation of virtual functions. + +As an alternative to the ``uio_pci_generic``, the DPDK also includes the igb_uio +module which can be found in the kmod subdirectory referred to above. It can +be loaded as shown below: + +.. code-block:: console + + sudo modprobe uio + sudo insmod kmod/igb_uio.ko + +.. note:: + + For some devices which lack support for legacy interrupts, e.g. virtual function + (VF) devices, the ``igb_uio`` module may be needed in place of ``uio_pci_generic``. + +.. note:: + + If UEFI secure boot is enabled, the Linux kernel may disallow the use of + UIO on the system. Therefore, devices for use by DPDK should be bound to the + ``vfio-pci`` kernel module rather than ``igb_uio`` or ``uio_pci_generic``. + For more details see :ref:`linux_gsg_binding_kernel` below. + +Since DPDK release 1.7 onward provides VFIO support, use of UIO is optional +for platforms that support using VFIO. + +VFIO +---- + +A more robust and secure driver in compare to the ``UIO``, relying on IOMMU protection. +To make use of VFIO, the ``vfio-pci`` module must be loaded: + +.. code-block:: console + + sudo modprobe vfio-pci + +Note that in order to use VFIO, your kernel must support it. +VFIO kernel modules have been included in the Linux kernel since version 3.6.0 and are usually present by default, +however please consult your distributions documentation to make sure that is the case. + +Also, to use VFIO, both kernel and BIOS must support and be configured to use IO virtualization (such as Intel® VT-d). + +.. note:: + + ``vfio-pci`` module doesn't support the creation of virtual functions. + +For proper operation of VFIO when running DPDK applications as a non-privileged user, correct permissions should also be set up. +This can be done by using the DPDK setup script (called dpdk-setup.sh and located in the usertools directory). + +.. note:: + + VFIO can be used without IOMMU. While this is just as unsafe as using UIO, it does make it possible for the user to keep the degree of device access and programming that VFIO has, in situations where IOMMU is not available. + +Bifurcated Driver +----------------- + +PMDs which use the bifurcated driver co-exists with the device kernel driver. +On such model the NIC is controlled by the kernel, while the data +path is performed by the PMD directly on top of the device. + +Such model has the following benefits: + + - It is secure and robust, as the memory management and isolation + is done by the kernel. + - It enables the user to use legacy linux tools such as ``ethtool`` or + ``ifconfig`` while running DPDK application on the same network ports. + - It enables the DPDK application to filter only part of the traffic, + While the rest will be directed and handled by the kernel driver. + +More about the bifurcated driver can be found in +`Mellanox Bifurcated DPDK PMD +<https://dpdksummit.com/Archive/pdf/2016Userspace/Day02-Session04-RonyEfraim-Userspace2016.pdf>`__. + +.. _linux_gsg_binding_kernel: + +Binding and Unbinding Network Ports to/from the Kernel Modules +-------------------------------------------------------------- + +.. note:: + + PMDs Which use the bifurcated driver should not be unbind from their kernel drivers. this section is for PMDs which use the UIO or VFIO drivers. + +As of release 1.4, DPDK applications no longer automatically unbind all supported network ports from the kernel driver in use. +Instead, in case the PMD being used use the UIO or VFIO drivers, all ports that are to be used by an DPDK application must be bound to the +``uio_pci_generic``, ``igb_uio`` or ``vfio-pci`` module before the application is run. +For such PMDs, any network ports under Linux* control will be ignored and cannot be used by the application. + +To bind ports to the ``uio_pci_generic``, ``igb_uio`` or ``vfio-pci`` module for DPDK use, +and then subsequently return ports to Linux* control, +a utility script called dpdk-devbind.py is provided in the usertools subdirectory. +This utility can be used to provide a view of the current state of the network ports on the system, +and to bind and unbind those ports from the different kernel modules, including the uio and vfio modules. +The following are some examples of how the script can be used. +A full description of the script and its parameters can be obtained by calling the script with the ``--help`` or ``--usage`` options. +Note that the uio or vfio kernel modules to be used, should be loaded into the kernel before +running the ``dpdk-devbind.py`` script. + +.. warning:: + + Due to the way VFIO works, there are certain limitations to which devices can be used with VFIO. + Mainly it comes down to how IOMMU groups work. + Any Virtual Function device can be used with VFIO on its own, but physical devices will require either all ports bound to VFIO, + or some of them bound to VFIO while others not being bound to anything at all. + + If your device is behind a PCI-to-PCI bridge, the bridge will then be part of the IOMMU group in which your device is in. + Therefore, the bridge driver should also be unbound from the bridge PCI device for VFIO to work with devices behind the bridge. + +.. warning:: + + While any user can run the dpdk-devbind.py script to view the status of the network ports, + binding or unbinding network ports requires root privileges. + +To see the status of all network ports on the system: + +.. code-block:: console + + ./usertools/dpdk-devbind.py --status + + Network devices using DPDK-compatible driver + ============================================ + 0000:82:00.0 '82599EB 10-GbE NIC' drv=uio_pci_generic unused=ixgbe + 0000:82:00.1 '82599EB 10-GbE NIC' drv=uio_pci_generic unused=ixgbe + + Network devices using kernel driver + =================================== + 0000:04:00.0 'I350 1-GbE NIC' if=em0 drv=igb unused=uio_pci_generic *Active* + 0000:04:00.1 'I350 1-GbE NIC' if=eth1 drv=igb unused=uio_pci_generic + 0000:04:00.2 'I350 1-GbE NIC' if=eth2 drv=igb unused=uio_pci_generic + 0000:04:00.3 'I350 1-GbE NIC' if=eth3 drv=igb unused=uio_pci_generic + + Other network devices + ===================== + <none> + +To bind device ``eth1``,``04:00.1``, to the ``uio_pci_generic`` driver: + +.. code-block:: console + + ./usertools/dpdk-devbind.py --bind=uio_pci_generic 04:00.1 + +or, alternatively, + +.. code-block:: console + + ./usertools/dpdk-devbind.py --bind=uio_pci_generic eth1 + +To restore device ``82:00.0`` to its original kernel binding: + +.. code-block:: console + + ./usertools/dpdk-devbind.py --bind=ixgbe 82:00.0 diff --git a/doc/guides/linux_gsg/nic_perf_intel_platform.rst b/doc/guides/linux_gsg/nic_perf_intel_platform.rst index 709113dc..febd7337 100644 --- a/doc/guides/linux_gsg/nic_perf_intel_platform.rst +++ b/doc/guides/linux_gsg/nic_perf_intel_platform.rst @@ -186,75 +186,5 @@ Configurations before running DPDK **Note**: To get the best performance, ensure that the core and NICs are in the same socket. In the example above ``85:00.0`` is on socket 1 and should be used by cores on socket 1 for the best performance. -4. Bind the test ports to DPDK compatible drivers, such as igb_uio. For example bind two ports to a DPDK compatible driver and check the status: - - .. code-block:: console - - - # Bind ports 82:00.0 and 85:00.0 to dpdk driver - ./dpdk_folder/usertools/dpdk-devbind.py -b igb_uio 82:00.0 85:00.0 - - # Check the port driver status - ./dpdk_folder/usertools/dpdk-devbind.py --status - - See ``dpdk-devbind.py --help`` for more details. - - -More details about DPDK setup and Linux kernel requirements see :ref:`linux_gsg_compiling_dpdk`. - - -Example of getting best performance for an Intel NIC ----------------------------------------------------- - -The following is an example of running the DPDK ``l3fwd`` sample application to get high performance with an -Intel server platform and Intel XL710 NICs. -For specific 40G NIC configuration please refer to the i40e NIC guide. - -The example scenario is to get best performance with two Intel XL710 40GbE ports. -See :numref:`figure_intel_perf_test_setup` for the performance test setup. - -.. _figure_intel_perf_test_setup: - -.. figure:: img/intel_perf_test_setup.* - - Performance Test Setup - - -1. Add two Intel XL710 NICs to the platform, and use one port per card to get best performance. - The reason for using two NICs is to overcome a PCIe Gen3's limitation since it cannot provide 80G bandwidth - for two 40G ports, but two different PCIe Gen3 x8 slot can. - Refer to the sample NICs output above, then we can select ``82:00.0`` and ``85:00.0`` as test ports:: - - 82:00.0 Ethernet [0200]: Intel XL710 for 40GbE QSFP+ [8086:1583] - 85:00.0 Ethernet [0200]: Intel XL710 for 40GbE QSFP+ [8086:1583] - -2. Connect the ports to the traffic generator. For high speed testing, it's best to use a hardware traffic generator. - -3. Check the PCI devices numa node (socket id) and get the cores number on the exact socket id. - In this case, ``82:00.0`` and ``85:00.0`` are both in socket 1, and the cores on socket 1 in the referenced platform - are 18-35 and 54-71. - Note: Don't use 2 logical cores on the same core (e.g core18 has 2 logical cores, core18 and core54), instead, use 2 logical - cores from different cores (e.g core18 and core19). - -4. Bind these two ports to igb_uio. - -5. As to XL710 40G port, we need at least two queue pairs to achieve best performance, then two queues per port - will be required, and each queue pair will need a dedicated CPU core for receiving/transmitting packets. - -6. The DPDK sample application ``l3fwd`` will be used for performance testing, with using two ports for bi-directional forwarding. - Compile the ``l3fwd sample`` with the default lpm mode. - -7. The command line of running l3fwd would be something like the followings:: - - ./l3fwd -l 18-21 -n 4 -w 82:00.0 -w 85:00.0 \ - -- -p 0x3 --config '(0,0,18),(0,1,19),(1,0,20),(1,1,21)' - - This means that the application uses core 18 for port 0, queue pair 0 forwarding, core 19 for port 0, queue pair 1 forwarding, - core 20 for port 1, queue pair 0 forwarding, and core 21 for port 1, queue pair 1 forwarding. - - -8. Configure the traffic at a traffic generator. - - * Start creating a stream on packet generator. - - * Set the Ethernet II type to 0x0800. +4. Check which kernel drivers needs to be loaded and whether there is a need to unbind the network ports from their kernel drivers. +More details about DPDK setup and Linux kernel requirements see :ref:`linux_gsg_compiling_dpdk` and :ref:`linux_gsg_linux_drivers`. diff --git a/doc/guides/linux_gsg/sys_reqs.rst b/doc/guides/linux_gsg/sys_reqs.rst index 3a28c9e5..eb8442c6 100644 --- a/doc/guides/linux_gsg/sys_reqs.rst +++ b/doc/guides/linux_gsg/sys_reqs.rst @@ -43,14 +43,21 @@ BIOS Setting Prerequisite on x86 For the majority of platforms, no special BIOS settings are needed to use basic DPDK functionality. However, for additional HPET timer and power management functionality, -and high performance of small packets on 40G NIC, BIOS setting changes may be needed. +and high performance of small packets, BIOS setting changes may be needed. Consult the section on :ref:`Enabling Additional Functionality <Enabling_Additional_Functionality>` for more information on the required changes. +.. note:: + + If UEFI secure boot is enabled, the Linux kernel may disallow the use of + UIO on the system. Therefore, devices for use by DPDK should be bound to the + ``vfio-pci`` kernel module rather than ``igb_uio`` or ``uio_pci_generic``. + For more details see :ref:`linux_gsg_binding_kernel`. + Compilation of the DPDK ----------------------- -**Required Tools:** +**Required Tools and Libraries:** .. note:: @@ -84,6 +91,8 @@ Compilation of the DPDK x86_x32 ABI is currently supported with distribution packages only on Ubuntu higher than 13.10 or recent Debian distribution. The only supported compiler is gcc 4.9+. +* libnuma-devel - library for handling NUMA (Non Uniform Memory Access). + * Python, version 2.7+ or 3.2+, to use various helper scripts included in the DPDK package. @@ -129,8 +138,6 @@ System Software For other kernel builds, options which should be enabled for DPDK include: - * UIO support - * HUGETLBFS * PROC_PAGE_MONITOR support diff --git a/doc/guides/nics/cxgbe.rst b/doc/guides/nics/cxgbe.rst index a205b43f..8651a7be 100644 --- a/doc/guides/nics/cxgbe.rst +++ b/doc/guides/nics/cxgbe.rst @@ -1,5 +1,5 @@ .. BSD LICENSE - Copyright 2015 Chelsio Communications. + Copyright 2015-2017 Chelsio Communications. All rights reserved. Redistribution and use in source and binary forms, with or without @@ -32,8 +32,8 @@ CXGBE Poll Mode Driver ====================== The CXGBE PMD (**librte_pmd_cxgbe**) provides poll mode driver support -for **Chelsio T5** 10/40 Gbps family of adapters. CXGBE PMD has support -for the latest Linux and FreeBSD operating systems. +for **Chelsio Terminator** 10/25/40/100 Gbps family of adapters. CXGBE PMD +has support for the latest Linux and FreeBSD operating systems. More information can be found at `Chelsio Communications Official Website <http://www.chelsio.com>`_. @@ -55,9 +55,10 @@ CXGBE PMD has support for: Limitations ----------- -The Chelsio T5 devices provide two/four ports but expose a single PCI bus -address, thus, librte_pmd_cxgbe registers itself as a -PCI driver that allocates one Ethernet device per detected port. +The Chelsio Terminator series of devices provide two/four ports but +expose a single PCI bus address, thus, librte_pmd_cxgbe registers +itself as a PCI driver that allocates one Ethernet device per detected +port. For this reason, one cannot whitelist/blacklist a single port without whitelisting/blacklisting the other ports on the same device. @@ -70,10 +71,16 @@ Supported Chelsio T5 NICs - 40G NICs: T580-CR, T580-LP-CR, T580-SO-CR - Other T5 NICs: T522-CR +Supported Chelsio T6 NICs +------------------------- + +- 25G NICs: T6425-CR, T6225-CR, T6225-LL-CR, T6225-SO-CR +- 100G NICs: T62100-CR, T62100-LP-CR, T62100-SO-CR + Prerequisites ------------- -- Requires firmware version **1.13.32.0** and higher. Visit +- Requires firmware version **1.16.43.0** and higher. Visit `Chelsio Download Center <http://service.chelsio.com>`_ to get latest firmware bundled with the latest Chelsio Unified Wire package. @@ -123,6 +130,10 @@ enabling debugging options may affect system performance. Toggle display of receiving data path run-time check messages. +- ``CONFIG_RTE_LIBRTE_CXGBE_TPUT`` (default **y**) + + Toggle behaviour to prefer Throughput or Latency. + .. _driver-compilation: Driver compilation and testing @@ -197,12 +208,12 @@ Unified Wire package for Linux operating system are as follows: .. code-block:: console - firmware-version: 1.13.32.0, TP 0.1.4.8 + firmware-version: 1.16.43.0, TP 0.1.4.9 Running testpmd ~~~~~~~~~~~~~~~ -This section demonstrates how to launch **testpmd** with Chelsio T5 +This section demonstrates how to launch **testpmd** with Chelsio devices managed by librte_pmd_cxgbe in Linux operating system. #. Load the kernel module: @@ -226,7 +237,7 @@ devices managed by librte_pmd_cxgbe in Linux operating system. .. note:: - Both the interfaces of a Chelsio T5 2-port adapter are bound to the + Both the interfaces of a Chelsio 2-port adapter are bound to the same PCI bus address. #. Unload the kernel module: @@ -243,7 +254,7 @@ devices managed by librte_pmd_cxgbe in Linux operating system. .. note:: - Currently, CXGBE PMD only supports the binding of PF4 for Chelsio T5 NICs. + Currently, CXGBE PMD only supports the binding of PF4 for Chelsio NICs. Example output: @@ -255,7 +266,7 @@ devices managed by librte_pmd_cxgbe in Linux operating system. EAL: PCI memory mapped at 0x7fd7c0200000 EAL: PCI memory mapped at 0x7fd77cdfd000 EAL: PCI memory mapped at 0x7fd7c10b7000 - PMD: rte_cxgbe_pmd: fw: 1.13.32.0, TP: 0.1.4.8 + PMD: rte_cxgbe_pmd: fw: 1.16.43.0, TP: 0.1.4.9 PMD: rte_cxgbe_pmd: Coming up as MASTER: Initializing adapter Interactive-mode selected Configuring Port 0 (socket 0) @@ -339,12 +350,12 @@ Unified Wire package for FreeBSD operating system are as follows: .. code-block:: console - dev.t5nex.0.firmware_version: 1.13.32.0 + dev.t5nex.0.firmware_version: 1.16.43.0 Running testpmd ~~~~~~~~~~~~~~~ -This section demonstrates how to launch **testpmd** with Chelsio T5 +This section demonstrates how to launch **testpmd** with Chelsio devices managed by librte_pmd_cxgbe in FreeBSD operating system. #. Change to DPDK source directory where the target has been compiled in @@ -413,7 +424,7 @@ devices managed by librte_pmd_cxgbe in FreeBSD operating system. .. note:: - Both the interfaces of a Chelsio T5 2-port adapter are bound to the + Both the interfaces of a Chelsio 2-port adapter are bound to the same PCI bus address. #. Unload the kernel module: @@ -433,7 +444,7 @@ devices managed by librte_pmd_cxgbe in FreeBSD operating system. .. note:: - Currently, CXGBE PMD only supports the binding of PF4 for Chelsio T5 NICs. + Currently, CXGBE PMD only supports the binding of PF4 for Chelsio NICs. #. Load nic_uio kernel driver: @@ -457,7 +468,7 @@ devices managed by librte_pmd_cxgbe in FreeBSD operating system. EAL: PCI memory mapped at 0x8007ec000 EAL: PCI memory mapped at 0x842800000 EAL: PCI memory mapped at 0x80086c000 - PMD: rte_cxgbe_pmd: fw: 1.13.32.0, TP: 0.1.4.8 + PMD: rte_cxgbe_pmd: fw: 1.16.43.0, TP: 0.1.4.9 PMD: rte_cxgbe_pmd: Coming up as MASTER: Initializing adapter Interactive-mode selected Configuring Port 0 (socket 0) diff --git a/doc/guides/nics/dpaa2.rst b/doc/guides/nics/dpaa2.rst index 1ca27d45..d6b1f21e 100644 --- a/doc/guides/nics/dpaa2.rst +++ b/doc/guides/nics/dpaa2.rst @@ -1,6 +1,5 @@ .. BSD LICENSE - Copyright (C) NXP. 2016. - All rights reserved. + Copyright 2016 NXP. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -423,9 +422,15 @@ Features of the DPAA2 PMD are: - Multiple queues for TX and RX - Receive Side Scaling (RSS) +- MAC/VLAN filtering - Packet type information - Checksum offload - Promiscuous mode +- Multicast mode +- Port hardware statistics +- Jumbo frames +- Link flow control +- Scattered and gather for TX and RX Supported DPAA2 SoCs -------------------- @@ -473,12 +478,12 @@ separately: SDK and related information can be obtained from: `NXP QorIQ SDK <http://www.nxp.com/products/software-and-tools/run-time-software/linux-sdk/linux-sdk-for-qoriq-processors:SDKLINUX>`_. -- **DPDK Helper Scripts** +- **DPDK Extra Scripts** DPAA2 based resources can be configured easily with the help of ready scripts - as provided in the DPDK helper repository. + as provided in the DPDK Extra repository. - `DPDK Helper Scripts <https://github.com/qoriq-open-source/dpdk-helper>`_. + `DPDK Extras Scripts <https://github.com/qoriq-open-source/dpdk-extras>`_. Currently supported by DPDK: @@ -592,3 +597,10 @@ The DPAA2 SoC family support a maximum of a 10240 jumbo frame. The value is fixed and cannot be changed. So, even when the ``rxmode.max_rx_pkt_len`` member of ``struct rte_eth_conf`` is set to a value lower than 10240, frames up to 10240 bytes can still reach the host interface. + +Other Limitations +~~~~~~~~~~~~~~~~~ + +- RSS hash key cannot be modified. +- RSS RETA cannot be configured. +- Secondary process packet I/O is not supported. diff --git a/doc/guides/nics/enic.rst b/doc/guides/nics/enic.rst index 89a30158..cb5ae125 100644 --- a/doc/guides/nics/enic.rst +++ b/doc/guides/nics/enic.rst @@ -213,6 +213,45 @@ or ``vfio`` in non-IOMMU mode. Please see :ref:`Limitations <enic_limitations>` for limitations in the use of SR-IOV. +.. _enic-genic-flow-api: + +Generic Flow API support +------------------------ + +Generic Flow API is supported. The baseline support is: + +- **1200 series VICs** + + 5-tuple exact Flow support for 1200 series adapters. This allows: + + - Attributes: ingress + - Items: ipv4, ipv6, udp, tcp (must exactly match src/dst IP + addresses and ports and all must be specified). + - Actions: queue and void + - Selectors: 'is' + +- **1300 series VICS with Advanced filters disabled** + + With advanced filters disabled, an IPv4 or IPv6 item must be specified + in the pattern. + + - Attributes: ingress + - Items: eth, ipv4, ipv6, udp, tcp, vxlan, inner eth, ipv4, ipv6, udp, tcp + - Actions: queue and void + - Selectors: 'is', 'spec' and 'mask'. 'last' is not supported + - In total, up to 64 bytes of mask is allowed across all haeders + +- **1300 series VICS with Advanced filters enabled** + + - Attributes: ingress + - Items: eth, ipv4, ipv6, udp, tcp, vxlan, inner eth, ipv4, ipv6, udp, tcp + - Actions: queue, mark, flag and void + - Selectors: 'is', 'spec' and 'mask'. 'last' is not supported + - In total, up to 64 bytes of mask is allowed across all haeders + +More features may be added in future firmware and new versions of the VIC. +Please refer to the release notes. + .. _enic_limitations: Limitations @@ -260,9 +299,21 @@ Limitations - The number of SR-IOV devices is limited to 256. Components on target system might limit this number to fewer than 256. +- **Flow API** + + - The number of filters that can be specified with the Generic Flow API is + dependent on how many header fields are being masked. Use 'flow create' in + a loop to determine how many filters your VIC will support (not more than + 1000 for 1300 series VICs). Filter are checked for matching in the order they + were added. Since there currently is no grouping or priority support, + 'catch-all' filters should be added last. + How to build the suite ---------------------- +The build instructions for the DPDK suite should be followed. By default +the ENIC PMD library will be built into the DPDK library. + Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>` for details. @@ -313,6 +364,7 @@ Supported features - Scattered Rx - MTU update - SR-IOV on UCS managed servers connected to Fabric Interconnects. +- Flow API Known bugs and unsupported features in this release --------------------------------------------------- diff --git a/doc/guides/nics/fail_safe.rst b/doc/guides/nics/fail_safe.rst new file mode 100644 index 00000000..34ecae2d --- /dev/null +++ b/doc/guides/nics/fail_safe.rst @@ -0,0 +1,221 @@ +.. BSD LICENSE + Copyright 2017 6WIND S.A. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of 6WIND S.A. nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +Fail-safe poll mode driver library +================================== + +The Fail-safe poll mode driver library (**librte_pmd_failsafe**) is a virtual +device that allows using any device supporting hotplug (sudden device removal +and plugging on its bus), without modifying other components relying on such +device (application, other PMDs). + +Additionally to the Seamless Hotplug feature, the Fail-safe PMD offers the +ability to redirect operations to secondary devices when the primary has been +removed from the system. + +.. note:: + + The library is enabled by default. You can enable it or disable it manually + by setting the ``CONFIG_RTE_LIBRTE_PMD_FAILSAFE`` configuration option. + +Features +-------- + +The Fail-safe PMD only supports a limited set of features. If you plan to use a +device underneath the Fail-safe PMD with a specific feature, this feature must +be supported by the Fail-safe PMD to avoid throwing any error. + +A notable exception is the device removal feature. The fail-safe PMD being a +virtual device, it cannot currently be removed in the sense of a specific bus +hotplug, like for PCI for example. It will however enable this feature for its +sub-device automatically, detecting those that are capable and register the +relevant callback for such event. + +Check the feature matrix for the complete set of supported features. + +Compilation option +------------------ + +This option can be modified in the ``$RTE_TARGET/build/.config`` file. + +- ``CONFIG_RTE_LIBRTE_PMD_FAILSAFE`` (default **y**) + + Toggle compiling librte_pmd_failsafe. + +Using the Fail-safe PMD from the EAL command line +------------------------------------------------- + +The Fail-safe PMD can be used like most other DPDK virtual devices, by passing a +``--vdev`` parameter to the EAL when starting the application. The device name +must start with the *net_failsafe* prefix, followed by numbers or letters. This +name must be unique for each device. Each fail-safe instance must have at least one +sub-device, up to ``RTE_MAX_ETHPORTS-1``. + +A sub-device can be any legal DPDK device, including possibly another fail-safe +instance. + +Fail-safe command line parameters +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- **dev(<iface>)** parameter + + This parameter allows the user to define a sub-device. The ``<iface>`` part of + this parameter must be a valid device definition. It could be the argument + provided to any ``-w`` device specification or the argument that would be + given to a ``--vdev`` parameter (including a fail-safe). + Enclosing the device definition within parenthesis here allows using + additional sub-device parameters if need be. They will be passed on to the + sub-device. + +- **exec(<shell command>)** parameter + + This parameter allows the user to provide a command to the fail-safe PMD to + execute and define a sub-device. + It is done within a regular shell context. + The first line of its output is read by the fail-safe PMD and otherwise + interpreted as if passed by the regular **dev** parameter. + Any other line is discarded. + If the command fail or output an incorrect string, the sub-device is not + initialized. + All commas within the ``shell command`` are replaced by spaces before + executing the command. This helps using scripts to specify devices. + +- **mac** parameter [MAC address] + + This parameter allows the user to set a default MAC address to the fail-safe + and all of its sub-devices. + If no default mac address is provided, the fail-safe PMD will read the MAC + address of the first of its sub-device to be successfully probed and use it as + its default MAC address, trying to set it to all of its other sub-devices. + If no sub-device was successfully probed at initialization, then a random MAC + address is generated, that will be subsequently applied to all sub-device once + they are probed. + +- **hotplug_poll** parameter [UINT64] (default **2000**) + + This parameter allows the user to configure the amount of time in milliseconds + between two slave upkeep round. + +Usage example +~~~~~~~~~~~~~ + +This section shows some example of using **testpmd** with a fail-safe PMD. + +#. To build a PMD and configure DPDK, refer to the document + :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>`. + +#. Start testpmd. The slave device should be blacklisted from normal EAL + operations to avoid probing it twice when in PCI blacklist mode. + + .. code-block:: console + + $RTE_TARGET/build/app/testpmd -c 0xff -n 4 \ + --vdev 'net_failsafe0,mac=de:ad:be:ef:01:02,dev(84:00.0),dev(net_ring0)' + -b 84:00.0 -b 00:04.0 -- -i + + If the slave device being used is not blacklisted, it will be probed by the + EAL first. When the fail-safe then tries to initialize it the probe operation + fails. + + Note that PCI blacklist mode is the default PCI operating mode. + +#. Alternatively, it can be used alongside any other device in whitelist mode. + + .. code-block:: console + + $RTE_TARGET/build/app/testpmd -c 0xff -n 4 \ + --vdev 'net_failsafe0,mac=de:ad:be:ef:01:02,dev(84:00.0),dev(net_ring0)' + -w 81:00.0 -- -i + +#. Start testpmd using a flexible device definition + + .. code-block:: console + + $RTE_TARGET/build/app/testpmd -c 0xff -n 4 --no-pci \ + --vdev='net_failsafe0,exec(echo 84:00.0)' -- -i + +Using the Fail-safe PMD from an application +------------------------------------------- + +This driver strives to be as seamless as possible to existing applications, in +order to propose the hotplug functionality in the easiest way possible. + +Care must be taken, however, to respect the **ether** API concerning device +access, and in particular, using the ``RTE_ETH_FOREACH_DEV`` macro to iterate +over ethernet devices, instead of directly accessing them or by writing one's +own device iterator. + +Plug-in feature +--------------- + +A sub-device can be defined without existing on the system when the fail-safe +PMD is initialized. Upon probing this device, the fail-safe PMD will detect its +absence and postpone its use. It will then register for a periodic check on any +missing sub-device. + +During this time, the fail-safe PMD can be used normally, configured and told to +emit and receive packets. It will store any applied configuration, and try to +apply it upon the probing of its missing sub-device. After this configuration +pass, the new sub-device will be synchronized with other sub-devices, i.e. be +started if the fail-safe PMD has been started by the user before. + +Plug-out feature +---------------- + +A sub-device supporting the device removal event can be removed from its bus at +any time. The fail-safe PMD will register a callback for such event and react +accordingly. It will try to safely stop, close and uninit the sub-device having +emitted this event, allowing it to free its eventual resources. + +Fail-safe glossary +------------------ + +Fallback device : Secondary device + The fail-safe will fail-over onto this device when the preferred device is + absent. + +Preferred device : Primary device + The first declared sub-device in the fail-safe parameters. + When this device is plugged, it is always used as emitting device. + It is the main sub-device and is used as target for configuration + operations if there is any ambiguity. + +Upkeep round + Periodical process when slaves are serviced. Each devices having a state + different to that of the fail-safe device itself, is synchronized with it. + Additionally, each slave having the remove flag set are cleaned-up. + +Slave + In the context of the fail-safe PMD, synonymous to sub-device. + +Sub-device + A device being utilized by the fail-safe PMD. + This is another PMD running underneath the fail-safe PMD. + Any sub-device can disappear at any time. The fail-safe will ensure + that the device removal happens gracefully. diff --git a/doc/guides/nics/features.rst b/doc/guides/nics/features.rst new file mode 100644 index 00000000..37ffbc68 --- /dev/null +++ b/doc/guides/nics/features.rst @@ -0,0 +1,890 @@ +.. BSD LICENSE + Copyright(c) 2017 Intel Corporation. All rights reserved. + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of Intel Corporation nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +Features Overview +================= + +This section explains the supported features that are listed in the +:doc:`overview`. + +As a guide to implementers it also shows the structs where the features are +defined and the APIs that can be use to get/set the values. + +Following tags used for feature details, these are from driver point of view: + +``[uses]`` : Driver uses some kind of input from the application. + +``[implements]`` : Driver implements a functionality. + +``[provides]`` : Driver provides some kind of data to the application. It is possible +to provide data by implementing some function, but "provides" is used +for cases where provided data can't be represented simply by a function. + +``[related]`` : Related API with that feature. + + +.. _nic_features_speed_capabilities: + +Speed capabilities +------------------ + +Supports getting the speed capabilities that the current device is capable of. + +* **[provides] rte_eth_dev_info**: ``speed_capa:ETH_LINK_SPEED_*``. +* **[related] API**: ``rte_eth_dev_info_get()``. + + +.. _nic_features_link_status: + +Link status +----------- + +Supports getting the link speed, duplex mode and link state (up/down). + +* **[implements] eth_dev_ops**: ``link_update``. +* **[implements] rte_eth_dev_data**: ``dev_link``. +* **[related] API**: ``rte_eth_link_get()``, ``rte_eth_link_get_nowait()``. + + +.. _nic_features_link_status_event: + +Link status event +----------------- + +Supports Link Status Change interrupts. + +* **[uses] user config**: ``dev_conf.intr_conf.lsc``. +* **[uses] rte_eth_dev_data**: ``dev_flags:RTE_ETH_DEV_INTR_LSC``. +* **[uses] rte_eth_event_type**: ``RTE_ETH_EVENT_INTR_LSC``. +* **[implements] rte_eth_dev_data**: ``dev_link``. +* **[provides] rte_pci_driver.drv_flags**: ``RTE_PCI_DRV_INTR_LSC``. +* **[related] API**: ``rte_eth_link_get()``, ``rte_eth_link_get_nowait()``. + + +.. _nic_features_removal_event: + +Removal event +------------- + +Supports device removal interrupts. + +* **[uses] user config**: ``dev_conf.intr_conf.rmv``. +* **[uses] rte_eth_dev_data**: ``dev_flags:RTE_ETH_DEV_INTR_RMV``. +* **[uses] rte_eth_event_type**: ``RTE_ETH_EVENT_INTR_RMV``. +* **[provides] rte_pci_driver.drv_flags**: ``RTE_PCI_DRV_INTR_RMV``. + + +.. _nic_features_queue_status_event: + +Queue status event +------------------ + +Supports queue enable/disable events. + +* **[uses] rte_eth_event_type**: ``RTE_ETH_EVENT_QUEUE_STATE``. + + +.. _nic_features_rx_interrupt: + +Rx interrupt +------------ + +Supports Rx interrupts. + +* **[uses] user config**: ``dev_conf.intr_conf.rxq``. +* **[implements] eth_dev_ops**: ``rx_queue_intr_enable``, ``rx_queue_intr_disable``. +* **[related] API**: ``rte_eth_dev_rx_intr_enable()``, ``rte_eth_dev_rx_intr_disable()``. + + +.. _nic_features_lock-free_tx_queue: + +Lock-free Tx queue +------------------ + +If a PMD advertises DEV_TX_OFFLOAD_MT_LOCKFREE capable, multiple threads can +invoke rte_eth_tx_burst() concurrently on the same Tx queue without SW lock. + +* **[provides] rte_eth_dev_info**: ``tx_offload_capa:DEV_TX_OFFLOAD_MT_LOCKFREE``. +* **[related] API**: ``rte_eth_tx_burst()``. + + +.. _nic_features_free_tx_mbuf_on_demand: + +Free Tx mbuf on demand +---------------------- + +Supports freeing consumed buffers on a Tx ring. + +* **[implements] eth_dev_ops**: ``tx_done_cleanup``. +* **[related] API**: ``rte_eth_tx_done_cleanup()``. + + +.. _nic_features_queue_start_stop: + +Queue start/stop +---------------- + +Supports starting/stopping a specific Rx/Tx queue of a port. + +* **[implements] eth_dev_ops**: ``rx_queue_start``, ``rx_queue_stop``, ``tx_queue_start``, + ``tx_queue_stop``. +* **[related] API**: ``rte_eth_dev_rx_queue_start()``, ``rte_eth_dev_rx_queue_stop()``, + ``rte_eth_dev_tx_queue_start()``, ``rte_eth_dev_tx_queue_stop()``. + + +.. _nic_features_mtu_update: + +MTU update +---------- + +Supports updating port MTU. + +* **[implements] eth_dev_ops**: ``mtu_set``. +* **[implements] rte_eth_dev_data**: ``mtu``. +* **[provides] rte_eth_dev_info**: ``max_rx_pktlen``. +* **[related] API**: ``rte_eth_dev_set_mtu()``, ``rte_eth_dev_get_mtu()``. + + +.. _nic_features_jumbo_frame: + +Jumbo frame +----------- + +Supports Rx jumbo frames. + +* **[uses] user config**: ``dev_conf.rxmode.jumbo_frame``, + ``dev_conf.rxmode.max_rx_pkt_len``. +* **[related] rte_eth_dev_info**: ``max_rx_pktlen``. +* **[related] API**: ``rte_eth_dev_set_mtu()``. + + +.. _nic_features_scattered_rx: + +Scattered Rx +------------ + +Supports receiving segmented mbufs. + +* **[uses] user config**: ``dev_conf.rxmode.enable_scatter``. +* **[implements] datapath**: ``Scattered Rx function``. +* **[implements] rte_eth_dev_data**: ``scattered_rx``. +* **[provides] eth_dev_ops**: ``rxq_info_get:scattered_rx``. +* **[related] eth_dev_ops**: ``rx_pkt_burst``. + + +.. _nic_features_lro: + +LRO +--- + +Supports Large Receive Offload. + +* **[uses] user config**: ``dev_conf.rxmode.enable_lro``. +* **[implements] datapath**: ``LRO functionality``. +* **[implements] rte_eth_dev_data**: ``lro``. +* **[provides] mbuf**: ``mbuf.ol_flags:PKT_RX_LRO``, ``mbuf.tso_segsz``. +* **[provides] rte_eth_dev_info**: ``rx_offload_capa:DEV_RX_OFFLOAD_TCP_LRO``. + + +.. _nic_features_tso: + +TSO +--- + +Supports TCP Segmentation Offloading. + +* **[uses] rte_eth_desc_lim**: ``nb_seg_max``, ``nb_mtu_seg_max``. +* **[uses] mbuf**: ``mbuf.ol_flags:PKT_TX_TCP_SEG``. +* **[uses] mbuf**: ``mbuf.tso_segsz``, ``mbuf.l2_len``, ``mbuf.l3_len``, ``mbuf.l4_len``. +* **[implements] datapath**: ``TSO functionality``. +* **[provides] rte_eth_dev_info**: ``tx_offload_capa:DEV_TX_OFFLOAD_TCP_TSO,DEV_TX_OFFLOAD_UDP_TSO``. + + +.. _nic_features_promiscuous_mode: + +Promiscuous mode +---------------- + +Supports enabling/disabling promiscuous mode for a port. + +* **[implements] eth_dev_ops**: ``promiscuous_enable``, ``promiscuous_disable``. +* **[implements] rte_eth_dev_data**: ``promiscuous``. +* **[related] API**: ``rte_eth_promiscuous_enable()``, ``rte_eth_promiscuous_disable()``, + ``rte_eth_promiscuous_get()``. + + +.. _nic_features_allmulticast_mode: + +Allmulticast mode +----------------- + +Supports enabling/disabling receiving multicast frames. + +* **[implements] eth_dev_ops**: ``allmulticast_enable``, ``allmulticast_disable``. +* **[implements] rte_eth_dev_data**: ``all_multicast``. +* **[related] API**: ``rte_eth_allmulticast_enable()``, + ``rte_eth_allmulticast_disable()``, ``rte_eth_allmulticast_get()``. + + +.. _nic_features_unicast_mac_filter: + +Unicast MAC filter +------------------ + +Supports adding MAC addresses to enable whitelist filtering to accept packets. + +* **[implements] eth_dev_ops**: ``mac_addr_set``, ``mac_addr_add``, ``mac_addr_remove``. +* **[implements] rte_eth_dev_data**: ``mac_addrs``. +* **[related] API**: ``rte_eth_dev_default_mac_addr_set()``, + ``rte_eth_dev_mac_addr_add()``, ``rte_eth_dev_mac_addr_remove()``, + ``rte_eth_macaddr_get()``. + + +.. _nic_features_multicast_mac_filter: + +Multicast MAC filter +-------------------- + +Supports setting multicast addresses to filter. + +* **[implements] eth_dev_ops**: ``set_mc_addr_list``. +* **[related] API**: ``rte_eth_dev_set_mc_addr_list()``. + + +.. _nic_features_rss_hash: + +RSS hash +-------- + +Supports RSS hashing on RX. + +* **[uses] user config**: ``dev_conf.rxmode.mq_mode`` = ``ETH_MQ_RX_RSS_FLAG``. +* **[uses] user config**: ``dev_conf.rx_adv_conf.rss_conf``. +* **[provides] rte_eth_dev_info**: ``flow_type_rss_offloads``. +* **[provides] mbuf**: ``mbuf.ol_flags:PKT_RX_RSS_HASH``, ``mbuf.rss``. + + +.. _nic_features_rss_key_update: + +RSS key update +-------------- + +Supports configuration of Receive Side Scaling (RSS) hash computation. Updating +Receive Side Scaling (RSS) hash key. + +* **[implements] eth_dev_ops**: ``rss_hash_update``, ``rss_hash_conf_get``. +* **[provides] rte_eth_dev_info**: ``hash_key_size``. +* **[related] API**: ``rte_eth_dev_rss_hash_update()``, + ``rte_eth_dev_rss_hash_conf_get()``. + + +.. _nic_features_rss_reta_update: + +RSS reta update +--------------- + +Supports updating Redirection Table of the Receive Side Scaling (RSS). + +* **[implements] eth_dev_ops**: ``reta_update``, ``reta_query``. +* **[provides] rte_eth_dev_info**: ``reta_size``. +* **[related] API**: ``rte_eth_dev_rss_reta_update()``, ``rte_eth_dev_rss_reta_query()``. + + +.. _nic_features_vmdq: + +VMDq +---- + +Supports Virtual Machine Device Queues (VMDq). + +* **[uses] user config**: ``dev_conf.rxmode.mq_mode`` = ``ETH_MQ_RX_VMDQ_FLAG``. +* **[uses] user config**: ``dev_conf.rx_adv_conf.vmdq_dcb_conf``. +* **[uses] user config**: ``dev_conf.rx_adv_conf.vmdq_rx_conf``. +* **[uses] user config**: ``dev_conf.tx_adv_conf.vmdq_dcb_tx_conf``. +* **[uses] user config**: ``dev_conf.tx_adv_conf.vmdq_tx_conf``. + + +.. _nic_features_sriov: + +SR-IOV +------ + +Driver supports creating Virtual Functions. + +* **[implements] rte_eth_dev_data**: ``sriov``. + +.. _nic_features_dcb: + +DCB +--- + +Supports Data Center Bridging (DCB). + +* **[uses] user config**: ``dev_conf.rxmode.mq_mode`` = ``ETH_MQ_RX_DCB_FLAG``. +* **[uses] user config**: ``dev_conf.rx_adv_conf.vmdq_dcb_conf``. +* **[uses] user config**: ``dev_conf.rx_adv_conf.dcb_rx_conf``. +* **[uses] user config**: ``dev_conf.tx_adv_conf.vmdq_dcb_tx_conf``. +* **[uses] user config**: ``dev_conf.tx_adv_conf.vmdq_tx_conf``. +* **[implements] eth_dev_ops**: ``get_dcb_info``. +* **[related] API**: ``rte_eth_dev_get_dcb_info()``. + + +.. _nic_features_vlan_filter: + +VLAN filter +----------- + +Supports filtering of a VLAN Tag identifier. + +* **[uses] user config**: ``dev_conf.rxmode.hw_vlan_filter``. +* **[implements] eth_dev_ops**: ``vlan_filter_set``. +* **[related] API**: ``rte_eth_dev_vlan_filter()``. + + +.. _nic_features_ethertype_filter: + +Ethertype filter +---------------- + +Supports filtering on Ethernet type. + +* **[implements] eth_dev_ops**: ``filter_ctrl:RTE_ETH_FILTER_ETHERTYPE``. +* **[related] API**: ``rte_eth_dev_filter_ctrl()``, ``rte_eth_dev_filter_supported()``. + +.. _nic_features_ntuple_filter: + +N-tuple filter +-------------- + +Supports filtering on N-tuple values. + +* **[implements] eth_dev_ops**: ``filter_ctrl:RTE_ETH_FILTER_NTUPLE``. +* **[related] API**: ``rte_eth_dev_filter_ctrl()``, ``rte_eth_dev_filter_supported()``. + + +.. _nic_features_syn_filter: + +SYN filter +---------- + +Supports TCP syn filtering. + +* **[implements] eth_dev_ops**: ``filter_ctrl:RTE_ETH_FILTER_SYN``. +* **[related] API**: ``rte_eth_dev_filter_ctrl()``, ``rte_eth_dev_filter_supported()``. + + +.. _nic_features_tunnel_filter: + +Tunnel filter +------------- + +Supports tunnel filtering. + +* **[implements] eth_dev_ops**: ``filter_ctrl:RTE_ETH_FILTER_TUNNEL``. +* **[related] API**: ``rte_eth_dev_filter_ctrl()``, ``rte_eth_dev_filter_supported()``. + + +.. _nic_features_flexible_filter: + +Flexible filter +--------------- + +Supports a flexible (non-tuple or Ethertype) filter. + +* **[implements] eth_dev_ops**: ``filter_ctrl:RTE_ETH_FILTER_FLEXIBLE``. +* **[related] API**: ``rte_eth_dev_filter_ctrl()``, ``rte_eth_dev_filter_supported()``. + + +.. _nic_features_hash_filter: + +Hash filter +----------- + +Supports Hash filtering. + +* **[implements] eth_dev_ops**: ``filter_ctrl:RTE_ETH_FILTER_HASH``. +* **[related] API**: ``rte_eth_dev_filter_ctrl()``, ``rte_eth_dev_filter_supported()``. + + +.. _nic_features_flow_director: + +Flow director +------------- + +Supports Flow Director style filtering to queues. + +* **[implements] eth_dev_ops**: ``filter_ctrl:RTE_ETH_FILTER_FDIR``. +* **[provides] mbuf**: ``mbuf.ol_flags:`` ``PKT_RX_FDIR``, ``PKT_RX_FDIR_ID``, + ``PKT_RX_FDIR_FLX``. +* **[related] API**: ``rte_eth_dev_filter_ctrl()``, ``rte_eth_dev_filter_supported()``. + + +.. _nic_features_flow_control: + +Flow control +------------ + +Supports configuring link flow control. + +* **[implements] eth_dev_ops**: ``flow_ctrl_get``, ``flow_ctrl_set``, + ``priority_flow_ctrl_set``. +* **[related] API**: ``rte_eth_dev_flow_ctrl_get()``, ``rte_eth_dev_flow_ctrl_set()``, + ``rte_eth_dev_priority_flow_ctrl_set()``. + + +.. _nic_features_flow_api: + +Flow API +-------- + +Supports the DPDK Flow API for generic filtering. + +* **[implements] eth_dev_ops**: ``filter_ctrl:RTE_ETH_FILTER_GENERIC``. +* **[implements] rte_flow_ops**: ``All``. + + +.. _nic_features_rate_limitation: + +Rate limitation +--------------- + +Supports Tx rate limitation for a queue. + +* **[implements] eth_dev_ops**: ``set_queue_rate_limit``. +* **[related] API**: ``rte_eth_set_queue_rate_limit()``. + + +.. _nic_features_traffic_mirroring: + +Traffic mirroring +----------------- + +Supports adding traffic mirroring rules. + +* **[implements] eth_dev_ops**: ``mirror_rule_set``, ``mirror_rule_reset``. +* **[related] API**: ``rte_eth_mirror_rule_set()``, ``rte_eth_mirror_rule_reset()``. + + +.. _nic_features_crc_offload: + +CRC offload +----------- + +Supports CRC stripping by hardware. + +* **[uses] user config**: ``dev_conf.rxmode.hw_strip_crc``. + + +.. _nic_features_vlan_offload: + +VLAN offload +------------ + +Supports VLAN offload to hardware. + +* **[uses] user config**: ``dev_conf.rxmode.hw_vlan_strip``, + ``dev_conf.rxmode.hw_vlan_filter``, ``dev_conf.rxmode.hw_vlan_extend``. +* **[implements] eth_dev_ops**: ``vlan_offload_set``. +* **[provides] mbuf**: ``mbuf.ol_flags:PKT_RX_VLAN_STRIPPED``, ``mbuf.vlan_tci``. +* **[provides] rte_eth_dev_info**: ``rx_offload_capa:DEV_RX_OFFLOAD_VLAN_STRIP``, + ``tx_offload_capa:DEV_TX_OFFLOAD_VLAN_INSERT``. +* **[related] API**: ``rte_eth_dev_set_vlan_offload()``, + ``rte_eth_dev_get_vlan_offload()``. + + +.. _nic_features_qinq_offload: + +QinQ offload +------------ + +Supports QinQ (queue in queue) offload. + +* **[uses] mbuf**: ``mbuf.ol_flags:PKT_TX_QINQ_PKT``. +* **[provides] mbuf**: ``mbuf.ol_flags:PKT_RX_QINQ_STRIPPED``, ``mbuf.vlan_tci``, + ``mbuf.vlan_tci_outer``. +* **[provides] rte_eth_dev_info**: ``rx_offload_capa:DEV_RX_OFFLOAD_QINQ_STRIP``, + ``tx_offload_capa:DEV_TX_OFFLOAD_QINQ_INSERT``. + + +.. _nic_features_l3_checksum_offload: + +L3 checksum offload +------------------- + +Supports L3 checksum offload. + +* **[uses] user config**: ``dev_conf.rxmode.hw_ip_checksum``. +* **[uses] mbuf**: ``mbuf.ol_flags:PKT_TX_IP_CKSUM``, + ``mbuf.ol_flags:PKT_TX_IPV4`` | ``PKT_TX_IPV6``. +* **[provides] mbuf**: ``mbuf.ol_flags:PKT_RX_IP_CKSUM_UNKNOWN`` | + ``PKT_RX_IP_CKSUM_BAD`` | ``PKT_RX_IP_CKSUM_GOOD`` | + ``PKT_RX_IP_CKSUM_NONE``. +* **[provides] rte_eth_dev_info**: ``rx_offload_capa:DEV_RX_OFFLOAD_IPV4_CKSUM``, + ``tx_offload_capa:DEV_TX_OFFLOAD_IPV4_CKSUM``. + + +.. _nic_features_l4_checksum_offload: + +L4 checksum offload +------------------- + +Supports L4 checksum offload. + +* **[uses] mbuf**: ``mbuf.ol_flags:PKT_TX_IPV4`` | ``PKT_TX_IPV6``, + ``mbuf.ol_flags:PKT_TX_L4_NO_CKSUM`` | ``PKT_TX_TCP_CKSUM`` | + ``PKT_TX_SCTP_CKSUM`` | ``PKT_TX_UDP_CKSUM``. +* **[provides] mbuf**: ``mbuf.ol_flags:PKT_RX_L4_CKSUM_UNKNOWN`` | + ``PKT_RX_L4_CKSUM_BAD`` | ``PKT_RX_L4_CKSUM_GOOD`` | + ``PKT_RX_L4_CKSUM_NONE``. +* **[provides] rte_eth_dev_info**: ``rx_offload_capa:DEV_RX_OFFLOAD_UDP_CKSUM,DEV_RX_OFFLOAD_TCP_CKSUM``, + ``tx_offload_capa:DEV_TX_OFFLOAD_UDP_CKSUM,DEV_TX_OFFLOAD_TCP_CKSUM,DEV_TX_OFFLOAD_SCTP_CKSUM``. + + +.. _nic_features_macsec_offload: + +MACsec offload +-------------- + +Supports MACsec. + +* **[uses] mbuf**: ``mbuf.ol_flags:PKT_TX_MACSEC``. +* **[provides] rte_eth_dev_info**: ``rx_offload_capa:DEV_RX_OFFLOAD_MACSEC_STRIP``, + ``tx_offload_capa:DEV_TX_OFFLOAD_MACSEC_INSERT``. + + +.. _nic_features_inner_l3_checksum: + +Inner L3 checksum +----------------- + +Supports inner packet L3 checksum. + +* **[uses] mbuf**: ``mbuf.ol_flags:PKT_TX_IP_CKSUM``, + ``mbuf.ol_flags:PKT_TX_IPV4`` | ``PKT_TX_IPV6``, + ``mbuf.ol_flags:PKT_TX_OUTER_IP_CKSUM``, + ``mbuf.ol_flags:PKT_TX_OUTER_IPV4`` | ``PKT_TX_OUTER_IPV6``. +* **[uses] mbuf**: ``mbuf.outer_l2_len``, ``mbuf.outer_l3_len``. +* **[provides] mbuf**: ``mbuf.ol_flags:PKT_RX_EIP_CKSUM_BAD``. +* **[provides] rte_eth_dev_info**: ``rx_offload_capa:DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM``, + ``tx_offload_capa:DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM``. + + +.. _nic_features_inner_l4_checksum: + +Inner L4 checksum +----------------- + +Supports inner packet L4 checksum. + + +.. _nic_features_packet_type_parsing: + +Packet type parsing +------------------- + +Supports packet type parsing and returns a list of supported types. + +* **[implements] eth_dev_ops**: ``dev_supported_ptypes_get``. +* **[related] API**: ``rte_eth_dev_get_supported_ptypes()``. + + +.. _nic_features_timesync: + +Timesync +-------- + +Supports IEEE1588/802.1AS timestamping. + +* **[implements] eth_dev_ops**: ``timesync_enable``, ``timesync_disable`` + ``timesync_read_rx_timestamp``, ``timesync_read_tx_timestamp``, + ``timesync_adjust_time``, ``timesync_read_time``, ``timesync_write_time``. +* **[related] API**: ``rte_eth_timesync_enable()``, ``rte_eth_timesync_disable()``, + ``rte_eth_timesync_read_rx_timestamp()``, + ``rte_eth_timesync_read_tx_timestamp``, ``rte_eth_timesync_adjust_time()``, + ``rte_eth_timesync_read_time()``, ``rte_eth_timesync_write_time()``. + + +.. _nic_features_rx_descriptor_status: + +Rx descriptor status +-------------------- + +Supports check the status of a Rx descriptor. When ``rx_descriptor_status`` is +used, status can be "Available", "Done" or "Unavailable". When +``rx_descriptor_done`` is used, status can be "DD bit is set" or "DD bit is +not set". + +* **[implements] eth_dev_ops**: ``rx_descriptor_status``. +* **[related] API**: ``rte_eth_rx_descriptor_status()``. +* **[implements] eth_dev_ops**: ``rx_descriptor_done``. +* **[related] API**: ``rte_eth_rx_descriptor_done()``. + + +.. _nic_features_tx_descriptor_status: + +Tx descriptor status +-------------------- + +Supports checking the status of a Tx descriptor. Status can be "Full", "Done" +or "Unavailable." + +* **[implements] eth_dev_ops**: ``tx_descriptor_status``. +* **[related] API**: ``rte_eth_tx_descriptor_status()``. + + +.. _nic_features_basic_stats: + +Basic stats +----------- + +Support basic statistics such as: ipackets, opackets, ibytes, obytes, +imissed, ierrors, oerrors, rx_nombuf. + +And per queue stats: q_ipackets, q_opackets, q_ibytes, q_obytes, q_errors. + +These apply to all drivers. + +* **[implements] eth_dev_ops**: ``stats_get``, ``stats_reset``. +* **[related] API**: ``rte_eth_stats_get``, ``rte_eth_stats_reset()``. + + +.. _nic_features_extended_stats: + +Extended stats +-------------- + +Supports Extended Statistics, changes from driver to driver. + +* **[implements] eth_dev_ops**: ``xstats_get``, ``xstats_reset``, ``xstats_get_names``. +* **[implements] eth_dev_ops**: ``xstats_get_by_id``, ``xstats_get_names_by_id``. +* **[related] API**: ``rte_eth_xstats_get()``, ``rte_eth_xstats_reset()``, + ``rte_eth_xstats_get_names``, ``rte_eth_xstats_get_by_id()``, + ``rte_eth_xstats_get_names_by_id()``, ``rte_eth_xstats_get_id_by_name()``. + + +.. _nic_features_stats_per_queue: + +Stats per queue +--------------- + +Supports configuring per-queue stat counter mapping. + +* **[implements] eth_dev_ops**: ``queue_stats_mapping_set``. +* **[related] API**: ``rte_eth_dev_set_rx_queue_stats_mapping()``, + ``rte_eth_dev_set_tx_queue_stats_mapping()``. + + +.. _nic_features_fw_version: + +FW version +---------- + +Supports getting device hardware firmware information. + +* **[implements] eth_dev_ops**: ``fw_version_get``. +* **[related] API**: ``rte_eth_dev_fw_version_get()``. + + +.. _nic_features_eeprom_dump: + +EEPROM dump +----------- + +Supports getting/setting device eeprom data. + +* **[implements] eth_dev_ops**: ``get_eeprom_length``, ``get_eeprom``, ``set_eeprom``. +* **[related] API**: ``rte_eth_dev_get_eeprom_length()``, ``rte_eth_dev_get_eeprom()``, + ``rte_eth_dev_set_eeprom()``. + + +.. _nic_features_register_dump: + +Registers dump +-------------- + +Supports retrieving device registers and registering attributes (number of +registers and register size). + +* **[implements] eth_dev_ops**: ``get_reg``. +* **[related] API**: ``rte_eth_dev_get_reg_info()``. + + +.. _nic_features_led: + +LED +--- + +Supports turning on/off a software controllable LED on a device. + +* **[implements] eth_dev_ops**: ``dev_led_on``, ``dev_led_off``. +* **[related] API**: ``rte_eth_led_on()``, ``rte_eth_led_off()``. + + +.. _nic_features_multiprocess_aware: + +Multiprocess aware +------------------ + +Driver can be used for primary-secondary process model. + + +.. _nic_features_bsd_nic_uio: + +BSD nic_uio +----------- + +BSD ``nic_uio`` module supported. + + +.. _nic_features_linux_uio: + +Linux UIO +--------- + +Works with ``igb_uio`` kernel module. + +* **[provides] RTE_PMD_REGISTER_KMOD_DEP**: ``igb_uio``. + +.. _nic_features_linux_vfio: + +Linux VFIO +---------- + +Works with ``vfio-pci`` kernel module. + +* **[provides] RTE_PMD_REGISTER_KMOD_DEP**: ``vfio-pci``. + +.. _nic_features_other_kdrv: + +Other kdrv +---------- + +Kernel module other than above ones supported. + + +.. _nic_features_armv7: + +ARMv7 +----- + +Support armv7 architecture. + +Use ``defconfig_arm-armv7a-*-*``. + + +.. _nic_features_armv8: + +ARMv8 +----- + +Support armv8a (64bit) architecture. + +Use ``defconfig_arm64-armv8a-*-*`` + + +.. _nic_features_power8: + +Power8 +------ + +Support PowerPC architecture. + +Use ``defconfig_ppc_64-power8-*-*`` + +.. _nic_features_x86-32: + +x86-32 +------ + +Support 32bits x86 architecture. + +Use ``defconfig_x86_x32-native-*-*`` and ``defconfig_i686-native-*-*``. + + +.. _nic_features_x86-64: + +x86-64 +------ + +Support 64bits x86 architecture. + +Use ``defconfig_x86_64-native-*-*``. + + +.. _nic_features_usage_doc: + +Usage doc +--------- + +Documentation describes usage. + +See ``doc/guides/nics/*.rst`` + + +.. _nic_features_design_doc: + +Design doc +---------- + +Documentation describes design. + +See ``doc/guides/nics/*.rst``. + + +.. _nic_features_perf_doc: + +Perf doc +-------- + +Documentation describes performance values. + +See ``dpdk.org/doc/perf/*``. + + + +.. _nic_features_other: + +Other dev ops not represented by a Feature +------------------------------------------ + +* ``rxq_info_get`` +* ``txq_info_get`` +* ``vlan_tpid_set`` +* ``vlan_strip_queue_set`` +* ``vlan_pvid_set`` +* ``rx_queue_count`` +* ``l2_tunnel_offload_set`` +* ``uc_hash_table_set`` +* ``uc_all_hash_table_set`` +* ``udp_tunnel_port_add`` +* ``udp_tunnel_port_del`` +* ``l2_tunnel_eth_type_conf`` +* ``l2_tunnel_offload_set`` +* ``tx_pkt_prepare`` diff --git a/doc/guides/nics/features/ark.ini b/doc/guides/nics/features/ark.ini index 31a35279..ec8a2b99 100644 --- a/doc/guides/nics/features/ark.ini +++ b/doc/guides/nics/features/ark.ini @@ -4,6 +4,7 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Speed capabilities = Y Queue start/stop = Y Jumbo frame = Y Scattered Rx = Y diff --git a/doc/guides/nics/features/bnx2x.ini b/doc/guides/nics/features/bnx2x.ini index 1ad8a3e8..3e33e9ab 100644 --- a/doc/guides/nics/features/bnx2x.ini +++ b/doc/guides/nics/features/bnx2x.ini @@ -4,6 +4,7 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Speed capabilities = P Link status = Y Link status event = Y Promiscuous mode = Y diff --git a/doc/guides/nics/features/bnx2x_vf.ini b/doc/guides/nics/features/bnx2x_vf.ini index da9168ea..c270902e 100644 --- a/doc/guides/nics/features/bnx2x_vf.ini +++ b/doc/guides/nics/features/bnx2x_vf.ini @@ -4,6 +4,7 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Speed capabilities = P Link status = Y Link status event = Y Promiscuous mode = Y diff --git a/doc/guides/nics/features/bnxt.ini b/doc/guides/nics/features/bnxt.ini index 013a9cda..119132e1 100644 --- a/doc/guides/nics/features/bnxt.ini +++ b/doc/guides/nics/features/bnxt.ini @@ -6,11 +6,21 @@ [Features] Link status = Y Queue start/stop = Y +MTU update = Y +Jumbo frame = Y +LRO = Y Promiscuous mode = Y +Allmulticast mode = Y Unicast MAC filter = Y Multicast MAC filter = Y RSS reta update = Y +SR-IOV = Y +VLAN filter = Y +VLAN offload = Y Basic stats = Y Extended stats = Y +FW version = Y +LED = Y Linux UIO = Y +Linux VFIO = Y x86-64 = Y diff --git a/doc/guides/nics/features/cxgbe.ini b/doc/guides/nics/features/cxgbe.ini index 2e72a107..3d0fde2f 100644 --- a/doc/guides/nics/features/cxgbe.ini +++ b/doc/guides/nics/features/cxgbe.ini @@ -4,6 +4,7 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Speed capabilities = Y Link status = Y Queue start/stop = Y MTU update = Y diff --git a/doc/guides/nics/features/default.ini b/doc/guides/nics/features/default.ini index cafc6c70..54243069 100644 --- a/doc/guides/nics/features/default.ini +++ b/doc/guides/nics/features/default.ini @@ -13,6 +13,7 @@ Link status event = Removal event = Queue status event = Rx interrupt = +Lock-free Tx queue = Free Tx mbuf on demand = Queue start/stop = MTU update = @@ -60,6 +61,7 @@ Stats per queue = FW version = EEPROM dump = Registers dump = +LED = Multiprocess aware = BSD nic_uio = Linux UIO = diff --git a/doc/guides/nics/features/dpaa2.ini b/doc/guides/nics/features/dpaa2.ini index d43f4046..146e087f 100644 --- a/doc/guides/nics/features/dpaa2.ini +++ b/doc/guides/nics/features/dpaa2.ini @@ -4,15 +4,23 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Speed capabilities = P Link status = Y Queue start/stop = Y +Jumbo frame = Y MTU update = Y Promiscuous mode = Y +Allmulticast mode = Y +Unicast MAC filter = Y RSS hash = Y +VLAN filter = Y +Flow control = Y +VLAN offload = Y L3 checksum offload = Y L4 checksum offload = Y Packet type parsing = Y Basic stats = Y +FW version = Y Linux VFIO = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/e1000.ini b/doc/guides/nics/features/e1000.ini index 260d46da..51ca580f 100644 --- a/doc/guides/nics/features/e1000.ini +++ b/doc/guides/nics/features/e1000.ini @@ -4,6 +4,7 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Speed capabilities = P Link status = Y Link status event = Y Rx interrupt = Y diff --git a/doc/guides/nics/features/ena.ini b/doc/guides/nics/features/ena.ini index 74969fd0..691c1e3d 100644 --- a/doc/guides/nics/features/ena.ini +++ b/doc/guides/nics/features/ena.ini @@ -4,6 +4,7 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Speed capabilities = P Queue start/stop = Y MTU update = Y Jumbo frame = Y diff --git a/doc/guides/nics/features/enic.ini b/doc/guides/nics/features/enic.ini index 94e7f3cb..0de3ef53 100644 --- a/doc/guides/nics/features/enic.ini +++ b/doc/guides/nics/features/enic.ini @@ -20,6 +20,7 @@ VLAN filter = Y CRC offload = Y VLAN offload = Y Flow director = Y +Flow API = Y L3 checksum offload = Y L4 checksum offload = Y Packet type parsing = Y diff --git a/doc/guides/nics/features/failsafe.ini b/doc/guides/nics/features/failsafe.ini new file mode 100644 index 00000000..a42e344a --- /dev/null +++ b/doc/guides/nics/features/failsafe.ini @@ -0,0 +1,26 @@ +; +; Supported features of the 'fail-safe' poll mode driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +Link status = Y +Link status event = Y +MTU update = Y +Jumbo frame = Y +Promiscuous mode = Y +Allmulticast mode = Y +Unicast MAC filter = Y +Multicast MAC filter = Y +VLAN filter = Y +Flow control = Y +Flow API = Y +Packet type parsing = Y +Basic stats = Y +Stats per queue = Y +ARMv7 = Y +ARMv8 = Y +Power8 = Y +x86-32 = Y +x86-64 = Y +Usage doc = Y diff --git a/doc/guides/nics/features/fm10k.ini b/doc/guides/nics/features/fm10k.ini index 9e1035f3..f0f61a7d 100644 --- a/doc/guides/nics/features/fm10k.ini +++ b/doc/guides/nics/features/fm10k.ini @@ -4,6 +4,7 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Speed capabilities = P Rx interrupt = Y Queue start/stop = Y Jumbo frame = Y diff --git a/doc/guides/nics/features/fm10k_vec.ini b/doc/guides/nics/features/fm10k_vec.ini index 1384ab15..4917e820 100644 --- a/doc/guides/nics/features/fm10k_vec.ini +++ b/doc/guides/nics/features/fm10k_vec.ini @@ -4,6 +4,7 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Speed capabilities = P Rx interrupt = Y Queue start/stop = Y Jumbo frame = Y diff --git a/doc/guides/nics/features/fm10k_vf.ini b/doc/guides/nics/features/fm10k_vf.ini index 15de536f..32b93df4 100644 --- a/doc/guides/nics/features/fm10k_vf.ini +++ b/doc/guides/nics/features/fm10k_vf.ini @@ -4,6 +4,7 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Speed capabilities = P Rx interrupt = Y Queue start/stop = Y Jumbo frame = Y diff --git a/doc/guides/nics/features/fm10k_vf_vec.ini b/doc/guides/nics/features/fm10k_vf_vec.ini index b32550cb..6f4a639a 100644 --- a/doc/guides/nics/features/fm10k_vf_vec.ini +++ b/doc/guides/nics/features/fm10k_vf_vec.ini @@ -4,6 +4,7 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Speed capabilities = P Rx interrupt = Y Queue start/stop = Y Jumbo frame = Y diff --git a/doc/guides/nics/features/i40e.ini b/doc/guides/nics/features/i40e.ini index ecabce0b..e862712c 100644 --- a/doc/guides/nics/features/i40e.ini +++ b/doc/guides/nics/features/i40e.ini @@ -4,6 +4,7 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Speed capabilities = Y Link status = Y Link status event = Y Rx interrupt = Y diff --git a/doc/guides/nics/features/i40e_vec.ini b/doc/guides/nics/features/i40e_vec.ini index 206f348b..7d7b3a92 100644 --- a/doc/guides/nics/features/i40e_vec.ini +++ b/doc/guides/nics/features/i40e_vec.ini @@ -4,6 +4,7 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Speed capabilities = Y Link status = Y Link status event = Y Rx interrupt = Y diff --git a/doc/guides/nics/features/igb.ini b/doc/guides/nics/features/igb.ini index 11450270..33d64d99 100644 --- a/doc/guides/nics/features/igb.ini +++ b/doc/guides/nics/features/igb.ini @@ -4,6 +4,7 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Speed capabilities = P Link status = Y Link status event = Y Rx interrupt = Y @@ -26,6 +27,7 @@ N-tuple filter = Y SYN filter = Y Flexible filter = Y Flow control = Y +Flow API = Y CRC offload = Y VLAN offload = Y QinQ offload = Y diff --git a/doc/guides/nics/features/ixgbe.ini b/doc/guides/nics/features/ixgbe.ini index 4aa7af6d..9ff5d8f8 100644 --- a/doc/guides/nics/features/ixgbe.ini +++ b/doc/guides/nics/features/ixgbe.ini @@ -4,6 +4,7 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Speed capabilities = Y Link status = Y Link status event = Y Rx interrupt = Y diff --git a/doc/guides/nics/features/ixgbe_vec.ini b/doc/guides/nics/features/ixgbe_vec.ini index 4da81182..4d56df4f 100644 --- a/doc/guides/nics/features/ixgbe_vec.ini +++ b/doc/guides/nics/features/ixgbe_vec.ini @@ -4,6 +4,7 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Speed capabilities = Y Link status = Y Link status event = Y Rx interrupt = Y diff --git a/doc/guides/nics/features/liquidio.ini b/doc/guides/nics/features/liquidio.ini index 49cc3566..3bea03a3 100644 --- a/doc/guides/nics/features/liquidio.ini +++ b/doc/guides/nics/features/liquidio.ini @@ -4,6 +4,7 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Speed capabilities = Y Link status = Y Link status event = Y Jumbo frame = Y diff --git a/doc/guides/nics/features/mlx4.ini b/doc/guides/nics/features/mlx4.ini index 285f0ecf..1d5f2668 100644 --- a/doc/guides/nics/features/mlx4.ini +++ b/doc/guides/nics/features/mlx4.ini @@ -4,9 +4,11 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Speed capabilities = P Link status = Y Link status event = Y Removal event = Y +Rx interrupt = Y Queue start/stop = Y MTU update = Y Jumbo frame = Y diff --git a/doc/guides/nics/features/mlx5.ini b/doc/guides/nics/features/mlx5.ini index e228c412..2913591c 100644 --- a/doc/guides/nics/features/mlx5.ini +++ b/doc/guides/nics/features/mlx5.ini @@ -28,15 +28,15 @@ CRC offload = Y VLAN offload = Y L3 checksum offload = Y L4 checksum offload = Y -Inner L3 checksum = Y -Inner L4 checksum = Y Packet type parsing = Y Rx descriptor status = Y Tx descriptor status = Y Basic stats = Y +Extended stats = Y Stats per queue = Y Multiprocess aware = Y Other kdrv = Y +ARMv8 = Y Power8 = Y x86-32 = Y x86-64 = Y diff --git a/doc/guides/nics/features/qede.ini b/doc/guides/nics/features/qede.ini index fba5dc33..cbadc194 100644 --- a/doc/guides/nics/features/qede.ini +++ b/doc/guides/nics/features/qede.ini @@ -32,6 +32,8 @@ Extended stats = Y Stats per queue = Y Multiprocess aware = Y Linux UIO = Y +ARMv8 = Y +x86-32 = Y x86-64 = Y Usage doc = Y N-tuple filter = Y diff --git a/doc/guides/nics/features/qede_vf.ini b/doc/guides/nics/features/qede_vf.ini index 21ec40fa..18857b6e 100644 --- a/doc/guides/nics/features/qede_vf.ini +++ b/doc/guides/nics/features/qede_vf.ini @@ -30,6 +30,8 @@ Extended stats = Y Stats per queue = Y Multiprocess aware = Y Linux UIO = Y +ARMv8 = Y +x86-32 = Y x86-64 = Y LRO = Y TSO = Y diff --git a/doc/guides/nics/features/sfc_efx.ini b/doc/guides/nics/features/sfc_efx.ini index 7957b5e9..1db7f67b 100644 --- a/doc/guides/nics/features/sfc_efx.ini +++ b/doc/guides/nics/features/sfc_efx.ini @@ -28,6 +28,7 @@ Packet type parsing = Y Basic stats = Y Extended stats = Y FW version = Y +Multiprocess aware = Y BSD nic_uio = Y Linux UIO = Y Linux VFIO = Y diff --git a/doc/guides/nics/features/szedata2.ini b/doc/guides/nics/features/szedata2.ini index 624314d3..a0e6f6e8 100644 --- a/doc/guides/nics/features/szedata2.ini +++ b/doc/guides/nics/features/szedata2.ini @@ -4,6 +4,7 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Speed capabilities = P Link status = Y Queue start/stop = Y Scattered Rx = Y diff --git a/doc/guides/nics/features/tap.ini b/doc/guides/nics/features/tap.ini index 3efae758..f0e893d6 100644 --- a/doc/guides/nics/features/tap.ini +++ b/doc/guides/nics/features/tap.ini @@ -4,6 +4,7 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Speed capabilities = P Link status = Y Link status event = Y Jumbo frame = Y @@ -11,9 +12,10 @@ Promiscuous mode = Y Allmulticast mode = Y Basic stats = Y Flow API = Y +L3 checksum offload = Y +L4 checksum offload = Y MTU update = Y Multicast MAC filter = Y -Speed capabilities = Y Unicast MAC filter = Y Packet type parsing = Y Flow control = Y diff --git a/doc/guides/nics/features/virtio.ini b/doc/guides/nics/features/virtio.ini index 8e3aca1d..16e577df 100644 --- a/doc/guides/nics/features/virtio.ini +++ b/doc/guides/nics/features/virtio.ini @@ -4,6 +4,7 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Speed capabilities = P Link status = Y Rx interrupt = Y Queue start/stop = Y diff --git a/doc/guides/nics/features/virtio_vec.ini b/doc/guides/nics/features/virtio_vec.ini index ec93f5c4..c06c860d 100644 --- a/doc/guides/nics/features/virtio_vec.ini +++ b/doc/guides/nics/features/virtio_vec.ini @@ -4,6 +4,7 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Speed capabilities = P Link status = Y Rx interrupt = Y Queue start/stop = Y diff --git a/doc/guides/nics/features/vmxnet3.ini b/doc/guides/nics/features/vmxnet3.ini index ef95932a..9a115138 100644 --- a/doc/guides/nics/features/vmxnet3.ini +++ b/doc/guides/nics/features/vmxnet3.ini @@ -4,6 +4,7 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Speed capabilities = P Link status = Y Link status event = Y Queue start/stop = Y @@ -20,6 +21,7 @@ VLAN offload = Y L4 checksum offload = Y Packet type parsing = Y Basic stats = Y +Extended stats = Y Stats per queue = Y Linux UIO = Y Linux VFIO = Y diff --git a/doc/guides/nics/i40e.rst b/doc/guides/nics/i40e.rst index 4d3c7ca0..bc200d39 100644 --- a/doc/guides/nics/i40e.rst +++ b/doc/guides/nics/i40e.rst @@ -404,16 +404,6 @@ is used as the VF driver, DPDK cannot choose 16 byte receive descriptor. That is to say, user should keep ``CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC=n`` in config file. -Link down with i40e kernel driver after DPDK application exit -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -After DPDK application quit, and the device is bound back to Linux i40e -kernel driver, the link cannot be up after ``ifconfig <dev> up``. -To work around this issue, ``ethtool -s <dev> autoneg on`` should be -set first and then the link can be brought up through ``ifconfig <dev> up``. - -NOTE: requires Linux kernel i40e driver version >= 1.4.X - Receive packets with Ethertype 0x88A8 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -447,3 +437,109 @@ It means if APP has set the max bandwidth for that TC, it comes to no effect. It's suggested to set the strict priority mode for a TC that is latency sensitive but no consuming much bandwidth. + +VF performance is impacted by PCI extended tag setting +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To reach maximum NIC performance in the VF the PCI extended tag must be +enabled. The DPDK I40E PF driver will set this feature during initialization, +but the kernel PF driver does not. So when running traffic on a VF which is +managed by the kernel PF driver, a significant NIC performance downgrade has +been observed (for 64 byte packets, there is about 25% linerate downgrade for +a 25G device and about 35% for a 40G device). + +For kernel version >= 4.11, the kernel's PCI driver will enable the extended +tag if it detects that the device supports it. So by default, this is not an +issue. For kernels <= 4.11 or when the PCI extended tag is disabled it can be +enabled using the steps below. + +#. Get the current value of the PCI configure register:: + + setpci -s <XX:XX.X> a8.w + +#. Set bit 8:: + + value = value | 0x100 + +#. Set the PCI configure register with new value:: + + setpci -s <XX:XX.X> a8.w=<value> + +High Performance of Small Packets on 40G NIC +-------------------------------------------- + +As there might be firmware fixes for performance enhancement in latest version +of firmware image, the firmware update might be needed for getting high performance. +Check with the local Intel's Network Division application engineers for firmware updates. +Users should consult the release notes specific to a DPDK release to identify +the validated firmware version for a NIC using the i40e driver. + +Use 16 Bytes RX Descriptor Size +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +As i40e PMD supports both 16 and 32 bytes RX descriptor sizes, and 16 bytes size can provide helps to high performance of small packets. +Configuration of ``CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC`` in config files can be changed to use 16 bytes size RX descriptors. + +High Performance and per Packet Latency Tradeoff +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Due to the hardware design, the interrupt signal inside NIC is needed for per +packet descriptor write-back. The minimum interval of interrupts could be set +at compile time by ``CONFIG_RTE_LIBRTE_I40E_ITR_INTERVAL`` in configuration files. +Though there is a default configuration, the interval could be tuned by the +users with that configuration item depends on what the user cares about more, +performance or per packet latency. + +Example of getting best performance with l3fwd example +------------------------------------------------------ + +The following is an example of running the DPDK ``l3fwd`` sample application to get high performance with an +Intel server platform and Intel XL710 NICs. + +The example scenario is to get best performance with two Intel XL710 40GbE ports. +See :numref:`figure_intel_perf_test_setup` for the performance test setup. + +.. _figure_intel_perf_test_setup: + +.. figure:: img/intel_perf_test_setup.* + + Performance Test Setup + + +1. Add two Intel XL710 NICs to the platform, and use one port per card to get best performance. + The reason for using two NICs is to overcome a PCIe Gen3's limitation since it cannot provide 80G bandwidth + for two 40G ports, but two different PCIe Gen3 x8 slot can. + Refer to the sample NICs output above, then we can select ``82:00.0`` and ``85:00.0`` as test ports:: + + 82:00.0 Ethernet [0200]: Intel XL710 for 40GbE QSFP+ [8086:1583] + 85:00.0 Ethernet [0200]: Intel XL710 for 40GbE QSFP+ [8086:1583] + +2. Connect the ports to the traffic generator. For high speed testing, it's best to use a hardware traffic generator. + +3. Check the PCI devices numa node (socket id) and get the cores number on the exact socket id. + In this case, ``82:00.0`` and ``85:00.0`` are both in socket 1, and the cores on socket 1 in the referenced platform + are 18-35 and 54-71. + Note: Don't use 2 logical cores on the same core (e.g core18 has 2 logical cores, core18 and core54), instead, use 2 logical + cores from different cores (e.g core18 and core19). + +4. Bind these two ports to igb_uio. + +5. As to XL710 40G port, we need at least two queue pairs to achieve best performance, then two queues per port + will be required, and each queue pair will need a dedicated CPU core for receiving/transmitting packets. + +6. The DPDK sample application ``l3fwd`` will be used for performance testing, with using two ports for bi-directional forwarding. + Compile the ``l3fwd sample`` with the default lpm mode. + +7. The command line of running l3fwd would be something like the following:: + + ./l3fwd -l 18-21 -n 4 -w 82:00.0 -w 85:00.0 \ + -- -p 0x3 --config '(0,0,18),(0,1,19),(1,0,20),(1,1,21)' + + This means that the application uses core 18 for port 0, queue pair 0 forwarding, core 19 for port 0, queue pair 1 forwarding, + core 20 for port 1, queue pair 0 forwarding, and core 21 for port 1, queue pair 1 forwarding. + +8. Configure the traffic at a traffic generator. + + * Start creating a stream on packet generator. + + * Set the Ethernet II type to 0x0800. diff --git a/doc/guides/linux_gsg/img/intel_perf_test_setup.svg b/doc/guides/nics/img/intel_perf_test_setup.svg index 27c3c1cd..27c3c1cd 100644 --- a/doc/guides/linux_gsg/img/intel_perf_test_setup.svg +++ b/doc/guides/nics/img/intel_perf_test_setup.svg diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst index 240d0824..36f4f3ff 100644 --- a/doc/guides/nics/index.rst +++ b/doc/guides/nics/index.rst @@ -36,6 +36,7 @@ Network Interface Controller Drivers :numbered: overview + features build_and_test ark avp @@ -64,6 +65,7 @@ Network Interface Controller Drivers vhost vmxnet3 pcap_ring + fail_safe **Figures** diff --git a/doc/guides/nics/mlx4.rst b/doc/guides/nics/mlx4.rst index f1f26d4f..f8885b23 100644 --- a/doc/guides/nics/mlx4.rst +++ b/doc/guides/nics/mlx4.rst @@ -1,5 +1,6 @@ .. BSD LICENSE Copyright 2012-2015 6WIND S.A. + Copyright 2015 Mellanox Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -76,6 +77,7 @@ Compiling librte_pmd_mlx4 causes DPDK to be linked against libibverbs. Features -------- +- Multi arch support: x86_64 and POWER8. - RSS, also known as RCA, is supported. In this mode the number of configured RX queues must be a power of two. - VLAN filtering is supported. @@ -87,6 +89,7 @@ Features - Inner L3/L4 (IP, TCP and UDP) TX/RX checksum offloading and validation. - Outer L3 (IP) TX/RX checksum offloading and validation for VXLAN frames. - Secondary process TX is supported. +- RX interrupts. Limitations ----------- @@ -116,6 +119,14 @@ These options can be modified in the ``.config`` file. adds additional run-time checks and debugging messages at the cost of lower performance. +- ``CONFIG_RTE_LIBRTE_MLX4_DEBUG_BROKEN_VERBS`` (default **n**) + + Mellanox OFED versions earlier than 4.2 may return false errors from + Verbs object destruction APIs after the device is plugged out. + Enabling this option replaces assertion checks that cause the program + to abort with harmless debugging messages as a workaround. + Relevant only when CONFIG_RTE_LIBRTE_MLX4_DEBUG is enabled. + - ``CONFIG_RTE_LIBRTE_MLX4_SGE_WR_N`` (default **4**) Number of scatter/gather elements (SGEs) per work request (WR). Lowering @@ -244,9 +255,8 @@ DPDK and must be installed separately: Currently supported by DPDK: -- Mellanox OFED **4.0-2.0.0.0**. -- Firmware version **2.40.7000**. -- Supported architectures: **x86_64** and **POWER8**. +- Mellanox OFED **4.1**. +- Firmware version **2.36.5000** and above. Getting Mellanox OFED ~~~~~~~~~~~~~~~~~~~~~ @@ -273,6 +283,138 @@ Supported NICs * Mellanox(R) ConnectX(R)-3 Pro 40G MCX354A-FCC_Ax (2*40G) +Quick Start Guide +----------------- + +1. Download latest Mellanox OFED. For more info check the `prerequisites`_. + +2. Install the required libraries and kernel modules either by installing + only the required set, or by installing the entire Mellanox OFED: + + For bare metal use: + + .. code-block:: console + + ./mlnxofedinstall + + For SR-IOV hypervisors use: + + .. code-block:: console + + ./mlnxofedinstall --enable-sriov -hypervisor + + For SR-IOV virtual machine use: + + .. code-block:: console + + ./mlnxofedinstall --guest + +3. Verify the firmware is the correct one: + + .. code-block:: console + + ibv_devinfo + +4. Set all ports links to Ethernet, follow instructions on the screen: + + .. code-block:: console + + connectx_port_config + + Or in the manual way: + + .. code-block:: console + + PCI=<NIC PCI address> + echo eth > "/sys/bus/pci/devices/$PCI/mlx4_port0" + echo eth > "/sys/bus/pci/devices/$PCI/mlx4_port1" + +5. In case of bare metal or hypervisor, configure optimized steering mode + by adding the following line to ``/etc/modprobe.d/mlx4_core.conf``: + + .. code-block:: console + + options mlx4_core log_num_mgm_entry_size=-7 + + .. note:: + + If VLAN filtering is used, set log_num_mgm_entry_size=-1. + Performance degradation can occur on this case. + +6. Restart the driver: + + .. code-block:: console + + /etc/init.d/openibd restart + + or: + + .. code-block:: console + + service openibd restart + +7. Compile DPDK and you are ready to go. See instructions on + :ref:`Development Kit Build System <Development_Kit_Build_System>` + +Performance tuning +------------------ + +1. Verify the optimized steering mode is configured: + + .. code-block:: console + + cat /sys/module/mlx4_core/parameters/log_num_mgm_entry_size + +2. Use environment variable MLX4_INLINE_RECV_SIZE=64 to get maximum + performance for 64B messages. + +3. Use the CPU near local NUMA node to which the PCIe adapter is connected, + for better performance. For VMs, verify that the right CPU + and NUMA node are pinned according to the above. Run: + + .. code-block:: console + + lstopo-no-graphics + + to identify the NUMA node to which the PCIe adapter is connected. + +4. If more than one adapter is used, and root complex capabilities allow + to put both adapters on the same NUMA node without PCI bandwidth degradation, + it is recommended to locate both adapters on the same NUMA node. + This in order to forward packets from one to the other without + NUMA performance penalty. + +5. Disable pause frames: + + .. code-block:: console + + ethtool -A <netdev> rx off tx off + +6. Verify IO non-posted prefetch is disabled by default. This can be checked + via the BIOS configuration. Please contact you server provider for more + information about the settings. + +.. note:: + + On some machines, depends on the machine integrator, it is beneficial + to set the PCI max read request parameter to 1K. This can be + done in the following way: + + To query the read request size use: + + .. code-block:: console + + setpci -s <NIC PCI address> 68.w + + If the output is different than 3XXX, set it by: + + .. code-block:: console + + setpci -s <NIC PCI address> 68.w=3XXX + + The XXX can be different on different systems. Make sure to configure + according to the setpci output. + Usage example ------------- diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index da6dc278..f4cb18bc 100644 --- a/doc/guides/nics/mlx5.rst +++ b/doc/guides/nics/mlx5.rst @@ -1,5 +1,6 @@ .. BSD LICENSE Copyright 2015 6WIND S.A. + Copyright 2015 Mellanox Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -64,6 +65,9 @@ physical memory (or memory that does not belong to the current process). This capability allows the PMD to coexist with kernel network interfaces which remain functional, although they stop receiving unicast packets as long as they share the same MAC address. +This means legacy linux control tools (for example: ethtool, ifconfig and +more) can operate on the same network interfaces that owned by the DPDK +application. Enabling librte_pmd_mlx5 causes DPDK applications to be linked against libibverbs. @@ -71,6 +75,7 @@ libibverbs. Features -------- +- Multi arch support: x86_64, POWER8, ARMv8. - Multiple TX and RX queues. - Support for scattered TX and RX frames. - IPv4, IPv6, TCPv4, TCPv6, UDPv4 and UDPv6 RSS on any number of queues. @@ -92,6 +97,8 @@ Features - RSS hash result is supported. - Hardware TSO. - Hardware checksum TX offload for VXLAN and GRE. +- RX interrupts. +- Statistics query including Basic, Extended and per queue. Limitations ----------- @@ -100,6 +107,22 @@ Limitations - Port statistics through software counters only. - Hardware checksum RX offloads for VXLAN inner header are not supported yet. - Secondary process RX is not supported. +- Flow pattern without any specific vlan will match for vlan packets as well: + + When VLAN spec is not specified in the pattern, the matching rule will be created with VLAN as a wild card. + Meaning, the flow rule:: + + flow create 0 ingress pattern eth / vlan vid is 3 / ipv4 / end ... + + Will only match vlan packets with vid=3. and the flow rules:: + + flow create 0 ingress pattern eth / ipv4 / end ... + + Or:: + + flow create 0 ingress pattern eth / vlan / ipv4 / end ... + + Will match any ipv4 packet (VLAN included). Configuration ------------- @@ -156,13 +179,12 @@ Run-time configuration - ``rxq_cqe_comp_en`` parameter [int] A nonzero value enables the compression of CQE on RX side. This feature - allows to save PCI bandwidth and improve performance at the cost of a - slightly higher CPU usage. Enabled by default. + allows to save PCI bandwidth and improve performance. Enabled by default. Supported on: - - x86_64 with ConnectX4 and ConnectX4 LX - - Power8 with ConnectX4 LX + - x86_64 with ConnectX-4, ConnectX-4 LX and ConnectX-5. + - POWER8 and ARMv8 with ConnectX-4 LX and ConnectX-5. - ``txq_inline`` parameter [int] @@ -170,8 +192,8 @@ Run-time configuration Can improve PPS performance when PCI back pressure is detected and may be useful for scenarios involving heavy traffic on many queues. - It is not enabled by default (set to 0) since the additional software - logic necessary to handle this mode can lower performance when back + Because additional software logic is necessary to handle this mode, this + option should be used with care, as it can lower performance when back pressure is not expected. - ``txqs_min_inline`` parameter [int] @@ -181,6 +203,15 @@ Run-time configuration This option should be used in combination with ``txq_inline`` above. + On ConnectX-4, ConnectX-4 LX and ConnectX-5 without Enhanced MPW: + + - Disabled by default. + - In case ``txq_inline`` is set recommendation is 4. + + On ConnectX-5 with Enhanced MPW: + + - Set to 8 by default. + - ``txq_mpw_en`` parameter [int] A nonzero value enables multi-packet send (MPS) for ConnectX-4 Lx and @@ -221,9 +252,21 @@ Run-time configuration A nonzero value enables hardware TSO. When hardware TSO is enabled, packets marked with TCP segmentation - offload will be divided into segments by the hardware. + offload will be divided into segments by the hardware. Disabled by default. + +- ``tx_vec_en`` parameter [int] - Disabled by default. + A nonzero value enables Tx vector on ConnectX-5 only NIC if the number of + global Tx queues on the port is lesser than MLX5_VPMD_MIN_TXQS. + + Enabled by default on ConnectX-5. + +- ``rx_vec_en`` parameter [int] + + A nonzero value enables Rx vector if the port is not configured in + multi-segment otherwise this parameter is ignored. + + Enabled by default. Prerequisites ------------- @@ -279,13 +322,13 @@ DPDK and must be installed separately: Currently supported by DPDK: -- Mellanox OFED version: **4.0-2.0.0.0** +- Mellanox OFED version: **4.1**. - firmware version: - - ConnectX-4: **12.18.2000** - - ConnectX-4 Lx: **14.18.2000** - - ConnectX-5: **16.19.1200** - - ConnectX-5 Ex: **16.19.1200** + - ConnectX-4: **12.20.1010** and above. + - ConnectX-4 Lx: **14.20.1010** and above. + - ConnectX-5: **16.20.1010** and above. + - ConnectX-5 Ex: **16.20.1010** and above. Getting Mellanox OFED ~~~~~~~~~~~~~~~~~~~~~ @@ -330,6 +373,155 @@ Supported NICs * Mellanox(R) ConnectX(R)-5 100G MCX556A-ECAT (2x100G) * Mellanox(R) ConnectX(R)-5 Ex EN 100G MCX516A-CDAT (2x100G) +Quick Start Guide +----------------- + +1. Download latest Mellanox OFED. For more info check the `prerequisites`_. + + +2. Install the required libraries and kernel modules either by installing + only the required set, or by installing the entire Mellanox OFED: + + .. code-block:: console + + ./mlnxofedinstall + +3. Verify the firmware is the correct one: + + .. code-block:: console + + ibv_devinfo + +4. Verify all ports links are set to Ethernet: + + .. code-block:: console + + mlxconfig -d <mst device> query | grep LINK_TYPE + LINK_TYPE_P1 ETH(2) + LINK_TYPE_P2 ETH(2) + + Link types may have to be configured to Ethernet: + + .. code-block:: console + + mlxconfig -d <mst device> set LINK_TYPE_P1/2=1/2/3 + + * LINK_TYPE_P1=<1|2|3> , 1=Infiniband 2=Ethernet 3=VPI(auto-sense) + + For hypervisors verify SR-IOV is enabled on the NIC: + + .. code-block:: console + + mlxconfig -d <mst device> query | grep SRIOV_EN + SRIOV_EN True(1) + + If needed, set enable the set the relevant fields: + + .. code-block:: console + + mlxconfig -d <mst device> set SRIOV_EN=1 NUM_OF_VFS=16 + mlxfwreset -d <mst device> reset + +5. Restart the driver: + + .. code-block:: console + + /etc/init.d/openibd restart + + or: + + .. code-block:: console + + service openibd restart + + If link type was changed, firmware must be reset as well: + + .. code-block:: console + + mlxfwreset -d <mst device> reset + + For hypervisors, after reset write the sysfs number of virtual functions + needed for the PF. + + To dynamically instantiate a given number of virtual functions (VFs): + + .. code-block:: console + + echo [num_vfs] > /sys/class/infiniband/mlx5_0/device/sriov_numvfs + +6. Compile DPDK and you are ready to go. See instructions on + :ref:`Development Kit Build System <Development_Kit_Build_System>` + +Performance tuning +------------------ + +1. Configure aggressive CQE Zipping for maximum performance: + + .. code-block:: console + + mlxconfig -d <mst device> s CQE_COMPRESSION=1 + + To set it back to the default CQE Zipping mode use: + + .. code-block:: console + + mlxconfig -d <mst device> s CQE_COMPRESSION=0 + +2. In case of virtualization: + + - Make sure that hypervisor kernel is 3.16 or newer. + - Configure boot with ``iommu=pt``. + - Use 1G huge pages. + - Make sure to allocate a VM on huge pages. + - Make sure to set CPU pinning. + +3. Use the CPU near local NUMA node to which the PCIe adapter is connected, + for better performance. For VMs, verify that the right CPU + and NUMA node are pinned according to the above. Run: + + .. code-block:: console + + lstopo-no-graphics + + to identify the NUMA node to which the PCIe adapter is connected. + +4. If more than one adapter is used, and root complex capabilities allow + to put both adapters on the same NUMA node without PCI bandwidth degradation, + it is recommended to locate both adapters on the same NUMA node. + This in order to forward packets from one to the other without + NUMA performance penalty. + +5. Disable pause frames: + + .. code-block:: console + + ethtool -A <netdev> rx off tx off + +6. Verify IO non-posted prefetch is disabled by default. This can be checked + via the BIOS configuration. Please contact you server provider for more + information about the settings. + +.. note:: + + On some machines, depends on the machine integrator, it is beneficial + to set the PCI max read request parameter to 1K. This can be + done in the following way: + + To query the read request size use: + + .. code-block:: console + + setpci -s <NIC PCI address> 68.w + + If the output is different than 3XXX, set it by: + + .. code-block:: console + + setpci -s <NIC PCI address> 68.w=3XXX + + The XXX can be different on different systems. Make sure to configure + according to the setpci output. + Notes for testpmd ----------------- diff --git a/doc/guides/nics/overview.rst b/doc/guides/nics/overview.rst index 757a3c90..0df0ef81 100644 --- a/doc/guides/nics/overview.rst +++ b/doc/guides/nics/overview.rst @@ -48,6 +48,8 @@ There are more differences between drivers regarding some internal properties, portability or even documentation availability. Most of these differences are summarized below. +More details about features can be found in :doc:`features`. + .. _table_net_pmd_features: .. include:: overview_table.txt diff --git a/doc/guides/nics/qede.rst b/doc/guides/nics/qede.rst index afe2df89..09a10be1 100644 --- a/doc/guides/nics/qede.rst +++ b/doc/guides/nics/qede.rst @@ -62,13 +62,13 @@ Supported Features - VXLAN tunneling offload - N-tuple filter and flow director (limited support) - LRO/TSO +- NPAR (NIC Partitioning) Non-supported Features ---------------------- - SR-IOV PF - GENEVE and NVGRE Tunneling offloads -- NPAR Supported QLogic Adapters ------------------------- diff --git a/doc/guides/nics/sfc_efx.rst b/doc/guides/nics/sfc_efx.rst index 5f825e9a..7761989c 100644 --- a/doc/guides/nics/sfc_efx.rst +++ b/doc/guides/nics/sfc_efx.rst @@ -259,7 +259,7 @@ boolean parameters value. - ``debug_init`` [bool] (default **n**) - Enable extra logging during device intialization and startup. + Enable extra logging during device initialization and startup. - ``mcdi_logging`` [bool] (default **n**) diff --git a/doc/guides/nics/szedata2.rst b/doc/guides/nics/szedata2.rst index 60080a9f..1a5d4138 100644 --- a/doc/guides/nics/szedata2.rst +++ b/doc/guides/nics/szedata2.rst @@ -91,14 +91,34 @@ These configuration options can be modified before compilation in the * ``CONFIG_RTE_LIBRTE_PMD_SZEDATA2_AS`` default value: **0** - This option defines type of firmware address space. - Currently supported value is: + This option defines type of firmware address space and must be set + according to the used card and mode. + Currently supported values are: - * **0** for firmwares: + * **0** - for cards (modes): - * NIC_100G1_LR4 - * HANIC_100G1_LR4 - * HANIC_100G1_SR10 + * NFB-100G1 (100G1) + + * **1** - for cards (modes): + + * NFB-100G2Q (100G1) + + * **2** - for cards (modes): + + * NFB-40G2 (40G2) + * NFB-100G2C (100G2) + * NFB-100G2Q (40G2) + + * **3** - for cards (modes): + + * NFB-40G2 (10G8) + * NFB-100G2Q (10G8) + + * **4** - for cards (modes): + + * NFB-100G1 (10G10) + + * **5** - for experimental firmwares and future use Using the SZEDATA2 PMD ---------------------- diff --git a/doc/guides/nics/tap.rst b/doc/guides/nics/tap.rst index 5c5ba535..f3ee95d2 100644 --- a/doc/guides/nics/tap.rst +++ b/doc/guides/nics/tap.rst @@ -46,7 +46,7 @@ These TAP interfaces can be used with Wireshark or tcpdump or Pktgen-DPDK along with being able to be used as a network connection to the DPDK application. The method enable one or more interfaces is to use the ``--vdev=net_tap0`` option on the DPDK application command line. Each -``--vdev=net_tap1`` option give will create an interface named dtap0, dtap1, +``--vdev=net_tap1`` option given will create an interface named dtap0, dtap1, and so on. The interface name can be changed by adding the ``iface=foo0``, for example:: @@ -58,6 +58,17 @@ needed, but the interface does not enforce that speed, for example:: --vdev=net_tap0,iface=foo0,speed=25000 +Normally the PMD will generate a random MAC address, but when testing or with +a static configuration the developer may need a fixed MAC address style. +Using the option ``mac=fixed`` you can create a fixed known MAC address:: + + --vdev=net_tap0,mac=fixed + +The MAC address will have a fixed value with the last octet incrementing by one +for each interface string containing ``mac=fixed``. The MAC address is formatted +as 00:'d':'t':'a':'p':[00-FF]. Convert the characters to hex and you get the +actual MAC address: ``00:64:74:61:70:[00-FF]``. + It is possible to specify a remote netdevice to capture packets from by adding ``remote=foo1``, for example:: diff --git a/doc/guides/nics/thunderx.rst b/doc/guides/nics/thunderx.rst index 4fa0039d..45bc690a 100644 --- a/doc/guides/nics/thunderx.rst +++ b/doc/guides/nics/thunderx.rst @@ -1,5 +1,5 @@ .. BSD LICENSE - Copyright (C) Cavium networks Ltd. 2016. + Copyright (C) Cavium, Inc. 2016. All rights reserved. Redistribution and use in source and binary forms, with or without @@ -12,7 +12,7 @@ notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. - * Neither the name of Cavium networks nor the names of its + * Neither the name of Cavium, Inc nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. @@ -35,7 +35,7 @@ The ThunderX NICVF PMD (**librte_pmd_thunderx_nicvf**) provides poll mode driver support for the inbuilt NIC found in the **Cavium ThunderX** SoC family as well as their virtual functions (VF) in SR-IOV context. -More information can be found at `Cavium Networks Official Website +More information can be found at `Cavium, Inc Official Website <http://www.cavium.com/ThunderX_ARM_Processors.html>`_. Features diff --git a/doc/guides/nics/virtio.rst b/doc/guides/nics/virtio.rst index 91bedea6..4d6a8376 100644 --- a/doc/guides/nics/virtio.rst +++ b/doc/guides/nics/virtio.rst @@ -34,6 +34,7 @@ Poll Mode Driver for Emulated Virtio NIC Virtio is a para-virtualization framework initiated by IBM, and supported by KVM hypervisor. In the Data Plane Development Kit (DPDK), we provide a virtio Poll Mode Driver (PMD) as a software solution, comparing to SRIOV hardware solution, + for fast guest VM to guest VM communication and guest VM to host communication. Vhost is a kernel acceleration module for virtio qemu backend. @@ -41,9 +42,6 @@ The DPDK extends kni to support vhost raw socket interface, which enables vhost to directly read/ write packets from/to a physical port. With this enhancement, virtio could achieve quite promising performance. -In future release, we will also make enhancement to vhost backend, -releasing peak performance of virtio PMD driver. - For basic qemu-KVM installation and other Intel EM poll mode driver in guest VM, please refer to Chapter "Driver for VM Emulated Devices". @@ -73,15 +71,20 @@ In this release, the virtio PMD driver provides the basic functionality of packe * It supports multicast packets and promiscuous mode. -* The descriptor number for the Rx/Tx queue is hard-coded to be 256 by qemu. +* The descriptor number for the Rx/Tx queue is hard-coded to be 256 by qemu 2.7 and below. If given a different descriptor number by the upper application, the virtio PMD generates a warning and fall back to the hard-coded value. + Rx queue size can be configureable and up to 1024 since qemu 2.8 and above. Rx queue size is 256 + by default. Tx queue size is still hard-coded to be 256. * Features of mac/vlan filter are supported, negotiation with vhost/backend are needed to support them. When backend can't support vlan filter, virtio app on guest should disable vlan filter to make sure the virtio port is configured correctly. E.g. specify '--disable-hw-vlan' in testpmd command line. -* RTE_PKTMBUF_HEADROOM should be defined larger than sizeof(struct virtio_net_hdr), which is 10 bytes. +* "RTE_PKTMBUF_HEADROOM" should be defined + no less than "sizeof(struct virtio_net_hdr_mrg_rxbuf)", which is 12 bytes when mergeable or + "VIRTIO_F_VERSION_1" is set. + no less than "sizeof(struct virtio_net_hdr)", which is 10 bytes, when using non-mergeable. * Virtio does not support runtime configuration. diff --git a/doc/guides/prog_guide/cryptodev_lib.rst b/doc/guides/prog_guide/cryptodev_lib.rst index 4f98f28c..75ae085f 100644 --- a/doc/guides/prog_guide/cryptodev_lib.rst +++ b/doc/guides/prog_guide/cryptodev_lib.rst @@ -1,5 +1,5 @@ .. BSD LICENSE - Copyright(c) 2016 Intel Corporation. All rights reserved. + Copyright(c) 2016-2017 Intel Corporation. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -68,13 +68,13 @@ From the command line using the --vdev EAL option .. code-block:: console - --vdev 'cryptodev_aesni_mb_pmd0,max_nb_queue_pairs=2,max_nb_sessions=1024,socket_id=0' + --vdev 'crypto_aesni_mb0,max_nb_queue_pairs=2,max_nb_sessions=1024,socket_id=0' Our using the rte_vdev_init API within the application code. .. code-block:: c - rte_vdev_init("cryptodev_aesni_mb_pmd", + rte_vdev_init("crypto_aesni_mb", "max_nb_queue_pairs=2,max_nb_sessions=1024,socket_id=0") All virtual Crypto devices support the following initialization parameters: @@ -114,9 +114,8 @@ The rte_cryptodev_configure API is used to configure a Crypto device. int rte_cryptodev_configure(uint8_t dev_id, struct rte_cryptodev_config *config) -The ``rte_cryptodev_config`` structure is used to pass the configuration parameters. -In contains parameter for socket selection, number of queue pairs and the -session mempool configuration. +The ``rte_cryptodev_config`` structure is used to pass the configuration +parameters for socket selection and number of queue pairs. .. code-block:: c @@ -125,12 +124,6 @@ session mempool configuration. /**< Socket to allocate resources on */ uint16_t nb_queue_pairs; /**< Number of queue pairs to configure on device */ - - struct { - uint32_t nb_objs; - uint32_t cache_size; - } session_mp; - /**< Session mempool configuration */ }; @@ -188,8 +181,9 @@ the device having hardware acceleration or supporting symmetric Crypto operations, The capabilities mechanism defines the individual algorithms/functions which -the device supports, such as a specific symmetric Crypto cipher or -authentication operation. +the device supports, such as a specific symmetric Crypto cipher, +authentication operation or Authenticated Encryption with Associated Data +(AEAD) operation. Device Features @@ -245,7 +239,8 @@ algorithm AES_CBC. .max = 12, .increment = 0 }, - .aad_size = { 0 } + .aad_size = { 0 }, + .iv_size = { 0 } } } }, @@ -291,7 +286,7 @@ relevant information for the device. struct rte_cryptodev_info { const char *driver_name; - enum rte_cryptodev_type dev_type; + uint8_t driver_id; struct rte_pci_device *pci_dev; uint64_t feature_flags; @@ -359,11 +354,11 @@ Crypto operation to be processed on a particular Crypto device poll mode driver. .. figure:: img/crypto_op.* -The operation structure includes the operation type and the operation status, -a reference to the operation specific data, which can vary in size and content -depending on the operation being provisioned. It also contains the source -mempool for the operation, if it allocate from a mempool. Finally an -opaque pointer for user specific data is provided. +The operation structure includes the operation type, the operation status +and the session type (session-based/less), a reference to the operation +specific data, which can vary in size and content depending on the operation +being provisioned. It also contains the source mempool for the operation, +if it allocated from a mempool. If Crypto operations are allocated from a Crypto operation mempool, see next section, there is also the ability to allocate private memory with the @@ -431,7 +426,7 @@ operations, as well as also supporting AEAD operations. Session and Session Management ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Session are used in symmetric cryptographic processing to store the immutable +Sessions are used in symmetric cryptographic processing to store the immutable data defined in a cryptographic transform which is used in the operation processing of a packet flow. Sessions are used to manage information such as expand cipher keys and HMAC IPADs and OPADs, which need to be calculated for a @@ -442,29 +437,33 @@ Crypto workloads. .. figure:: img/cryptodev_sym_sess.* -The Crypto device framework provides a set of session pool management APIs for -the creation and freeing of the sessions, utilizing the Mempool Library. - -The framework also provides hooks so the PMDs can pass the amount of memory -required for that PMDs private session parameters, as well as initialization -functions for the configuration of the session parameters and freeing function -so the PMD can managed the memory on destruction of a session. - -**Note**: Sessions created on a particular device can only be used on Crypto -devices of the same type, and if you try to use a session on a device different -to that on which it was created then the Crypto operation will fail. - -``rte_cryptodev_sym_session_create()`` is used to create a symmetric session on -Crypto device. A symmetric transform chain is used to specify the particular -operation and its parameters. See the section below for details on transforms. - -.. code-block:: c - - struct rte_cryptodev_sym_session * rte_cryptodev_sym_session_create( - uint8_t dev_id, struct rte_crypto_sym_xform *xform); - -**Note**: For AEAD operations the algorithm selected for authentication and -ciphering must aligned, eg AES_GCM. +The Crypto device framework provides APIs to allocate and initizalize sessions +for crypto devices, where sessions are mempool objects. +It is the application's responsibility to create and manage the session mempools. +This approach allows for different scenarios such as having a single session +mempool for all crypto devices (where the mempool object size is big +enough to hold the private session of any crypto device), as well as having +multiple session mempools of different sizes for better memory usage. + +An application can use ``rte_cryptodev_get_private_session_size()`` to +get the private session size of given crypto device. This function would allow +an application to calculate the max device session size of all crypto devices +to create a single session mempool. +If instead an application creates multiple session mempools, the Crypto device +framework also provides ``rte_cryptodev_get_header_session_size`` to get +the size of an uninitialized session. + +Once the session mempools have been created, ``rte_cryptodev_sym_session_create()`` +is used to allocate an uninitialized session from the given mempool. +The session then must be initialized using ``rte_cryptodev_sym_session_init()`` +for each of the required crypto devices. A symmetric transform chain +is used to specify the operation and its parameters. See the section below for +details on transforms. + +When a session is no longer used, user must call ``rte_cryptodev_sym_session_clear()`` +for each of the crypto devices that are using the session, to free all driver +private session data. Once this is done, session should be freed using +``rte_cryptodev_sym_session_free`` which returns them to their mempool. Transforms and Transform Chaining @@ -476,9 +475,8 @@ operations such as cipher encrypt and authentication generate, the next pointer allows transform to be chained together. Crypto devices which support chaining must publish the chaining of symmetric Crypto operations feature flag. -Currently there are two transforms types cipher and authentication, to specify -an AEAD operation it is required to chain a cipher and an authentication -transform together. Also it is important to note that the order in which the +Currently there are three transforms types cipher, authentication and AEAD. +Also it is important to note that the order in which the transforms are passed indicates the order of the chaining. .. code-block:: c @@ -493,6 +491,8 @@ transforms are passed indicates the order of the chaining. /**< Authentication / hash xform */ struct rte_crypto_cipher_xform cipher; /**< Cipher xform */ + struct rte_crypto_aead_xform aead; + /**< AEAD xform */ }; }; @@ -512,9 +512,9 @@ buffer. It is used for either cipher, authentication, AEAD and chained operations. As a minimum the symmetric operation must have a source data buffer (``m_src``), -the session type (session-based/less), a valid session (or transform chain if in -session-less mode) and the minimum authentication/ cipher parameters required -depending on the type of operation specified in the session or the transform +a valid session (or transform chain if in session-less mode) and the minimum +authentication/ cipher/ AEAD parameters required depending on the type of operation +specified in the session or the transform chain. .. code-block:: c @@ -523,8 +523,6 @@ chain. struct rte_mbuf *m_src; struct rte_mbuf *m_dst; - enum rte_crypto_sym_op_sess_type type; - union { struct rte_cryptodev_sym_session *session; /**< Handle for the initialised session context */ @@ -532,39 +530,255 @@ chain. /**< Session-less API Crypto operation parameters */ }; - struct { + union { struct { - uint32_t offset; - uint32_t length; - } data; /**< Data offsets and length for ciphering */ + struct { + uint32_t offset; + uint32_t length; + } data; /**< Data offsets and length for AEAD */ + + struct { + uint8_t *data; + phys_addr_t phys_addr; + } digest; /**< Digest parameters */ + + struct { + uint8_t *data; + phys_addr_t phys_addr; + } aad; + /**< Additional authentication parameters */ + } aead; struct { - uint8_t *data; - phys_addr_t phys_addr; - uint16_t length; - } iv; /**< Initialisation vector parameters */ - } cipher; + struct { + struct { + uint32_t offset; + uint32_t length; + } data; /**< Data offsets and length for ciphering */ + } cipher; + + struct { + struct { + uint32_t offset; + uint32_t length; + } data; + /**< Data offsets and length for authentication */ + + struct { + uint8_t *data; + phys_addr_t phys_addr; + } digest; /**< Digest parameters */ + } auth; + }; + }; + }; - struct { - struct { - uint32_t offset; - uint32_t length; - } data; /**< Data offsets and length for authentication */ +Sample code +----------- - struct { - uint8_t *data; - phys_addr_t phys_addr; - uint16_t length; - } digest; /**< Digest parameters */ +There are various sample applications that show how to use the cryptodev library, +such as the L2fwd with Crypto sample application (L2fwd-crypto) and +the IPSec Security Gateway application (ipsec-secgw). - struct { - uint8_t *data; - phys_addr_t phys_addr; - uint16_t length; - } aad; /**< Additional authentication parameters */ - } auth; +While these applications demonstrate how an application can be created to perform +generic crypto operation, the required complexity hides the basic steps of +how to use the cryptodev APIs. + +The following sample code shows the basic steps to encrypt several buffers +with AES-CBC (although performing other crypto operations is similar), +using one of the crypto PMDs available in DPDK. + +.. code-block:: c + + /* + * Simple example to encrypt several buffers with AES-CBC using + * the Cryptodev APIs. + */ + + #define MAX_SESSIONS 1024 + #define NUM_MBUFS 1024 + #define POOL_CACHE_SIZE 128 + #define BURST_SIZE 32 + #define BUFFER_SIZE 1024 + #define AES_CBC_IV_LENGTH 16 + #define AES_CBC_KEY_LENGTH 16 + #define IV_OFFSET (sizeof(struct rte_crypto_op) + \ + sizeof(struct rte_crypto_sym_op)) + + struct rte_mempool *mbuf_pool, *crypto_op_pool, *session_pool; + unsigned int session_size; + int ret; + + /* Initialize EAL. */ + ret = rte_eal_init(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "Invalid EAL arguments\n"); + + uint8_t socket_id = rte_socket_id(); + + /* Create the mbuf pool. */ + mbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", + NUM_MBUFS, + POOL_CACHE_SIZE, + 0, + RTE_MBUF_DEFAULT_BUF_SIZE, + socket_id); + if (mbuf_pool == NULL) + rte_exit(EXIT_FAILURE, "Cannot create mbuf pool\n"); + + /* + * The IV is always placed after the crypto operation, + * so some private data is required to be reserved. + */ + unsigned int crypto_op_private_data = AES_CBC_IV_LENGTH; + + /* Create crypto operation pool. */ + crypto_op_pool = rte_crypto_op_pool_create("crypto_op_pool", + RTE_CRYPTO_OP_TYPE_SYMMETRIC, + NUM_MBUFS, + POOL_CACHE_SIZE, + crypto_op_private_data, + socket_id); + if (crypto_op_pool == NULL) + rte_exit(EXIT_FAILURE, "Cannot create crypto op pool\n"); + + /* Create the virtual crypto device. */ + char args[128]; + const char *crypto_name = "crypto_aesni_mb0"; + snprintf(args, sizeof(args), "socket_id=%d", socket_id); + ret = rte_vdev_init(crypto_name, args); + if (ret != 0) + rte_exit(EXIT_FAILURE, "Cannot create virtual device"); + + uint8_t cdev_id = rte_cryptodev_get_dev_id(crypto_name); + + /* Get private session data size. */ + session_size = rte_cryptodev_get_private_session_size(cdev_id); + + /* + * Create session mempool, with two objects per session, + * one for the session header and another one for the + * private session data for the crypto device. + */ + session_pool = rte_mempool_create("session_pool", + MAX_SESSIONS * 2, + session_size, + POOL_CACHE_SIZE, + 0, NULL, NULL, NULL, + NULL, socket_id, + 0); + + /* Configure the crypto device. */ + struct rte_cryptodev_config conf = { + .nb_queue_pairs = 1, + .socket_id = socket_id + }; + struct rte_cryptodev_qp_conf qp_conf = { + .nb_descriptors = 2048 + }; + + if (rte_cryptodev_configure(cdev_id, &conf) < 0) + rte_exit(EXIT_FAILURE, "Failed to configure cryptodev %u", cdev_id); + + if (rte_cryptodev_queue_pair_setup(cdev_id, 0, &qp_conf, + socket_id, session_pool) < 0) + rte_exit(EXIT_FAILURE, "Failed to setup queue pair\n"); + + if (rte_cryptodev_start(cdev_id) < 0) + rte_exit(EXIT_FAILURE, "Failed to start device\n"); + + /* Create the crypto transform. */ + uint8_t cipher_key[16] = {0}; + struct rte_crypto_sym_xform cipher_xform = { + .next = NULL, + .type = RTE_CRYPTO_SYM_XFORM_CIPHER, + .cipher = { + .op = RTE_CRYPTO_CIPHER_OP_ENCRYPT, + .algo = RTE_CRYPTO_CIPHER_AES_CBC, + .key = { + .data = cipher_key, + .length = AES_CBC_KEY_LENGTH + }, + .iv = { + .offset = IV_OFFSET, + .length = AES_CBC_IV_LENGTH + } + } + }; + + /* Create crypto session and initialize it for the crypto device. */ + struct rte_cryptodev_sym_session *session; + session = rte_cryptodev_sym_session_create(session_pool); + if (session == NULL) + rte_exit(EXIT_FAILURE, "Session could not be created\n"); + + if (rte_cryptodev_sym_session_init(cdev_id, session, + &cipher_xform, session_pool) < 0) + rte_exit(EXIT_FAILURE, "Session could not be initialized " + "for the crypto device\n"); + + /* Get a burst of crypto operations. */ + struct rte_crypto_op *crypto_ops[BURST_SIZE]; + if (rte_crypto_op_bulk_alloc(crypto_op_pool, + RTE_CRYPTO_OP_TYPE_SYMMETRIC, + crypto_ops, BURST_SIZE) == 0) + rte_exit(EXIT_FAILURE, "Not enough crypto operations available\n"); + + /* Get a burst of mbufs. */ + struct rte_mbuf *mbufs[BURST_SIZE]; + if (rte_pktmbuf_alloc_bulk(mbuf_pool, mbufs, BURST_SIZE) < 0) + rte_exit(EXIT_FAILURE, "Not enough mbufs available"); + + /* Initialize the mbufs and append them to the crypto operations. */ + unsigned int i; + for (i = 0; i < BURST_SIZE; i++) { + if (rte_pktmbuf_append(mbufs[i], BUFFER_SIZE) == NULL) + rte_exit(EXIT_FAILURE, "Not enough room in the mbuf\n"); + crypto_ops[i]->sym->m_src = mbufs[i]; } + /* Set up the crypto operations. */ + for (i = 0; i < BURST_SIZE; i++) { + struct rte_crypto_op *op = crypto_ops[i]; + /* Modify bytes of the IV at the end of the crypto operation */ + uint8_t *iv_ptr = rte_crypto_op_ctod_offset(op, uint8_t *, + IV_OFFSET); + + generate_random_bytes(iv_ptr, AES_CBC_IV_LENGTH); + + op->sym->cipher.data.offset = 0; + op->sym->cipher.data.length = BUFFER_SIZE; + + /* Attach the crypto session to the operation */ + rte_crypto_op_attach_sym_session(op, session); + } + + /* Enqueue the crypto operations in the crypto device. */ + uint16_t num_enqueued_ops = rte_cryptodev_enqueue_burst(cdev_id, 0, + crypto_ops, BURST_SIZE); + + /* + * Dequeue the crypto operations until all the operations + * are proccessed in the crypto device. + */ + uint16_t num_dequeued_ops, total_num_dequeued_ops = 0; + do { + struct rte_crypto_op *dequeued_ops[BURST_SIZE]; + num_dequeued_ops = rte_cryptodev_dequeue_burst(cdev_id, 0, + dequeued_ops, BURST_SIZE); + total_num_dequeued_ops += num_dequeued_ops; + + /* Check if operation was processed successfully */ + for (i = 0; i < num_dequeued_ops; i++) { + if (dequeued_ops[i]->status != RTE_CRYPTO_OP_STATUS_SUCCESS) + rte_exit(EXIT_FAILURE, + "Some operations were not processed correctly"); + } + + rte_mempool_put_bulk(crypto_op_pool, (void **)dequeued_ops, + num_dequeued_ops); + } while (total_num_dequeued_ops < num_enqueued_ops); + Asymmetric Cryptography ----------------------- diff --git a/doc/guides/prog_guide/eventdev.rst b/doc/guides/prog_guide/eventdev.rst new file mode 100644 index 00000000..908d123a --- /dev/null +++ b/doc/guides/prog_guide/eventdev.rst @@ -0,0 +1,394 @@ +.. BSD LICENSE + Copyright(c) 2017 Intel Corporation. All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of Intel Corporation nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +Event Device Library +==================== + +The DPDK Event device library is an abstraction that provides the application +with features to schedule events. This is achieved using the PMD architecture +similar to the ethdev or cryptodev APIs, which may already be familiar to the +reader. + +The eventdev framework introduces the event driven programming model. In a +polling model, lcores poll ethdev ports and associated Rx queues directly +to look for a packet. By contrast in an event driven model, lcores call the +scheduler that selects packets for them based on programmer-specified criteria. +The Eventdev library adds support for an event driven programming model, which +offers applications automatic multicore scaling, dynamic load balancing, +pipelining, packet ingress order maintenance and synchronization services to +simplify application packet processing. + +By introducing an event driven programming model, DPDK can support both polling +and event driven programming models for packet processing, and applications are +free to choose whatever model (or combination of the two) best suits their +needs. + +Step-by-step instructions of the eventdev design is available in the `API +Walk-through`_ section later in this document. + +Event struct +------------ + +The eventdev API represents each event with a generic struct, which contains a +payload and metadata required for scheduling by an eventdev. The +``rte_event`` struct is a 16 byte C structure, defined in +``libs/librte_eventdev/rte_eventdev.h``. + +Event Metadata +~~~~~~~~~~~~~~ + +The rte_event structure contains the following metadata fields, which the +application fills in to have the event scheduled as required: + +* ``flow_id`` - The targeted flow identifier for the enq/deq operation. +* ``event_type`` - The source of this event, eg RTE_EVENT_TYPE_ETHDEV or CPU. +* ``sub_event_type`` - Distinguishes events inside the application, that have + the same event_type (see above) +* ``op`` - This field takes one of the RTE_EVENT_OP_* values, and tells the + eventdev about the status of the event - valid values are NEW, FORWARD or + RELEASE. +* ``sched_type`` - Represents the type of scheduling that should be performed + on this event, valid values are the RTE_SCHED_TYPE_ORDERED, ATOMIC and + PARALLEL. +* ``queue_id`` - The identifier for the event queue that the event is sent to. +* ``priority`` - The priority of this event, see RTE_EVENT_DEV_PRIORITY. + +Event Payload +~~~~~~~~~~~~~ + +The rte_event struct contains a union for payload, allowing flexibility in what +the actual event being scheduled is. The payload is a union of the following: + +* ``uint64_t u64`` +* ``void *event_ptr`` +* ``struct rte_mbuf *mbuf`` + +These three items in a union occupy the same 64 bits at the end of the rte_event +structure. The application can utilize the 64 bits directly by accessing the +u64 variable, while the event_ptr and mbuf are provided as convenience +variables. For example the mbuf pointer in the union can used to schedule a +DPDK packet. + +Queues +~~~~~~ + +An event queue is a queue containing events that are scheduled by the event +device. An event queue contains events of different flows associated with +scheduling types, such as atomic, ordered, or parallel. + +Queue All Types Capable +^^^^^^^^^^^^^^^^^^^^^^^ + +If RTE_EVENT_DEV_CAP_QUEUE_ALL_TYPES capability bit is set in the event device, +then events of any type may be sent to any queue. Otherwise, the queues only +support events of the type that it was created with. + +Queue All Types Incapable +^^^^^^^^^^^^^^^^^^^^^^^^^ + +In this case, each stage has a specified scheduling type. The application +configures each queue for a specific type of scheduling, and just enqueues all +events to the eventdev. An example of a PMD of this type is the eventdev +software PMD. + +The Eventdev API supports the following scheduling types per queue: + +* Atomic +* Ordered +* Parallel + +Atomic, Ordered and Parallel are load-balanced scheduling types: the output +of the queue can be spread out over multiple CPU cores. + +Atomic scheduling on a queue ensures that a single flow is not present on two +different CPU cores at the same time. Ordered allows sending all flows to any +core, but the scheduler must ensure that on egress the packets are returned to +ingress order on downstream queue enqueue. Parallel allows sending all flows +to all CPU cores, without any re-ordering guarantees. + +Single Link Flag +^^^^^^^^^^^^^^^^ + +There is a SINGLE_LINK flag which allows an application to indicate that only +one port will be connected to a queue. Queues configured with the single-link +flag follow a FIFO like structure, maintaining ordering but it is only capable +of being linked to a single port (see below for port and queue linking details). + + +Ports +~~~~~ + +Ports are the points of contact between worker cores and the eventdev. The +general use-case will see one CPU core using one port to enqueue and dequeue +events from an eventdev. Ports are linked to queues in order to retrieve events +from those queues (more details in `Linking Queues and Ports`_ below). + + +API Walk-through +---------------- + +This section will introduce the reader to the eventdev API, showing how to +create and configure an eventdev and use it for a two-stage atomic pipeline +with a single core for TX. The diagram below shows the final state of the +application after this walk-through: + +.. _figure_eventdev-usage1: + +.. figure:: img/eventdev_usage.* + + Sample eventdev usage, with RX, two atomic stages and a single-link to TX. + + +A high level overview of the setup steps are: + +* rte_event_dev_configure() +* rte_event_queue_setup() +* rte_event_port_setup() +* rte_event_port_link() +* rte_event_dev_start() + + +Init and Config +~~~~~~~~~~~~~~~ + +The eventdev library uses vdev options to add devices to the DPDK application. +The ``--vdev`` EAL option allows adding eventdev instances to your DPDK +application, using the name of the eventdev PMD as an argument. + +For example, to create an instance of the software eventdev scheduler, the +following vdev arguments should be provided to the application EAL command line: + +.. code-block:: console + + ./dpdk_application --vdev="event_sw0" + +In the following code, we configure eventdev instance with 3 queues +and 6 ports as follows. The 3 queues consist of 2 Atomic and 1 Single-Link, +while the 6 ports consist of 4 workers, 1 RX and 1 TX. + +.. code-block:: c + + const struct rte_event_dev_config config = { + .nb_event_queues = 3, + .nb_event_ports = 6, + .nb_events_limit = 4096, + .nb_event_queue_flows = 1024, + .nb_event_port_dequeue_depth = 128, + .nb_event_port_enqueue_depth = 128, + }; + int err = rte_event_dev_configure(dev_id, &config); + +The remainder of this walk-through assumes that dev_id is 0. + +Setting up Queues +~~~~~~~~~~~~~~~~~ + +Once the eventdev itself is configured, the next step is to configure queues. +This is done by setting the appropriate values in a queue_conf structure, and +calling the setup function. Repeat this step for each queue, starting from +0 and ending at ``nb_event_queues - 1`` from the event_dev config above. + +.. code-block:: c + + struct rte_event_queue_conf atomic_conf = { + .event_queue_cfg = RTE_EVENT_QUEUE_CFG_ATOMIC_ONLY, + .priority = RTE_EVENT_DEV_PRIORITY_NORMAL, + .nb_atomic_flows = 1024, + .nb_atomic_order_sequences = 1024, + }; + int dev_id = 0; + int queue_id = 0; + int err = rte_event_queue_setup(dev_id, queue_id, &atomic_conf); + +The remainder of this walk-through assumes that the queues are configured as +follows: + + * id 0, atomic queue #1 + * id 1, atomic queue #2 + * id 2, single-link queue + +Setting up Ports +~~~~~~~~~~~~~~~~ + +Once queues are set up successfully, create the ports as required. Each port +should be set up with its corresponding port_conf type, worker for worker cores, +rx and tx for the RX and TX cores: + +.. code-block:: c + + struct rte_event_port_conf rx_conf = { + .dequeue_depth = 128, + .enqueue_depth = 128, + .new_event_threshold = 1024, + }; + struct rte_event_port_conf worker_conf = { + .dequeue_depth = 16, + .enqueue_depth = 64, + .new_event_threshold = 4096, + }; + struct rte_event_port_conf tx_conf = { + .dequeue_depth = 128, + .enqueue_depth = 128, + .new_event_threshold = 4096, + }; + int dev_id = 0; + int port_id = 0; + int err = rte_event_port_setup(dev_id, port_id, &CORE_FUNCTION_conf); + +It is now assumed that: + + * port 0: RX core + * ports 1,2,3,4: Workers + * port 5: TX core + +Linking Queues and Ports +~~~~~~~~~~~~~~~~~~~~~~~~ + +The final step is to "wire up" the ports to the queues. After this, the +eventdev is capable of scheduling events, and when cores request work to do, +the correct events are provided to that core. Note that the RX core takes input +from eg: a NIC so it is not linked to any eventdev queues. + +Linking all workers to atomic queues, and the TX core to the single-link queue +can be achieved like this: + +.. code-block:: c + + uint8_t port_id = 0; + uint8_t atomic_qs[] = {0, 1}; + uint8_t single_link_q = 2; + uint8_t tx_port_id = 5; + uin8t_t priority = RTE_EVENT_DEV_PRIORITY_NORMAL; + + for(int i = 0; i < 4; i++) { + int worker_port = i + 1; + int links_made = rte_event_port_link(dev_id, worker_port, atomic_qs, NULL, 2); + } + int links_made = rte_event_port_link(dev_id, tx_port_id, &single_link_q, &priority, 1); + +Starting the EventDev +~~~~~~~~~~~~~~~~~~~~~ + +A single function call tells the eventdev instance to start processing +events. Note that all queues must be linked to for the instance to start, as +if any queue is not linked to, enqueuing to that queue will cause the +application to backpressure and eventually stall due to no space in the +eventdev. + +.. code-block:: c + + int err = rte_event_dev_start(dev_id); + +Ingress of New Events +~~~~~~~~~~~~~~~~~~~~~ + +Now that the eventdev is set up, and ready to receive events, the RX core must +enqueue some events into the system for it to schedule. The events to be +scheduled are ordinary DPDK packets, received from an eth_rx_burst() as normal. +The following code shows how those packets can be enqueued into the eventdev: + +.. code-block:: c + + const uint16_t nb_rx = rte_eth_rx_burst(eth_port, 0, mbufs, BATCH_SIZE); + + for (i = 0; i < nb_rx; i++) { + ev[i].flow_id = mbufs[i]->hash.rss; + ev[i].op = RTE_EVENT_OP_NEW; + ev[i].sched_type = RTE_EVENT_QUEUE_CFG_ATOMIC_ONLY; + ev[i].queue_id = 0; + ev[i].event_type = RTE_EVENT_TYPE_ETHDEV; + ev[i].sub_event_type = 0; + ev[i].priority = RTE_EVENT_DEV_PRIORITY_NORMAL; + ev[i].mbuf = mbufs[i]; + } + + const int nb_tx = rte_event_enqueue_burst(dev_id, port_id, ev, nb_rx); + if (nb_tx != nb_rx) { + for(i = nb_tx; i < nb_rx; i++) + rte_pktmbuf_free(mbufs[i]); + } + +Forwarding of Events +~~~~~~~~~~~~~~~~~~~~ + +Now that the RX core has injected events, there is work to be done by the +workers. Note that each worker will dequeue as many events as it can in a burst, +process each one individually, and then burst the packets back into the +eventdev. + +The worker can lookup the events source from ``event.queue_id``, which should +indicate to the worker what workload needs to be performed on the event. +Once done, the worker can update the ``event.queue_id`` to a new value, to send +the event to the next stage in the pipeline. + +.. code-block:: c + + int timeout = 0; + struct rte_event events[BATCH_SIZE]; + uint16_t nb_rx = rte_event_dequeue_burst(dev_id, worker_port_id, events, BATCH_SIZE, timeout); + + for (i = 0; i < nb_rx; i++) { + /* process mbuf using events[i].queue_id as pipeline stage */ + struct rte_mbuf *mbuf = events[i].mbuf; + /* Send event to next stage in pipeline */ + events[i].queue_id++; + } + + uint16_t nb_tx = rte_event_enqueue_burst(dev_id, port_id, events, nb_rx); + + +Egress of Events +~~~~~~~~~~~~~~~~ + +Finally, when the packet is ready for egress or needs to be dropped, we need +to inform the eventdev that the packet is no longer being handled by the +application. This can be done by calling dequeue() or dequeue_burst(), which +indicates that the previous burst of packets is no longer in use by the +application. + +An event driven worker thread has following typical workflow on fastpath: + +.. code-block:: c + + while (1) { + rte_event_dequeue_burst(...); + (event processing) + rte_event_enqueue_burst(...); + } + + +Summary +------- + +The eventdev library allows an application to easily schedule events as it +requires, either using a run-to-completion or pipeline processing model. The +queues and ports abstract the logical functionality of an eventdev, providing +the application with a generic method to schedule events. With the flexible +PMD infrastructure applications benefit of improvements in existing eventdevs +and additions of new ones without modification. diff --git a/doc/guides/prog_guide/generic_receive_offload_lib.rst b/doc/guides/prog_guide/generic_receive_offload_lib.rst new file mode 100644 index 00000000..22e50ecb --- /dev/null +++ b/doc/guides/prog_guide/generic_receive_offload_lib.rst @@ -0,0 +1,159 @@ +.. BSD LICENSE + Copyright(c) 2017 Intel Corporation. All rights reserved. + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of Intel Corporation nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +Generic Receive Offload Library +=============================== + +Generic Receive Offload (GRO) is a widely used SW-based offloading +technique to reduce per-packet processing overhead. It gains performance +by reassembling small packets into large ones. To enable more flexibility +to applications, DPDK implements GRO as a standalone library. Applications +explicitly use the GRO library to merge small packets into large ones. + +The GRO library assumes all input packets have correct checksums. In +addition, the GRO library doesn't re-calculate checksums for merged +packets. If input packets are IP fragmented, the GRO library assumes +they are complete packets (i.e. with L4 headers). + +Currently, the GRO library implements TCP/IPv4 packet reassembly. + +Reassembly Modes +---------------- + +The GRO library provides two reassembly modes: lightweight and +heavyweight mode. If applications want to merge packets in a simple way, +they can use the lightweight mode API. If applications want more +fine-grained controls, they can choose the heavyweight mode API. + +Lightweight Mode +~~~~~~~~~~~~~~~~ + +The ``rte_gro_reassemble_burst()`` function is used for reassembly in +lightweight mode. It tries to merge N input packets at a time, where +N should be less than or equal to ``RTE_GRO_MAX_BURST_ITEM_NUM``. + +In each invocation, ``rte_gro_reassemble_burst()`` allocates temporary +reassembly tables for the desired GRO types. Note that the reassembly +table is a table structure used to reassemble packets and different GRO +types (e.g. TCP/IPv4 GRO and TCP/IPv6 GRO) have different reassembly table +structures. The ``rte_gro_reassemble_burst()`` function uses the reassembly +tables to merge the N input packets. + +For applications, performing GRO in lightweight mode is simple. They +just need to invoke ``rte_gro_reassemble_burst()``. Applications can get +GROed packets as soon as ``rte_gro_reassemble_burst()`` returns. + +Heavyweight Mode +~~~~~~~~~~~~~~~~ + +The ``rte_gro_reassemble()`` function is used for reassembly in heavyweight +mode. Compared with the lightweight mode, performing GRO in heavyweight mode +is relatively complicated. + +Before performing GRO, applications need to create a GRO context object +by calling ``rte_gro_ctx_create()``. A GRO context object holds the +reassembly tables of desired GRO types. Note that all update/lookup +operations on the context object are not thread safe. So if different +processes or threads want to access the same context object simultaneously, +some external syncing mechanisms must be used. + +Once the GRO context is created, applications can then use the +``rte_gro_reassemble()`` function to merge packets. In each invocation, +``rte_gro_reassemble()`` tries to merge input packets with the packets +in the reassembly tables. If an input packet is an unsupported GRO type, +or other errors happen (e.g. SYN bit is set), ``rte_gro_reassemble()`` +returns the packet to applications. Otherwise, the input packet is either +merged or inserted into a reassembly table. + +When applications want to get GRO processed packets, they need to use +``rte_gro_timeout_flush()`` to flush them from the tables manually. + +TCP/IPv4 GRO +------------ + +TCP/IPv4 GRO supports merging small TCP/IPv4 packets into large ones, +using a table structure called the TCP/IPv4 reassembly table. + +TCP/IPv4 Reassembly Table +~~~~~~~~~~~~~~~~~~~~~~~~~ + +A TCP/IPv4 reassembly table includes a "key" array and an "item" array. +The key array keeps the criteria to merge packets and the item array +keeps the packet information. + +Each key in the key array points to an item group, which consists of +packets which have the same criteria values but can't be merged. A key +in the key array includes two parts: + +* ``criteria``: the criteria to merge packets. If two packets can be + merged, they must have the same criteria values. + +* ``start_index``: the item array index of the first packet in the item + group. + +Each element in the item array keeps the information of a packet. An item +in the item array mainly includes three parts: + +* ``firstseg``: the mbuf address of the first segment of the packet. + +* ``lastseg``: the mbuf address of the last segment of the packet. + +* ``next_pkt_index``: the item array index of the next packet in the same + item group. TCP/IPv4 GRO uses ``next_pkt_index`` to chain the packets + that have the same criteria value but can't be merged together. + +Procedure to Reassemble a Packet +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To reassemble an incoming packet needs three steps: + +#. Check if the packet should be processed. Packets with one of the + following properties aren't processed and are returned immediately: + + * FIN, SYN, RST, URG, PSH, ECE or CWR bit is set. + + * L4 payload length is 0. + +#. Traverse the key array to find a key which has the same criteria + value with the incoming packet. If found, go to the next step. + Otherwise, insert a new key and a new item for the packet. + +#. Locate the first packet in the item group via ``start_index``. Then + traverse all packets in the item group via ``next_pkt_index``. If a + packet is found which can be merged with the incoming one, merge them + together. If one isn't found, insert the packet into this item group. + Note that to merge two packets is to link them together via mbuf's + ``next`` field. + +When packets are flushed from the reassembly table, TCP/IPv4 GRO updates +packet header fields for the merged packets. Note that before reassembling +the packet, TCP/IPv4 GRO doesn't check if the checksums of packets are +correct. Also, TCP/IPv4 GRO doesn't re-calculate checksums for merged +packets. diff --git a/doc/guides/prog_guide/img/crypto_xform_chain.svg b/doc/guides/prog_guide/img/crypto_xform_chain.svg index 4670a07e..13681631 100644 --- a/doc/guides/prog_guide/img/crypto_xform_chain.svg +++ b/doc/guides/prog_guide/img/crypto_xform_chain.svg @@ -69,7 +69,9 @@ class="st3">auth</tspan><tspan class="st3">_</tspan><tspan class="st3">xform </tspan><tspan x="16.02" dy="1.425em" class="st4">struct </tspan><tspan class="st3">rte</tspan><tspan class="st3">_</tspan><tspan class="st3">crypto</tspan><tspan class="st3">_</tspan><tspan class="st3">cipher</tspan><tspan class="st3">_</tspan><tspan - class="st3">xform</tspan></text> </g> + class="st3">xform</tspan><tspan x="18.76" dy="1.425em" class="st4">struct </tspan><tspan + class="st3">rte</tspan><tspan class="st3">_</tspan><tspan class="st3">crypto</tspan><tspan class="st3">_</tspan><tspan + class="st3">aead</tspan><tspan class="st3">_</tspan><tspan class="st3">xform</tspan></text> </g> <g id="shape11-38" transform="translate(10.6711,-238.133)"> <title>Rounded Rectangle.26</title> <desc>next transform (struct rte_crypto_sym_xform *)</desc> @@ -116,7 +118,9 @@ class="st3">auth</tspan><tspan class="st3">_</tspan><tspan class="st3">xform </tspan><tspan x="16.02" dy="1.425em" class="st4">struct </tspan><tspan class="st3">rte</tspan><tspan class="st3">_</tspan><tspan class="st3">crypto</tspan><tspan class="st3">_</tspan><tspan class="st3">cipher</tspan><tspan class="st3">_</tspan><tspan - class="st3">xform</tspan></text> </g> + class="st3">xform</tspan><tspan x="18.76" dy="1.425em" class="st4">struct </tspan><tspan + class="st3">rte</tspan><tspan class="st3">_</tspan><tspan class="st3">crypto</tspan><tspan class="st3">_</tspan><tspan + class="st3">aead</tspan><tspan class="st3">_</tspan><tspan class="st3">xform</tspan></text> </g> <g id="shape15-102" transform="translate(209.592,-163.865)"> <title>Rounded Rectangle.32</title> <desc>next transform (struct rte_crypto_sym_xform *)</desc> diff --git a/doc/guides/prog_guide/img/cryptodev_sym_sess.svg b/doc/guides/prog_guide/img/cryptodev_sym_sess.svg index e5f41ec5..a807ceba 100644 --- a/doc/guides/prog_guide/img/cryptodev_sym_sess.svg +++ b/doc/guides/prog_guide/img/cryptodev_sym_sess.svg @@ -1,10 +1,50 @@ <?xml version="1.0" encoding="UTF-8" standalone="no"?> -<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"> <!-- Generated by Microsoft Visio, SVG Export cryptodev_sym_sess.svg Page-1 --> -<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events" - width="2.17241in" height="2.8102in" viewBox="0 0 156.413 202.335" xml:space="preserve" color-interpolation-filters="sRGB" - class="st10"> - <style type="text/css"> + +<svg + xmlns:dc="http://purl.org/dc/elements/1.1/" + xmlns:cc="http://creativecommons.org/ns#" + xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" + xmlns:svg="http://www.w3.org/2000/svg" + xmlns="http://www.w3.org/2000/svg" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" + xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" + width="4.8933434in" + height="3.8972795in" + viewBox="0 0 352.31955 280.60496" + xml:space="preserve" + class="st10" + version="1.1" + id="svg70" + sodipodi:docname="cryptodev_sym_sess.svg" + style="font-size:12px;overflow:visible;color-interpolation-filters:sRGB;fill:none;fill-rule:evenodd;stroke-linecap:square;stroke-miterlimit:3" + inkscape:version="0.92.1 r15371"><metadata + id="metadata74"><rdf:RDF><cc:Work + rdf:about=""><dc:format>image/svg+xml</dc:format><dc:type + rdf:resource="http://purl.org/dc/dcmitype/StillImage" /><dc:title /></cc:Work></rdf:RDF></metadata><sodipodi:namedview + pagecolor="#ffffff" + bordercolor="#666666" + borderopacity="1" + objecttolerance="10" + gridtolerance="10" + guidetolerance="10" + inkscape:pageopacity="0" + inkscape:pageshadow="2" + inkscape:window-width="1920" + inkscape:window-height="1051" + id="namedview72" + showgrid="false" + inkscape:zoom="1.7495789" + inkscape:cx="208.74719" + inkscape:cy="216.52777" + inkscape:window-x="-9" + inkscape:window-y="-9" + inkscape:window-maximized="0" + inkscape:current-layer="g68-0" /> + <style + type="text/css" + id="style2"> <![CDATA[ .st1 {fill:url(#grad0-4);stroke:#386288;stroke-width:0.75} .st2 {fill:#386288;font-family:Calibri;font-size:0.833336em} @@ -19,48 +59,334 @@ ]]> </style> - <defs id="Patterns_And_Gradients"> - <linearGradient id="grad0-4" x1="0" y1="0" x2="1" y2="0" gradientTransform="rotate(60 0.5 0.5)"> - <stop offset="0" stop-color="#e8ebef" stop-opacity="1"/> - <stop offset="0.24" stop-color="#f4f5f7" stop-opacity="1"/> - <stop offset="0.54" stop-color="#feffff" stop-opacity="1"/> + <defs + id="Patterns_And_Gradients"><marker + inkscape:isstock="true" + style="overflow:visible" + id="marker5421" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="Arrow2Lend"><path + transform="matrix(-1.1,0,0,-1.1,-1.1,0)" + d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" + style="fill:#41719c;fill-opacity:1;fill-rule:evenodd;stroke:#41719c;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" + id="path5419" + inkscape:connector-curvature="0" /></marker><marker + inkscape:stockid="Arrow2Lend" + orient="auto" + refY="0" + refX="0" + id="Arrow2Lend" + style="overflow:visible" + inkscape:isstock="true"><path + id="path5004" + style="fill:#41719c;fill-opacity:1;fill-rule:evenodd;stroke:#41719c;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" + d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" + transform="matrix(-1.1,0,0,-1.1,-1.1,0)" + inkscape:connector-curvature="0" /></marker><marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Lend" + style="overflow:visible" + inkscape:isstock="true"><path + id="path4986" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /></marker> + <linearGradient + id="grad0-4" + x1="0" + y1="0" + x2="1" + y2="0" + gradientTransform="rotate(60,0.5,0.5)"> + <stop + offset="0" + stop-color="#e8ebef" + stop-opacity="1" + id="stop4" /> + <stop + offset="0.24" + stop-color="#f4f5f7" + stop-opacity="1" + id="stop6" /> + <stop + offset="0.54" + stop-color="#feffff" + stop-opacity="1" + id="stop8" /> </linearGradient> - </defs> - <defs id="Filters"> - <filter id="filter_2"> - <feGaussianBlur stdDeviation="2"/> + <filter + id="filter_2-4"><feGaussianBlur + stdDeviation="2" + id="feGaussianBlur12-0" /></filter><linearGradient + inkscape:collect="always" + xlink:href="#grad0-4" + id="linearGradient189" + gradientTransform="scale(0.8787489,1.1379815)" + x1="-0.42674366" + y1="0.98859203" + x2="176.71146" + y2="0.98859203" + gradientUnits="userSpaceOnUse" /><filter + id="filter_2-5"><feGaussianBlur + stdDeviation="2" + id="feGaussianBlur12-8" /></filter><filter + id="filter_2-3"><feGaussianBlur + stdDeviation="2" + id="feGaussianBlur12-2" /></filter><linearGradient + inkscape:collect="always" + xlink:href="#grad0-4" + id="linearGradient189-7" + gradientTransform="scale(0.8787489,1.1379815)" + x1="-0.42674366" + y1="0.98859203" + x2="176.71146" + y2="0.98859203" + gradientUnits="userSpaceOnUse" /><linearGradient + inkscape:collect="always" + xlink:href="#grad0-4" + id="linearGradient500" + gradientTransform="matrix(0.8787489,0,0,1.1379815,12.431599,21.739241)" + x1="-0.42674366" + y1="0.98859203" + x2="176.71146" + y2="0.98859203" + gradientUnits="userSpaceOnUse" /></defs> + <defs + id="Filters"> + <filter + id="filter_2"> + <feGaussianBlur + stdDeviation="2" + id="feGaussianBlur12" /> </filter> </defs> - <g> - <title>Page-1</title> - <g id="shape18-1" transform="translate(0.749889,-0.75)"> - <title>Rounded Rectangle.12</title> - <desc>Crypto Symmetric Session</desc> - <path d="M6.78 202.33 L148.14 202.33 A6.77735 6.77735 -180 0 0 154.91 195.56 L154.91 8.28 A6.77735 6.77735 -180 0 0 148.14 - 1.5 L6.78 1.5 A6.77735 6.77735 -180 0 0 -0 8.28 L0 195.56 A6.77735 6.77735 -180 0 0 6.78 202.33 Z" - class="st1"/> - <text x="24.76" y="14.5" class="st2">Crypto Symmetric Session</text> </g> - <g id="shape19-6" transform="translate(10.6711,-9.82087)"> - <title>Rounded Rectangle.13</title> - <desc>Private Session Data</desc> - <g id="shadow19-7" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st3"> - <path d="M5.91 202.33 L129.16 202.33 A5.90925 5.90925 -180 0 0 135.07 196.43 L135.07 103.65 A5.90925 5.90925 -180 - 0 0 129.16 97.74 L5.91 97.74 A5.90925 5.90925 -180 0 0 -0 103.65 L0 196.43 A5.90925 5.90925 -180 0 0 - 5.91 202.33 Z" class="st4"/> - </g> - <path d="M5.91 202.33 L129.16 202.33 A5.90925 5.90925 -180 0 0 135.07 196.43 L135.07 103.65 A5.90925 5.90925 -180 0 0 - 129.16 97.74 L5.91 97.74 A5.90925 5.90925 -180 0 0 -0 103.65 L0 196.43 A5.90925 5.90925 -180 0 0 5.91 202.33 - Z" class="st5"/> - <text x="26.34" y="153.04" class="st6">Private Session Data</text> </g> - <g id="shape20-12" transform="translate(10.6711,-122.923)"> - <title>Rounded Rectangle.15</title> - <desc>General Session Data (struct rte_cryptodev_sym_session)</desc> - <path d="M5.91 202.33 L129.16 202.33 A5.90925 5.90925 -180 0 0 135.07 196.43 L135.07 160.06 A5.90925 5.90925 -180 0 0 - 129.16 154.15 L5.91 154.15 A5.90925 5.90925 -180 0 0 -0 160.06 L0 196.43 A5.90925 5.90925 -180 0 0 5.91 - 202.33 Z" class="st7"/> - <text x="24.58" y="175.24" class="st6">General Session Data <tspan x="9.16" dy="1.5em" class="st8">(</tspan><tspan - class="st9">struct </tspan><tspan class="st8">rte</tspan><tspan class="st8">_</tspan><tspan class="st8">cryptodev</tspan><tspan - class="st8">_</tspan><tspan class="st8">sym</tspan><tspan class="st8">_</tspan><tspan class="st8">session</tspan><tspan - class="st8">)</tspan></text> </g> + <g + id="g68" + transform="matrix(1,0,0,0.41409874,-12.807629,-5.4621159)"> + <title + id="title16">Page-1</title> + <g + id="shape18-1" + transform="translate(0.749889,-0.75)"> + <title + id="title18">Rounded Rectangle.12</title> + <desc + id="desc20">Crypto Symmetric Session</desc> + <path + d="M 19.211599,224.06924 H 160.5716 a 6.77735,6.77735 0 0 0 6.77,-6.77 V 30.019241 a 6.77735,6.77735 0 0 0 -6.77,-6.78 H 19.211599 a 6.77735,6.77735 0 0 0 -6.78,6.78 V 217.29924 a 6.77735,6.77735 0 0 0 6.78,6.77 z" + class="st1" + id="path22" + style="fill:url(#linearGradient500);stroke:#386288;stroke-width:0.75" + inkscape:connector-curvature="0" /> + <text + x="63.123039" + y="28.531481" + class="st2" + id="text24" + style="font-size:16.97244835px;font-family:Calibri;fill:#386288;stroke-width:1.69723928" + transform="scale(0.58919214,1.6972392)">Crypto Symmetric Session</text> + + </g> + <g + id="shape19-6" + transform="translate(10.6711,-9.82087)"> + <title + id="title27">Rounded Rectangle.13</title> + <desc + id="desc29">Private Session Data</desc> + </g> + <g + id="shape20-12" + transform="matrix(1,0,0,2.5278193,23.531375,-309.78186)"> + <title + id="title39">Rounded Rectangle.15</title> + <desc + id="desc41">void *sess_private_data[]</desc> + <path + d="m 5.91,202.33 h 123.25 a 5.90925,5.90925 -180 0 0 5.91,-5.9 v -36.37 a 5.90925,5.90925 -180 0 0 -5.91,-5.91 H 5.91 A 5.90925,5.90925 -180 0 0 0,160.06 v 36.37 a 5.90925,5.90925 -180 0 0 5.91,5.9 z" + class="st7" + id="path43" + inkscape:connector-curvature="0" + style="fill:#ffffff;stroke:#41719c;stroke-width:0.75" /> + <text + x="14.072042" + y="159.1931" + class="st6" + id="text65" + style="font-size:11.41061592px;font-family:Calibri;fill:#41719c;stroke-width:1.14105785" + transform="scale(0.92359087,1.0827305)">void *sess_private_data[] <tspan + x="-3.5230706" + class="st9" + id="tspan47" + style="font-weight:bold;font-size:9.12843513px;stroke-width:1.14105785" /></text> + + <rect + style="fill:none;fill-opacity:1;stroke:#41719c;stroke-width:0.73305672;stroke-opacity:1" + id="rect4604" + width="15.968175" + height="14.230948" + x="13.494645" + y="181.68814" /><rect + style="font-size:12px;overflow:visible;color-interpolation-filters:sRGB;fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#41719c;stroke-width:0.73305672;stroke-linecap:square;stroke-miterlimit:3;stroke-opacity:1" + id="rect4604-7" + width="15.968174" + height="14.230948" + x="29.46282" + y="181.68814" /><rect + style="font-size:12px;overflow:visible;color-interpolation-filters:sRGB;fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#41719c;stroke-width:0.73305672;stroke-linecap:square;stroke-miterlimit:3;stroke-opacity:1" + id="rect4604-7-6" + width="15.968174" + height="14.230948" + x="45.430992" + y="181.68814" /><rect + style="font-size:12px;overflow:visible;color-interpolation-filters:sRGB;fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#41719c;stroke-width:0.73305672;stroke-linecap:square;stroke-miterlimit:3;stroke-opacity:1" + id="rect4604-7-6-9" + width="15.968174" + height="14.230948" + x="61.399166" + y="181.68814" /><rect + style="font-size:12px;overflow:visible;color-interpolation-filters:sRGB;fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#41719c;stroke-width:0.73305672;stroke-linecap:square;stroke-miterlimit:3;stroke-opacity:1" + id="rect4604-7-6-9-8" + width="15.968174" + height="14.230948" + x="77.36734" + y="181.68814" /><rect + style="font-size:12px;overflow:visible;color-interpolation-filters:sRGB;fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#41719c;stroke-width:0.73305672;stroke-linecap:square;stroke-miterlimit:3;stroke-opacity:1" + id="rect4604-7-6-9-8-9" + width="15.968174" + height="14.230948" + x="93.33551" + y="181.68814" /><rect + style="font-size:12px;overflow:visible;color-interpolation-filters:sRGB;fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#41719c;stroke-width:0.73305672;stroke-linecap:square;stroke-miterlimit:3;stroke-opacity:1" + id="rect4604-7-6-9-8-9-6" + width="15.968174" + height="14.230948" + x="109.30369" + y="181.68814" /><path + style="fill:none;fill-opacity:1;stroke:#41719c;stroke-width:0.72427988px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow2Lend)" + d="m 117.64885,196.01764 0.22164,18.77485 44.6966,-0.0725 -0.22163,-20.00716 16.84434,0.43494" + id="path5030" + inkscape:connector-curvature="0" /></g> </g> -</svg> +<g + transform="translate(190.70887,-0.53319281)" + id="g68-0"><title + id="title16-2">Page-1</title><g + id="shape18-1-4" + transform="matrix(1,0,0,0.57815109,0.749889,-0.11722686)"><title + id="title18-4">Rounded Rectangle.12</title><desc + id="desc20-6">Crypto Symmetric Session</desc><path + inkscape:connector-curvature="0" + d="m 6.78,202.33 h 141.36 a 6.77735,6.77735 -180 0 0 6.77,-6.77 V 8.28 A 6.77735,6.77735 -180 0 0 148.14,1.5 H 6.78 A 6.77735,6.77735 -180 0 0 0,8.28 v 187.28 a 6.77735,6.77735 -180 0 0 6.78,6.77 z" + class="st1" + id="path22-0" + style="fill:url(#linearGradient189);stroke:#386288;stroke-width:0.75" /><text + x="26.317923" + y="17.335487" + class="st2" + id="text24-5" + style="font-size:14.02988338px;font-family:Calibri;fill:#386288;stroke-width:1.40298378" + transform="scale(0.71276665,1.4029837)">Crypto Driver Private Session</text> + +</g><g + id="shape19-6-5" + transform="matrix(1.022976,0,0,0.71529071,9.1114734,-39.403506)"><title + id="title27-2">Rounded Rectangle.13</title><desc + id="desc29-0">Private Session Data</desc><g + id="shadow19-7-1" + transform="translate(0.345598,1.97279)" + class="st3" + style="visibility:visible"><path + inkscape:connector-curvature="0" + d="m 5.91,202.33 h 123.25 a 5.90925,5.90925 -180 0 0 5.91,-5.9 v -92.78 a 5.90925,5.90925 -180 0 0 -5.91,-5.91 H 5.91 A 5.90925,5.90925 -180 0 0 0,103.65 v 92.78 a 5.90925,5.90925 -180 0 0 5.91,5.9 z" + class="st4" + id="path31-8" + style="fill:#bdd0e9;fill-opacity:0.25;stroke:#bdd0e9;stroke-opacity:0.25;filter:url(#filter_2)" /></g><path + inkscape:connector-curvature="0" + d="m 5.91,202.33 h 123.25 a 5.90925,5.90925 -180 0 0 5.91,-5.9 v -92.78 a 5.90925,5.90925 -180 0 0 -5.91,-5.91 H 5.91 A 5.90925,5.90925 -180 0 0 0,103.65 v 92.78 a 5.90925,5.90925 -180 0 0 5.91,5.9 z" + class="st5" + id="path34-8" + style="fill:#a6b6cd;stroke:#41719c;stroke-width:0.75" /><text + x="34.639763" + y="119.96548" + class="st6" + id="text36-7" + style="font-size:13.15105343px;font-family:Calibri;fill:#41719c;stroke-width:1.31510115" + transform="scale(0.76039781,1.3151011)">Private Session Data</text> + +</g><g + id="shape18-1-4-7" + transform="matrix(1,0,0,0.57815109,0.90591369,163.94402)"><title + id="title18-4-3">Rounded Rectangle.12</title><desc + id="desc20-6-5">Crypto Symmetric Session</desc><path + inkscape:connector-curvature="0" + d="m 6.78,202.33 h 141.36 a 6.77735,6.77735 -180 0 0 6.77,-6.77 V 8.28 A 6.77735,6.77735 -180 0 0 148.14,1.5 H 6.78 A 6.77735,6.77735 -180 0 0 0,8.28 v 187.28 a 6.77735,6.77735 -180 0 0 6.78,6.77 z" + class="st1" + id="path22-0-8" + style="fill:url(#linearGradient189-7);stroke:#386288;stroke-width:0.75" /><text + x="26.317923" + y="17.335487" + class="st2" + id="text24-5-1" + style="font-size:14.02988338px;font-family:Calibri;fill:#386288;stroke-width:1.40298378" + transform="scale(0.71276665,1.4029837)">Crypto Driver Private Session</text> + +</g><g + id="shape19-6-5-1" + transform="matrix(1.022976,0,0,0.71529071,9.2675037,124.65774)"><title + id="title27-2-4">Rounded Rectangle.13</title><desc + id="desc29-0-9">Private Session Data</desc><g + id="shadow19-7-1-8" + transform="translate(0.345598,1.97279)" + class="st3" + style="visibility:visible"><path + inkscape:connector-curvature="0" + d="m 5.91,202.33 h 123.25 a 5.90925,5.90925 -180 0 0 5.91,-5.9 v -92.78 a 5.90925,5.90925 -180 0 0 -5.91,-5.91 H 5.91 A 5.90925,5.90925 -180 0 0 0,103.65 v 92.78 a 5.90925,5.90925 -180 0 0 5.91,5.9 z" + class="st4" + id="path31-8-4" + style="fill:#bdd0e9;fill-opacity:0.25;stroke:#bdd0e9;stroke-opacity:0.25;filter:url(#filter_2-3)" /></g><path + inkscape:connector-curvature="0" + d="m 5.91,202.33 h 123.25 a 5.90925,5.90925 -180 0 0 5.91,-5.9 v -92.78 a 5.90925,5.90925 -180 0 0 -5.91,-5.91 H 5.91 A 5.90925,5.90925 -180 0 0 0,103.65 v 92.78 a 5.90925,5.90925 -180 0 0 5.91,5.9 z" + class="st5" + id="path34-8-3" + style="fill:#a6b6cd;stroke:#41719c;stroke-width:0.75" /><text + x="34.639763" + y="119.96548" + class="st6" + id="text36-7-6" + style="font-size:13.15105343px;font-family:Calibri;fill:#41719c;stroke-width:1.31510115" + transform="scale(0.76039781,1.3151011)">Private Session Data</text> + +</g><text + xml:space="preserve" + style="font-style:normal;font-weight:normal;font-size:30.00008774px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75000221" + x="57.540585" + y="145.94679" + id="text5070"><tspan + sodipodi:role="line" + id="tspan5068" + x="57.540585" + y="173.31679" + style="stroke-width:0.75000221"></tspan></text> +<text + xml:space="preserve" + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:22.00006485px;line-height:1.25;font-family:sans-serif;-inkscape-font-specification:'sans-serif, Normal';font-variant-ligatures:normal;font-variant-caps:normal;font-variant-numeric:normal;font-feature-settings:normal;text-align:start;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75000221" + x="60.571766" + y="143.21872" + id="text5074"><tspan + sodipodi:role="line" + id="tspan5072" + x="60.571766" + y="143.21872" + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:22.00006485px;font-family:sans-serif;-inkscape-font-specification:'sans-serif, Normal';font-variant-ligatures:normal;font-variant-caps:normal;font-variant-numeric:normal;font-feature-settings:normal;text-align:start;writing-mode:lr-tb;text-anchor:start;stroke-width:0.75000221">...</tspan></text> +<path + style="fill:none;stroke:#41719c;stroke-width:0.74499911px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#marker5421)" + d="M -158.57624,71.371238 -157.38,232.04055 -1.1215065,232.19212" + id="path5076" + inkscape:connector-curvature="0" /></g></svg> diff --git a/doc/guides/prog_guide/img/eventdev_usage.svg b/doc/guides/prog_guide/img/eventdev_usage.svg new file mode 100644 index 00000000..7765649b --- /dev/null +++ b/doc/guides/prog_guide/img/eventdev_usage.svg @@ -0,0 +1,994 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> + +<svg + xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" + xmlns:dc="http://purl.org/dc/elements/1.1/" + xmlns:cc="http://creativecommons.org/ns#" + xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" + xmlns:svg="http://www.w3.org/2000/svg" + xmlns="http://www.w3.org/2000/svg" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" + xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" + width="683.12061" + height="184.672" + viewBox="0 0 546.49648 147.7376" + xml:space="preserve" + color-interpolation-filters="sRGB" + class="st9" + id="svg2" + version="1.1" + inkscape:version="0.48.4 r9939" + sodipodi:docname="eventdev_usage.svg" + style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"><metadata + id="metadata214"><rdf:RDF><cc:Work + rdf:about=""><dc:format>image/svg+xml</dc:format><dc:type + rdf:resource="http://purl.org/dc/dcmitype/StillImage" /></cc:Work></rdf:RDF></metadata><sodipodi:namedview + pagecolor="#ffffff" + bordercolor="#666666" + borderopacity="1" + objecttolerance="10" + gridtolerance="10" + guidetolerance="10" + inkscape:pageopacity="0" + inkscape:pageshadow="2" + inkscape:window-width="1920" + inkscape:window-height="1017" + id="namedview212" + showgrid="false" + fit-margin-top="2" + fit-margin-left="2" + fit-margin-right="2" + fit-margin-bottom="2" + inkscape:zoom="1.2339869" + inkscape:cx="501.15554" + inkscape:cy="164.17693" + inkscape:window-x="-8" + inkscape:window-y="406" + inkscape:window-maximized="1" + inkscape:current-layer="g17" /> + <v:documentProperties + v:langID="1033" + v:viewMarkup="false"> + <v:userDefs> + <v:ud + v:nameU="msvSubprocessMaster" + v:prompt="" + v:val="VT4(Rectangle)" /> + <v:ud + v:nameU="msvNoAutoConnect" + v:val="VT0(1):26" /> + </v:userDefs> + </v:documentProperties> + + <style + type="text/css" + id="style4"> + + .st1 {visibility:visible} + .st2 {fill:#5b9bd5;fill-opacity:0.22;filter:url(#filter_2);stroke:#5b9bd5;stroke-opacity:0.22} + .st3 {fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25} + .st4 {fill:#feffff;font-family:Calibri;font-size:0.833336em} + .st5 {font-size:1em} + .st6 {fill:#ffd965;stroke:#c7c8c8;stroke-width:0.25} + .st7 {marker-end:url(#mrkr4-33);stroke:#5b9bd5;stroke-linecap:round;stroke-linejoin:round;stroke-width:1} + .st8 {fill:#5b9bd5;fill-opacity:1;stroke:#5b9bd5;stroke-opacity:1;stroke-width:0.28409090909091} + .st9 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3} + + </style> + + <defs + id="Markers"> + <g + id="lend4"> + <path + d="M 2,1 0,0 2,-1 2,1" + style="stroke:none" + id="path8" + inkscape:connector-curvature="0" /> + </g> + <marker + id="mrkr4-33" + class="st8" + v:arrowType="4" + v:arrowSize="2" + v:setback="7.04" + refX="-7.04" + orient="auto" + markerUnits="strokeWidth" + overflow="visible" + style="fill:#5b9bd5;fill-opacity:1;stroke:#5b9bd5;stroke-width:0.28409091;stroke-opacity:1;overflow:visible"> + <use + xlink:href="#lend4" + transform="scale(-3.52,-3.52)" + id="use11" + x="0" + y="0" + width="3" + height="3" /> + </marker> + <filter + id="filter_2-7" + color-interpolation-filters="sRGB"><feGaussianBlur + stdDeviation="2" + id="feGaussianBlur15-1" /></filter><marker + id="mrkr4-33-2" + class="st8" + v:arrowType="4" + v:arrowSize="2" + v:setback="7.04" + refX="-7.04" + orient="auto" + markerUnits="strokeWidth" + overflow="visible" + style="fill:#5b9bd5;fill-opacity:1;stroke:#5b9bd5;stroke-width:0.28409091;stroke-opacity:1;overflow:visible"><use + xlink:href="#lend4" + transform="scale(-3.52,-3.52)" + id="use11-3" + x="0" + y="0" + width="3" + height="3" /></marker><marker + id="mrkr4-33-6" + class="st8" + v:arrowType="4" + v:arrowSize="2" + v:setback="7.04" + refX="-7.04" + orient="auto" + markerUnits="strokeWidth" + overflow="visible" + style="fill:#5b9bd5;fill-opacity:1;stroke:#5b9bd5;stroke-width:0.28409091;stroke-opacity:1;overflow:visible"><use + xlink:href="#lend4" + transform="scale(-3.52,-3.52)" + id="use11-8" + x="0" + y="0" + width="3" + height="3" /></marker></defs> + <defs + id="Filters"> + <filter + id="filter_2" + color-interpolation-filters="sRGB"> + <feGaussianBlur + stdDeviation="2" + id="feGaussianBlur15" /> + </filter> + </defs> + <g + v:mID="0" + v:index="1" + v:groupContext="foregroundPage" + id="g17" + transform="translate(-47.323579,-90.784072)"> + <v:userDefs> + <v:ud + v:nameU="msvThemeOrder" + v:val="VT0(0):26" /> + </v:userDefs> + <title + id="title19">Page-1</title> + <v:pageProperties + v:drawingScale="1" + v:pageScale="1" + v:drawingUnits="0" + v:shadowOffsetX="9" + v:shadowOffsetY="-9" /> + <v:layer + v:name="Connector" + v:index="0" /> + <g + id="shape1-1" + v:mID="1" + v:groupContext="shape" + transform="matrix(0.77644652,0,0,0.77644652,128.62352,-288.18843)"> + <title + id="title22">Square</title> + <desc + id="desc24">Atomic Queue #1</desc> + <v:userDefs> + <v:ud + v:nameU="visVersion" + v:val="VT0(15):26" /> + </v:userDefs> + <v:textBlock + v:margins="rect(4,4,4,4)" /> + <v:textRect + cx="30.75" + cy="581.25" + width="61.5" + height="61.5" /> + <g + id="shadow1-2" + v:groupContext="shadow" + v:shadowOffsetX="0.345598" + v:shadowOffsetY="-1.97279" + v:shadowType="1" + transform="translate(0.345598,1.97279)" + class="st1" + style="visibility:visible"> + <rect + x="0" + y="550.5" + width="61.5" + height="61.5" + class="st2" + id="rect27" + style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2)" /> + </g> + <rect + x="0" + y="550.5" + width="61.5" + height="61.5" + class="st3" + id="rect29" + style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" /> + + </g> + <g + id="shape3-8" + v:mID="3" + v:groupContext="shape" + transform="matrix(0.77644652,0,0,0.77644652,297.37175,-288.18843)"> + <title + id="title36">Square.3</title> + <desc + id="desc38">Atomic Queue #2</desc> + <v:userDefs> + <v:ud + v:nameU="visVersion" + v:val="VT0(15):26" /> + </v:userDefs> + <v:textBlock + v:margins="rect(4,4,4,4)" /> + <v:textRect + cx="30.75" + cy="581.25" + width="61.5" + height="61.5" /> + <g + id="shadow3-9" + v:groupContext="shadow" + v:shadowOffsetX="0.345598" + v:shadowOffsetY="-1.97279" + v:shadowType="1" + transform="translate(0.345598,1.97279)" + class="st1" + style="visibility:visible"> + <rect + x="0" + y="550.5" + width="61.5" + height="61.5" + class="st2" + id="rect41" + style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2)" /> + </g> + <rect + x="0" + y="550.5" + width="61.5" + height="61.5" + class="st3" + id="rect43" + style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" /> + + </g> + <g + id="shape4-15" + v:mID="4" + v:groupContext="shape" + transform="matrix(0.77644652,0,0,0.77644652,466.1192,-288.18843)"> + <title + id="title50">Square.4</title> + <desc + id="desc52">Single Link Queue # 1</desc> + <v:userDefs> + <v:ud + v:nameU="visVersion" + v:val="VT0(15):26" /> + </v:userDefs> + <v:textBlock + v:margins="rect(4,4,4,4)" /> + <v:textRect + cx="30.75" + cy="581.25" + width="61.5" + height="61.5" /> + <g + id="shadow4-16" + v:groupContext="shadow" + v:shadowOffsetX="0.345598" + v:shadowOffsetY="-1.97279" + v:shadowType="1" + transform="translate(0.345598,1.97279)" + class="st1" + style="visibility:visible"> + <rect + x="0" + y="550.5" + width="61.5" + height="61.5" + class="st2" + id="rect55" + style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2)" /> + </g> + <rect + x="0" + y="550.5" + width="61.5" + height="61.5" + class="st3" + id="rect57" + style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" /> + + </g> + <g + id="shape5-22" + v:mID="5" + v:groupContext="shape" + transform="matrix(0.77644652,0,0,0.77644652,52.208527,-296.14701)"> + <title + id="title64">Circle</title> + <desc + id="desc66">RX</desc> + <v:userDefs> + <v:ud + v:nameU="visVersion" + v:val="VT0(15):26" /> + </v:userDefs> + <v:textBlock + v:margins="rect(4,4,4,4)" /> + <v:textRect + cx="20.5" + cy="591.5" + width="35.88" + height="30.75" /> + <g + id="shadow5-23" + v:groupContext="shadow" + v:shadowOffsetX="0.345598" + v:shadowOffsetY="-1.97279" + v:shadowType="1" + transform="translate(0.345598,1.97279)" + class="st1" + style="visibility:visible"> + <path + d="m 0,591.5 a 20.5,20.5 0 0 1 41,0 20.5,20.5 0 1 1 -41,0 z" + class="st2" + id="path69" + inkscape:connector-curvature="0" + style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2)" /> + </g> + <path + d="m 0,591.5 a 20.5,20.5 0 0 1 41,0 20.5,20.5 0 1 1 -41,0 z" + class="st6" + id="path71" + inkscape:connector-curvature="0" + style="fill:#ffd965;stroke:#c7c8c8;stroke-width:0.25" /> + <text + x="15.19" + y="594.5" + class="st4" + v:langID="1033" + id="text73" + style="fill:#feffff;font-family:Calibri"><v:paragraph + v:horizAlign="1" /><v:tabList />RX</text> + + </g> + <g + id="shape6-28" + v:mID="6" + v:groupContext="shape" + v:layerMember="0" + transform="matrix(0.77644652,0,0,0.77644652,84.042834,-305.07614)"> + <title + id="title76">Dynamic connector</title> + <path + d="m 0,603 50.38,0" + class="st7" + id="path78" + inkscape:connector-curvature="0" + style="stroke:#5b9bd5;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;marker-end:url(#mrkr4-33)" /> + </g> + <g + id="shape7-34" + v:mID="7" + v:groupContext="shape" + transform="matrix(0.77644652,0,0,0.77644652,220.95621,-296.14701)"> + <title + id="title81">Circle.7</title> + <desc + id="desc83">W ..</desc> + <v:userDefs> + <v:ud + v:nameU="visVersion" + v:val="VT0(15):26" /> + </v:userDefs> + <v:textBlock + v:margins="rect(4,4,4,4)" /> + <v:textRect + cx="20.5" + cy="591.5" + width="35.88" + height="30.75" /> + <g + id="shadow7-35" + v:groupContext="shadow" + v:shadowOffsetX="0.345598" + v:shadowOffsetY="-1.97279" + v:shadowType="1" + transform="translate(0.345598,1.97279)" + class="st1" + style="visibility:visible"> + <path + d="m 0,591.5 a 20.5,20.5 0 0 1 41,0 20.5,20.5 0 1 1 -41,0 z" + class="st2" + id="path86" + inkscape:connector-curvature="0" + style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2)" /> + </g> + <path + d="m 0,591.5 a 20.5,20.5 0 0 1 41,0 20.5,20.5 0 1 1 -41,0 z" + class="st6" + id="path88" + inkscape:connector-curvature="0" + style="fill:#ffd965;stroke:#c7c8c8;stroke-width:0.25" /> + <text + x="12.4" + y="594.5" + class="st4" + v:langID="1033" + id="text90" + style="fill:#feffff;font-family:Calibri"><v:paragraph + v:horizAlign="1" /><v:tabList />W ..</text> + + </g> + <g + id="shape9-40" + v:mID="9" + v:groupContext="shape" + transform="matrix(0.77644652,0,0,0.77644652,220.95621,-243.34865)"> + <title + id="title93">Circle.9</title> + <desc + id="desc95">W N</desc> + <v:userDefs> + <v:ud + v:nameU="visVersion" + v:val="VT0(15):26" /> + </v:userDefs> + <v:textBlock + v:margins="rect(4,4,4,4)" /> + <v:textRect + cx="20.5" + cy="591.5" + width="35.88" + height="30.75" /> + <g + id="shadow9-41" + v:groupContext="shadow" + v:shadowOffsetX="0.345598" + v:shadowOffsetY="-1.97279" + v:shadowType="1" + transform="translate(0.345598,1.97279)" + class="st1" + style="visibility:visible"> + <path + d="m 0,591.5 a 20.5,20.5 0 0 1 41,0 20.5,20.5 0 1 1 -41,0 z" + class="st2" + id="path98" + inkscape:connector-curvature="0" + style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2)" /> + </g> + <path + d="m 0,591.5 a 20.5,20.5 0 0 1 41,0 20.5,20.5 0 1 1 -41,0 z" + class="st6" + id="path100" + inkscape:connector-curvature="0" + style="fill:#ffd965;stroke:#c7c8c8;stroke-width:0.25" /> + <text + x="11.69" + y="594.5" + class="st4" + v:langID="1033" + id="text102" + style="fill:#feffff;font-family:Calibri"><v:paragraph + v:horizAlign="1" /><v:tabList />W N</text> + + </g> + <g + id="shape10-46" + v:mID="10" + v:groupContext="shape" + transform="matrix(0.77644652,0,0,0.77644652,220.95621,-348.94537)"> + <title + id="title105">Circle.10</title> + <desc + id="desc107">W 1</desc> + <v:userDefs> + <v:ud + v:nameU="visVersion" + v:val="VT0(15):26" /> + </v:userDefs> + <v:textBlock + v:margins="rect(4,4,4,4)" /> + <v:textRect + cx="20.5" + cy="591.5" + width="35.88" + height="30.75" /> + <g + id="shadow10-47" + v:groupContext="shadow" + v:shadowOffsetX="0.345598" + v:shadowOffsetY="-1.97279" + v:shadowType="1" + transform="translate(0.345598,1.97279)" + class="st1" + style="visibility:visible"> + <path + d="m 0,591.5 a 20.5,20.5 0 0 1 41,0 20.5,20.5 0 1 1 -41,0 z" + class="st2" + id="path110" + inkscape:connector-curvature="0" + style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2)" /> + </g> + <path + d="m 0,591.5 a 20.5,20.5 0 0 1 41,0 20.5,20.5 0 1 1 -41,0 z" + class="st6" + id="path112" + inkscape:connector-curvature="0" + style="fill:#ffd965;stroke:#c7c8c8;stroke-width:0.25" /> + <text + x="12.39" + y="594.5" + class="st4" + v:langID="1033" + id="text114" + style="fill:#feffff;font-family:Calibri"><v:paragraph + v:horizAlign="1" /><v:tabList />W 1</text> + + </g> + <g + id="shape11-52" + v:mID="11" + v:groupContext="shape" + v:layerMember="0" + transform="matrix(0.77644652,0,0,0.77644652,195.91581,-312.06416)"> + <title + id="title117">Dynamic connector.11</title> + <path + d="m 0,612 0,-68 25.21,0" + class="st7" + id="path119" + inkscape:connector-curvature="0" + style="stroke:#5b9bd5;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;marker-end:url(#mrkr4-33)" /> + </g> + <g + id="shape12-57" + v:mID="12" + v:groupContext="shape" + v:layerMember="0" + transform="matrix(0.77644652,0,0,0.77644652,176.37498,-305.07614)"> + <title + id="title122">Dynamic connector.12</title> + <path + d="m 0,603 50.38,0" + class="st7" + id="path124" + inkscape:connector-curvature="0" + style="stroke:#5b9bd5;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;marker-end:url(#mrkr4-33)" /> + </g> + <g + id="shape13-62" + v:mID="13" + v:groupContext="shape" + v:layerMember="0" + transform="matrix(0.77644652,0,0,0.77644652,176.37498,-312.06416)"> + <title + id="title127">Dynamic connector.13</title> + <path + d="m 0,612 25.17,0 0,68 25.21,0" + class="st7" + id="path129" + inkscape:connector-curvature="0" + style="stroke:#5b9bd5;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;marker-end:url(#mrkr4-33)" /> + </g> + <g + id="shape14-67" + v:mID="14" + v:groupContext="shape" + v:layerMember="0" + transform="matrix(0.77644652,0,0,0.77644652,252.79052,-259.2658)"> + <title + id="title132">Dynamic connector.14</title> + <path + d="m 0,612 26.88,0 0,-68 23.5,0" + class="st7" + id="path134" + inkscape:connector-curvature="0" + style="stroke:#5b9bd5;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;marker-end:url(#mrkr4-33)" /> + </g> + <g + id="shape15-72" + v:mID="15" + v:groupContext="shape" + v:layerMember="0" + transform="matrix(0.77644652,0,0,0.77644652,252.79052,-305.07614)"> + <title + id="title137">Dynamic connector.15</title> + <path + d="m 0,603 50.38,0" + class="st7" + id="path139" + inkscape:connector-curvature="0" + style="stroke:#5b9bd5;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;marker-end:url(#mrkr4-33)" /> + </g> + <g + id="shape19-77" + v:mID="19" + v:groupContext="shape" + transform="matrix(0.77644652,0,0,0.77644652,389.70366,-296.14701)"> + <title + id="title142">Circle.19</title> + <desc + id="desc144">W ..</desc> + <v:userDefs> + <v:ud + v:nameU="visVersion" + v:val="VT0(15):26" /> + </v:userDefs> + <v:textBlock + v:margins="rect(4,4,4,4)" /> + <v:textRect + cx="20.5" + cy="591.5" + width="35.88" + height="30.75" /> + <g + id="shadow19-78" + v:groupContext="shadow" + v:shadowOffsetX="0.345598" + v:shadowOffsetY="-1.97279" + v:shadowType="1" + transform="translate(0.345598,1.97279)" + class="st1" + style="visibility:visible"> + <path + d="m 0,591.5 a 20.5,20.5 0 0 1 41,0 20.5,20.5 0 1 1 -41,0 z" + class="st2" + id="path147" + inkscape:connector-curvature="0" + style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2)" /> + </g> + <path + d="m 0,591.5 a 20.5,20.5 0 0 1 41,0 20.5,20.5 0 1 1 -41,0 z" + class="st6" + id="path149" + inkscape:connector-curvature="0" + style="fill:#ffd965;stroke:#c7c8c8;stroke-width:0.25" /> + <text + x="12.4" + y="594.5" + class="st4" + v:langID="1033" + id="text151" + style="fill:#feffff;font-family:Calibri"><v:paragraph + v:horizAlign="1" /><v:tabList />W ..</text> + + </g> + <g + id="shape20-83" + v:mID="20" + v:groupContext="shape" + transform="matrix(0.77644652,0,0,0.77644652,389.70366,-243.34865)"> + <title + id="title154">Circle.20</title> + <desc + id="desc156">W N</desc> + <v:userDefs> + <v:ud + v:nameU="visVersion" + v:val="VT0(15):26" /> + </v:userDefs> + <v:textBlock + v:margins="rect(4,4,4,4)" /> + <v:textRect + cx="20.5" + cy="591.5" + width="35.88" + height="30.75" /> + <g + id="shadow20-84" + v:groupContext="shadow" + v:shadowOffsetX="0.345598" + v:shadowOffsetY="-1.97279" + v:shadowType="1" + transform="translate(0.345598,1.97279)" + class="st1" + style="visibility:visible"> + <path + d="m 0,591.5 a 20.5,20.5 0 0 1 41,0 20.5,20.5 0 1 1 -41,0 z" + class="st2" + id="path159" + inkscape:connector-curvature="0" + style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2)" /> + </g> + <path + d="m 0,591.5 a 20.5,20.5 0 0 1 41,0 20.5,20.5 0 1 1 -41,0 z" + class="st6" + id="path161" + inkscape:connector-curvature="0" + style="fill:#ffd965;stroke:#c7c8c8;stroke-width:0.25" /> + <text + x="11.69" + y="594.5" + class="st4" + v:langID="1033" + id="text163" + style="fill:#feffff;font-family:Calibri"><v:paragraph + v:horizAlign="1" /><v:tabList />W N</text> + + </g> + <g + id="shape21-89" + v:mID="21" + v:groupContext="shape" + transform="matrix(0.77644652,0,0,0.77644652,389.70366,-348.94537)"> + <title + id="title166">Circle.21</title> + <desc + id="desc168">W 1</desc> + <v:userDefs> + <v:ud + v:nameU="visVersion" + v:val="VT0(15):26" /> + </v:userDefs> + <v:textBlock + v:margins="rect(4,4,4,4)" /> + <v:textRect + cx="20.5" + cy="591.5" + width="35.88" + height="30.75" /> + <g + id="shadow21-90" + v:groupContext="shadow" + v:shadowOffsetX="0.345598" + v:shadowOffsetY="-1.97279" + v:shadowType="1" + transform="translate(0.345598,1.97279)" + class="st1" + style="visibility:visible"> + <path + d="m 0,591.5 a 20.5,20.5 0 0 1 41,0 20.5,20.5 0 1 1 -41,0 z" + class="st2" + id="path171" + inkscape:connector-curvature="0" + style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2)" /> + </g> + <path + d="m 0,591.5 a 20.5,20.5 0 0 1 41,0 20.5,20.5 0 1 1 -41,0 z" + class="st6" + id="path173" + inkscape:connector-curvature="0" + style="fill:#ffd965;stroke:#c7c8c8;stroke-width:0.25" /> + <text + x="12.39" + y="594.5" + class="st4" + v:langID="1033" + id="text175" + style="fill:#feffff;font-family:Calibri"><v:paragraph + v:horizAlign="1" /><v:tabList />W 1</text> + + </g> + <g + id="shape28-95" + v:mID="28" + v:groupContext="shape" + v:layerMember="0" + transform="matrix(0.77644652,0,0,0.77644652,345.12321,-305.07614)"> + <title + id="title178">Dynamic connector.28</title> + <path + d="m 0,603 50.38,0" + class="st7" + id="path180" + inkscape:connector-curvature="0" + style="stroke:#5b9bd5;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;marker-end:url(#mrkr4-33)" /> + </g> + <g + id="shape29-100" + v:mID="29" + v:groupContext="shape" + v:layerMember="0" + transform="matrix(0.77644652,0,0,0.77644652,345.12321,-312.06416)"> + <title + id="title183">Dynamic connector.29</title> + <path + d="m 0,612 28.33,0 0,-68 22.05,0" + class="st7" + id="path185" + inkscape:connector-curvature="0" + style="stroke:#5b9bd5;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;marker-end:url(#mrkr4-33)" /> + </g> + <g + id="shape30-105" + v:mID="30" + v:groupContext="shape" + v:layerMember="0" + transform="matrix(0.77644652,0,0,0.77644652,345.12321,-312.06416)"> + <title + id="title188">Dynamic connector.30</title> + <path + d="m 0,612 28.33,0 0,68 22.05,0" + class="st7" + id="path190" + inkscape:connector-curvature="0" + style="stroke:#5b9bd5;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;marker-end:url(#mrkr4-33)" /> + </g> + <g + id="shape31-110" + v:mID="31" + v:groupContext="shape" + v:layerMember="0" + transform="matrix(0.77644652,0,0,0.77644652,421.53797,-259.2658)"> + <title + id="title193">Dynamic connector.31</title> + <path + d="m 0,612 24.42,0 0,-68 25.96,0" + class="st7" + id="path195" + inkscape:connector-curvature="0" + style="stroke:#5b9bd5;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;marker-end:url(#mrkr4-33)" /> + </g> + <g + id="shape32-115" + v:mID="32" + v:groupContext="shape" + v:layerMember="0" + transform="matrix(0.77644652,0,0,0.77644652,421.53797,-305.07614)"> + <title + id="title198">Dynamic connector.32</title> + <path + d="m 0,603 50.38,0" + class="st7" + id="path200" + inkscape:connector-curvature="0" + style="stroke:#5b9bd5;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;marker-end:url(#mrkr4-33)" /> + </g> + <g + id="shape33-120" + v:mID="33" + v:groupContext="shape" + v:layerMember="0" + transform="matrix(0.77644652,0,0,0.77644652,421.53797,-364.86253)"> + <title + id="title203">Dynamic connector.33</title> + <path + d="m 0,612 24.42,0 0,68 25.96,0" + class="st7" + id="path205" + inkscape:connector-curvature="0" + style="stroke:#5b9bd5;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;marker-end:url(#mrkr4-33)" /> + </g> + <g + id="shape34-125" + v:mID="34" + v:groupContext="shape" + v:layerMember="0" + transform="matrix(0.77644652,0,0,0.77644652,252.79052,-364.86253)"> + <title + id="title208">Dynamic connector.34</title> + <path + d="m 0,612 26.88,0 0,68 23.5,0" + class="st7" + id="path210" + inkscape:connector-curvature="0" + style="stroke:#5b9bd5;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;marker-end:url(#mrkr4-33)" /> + </g> + <text + xml:space="preserve" + style="font-size:24.84628868px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + x="153.38116" + y="165.90149" + id="text3106" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + x="153.38116" + y="165.90149" + id="tspan3110" + style="font-size:8.69620132px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;writing-mode:lr-tb;text-anchor:middle;fill:#ffffff;fill-opacity:1;font-family:Sans;-inkscape-font-specification:Sans">Atomic #1</tspan></text> +<text + xml:space="preserve" + style="font-size:24.84628868px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;overflow:visible;font-family:Sans" + x="322.12939" + y="165.90149" + id="text3106-1" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + x="322.12939" + y="165.90149" + id="tspan3110-4" + style="font-size:8.69620132px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;writing-mode:lr-tb;text-anchor:middle;fill:#ffffff;fill-opacity:1;font-family:Sans;-inkscape-font-specification:Sans">Atomic #2</tspan></text> +<text + xml:space="preserve" + style="font-size:24.84628868px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;overflow:visible;font-family:Sans" + x="491.82089" + y="172.79289" + id="text3106-0" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + x="491.82089" + y="172.79289" + style="font-size:8.69620132px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;writing-mode:lr-tb;text-anchor:middle;fill:#ffffff;fill-opacity:1;font-family:Sans;-inkscape-font-specification:Sans" + id="tspan3923" /></text> +<text + xml:space="preserve" + style="font-size:24.84628868px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;overflow:visible;font-family:Sans" + x="491.02899" + y="165.03951" + id="text3106-8-5" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + x="491.02899" + y="165.03951" + id="tspan3110-2-1" + style="font-size:8.69620132px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;writing-mode:lr-tb;text-anchor:middle;fill:#ffffff;fill-opacity:1;font-family:Sans;-inkscape-font-specification:Sans">Single Link</tspan></text> +<g + style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible" + id="shape5-22-1" + v:mID="5" + v:groupContext="shape" + transform="matrix(0.77644652,0,0,0.77644652,556.00223,-296.89447)"><title + id="title64-5">Circle</title><desc + id="desc66-2">RX</desc><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(15):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="20.5" + cy="591.5" + width="35.88" + height="30.75" /><g + id="shadow5-23-7" + v:groupContext="shadow" + v:shadowOffsetX="0.345598" + v:shadowOffsetY="-1.97279" + v:shadowType="1" + transform="translate(0.345598,1.97279)" + class="st1" + style="visibility:visible"><path + d="m 0,591.5 a 20.5,20.5 0 0 1 41,0 20.5,20.5 0 1 1 -41,0 z" + class="st2" + id="path69-6" + inkscape:connector-curvature="0" + style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-7)" /></g><path + d="m 0,591.5 a 20.5,20.5 0 0 1 41,0 20.5,20.5 0 1 1 -41,0 z" + class="st6" + id="path71-1" + inkscape:connector-curvature="0" + style="fill:#ffd965;stroke:#c7c8c8;stroke-width:0.25" /><text + x="11.06866" + y="596.56067" + class="st4" + v:langID="1033" + id="text73-4" + style="fill:#feffff;font-family:Calibri"> TX</text> +</g><g + style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible" + id="shape28-95-5" + v:mID="28" + v:groupContext="shape" + v:layerMember="0" + transform="matrix(0.77644652,0,0,0.77644652,512.00213,-305.42637)"><title + id="title178-7">Dynamic connector.28</title><path + d="m 0,603 50.38,0" + class="st7" + id="path180-6" + inkscape:connector-curvature="0" + style="stroke:#5b9bd5;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;marker-end:url(#mrkr4-33)" /></g></g> +</svg> diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index ef5a02ad..40f04a10 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -38,11 +38,13 @@ Programmer's Guide intro overview env_abstraction_layer + service_cores ring_lib mempool_lib mbuf_lib poll_mode_drv rte_flow + traffic_management cryptodev_lib link_bonding_poll_mode_drv_lib timer_lib @@ -53,10 +55,12 @@ Programmer's Guide packet_distrib_lib reorder_lib ip_fragment_reassembly_lib + generic_receive_offload_lib pdump_lib multi_proc_support kernel_nic_interface thread_safety_dpdk_functions + eventdev qos_framework power_man packet_classif_access_ctrl diff --git a/doc/guides/prog_guide/kernel_nic_interface.rst b/doc/guides/prog_guide/kernel_nic_interface.rst index 6f7fd287..4b2dd770 100644 --- a/doc/guides/prog_guide/kernel_nic_interface.rst +++ b/doc/guides/prog_guide/kernel_nic_interface.rst @@ -69,7 +69,7 @@ The KNI kernel loadable module provides support for two types of devices: (simulating the RX side of the net driver). * For multiple kernel thread mode, maintains a kernel thread context for each KNI instance - (simulating the RX side of the new driver). + (simulating the RX side of the net driver). * Net device: diff --git a/doc/guides/prog_guide/metrics_lib.rst b/doc/guides/prog_guide/metrics_lib.rst index 702c29d9..d52204fd 100644 --- a/doc/guides/prog_guide/metrics_lib.rst +++ b/doc/guides/prog_guide/metrics_lib.rst @@ -144,6 +144,7 @@ print out all metrics for a given port: .. code-block:: c void print_metrics() { + struct rte_metric_value *metrics; struct rte_metric_name *names; int len; diff --git a/doc/guides/prog_guide/perf_opt_guidelines.rst b/doc/guides/prog_guide/perf_opt_guidelines.rst index 7c24aa48..286bf983 100644 --- a/doc/guides/prog_guide/perf_opt_guidelines.rst +++ b/doc/guides/prog_guide/perf_opt_guidelines.rst @@ -38,7 +38,7 @@ Performance Optimization Guidelines Introduction ------------ -The following sections describe optimizations used in the DPDK and optimizations that should be considered for a new applications. +The following sections describe optimizations used in DPDK and optimizations that should be considered for new applications. They also highlight the performance-impacting coding techniques that should, and should not be, used when developing an application using the DPDK. diff --git a/doc/guides/prog_guide/poll_mode_drv.rst b/doc/guides/prog_guide/poll_mode_drv.rst index 4987f70a..1ac8f7eb 100644 --- a/doc/guides/prog_guide/poll_mode_drv.rst +++ b/doc/guides/prog_guide/poll_mode_drv.rst @@ -84,7 +84,7 @@ Whenever needed and appropriate, asynchronous communication should be introduced Avoiding lock contention is a key issue in a multi-core environment. To address this issue, PMDs are designed to work with per-core private resources as much as possible. -For example, a PMD maintains a separate transmit queue per-core, per-port. +For example, a PMD maintains a separate transmit queue per-core, per-port, if the PMD is not ``DEV_TX_OFFLOAD_MT_LOCKFREE`` capable. In the same way, every receive queue of a port is assigned to and polled by a single logical core (lcore). To comply with Non-Uniform Memory Access (NUMA), memory management is designed to assign to each logical core @@ -146,6 +146,16 @@ This is also true for the pipe-line model provided all logical cores used are lo Multiple logical cores should never share receive or transmit queues for interfaces since this would require global locks and hinder performance. +If the PMD is ``DEV_TX_OFFLOAD_MT_LOCKFREE`` capable, multiple threads can invoke ``rte_eth_tx_burst()`` +concurrently on the same tx queue without SW lock. This PMD feature found in some NICs and useful in the following use cases: + +* Remove explicit spinlock in some applications where lcores are not mapped to Tx queues with 1:1 relation. + +* In the eventdev use case, avoid dedicating a separate TX core for transmitting and thus + enables more scaling as all workers can send the packets. + +See `Hardware Offload`_ for ``DEV_TX_OFFLOAD_MT_LOCKFREE`` capability probing details. + Device Identification and Configuration --------------------------------------- @@ -290,7 +300,8 @@ Hardware Offload Depending on driver capabilities advertised by ``rte_eth_dev_info_get()``, the PMD may support hardware offloading -feature like checksumming, TCP segmentation or VLAN insertion. +feature like checksumming, TCP segmentation, VLAN insertion or +lockfree multithreaded TX burst on the same TX queue. The support of these offload features implies the addition of dedicated status bit(s) and value field(s) into the rte_mbuf data structure, along diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst index b587ba99..662a9123 100644 --- a/doc/guides/prog_guide/rte_flow.rst +++ b/doc/guides/prog_guide/rte_flow.rst @@ -898,7 +898,7 @@ Matches a MPLS header. - Default ``mask`` matches label only. Item: ``GRE`` -^^^^^^^^^^^^^^ +^^^^^^^^^^^^^ Matches a GRE header. @@ -906,6 +906,55 @@ Matches a GRE header. - ``protocol``: protocol type. - Default ``mask`` matches protocol only. +Item: ``FUZZY`` +^^^^^^^^^^^^^^^ + +Fuzzy pattern match, expect faster than default. + +This is for device that support fuzzy match option. Usually a fuzzy match is +fast but the cost is accuracy. i.e. Signature Match only match pattern's hash +value, but it is possible two different patterns have the same hash value. + +Matching accuracy level can be configured by threshold. Driver can divide the +range of threshold and map to different accuracy levels that device support. + +Threshold 0 means perfect match (no fuzziness), while threshold 0xffffffff +means fuzziest match. + +.. _table_rte_flow_item_fuzzy: + +.. table:: FUZZY + + +----------+---------------+--------------------------------------------------+ + | Field | Subfield | Value | + +==========+===============+==================================================+ + | ``spec`` | ``threshold`` | 0 as perfect match, 0xffffffff as fuzziest match | + +----------+---------------+--------------------------------------------------+ + | ``last`` | ``threshold`` | upper range value | + +----------+---------------+--------------------------------------------------+ + | ``mask`` | ``threshold`` | bit-mask apply to "spec" and "last" | + +----------+---------------+--------------------------------------------------+ + +Usage example, fuzzy match a TCPv4 packets: + +.. _table_rte_flow_item_fuzzy_example: + +.. table:: Fuzzy matching + + +-------+----------+ + | Index | Item | + +=======+==========+ + | 0 | FUZZY | + +-------+----------+ + | 1 | Ethernet | + +-------+----------+ + | 2 | IPv4 | + +-------+----------+ + | 3 | TCP | + +-------+----------+ + | 4 | END | + +-------+----------+ + Actions ~~~~~~~ @@ -1517,6 +1566,73 @@ Return values: - 0 on success, a negative errno value otherwise and ``rte_errno`` is set. +Isolated mode +------------- + +The general expectation for ingress traffic is that flow rules process it +first; the remaining unmatched or pass-through traffic usually ends up in a +queue (with or without RSS, locally or in some sub-device instance) +depending on the global configuration settings of a port. + +While fine from a compatibility standpoint, this approach makes drivers more +complex as they have to check for possible side effects outside of this API +when creating or destroying flow rules. It results in a more limited set of +available rule types due to the way device resources are assigned (e.g. no +support for the RSS action even on capable hardware). + +Given that nonspecific traffic can be handled by flow rules as well, +isolated mode is a means for applications to tell a driver that ingress on +the underlying port must be injected from the defined flow rules only; that +no default traffic is expected outside those rules. + +This has the following benefits: + +- Applications get finer-grained control over the kind of traffic they want + to receive (no traffic by default). + +- More importantly they control at what point nonspecific traffic is handled + relative to other flow rules, by adjusting priority levels. + +- Drivers can assign more hardware resources to flow rules and expand the + set of supported rule types. + +Because toggling isolated mode may cause profound changes to the ingress +processing path of a driver, it may not be possible to leave it once +entered. Likewise, existing flow rules or global configuration settings may +prevent a driver from entering isolated mode. + +Applications relying on this mode are therefore encouraged to toggle it as +soon as possible after device initialization, ideally before the first call +to ``rte_eth_dev_configure()`` to avoid possible failures due to conflicting +settings. + +Once effective, the following functionality has no effect on the underlying +port and may return errors such as ``ENOTSUP`` ("not supported"): + +- Toggling promiscuous mode. +- Toggling allmulticast mode. +- Configuring MAC addresses. +- Configuring multicast addresses. +- Configuring VLAN filters. +- Configuring Rx filters through the legacy API (e.g. FDIR). +- Configuring global RSS settings. + +.. code-block:: c + + int + rte_flow_isolate(uint8_t port_id, int set, struct rte_flow_error *error); + +Arguments: + +- ``port_id``: port identifier of Ethernet device. +- ``set``: nonzero to enter isolated mode, attempt to leave it otherwise. +- ``error``: perform verbose error reporting if not NULL. PMDs initialize + this structure in case of error only. + +Return values: + +- 0 on success, a negative errno value otherwise and ``rte_errno`` is set. + Verbose error reporting ----------------------- @@ -2026,8 +2142,8 @@ A few features are intentionally not supported: - "MAC VLAN" or "tunnel" perfect matching modes should be automatically set according to the created flow rules. -- Signature mode of operation is not defined but could be handled through a - specific item type if needed. +- Signature mode of operation is not defined but could be handled through + "FUZZY" item. .. _table_rte_flow_migration_fdir: @@ -2054,8 +2170,8 @@ A few features are intentionally not supported: | | +----------+-----+ | | | | ``mask`` | any | | +---+-------------------+----------+-----+ | - | 3 | VF, PF (optional) | ``spec`` | any | | - | | +----------+-----+ | + | 3 | VF, PF, FUZZY | ``spec`` | any | | + | | (optional) +----------+-----+ | | | | ``last`` | N/A | | | | +----------+-----+ | | | | ``mask`` | any | | diff --git a/doc/guides/prog_guide/service_cores.rst b/doc/guides/prog_guide/service_cores.rst new file mode 100644 index 00000000..3a029ba9 --- /dev/null +++ b/doc/guides/prog_guide/service_cores.rst @@ -0,0 +1,81 @@ +.. BSD LICENSE + Copyright(c) 2017 Intel Corporation. All rights reserved. + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of Intel Corporation nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +Service Cores +============= + +DPDK has a concept known as service cores, which enables a dynamic way of +performing work on DPDK lcores. Service core support is built into the EAL, and +an API is provided to optionally allow applications to control how the service +cores are used at runtime. + +The service cores concept is built up out of services (components of DPDK that +require CPU cycles to operate) and service cores (DPDK lcores, tasked with +running services). The power of the service core concept is that the mapping +between service cores and services can be configured to abstract away the +difference between platforms and environments. + +For example, the Eventdev has hardware and software PMDs. Of these the software +PMD requires an lcore to perform the scheduling operations, while the hardware +PMD does not. With service cores, the application would not directly notice +that the scheduling is done in software. + +For detailed information about the service core API, please refer to the docs. + +Service Core Initialization +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +There are two methods to having service cores in a DPDK application, either by +using the service coremask, or by dynamically adding cores using the API. +The simpler of the two is to pass the `-s` coremask argument to EAL, which will +take any cores available in the main DPDK coremask, an if the bits are also set +in the service coremask the cores become service-cores instead of DPDK +application lcores. + +Enabling Services on Cores +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Each registered service can be individually mapped to a service core, or set of +service cores. Enabling a service on a particular core means that the lcore in +question will run the service. Disabling that core on the service stops the +lcore in question from running the service. + +Using this method, it is possible to assign specific workloads to each +service core, and map N workloads to M number of service cores. Each service +lcore loops over the services that are enabled for that core, and invokes the +function to run the service. + +Service Core Statistics +~~~~~~~~~~~~~~~~~~~~~~~ + +The service core library is capable of collecting runtime statistics like number +of calls to a specific service, and number of cycles used by the service. The +cycle count collection is dynamically configurable, allowing any application to +profile the services running on the system at any time. diff --git a/doc/guides/prog_guide/traffic_management.rst b/doc/guides/prog_guide/traffic_management.rst new file mode 100644 index 00000000..c0dc235e --- /dev/null +++ b/doc/guides/prog_guide/traffic_management.rst @@ -0,0 +1,251 @@ +.. BSD LICENSE + Copyright(c) 2017 Intel Corporation. All rights reserved. + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of Intel Corporation nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + +Traffic Management API +====================== + + +Overview +-------- + +This is the generic API for the Quality of Service (QoS) Traffic Management of +Ethernet devices, which includes the following main features: hierarchical +scheduling, traffic shaping, congestion management, packet marking. This API +is agnostic of the underlying HW, SW or mixed HW-SW implementation. + +Main features: + +* Part of DPDK rte_ethdev API +* Capability query API per port, per hierarchy level and per hierarchy node +* Scheduling algorithms: Strict Priority (SP), Weighed Fair Queuing (WFQ) +* Traffic shaping: single/dual rate, private (per node) and + shared (by multiple nodes) shapers +* Congestion management for hierarchy leaf nodes: algorithms of tail drop, head + drop, WRED, private (per node) and shared (by multiple nodes) WRED contexts +* Packet marking: IEEE 802.1q (VLAN DEI), IETF RFC 3168 (IPv4/IPv6 ECN for TCP + and SCTP), IETF RFC 2597 (IPv4 / IPv6 DSCP) + + +Capability API +-------------- + +The aim of these APIs is to advertise the capability information (i.e critical +parameter values) that the TM implementation (HW/SW) is able to support for the +application. The APIs supports the information disclosure at the TM level, at +any hierarchical level of the TM and at any node level of the specific +hierarchical level. Such information helps towards rapid understanding of +whether a specific implementation does meet the needs to the user application. + +At the TM level, users can get high level idea with the help of various +parameters such as maximum number of nodes, maximum number of hierarchical +levels, maximum number of shapers, maximum number of private shapers, type of +scheduling algorithm (Strict Priority, Weighted Fair Queueing , etc.), etc., +supported by the implementation. + +Likewise, users can query the capability of the TM at the hierarchical level to +have more granular knowledge about the specific level. The various parameters +such as maximum number of nodes at the level, maximum number of leaf/non-leaf +nodes at the level, type of the shaper(dual rate, single rate) supported at +the level if node is non-leaf type etc., are exposed as a result of +hierarchical level capability query. + +Finally, the node level capability API offers knowledge about the capability +supported by the node at any specific level. The information whether the +support is available for private shaper, dual rate shaper, maximum and minimum +shaper rate, etc. is exposed by node level capability API. + + +Scheduling Algorithms +--------------------- + +The fundamental scheduling algorithms that are supported are Strict Priority +(SP) and Weighted Fair Queuing (WFQ). The SP and WFQ algorithms are supported +at the level of each node of the scheduling hierarchy, regardless of the node +level/position in the tree. The SP algorithm is used to schedule between +sibling nodes with different priority, while WFQ is used to schedule between +groups of siblings that have the same priority. + +Algorithms such as Weighed Round Robin (WRR), byte-level WRR, Deficit WRR +(DWRR), etc are considered approximations of the ideal WFQ and are therefore +assimilated to WFQ, although an associated implementation-dependent accuracy, +performance and resource usage trade-off might exist. + + +Traffic Shaping +--------------- + +The TM API provides support for single rate and dual rate shapers (rate +limiters) for the hierarchy nodes, subject to the specific implementation +support being available. + +Each hierarchy node has zero or one private shaper (only one node using it) +and/or zero, one or several shared shapers (multiple nodes use the same shaper +instance). A private shaper is used to perform traffic shaping for a single +node, while a shared shaper is used to perform traffic shaping for a group of +nodes. + +The configuration of private and shared shapers is done through the definition +of shaper profiles. Any shaper profile (single rate or dual rate shaper) can be +used by one or several shaper instances (either private or shared). + +Single rate shapers use a single token bucket. Therefore, single rate shaper is +configured by setting the rate of the committed bucket to zero, which +effectively disables this bucket. The peak bucket is used to limit the rate +and the burst size for the single rate shaper. Dual rate shapers use both the +committed and the peak token buckets. The rate of the peak bucket has to be +bigger than zero, as well as greater than or equal to the rate of the committed +bucket. + + +Congestion Management +--------------------- + +Congestion management is used to control the admission of packets into a packet +queue or group of packet queues on congestion. The congestion management +algorithms that are supported are: Tail Drop, Head Drop and Weighted Random +Early Detection (WRED). They are made available for every leaf node in the +hierarchy, subject to the specific implementation supporting them. +On request of writing a new packet into the current queue while the queue is +full, the Tail Drop algorithm drops the new packet while leaving the queue +unmodified, as opposed to the Head Drop* algorithm, which drops the packet +at the head of the queue (the oldest packet waiting in the queue) and admits +the new packet at the tail of the queue. + +The Random Early Detection (RED) algorithm works by proactively dropping more +and more input packets as the queue occupancy builds up. When the queue is full +or almost full, RED effectively works as Tail Drop. The Weighted RED (WRED) +algorithm uses a separate set of RED thresholds for each packet color and uses +separate set of RED thresholds for each packet color. + +Each hierarchy leaf node with WRED enabled as its congestion management mode +has zero or one private WRED context (only one leaf node using it) and/or zero, +one or several shared WRED contexts (multiple leaf nodes use the same WRED +context). A private WRED context is used to perform congestion management for +a single leaf node, while a shared WRED context is used to perform congestion +management for a group of leaf nodes. + +The configuration of WRED private and shared contexts is done through the +definition of WRED profiles. Any WRED profile can be used by one or several +WRED contexts (either private or shared). + + +Packet Marking +-------------- +The TM APIs have been provided to support various types of packet marking such +as VLAN DEI packet marking (IEEE 802.1Q), IPv4/IPv6 ECN marking of TCP and SCTP +packets (IETF RFC 3168) and IPv4/IPv6 DSCP packet marking (IETF RFC 2597). +All VLAN frames of a given color get their DEI bit set if marking is enabled +for this color. In case, when marking for a given color is not enabled, the +DEI bit is left as is (either set or not). + +All IPv4/IPv6 packets of a given color with ECN set to 2’b01 or 2’b10 carrying +TCP or SCTP have their ECN set to 2’b11 if the marking feature is enabled for +the current color, otherwise the ECN field is left as is. + +All IPv4/IPv6 packets have their color marked into DSCP bits 3 and 4 as +follows: green mapped to Low Drop Precedence (2’b01), yellow to Medium (2’b10) +and red to High (2’b11). Marking needs to be explicitly enabled for each color; +when not enabled for a given color, the DSCP field of all packets with that +color is left as is. + + +Steps to Setup the Hierarchy +---------------------------- + +The TM hierarchical tree consists of leaf nodes and non-leaf nodes. Each leaf +node sits on top of a scheduling queue of the current Ethernet port. Therefore, +the leaf nodes have predefined IDs in the range of 0... (N-1), where N is the +number of scheduling queues of the current Ethernet port. The non-leaf nodes +have their IDs generated by the application outside of the above range, which +is reserved for leaf nodes. + +Each non-leaf node has multiple inputs (its children nodes) and single output +(which is input to its parent node). It arbitrates its inputs using Strict +Priority (SP) and Weighted Fair Queuing (WFQ) algorithms to schedule input +packets to its output while observing its shaping (rate limiting) constraints. + +The children nodes with different priorities are scheduled using the SP +algorithm based on their priority, with 0 as the highest priority. Children +with the same priority are scheduled using the WFQ algorithm according to their +weights. The WFQ weight of a given child node is relative to the sum of the +weights of all its sibling nodes that have the same priority, with 1 as the +lowest weight. For each SP priority, the WFQ weight mode can be set as either +byte-based or packet-based. + + +Initial Hierarchy Specification +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The hierarchy is specified by incrementally adding nodes to build up the +scheduling tree. The first node that is added to the hierarchy becomes the root +node and all the nodes that are subsequently added have to be added as +descendants of the root node. The parent of the root node has to be specified +as RTE_TM_NODE_ID_NULL and there can only be one node with this parent ID +(i.e. the root node). The unique ID that is assigned to each node when the node +is created is further used to update the node configuration or to connect +children nodes to it. + +During this phase, some limited checks on the hierarchy specification can be +conducted, usually limited in scope to the current node, its parent node and +its sibling nodes. At this time, since the hierarchy is not fully defined, +there is typically no real action performed by the underlying implementation. + + +Hierarchy Commit +~~~~~~~~~~~~~~~~ + +The hierarchy commit API is called during the port initialization phase (before +the Ethernet port is started) to freeze the start-up hierarchy. This function +typically performs the following steps: + +* It validates the start-up hierarchy that was previously defined for the + current port through successive node add API invocations. +* Assuming successful validation, it performs all the necessary implementation + specific operations to install the specified hierarchy on the current port, + with immediate effect once the port is started. + +This function fails when the currently configured hierarchy is not supported by +the Ethernet port, in which case the user can abort or try out another +hierarchy configuration (e.g. a hierarchy with less leaf nodes), which can be +built from scratch or by modifying the existing hierarchy configuration. Note +that this function can still fail due to other causes (e.g. not enough memory +available in the system, etc.), even though the specified hierarchy is +supported in principle by the current port. + + +Run-Time Hierarchy Updates +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The TM API provides support for on-the-fly changes to the scheduling hierarchy, +thus operations such as node add/delete, node suspend/resume, parent node +update, etc., can be invoked after the Ethernet port has been started, subject +to the specific implementation supporting them. The set of dynamic updates +supported by the implementation is advertised through the port capability set. diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst index ba9b5a21..3362f335 100644 --- a/doc/guides/rel_notes/deprecation.rst +++ b/doc/guides/rel_notes/deprecation.rst @@ -9,64 +9,51 @@ Deprecation Notices ------------------- * eal: the following functions are deprecated starting from 17.05 and will - be removed in 17.08: + be removed in 17.11: - ``rte_set_log_level``, replaced by ``rte_log_set_global_level`` - ``rte_get_log_level``, replaced by ``rte_log_get_global_level`` - ``rte_set_log_type``, replaced by ``rte_log_set_level`` - ``rte_get_log_type``, replaced by ``rte_log_get_level`` -* devargs: An ABI change is planned for 17.08 for the structure ``rte_devargs``. - The current version is dependent on bus-specific device identifier, which will - be made generic and abstracted, in order to make the EAL bus-agnostic. - - Accompanying this evolution, device command line parameters will thus support - explicit bus definition in a device declaration. - -* igb_uio: iomem mapping and sysfs files created for iomem and ioport in - igb_uio will be removed, because we are able to detect these from what Linux - has exposed, like the way we have done with uio-pci-generic. This change - targets release 17.05. - -* The VDEV subsystem will be converted as driver of the new bus model. - It may imply some EAL API changes in 17.08. - -* The struct ``rte_pci_driver`` is planned to be removed from - ``rte_cryptodev_driver`` and ``rte_eventdev_driver`` in 17.08. - -* ethdev: An API change is planned for 17.08 for the function - ``_rte_eth_dev_callback_process``. In 17.08 the function will return an ``int`` - instead of ``void`` and a fourth parameter ``void *ret_param`` will be added. - -* ethdev: for 17.08 it is planned to deprecate the following nine rte_eth_dev_* - functions and move them into the ixgbe PMD: - - ``rte_eth_dev_bypass_init``, ``rte_eth_dev_bypass_state_set``, - ``rte_eth_dev_bypass_state_show``, ``rte_eth_dev_bypass_event_store``, - ``rte_eth_dev_bypass_event_show``, ``rte_eth_dev_wd_timeout_store``, - ``rte_eth_dev_bypass_wd_timeout_show``, ``rte_eth_dev_bypass_ver_show``, - ``rte_eth_dev_bypass_wd_reset``. - - The following fields will be removed from ``struct eth_dev_ops``: - - ``bypass_init_t``, ``bypass_state_set_t``, ``bypass_state_show_t``, - ``bypass_event_set_t``, ``bypass_event_show_t``, ``bypass_wd_timeout_set_t``, - ``bypass_wd_timeout_show_t``, ``bypass_ver_show_t``, ``bypass_wd_reset_t``. - - The functions will be renamed to the following, and moved to the ``ixgbe`` PMD: - - ``rte_pmd_ixgbe_bypass_init``, ``rte_pmd_ixgbe_bypass_state_set``, - ``rte_pmd_ixgbe_bypass_state_show``, ``rte_pmd_ixgbe_bypass_event_set``, - ``rte_pmd_ixgbe_bypass_event_show``, ``rte_pmd_ixgbe_bypass_wd_timeout_set``, - ``rte_pmd_ixgbe_bypass_wd_timeout_show``, ``rte_pmd_ixgbe_bypass_ver_show``, - ``rte_pmd_ixgbe_bypass_wd_reset``. +* eal: several API and ABI changes are planned for ``rte_devargs`` in v17.11. + The format of device command line parameters will change. The bus will need + to be explicitly stated in the device declaration. The enum ``rte_devtype`` + was used to identify a bus and will disappear. + The structure ``rte_devargs`` will change. + The ``rte_devargs_list`` will be made private. + The following functions are deprecated starting from 17.08 and will either be + modified or removed in 17.11: + + - ``rte_eal_devargs_add`` + - ``rte_eal_devargs_type_count`` + - ``rte_eal_parse_devargs_str``, replaced by ``rte_eal_devargs_parse`` + +* eal: the support of Xen dom0 will be removed from EAL in 17.11; and with + that, drivers/net/xenvirt and examples/vhost_xen will also be removed. + +* eal: An ABI change is planned for 17.11 to make DPDK aware of IOVA address + translation scheme. + Reference to phys address in EAL data-structure or functions may change to + IOVA address or more appropriate name. + The change will be only for the name. + Functional aspects of the API or data-structure will remain same. * The mbuf flags PKT_RX_VLAN_PKT and PKT_RX_QINQ_PKT are deprecated and are respectively replaced by PKT_RX_VLAN_STRIPPED and PKT_RX_QINQ_STRIPPED, that are better described. The old flags and - their behavior will be kept until 17.05 and will be removed in 17.08. + their behavior will be kept until 17.08 and will be removed in 17.11. + +* mempool: The following will be modified in 17.11: + + - ``rte_mempool_xmem_size`` and ``rte_mempool_xmem_usage`` need to know + the mempool flag status so adding new param rte_mempool in those API. + - Removing __rte_unused int flag param from ``rte_mempool_generic_put`` + and ``rte_mempool_generic_get`` API. + - ``rte_mempool`` flags data type will changed from int to + unsigned int. -* ethdev: Tx offloads will no longer be enabled by default in 17.08. +* ethdev: Tx offloads will no longer be enabled by default in 17.11. Instead, the ``rte_eth_txmode`` structure will be extended with bit field to enable each Tx offload. Besides of making the Rx/Tx configuration API more consistent for the @@ -81,47 +68,52 @@ Deprecation Notices Target release for removal of the legacy API will be defined once most PMDs have switched to rte_flow. -* cryptodev: All PMD names definitions will be moved to the individual PMDs - in 17.08. +* ethdev: The device flag advertizing hotplug capability + ``RTE_ETH_DEV_DETACHABLE`` is not needed anymore and will be removed in + v17.11. + This capability is verified upon calling the relevant hotplug functions in EAL + by checking that the ``unplug`` ops is set in the bus. This verification is + done by the EAL and not by the ``ethdev`` layer anymore. Users relying on this + flag being present only have to remove their checks to follow the change. -* cryptodev: The following changes will be done in in 17.08: +* ABI/API changes are planned for 17.11 in all structures which include port_id + definition such as "rte_eth_dev_data", "rte_port_ethdev_reader_params", + "rte_port_ethdev_writer_params", and so on. The definition of port_id will be + changed from 8 bits to 16 bits in order to support more than 256 ports in + DPDK. All APIs which have port_id parameter will be changed at the same time. - - the device type enumeration ``rte_cryptodev_type`` will be removed - - the following structures will be changed: ``rte_cryptodev_session``, - ``rte_cryptodev_sym_session``, ``rte_cryptodev_info``, ``rte_cryptodev`` - - the function ``rte_cryptodev_count_devtype`` will be replaced by - ``rte_cryptodev_device_count_by_driver`` +* ethdev: An ABI change is planned for 17.11 for the structure rte_eth_dev_data. + The size of the unique name will increase RTE_ETH_NAME_MAX_LEN from 32 to + 64 characters to allow using a globally unique identifier (GUID) in this field. -* cryptodev: API changes are planned for 17.08 for the sessions management - to make it agnostic to the underlying devices, removing coupling with - crypto PMDs, so a single session can be used on multiple devices. +* ethdev: new parameters - ``rte_security_capabilities`` and + ``rte_security_ops`` will be added to ``rte_eth_dev_info`` and + ``rte_eth_dev`` respectively to support security operations like + ipsec inline. - - ``struct rte_cryptodev_sym_session``, dev_id, dev_type will be removed, - _private field changed to the indirect array of private data pointers of - all supported devices +* cryptodev: new parameters - ``rte_security_capabilities`` and + ``rte_security_ops`` will be added to ``rte_cryptodev_info`` and + ``rte_cryptodev`` respectively to support security protocol offloaded + operations. - An API of followed functions will be changed to allow operate on multiple - devices with one session: +* cryptodev: the following function is deprecated starting from 17.08 and will + be removed in 17.11: - - ``rte_cryptodev_sym_session_create`` - - ``rte_cryptodev_sym_session_free`` - - ``rte_cryptodev_sym_session_pool_create`` + - ``rte_cryptodev_create_vdev`` - While dev_id will not be stored in the ``struct rte_cryptodev_sym_session``, - directly, the change of followed API is required: +* cryptodev: the following function will be static in 17.11 and included + by all crypto drivers, therefore, will not be public: - - ``rte_cryptodev_queue_pair_attach_sym_session`` - - ``rte_cryptodev_queue_pair_detach_sym_session`` + - ``rte_cryptodev_vdev_pmd_init`` -* cryptodev: the structures ``rte_crypto_op``, ``rte_crypto_sym_op`` - and ``rte_crypto_sym_xform`` will be restructured in 17.08, - for correctness and improvement. +* cryptodev: the following function will have an extra parameter, passing a + statically allocated crypto driver structure, instead of calling malloc, + in 17.11: -* crypto/scheduler: the following two functions are deprecated starting - from 17.05 and will be removed in 17.08: + - ``rte_cryptodev_allocate_driver`` - - ``rte_crpytodev_scheduler_mode_get``, replaced by ``rte_cryptodev_scheduler_mode_get`` - - ``rte_crpytodev_scheduler_mode_set``, replaced by ``rte_cryptodev_scheduler_mode_set`` +* librte_meter: The API will change to accommodate configuration profiles. + Most of the API functions will have an additional opaque parameter. * librte_table: The ``key_mask`` parameter will be added to all the hash tables that currently do not have it, as well as to the hash compute function prototype. diff --git a/doc/guides/rel_notes/index.rst b/doc/guides/rel_notes/index.rst index c4d243cd..b3c8090a 100644 --- a/doc/guides/rel_notes/index.rst +++ b/doc/guides/rel_notes/index.rst @@ -36,6 +36,7 @@ Release Notes :numbered: rel_description + release_17_08 release_17_05 release_17_02 release_16_11 @@ -45,6 +46,5 @@ Release Notes release_2_1 release_2_0 release_1_8 - supported_os known_issues deprecation diff --git a/doc/guides/rel_notes/release_17_05.rst b/doc/guides/rel_notes/release_17_05.rst index ef6211b6..87b3f927 100644 --- a/doc/guides/rel_notes/release_17_05.rst +++ b/doc/guides/rel_notes/release_17_05.rst @@ -518,6 +518,7 @@ The libraries prepended with a plus sign were incremented in this version. librte_distributor.so.1 + librte_eal.so.4 librte_ethdev.so.6 + + librte_eventdev.so.1 librte_hash.so.2 librte_ip_frag.so.1 librte_jobstats.so.1 @@ -827,49 +828,3 @@ Tested Platforms * Host interface: PCI Express 3.0 x16 * Device ID: 15b3:1013 * Firmware version: 12.18.2000 - -Fixes in 17.05 Stable Release ------------------------------ - -17.05.1 -~~~~~~~ - -* app/testpmd: fix creating E-Tag and NVGRE flow rules -* drivers/net: fix vfio kmod dependency -* examples/vhost: fix uninitialized descriptor indexes -* kni: fix build on RHEL 7.4 -* kni: fix build with gcc 7.1 -* lpm: fix index of tbl8 -* net/af_packet: fix packet bytes counting -* net/af_packet: handle possible null pointer -* net/ark: fix buffer not null terminated -* net/ark: fix null pointer dereference -* net/ark: fix return code not checked -* net/ark: fix return value of null not checked -* net/bnxt: fix reporting of link status -* net/cxgbe: fix port statistics -* net/cxgbe: fix rxq default params for ports under same PF -* net/enic: fix build with gcc 7.1 -* net/i40e/base: fix Tx error stats on VF -* net/i40e: exclude internal packet's byte count -* net/i40e: fix VF statistics -* net/igb: fix add/delete of flex filters -* net/igb: fix checksum valid flags -* net/ixgbe: fix fdir mask not be reset -* net/liquidio: fix MTU calculation from port configuration -* net/mlx5: fix build with gcc 7.1 -* net/mlx5: fix completion buffer size -* net/mlx5: fix exception handling -* net/mlx5: fix flow application order on stop/start -* net/mlx5: fix redundant free of Tx buffer -* net/qede: fix VXLAN tunnel Tx offload flag setting -* net/ring: fix adding MAC addresses -* net/sfc: add Tx queue flush failed flag for sanity -* net/sfc/base: fix error code usage in common code -* net/sfc/base: let caller know that queue is already flushed -* net/tap: fix some flow collision -* net/virtio: zero the whole memory zone -* vfio: fix array bounds check -* vhost: fix crash on NUMA -* vhost: fix guest pages memory leak -* vhost: fix malloc size too small diff --git a/doc/guides/rel_notes/release_17_08.rst b/doc/guides/rel_notes/release_17_08.rst new file mode 100644 index 00000000..0bcdfb7b --- /dev/null +++ b/doc/guides/rel_notes/release_17_08.rst @@ -0,0 +1,614 @@ +DPDK Release 17.08 +================== + +.. **Read this first.** + + The text in the sections below explains how to update the release notes. + + Use proper spelling, capitalization and punctuation in all sections. + + Variable and config names should be quoted as fixed width text: + ``LIKE_THIS``. + + Build the docs and view the output file to ensure the changes are correct:: + + make doc-guides-html + + xdg-open build/doc/html/guides/rel_notes/release_17_08.html + + +New Features +------------ + +.. This section should contain new features added in this release. Sample + format: + + * **Add a title in the past tense with a full stop.** + + Add a short 1-2 sentence description in the past tense. The description + should be enough to allow someone scanning the release notes to + understand the new feature. + + If the feature adds a lot of sub-features you can use a bullet list like + this: + + * Added feature foo to do something. + * Enhanced feature bar to do something else. + + Refer to the previous release notes for examples. + + This section is a comment. do not overwrite or remove it. + Also, make sure to start the actual text at the margin. + ========================================================= + +* **Increase minimum x86 ISA version to SSE4.2.** + + Starting with version 17.08, DPDK requires SSE4.2 to run on x86. + Previous versions required SSE3. + +* **Added Service Core functionality.** + + The service core functionality added to EAL allows DPDK to run services such + as software PMDs on lcores without the application manually running them. The + service core infrastructure allows flexibility of running multiple services + on the same service lcore, and provides the application with powerful APIs to + configure the mapping from service lcores to services. + +* **Added Generic Receive Offload API.** + + Added Generic Receive Offload (GRO) API support to reassemble TCP/IPv4 + packets. The GRO API assumes all input packets have the correct + checksums. GRO API doesn't update checksums for merged packets. If + input packets are IP fragmented, the GRO API assumes they are complete + packets (i.e. with L4 headers). + +* **Added Fail-Safe PMD** + + Added the new Fail-Safe PMD. This virtual device allows applications to + support seamless hotplug of devices. + See the :doc:`/nics/fail_safe` guide for more details about this driver. + +* **Added support for generic flow API (rte_flow) on igb NICs.** + + This API provides a generic means of configuring hardware to match specific + ingress or egress traffic, altering its behavior and querying related counters + according to any number of user-defined rules. + + Added generic flow API support for Ethernet, IPv4, UDP, TCP and RAW pattern + items with QUEUE actions. There are four types of filter support for this + feature on igb. + +* **Added support for generic flow API (rte_flow) on enic.** + + Added flow API support for outer Ethernet, VLAN, IPv4, IPv6, UDP, TCP, SCTP, + VxLAN and inner Ethernet, VLAN, IPv4, IPv6, UDP and TCP pattern items with + QUEUE, MARK, FLAG and VOID actions for ingress traffic. + +* **Added support for Chelsio T6 family of adapters** + + The CXGBE PMD was updated to run Chelsio T6 family of adapters. + +* **Added latency and performance improvements for cxgbe** + + the Tx and Rx path in cxgbe were reworked to improve performance. In + addition the latency was reduced for slow traffic. + +* **Updated the bnxt PMD.** + + Updated the bnxt PMD. The major enhancements include: + + * Support MTU modification. + * Add support for LRO. + * Add support for VLAN filter and strip functionality. + * Additional enhancements to add support for more dev_ops. + * Added PMD specific APIs mainly to control VF from PF. + * Update HWRM version to 1.7.7 + +* **Added support for Rx interrupts on mlx4 driver.** + + Rx queues can be now be armed with an interrupt which will trigger on the + next packet arrival. + +* **Updated mlx5 driver.** + + Updated the mlx5 driver including the following changes: + + * Added vectorized Rx/Tx burst for x86. + * Added support for isolated mode from flow API. + * Reworked the flow drop action to implement in hardware classifier. + * Improved Rx interrupts management. + +* **Updated szedata2 PMD.** + + Added support for firmware with multiple Ethernet ports per physical port. + +* **Updated dpaa2 PMD.** + + Updated dpaa2 PMD. Major enhancements include: + + * Added support for MAC Filter configuration. + * Added support for Segmented Buffers. + * Added support for VLAN filter and strip functionality. + * Additional enhancements to add support for more dev_ops. + * Optimized the packet receive path + +* **Reorganized the symmetric crypto operation structure.** + + The crypto operation (``rte_crypto_sym_op``) has been reorganized as follows: + + * Removed the ``rte_crypto_sym_op_sess_type`` field. + * Replaced the pointer and physical address of IV with offset from the start + of the crypto operation. + * Moved length and offset of cipher IV to ``rte_crypto_cipher_xform``. + * Removed "Additional Authentication Data" (AAD) length. + * Removed digest length. + * Removed AAD pointer and physical address from ``auth`` structure. + * Added ``aead`` structure, containing parameters for AEAD algorithms. + +* **Reorganized the crypto operation structure.** + + The crypto operation (``rte_crypto_op``) has been reorganized as follows: + + * Added the ``rte_crypto_op_sess_type`` field. + * The enumerations ``rte_crypto_op_status`` and ``rte_crypto_op_type`` + have been modified to be ``uint8_t`` values. + * Removed the field ``opaque_data``. + * Pointer to ``rte_crypto_sym_op`` has been replaced with a zero length array. + +* **Reorganized the crypto symmetric session structure.** + + The crypto symmetric session structure (``rte_cryptodev_sym_session``) has + been reorganized as follows: + + * The ``dev_id`` field has been removed. + * The ``driver_id`` field has been removed. + * The mempool pointer ``mp`` has been removed. + * Replaced ``private`` marker with array of pointers to private data sessions + ``sess_private_data``. + +* **Updated cryptodev library.** + + * Added AEAD algorithm specific functions and structures, so it is not + necessary to use a combination of cipher and authentication + structures anymore. + * Added helper functions for crypto device driver identification. + * Added support for multi-device sessions, so a single session can be + used in multiple drivers. + * Added functions to initialize and free individual driver private data + with the same session. + +* **Updated dpaa2_sec crypto PMD.** + + Added support for AES-GCM and AES-CTR. + +* **Updated the AESNI MB PMD.** + + The AESNI MB PMD has been updated with additional support for: + + * 12-byte IV on AES Counter Mode, apart from the previous 16-byte IV. + +* **Updated the AES-NI GCM PMD.** + + The AES-NI GCM PMD was migrated from the ISA-L library to the Multi Buffer + library, as the latter library has Scatter Gather List support + now. The migration entailed adding additional support for 192-bit keys. + +* **Updated the Cryptodev Scheduler PMD.** + + Added a multicore based distribution mode, which distributes the enqueued + crypto operations among several slaves, running on different logical cores. + +* **Added NXP DPAA2 Eventdev PMD.** + + Added the new dpaa2 eventdev driver for NXP DPAA2 devices. See the + "Event Device Drivers" document for more details on this new driver. + +* **Added dpdk-test-eventdev test application.** + + The dpdk-test-eventdev tool is a Data Plane Development Kit (DPDK) application + that allows exercising various eventdev use cases. + This application has a generic framework to add new eventdev based test cases + to verify functionality and measure the performance parameters of DPDK + eventdev devices. + + +Known Issues +------------ + +.. This section should contain new known issues in this release. Sample format: + + * **Add title in present tense with full stop.** + + Add a short 1-2 sentence description of the known issue in the present + tense. Add information on any known workarounds. + + This section is a comment. do not overwrite or remove it. + Also, make sure to start the actual text at the margin. + ========================================================= + +* **Starting with version 17.08, libnuma is required to build DPDK.** + + +API Changes +----------- + +.. This section should contain API changes. Sample format: + + * Add a short 1-2 sentence description of the API change. Use fixed width + quotes for ``rte_function_names`` or ``rte_struct_names``. Use the past + tense. + + This section is a comment. do not overwrite or remove it. + Also, make sure to start the actual text at the margin. + ========================================================= + +* **Modified the _rte_eth_dev_callback_process function in the ethdev library.** + + The function ``_rte_eth_dev_callback_process()`` has been modified. The + return value has been changed from void to int and an extra parameter ``void + *ret_param`` has been added. + +* **Moved bypass functions from the rte_ethdev library to ixgbe PMD** + + * The following rte_ethdev library functions were removed: + + * ``rte_eth_dev_bypass_event_show()`` + * ``rte_eth_dev_bypass_event_store()`` + * ``rte_eth_dev_bypass_init()`` + * ``rte_eth_dev_bypass_state_set()`` + * ``rte_eth_dev_bypass_state_show()`` + * ``rte_eth_dev_bypass_ver_show()`` + * ``rte_eth_dev_bypass_wd_reset()`` + * ``rte_eth_dev_bypass_wd_timeout_show()`` + * ``rte_eth_dev_wd_timeout_store()`` + + * The following ixgbe PMD functions were added: + + * ``rte_pmd_ixgbe_bypass_event_show()`` + * ``rte_pmd_ixgbe_bypass_event_store()`` + * ``rte_pmd_ixgbe_bypass_init()`` + * ``rte_pmd_ixgbe_bypass_state_set()`` + * ``rte_pmd_ixgbe_bypass_state_show()`` + * ``rte_pmd_ixgbe_bypass_ver_show()`` + * ``rte_pmd_ixgbe_bypass_wd_reset()`` + * ``rte_pmd_ixgbe_bypass_wd_timeout_show()`` + * ``rte_pmd_ixgbe_bypass_wd_timeout_store()`` + +* **Reworked rte_cryptodev library.** + + The rte_cryptodev library has been reworked and updated. The following changes + have been made to it: + + * The crypto device type enumeration has been removed from cryptodev library. + * The function ``rte_crypto_count_devtype()`` has been removed, and replaced + by the new function ``rte_crypto_count_by_driver()``. + * Moved crypto device driver names definitions to the particular PMDs. + These names are not public anymore. + * The ``rte_cryptodev_configure()`` function does not create the session + mempool for the device anymore. + * The ``rte_cryptodev_queue_pair_attach_sym_session()`` and + ``rte_cryptodev_queue_pair_dettach_sym_session()`` functions require + the new parameter ``device id``. + * Parameters of ``rte_cryptodev_sym_session_create()`` were modified to + accept ``mempool``, instead of ``device id`` and ``rte_crypto_sym_xform``. + * Removed ``device id`` parameter from ``rte_cryptodev_sym_session_free()``. + * Added a new field ``session_pool`` to ``rte_cryptodev_queue_pair_setup()``. + * Removed ``aad_size`` parameter from + ``rte_cryptodev_sym_capability_check_auth()``. + * Added ``iv_size`` parameter to + ``rte_cryptodev_sym_capability_check_auth()``. + * Removed ``RTE_CRYPTO_OP_STATUS_ENQUEUED`` from enum + ``rte_crypto_op_status``. + + +ABI Changes +----------- + +.. This section should contain ABI changes. Sample format: + + * Add a short 1-2 sentence description of the ABI change that was announced + in the previous releases and made in this release. Use fixed width quotes + for ``rte_function_names`` or ``rte_struct_names``. Use the past tense. + + This section is a comment. do not overwrite or remove it. + Also, make sure to start the actual text at the margin. + ========================================================= + +* Changed type of ``domain`` field in ``rte_pci_addr`` to ``uint32_t`` + to follow the PCI standard. + +* Added new ``rte_bus`` experimental APIs available as operators within the + ``rte_bus`` structure. + +* Made ``rte_devargs`` structure internal device representation generic to + prepare for a bus-agnostic EAL. + +* **Reorganized the crypto operation structures.** + + Some fields have been modified in the ``rte_crypto_op`` and + ``rte_crypto_sym_op`` structures, as described in the `New Features`_ + section. + +* **Reorganized the crypto symmetric session structure.** + + Some fields have been modified in the ``rte_cryptodev_sym_session`` + structure, as described in the `New Features`_ section. + +* **Reorganized the rte_crypto_sym_cipher_xform structure.** + + * Added cipher IV length and offset parameters. + * Changed field size of key length from ``size_t`` to ``uint16_t``. + +* **Reorganized the rte_crypto_sym_auth_xform structure.** + + * Added authentication IV length and offset parameters. + * Changed field size of AAD length from ``uint32_t`` to ``uint16_t``. + * Changed field size of digest length from ``uint32_t`` to ``uint16_t``. + * Removed AAD length. + * Changed field size of key length from ``size_t`` to ``uint16_t``. + +* Replaced ``dev_type`` enumeration with ``uint8_t`` ``driver_id`` in + ``rte_cryptodev_info`` and ``rte_cryptodev`` structures. + +* Removed ``session_mp`` from ``rte_cryptodev_config``. + + +Shared Library Versions +----------------------- + +.. Update any library version updated in this release and prepend with a ``+`` + sign, like this: + + librte_acl.so.2 + + librte_cfgfile.so.2 + librte_cmdline.so.2 + + This section is a comment. do not overwrite or remove it. + ========================================================= + + +The libraries prepended with a plus sign were incremented in this version. + +.. code-block:: diff + + librte_acl.so.2 + librte_bitratestats.so.1 + librte_cfgfile.so.2 + librte_cmdline.so.2 + + librte_cryptodev.so.3 + librte_distributor.so.1 + + librte_eal.so.5 + + librte_ethdev.so.7 + + librte_eventdev.so.2 + + librte_gro.so.1 + librte_hash.so.2 + librte_ip_frag.so.1 + librte_jobstats.so.1 + librte_kni.so.2 + librte_kvargs.so.1 + librte_latencystats.so.1 + librte_lpm.so.2 + librte_mbuf.so.3 + librte_mempool.so.2 + librte_meter.so.1 + librte_metrics.so.1 + librte_net.so.1 + librte_pdump.so.1 + librte_pipeline.so.3 + librte_pmd_bond.so.1 + librte_pmd_ring.so.2 + librte_port.so.3 + librte_power.so.1 + librte_reorder.so.1 + librte_ring.so.1 + librte_sched.so.1 + librte_table.so.2 + librte_timer.so.1 + librte_vhost.so.3 + + +Tested Platforms +---------------- + +.. This section should contain a list of platforms that were tested with this + release. + + The format is: + + * <vendor> platform with <vendor> <type of devices> combinations + + * List of CPU + * List of OS + * List of devices + * Other relevant details... + + This section is a comment. do not overwrite or remove it. + Also, make sure to start the actual text at the margin. + ========================================================= + +* Intel(R) platforms with Mellanox(R) NICs combinations + + * Platform details: + + * Intel(R) Xeon(R) CPU E5-2697A v4 @ 2.60GHz + * Intel(R) Xeon(R) CPU E5-2697 v3 @ 2.60GHz + * Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz + * Intel(R) Xeon(R) CPU E5-2640 @ 2.50GHz + + * OS: + + * Red Hat Enterprise Linux Server release 7.3 (Maipo) + * Red Hat Enterprise Linux Server release 7.2 (Maipo) + * Ubuntu 16.10 + * Ubuntu 16.04 + * Ubuntu 14.04 + + * MLNX_OFED: 4.1-1.0.2.0 + + * NICs: + + * Mellanox(R) ConnectX(R)-3 Pro 40G MCX354A-FCC_Ax (2x40G) + + * Host interface: PCI Express 3.0 x8 + * Device ID: 15b3:1007 + * Firmware version: 2.40.5030 + + * Mellanox(R) ConnectX(R)-4 10G MCX4111A-XCAT (1x10G) + + * Host interface: PCI Express 3.0 x8 + * Device ID: 15b3:1013 + * Firmware version: 12.18.2000 + + * Mellanox(R) ConnectX(R)-4 10G MCX4121A-XCAT (2x10G) + + * Host interface: PCI Express 3.0 x8 + * Device ID: 15b3:1013 + * Firmware version: 12.18.2000 + + * Mellanox(R) ConnectX(R)-4 25G MCX4111A-ACAT (1x25G) + + * Host interface: PCI Express 3.0 x8 + * Device ID: 15b3:1013 + * Firmware version: 12.18.2000 + + * Mellanox(R) ConnectX(R)-4 25G MCX4121A-ACAT (2x25G) + + * Host interface: PCI Express 3.0 x8 + * Device ID: 15b3:1013 + * Firmware version: 12.18.2000 + + * Mellanox(R) ConnectX(R)-4 40G MCX4131A-BCAT/MCX413A-BCAT (1x40G) + + * Host interface: PCI Express 3.0 x8 + * Device ID: 15b3:1013 + * Firmware version: 12.18.2000 + + * Mellanox(R) ConnectX(R)-4 40G MCX415A-BCAT (1x40G) + + * Host interface: PCI Express 3.0 x16 + * Device ID: 15b3:1013 + * Firmware version: 12.18.2000 + + * Mellanox(R) ConnectX(R)-4 50G MCX4131A-GCAT/MCX413A-GCAT (1x50G) + + * Host interface: PCI Express 3.0 x8 + * Device ID: 15b3:1013 + * Firmware version: 12.18.2000 + + * Mellanox(R) ConnectX(R)-4 50G MCX414A-BCAT (2x50G) + + * Host interface: PCI Express 3.0 x8 + * Device ID: 15b3:1013 + * Firmware version: 12.18.2000 + + * Mellanox(R) ConnectX(R)-4 50G MCX415A-GCAT/MCX416A-BCAT/MCX416A-GCAT + (2x50G) + + * Host interface: PCI Express 3.0 x16 + * Device ID: 15b3:1013 + * Firmware version: 12.18.2000 + + * Mellanox(R) ConnectX(R)-4 50G MCX415A-CCAT (1x100G) + + * Host interface: PCI Express 3.0 x16 + * Device ID: 15b3:1013 + * Firmware version: 12.18.2000 + + * Mellanox(R) ConnectX(R)-4 100G MCX416A-CCAT (2x100G) + + * Host interface: PCI Express 3.0 x16 + * Device ID: 15b3:1013 + * Firmware version: 12.18.2000 + + * Mellanox(R) ConnectX(R)-4 Lx 10G MCX4121A-XCAT (2x10G) + + * Host interface: PCI Express 3.0 x8 + * Device ID: 15b3:1015 + * Firmware version: 14.18.2000 + + * Mellanox(R) ConnectX(R)-4 Lx 25G MCX4121A-ACAT (2x25G) + + * Host interface: PCI Express 3.0 x8 + * Device ID: 15b3:1015 + * Firmware version: 14.18.2000 + + * Mellanox(R) ConnectX(R)-5 100G MCX556A-ECAT (2x100G) + + * Host interface: PCI Express 3.0 x16 + * Device ID: 15b3:1017 + * Firmware version: 16.19.1200 + + * Mellanox(R) ConnectX-5 Ex EN 100G MCX516A-CDAT (2x100G) + + * Host interface: PCI Express 4.0 x16 + * Device ID: 15b3:1019 + * Firmware version: 16.19.1200 + +* Intel(R) platforms with Intel(R) NICs combinations + + * CPU + + * Intel(R) Atom(TM) CPU C2758 @ 2.40GHz + * Intel(R) Xeon(R) CPU D-1540 @ 2.00GHz + * Intel(R) Xeon(R) CPU D-1541 @ 2.10GHz + * Intel(R) Xeon(R) CPU E5-4667 v3 @ 2.00GHz + * Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz + * Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz + * Intel(R) Xeon(R) CPU E5-2695 v4 @ 2.10GHz + * Intel(R) Xeon(R) CPU E5-2658 v2 @ 2.40GHz + * Intel(R) Xeon(R) CPU E5-2658 v3 @ 2.20GHz + + * OS: + + * CentOS 7.2 + * Fedora 25 + * FreeBSD 11 + * Red Hat Enterprise Linux Server release 7.3 + * SUSE Enterprise Linux 12 + * Wind River Linux 8 + * Ubuntu 16.04 + * Ubuntu 16.10 + + * NICs: + + * Intel(R) 82599ES 10 Gigabit Ethernet Controller + + * Firmware version: 0x61bf0001 + * Device id (pf/vf): 8086:10fb / 8086:10ed + * Driver version: 4.0.1-k (ixgbe) + + * Intel(R) Corporation Ethernet Connection X552/X557-AT 10GBASE-T + + * Firmware version: 0x800001cf + * Device id (pf/vf): 8086:15ad / 8086:15a8 + * Driver version: 4.2.5 (ixgbe) + + * Intel(R) Ethernet Converged Network Adapter X710-DA4 (4x10G) + + * Firmware version: 6.01 0x80003205 + * Device id (pf/vf): 8086:1572 / 8086:154c + * Driver version: 2.0.19 (i40e) + + * Intel(R) Ethernet Converged Network Adapter X710-DA2 (2x10G) + + * Firmware version: 6.01 0x80003204 + * Device id (pf/vf): 8086:1572 / 8086:154c + * Driver version: 2.0.19 (i40e) + + * Intel(R) Ethernet Converged Network Adapter XXV710-DA2 (2x25G) + + * Firmware version: 6.01 0x80003221 + * Device id (pf/vf): 8086:158b + * Driver version: 2.0.19 (i40e) + + * Intel(R) Ethernet Converged Network Adapter XL710-QDA2 (2X40G) + + * Firmware version: 6.01 0x8000321c + * Device id (pf/vf): 8086:1583 / 8086:154c + * Driver version: 2.0.19 (i40e) + + * Intel(R) Corporation I350 Gigabit Network Connection + + * Firmware version: 1.48, 0x800006e7 + * Device id (pf/vf): 8086:1521 / 8086:1520 + * Driver version: 5.2.13-k (igb) diff --git a/doc/guides/rel_notes/supported_os.rst b/doc/guides/rel_notes/supported_os.rst deleted file mode 100644 index 92218c18..00000000 --- a/doc/guides/rel_notes/supported_os.rst +++ /dev/null @@ -1,49 +0,0 @@ -.. BSD LICENSE - Copyright(c) 2010-2014 Intel Corporation. All rights reserved. - All rights reserved. - - Redistribution and use in source and binary forms, with or without - modification, are permitted provided that the following conditions - are met: - - * Redistributions of source code must retain the above copyright - notice, this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright - notice, this list of conditions and the following disclaimer in - the documentation and/or other materials provided with the - distribution. - * Neither the name of Intel Corporation nor the names of its - contributors may be used to endorse or promote products derived - from this software without specific prior written permission. - - THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS - "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT - LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR - A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT - OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, - SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT - LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, - DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY - THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -Supported Operating Systems -=========================== - -The following Linux distributions were successfully used to compiler or run DPDK. - -* FreeBSD 10 - -* Fedora release 20 - -* Ubuntu 14.04 LTS - -* Wind River Linux 6 - -* Red Hat Enterprise Linux 6.5 - -* SUSE Enterprise Linux 11 SP3 - -These distributions may need additional packages that are not installed by default, or a specific kernel. -Refer to the :ref:`Linux guide <linux_gsg>` and :ref:`FreeBSD guide <freebsd_gsg>` for details. diff --git a/doc/guides/sample_app_ug/eventdev_pipeline_sw_pmd.rst b/doc/guides/sample_app_ug/eventdev_pipeline_sw_pmd.rst new file mode 100644 index 00000000..b1b18dd0 --- /dev/null +++ b/doc/guides/sample_app_ug/eventdev_pipeline_sw_pmd.rst @@ -0,0 +1,190 @@ + +.. BSD LICENSE + Copyright(c) 2017 Intel Corporation. All rights reserved. + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of Intel Corporation nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +Eventdev Pipeline SW PMD Sample Application +=========================================== + +The eventdev pipeline sample application is a sample app that demonstrates +the usage of the eventdev API using the software PMD. It shows how an +application can configure a pipeline and assign a set of worker cores to +perform the processing required. + +The application has a range of command line arguments allowing it to be +configured for various numbers worker cores, stages,queue depths and cycles per +stage of work. This is useful for performance testing as well as quickly testing +a particular pipeline configuration. + + +Compiling the Application +------------------------- + +To compile the application: + +#. Go to the sample application directory: + + .. code-block:: console + + export RTE_SDK=/path/to/rte_sdk + cd ${RTE_SDK}/examples/eventdev_pipeline_sw_pmd + +#. Set the target (a default target is used if not specified). For example: + + .. code-block:: console + + export RTE_TARGET=x86_64-native-linuxapp-gcc + + See the *DPDK Getting Started Guide* for possible RTE_TARGET values. + +#. Build the application: + + .. code-block:: console + + make + +Running the Application +----------------------- + +The application has a lot of command line options. This allows specification of +the eventdev PMD to use, and a number of attributes of the processing pipeline +options. + +An example eventdev pipeline running with the software eventdev PMD using +these settings is shown below: + + * ``-r1``: core mask 0x1 for RX + * ``-t1``: core mask 0x1 for TX + * ``-e4``: core mask 0x4 for the software scheduler + * ``-w FF00``: core mask for worker cores, 8 cores from 8th to 16th + * ``-s4``: 4 atomic stages + * ``-n0``: process infinite packets (run forever) + * ``-c32``: worker dequeue depth of 32 + * ``-W1000``: do 1000 cycles of work per packet in each stage + * ``-D``: dump statistics on exit + +.. code-block:: console + + ./build/eventdev_pipeline_sw_pmd --vdev event_sw0 -- -r1 -t1 -e4 -w FF00 -s4 -n0 -c32 -W1000 -D + +The application has some sanity checking built-in, so if there is a function +(eg; the RX core) which doesn't have a cpu core mask assigned, the application +will print an error message: + +.. code-block:: console + + Core part of pipeline was not assigned any cores. This will stall the + pipeline, please check core masks (use -h for details on setting core masks): + rx: 0 + tx: 1 + +Configuration of the eventdev is covered in detail in the programmers guide, +see the Event Device Library section. + + +Observing the Application +------------------------- + +At runtime the eventdev pipeline application prints out a summary of the +configuration, and some runtime statistics like packets per second. On exit the +worker statistics are printed, along with a full dump of the PMD statistics if +required. The following sections show sample output for each of the output +types. + +Configuration +~~~~~~~~~~~~~ + +This provides an overview of the pipeline, +scheduling type at each stage, and parameters to options such as how many +flows to use and what eventdev PMD is in use. See the following sample output +for details: + +.. code-block:: console + + Config: + ports: 2 + workers: 8 + packets: 0 + priorities: 1 + Queue-prio: 0 + qid0 type: atomic + Cores available: 44 + Cores used: 10 + Eventdev 0: event_sw + Stages: + Stage 0, Type Atomic Priority = 128 + Stage 1, Type Atomic Priority = 128 + Stage 2, Type Atomic Priority = 128 + Stage 3, Type Atomic Priority = 128 + +Runtime +~~~~~~~ + +At runtime, the statistics of the consumer are printed, stating the number of +packets received, runtime in milliseconds, average mpps, and current mpps. + +.. code-block:: console + + # consumer RX= xxxxxxx, time yyyy ms, avg z.zzz mpps [current w.www mpps] + +Shutdown +~~~~~~~~ + +At shutdown, the application prints the number of packets received and +transmitted, and an overview of the distribution of work across worker cores. + +.. code-block:: console + + Signal 2 received, preparing to exit... + worker 12 thread done. RX=4966581 TX=4966581 + worker 13 thread done. RX=4963329 TX=4963329 + worker 14 thread done. RX=4953614 TX=4953614 + worker 0 thread done. RX=0 TX=0 + worker 11 thread done. RX=4970549 TX=4970549 + worker 10 thread done. RX=4986391 TX=4986391 + worker 9 thread done. RX=4970528 TX=4970528 + worker 15 thread done. RX=4974087 TX=4974087 + worker 8 thread done. RX=4979908 TX=4979908 + worker 2 thread done. RX=0 TX=0 + + Port Workload distribution: + worker 0 : 12.5 % (4979876 pkts) + worker 1 : 12.5 % (4970497 pkts) + worker 2 : 12.5 % (4986359 pkts) + worker 3 : 12.5 % (4970517 pkts) + worker 4 : 12.5 % (4966566 pkts) + worker 5 : 12.5 % (4963297 pkts) + worker 6 : 12.5 % (4953598 pkts) + worker 7 : 12.5 % (4974055 pkts) + +To get a full dump of the state of the eventdev PMD, pass the ``-D`` flag to +this application. When the app is terminated using ``Ctrl+C``, the +``rte_event_dev_dump()`` function is called, resulting in a dump of the +statistics that the PMD provides. The statistics provided depend on the PMD +used, see the Event Device Drivers section for a list of eventdev PMDs. diff --git a/doc/guides/sample_app_ug/index.rst b/doc/guides/sample_app_ug/index.rst index 02611ef8..069d4f12 100644 --- a/doc/guides/sample_app_ug/index.rst +++ b/doc/guides/sample_app_ug/index.rst @@ -66,9 +66,11 @@ Sample Applications User Guides packet_ordering vmdq_dcb_forwarding vhost + vhost_scsi netmap_compatibility ip_pipeline test_pipeline + eventdev_pipeline_sw_pmd dist_app vm_power_management tep_termination diff --git a/doc/guides/sample_app_ug/ipsec_secgw.rst b/doc/guides/sample_app_ug/ipsec_secgw.rst index 885c77e3..b675cbae 100644 --- a/doc/guides/sample_app_ug/ipsec_secgw.rst +++ b/doc/guides/sample_app_ug/ipsec_secgw.rst @@ -1,5 +1,5 @@ .. BSD LICENSE - Copyright(c) 2016 Intel Corporation. All rights reserved. + Copyright(c) 2016-2017 Intel Corporation. All rights reserved. All rights reserved. Redistribution and use in source and binary forms, with or without @@ -153,7 +153,7 @@ The mapping of lcores to port/queues is similar to other l3fwd applications. For example, given the following command line:: ./build/ipsec-secgw -l 20,21 -n 4 --socket-mem 0,2048 \ - --vdev "cryptodev_null_pmd" -- -p 0xf -P -u 0x3 \ + --vdev "crypto_null" -- -p 0xf -P -u 0x3 \ --config="(0,0,20),(1,0,20),(2,0,21),(3,0,21)" \ -f /path/to/config_file \ @@ -165,7 +165,7 @@ where each options means: * The ``--socket-mem`` to use 2GB on socket 1. -* The ``--vdev "cryptodev_null_pmd"`` option creates virtual NULL cryptodev PMD. +* The ``--vdev "crypto_null"`` option creates virtual NULL cryptodev PMD. * The ``-p`` option enables ports (detected) 0, 1, 2 and 3. @@ -218,7 +218,7 @@ For example, something like the following command line: ./build/ipsec-secgw -l 20,21 -n 4 --socket-mem 0,2048 \ -w 81:00.0 -w 81:00.1 -w 81:00.2 -w 81:00.3 \ - --vdev "cryptodev_aesni_mb_pmd" --vdev "cryptodev_null_pmd" \ + --vdev "crypto_aesni_mb" --vdev "crypto_null" \ -- \ -p 0xf -P -u 0x3 --config="(0,0,20),(1,0,20),(2,0,21),(3,0,21)" \ -f sample.cfg @@ -412,14 +412,13 @@ where each options means: * Cipher algorithm - * Optional: No + * Optional: Yes, unless <aead_algo> is not used * Available options: * *null*: NULL algorithm * *aes-128-cbc*: AES-CBC 128-bit algorithm * *aes-128-ctr*: AES-CTR 128-bit algorithm - * *aes-128-gcm*: AES-GCM 128-bit algorithm * Syntax: *cipher_algo <your algorithm>* @@ -427,7 +426,8 @@ where each options means: * Cipher key, NOT available when 'null' algorithm is used - * Optional: No, must followed by <cipher_algo> option + * Optional: Yes, unless <aead_algo> is not used. + Must be followed by <cipher_algo> option * Syntax: Hexadecimal bytes (0x0-0xFF) concatenate by colon symbol ':'. The number of bytes should be as same as the specified cipher algorithm @@ -440,20 +440,20 @@ where each options means: * Authentication algorithm - * Optional: No + * Optional: Yes, unless <aead_algo> is not used * Available options: * *null*: NULL algorithm * *sha1-hmac*: HMAC SHA1 algorithm - * *aes-128-gcm*: AES-GCM 128-bit algorithm ``<auth_key>`` * Authentication key, NOT available when 'null' or 'aes-128-gcm' algorithm is used. - * Optional: No, must followed by <auth_algo> option + * Optional: Yes, unless <aead_algo> is not used. + Must be followed by <auth_algo> option * Syntax: Hexadecimal bytes (0x0-0xFF) concatenate by colon symbol ':'. The number of bytes should be as same as the specified authentication @@ -462,6 +462,32 @@ where each options means: For example: *auth_key A1:B2:C3:D4:A1:B2:C3:D4:A1:B2:C3:D4:A1:B2:C3:D4: A1:B2:C3:D4* +``<aead_algo>`` + + * AEAD algorithm + + * Optional: Yes, unless <cipher_algo> and <auth_algo> are not used + + * Available options: + + * *aes-128-gcm*: AES-GCM 128-bit algorithm + + * Syntax: *cipher_algo <your algorithm>* + +``<aead_key>`` + + * Cipher key, NOT available when 'null' algorithm is used + + * Optional: Yes, unless <cipher_algo> and <auth_algo> are not used. + Must be followed by <aead_algo> option + + * Syntax: Hexadecimal bytes (0x0-0xFF) concatenate by colon symbol ':'. + The number of bytes should be as same as the specified AEAD algorithm + key size. + + For example: *aead_key A1:B2:C3:D4:A1:B2:C3:D4:A1:B2:C3:D4: + A1:B2:C3:D4* + ``<mode>`` * The operation mode @@ -515,9 +541,8 @@ Example SA rules: src 1111:1111:1111:1111:1111:1111:1111:5555 \ dst 2222:2222:2222:2222:2222:2222:2222:5555 - sa in 105 cipher_algo aes-128-gcm \ - cipher_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \ - auth_algo aes-128-gcm \ + sa in 105 aead_algo aes-128-gcm \ + aead_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \ mode ipv4-tunnel src 172.16.2.5 dst 172.16.1.5 Routing rule syntax diff --git a/doc/guides/sample_app_ug/ipv4_multicast.rst b/doc/guides/sample_app_ug/ipv4_multicast.rst index 3e30f509..49712a0f 100644 --- a/doc/guides/sample_app_ug/ipv4_multicast.rst +++ b/doc/guides/sample_app_ug/ipv4_multicast.rst @@ -51,6 +51,12 @@ The lookup method is the Four-byte Key (FBK) hash-based method. The lookup table is composed of pairs of destination IPv4 address (the FBK) and a port mask associated with that IPv4 address. +.. note:: + + The max port mask supported in the given hash table is 0xf, so only first + four ports can be supported. + If using non-consecutive ports, use the destination IPv4 address accordingly. + For convenience and simplicity, this sample application does not take IANA-assigned multicast addresses into account, but instead equates the last four bytes of the multicast group (that is, the last four bytes of the destination IP address) with the mask of ports to multicast packets to. @@ -117,7 +123,7 @@ Typically, to run the IPv4 Multicast sample application, issue the following com In this command: -* The -c option enables cores 0, 1, 2 and 3 +* The -l option enables cores 0, 1, 2 and 3 * The -n option specifies 3 memory channels diff --git a/doc/guides/sample_app_ug/l2_forward_crypto.rst b/doc/guides/sample_app_ug/l2_forward_crypto.rst index d6df36b5..158f4b44 100644 --- a/doc/guides/sample_app_ug/l2_forward_crypto.rst +++ b/doc/guides/sample_app_ug/l2_forward_crypto.rst @@ -46,7 +46,7 @@ for each packet that is received on a RX_PORT and performs L2 forwarding. The destination port is the adjacent port from the enabled portmask, that is, if the first four ports are enabled (portmask 0xf), ports 0 and 1 forward into each other, and ports 2 and 3 forward into each other. -Also, the MAC addresses are affected as follows: +Also, if MAC addresses updating is enabled, the MAC addresses are affected as follows: * The source MAC address is replaced by the TX_PORT MAC address @@ -84,12 +84,16 @@ The application requires a number of command line options: .. code-block:: console ./build/l2fwd-crypto [EAL options] -- [-p PORTMASK] [-q NQ] [-s] [-T PERIOD] / - [--cdev_type HW/SW/ANY] [--chain HASH_CIPHER/CIPHER_HASH/CIPHER_ONLY/HASH_ONLY] / + [--cdev_type HW/SW/ANY] [--chain HASH_CIPHER/CIPHER_HASH/CIPHER_ONLY/HASH_ONLY/AEAD] / [--cipher_algo ALGO] [--cipher_op ENCRYPT/DECRYPT] [--cipher_key KEY] / - [--cipher_key_random_size SIZE] [--iv IV] [--iv_random_size SIZE] / + [--cipher_key_random_size SIZE] [--cipher_iv IV] [--cipher_iv_random_size SIZE] / [--auth_algo ALGO] [--auth_op GENERATE/VERIFY] [--auth_key KEY] / - [--auth_key_random_size SIZE] [--aad AAD] [--aad_random_size SIZE] / - [--digest size SIZE] [--sessionless] [--cryptodev_mask MASK] + [--auth_key_random_size SIZE] [--auth_iv IV] [--auth_iv_random_size SIZE] / + [--aead_algo ALGO] [--aead_op ENCRYPT/DECRYPT] [--aead_key KEY] / + [--aead_key_random_size SIZE] [--aead_iv] [--aead_iv_random_size SIZE] / + [--aad AAD] [--aad_random_size SIZE] / + [--digest size SIZE] [--sessionless] [--cryptodev_mask MASK] / + [--mac-updating] [--no-mac-updating] where, @@ -109,7 +113,9 @@ where, * chain: select the operation chaining to perform: Cipher->Hash (CIPHER_HASH), - Hash->Cipher (HASH_CIPHER), Cipher (CIPHER_ONLY), Hash(HASH_ONLY) + Hash->Cipher (HASH_CIPHER), Cipher (CIPHER_ONLY), Hash (HASH_ONLY) + + or AEAD (AEAD) (default is Cipher->Hash) @@ -127,15 +133,15 @@ where, Note that if --cipher_key is used, this will be ignored. -* iv: set the IV to be used. Bytes has to be separated with ":" +* cipher_iv: set the cipher IV to be used. Bytes has to be separated with ":" -* iv_random_size: set the size of the IV, which will be generated randomly. +* cipher_iv_random_size: set the size of the cipher IV, which will be generated randomly. - Note that if --iv is used, this will be ignored. + Note that if --cipher_iv is used, this will be ignored. * auth_algo: select the authentication algorithm (default is sha1-hmac) -* cipher_op: select the authentication operation to perform: GENERATE or VERIFY +* auth_op: select the authentication operation to perform: GENERATE or VERIFY (default is GENERATE) @@ -147,6 +153,32 @@ where, Note that if --auth_key is used, this will be ignored. +* auth_iv: set the auth IV to be used. Bytes has to be separated with ":" + +* auth_iv_random_size: set the size of the auth IV, which will be generated randomly. + + Note that if --auth_iv is used, this will be ignored. + +* aead_algo: select the AEAD algorithm (default is aes-gcm) + +* aead_op: select the AEAD operation to perform: ENCRYPT or DECRYPT + + (default is ENCRYPT) + +* aead_key: set the AEAD key to be used. Bytes has to be separated with ":" + +* aead_key_random_size: set the size of the AEAD key, + + which will be generated randomly. + + Note that if --aead_key is used, this will be ignored. + +* aead_iv: set the AEAD IV to be used. Bytes has to be separated with ":" + +* aead_iv_random_size: set the size of the AEAD IV, which will be generated randomly. + + Note that if --aead_iv is used, this will be ignored. + * aad: set the AAD to be used. Bytes has to be separated with ":" * aad_random_size: set the size of the AAD, which will be generated randomly. @@ -162,6 +194,8 @@ where, (default is all cryptodevs). +* [no-]mac-updating: Enable or disable MAC addresses updating (enabled by default). + The application requires that crypto devices capable of performing the specified crypto operation are available on application initialization. @@ -172,8 +206,8 @@ To run the application in linuxapp environment with 2 lcores, 2 ports and 2 cryp .. code-block:: console - $ ./build/l2fwd-crypto -l 0-1 -n 4 --vdev "cryptodev_aesni_mb_pmd" \ - --vdev "cryptodev_aesni_mb_pmd" -- -p 0x3 --chain CIPHER_HASH \ + $ ./build/l2fwd-crypto -l 0-1 -n 4 --vdev "crypto_aesni_mb0" \ + --vdev "crypto_aesni_mb1" -- -p 0x3 --chain CIPHER_HASH \ --cipher_op ENCRYPT --cipher_algo aes-cbc \ --cipher_key 00:01:02:03:04:05:06:07:08:09:0a:0b:0c:0d:0e:0f \ --auth_op GENERATE --auth_algo aes-xcbc-mac \ @@ -331,8 +365,14 @@ This session is created and is later attached to the crypto operation: uint8_t cdev_id) { struct rte_crypto_sym_xform *first_xform; + struct rte_cryptodev_sym_session *session; + uint8_t socket_id = rte_cryptodev_socket_id(cdev_id); + struct rte_mempool *sess_mp = session_pool_socket[socket_id]; + - if (options->xform_chain == L2FWD_CRYPTO_CIPHER_HASH) { + if (options->xform_chain == L2FWD_CRYPTO_AEAD) { + first_xform = &options->aead_xform; + } else if (options->xform_chain == L2FWD_CRYPTO_CIPHER_HASH) { first_xform = &options->cipher_xform; first_xform->next = &options->auth_xform; } else if (options->xform_chain == L2FWD_CRYPTO_HASH_CIPHER) { @@ -344,8 +384,16 @@ This session is created and is later attached to the crypto operation: first_xform = &options->auth_xform; } - /* Setup Cipher Parameters */ - return rte_cryptodev_sym_session_create(cdev_id, first_xform); + session = rte_cryptodev_sym_session_create(sess_mp); + + if (session == NULL) + return NULL; + + if (rte_cryptodev_sym_session_init(cdev_id, session, + first_xform, sess_mp) < 0) + return NULL; + + return session; } ... diff --git a/doc/guides/sample_app_ug/vhost_scsi.rst b/doc/guides/sample_app_ug/vhost_scsi.rst new file mode 100644 index 00000000..8be069e0 --- /dev/null +++ b/doc/guides/sample_app_ug/vhost_scsi.rst @@ -0,0 +1,115 @@ + +.. BSD LICENSE + Copyright(c) 2010-2017 Intel Corporation. All rights reserved. + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of Intel Corporation nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + +Vhost_scsi Sample Application +============================= + +The vhost_scsi sample application implemented a simple SCSI block device, +which used as the backend of Qemu vhost-user-scsi device. Users can extend +the exist example to use other type of block device(e.g. AIO) besides +memory based block device. Similar with vhost-user-net device, the sample +application used domain socket to communicate with Qemu, and the virtio +ring was processed by vhost_scsi sample application. + +The sample application reuse lots codes from SPDK(Storage Performance +Development Kit, https://github.com/spdk/spdk) vhost-user-scsi target, +for DPDK vhost library used in storage area, user can take SPDK as +reference as well. + +Testing steps +------------- + +This section shows the steps how to start a VM with the block device as +fast data path for critical application. + +Build +~~~~~ + +Follow the *Getting Started Guide for Linux* on generic info about +environment setup and building DPDK from source. + +In this example, you need build DPDK both on the host and inside guest. +Also, you need build this example. + +.. code-block:: console + + export RTE_SDK=/path/to/dpdk_source + export RTE_TARGET=x86_64-native-linuxapp-gcc + + cd ${RTE_SDK}/examples/vhost_scsi + make + + +Start the vhost_scsi example +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: console + + ./vhost_scsi -m 1024 + +.. _vhost_scsi_app_run_vm: + +Start the VM +~~~~~~~~~~~~ + +.. code-block:: console + + qemu-system-x86_64 -machine accel=kvm \ + -m $mem -object memory-backend-file,id=mem,size=$mem,\ + mem-path=/dev/hugepages,share=on -numa node,memdev=mem \ + -drive file=os.img,if=none,id=disk \ + -device ide-hd,drive=disk,bootindex=0 \ + -chardev socket,id=char0,path=/tmp/vhost.socket \ + -device vhost-user-scsi-pci,chardev=char0,bootindex=2 \ + ... + +.. note:: + You must check whether your Qemu can support "vhost-user-scsi" or not, + Qemu v2.10 or newer version is required. + +Vhost_scsi Common Issues +------------------------ + +* vhost_scsi can not start with block size 512 Bytes: + + Currently DPDK vhost library was designed for NET device(althrough the APIs + are generic now), for 512 Bytes block device, Qemu BIOS(x86 BIOS Enhanced + Disk Device) will enumerate all block device and do some IOs to those block + devices with 512 Bytes sector size. DPDK vhost library can not process such + scenarios(both BIOS and OS will enumerate the block device), so as a + workaround, the vhost_scsi example application hardcoded the block size + with 4096 Bytes. + +* vhost_scsi can only support the block device as fast data disk(non OS image): + + Make sure ``bootindex=2`` Qemu option is given to vhost-user-scsi-pci device. + diff --git a/doc/guides/testpmd_app_ug/run_app.rst b/doc/guides/testpmd_app_ug/run_app.rst index 2a43214e..e8303f3b 100644 --- a/doc/guides/testpmd_app_ug/run_app.rst +++ b/doc/guides/testpmd_app_ug/run_app.rst @@ -188,6 +188,19 @@ The commandline options are: Start forwarding on initialization. +* ``--tx-first`` + + Start forwarding, after sending a burst of packets first. + +.. Note:: + + This flag should be only used in non-interactive mode. + +* ``--stats-period PERIOD`` + + Display statistics every PERIOD seconds, if interactive mode is disabled. + The default value is 0, which means that the statistics will not be displayed. + * ``--nb-cores=N`` Set the number of forwarding cores, @@ -487,3 +500,11 @@ The commandline options are: Disable printing the occurrence of the designated event. Using all will disable all of them. + +* ``--flow-isolate-all`` + + Providing this parameter requests flow API isolated mode on all ports at + initialization time. It ensures all traffic is received through the + configured flow rules only (see flow command). + + Ports that do not support this mode are automatically discarded. diff --git a/doc/guides/testpmd_app_ug/testpmd_funcs.rst b/doc/guides/testpmd_app_ug/testpmd_funcs.rst index 0e50c107..2ed62f5b 100644 --- a/doc/guides/testpmd_app_ug/testpmd_funcs.rst +++ b/doc/guides/testpmd_app_ug/testpmd_funcs.rst @@ -380,6 +380,20 @@ For example:: testpmd> read txd 0 0 4 0x00000001 - 0x24C3C440 / 0x000F0000 - 0x2330003C +ddp get list +~~~~~~~~~~~~ + +Get loaded dynamic device personalization (DDP) package info list:: + + testpmd> ddp get list (port_id) + +ddp get info +~~~~~~~~~~~~ + +Display information about dynamic device personalization (DDP) profile:: + + testpmd> ddp get info (profile_patch) + show vf stats ~~~~~~~~~~~~~ @@ -884,6 +898,40 @@ Display the status of TCP Segmentation Offload:: testpmd> tso show (port_id) +gro +~~~ + +Enable or disable GRO in ``csum`` forwarding engine:: + + testpmd> gro (on|off) (port_id) + +If enabled, the csum forwarding engine will perform GRO on the TCP/IPv4 +packets received from the given port. + +If disabled, packets received from the given port won't be performed +GRO. By default, GRO is disabled for all ports. + +.. note:: + + When enable GRO for a port, TCP/IPv4 packets received from the port + will be performed GRO. After GRO, the merged packets are multi-segments. + But csum forwarding engine doesn't support to calculate TCP checksum + for multi-segment packets in SW. So please select TCP HW checksum + calculation for the port which GROed packets are transmitted to. + +gro set +~~~~~~~ + +Set max flow number and max packet number per-flow for GRO:: + + testpmd> gro set (max_flow_num) (max_item_num_per_flow) (port_id) + +The product of ``max_flow_num`` and ``max_item_num_per_flow`` is the max +number of packets a GRO table can store. + +If current packet number is greater than or equal to the max value, GRO +will stop processing incoming packets. + mac_addr add ~~~~~~~~~~~~ @@ -1211,6 +1259,20 @@ Add an E-tag forwarding filter on a port:: Delete an E-tag forwarding filter on a port:: testpmd> E-tag set filter del e-tag-id (value) port (port_id) +ddp add +~~~~~~~ + +Load a dynamic device personalization (DDP) package:: + + testpmd> ddp add (port_id) (package_path[,output_path]) + +ddp del +~~~~~~~ + +Delete a dynamic device personalization package:: + + testpmd> ddp del (port_id) (package_path) + ptype mapping ~~~~~~~~~~~~~ @@ -1738,6 +1800,23 @@ For example, to set the link status monitoring polling period of bonded device ( testpmd> set bonding mon_period 5 150 +set bonding lacp dedicated_queue +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Enable dedicated tx/rx queues on bonding devices slaves to handle LACP control plane traffic +when in mode 4 (link-aggregration-802.3ad):: + + testpmd> set bonding lacp dedicated_queues (port_id) (enable|disable) + + +set bonding agg_mode +~~~~~~~~~~~~~~~~~~~~ + +Enable one of the specific aggregators mode when in mode 4 (link-aggregration-802.3ad):: + + testpmd> set bonding agg_mode (port_id) (bandwidth|count|stable) + + show bonding config ~~~~~~~~~~~~~~~~~~~ @@ -2264,7 +2343,8 @@ Flow rules management --------------------- Control of the generic flow API (*rte_flow*) is fully exposed through the -``flow`` command (validation, creation, destruction and queries). +``flow`` command (validation, creation, destruction, queries and operation +modes). Considering *rte_flow* overlaps with all `Filter Functions`_, using both features simultaneously may cause undefined side-effects and is therefore @@ -2318,6 +2398,10 @@ following sections. flow list {port_id} [group {group_id}] [...] +- Restrict ingress traffic to the defined flow rules:: + + flow isolate {port_id} {boolean} + Validating flow rules ~~~~~~~~~~~~~~~~~~~~~ @@ -2608,6 +2692,10 @@ This section lists supported pattern items and their attributes, if any. - ``protocol {unsigned}``: protocol type. +- ``fuzzy``: fuzzy pattern match, expect faster than default. + + - ``thresh {unsigned}``: accuracy threshold. + Actions list ^^^^^^^^^^^^ @@ -2894,6 +2982,46 @@ Output can be limited to specific groups:: 7 63 0 i- ETH IPV6 UDP VXLAN => MARK QUEUE testpmd> +Toggling isolated mode +~~~~~~~~~~~~~~~~~~~~~~ + +``flow isolate`` can be used to tell the underlying PMD that ingress traffic +must only be injected from the defined flow rules; that no default traffic +is expected outside those rules and the driver is free to assign more +resources to handle them. It is bound to ``rte_flow_isolate()``:: + + flow isolate {port_id} {boolean} + +If successful, enabling or disabling isolated mode shows either:: + + Ingress traffic on port [...] + is now restricted to the defined flow rules + +Or:: + + Ingress traffic on port [...] + is not restricted anymore to the defined flow rules + +Otherwise, in case of error:: + + Caught error type [...] ([...]): [...] + +Mainly due to its side effects, PMDs supporting this mode may not have the +ability to toggle it more than once without reinitializing affected ports +first (e.g. by exiting testpmd). + +Enabling isolated mode:: + + testpmd> flow isolate 0 true + Ingress traffic on port 0 is now restricted to the defined flow rules + testpmd> + +Disabling isolated mode:: + + testpmd> flow isolate 0 false + Ingress traffic on port 0 is not restricted anymore to the defined flow rules + testpmd> + Sample QinQ flow rules ~~~~~~~~~~~~~~~~~~~~~~ @@ -2942,4 +3070,3 @@ Validate and create a QinQ rule on port 0 to steer traffic to a queue on the hos ID Group Prio Attr Rule 0 0 0 i- ETH VLAN VLAN=>VF QUEUE 1 0 0 i- ETH VLAN VLAN=>PF QUEUE - diff --git a/doc/guides/tools/cryptoperf.rst b/doc/guides/tools/cryptoperf.rst index 2d225d56..457f817c 100644 --- a/doc/guides/tools/cryptoperf.rst +++ b/doc/guides/tools/cryptoperf.rst @@ -186,6 +186,9 @@ The following are the appication command-line options: crypto_snow3g crypto_kasumi crypto_zuc + crypto_dpaa2_sec + crypto_armv8 + crypto_scheduler * ``--optype <name>`` @@ -223,10 +226,8 @@ The following are the appication command-line options: 3des-ecb 3des-ctr aes-cbc - aes-ccm aes-ctr aes-ecb - aes-gcm aes-f8 aes-xts arc4 @@ -257,9 +258,7 @@ The following are the appication command-line options: 3des-cbc aes-cbc-mac - aes-ccm aes-cmac - aes-gcm aes-gmac aes-xcbc-mac md5 @@ -290,13 +289,41 @@ The following are the appication command-line options: Set the size of authentication key. -* ``--auth-digest-sz <n>`` +* ``--auth-iv-sz <n>`` + + Set the size of auth iv. + +* ``--aead-algo <name>`` + + Set AEAD algorithm name, where ``name`` is one + of the following:: + + aes-ccm + aes-gcm + +* ``--aead-op <mode>`` - Set the size of authentication digest. + Set AEAD operation mode, where ``mode`` is one of + the following:: + + encrypt + decrypt + +* ``--aead-key-sz <n>`` + + Set the size of AEAD key. -* ``--auth-aad-sz <n>`` +* ``--aead-iv-sz <n>`` - Set the size of authentication aad. + Set the size of AEAD iv. + +* ``--aead-aad-sz <n>`` + + Set the size of AEAD aad. + +* ``--digest-sz <n>`` + + Set the size of digest. * ``--csv-friendly`` @@ -308,9 +335,9 @@ Test Vector File The test vector file is a text file contain information about test vectors. The file is made of the sections. The first section doesn't have header. It contain global information used in each test variant vectors - -typicaly information about plaintext, ciphertext, cipher key, aut key, +typically information about plaintext, ciphertext, cipher key, aut key, initial vector. All other sections begin header. -The sections contain particular information typicaly digest. +The sections contain particular information typically digest. **Format of the file:** @@ -345,9 +372,13 @@ a string of bytes in C byte array format:: Key used in auth operation. -* ``iv`` +* ``cipher_iv`` + + Cipher Initial Vector. + +* ``auth_iv`` - Initial vector. + Auth Initial Vector. * ``aad`` @@ -362,19 +393,19 @@ Examples Call application for performance throughput test of single Aesni MB PMD for cipher encryption aes-cbc and auth generation sha1-hmac, -one milion operations, burst size 32, packet size 64:: +one million operations, burst size 32, packet size 64:: - dpdk-test-crypto-perf -l 6-7 --vdev crypto_aesni_mb_pmd -w 0000:00:00.0 -- + dpdk-test-crypto-perf -l 6-7 --vdev crypto_aesni_mb -w 0000:00:00.0 -- --ptest throughput --devtype crypto_aesni_mb --optype cipher-then-auth --cipher-algo aes-cbc --cipher-op encrypt --cipher-key-sz 16 --auth-algo - sha1-hmac --auth-op generate --auth-key-sz 64 --auth-digest-sz 12 + sha1-hmac --auth-op generate --auth-key-sz 64 --digest-sz 12 --total-ops 10000000 --burst-sz 32 --buffer-sz 64 Call application for performance latency test of two Aesni MB PMD executed on two cores for cipher encryption aes-cbc, ten operations in silent mode:: - dpdk-test-crypto-perf -l 4-7 --vdev crypto_aesni_mb_pmd1 - --vdev crypto_aesni_mb_pmd2 -w 0000:00:00.0 -- --devtype crypto_aesni_mb + dpdk-test-crypto-perf -l 4-7 --vdev crypto_aesni_mb1 + --vdev crypto_aesni_mb2 -w 0000:00:00.0 -- --devtype crypto_aesni_mb --cipher-algo aes-cbc --cipher-key-sz 16 --cipher-iv-sz 16 --cipher-op encrypt --optype cipher-only --silent --ptest latency --total-ops 10 @@ -385,10 +416,9 @@ in silent mode, test vector provide in file "test_aes_gcm.data" with packet verification:: dpdk-test-crypto-perf -l 4-7 --vdev crypto_openssl -w 0000:00:00.0 -- - --devtype crypto_openssl --cipher-algo aes-gcm --cipher-key-sz 16 - --cipher-iv-sz 16 --cipher-op encrypt --auth-algo aes-gcm --auth-key-sz 16 - --auth-digest-sz 16 --auth-aad-sz 16 --auth-op generate --optype aead - --silent --ptest verify --total-ops 10 + --devtype crypto_openssl --aead-algo aes-gcm --aead-key-sz 16 + --aead-iv-sz 16 --aead-op encrypt --aead-aad-sz 16 --digest-sz 16 + --optype aead --silent --ptest verify --total-ops 10 --test-file test_aes_gcm.data Test vector file for cipher algorithm aes cbc 256 with authorization sha:: @@ -412,7 +442,7 @@ Test vector file for cipher algorithm aes cbc 256 with authorization sha:: 0xf5, 0x0c, 0xe7, 0xa2, 0xa6, 0x23, 0xd5, 0x3d, 0x95, 0xd8, 0xcd, 0x86, 0x79, 0xf5, 0x01, 0x47, 0x4f, 0xf9, 0x1d, 0x9d, 0x36, 0xf7, 0x68, 0x1a, 0x64, 0x44, 0x58, 0x5d, 0xe5, 0x81, 0x15, 0x2a, 0x41, 0xe4, 0x0e, 0xaa, 0x1f, 0x04, 0x21, 0xff, 0x2c, 0xf3, 0x73, 0x2b, 0x48, 0x1e, 0xd2, 0xf7 - iv = + cipher_iv = 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0A, 0x0B, 0x0C, 0x0D, 0x0E, 0x0F # Section sha 1 hmac buff 32 [sha1_hmac_buff_32] diff --git a/doc/guides/tools/img/eventdev_order_atq_test.svg b/doc/guides/tools/img/eventdev_order_atq_test.svg new file mode 100644 index 00000000..a36eafa7 --- /dev/null +++ b/doc/guides/tools/img/eventdev_order_atq_test.svg @@ -0,0 +1,1576 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<!-- Created with Inkscape (http://www.inkscape.org/) --> + +<!-- +# BSD LICENSE +# +# Copyright (c) 2017, Cavium, Inc +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# +# - Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# +# - Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in +# the documentation and/or other materials provided with the +# distribution. +# +# - Neither the name of Cavium, Inc nor the names of its +# contributors may be used to endorse or promote products derived +# from this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS +# FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE +# COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, +# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES +# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +# STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) +# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED +# OF THE POSSIBILITY OF SUCH DAMAGE. +--> + +<svg + xmlns:osb="http://www.openswatchbook.org/uri/2009/osb" + xmlns:dc="http://purl.org/dc/elements/1.1/" + xmlns:cc="http://creativecommons.org/ns#" + xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" + xmlns:svg="http://www.w3.org/2000/svg" + xmlns="http://www.w3.org/2000/svg" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" + xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" + width="631.91431" + height="288.34286" + id="svg3868" + version="1.1" + inkscape:version="0.92.1 r" + sodipodi:docname="order_atq.svg" + sodipodi:version="0.32" + inkscape:output_extension="org.inkscape.output.svg.inkscape"> + <defs + id="defs3870"> + <linearGradient + id="linearGradient6545" + osb:paint="solid"> + <stop + style="stop-color:#ffa600;stop-opacity:1;" + offset="0" + id="stop6543" /> + </linearGradient> + <inkscape:path-effect + effect="spiro" + id="path-effect3302" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3294" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3290" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3286" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3228" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3188" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3184" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3180" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3176" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3172" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3168" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3164" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3160" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3120" + is_visible="true" /> + <linearGradient + id="linearGradient3114" + osb:paint="solid"> + <stop + style="stop-color:#00f900;stop-opacity:1;" + offset="0" + id="stop3112" /> + </linearGradient> + <linearGradient + id="linearGradient3088" + osb:paint="solid"> + <stop + style="stop-color:#00f900;stop-opacity:1;" + offset="0" + id="stop3086" /> + </linearGradient> + <linearGradient + id="linearGradient3058" + osb:paint="solid"> + <stop + style="stop-color:#00f900;stop-opacity:1;" + offset="0" + id="stop3056" /> + </linearGradient> + <inkscape:path-effect + effect="spiro" + id="path-effect3054" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3050" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3046" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3042" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3038" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3034" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3030" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3008" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3004" + is_visible="true" /> + <linearGradient + id="linearGradient2975" + osb:paint="solid"> + <stop + style="stop-color:#ff2200;stop-opacity:1;" + offset="0" + id="stop2973" /> + </linearGradient> + <linearGradient + id="linearGradient2969" + osb:paint="solid"> + <stop + style="stop-color:#69ff72;stop-opacity:1;" + offset="0" + id="stop2967" /> + </linearGradient> + <linearGradient + id="linearGradient2963" + osb:paint="solid"> + <stop + style="stop-color:#000000;stop-opacity:1;" + offset="0" + id="stop2961" /> + </linearGradient> + <linearGradient + id="linearGradient2929" + osb:paint="solid"> + <stop + style="stop-color:#ff2d00;stop-opacity:1;" + offset="0" + id="stop2927" /> + </linearGradient> + <linearGradient + id="linearGradient4610" + osb:paint="solid"> + <stop + style="stop-color:#00ffff;stop-opacity:1;" + offset="0" + id="stop4608" /> + </linearGradient> + <linearGradient + id="linearGradient3993" + osb:paint="solid"> + <stop + style="stop-color:#6ba6fd;stop-opacity:1;" + offset="0" + id="stop3991" /> + </linearGradient> + <linearGradient + id="linearGradient3808" + osb:paint="solid"> + <stop + style="stop-color:#6ba6fd;stop-opacity:1;" + offset="0" + id="stop3806" /> + </linearGradient> + <linearGradient + id="linearGradient3776" + osb:paint="solid"> + <stop + style="stop-color:#fc0000;stop-opacity:1;" + offset="0" + id="stop3774" /> + </linearGradient> + <linearGradient + id="linearGradient3438" + osb:paint="solid"> + <stop + style="stop-color:#b8e132;stop-opacity:1;" + offset="0" + id="stop3436" /> + </linearGradient> + <inkscape:path-effect + effect="spiro" + id="path-effect3408" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3404" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3400" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3392" + is_visible="true" /> + <inkscape:path-effect + effect="bspline" + id="path-effect3376" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect3044" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect3040" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect3036" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect3032" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect3028" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect3024" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3020" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect2858" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect2854" + is_visible="true" /> + <inkscape:path-effect + effect="bspline" + id="path-effect2844" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <linearGradient + id="linearGradient2828" + osb:paint="solid"> + <stop + style="stop-color:#ff0000;stop-opacity:1;" + offset="0" + id="stop2826" /> + </linearGradient> + <inkscape:path-effect + effect="bspline" + id="path-effect329" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <marker + inkscape:stockid="Arrow1Mstart" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mstart" + style="overflow:visible"> + <path + id="path4530" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(0.4,0,0,0.4,4,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend" + style="overflow:visible"> + <path + id="path4533" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <linearGradient + id="linearGradient4513"> + <stop + style="stop-color:#fdffdb;stop-opacity:1;" + offset="0" + id="stop4515" /> + <stop + style="stop-color:#dfe2d8;stop-opacity:0;" + offset="1" + id="stop4517" /> + </linearGradient> + <inkscape:perspective + sodipodi:type="inkscape:persp3d" + inkscape:vp_x="0 : 526.18109 : 1" + inkscape:vp_y="0 : 1000 : 0" + inkscape:vp_z="744.09448 : 526.18109 : 1" + inkscape:persp3d-origin="372.04724 : 350.78739 : 1" + id="perspective3876" /> + <inkscape:perspective + id="perspective3886" + inkscape:persp3d-origin="0.5 : 0.33333333 : 1" + inkscape:vp_z="1 : 0.5 : 1" + inkscape:vp_y="0 : 1000 : 0" + inkscape:vp_x="0 : 0.5 : 1" + sodipodi:type="inkscape:persp3d" /> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Lend" + style="overflow:visible"> + <path + id="path3211" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker3892" + style="overflow:visible"> + <path + id="path3894" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker3896" + style="overflow:visible"> + <path + id="path3898" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lstart" + orient="auto" + refY="0" + refX="0" + id="Arrow1Lstart" + style="overflow:visible"> + <path + id="path3208" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(0.8,0,0,0.8,10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker3902" + style="overflow:visible"> + <path + id="path3904" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lstart" + orient="auto" + refY="0" + refX="0" + id="marker3906" + style="overflow:visible"> + <path + id="path3908" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(0.8,0,0,0.8,10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker3910" + style="overflow:visible"> + <path + id="path3912" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:perspective + id="perspective4086" + inkscape:persp3d-origin="0.5 : 0.33333333 : 1" + inkscape:vp_z="1 : 0.5 : 1" + inkscape:vp_y="0 : 1000 : 0" + inkscape:vp_x="0 : 0.5 : 1" + sodipodi:type="inkscape:persp3d" /> + <inkscape:perspective + id="perspective4113" + inkscape:persp3d-origin="0.5 : 0.33333333 : 1" + inkscape:vp_z="1 : 0.5 : 1" + inkscape:vp_y="0 : 1000 : 0" + inkscape:vp_x="0 : 0.5 : 1" + sodipodi:type="inkscape:persp3d" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient4513" + id="linearGradient4519" + x1="47.142857" + y1="244.50504" + x2="677.85718" + y2="244.50504" + gradientUnits="userSpaceOnUse" + gradientTransform="matrix(0.99477436,0,0,0.98597786,2.8382132,3.7730937)" /> + <inkscape:perspective + id="perspective5195" + inkscape:persp3d-origin="0.5 : 0.33333333 : 1" + inkscape:vp_z="1 : 0.5 : 1" + inkscape:vp_y="0 : 1000 : 0" + inkscape:vp_x="0 : 0.5 : 1" + sodipodi:type="inkscape:persp3d" /> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-4" + style="overflow:visible"> + <path + id="path4533-7" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:perspective + id="perspective5272" + inkscape:persp3d-origin="0.5 : 0.33333333 : 1" + inkscape:vp_z="1 : 0.5 : 1" + inkscape:vp_y="0 : 1000 : 0" + inkscape:vp_x="0 : 0.5 : 1" + sodipodi:type="inkscape:persp3d" /> + <marker + inkscape:stockid="Arrow1Mstart" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mstart-4" + style="overflow:visible"> + <path + id="path4530-5" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(0.4,0,0,0.4,4,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-0" + style="overflow:visible"> + <path + id="path4533-3" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:perspective + id="perspective5317" + inkscape:persp3d-origin="0.5 : 0.33333333 : 1" + inkscape:vp_z="1 : 0.5 : 1" + inkscape:vp_y="0 : 1000 : 0" + inkscape:vp_x="0 : 0.5 : 1" + sodipodi:type="inkscape:persp3d" /> + <marker + inkscape:stockid="Arrow1Mstart" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mstart-3" + style="overflow:visible"> + <path + id="path4530-2" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(0.4,0,0,0.4,4,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-06" + style="overflow:visible"> + <path + id="path4533-1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Mstart" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mstart-8" + style="overflow:visible"> + <path + id="path4530-7" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(0.4,0,0,0.4,4,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-9" + style="overflow:visible"> + <path + id="path4533-2" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:path-effect + effect="spiro" + id="path-effect2858-0" + is_visible="true" /> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-3" + style="overflow:visible"> + <path + id="path4533-75" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:path-effect + effect="bspline" + id="path-effect3044-9" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-3-2" + style="overflow:visible"> + <path + id="path4533-75-8" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:path-effect + effect="bspline" + id="path-effect3044-9-9" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3808" + id="linearGradient3810" + x1="61.233804" + y1="153.47966" + x2="308.87187" + y2="153.47966" + gradientUnits="userSpaceOnUse" + gradientTransform="matrix(0.97704237,0,0,1.0002563,1.4114958,-0.03933915)" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3993" + id="linearGradient3995" + x1="155.21328" + y1="231.61366" + x2="207.95523" + y2="231.61366" + gradientUnits="userSpaceOnUse" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3438" + id="linearGradient4612" + x1="594.77722" + y1="232.19244" + x2="647.51917" + y2="232.19244" + gradientUnits="userSpaceOnUse" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3438" + id="linearGradient4614" + x1="530.03839" + y1="232.3177" + x2="582.78033" + y2="232.3177" + gradientUnits="userSpaceOnUse" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3438" + id="linearGradient4616" + x1="468.32343" + y1="232.3177" + x2="521.06543" + y2="232.3177" + gradientUnits="userSpaceOnUse" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3438" + id="linearGradient4618" + x1="405.4682" + y1="232.36095" + x2="458.21014" + y2="232.36095" + gradientUnits="userSpaceOnUse" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient2963" + id="linearGradient2965" + x1="49.239535" + y1="244.84964" + x2="677.6483" + y2="244.84964" + gradientUnits="userSpaceOnUse" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3438" + id="linearGradient2971" + x1="372.12488" + y1="333.32864" + x2="476.58179" + y2="333.32864" + gradientUnits="userSpaceOnUse" + gradientTransform="translate(95.459415,-18.384776)" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3008-3" + is_visible="true" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3114" + id="linearGradient3104" + x1="570.76388" + y1="265.59842" + x2="613.45706" + y2="265.59842" + gradientUnits="userSpaceOnUse" + gradientTransform="translate(-4)" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3114" + id="linearGradient3106" + x1="540.94574" + y1="265.3059" + x2="548.29097" + y2="265.3059" + gradientUnits="userSpaceOnUse" + gradientTransform="translate(-2)" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3114" + id="linearGradient3108" + x1="429.95148" + y1="274.65289" + x2="467.35629" + y2="274.65289" + gradientUnits="userSpaceOnUse" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3114" + id="linearGradient3116" + gradientUnits="userSpaceOnUse" + x1="481.13927" + y1="264.7722" + x2="491.08518" + y2="264.7722" + gradientTransform="translate(0,2)" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3120-7" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3120-7-3" + is_visible="true" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3993" + id="linearGradient6547" + x1="568.54004" + y1="280.793" + x2="630.64801" + y2="280.793" + gradientUnits="userSpaceOnUse" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3993" + id="linearGradient6549" + x1="499.98608" + y1="268.21176" + x2="522.65869" + y2="268.21176" + gradientUnits="userSpaceOnUse" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3993" + id="linearGradient6551" + x1="449.72733" + y1="267.29733" + x2="480.36688" + y2="267.29733" + gradientUnits="userSpaceOnUse" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3993" + id="linearGradient6553" + x1="554.2403" + y1="266.57718" + x2="565.97662" + y2="266.57718" + gradientUnits="userSpaceOnUse" /> + </defs> + <sodipodi:namedview + id="base" + pagecolor="#ffffff" + bordercolor="#666666" + borderopacity="1.0" + inkscape:pageopacity="0.0" + inkscape:pageshadow="2" + inkscape:zoom="1.4142136" + inkscape:cx="339.80121" + inkscape:cy="150.23618" + inkscape:document-units="px" + inkscape:current-layer="layer1" + showgrid="false" + inkscape:window-width="1360" + inkscape:window-height="724" + inkscape:window-x="0" + inkscape:window-y="20" + inkscape:window-maximized="0" + fit-margin-top="0.1" + fit-margin-left="0.1" + fit-margin-right="0.1" + fit-margin-bottom="0.1" + inkscape:measure-start="-29.078,219.858" + inkscape:measure-end="346.809,219.858" + showguides="false" /> + <metadata + id="metadata3873"> + <rdf:RDF> + <cc:Work + rdf:about=""> + <dc:format>image/svg+xml</dc:format> + <dc:type + rdf:resource="http://purl.org/dc/dcmitype/StillImage" /> + <dc:title></dc:title> + </cc:Work> + </rdf:RDF> + </metadata> + <g + inkscape:label="Layer 1" + inkscape:groupmode="layer" + id="layer1" + transform="translate(-46.542857,-100.33361)"> + <rect + style="fill:url(#linearGradient4519);fill-opacity:1;stroke:url(#linearGradient2965);stroke-width:0.99036628;stroke-opacity:1" + id="rect3697" + width="627.4184" + height="283.11649" + x="49.734718" + y="103.2914" + rx="0" + ry="0" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="436.47687" + y="382.4664" + id="text2912" + inkscape:export-filename="/home/matz/barracuda/rapports/mbuf-api-v2-images/octeon_multi.png" + inkscape:export-xdpi="112" + inkscape:export-ydpi="112"><tspan + sodipodi:role="line" + x="436.47687" + y="382.4664" + id="tspan2916" + style="font-weight:bold;font-size:13.33333302px;line-height:1.25">test: order_atq(all types queue)</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="99.327995" + y="317.25745" + id="text2978" + inkscape:export-filename="/home/matz/barracuda/rapports/mbuf-api-v2-images/octeon_multi.png" + inkscape:export-xdpi="112" + inkscape:export-ydpi="112"><tspan + sodipodi:role="line" + x="99.327995" + y="317.25745" + id="tspan3006" + style="font-size:15.22520161px;line-height:1.25"> </tspan></text> + <rect + style="fill:#ffffff;fill-opacity:1;stroke:url(#linearGradient4614);stroke-width:1.02699995;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect87" + width="51.714954" + height="32.587509" + x="530.55188" + y="216.02396" + rx="11.6051" + ry="16.293755" /> + <rect + style="fill:#ffffff;fill-opacity:1;stroke:url(#linearGradient4612);stroke-width:1.02699995;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect87-3" + width="51.714954" + height="32.587509" + x="595.29071" + y="215.89868" + rx="11.6051" + ry="16.293755" /> + <rect + style="fill:#ffffff;fill-opacity:1;stroke:url(#linearGradient4616);stroke-width:1.02699995;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect87-6" + width="51.714954" + height="32.587509" + x="468.83694" + y="216.02396" + rx="11.6051" + ry="16.293755" /> + <rect + style="fill:#ffffff;fill-opacity:1;stroke:url(#linearGradient2971);stroke-width:1.02699995;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect128-7" + width="103.42992" + height="57.382355" + x="468.09781" + y="286.25269" + rx="8.5874901" + ry="10.712767" /> + <rect + style="fill:#ffffff;fill-opacity:1;stroke:url(#linearGradient4618);stroke-width:1.02699995;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect87-6-5" + width="51.714954" + height="32.587509" + x="405.98169" + y="216.06718" + rx="11.6051" + ry="16.293755" /> + <rect + style="fill:#ffffff;fill-opacity:1;stroke:url(#linearGradient3995);stroke-width:1.02699995;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect87-6-5-3" + width="51.714954" + height="32.587509" + x="155.72678" + y="215.3199" + rx="11.6051" + ry="16.293755" /> + <path + style="fill:none;stroke:#009587;stroke-width:0.93883556;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#Arrow1Mend)" + d="m 183.25449,249.08205 c 3.74662,22.85489 17.60919,43.86172 37.14916,56.29446 18.31316,11.65216 40.37703,15.62026 61.91526,18.31267 12.34123,1.54273 24.72858,2.74691 37.14916,3.39123 13.45494,0.69797 26.93497,0.73841 40.40786,0.67825 36.04931,-0.16098 72.09541,-1.04105 108.10962,-2.63952" + id="path2852" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect2854" + inkscape:original-d="m 183.25449,249.08205 c 12.38397,18.76387 24.76701,37.52867 37.14916,56.29446 12.38211,18.76578 41.27777,12.20748 61.91526,18.31267 20.6375,6.10516 24.76702,2.25985 37.14916,3.39123 12.38215,1.13134 26.9395,0.4512 40.40786,0.67825 13.46837,0.22702 106.15533,-2.64046 108.10962,-2.63952" + sodipodi:nodetypes="csscsc" /> + <g + id="g4374"> + <text + id="text5219-3" + y="187.92023" + x="132.8121" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + xml:space="preserve"><tspan + style="font-size:10px;line-height:1.25" + id="tspan5223-6" + y="187.92023" + x="132.8121" + sodipodi:role="line">producer_flow_seq</tspan></text> + <g + id="g4286"> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="67.609619" + y="125.91534" + id="text5219"><tspan + sodipodi:role="line" + x="67.609619" + y="125.91534" + id="tspan5223" + style="font-size:10px;line-height:1.25">producer maintains per flow sequence number</tspan></text> + <rect + style="fill:#ffffff;fill-opacity:1;fill-rule:evenodd;stroke:url(#linearGradient3810);stroke-width:0.97884095;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect2896" + width="240.98547" + height="44.122215" + x="61.723225" + y="131.41856" + ry="8.8282356" + rx="9.0800323" + inkscape:export-filename="/home/matz/barracuda/rapports/mbuf-api-v2-images/octeon_multi.png" + inkscape:export-xdpi="112" + inkscape:export-ydpi="112" /> + <rect + style="fill:none;fill-opacity:1;stroke:#6ba6fd;stroke-width:1.06814909;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect3736" + width="39.065548" + height="24.347494" + x="70.045547" + y="143.98941" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="76.606445" + y="141.62436" + id="text5219-1-9"><tspan + sodipodi:role="line" + x="76.606445" + y="141.62436" + id="tspan5223-2-3" + style="font-size:10px;line-height:1.25">flow 0</tspan></text> + <rect + style="fill:none;fill-opacity:1;stroke:#6ba6fd;stroke-width:1.06814909;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect3736-8" + width="39.065548" + height="24.347494" + x="129.42143" + y="144.7206" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="131.98233" + y="142.35555" + id="text5219-1-9-4"><tspan + sodipodi:role="line" + x="131.98233" + y="142.35555" + id="tspan5223-2-3-5" + style="font-size:10px;line-height:1.25">flow 1</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="195.98233" + y="142.35555" + id="text5219-1-9-4-3"><tspan + sodipodi:role="line" + x="195.98233" + y="142.35555" + id="tspan5223-2-3-5-6" + style="font-size:10px;line-height:1.25">flow 2</tspan></text> + <rect + style="fill:none;fill-opacity:1;stroke:#6ba6fd;stroke-width:1.06814909;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect3736-8-0-1" + width="39.065548" + height="24.347494" + x="251.42145" + y="144.7206" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="257.98233" + y="142.35555" + id="text5219-1-9-4-3-0"><tspan + sodipodi:role="line" + x="257.98233" + y="142.35555" + id="tspan5223-2-3-5-6-6" + style="font-size:10px;line-height:1.25">flow n</tspan></text> + <rect + style="fill:none;fill-opacity:1;stroke:#6ba6fd;stroke-width:1.06814909;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect3736-8-3" + width="39.065548" + height="24.347494" + x="192.15901" + y="144.7155" /> + </g> + </g> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="157.0374" + y="258.07278" + id="text5219-2"><tspan + sodipodi:role="line" + x="157.0374" + y="258.07278" + id="tspan5223-0" + style="font-size:10px;line-height:1.25">producer0</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="477.25565" + y="316.59613" + id="text5219-6"><tspan + sodipodi:role="line" + x="477.25565" + y="316.59613" + id="tspan5223-1" + style="font-size:10px;line-height:1.25">all_types_queue0</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="410.87885" + y="213.34842" + id="text5219-2-4"><tspan + sodipodi:role="line" + x="410.87885" + y="213.34842" + id="tspan5223-0-7" + style="font-size:10px;line-height:1.25">worker 0</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="157.44383" + y="236.49918" + id="text5219-2-6"><tspan + sodipodi:role="line" + x="157.44383" + y="236.49918" + id="tspan5223-0-9" + style="font-size:10px;line-height:1.25">port n+1</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="472.61508" + y="213.66943" + id="text5219-2-4-3"><tspan + sodipodi:role="line" + x="472.61508" + y="213.66943" + id="tspan5223-0-7-7" + style="font-size:10px;line-height:1.25">worker 1</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="534.61511" + y="213.66943" + id="text5219-2-4-3-4"><tspan + sodipodi:role="line" + x="534.61511" + y="213.66943" + id="tspan5223-0-7-7-5" + style="font-size:10px;line-height:1.25">worker 2</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="600.61511" + y="213.66943" + id="text5219-2-4-3-4-2"><tspan + sodipodi:role="line" + x="600.61511" + y="213.66943" + id="tspan5223-0-7-7-5-5" + style="font-size:10px;line-height:1.25">worker n</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="420.13348" + y="234.8974" + id="text5219-2-6-4"><tspan + sodipodi:role="line" + x="420.13348" + y="234.8974" + id="tspan5223-0-9-7" + style="font-size:10px;line-height:1.25">port 0</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="477.25241" + y="234.85495" + id="text5219-2-6-4-4"><tspan + sodipodi:role="line" + x="477.25241" + y="234.85495" + id="tspan5223-0-9-7-4" + style="font-size:10px;line-height:1.25">port 1</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="539.25244" + y="234.85495" + id="text5219-2-6-4-4-3"><tspan + sodipodi:role="line" + x="539.25244" + y="234.85495" + id="tspan5223-0-9-7-4-0" + style="font-size:10px;line-height:1.25">port 2</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="607.25244" + y="234.85495" + id="text5219-2-6-4-4-3-7"><tspan + sodipodi:role="line" + x="607.25244" + y="234.85495" + id="tspan5223-0-9-7-4-0-8" + style="font-size:10px;line-height:1.25">port n</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="478.92789" + y="188.00357" + id="text5219-3-2"><tspan + sodipodi:role="line" + x="478.92789" + y="188.00357" + id="tspan5223-6-7" + style="font-size:10px;line-height:1.25">expected_flow_seq</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="433.7254" + y="125.99867" + id="text5219-26"><tspan + sodipodi:role="line" + x="433.7254" + y="125.99867" + id="tspan5223-10" + style="font-size:10px;line-height:1.25">per flow expected sequence number</tspan></text> + <rect + style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff2d00;stroke-width:0.97884095;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect2896-6" + width="240.98547" + height="44.122215" + x="407.83902" + y="131.50191" + ry="8.8282356" + rx="9.0800323" + inkscape:export-filename="/home/matz/barracuda/rapports/mbuf-api-v2-images/octeon_multi.png" + inkscape:export-xdpi="112" + inkscape:export-ydpi="112" /> + <rect + style="fill:none;fill-opacity:1;stroke:#ff2d00;stroke-width:1.06814909;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect3736-1" + width="39.065548" + height="24.347494" + x="416.16132" + y="144.07275" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="422.72223" + y="141.7077" + id="text5219-1-9-5"><tspan + sodipodi:role="line" + x="422.72223" + y="141.7077" + id="tspan5223-2-3-9" + style="font-size:10px;line-height:1.25">flow 0</tspan></text> + <rect + style="fill:none;fill-opacity:1;stroke:#ff2d00;stroke-width:1.06814909;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect3736-8-4" + width="39.065548" + height="24.347494" + x="475.5372" + y="144.80394" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="478.09811" + y="142.43889" + id="text5219-1-9-4-9"><tspan + sodipodi:role="line" + x="478.09811" + y="142.43889" + id="tspan5223-2-3-5-0" + style="font-size:10px;line-height:1.25">flow 1</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="542.09808" + y="142.43889" + id="text5219-1-9-4-3-9"><tspan + sodipodi:role="line" + x="542.09808" + y="142.43889" + id="tspan5223-2-3-5-6-1" + style="font-size:10px;line-height:1.25">flow 2</tspan></text> + <rect + style="fill:none;fill-opacity:1;stroke:#ff2d00;stroke-width:1.06814909;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect3736-8-0-1-7" + width="39.065548" + height="24.347494" + x="597.53723" + y="144.80394" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="604.09808" + y="142.43889" + id="text5219-1-9-4-3-0-7"><tspan + sodipodi:role="line" + x="604.09808" + y="142.43889" + id="tspan5223-2-3-5-6-6-1" + style="font-size:10px;line-height:1.25">flow n</tspan></text> + <rect + style="fill:none;fill-opacity:1;stroke:#ff2d00;stroke-width:1.06814909;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect3736-8-3-1" + width="39.065548" + height="24.347494" + x="538.27478" + y="144.79884" /> + <path + style="fill:none;stroke:#5cdcff;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-start:url(#Arrow1Mstart);marker-end:url(#Arrow1Mend)" + d="m 86.923031,168.93973 c 2.833543,14.16771 5.667239,28.33619 16.884859,38.84515 11.21761,10.50897 30.81628,17.35669 50.41543,24.20459" + id="path3022" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect3024" + inkscape:original-d="m 86.923031,168.93973 c 2.834697,14.16748 5.668393,28.33596 8.50109,42.50544 19.601799,6.84748 39.200469,13.6952 58.799199,20.5443" /> + <path + style="fill:none;stroke:#5cdcff;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Mend);marker-start:url(#Arrow1Mstart)" + d="m 146.43066,168.35658 c 2.36123,9.20881 4.72265,18.41832 9.20969,26.09352 4.48705,7.67519 11.09851,13.8144 17.71043,19.95404" + id="path3026" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect3028" + inkscape:original-d="m 146.43066,168.35658 c 2.36241,9.20851 4.72383,18.41802 7.08424,27.62854 6.61346,6.13914 13.22492,12.27835 19.83588,18.41902" /> + <path + style="fill:none;stroke:#5cdcff;stroke-width:0.81213671px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-start:url(#Arrow1Mstart);marker-end:url(#Arrow1Mend)" + d="m 217.48983,176.52088 c -8.64146,12.7325 -17.28354,25.46592 -25.92626,38.20028" + id="path3034" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect3036" + inkscape:original-d="m 217.48983,176.52088 c -8.64125,12.73264 -17.28334,25.46606 -25.92626,38.20028" /> + <path + style="fill:none;stroke:#5cdcff;stroke-width:0.86425042px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-start:url(#Arrow1Mstart);marker-end:url(#Arrow1Mend)" + d="m 272.10856,176.69086 c -0.83331,11.39414 -1.66669,22.78917 -12.50095,31.58588 -10.83426,8.79671 -31.66708,14.99352 -52.5026,21.19113" + id="path3038" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect3040" + inkscape:original-d="m 272.10856,176.69086 c -0.83249,11.3942 -1.66587,22.78923 -2.50014,34.18511 -20.83523,6.19695 -41.66805,12.39376 -62.50341,18.5919" /> + <path + style="fill:none;stroke:url(#linearGradient3108);stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Mend)" + d="m 467.27138,304.53077 c -4.47171,0.79203 -9.17911,0.19735 -13.31337,-1.68186 -4.13426,-1.8792 -7.67758,-5.03486 -10.02115,-8.92474 -2.70468,-4.48926 -3.7629,-9.74432 -4.94975,-14.84924 -2.2305,-9.59386 -5.06642,-19.04692 -8.48528,-28.28427" + id="path3040" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect3042" + inkscape:original-d="m 467.27138,304.53077 c -7.54147,-3.30083 -15.55535,-7.07207 -23.33452,-10.6066 -7.77917,-3.53453 -3.29883,-9.9005 -4.94975,-14.84924 -1.65091,-4.94875 -5.65585,-17.20727 -8.48528,-28.28427" + sodipodi:nodetypes="cscc" /> + <path + style="fill:none;stroke:url(#linearGradient3116);stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Mend)" + d="m 490.60591,286.02467 c -1.19028,-4.00346 -2.3688,-8.01042 -3.53554,-12.02081 -1.28128,-4.40407 -2.55618,-8.85645 -2.82842,-13.43503 -0.22685,-3.81532 0.25518,-7.67163 1.41421,-11.31371" + id="path3044" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect3046" + inkscape:original-d="m 490.60591,286.02467 c -1.17751,-3.53653 -2.35603,-8.01487 -3.53554,-12.02081 -1.17951,-4.00594 -1.88462,-8.95769 -2.82842,-13.43503 -0.94381,-4.47734 0.9438,-7.54347 1.41421,-11.31371" /> + <path + style="fill:none;stroke:url(#linearGradient3106);stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Mend)" + d="m 543.76023,283.31757 c -3.17461,-5.53504 -4.67076,-12.01835 -4.24264,-18.38478 0.38974,-5.79571 2.3658,-11.4769 5.65686,-16.26345" + id="path3048" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect3050" + inkscape:original-d="m 543.76023,283.31757 c -1.17751,-5.89356 -2.82742,-12.25752 -4.24264,-18.38478 -1.41521,-6.12726 3.77224,-10.8433 5.65686,-16.26345" /> + <path + style="fill:none;stroke:url(#linearGradient3104);stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Mend)" + d="m 567.52771,286.25269 c -0.89405,-7.05499 0.50327,-14.382 3.93101,-20.61279 3.42237,-6.22103 8.85117,-11.31764 15.27563,-14.34091 6.42445,-3.02328 13.81187,-3.95783 20.78681,-2.62965" + id="path3052" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect3054" + inkscape:original-d="m 567.52771,286.25269 c 1.31134,-6.87193 2.62167,-13.74286 3.93101,-20.61279 1.30933,-6.86993 24.04263,-11.31471 36.06244,-16.97056" /> + <path + style="fill:none;stroke:url(#linearGradient6551);stroke-width:1.23147655px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Mend-3)" + d="m 449.95502,247.97701 c 7.55606,3.00738 14.27612,8.08523 19.23272,14.53274 4.94601,6.43374 8.12285,14.21372 9.09385,22.27059" + id="path3118" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect3120" + inkscape:original-d="m 449.95502,247.97701 c 6.41168,4.84269 12.82258,9.68693 19.23272,14.53274 6.41012,4.84582 6.06333,14.84549 9.09385,22.27059" /> + <path + style="fill:none;stroke:url(#linearGradient6549);stroke-width:1.02635109px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Mend-3)" + d="m 500.27242,249.30514 c 5.49861,3.69701 10.16955,8.61776 13.57532,14.30137 3.95545,6.60092 6.18818,14.22417 6.41885,21.91602" + id="path3118-5" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect3120-7" + inkscape:original-d="m 500.27242,249.30514 c 4.52565,4.76559 9.05076,9.5327 13.57532,14.30137 4.52456,4.76867 4.27978,14.60913 6.41885,21.91602" /> + <path + style="fill:none;stroke:url(#linearGradient6553);stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Mend-3)" + d="m 563.4379,247.96223 c -0.93075,1.47255 -1.40195,3.23109 -1.33217,4.97173 0.0873,2.17847 0.98613,4.22982 1.82529,6.24207 0.83917,2.01226 1.64627,4.12194 1.53517,6.29933 -0.10557,2.06901 -1.03996,4.01595 -2.21955,5.71904 -1.17958,1.70309 -2.61086,3.2153 -3.88281,4.85056 -1.79899,2.31284 -3.27787,4.87432 -4.38135,7.58871" + id="path3158" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect3160" + inkscape:original-d="m 563.4379,247.96223 c -0.44305,1.65624 -0.88711,3.31349 -1.33217,4.97173 -0.44505,1.65824 2.24131,8.35993 3.36046,12.5414 1.11915,4.18146 -4.06724,7.0454 -6.10236,10.5696 -2.03512,3.5242 -2.9199,5.05814 -4.38135,7.58871" /> + <path + style="fill:none;stroke:url(#linearGradient6547);stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Mend-9)" + d="m 627.07751,247.25512 c 2.57858,5.21574 3.57603,11.20045 2.82843,16.97056 -0.64544,4.9816 -2.54874,9.72988 -4.94975,14.14214 -5.34434,9.82114 -13.26591,18.22509 -22.75437,24.13997 -9.48846,5.91488 -20.52182,9.327 -31.69285,9.80115" + id="path3170" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect3172" + inkscape:original-d="m 627.07751,247.25512 c 0.94381,5.65586 1.88662,11.31271 2.82843,16.97056 0.94181,5.65786 -3.29883,9.42709 -4.94975,14.14214 -1.65091,4.71505 -36.29715,22.62642 -54.44722,33.94112" /> + <path + style="fill:none;stroke:#ff0009;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Mend-3)" + d="m 472.28159,286.78034 c -4.73891,1.38236 -9.8908,1.31285 -14.59068,-0.19687 -4.69989,-1.50972 -8.9285,-4.45346 -11.97588,-8.33697 -4.6972,-5.98601 -6.39497,-13.73104 -7.77817,-21.2132 -4.74217,-25.65195 -7.34684,-51.69871 -7.77817,-77.78175" + id="path3174" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect3176" + inkscape:original-d="m 472.28159,286.78034 c -8.85452,-2.84561 -17.71004,-5.69023 -26.56656,-8.53384 -8.85651,-2.84361 -5.18444,-18.15007 -7.77817,-21.2132 -2.59372,-3.06313 -5.18445,-47.84856 -7.77817,-77.78175" + sodipodi:nodetypes="cscc" /> + <path + style="fill:none;stroke:#ff0009;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Mend-3)" + d="m 520.26659,285.52253 c -5.12949,-4.21044 -10.95341,-7.57288 -17.1645,-9.90993 -1.06939,-0.40238 -2.15342,-0.77603 -3.17262,-1.29248 -1.0192,-0.51645 -1.98094,-1.18681 -2.68299,-2.08826 -0.72153,-0.92647 -1.14059,-2.06537 -1.31508,-3.22662 -0.1745,-1.16126 -0.1134,-2.34757 0.0547,-3.50977 0.33614,-2.32441 1.09651,-4.58378 1.26041,-6.92664 0.17202,-2.45897 -0.32204,-4.92427 -1.08174,-7.26926 -0.75971,-2.34499 -1.78291,-4.59423 -2.70916,-6.87857 -3.13866,-7.7406 -5.16733,-15.90124 -6.47139,-24.15154 -2.35876,-14.92295 -2.35876,-30.21628 0,-45.13923" + id="path3178" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect3180" + inkscape:original-d="m 520.26659,285.52253 c -6.30121,-1.68967 -11.442,-6.60762 -17.1645,-9.90993 -5.7225,-3.30231 -3.90274,-2.25483 -5.85561,-3.38074 -1.95287,-1.12591 10e-4,-9.10969 0,-13.66303 -0.001,-4.55334 -2.52627,-9.43288 -3.7909,-14.14783 -1.26463,-4.71494 -4.31326,-16.10203 -6.47139,-24.15154 -2.15813,-8.04952 10e-4,-30.09382 0,-45.13923" /> + <path + style="fill:none;stroke:#ff0009;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Mend-3)" + d="m 550.98248,285.63367 c -2.92905,-0.67285 -5.54573,-2.60689 -7.0484,-5.20959 -1.50267,-2.60269 -1.86925,-5.83582 -0.98743,-8.70888 0.60067,-1.95707 1.7332,-3.7028 2.90087,-5.38431 1.16766,-1.68151 2.39383,-3.34436 3.22004,-5.21741 1.05624,-2.39454 1.4169,-5.05627 1.32027,-7.67164 -0.0966,-2.61537 -0.63688,-5.1959 -1.32027,-7.72225 -2.02251,-7.47675 -5.29434,-14.54655 -7.92814,-21.83047 -2.63379,-7.28391 -4.65127,-14.98425 -4.00448,-22.70266 0.8282,-9.88322 6.25638,-19.28511 14.4014,-24.94396" + id="path3182" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect3184" + inkscape:original-d="m 550.98248,285.63367 c -2.67761,-4.64049 -5.35622,-9.27998 -8.03583,-13.91847 -2.67961,-4.63849 4.0816,-7.06881 6.12091,-10.60172 2.0393,-3.53291 0.001,-10.26359 0,-15.39389 -10e-4,-5.1303 -7.95408,-29.68975 -11.93262,-44.53313 -3.97854,-14.84337 9.60194,-16.63031 14.4014,-24.94396" /> + <path + style="fill:none;stroke:#ff0009;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Mend-3)" + d="m 570.50897,312.30894 15.9099,-15.9099 c 1.60179,-1.60179 3.18026,-3.22794 4.83149,-4.77871 1.65122,-1.55077 3.4059,-3.02641 5.42156,-4.06012 3.98852,-2.04548 8.73787,-2.20014 12.72792,-4.24264 2.36474,-1.21051 4.3875,-3.06569 5.84524,-5.28657 1.45774,-2.22089 2.35254,-4.80039 2.64004,-7.44135 0.22981,-2.11099 0.0784,-4.24195 0,-6.36397 -0.10438,-2.827 -0.0784,-5.65744 0,-8.48528 0.10462,-3.77187 0.30241,-7.55251 0,-11.3137 -0.66504,-8.27138 -3.7123,-16.13228 -6.42402,-23.97477 -4.92134,-14.23288 -8.82892,-28.81618 -11.68335,-43.60288" + id="path3186" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect3188" + inkscape:original-d="m 570.50897,312.30894 c 5.3043,-5.3043 10.6076,-10.6076 15.9099,-15.9099 5.3023,-5.3023 6.83637,-5.89355 10.25305,-8.83883 3.41668,-2.94528 8.48628,-2.82943 12.72792,-4.24264 4.24164,-1.41322 5.65786,-8.48628 8.48528,-12.72792 2.82743,-4.24164 10e-4,-4.24364 0,-6.36397 -10e-4,-2.12032 10e-4,-5.65785 0,-8.48528 -10e-4,-2.82742 10e-4,-7.54347 0,-11.3137 -10e-4,-3.77024 -4.28168,-15.98418 -6.42402,-23.97477 -2.14234,-7.99059 -7.7879,-29.06959 -11.68335,-43.60288" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="274.48175" + y="311.57025" + id="text5219-2-62"><tspan + sodipodi:role="line" + x="274.48175" + y="311.57025" + id="tspan5223-0-91" + style="font-size:10px;line-height:1.25">dequeue_ordered_flow(step 2)</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="230.38838" + y="349.16824" + id="text5219-2-62-2"><tspan + sodipodi:role="line" + x="230.38838" + y="349.16824" + id="tspan5223-0-91-7" + style="font-size:10px;line-height:1.25">enqueue ordered flow(step 1)</tspan></text> + <path + style="fill:none;stroke:#000000;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;stroke-miterlimit:4;stroke-dasharray:1, 2;stroke-dashoffset:0;marker-end:url(#Arrow1Mend)" + d="m 284.13073,339.88611 c 0.17405,-3.87643 1.69528,-7.6795 4.24264,-10.6066 1.21368,-1.3946 2.65204,-2.59324 4.24264,-3.53554" + id="path3226" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect3228" + inkscape:original-d="m 284.13073,339.88611 c 1.65092,-3.53654 2.82943,-7.07207 4.24264,-10.6066 1.41322,-3.53454 2.82943,-2.35803 4.24264,-3.53554" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="54.031021" + y="305.13019" + id="text5219-2-62-2-0"><tspan + sodipodi:role="line" + x="54.031021" + y="305.13019" + id="tspan5223-0-91-7-9" + style="font-size:10px;line-height:1.25">produce ordered flows(step 0)</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="213.00854" + y="270.78644" + id="text5219-2-62-3"><tspan + sodipodi:role="line" + x="213.00854" + y="270.78644" + id="tspan5223-0-91-6" + style="font-size:10px;line-height:1.25">change to atomic flow and enqueue(step 3)</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="245.69855" + y="229.93423" + id="text5219-2-62-3-0"><tspan + sodipodi:role="line" + x="245.69855" + y="229.93423" + id="tspan5223-0-91-6-6" + style="font-size:10px;line-height:1.25">dequeue_atomic_flow (step 4)</tspan></text> + <path + style="fill:none;stroke:#000000;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:1, 2;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#Arrow1Mend)" + d="m 125.03171,296.7526 c 3.74786,-3.82704 6.25815,-8.84762 7.07106,-14.14214 0.89616,-5.83674 -0.22472,-11.84652 0.70712,-17.67767 0.88602,-5.54438 3.67535,-10.76654 7.79086,-14.58594 4.11551,-3.81939 9.53103,-6.21176 15.12603,-6.68208" + id="path3284" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect3286" + inkscape:original-d="m 125.03171,296.7526 c 2.35802,-4.71505 4.71504,-9.42909 7.07106,-14.14214 2.35603,-4.71304 -4.47734,-16.97156 0.70712,-17.67767 5.18445,-0.70611 8.30435,-19.6191 22.91689,-21.26802" + sodipodi:nodetypes="ccsc" /> + <path + style="fill:none;stroke:#000000;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;stroke-miterlimit:4;stroke-dasharray:1, 2;stroke-dashoffset:0;marker-end:url(#Arrow1Mend)" + d="m 400.09624,301.70234 c 2.99719,-1.5536 6.06561,-2.9698 9.19239,-4.24264 10.36506,-4.21939 21.37433,-6.85204 32.52691,-7.77817" + id="path3288" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect3290" + inkscape:original-d="m 400.09624,301.70234 c 3.06513,-1.41521 6.12926,-2.82942 9.19239,-4.24264 3.06313,-1.41321 21.68561,-5.18645 32.52691,-7.77817" /> + <path + style="fill:none;stroke:#000000;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;stroke-miterlimit:4;stroke-dasharray:1, 2;stroke-dashoffset:0;marker-end:url(#Arrow1Mend)" + d="m 366.15512,272.71097 c 3.81527,2.26146 7.84644,4.15848 12.02081,5.65685 11.69951,4.19948 24.37655,5.20587 36.76955,4.24264 17.71147,-1.3766 35.17977,-6.78471 50.20458,-16.26345 1.43767,-0.90698 2.85248,-1.85019 4.24264,-2.82843" + id="path3292" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect3294" + inkscape:original-d="m 366.15512,272.71097 c 4.00793,1.88461 8.01487,3.77023 12.02081,5.65685 4.00594,1.88662 24.51404,2.82743 36.76955,4.24264 12.25552,1.41521 33.47072,-10.8433 50.20458,-16.26345 16.73386,-5.42016 2.82943,-1.88662 4.24264,-2.82843" /> + <path + style="fill:none;stroke:#000000;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;stroke-miterlimit:4;stroke-dasharray:1, 2;stroke-dashoffset:0;marker-end:url(#Arrow1Mend)" + d="m 336.45663,221.09217 c 2.28482,-4.91581 5.69736,-9.30336 9.8995,-12.72792 8.26499,-6.7356 19.09721,-9.47021 29.69848,-10.6066 11.02462,-1.18177 22.14702,-0.83857 33.23402,-0.70711 6.83505,0.081 13.67105,0.081 20.5061,0" + id="path3300" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect3302" + inkscape:original-d="m 336.45663,221.09217 c 3.30083,-4.24364 6.60067,-8.48628 9.8995,-12.72792 3.29883,-4.24164 19.79999,-7.07207 29.69848,-10.6066 9.8985,-3.53454 22.15701,-0.47241 33.23402,-0.70711 11.07701,-0.2347 13.67173,-0.001 20.5061,0" /> + </g> +</svg> diff --git a/doc/guides/tools/img/eventdev_order_queue_test.svg b/doc/guides/tools/img/eventdev_order_queue_test.svg new file mode 100644 index 00000000..3e2113cd --- /dev/null +++ b/doc/guides/tools/img/eventdev_order_queue_test.svg @@ -0,0 +1,1673 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<!-- Created with Inkscape (http://www.inkscape.org/) --> + +<!-- +# BSD LICENSE +# +# Copyright (c) 2017, Cavium, Inc +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# +# - Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# +# - Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in +# the documentation and/or other materials provided with the +# distribution. +# +# - Neither the name of Cavium, Inc nor the names of its +# contributors may be used to endorse or promote products derived +# from this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS +# FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE +# COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, +# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES +# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +# STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) +# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED +# OF THE POSSIBILITY OF SUCH DAMAGE. +--> + +<svg + xmlns:osb="http://www.openswatchbook.org/uri/2009/osb" + xmlns:dc="http://purl.org/dc/elements/1.1/" + xmlns:cc="http://creativecommons.org/ns#" + xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" + xmlns:svg="http://www.w3.org/2000/svg" + xmlns="http://www.w3.org/2000/svg" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" + xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" + width="631.91431" + height="288.34286" + id="svg3868" + version="1.1" + inkscape:version="0.92.1 r" + sodipodi:docname="order_queue.svg" + sodipodi:version="0.32" + inkscape:output_extension="org.inkscape.output.svg.inkscape"> + <defs + id="defs3870"> + <linearGradient + id="linearGradient6425" + osb:paint="solid"> + <stop + style="stop-color:#e6860b;stop-opacity:1;" + offset="0" + id="stop6423" /> + </linearGradient> + <linearGradient + id="linearGradient6391" + osb:paint="solid"> + <stop + style="stop-color:#000000;stop-opacity:1;" + offset="0" + id="stop6389" /> + </linearGradient> + <inkscape:path-effect + effect="spiro" + id="path-effect6387" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect6037" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect6033" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect6029" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect6025" + is_visible="true" /> + <linearGradient + id="linearGradient5213" + osb:paint="solid"> + <stop + style="stop-color:#ff0009;stop-opacity:1;" + offset="0" + id="stop5211" /> + </linearGradient> + <inkscape:path-effect + effect="spiro" + id="path-effect4276" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect4272" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect4268" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect4264" + is_visible="true" /> + <linearGradient + id="linearGradient2975" + osb:paint="solid"> + <stop + style="stop-color:#ff2200;stop-opacity:1;" + offset="0" + id="stop2973" /> + </linearGradient> + <linearGradient + id="linearGradient2969" + osb:paint="solid"> + <stop + style="stop-color:#009a08;stop-opacity:1;" + offset="0" + id="stop2967" /> + </linearGradient> + <linearGradient + id="linearGradient2963" + osb:paint="solid"> + <stop + style="stop-color:#000000;stop-opacity:1;" + offset="0" + id="stop2961" /> + </linearGradient> + <linearGradient + id="linearGradient2929" + osb:paint="solid"> + <stop + style="stop-color:#ff2d00;stop-opacity:1;" + offset="0" + id="stop2927" /> + </linearGradient> + <linearGradient + id="linearGradient4610" + osb:paint="solid"> + <stop + style="stop-color:#00ffff;stop-opacity:1;" + offset="0" + id="stop4608" /> + </linearGradient> + <linearGradient + id="linearGradient3993" + osb:paint="solid"> + <stop + style="stop-color:#6ba6fd;stop-opacity:1;" + offset="0" + id="stop3991" /> + </linearGradient> + <linearGradient + id="linearGradient3808" + osb:paint="solid"> + <stop + style="stop-color:#6ba6fd;stop-opacity:1;" + offset="0" + id="stop3806" /> + </linearGradient> + <linearGradient + id="linearGradient3776" + osb:paint="solid"> + <stop + style="stop-color:#fc0000;stop-opacity:1;" + offset="0" + id="stop3774" /> + </linearGradient> + <linearGradient + id="linearGradient3438" + osb:paint="solid"> + <stop + style="stop-color:#d18f21;stop-opacity:1;" + offset="0" + id="stop3436" /> + </linearGradient> + <inkscape:path-effect + effect="spiro" + id="path-effect3408" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3404" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3400" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3392" + is_visible="true" /> + <inkscape:path-effect + effect="bspline" + id="path-effect3376" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect3044" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect3040" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect3036" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect3032" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect3028" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect3024" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3020" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect2858" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect2854" + is_visible="true" /> + <inkscape:path-effect + effect="bspline" + id="path-effect2844" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <linearGradient + id="linearGradient2828" + osb:paint="solid"> + <stop + style="stop-color:#ff0000;stop-opacity:1;" + offset="0" + id="stop2826" /> + </linearGradient> + <inkscape:path-effect + effect="bspline" + id="path-effect329" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <marker + inkscape:stockid="Arrow1Mstart" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mstart" + style="overflow:visible"> + <path + id="path4530" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(0.4,0,0,0.4,4,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend" + style="overflow:visible"> + <path + id="path4533" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <linearGradient + id="linearGradient4513"> + <stop + style="stop-color:#fdffdb;stop-opacity:1;" + offset="0" + id="stop4515" /> + <stop + style="stop-color:#dfe2d8;stop-opacity:0;" + offset="1" + id="stop4517" /> + </linearGradient> + <inkscape:perspective + sodipodi:type="inkscape:persp3d" + inkscape:vp_x="0 : 526.18109 : 1" + inkscape:vp_y="0 : 1000 : 0" + inkscape:vp_z="744.09448 : 526.18109 : 1" + inkscape:persp3d-origin="372.04724 : 350.78739 : 1" + id="perspective3876" /> + <inkscape:perspective + id="perspective3886" + inkscape:persp3d-origin="0.5 : 0.33333333 : 1" + inkscape:vp_z="1 : 0.5 : 1" + inkscape:vp_y="0 : 1000 : 0" + inkscape:vp_x="0 : 0.5 : 1" + sodipodi:type="inkscape:persp3d" /> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Lend" + style="overflow:visible"> + <path + id="path3211" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker3892" + style="overflow:visible"> + <path + id="path3894" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker3896" + style="overflow:visible"> + <path + id="path3898" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lstart" + orient="auto" + refY="0" + refX="0" + id="Arrow1Lstart" + style="overflow:visible"> + <path + id="path3208" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(0.8,0,0,0.8,10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker3902" + style="overflow:visible"> + <path + id="path3904" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lstart" + orient="auto" + refY="0" + refX="0" + id="marker3906" + style="overflow:visible"> + <path + id="path3908" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(0.8,0,0,0.8,10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker3910" + style="overflow:visible"> + <path + id="path3912" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:perspective + id="perspective4086" + inkscape:persp3d-origin="0.5 : 0.33333333 : 1" + inkscape:vp_z="1 : 0.5 : 1" + inkscape:vp_y="0 : 1000 : 0" + inkscape:vp_x="0 : 0.5 : 1" + sodipodi:type="inkscape:persp3d" /> + <inkscape:perspective + id="perspective4113" + inkscape:persp3d-origin="0.5 : 0.33333333 : 1" + inkscape:vp_z="1 : 0.5 : 1" + inkscape:vp_y="0 : 1000 : 0" + inkscape:vp_x="0 : 0.5 : 1" + sodipodi:type="inkscape:persp3d" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient4513" + id="linearGradient4519" + x1="47.142857" + y1="244.50504" + x2="677.85718" + y2="244.50504" + gradientUnits="userSpaceOnUse" + gradientTransform="matrix(0.99477436,0,0,0.98597786,2.8382132,3.7730937)" /> + <inkscape:perspective + id="perspective5195" + inkscape:persp3d-origin="0.5 : 0.33333333 : 1" + inkscape:vp_z="1 : 0.5 : 1" + inkscape:vp_y="0 : 1000 : 0" + inkscape:vp_x="0 : 0.5 : 1" + sodipodi:type="inkscape:persp3d" /> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-4" + style="overflow:visible"> + <path + id="path4533-7" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:perspective + id="perspective5272" + inkscape:persp3d-origin="0.5 : 0.33333333 : 1" + inkscape:vp_z="1 : 0.5 : 1" + inkscape:vp_y="0 : 1000 : 0" + inkscape:vp_x="0 : 0.5 : 1" + sodipodi:type="inkscape:persp3d" /> + <marker + inkscape:stockid="Arrow1Mstart" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mstart-4" + style="overflow:visible"> + <path + id="path4530-5" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(0.4,0,0,0.4,4,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-0" + style="overflow:visible"> + <path + id="path4533-3" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:perspective + id="perspective5317" + inkscape:persp3d-origin="0.5 : 0.33333333 : 1" + inkscape:vp_z="1 : 0.5 : 1" + inkscape:vp_y="0 : 1000 : 0" + inkscape:vp_x="0 : 0.5 : 1" + sodipodi:type="inkscape:persp3d" /> + <marker + inkscape:stockid="Arrow1Mstart" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mstart-3" + style="overflow:visible"> + <path + id="path4530-2" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(0.4,0,0,0.4,4,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-06" + style="overflow:visible"> + <path + id="path4533-1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Mstart" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mstart-8" + style="overflow:visible"> + <path + id="path4530-7" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(0.4,0,0,0.4,4,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-9" + style="overflow:visible"> + <path + id="path4533-2" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:path-effect + effect="spiro" + id="path-effect2858-0" + is_visible="true" /> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-3" + style="overflow:visible"> + <path + id="path4533-75" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:path-effect + effect="bspline" + id="path-effect3044-9" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-3-2" + style="overflow:visible"> + <path + id="path4533-75-8" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:path-effect + effect="bspline" + id="path-effect3044-9-9" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3808" + id="linearGradient3810" + x1="61.233804" + y1="153.47966" + x2="308.87187" + y2="153.47966" + gradientUnits="userSpaceOnUse" + gradientTransform="matrix(0.97704237,0,0,1.0002563,1.4114958,-0.03933915)" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3993" + id="linearGradient3995" + x1="155.21328" + y1="231.61366" + x2="207.95523" + y2="231.61366" + gradientUnits="userSpaceOnUse" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3438" + id="linearGradient4612" + x1="594.77722" + y1="232.19244" + x2="647.51917" + y2="232.19244" + gradientUnits="userSpaceOnUse" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3438" + id="linearGradient4614" + x1="530.03839" + y1="232.3177" + x2="582.78033" + y2="232.3177" + gradientUnits="userSpaceOnUse" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3438" + id="linearGradient4616" + x1="468.32343" + y1="232.3177" + x2="521.06543" + y2="232.3177" + gradientUnits="userSpaceOnUse" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3438" + id="linearGradient4618" + x1="405.4682" + y1="232.36095" + x2="458.21014" + y2="232.36095" + gradientUnits="userSpaceOnUse" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient2963" + id="linearGradient2965" + x1="49.239535" + y1="244.84964" + x2="677.6483" + y2="244.84964" + gradientUnits="userSpaceOnUse" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient2969" + id="linearGradient2971" + x1="372.12487" + y1="333.32863" + x2="476.58178" + y2="333.32863" + gradientUnits="userSpaceOnUse" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient2975" + id="linearGradient2977" + x1="558.08159" + y1="336.1407" + x2="662.53851" + y2="336.1407" + gradientUnits="userSpaceOnUse" /> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-2" + style="overflow:visible"> + <path + id="path4533-6" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:path-effect + effect="spiro" + id="path-effect3228" + is_visible="true" /> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="marker3706" + style="overflow:visible"> + <path + id="path3704" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:path-effect + effect="spiro" + id="path-effect3286" + is_visible="true" /> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-1" + style="overflow:visible"> + <path + id="path4533-8" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:path-effect + effect="spiro" + id="path-effect3290" + is_visible="true" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient2969" + id="linearGradient5215" + x1="474.25354" + y1="288.07208" + x2="607.70117" + y2="288.07208" + gradientUnits="userSpaceOnUse" + gradientTransform="translate(2.9619308,1.9381716)" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient2969" + id="linearGradient5217" + x1="475.90207" + y1="275.55313" + x2="550.59595" + y2="275.55313" + gradientUnits="userSpaceOnUse" + gradientTransform="matrix(1.0378669,0,0,1.0378669,-20.849369,-9.3151532)" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient2969" + id="linearGradient5219" + x1="430.01959" + y1="275.94962" + x2="483.12329" + y2="275.94962" + gradientUnits="userSpaceOnUse" + gradientTransform="matrix(1.0526015,0,0,1.1085927,-22.60217,-28.51638)" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient2969" + id="linearGradient5221" + x1="409.40347" + y1="274.47592" + x2="424.67188" + y2="274.47592" + gradientUnits="userSpaceOnUse" + gradientTransform="matrix(0.99688019,0,0,1.0540252,2.0081849,-13.414405)" /> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-3-7" + style="overflow:visible"> + <path + id="path4533-75-9" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:path-effect + effect="spiro" + id="path-effect3120-7" + is_visible="true" /> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-4-2" + style="overflow:visible"> + <path + id="path4533-7-0" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:path-effect + effect="spiro" + id="path-effect6025-2" + is_visible="true" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3993" + id="linearGradient6427" + x1="629.66772" + y1="279.10413" + x2="652.93823" + y2="279.10413" + gradientUnits="userSpaceOnUse" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3993" + id="linearGradient6429" + x1="548.02209" + y1="278.62817" + x2="594.85144" + y2="278.62817" + gradientUnits="userSpaceOnUse" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3993" + id="linearGradient6431" + x1="439.92499" + y1="294.88806" + x2="559.63593" + y2="294.88806" + gradientUnits="userSpaceOnUse" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3993" + id="linearGradient6433" + x1="483.44641" + y1="280.99118" + x2="564.04688" + y2="280.99118" + gradientUnits="userSpaceOnUse" /> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-7" + style="overflow:visible"> + <path + id="path4533-5" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:path-effect + effect="spiro" + id="path-effect3294" + is_visible="true" /> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-92" + style="overflow:visible"> + <path + id="path4533-28" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:path-effect + effect="spiro" + id="path-effect3302" + is_visible="true" /> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-97" + style="overflow:visible"> + <path + id="path4533-36" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:path-effect + effect="spiro" + id="path-effect3228-1" + is_visible="true" /> + </defs> + <sodipodi:namedview + id="base" + pagecolor="#ffffff" + bordercolor="#666666" + borderopacity="1.0" + inkscape:pageopacity="0.0" + inkscape:pageshadow="2" + inkscape:zoom="1.4142136" + inkscape:cx="361.03715" + inkscape:cy="144.93288" + inkscape:document-units="px" + inkscape:current-layer="layer1" + showgrid="false" + inkscape:window-width="1360" + inkscape:window-height="724" + inkscape:window-x="0" + inkscape:window-y="20" + inkscape:window-maximized="0" + fit-margin-top="0.1" + fit-margin-left="0.1" + fit-margin-right="0.1" + fit-margin-bottom="0.1" + inkscape:measure-start="-29.078,219.858" + inkscape:measure-end="346.809,219.858" + showguides="false" /> + <metadata + id="metadata3873"> + <rdf:RDF> + <cc:Work + rdf:about=""> + <dc:format>image/svg+xml</dc:format> + <dc:type + rdf:resource="http://purl.org/dc/dcmitype/StillImage" /> + <dc:title /> + </cc:Work> + </rdf:RDF> + </metadata> + <g + inkscape:label="Layer 1" + inkscape:groupmode="layer" + id="layer1" + transform="translate(-46.542857,-100.33361)"> + <rect + style="fill:url(#linearGradient4519);fill-opacity:1;stroke:url(#linearGradient2965);stroke-width:0.99036628;stroke-opacity:1" + id="rect3697" + width="627.4184" + height="283.11649" + x="49.734718" + y="103.2914" + rx="0" + ry="0" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="540.47687" + y="380.4664" + id="text2912" + inkscape:export-filename="/home/matz/barracuda/rapports/mbuf-api-v2-images/octeon_multi.png" + inkscape:export-xdpi="112" + inkscape:export-ydpi="112"><tspan + sodipodi:role="line" + x="540.47687" + y="380.4664" + id="tspan2916" + style="font-weight:bold;font-size:13.33333302px;line-height:1.25">test: order_queue</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="99.327995" + y="317.25745" + id="text2978" + inkscape:export-filename="/home/matz/barracuda/rapports/mbuf-api-v2-images/octeon_multi.png" + inkscape:export-xdpi="112" + inkscape:export-ydpi="112"><tspan + sodipodi:role="line" + x="99.327995" + y="317.25745" + id="tspan3006" + style="font-size:15.22520161px;line-height:1.25"> </tspan></text> + <rect + style="fill:#ffffff;fill-opacity:1;stroke:url(#linearGradient4614);stroke-width:1.02699995;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect87" + width="51.714954" + height="32.587509" + x="530.55188" + y="216.02396" + rx="11.6051" + ry="16.293755" /> + <rect + style="fill:#ffffff;fill-opacity:1;stroke:url(#linearGradient4612);stroke-width:1.02699995;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect87-3" + width="51.714954" + height="32.587509" + x="595.29071" + y="215.89868" + rx="11.6051" + ry="16.293755" /> + <rect + style="fill:#ffffff;fill-opacity:1;stroke:url(#linearGradient4616);stroke-width:1.02699995;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect87-6" + width="51.714954" + height="32.587509" + x="468.83694" + y="216.02396" + rx="11.6051" + ry="16.293755" /> + <rect + style="fill:#ffffff;fill-opacity:1;stroke:url(#linearGradient2977);stroke-width:1.02699995;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect128" + width="103.42992" + height="57.382355" + x="558.59509" + y="307.44952" + rx="8.5874901" + ry="10.712767" /> + <rect + style="fill:#ffffff;fill-opacity:1;stroke:url(#linearGradient2971);stroke-width:1.02699995;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect128-7" + width="103.42992" + height="57.382355" + x="372.63837" + y="304.63745" + rx="8.5874901" + ry="10.712767" /> + <rect + style="fill:#ffffff;fill-opacity:1;stroke:url(#linearGradient4618);stroke-width:1.02699995;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect87-6-5" + width="51.714954" + height="32.587509" + x="405.98169" + y="216.06718" + rx="11.6051" + ry="16.293755" /> + <rect + style="fill:#ffffff;fill-opacity:1;stroke:url(#linearGradient3995);stroke-width:1.02699995;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect87-6-5-3" + width="51.714954" + height="32.587509" + x="155.72678" + y="215.3199" + rx="11.6051" + ry="16.293755" /> + <path + style="fill:none;stroke:#009587;stroke-width:0.9931457;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#Arrow1Mend)" + d="m 181.86811,247.66582 c 3.58556,24.38192 18.0972,46.94673 38.79478,60.32374 19.0792,12.33104 42.1302,16.69577 64.65795,19.6234 12.88313,1.67425 25.82062,2.9633 38.79477,3.63396 14.05094,0.72632 28.12813,0.72676 42.19783,0.72679 2.04183,0 4.08366,0 6.12549,0" + id="path2852" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect2854" + inkscape:original-d="m 181.86811,247.66582 c 12.93255,20.10689 25.86414,40.2148 38.79478,60.32374 12.93062,20.10895 43.10626,13.08124 64.65795,19.6234 21.55169,6.54215 25.86414,2.42161 38.79477,3.63396 12.93064,1.21233 28.13285,0.4835 42.19783,0.72679 14.06498,0.24328 4.08462,-10e-4 6.12549,0" /> + <path + style="fill:none;stroke:#f70000;stroke-width:0.981;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;stroke-miterlimit:4;stroke-dasharray:none;marker-end:url(#Arrow1Mend);marker-start:" + d="m 640.02293,307.22481 c -12.09421,-24.58854 -24.27852,-49.36025 -30.348,-70.97018 -6.06948,-21.60992 -6.06948,-40.14987 -6.06948,-58.68854" + id="path3042" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect3044" + inkscape:original-d="m 640.02293,307.22481 c -11.90643,-24.6809 -24.27734,-49.36083 -36.41748,-74.03977 9.8e-4,-18.54033 9.8e-4,-37.08028 0,-55.61895" /> + <path + style="stroke:#f90000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;fill-opacity:1;fill:none;marker-start:url(#Arrow1Mstart)" + d="m 541.5074,178.03818 c -5.9138,7.73622 -8.0643,18.21989 -5.67376,27.65957 1.48599,5.86783 4.57531,11.19036 7.80142,16.31206 21.74916,34.52845 51.56536,63.93984 86.38787,85.215" + id="path3398" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect3400" + inkscape:original-d="m 541.5074,178.03818 c -2.36307,8.74604 -3.78151,18.43871 -5.67376,27.65957 -1.89225,9.22086 5.20195,10.87371 7.80142,16.31206 2.59947,5.43835 57.59291,56.809 86.38787,85.215" /> + <path + style="fill:none;stroke:#fc0000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-start:url(#Arrow1Mstart)" + d="m 486.31591,175.32896 c -4.56701,3.4939 -7.88094,8.59224 -9.21986,14.18439 -1.17323,4.90013 -0.85198,10.07279 0.32038,14.97313 1.17237,4.90034 3.17163,9.56311 5.35338,14.10489 18.70771,38.94408 52.03948,70.64767 91.8709,87.38319" + id="path3402" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect3404" + inkscape:original-d="m 486.31591,175.32896 c -4.01791,5.67276 -5.19994,8.50963 -9.21986,14.18439 -4.01991,5.67476 2.12866,20.32997 5.67376,29.07802 3.5451,8.74804 61.24827,58.25446 91.8709,87.38319" + sodipodi:nodetypes="ccsc" /> + <path + style="fill:none;stroke:#fc0000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-start:url(#Arrow1Mstart)" + d="m 438.74144,177.32896 c -2.99229,7.65136 -4.44794,15.90007 -4.25532,24.11347 0.26469,11.28631 3.68787,22.50755 9.92908,31.9149 7.88401,11.88353 19.75202,20.40996 30.49645,29.78723 16.28636,14.21403 30.48909,30.90719 48.22696,43.26242 11.01957,7.67563 23.28348,13.56063 36.16571,17.3546" + id="path3406" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect3408" + inkscape:original-d="m 438.74144,177.32896 c -3.5451,5.90916 -0.2354,18.20231 -4.25532,24.11347 -4.01991,5.91117 4.01991,19.14794 9.92908,31.9149 5.90917,12.76695 20.33197,19.85715 30.49645,29.78723 10.16449,9.93008 36.88044,34.98718 48.22696,43.26242 11.34652,8.27523 19.61974,10.85951 36.16571,17.3546" + sodipodi:nodetypes="ccsssc" /> + <g + id="g4374"> + <text + id="text5219-3" + y="187.92023" + x="132.8121" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + xml:space="preserve"><tspan + style="font-size:10px;line-height:1.25" + id="tspan5223-6" + y="187.92023" + x="132.8121" + sodipodi:role="line">producer_flow_seq</tspan></text> + <g + id="g4286"> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="67.609619" + y="125.91534" + id="text5219"><tspan + sodipodi:role="line" + x="67.609619" + y="125.91534" + id="tspan5223" + style="font-size:10px;line-height:1.25">producer maintains per flow sequence number</tspan></text> + <rect + style="fill:#ffffff;fill-opacity:1;fill-rule:evenodd;stroke:url(#linearGradient3810);stroke-width:0.97884095;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect2896" + width="240.98547" + height="44.122215" + x="61.723225" + y="131.41856" + ry="8.8282356" + rx="9.0800323" + inkscape:export-filename="/home/matz/barracuda/rapports/mbuf-api-v2-images/octeon_multi.png" + inkscape:export-xdpi="112" + inkscape:export-ydpi="112" /> + <rect + style="fill:none;fill-opacity:1;stroke:#6ba6fd;stroke-width:1.06814909;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect3736" + width="39.065548" + height="24.347494" + x="70.045547" + y="143.98941" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="76.606445" + y="141.62436" + id="text5219-1-9"><tspan + sodipodi:role="line" + x="76.606445" + y="141.62436" + id="tspan5223-2-3" + style="font-size:10px;line-height:1.25">flow 0</tspan></text> + <rect + style="fill:none;fill-opacity:1;stroke:#6ba6fd;stroke-width:1.06814909;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect3736-8" + width="39.065548" + height="24.347494" + x="129.42143" + y="144.7206" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="131.98233" + y="142.35555" + id="text5219-1-9-4"><tspan + sodipodi:role="line" + x="131.98233" + y="142.35555" + id="tspan5223-2-3-5" + style="font-size:10px;line-height:1.25">flow 1</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="195.98233" + y="142.35555" + id="text5219-1-9-4-3"><tspan + sodipodi:role="line" + x="195.98233" + y="142.35555" + id="tspan5223-2-3-5-6" + style="font-size:10px;line-height:1.25">flow 2</tspan></text> + <rect + style="fill:none;fill-opacity:1;stroke:#6ba6fd;stroke-width:1.06814909;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect3736-8-0-1" + width="39.065548" + height="24.347494" + x="251.42145" + y="144.7206" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="257.98233" + y="142.35555" + id="text5219-1-9-4-3-0"><tspan + sodipodi:role="line" + x="257.98233" + y="142.35555" + id="tspan5223-2-3-5-6-6" + style="font-size:10px;line-height:1.25">flow n</tspan></text> + <rect + style="fill:none;fill-opacity:1;stroke:#6ba6fd;stroke-width:1.06814909;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect3736-8-3" + width="39.065548" + height="24.347494" + x="192.15901" + y="144.7155" /> + </g> + </g> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="157.0374" + y="258.07278" + id="text5219-2"><tspan + sodipodi:role="line" + x="157.0374" + y="258.07278" + id="tspan5223-0" + style="font-size:10px;line-height:1.25">producer0</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="384.74597" + y="334.61694" + id="text5219-6"><tspan + sodipodi:role="line" + x="384.74597" + y="334.61694" + id="tspan5223-1" + style="font-size:10px;line-height:1.25">ordered queue 0</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="569.64355" + y="336.42307" + id="text5219-6-5"><tspan + sodipodi:role="line" + x="569.64355" + y="336.42307" + id="tspan5223-1-5" + style="font-size:10px;line-height:1.25">atomic queue 1</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="410.87885" + y="213.34842" + id="text5219-2-4"><tspan + sodipodi:role="line" + x="410.87885" + y="213.34842" + id="tspan5223-0-7" + style="font-size:10px;line-height:1.25">worker 0</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="157.44383" + y="236.49918" + id="text5219-2-6"><tspan + sodipodi:role="line" + x="157.44383" + y="236.49918" + id="tspan5223-0-9" + style="font-size:10px;line-height:1.25">port n+1</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="472.61508" + y="213.66943" + id="text5219-2-4-3"><tspan + sodipodi:role="line" + x="472.61508" + y="213.66943" + id="tspan5223-0-7-7" + style="font-size:10px;line-height:1.25">worker 1</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="534.61511" + y="213.66943" + id="text5219-2-4-3-4"><tspan + sodipodi:role="line" + x="534.61511" + y="213.66943" + id="tspan5223-0-7-7-5" + style="font-size:10px;line-height:1.25">worker 2</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="600.61511" + y="213.66943" + id="text5219-2-4-3-4-2"><tspan + sodipodi:role="line" + x="600.61511" + y="213.66943" + id="tspan5223-0-7-7-5-5" + style="font-size:10px;line-height:1.25">worker n</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="420.13348" + y="234.8974" + id="text5219-2-6-4"><tspan + sodipodi:role="line" + x="420.13348" + y="234.8974" + id="tspan5223-0-9-7" + style="font-size:10px;line-height:1.25">port 0</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="477.25241" + y="234.85495" + id="text5219-2-6-4-4"><tspan + sodipodi:role="line" + x="477.25241" + y="234.85495" + id="tspan5223-0-9-7-4" + style="font-size:10px;line-height:1.25">port 1</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="539.25244" + y="234.85495" + id="text5219-2-6-4-4-3"><tspan + sodipodi:role="line" + x="539.25244" + y="234.85495" + id="tspan5223-0-9-7-4-0" + style="font-size:10px;line-height:1.25">port 2</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="607.25244" + y="234.85495" + id="text5219-2-6-4-4-3-7"><tspan + sodipodi:role="line" + x="607.25244" + y="234.85495" + id="tspan5223-0-9-7-4-0-8" + style="font-size:10px;line-height:1.25">port n</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="478.92789" + y="188.00357" + id="text5219-3-2"><tspan + sodipodi:role="line" + x="478.92789" + y="188.00357" + id="tspan5223-6-7" + style="font-size:10px;line-height:1.25">expected_flow_seq</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="433.7254" + y="125.99867" + id="text5219-26"><tspan + sodipodi:role="line" + x="433.7254" + y="125.99867" + id="tspan5223-10" + style="font-size:10px;line-height:1.25">per flow expected sequence number</tspan></text> + <rect + style="fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#ff2d00;stroke-width:0.97884095;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect2896-6" + width="240.98547" + height="44.122215" + x="407.83902" + y="131.50191" + ry="8.8282356" + rx="9.0800323" + inkscape:export-filename="/home/matz/barracuda/rapports/mbuf-api-v2-images/octeon_multi.png" + inkscape:export-xdpi="112" + inkscape:export-ydpi="112" /> + <rect + style="fill:none;fill-opacity:1;stroke:#ff2d00;stroke-width:1.06814909;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect3736-1" + width="39.065548" + height="24.347494" + x="416.16132" + y="144.07275" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="422.72223" + y="141.7077" + id="text5219-1-9-5"><tspan + sodipodi:role="line" + x="422.72223" + y="141.7077" + id="tspan5223-2-3-9" + style="font-size:10px;line-height:1.25">flow 0</tspan></text> + <rect + style="fill:none;fill-opacity:1;stroke:#ff2d00;stroke-width:1.06814909;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect3736-8-4" + width="39.065548" + height="24.347494" + x="475.5372" + y="144.80394" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="478.09811" + y="142.43889" + id="text5219-1-9-4-9"><tspan + sodipodi:role="line" + x="478.09811" + y="142.43889" + id="tspan5223-2-3-5-0" + style="font-size:10px;line-height:1.25">flow 1</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="542.09808" + y="142.43889" + id="text5219-1-9-4-3-9"><tspan + sodipodi:role="line" + x="542.09808" + y="142.43889" + id="tspan5223-2-3-5-6-1" + style="font-size:10px;line-height:1.25">flow 2</tspan></text> + <rect + style="fill:none;fill-opacity:1;stroke:#ff2d00;stroke-width:1.06814909;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect3736-8-0-1-7" + width="39.065548" + height="24.347494" + x="597.53723" + y="144.80394" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="604.09808" + y="142.43889" + id="text5219-1-9-4-3-0-7"><tspan + sodipodi:role="line" + x="604.09808" + y="142.43889" + id="tspan5223-2-3-5-6-6-1" + style="font-size:10px;line-height:1.25">flow n</tspan></text> + <rect + style="fill:none;fill-opacity:1;stroke:#ff2d00;stroke-width:1.06814909;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect3736-8-3-1" + width="39.065548" + height="24.347494" + x="538.27478" + y="144.79884" /> + <path + style="fill:none;stroke:#2ce3ea;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-start:url(#Arrow1Mstart);marker-end:url(#Arrow1Mend)" + d="m 86.923031,168.93973 c 2.833543,14.16771 5.667239,28.33619 16.884859,38.84515 11.21761,10.50897 30.81628,17.35669 50.41543,24.20459" + id="path3022" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect3024" + inkscape:original-d="m 86.923031,168.93973 c 2.834697,14.16748 5.668393,28.33596 8.50109,42.50544 19.601799,6.84748 39.200469,13.6952 58.799199,20.5443" /> + <path + style="fill:none;stroke:#2ce3ea;stroke-width:0.94190133px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-start:url(#Arrow1Mstart);marker-end:url(#Arrow1Mend)" + d="m 146.34977,174.57512 c 2.37508,8.12236 4.75033,16.24527 9.26371,23.01491 4.51339,6.76964 11.16356,12.18449 17.81407,17.59962" + id="path3026" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect3028" + inkscape:original-d="m 146.34977,174.57512 c 2.37625,8.12202 4.7515,16.24493 7.12573,24.36872 6.6522,5.4148 13.30237,10.82966 19.95205,16.24581" /> + <path + style="fill:none;stroke:#2ce3ea;stroke-width:0.80414414px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-start:url(#Arrow1Mstart);marker-end:url(#Arrow1Mend)" + d="m 216.07443,175.10554 c -8.16931,13.20464 -16.33919,26.41022 -24.50966,39.61674" + id="path3034" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect3036" + inkscape:original-d="m 216.07443,175.10554 c -8.1691,13.20477 -16.33899,26.41034 -24.50966,39.61674" /> + <path + style="fill:none;stroke:#2ce3ea;stroke-width:0.77416188px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-start:url(#Arrow1Mstart);marker-end:url(#Arrow1Mend)" + d="m 275.03639,177.69945 c -3.2253,11.1515 -6.45088,22.30399 -17.30887,30.95868 -10.85798,8.65469 -29.34621,14.80993 -47.83697,20.96602" + id="path3038" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect3040" + inkscape:original-d="m 275.03639,177.69945 c -3.22467,11.15168 -6.45026,22.30417 -9.67676,33.4575 -18.49025,6.1554 -36.97848,12.31064 -55.46908,18.4672" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="215.93097" + y="349.08289" + id="text5219-2-62-2"><tspan + sodipodi:role="line" + x="215.93097" + y="349.08289" + id="tspan5223-0-91-7" + style="font-size:10px;line-height:1.25">enqueue ordered flow(step 1)</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="53.573601" + y="307.04483" + id="text5219-2-62-2-0"><tspan + sodipodi:role="line" + x="53.573601" + y="307.04483" + id="tspan5223-0-91-7-9" + style="font-size:10px;line-height:1.25">produce ordered flows(step 0)</tspan></text> + <path + style="fill:none;stroke:#000000;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:1, 2;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#Arrow1Mend-2)" + d="m 124.57429,298.66726 c 3.67724,-3.88246 6.17144,-8.87087 7.07106,-14.14214 0.99323,-5.81974 0.0756,-11.80766 0.70712,-17.67767 0.68671,-6.38281 3.2487,-12.55246 7.28535,-17.54419 4.03665,-4.99173 9.53369,-8.7879 15.63155,-10.7949" + id="path3284" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect3286" + inkscape:original-d="m 124.57429,298.66726 c 2.35802,-4.71505 4.71504,-9.42909 7.07106,-14.14214 2.35603,-4.71304 -4.47734,-16.97156 0.70712,-17.67767 5.18445,-0.70611 8.30435,-26.69017 22.9169,-28.33909" + sodipodi:nodetypes="ccsc" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="243.45221" + y="299.112" + id="text5219-2-62"><tspan + sodipodi:role="line" + x="243.45221" + y="299.112" + id="tspan5223-0-91" + style="font-size:10px;line-height:1.25">dequeue_ordered_flow(step 2)</tspan></text> + <path + style="fill:none;stroke:#000000;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:1, 2;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#Arrow1Mend-1)" + d="m 369.06669,289.2441 c 2.99719,-1.5536 6.06561,-2.9698 9.19239,-4.24264 10.36506,-4.21939 21.37433,-6.85204 32.52691,-7.77817" + id="path3288" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect3290" + inkscape:original-d="m 369.06669,289.2441 c 3.06513,-1.41521 6.12926,-2.82942 9.19239,-4.24264 3.06313,-1.41321 21.68561,-5.18645 32.52691,-7.77817" /> + <path + style="fill:none;stroke:url(#linearGradient5221);stroke-width:1.02505457px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Mend)" + d="m 412.13462,303.84217 c -1.78221,-8.21339 -1.99449,-16.76454 -0.62202,-25.05624 1.75585,-10.60783 6.12178,-20.77383 12.60532,-29.35128" + id="path4262" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect4264" + inkscape:original-d="m 412.13462,303.84217 c -0.20635,-8.35314 -0.41368,-16.70522 -0.62202,-25.05624 -0.20832,-8.35102 8.40455,-19.56856 12.60532,-29.35128" /> + <path + style="fill:none;stroke:url(#linearGradient5219);stroke-width:1.08023429px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Mend)" + d="m 430.54474,305.94863 c 1.69515,-6.54525 4.20133,-12.88001 7.44301,-18.81342 3.60164,-6.59226 8.11378,-12.68982 13.39743,-18.02956 9.02277,-9.11854 20.30848,-15.98095 32.55605,-19.79607" + id="path4266" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect4268" + inkscape:original-d="m 430.54474,305.94863 c 2.23395,-6.01096 4.96307,-12.54338 7.44301,-18.81342 2.47997,-6.27005 8.93268,-12.02081 13.39743,-18.02956 4.46476,-6.00874 21.70509,-13.19849 32.55605,-19.79607" /> + <path + style="fill:none;stroke:url(#linearGradient5217);stroke-width:1.03786695px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Mend)" + d="m 473.24617,306.85798 c 13.16685,-4.64153 26.0551,-10.07323 38.57234,-16.25615 11.0872,-5.47655 22.26981,-11.88166 29.35531,-22.01647 4.21744,-6.03245 6.78064,-13.2094 7.33883,-20.54872" + id="path4270" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect4272" + inkscape:original-d="m 473.24617,306.85798 c 12.85848,-5.41976 25.71593,-10.83847 38.57234,-16.25615 12.85641,-5.41767 19.57124,-14.67868 29.35531,-22.01647 9.78406,-7.3378 4.89359,-13.70019 7.33883,-20.54872" /> + <path + style="fill:none;stroke:url(#linearGradient5215);stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Mend)" + d="m 477.30438,331.21768 c 12.46907,4.50534 26.59382,4.24853 38.89087,-0.70711 15.87809,-6.39877 27.91048,-19.8678 43.24046,-27.48684 5.17938,-2.57417 10.67531,-4.44881 15.95548,-6.80936 5.28016,-2.36055 10.43559,-5.28025 14.34317,-9.54442 5.02516,-5.48374 7.59372,-12.72742 9.8995,-19.79898 1.98775,-6.09621 3.87372,-12.22561 5.65685,-18.38478" + id="path4274" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect4276" + inkscape:original-d="m 477.30438,331.21768 c 12.96463,-0.23671 25.92825,-0.47241 38.89087,-0.70711 12.96263,-0.2347 28.12086,-20.44688 43.24046,-27.48684 15.11959,-7.03995 20.9072,-8.7822 30.29865,-16.35378 9.39144,-7.57158 4.71505,-14.37883 9.8995,-19.79898 5.18444,-5.42016 5.65785,-11.07901 5.65685,-18.38478" + sodipodi:nodetypes="cssscc" /> + <path + style="fill:none;stroke:url(#linearGradient6431);stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Mend-4)" + d="m 440.40133,248.66934 c 3.19162,10.00334 8.25468,19.40617 14.84924,27.57716 3.66774,4.54451 7.79314,8.69906 12.02081,12.72792 10.2267,9.74579 21.15045,18.84495 33.23402,26.16295 8.57229,5.19151 17.67288,9.45409 26.87006,13.43503 10.00349,4.32995 20.14561,8.33962 30.40559,12.02082" + id="path6023" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect6025" + inkscape:original-d="m 440.40133,248.66934 c 4.95074,9.19138 9.90049,18.38377 14.84924,27.57716 4.94875,9.19339 8.01488,8.48428 12.02081,12.72792 4.00594,4.24364 22.15702,17.44097 33.23402,26.16295 11.07701,8.72199 17.91437,8.95569 26.87006,13.43503 8.95568,4.47934 20.27139,8.01288 30.40559,12.02082" /> + <path + style="fill:none;stroke:url(#linearGradient6429);stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Mend-4)" + d="m 548.51265,248.03664 c 1.13857,5.77255 4.23753,11.14011 8.66742,15.01241 4.18454,3.65784 9.3801,5.91956 14.03601,8.95481 11.99609,7.82041 20.2499,21.13301 21.92031,35.35534" + id="path6031" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect6033" + inkscape:original-d="m 548.51265,248.03664 c 2.72916,4.7243 5.77928,10.00728 8.66742,15.01241 2.88814,5.00514 9.35834,5.96887 14.03601,8.95481 4.67767,2.98593 14.61454,23.56922 21.92031,35.35534" /> + <path + style="fill:none;stroke:url(#linearGradient6427);stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Mend-4)" + d="m 629.90594,247.96223 c 4.16076,2.2543 7.64519,5.73873 9.89949,9.89949 3.81368,7.0389 3.96402,15.38981 4.94975,23.33453 1.17967,9.50784 3.68303,18.85051 7.41533,27.67438" + id="path6035" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect6037" + inkscape:original-d="m 629.90594,247.96223 c 3.18298,3.18098 6.60066,6.59866 9.89949,9.89949 3.29884,3.30084 3.30084,15.55535 4.94975,23.33453 1.64892,7.77917 4.94455,18.44859 7.41533,27.67438" /> + <path + style="fill:none;stroke:url(#linearGradient6433);stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Mend)" + d="m 483.94123,249.30958 c 1.1199,7.72101 5.29709,14.95611 11.42373,19.78648 5.29578,4.1753 11.79761,6.49938 17.49811,10.10253 5.44652,3.44261 10.08603,8.00309 15.2195,11.89715 3.38678,2.56908 6.98502,4.84517 10.6066,7.07107 7.74785,4.76198 15.62669,9.31084 23.62461,13.63968" + id="path6385" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect6387" + inkscape:original-d="m 483.94123,249.30958 c 5.24772,5.24571 7.61682,13.18999 11.42373,19.78648 3.80691,6.59649 11.6664,6.73402 17.49811,10.10253 5.8317,3.36852 10.14733,7.93044 15.2195,11.89715 5.07216,3.96672 7.07206,4.71305 10.6066,7.07107 3.53453,2.35802 15.75074,9.09212 23.62461,13.63968" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="191.27325" + y="270.74423" + id="text5219-2-62-3"><tspan + sodipodi:role="line" + x="191.27325" + y="270.74423" + id="tspan5223-0-91-6" + style="font-size:10px;line-height:1.25">change to atomic flow and enqueue(step 3)</tspan></text> + <path + style="fill:none;stroke:#000000;stroke-width:0.76920223;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.76920221, 1.53840441;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#Arrow1Mend-7)" + d="m 357.93205,263.09044 c 3.35603,-1.55628 6.83267,-2.85241 10.38844,-3.87293 10.27369,-2.94859 21.10841,-3.56584 31.77639,-2.90469 15.00358,0.92985 29.94516,4.405 43.38701,11.13467 1.23601,0.61881 2.45857,1.2645 3.66651,1.93647" + id="path3292" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect3294" + inkscape:original-d="m 357.93205,263.09044 c 3.46367,-1.29029 6.92649,-2.58127 10.38844,-3.87293 3.46195,-1.29167 21.18514,-1.93578 31.77639,-2.90469 10.59128,-0.96893 28.92555,7.4238 43.38701,11.13467 14.46147,3.71087 2.44521,1.29166 3.66651,1.93647" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="280.1011" + y="200.31314" + id="text5219-2-62-3-0"><tspan + sodipodi:role="line" + x="280.1011" + y="200.31314" + id="tspan5223-0-91-6-6" + style="font-size:10px;line-height:1.25">dequeue_atomic_flow (step 4)</tspan></text> + <path + style="fill:none;stroke:#000000;stroke-width:0.64963406;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.64963409, 1.29926818;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#Arrow1Mend-92)" + d="m 347.02887,196.67228 c 2.73089,-2.27942 5.78054,-4.1764 9.03233,-5.6184 8.65182,-3.83663 18.36723,-4.34297 27.82919,-4.5551 10.47734,-0.23489 20.95878,-0.18017 31.43857,-0.11877 6.46997,0.0379 12.93992,0.0784 19.40986,0.12133" + id="path3300" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect3302" + inkscape:original-d="m 347.02887,196.67228 c 3.0117,-1.87323 6.02249,-3.74603 9.03233,-5.6184 3.00986,-1.87236 18.55372,-3.03718 27.82919,-4.5551 9.27549,-1.51794 20.95997,-0.0796 31.43857,-0.11877 10.47862,-0.0391 12.94082,0.0804 19.40986,0.12133" /> + <path + style="fill:none;stroke:#000000;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:1, 2;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#Arrow1Mend-97)" + d="m 281.77537,342.34916 c 0.17405,-3.87643 1.69528,-7.6795 4.24264,-10.6066 1.21368,-1.3946 2.65204,-2.59324 4.24264,-3.53554" + id="path3226" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect3228-1" + inkscape:original-d="m 281.77537,342.34916 c 1.65092,-3.53654 2.82943,-7.07207 4.24264,-10.6066 1.41322,-3.53454 2.82943,-2.35803 4.24264,-3.53554" /> + </g> +</svg> diff --git a/doc/guides/tools/img/eventdev_perf_atq_test.svg b/doc/guides/tools/img/eventdev_perf_atq_test.svg new file mode 100644 index 00000000..404b572c --- /dev/null +++ b/doc/guides/tools/img/eventdev_perf_atq_test.svg @@ -0,0 +1,3188 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<!-- Created with Inkscape (http://www.inkscape.org/) --> + +<!-- +# BSD LICENSE +# +# Copyright (c) 2017, Cavium, Inc +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# +# - Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# +# - Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in +# the documentation and/or other materials provided with the +# distribution. +# +# - Neither the name of Cavium, Inc nor the names of its +# contributors may be used to endorse or promote products derived +# from this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS +# FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE +# COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, +# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES +# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +# STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) +# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED +# OF THE POSSIBILITY OF SUCH DAMAGE. +--> + +<svg + xmlns:osb="http://www.openswatchbook.org/uri/2009/osb" + xmlns:dc="http://purl.org/dc/elements/1.1/" + xmlns:cc="http://creativecommons.org/ns#" + xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" + xmlns:svg="http://www.w3.org/2000/svg" + xmlns="http://www.w3.org/2000/svg" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" + xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" + width="631.91431" + height="288.34286" + id="svg3868" + version="1.1" + inkscape:version="0.92.1 r" + sodipodi:docname="perf_atq.svg" + sodipodi:version="0.32" + inkscape:output_extension="org.inkscape.output.svg.inkscape" + enable-background="new"> + <defs + id="defs3870"> + <marker + inkscape:stockid="TriangleOutM" + orient="auto" + refY="0.0" + refX="0.0" + id="marker7126" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path7124" + d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z " + style="fill-rule:evenodd;stroke:#12efe9;stroke-width:1pt;stroke-opacity:1;fill:#12efe9;fill-opacity:1" + transform="scale(0.4)" /> + </marker> + <marker + inkscape:stockid="TriangleOutM" + orient="auto" + refY="0.0" + refX="0.0" + id="marker92948" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path92946" + d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z " + style="fill-rule:evenodd;stroke:#12efe9;stroke-width:1pt;stroke-opacity:1;fill:#12efe9;fill-opacity:1" + transform="scale(0.4)" /> + </marker> + <marker + inkscape:stockid="TriangleInM" + orient="auto" + refY="0.0" + refX="0.0" + id="marker92278" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path92276" + d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z " + style="fill-rule:evenodd;stroke:#12efe9;stroke-width:1pt;stroke-opacity:1;fill:#12efe9;fill-opacity:1" + transform="scale(-0.4)" /> + </marker> + <marker + inkscape:stockid="TriangleInM" + orient="auto" + refY="0.0" + refX="0.0" + id="marker91638" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path91636" + d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z " + style="fill-rule:evenodd;stroke:#00ffff;stroke-width:1pt;stroke-opacity:1;fill:#00ffff;fill-opacity:1" + transform="scale(-0.4)" /> + </marker> + <inkscape:path-effect + effect="bspline" + id="path-effect91628" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect91624" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker90762" + refX="0.0" + refY="0.0" + orient="auto" + inkscape:stockid="TriangleOutM" + inkscape:collect="always"> + <path + transform="scale(0.4)" + style="fill-rule:evenodd;stroke:#00ffff;stroke-width:1pt;stroke-opacity:1;fill:#00ffff;fill-opacity:1" + d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z " + id="path90760" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker90128" + refX="0.0" + refY="0.0" + orient="auto" + inkscape:stockid="TriangleOutM"> + <path + transform="scale(0.4)" + style="fill-rule:evenodd;stroke:#00ffff;stroke-width:1pt;stroke-opacity:1;fill:#00ffff;fill-opacity:1" + d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z " + id="path90126" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker89506" + refX="0.0" + refY="0.0" + orient="auto" + inkscape:stockid="TriangleOutM"> + <path + transform="scale(0.4)" + style="fill-rule:evenodd;stroke:#00ffff;stroke-width:1pt;stroke-opacity:1;fill:#00ffff;fill-opacity:1" + d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z " + id="path89504" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker88280" + refX="0.0" + refY="0.0" + orient="auto" + inkscape:stockid="TriangleOutM"> + <path + transform="scale(0.4)" + style="fill-rule:evenodd;stroke:#00ffff;stroke-width:1pt;stroke-opacity:1;fill:#00ffff;fill-opacity:1" + d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z " + id="path88278" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker87676" + refX="0.0" + refY="0.0" + orient="auto" + inkscape:stockid="TriangleOutM"> + <path + transform="scale(0.4)" + style="fill-rule:evenodd;stroke:#00ffff;stroke-width:1pt;stroke-opacity:1;fill:#00ffff;fill-opacity:1" + d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z " + id="path87674" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker86468" + refX="0.0" + refY="0.0" + orient="auto" + inkscape:stockid="TriangleInM"> + <path + transform="scale(-0.4)" + style="fill-rule:evenodd;stroke:#00ffff;stroke-width:1pt;stroke-opacity:1;fill:#00ffff;fill-opacity:1" + d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z " + id="path86466" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker85882" + refX="0.0" + refY="0.0" + orient="auto" + inkscape:stockid="TriangleInM"> + <path + transform="scale(-0.4)" + style="fill-rule:evenodd;stroke:#00ffff;stroke-width:1pt;stroke-opacity:1;fill:#00ffff;fill-opacity:1" + d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z " + id="path85880" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker85302" + refX="0.0" + refY="0.0" + orient="auto" + inkscape:stockid="TriangleInM"> + <path + transform="scale(-0.4)" + style="fill-rule:evenodd;stroke:#00ffff;stroke-width:1pt;stroke-opacity:1;fill:#00ffff;fill-opacity:1" + d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z " + id="path85300" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker84728" + refX="0.0" + refY="0.0" + orient="auto" + inkscape:stockid="TriangleInM"> + <path + transform="scale(-0.4)" + style="fill-rule:evenodd;stroke:#00ffff;stroke-width:1pt;stroke-opacity:1;fill:#00ffff;fill-opacity:1" + d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z " + id="path84726" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker84160" + refX="0.0" + refY="0.0" + orient="auto" + inkscape:stockid="TriangleInM"> + <path + transform="scale(-0.4)" + style="fill-rule:evenodd;stroke:#00ffff;stroke-width:1pt;stroke-opacity:1;fill:#00ffff;fill-opacity:1" + d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z " + id="path84158" /> + </marker> + <linearGradient + id="linearGradient84130" + osb:paint="solid"> + <stop + style="stop-color:#01fcff;stop-opacity:1;" + offset="0" + id="stop84128" /> + </linearGradient> + <inkscape:path-effect + effect="bspline" + id="path-effect82658" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect82654" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect82650" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect82616" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect82612" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect82608" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect82544" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect78438" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect78434" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect78430" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect78426" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <marker + inkscape:stockid="TriangleOutM" + orient="auto" + refY="0.0" + refX="0.0" + id="marker75328" + style="overflow:visible" + inkscape:isstock="true" + inkscape:collect="always"> + <path + id="path75326" + d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z " + style="fill-rule:evenodd;stroke:#00ffff;stroke-width:1pt;stroke-opacity:1;fill:#00ffff;fill-opacity:1" + transform="scale(0.4)" /> + </marker> + <marker + inkscape:stockid="TriangleOutM" + orient="auto" + refY="0.0" + refX="0.0" + id="marker74790" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path74788" + d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z " + style="fill-rule:evenodd;stroke:#01fcff;stroke-width:1pt;stroke-opacity:1;fill:#01fcff;fill-opacity:1" + transform="scale(0.4)" /> + </marker> + <marker + inkscape:stockid="TriangleInM" + orient="auto" + refY="0.0" + refX="0.0" + id="marker74246" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path74244" + d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z " + style="fill-rule:evenodd;stroke:#01fcff;stroke-width:1pt;stroke-opacity:1;fill:#01fcff;fill-opacity:1" + transform="scale(-0.4)" /> + </marker> + <inkscape:path-effect + effect="bspline" + id="path-effect73710" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect73706" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect73702" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect66544" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect65984" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker49921" + refX="0.0" + refY="0.0" + orient="auto" + inkscape:stockid="TriangleInM"> + <path + transform="scale(-0.4)" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1;fill:#000000;fill-opacity:1" + d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z " + id="path49919" /> + </marker> + <inkscape:path-effect + effect="bspline" + id="path-effect49911" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <marker + inkscape:stockid="TriangleInM" + orient="auto" + refY="0.0" + refX="0.0" + id="TriangleInM" + style="overflow:visible" + inkscape:isstock="true" + inkscape:collect="always"> + <path + id="path2114" + d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z " + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1;fill:#000000;fill-opacity:1" + transform="scale(-0.4)" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker46725" + refX="0.0" + refY="0.0" + orient="auto" + inkscape:stockid="Arrow2Mstart"> + <path + transform="scale(0.6) translate(0,0)" + d="M 8.7185878,4.0337352 L -2.2072895,0.016013256 L 8.7185884,-4.0017078 C 6.9730900,-1.6296469 6.9831476,1.6157441 8.7185878,4.0337352 z " + style="fill-rule:evenodd;stroke-width:0.625;stroke-linejoin:round;stroke:#000000;stroke-opacity:1;fill:#000000;fill-opacity:1" + id="path46723" /> + </marker> + <inkscape:path-effect + effect="bspline" + id="path-effect46703" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <marker + inkscape:stockid="Arrow2Mstart" + orient="auto" + refY="0.0" + refX="0.0" + id="marker42177" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path42175" + style="fill-rule:evenodd;stroke-width:0.625;stroke-linejoin:round;stroke:#000000;stroke-opacity:1;fill:#000000;fill-opacity:1" + d="M 8.7185878,4.0337352 L -2.2072895,0.016013256 L 8.7185884,-4.0017078 C 6.9730900,-1.6296469 6.9831476,1.6157441 8.7185878,4.0337352 z " + transform="scale(0.6) translate(0,0)" /> + </marker> + <marker + inkscape:stockid="TriangleOutM" + orient="auto" + refY="0.0" + refX="0.0" + id="marker41759" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path41757" + d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z " + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1;fill:#000000;fill-opacity:1" + transform="scale(0.4)" /> + </marker> + <inkscape:path-effect + effect="bspline" + id="path-effect41749" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="spiro" + id="path-effect41745" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect41450" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect41446" + is_visible="true" /> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker28236" + refX="0.0" + refY="0.0" + orient="auto" + inkscape:stockid="Arrow2Mstart"> + <path + transform="scale(0.6) translate(0,0)" + d="M 8.7185878,4.0337352 L -2.2072895,0.016013256 L 8.7185884,-4.0017078 C 6.9730900,-1.6296469 6.9831476,1.6157441 8.7185878,4.0337352 z " + style="fill-rule:evenodd;stroke-width:0.625;stroke-linejoin:round;stroke:#000000;stroke-opacity:1;fill:#000000;fill-opacity:1" + id="path28234" /> + </marker> + <marker + inkscape:isstock="true" + orient="auto" + refY="0" + refX="0" + id="marker20023" + inkscape:stockid="InfiniteLineStart" + style="overflow:visible"> + <g + transform="translate(-13,0)" + style="fill:#000000;stroke:#000000;stroke-opacity:1;fill-opacity:1" + id="g20021"> + <circle + style="stroke:#000000;stroke-opacity:1;fill:#000000;fill-opacity:1" + cx="3" + cy="0" + r="0.8" + id="circle20015" /> + <circle + style="stroke:#000000;stroke-opacity:1;fill:#000000;fill-opacity:1" + cx="6.5" + cy="0" + r="0.8" + id="circle20017" /> + <circle + style="stroke:#000000;stroke-opacity:1;fill:#000000;fill-opacity:1" + cx="10" + cy="0" + r="0.8" + id="circle20019" /> + </g> + </marker> + <marker + inkscape:isstock="true" + orient="auto" + refY="0" + refX="0" + id="marker19992" + inkscape:stockid="InfiniteLineStart" + style="overflow:visible"> + <g + transform="translate(-13,0)" + style="fill:#000000;stroke:#000000;stroke-opacity:1;fill-opacity:1" + id="g19990"> + <circle + style="stroke:#000000;stroke-opacity:1;fill:#000000;fill-opacity:1" + cx="3" + cy="0" + r="0.8" + id="circle19984" /> + <circle + style="stroke:#000000;stroke-opacity:1;fill:#000000;fill-opacity:1" + cx="6.5" + cy="0" + r="0.8" + id="circle19986" /> + <circle + style="stroke:#000000;stroke-opacity:1;fill:#000000;fill-opacity:1" + cx="10" + cy="0" + r="0.8" + id="circle19988" /> + </g> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker18966" + refX="0.0" + refY="0.0" + orient="auto" + inkscape:stockid="Tail"> + <g + style="stroke:#000000;stroke-opacity:1;fill:#000000;fill-opacity:1" + transform="scale(-1.2)" + id="g18964"> + <path + style="fill:#000000;fill-rule:evenodd;stroke:#000000;stroke-width:0.8;stroke-linecap:round;stroke-opacity:1;fill-opacity:1" + d="M -3.8048674,-3.9585227 L 0.54352094,0" + id="path18952" /> + <path + style="fill:#000000;fill-rule:evenodd;stroke:#000000;stroke-width:0.8;stroke-linecap:round;stroke-opacity:1;fill-opacity:1" + d="M -1.2866832,-3.9585227 L 3.0617053,0" + id="path18954" /> + <path + style="fill:#000000;fill-rule:evenodd;stroke:#000000;stroke-width:0.8;stroke-linecap:round;stroke-opacity:1;fill-opacity:1" + d="M 1.3053582,-3.9585227 L 5.6537466,0" + id="path18956" /> + <path + style="fill:#000000;fill-rule:evenodd;stroke:#000000;stroke-width:0.8;stroke-linecap:round;stroke-opacity:1;fill-opacity:1" + d="M -3.8048674,4.1775838 L 0.54352094,0.21974226" + id="path18958" /> + <path + style="fill:#000000;fill-rule:evenodd;stroke:#000000;stroke-width:0.8;stroke-linecap:round;stroke-opacity:1;fill-opacity:1" + d="M -1.2866832,4.1775838 L 3.0617053,0.21974226" + id="path18960" /> + <path + style="fill:#000000;fill-rule:evenodd;stroke:#000000;stroke-width:0.8;stroke-linecap:round;stroke-opacity:1;fill-opacity:1" + d="M 1.3053582,4.1775838 L 5.6537466,0.21974226" + id="path18962" /> + </g> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker18494" + refX="0.0" + refY="0.0" + orient="auto" + inkscape:stockid="Tail"> + <g + style="stroke:#000000;stroke-opacity:1;fill:#000000;fill-opacity:1" + transform="scale(-1.2)" + id="g18492"> + <path + style="fill:#000000;fill-rule:evenodd;stroke:#000000;stroke-width:0.8;stroke-linecap:round;stroke-opacity:1;fill-opacity:1" + d="M -3.8048674,-3.9585227 L 0.54352094,0" + id="path18480" /> + <path + style="fill:#000000;fill-rule:evenodd;stroke:#000000;stroke-width:0.8;stroke-linecap:round;stroke-opacity:1;fill-opacity:1" + d="M -1.2866832,-3.9585227 L 3.0617053,0" + id="path18482" /> + <path + style="fill:#000000;fill-rule:evenodd;stroke:#000000;stroke-width:0.8;stroke-linecap:round;stroke-opacity:1;fill-opacity:1" + d="M 1.3053582,-3.9585227 L 5.6537466,0" + id="path18484" /> + <path + style="fill:#000000;fill-rule:evenodd;stroke:#000000;stroke-width:0.8;stroke-linecap:round;stroke-opacity:1;fill-opacity:1" + d="M -3.8048674,4.1775838 L 0.54352094,0.21974226" + id="path18486" /> + <path + style="fill:#000000;fill-rule:evenodd;stroke:#000000;stroke-width:0.8;stroke-linecap:round;stroke-opacity:1;fill-opacity:1" + d="M -1.2866832,4.1775838 L 3.0617053,0.21974226" + id="path18488" /> + <path + style="fill:#000000;fill-rule:evenodd;stroke:#000000;stroke-width:0.8;stroke-linecap:round;stroke-opacity:1;fill-opacity:1" + d="M 1.3053582,4.1775838 L 5.6537466,0.21974226" + id="path18490" /> + </g> + </marker> + <marker + inkscape:stockid="TriangleOutM" + orient="auto" + refY="0.0" + refX="0.0" + id="marker17998" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path17996" + d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z " + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1;fill:#000000;fill-opacity:1" + transform="scale(0.4)" /> + </marker> + <marker + inkscape:stockid="TriangleOutM" + orient="auto" + refY="0.0" + refX="0.0" + id="marker17586" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path17584" + d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z " + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1;fill:#000000;fill-opacity:1" + transform="scale(0.4)" /> + </marker> + <marker + inkscape:stockid="TriangleOutM" + orient="auto" + refY="0.0" + refX="0.0" + id="marker17186" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path17184" + d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z " + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1;fill:#000000;fill-opacity:1" + transform="scale(0.4)" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker16768" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="EmptyTriangleOutM"> + <path + transform="matrix(0.4,0,0,0.4,-1.8,0)" + style="fill:#ffffff;fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;stroke-opacity:1" + d="M 5.77,0 -2.88,5 V -5 Z" + id="path16766" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker16380" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="EmptyTriangleOutM"> + <path + transform="matrix(0.4,0,0,0.4,-1.8,0)" + style="fill:#ffffff;fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;stroke-opacity:1" + d="M 5.77,0 -2.88,5 V -5 Z" + id="path16378" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker15998" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="EmptyTriangleOutM"> + <path + transform="matrix(0.4,0,0,0.4,-1.8,0)" + style="fill:#ffffff;fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;stroke-opacity:1" + d="M 5.77,0 -2.88,5 V -5 Z" + id="path15996" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="EmptyTriangleOutM" + orient="auto" + refY="0.0" + refX="0.0" + id="marker15604" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path15602" + d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z " + style="fill-rule:evenodd;fill:#ffffff;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="scale(0.4) translate(-4.5,0)" /> + </marker> + <marker + inkscape:stockid="EmptyTriangleOutM" + orient="auto" + refY="0.0" + refX="0.0" + id="marker15234" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path15232" + d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z " + style="fill-rule:evenodd;fill:#ffffff;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="scale(0.4) translate(-4.5,0)" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker14500" + refX="0.0" + refY="0.0" + orient="auto" + inkscape:stockid="EmptyTriangleOutM"> + <path + transform="scale(0.4) translate(-4.5,0)" + style="fill-rule:evenodd;fill:#ffffff;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z " + id="path14498" /> + </marker> + <inkscape:path-effect + effect="spiro" + id="path-effect14484" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect14480" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect14473" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect14469" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect14461" + is_visible="true" /> + <marker + inkscape:stockid="TriangleOutM" + orient="auto" + refY="0.0" + refX="0.0" + id="marker13075" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path13073" + d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z " + style="fill-rule:evenodd;stroke:#22f00d;stroke-width:1pt;stroke-opacity:1;fill:#22f00d;fill-opacity:1" + transform="scale(0.4)" /> + </marker> + <inkscape:path-effect + effect="spiro" + id="path-effect13065" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect13061" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect13057" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect13053" + is_visible="true" /> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker7719" + refX="0.0" + refY="0.0" + orient="auto" + inkscape:stockid="TriangleOutM" + inkscape:collect="always"> + <path + transform="scale(0.4)" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1;fill:#000000;fill-opacity:1" + d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z " + id="path7717" /> + </marker> + <marker + inkscape:stockid="Arrow1Send" + orient="auto" + refY="0.0" + refX="0.0" + id="marker7179" + style="overflow:visible;" + inkscape:isstock="true"> + <path + id="path7177" + d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z " + style="fill-rule:evenodd;stroke:#f78202;stroke-width:1pt;stroke-opacity:1;fill:#f78202;fill-opacity:1" + transform="scale(0.2) rotate(180) translate(6,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Send" + orient="auto" + refY="0.0" + refX="0.0" + id="Arrow1Send" + style="overflow:visible;" + inkscape:isstock="true"> + <path + id="path1993" + d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z " + style="fill-rule:evenodd;stroke:#ff141a;stroke-width:1pt;stroke-opacity:1;fill:#ff141a;fill-opacity:1" + transform="scale(0.2) rotate(180) translate(6,0)" /> + </marker> + <inkscape:path-effect + effect="bspline" + id="path-effect5228" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <marker + inkscape:stockid="DotM" + orient="auto" + refY="0.0" + refX="0.0" + id="DotM" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path2042" + d="M -2.5,-1.0 C -2.5,1.7600000 -4.7400000,4.0 -7.5,4.0 C -10.260000,4.0 -12.5,1.7600000 -12.5,-1.0 C -12.5,-3.7600000 -10.260000,-6.0 -7.5,-6.0 C -4.7400000,-6.0 -2.5,-3.7600000 -2.5,-1.0 z " + style="fill-rule:evenodd;stroke:#ff141a;stroke-width:1pt;stroke-opacity:1;fill:#ff141a;fill-opacity:1" + transform="scale(0.4) translate(7.4, 1)" /> + </marker> + <marker + inkscape:stockid="DiamondS" + orient="auto" + refY="0.0" + refX="0.0" + id="DiamondS" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path2063" + d="M 0,-7.0710768 L -7.0710894,0 L 0,7.0710589 L 7.0710462,0 L 0,-7.0710768 z " + style="fill-rule:evenodd;stroke:#ff141a;stroke-width:1pt;stroke-opacity:1;fill:#ff141a;fill-opacity:1" + transform="scale(0.2)" /> + </marker> + <marker + inkscape:stockid="StopL" + orient="auto" + refY="0.0" + refX="0.0" + id="StopL" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path2147" + d="M 0.0,5.65 L 0.0,-5.65" + style="fill:none;fill-opacity:0.75;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="scale(0.8)" /> + </marker> + <marker + inkscape:stockid="Tail" + orient="auto" + refY="0.0" + refX="0.0" + id="Tail" + style="overflow:visible" + inkscape:isstock="true"> + <g + id="g2026" + transform="scale(-1.2)" + style="stroke:#000000;stroke-opacity:1;fill:#000000;fill-opacity:1"> + <path + id="path2014" + d="M -3.8048674,-3.9585227 L 0.54352094,0" + style="fill:#000000;fill-rule:evenodd;stroke:#000000;stroke-width:0.8;stroke-linecap:round;stroke-opacity:1;fill-opacity:1" /> + <path + id="path2016" + d="M -1.2866832,-3.9585227 L 3.0617053,0" + style="fill:#000000;fill-rule:evenodd;stroke:#000000;stroke-width:0.8;stroke-linecap:round;stroke-opacity:1;fill-opacity:1" /> + <path + id="path2018" + d="M 1.3053582,-3.9585227 L 5.6537466,0" + style="fill:#000000;fill-rule:evenodd;stroke:#000000;stroke-width:0.8;stroke-linecap:round;stroke-opacity:1;fill-opacity:1" /> + <path + id="path2020" + d="M -3.8048674,4.1775838 L 0.54352094,0.21974226" + style="fill:#000000;fill-rule:evenodd;stroke:#000000;stroke-width:0.8;stroke-linecap:round;stroke-opacity:1;fill-opacity:1" /> + <path + id="path2022" + d="M -1.2866832,4.1775838 L 3.0617053,0.21974226" + style="fill:#000000;fill-rule:evenodd;stroke:#000000;stroke-width:0.8;stroke-linecap:round;stroke-opacity:1;fill-opacity:1" /> + <path + id="path2024" + d="M 1.3053582,4.1775838 L 5.6537466,0.21974226" + style="fill:#000000;fill-rule:evenodd;stroke:#000000;stroke-width:0.8;stroke-linecap:round;stroke-opacity:1;fill-opacity:1" /> + </g> + </marker> + <inkscape:path-effect + effect="bspline" + id="path-effect2658" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect1940" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="spiro" + id="path-effect1932" + is_visible="true" /> + <linearGradient + id="linearGradient1758" + osb:paint="solid"> + <stop + style="stop-color:#ff2d00;stop-opacity:1;" + offset="0" + id="stop1756" /> + </linearGradient> + <linearGradient + id="linearGradient6425" + osb:paint="solid"> + <stop + style="stop-color:#e6860b;stop-opacity:1;" + offset="0" + id="stop6423" /> + </linearGradient> + <linearGradient + id="linearGradient6391" + osb:paint="solid"> + <stop + style="stop-color:#000000;stop-opacity:1;" + offset="0" + id="stop6389" /> + </linearGradient> + <inkscape:path-effect + effect="spiro" + id="path-effect6387" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect6037" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect6033" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect6029" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect6025" + is_visible="true" /> + <linearGradient + id="linearGradient5213" + osb:paint="solid"> + <stop + style="stop-color:#ff0009;stop-opacity:1;" + offset="0" + id="stop5211" /> + </linearGradient> + <inkscape:path-effect + effect="spiro" + id="path-effect4276" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect4272" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect4268" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect4264" + is_visible="true" /> + <linearGradient + id="linearGradient2975" + osb:paint="solid"> + <stop + style="stop-color:#ff2200;stop-opacity:1;" + offset="0" + id="stop2973" /> + </linearGradient> + <linearGradient + id="linearGradient2969" + osb:paint="solid"> + <stop + style="stop-color:#009a08;stop-opacity:1;" + offset="0" + id="stop2967" /> + </linearGradient> + <linearGradient + id="linearGradient2963" + osb:paint="solid"> + <stop + style="stop-color:#000000;stop-opacity:1;" + offset="0" + id="stop2961" /> + </linearGradient> + <linearGradient + id="linearGradient2929" + osb:paint="solid"> + <stop + style="stop-color:#ff2d00;stop-opacity:1;" + offset="0" + id="stop2927" /> + </linearGradient> + <linearGradient + id="linearGradient4610" + osb:paint="solid"> + <stop + style="stop-color:#00ffff;stop-opacity:1;" + offset="0" + id="stop4608" /> + </linearGradient> + <linearGradient + id="linearGradient3993" + osb:paint="solid"> + <stop + style="stop-color:#6ba6fd;stop-opacity:1;" + offset="0" + id="stop3991" /> + </linearGradient> + <linearGradient + id="linearGradient3808" + osb:paint="solid"> + <stop + style="stop-color:#6ba6fd;stop-opacity:1;" + offset="0" + id="stop3806" /> + </linearGradient> + <linearGradient + id="linearGradient3776" + osb:paint="solid"> + <stop + style="stop-color:#fc0000;stop-opacity:1;" + offset="0" + id="stop3774" /> + </linearGradient> + <linearGradient + id="linearGradient3438" + osb:paint="solid"> + <stop + style="stop-color:#d18f21;stop-opacity:1;" + offset="0" + id="stop3436" /> + </linearGradient> + <inkscape:path-effect + effect="spiro" + id="path-effect3408" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3404" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3400" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3392" + is_visible="true" /> + <inkscape:path-effect + effect="bspline" + id="path-effect3376" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect3044" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect3040" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect3036" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect3032" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect3028" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect3024" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3020" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect2858" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect2854" + is_visible="true" /> + <inkscape:path-effect + effect="bspline" + id="path-effect2844" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <linearGradient + id="linearGradient2828" + osb:paint="solid"> + <stop + style="stop-color:#ff0000;stop-opacity:1;" + offset="0" + id="stop2826" /> + </linearGradient> + <inkscape:path-effect + effect="bspline" + id="path-effect329" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <marker + inkscape:stockid="Arrow1Mstart" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mstart" + style="overflow:visible"> + <path + id="path4530" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(0.4,0,0,0.4,4,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend" + style="overflow:visible"> + <path + id="path4533" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <linearGradient + id="linearGradient4513"> + <stop + style="stop-color:#fdffdb;stop-opacity:1;" + offset="0" + id="stop4515" /> + <stop + style="stop-color:#dfe2d8;stop-opacity:0;" + offset="1" + id="stop4517" /> + </linearGradient> + <inkscape:perspective + sodipodi:type="inkscape:persp3d" + inkscape:vp_x="0 : 526.18109 : 1" + inkscape:vp_y="0 : 1000 : 0" + inkscape:vp_z="744.09448 : 526.18109 : 1" + inkscape:persp3d-origin="372.04724 : 350.78739 : 1" + id="perspective3876" /> + <inkscape:perspective + id="perspective3886" + inkscape:persp3d-origin="0.5 : 0.33333333 : 1" + inkscape:vp_z="1 : 0.5 : 1" + inkscape:vp_y="0 : 1000 : 0" + inkscape:vp_x="0 : 0.5 : 1" + sodipodi:type="inkscape:persp3d" /> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Lend" + style="overflow:visible"> + <path + id="path3211" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker3892" + style="overflow:visible"> + <path + id="path3894" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker3896" + style="overflow:visible"> + <path + id="path3898" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lstart" + orient="auto" + refY="0" + refX="0" + id="Arrow1Lstart" + style="overflow:visible"> + <path + id="path3208" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(0.8,0,0,0.8,10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker3902" + style="overflow:visible"> + <path + id="path3904" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lstart" + orient="auto" + refY="0" + refX="0" + id="marker3906" + style="overflow:visible"> + <path + id="path3908" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(0.8,0,0,0.8,10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker3910" + style="overflow:visible"> + <path + id="path3912" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:perspective + id="perspective4086" + inkscape:persp3d-origin="0.5 : 0.33333333 : 1" + inkscape:vp_z="1 : 0.5 : 1" + inkscape:vp_y="0 : 1000 : 0" + inkscape:vp_x="0 : 0.5 : 1" + sodipodi:type="inkscape:persp3d" /> + <inkscape:perspective + id="perspective4113" + inkscape:persp3d-origin="0.5 : 0.33333333 : 1" + inkscape:vp_z="1 : 0.5 : 1" + inkscape:vp_y="0 : 1000 : 0" + inkscape:vp_x="0 : 0.5 : 1" + sodipodi:type="inkscape:persp3d" /> + <inkscape:perspective + id="perspective5195" + inkscape:persp3d-origin="0.5 : 0.33333333 : 1" + inkscape:vp_z="1 : 0.5 : 1" + inkscape:vp_y="0 : 1000 : 0" + inkscape:vp_x="0 : 0.5 : 1" + sodipodi:type="inkscape:persp3d" /> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-4" + style="overflow:visible"> + <path + id="path4533-7" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:perspective + id="perspective5272" + inkscape:persp3d-origin="0.5 : 0.33333333 : 1" + inkscape:vp_z="1 : 0.5 : 1" + inkscape:vp_y="0 : 1000 : 0" + inkscape:vp_x="0 : 0.5 : 1" + sodipodi:type="inkscape:persp3d" /> + <marker + inkscape:stockid="Arrow1Mstart" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mstart-4" + style="overflow:visible"> + <path + id="path4530-5" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(0.4,0,0,0.4,4,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-0" + style="overflow:visible"> + <path + id="path4533-3" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:perspective + id="perspective5317" + inkscape:persp3d-origin="0.5 : 0.33333333 : 1" + inkscape:vp_z="1 : 0.5 : 1" + inkscape:vp_y="0 : 1000 : 0" + inkscape:vp_x="0 : 0.5 : 1" + sodipodi:type="inkscape:persp3d" /> + <marker + inkscape:stockid="Arrow1Mstart" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mstart-3" + style="overflow:visible"> + <path + id="path4530-2" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(0.4,0,0,0.4,4,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-06" + style="overflow:visible"> + <path + id="path4533-1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Mstart" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mstart-8" + style="overflow:visible"> + <path + id="path4530-7" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(0.4,0,0,0.4,4,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-9" + style="overflow:visible"> + <path + id="path4533-2" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:path-effect + effect="spiro" + id="path-effect2858-0" + is_visible="true" /> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-3" + style="overflow:visible"> + <path + id="path4533-75" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:path-effect + effect="bspline" + id="path-effect3044-9" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-3-2" + style="overflow:visible"> + <path + id="path4533-75-8" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:path-effect + effect="bspline" + id="path-effect3044-9-9" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3993" + id="linearGradient3995" + x1="155.21328" + y1="231.61366" + x2="207.95523" + y2="231.61366" + gradientUnits="userSpaceOnUse" + gradientTransform="translate(-14,-74)" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3438" + id="linearGradient4612" + x1="594.77722" + y1="232.19244" + x2="647.51917" + y2="232.19244" + gradientUnits="userSpaceOnUse" + gradientTransform="translate(-38,66)" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3438" + id="linearGradient4616" + x1="468.32343" + y1="232.3177" + x2="521.06543" + y2="232.3177" + gradientUnits="userSpaceOnUse" + gradientTransform="translate(88,10)" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3438" + id="linearGradient4618" + x1="405.4682" + y1="232.36095" + x2="458.21014" + y2="232.36095" + gradientUnits="userSpaceOnUse" + gradientTransform="translate(150,-46)" /> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-2" + style="overflow:visible"> + <path + id="path4533-6" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:path-effect + effect="spiro" + id="path-effect3228" + is_visible="true" /> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="marker3706" + style="overflow:visible"> + <path + id="path3704" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:path-effect + effect="spiro" + id="path-effect3286" + is_visible="true" /> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-1" + style="overflow:visible"> + <path + id="path4533-8" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:path-effect + effect="spiro" + id="path-effect3290" + is_visible="true" /> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-3-7" + style="overflow:visible"> + <path + id="path4533-75-9" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:path-effect + effect="spiro" + id="path-effect3120-7" + is_visible="true" /> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-4-2" + style="overflow:visible"> + <path + id="path4533-7-0" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:path-effect + effect="spiro" + id="path-effect6025-2" + is_visible="true" /> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-7" + style="overflow:visible"> + <path + id="path4533-5" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:path-effect + effect="spiro" + id="path-effect3294" + is_visible="true" /> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-92" + style="overflow:visible"> + <path + id="path4533-28" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:path-effect + effect="spiro" + id="path-effect3302" + is_visible="true" /> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-97" + style="overflow:visible"> + <path + id="path4533-36" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:path-effect + effect="spiro" + id="path-effect3228-1" + is_visible="true" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient4513" + id="linearGradient4519" + x1="47.142857" + y1="244.50504" + x2="677.85718" + y2="244.50504" + gradientUnits="userSpaceOnUse" + gradientTransform="matrix(0.98357201,0,0,0.98599728,7.8873958,3.6023064)" /> + <linearGradient + gradientTransform="matrix(0.9887388,0,0,1.0000197,5.0811445,-0.1708579)" + inkscape:collect="always" + xlink:href="#linearGradient6391" + id="linearGradient2965" + x1="49.239536" + y1="244.84964" + x2="677.64832" + y2="244.84964" + gradientUnits="userSpaceOnUse" /> + <inkscape:path-effect + effect="bspline" + id="path-effect5228-5" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect5228-5-1" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect2658-9" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect1940-3" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect5228-5-1-6" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect5228-5-9" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect5228-59" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker7719-2" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="TriangleOutM"> + <path + inkscape:connector-curvature="0" + transform="scale(0.4)" + style="fill:#ff141a;fill-opacity:1;fill-rule:evenodd;stroke:#ff141a;stroke-width:1.00000003pt;stroke-opacity:1" + d="M 5.77,0 -2.88,5 V -5 Z" + id="path7717-2" /> + </marker> + <inkscape:path-effect + effect="spiro" + id="path-effect1932-8" + is_visible="true" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3993" + id="linearGradient3995-9" + x1="155.21329" + y1="231.61366" + x2="207.95523" + y2="231.61366" + gradientUnits="userSpaceOnUse" + gradientTransform="translate(-14.08539,16.056541)" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3993" + id="linearGradient3995-9-5" + x1="155.21329" + y1="231.61366" + x2="207.95523" + y2="231.61366" + gradientUnits="userSpaceOnUse" + gradientTransform="translate(-14.08539,104.05655)" /> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker7719-2-7" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="TriangleOutM"> + <path + inkscape:connector-curvature="0" + transform="scale(0.4)" + style="fill:#ff141a;fill-opacity:1;fill-rule:evenodd;stroke:#ff141a;stroke-width:1.00000003pt;stroke-opacity:1" + d="M 5.77,0 -2.88,5 V -5 Z" + id="path7717-2-6" /> + </marker> + <inkscape:path-effect + effect="spiro" + id="path-effect1932-8-5" + is_visible="true" /> + <inkscape:path-effect + effect="bspline" + id="path-effect1940-3-6" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect2658-9-7" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect5228-59-7" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect5228-5-9-3" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect5228-5-1-6-8" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient4513" + id="linearGradient38222" + gradientUnits="userSpaceOnUse" + gradientTransform="matrix(0.98357201,0,0,0.98599728,8.4731825,1.792165)" + x1="47.142857" + y1="244.50504" + x2="677.85718" + y2="244.50504" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient6391" + id="linearGradient38224" + gradientUnits="userSpaceOnUse" + gradientTransform="matrix(0.9887388,0,0,1.0000197,5.6669309,-1.980995)" + x1="49.239536" + y1="244.84964" + x2="677.64832" + y2="244.84964" /> + <linearGradient + gradientTransform="matrix(0.9987348,0,0,1.2726851,-111.27358,-86.656802)" + inkscape:collect="always" + xlink:href="#linearGradient2969" + id="linearGradient2971-1" + x1="372.12488" + y1="333.32864" + x2="476.58179" + y2="333.32864" + gradientUnits="userSpaceOnUse" /> + <linearGradient + gradientTransform="matrix(0.9987348,0,0,1.2726851,-111.62324,-175.91341)" + inkscape:collect="always" + xlink:href="#linearGradient2969" + id="linearGradient2971-1-7" + x1="372.12488" + y1="333.32864" + x2="476.58179" + y2="333.32864" + gradientUnits="userSpaceOnUse" /> + <linearGradient + gradientTransform="matrix(0.9987348,0,0,1.2726851,-111.62323,-263.9134)" + inkscape:collect="always" + xlink:href="#linearGradient2969" + id="linearGradient2971-1-7-1" + x1="372.12488" + y1="333.32864" + x2="476.58179" + y2="333.32864" + gradientUnits="userSpaceOnUse" /> + <marker + inkscape:stockid="TriangleOutM" + orient="auto" + refY="0" + refX="0" + id="marker13075-7" + style="overflow:visible" + inkscape:isstock="true"> + <path + inkscape:connector-curvature="0" + id="path13073-7" + d="M 5.77,0 -2.88,5 V -5 Z" + style="fill:#f00d28;fill-opacity:1;fill-rule:evenodd;stroke:#f00d28;stroke-width:1.00000003pt;stroke-opacity:1" + transform="scale(0.4)" /> + </marker> + <inkscape:path-effect + effect="bspline" + id="path-effect41749-6" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <marker + inkscape:stockid="TriangleOutM" + orient="auto" + refY="0" + refX="0" + id="marker13075-7-3" + style="overflow:visible" + inkscape:isstock="true"> + <path + inkscape:connector-curvature="0" + id="path13073-7-6" + d="M 5.77,0 -2.88,5 V -5 Z" + style="fill:#280df0;fill-opacity:1;fill-rule:evenodd;stroke:#280df0;stroke-width:1.00000003pt;stroke-opacity:1" + transform="scale(0.4)" /> + </marker> + <inkscape:path-effect + effect="bspline" + id="path-effect41749-6-5" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <marker + inkscape:stockid="TriangleOutM" + orient="auto" + refY="0" + refX="0" + id="marker13075-7-3-3" + style="overflow:visible" + inkscape:isstock="true"> + <path + inkscape:connector-curvature="0" + id="path13073-7-6-9" + d="M 5.77,0 -2.88,5 V -5 Z" + style="fill:#22f00d;fill-opacity:1;fill-rule:evenodd;stroke:#22f00d;stroke-width:1.00000003pt;stroke-opacity:1" + transform="scale(0.4)" /> + </marker> + <inkscape:path-effect + effect="bspline" + id="path-effect41749-6-5-4" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <marker + inkscape:stockid="TriangleInM" + orient="auto" + refY="0" + refX="0" + id="TriangleInM-8" + style="overflow:visible" + inkscape:isstock="true"> + <path + inkscape:connector-curvature="0" + id="path2114-5" + d="M 5.77,0 -2.88,5 V -5 Z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;stroke-opacity:1" + transform="scale(-0.4)" /> + </marker> + <marker + inkscape:stockid="TriangleOutM" + orient="auto" + refY="0" + refX="0" + id="marker42625-6" + style="overflow:visible" + inkscape:isstock="true"> + <path + inkscape:connector-curvature="0" + id="path42623-1" + d="M 5.77,0 -2.88,5 V -5 Z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;stroke-opacity:1" + transform="scale(0.4)" /> + </marker> + <inkscape:path-effect + effect="bspline" + id="path-effect46703-1" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <marker + inkscape:stockid="TriangleInM" + orient="auto" + refY="0" + refX="0" + id="TriangleInM-8-9" + style="overflow:visible" + inkscape:isstock="true"> + <path + inkscape:connector-curvature="0" + id="path2114-5-8" + d="M 5.77,0 -2.88,5 V -5 Z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;stroke-opacity:1" + transform="scale(-0.4)" /> + </marker> + <marker + inkscape:stockid="TriangleOutM" + orient="auto" + refY="0" + refX="0" + id="marker42625-6-4" + style="overflow:visible" + inkscape:isstock="true"> + <path + inkscape:connector-curvature="0" + id="path42623-1-8" + d="M 5.77,0 -2.88,5 V -5 Z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;stroke-opacity:1" + transform="scale(0.4)" /> + </marker> + <inkscape:path-effect + effect="bspline" + id="path-effect46703-1-1" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <marker + inkscape:stockid="TriangleOutM" + orient="auto" + refY="0" + refX="0" + id="marker13075-7-3-9" + style="overflow:visible" + inkscape:isstock="true"> + <path + inkscape:connector-curvature="0" + id="path13073-7-6-7" + d="M 5.77,0 -2.88,5 V -5 Z" + style="fill:#280df0;fill-opacity:1;fill-rule:evenodd;stroke:#280df0;stroke-width:1.00000003pt;stroke-opacity:1" + transform="scale(0.4)" /> + </marker> + <inkscape:path-effect + effect="bspline" + id="path-effect41749-6-5-5" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <marker + inkscape:stockid="TriangleOutM" + orient="auto" + refY="0" + refX="0" + id="marker13075-7-38" + style="overflow:visible" + inkscape:isstock="true"> + <path + inkscape:connector-curvature="0" + id="path13073-7-8" + d="M 5.77,0 -2.88,5 V -5 Z" + style="fill:#f00d28;fill-opacity:1;fill-rule:evenodd;stroke:#f00d28;stroke-width:1.00000003pt;stroke-opacity:1" + transform="scale(0.4)" /> + </marker> + <inkscape:path-effect + effect="bspline" + id="path-effect41749-6-3" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <marker + inkscape:stockid="TriangleOutM" + orient="auto" + refY="0" + refX="0" + id="marker13075-1" + style="overflow:visible" + inkscape:isstock="true"> + <path + inkscape:connector-curvature="0" + id="path13073-8" + d="M 5.77,0 -2.88,5 V -5 Z" + style="fill:#22f00d;fill-opacity:1;fill-rule:evenodd;stroke:#22f00d;stroke-width:1.00000003pt;stroke-opacity:1" + transform="scale(0.4)" /> + </marker> + <inkscape:path-effect + effect="bspline" + id="path-effect41749-9" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <marker + inkscape:stockid="TriangleOutM" + orient="auto" + refY="0" + refX="0" + id="marker13075-1-3" + style="overflow:visible" + inkscape:isstock="true"> + <path + inkscape:connector-curvature="0" + id="path13073-8-3" + d="M 5.77,0 -2.88,5 V -5 Z" + style="fill:#22f00d;fill-opacity:1;fill-rule:evenodd;stroke:#22f00d;stroke-width:1.00000003pt;stroke-opacity:1" + transform="scale(0.4)" /> + </marker> + <inkscape:path-effect + effect="bspline" + id="path-effect41749-9-8" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <marker + inkscape:stockid="TriangleOutM" + orient="auto" + refY="0" + refX="0" + id="marker13075-7-38-6" + style="overflow:visible" + inkscape:isstock="true"> + <path + inkscape:connector-curvature="0" + id="path13073-7-8-0" + d="M 5.77,0 -2.88,5 V -5 Z" + style="fill:#f00d28;fill-opacity:1;fill-rule:evenodd;stroke:#f00d28;stroke-width:1.00000003pt;stroke-opacity:1" + transform="scale(0.4)" /> + </marker> + <inkscape:path-effect + effect="bspline" + id="path-effect41749-6-3-4" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <marker + inkscape:stockid="TriangleOutM" + orient="auto" + refY="0" + refX="0" + id="marker13075-7-3-9-8" + style="overflow:visible" + inkscape:isstock="true"> + <path + inkscape:connector-curvature="0" + id="path13073-7-6-7-8" + d="M 5.77,0 -2.88,5 V -5 Z" + style="fill:#280df0;fill-opacity:1;fill-rule:evenodd;stroke:#280df0;stroke-width:1.00000003pt;stroke-opacity:1" + transform="scale(0.4)" /> + </marker> + <inkscape:path-effect + effect="bspline" + id="path-effect41749-6-5-5-8" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="marker3706-3" + style="overflow:visible"> + <path + id="path3704-0" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:path-effect + effect="spiro" + id="path-effect14484-3" + is_visible="true" /> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="marker3706-5" + style="overflow:visible"> + <path + id="path3704-4" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:path-effect + effect="spiro" + id="path-effect14484-0" + is_visible="true" /> + <marker + inkscape:stockid="TriangleInM" + orient="auto" + refY="0" + refX="0" + id="marker74246-9" + style="overflow:visible" + inkscape:isstock="true"> + <path + inkscape:connector-curvature="0" + id="path74244-4" + d="M 5.77,0 -2.88,5 V -5 Z" + style="fill:#01fcff;fill-opacity:1;fill-rule:evenodd;stroke:#01fcff;stroke-width:1.00000003pt;stroke-opacity:1" + transform="scale(-0.4)" /> + </marker> + <marker + inkscape:stockid="TriangleOutM" + orient="auto" + refY="0" + refX="0" + id="marker75328-6" + style="overflow:visible" + inkscape:isstock="true"> + <path + inkscape:connector-curvature="0" + id="path75326-9" + d="M 5.77,0 -2.88,5 V -5 Z" + style="fill:#01fcff;fill-opacity:1;fill-rule:evenodd;stroke:#01fcff;stroke-width:1.00000003pt;stroke-opacity:1" + transform="scale(0.4)" /> + </marker> + <inkscape:path-effect + effect="bspline" + id="path-effect73702-2" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <marker + inkscape:stockid="TriangleInM" + orient="auto" + refY="0" + refX="0" + id="marker74246-9-4" + style="overflow:visible" + inkscape:isstock="true"> + <path + inkscape:connector-curvature="0" + id="path74244-4-7" + d="M 5.77,0 -2.88,5 V -5 Z" + style="fill:#01fcff;fill-opacity:1;fill-rule:evenodd;stroke:#01fcff;stroke-width:1.00000003pt;stroke-opacity:1" + transform="scale(-0.4)" /> + </marker> + <marker + inkscape:stockid="TriangleOutM" + orient="auto" + refY="0" + refX="0" + id="marker75328-6-7" + style="overflow:visible" + inkscape:isstock="true"> + <path + inkscape:connector-curvature="0" + id="path75326-9-5" + d="M 5.77,0 -2.88,5 V -5 Z" + style="fill:#01fcff;fill-opacity:1;fill-rule:evenodd;stroke:#01fcff;stroke-width:1.00000003pt;stroke-opacity:1" + transform="scale(0.4)" /> + </marker> + <inkscape:path-effect + effect="bspline" + id="path-effect73702-2-4" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <marker + inkscape:stockid="TriangleInM" + orient="auto" + refY="0" + refX="0" + id="marker74246-9-4-1" + style="overflow:visible" + inkscape:isstock="true"> + <path + inkscape:connector-curvature="0" + id="path74244-4-7-2" + d="M 5.77,0 -2.88,5 V -5 Z" + style="fill:#01fcff;fill-opacity:1;fill-rule:evenodd;stroke:#01fcff;stroke-width:1.00000003pt;stroke-opacity:1" + transform="scale(-0.4)" /> + </marker> + <marker + inkscape:stockid="TriangleOutM" + orient="auto" + refY="0" + refX="0" + id="marker75328-6-7-8" + style="overflow:visible" + inkscape:isstock="true"> + <path + inkscape:connector-curvature="0" + id="path75326-9-5-9" + d="M 5.77,0 -2.88,5 V -5 Z" + style="fill:#01fcff;fill-opacity:1;fill-rule:evenodd;stroke:#01fcff;stroke-width:1.00000003pt;stroke-opacity:1" + transform="scale(0.4)" /> + </marker> + <inkscape:path-effect + effect="bspline" + id="path-effect73702-2-4-3" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect82544-8" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect82544-2" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect82616-0" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + </defs> + <sodipodi:namedview + id="base" + pagecolor="#ffffff" + bordercolor="#666666" + borderopacity="1.0" + inkscape:pageopacity="0.0" + inkscape:pageshadow="2" + inkscape:zoom="1.4142136" + inkscape:cx="456.95602" + inkscape:cy="142.49349" + inkscape:document-units="px" + inkscape:current-layer="layer1" + showgrid="false" + inkscape:window-width="1360" + inkscape:window-height="724" + inkscape:window-x="0" + inkscape:window-y="20" + inkscape:window-maximized="0" + fit-margin-top="0.1" + fit-margin-left="0.1" + fit-margin-right="0.1" + fit-margin-bottom="0.1" + inkscape:measure-start="-29.078,219.858" + inkscape:measure-end="346.809,219.858" + showguides="false" /> + <metadata + id="metadata3873"> + <rdf:RDF> + <cc:Work + rdf:about=""> + <dc:format>image/svg+xml</dc:format> + <dc:type + rdf:resource="http://purl.org/dc/dcmitype/StillImage" /> + <dc:title /> + </cc:Work> + </rdf:RDF> + </metadata> + <g + inkscape:label="Layer 1" + inkscape:groupmode="layer" + id="layer1" + transform="translate(-46.542857,-100.33361)" + style="display:inline;opacity:1"> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="117.328" + y="-14.742554" + id="text2978" + inkscape:export-filename="/home/matz/barracuda/rapports/mbuf-api-v2-images/octeon_multi.png" + inkscape:export-xdpi="112" + inkscape:export-ydpi="112"><tspan + sodipodi:role="line" + x="117.328" + y="-14.742554" + id="tspan3006" + style="font-size:15.22520161px;line-height:1.25"> </tspan></text> + <rect + style="fill:url(#linearGradient38222);fill-opacity:1;stroke:url(#linearGradient38224);stroke-width:0.98478383;stroke-opacity:1" + id="rect3697" + width="620.35291" + height="283.12207" + x="54.841576" + y="101.31245" + rx="0" + ry="0" /> + <rect + style="fill:#ffffff;fill-opacity:1;stroke:url(#linearGradient4612);stroke-width:1.02699995;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect87-3" + width="51.714954" + height="32.587509" + x="557.29071" + y="281.89868" + rx="11.6051" + ry="16.293755" /> + <rect + style="fill:#ffffff;fill-opacity:1;stroke:url(#linearGradient4616);stroke-width:1.02699995;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect87-6" + width="51.714954" + height="32.587509" + x="556.83691" + y="226.02396" + rx="11.6051" + ry="16.293755" /> + <rect + style="fill:#ffffff;fill-opacity:1;stroke:url(#linearGradient4618);stroke-width:1.02699995;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect87-6-5" + width="51.714954" + height="32.587509" + x="555.98169" + y="170.06718" + rx="11.6051" + ry="16.293755" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="560.87885" + y="167.34842" + id="text5219-2-4"><tspan + sodipodi:role="line" + x="560.87885" + y="167.34842" + id="tspan5223-0-7" + style="font-size:10px;line-height:1.25">worker 0</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="560.61511" + y="223.66943" + id="text5219-2-4-3"><tspan + sodipodi:role="line" + x="560.61511" + y="223.66943" + id="tspan5223-0-7-7" + style="font-size:10px;line-height:1.25">worker 1</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="562.61511" + y="277.66943" + id="text5219-2-4-3-4-2"><tspan + sodipodi:role="line" + x="562.61511" + y="277.66943" + id="tspan5223-0-7-7-5-5" + style="font-size:10px;line-height:1.25">worker n</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="570.13348" + y="188.8974" + id="text5219-2-6-4"><tspan + sodipodi:role="line" + x="570.13348" + y="188.8974" + id="tspan5223-0-9-7" + style="font-size:10px;line-height:1.25">port 0</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="565.25244" + y="244.85495" + id="text5219-2-6-4-4"><tspan + sodipodi:role="line" + x="565.25244" + y="244.85495" + id="tspan5223-0-9-7-4" + style="font-size:10px;line-height:1.25">port 1</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="569.25244" + y="296.85495" + id="text5219-2-6-4-4-3-7"><tspan + sodipodi:role="line" + x="569.25244" + y="296.85495" + id="tspan5223-0-9-7-4-0-8" + style="font-size:10px;line-height:1.25">port n</tspan></text> + <rect + style="display:inline;opacity:1;fill:#ffffff;fill-opacity:1;stroke:url(#linearGradient3995);stroke-width:1.02699995;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect87-6-5-3" + width="51.714954" + height="32.587509" + x="141.72678" + y="141.31989" + rx="11.6051" + ry="16.293755" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';display:inline;opacity:1;fill:#000000;fill-opacity:1;stroke:none" + x="143.03741" + y="182.07278" + id="text5219-2"><tspan + sodipodi:role="line" + x="143.03741" + y="182.07278" + id="tspan5223-0" + style="font-size:10px;line-height:1.25">producer 0</tspan></text> + <path + style="display:inline;opacity:1;fill:none;stroke:#ff141a;stroke-width:0.89999998;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#marker7719)" + d="m 192.59877,157.45256 h 65.05382" + id="path1930" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect1932" + inkscape:original-d="m 192.59877,157.45256 c 21.68561,-0.001 43.37021,-0.001 65.05382,0" + sodipodi:nodetypes="cc" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';display:inline;opacity:1;fill:#000000;fill-opacity:1;stroke:none" + x="145.44385" + y="160.49918" + id="text5219-2-6"><tspan + sodipodi:role="line" + x="145.44385" + y="160.49918" + id="tspan5223-0-9" + style="font-size:10px;line-height:1.25">port n+1</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';display:inline;opacity:1;fill:#000000;fill-opacity:1;stroke:none" + x="436.47687" + y="380.4664" + id="text2912" + inkscape:export-filename="/home/matz/barracuda/rapports/mbuf-api-v2-images/octeon_multi.png" + inkscape:export-xdpi="112" + inkscape:export-ydpi="112"><tspan + sodipodi:role="line" + x="436.47687" + y="380.4664" + id="tspan2916" + style="font-weight:bold;font-size:13.33333302px;line-height:1.25">test: perf_atq(all types queues)</tspan></text> + <rect + style="display:inline;opacity:1;fill:#ffffff;fill-opacity:1;stroke:url(#linearGradient3995-9);stroke-width:1.02699995;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect87-6-5-3-3" + width="51.714954" + height="32.587509" + x="141.64139" + y="231.3764" + rx="11.6051" + ry="16.293755" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';display:inline;opacity:1;fill:#000000;fill-opacity:1;stroke:none" + x="142.95203" + y="274.12933" + id="text5219-2-61"><tspan + sodipodi:role="line" + x="142.95203" + y="274.12933" + id="tspan5223-0-2" + style="font-size:10px;line-height:1.25">producer 1</tspan></text> + <path + style="display:inline;opacity:1;fill:none;stroke:#ff141a;stroke-width:0.89999998;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#marker7719-2)" + d="M 192.51338,247.5091 H 257.5672" + id="path1930-0" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect1932-8" + inkscape:original-d="m 192.51338,247.5091 c 21.68561,-10e-4 43.37021,-10e-4 65.05382,0" + sodipodi:nodetypes="cc" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';display:inline;opacity:1;fill:#000000;fill-opacity:1;stroke:none" + x="143.35846" + y="250.55573" + id="text5219-2-6-1"><tspan + sodipodi:role="line" + x="143.35846" + y="250.55573" + id="tspan5223-0-9-0" + style="font-size:10px;line-height:1.25">port n+2</tspan></text> + <rect + style="display:inline;opacity:1;fill:#ffffff;fill-opacity:1;stroke:url(#linearGradient3995-9-5);stroke-width:1.02699995;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect87-6-5-3-3-6" + width="51.714954" + height="32.587509" + x="141.64139" + y="319.3764" + rx="11.6051" + ry="16.293755" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';display:inline;opacity:1;fill:#000000;fill-opacity:1;stroke:none" + x="142.95203" + y="362.12933" + id="text5219-2-61-8"><tspan + sodipodi:role="line" + x="142.95203" + y="362.12933" + id="tspan5223-0-2-8" + style="font-size:10px;line-height:1.25">producer m</tspan></text> + <path + style="display:inline;opacity:1;fill:none;stroke:#ff141a;stroke-width:0.89999998;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#marker7719-2-7)" + d="M 192.51338,335.50911 H 257.5672" + id="path1930-0-9" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect1932-8-5" + inkscape:original-d="m 192.51338,335.50911 c 21.68561,-0.001 43.37021,-0.001 65.05382,0" + sodipodi:nodetypes="cc" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';display:inline;opacity:1;fill:#000000;fill-opacity:1;stroke:none" + x="143.35846" + y="338.55573" + id="text5219-2-6-1-6"><tspan + sodipodi:role="line" + x="143.35846" + y="338.55573" + id="tspan5223-0-9-0-4" + style="font-size:10px;line-height:1.25">port n+m</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;font-size:40px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none" + x="193.35634" + y="277.3764" + id="text21302"><tspan + sodipodi:role="line" + id="tspan21300" + x="193.35634" + y="277.3764" /></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;font-size:40px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none" + x="344.2348" + y="276.24649" + id="text21306"><tspan + sodipodi:role="line" + id="tspan21304" + x="344.2348" + y="311.63712" /></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;font-size:40px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none" + x="453.83633" + y="276.95361" + id="text21310"><tspan + sodipodi:role="line" + id="tspan21308" + x="453.83633" + y="312.34424" /></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';display:inline;opacity:1;fill:#000000;fill-opacity:1;stroke:none" + x="240.53555" + y="116.40381" + id="text5219-26"><tspan + sodipodi:role="line" + x="240.53555" + y="116.40381" + id="tspan5223-10" + style="font-size:10px;line-height:1.25">total queues = number of producers</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';display:inline;opacity:1;fill:#000000;fill-opacity:1;stroke:none" + x="493.64252" + y="211.9931" + id="text5219-26-2"><tspan + sodipodi:role="line" + x="493.64252" + y="211.9931" + id="tspan5223-10-7" + style="font-size:10px;line-height:1.25">All workers are linked to all queues</tspan></text> + <rect + style="display:inline;opacity:1;fill:#ffffff;fill-opacity:1;stroke:url(#linearGradient2971-1);stroke-width:1.1578598;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect128-7-9" + width="103.29906" + height="73.029671" + x="260.89331" + y="301.05072" + rx="8.5766249" + ry="13.633979" /> + <rect + style="display:inline;opacity:1;fill:#ffffff;fill-opacity:1;stroke:url(#linearGradient2971-1-7);stroke-width:1.1578598;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect128-7-9-1" + width="103.29906" + height="73.029671" + x="260.54364" + y="211.7941" + rx="8.5766249" + ry="13.633979" /> + <rect + style="display:inline;opacity:1;fill:#ffffff;fill-opacity:1;stroke:url(#linearGradient2971-1-7-1);stroke-width:1.1578598;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect128-7-9-1-9" + width="103.29906" + height="73.029671" + x="260.54364" + y="123.7941" + rx="8.5766249" + ry="13.633979" /> + <path + style="fill:none;stroke:#22f00d;stroke-width:0.89999998;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-mid:url(#marker13075)" + d="m 365.1356,144.98649 c 17.50681,-3.15856 35.01246,-6.31691 50.6001,-6.83532 15.58765,-0.51841 29.25916,1.60303 35.74063,5.72722 6.48148,4.12418 5.77447,10.25151 -1.53293,13.67023 -7.30741,3.41872 -21.21016,4.12564 -33.35024,4.47926 -12.14008,0.35362 -22.50881,0.35362 -30.995,-0.23562 -8.48618,-0.58924 -15.08602,-1.76779 -21.68568,-2.9463 0,0 0,0 0,0 0,0 2.12132,0 2.12132,0" + id="path41747" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect41749" + inkscape:original-d="m 365.1356,144.98649 c 17.50667,-3.15935 35.01232,-6.3177 52.51699,-9.47505 13.67093,2.1202 27.34244,4.24164 41.0122,6.36396 -0.70611,6.12727 -1.41312,12.2546 -2.12133,18.38478 -13.90752,0.70621 -27.81027,1.41313 -41.71929,2.12132 -10.37109,-10e-4 -20.73982,-10e-4 -31.11271,0 -6.59928,-1.17962 -19.79898,-3.53554 -19.79898,-3.53554 v 0 h 2.12132" + sodipodi:nodetypes="ccccccccc" /> + <path + style="display:inline;opacity:1;fill:none;stroke:#f00d28;stroke-width:0.89999998;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-mid:url(#marker13075-7)" + d="m 365.75435,154.89448 c 17.50681,-3.15856 35.01246,-6.3169 50.60011,-6.83532 15.58765,-0.51841 29.25916,1.60303 35.74063,5.72722 6.48148,4.12418 5.77447,10.25151 -1.53293,13.67023 -7.30741,3.41872 -21.21016,4.12564 -33.35025,4.47926 -12.14008,0.35362 -22.50881,0.35362 -30.995,-0.23562 -8.48618,-0.58924 -15.08602,-1.76779 -21.68568,-2.9463 0,0 0,0 0,0 0,0 2.12132,0 2.12132,0" + id="path41747-7" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect41749-6" + inkscape:original-d="m 365.75435,154.89448 c 17.50667,-3.15935 35.01232,-6.3177 52.517,-9.47505 13.67093,2.1202 27.34244,4.24164 41.0122,6.36396 -0.70611,6.12727 -1.41312,12.2546 -2.12133,18.38478 -13.90752,0.70621 -27.81027,1.41313 -41.7193,2.12132 -10.37109,-0.001 -20.73982,-0.001 -31.11271,0 -6.59928,-1.17962 -19.79898,-3.53554 -19.79898,-3.53554 v 0 h 2.12132" + sodipodi:nodetypes="ccccccccc" /> + <path + style="display:inline;opacity:1;fill:none;stroke:#280df0;stroke-width:0.89999998;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-mid:url(#marker13075-7-3)" + d="m 365.75435,162.89448 c 17.50681,-3.15856 35.01246,-6.3169 50.60011,-6.83532 15.58765,-0.51841 29.25916,1.60303 35.74063,5.72722 6.48148,4.12418 5.77447,10.25151 -1.53293,13.67023 -7.30741,3.41872 -21.21016,4.12564 -33.35025,4.47926 -12.14008,0.35362 -22.50881,0.35362 -30.995,-0.23562 -8.48618,-0.58924 -15.08602,-1.76779 -21.68568,-2.9463 0,0 0,0 0,0 0,0 2.12132,0 2.12132,0" + id="path41747-7-6" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect41749-6-5" + inkscape:original-d="m 365.75435,162.89448 c 17.50667,-3.15935 35.01232,-6.3177 52.517,-9.47505 13.67093,2.1202 27.34244,4.24164 41.0122,6.36396 -0.70611,6.12727 -1.41312,12.2546 -2.12133,18.38478 -13.90752,0.70621 -27.81027,1.41313 -41.7193,2.12132 -10.37109,-0.001 -20.73982,-0.001 -31.11271,0 -6.59928,-1.17962 -19.79898,-3.53554 -19.79898,-3.53554 v 0 h 2.12132" + sodipodi:nodetypes="ccccccccc" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';display:inline;opacity:1;fill:#000000;fill-opacity:1;stroke:none" + x="272.16205" + y="162.59613" + id="text5219-2-1"><tspan + sodipodi:role="line" + x="272.16205" + y="162.59613" + id="tspan5223-0-29" + style="font-size:10px;line-height:1.25">all types queue 0</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';display:inline;opacity:1;fill:#000000;fill-opacity:1;stroke:none" + x="269.43988" + y="253.62556" + id="text5219-2-1-3"><tspan + sodipodi:role="line" + x="269.43988" + y="253.62556" + id="tspan5223-0-29-9" + style="font-size:10px;line-height:1.25">all types queue 1</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';display:inline;opacity:1;fill:#000000;fill-opacity:1;stroke:none" + x="267.29773" + y="336.96365" + id="text5219-2-1-3-0"><tspan + sodipodi:role="line" + x="267.29773" + y="336.96365" + id="tspan5223-0-29-9-8" + style="font-size:10px;line-height:1.25">all types queue n</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';display:inline;opacity:1;fill:#000000;fill-opacity:1;stroke:none" + x="472.35513" + y="126.43675" + id="text5219-2-1-8"><tspan + sodipodi:role="line" + x="472.35513" + y="126.43675" + id="tspan5223-0-29-5" + style="font-size:10px;line-height:1.25">stage 0</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';display:inline;opacity:1;fill:#000000;fill-opacity:1;stroke:none" + x="471.03671" + y="148.78894" + id="text5219-2-1-8-0"><tspan + sodipodi:role="line" + x="471.03671" + y="148.78894" + id="tspan5223-0-29-5-9" + style="font-size:10px;line-height:1.25">stage 1</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';display:inline;opacity:1;fill:#000000;fill-opacity:1;stroke:none" + x="471.07834" + y="170.80975" + id="text5219-2-1-8-0-6"><tspan + sodipodi:role="line" + x="471.07834" + y="170.80975" + id="tspan5223-0-29-5-9-3" + style="font-size:10px;line-height:1.25">stage n</tspan></text> + <path + style="fill:none;stroke:#000000;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:1, 1;stroke-dashoffset:0;stroke-opacity:1;marker-start:url(#TriangleInM);marker-end:" + d="m 432.03737,136.70383 c 0,0 0,0 0.47136,-0.82489 0.47137,-0.82489 1.41493,-2.47613 1.886,-3.3005 0.47106,-0.82436 5.42081,-5.77411 10.60366,-6.36307 5.18286,-0.58896 15.56005,-1.76818 20.74495,-2.35738 5.1849,-0.58919 5.1849,-0.58919 5.1849,-0.58919" + id="path46701" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect46703" + inkscape:original-d="m 432.03737,136.70383 c 0,0 10e-4,-0.001 0,0 0.94305,-1.64959 1.88661,-3.30084 2.82842,-4.94975 l 4.94975,-4.94975 c 10.36561,-1.17879 20.7428,-2.35802 31.1127,-3.53553 10e-4,-0.001 0,0 0,0" + sodipodi:nodetypes="cccccc" /> + <path + style="display:inline;opacity:1;fill:none;stroke:#000000;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:1, 1;stroke-dashoffset:0;stroke-opacity:1;marker-start:url(#TriangleInM-8);marker-end:" + d="m 431.13155,147.95859 c 0,0 0,0 0.47136,-0.82489 0.47137,-0.82489 1.41493,-2.47613 1.886,-3.3005 0.47106,-0.82436 5.42081,-5.77411 10.60366,-6.36307 5.18286,-0.58896 15.56005,-1.76818 22.74852,-0.94309 7.18847,0.82509 11.19521,3.65337 15.20215,6.4818" + id="path46701-5" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect46703-1" + inkscape:original-d="m 431.13155,147.95859 c 0,0 10e-4,-10e-4 0,0 0.94305,-1.64959 1.88661,-3.30084 2.82842,-4.94975 l 4.94975,-4.94975 c 10.36561,-1.17879 20.7428,-2.35802 31.1127,-3.53553 4.00794,2.82743 12.02082,8.48528 12.02082,8.48528" + sodipodi:nodetypes="cccccc" /> + <path + style="fill:none;stroke:#000000;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:1, 1;stroke-dashoffset:0;stroke-opacity:1;marker-start:url(#marker49921);marker-end:" + d="m 426.25919,180.07998 c 17.20698,4.24282 34.41324,8.48545 46.19849,7.30635 11.78525,-1.17911 18.14921,-7.77878 21.3307,-11.0781 3.18149,-3.29932 3.18149,-3.29932 3.18149,-3.29932 0,0 0,0 0,0 0,0 0,0 0,0" + id="path49909" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect49911" + inkscape:original-d="m 426.25919,180.07998 c 17.20727,4.24164 34.41353,8.48428 51.6188,12.72792 6.36496,-6.60066 12.72892,-13.20033 19.09188,-19.79899 10e-4,-10e-4 0,0 0,0 10e-4,-10e-4 10e-4,-10e-4 0,0 v 0" + sodipodi:nodetypes="cccccc" /> + <path + style="display:inline;opacity:1;fill:none;stroke:#22f00d;stroke-width:0.89999998;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-mid:url(#marker13075-1)" + d="m 367.96475,228.58515 c 17.50681,-3.15856 35.01246,-6.31691 50.6001,-6.83532 15.58765,-0.51841 29.25916,1.60303 35.74063,5.72722 6.48148,4.12418 5.77447,10.25151 -1.53293,13.67023 -7.30741,3.41872 -21.21016,4.12564 -33.35024,4.47926 -12.14008,0.35362 -22.50881,0.35362 -30.995,-0.23562 -8.48618,-0.58924 -15.08602,-1.76779 -21.68568,-2.9463 0,0 0,0 0,0 0,0 2.12132,0 2.12132,0" + id="path41747-6" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect41749-9" + inkscape:original-d="m 367.96475,228.58515 c 17.50667,-3.15935 35.01232,-6.3177 52.51699,-9.47505 13.67093,2.1202 27.34244,4.24164 41.0122,6.36396 -0.70611,6.12727 -1.41312,12.2546 -2.12133,18.38478 -13.90752,0.70621 -27.81027,1.41313 -41.71929,2.12132 -10.37109,-0.001 -20.73982,-0.001 -31.11271,0 -6.59928,-1.17962 -19.79898,-3.53554 -19.79898,-3.53554 v 0 h 2.12132" + sodipodi:nodetypes="ccccccccc" /> + <path + style="display:inline;opacity:1;fill:none;stroke:#f00d28;stroke-width:0.89999998;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-mid:url(#marker13075-7-38)" + d="m 368.5835,238.49314 c 17.50681,-3.15856 35.01246,-6.3169 50.60011,-6.83532 15.58765,-0.51841 29.25916,1.60303 35.74063,5.72722 6.48148,4.12418 5.77447,10.25151 -1.53293,13.67023 -7.30741,3.41872 -21.21016,4.12564 -33.35025,4.47926 -12.14008,0.35362 -22.50881,0.35362 -30.995,-0.23562 -8.48618,-0.58924 -15.08602,-1.76779 -21.68568,-2.9463 0,0 0,0 0,0 0,0 2.12132,0 2.12132,0" + id="path41747-7-4" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect41749-6-3" + inkscape:original-d="m 368.5835,238.49314 c 17.50667,-3.15935 35.01232,-6.3177 52.517,-9.47505 13.67093,2.1202 27.34244,4.24164 41.0122,6.36396 -0.70611,6.12727 -1.41312,12.2546 -2.12133,18.38478 -13.90752,0.70621 -27.81027,1.41313 -41.7193,2.12132 -10.37109,-0.001 -20.73982,-0.001 -31.11271,0 -6.59928,-1.17962 -19.79898,-3.53554 -19.79898,-3.53554 v 0 h 2.12132" + sodipodi:nodetypes="ccccccccc" /> + <path + style="display:inline;opacity:1;fill:none;stroke:#280df0;stroke-width:0.89999998;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-mid:url(#marker13075-7-3-9)" + d="m 368.5835,246.49314 c 17.50681,-3.15856 35.01246,-6.3169 50.60011,-6.83532 15.58765,-0.51841 29.25916,1.60303 35.74063,5.72722 6.48148,4.12418 5.77447,10.25151 -1.53293,13.67023 -7.30741,3.41872 -21.21016,4.12564 -33.35025,4.47926 -12.14008,0.35362 -22.50881,0.35362 -30.995,-0.23562 -8.48618,-0.58924 -15.08602,-1.76779 -21.68568,-2.9463 0,0 0,0 0,0 0,0 2.12132,0 2.12132,0" + id="path41747-7-6-3" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect41749-6-5-5" + inkscape:original-d="m 368.5835,246.49314 c 17.50667,-3.15935 35.01232,-6.3177 52.517,-9.47505 13.67093,2.1202 27.34244,4.24164 41.0122,6.36396 -0.70611,6.12727 -1.41312,12.2546 -2.12133,18.38478 -13.90752,0.70621 -27.81027,1.41313 -41.7193,2.12132 -10.37109,-10e-4 -20.73982,-10e-4 -31.11271,0 -6.59928,-1.17962 -19.79898,-3.53554 -19.79898,-3.53554 v 0 h 2.12132" + sodipodi:nodetypes="ccccccccc" /> + <path + style="display:inline;opacity:1;fill:none;stroke:#22f00d;stroke-width:0.89999998;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-mid:url(#marker13075-1-3)" + d="m 367.96475,320.58515 c 17.50681,-3.15856 35.01246,-6.31691 50.6001,-6.83532 15.58765,-0.51841 29.25916,1.60303 35.74063,5.72722 6.48148,4.12418 5.77447,10.25151 -1.53293,13.67023 -7.30741,3.41872 -21.21016,4.12564 -33.35024,4.47926 -12.14008,0.35362 -22.50881,0.35362 -30.995,-0.23562 -8.48618,-0.58924 -15.08602,-1.76779 -21.68568,-2.9463 0,0 0,0 0,0 0,0 2.12132,0 2.12132,0" + id="path41747-6-9" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect41749-9-8" + inkscape:original-d="m 367.96475,320.58515 c 17.50667,-3.15935 35.01232,-6.3177 52.51699,-9.47505 13.67093,2.1202 27.34244,4.24164 41.0122,6.36396 -0.70611,6.12727 -1.41312,12.2546 -2.12133,18.38478 -13.90752,0.70621 -27.81027,1.41313 -41.71929,2.12132 -10.37109,-0.001 -20.73982,-0.001 -31.11271,0 -6.59928,-1.17962 -19.79898,-3.53554 -19.79898,-3.53554 v 0 h 2.12132" + sodipodi:nodetypes="ccccccccc" /> + <path + style="display:inline;opacity:1;fill:none;stroke:#f00d28;stroke-width:0.89999998;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-mid:url(#marker13075-7-38-6)" + d="m 368.5835,330.49314 c 17.50681,-3.15856 35.01246,-6.3169 50.60011,-6.83532 15.58765,-0.51841 29.25916,1.60303 35.74063,5.72722 6.48148,4.12418 5.77447,10.25151 -1.53293,13.67023 -7.30741,3.41872 -21.21016,4.12564 -33.35025,4.47926 -12.14008,0.35362 -22.50881,0.35362 -30.995,-0.23562 -8.48618,-0.58924 -15.08602,-1.76779 -21.68568,-2.9463 0,0 0,0 0,0 0,0 2.12132,0 2.12132,0" + id="path41747-7-4-7" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect41749-6-3-4" + inkscape:original-d="m 368.5835,330.49314 c 17.50667,-3.15935 35.01232,-6.3177 52.517,-9.47505 13.67093,2.1202 27.34244,4.24164 41.0122,6.36396 -0.70611,6.12727 -1.41312,12.2546 -2.12133,18.38478 -13.90752,0.70621 -27.81027,1.41313 -41.7193,2.12132 -10.37109,-10e-4 -20.73982,-10e-4 -31.11271,0 -6.59928,-1.17962 -19.79898,-3.53554 -19.79898,-3.53554 v 0 h 2.12132" + sodipodi:nodetypes="ccccccccc" /> + <path + style="display:inline;opacity:1;fill:none;stroke:#280df0;stroke-width:0.89999998;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-mid:url(#marker13075-7-3-9-8)" + d="m 368.5835,338.49314 c 17.50681,-3.15856 35.01246,-6.3169 50.60011,-6.83532 15.58765,-0.51841 29.25916,1.60303 35.74063,5.72722 6.48148,4.12418 5.77447,10.25151 -1.53293,13.67023 -7.30741,3.41872 -21.21016,4.12564 -33.35025,4.47926 -12.14008,0.35362 -22.50881,0.35362 -30.995,-0.23562 -8.48618,-0.58924 -15.08602,-1.76779 -21.68568,-2.9463 0,0 0,0 0,0 0,0 2.12132,0 2.12132,0" + id="path41747-7-6-3-7" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect41749-6-5-5-8" + inkscape:original-d="m 368.5835,338.49314 c 17.50667,-3.15935 35.01232,-6.3177 52.517,-9.47505 13.67093,2.1202 27.34244,4.24164 41.0122,6.36396 -0.70611,6.12727 -1.41312,12.2546 -2.12133,18.38478 -13.90752,0.70621 -27.81027,1.41313 -41.7193,2.12132 -10.37109,-10e-4 -20.73982,-10e-4 -31.11271,0 -6.59928,-1.17962 -19.79898,-3.53554 -19.79898,-3.53554 v 0 h 2.12132" + sodipodi:nodetypes="ccccccccc" /> + <path + style="fill:none;stroke:#00ffff;stroke-width:0.99599999;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.996, 1.992;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#marker75328)" + d="m 517.47596,257.39726 c -6.36289,5.42024 -12.72685,10.84139 -27.92958,17.20562 -15.20274,6.36424 -39.24437,13.67101 -55.74376,18.03162 -16.49939,4.36062 -25.45567,5.77477 -35.56404,8.14827 -10.10838,2.3735 -21.36568,5.70562 -32.62558,9.03852" + id="path82648" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect82650" + inkscape:original-d="m 517.47596,257.39726 c -6.36296,5.42016 -12.72692,10.84131 -19.09188,16.26345 -24.04063,7.30577 -48.08226,14.61254 -72.12489,21.92031 -8.95609,1.41328 -17.91237,2.82743 -26.87006,4.24264 -11.25912,3.33196 -22.51642,6.66409 -33.77613,9.99763" + sodipodi:nodetypes="ccccc" /> + <path + style="fill:none;stroke:#00ffff;stroke-width:0.99600399;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.99600399, 1.99200797;stroke-dashoffset:0;stroke-opacity:1;marker-start:url(#marker91638);marker-end:url(#marker90762)" + d="m 555.30362,244.42669 c -47.49196,14.92975 -94.98511,29.85987 -126.06777,36.66718 -31.08266,6.80731 -49.06508,5.19441 -65.39314,3.72989" + id="path82652" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect82654" + inkscape:original-d="m 555.30362,244.42669 c -47.49216,14.92912 -94.9853,29.85925 -142.47946,44.79037 -14.67087,-1.31697 -32.65329,-2.92987 -48.98145,-4.3933" + sodipodi:nodetypes="ccc" /> + <path + style="fill:none;stroke:#00ffff;stroke-width:0.99600399;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.99600399, 1.99200797;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#marker90128)" + d="m 517.47596,257.39726 c -11.27308,-12.19333 -23.09732,-24.98281 -44.07722,-34.52993 -20.97991,-9.54711 -51.37607,-16.14473 -61.1594,-18.62006 -9.78333,-2.47533 1.05705,-0.8257 1.05672,-0.82575 -3.2e-4,-5e-5 -10.84089,-1.6497 -20.89115,-3.69115 -10.05026,-2.04144 -19.30542,-4.47381 -28.56219,-6.90661" + id="path82656" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect82658" + inkscape:original-d="m 517.47596,257.39726 c -11.27204,-12.19429 -23.09628,-24.98377 -34.64823,-37.47666 -30.40865,-6.60154 -60.80481,-13.19916 -91.21677,-19.79899 10.84522,1.64921 21.6856,3.29883 32.52691,4.94975 -10.84196,-1.65102 -21.68253,-3.30067 -32.52691,-4.94975 -9.256,-2.43386 -18.51116,-4.86623 -27.76824,-7.29785" + sodipodi:nodetypes="cccccc" /> + <path + style="fill:none;stroke:#12efe9;stroke-width:0.8;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;stroke-miterlimit:4;stroke-dasharray:0.8, 0.80000000000000004;stroke-dashoffset:0;marker-end:url(#marker7126);marker-start:url(#marker92278)" + d="m 552.8313,186.44394 c -1.88462,0 -3.77023,0 -8.35845,1.03362 -4.58822,1.03362 -16.15339,4.31326 -20.51447,10.67756 -4.36107,6.3643 -3.65405,16.41734 -4.36114,28.39826 -0.70708,11.98091 -2.82821,25.88606 -3.18187,36.41572 -0.35366,10.52966 1.06044,17.68103 8.01475,22.985 6.9543,5.30396 19.44517,8.75824 31.93672,12.21271" + id="path91622" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect91624" + inkscape:original-d="m 552.8313,186.44394 c -1.88462,-0.001 -3.77023,-0.001 -5.65685,0 -7.07382,2.82893 -18.85518,5.34621 -28.28427,8.02082 0.7082,10.05458 1.41521,20.10763 2.12132,30.16295 -2.12052,13.90671 -4.24164,27.81186 -6.36396,41.7193 1.41533,7.15152 2.82943,14.30289 4.24264,21.45584 12.49457,3.45403 24.98544,6.90831 37.47666,10.36396" /> + <rect + style="fill:#ffffff;fill-opacity:0;stroke:#00ffff;stroke-width:0.80000001;stroke-miterlimit:4;stroke-dasharray:0.8, 0.8;stroke-dashoffset:0;stroke-opacity:1" + id="rect93634" + width="0.70710677" + height="3.5355339" + x="615.0567" + y="54.214977" /> + </g> +</svg> diff --git a/doc/guides/tools/img/eventdev_perf_queue_test.svg b/doc/guides/tools/img/eventdev_perf_queue_test.svg new file mode 100644 index 00000000..f87ee367 --- /dev/null +++ b/doc/guides/tools/img/eventdev_perf_queue_test.svg @@ -0,0 +1,2599 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<!-- Created with Inkscape (http://www.inkscape.org/) --> + +<!-- +# BSD LICENSE +# +# Copyright (c) 2017, Cavium, Inc +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# +# - Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# +# - Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in +# the documentation and/or other materials provided with the +# distribution. +# +# - Neither the name of Cavium, Inc nor the names of its +# contributors may be used to endorse or promote products derived +# from this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS +# FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE +# COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, +# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES +# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +# STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) +# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED +# OF THE POSSIBILITY OF SUCH DAMAGE. +--> + +<svg + xmlns:osb="http://www.openswatchbook.org/uri/2009/osb" + xmlns:dc="http://purl.org/dc/elements/1.1/" + xmlns:cc="http://creativecommons.org/ns#" + xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" + xmlns:svg="http://www.w3.org/2000/svg" + xmlns="http://www.w3.org/2000/svg" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" + xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" + width="631.91431" + height="288.34286" + id="svg3868" + version="1.1" + inkscape:version="0.92.1 r" + sodipodi:docname="perf_queue.svg" + sodipodi:version="0.32" + inkscape:output_extension="org.inkscape.output.svg.inkscape" + enable-background="new"> + <defs + id="defs3870"> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker28236" + refX="0.0" + refY="0.0" + orient="auto" + inkscape:stockid="Arrow2Mstart"> + <path + transform="scale(0.6) translate(0,0)" + d="M 8.7185878,4.0337352 L -2.2072895,0.016013256 L 8.7185884,-4.0017078 C 6.9730900,-1.6296469 6.9831476,1.6157441 8.7185878,4.0337352 z " + style="fill-rule:evenodd;stroke-width:0.625;stroke-linejoin:round;stroke:#000000;stroke-opacity:1;fill:#000000;fill-opacity:1" + id="path28234" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker27764" + refX="0.0" + refY="0.0" + orient="auto" + inkscape:stockid="Arrow2Mstart"> + <path + transform="scale(0.6) translate(0,0)" + d="M 8.7185878,4.0337352 L -2.2072895,0.016013256 L 8.7185884,-4.0017078 C 6.9730900,-1.6296469 6.9831476,1.6157441 8.7185878,4.0337352 z " + style="fill-rule:evenodd;stroke-width:0.625;stroke-linejoin:round;stroke:#000000;stroke-opacity:1;fill:#000000;fill-opacity:1" + id="path27762" /> + </marker> + <marker + inkscape:isstock="true" + orient="auto" + refY="0" + refX="0" + id="marker20023" + inkscape:stockid="InfiniteLineStart" + style="overflow:visible"> + <g + transform="translate(-13,0)" + style="fill:#000000;stroke:#000000;stroke-opacity:1;fill-opacity:1" + id="g20021"> + <circle + style="stroke:#000000;stroke-opacity:1;fill:#000000;fill-opacity:1" + cx="3" + cy="0" + r="0.8" + id="circle20015" /> + <circle + style="stroke:#000000;stroke-opacity:1;fill:#000000;fill-opacity:1" + cx="6.5" + cy="0" + r="0.8" + id="circle20017" /> + <circle + style="stroke:#000000;stroke-opacity:1;fill:#000000;fill-opacity:1" + cx="10" + cy="0" + r="0.8" + id="circle20019" /> + </g> + </marker> + <marker + inkscape:isstock="true" + orient="auto" + refY="0" + refX="0" + id="marker19992" + inkscape:stockid="InfiniteLineStart" + style="overflow:visible"> + <g + transform="translate(-13,0)" + style="fill:#000000;stroke:#000000;stroke-opacity:1;fill-opacity:1" + id="g19990"> + <circle + style="stroke:#000000;stroke-opacity:1;fill:#000000;fill-opacity:1" + cx="3" + cy="0" + r="0.8" + id="circle19984" /> + <circle + style="stroke:#000000;stroke-opacity:1;fill:#000000;fill-opacity:1" + cx="6.5" + cy="0" + r="0.8" + id="circle19986" /> + <circle + style="stroke:#000000;stroke-opacity:1;fill:#000000;fill-opacity:1" + cx="10" + cy="0" + r="0.8" + id="circle19988" /> + </g> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker18966" + refX="0.0" + refY="0.0" + orient="auto" + inkscape:stockid="Tail"> + <g + style="stroke:#000000;stroke-opacity:1;fill:#000000;fill-opacity:1" + transform="scale(-1.2)" + id="g18964"> + <path + style="fill:#000000;fill-rule:evenodd;stroke:#000000;stroke-width:0.8;stroke-linecap:round;stroke-opacity:1;fill-opacity:1" + d="M -3.8048674,-3.9585227 L 0.54352094,0" + id="path18952" /> + <path + style="fill:#000000;fill-rule:evenodd;stroke:#000000;stroke-width:0.8;stroke-linecap:round;stroke-opacity:1;fill-opacity:1" + d="M -1.2866832,-3.9585227 L 3.0617053,0" + id="path18954" /> + <path + style="fill:#000000;fill-rule:evenodd;stroke:#000000;stroke-width:0.8;stroke-linecap:round;stroke-opacity:1;fill-opacity:1" + d="M 1.3053582,-3.9585227 L 5.6537466,0" + id="path18956" /> + <path + style="fill:#000000;fill-rule:evenodd;stroke:#000000;stroke-width:0.8;stroke-linecap:round;stroke-opacity:1;fill-opacity:1" + d="M -3.8048674,4.1775838 L 0.54352094,0.21974226" + id="path18958" /> + <path + style="fill:#000000;fill-rule:evenodd;stroke:#000000;stroke-width:0.8;stroke-linecap:round;stroke-opacity:1;fill-opacity:1" + d="M -1.2866832,4.1775838 L 3.0617053,0.21974226" + id="path18960" /> + <path + style="fill:#000000;fill-rule:evenodd;stroke:#000000;stroke-width:0.8;stroke-linecap:round;stroke-opacity:1;fill-opacity:1" + d="M 1.3053582,4.1775838 L 5.6537466,0.21974226" + id="path18962" /> + </g> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker18494" + refX="0.0" + refY="0.0" + orient="auto" + inkscape:stockid="Tail"> + <g + style="stroke:#000000;stroke-opacity:1;fill:#000000;fill-opacity:1" + transform="scale(-1.2)" + id="g18492"> + <path + style="fill:#000000;fill-rule:evenodd;stroke:#000000;stroke-width:0.8;stroke-linecap:round;stroke-opacity:1;fill-opacity:1" + d="M -3.8048674,-3.9585227 L 0.54352094,0" + id="path18480" /> + <path + style="fill:#000000;fill-rule:evenodd;stroke:#000000;stroke-width:0.8;stroke-linecap:round;stroke-opacity:1;fill-opacity:1" + d="M -1.2866832,-3.9585227 L 3.0617053,0" + id="path18482" /> + <path + style="fill:#000000;fill-rule:evenodd;stroke:#000000;stroke-width:0.8;stroke-linecap:round;stroke-opacity:1;fill-opacity:1" + d="M 1.3053582,-3.9585227 L 5.6537466,0" + id="path18484" /> + <path + style="fill:#000000;fill-rule:evenodd;stroke:#000000;stroke-width:0.8;stroke-linecap:round;stroke-opacity:1;fill-opacity:1" + d="M -3.8048674,4.1775838 L 0.54352094,0.21974226" + id="path18486" /> + <path + style="fill:#000000;fill-rule:evenodd;stroke:#000000;stroke-width:0.8;stroke-linecap:round;stroke-opacity:1;fill-opacity:1" + d="M -1.2866832,4.1775838 L 3.0617053,0.21974226" + id="path18488" /> + <path + style="fill:#000000;fill-rule:evenodd;stroke:#000000;stroke-width:0.8;stroke-linecap:round;stroke-opacity:1;fill-opacity:1" + d="M 1.3053582,4.1775838 L 5.6537466,0.21974226" + id="path18490" /> + </g> + </marker> + <marker + inkscape:stockid="TriangleOutM" + orient="auto" + refY="0.0" + refX="0.0" + id="marker17998" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path17996" + d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z " + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1;fill:#000000;fill-opacity:1" + transform="scale(0.4)" /> + </marker> + <marker + inkscape:stockid="TriangleOutM" + orient="auto" + refY="0.0" + refX="0.0" + id="marker17586" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path17584" + d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z " + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1;fill:#000000;fill-opacity:1" + transform="scale(0.4)" /> + </marker> + <marker + inkscape:stockid="TriangleOutM" + orient="auto" + refY="0.0" + refX="0.0" + id="marker17186" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path17184" + d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z " + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1;fill:#000000;fill-opacity:1" + transform="scale(0.4)" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker16768" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="EmptyTriangleOutM"> + <path + transform="matrix(0.4,0,0,0.4,-1.8,0)" + style="fill:#ffffff;fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;stroke-opacity:1" + d="M 5.77,0 -2.88,5 V -5 Z" + id="path16766" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker16380" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="EmptyTriangleOutM"> + <path + transform="matrix(0.4,0,0,0.4,-1.8,0)" + style="fill:#ffffff;fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;stroke-opacity:1" + d="M 5.77,0 -2.88,5 V -5 Z" + id="path16378" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker15998" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="EmptyTriangleOutM"> + <path + transform="matrix(0.4,0,0,0.4,-1.8,0)" + style="fill:#ffffff;fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;stroke-opacity:1" + d="M 5.77,0 -2.88,5 V -5 Z" + id="path15996" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="EmptyTriangleOutM" + orient="auto" + refY="0.0" + refX="0.0" + id="marker15604" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path15602" + d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z " + style="fill-rule:evenodd;fill:#ffffff;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="scale(0.4) translate(-4.5,0)" /> + </marker> + <marker + inkscape:stockid="EmptyTriangleOutM" + orient="auto" + refY="0.0" + refX="0.0" + id="marker15234" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path15232" + d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z " + style="fill-rule:evenodd;fill:#ffffff;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="scale(0.4) translate(-4.5,0)" /> + </marker> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker14500" + refX="0.0" + refY="0.0" + orient="auto" + inkscape:stockid="EmptyTriangleOutM"> + <path + transform="scale(0.4) translate(-4.5,0)" + style="fill-rule:evenodd;fill:#ffffff;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z " + id="path14498" /> + </marker> + <inkscape:path-effect + effect="spiro" + id="path-effect14484" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect14480" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect14473" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect14469" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect14461" + is_visible="true" /> + <marker + inkscape:stockid="Arrow2Mstart" + orient="auto" + refY="0.0" + refX="0.0" + id="Arrow2Mstart" + style="overflow:visible" + inkscape:isstock="true" + inkscape:collect="always"> + <path + id="path2002" + style="fill-rule:evenodd;stroke-width:0.625;stroke-linejoin:round;stroke:#000000;stroke-opacity:1;fill:#000000;fill-opacity:1" + d="M 8.7185878,4.0337352 L -2.2072895,0.016013256 L 8.7185884,-4.0017078 C 6.9730900,-1.6296469 6.9831476,1.6157441 8.7185878,4.0337352 z " + transform="scale(0.6) translate(0,0)" /> + </marker> + <marker + inkscape:stockid="TriangleOutM" + orient="auto" + refY="0.0" + refX="0.0" + id="marker13075" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path13073" + d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z " + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1;fill:#000000;fill-opacity:1" + transform="scale(0.4)" /> + </marker> + <inkscape:path-effect + effect="spiro" + id="path-effect13065" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect13061" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect13057" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect13053" + is_visible="true" /> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker7719" + refX="0.0" + refY="0.0" + orient="auto" + inkscape:stockid="TriangleOutM" + inkscape:collect="always"> + <path + transform="scale(0.4)" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1;fill:#000000;fill-opacity:1" + d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z " + id="path7717" /> + </marker> + <marker + inkscape:stockid="TriangleOutM" + orient="auto" + refY="0.0" + refX="0.0" + id="TriangleOutM" + style="overflow:visible" + inkscape:isstock="true" + inkscape:collect="always"> + <path + id="path2123" + d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z " + style="fill-rule:evenodd;stroke:#f78202;stroke-width:1pt;stroke-opacity:1;fill:#f78202;fill-opacity:1" + transform="scale(0.4)" /> + </marker> + <marker + inkscape:stockid="Arrow1Send" + orient="auto" + refY="0.0" + refX="0.0" + id="marker7179" + style="overflow:visible;" + inkscape:isstock="true"> + <path + id="path7177" + d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z " + style="fill-rule:evenodd;stroke:#f78202;stroke-width:1pt;stroke-opacity:1;fill:#f78202;fill-opacity:1" + transform="scale(0.2) rotate(180) translate(6,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Send" + orient="auto" + refY="0.0" + refX="0.0" + id="Arrow1Send" + style="overflow:visible;" + inkscape:isstock="true"> + <path + id="path1993" + d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z " + style="fill-rule:evenodd;stroke:#ff141a;stroke-width:1pt;stroke-opacity:1;fill:#ff141a;fill-opacity:1" + transform="scale(0.2) rotate(180) translate(6,0)" /> + </marker> + <inkscape:path-effect + effect="bspline" + id="path-effect5228" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <marker + inkscape:stockid="DotM" + orient="auto" + refY="0.0" + refX="0.0" + id="DotM" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path2042" + d="M -2.5,-1.0 C -2.5,1.7600000 -4.7400000,4.0 -7.5,4.0 C -10.260000,4.0 -12.5,1.7600000 -12.5,-1.0 C -12.5,-3.7600000 -10.260000,-6.0 -7.5,-6.0 C -4.7400000,-6.0 -2.5,-3.7600000 -2.5,-1.0 z " + style="fill-rule:evenodd;stroke:#ff141a;stroke-width:1pt;stroke-opacity:1;fill:#ff141a;fill-opacity:1" + transform="scale(0.4) translate(7.4, 1)" /> + </marker> + <marker + inkscape:stockid="DiamondS" + orient="auto" + refY="0.0" + refX="0.0" + id="DiamondS" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path2063" + d="M 0,-7.0710768 L -7.0710894,0 L 0,7.0710589 L 7.0710462,0 L 0,-7.0710768 z " + style="fill-rule:evenodd;stroke:#ff141a;stroke-width:1pt;stroke-opacity:1;fill:#ff141a;fill-opacity:1" + transform="scale(0.2)" /> + </marker> + <marker + inkscape:stockid="EmptyTriangleOutM" + orient="auto" + refY="0.0" + refX="0.0" + id="EmptyTriangleOutM" + style="overflow:visible" + inkscape:isstock="true" + inkscape:collect="always"> + <path + id="path2141" + d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z " + style="fill-rule:evenodd;fill:#ffffff;stroke:#358611;stroke-width:1pt;stroke-opacity:0.95703125" + transform="scale(0.4) translate(-4.5,0)" /> + </marker> + <marker + inkscape:stockid="StopL" + orient="auto" + refY="0.0" + refX="0.0" + id="StopL" + style="overflow:visible" + inkscape:isstock="true"> + <path + id="path2147" + d="M 0.0,5.65 L 0.0,-5.65" + style="fill:none;fill-opacity:0.75;fill-rule:evenodd;stroke:#000000;stroke-width:1pt;stroke-opacity:1" + transform="scale(0.8)" /> + </marker> + <marker + inkscape:stockid="Tail" + orient="auto" + refY="0.0" + refX="0.0" + id="Tail" + style="overflow:visible" + inkscape:isstock="true"> + <g + id="g2026" + transform="scale(-1.2)" + style="stroke:#000000;stroke-opacity:1;fill:#000000;fill-opacity:1"> + <path + id="path2014" + d="M -3.8048674,-3.9585227 L 0.54352094,0" + style="fill:#000000;fill-rule:evenodd;stroke:#000000;stroke-width:0.8;stroke-linecap:round;stroke-opacity:1;fill-opacity:1" /> + <path + id="path2016" + d="M -1.2866832,-3.9585227 L 3.0617053,0" + style="fill:#000000;fill-rule:evenodd;stroke:#000000;stroke-width:0.8;stroke-linecap:round;stroke-opacity:1;fill-opacity:1" /> + <path + id="path2018" + d="M 1.3053582,-3.9585227 L 5.6537466,0" + style="fill:#000000;fill-rule:evenodd;stroke:#000000;stroke-width:0.8;stroke-linecap:round;stroke-opacity:1;fill-opacity:1" /> + <path + id="path2020" + d="M -3.8048674,4.1775838 L 0.54352094,0.21974226" + style="fill:#000000;fill-rule:evenodd;stroke:#000000;stroke-width:0.8;stroke-linecap:round;stroke-opacity:1;fill-opacity:1" /> + <path + id="path2022" + d="M -1.2866832,4.1775838 L 3.0617053,0.21974226" + style="fill:#000000;fill-rule:evenodd;stroke:#000000;stroke-width:0.8;stroke-linecap:round;stroke-opacity:1;fill-opacity:1" /> + <path + id="path2024" + d="M 1.3053582,4.1775838 L 5.6537466,0.21974226" + style="fill:#000000;fill-rule:evenodd;stroke:#000000;stroke-width:0.8;stroke-linecap:round;stroke-opacity:1;fill-opacity:1" /> + </g> + </marker> + <inkscape:path-effect + effect="bspline" + id="path-effect2658" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect1940" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="spiro" + id="path-effect1932" + is_visible="true" /> + <linearGradient + id="linearGradient1758" + osb:paint="solid"> + <stop + style="stop-color:#ff2d00;stop-opacity:1;" + offset="0" + id="stop1756" /> + </linearGradient> + <linearGradient + id="linearGradient6425" + osb:paint="solid"> + <stop + style="stop-color:#e6860b;stop-opacity:1;" + offset="0" + id="stop6423" /> + </linearGradient> + <linearGradient + id="linearGradient6391" + osb:paint="solid"> + <stop + style="stop-color:#000000;stop-opacity:1;" + offset="0" + id="stop6389" /> + </linearGradient> + <inkscape:path-effect + effect="spiro" + id="path-effect6387" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect6037" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect6033" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect6029" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect6025" + is_visible="true" /> + <linearGradient + id="linearGradient5213" + osb:paint="solid"> + <stop + style="stop-color:#ff0009;stop-opacity:1;" + offset="0" + id="stop5211" /> + </linearGradient> + <inkscape:path-effect + effect="spiro" + id="path-effect4276" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect4272" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect4268" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect4264" + is_visible="true" /> + <linearGradient + id="linearGradient2975" + osb:paint="solid"> + <stop + style="stop-color:#ff2200;stop-opacity:1;" + offset="0" + id="stop2973" /> + </linearGradient> + <linearGradient + id="linearGradient2969" + osb:paint="solid"> + <stop + style="stop-color:#009a08;stop-opacity:1;" + offset="0" + id="stop2967" /> + </linearGradient> + <linearGradient + id="linearGradient2963" + osb:paint="solid"> + <stop + style="stop-color:#000000;stop-opacity:1;" + offset="0" + id="stop2961" /> + </linearGradient> + <linearGradient + id="linearGradient2929" + osb:paint="solid"> + <stop + style="stop-color:#ff2d00;stop-opacity:1;" + offset="0" + id="stop2927" /> + </linearGradient> + <linearGradient + id="linearGradient4610" + osb:paint="solid"> + <stop + style="stop-color:#00ffff;stop-opacity:1;" + offset="0" + id="stop4608" /> + </linearGradient> + <linearGradient + id="linearGradient3993" + osb:paint="solid"> + <stop + style="stop-color:#6ba6fd;stop-opacity:1;" + offset="0" + id="stop3991" /> + </linearGradient> + <linearGradient + id="linearGradient3808" + osb:paint="solid"> + <stop + style="stop-color:#6ba6fd;stop-opacity:1;" + offset="0" + id="stop3806" /> + </linearGradient> + <linearGradient + id="linearGradient3776" + osb:paint="solid"> + <stop + style="stop-color:#fc0000;stop-opacity:1;" + offset="0" + id="stop3774" /> + </linearGradient> + <linearGradient + id="linearGradient3438" + osb:paint="solid"> + <stop + style="stop-color:#d18f21;stop-opacity:1;" + offset="0" + id="stop3436" /> + </linearGradient> + <inkscape:path-effect + effect="spiro" + id="path-effect3408" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3404" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3400" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3392" + is_visible="true" /> + <inkscape:path-effect + effect="bspline" + id="path-effect3376" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect3044" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect3040" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect3036" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect3032" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect3028" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect3024" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="spiro" + id="path-effect3020" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect2858" + is_visible="true" /> + <inkscape:path-effect + effect="spiro" + id="path-effect2854" + is_visible="true" /> + <inkscape:path-effect + effect="bspline" + id="path-effect2844" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <linearGradient + id="linearGradient2828" + osb:paint="solid"> + <stop + style="stop-color:#ff0000;stop-opacity:1;" + offset="0" + id="stop2826" /> + </linearGradient> + <inkscape:path-effect + effect="bspline" + id="path-effect329" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <marker + inkscape:stockid="Arrow1Mstart" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mstart" + style="overflow:visible"> + <path + id="path4530" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(0.4,0,0,0.4,4,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend" + style="overflow:visible"> + <path + id="path4533" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <linearGradient + id="linearGradient4513"> + <stop + style="stop-color:#fdffdb;stop-opacity:1;" + offset="0" + id="stop4515" /> + <stop + style="stop-color:#dfe2d8;stop-opacity:0;" + offset="1" + id="stop4517" /> + </linearGradient> + <inkscape:perspective + sodipodi:type="inkscape:persp3d" + inkscape:vp_x="0 : 526.18109 : 1" + inkscape:vp_y="0 : 1000 : 0" + inkscape:vp_z="744.09448 : 526.18109 : 1" + inkscape:persp3d-origin="372.04724 : 350.78739 : 1" + id="perspective3876" /> + <inkscape:perspective + id="perspective3886" + inkscape:persp3d-origin="0.5 : 0.33333333 : 1" + inkscape:vp_z="1 : 0.5 : 1" + inkscape:vp_y="0 : 1000 : 0" + inkscape:vp_x="0 : 0.5 : 1" + sodipodi:type="inkscape:persp3d" /> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Lend" + style="overflow:visible"> + <path + id="path3211" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker3892" + style="overflow:visible"> + <path + id="path3894" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker3896" + style="overflow:visible"> + <path + id="path3898" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lstart" + orient="auto" + refY="0" + refX="0" + id="Arrow1Lstart" + style="overflow:visible"> + <path + id="path3208" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(0.8,0,0,0.8,10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker3902" + style="overflow:visible"> + <path + id="path3904" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lstart" + orient="auto" + refY="0" + refX="0" + id="marker3906" + style="overflow:visible"> + <path + id="path3908" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(0.8,0,0,0.8,10,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="marker3910" + style="overflow:visible"> + <path + id="path3912" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:perspective + id="perspective4086" + inkscape:persp3d-origin="0.5 : 0.33333333 : 1" + inkscape:vp_z="1 : 0.5 : 1" + inkscape:vp_y="0 : 1000 : 0" + inkscape:vp_x="0 : 0.5 : 1" + sodipodi:type="inkscape:persp3d" /> + <inkscape:perspective + id="perspective4113" + inkscape:persp3d-origin="0.5 : 0.33333333 : 1" + inkscape:vp_z="1 : 0.5 : 1" + inkscape:vp_y="0 : 1000 : 0" + inkscape:vp_x="0 : 0.5 : 1" + sodipodi:type="inkscape:persp3d" /> + <inkscape:perspective + id="perspective5195" + inkscape:persp3d-origin="0.5 : 0.33333333 : 1" + inkscape:vp_z="1 : 0.5 : 1" + inkscape:vp_y="0 : 1000 : 0" + inkscape:vp_x="0 : 0.5 : 1" + sodipodi:type="inkscape:persp3d" /> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-4" + style="overflow:visible"> + <path + id="path4533-7" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:perspective + id="perspective5272" + inkscape:persp3d-origin="0.5 : 0.33333333 : 1" + inkscape:vp_z="1 : 0.5 : 1" + inkscape:vp_y="0 : 1000 : 0" + inkscape:vp_x="0 : 0.5 : 1" + sodipodi:type="inkscape:persp3d" /> + <marker + inkscape:stockid="Arrow1Mstart" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mstart-4" + style="overflow:visible"> + <path + id="path4530-5" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(0.4,0,0,0.4,4,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-0" + style="overflow:visible"> + <path + id="path4533-3" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:perspective + id="perspective5317" + inkscape:persp3d-origin="0.5 : 0.33333333 : 1" + inkscape:vp_z="1 : 0.5 : 1" + inkscape:vp_y="0 : 1000 : 0" + inkscape:vp_x="0 : 0.5 : 1" + sodipodi:type="inkscape:persp3d" /> + <marker + inkscape:stockid="Arrow1Mstart" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mstart-3" + style="overflow:visible"> + <path + id="path4530-2" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(0.4,0,0,0.4,4,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-06" + style="overflow:visible"> + <path + id="path4533-1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Mstart" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mstart-8" + style="overflow:visible"> + <path + id="path4530-7" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(0.4,0,0,0.4,4,0)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-9" + style="overflow:visible"> + <path + id="path4533-2" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:path-effect + effect="spiro" + id="path-effect2858-0" + is_visible="true" /> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-3" + style="overflow:visible"> + <path + id="path4533-75" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:path-effect + effect="bspline" + id="path-effect3044-9" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-3-2" + style="overflow:visible"> + <path + id="path4533-75-8" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:path-effect + effect="bspline" + id="path-effect3044-9-9" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3993" + id="linearGradient3995" + x1="155.21328" + y1="231.61366" + x2="207.95523" + y2="231.61366" + gradientUnits="userSpaceOnUse" + gradientTransform="translate(-14,-48)" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3438" + id="linearGradient4612" + x1="594.77722" + y1="232.19244" + x2="647.51917" + y2="232.19244" + gradientUnits="userSpaceOnUse" + gradientTransform="translate(-40,68)" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3438" + id="linearGradient4616" + x1="468.32343" + y1="232.3177" + x2="521.06543" + y2="232.3177" + gradientUnits="userSpaceOnUse" + gradientTransform="translate(86,14)" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3438" + id="linearGradient4618" + x1="405.4682" + y1="232.36095" + x2="458.21014" + y2="232.36095" + gradientUnits="userSpaceOnUse" + gradientTransform="translate(148,-46)" /> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-2" + style="overflow:visible"> + <path + id="path4533-6" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:path-effect + effect="spiro" + id="path-effect3228" + is_visible="true" /> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="marker3706" + style="overflow:visible"> + <path + id="path3704" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:path-effect + effect="spiro" + id="path-effect3286" + is_visible="true" /> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-1" + style="overflow:visible"> + <path + id="path4533-8" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:path-effect + effect="spiro" + id="path-effect3290" + is_visible="true" /> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-3-7" + style="overflow:visible"> + <path + id="path4533-75-9" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:path-effect + effect="spiro" + id="path-effect3120-7" + is_visible="true" /> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-4-2" + style="overflow:visible"> + <path + id="path4533-7-0" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:path-effect + effect="spiro" + id="path-effect6025-2" + is_visible="true" /> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-7" + style="overflow:visible"> + <path + id="path4533-5" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:path-effect + effect="spiro" + id="path-effect3294" + is_visible="true" /> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-92" + style="overflow:visible"> + <path + id="path4533-28" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:path-effect + effect="spiro" + id="path-effect3302" + is_visible="true" /> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Mend-97" + style="overflow:visible"> + <path + id="path4533-36" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;marker-start:none" + transform="matrix(-0.4,0,0,-0.4,-4,0)" + inkscape:connector-curvature="0" /> + </marker> + <inkscape:path-effect + effect="spiro" + id="path-effect3228-1" + is_visible="true" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3808" + id="linearGradient1760" + x1="405.34961" + y1="243.36557" + x2="651.55652" + y2="243.36557" + gradientUnits="userSpaceOnUse" + gradientTransform="translate(-158,2)" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3808" + id="linearGradient1918" + x1="415.62723" + y1="156.24651" + x2="455.76093" + y2="156.24651" + gradientUnits="userSpaceOnUse" + gradientTransform="translate(-156,28)" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3808" + id="linearGradient1920" + x1="475.00314" + y1="156.97769" + x2="515.13684" + y2="156.97769" + gradientUnits="userSpaceOnUse" + gradientTransform="translate(-154,28)" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3808" + id="linearGradient1922" + x1="537.74072" + y1="156.9726" + x2="577.87439" + y2="156.9726" + gradientUnits="userSpaceOnUse" + gradientTransform="translate(-100,28)" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3808" + id="linearGradient1924" + x1="597.00317" + y1="156.97769" + x2="637.13684" + y2="156.97769" + gradientUnits="userSpaceOnUse" + gradientTransform="translate(-218,28)" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient4513" + id="linearGradient4519" + x1="47.142857" + y1="244.50504" + x2="677.85718" + y2="244.50504" + gradientUnits="userSpaceOnUse" + gradientTransform="matrix(0.98357201,0,0,0.98599728,7.8873958,3.6023064)" /> + <linearGradient + gradientTransform="matrix(0.9887388,0,0,1.0000197,5.0811445,-0.1708579)" + inkscape:collect="always" + xlink:href="#linearGradient6391" + id="linearGradient2965" + x1="49.239536" + y1="244.84964" + x2="677.64832" + y2="244.84964" + gradientUnits="userSpaceOnUse" /> + <marker + inkscape:stockid="TriangleOutM" + orient="auto" + refY="0" + refX="0" + id="TriangleOutM-5" + style="overflow:visible" + inkscape:isstock="true" + inkscape:collect="always"> + <path + inkscape:connector-curvature="0" + id="path2123-3" + d="M 5.77,0 -2.88,5 V -5 Z" + style="fill:#f78202;fill-opacity:1;fill-rule:evenodd;stroke:#f78202;stroke-width:1.00000003pt;stroke-opacity:1" + transform="scale(0.4)" /> + </marker> + <inkscape:path-effect + effect="bspline" + id="path-effect5228-5" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <marker + inkscape:stockid="TriangleOutM" + orient="auto" + refY="0" + refX="0" + id="TriangleOutM-5-2" + style="overflow:visible" + inkscape:isstock="true" + inkscape:collect="always"> + <path + inkscape:connector-curvature="0" + id="path2123-3-9" + d="M 5.77,0 -2.88,5 V -5 Z" + style="fill:#f78202;fill-opacity:1;fill-rule:evenodd;stroke:#f78202;stroke-width:1.00000003pt;stroke-opacity:1" + transform="scale(0.4)" /> + </marker> + <inkscape:path-effect + effect="bspline" + id="path-effect5228-5-1" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <marker + inkscape:stockid="EmptyTriangleOutM" + orient="auto" + refY="0" + refX="0" + id="EmptyTriangleOutM-7" + style="overflow:visible" + inkscape:isstock="true" + inkscape:collect="always"> + <path + inkscape:connector-curvature="0" + id="path2141-0" + d="M 5.77,0 -2.88,5 V -5 Z" + style="fill:#ffffff;fill-rule:evenodd;stroke:#358611;stroke-width:1.00000003pt;stroke-opacity:0.95703125" + transform="matrix(0.4,0,0,0.4,-1.8,0)" /> + </marker> + <inkscape:path-effect + effect="bspline" + id="path-effect2658-9" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <inkscape:path-effect + effect="bspline" + id="path-effect1940-3" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3993" + id="linearGradient1924-6" + x1="597.00317" + y1="156.97769" + x2="637.13684" + y2="156.97769" + gradientUnits="userSpaceOnUse" + gradientTransform="translate(-158.08539,84.05654)" /> + <marker + inkscape:stockid="TriangleOutM" + orient="auto" + refY="0" + refX="0" + id="TriangleOutM-5-2-6" + style="overflow:visible" + inkscape:isstock="true" + inkscape:collect="always"> + <path + inkscape:connector-curvature="0" + id="path2123-3-9-2" + d="M 5.77,0 -2.88,5 V -5 Z" + style="fill:#f78202;fill-opacity:1;fill-rule:evenodd;stroke:#f78202;stroke-width:1.00000003pt;stroke-opacity:1" + transform="scale(0.4)" /> + </marker> + <inkscape:path-effect + effect="bspline" + id="path-effect5228-5-1-6" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3993" + id="linearGradient1922-1" + x1="537.74072" + y1="156.9726" + x2="577.87439" + y2="156.9726" + gradientUnits="userSpaceOnUse" + gradientTransform="translate(-158.08539,84.05654)" /> + <marker + inkscape:stockid="TriangleOutM" + orient="auto" + refY="0" + refX="0" + id="TriangleOutM-5-8" + style="overflow:visible" + inkscape:isstock="true" + inkscape:collect="always"> + <path + inkscape:connector-curvature="0" + id="path2123-3-7" + d="M 5.77,0 -2.88,5 V -5 Z" + style="fill:#f78202;fill-opacity:1;fill-rule:evenodd;stroke:#f78202;stroke-width:1.00000003pt;stroke-opacity:1" + transform="scale(0.4)" /> + </marker> + <inkscape:path-effect + effect="bspline" + id="path-effect5228-5-9" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3993" + id="linearGradient1920-2" + x1="475.00314" + y1="156.97769" + x2="515.13684" + y2="156.97769" + gradientUnits="userSpaceOnUse" + gradientTransform="translate(-154.08539,84.05654)" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3993" + id="linearGradient1918-0" + x1="415.62723" + y1="156.24651" + x2="455.76093" + y2="156.24651" + gradientUnits="userSpaceOnUse" + gradientTransform="translate(-156.08539,84.05654)" /> + <marker + inkscape:stockid="TriangleOutM" + orient="auto" + refY="0" + refX="0" + id="TriangleOutM-2" + style="overflow:visible" + inkscape:isstock="true" + inkscape:collect="always"> + <path + inkscape:connector-curvature="0" + id="path2123-37" + d="M 5.77,0 -2.88,5 V -5 Z" + style="fill:#f78202;fill-opacity:1;fill-rule:evenodd;stroke:#f78202;stroke-width:1.00000003pt;stroke-opacity:1" + transform="scale(0.4)" /> + </marker> + <inkscape:path-effect + effect="bspline" + id="path-effect5228-59" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker7719-2" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="TriangleOutM"> + <path + inkscape:connector-curvature="0" + transform="scale(0.4)" + style="fill:#ff141a;fill-opacity:1;fill-rule:evenodd;stroke:#ff141a;stroke-width:1.00000003pt;stroke-opacity:1" + d="M 5.77,0 -2.88,5 V -5 Z" + id="path7717-2" /> + </marker> + <inkscape:path-effect + effect="spiro" + id="path-effect1932-8" + is_visible="true" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3993" + id="linearGradient3995-9" + x1="155.21329" + y1="231.61366" + x2="207.95523" + y2="231.61366" + gradientUnits="userSpaceOnUse" + gradientTransform="translate(-14.08539,8.056541)" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3993" + id="linearGradient3995-9-5" + x1="155.21329" + y1="231.61366" + x2="207.95523" + y2="231.61366" + gradientUnits="userSpaceOnUse" + gradientTransform="translate(-14.08539,62.056546)" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3993" + id="linearGradient1918-0-4" + x1="415.62723" + y1="156.24651" + x2="455.76093" + y2="156.24651" + gradientUnits="userSpaceOnUse" + gradientTransform="translate(-156.08539,138.05655)" /> + <marker + inkscape:isstock="true" + style="overflow:visible" + id="marker7719-2-7" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="TriangleOutM"> + <path + inkscape:connector-curvature="0" + transform="scale(0.4)" + style="fill:#ff141a;fill-opacity:1;fill-rule:evenodd;stroke:#ff141a;stroke-width:1.00000003pt;stroke-opacity:1" + d="M 5.77,0 -2.88,5 V -5 Z" + id="path7717-2-6" /> + </marker> + <inkscape:path-effect + effect="spiro" + id="path-effect1932-8-5" + is_visible="true" /> + <inkscape:path-effect + effect="bspline" + id="path-effect1940-3-6" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <marker + inkscape:stockid="EmptyTriangleOutM" + orient="auto" + refY="0" + refX="0" + id="EmptyTriangleOutM-7-9" + style="overflow:visible" + inkscape:isstock="true" + inkscape:collect="always"> + <path + inkscape:connector-curvature="0" + id="path2141-0-3" + d="M 5.77,0 -2.88,5 V -5 Z" + style="fill:#ffffff;fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;stroke-opacity:1" + transform="matrix(0.4,0,0,0.4,-1.8,0)" /> + </marker> + <inkscape:path-effect + effect="bspline" + id="path-effect2658-9-7" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3993" + id="linearGradient1920-2-4" + x1="475.00314" + y1="156.97769" + x2="515.13684" + y2="156.97769" + gradientUnits="userSpaceOnUse" + gradientTransform="translate(-154.08539,138.05655)" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3993" + id="linearGradient1924-6-5" + x1="597.00317" + y1="156.97769" + x2="637.13684" + y2="156.97769" + gradientUnits="userSpaceOnUse" + gradientTransform="translate(-158.08539,138.05655)" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient3993" + id="linearGradient1922-1-2" + x1="537.74072" + y1="156.9726" + x2="577.87439" + y2="156.9726" + gradientUnits="userSpaceOnUse" + gradientTransform="translate(-158.08539,138.05655)" /> + <marker + inkscape:stockid="TriangleOutM" + orient="auto" + refY="0" + refX="0" + id="TriangleOutM-2-5" + style="overflow:visible" + inkscape:isstock="true" + inkscape:collect="always"> + <path + inkscape:connector-curvature="0" + id="path2123-37-4" + d="M 5.77,0 -2.88,5 V -5 Z" + style="fill:#f78202;fill-opacity:1;fill-rule:evenodd;stroke:#f78202;stroke-width:1.00000003pt;stroke-opacity:1" + transform="scale(0.4)" /> + </marker> + <inkscape:path-effect + effect="bspline" + id="path-effect5228-59-7" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <marker + inkscape:stockid="TriangleOutM" + orient="auto" + refY="0" + refX="0" + id="TriangleOutM-5-8-4" + style="overflow:visible" + inkscape:isstock="true" + inkscape:collect="always"> + <path + inkscape:connector-curvature="0" + id="path2123-3-7-4" + d="M 5.77,0 -2.88,5 V -5 Z" + style="fill:#f78202;fill-opacity:1;fill-rule:evenodd;stroke:#f78202;stroke-width:1.00000003pt;stroke-opacity:1" + transform="scale(0.4)" /> + </marker> + <inkscape:path-effect + effect="bspline" + id="path-effect5228-5-9-3" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + <marker + inkscape:stockid="TriangleOutM" + orient="auto" + refY="0" + refX="0" + id="TriangleOutM-5-2-6-0" + style="overflow:visible" + inkscape:isstock="true" + inkscape:collect="always"> + <path + inkscape:connector-curvature="0" + id="path2123-3-9-2-7" + d="M 5.77,0 -2.88,5 V -5 Z" + style="fill:#f78202;fill-opacity:1;fill-rule:evenodd;stroke:#f78202;stroke-width:1.00000003pt;stroke-opacity:1" + transform="scale(0.4)" /> + </marker> + <inkscape:path-effect + effect="bspline" + id="path-effect5228-5-1-6-8" + is_visible="true" + weight="33.333333" + steps="2" + helper_size="0" + apply_no_weight="true" + apply_with_weight="true" + only_selected="false" /> + </defs> + <sodipodi:namedview + id="base" + pagecolor="#ffffff" + bordercolor="#666666" + borderopacity="1.0" + inkscape:pageopacity="0.0" + inkscape:pageshadow="2" + inkscape:zoom="1.4142136" + inkscape:cx="507.83223" + inkscape:cy="201.88318" + inkscape:document-units="px" + inkscape:current-layer="layer1" + showgrid="false" + inkscape:window-width="1360" + inkscape:window-height="724" + inkscape:window-x="0" + inkscape:window-y="20" + inkscape:window-maximized="0" + fit-margin-top="0.1" + fit-margin-left="0.1" + fit-margin-right="0.1" + fit-margin-bottom="0.1" + inkscape:measure-start="-29.078,219.858" + inkscape:measure-end="346.809,219.858" + showguides="false" /> + <metadata + id="metadata3873"> + <rdf:RDF> + <cc:Work + rdf:about=""> + <dc:format>image/svg+xml</dc:format> + <dc:type + rdf:resource="http://purl.org/dc/dcmitype/StillImage" /> + <dc:title /> + </cc:Work> + </rdf:RDF> + </metadata> + <g + inkscape:label="Layer 1" + inkscape:groupmode="layer" + id="layer1" + transform="translate(-46.542857,-100.33361)" + style="display:inline;opacity:1"> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="117.328" + y="-14.742554" + id="text2978" + inkscape:export-filename="/home/matz/barracuda/rapports/mbuf-api-v2-images/octeon_multi.png" + inkscape:export-xdpi="112" + inkscape:export-ydpi="112"><tspan + sodipodi:role="line" + x="117.328" + y="-14.742554" + id="tspan3006" + style="font-size:15.22520161px;line-height:1.25"> </tspan></text> + <rect + style="display:inline;opacity:1;fill:none;fill-opacity:1;stroke:url(#linearGradient1922);stroke-width:1.06814909;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect3736-8-3-1" + width="39.065548" + height="24.347494" + x="438.27478" + y="172.79883" /> + <rect + style="fill:url(#linearGradient4519);fill-opacity:1;stroke:url(#linearGradient2965);stroke-width:0.98478383;stroke-opacity:1" + id="rect3697" + width="620.35291" + height="283.12207" + x="54.255791" + y="103.1226" + rx="0" + ry="0" /> + <rect + style="fill:#ffffff;fill-opacity:1;stroke:url(#linearGradient4612);stroke-width:1.02699995;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect87-3" + width="51.714954" + height="32.587509" + x="555.29071" + y="283.89868" + rx="11.6051" + ry="16.293755" /> + <rect + style="fill:#ffffff;fill-opacity:1;stroke:url(#linearGradient4616);stroke-width:1.02699995;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect87-6" + width="51.714954" + height="32.587509" + x="554.83691" + y="230.02396" + rx="11.6051" + ry="16.293755" /> + <rect + style="fill:#ffffff;fill-opacity:1;stroke:url(#linearGradient4618);stroke-width:1.02699995;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect87-6-5" + width="51.714954" + height="32.587509" + x="553.98169" + y="170.06718" + rx="11.6051" + ry="16.293755" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="558.87885" + y="167.34842" + id="text5219-2-4"><tspan + sodipodi:role="line" + x="558.87885" + y="167.34842" + id="tspan5223-0-7" + style="font-size:10px;line-height:1.25">worker 0</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="558.61511" + y="227.66943" + id="text5219-2-4-3"><tspan + sodipodi:role="line" + x="558.61511" + y="227.66943" + id="tspan5223-0-7-7" + style="font-size:10px;line-height:1.25">worker 1</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="560.61511" + y="281.66943" + id="text5219-2-4-3-4-2"><tspan + sodipodi:role="line" + x="560.61511" + y="281.66943" + id="tspan5223-0-7-7-5-5" + style="font-size:10px;line-height:1.25">worker n</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="568.13348" + y="188.8974" + id="text5219-2-6-4"><tspan + sodipodi:role="line" + x="568.13348" + y="188.8974" + id="tspan5223-0-9-7" + style="font-size:10px;line-height:1.25">port 0</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="563.25244" + y="248.85495" + id="text5219-2-6-4-4"><tspan + sodipodi:role="line" + x="563.25244" + y="248.85495" + id="tspan5223-0-9-7-4" + style="font-size:10px;line-height:1.25">port 1</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';fill:#000000;fill-opacity:1;stroke:none" + x="567.25244" + y="302.85495" + id="text5219-2-6-4-4-3-7"><tspan + sodipodi:role="line" + x="567.25244" + y="302.85495" + id="tspan5223-0-9-7-4-0-8" + style="font-size:10px;line-height:1.25">port n</tspan></text> + <rect + style="display:inline;opacity:1;fill:#ffffff;fill-opacity:1;stroke:url(#linearGradient3995);stroke-width:1.02699995;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect87-6-5-3" + width="51.714954" + height="32.587509" + x="141.72678" + y="167.31989" + rx="11.6051" + ry="16.293755" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';display:inline;opacity:1;fill:#000000;fill-opacity:1;stroke:none" + x="143.03741" + y="210.07278" + id="text5219-2"><tspan + sodipodi:role="line" + x="143.03741" + y="210.07278" + id="tspan5223-0" + style="font-size:10px;line-height:1.25">producer 0</tspan></text> + <rect + style="display:inline;opacity:1;fill:none;fill-opacity:1;fill-rule:evenodd;stroke:url(#linearGradient1760);stroke-width:0.97884095;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect2896-6" + width="245.22809" + height="223.72733" + x="247.83902" + y="133.50191" + ry="5.6568542" + rx="9.0800323" + inkscape:export-filename="/home/matz/barracuda/rapports/mbuf-api-v2-images/octeon_multi.png" + inkscape:export-xdpi="112" + inkscape:export-ydpi="112" /> + <rect + style="display:inline;opacity:1;fill:none;fill-opacity:1;stroke:url(#linearGradient1918);stroke-width:1.06814909;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect3736-1" + width="39.065548" + height="24.347494" + x="260.16132" + y="172.07275" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';display:inline;opacity:1;fill:#000000;fill-opacity:1;stroke:none" + x="270.72223" + y="169.7077" + id="text5219-1-9-5"><tspan + sodipodi:role="line" + x="270.72223" + y="169.7077" + id="tspan5223-2-3-9" + style="font-size:10px;line-height:1.25">q0</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';display:inline;opacity:1;fill:#000000;fill-opacity:1;stroke:none" + x="326.09811" + y="168.4389" + id="text5219-1-9-4-9"><tspan + sodipodi:role="line" + x="326.09811" + y="168.4389" + id="tspan5223-2-3-5-0" + style="font-size:10px;line-height:1.25">q1</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';display:inline;opacity:1;fill:#000000;fill-opacity:1;stroke:none" + x="392.09808" + y="170.4389" + id="text5219-1-9-4-3-9"><tspan + sodipodi:role="line" + x="392.09808" + y="170.4389" + id="tspan5223-2-3-5-6-1" + style="font-size:10px;line-height:1.25">q2</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';display:inline;opacity:1;fill:#000000;fill-opacity:1;stroke:none" + x="446.09808" + y="170.4389" + id="text5219-1-9-4-3-0-7"><tspan + sodipodi:role="line" + x="446.09808" + y="170.4389" + id="tspan5223-2-3-5-6-6-1" + style="font-size:10px;line-height:1.25">qs-1</tspan></text> + <path + style="display:inline;opacity:1;fill:none;stroke:#ff141a;stroke-width:0.89999998;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#marker7719)" + d="m 192.59877,183.45256 h 65.05382" + id="path1930" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect1932" + inkscape:original-d="m 192.59877,183.45256 c 21.68561,-0.001 43.37021,-0.001 65.05382,0" + sodipodi:nodetypes="cc" /> + <path + style="display:inline;opacity:1;fill:none;stroke:#358611;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:1, 1;stroke-dashoffset:0;stroke-opacity:1" + d="m 478.2462,184.07275 c 9.43699,0 18.87298,0 23.47261,-0.12707 4.59962,-0.12707 4.36395,-0.38114 4.12825,-0.63524" + id="path1938" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect1940" + inkscape:original-d="m 478.2462,184.07275 c 9.43699,-0.001 18.87298,-0.001 28.30797,0 -0.2347,-0.2551 -0.70711,-0.76231 -0.70711,-0.76231" + sodipodi:nodetypes="ccc" /> + <path + style="display:inline;opacity:1;fill:none;stroke:#358611;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:1, 1;stroke-dashoffset:0;stroke-opacity:0.95703125;marker-end:url(#EmptyTriangleOutM)" + d="m 505.84632,184.68305 c 0,8.01981 0,16.04062 0,24.06243" + id="path2656" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect2658" + inkscape:original-d="m 505.84632,184.68305 c 0.001,8.01981 0.001,16.04062 0,24.06243" + sodipodi:nodetypes="cc" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';display:inline;opacity:1;fill:#000000;fill-opacity:1;stroke:none" + x="143.44385" + y="186.49918" + id="text5219-2-6"><tspan + sodipodi:role="line" + x="143.44385" + y="186.49918" + id="tspan5223-0-9" + style="font-size:10px;line-height:1.25">port n+1</tspan></text> + <rect + style="display:inline;opacity:1;fill:none;fill-opacity:1;stroke:url(#linearGradient1920);stroke-width:1.06814909;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect3736-8-4" + width="39.065548" + height="24.347494" + x="321.5372" + y="172.80396" /> + <rect + style="display:inline;opacity:1;fill:none;fill-opacity:1;stroke:url(#linearGradient1924);stroke-width:1.06814909;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect3736-8-0-1-7" + width="39.065548" + height="24.347494" + x="379.53723" + y="172.80396" /> + <path + style="display:inline;opacity:1;fill:none;stroke:#f78202;stroke-width:1.08672047;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleOutM)" + d="m 299.22687,182.77736 c 6.46827,0.01 12.93534,0.0194 19.40121,0.0291" + id="path5226" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect5228" + inkscape:original-d="m 299.22687,182.77736 c 6.46827,0.008 12.93534,0.0182 19.40121,0.0291" + sodipodi:nodetypes="cc" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';display:inline;opacity:1;fill:#000000;fill-opacity:1;stroke:none" + x="540.47687" + y="378.4664" + id="text2912" + inkscape:export-filename="/home/matz/barracuda/rapports/mbuf-api-v2-images/octeon_multi.png" + inkscape:export-xdpi="112" + inkscape:export-ydpi="112"><tspan + sodipodi:role="line" + x="540.47687" + y="378.4664" + id="tspan2916" + style="font-weight:bold;font-size:13.33333302px;line-height:1.25">test: perf_queue</tspan></text> + <path + style="display:inline;opacity:1;fill:none;stroke:#f78202;stroke-width:0.89999998;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleOutM-5)" + d="m 360.66672,182.86561 c 5.35689,0.008 10.71279,0.0161 16.06769,0.0241" + id="path5226-6" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect5228-5" + inkscape:original-d="m 360.66672,182.86561 c 5.35689,0.007 10.71279,0.0151 16.06769,0.0241" + sodipodi:nodetypes="cc" /> + <path + style="display:inline;opacity:1;fill:none;stroke:#f78202;stroke-width:0.89999998;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleOutM-5-2)" + d="m 419.73779,183.57272 c 5.35689,0.008 10.71279,0.0161 16.06769,0.0241" + id="path5226-6-2" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect5228-5-1" + inkscape:original-d="m 419.73779,183.57272 c 5.35689,0.007 10.71279,0.0151 16.06769,0.0241" + sodipodi:nodetypes="cc" /> + <rect + style="display:inline;opacity:1;fill:#ffffff;fill-opacity:1;stroke:url(#linearGradient3995-9);stroke-width:1.02699995;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect87-6-5-3-3" + width="51.714954" + height="32.587509" + x="141.64139" + y="223.3764" + rx="11.6051" + ry="16.293755" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';display:inline;opacity:1;fill:#000000;fill-opacity:1;stroke:none" + x="142.95203" + y="266.12933" + id="text5219-2-61"><tspan + sodipodi:role="line" + x="142.95203" + y="266.12933" + id="tspan5223-0-2" + style="font-size:10px;line-height:1.25">producer 1</tspan></text> + <rect + style="display:inline;opacity:1;fill:none;fill-opacity:1;stroke:url(#linearGradient1918-0);stroke-width:1.06814909;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect3736-1-9" + width="39.065548" + height="24.347494" + x="260.07593" + y="228.12927" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';display:inline;opacity:1;fill:#000000;fill-opacity:1;stroke:none" + x="270.63684" + y="225.76422" + id="text5219-1-9-5-3"><tspan + sodipodi:role="line" + x="270.63684" + y="225.76422" + id="tspan5223-2-3-9-1" + style="font-size:10px;line-height:1.25">qs</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';display:inline;opacity:1;fill:#000000;fill-opacity:1;stroke:none" + x="326.01276" + y="224.49542" + id="text5219-1-9-4-9-9"><tspan + sodipodi:role="line" + x="326.01276" + y="224.49542" + id="tspan5223-2-3-5-0-4" + style="font-size:10px;line-height:1.25">qs+1</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';display:inline;opacity:1;fill:#000000;fill-opacity:1;stroke:none" + x="386.0127" + y="226.49542" + id="text5219-1-9-4-3-9-7"><tspan + sodipodi:role="line" + x="386.0127" + y="226.49542" + id="tspan5223-2-3-5-6-1-8" + style="font-size:10px;line-height:1.25">qs+2</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';display:inline;opacity:1;fill:#000000;fill-opacity:1;stroke:none" + x="446.0127" + y="226.49542" + id="text5219-1-9-4-3-0-7-4"><tspan + sodipodi:role="line" + x="446.0127" + y="226.49542" + id="tspan5223-2-3-5-6-6-1-5" + style="font-size:10px;line-height:1.25">q2s-1</tspan></text> + <path + style="display:inline;opacity:1;fill:none;stroke:#ff141a;stroke-width:0.89999998;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#marker7719-2)" + d="M 192.51338,239.5091 H 257.5672" + id="path1930-0" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect1932-8" + inkscape:original-d="m 192.51338,239.5091 c 21.68561,-10e-4 43.37021,-10e-4 65.05382,0" + sodipodi:nodetypes="cc" /> + <path + style="display:inline;opacity:1;fill:none;stroke:#358611;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:1, 1;stroke-dashoffset:0;stroke-opacity:1" + d="m 478.16081,240.12929 c 9.43699,0 18.87298,0 23.47261,-0.12707 4.59962,-0.12707 4.36395,-0.38114 4.12825,-0.63524" + id="path1938-3" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect1940-3" + inkscape:original-d="m 478.16081,240.12929 c 9.43699,-0.001 18.87298,-0.001 28.30797,0 -0.2347,-0.2551 -0.70711,-0.76231 -0.70711,-0.76231" + sodipodi:nodetypes="ccc" /> + <path + style="display:inline;opacity:1;fill:none;stroke:#358611;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:1, 1;stroke-dashoffset:0;stroke-opacity:0.95703125;marker-end:url(#EmptyTriangleOutM-7)" + d="m 505.76093,240.73959 c 0,8.0198 0,16.04062 0,24.06242" + id="path2656-6" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect2658-9" + inkscape:original-d="m 505.76093,240.73959 c 0.001,8.0198 0.001,16.04062 0,24.06242" + sodipodi:nodetypes="cc" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';display:inline;opacity:1;fill:#000000;fill-opacity:1;stroke:none" + x="143.35846" + y="242.55573" + id="text5219-2-6-1"><tspan + sodipodi:role="line" + x="143.35846" + y="242.55573" + id="tspan5223-0-9-0" + style="font-size:10px;line-height:1.25">port n+2</tspan></text> + <rect + style="display:inline;opacity:1;fill:none;fill-opacity:1;stroke:url(#linearGradient1920-2);stroke-width:1.06814909;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect3736-8-4-6" + width="39.065548" + height="24.347494" + x="321.45184" + y="228.86047" /> + <rect + style="display:inline;opacity:1;fill:none;fill-opacity:1;stroke:url(#linearGradient1924-6);stroke-width:1.06814909;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect3736-8-0-1-7-3" + width="39.065548" + height="24.347494" + x="439.45184" + y="228.86047" /> + <rect + style="display:inline;opacity:1;fill:none;fill-opacity:1;stroke:url(#linearGradient1922-1);stroke-width:1.06814909;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect3736-8-3-1-2" + width="39.065548" + height="24.347494" + x="380.18939" + y="228.85535" /> + <path + style="display:inline;opacity:1;fill:none;stroke:#f78202;stroke-width:1.05190074;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleOutM-2)" + d="m 299.14148,238.83437 c 6.26102,0.009 12.52088,0.0188 18.77957,0.0282" + id="path5226-0" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect5228-59" + inkscape:original-d="m 299.14148,238.83437 c 6.26102,0.008 12.52088,0.0176 18.77957,0.0282" + sodipodi:nodetypes="cc" /> + <path + style="display:inline;opacity:1;fill:none;stroke:#f78202;stroke-width:0.89999998;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleOutM-5-8)" + d="m 360.58133,238.92215 c 5.35689,0.008 10.71279,0.0161 16.06769,0.0241" + id="path5226-6-6" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect5228-5-9" + inkscape:original-d="m 360.58133,238.92215 c 5.35689,0.007 10.71279,0.0151 16.06769,0.0241" + sodipodi:nodetypes="cc" /> + <path + style="display:inline;opacity:1;fill:none;stroke:#f78202;stroke-width:0.89999998;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleOutM-5-2-6)" + d="m 419.6524,239.62926 c 5.35689,0.008 10.71279,0.0161 16.06769,0.0241" + id="path5226-6-2-1" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect5228-5-1-6" + inkscape:original-d="m 419.6524,239.62926 c 5.35689,0.007 10.71279,0.0151 16.06769,0.0241" + sodipodi:nodetypes="cc" /> + <rect + style="display:inline;opacity:1;fill:#ffffff;fill-opacity:1;stroke:url(#linearGradient3995-9-5);stroke-width:1.02699995;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect87-6-5-3-3-6" + width="51.714954" + height="32.587509" + x="141.64139" + y="277.3764" + rx="11.6051" + ry="16.293755" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';display:inline;opacity:1;fill:#000000;fill-opacity:1;stroke:none" + x="142.95203" + y="320.12933" + id="text5219-2-61-8"><tspan + sodipodi:role="line" + x="142.95203" + y="320.12933" + id="tspan5223-0-2-8" + style="font-size:10px;line-height:1.25">producer m</tspan></text> + <rect + style="display:inline;opacity:1;fill:none;fill-opacity:1;stroke:url(#linearGradient1918-0-4);stroke-width:1.06814909;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect3736-1-9-4" + width="39.065548" + height="24.347494" + x="260.07593" + y="282.12927" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';display:inline;opacity:1;fill:#000000;fill-opacity:1;stroke:none" + x="270.63684" + y="279.76422" + id="text5219-1-9-5-3-3"><tspan + sodipodi:role="line" + x="270.63684" + y="279.76422" + id="tspan5223-2-3-9-1-1" + style="font-size:10px;line-height:1.25">q2s</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';display:inline;opacity:1;fill:#000000;fill-opacity:1;stroke:none" + x="326.01276" + y="278.49542" + id="text5219-1-9-4-9-9-4"><tspan + sodipodi:role="line" + x="326.01276" + y="278.49542" + id="tspan5223-2-3-5-0-4-9" + style="font-size:10px;line-height:1.25">q2s+1</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';display:inline;opacity:1;fill:#000000;fill-opacity:1;stroke:none" + x="386.0127" + y="280.49542" + id="text5219-1-9-4-3-9-7-2"><tspan + sodipodi:role="line" + x="386.0127" + y="280.49542" + id="tspan5223-2-3-5-6-1-8-0" + style="font-size:10px;line-height:1.25">q2s+2</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';display:inline;opacity:1;fill:#000000;fill-opacity:1;stroke:none" + x="446.0127" + y="280.49542" + id="text5219-1-9-4-3-0-7-4-6"><tspan + sodipodi:role="line" + x="446.0127" + y="280.49542" + id="tspan5223-2-3-5-6-6-1-5-8" + style="font-size:10px;line-height:1.25">q3s-1</tspan></text> + <path + style="display:inline;opacity:1;fill:none;stroke:#ff141a;stroke-width:0.89999998;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#marker7719-2-7)" + d="M 192.51338,293.50911 H 257.5672" + id="path1930-0-9" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect1932-8-5" + inkscape:original-d="m 192.51338,293.50911 c 21.68561,-0.001 43.37021,-0.001 65.05382,0" + sodipodi:nodetypes="cc" /> + <path + style="display:inline;opacity:1;fill:none;stroke:#358611;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:1, 1;stroke-dashoffset:0;stroke-opacity:1" + d="m 478.16081,294.1293 c 9.43699,0 18.87298,0 23.47261,-0.12707 4.59962,-0.12707 4.36395,-0.38114 4.12825,-0.63524" + id="path1938-3-2" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect1940-3-6" + inkscape:original-d="m 478.16081,294.1293 c 9.43699,-10e-4 18.87298,-10e-4 28.30797,0 -0.2347,-0.2551 -0.70711,-0.76231 -0.70711,-0.76231" + sodipodi:nodetypes="ccc" /> + <path + style="display:inline;opacity:1;fill:none;stroke:#358611;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:1, 1;stroke-dashoffset:0;stroke-opacity:0.95703125;marker-end:url(#EmptyTriangleOutM-7-9)" + d="m 505.76093,294.7396 c 0,8.0198 0,16.04062 0,24.06242" + id="path2656-6-6" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect2658-9-7" + inkscape:original-d="m 505.76093,294.7396 c 0.001,8.0198 0.001,16.04062 0,24.06242" + sodipodi:nodetypes="cc" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';display:inline;opacity:1;fill:#000000;fill-opacity:1;stroke:none" + x="143.35846" + y="296.55573" + id="text5219-2-6-1-6"><tspan + sodipodi:role="line" + x="143.35846" + y="296.55573" + id="tspan5223-0-9-0-4" + style="font-size:10px;line-height:1.25">port n+m</tspan></text> + <rect + style="display:inline;opacity:1;fill:none;fill-opacity:1;stroke:url(#linearGradient1920-2-4);stroke-width:1.06814909;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect3736-8-4-6-9" + width="39.065548" + height="24.347494" + x="321.45184" + y="282.86047" /> + <rect + style="display:inline;opacity:1;fill:none;fill-opacity:1;stroke:url(#linearGradient1924-6-5);stroke-width:1.06814909;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect3736-8-0-1-7-3-5" + width="39.065548" + height="24.347494" + x="439.45184" + y="282.86047" /> + <rect + style="display:inline;opacity:1;fill:none;fill-opacity:1;stroke:url(#linearGradient1922-1-2);stroke-width:1.06814909;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1" + id="rect3736-8-3-1-2-0" + width="39.065548" + height="24.347494" + x="380.18939" + y="282.85535" /> + <path + style="display:inline;opacity:1;fill:none;stroke:#f78202;stroke-width:1.05190074;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleOutM-2-5)" + d="m 299.14148,294.24859 c 6.26102,0.009 12.52088,0.0188 18.77957,0.0282" + id="path5226-0-4" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect5228-59-7" + inkscape:original-d="m 299.14148,294.24859 c 6.26102,0.008 12.52088,0.0176 18.77957,0.0282" + sodipodi:nodetypes="cc" /> + <path + style="display:inline;opacity:1;fill:none;stroke:#f78202;stroke-width:0.89999998;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleOutM-5-8-4)" + d="m 360.58133,292.92216 c 5.35689,0.008 10.71279,0.0161 16.06769,0.0241" + id="path5226-6-6-8" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect5228-5-9-3" + inkscape:original-d="m 360.58133,292.92216 c 5.35689,0.007 10.71279,0.0151 16.06769,0.0241" + sodipodi:nodetypes="cc" /> + <path + style="display:inline;opacity:1;fill:none;stroke:#f78202;stroke-width:0.89999998;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#TriangleOutM-5-2-6-0)" + d="m 419.6524,293.62927 c 5.35689,0.008 10.71279,0.0161 16.06769,0.0241" + id="path5226-6-2-1-7" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect5228-5-1-6-8" + inkscape:original-d="m 419.6524,293.62927 c 5.35689,0.007 10.71279,0.0151 16.06769,0.0241" + sodipodi:nodetypes="cc" /> + <path + style="fill:none;stroke:#000000;stroke-width:0.93284476;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.93284469, 0.93284469;stroke-dashoffset:0;stroke-opacity:1;marker-start:url(#marker28236);marker-end:url(#marker3706)" + d="m 493.60937,225.85078 c 6.17895,1.39044 12.5936,1.72719 18.88417,0.99136 9.68216,-1.13256 19.05181,-4.83584 26.89197,-10.62883 7.84016,-5.79299 14.13198,-13.66177 18.05824,-22.58429" + id="path14459" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect14461" + inkscape:original-d="m 493.60937,225.85078 c 4.17466,-11.99492 8.79442,4.39475 18.88417,0.99136 60.98518,-20.57101 6.8766,-33.21442 44.95021,-33.21312" + sodipodi:nodetypes="csc" /> + <path + style="fill:none;stroke:#000000;stroke-width:0.74085319;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.74085314, 0.74085314;stroke-dashoffset:0;stroke-opacity:1;marker-start:url(#marker27764);marker-end:url(#marker3706)" + d="m 472.15845,359.3562 c 7.70444,3.67634 16.17823,5.73067 24.71089,5.99076 5.72629,0.17454 11.49119,-0.45602 16.99344,-2.05167 10.09944,-2.92884 19.04178,-9.02089 26.75302,-16.17026 3.94036,-3.65325 7.6018,-7.59753 11.1291,-11.65103 4.51116,-5.18413 8.81657,-10.56332 12.57247,-16.31823 0.43414,-0.6652 0.86084,-1.33527 1.27998,-2.01002" + id="path14478" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect14480" + inkscape:original-d="m 472.15845,359.3562 c 5.89123,3.55932 18.82146,2.42954 24.71089,5.99076 5.88941,3.56122 8.98322,0.19463 16.99344,-2.05167 8.01021,-2.2463 17.83625,-10.78112 26.75302,-16.17026 8.91676,-5.38914 7.4203,-7.76831 11.1291,-11.65103 3.7088,-3.88274 8.38255,-10.87977 12.57247,-16.31823 4.18992,-5.43845 0.85422,-1.34095 1.27998,-2.01002" + sodipodi:nodetypes="cssccsc" /> + <path + style="fill:none;stroke:#000000;stroke-width:0.8;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;stroke-miterlimit:4;stroke-dasharray:0.8, 0.80000000000000004;stroke-dashoffset:0;marker-end:url(#marker3706);marker-start:url(#Arrow2Mstart)" + d="m 492.02012,273.41807 c 3.53022,2.92401 7.55595,5.24827 11.85333,6.84353 10.62484,3.94412 22.55621,3.28983 33.4015,0 10.60649,-3.21739 20.4556,-8.90378 28.54519,-16.48057" + id="path14482" + inkscape:connector-curvature="0" + inkscape:path-effect="#path-effect14484" + inkscape:original-d="m 492.02012,273.41807 c 3.95211,2.28018 7.90322,4.56135 11.85333,6.84353 3.95011,2.28217 22.26867,-10e-4 33.4015,0 11.13284,0.001 19.03113,-10.98805 28.54519,-16.48057" /> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;font-size:40px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none" + x="193.35634" + y="277.3764" + id="text21302"><tspan + sodipodi:role="line" + id="tspan21300" + x="193.35634" + y="277.3764"> </tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;font-size:40px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none" + x="344.2348" + y="276.24649" + id="text21306"><tspan + sodipodi:role="line" + id="tspan21304" + x="344.2348" + y="311.63712"></tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;font-size:40px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none" + x="453.83633" + y="276.95361" + id="text21310"><tspan + sodipodi:role="line" + id="tspan21308" + x="453.83633" + y="312.34424"></tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';display:inline;opacity:1;fill:#000000;fill-opacity:1;stroke:none" + x="243.03555" + y="126.90381" + id="text5219-26"><tspan + sodipodi:role="line" + x="243.03555" + y="126.90381" + id="tspan5223-10" + style="font-size:10px;line-height:1.25">total queues = number of stages * number of producers</tspan></text> + <text + xml:space="preserve" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:'Bitstream Vera Sans';display:inline;opacity:1;fill:#000000;fill-opacity:1;stroke:none" + x="495.66333" + y="349.67435" + id="text5219-26-2"><tspan + sodipodi:role="line" + x="495.66333" + y="349.67435" + id="tspan5223-10-7" + style="font-size:10px;line-height:1.25">All workers are linked to all queues</tspan></text> + </g> +</svg> diff --git a/doc/guides/tools/index.rst b/doc/guides/tools/index.rst index 6dc5d202..c9133ec8 100644 --- a/doc/guides/tools/index.rst +++ b/doc/guides/tools/index.rst @@ -40,4 +40,4 @@ DPDK Tools User Guides pmdinfo devbind cryptoperf - + testeventdev diff --git a/doc/guides/tools/pdump.rst b/doc/guides/tools/pdump.rst index 57097e43..16502f7a 100644 --- a/doc/guides/tools/pdump.rst +++ b/doc/guides/tools/pdump.rst @@ -28,6 +28,7 @@ (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +.. _pdump_tool: dpdk-pdump Application ====================== @@ -38,6 +39,12 @@ a DPDK secondary process and is capable of enabling packet capture on dpdk ports .. Note:: * The ``dpdk-pdump`` tool can only be used in conjunction with a primary application which has the packet capture framework initialized already. + In dpdk, only the ``testpmd`` is modified to initialize packet capture + framework, other applications remain untouched. So, if the ``dpdk-pdump`` + tool has to be used with any application other than the testpmd, user + needs to explicitly modify that application to call packet capture + framework initialization code. Refer ``app/test-pmd/testpmd.c`` + code to see how this is done. * The ``dpdk-pdump`` tool depends on libpcap based PMD which is disabled by default in the build configuration files, diff --git a/doc/guides/tools/testeventdev.rst b/doc/guides/tools/testeventdev.rst new file mode 100644 index 00000000..34b1c318 --- /dev/null +++ b/doc/guides/tools/testeventdev.rst @@ -0,0 +1,461 @@ +.. BSD LICENSE + Copyright(c) 2017 Cavium, Inc. All rights reserved. + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of Cavium, Inc nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +dpdk-test-eventdev Application +============================== + +The ``dpdk-test-eventdev`` tool is a Data Plane Development Kit (DPDK) +application that allows exercising various eventdev use cases. +This application has a generic framework to add new eventdev based test cases to +verify functionality and measure the performance parameters of DPDK eventdev +devices. + +Compiling the Application +------------------------- + +**Build the application** + +Execute the ``dpdk-setup.sh`` script to build the DPDK library together with the +``dpdk-test-eventdev`` application. + +Initially, the user must select a DPDK target to choose the correct target type +and compiler options to use when building the libraries. +The user must have all libraries, modules, updates and compilers installed +in the system prior to this, +as described in the earlier chapters in this Getting Started Guide. + +Running the Application +----------------------- + +The application has a number of command line options: + +.. code-block:: console + + dpdk-test-eventdev [EAL Options] -- [application options] + +EAL Options +~~~~~~~~~~~ + +The following are the EAL command-line options that can be used in conjunction +with the ``dpdk-test-eventdev`` application. +See the DPDK Getting Started Guides for more information on these options. + +* ``-c <COREMASK>`` or ``-l <CORELIST>`` + + Set the hexadecimal bitmask of the cores to run on. The corelist is a + list of cores to use. + +* ``--vdev <driver><id>`` + + Add a virtual eventdev device. + +Application Options +~~~~~~~~~~~~~~~~~~~ + +The following are the application command-line options: + +* ``--verbose`` + + Set verbose level. Default is 1. Value > 1 displays more details. + +* ``--dev <n>`` + + Set the device id of the event device. + +* ``--test <name>`` + + Set test name, where ``name`` is one of the following:: + + order_queue + order_atq + perf_queue + perf_atq + +* ``--socket_id <n>`` + + Set the socket id of the application resources. + +* ``--pool-sz <n>`` + + Set the number of mbufs to be allocated from the mempool. + +* ``--slcore <n>`` + + Set the scheduler lcore id.(Valid when eventdev is not RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED capable) + +* ``--plcores <CORELIST>`` + + Set the list of cores to be used as producers. + +* ``--wlcores <CORELIST>`` + + Set the list of cores to be used as workers. + +* ``--stlist <type_list>`` + + Set the scheduled type of each stage where ``type_list`` size + determines the number of stages used in the test application. + Each type_list member can be one of the following:: + + P or p : Parallel schedule type + O or o : Ordered schedule type + A or a : Atomic schedule type + + Application expects the ``type_list`` in comma separated form (i.e. ``--stlist o,a,a,a``) + +* ``--nb_flows <n>`` + + Set the number of flows to produce. + +* ``--nb_pkts <n>`` + + Set the number of packets to produce. 0 implies no limit. + +* ``--worker_deq_depth <n>`` + + Set the dequeue depth of the worker. + +* ``--fwd_latency`` + + Perform forward latency measurement. + +* ``--queue_priority`` + + Enable queue priority. + + +Eventdev Tests +-------------- + +ORDER_QUEUE Test +~~~~~~~~~~~~~~~~ + +This is a functional test case that aims at testing the following: + +#. Verify the ingress order maintenance. +#. Verify the exclusive(atomic) access to given atomic flow per eventdev port. + +.. _table_eventdev_order_queue_test: + +.. table:: Order queue test eventdev configuration. + + +---+--------------+----------------+------------------------+ + | # | Items | Value | Comments | + | | | | | + +===+==============+================+========================+ + | 1 | nb_queues | 2 | q0(ordered), q1(atomic)| + | | | | | + +---+--------------+----------------+------------------------+ + | 2 | nb_producers | 1 | | + | | | | | + +---+--------------+----------------+------------------------+ + | 3 | nb_workers | >= 1 | | + | | | | | + +---+--------------+----------------+------------------------+ + | 4 | nb_ports | nb_workers + | Workers use port 0 to | + | | | 1 | port n-1. Producer uses| + | | | | port n | + +---+--------------+----------------+------------------------+ + +.. _figure_eventdev_order_queue_test: + +.. figure:: img/eventdev_order_queue_test.* + + order queue test operation. + +The order queue test configures the eventdev with two queues and an event +producer to inject the events to q0(ordered) queue. Both q0(ordered) and +q1(atomic) are linked to all the workers. + +The event producer maintains a sequence number per flow and injects the events +to the ordered queue. The worker receives the events from ordered queue and +forwards to atomic queue. Since the events from an ordered queue can be +processed in parallel on the different workers, the ingress order of events +might have changed on the downstream atomic queue enqueue. On enqueue to the +atomic queue, the eventdev PMD driver reorders the event to the original +ingress order(i.e producer ingress order). + +When the event is dequeued from the atomic queue by the worker, this test +verifies the expected sequence number of associated event per flow by comparing +the free running expected sequence number per flow. + +Application options +^^^^^^^^^^^^^^^^^^^ + +Supported application command line options are following:: + + --verbose + --dev + --test + --socket_id + --pool_sz + --plcores + --wlcores + --nb_flows + --nb_pkts + --worker_deq_depth + +Example +^^^^^^^ + +Example command to run order queue test: + +.. code-block:: console + + sudo build/app/dpdk-test-eventdev --vdev=event_sw0 -- \ + --test=order_queue --plcores 1 --wlcores 2,3 + + +ORDER_ATQ Test +~~~~~~~~~~~~~~ + +This test verifies the same aspects of ``order_queue`` test, the difference is +the number of queues used, this test operates on a single ``all types queue(atq)`` +instead of two different queues for ordered and atomic. + +.. _table_eventdev_order_atq_test: + +.. table:: Order all types queue test eventdev configuration. + + +---+--------------+----------------+------------------------+ + | # | Items | Value | Comments | + | | | | | + +===+==============+================+========================+ + | 1 | nb_queues | 1 | q0(all types queue) | + | | | | | + +---+--------------+----------------+------------------------+ + | 2 | nb_producers | 1 | | + | | | | | + +---+--------------+----------------+------------------------+ + | 3 | nb_workers | >= 1 | | + | | | | | + +---+--------------+----------------+------------------------+ + | 4 | nb_ports | nb_workers + | Workers use port 0 to | + | | | 1 | port n-1.Producer uses | + | | | | port n. | + +---+--------------+----------------+------------------------+ + +.. _figure_eventdev_order_atq_test: + +.. figure:: img/eventdev_order_atq_test.* + + order all types queue test operation. + +Application options +^^^^^^^^^^^^^^^^^^^ + +Supported application command line options are following:: + + --verbose + --dev + --test + --socket_id + --pool_sz + --plcores + --wlcores + --nb_flows + --nb_pkts + --worker_deq_depth + +Example +^^^^^^^ + +Example command to run order ``all types queue`` test: + +.. code-block:: console + + sudo build/app/dpdk-test-eventdev --vdev=event_octeontx -- \ + --test=order_atq --plcores 1 --wlcores 2,3 + + +PERF_QUEUE Test +~~~~~~~~~~~~~~~ + +This is a performance test case that aims at testing the following: + +#. Measure the number of events can be processed in a second. +#. Measure the latency to forward an event. + +.. _table_eventdev_perf_queue_test: + +.. table:: Perf queue test eventdev configuration. + + +---+--------------+----------------+-----------------------------------------+ + | # | Items | Value | Comments | + | | | | | + +===+==============+================+=========================================+ + | 1 | nb_queues | nb_producers * | Queues will be configured based on the | + | | | nb_stages | user requested sched type list(--stlist)| + +---+--------------+----------------+-----------------------------------------+ + | 2 | nb_producers | >= 1 | Selected through --plcores command line | + | | | | argument. | + +---+--------------+----------------+-----------------------------------------+ + | 3 | nb_workers | >= 1 | Selected through --wlcores command line | + | | | | argument | + +---+--------------+----------------+-----------------------------------------+ + | 4 | nb_ports | nb_workers + | Workers use port 0 to port n-1. | + | | | nb_producers | Producers use port n to port p | + +---+--------------+----------------+-----------------------------------------+ + +.. _figure_eventdev_perf_queue_test: + +.. figure:: img/eventdev_perf_queue_test.* + + perf queue test operation. + +The perf queue test configures the eventdev with Q queues and P ports, where +Q and P is a function of the number of workers, the number of producers and +number of stages as mentioned in :numref:`table_eventdev_perf_queue_test`. + +The user can choose the number of workers, the number of producers and number of +stages through the ``--wlcores``, ``--plcores`` and the ``--stlist`` application +command line arguments respectively. + +The producer(s) injects the events to eventdev based the first stage sched type +list requested by the user through ``--stlist`` the command line argument. + +Based on the number of stages to process(selected through ``--stlist``), +The application forwards the event to next upstream queue and terminates when it +reaches the last stage in the pipeline. On event termination, application +increments the number events processed and print periodically in one second +to get the number of events processed in one second. + +When ``--fwd_latency`` command line option selected, the application inserts +the timestamp in the event on the first stage and then on termination, it +updates the number of cycles to forward a packet. The application uses this +value to compute the average latency to a forward packet. + +Application options +^^^^^^^^^^^^^^^^^^^ + +Supported application command line options are following:: + + --verbose + --dev + --test + --socket_id + --pool_sz + --slcore (Valid when eventdev is not RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED capable) + --plcores + --wlcores + --stlist + --nb_flows + --nb_pkts + --worker_deq_depth + --fwd_latency + --queue_priority + +Example +^^^^^^^ + +Example command to run perf queue test: + +.. code-block:: console + + sudo build/app/dpdk-test-eventdev --vdev=event_sw0 -- \ + --test=perf_queue --slcore=1 --plcores=2 --wlcore=3 --stlist=p --nb_pkts=0 + + +PERF_ATQ Test +~~~~~~~~~~~~~~~ + +This is a performance test case that aims at testing the following with +``all types queue`` eventdev scheme. + +#. Measure the number of events can be processed in a second. +#. Measure the latency to forward an event. + +.. _table_eventdev_perf_atq_test: + +.. table:: Perf all types queue test eventdev configuration. + + +---+--------------+----------------+-----------------------------------------+ + | # | Items | Value | Comments | + | | | | | + +===+==============+================+=========================================+ + | 1 | nb_queues | nb_producers | Queues will be configured based on the | + | | | | user requested sched type list(--stlist)| + +---+--------------+----------------+-----------------------------------------+ + | 2 | nb_producers | >= 1 | Selected through --plcores command line | + | | | | argument. | + +---+--------------+----------------+-----------------------------------------+ + | 3 | nb_workers | >= 1 | Selected through --wlcores command line | + | | | | argument | + +---+--------------+----------------+-----------------------------------------+ + | 4 | nb_ports | nb_workers + | Workers use port 0 to port n-1. | + | | | nb_producers | Producers use port n to port p | + +---+--------------+----------------+-----------------------------------------+ + +.. _figure_eventdev_perf_atq_test: + +.. figure:: img/eventdev_perf_atq_test.* + + perf all types queue test operation. + + +The ``all types queues(atq)`` perf test configures the eventdev with Q queues +and P ports, where Q and P is a function of the number of workers and number of +producers as mentioned in :numref:`table_eventdev_perf_atq_test`. + + +The atq queue test functions as same as ``perf_queue`` test. The difference +is, It uses, ``all type queue scheme`` instead of separate queues for each +stage and thus reduces the number of queues required to realize the use case +and enables flow pinning as the event does not move to the next queue. + + +Application options +^^^^^^^^^^^^^^^^^^^ + +Supported application command line options are following:: + + --verbose + --dev + --test + --socket_id + --pool_sz + --slcore (Valid when eventdev is not RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED capable) + --plcores + --wlcores + --stlist + --nb_flows + --nb_pkts + --worker_deq_depth + --fwd_latency + +Example +^^^^^^^ + +Example command to run perf ``all types queue`` test: + +.. code-block:: console + + sudo build/app/dpdk-test-eventdev --vdev=event_octeontx -- \ + --test=perf_atq --plcores=2 --wlcore=3 --stlist=p --nb_pkts=0 |
