diff options
Diffstat (limited to 'doc/guides/prog_guide')
| -rw-r--r-- | doc/guides/prog_guide/cryptodev_lib.rst | 370 | ||||
| -rw-r--r-- | doc/guides/prog_guide/eventdev.rst | 394 | ||||
| -rw-r--r-- | doc/guides/prog_guide/generic_receive_offload_lib.rst | 159 | ||||
| -rw-r--r-- | doc/guides/prog_guide/img/crypto_xform_chain.svg | 8 | ||||
| -rw-r--r-- | doc/guides/prog_guide/img/cryptodev_sym_sess.svg | 418 | ||||
| -rw-r--r-- | doc/guides/prog_guide/img/eventdev_usage.svg | 994 | ||||
| -rw-r--r-- | doc/guides/prog_guide/index.rst | 4 | ||||
| -rw-r--r-- | doc/guides/prog_guide/kernel_nic_interface.rst | 2 | ||||
| -rw-r--r-- | doc/guides/prog_guide/metrics_lib.rst | 1 | ||||
| -rw-r--r-- | doc/guides/prog_guide/perf_opt_guidelines.rst | 2 | ||||
| -rw-r--r-- | doc/guides/prog_guide/poll_mode_drv.rst | 15 | ||||
| -rw-r--r-- | doc/guides/prog_guide/rte_flow.rst | 126 | ||||
| -rw-r--r-- | doc/guides/prog_guide/service_cores.rst | 81 | ||||
| -rw-r--r-- | doc/guides/prog_guide/traffic_management.rst | 251 |
14 files changed, 2690 insertions, 135 deletions
diff --git a/doc/guides/prog_guide/cryptodev_lib.rst b/doc/guides/prog_guide/cryptodev_lib.rst index 4f98f28c..75ae085f 100644 --- a/doc/guides/prog_guide/cryptodev_lib.rst +++ b/doc/guides/prog_guide/cryptodev_lib.rst @@ -1,5 +1,5 @@ .. BSD LICENSE - Copyright(c) 2016 Intel Corporation. All rights reserved. + Copyright(c) 2016-2017 Intel Corporation. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -68,13 +68,13 @@ From the command line using the --vdev EAL option .. code-block:: console - --vdev 'cryptodev_aesni_mb_pmd0,max_nb_queue_pairs=2,max_nb_sessions=1024,socket_id=0' + --vdev 'crypto_aesni_mb0,max_nb_queue_pairs=2,max_nb_sessions=1024,socket_id=0' Our using the rte_vdev_init API within the application code. .. code-block:: c - rte_vdev_init("cryptodev_aesni_mb_pmd", + rte_vdev_init("crypto_aesni_mb", "max_nb_queue_pairs=2,max_nb_sessions=1024,socket_id=0") All virtual Crypto devices support the following initialization parameters: @@ -114,9 +114,8 @@ The rte_cryptodev_configure API is used to configure a Crypto device. int rte_cryptodev_configure(uint8_t dev_id, struct rte_cryptodev_config *config) -The ``rte_cryptodev_config`` structure is used to pass the configuration parameters. -In contains parameter for socket selection, number of queue pairs and the -session mempool configuration. +The ``rte_cryptodev_config`` structure is used to pass the configuration +parameters for socket selection and number of queue pairs. .. code-block:: c @@ -125,12 +124,6 @@ session mempool configuration. /**< Socket to allocate resources on */ uint16_t nb_queue_pairs; /**< Number of queue pairs to configure on device */ - - struct { - uint32_t nb_objs; - uint32_t cache_size; - } session_mp; - /**< Session mempool configuration */ }; @@ -188,8 +181,9 @@ the device having hardware acceleration or supporting symmetric Crypto operations, The capabilities mechanism defines the individual algorithms/functions which -the device supports, such as a specific symmetric Crypto cipher or -authentication operation. +the device supports, such as a specific symmetric Crypto cipher, +authentication operation or Authenticated Encryption with Associated Data +(AEAD) operation. Device Features @@ -245,7 +239,8 @@ algorithm AES_CBC. .max = 12, .increment = 0 }, - .aad_size = { 0 } + .aad_size = { 0 }, + .iv_size = { 0 } } } }, @@ -291,7 +286,7 @@ relevant information for the device. struct rte_cryptodev_info { const char *driver_name; - enum rte_cryptodev_type dev_type; + uint8_t driver_id; struct rte_pci_device *pci_dev; uint64_t feature_flags; @@ -359,11 +354,11 @@ Crypto operation to be processed on a particular Crypto device poll mode driver. .. figure:: img/crypto_op.* -The operation structure includes the operation type and the operation status, -a reference to the operation specific data, which can vary in size and content -depending on the operation being provisioned. It also contains the source -mempool for the operation, if it allocate from a mempool. Finally an -opaque pointer for user specific data is provided. +The operation structure includes the operation type, the operation status +and the session type (session-based/less), a reference to the operation +specific data, which can vary in size and content depending on the operation +being provisioned. It also contains the source mempool for the operation, +if it allocated from a mempool. If Crypto operations are allocated from a Crypto operation mempool, see next section, there is also the ability to allocate private memory with the @@ -431,7 +426,7 @@ operations, as well as also supporting AEAD operations. Session and Session Management ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Session are used in symmetric cryptographic processing to store the immutable +Sessions are used in symmetric cryptographic processing to store the immutable data defined in a cryptographic transform which is used in the operation processing of a packet flow. Sessions are used to manage information such as expand cipher keys and HMAC IPADs and OPADs, which need to be calculated for a @@ -442,29 +437,33 @@ Crypto workloads. .. figure:: img/cryptodev_sym_sess.* -The Crypto device framework provides a set of session pool management APIs for -the creation and freeing of the sessions, utilizing the Mempool Library. - -The framework also provides hooks so the PMDs can pass the amount of memory -required for that PMDs private session parameters, as well as initialization -functions for the configuration of the session parameters and freeing function -so the PMD can managed the memory on destruction of a session. - -**Note**: Sessions created on a particular device can only be used on Crypto -devices of the same type, and if you try to use a session on a device different -to that on which it was created then the Crypto operation will fail. - -``rte_cryptodev_sym_session_create()`` is used to create a symmetric session on -Crypto device. A symmetric transform chain is used to specify the particular -operation and its parameters. See the section below for details on transforms. - -.. code-block:: c - - struct rte_cryptodev_sym_session * rte_cryptodev_sym_session_create( - uint8_t dev_id, struct rte_crypto_sym_xform *xform); - -**Note**: For AEAD operations the algorithm selected for authentication and -ciphering must aligned, eg AES_GCM. +The Crypto device framework provides APIs to allocate and initizalize sessions +for crypto devices, where sessions are mempool objects. +It is the application's responsibility to create and manage the session mempools. +This approach allows for different scenarios such as having a single session +mempool for all crypto devices (where the mempool object size is big +enough to hold the private session of any crypto device), as well as having +multiple session mempools of different sizes for better memory usage. + +An application can use ``rte_cryptodev_get_private_session_size()`` to +get the private session size of given crypto device. This function would allow +an application to calculate the max device session size of all crypto devices +to create a single session mempool. +If instead an application creates multiple session mempools, the Crypto device +framework also provides ``rte_cryptodev_get_header_session_size`` to get +the size of an uninitialized session. + +Once the session mempools have been created, ``rte_cryptodev_sym_session_create()`` +is used to allocate an uninitialized session from the given mempool. +The session then must be initialized using ``rte_cryptodev_sym_session_init()`` +for each of the required crypto devices. A symmetric transform chain +is used to specify the operation and its parameters. See the section below for +details on transforms. + +When a session is no longer used, user must call ``rte_cryptodev_sym_session_clear()`` +for each of the crypto devices that are using the session, to free all driver +private session data. Once this is done, session should be freed using +``rte_cryptodev_sym_session_free`` which returns them to their mempool. Transforms and Transform Chaining @@ -476,9 +475,8 @@ operations such as cipher encrypt and authentication generate, the next pointer allows transform to be chained together. Crypto devices which support chaining must publish the chaining of symmetric Crypto operations feature flag. -Currently there are two transforms types cipher and authentication, to specify -an AEAD operation it is required to chain a cipher and an authentication -transform together. Also it is important to note that the order in which the +Currently there are three transforms types cipher, authentication and AEAD. +Also it is important to note that the order in which the transforms are passed indicates the order of the chaining. .. code-block:: c @@ -493,6 +491,8 @@ transforms are passed indicates the order of the chaining. /**< Authentication / hash xform */ struct rte_crypto_cipher_xform cipher; /**< Cipher xform */ + struct rte_crypto_aead_xform aead; + /**< AEAD xform */ }; }; @@ -512,9 +512,9 @@ buffer. It is used for either cipher, authentication, AEAD and chained operations. As a minimum the symmetric operation must have a source data buffer (``m_src``), -the session type (session-based/less), a valid session (or transform chain if in -session-less mode) and the minimum authentication/ cipher parameters required -depending on the type of operation specified in the session or the transform +a valid session (or transform chain if in session-less mode) and the minimum +authentication/ cipher/ AEAD parameters required depending on the type of operation +specified in the session or the transform chain. .. code-block:: c @@ -523,8 +523,6 @@ chain. struct rte_mbuf *m_src; struct rte_mbuf *m_dst; - enum rte_crypto_sym_op_sess_type type; - union { struct rte_cryptodev_sym_session *session; /**< Handle for the initialised session context */ @@ -532,39 +530,255 @@ chain. /**< Session-less API Crypto operation parameters */ }; - struct { + union { struct { - uint32_t offset; - uint32_t length; - } data; /**< Data offsets and length for ciphering */ + struct { + uint32_t offset; + uint32_t length; + } data; /**< Data offsets and length for AEAD */ + + struct { + uint8_t *data; + phys_addr_t phys_addr; + } digest; /**< Digest parameters */ + + struct { + uint8_t *data; + phys_addr_t phys_addr; + } aad; + /**< Additional authentication parameters */ + } aead; struct { - uint8_t *data; - phys_addr_t phys_addr; - uint16_t length; - } iv; /**< Initialisation vector parameters */ - } cipher; + struct { + struct { + uint32_t offset; + uint32_t length; + } data; /**< Data offsets and length for ciphering */ + } cipher; + + struct { + struct { + uint32_t offset; + uint32_t length; + } data; + /**< Data offsets and length for authentication */ + + struct { + uint8_t *data; + phys_addr_t phys_addr; + } digest; /**< Digest parameters */ + } auth; + }; + }; + }; - struct { - struct { - uint32_t offset; - uint32_t length; - } data; /**< Data offsets and length for authentication */ +Sample code +----------- - struct { - uint8_t *data; - phys_addr_t phys_addr; - uint16_t length; - } digest; /**< Digest parameters */ +There are various sample applications that show how to use the cryptodev library, +such as the L2fwd with Crypto sample application (L2fwd-crypto) and +the IPSec Security Gateway application (ipsec-secgw). - struct { - uint8_t *data; - phys_addr_t phys_addr; - uint16_t length; - } aad; /**< Additional authentication parameters */ - } auth; +While these applications demonstrate how an application can be created to perform +generic crypto operation, the required complexity hides the basic steps of +how to use the cryptodev APIs. + +The following sample code shows the basic steps to encrypt several buffers +with AES-CBC (although performing other crypto operations is similar), +using one of the crypto PMDs available in DPDK. + +.. code-block:: c + + /* + * Simple example to encrypt several buffers with AES-CBC using + * the Cryptodev APIs. + */ + + #define MAX_SESSIONS 1024 + #define NUM_MBUFS 1024 + #define POOL_CACHE_SIZE 128 + #define BURST_SIZE 32 + #define BUFFER_SIZE 1024 + #define AES_CBC_IV_LENGTH 16 + #define AES_CBC_KEY_LENGTH 16 + #define IV_OFFSET (sizeof(struct rte_crypto_op) + \ + sizeof(struct rte_crypto_sym_op)) + + struct rte_mempool *mbuf_pool, *crypto_op_pool, *session_pool; + unsigned int session_size; + int ret; + + /* Initialize EAL. */ + ret = rte_eal_init(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "Invalid EAL arguments\n"); + + uint8_t socket_id = rte_socket_id(); + + /* Create the mbuf pool. */ + mbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", + NUM_MBUFS, + POOL_CACHE_SIZE, + 0, + RTE_MBUF_DEFAULT_BUF_SIZE, + socket_id); + if (mbuf_pool == NULL) + rte_exit(EXIT_FAILURE, "Cannot create mbuf pool\n"); + + /* + * The IV is always placed after the crypto operation, + * so some private data is required to be reserved. + */ + unsigned int crypto_op_private_data = AES_CBC_IV_LENGTH; + + /* Create crypto operation pool. */ + crypto_op_pool = rte_crypto_op_pool_create("crypto_op_pool", + RTE_CRYPTO_OP_TYPE_SYMMETRIC, + NUM_MBUFS, + POOL_CACHE_SIZE, + crypto_op_private_data, + socket_id); + if (crypto_op_pool == NULL) + rte_exit(EXIT_FAILURE, "Cannot create crypto op pool\n"); + + /* Create the virtual crypto device. */ + char args[128]; + const char *crypto_name = "crypto_aesni_mb0"; + snprintf(args, sizeof(args), "socket_id=%d", socket_id); + ret = rte_vdev_init(crypto_name, args); + if (ret != 0) + rte_exit(EXIT_FAILURE, "Cannot create virtual device"); + + uint8_t cdev_id = rte_cryptodev_get_dev_id(crypto_name); + + /* Get private session data size. */ + session_size = rte_cryptodev_get_private_session_size(cdev_id); + + /* + * Create session mempool, with two objects per session, + * one for the session header and another one for the + * private session data for the crypto device. + */ + session_pool = rte_mempool_create("session_pool", + MAX_SESSIONS * 2, + session_size, + POOL_CACHE_SIZE, + 0, NULL, NULL, NULL, + NULL, socket_id, + 0); + + /* Configure the crypto device. */ + struct rte_cryptodev_config conf = { + .nb_queue_pairs = 1, + .socket_id = socket_id + }; + struct rte_cryptodev_qp_conf qp_conf = { + .nb_descriptors = 2048 + }; + + if (rte_cryptodev_configure(cdev_id, &conf) < 0) + rte_exit(EXIT_FAILURE, "Failed to configure cryptodev %u", cdev_id); + + if (rte_cryptodev_queue_pair_setup(cdev_id, 0, &qp_conf, + socket_id, session_pool) < 0) + rte_exit(EXIT_FAILURE, "Failed to setup queue pair\n"); + + if (rte_cryptodev_start(cdev_id) < 0) + rte_exit(EXIT_FAILURE, "Failed to start device\n"); + + /* Create the crypto transform. */ + uint8_t cipher_key[16] = {0}; + struct rte_crypto_sym_xform cipher_xform = { + .next = NULL, + .type = RTE_CRYPTO_SYM_XFORM_CIPHER, + .cipher = { + .op = RTE_CRYPTO_CIPHER_OP_ENCRYPT, + .algo = RTE_CRYPTO_CIPHER_AES_CBC, + .key = { + .data = cipher_key, + .length = AES_CBC_KEY_LENGTH + }, + .iv = { + .offset = IV_OFFSET, + .length = AES_CBC_IV_LENGTH + } + } + }; + + /* Create crypto session and initialize it for the crypto device. */ + struct rte_cryptodev_sym_session *session; + session = rte_cryptodev_sym_session_create(session_pool); + if (session == NULL) + rte_exit(EXIT_FAILURE, "Session could not be created\n"); + + if (rte_cryptodev_sym_session_init(cdev_id, session, + &cipher_xform, session_pool) < 0) + rte_exit(EXIT_FAILURE, "Session could not be initialized " + "for the crypto device\n"); + + /* Get a burst of crypto operations. */ + struct rte_crypto_op *crypto_ops[BURST_SIZE]; + if (rte_crypto_op_bulk_alloc(crypto_op_pool, + RTE_CRYPTO_OP_TYPE_SYMMETRIC, + crypto_ops, BURST_SIZE) == 0) + rte_exit(EXIT_FAILURE, "Not enough crypto operations available\n"); + + /* Get a burst of mbufs. */ + struct rte_mbuf *mbufs[BURST_SIZE]; + if (rte_pktmbuf_alloc_bulk(mbuf_pool, mbufs, BURST_SIZE) < 0) + rte_exit(EXIT_FAILURE, "Not enough mbufs available"); + + /* Initialize the mbufs and append them to the crypto operations. */ + unsigned int i; + for (i = 0; i < BURST_SIZE; i++) { + if (rte_pktmbuf_append(mbufs[i], BUFFER_SIZE) == NULL) + rte_exit(EXIT_FAILURE, "Not enough room in the mbuf\n"); + crypto_ops[i]->sym->m_src = mbufs[i]; } + /* Set up the crypto operations. */ + for (i = 0; i < BURST_SIZE; i++) { + struct rte_crypto_op *op = crypto_ops[i]; + /* Modify bytes of the IV at the end of the crypto operation */ + uint8_t *iv_ptr = rte_crypto_op_ctod_offset(op, uint8_t *, + IV_OFFSET); + + generate_random_bytes(iv_ptr, AES_CBC_IV_LENGTH); + + op->sym->cipher.data.offset = 0; + op->sym->cipher.data.length = BUFFER_SIZE; + + /* Attach the crypto session to the operation */ + rte_crypto_op_attach_sym_session(op, session); + } + + /* Enqueue the crypto operations in the crypto device. */ + uint16_t num_enqueued_ops = rte_cryptodev_enqueue_burst(cdev_id, 0, + crypto_ops, BURST_SIZE); + + /* + * Dequeue the crypto operations until all the operations + * are proccessed in the crypto device. + */ + uint16_t num_dequeued_ops, total_num_dequeued_ops = 0; + do { + struct rte_crypto_op *dequeued_ops[BURST_SIZE]; + num_dequeued_ops = rte_cryptodev_dequeue_burst(cdev_id, 0, + dequeued_ops, BURST_SIZE); + total_num_dequeued_ops += num_dequeued_ops; + + /* Check if operation was processed successfully */ + for (i = 0; i < num_dequeued_ops; i++) { + if (dequeued_ops[i]->status != RTE_CRYPTO_OP_STATUS_SUCCESS) + rte_exit(EXIT_FAILURE, + "Some operations were not processed correctly"); + } + + rte_mempool_put_bulk(crypto_op_pool, (void **)dequeued_ops, + num_dequeued_ops); + } while (total_num_dequeued_ops < num_enqueued_ops); + Asymmetric Cryptography ----------------------- diff --git a/doc/guides/prog_guide/eventdev.rst b/doc/guides/prog_guide/eventdev.rst new file mode 100644 index 00000000..908d123a --- /dev/null +++ b/doc/guides/prog_guide/eventdev.rst @@ -0,0 +1,394 @@ +.. BSD LICENSE + Copyright(c) 2017 Intel Corporation. All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of Intel Corporation nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +Event Device Library +==================== + +The DPDK Event device library is an abstraction that provides the application +with features to schedule events. This is achieved using the PMD architecture +similar to the ethdev or cryptodev APIs, which may already be familiar to the +reader. + +The eventdev framework introduces the event driven programming model. In a +polling model, lcores poll ethdev ports and associated Rx queues directly +to look for a packet. By contrast in an event driven model, lcores call the +scheduler that selects packets for them based on programmer-specified criteria. +The Eventdev library adds support for an event driven programming model, which +offers applications automatic multicore scaling, dynamic load balancing, +pipelining, packet ingress order maintenance and synchronization services to +simplify application packet processing. + +By introducing an event driven programming model, DPDK can support both polling +and event driven programming models for packet processing, and applications are +free to choose whatever model (or combination of the two) best suits their +needs. + +Step-by-step instructions of the eventdev design is available in the `API +Walk-through`_ section later in this document. + +Event struct +------------ + +The eventdev API represents each event with a generic struct, which contains a +payload and metadata required for scheduling by an eventdev. The +``rte_event`` struct is a 16 byte C structure, defined in +``libs/librte_eventdev/rte_eventdev.h``. + +Event Metadata +~~~~~~~~~~~~~~ + +The rte_event structure contains the following metadata fields, which the +application fills in to have the event scheduled as required: + +* ``flow_id`` - The targeted flow identifier for the enq/deq operation. +* ``event_type`` - The source of this event, eg RTE_EVENT_TYPE_ETHDEV or CPU. +* ``sub_event_type`` - Distinguishes events inside the application, that have + the same event_type (see above) +* ``op`` - This field takes one of the RTE_EVENT_OP_* values, and tells the + eventdev about the status of the event - valid values are NEW, FORWARD or + RELEASE. +* ``sched_type`` - Represents the type of scheduling that should be performed + on this event, valid values are the RTE_SCHED_TYPE_ORDERED, ATOMIC and + PARALLEL. +* ``queue_id`` - The identifier for the event queue that the event is sent to. +* ``priority`` - The priority of this event, see RTE_EVENT_DEV_PRIORITY. + +Event Payload +~~~~~~~~~~~~~ + +The rte_event struct contains a union for payload, allowing flexibility in what +the actual event being scheduled is. The payload is a union of the following: + +* ``uint64_t u64`` +* ``void *event_ptr`` +* ``struct rte_mbuf *mbuf`` + +These three items in a union occupy the same 64 bits at the end of the rte_event +structure. The application can utilize the 64 bits directly by accessing the +u64 variable, while the event_ptr and mbuf are provided as convenience +variables. For example the mbuf pointer in the union can used to schedule a +DPDK packet. + +Queues +~~~~~~ + +An event queue is a queue containing events that are scheduled by the event +device. An event queue contains events of different flows associated with +scheduling types, such as atomic, ordered, or parallel. + +Queue All Types Capable +^^^^^^^^^^^^^^^^^^^^^^^ + +If RTE_EVENT_DEV_CAP_QUEUE_ALL_TYPES capability bit is set in the event device, +then events of any type may be sent to any queue. Otherwise, the queues only +support events of the type that it was created with. + +Queue All Types Incapable +^^^^^^^^^^^^^^^^^^^^^^^^^ + +In this case, each stage has a specified scheduling type. The application +configures each queue for a specific type of scheduling, and just enqueues all +events to the eventdev. An example of a PMD of this type is the eventdev +software PMD. + +The Eventdev API supports the following scheduling types per queue: + +* Atomic +* Ordered +* Parallel + +Atomic, Ordered and Parallel are load-balanced scheduling types: the output +of the queue can be spread out over multiple CPU cores. + +Atomic scheduling on a queue ensures that a single flow is not present on two +different CPU cores at the same time. Ordered allows sending all flows to any +core, but the scheduler must ensure that on egress the packets are returned to +ingress order on downstream queue enqueue. Parallel allows sending all flows +to all CPU cores, without any re-ordering guarantees. + +Single Link Flag +^^^^^^^^^^^^^^^^ + +There is a SINGLE_LINK flag which allows an application to indicate that only +one port will be connected to a queue. Queues configured with the single-link +flag follow a FIFO like structure, maintaining ordering but it is only capable +of being linked to a single port (see below for port and queue linking details). + + +Ports +~~~~~ + +Ports are the points of contact between worker cores and the eventdev. The +general use-case will see one CPU core using one port to enqueue and dequeue +events from an eventdev. Ports are linked to queues in order to retrieve events +from those queues (more details in `Linking Queues and Ports`_ below). + + +API Walk-through +---------------- + +This section will introduce the reader to the eventdev API, showing how to +create and configure an eventdev and use it for a two-stage atomic pipeline +with a single core for TX. The diagram below shows the final state of the +application after this walk-through: + +.. _figure_eventdev-usage1: + +.. figure:: img/eventdev_usage.* + + Sample eventdev usage, with RX, two atomic stages and a single-link to TX. + + +A high level overview of the setup steps are: + +* rte_event_dev_configure() +* rte_event_queue_setup() +* rte_event_port_setup() +* rte_event_port_link() +* rte_event_dev_start() + + +Init and Config +~~~~~~~~~~~~~~~ + +The eventdev library uses vdev options to add devices to the DPDK application. +The ``--vdev`` EAL option allows adding eventdev instances to your DPDK +application, using the name of the eventdev PMD as an argument. + +For example, to create an instance of the software eventdev scheduler, the +following vdev arguments should be provided to the application EAL command line: + +.. code-block:: console + + ./dpdk_application --vdev="event_sw0" + +In the following code, we configure eventdev instance with 3 queues +and 6 ports as follows. The 3 queues consist of 2 Atomic and 1 Single-Link, +while the 6 ports consist of 4 workers, 1 RX and 1 TX. + +.. code-block:: c + + const struct rte_event_dev_config config = { + .nb_event_queues = 3, + .nb_event_ports = 6, + .nb_events_limit = 4096, + .nb_event_queue_flows = 1024, + .nb_event_port_dequeue_depth = 128, + .nb_event_port_enqueue_depth = 128, + }; + int err = rte_event_dev_configure(dev_id, &config); + +The remainder of this walk-through assumes that dev_id is 0. + +Setting up Queues +~~~~~~~~~~~~~~~~~ + +Once the eventdev itself is configured, the next step is to configure queues. +This is done by setting the appropriate values in a queue_conf structure, and +calling the setup function. Repeat this step for each queue, starting from +0 and ending at ``nb_event_queues - 1`` from the event_dev config above. + +.. code-block:: c + + struct rte_event_queue_conf atomic_conf = { + .event_queue_cfg = RTE_EVENT_QUEUE_CFG_ATOMIC_ONLY, + .priority = RTE_EVENT_DEV_PRIORITY_NORMAL, + .nb_atomic_flows = 1024, + .nb_atomic_order_sequences = 1024, + }; + int dev_id = 0; + int queue_id = 0; + int err = rte_event_queue_setup(dev_id, queue_id, &atomic_conf); + +The remainder of this walk-through assumes that the queues are configured as +follows: + + * id 0, atomic queue #1 + * id 1, atomic queue #2 + * id 2, single-link queue + +Setting up Ports +~~~~~~~~~~~~~~~~ + +Once queues are set up successfully, create the ports as required. Each port +should be set up with its corresponding port_conf type, worker for worker cores, +rx and tx for the RX and TX cores: + +.. code-block:: c + + struct rte_event_port_conf rx_conf = { + .dequeue_depth = 128, + .enqueue_depth = 128, + .new_event_threshold = 1024, + }; + struct rte_event_port_conf worker_conf = { + .dequeue_depth = 16, + .enqueue_depth = 64, + .new_event_threshold = 4096, + }; + struct rte_event_port_conf tx_conf = { + .dequeue_depth = 128, + .enqueue_depth = 128, + .new_event_threshold = 4096, + }; + int dev_id = 0; + int port_id = 0; + int err = rte_event_port_setup(dev_id, port_id, &CORE_FUNCTION_conf); + +It is now assumed that: + + * port 0: RX core + * ports 1,2,3,4: Workers + * port 5: TX core + +Linking Queues and Ports +~~~~~~~~~~~~~~~~~~~~~~~~ + +The final step is to "wire up" the ports to the queues. After this, the +eventdev is capable of scheduling events, and when cores request work to do, +the correct events are provided to that core. Note that the RX core takes input +from eg: a NIC so it is not linked to any eventdev queues. + +Linking all workers to atomic queues, and the TX core to the single-link queue +can be achieved like this: + +.. code-block:: c + + uint8_t port_id = 0; + uint8_t atomic_qs[] = {0, 1}; + uint8_t single_link_q = 2; + uint8_t tx_port_id = 5; + uin8t_t priority = RTE_EVENT_DEV_PRIORITY_NORMAL; + + for(int i = 0; i < 4; i++) { + int worker_port = i + 1; + int links_made = rte_event_port_link(dev_id, worker_port, atomic_qs, NULL, 2); + } + int links_made = rte_event_port_link(dev_id, tx_port_id, &single_link_q, &priority, 1); + +Starting the EventDev +~~~~~~~~~~~~~~~~~~~~~ + +A single function call tells the eventdev instance to start processing +events. Note that all queues must be linked to for the instance to start, as +if any queue is not linked to, enqueuing to that queue will cause the +application to backpressure and eventually stall due to no space in the +eventdev. + +.. code-block:: c + + int err = rte_event_dev_start(dev_id); + +Ingress of New Events +~~~~~~~~~~~~~~~~~~~~~ + +Now that the eventdev is set up, and ready to receive events, the RX core must +enqueue some events into the system for it to schedule. The events to be +scheduled are ordinary DPDK packets, received from an eth_rx_burst() as normal. +The following code shows how those packets can be enqueued into the eventdev: + +.. code-block:: c + + const uint16_t nb_rx = rte_eth_rx_burst(eth_port, 0, mbufs, BATCH_SIZE); + + for (i = 0; i < nb_rx; i++) { + ev[i].flow_id = mbufs[i]->hash.rss; + ev[i].op = RTE_EVENT_OP_NEW; + ev[i].sched_type = RTE_EVENT_QUEUE_CFG_ATOMIC_ONLY; + ev[i].queue_id = 0; + ev[i].event_type = RTE_EVENT_TYPE_ETHDEV; + ev[i].sub_event_type = 0; + ev[i].priority = RTE_EVENT_DEV_PRIORITY_NORMAL; + ev[i].mbuf = mbufs[i]; + } + + const int nb_tx = rte_event_enqueue_burst(dev_id, port_id, ev, nb_rx); + if (nb_tx != nb_rx) { + for(i = nb_tx; i < nb_rx; i++) + rte_pktmbuf_free(mbufs[i]); + } + +Forwarding of Events +~~~~~~~~~~~~~~~~~~~~ + +Now that the RX core has injected events, there is work to be done by the +workers. Note that each worker will dequeue as many events as it can in a burst, +process each one individually, and then burst the packets back into the +eventdev. + +The worker can lookup the events source from ``event.queue_id``, which should +indicate to the worker what workload needs to be performed on the event. +Once done, the worker can update the ``event.queue_id`` to a new value, to send +the event to the next stage in the pipeline. + +.. code-block:: c + + int timeout = 0; + struct rte_event events[BATCH_SIZE]; + uint16_t nb_rx = rte_event_dequeue_burst(dev_id, worker_port_id, events, BATCH_SIZE, timeout); + + for (i = 0; i < nb_rx; i++) { + /* process mbuf using events[i].queue_id as pipeline stage */ + struct rte_mbuf *mbuf = events[i].mbuf; + /* Send event to next stage in pipeline */ + events[i].queue_id++; + } + + uint16_t nb_tx = rte_event_enqueue_burst(dev_id, port_id, events, nb_rx); + + +Egress of Events +~~~~~~~~~~~~~~~~ + +Finally, when the packet is ready for egress or needs to be dropped, we need +to inform the eventdev that the packet is no longer being handled by the +application. This can be done by calling dequeue() or dequeue_burst(), which +indicates that the previous burst of packets is no longer in use by the +application. + +An event driven worker thread has following typical workflow on fastpath: + +.. code-block:: c + + while (1) { + rte_event_dequeue_burst(...); + (event processing) + rte_event_enqueue_burst(...); + } + + +Summary +------- + +The eventdev library allows an application to easily schedule events as it +requires, either using a run-to-completion or pipeline processing model. The +queues and ports abstract the logical functionality of an eventdev, providing +the application with a generic method to schedule events. With the flexible +PMD infrastructure applications benefit of improvements in existing eventdevs +and additions of new ones without modification. diff --git a/doc/guides/prog_guide/generic_receive_offload_lib.rst b/doc/guides/prog_guide/generic_receive_offload_lib.rst new file mode 100644 index 00000000..22e50ecb --- /dev/null +++ b/doc/guides/prog_guide/generic_receive_offload_lib.rst @@ -0,0 +1,159 @@ +.. BSD LICENSE + Copyright(c) 2017 Intel Corporation. All rights reserved. + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of Intel Corporation nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +Generic Receive Offload Library +=============================== + +Generic Receive Offload (GRO) is a widely used SW-based offloading +technique to reduce per-packet processing overhead. It gains performance +by reassembling small packets into large ones. To enable more flexibility +to applications, DPDK implements GRO as a standalone library. Applications +explicitly use the GRO library to merge small packets into large ones. + +The GRO library assumes all input packets have correct checksums. In +addition, the GRO library doesn't re-calculate checksums for merged +packets. If input packets are IP fragmented, the GRO library assumes +they are complete packets (i.e. with L4 headers). + +Currently, the GRO library implements TCP/IPv4 packet reassembly. + +Reassembly Modes +---------------- + +The GRO library provides two reassembly modes: lightweight and +heavyweight mode. If applications want to merge packets in a simple way, +they can use the lightweight mode API. If applications want more +fine-grained controls, they can choose the heavyweight mode API. + +Lightweight Mode +~~~~~~~~~~~~~~~~ + +The ``rte_gro_reassemble_burst()`` function is used for reassembly in +lightweight mode. It tries to merge N input packets at a time, where +N should be less than or equal to ``RTE_GRO_MAX_BURST_ITEM_NUM``. + +In each invocation, ``rte_gro_reassemble_burst()`` allocates temporary +reassembly tables for the desired GRO types. Note that the reassembly +table is a table structure used to reassemble packets and different GRO +types (e.g. TCP/IPv4 GRO and TCP/IPv6 GRO) have different reassembly table +structures. The ``rte_gro_reassemble_burst()`` function uses the reassembly +tables to merge the N input packets. + +For applications, performing GRO in lightweight mode is simple. They +just need to invoke ``rte_gro_reassemble_burst()``. Applications can get +GROed packets as soon as ``rte_gro_reassemble_burst()`` returns. + +Heavyweight Mode +~~~~~~~~~~~~~~~~ + +The ``rte_gro_reassemble()`` function is used for reassembly in heavyweight +mode. Compared with the lightweight mode, performing GRO in heavyweight mode +is relatively complicated. + +Before performing GRO, applications need to create a GRO context object +by calling ``rte_gro_ctx_create()``. A GRO context object holds the +reassembly tables of desired GRO types. Note that all update/lookup +operations on the context object are not thread safe. So if different +processes or threads want to access the same context object simultaneously, +some external syncing mechanisms must be used. + +Once the GRO context is created, applications can then use the +``rte_gro_reassemble()`` function to merge packets. In each invocation, +``rte_gro_reassemble()`` tries to merge input packets with the packets +in the reassembly tables. If an input packet is an unsupported GRO type, +or other errors happen (e.g. SYN bit is set), ``rte_gro_reassemble()`` +returns the packet to applications. Otherwise, the input packet is either +merged or inserted into a reassembly table. + +When applications want to get GRO processed packets, they need to use +``rte_gro_timeout_flush()`` to flush them from the tables manually. + +TCP/IPv4 GRO +------------ + +TCP/IPv4 GRO supports merging small TCP/IPv4 packets into large ones, +using a table structure called the TCP/IPv4 reassembly table. + +TCP/IPv4 Reassembly Table +~~~~~~~~~~~~~~~~~~~~~~~~~ + +A TCP/IPv4 reassembly table includes a "key" array and an "item" array. +The key array keeps the criteria to merge packets and the item array +keeps the packet information. + +Each key in the key array points to an item group, which consists of +packets which have the same criteria values but can't be merged. A key +in the key array includes two parts: + +* ``criteria``: the criteria to merge packets. If two packets can be + merged, they must have the same criteria values. + +* ``start_index``: the item array index of the first packet in the item + group. + +Each element in the item array keeps the information of a packet. An item +in the item array mainly includes three parts: + +* ``firstseg``: the mbuf address of the first segment of the packet. + +* ``lastseg``: the mbuf address of the last segment of the packet. + +* ``next_pkt_index``: the item array index of the next packet in the same + item group. TCP/IPv4 GRO uses ``next_pkt_index`` to chain the packets + that have the same criteria value but can't be merged together. + +Procedure to Reassemble a Packet +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To reassemble an incoming packet needs three steps: + +#. Check if the packet should be processed. Packets with one of the + following properties aren't processed and are returned immediately: + + * FIN, SYN, RST, URG, PSH, ECE or CWR bit is set. + + * L4 payload length is 0. + +#. Traverse the key array to find a key which has the same criteria + value with the incoming packet. If found, go to the next step. + Otherwise, insert a new key and a new item for the packet. + +#. Locate the first packet in the item group via ``start_index``. Then + traverse all packets in the item group via ``next_pkt_index``. If a + packet is found which can be merged with the incoming one, merge them + together. If one isn't found, insert the packet into this item group. + Note that to merge two packets is to link them together via mbuf's + ``next`` field. + +When packets are flushed from the reassembly table, TCP/IPv4 GRO updates +packet header fields for the merged packets. Note that before reassembling +the packet, TCP/IPv4 GRO doesn't check if the checksums of packets are +correct. Also, TCP/IPv4 GRO doesn't re-calculate checksums for merged +packets. diff --git a/doc/guides/prog_guide/img/crypto_xform_chain.svg b/doc/guides/prog_guide/img/crypto_xform_chain.svg index 4670a07e..13681631 100644 --- a/doc/guides/prog_guide/img/crypto_xform_chain.svg +++ b/doc/guides/prog_guide/img/crypto_xform_chain.svg @@ -69,7 +69,9 @@ class="st3">auth</tspan><tspan class="st3">_</tspan><tspan class="st3">xform </tspan><tspan x="16.02" dy="1.425em" class="st4">struct </tspan><tspan class="st3">rte</tspan><tspan class="st3">_</tspan><tspan class="st3">crypto</tspan><tspan class="st3">_</tspan><tspan class="st3">cipher</tspan><tspan class="st3">_</tspan><tspan - class="st3">xform</tspan></text> </g> + class="st3">xform</tspan><tspan x="18.76" dy="1.425em" class="st4">struct </tspan><tspan + class="st3">rte</tspan><tspan class="st3">_</tspan><tspan class="st3">crypto</tspan><tspan class="st3">_</tspan><tspan + class="st3">aead</tspan><tspan class="st3">_</tspan><tspan class="st3">xform</tspan></text> </g> <g id="shape11-38" transform="translate(10.6711,-238.133)"> <title>Rounded Rectangle.26</title> <desc>next transform (struct rte_crypto_sym_xform *)</desc> @@ -116,7 +118,9 @@ class="st3">auth</tspan><tspan class="st3">_</tspan><tspan class="st3">xform </tspan><tspan x="16.02" dy="1.425em" class="st4">struct </tspan><tspan class="st3">rte</tspan><tspan class="st3">_</tspan><tspan class="st3">crypto</tspan><tspan class="st3">_</tspan><tspan class="st3">cipher</tspan><tspan class="st3">_</tspan><tspan - class="st3">xform</tspan></text> </g> + class="st3">xform</tspan><tspan x="18.76" dy="1.425em" class="st4">struct </tspan><tspan + class="st3">rte</tspan><tspan class="st3">_</tspan><tspan class="st3">crypto</tspan><tspan class="st3">_</tspan><tspan + class="st3">aead</tspan><tspan class="st3">_</tspan><tspan class="st3">xform</tspan></text> </g> <g id="shape15-102" transform="translate(209.592,-163.865)"> <title>Rounded Rectangle.32</title> <desc>next transform (struct rte_crypto_sym_xform *)</desc> diff --git a/doc/guides/prog_guide/img/cryptodev_sym_sess.svg b/doc/guides/prog_guide/img/cryptodev_sym_sess.svg index e5f41ec5..a807ceba 100644 --- a/doc/guides/prog_guide/img/cryptodev_sym_sess.svg +++ b/doc/guides/prog_guide/img/cryptodev_sym_sess.svg @@ -1,10 +1,50 @@ <?xml version="1.0" encoding="UTF-8" standalone="no"?> -<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"> <!-- Generated by Microsoft Visio, SVG Export cryptodev_sym_sess.svg Page-1 --> -<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events" - width="2.17241in" height="2.8102in" viewBox="0 0 156.413 202.335" xml:space="preserve" color-interpolation-filters="sRGB" - class="st10"> - <style type="text/css"> + +<svg + xmlns:dc="http://purl.org/dc/elements/1.1/" + xmlns:cc="http://creativecommons.org/ns#" + xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" + xmlns:svg="http://www.w3.org/2000/svg" + xmlns="http://www.w3.org/2000/svg" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" + xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" + width="4.8933434in" + height="3.8972795in" + viewBox="0 0 352.31955 280.60496" + xml:space="preserve" + class="st10" + version="1.1" + id="svg70" + sodipodi:docname="cryptodev_sym_sess.svg" + style="font-size:12px;overflow:visible;color-interpolation-filters:sRGB;fill:none;fill-rule:evenodd;stroke-linecap:square;stroke-miterlimit:3" + inkscape:version="0.92.1 r15371"><metadata + id="metadata74"><rdf:RDF><cc:Work + rdf:about=""><dc:format>image/svg+xml</dc:format><dc:type + rdf:resource="http://purl.org/dc/dcmitype/StillImage" /><dc:title /></cc:Work></rdf:RDF></metadata><sodipodi:namedview + pagecolor="#ffffff" + bordercolor="#666666" + borderopacity="1" + objecttolerance="10" + gridtolerance="10" + guidetolerance="10" + inkscape:pageopacity="0" + inkscape:pageshadow="2" + inkscape:window-width="1920" + inkscape:window-height="1051" + id="namedview72" + showgrid="false" + inkscape:zoom="1.7495789" + inkscape:cx="208.74719" + inkscape:cy="216.52777" + inkscape:window-x="-9" + inkscape:window-y="-9" + inkscape:window-maximized="0" + inkscape:current-layer="g68-0" /> + <style + type="text/css" + id="style2"> <![CDATA[ .st1 {fill:url(#grad0-4);stroke:#386288;stroke-width:0.75} .st2 {fill:#386288;font-family:Calibri;font-size:0.833336em} @@ -19,48 +59,334 @@ ]]> </style> - <defs id="Patterns_And_Gradients"> - <linearGradient id="grad0-4" x1="0" y1="0" x2="1" y2="0" gradientTransform="rotate(60 0.5 0.5)"> - <stop offset="0" stop-color="#e8ebef" stop-opacity="1"/> - <stop offset="0.24" stop-color="#f4f5f7" stop-opacity="1"/> - <stop offset="0.54" stop-color="#feffff" stop-opacity="1"/> + <defs + id="Patterns_And_Gradients"><marker + inkscape:isstock="true" + style="overflow:visible" + id="marker5421" + refX="0" + refY="0" + orient="auto" + inkscape:stockid="Arrow2Lend"><path + transform="matrix(-1.1,0,0,-1.1,-1.1,0)" + d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" + style="fill:#41719c;fill-opacity:1;fill-rule:evenodd;stroke:#41719c;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" + id="path5419" + inkscape:connector-curvature="0" /></marker><marker + inkscape:stockid="Arrow2Lend" + orient="auto" + refY="0" + refX="0" + id="Arrow2Lend" + style="overflow:visible" + inkscape:isstock="true"><path + id="path5004" + style="fill:#41719c;fill-opacity:1;fill-rule:evenodd;stroke:#41719c;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" + d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" + transform="matrix(-1.1,0,0,-1.1,-1.1,0)" + inkscape:connector-curvature="0" /></marker><marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Lend" + style="overflow:visible" + inkscape:isstock="true"><path + id="path4986" + d="M 0,0 5,-5 -12.5,0 5,5 Z" + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;stroke-opacity:1" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /></marker> + <linearGradient + id="grad0-4" + x1="0" + y1="0" + x2="1" + y2="0" + gradientTransform="rotate(60,0.5,0.5)"> + <stop + offset="0" + stop-color="#e8ebef" + stop-opacity="1" + id="stop4" /> + <stop + offset="0.24" + stop-color="#f4f5f7" + stop-opacity="1" + id="stop6" /> + <stop + offset="0.54" + stop-color="#feffff" + stop-opacity="1" + id="stop8" /> </linearGradient> - </defs> - <defs id="Filters"> - <filter id="filter_2"> - <feGaussianBlur stdDeviation="2"/> + <filter + id="filter_2-4"><feGaussianBlur + stdDeviation="2" + id="feGaussianBlur12-0" /></filter><linearGradient + inkscape:collect="always" + xlink:href="#grad0-4" + id="linearGradient189" + gradientTransform="scale(0.8787489,1.1379815)" + x1="-0.42674366" + y1="0.98859203" + x2="176.71146" + y2="0.98859203" + gradientUnits="userSpaceOnUse" /><filter + id="filter_2-5"><feGaussianBlur + stdDeviation="2" + id="feGaussianBlur12-8" /></filter><filter + id="filter_2-3"><feGaussianBlur + stdDeviation="2" + id="feGaussianBlur12-2" /></filter><linearGradient + inkscape:collect="always" + xlink:href="#grad0-4" + id="linearGradient189-7" + gradientTransform="scale(0.8787489,1.1379815)" + x1="-0.42674366" + y1="0.98859203" + x2="176.71146" + y2="0.98859203" + gradientUnits="userSpaceOnUse" /><linearGradient + inkscape:collect="always" + xlink:href="#grad0-4" + id="linearGradient500" + gradientTransform="matrix(0.8787489,0,0,1.1379815,12.431599,21.739241)" + x1="-0.42674366" + y1="0.98859203" + x2="176.71146" + y2="0.98859203" + gradientUnits="userSpaceOnUse" /></defs> + <defs + id="Filters"> + <filter + id="filter_2"> + <feGaussianBlur + stdDeviation="2" + id="feGaussianBlur12" /> </filter> </defs> - <g> - <title>Page-1</title> - <g id="shape18-1" transform="translate(0.749889,-0.75)"> - <title>Rounded Rectangle.12</title> - <desc>Crypto Symmetric Session</desc> - <path d="M6.78 202.33 L148.14 202.33 A6.77735 6.77735 -180 0 0 154.91 195.56 L154.91 8.28 A6.77735 6.77735 -180 0 0 148.14 - 1.5 L6.78 1.5 A6.77735 6.77735 -180 0 0 -0 8.28 L0 195.56 A6.77735 6.77735 -180 0 0 6.78 202.33 Z" - class="st1"/> - <text x="24.76" y="14.5" class="st2">Crypto Symmetric Session</text> </g> - <g id="shape19-6" transform="translate(10.6711,-9.82087)"> - <title>Rounded Rectangle.13</title> - <desc>Private Session Data</desc> - <g id="shadow19-7" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st3"> - <path d="M5.91 202.33 L129.16 202.33 A5.90925 5.90925 -180 0 0 135.07 196.43 L135.07 103.65 A5.90925 5.90925 -180 - 0 0 129.16 97.74 L5.91 97.74 A5.90925 5.90925 -180 0 0 -0 103.65 L0 196.43 A5.90925 5.90925 -180 0 0 - 5.91 202.33 Z" class="st4"/> - </g> - <path d="M5.91 202.33 L129.16 202.33 A5.90925 5.90925 -180 0 0 135.07 196.43 L135.07 103.65 A5.90925 5.90925 -180 0 0 - 129.16 97.74 L5.91 97.74 A5.90925 5.90925 -180 0 0 -0 103.65 L0 196.43 A5.90925 5.90925 -180 0 0 5.91 202.33 - Z" class="st5"/> - <text x="26.34" y="153.04" class="st6">Private Session Data</text> </g> - <g id="shape20-12" transform="translate(10.6711,-122.923)"> - <title>Rounded Rectangle.15</title> - <desc>General Session Data (struct rte_cryptodev_sym_session)</desc> - <path d="M5.91 202.33 L129.16 202.33 A5.90925 5.90925 -180 0 0 135.07 196.43 L135.07 160.06 A5.90925 5.90925 -180 0 0 - 129.16 154.15 L5.91 154.15 A5.90925 5.90925 -180 0 0 -0 160.06 L0 196.43 A5.90925 5.90925 -180 0 0 5.91 - 202.33 Z" class="st7"/> - <text x="24.58" y="175.24" class="st6">General Session Data <tspan x="9.16" dy="1.5em" class="st8">(</tspan><tspan - class="st9">struct </tspan><tspan class="st8">rte</tspan><tspan class="st8">_</tspan><tspan class="st8">cryptodev</tspan><tspan - class="st8">_</tspan><tspan class="st8">sym</tspan><tspan class="st8">_</tspan><tspan class="st8">session</tspan><tspan - class="st8">)</tspan></text> </g> + <g + id="g68" + transform="matrix(1,0,0,0.41409874,-12.807629,-5.4621159)"> + <title + id="title16">Page-1</title> + <g + id="shape18-1" + transform="translate(0.749889,-0.75)"> + <title + id="title18">Rounded Rectangle.12</title> + <desc + id="desc20">Crypto Symmetric Session</desc> + <path + d="M 19.211599,224.06924 H 160.5716 a 6.77735,6.77735 0 0 0 6.77,-6.77 V 30.019241 a 6.77735,6.77735 0 0 0 -6.77,-6.78 H 19.211599 a 6.77735,6.77735 0 0 0 -6.78,6.78 V 217.29924 a 6.77735,6.77735 0 0 0 6.78,6.77 z" + class="st1" + id="path22" + style="fill:url(#linearGradient500);stroke:#386288;stroke-width:0.75" + inkscape:connector-curvature="0" /> + <text + x="63.123039" + y="28.531481" + class="st2" + id="text24" + style="font-size:16.97244835px;font-family:Calibri;fill:#386288;stroke-width:1.69723928" + transform="scale(0.58919214,1.6972392)">Crypto Symmetric Session</text> + + </g> + <g + id="shape19-6" + transform="translate(10.6711,-9.82087)"> + <title + id="title27">Rounded Rectangle.13</title> + <desc + id="desc29">Private Session Data</desc> + </g> + <g + id="shape20-12" + transform="matrix(1,0,0,2.5278193,23.531375,-309.78186)"> + <title + id="title39">Rounded Rectangle.15</title> + <desc + id="desc41">void *sess_private_data[]</desc> + <path + d="m 5.91,202.33 h 123.25 a 5.90925,5.90925 -180 0 0 5.91,-5.9 v -36.37 a 5.90925,5.90925 -180 0 0 -5.91,-5.91 H 5.91 A 5.90925,5.90925 -180 0 0 0,160.06 v 36.37 a 5.90925,5.90925 -180 0 0 5.91,5.9 z" + class="st7" + id="path43" + inkscape:connector-curvature="0" + style="fill:#ffffff;stroke:#41719c;stroke-width:0.75" /> + <text + x="14.072042" + y="159.1931" + class="st6" + id="text65" + style="font-size:11.41061592px;font-family:Calibri;fill:#41719c;stroke-width:1.14105785" + transform="scale(0.92359087,1.0827305)">void *sess_private_data[] <tspan + x="-3.5230706" + class="st9" + id="tspan47" + style="font-weight:bold;font-size:9.12843513px;stroke-width:1.14105785" /></text> + + <rect + style="fill:none;fill-opacity:1;stroke:#41719c;stroke-width:0.73305672;stroke-opacity:1" + id="rect4604" + width="15.968175" + height="14.230948" + x="13.494645" + y="181.68814" /><rect + style="font-size:12px;overflow:visible;color-interpolation-filters:sRGB;fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#41719c;stroke-width:0.73305672;stroke-linecap:square;stroke-miterlimit:3;stroke-opacity:1" + id="rect4604-7" + width="15.968174" + height="14.230948" + x="29.46282" + y="181.68814" /><rect + style="font-size:12px;overflow:visible;color-interpolation-filters:sRGB;fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#41719c;stroke-width:0.73305672;stroke-linecap:square;stroke-miterlimit:3;stroke-opacity:1" + id="rect4604-7-6" + width="15.968174" + height="14.230948" + x="45.430992" + y="181.68814" /><rect + style="font-size:12px;overflow:visible;color-interpolation-filters:sRGB;fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#41719c;stroke-width:0.73305672;stroke-linecap:square;stroke-miterlimit:3;stroke-opacity:1" + id="rect4604-7-6-9" + width="15.968174" + height="14.230948" + x="61.399166" + y="181.68814" /><rect + style="font-size:12px;overflow:visible;color-interpolation-filters:sRGB;fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#41719c;stroke-width:0.73305672;stroke-linecap:square;stroke-miterlimit:3;stroke-opacity:1" + id="rect4604-7-6-9-8" + width="15.968174" + height="14.230948" + x="77.36734" + y="181.68814" /><rect + style="font-size:12px;overflow:visible;color-interpolation-filters:sRGB;fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#41719c;stroke-width:0.73305672;stroke-linecap:square;stroke-miterlimit:3;stroke-opacity:1" + id="rect4604-7-6-9-8-9" + width="15.968174" + height="14.230948" + x="93.33551" + y="181.68814" /><rect + style="font-size:12px;overflow:visible;color-interpolation-filters:sRGB;fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#41719c;stroke-width:0.73305672;stroke-linecap:square;stroke-miterlimit:3;stroke-opacity:1" + id="rect4604-7-6-9-8-9-6" + width="15.968174" + height="14.230948" + x="109.30369" + y="181.68814" /><path + style="fill:none;fill-opacity:1;stroke:#41719c;stroke-width:0.72427988px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow2Lend)" + d="m 117.64885,196.01764 0.22164,18.77485 44.6966,-0.0725 -0.22163,-20.00716 16.84434,0.43494" + id="path5030" + inkscape:connector-curvature="0" /></g> </g> -</svg> +<g + transform="translate(190.70887,-0.53319281)" + id="g68-0"><title + id="title16-2">Page-1</title><g + id="shape18-1-4" + transform="matrix(1,0,0,0.57815109,0.749889,-0.11722686)"><title + id="title18-4">Rounded Rectangle.12</title><desc + id="desc20-6">Crypto Symmetric Session</desc><path + inkscape:connector-curvature="0" + d="m 6.78,202.33 h 141.36 a 6.77735,6.77735 -180 0 0 6.77,-6.77 V 8.28 A 6.77735,6.77735 -180 0 0 148.14,1.5 H 6.78 A 6.77735,6.77735 -180 0 0 0,8.28 v 187.28 a 6.77735,6.77735 -180 0 0 6.78,6.77 z" + class="st1" + id="path22-0" + style="fill:url(#linearGradient189);stroke:#386288;stroke-width:0.75" /><text + x="26.317923" + y="17.335487" + class="st2" + id="text24-5" + style="font-size:14.02988338px;font-family:Calibri;fill:#386288;stroke-width:1.40298378" + transform="scale(0.71276665,1.4029837)">Crypto Driver Private Session</text> + +</g><g + id="shape19-6-5" + transform="matrix(1.022976,0,0,0.71529071,9.1114734,-39.403506)"><title + id="title27-2">Rounded Rectangle.13</title><desc + id="desc29-0">Private Session Data</desc><g + id="shadow19-7-1" + transform="translate(0.345598,1.97279)" + class="st3" + style="visibility:visible"><path + inkscape:connector-curvature="0" + d="m 5.91,202.33 h 123.25 a 5.90925,5.90925 -180 0 0 5.91,-5.9 v -92.78 a 5.90925,5.90925 -180 0 0 -5.91,-5.91 H 5.91 A 5.90925,5.90925 -180 0 0 0,103.65 v 92.78 a 5.90925,5.90925 -180 0 0 5.91,5.9 z" + class="st4" + id="path31-8" + style="fill:#bdd0e9;fill-opacity:0.25;stroke:#bdd0e9;stroke-opacity:0.25;filter:url(#filter_2)" /></g><path + inkscape:connector-curvature="0" + d="m 5.91,202.33 h 123.25 a 5.90925,5.90925 -180 0 0 5.91,-5.9 v -92.78 a 5.90925,5.90925 -180 0 0 -5.91,-5.91 H 5.91 A 5.90925,5.90925 -180 0 0 0,103.65 v 92.78 a 5.90925,5.90925 -180 0 0 5.91,5.9 z" + class="st5" + id="path34-8" + style="fill:#a6b6cd;stroke:#41719c;stroke-width:0.75" /><text + x="34.639763" + y="119.96548" + class="st6" + id="text36-7" + style="font-size:13.15105343px;font-family:Calibri;fill:#41719c;stroke-width:1.31510115" + transform="scale(0.76039781,1.3151011)">Private Session Data</text> + +</g><g + id="shape18-1-4-7" + transform="matrix(1,0,0,0.57815109,0.90591369,163.94402)"><title + id="title18-4-3">Rounded Rectangle.12</title><desc + id="desc20-6-5">Crypto Symmetric Session</desc><path + inkscape:connector-curvature="0" + d="m 6.78,202.33 h 141.36 a 6.77735,6.77735 -180 0 0 6.77,-6.77 V 8.28 A 6.77735,6.77735 -180 0 0 148.14,1.5 H 6.78 A 6.77735,6.77735 -180 0 0 0,8.28 v 187.28 a 6.77735,6.77735 -180 0 0 6.78,6.77 z" + class="st1" + id="path22-0-8" + style="fill:url(#linearGradient189-7);stroke:#386288;stroke-width:0.75" /><text + x="26.317923" + y="17.335487" + class="st2" + id="text24-5-1" + style="font-size:14.02988338px;font-family:Calibri;fill:#386288;stroke-width:1.40298378" + transform="scale(0.71276665,1.4029837)">Crypto Driver Private Session</text> + +</g><g + id="shape19-6-5-1" + transform="matrix(1.022976,0,0,0.71529071,9.2675037,124.65774)"><title + id="title27-2-4">Rounded Rectangle.13</title><desc + id="desc29-0-9">Private Session Data</desc><g + id="shadow19-7-1-8" + transform="translate(0.345598,1.97279)" + class="st3" + style="visibility:visible"><path + inkscape:connector-curvature="0" + d="m 5.91,202.33 h 123.25 a 5.90925,5.90925 -180 0 0 5.91,-5.9 v -92.78 a 5.90925,5.90925 -180 0 0 -5.91,-5.91 H 5.91 A 5.90925,5.90925 -180 0 0 0,103.65 v 92.78 a 5.90925,5.90925 -180 0 0 5.91,5.9 z" + class="st4" + id="path31-8-4" + style="fill:#bdd0e9;fill-opacity:0.25;stroke:#bdd0e9;stroke-opacity:0.25;filter:url(#filter_2-3)" /></g><path + inkscape:connector-curvature="0" + d="m 5.91,202.33 h 123.25 a 5.90925,5.90925 -180 0 0 5.91,-5.9 v -92.78 a 5.90925,5.90925 -180 0 0 -5.91,-5.91 H 5.91 A 5.90925,5.90925 -180 0 0 0,103.65 v 92.78 a 5.90925,5.90925 -180 0 0 5.91,5.9 z" + class="st5" + id="path34-8-3" + style="fill:#a6b6cd;stroke:#41719c;stroke-width:0.75" /><text + x="34.639763" + y="119.96548" + class="st6" + id="text36-7-6" + style="font-size:13.15105343px;font-family:Calibri;fill:#41719c;stroke-width:1.31510115" + transform="scale(0.76039781,1.3151011)">Private Session Data</text> + +</g><text + xml:space="preserve" + style="font-style:normal;font-weight:normal;font-size:30.00008774px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75000221" + x="57.540585" + y="145.94679" + id="text5070"><tspan + sodipodi:role="line" + id="tspan5068" + x="57.540585" + y="173.31679" + style="stroke-width:0.75000221"></tspan></text> +<text + xml:space="preserve" + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:22.00006485px;line-height:1.25;font-family:sans-serif;-inkscape-font-specification:'sans-serif, Normal';font-variant-ligatures:normal;font-variant-caps:normal;font-variant-numeric:normal;font-feature-settings:normal;text-align:start;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.75000221" + x="60.571766" + y="143.21872" + id="text5074"><tspan + sodipodi:role="line" + id="tspan5072" + x="60.571766" + y="143.21872" + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:22.00006485px;font-family:sans-serif;-inkscape-font-specification:'sans-serif, Normal';font-variant-ligatures:normal;font-variant-caps:normal;font-variant-numeric:normal;font-feature-settings:normal;text-align:start;writing-mode:lr-tb;text-anchor:start;stroke-width:0.75000221">...</tspan></text> +<path + style="fill:none;stroke:#41719c;stroke-width:0.74499911px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#marker5421)" + d="M -158.57624,71.371238 -157.38,232.04055 -1.1215065,232.19212" + id="path5076" + inkscape:connector-curvature="0" /></g></svg> diff --git a/doc/guides/prog_guide/img/eventdev_usage.svg b/doc/guides/prog_guide/img/eventdev_usage.svg new file mode 100644 index 00000000..7765649b --- /dev/null +++ b/doc/guides/prog_guide/img/eventdev_usage.svg @@ -0,0 +1,994 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> + +<svg + xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" + xmlns:dc="http://purl.org/dc/elements/1.1/" + xmlns:cc="http://creativecommons.org/ns#" + xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" + xmlns:svg="http://www.w3.org/2000/svg" + xmlns="http://www.w3.org/2000/svg" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" + xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" + width="683.12061" + height="184.672" + viewBox="0 0 546.49648 147.7376" + xml:space="preserve" + color-interpolation-filters="sRGB" + class="st9" + id="svg2" + version="1.1" + inkscape:version="0.48.4 r9939" + sodipodi:docname="eventdev_usage.svg" + style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible"><metadata + id="metadata214"><rdf:RDF><cc:Work + rdf:about=""><dc:format>image/svg+xml</dc:format><dc:type + rdf:resource="http://purl.org/dc/dcmitype/StillImage" /></cc:Work></rdf:RDF></metadata><sodipodi:namedview + pagecolor="#ffffff" + bordercolor="#666666" + borderopacity="1" + objecttolerance="10" + gridtolerance="10" + guidetolerance="10" + inkscape:pageopacity="0" + inkscape:pageshadow="2" + inkscape:window-width="1920" + inkscape:window-height="1017" + id="namedview212" + showgrid="false" + fit-margin-top="2" + fit-margin-left="2" + fit-margin-right="2" + fit-margin-bottom="2" + inkscape:zoom="1.2339869" + inkscape:cx="501.15554" + inkscape:cy="164.17693" + inkscape:window-x="-8" + inkscape:window-y="406" + inkscape:window-maximized="1" + inkscape:current-layer="g17" /> + <v:documentProperties + v:langID="1033" + v:viewMarkup="false"> + <v:userDefs> + <v:ud + v:nameU="msvSubprocessMaster" + v:prompt="" + v:val="VT4(Rectangle)" /> + <v:ud + v:nameU="msvNoAutoConnect" + v:val="VT0(1):26" /> + </v:userDefs> + </v:documentProperties> + + <style + type="text/css" + id="style4"> + + .st1 {visibility:visible} + .st2 {fill:#5b9bd5;fill-opacity:0.22;filter:url(#filter_2);stroke:#5b9bd5;stroke-opacity:0.22} + .st3 {fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25} + .st4 {fill:#feffff;font-family:Calibri;font-size:0.833336em} + .st5 {font-size:1em} + .st6 {fill:#ffd965;stroke:#c7c8c8;stroke-width:0.25} + .st7 {marker-end:url(#mrkr4-33);stroke:#5b9bd5;stroke-linecap:round;stroke-linejoin:round;stroke-width:1} + .st8 {fill:#5b9bd5;fill-opacity:1;stroke:#5b9bd5;stroke-opacity:1;stroke-width:0.28409090909091} + .st9 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3} + + </style> + + <defs + id="Markers"> + <g + id="lend4"> + <path + d="M 2,1 0,0 2,-1 2,1" + style="stroke:none" + id="path8" + inkscape:connector-curvature="0" /> + </g> + <marker + id="mrkr4-33" + class="st8" + v:arrowType="4" + v:arrowSize="2" + v:setback="7.04" + refX="-7.04" + orient="auto" + markerUnits="strokeWidth" + overflow="visible" + style="fill:#5b9bd5;fill-opacity:1;stroke:#5b9bd5;stroke-width:0.28409091;stroke-opacity:1;overflow:visible"> + <use + xlink:href="#lend4" + transform="scale(-3.52,-3.52)" + id="use11" + x="0" + y="0" + width="3" + height="3" /> + </marker> + <filter + id="filter_2-7" + color-interpolation-filters="sRGB"><feGaussianBlur + stdDeviation="2" + id="feGaussianBlur15-1" /></filter><marker + id="mrkr4-33-2" + class="st8" + v:arrowType="4" + v:arrowSize="2" + v:setback="7.04" + refX="-7.04" + orient="auto" + markerUnits="strokeWidth" + overflow="visible" + style="fill:#5b9bd5;fill-opacity:1;stroke:#5b9bd5;stroke-width:0.28409091;stroke-opacity:1;overflow:visible"><use + xlink:href="#lend4" + transform="scale(-3.52,-3.52)" + id="use11-3" + x="0" + y="0" + width="3" + height="3" /></marker><marker + id="mrkr4-33-6" + class="st8" + v:arrowType="4" + v:arrowSize="2" + v:setback="7.04" + refX="-7.04" + orient="auto" + markerUnits="strokeWidth" + overflow="visible" + style="fill:#5b9bd5;fill-opacity:1;stroke:#5b9bd5;stroke-width:0.28409091;stroke-opacity:1;overflow:visible"><use + xlink:href="#lend4" + transform="scale(-3.52,-3.52)" + id="use11-8" + x="0" + y="0" + width="3" + height="3" /></marker></defs> + <defs + id="Filters"> + <filter + id="filter_2" + color-interpolation-filters="sRGB"> + <feGaussianBlur + stdDeviation="2" + id="feGaussianBlur15" /> + </filter> + </defs> + <g + v:mID="0" + v:index="1" + v:groupContext="foregroundPage" + id="g17" + transform="translate(-47.323579,-90.784072)"> + <v:userDefs> + <v:ud + v:nameU="msvThemeOrder" + v:val="VT0(0):26" /> + </v:userDefs> + <title + id="title19">Page-1</title> + <v:pageProperties + v:drawingScale="1" + v:pageScale="1" + v:drawingUnits="0" + v:shadowOffsetX="9" + v:shadowOffsetY="-9" /> + <v:layer + v:name="Connector" + v:index="0" /> + <g + id="shape1-1" + v:mID="1" + v:groupContext="shape" + transform="matrix(0.77644652,0,0,0.77644652,128.62352,-288.18843)"> + <title + id="title22">Square</title> + <desc + id="desc24">Atomic Queue #1</desc> + <v:userDefs> + <v:ud + v:nameU="visVersion" + v:val="VT0(15):26" /> + </v:userDefs> + <v:textBlock + v:margins="rect(4,4,4,4)" /> + <v:textRect + cx="30.75" + cy="581.25" + width="61.5" + height="61.5" /> + <g + id="shadow1-2" + v:groupContext="shadow" + v:shadowOffsetX="0.345598" + v:shadowOffsetY="-1.97279" + v:shadowType="1" + transform="translate(0.345598,1.97279)" + class="st1" + style="visibility:visible"> + <rect + x="0" + y="550.5" + width="61.5" + height="61.5" + class="st2" + id="rect27" + style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2)" /> + </g> + <rect + x="0" + y="550.5" + width="61.5" + height="61.5" + class="st3" + id="rect29" + style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" /> + + </g> + <g + id="shape3-8" + v:mID="3" + v:groupContext="shape" + transform="matrix(0.77644652,0,0,0.77644652,297.37175,-288.18843)"> + <title + id="title36">Square.3</title> + <desc + id="desc38">Atomic Queue #2</desc> + <v:userDefs> + <v:ud + v:nameU="visVersion" + v:val="VT0(15):26" /> + </v:userDefs> + <v:textBlock + v:margins="rect(4,4,4,4)" /> + <v:textRect + cx="30.75" + cy="581.25" + width="61.5" + height="61.5" /> + <g + id="shadow3-9" + v:groupContext="shadow" + v:shadowOffsetX="0.345598" + v:shadowOffsetY="-1.97279" + v:shadowType="1" + transform="translate(0.345598,1.97279)" + class="st1" + style="visibility:visible"> + <rect + x="0" + y="550.5" + width="61.5" + height="61.5" + class="st2" + id="rect41" + style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2)" /> + </g> + <rect + x="0" + y="550.5" + width="61.5" + height="61.5" + class="st3" + id="rect43" + style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" /> + + </g> + <g + id="shape4-15" + v:mID="4" + v:groupContext="shape" + transform="matrix(0.77644652,0,0,0.77644652,466.1192,-288.18843)"> + <title + id="title50">Square.4</title> + <desc + id="desc52">Single Link Queue # 1</desc> + <v:userDefs> + <v:ud + v:nameU="visVersion" + v:val="VT0(15):26" /> + </v:userDefs> + <v:textBlock + v:margins="rect(4,4,4,4)" /> + <v:textRect + cx="30.75" + cy="581.25" + width="61.5" + height="61.5" /> + <g + id="shadow4-16" + v:groupContext="shadow" + v:shadowOffsetX="0.345598" + v:shadowOffsetY="-1.97279" + v:shadowType="1" + transform="translate(0.345598,1.97279)" + class="st1" + style="visibility:visible"> + <rect + x="0" + y="550.5" + width="61.5" + height="61.5" + class="st2" + id="rect55" + style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2)" /> + </g> + <rect + x="0" + y="550.5" + width="61.5" + height="61.5" + class="st3" + id="rect57" + style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" /> + + </g> + <g + id="shape5-22" + v:mID="5" + v:groupContext="shape" + transform="matrix(0.77644652,0,0,0.77644652,52.208527,-296.14701)"> + <title + id="title64">Circle</title> + <desc + id="desc66">RX</desc> + <v:userDefs> + <v:ud + v:nameU="visVersion" + v:val="VT0(15):26" /> + </v:userDefs> + <v:textBlock + v:margins="rect(4,4,4,4)" /> + <v:textRect + cx="20.5" + cy="591.5" + width="35.88" + height="30.75" /> + <g + id="shadow5-23" + v:groupContext="shadow" + v:shadowOffsetX="0.345598" + v:shadowOffsetY="-1.97279" + v:shadowType="1" + transform="translate(0.345598,1.97279)" + class="st1" + style="visibility:visible"> + <path + d="m 0,591.5 a 20.5,20.5 0 0 1 41,0 20.5,20.5 0 1 1 -41,0 z" + class="st2" + id="path69" + inkscape:connector-curvature="0" + style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2)" /> + </g> + <path + d="m 0,591.5 a 20.5,20.5 0 0 1 41,0 20.5,20.5 0 1 1 -41,0 z" + class="st6" + id="path71" + inkscape:connector-curvature="0" + style="fill:#ffd965;stroke:#c7c8c8;stroke-width:0.25" /> + <text + x="15.19" + y="594.5" + class="st4" + v:langID="1033" + id="text73" + style="fill:#feffff;font-family:Calibri"><v:paragraph + v:horizAlign="1" /><v:tabList />RX</text> + + </g> + <g + id="shape6-28" + v:mID="6" + v:groupContext="shape" + v:layerMember="0" + transform="matrix(0.77644652,0,0,0.77644652,84.042834,-305.07614)"> + <title + id="title76">Dynamic connector</title> + <path + d="m 0,603 50.38,0" + class="st7" + id="path78" + inkscape:connector-curvature="0" + style="stroke:#5b9bd5;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;marker-end:url(#mrkr4-33)" /> + </g> + <g + id="shape7-34" + v:mID="7" + v:groupContext="shape" + transform="matrix(0.77644652,0,0,0.77644652,220.95621,-296.14701)"> + <title + id="title81">Circle.7</title> + <desc + id="desc83">W ..</desc> + <v:userDefs> + <v:ud + v:nameU="visVersion" + v:val="VT0(15):26" /> + </v:userDefs> + <v:textBlock + v:margins="rect(4,4,4,4)" /> + <v:textRect + cx="20.5" + cy="591.5" + width="35.88" + height="30.75" /> + <g + id="shadow7-35" + v:groupContext="shadow" + v:shadowOffsetX="0.345598" + v:shadowOffsetY="-1.97279" + v:shadowType="1" + transform="translate(0.345598,1.97279)" + class="st1" + style="visibility:visible"> + <path + d="m 0,591.5 a 20.5,20.5 0 0 1 41,0 20.5,20.5 0 1 1 -41,0 z" + class="st2" + id="path86" + inkscape:connector-curvature="0" + style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2)" /> + </g> + <path + d="m 0,591.5 a 20.5,20.5 0 0 1 41,0 20.5,20.5 0 1 1 -41,0 z" + class="st6" + id="path88" + inkscape:connector-curvature="0" + style="fill:#ffd965;stroke:#c7c8c8;stroke-width:0.25" /> + <text + x="12.4" + y="594.5" + class="st4" + v:langID="1033" + id="text90" + style="fill:#feffff;font-family:Calibri"><v:paragraph + v:horizAlign="1" /><v:tabList />W ..</text> + + </g> + <g + id="shape9-40" + v:mID="9" + v:groupContext="shape" + transform="matrix(0.77644652,0,0,0.77644652,220.95621,-243.34865)"> + <title + id="title93">Circle.9</title> + <desc + id="desc95">W N</desc> + <v:userDefs> + <v:ud + v:nameU="visVersion" + v:val="VT0(15):26" /> + </v:userDefs> + <v:textBlock + v:margins="rect(4,4,4,4)" /> + <v:textRect + cx="20.5" + cy="591.5" + width="35.88" + height="30.75" /> + <g + id="shadow9-41" + v:groupContext="shadow" + v:shadowOffsetX="0.345598" + v:shadowOffsetY="-1.97279" + v:shadowType="1" + transform="translate(0.345598,1.97279)" + class="st1" + style="visibility:visible"> + <path + d="m 0,591.5 a 20.5,20.5 0 0 1 41,0 20.5,20.5 0 1 1 -41,0 z" + class="st2" + id="path98" + inkscape:connector-curvature="0" + style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2)" /> + </g> + <path + d="m 0,591.5 a 20.5,20.5 0 0 1 41,0 20.5,20.5 0 1 1 -41,0 z" + class="st6" + id="path100" + inkscape:connector-curvature="0" + style="fill:#ffd965;stroke:#c7c8c8;stroke-width:0.25" /> + <text + x="11.69" + y="594.5" + class="st4" + v:langID="1033" + id="text102" + style="fill:#feffff;font-family:Calibri"><v:paragraph + v:horizAlign="1" /><v:tabList />W N</text> + + </g> + <g + id="shape10-46" + v:mID="10" + v:groupContext="shape" + transform="matrix(0.77644652,0,0,0.77644652,220.95621,-348.94537)"> + <title + id="title105">Circle.10</title> + <desc + id="desc107">W 1</desc> + <v:userDefs> + <v:ud + v:nameU="visVersion" + v:val="VT0(15):26" /> + </v:userDefs> + <v:textBlock + v:margins="rect(4,4,4,4)" /> + <v:textRect + cx="20.5" + cy="591.5" + width="35.88" + height="30.75" /> + <g + id="shadow10-47" + v:groupContext="shadow" + v:shadowOffsetX="0.345598" + v:shadowOffsetY="-1.97279" + v:shadowType="1" + transform="translate(0.345598,1.97279)" + class="st1" + style="visibility:visible"> + <path + d="m 0,591.5 a 20.5,20.5 0 0 1 41,0 20.5,20.5 0 1 1 -41,0 z" + class="st2" + id="path110" + inkscape:connector-curvature="0" + style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2)" /> + </g> + <path + d="m 0,591.5 a 20.5,20.5 0 0 1 41,0 20.5,20.5 0 1 1 -41,0 z" + class="st6" + id="path112" + inkscape:connector-curvature="0" + style="fill:#ffd965;stroke:#c7c8c8;stroke-width:0.25" /> + <text + x="12.39" + y="594.5" + class="st4" + v:langID="1033" + id="text114" + style="fill:#feffff;font-family:Calibri"><v:paragraph + v:horizAlign="1" /><v:tabList />W 1</text> + + </g> + <g + id="shape11-52" + v:mID="11" + v:groupContext="shape" + v:layerMember="0" + transform="matrix(0.77644652,0,0,0.77644652,195.91581,-312.06416)"> + <title + id="title117">Dynamic connector.11</title> + <path + d="m 0,612 0,-68 25.21,0" + class="st7" + id="path119" + inkscape:connector-curvature="0" + style="stroke:#5b9bd5;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;marker-end:url(#mrkr4-33)" /> + </g> + <g + id="shape12-57" + v:mID="12" + v:groupContext="shape" + v:layerMember="0" + transform="matrix(0.77644652,0,0,0.77644652,176.37498,-305.07614)"> + <title + id="title122">Dynamic connector.12</title> + <path + d="m 0,603 50.38,0" + class="st7" + id="path124" + inkscape:connector-curvature="0" + style="stroke:#5b9bd5;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;marker-end:url(#mrkr4-33)" /> + </g> + <g + id="shape13-62" + v:mID="13" + v:groupContext="shape" + v:layerMember="0" + transform="matrix(0.77644652,0,0,0.77644652,176.37498,-312.06416)"> + <title + id="title127">Dynamic connector.13</title> + <path + d="m 0,612 25.17,0 0,68 25.21,0" + class="st7" + id="path129" + inkscape:connector-curvature="0" + style="stroke:#5b9bd5;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;marker-end:url(#mrkr4-33)" /> + </g> + <g + id="shape14-67" + v:mID="14" + v:groupContext="shape" + v:layerMember="0" + transform="matrix(0.77644652,0,0,0.77644652,252.79052,-259.2658)"> + <title + id="title132">Dynamic connector.14</title> + <path + d="m 0,612 26.88,0 0,-68 23.5,0" + class="st7" + id="path134" + inkscape:connector-curvature="0" + style="stroke:#5b9bd5;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;marker-end:url(#mrkr4-33)" /> + </g> + <g + id="shape15-72" + v:mID="15" + v:groupContext="shape" + v:layerMember="0" + transform="matrix(0.77644652,0,0,0.77644652,252.79052,-305.07614)"> + <title + id="title137">Dynamic connector.15</title> + <path + d="m 0,603 50.38,0" + class="st7" + id="path139" + inkscape:connector-curvature="0" + style="stroke:#5b9bd5;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;marker-end:url(#mrkr4-33)" /> + </g> + <g + id="shape19-77" + v:mID="19" + v:groupContext="shape" + transform="matrix(0.77644652,0,0,0.77644652,389.70366,-296.14701)"> + <title + id="title142">Circle.19</title> + <desc + id="desc144">W ..</desc> + <v:userDefs> + <v:ud + v:nameU="visVersion" + v:val="VT0(15):26" /> + </v:userDefs> + <v:textBlock + v:margins="rect(4,4,4,4)" /> + <v:textRect + cx="20.5" + cy="591.5" + width="35.88" + height="30.75" /> + <g + id="shadow19-78" + v:groupContext="shadow" + v:shadowOffsetX="0.345598" + v:shadowOffsetY="-1.97279" + v:shadowType="1" + transform="translate(0.345598,1.97279)" + class="st1" + style="visibility:visible"> + <path + d="m 0,591.5 a 20.5,20.5 0 0 1 41,0 20.5,20.5 0 1 1 -41,0 z" + class="st2" + id="path147" + inkscape:connector-curvature="0" + style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2)" /> + </g> + <path + d="m 0,591.5 a 20.5,20.5 0 0 1 41,0 20.5,20.5 0 1 1 -41,0 z" + class="st6" + id="path149" + inkscape:connector-curvature="0" + style="fill:#ffd965;stroke:#c7c8c8;stroke-width:0.25" /> + <text + x="12.4" + y="594.5" + class="st4" + v:langID="1033" + id="text151" + style="fill:#feffff;font-family:Calibri"><v:paragraph + v:horizAlign="1" /><v:tabList />W ..</text> + + </g> + <g + id="shape20-83" + v:mID="20" + v:groupContext="shape" + transform="matrix(0.77644652,0,0,0.77644652,389.70366,-243.34865)"> + <title + id="title154">Circle.20</title> + <desc + id="desc156">W N</desc> + <v:userDefs> + <v:ud + v:nameU="visVersion" + v:val="VT0(15):26" /> + </v:userDefs> + <v:textBlock + v:margins="rect(4,4,4,4)" /> + <v:textRect + cx="20.5" + cy="591.5" + width="35.88" + height="30.75" /> + <g + id="shadow20-84" + v:groupContext="shadow" + v:shadowOffsetX="0.345598" + v:shadowOffsetY="-1.97279" + v:shadowType="1" + transform="translate(0.345598,1.97279)" + class="st1" + style="visibility:visible"> + <path + d="m 0,591.5 a 20.5,20.5 0 0 1 41,0 20.5,20.5 0 1 1 -41,0 z" + class="st2" + id="path159" + inkscape:connector-curvature="0" + style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2)" /> + </g> + <path + d="m 0,591.5 a 20.5,20.5 0 0 1 41,0 20.5,20.5 0 1 1 -41,0 z" + class="st6" + id="path161" + inkscape:connector-curvature="0" + style="fill:#ffd965;stroke:#c7c8c8;stroke-width:0.25" /> + <text + x="11.69" + y="594.5" + class="st4" + v:langID="1033" + id="text163" + style="fill:#feffff;font-family:Calibri"><v:paragraph + v:horizAlign="1" /><v:tabList />W N</text> + + </g> + <g + id="shape21-89" + v:mID="21" + v:groupContext="shape" + transform="matrix(0.77644652,0,0,0.77644652,389.70366,-348.94537)"> + <title + id="title166">Circle.21</title> + <desc + id="desc168">W 1</desc> + <v:userDefs> + <v:ud + v:nameU="visVersion" + v:val="VT0(15):26" /> + </v:userDefs> + <v:textBlock + v:margins="rect(4,4,4,4)" /> + <v:textRect + cx="20.5" + cy="591.5" + width="35.88" + height="30.75" /> + <g + id="shadow21-90" + v:groupContext="shadow" + v:shadowOffsetX="0.345598" + v:shadowOffsetY="-1.97279" + v:shadowType="1" + transform="translate(0.345598,1.97279)" + class="st1" + style="visibility:visible"> + <path + d="m 0,591.5 a 20.5,20.5 0 0 1 41,0 20.5,20.5 0 1 1 -41,0 z" + class="st2" + id="path171" + inkscape:connector-curvature="0" + style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2)" /> + </g> + <path + d="m 0,591.5 a 20.5,20.5 0 0 1 41,0 20.5,20.5 0 1 1 -41,0 z" + class="st6" + id="path173" + inkscape:connector-curvature="0" + style="fill:#ffd965;stroke:#c7c8c8;stroke-width:0.25" /> + <text + x="12.39" + y="594.5" + class="st4" + v:langID="1033" + id="text175" + style="fill:#feffff;font-family:Calibri"><v:paragraph + v:horizAlign="1" /><v:tabList />W 1</text> + + </g> + <g + id="shape28-95" + v:mID="28" + v:groupContext="shape" + v:layerMember="0" + transform="matrix(0.77644652,0,0,0.77644652,345.12321,-305.07614)"> + <title + id="title178">Dynamic connector.28</title> + <path + d="m 0,603 50.38,0" + class="st7" + id="path180" + inkscape:connector-curvature="0" + style="stroke:#5b9bd5;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;marker-end:url(#mrkr4-33)" /> + </g> + <g + id="shape29-100" + v:mID="29" + v:groupContext="shape" + v:layerMember="0" + transform="matrix(0.77644652,0,0,0.77644652,345.12321,-312.06416)"> + <title + id="title183">Dynamic connector.29</title> + <path + d="m 0,612 28.33,0 0,-68 22.05,0" + class="st7" + id="path185" + inkscape:connector-curvature="0" + style="stroke:#5b9bd5;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;marker-end:url(#mrkr4-33)" /> + </g> + <g + id="shape30-105" + v:mID="30" + v:groupContext="shape" + v:layerMember="0" + transform="matrix(0.77644652,0,0,0.77644652,345.12321,-312.06416)"> + <title + id="title188">Dynamic connector.30</title> + <path + d="m 0,612 28.33,0 0,68 22.05,0" + class="st7" + id="path190" + inkscape:connector-curvature="0" + style="stroke:#5b9bd5;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;marker-end:url(#mrkr4-33)" /> + </g> + <g + id="shape31-110" + v:mID="31" + v:groupContext="shape" + v:layerMember="0" + transform="matrix(0.77644652,0,0,0.77644652,421.53797,-259.2658)"> + <title + id="title193">Dynamic connector.31</title> + <path + d="m 0,612 24.42,0 0,-68 25.96,0" + class="st7" + id="path195" + inkscape:connector-curvature="0" + style="stroke:#5b9bd5;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;marker-end:url(#mrkr4-33)" /> + </g> + <g + id="shape32-115" + v:mID="32" + v:groupContext="shape" + v:layerMember="0" + transform="matrix(0.77644652,0,0,0.77644652,421.53797,-305.07614)"> + <title + id="title198">Dynamic connector.32</title> + <path + d="m 0,603 50.38,0" + class="st7" + id="path200" + inkscape:connector-curvature="0" + style="stroke:#5b9bd5;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;marker-end:url(#mrkr4-33)" /> + </g> + <g + id="shape33-120" + v:mID="33" + v:groupContext="shape" + v:layerMember="0" + transform="matrix(0.77644652,0,0,0.77644652,421.53797,-364.86253)"> + <title + id="title203">Dynamic connector.33</title> + <path + d="m 0,612 24.42,0 0,68 25.96,0" + class="st7" + id="path205" + inkscape:connector-curvature="0" + style="stroke:#5b9bd5;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;marker-end:url(#mrkr4-33)" /> + </g> + <g + id="shape34-125" + v:mID="34" + v:groupContext="shape" + v:layerMember="0" + transform="matrix(0.77644652,0,0,0.77644652,252.79052,-364.86253)"> + <title + id="title208">Dynamic connector.34</title> + <path + d="m 0,612 26.88,0 0,68 23.5,0" + class="st7" + id="path210" + inkscape:connector-curvature="0" + style="stroke:#5b9bd5;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;marker-end:url(#mrkr4-33)" /> + </g> + <text + xml:space="preserve" + style="font-size:24.84628868px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + x="153.38116" + y="165.90149" + id="text3106" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + x="153.38116" + y="165.90149" + id="tspan3110" + style="font-size:8.69620132px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;writing-mode:lr-tb;text-anchor:middle;fill:#ffffff;fill-opacity:1;font-family:Sans;-inkscape-font-specification:Sans">Atomic #1</tspan></text> +<text + xml:space="preserve" + style="font-size:24.84628868px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;overflow:visible;font-family:Sans" + x="322.12939" + y="165.90149" + id="text3106-1" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + x="322.12939" + y="165.90149" + id="tspan3110-4" + style="font-size:8.69620132px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;writing-mode:lr-tb;text-anchor:middle;fill:#ffffff;fill-opacity:1;font-family:Sans;-inkscape-font-specification:Sans">Atomic #2</tspan></text> +<text + xml:space="preserve" + style="font-size:24.84628868px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;overflow:visible;font-family:Sans" + x="491.82089" + y="172.79289" + id="text3106-0" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + x="491.82089" + y="172.79289" + style="font-size:8.69620132px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;writing-mode:lr-tb;text-anchor:middle;fill:#ffffff;fill-opacity:1;font-family:Sans;-inkscape-font-specification:Sans" + id="tspan3923" /></text> +<text + xml:space="preserve" + style="font-size:24.84628868px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;overflow:visible;font-family:Sans" + x="491.02899" + y="165.03951" + id="text3106-8-5" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + x="491.02899" + y="165.03951" + id="tspan3110-2-1" + style="font-size:8.69620132px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;writing-mode:lr-tb;text-anchor:middle;fill:#ffffff;fill-opacity:1;font-family:Sans;-inkscape-font-specification:Sans">Single Link</tspan></text> +<g + style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible" + id="shape5-22-1" + v:mID="5" + v:groupContext="shape" + transform="matrix(0.77644652,0,0,0.77644652,556.00223,-296.89447)"><title + id="title64-5">Circle</title><desc + id="desc66-2">RX</desc><v:userDefs><v:ud + v:nameU="visVersion" + v:val="VT0(15):26" /></v:userDefs><v:textBlock + v:margins="rect(4,4,4,4)" /><v:textRect + cx="20.5" + cy="591.5" + width="35.88" + height="30.75" /><g + id="shadow5-23-7" + v:groupContext="shadow" + v:shadowOffsetX="0.345598" + v:shadowOffsetY="-1.97279" + v:shadowType="1" + transform="translate(0.345598,1.97279)" + class="st1" + style="visibility:visible"><path + d="m 0,591.5 a 20.5,20.5 0 0 1 41,0 20.5,20.5 0 1 1 -41,0 z" + class="st2" + id="path69-6" + inkscape:connector-curvature="0" + style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-7)" /></g><path + d="m 0,591.5 a 20.5,20.5 0 0 1 41,0 20.5,20.5 0 1 1 -41,0 z" + class="st6" + id="path71-1" + inkscape:connector-curvature="0" + style="fill:#ffd965;stroke:#c7c8c8;stroke-width:0.25" /><text + x="11.06866" + y="596.56067" + class="st4" + v:langID="1033" + id="text73-4" + style="fill:#feffff;font-family:Calibri"> TX</text> +</g><g + style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible" + id="shape28-95-5" + v:mID="28" + v:groupContext="shape" + v:layerMember="0" + transform="matrix(0.77644652,0,0,0.77644652,512.00213,-305.42637)"><title + id="title178-7">Dynamic connector.28</title><path + d="m 0,603 50.38,0" + class="st7" + id="path180-6" + inkscape:connector-curvature="0" + style="stroke:#5b9bd5;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;marker-end:url(#mrkr4-33)" /></g></g> +</svg> diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index ef5a02ad..40f04a10 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -38,11 +38,13 @@ Programmer's Guide intro overview env_abstraction_layer + service_cores ring_lib mempool_lib mbuf_lib poll_mode_drv rte_flow + traffic_management cryptodev_lib link_bonding_poll_mode_drv_lib timer_lib @@ -53,10 +55,12 @@ Programmer's Guide packet_distrib_lib reorder_lib ip_fragment_reassembly_lib + generic_receive_offload_lib pdump_lib multi_proc_support kernel_nic_interface thread_safety_dpdk_functions + eventdev qos_framework power_man packet_classif_access_ctrl diff --git a/doc/guides/prog_guide/kernel_nic_interface.rst b/doc/guides/prog_guide/kernel_nic_interface.rst index 6f7fd287..4b2dd770 100644 --- a/doc/guides/prog_guide/kernel_nic_interface.rst +++ b/doc/guides/prog_guide/kernel_nic_interface.rst @@ -69,7 +69,7 @@ The KNI kernel loadable module provides support for two types of devices: (simulating the RX side of the net driver). * For multiple kernel thread mode, maintains a kernel thread context for each KNI instance - (simulating the RX side of the new driver). + (simulating the RX side of the net driver). * Net device: diff --git a/doc/guides/prog_guide/metrics_lib.rst b/doc/guides/prog_guide/metrics_lib.rst index 702c29d9..d52204fd 100644 --- a/doc/guides/prog_guide/metrics_lib.rst +++ b/doc/guides/prog_guide/metrics_lib.rst @@ -144,6 +144,7 @@ print out all metrics for a given port: .. code-block:: c void print_metrics() { + struct rte_metric_value *metrics; struct rte_metric_name *names; int len; diff --git a/doc/guides/prog_guide/perf_opt_guidelines.rst b/doc/guides/prog_guide/perf_opt_guidelines.rst index 7c24aa48..286bf983 100644 --- a/doc/guides/prog_guide/perf_opt_guidelines.rst +++ b/doc/guides/prog_guide/perf_opt_guidelines.rst @@ -38,7 +38,7 @@ Performance Optimization Guidelines Introduction ------------ -The following sections describe optimizations used in the DPDK and optimizations that should be considered for a new applications. +The following sections describe optimizations used in DPDK and optimizations that should be considered for new applications. They also highlight the performance-impacting coding techniques that should, and should not be, used when developing an application using the DPDK. diff --git a/doc/guides/prog_guide/poll_mode_drv.rst b/doc/guides/prog_guide/poll_mode_drv.rst index 4987f70a..1ac8f7eb 100644 --- a/doc/guides/prog_guide/poll_mode_drv.rst +++ b/doc/guides/prog_guide/poll_mode_drv.rst @@ -84,7 +84,7 @@ Whenever needed and appropriate, asynchronous communication should be introduced Avoiding lock contention is a key issue in a multi-core environment. To address this issue, PMDs are designed to work with per-core private resources as much as possible. -For example, a PMD maintains a separate transmit queue per-core, per-port. +For example, a PMD maintains a separate transmit queue per-core, per-port, if the PMD is not ``DEV_TX_OFFLOAD_MT_LOCKFREE`` capable. In the same way, every receive queue of a port is assigned to and polled by a single logical core (lcore). To comply with Non-Uniform Memory Access (NUMA), memory management is designed to assign to each logical core @@ -146,6 +146,16 @@ This is also true for the pipe-line model provided all logical cores used are lo Multiple logical cores should never share receive or transmit queues for interfaces since this would require global locks and hinder performance. +If the PMD is ``DEV_TX_OFFLOAD_MT_LOCKFREE`` capable, multiple threads can invoke ``rte_eth_tx_burst()`` +concurrently on the same tx queue without SW lock. This PMD feature found in some NICs and useful in the following use cases: + +* Remove explicit spinlock in some applications where lcores are not mapped to Tx queues with 1:1 relation. + +* In the eventdev use case, avoid dedicating a separate TX core for transmitting and thus + enables more scaling as all workers can send the packets. + +See `Hardware Offload`_ for ``DEV_TX_OFFLOAD_MT_LOCKFREE`` capability probing details. + Device Identification and Configuration --------------------------------------- @@ -290,7 +300,8 @@ Hardware Offload Depending on driver capabilities advertised by ``rte_eth_dev_info_get()``, the PMD may support hardware offloading -feature like checksumming, TCP segmentation or VLAN insertion. +feature like checksumming, TCP segmentation, VLAN insertion or +lockfree multithreaded TX burst on the same TX queue. The support of these offload features implies the addition of dedicated status bit(s) and value field(s) into the rte_mbuf data structure, along diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst index b587ba99..662a9123 100644 --- a/doc/guides/prog_guide/rte_flow.rst +++ b/doc/guides/prog_guide/rte_flow.rst @@ -898,7 +898,7 @@ Matches a MPLS header. - Default ``mask`` matches label only. Item: ``GRE`` -^^^^^^^^^^^^^^ +^^^^^^^^^^^^^ Matches a GRE header. @@ -906,6 +906,55 @@ Matches a GRE header. - ``protocol``: protocol type. - Default ``mask`` matches protocol only. +Item: ``FUZZY`` +^^^^^^^^^^^^^^^ + +Fuzzy pattern match, expect faster than default. + +This is for device that support fuzzy match option. Usually a fuzzy match is +fast but the cost is accuracy. i.e. Signature Match only match pattern's hash +value, but it is possible two different patterns have the same hash value. + +Matching accuracy level can be configured by threshold. Driver can divide the +range of threshold and map to different accuracy levels that device support. + +Threshold 0 means perfect match (no fuzziness), while threshold 0xffffffff +means fuzziest match. + +.. _table_rte_flow_item_fuzzy: + +.. table:: FUZZY + + +----------+---------------+--------------------------------------------------+ + | Field | Subfield | Value | + +==========+===============+==================================================+ + | ``spec`` | ``threshold`` | 0 as perfect match, 0xffffffff as fuzziest match | + +----------+---------------+--------------------------------------------------+ + | ``last`` | ``threshold`` | upper range value | + +----------+---------------+--------------------------------------------------+ + | ``mask`` | ``threshold`` | bit-mask apply to "spec" and "last" | + +----------+---------------+--------------------------------------------------+ + +Usage example, fuzzy match a TCPv4 packets: + +.. _table_rte_flow_item_fuzzy_example: + +.. table:: Fuzzy matching + + +-------+----------+ + | Index | Item | + +=======+==========+ + | 0 | FUZZY | + +-------+----------+ + | 1 | Ethernet | + +-------+----------+ + | 2 | IPv4 | + +-------+----------+ + | 3 | TCP | + +-------+----------+ + | 4 | END | + +-------+----------+ + Actions ~~~~~~~ @@ -1517,6 +1566,73 @@ Return values: - 0 on success, a negative errno value otherwise and ``rte_errno`` is set. +Isolated mode +------------- + +The general expectation for ingress traffic is that flow rules process it +first; the remaining unmatched or pass-through traffic usually ends up in a +queue (with or without RSS, locally or in some sub-device instance) +depending on the global configuration settings of a port. + +While fine from a compatibility standpoint, this approach makes drivers more +complex as they have to check for possible side effects outside of this API +when creating or destroying flow rules. It results in a more limited set of +available rule types due to the way device resources are assigned (e.g. no +support for the RSS action even on capable hardware). + +Given that nonspecific traffic can be handled by flow rules as well, +isolated mode is a means for applications to tell a driver that ingress on +the underlying port must be injected from the defined flow rules only; that +no default traffic is expected outside those rules. + +This has the following benefits: + +- Applications get finer-grained control over the kind of traffic they want + to receive (no traffic by default). + +- More importantly they control at what point nonspecific traffic is handled + relative to other flow rules, by adjusting priority levels. + +- Drivers can assign more hardware resources to flow rules and expand the + set of supported rule types. + +Because toggling isolated mode may cause profound changes to the ingress +processing path of a driver, it may not be possible to leave it once +entered. Likewise, existing flow rules or global configuration settings may +prevent a driver from entering isolated mode. + +Applications relying on this mode are therefore encouraged to toggle it as +soon as possible after device initialization, ideally before the first call +to ``rte_eth_dev_configure()`` to avoid possible failures due to conflicting +settings. + +Once effective, the following functionality has no effect on the underlying +port and may return errors such as ``ENOTSUP`` ("not supported"): + +- Toggling promiscuous mode. +- Toggling allmulticast mode. +- Configuring MAC addresses. +- Configuring multicast addresses. +- Configuring VLAN filters. +- Configuring Rx filters through the legacy API (e.g. FDIR). +- Configuring global RSS settings. + +.. code-block:: c + + int + rte_flow_isolate(uint8_t port_id, int set, struct rte_flow_error *error); + +Arguments: + +- ``port_id``: port identifier of Ethernet device. +- ``set``: nonzero to enter isolated mode, attempt to leave it otherwise. +- ``error``: perform verbose error reporting if not NULL. PMDs initialize + this structure in case of error only. + +Return values: + +- 0 on success, a negative errno value otherwise and ``rte_errno`` is set. + Verbose error reporting ----------------------- @@ -2026,8 +2142,8 @@ A few features are intentionally not supported: - "MAC VLAN" or "tunnel" perfect matching modes should be automatically set according to the created flow rules. -- Signature mode of operation is not defined but could be handled through a - specific item type if needed. +- Signature mode of operation is not defined but could be handled through + "FUZZY" item. .. _table_rte_flow_migration_fdir: @@ -2054,8 +2170,8 @@ A few features are intentionally not supported: | | +----------+-----+ | | | | ``mask`` | any | | +---+-------------------+----------+-----+ | - | 3 | VF, PF (optional) | ``spec`` | any | | - | | +----------+-----+ | + | 3 | VF, PF, FUZZY | ``spec`` | any | | + | | (optional) +----------+-----+ | | | | ``last`` | N/A | | | | +----------+-----+ | | | | ``mask`` | any | | diff --git a/doc/guides/prog_guide/service_cores.rst b/doc/guides/prog_guide/service_cores.rst new file mode 100644 index 00000000..3a029ba9 --- /dev/null +++ b/doc/guides/prog_guide/service_cores.rst @@ -0,0 +1,81 @@ +.. BSD LICENSE + Copyright(c) 2017 Intel Corporation. All rights reserved. + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of Intel Corporation nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +Service Cores +============= + +DPDK has a concept known as service cores, which enables a dynamic way of +performing work on DPDK lcores. Service core support is built into the EAL, and +an API is provided to optionally allow applications to control how the service +cores are used at runtime. + +The service cores concept is built up out of services (components of DPDK that +require CPU cycles to operate) and service cores (DPDK lcores, tasked with +running services). The power of the service core concept is that the mapping +between service cores and services can be configured to abstract away the +difference between platforms and environments. + +For example, the Eventdev has hardware and software PMDs. Of these the software +PMD requires an lcore to perform the scheduling operations, while the hardware +PMD does not. With service cores, the application would not directly notice +that the scheduling is done in software. + +For detailed information about the service core API, please refer to the docs. + +Service Core Initialization +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +There are two methods to having service cores in a DPDK application, either by +using the service coremask, or by dynamically adding cores using the API. +The simpler of the two is to pass the `-s` coremask argument to EAL, which will +take any cores available in the main DPDK coremask, an if the bits are also set +in the service coremask the cores become service-cores instead of DPDK +application lcores. + +Enabling Services on Cores +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Each registered service can be individually mapped to a service core, or set of +service cores. Enabling a service on a particular core means that the lcore in +question will run the service. Disabling that core on the service stops the +lcore in question from running the service. + +Using this method, it is possible to assign specific workloads to each +service core, and map N workloads to M number of service cores. Each service +lcore loops over the services that are enabled for that core, and invokes the +function to run the service. + +Service Core Statistics +~~~~~~~~~~~~~~~~~~~~~~~ + +The service core library is capable of collecting runtime statistics like number +of calls to a specific service, and number of cycles used by the service. The +cycle count collection is dynamically configurable, allowing any application to +profile the services running on the system at any time. diff --git a/doc/guides/prog_guide/traffic_management.rst b/doc/guides/prog_guide/traffic_management.rst new file mode 100644 index 00000000..c0dc235e --- /dev/null +++ b/doc/guides/prog_guide/traffic_management.rst @@ -0,0 +1,251 @@ +.. BSD LICENSE + Copyright(c) 2017 Intel Corporation. All rights reserved. + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of Intel Corporation nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + +Traffic Management API +====================== + + +Overview +-------- + +This is the generic API for the Quality of Service (QoS) Traffic Management of +Ethernet devices, which includes the following main features: hierarchical +scheduling, traffic shaping, congestion management, packet marking. This API +is agnostic of the underlying HW, SW or mixed HW-SW implementation. + +Main features: + +* Part of DPDK rte_ethdev API +* Capability query API per port, per hierarchy level and per hierarchy node +* Scheduling algorithms: Strict Priority (SP), Weighed Fair Queuing (WFQ) +* Traffic shaping: single/dual rate, private (per node) and + shared (by multiple nodes) shapers +* Congestion management for hierarchy leaf nodes: algorithms of tail drop, head + drop, WRED, private (per node) and shared (by multiple nodes) WRED contexts +* Packet marking: IEEE 802.1q (VLAN DEI), IETF RFC 3168 (IPv4/IPv6 ECN for TCP + and SCTP), IETF RFC 2597 (IPv4 / IPv6 DSCP) + + +Capability API +-------------- + +The aim of these APIs is to advertise the capability information (i.e critical +parameter values) that the TM implementation (HW/SW) is able to support for the +application. The APIs supports the information disclosure at the TM level, at +any hierarchical level of the TM and at any node level of the specific +hierarchical level. Such information helps towards rapid understanding of +whether a specific implementation does meet the needs to the user application. + +At the TM level, users can get high level idea with the help of various +parameters such as maximum number of nodes, maximum number of hierarchical +levels, maximum number of shapers, maximum number of private shapers, type of +scheduling algorithm (Strict Priority, Weighted Fair Queueing , etc.), etc., +supported by the implementation. + +Likewise, users can query the capability of the TM at the hierarchical level to +have more granular knowledge about the specific level. The various parameters +such as maximum number of nodes at the level, maximum number of leaf/non-leaf +nodes at the level, type of the shaper(dual rate, single rate) supported at +the level if node is non-leaf type etc., are exposed as a result of +hierarchical level capability query. + +Finally, the node level capability API offers knowledge about the capability +supported by the node at any specific level. The information whether the +support is available for private shaper, dual rate shaper, maximum and minimum +shaper rate, etc. is exposed by node level capability API. + + +Scheduling Algorithms +--------------------- + +The fundamental scheduling algorithms that are supported are Strict Priority +(SP) and Weighted Fair Queuing (WFQ). The SP and WFQ algorithms are supported +at the level of each node of the scheduling hierarchy, regardless of the node +level/position in the tree. The SP algorithm is used to schedule between +sibling nodes with different priority, while WFQ is used to schedule between +groups of siblings that have the same priority. + +Algorithms such as Weighed Round Robin (WRR), byte-level WRR, Deficit WRR +(DWRR), etc are considered approximations of the ideal WFQ and are therefore +assimilated to WFQ, although an associated implementation-dependent accuracy, +performance and resource usage trade-off might exist. + + +Traffic Shaping +--------------- + +The TM API provides support for single rate and dual rate shapers (rate +limiters) for the hierarchy nodes, subject to the specific implementation +support being available. + +Each hierarchy node has zero or one private shaper (only one node using it) +and/or zero, one or several shared shapers (multiple nodes use the same shaper +instance). A private shaper is used to perform traffic shaping for a single +node, while a shared shaper is used to perform traffic shaping for a group of +nodes. + +The configuration of private and shared shapers is done through the definition +of shaper profiles. Any shaper profile (single rate or dual rate shaper) can be +used by one or several shaper instances (either private or shared). + +Single rate shapers use a single token bucket. Therefore, single rate shaper is +configured by setting the rate of the committed bucket to zero, which +effectively disables this bucket. The peak bucket is used to limit the rate +and the burst size for the single rate shaper. Dual rate shapers use both the +committed and the peak token buckets. The rate of the peak bucket has to be +bigger than zero, as well as greater than or equal to the rate of the committed +bucket. + + +Congestion Management +--------------------- + +Congestion management is used to control the admission of packets into a packet +queue or group of packet queues on congestion. The congestion management +algorithms that are supported are: Tail Drop, Head Drop and Weighted Random +Early Detection (WRED). They are made available for every leaf node in the +hierarchy, subject to the specific implementation supporting them. +On request of writing a new packet into the current queue while the queue is +full, the Tail Drop algorithm drops the new packet while leaving the queue +unmodified, as opposed to the Head Drop* algorithm, which drops the packet +at the head of the queue (the oldest packet waiting in the queue) and admits +the new packet at the tail of the queue. + +The Random Early Detection (RED) algorithm works by proactively dropping more +and more input packets as the queue occupancy builds up. When the queue is full +or almost full, RED effectively works as Tail Drop. The Weighted RED (WRED) +algorithm uses a separate set of RED thresholds for each packet color and uses +separate set of RED thresholds for each packet color. + +Each hierarchy leaf node with WRED enabled as its congestion management mode +has zero or one private WRED context (only one leaf node using it) and/or zero, +one or several shared WRED contexts (multiple leaf nodes use the same WRED +context). A private WRED context is used to perform congestion management for +a single leaf node, while a shared WRED context is used to perform congestion +management for a group of leaf nodes. + +The configuration of WRED private and shared contexts is done through the +definition of WRED profiles. Any WRED profile can be used by one or several +WRED contexts (either private or shared). + + +Packet Marking +-------------- +The TM APIs have been provided to support various types of packet marking such +as VLAN DEI packet marking (IEEE 802.1Q), IPv4/IPv6 ECN marking of TCP and SCTP +packets (IETF RFC 3168) and IPv4/IPv6 DSCP packet marking (IETF RFC 2597). +All VLAN frames of a given color get their DEI bit set if marking is enabled +for this color. In case, when marking for a given color is not enabled, the +DEI bit is left as is (either set or not). + +All IPv4/IPv6 packets of a given color with ECN set to 2’b01 or 2’b10 carrying +TCP or SCTP have their ECN set to 2’b11 if the marking feature is enabled for +the current color, otherwise the ECN field is left as is. + +All IPv4/IPv6 packets have their color marked into DSCP bits 3 and 4 as +follows: green mapped to Low Drop Precedence (2’b01), yellow to Medium (2’b10) +and red to High (2’b11). Marking needs to be explicitly enabled for each color; +when not enabled for a given color, the DSCP field of all packets with that +color is left as is. + + +Steps to Setup the Hierarchy +---------------------------- + +The TM hierarchical tree consists of leaf nodes and non-leaf nodes. Each leaf +node sits on top of a scheduling queue of the current Ethernet port. Therefore, +the leaf nodes have predefined IDs in the range of 0... (N-1), where N is the +number of scheduling queues of the current Ethernet port. The non-leaf nodes +have their IDs generated by the application outside of the above range, which +is reserved for leaf nodes. + +Each non-leaf node has multiple inputs (its children nodes) and single output +(which is input to its parent node). It arbitrates its inputs using Strict +Priority (SP) and Weighted Fair Queuing (WFQ) algorithms to schedule input +packets to its output while observing its shaping (rate limiting) constraints. + +The children nodes with different priorities are scheduled using the SP +algorithm based on their priority, with 0 as the highest priority. Children +with the same priority are scheduled using the WFQ algorithm according to their +weights. The WFQ weight of a given child node is relative to the sum of the +weights of all its sibling nodes that have the same priority, with 1 as the +lowest weight. For each SP priority, the WFQ weight mode can be set as either +byte-based or packet-based. + + +Initial Hierarchy Specification +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The hierarchy is specified by incrementally adding nodes to build up the +scheduling tree. The first node that is added to the hierarchy becomes the root +node and all the nodes that are subsequently added have to be added as +descendants of the root node. The parent of the root node has to be specified +as RTE_TM_NODE_ID_NULL and there can only be one node with this parent ID +(i.e. the root node). The unique ID that is assigned to each node when the node +is created is further used to update the node configuration or to connect +children nodes to it. + +During this phase, some limited checks on the hierarchy specification can be +conducted, usually limited in scope to the current node, its parent node and +its sibling nodes. At this time, since the hierarchy is not fully defined, +there is typically no real action performed by the underlying implementation. + + +Hierarchy Commit +~~~~~~~~~~~~~~~~ + +The hierarchy commit API is called during the port initialization phase (before +the Ethernet port is started) to freeze the start-up hierarchy. This function +typically performs the following steps: + +* It validates the start-up hierarchy that was previously defined for the + current port through successive node add API invocations. +* Assuming successful validation, it performs all the necessary implementation + specific operations to install the specified hierarchy on the current port, + with immediate effect once the port is started. + +This function fails when the currently configured hierarchy is not supported by +the Ethernet port, in which case the user can abort or try out another +hierarchy configuration (e.g. a hierarchy with less leaf nodes), which can be +built from scratch or by modifying the existing hierarchy configuration. Note +that this function can still fail due to other causes (e.g. not enough memory +available in the system, etc.), even though the specified hierarchy is +supported in principle by the current port. + + +Run-Time Hierarchy Updates +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The TM API provides support for on-the-fly changes to the scheduling hierarchy, +thus operations such as node add/delete, node suspend/resume, parent node +update, etc., can be invoked after the Ethernet port has been started, subject +to the specific implementation supporting them. The set of dynamic updates +supported by the implementation is advertised through the port capability set. |
