diff options
Diffstat (limited to 'src/vnet')
| -rw-r--r-- | src/vnet/interface/tx_queue.rst | 159 |
1 files changed, 159 insertions, 0 deletions
diff --git a/src/vnet/interface/tx_queue.rst b/src/vnet/interface/tx_queue.rst new file mode 100644 index 00000000000..e8f0e039b8e --- /dev/null +++ b/src/vnet/interface/tx_queue.rst @@ -0,0 +1,159 @@ +.. _TX_Queue_doc: + +Transmit Queues +=============== + +Overview +________ + +VPP implements Transmit queues infra to access and manage them. It provides +common registration functions to register or unregister interfaces’ transmit +queues. It also provides functions for queues placement on given thread(s). + +The TXQ Infrastructure +_______________________ + +Infra registers each queue using a unique key which is formed by concatenating +the hardware interface index ``hw_if_index`` and unique queue identifier for +given interface ``queue_id``. As a result of registration of queue, infra +returns back a unique global ``queue_index`` which can be used by driver to +access that queue later. + +Interface output node uses pre-computed ``output_node_thread_runtime`` data +which provides essential information related to queue placements on given +thread of given interface. Transmit queue infra implements an algorithm to +pre-compute this information. It also pre-computes scalar arguments of frame +``vnet_hw_if_tx_frame_t``. It also pre-calculates a ``lookup_table`` for +thread if there are multiple transmit queues are placed on that thread. +Interface drivers call ``vnet_hw_if_update_runtime_data()`` to execute that +algorithm after registering the transmit queues to TXQ infra. + +The algorithm makes the copy of existing runtime data and iterate through them +for each vpp main and worker thread. In each iteration, algorithm loop through +all the tx queues of given interface to fill the information in the frame data +structure ``vnet_hw_if_tx_frame_t``. Algorithm also updates the information +related to number of transmit queues of given interface on given vpp thread in +data structure ``output_node_thread_runtime``. As a consequence of any update +to the copy, triggers the function to update the actual working copy by taking +the worker barrier and free the old copy of ``output_node_thread_runtime``. + +Multi-TXQ infra +^^^^^^^^^^^^^^^ + +Interface output node uses packet flow hash using hash infra in case of multi-txq +on given thread. Each hardware interface class contains type of the hash required +for interfaces from that hardware interface class i.e. ethernet interface hardware +class contains type ``VNET_HASH_FN_TYPE_ETHERNET``. Though, the hash function +itself is contained by hardware interface data structure of given interface. Default +hashing function is selected upon interface creation based on priority. User can +configure a different hash to an interface for multi-txq use case. + +Interface output node uses packet flow hash as an index to the pre-calculated lookup +table to get the queue identifier for given transmit queue. Interface output node +enqueues the packets to respective frame and also copies the ``vnet_hw_if_tx_frame_t`` +to frame scalar arguments. Drivers use scalar arguments ``vnet_hw_if_tx_frame_t`` +of the given frame to extract the information about the transmit queue to be used to +transmit the packets. Drivers may need to acquire a lock on given queue before +transmitting the packets based on the ``shared_queue`` bit status. + +Data structures +^^^^^^^^^^^^^^^ + +Queue information is stored in data structure ``vnet_hw_if_tx_queue_t``: + +.. code:: c + + typedef struct + { + /* either this queue is shared among multiple threads */ + u8 shared_queue : 1; + /* hw interface index */ + u32 hw_if_index; + + /* hardware queue identifier */ + u32 queue_id; + + /* bitmap of threads which use this queue */ + clib_bitmap_t *threads; + } vnet_hw_if_tx_queue_t; + + +Frame information is stored in data structure: ``vnet_hw_if_tx_frame_t``: + +.. code:: c + + typedef enum + { + VNET_HW_IF_TX_FRAME_HINT_NOT_CHAINED = (1 << 0), + VNET_HW_IF_TX_FRAME_HINT_NO_GSO = (1 << 1), + VNET_HW_IF_TX_FRAME_HINT_NO_CKSUM_OFFLOAD = (1 << 2), + } vnet_hw_if_tx_frame_hint_t; + + typedef struct + { + u8 shared_queue : 1; + vnet_hw_if_tx_frame_hint_t hints : 16; + u32 queue_id; + } vnet_hw_if_tx_frame_t; + +Output node runtime information is stored in data structure: ``output_node_thread_runtime``: + +.. code:: c + + typedef struct + { + CLIB_CACHE_LINE_ALIGN_MARK (cacheline0); + vnet_hw_if_tx_frame_t *frame; + u32 *lookup_table; + u32 n_queues; + } vnet_hw_if_output_node_runtime_t; + + +MultiTXQ API +^^^^^^^^^^^^ + +This API message is used to place tx queue of an interface to vpp main or worker(s) thread(s). + +.. code:: c + + autoendian autoreply define sw_interface_set_tx_placement + { + u32 client_index; + u32 context; + vl_api_interface_index_t sw_if_index; + u32 queue_id; + u32 array_size; + u32 threads[array_size]; + option vat_help = "<interface | sw_if_index <index>> queue <n> [threads <list> | mask <hex>]"; + }; + +Multi-TXQ CLI +^^^^^^^^^^^^^ + +:: + + set interface tx-queue set interface tx-queue <interface> queue <n> [threads <list>] + set interface tx-hash set interface tx-hash <interface> hash-name <hash-name> + +:: + + show hardware-interfaces + + Name Idx Link Hardware + tap0 1 up tap0 + Link speed: unknown + RX Queues: + queue thread mode + 0 main (0) polling + TX Queues: + TX Hash: [name: crc32c-5tuple priority: 50 description: IPv4/IPv6 header and TCP/UDP ports] + queue shared thread(s) + 0 no 0 + Ethernet address 02:fe:27:69:5a:b5 + VIRTIO interface + instance 0 + RX QUEUE : Total Packets + 0 : 0 + TX QUEUE : Total Packets + 0 : 0 + |