diff options
| author |   Luca Boccassi <luca.boccassi@gmail.com> | 2018-08-14 18:52:30 +0100 |
|---|---|---|
| committer | Luca Boccassi <luca.boccassi@gmail.com> | 2018-08-14 18:53:17 +0100 |
| commit | b63264c8342e6a1b6971c79550d2af2024b6a4de (patch) | |
| tree | 83114aac64286fe616506c0b3dfaec2ab86ef835 /doc/guides/sample_app_ug/ip_pipeline.rst | |
| parent | ca33590b6af032bff57d9cc70455660466a654b2 (diff) | |
New upstream version 18.08upstream/18.08
Change-Id: I32fdf5e5016556d9c0a6d88ddaf1fc468961790a
Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>
Diffstat (limited to 'doc/guides/sample_app_ug/ip_pipeline.rst')
| -rw-r--r-- | doc/guides/sample_app_ug/ip_pipeline.rst | 1448 |
1 files changed, 404 insertions, 1044 deletions
diff --git a/doc/guides/sample_app_ug/ip_pipeline.rst b/doc/guides/sample_app_ug/ip_pipeline.rst index 69af6b8d..b75509a0 100644 --- a/doc/guides/sample_app_ug/ip_pipeline.rst +++ b/doc/guides/sample_app_ug/ip_pipeline.rst @@ -1,5 +1,5 @@ .. SPDX-License-Identifier: BSD-3-Clause - Copyright(c) 2015-2016 Intel Corporation. + Copyright(c) 2015-2018 Intel Corporation. Internet Protocol (IP) Pipeline Application =========================================== @@ -8,1163 +8,523 @@ Application overview -------------------- The *Internet Protocol (IP) Pipeline* application is intended to be a vehicle for rapid development of packet processing -applications running on multi-core CPUs. +applications on multi-core CPUs. -The application provides a library of reusable functional blocks called pipelines. -These pipelines can be seen as prefabricated blocks that can be instantiated and inter-connected through packet queues +Following OpenFlow and P4 design principles, the application can be used to create functional blocks called pipelines out +of input/output ports, tables and actions in a modular way. Multiple pipelines can be inter-connected through packet queues to create complete applications (super-pipelines). -Pipelines are created and inter-connected through the application configuration file. -By using different configuration files, different applications are effectively created, therefore this application -can be seen as an application generator. -The configuration of each pipeline can be updated at run-time through the application Command Line Interface (CLI). +The pipelines are mapped to application threads, with each pipeline executed by a single thread and each thread able to run +one or several pipelines. The possibilities of creating pipelines out of ports, tables and actions, connecting multiple +pipelines together and mapping the pipelines to execution threads are endless, therefore this application can be seen as +a true application generator. -Main application components are: +Pipelines are created and managed through Command Line Interface (CLI): -**A Library of reusable pipelines** + * Any standard TCP client (e.g. telnet, netcat, custom script, etc) is typically able to connect to the application, send + commands through the network and wait for the response before pushing the next command. - * Each pipeline represents a functional block, e.g. flow classification, firewall, routing, master, etc. - - * Each pipeline type can be instantiated several times in the same application, which each instance configured - separately and mapped to a single CPU core. - Each CPU core can run one or several pipeline instances, which can be of same or different type. - - * Pipeline instances are inter-connected through packet queues (for packet processing) and message queues - (for run-time configuration). - - * Pipelines are implemented using DPDK Packet Framework. - - * More pipeline types can always be built and added to the existing pipeline types. - -**The Configuration file** - - * The configuration file defines the application structure. - By using different configuration files, different applications are created. - - * All the application resources are created and configured through the application configuration file: - pipeline instances, buffer pools, links (i.e. network interfaces), hardware device RX/TX queues, - software queues, traffic manager devices, EAL startup arguments, etc. - - * The configuration file syntax is “define by reference”, meaning that resources are defined as they are referenced. - First time a resource name is detected, it is registered with default parameters. - Optionally, the resource parameters can be further refined through a configuration file section dedicated to - that resource. - - * Command Line Interface (CLI) - -**Global CLI commands: link configuration, etc.** - - * Common pipeline CLI commands: ping (keep-alive), statistics, etc. - - * Pipeline type specific CLI commands: used to configure instances of specific pipeline type. - These commands are registered with the application when the pipeline type is registered. - For example, the commands for routing pipeline instances include: route add, route delete, route list, etc. - - * CLI commands can be grouped into scripts that can be invoked at initialization and at runtime. - - -Design goals ------------- - - -Rapid development -~~~~~~~~~~~~~~~~~ - -This application enables rapid development through quick connectivity of standard components called pipelines. -These components are built using DPDK Packet Framework and encapsulate packet processing features at different levels: -ports, tables, actions, pipelines and complete applications. - -Pipeline instances are instantiated, configured and inter-connected through low complexity configuration files loaded -during application initialization. -Each pipeline instance is mapped to a single CPU core, with each CPU core able to run one or multiple pipeline -instances of same or different types. By loading a different configuration file, a different application is -effectively started. - - -Flexibility -~~~~~~~~~~~ - -Each packet processing application is typically represented as a chain of functional stages which is often called -the functional pipeline of the application. -These stages are mapped to CPU cores to create chains of CPU cores (pipeline model), clusters of CPU cores -(run-to-completion model) or chains of clusters of CPU cores (hybrid model). - -This application allows all the above programming models. -By applying changes to the configuration file, the application provides the flexibility to reshuffle its -building blocks in different ways until the configuration providing the best performance is identified. - - -Move pipelines around -^^^^^^^^^^^^^^^^^^^^^ - -The mapping of pipeline instances to CPU cores can be reshuffled through the configuration file. -One or several pipeline instances can be mapped to the same CPU core. - -.. _figure_ip_pipelines_1: - -.. figure:: img/ip_pipelines_1.* - - Example of moving pipeline instances across different CPU cores - - -Move tables around -^^^^^^^^^^^^^^^^^^ - -There is some degree of flexibility for moving tables from one pipeline instance to another. -Based on the configuration arguments passed to each pipeline instance in the configuration file, specific tables -can be enabled or disabled. -This way, a specific table can be “moved” from pipeline instance A to pipeline instance B by simply disabling its -associated functionality for pipeline instance A while enabling it for pipeline instance B. - -Due to requirement to have simple syntax for the configuration file, moving tables across different pipeline -instances is not as flexible as the mapping of pipeline instances to CPU cores, or mapping actions to pipeline tables. -Complete flexibility in moving tables from one pipeline to another could be achieved through a complex pipeline -description language that would detail the structural elements of the pipeline (ports, tables and actions) and -their connectivity, resulting in complex syntax for the configuration file, which is not acceptable. -Good configuration file readability through simple syntax is preferred. - -*Example*: the IP routing pipeline can run the routing function only (with ARP function run by a different -pipeline instance), or it can run both the routing and ARP functions as part of the same pipeline instance. - - -.. _figure_ip_pipelines_2: - -.. figure:: img/ip_pipelines_2.* - - Example of moving tables across different pipeline instances - - -Move actions around -^^^^^^^^^^^^^^^^^^^ - -When it makes sense, packet processing actions can be moved from one pipeline instance to another. -Based on the configuration arguments passed to each pipeline instance in the configuration file, specific actions -can be enabled or disabled. -This way, a specific action can be "moved" from pipeline instance A to pipeline instance B by simply disabling its -associated functionality for pipeline instance A while enabling it for pipeline instance B. - -*Example*: The flow actions of accounting, traffic metering, application identification, NAT, etc can be run as part -of the flow classification pipeline instance or split across several flow actions pipeline instances, depending on -the number of flow instances and their compute requirements. - - -.. _figure_ip_pipelines_3: - -.. figure:: img/ip_pipelines_3.* - - Example of moving actions across different tables and pipeline instances - - -Performance -~~~~~~~~~~~ - -Performance of the application is the highest priority requirement. -Flexibility is not provided at the expense of performance. - -The purpose of flexibility is to provide an incremental development methodology that allows monitoring the -performance evolution: - -* Apply incremental changes in the configuration (e.g. mapping on pipeline instances to CPU cores) - in order to identify the configuration providing the best performance for a given application; - -* Add more processing incrementally (e.g. by enabling more actions for specific pipeline instances) until - the application is feature complete while checking the performance impact at each step. - - -Debug capabilities -~~~~~~~~~~~~~~~~~~ - -The application provides a significant set of debug capabilities: - -* Command Line Interface (CLI) support for statistics polling: pipeline instance ping (keep-alive checks), - pipeline instance statistics per input port/output port/table, link statistics, etc; - -* Logging: Turn on/off application log messages based on priority level; + * All the application objects are created and managed through CLI commands: + * 'Primitive' objects used to create pipeline ports: memory pools, links (i.e. network interfaces), SW queues, traffic managers, etc. + * Action profiles: used to define the actions to be executed by pipeline input/output ports and tables. + * Pipeline components: input/output ports, tables, pipelines, mapping of pipelines to execution threads. Running the application ----------------------- The application startup command line is:: - ip_pipeline [-f CONFIG_FILE] [-s SCRIPT_FILE] -p PORT_MASK [-l LOG_LEVEL] + ip_pipeline [EAL_ARGS] -- [-s SCRIPT_FILE] [-h HOST] [-p PORT] The application startup arguments are: -``-f CONFIG_FILE`` - - * Optional: Yes - - * Default: ``./config/ip_pipeline.cfg`` - - * Argument: Path to the configuration file to be loaded by the application. - Please refer to the :ref:`ip_pipeline_configuration_file` for details on how to write the configuration file. - ``-s SCRIPT_FILE`` * Optional: Yes * Default: Not present - * Argument: Path to the CLI script file to be run by the master pipeline at application startup. - No CLI script file will be run at startup if this argument is not present. - -``-p PORT_MASK`` - - * Optional: No + * Argument: Path to the CLI script file to be run at application startup. + No CLI script file will run at startup if this argument is not present. - * Default: N/A - - * Argument: Hexadecimal mask of NIC port IDs to be used by the application. - First port enabled in this mask will be referenced as LINK0 as part of the application configuration file, - next port as LINK1, etc. - -``-l LOG_LEVEL`` +``-h HOST`` * Optional: Yes - * Default: 1 (High priority) - - * Argument: Log level to determine which application messages are to be printed to standard output. - Available log levels are: 0 (None), 1 (High priority), 2 (Low priority). - Only application messages whose priority is higher than or equal to the application log level will be printed. - - -Application stages ------------------- - - -Configuration -~~~~~~~~~~~~~ - -During this stage, the application configuration file is parsed and its content is loaded into the application data -structures. -In case of any configuration file parse error, an error message is displayed and the application is terminated. -Please refer to the :ref:`ip_pipeline_configuration_file` for a description of the application configuration file format. - - -Configuration checking -~~~~~~~~~~~~~~~~~~~~~~ - -In the absence of any parse errors, the loaded content of application data structures is checked for overall consistency. -In case of any configuration check error, an error message is displayed and the application is terminated. - - -Initialization -~~~~~~~~~~~~~~ - -During this stage, the application resources are initialized and the handles to access them are saved into the -application data structures. -In case of any initialization error, an error message is displayed and the application is terminated. - -The typical resources to be initialized are: pipeline instances, buffer pools, links (i.e. network interfaces), -hardware device RX/TX queues, software queues, traffic management devices, etc. - - -.. _ip_pipeline_runtime: + * Default: ``0.0.0.0`` -Run-time -~~~~~~~~ + * Argument: IP Address of the host running ip pipeline application to be used by + remote TCP based client (telnet, netcat, etc.) for connection. -Each CPU core runs the pipeline instances assigned to it in time sharing mode and in round robin order: +``-p PORT`` -1. *Packet processing task*: The pipeline run-time code is typically a packet *processing* task built on top of - DPDK Packet Framework rte_pipeline library, which reads bursts of packets from the pipeline input ports, - performs table lookups and executes the identified actions for all tables in the pipeline, with packet - eventually written to pipeline output ports or dropped. + * Optional: Yes -2. *Message handling task*: Each CPU core will also periodically execute the *message handling* code of each - of the pipelines mapped to it. - The pipeline message handling code is processing the messages that are pending in the pipeline input message - queues, which are typically sent by the master CPU core for the on-the-fly pipeline configuration: check - that pipeline is still alive (ping), add/delete entries in the pipeline tables, get statistics, etc. - The frequency of executing the message handling code is usually much smaller than the frequency of executing - the packet processing work. + * Default: ``8086`` -Please refer to the :ref:`ip_pipeline_pipeline_section` for more details about the application pipeline module encapsulation. + * Argument: TCP port number at which the ip pipeline is running. + This port number should be used by remote TCP client (such as telnet, netcat, etc.) to connect to host application. -.. _ip_pipeline_configuration_file: +Refer to *DPDK Getting Started Guide* for general information on running applications and the Environment Abstraction Layer (EAL) options. -Configuration file syntax -------------------------- +The following is an example command to run ip pipeline application configured for layer 2 forwarding: -Syntax overview -~~~~~~~~~~~~~~~ +.. code-block:: console -The syntax of the configuration file is designed to be simple, which favors readability. -The configuration file is parsed using the DPDK library librte_cfgfile, which supports simple -`INI file format <http://en.wikipedia.org/wiki/INI_file>`__ for configuration files. + $ ./build/ip_pipeline -c 0x3 -- -s examples/route_ecmp.cli -As result, the configuration file is split into several sections, with each section containing one or more entries. -The scope of each entry is its section, and each entry specifies a variable that is assigned a specific value. -Any text after the ``;`` character is considered a comment and is therefore ignored. +The application should start successfully and display as follows: -The following are application specific: number of sections, name of each section, number of entries of each section, -name of the variables used for each section entry, the value format (e.g. signed/unsigned integer, string, etc) -and range of each section entry variable. +.. code-block:: console -Generic example of configuration file section: + EAL: Detected 40 lcore(s) + EAL: Detected 2 NUMA nodes + EAL: Multi-process socket /var/run/.rte_unix + EAL: Probing VFIO support... + EAL: PCI device 0000:02:00.0 on NUMA socket 0 + EAL: probe driver: 8086:10fb net_ixgbe + ... -.. code-block:: ini +To run remote client (e.g. telnet) to communicate with the ip pipeline application: - [<section_name>] +.. code-block:: console - <variable_name_1> = <value_1> + $ telnet 127.0.0.1 8086 - ; ... +When running a telnet client as above, command prompt is displayed: - <variable_name_N> = <value_N> +.. code-block:: console + Trying 127.0.0.1... + Connected to 127.0.0.1. + Escape character is '^]'. -Application resources present in the configuration file -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + Welcome to IP Pipeline! -.. _table_ip_pipelines_resource_name: + pipeline> -.. tabularcolumns:: |p{4cm}|p{6cm}|p{6cm}| +Once application and telnet client start running, messages can be sent from client to application. +At any stage, telnet client can be terminated using the quit command. -.. table:: Application resource names in the configuration file - +----------------------------+-----------------------------+-------------------------------------------------+ - | Resource type | Format | Examples | - +============================+=============================+=================================================+ - | Pipeline | ``PIPELINE<ID>`` | ``PIPELINE0``, ``PIPELINE1`` | - +----------------------------+-----------------------------+-------------------------------------------------+ - | Mempool | ``MEMPOOL<ID>`` | ``MEMPOOL0``, ``MEMPOOL1`` | - +----------------------------+-----------------------------+-------------------------------------------------+ - | Link (network interface) | ``LINK<ID>`` | ``LINK0``, ``LINK1`` | - +----------------------------+-----------------------------+-------------------------------------------------+ - | Link RX queue | ``RXQ<LINK_ID>.<QUEUE_ID>`` | ``RXQ0.0``, ``RXQ1.5`` | - +----------------------------+-----------------------------+-------------------------------------------------+ - | Link TX queue | ``TXQ<LINK_ID>.<QUEUE_ID>`` | ``TXQ0.0``, ``TXQ1.5`` | - +----------------------------+-----------------------------+-------------------------------------------------+ - | Software queue | ``SWQ<ID>`` | ``SWQ0``, ``SWQ1`` | - +----------------------------+-----------------------------+-------------------------------------------------+ - | Traffic Manager | ``TM<LINK_ID>`` | ``TM0``, ``TM1`` | - +----------------------------+-----------------------------+-------------------------------------------------+ - | KNI (kernel NIC interface) | ``KNI<LINK_ID>`` | ``KNI0``, ``KNI1`` | - +----------------------------+-----------------------------+-------------------------------------------------+ - | Source | ``SOURCE<ID>`` | ``SOURCE0``, ``SOURCE1`` | - +----------------------------+-----------------------------+-------------------------------------------------+ - | Sink | ``SINK<ID>`` | ``SINK0``, ``SINK1`` | - +----------------------------+-----------------------------+-------------------------------------------------+ - | Message queue | ``MSGQ<ID>`` | ``MSGQ0``, ``MSGQ1``, | - | | ``MSGQ-REQ-PIPELINE<ID>`` | ``MSGQ-REQ-PIPELINE2``, ``MSGQ-RSP-PIPELINE2,`` | - | | ``MSGQ-RSP-PIPELINE<ID>`` | ``MSGQ-REQ-CORE-s0c1``, ``MSGQ-RSP-CORE-s0c1`` | - | | ``MSGQ-REQ-CORE-<CORE_ID>`` | | - | | ``MSGQ-RSP-CORE-<CORE_ID>`` | | - +----------------------------+-----------------------------+-------------------------------------------------+ - -``LINK`` instances are created implicitly based on the ``PORT_MASK`` application startup argument. -``LINK0`` is the first port enabled in the ``PORT_MASK``, port 1 is the next one, etc. -The LINK ID is different than the DPDK PMD-level NIC port ID, which is the actual position in the bitmask mentioned above. -For example, if bit 5 is the first bit set in the bitmask, then ``LINK0`` is having the PMD ID of 5. -This mechanism creates a contiguous LINK ID space and isolates the configuration file against changes in the board -PCIe slots where NICs are plugged in. - -``RXQ``, ``TXQ``, ``TM`` and ``KNI`` instances have the LINK ID as part of their name. -For example, ``RXQ2.1``, ``TXQ2.1`` and ``TM2`` are all associated with ``LINK2``. - - -Rules to parse the configuration file -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The main rules used to parse the configuration file are: - -1. Application resource name determines the type of resource based on the name prefix. - - *Example*: all software queues need to start with ``SWQ`` prefix, so ``SWQ0`` and ``SWQ5`` are valid software - queue names. - -2. An application resource is defined by creating a configuration file section with its name. - The configuration file section allows fine tuning on any of the resource parameters. - Some resource parameters are mandatory, in which case it is required to have them specified as part of the - section, while some others are optional, in which case they get assigned their default value when not present. - - *Example*: section ``SWQ0`` defines a software queue named SWQ0, whose parameters are detailed as part of this section. - -3. An application resource can also be defined by referencing it. - Referencing a resource takes place by simply using its name as part of the value assigned to a variable in any - configuration file section. - In this case, the resource is registered with all its parameters having their default values. - Optionally, a section with the resource name can be added to the configuration file to fine tune some or all - of the resource parameters. - - *Example*: in section ``PIPELINE3``, variable ``pktq_in`` includes ``SWQ5`` as part of its list, which results - in defining a software queue named ``SWQ5``; when there is no ``SWQ5`` section present in the configuration file, - ``SWQ5`` gets registered with default parameters. - - -.. _ip_pipeline_pipeline_section: - -PIPELINE section -~~~~~~~~~~~~~~~~ - -.. _table_ip_pipelines_pipeline_section_1: - -.. tabularcolumns:: |p{2.5cm}|p{7cm}|p{1.5cm}|p{1.5cm}|p{1.5cm}| - -.. table:: Configuration file PIPELINE section (1/2) - - +---------------+-----------------------------------------------------------+---------------+------------------------+----------------+ - | Section | Description | Optional | Range | Default value | - +===============+===========================================================+===============+========================+================+ - | type | Pipeline type. Defines the functionality to be | NO | See "List | N/A | - | | executed. | | of pipeline types" | | - +---------------+-----------------------------------------------------------+---------------+------------------------+----------------+ - | core | CPU core to run the current pipeline. | YES | See "CPU Core | CPU socket 0, | - | | | | notation" | core 0, | - | | | | | hyper-thread 0 | - +---------------+-----------------------------------------------------------+---------------+------------------------+----------------+ - | pktq_in | Packet queues to serve as input ports for the | YES | List of input | Empty list | - | | current pipeline instance. The acceptable packet | | packet queue IDs | | - | | queue types are: ``RXQ``, ``SWQ``, ``TM`` and ``SOURCE``. | | | | - | | First device in this list is used as pipeline input port | | | | - | | 0, second as pipeline input port 1, etc. | | | | - +---------------+-----------------------------------------------------------+---------------+------------------------+----------------+ - | pktq_out | Packet queues to serve as output ports for the | YES | List of output | Empty list | - | | current pipeline instance. The acceptable packet | | packet queue IDs. | | - | | queue types are: ``TXQ``, ``SWQ``, ``TM`` and ``SINK``. | | | | - | | First device in this list is used as pipeline output | | | | - | | port 0, second as pipeline output port 1, etc. | | | | - +---------------+-----------------------------------------------------------+---------------+------------------------+----------------+ - -.. _table_ip_pipelines_pipeline_section_2: - -.. tabularcolumns:: |p{2.5cm}|p{7cm}|p{1.5cm}|p{1.5cm}|p{1.5cm}| - -.. table:: Configuration file PIPELINE section (2/2) - - +---------------+-----------------------------------------------------------+---------------+------------------------+----------------+ - | Section | Description | Optional | Range | Default value | - +===============+===========================================================+===============+========================+================+ - | msgq_in | Input message queues. These queues contain | YES | List of message | Empty list | - | | request messages that need to be handled by the | | queue IDs | | - | | current pipeline instance. The type and format of | | | | - | | request messages is defined by the pipeline type. | | | | - | | For each pipeline instance, there is an input | | | | - | | message queue defined implicitly, whose name is: | | | | - | | ``MSGQ-REQ-<PIPELINE_ID>``. This message queue | | | | - | | should not be mentioned as part of msgq_in list. | | | | - +---------------+-----------------------------------------------------------+---------------+------------------------+----------------+ - | msgq_out | Output message queues. These queues are used by | YES | List of message | Empty list | - | | the current pipeline instance to write response | | queue IDs | | - | | messages as result of request messages being | | | | - | | handled. The type and format of response | | | | - | | messages is defined by the pipeline type. | | | | - | | For each pipeline instance, there is an output | | | | - | | message queue defined implicitly, whose name is: | | | | - | | ``MSGQ-RSP-<PIPELINE_ID>``. This message queue | | | | - | | should not be mentioned as part of msgq_out list. | | | | - +---------------+-----------------------------------------------------------+---------------+------------------------+----------------+ - | timer_period | Time period, measured in milliseconds, | YES | milliseconds | 1 ms | - | | for handling the input message queues. | | | | - +---------------+-----------------------------------------------------------+---------------+------------------------+----------------+ - | <any other> | Arguments to be passed to the current pipeline | Depends on | Depends on | Depends on | - | | instance. Format of the arguments, their type, | pipeline type | pipeline type | pipeline type | - | | whether each argument is optional or mandatory | | | | - | | and its default value (when optional) are defined | | | | - | | by the pipeline type. | | | | - | | The value of the arguments is applicable to the | | | | - | | current pipeline instance only. | | | | - +---------------+-----------------------------------------------------------+---------------+------------------------+----------------+ - - -CPU core notation -^^^^^^^^^^^^^^^^^ - -The CPU Core notation is:: - - <CPU core> ::= [s|S<CPU socket ID>][c|C]<CPU core ID>[h|H] - -For example:: - - CPU socket 0, core 0, hyper-thread 0: 0, c0, s0c0 - - CPU socket 0, core 0, hyper-thread 1: 0h, c0h, s0c0h - - CPU socket 3, core 9, hyper-thread 1: s3c9h - - -MEMPOOL section -~~~~~~~~~~~~~~~ +Application stages +------------------ -.. _table_ip_pipelines_mempool_section: - -.. tabularcolumns:: |p{2.5cm}|p{6cm}|p{1.5cm}|p{1.5cm}|p{3cm}| - -.. table:: Configuration file MEMPOOL section - - +---------------+-----------------------------------------------+----------+----------+---------------------------+ - | Section | Description | Optional | Type | Default value | - +===============+===============================================+==========+==========+===========================+ - | buffer_size | Buffer size (in bytes) for the current | YES | uint32_t | 2048 | - | | buffer pool. | | | + sizeof(struct rte_mbuf) | - | | | | | + HEADROOM | - +---------------+-----------------------------------------------+----------+----------+---------------------------+ - | pool_size | Number of buffers in the current buffer pool. | YES | uint32_t | 32K | - +---------------+-----------------------------------------------+----------+----------+---------------------------+ - | cache_size | Per CPU thread cache size (in number of | YES | uint32_t | 256 | - | | buffers) for the current buffer pool. | | | | - +---------------+-----------------------------------------------+----------+----------+---------------------------+ - | cpu | CPU socket ID where to allocate memory for | YES | uint32_t | 0 | - | | the current buffer pool. | | | | - +---------------+-----------------------------------------------+----------+----------+---------------------------+ - - -LINK section -~~~~~~~~~~~~ - -.. _table_ip_pipelines_link_section: - -.. tabularcolumns:: |p{3cm}|p{7cm}|p{1.5cm}|p{1.5cm}|p{2cm}| - -.. table:: Configuration file LINK section - - +-----------------+----------------------------------------------+----------+----------+-------------------+ - | Section entry | Description | Optional | Type | Default value | - +=================+==============================================+==========+==========+===================+ - | arp_q | NIC RX queue where ARP packets should | YES | 0 .. 127 | 0 (default queue) | - | | be filtered. | | | | - +-----------------+----------------------------------------------+----------+----------+-------------------+ - | tcp_syn_local_q | NIC RX queue where TCP packets with SYN | YES | 0 .. 127 | 0 (default queue) | - | | flag should be filtered. | | | | - +-----------------+----------------------------------------------+----------+----------+-------------------+ - | ip_local_q | NIC RX queue where IP packets with local | YES | 0 .. 127 | 0 (default queue) | - | | destination should be filtered. | | | | - | | When TCP, UDP and SCTP local queues are | | | | - | | defined, they take higher priority than this | | | | - | | queue. | | | | - +-----------------+----------------------------------------------+----------+----------+-------------------+ - | tcp_local_q | NIC RX queue where TCP packets with local | YES | 0 .. 127 | 0 (default queue) | - | | destination should be filtered. | | | | - +-----------------+----------------------------------------------+----------+----------+-------------------+ - | udp_local_q | NIC RX queue where TCP packets with local | YES | 0 .. 127 | 0 (default queue) | - | | destination should be filtered. | | | | - +-----------------+----------------------------------------------+----------+----------+-------------------+ - | sctp_local_q | NIC RX queue where TCP packets with local | YES | 0 .. 127 | 0 (default queue) | - | | destination should be filtered. | | | | - +-----------------+----------------------------------------------+----------+----------+-------------------+ - | promisc | Indicates whether current link should be | YES | YES/NO | YES | - | | started in promiscuous mode. | | | | - +-----------------+----------------------------------------------+----------+----------+-------------------+ - - -RXQ section -~~~~~~~~~~~ - -.. _table_ip_pipelines_rxq_section: - -.. tabularcolumns:: |p{3cm}|p{7cm}|p{1.5cm}|p{1.5cm}|p{2cm}| - -.. table:: Configuration file RXQ section - - +---------------+--------------------------------------------+----------+----------+---------------+ - | Section | Description | Optional | Type | Default value | - +===============+============================================+==========+==========+===============+ - | mempool | Mempool to use for buffer allocation for | YES | uint32_t | MEMPOOL0 | - | | current NIC RX queue. The mempool ID has | | | | - | | to be associated with a valid instance | | | | - | | defined in the mempool entry of the global | | | | - | | section. | | | | - +---------------+--------------------------------------------+----------+----------+---------------+ - | Size | NIC RX queue size (number of descriptors) | YES | uint32_t | 128 | - +---------------+--------------------------------------------+----------+----------+---------------+ - | burst | Read burst size (number of descriptors) | YES | uint32_t | 32 | - +---------------+--------------------------------------------+----------+----------+---------------+ - - -TXQ section -~~~~~~~~~~~ - -.. _table_ip_pipelines_txq_section: - -.. tabularcolumns:: |p{2.5cm}|p{7cm}|p{1.5cm}|p{2cm}|p{1.5cm}| - -.. table:: Configuration file TXQ section - - +---------------+----------------------------------------------+----------+------------------+---------------+ - | Section | Description | Optional | Type | Default value | - +===============+==============================================+==========+==================+===============+ - | size | NIC TX queue size (number of descriptors) | YES | uint32_t | 512 | - | | | | power of 2 | | - | | | | > 0 | | - +---------------+----------------------------------------------+----------+------------------+---------------+ - | burst | Write burst size (number of descriptors) | YES | uint32_t | 32 | - | | | | power of 2 | | - | | | | 0 < burst < size | | - +---------------+----------------------------------------------+----------+------------------+---------------+ - | dropless | When dropless is set to NO, packets can be | YES | YES/NO | NO | - | | dropped if not enough free slots are | | | | - | | currently available in the queue, so the | | | | - | | write operation to the queue is non- | | | | - | | blocking. | | | | - | | When dropless is set to YES, packets cannot | | | | - | | be dropped if not enough free slots are | | | | - | | currently available in the queue, so the | | | | - | | write operation to the queue is blocking, as | | | | - | | the write operation is retried until enough | | | | - | | free slots become available and all the | | | | - | | packets are successfully written to the | | | | - | | queue. | | | | - +---------------+----------------------------------------------+----------+------------------+---------------+ - | n_retries | Number of retries. Valid only when dropless | YES | uint32_t | 0 | - | | is set to YES. When set to 0, it indicates | | | | - | | unlimited number of retries. | | | | - +---------------+----------------------------------------------+----------+------------------+---------------+ - - -SWQ section -~~~~~~~~~~~ - -.. _table_ip_pipelines_swq_section: - -.. tabularcolumns:: |p{2.5cm}|p{7cm}|p{1.5cm}|p{1.5cm}|p{1.5cm}| - -.. table:: Configuration file SWQ section - - +---------------+----------------------------------------------+----------+------------------+---------------+ - | Section | Description | Optional | Type | Default value | - +===============+==============================================+==========+==================+===============+ - | size | Queue size (number of packets) | YES | uint32_t | 256 | - | | | | power of 2 | | - +---------------+----------------------------------------------+----------+------------------+---------------+ - | burst_read | Read burst size (number of packets) | YES | uint32_t | 32 | - | | | | power of 2 | | - | | | | 0 < burst < size | | - +---------------+----------------------------------------------+----------+------------------+---------------+ - | burst_write | Write burst size (number of packets) | YES | uint32_t | 32 | - | | | | power of 2 | | - | | | | 0 < burst < size | | - +---------------+----------------------------------------------+----------+------------------+---------------+ - | dropless | When dropless is set to NO, packets can be | YES | YES/NO | NO | - | | dropped if not enough free slots are | | | | - | | currently available in the queue, so the | | | | - | | write operation to the queue is non- | | | | - | | blocking. | | | | - | | When dropless is set to YES, packets cannot | | | | - | | be dropped if not enough free slots are | | | | - | | currently available in the queue, so the | | | | - | | write operation to the queue is blocking, as | | | | - | | the write operation is retried until enough | | | | - | | free slots become available and all the | | | | - | | packets are successfully written to the | | | | - | | queue. | | | | - +---------------+----------------------------------------------+----------+------------------+---------------+ - | n_retries | Number of retries. Valid only when dropless | YES | uint32_t | 0 | - | | is set to YES. When set to 0, it indicates | | | | - | | unlimited number of retries. | | | | - +---------------+----------------------------------------------+----------+------------------+---------------+ - | cpu | CPU socket ID where to allocate memory | YES | uint32_t | 0 | - | | for this SWQ. | | | | - +---------------+----------------------------------------------+----------+------------------+---------------+ - - -TM section -~~~~~~~~~~ - -.. _table_ip_pipelines_tm_section: - -.. tabularcolumns:: |p{2.5cm}|p{7cm}|p{1.5cm}|p{1.5cm}|p{1.5cm}| - -.. table:: Configuration file TM section - - +---------------+---------------------------------------------+----------+----------+---------------+ - | Section | Description | Optional | Type | Default value | - +===============+=============================================+==========+==========+===============+ - | Cfg | File name to parse for the TM configuration | YES | string | tm_profile | - | | to be applied. The syntax of this file is | | | | - | | described in the examples/qos_sched DPDK | | | | - | | application documentation. | | | | - +---------------+---------------------------------------------+----------+----------+---------------+ - | burst_read | Read burst size (number of packets) | YES | uint32_t | 64 | - +---------------+---------------------------------------------+----------+----------+---------------+ - | burst_write | Write burst size (number of packets) | YES | uint32_t | 32 | - +---------------+---------------------------------------------+----------+----------+---------------+ - - -KNI section -~~~~~~~~~~~ - -.. _table_ip_pipelines_kni_section: - -.. tabularcolumns:: |p{2.5cm}|p{7cm}|p{1.5cm}|p{1.5cm}|p{1.5cm}| - -.. table:: Configuration file KNI section - - +---------------+----------------------------------------------+----------+------------------+---------------+ - | Section | Description | Optional | Type | Default value | - +===============+==============================================+==========+==================+===============+ - | core | CPU core to run the KNI kernel thread. | YES | See "CPU Core | Not set | - | | When core config is set, the KNI kernel | | notation" | | - | | thread will be bound to the particular core. | | | | - | | When core config is not set, the KNI kernel | | | | - | | thread will be scheduled by the OS. | | | | - +---------------+----------------------------------------------+----------+------------------+---------------+ - | mempool | Mempool to use for buffer allocation for | YES | uint32_t | MEMPOOL0 | - | | current KNI port. The mempool ID has | | | | - | | to be associated with a valid instance | | | | - | | defined in the mempool entry of the global | | | | - | | section. | | | | - +---------------+----------------------------------------------+----------+------------------+---------------+ - | burst_read | Read burst size (number of packets) | YES | uint32_t | 32 | - | | | | power of 2 | | - | | | | 0 < burst < size | | - +---------------+----------------------------------------------+----------+------------------+---------------+ - | burst_write | Write burst size (number of packets) | YES | uint32_t | 32 | - | | | | power of 2 | | - | | | | 0 < burst < size | | - +---------------+----------------------------------------------+----------+------------------+---------------+ - | dropless | When dropless is set to NO, packets can be | YES | YES/NO | NO | - | | dropped if not enough free slots are | | | | - | | currently available in the queue, so the | | | | - | | write operation to the queue is non- | | | | - | | blocking. | | | | - | | When dropless is set to YES, packets cannot | | | | - | | be dropped if not enough free slots are | | | | - | | currently available in the queue, so the | | | | - | | write operation to the queue is blocking, as | | | | - | | the write operation is retried until enough | | | | - | | free slots become available and all the | | | | - | | packets are successfully written to the | | | | - | | queue. | | | | - +---------------+----------------------------------------------+----------+------------------+---------------+ - | n_retries | Number of retries. Valid only when dropless | YES | uint64_t | 0 | - | | is set to YES. When set to 0, it indicates | | | | - | | unlimited number of retries. | | | | - +---------------+----------------------------------------------+----------+------------------+---------------+ - - -SOURCE section +Initialization ~~~~~~~~~~~~~~ -.. _table_ip_pipelines_source_section: - -.. tabularcolumns:: |p{2.5cm}|p{7cm}|p{1.5cm}|p{1.5cm}|p{2cm}| - -.. table:: Configuration file SOURCE section - - +---------------+---------------------------------------+----------+----------+---------------+ - | Section | Description | Optional | Type | Default value | - +===============+=======================================+==========+==========+===============+ - | Mempool | Mempool to use for buffer allocation. | YES | uint32_t | MEMPOOL0 | - +---------------+---------------------------------------+----------+----------+---------------+ - | Burst | Read burst size (number of packets) | | uint32_t | 32 | - +---------------+---------------------------------------+----------+----------+---------------+ - - -SINK section -~~~~~~~~~~~~ +During this stage, EAL layer is initialised and application specific arguments are parsed. Furthermore, the data strcutures +(i.e. linked lists) for application objects are initialized. In case of any initialization error, an error message +is displayed and the application is terminated. -Currently, there are no parameters to be passed to a sink device, so -SINK section is not allowed. +.. _ip_pipeline_runtime: -MSGQ section -~~~~~~~~~~~~ +Run-time +~~~~~~~~ -.. _table_ip_pipelines_msgq_section: +The master thread is creating and managing all the application objects based on CLI input. -.. tabularcolumns:: |p{2.5cm}|p{7cm}|p{1.5cm}|p{1.5cm}|p{1.5cm}| +Each data plane thread runs one or several pipelines previously assigned to it in round-robin order. Each data plane thread +executes two tasks in time-sharing mode: -.. table:: Configuration file MSGQ section +1. *Packet processing task*: Process bursts of input packets read from the pipeline input ports. - +---------+--------------------------------------------+----------+------------+---------------+ - | Section | Description | Optional | Type | Default value | - +=========+============================================+==========+============+===============+ - | size | Queue size (number of packets) | YES | uint32_t | 64 | - | | | | != 0 | | - | | | | power of 2 | | - +---------+--------------------------------------------+----------+------------+---------------+ - | cpu | CPU socket ID where to allocate memory for | YES | uint32_t | 0 | - | | the current queue. | | | | - +---------+--------------------------------------------+----------+------------+---------------+ +2. *Message handling task*: Periodically, the data plane thread pauses the packet processing task and polls for request + messages send by the master thread. Examples: add/remove pipeline to/from current data plane thread, add/delete rules + to/from given table of a specific pipeline owned by the current data plane thread, read statistics, etc. +Examples +-------- -EAL section -~~~~~~~~~~~ +.. _table_examples: -The application generates the EAL parameters rather than reading them from the command line. +.. tabularcolumns:: |p{3cm}|p{5cm}|p{4cm}|p{4cm}| -The CPU core mask parameter is generated based on the core entry of all PIPELINE sections. -All the other EAL parameters can be set from this section of the application configuration file. +.. table:: Pipeline examples provided with the application + + +-----------------------+----------------------+----------------+------------------------------------+ + | Name | Table(s) | Actions | Messages | + +=======================+======================+================+====================================+ + | L2fwd | Stub | Forward | 1. Mempool create | + | | | | 2. Link create | + | Note: Implemented | | | 3. Pipeline create | + | using pipeline with | | | 4. Pipeline port in/out | + | a simple pass-through | | | 5. Pipeline table | + | connection between | | | 6. Pipeline port in table | + | input and output | | | 7. Pipeline enable | + | ports. | | | 8. Pipeline table rule add | + +-----------------------+----------------------+----------------+------------------------------------+ + | Flow classification | Exact match | Forward | 1. Mempool create | + | | | | 2. Link create | + | | * Key = byte array | | 3. Pipeline create | + | | (16 bytes) | | 4. Pipeline port in/out | + | | * Offset = 278 | | 5. Pipeline table | + | | * Table size = 64K | | 6. Pipeline port in table | + | | | | 7. Pipeline enable | + | | | | 8. Pipeline table rule add default | + | | | | 9. Pipeline table rule add | + +-----------------------+----------------------+----------------+------------------------------------+ + | KNI | Stub | Forward | 1. Mempool create | + | | | | 2. Link create | + | | | | 3. Pipeline create | + | | | | 4. Pipeline port in/out | + | | | | 5. Pipeline table | + | | | | 6. Pipeline port in table | + | | | | 7. Pipeline enable | + | | | | 8. Pipeline table rule add | + +-----------------------+----------------------+----------------+------------------------------------+ + | Firewall | ACL | Allow/Drop | 1. Mempool create | + | | | | 2. Link create | + | | * Key = n-tuple | | 3. Pipeline create | + | | * Offset = 270 | | 4. Pipeline port in/out | + | | * Table size = 4K | | 5. Pipeline table | + | | | | 6. Pipeline port in table | + | | | | 7. Pipeline enable | + | | | | 8. Pipeline table rule add default | + | | | | 9. Pipeline table rule add | + +-----------------------+----------------------+----------------+------------------------------------+ + | IP routing | LPM (IPv4) | Forward | 1. Mempool Create | + | | | | 2. Link create | + | | * Key = IP dest addr | | 3. Pipeline creat | + | | * Offset = 286 | | 4. Pipeline port in/out | + | | * Table size = 4K | | 5. Pipeline table | + | | | | 6. Pipeline port in table | + | | | | 7. Pipeline enable | + | | | | 8. Pipeline table rule add default | + | | | | 9. Pipeline table rule add | + +-----------------------+----------------------+----------------+------------------------------------+ + | Equal-cost multi-path | LPM (IPv4) | Forward, | 1. Mempool Create | + | routing (ECMP) | | load balance, | 2. Link create | + | | * Key = IP dest addr | encap ether | 3. Pipeline create | + | | * Offset = 286 | | 4. Pipeline port in/out | + | | * Table size = 4K | | 5. Pipeline table (LPM) | + | | | | 6. Pipeline table (Array) | + | | | | 7. Pipeline port in table (LPM) | + | | Array | | 8. Pipeline enable | + | | | | 9. Pipeline table rule add default | + | | * Key = Array index | | 10. Pipeline table rule add(LPM) | + | | * Offset = 256 | | 11. Pipeline table rule add(Array) | + | | * Size = 64K | | | + | | | | | + +-----------------------+----------------------+----------------+------------------------------------+ +Command Line Interface (CLI) +---------------------------- -Library of pipeline types -------------------------- +Link +~~~~ -Pipeline module -~~~~~~~~~~~~~~~ - -A pipeline is a self-contained module that implements a packet processing function and is typically implemented on -top of the DPDK Packet Framework *librte_pipeline* library. -The application provides a run-time mechanism to register different pipeline types. - -Depending on the required configuration, each registered pipeline type (pipeline class) is instantiated one or -several times, with each pipeline instance (pipeline object) assigned to one of the available CPU cores. -Each CPU core can run one or more pipeline instances, which might be of same or different types. -For more information of the CPU core threading model, please refer to the :ref:`ip_pipeline_runtime` section. - - -Pipeline type -^^^^^^^^^^^^^ - -Each pipeline type is made up of a back-end and a front-end. The back-end represents the packet processing engine -of the pipeline, typically implemented using the DPDK Packet Framework libraries, which reads packets from the -input packet queues, handles them and eventually writes them to the output packet queues or drops them. -The front-end represents the run-time configuration interface of the pipeline, which is exposed as CLI commands. -The front-end communicates with the back-end through message queues. - -.. _table_ip_pipelines_back_end: - -.. tabularcolumns:: |p{1cm}|p{2cm}|p{12cm}| - -.. table:: Pipeline back-end - - +------------+------------------+--------------------------------------------------------------------+ - | Field name | Field type | Description | - +============+==================+====================================================================+ - | f_init | Function pointer | Function to initialize the back-end of the current pipeline | - | | | instance. Typical work implemented by this function for the | - | | | current pipeline instance: | - | | | Memory allocation; | - | | | Parse the pipeline type specific arguments; | - | | | Initialize the pipeline input ports, output ports and tables, | - | | | interconnect input ports to tables; | - | | | Set the message handlers. | - +------------+------------------+--------------------------------------------------------------------+ - | f_free | Function pointer | Function to free the resources allocated by the back-end of the | - | | | current pipeline instance. | - +------------+------------------+--------------------------------------------------------------------+ - | f_run | Function pointer | Set to NULL for pipelines implemented using the DPDK library | - | | | librte_pipeline (typical case), and to non-NULL otherwise. This | - | | | mechanism is made available to support quick integration of | - | | | legacy code. | - | | | This function is expected to provide the packet processing | - | | | related code to be called as part of the CPU thread dispatch | - | | | loop, so this function is not allowed to contain an infinite loop. | - +------------+------------------+--------------------------------------------------------------------+ - | f_timer | Function pointer | Function to read the pipeline input message queues, handle | - | | | the request messages, create response messages and write | - | | | the response queues. The format of request and response | - | | | messages is defined by each pipeline type, with the exception | - | | | of some requests which are mandatory for all pipelines (e.g. | - | | | ping, statistics). | - +------------+------------------+--------------------------------------------------------------------+ + Link configuration :: + link <link_name> + dev <device_name>|port <port_id> + rxq <n_queues> <queue_size> <mempool_name> + txq <n_queues> <queue_size> promiscuous on | off + [rss <qid_0> ... <qid_n>] -.. _table_ip_pipelines_front_end: - -.. tabularcolumns:: |p{1cm}|p{2cm}|p{12cm}| + Note: The PCI device name must be specified in the Domain:Bus:Device.Function format. -.. table:: Pipeline front-end - - +------------+-----------------------+-------------------------------------------------------------------+ - | Field name | Field type | Description | - +============+=======================+===================================================================+ - | f_init | Function pointer | Function to initialize the front-end of the current pipeline | - | | | instance. | - +------------+-----------------------+-------------------------------------------------------------------+ - | f_post_init| Function pointer | Function to run once after f_init. | - +------------+-----------------------+-------------------------------------------------------------------+ - | f_track | Function pointer | See section Tracking pipeline output port to physical link. | - +------------+-----------------------+-------------------------------------------------------------------+ - | f_free | Function pointer | Function to free the resources allocated by the front-end of | - | | | the current pipeline instance. | - +------------+-----------------------+-------------------------------------------------------------------+ - | cmds | Array of CLI commands | Array of CLI commands to be registered to the application CLI | - | | | for the current pipeline type. Even though the CLI is executed | - | | | by a different pipeline (typically, this is the master pipeline), | - | | | from modularity perspective is more efficient to keep the | - | | | message client side (part of the front-end) together with the | - | | | message server side (part of the back-end). | - +------------+-----------------------+-------------------------------------------------------------------+ +Mempool +~~~~~~~ -Tracking pipeline output port to physical link -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Each pipeline instance is a standalone block that does not have visibility into the other pipeline instances or -the application-level pipeline inter-connectivity. -In some cases, it is useful for a pipeline instance to get application level information related to pipeline -connectivity, such as to identify the output link (e.g. physical NIC port) where one of its output ports connected, -either directly or indirectly by traversing other pipeline instances. + Mempool create :: -Tracking can be successful or unsuccessful. -Typically, tracking for a specific pipeline instance is successful when each one of its input ports can be mapped -to a single output port, meaning that all packets read from the current input port can only go out on a single -output port. -Depending on the pipeline type, some exceptions may be allowed: a small portion of the packets, considered exception -packets, are sent out on an output port that is pre-configured for this purpose. - -For pass-through pipeline type, the tracking is always successful. -For pipeline types as flow classification, firewall or routing, the tracking is only successful when the number of -output ports for the current pipeline instance is 1. - -This feature is used by the IP routing pipeline for adding/removing implicit routes every time a link is brought -up/down. + mempool <mempool_name> buffer <buffer_size> + pool <pool_size> cache <cache_size> cpu <cpu_id> -Table copies -^^^^^^^^^^^^ - -Fast table copy: pipeline table used by pipeline for the packet processing task, updated through messages, table -data structures are optimized for lookup operation. - -Slow table copy: used by the configuration layer, typically updated through CLI commands, kept in sync with the fast -copy (its update triggers the fast copy update). -Required for executing advanced table queries without impacting the packet processing task, therefore the slow copy -is typically organized using different criteria than the fast copy. - -Examples: - -* Flow classification: Search through current set of flows (e.g. list all flows with a specific source IP address); - -* Firewall: List rules in descending order of priority; - -* Routing table: List routes sorted by prefix depth and their type (local, remote, default); - -* ARP: List entries sorted per output interface. - - -Packet meta-data -^^^^^^^^^^^^^^^^ - -Packet meta-data field offsets provided as argument to pipeline instances are essentially defining the data structure -for the packet meta-data used by the current application use-case. -It is very useful to put it in the configuration file as a comment in order to facilitate the readability of the -configuration file. - -The reason to use field offsets for defining the data structure for the packet meta-data is due to the C language -limitation of not being able to define data structures at run-time. -Feature to consider: have the configuration file parser automatically generate and print the data structure defining -the packet meta-data for the current application use-case. - -Packet meta-data typically contains: +Software queue +~~~~~~~~~~~~~~ -1. Pure meta-data: intermediate data per packet that is computed internally, passed between different tables of - the same pipeline instance (e.g. lookup key for the ARP table is obtained from the routing table), or between - different pipeline instances (e.g. flow ID, traffic metering color, etc); + Create software queue :: -2. Packet fields: typically, packet header fields that are read directly from the packet, or read from the packet - and saved (duplicated) as a working copy at a different location within the packet meta-data (e.g. Diffserv - 5-tuple, IP destination address, etc). + swq <swq_name> size <size> cpu <cpu_id> -Several strategies are used to design the packet meta-data, as described in the next subsections. +Traffic manager +~~~~~~~~~~~~~~~ -Store packet meta-data in a different cache line as the packet headers -"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" + Add traffic manager subport profile :: -This approach is able to support protocols with variable header length, like MPLS, where the offset of IP header -from the start of the packet (and, implicitly, the offset of the IP header in the packet buffer) is not fixed. -Since the pipelines typically require the specification of a fixed offset to the packet fields (e.g. Diffserv -5-tuple, used by the flow classification pipeline, or the IP destination address, used by the IP routing pipeline), -the workaround is to have the packet RX pipeline copy these fields at fixed offsets within the packet meta-data. + tmgr subport profile + <tb_rate> <tb_size> + <tc0_rate> <tc1_rate> <tc2_rate> <tc3_rate> + <tc_period> -As this approach duplicates some of the packet fields, it requires accessing more cache lines per packet for filling -in selected packet meta-data fields (on RX), as well as flushing selected packet meta-data fields into the -packet (on TX). -Example: + Add traffic manager pipe profile :: -.. code-block:: ini + tmgr pipe profile + <tb_rate> <tb_size> + <tc0_rate> <tc1_rate> <tc2_rate> <tc3_rate> + <tc_period> + <tc_ov_weight> <wrr_weight0..15> + Create traffic manager port :: - ; struct app_pkt_metadata { - ; uint32_t ip_da; - ; uint32_t hash; - ; uint32_t flow_id; - ; uint32_t color; - ; } __attribute__((__packed__)); - ; + tmgr <tmgr_name> + rate <rate> + spp <n_subports_per_port> + pps <n_pipes_per_subport> + qsize <qsize_tc0> + <qsize_tc1> <qsize_tc2> <qsize_tc3> + fo <frame_overhead> mtu <mtu> cpu <cpu_id> - [PIPELINE1] - ; Packet meta-data offsets - ip_da_offset = 0; Used by: routing - hash_offset = 4; Used by: RX, flow classification - flow_id_offset = 8; Used by: flow classification, flow actions - color_offset = 12; Used by: flow actions, routing + Configure traffic manager subport :: + tmgr <tmgr_name> + subport <subport_id> + profile <subport_profile_id> -Overlay the packet meta-data in the same cache line with the packet headers -""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" + Configure traffic manager pipe :: -This approach is minimizing the number of cache line accessed per packet by storing the packet metadata in the -same cache line with the packet headers. -To enable this strategy, either some headroom is reserved for meta-data at the beginning of the packet headers -cache line (e.g. if 16 bytes are needed for meta-data, then the packet headroom can be set to 128+16 bytes, so -that NIC writes the first byte of the packet at offset 16 from the start of the first packet cache line), -or meta-data is reusing the space of some packet headers that are discarded from the packet (e.g. input Ethernet -header). + tmgr <tmgr_name> + subport <subport_id> + pipe from <pipe_id_first> to <pipe_id_last> + profile <pipe_profile_id> -Example: -.. code-block:: ini +Tap +~~~ - ; struct app_pkt_metadata { - ; uint8_t headroom[RTE_PKTMBUF_HEADROOM]; /* 128 bytes (default) */ - ; union { - ; struct { - ; struct ether_hdr ether; /* 14 bytes */ - ; struct qinq_hdr qinq; /* 8 bytes */ - ; }; - ; struct { - ; uint32_t hash; - ; uint32_t flow_id; - ; uint32_t color; - ; }; - ; }; - ; struct ipv4_hdr ip; /* 20 bytes */ - ; } __attribute__((__packed__)); - ; - [PIPELINE2] - ; Packet meta-data offsets - qinq_offset = 142; Used by: RX, flow classification - ip_da_offset = 166; Used by: routing - hash_offset = 128; Used by: RX, flow classification - flow_id_offset = 132; Used by: flow classification, flow actions - color_offset = 136; Used by: flow actions, routing + Create tap port :: + tap <name> -List of pipeline types -~~~~~~~~~~~~~~~~~~~~~~ -.. _table_ip_pipelines_types: +Kni +~~~ -.. tabularcolumns:: |p{3cm}|p{5cm}|p{4cm}|p{4cm}| + Create kni port :: -.. table:: List of pipeline types provided with the application - - +-----------------------+-----------------------------+-----------------------+------------------------------------------+ - | Name | Table(s) | Actions | Messages | - +=======================+=============================+=======================+==========================================+ - | Pass-through | Passthrough | 1. Pkt metadata build | 1. Ping | - | | | 2. Flow hash | 2. Stats | - | Note: depending on | | 3. Pkt checks | | - | port type, can be | | 4. Load balancing | | - | used for RX, TX, IP | | | | - | fragmentation, IP | | | | - | reassembly or Traffic | | | | - | Management | | | | - +-----------------------+-----------------------------+-----------------------+------------------------------------------+ - | Flow classification | Exact match | 1. Flow ID | 1. Ping | - | | | | | - | | * Key = byte array | 2. Flow stats | 2. Stats | - | | (source: pkt metadata) | 3. Metering | 3. Flow stats | - | | * Data = action dependent | 4. Network Address | 4. Action stats | - | | | 5. Translation (NAT) | 5. Flow add/ update/ delete | - | | | | 6. Default flow add/ update/ delete | - | | | | 7. Action update | - +-----------------------+-----------------------------+-----------------------+------------------------------------------+ - | Flow actions | Array | 1. Flow stats | 1. Ping | - | | | | | - | | * Key = Flow ID | 2. Metering | 2. Stats | - | | (source: pkt metadata) | 3. Network Address | 3. Action stats | - | | * Data = action dependent | 4. Translation (NAT) | 4. Action update | - +-----------------------+-----------------------------+-----------------------+------------------------------------------+ - | Firewall | ACL | 1. Allow/Drop | 1. Ping | - | | | | | - | | * Key = n-tuple | | 2. Stats | - | | (source: pkt headers) | | 3. Rule add/ update/ delete | - | | * Data = none | | 4. Default rule add/ update/ delete | - +-----------------------+-----------------------------+-----------------------+------------------------------------------+ - | IP routing | LPM (IPv4 or IPv6, | 1. TTL decrement and | 1. Ping | - | | depending on pipeline type) | 2. IPv4 checksum | 2. Stats | - | | | | | - | | * Key = IP destination | 3. update | 3. Route add/ update/ delete | - | | (source: pkt metadata) | 4. Header | 4. Default route add/ update/ delete | - | | * Data = Dependent on | 5. encapsulation | 5. ARP entry add/ update/ delete | - | | actions and next hop | 6. (based on next hop | 6. Default ARP entry add/ update/ delete | - | | type | 7. type) | | - | | | | | - | | Hash table (for ARP, only | | | - | | | | | - | | when ARP is enabled) | | | - | | | | | - | | * Key = (Port ID, | | | - | | next hop IP address) | | | - | | (source: pkt meta-data) | | | - | | * Data: MAC address | | | - +-----------------------+-----------------------------+-----------------------+------------------------------------------+ + kni <kni_name> + link <link_name> + mempool <mempool_name> + [thread <thread_id>] +Action profile +~~~~~~~~~~~~~~ -Command Line Interface (CLI) ----------------------------- + Create action profile for pipeline input port :: + + port in action profile <profile_name> + [filter match | mismatch offset <key_offset> mask <key_mask> key <key_value> port <port_id>] + [balance offset <key_offset> mask <key_mask> port <port_id0> ... <port_id15>] + + Create action profile for the pipeline table :: + + table action profile <profile_name> + ipv4 | ipv6 + offset <ip_offset> + fwd + [balance offset <key_offset> mask <key_mask> outoffset <out_offset>] + [meter srtcm | trtcm + tc <n_tc> + stats none | pkts | bytes | both] + [tm spp <n_subports_per_port> pps <n_pipes_per_subport>] + [encap ether | vlan | qinq | mpls | pppoe] + [nat src | dst + proto udp | tcp] + [ttl drop | fwd + stats none | pkts] + [stats pkts | bytes | both] + [time] + + +Pipeline +~~~~~~~~ -Global CLI commands -~~~~~~~~~~~~~~~~~~~ +Create pipeline :: + + pipeline <pipeline_name> + period <timer_period_ms> + offset_port_id <offset_port_id> + cpu <cpu_id> + +Create pipeline input port :: + + pipeline <pipeline_name> port in + bsz <burst_size> + link <link_name> rxq <queue_id> + | swq <swq_name> + | tmgr <tmgr_name> + | tap <tap_name> mempool <mempool_name> mtu <mtu> + | kni <kni_name> + | source mempool <mempool_name> file <file_name> bpp <n_bytes_per_pkt> + [action <port_in_action_profile_name>] + [disabled] + +Create pipeline output port :: + + pipeline <pipeline_name> port out + bsz <burst_size> + link <link_name> txq <txq_id> + | swq <swq_name> + | tmgr <tmgr_name> + | tap <tap_name> + | kni <kni_name> + | sink [file <file_name> pkts <max_n_pkts>] + +Create pipeline table :: + + pipeline <pipeline_name> table + match + acl + ipv4 | ipv6 + offset <ip_header_offset> + size <n_rules> + | array + offset <key_offset> + size <n_keys> + | hash + ext | lru + key <key_size> + mask <key_mask> + offset <key_offset> + buckets <n_buckets> + size <n_keys> + | lpm + ipv4 | ipv6 + offset <ip_header_offset> + size <n_rules> + | stub + [action <table_action_profile_name>] + +Connect pipeline input port to table :: + + pipeline <pipeline_name> port in <port_id> table <table_id> + +Display statistics for specific pipeline input port, output port +or table :: + + pipeline <pipeline_name> port in <port_id> stats read [clear] + pipeline <pipeline_name> port out <port_id> stats read [clear] + pipeline <pipeline_name> table <table_id> stats read [clear] + +Enable given input port for specific pipeline instance :: + + pipeline <pipeline_name> port out <port_id> disable + +Disable given input port for specific pipeline instance :: + + pipeline <pipeline_name> port out <port_id> disable + +Add default rule to table for specific pipeline instance :: + + pipeline <pipeline_name> table <table_id> rule add + match + default + action + fwd + drop + | port <port_id> + | meta + | table <table_id> + +Add rule to table for specific pipeline instance :: + + pipeline <pipeline_name> table <table_id> rule add + + match + acl + priority <priority> + ipv4 | ipv6 <sa> <sa_depth> <da> <da_depth> + <sp0> <sp1> <dp0> <dp1> <proto> + | array <pos> + | hash + raw <key> + | ipv4_5tuple <sa> <da> <sp> <dp> <proto> + | ipv6_5tuple <sa> <da> <sp> <dp> <proto> + | ipv4_addr <addr> + | ipv6_addr <addr> + | qinq <svlan> <cvlan> + | lpm + ipv4 | ipv6 <addr> <depth> + + action + fwd + drop + | port <port_id> + | meta + | table <table_id> + [balance <out0> ... <out7>] + [meter + tc0 meter <meter_profile_id> policer g <pa> y <pa> r <pa> + [tc1 meter <meter_profile_id> policer g <pa> y <pa> r <pa> + tc2 meter <meter_profile_id> policer g <pa> y <pa> r <pa> + tc3 meter <meter_profile_id> policer g <pa> y <pa> r <pa>]] + [tm subport <subport_id> pipe <pipe_id>] + [encap + ether <da> <sa> + | vlan <da> <sa> <pcp> <dei> <vid> + | qinq <da> <sa> <pcp> <dei> <vid> <pcp> <dei> <vid> + | mpls unicast | multicast + <da> <sa> + label0 <label> <tc> <ttl> + [label1 <label> <tc> <ttl> + [label2 <label> <tc> <ttl> + [label3 <label> <tc> <ttl>]]] + | pppoe <da> <sa> <session_id>] + [nat ipv4 | ipv6 <addr> <port>] + [ttl dec | keep] + [stats] + [time] + + where: + <pa> ::= g | y | r | drop + +Add bulk rules to table for specific pipeline instance :: + + pipeline <pipeline_name> table <table_id> rule add bulk <file_name> <n_rules> -.. _table_ip_pipelines_cli_commands: + Where: + - file_name = path to file + - File line format = match <match> action <action> -.. tabularcolumns:: |p{3cm}|p{6cm}|p{6cm}| +Delete table rule for specific pipeline instance :: -.. table:: Global CLI commands + pipeline <pipeline_name> table <table_id> rule delete + match <match> - +---------+---------------------------------------+--------------------------------------------+ - | Command | Description | Syntax | - +=========+=======================================+============================================+ - | run | Run CLI commands script file. | run <file> | - | | | <file> = path to file with CLI commands to | - | | | execute | - +---------+---------------------------------------+--------------------------------------------+ - | quit | Gracefully terminate the application. | quit | - +---------+---------------------------------------+--------------------------------------------+ +Delete default table rule for specific pipeline instance :: + pipeline <pipeline_name> table <table_id> rule delete + match + default -CLI commands for link configuration -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Add meter profile to the table for specific pipeline instance :: -.. _table_ip_pipelines_runtime_config: + pipeline <pipeline_name> table <table_id> meter profile <meter_profile_id> + add srtcm cir <cir> cbs <cbs> ebs <ebs> + | trtcm cir <cir> pir <pir> cbs <cbs> pbs <pbs> -.. tabularcolumns:: |p{3cm}|p{6cm}|p{6cm}| +Delete meter profile from the table for specific pipeline instance :: -.. table:: List of run-time configuration commands for link configuration + pipeline <pipeline_name> table <table_id> + meter profile <meter_profile_id> delete - +-------------+--------------------+--------------------------------------------+ - | Command | Description | Syntax | - +=============+====================+============================================+ - | link config | Link configuration | link <link ID> config <IP address> <depth> | - +-------------+--------------------+--------------------------------------------+ - | link up | Link up | link <link ID> up | - +-------------+--------------------+--------------------------------------------+ - | link down | Link down | link <link ID> down | - +-------------+--------------------+--------------------------------------------+ - | link ls | Link list | link ls | - +-------------+--------------------+--------------------------------------------+ +Update the dscp table for meter or traffic manager action for specific +pipeline instance :: -CLI commands common for all pipeline types -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + pipeline <pipeline_name> table <table_id> dscp <file_name> -.. _table_ip_pipelines_mandatory: + Where: + - file_name = path to file + - exactly 64 lines + - File line format = <tc_id> <tc_queue_id> <color>, with <color> as: g | y | r + + +Pipeline enable/disable +~~~~~~~~~~~~~~~~~~~~~~~ -.. tabularcolumns:: |p{3cm}|p{6cm}|p{6cm}| + Enable given pipeline instance for specific data plane thread :: -.. table:: CLI commands mandatory for all pipelines + thread <thread_id> pipeline <pipeline_name> enable - +--------------------+------------------------------------------------------+----------------------------------------------+ - | Command | Description | Syntax | - +====================+======================================================+==============================================+ - | ping | Check whether specific pipeline instance is alive. | p <pipeline ID> ping | - | | The master pipeline sends a ping request | | - | | message to given pipeline instance and waits for | | - | | a response message back. | | - | | Timeout message is displayed when the response | | - | | message is not received before the timer | | - | | expires. | | - +--------------------+------------------------------------------------------+----------------------------------------------+ - | stats | Display statistics for specific pipeline input port, | p <pipeline ID> stats port in <port in ID> | - | | output port or table. | p <pipeline ID> stats port out <port out ID> | - | | | p <pipeline ID> stats table <table ID> | - +--------------------+------------------------------------------------------+----------------------------------------------+ - | input port enable | Enable given input port for specific pipeline | p <pipeline ID> port in <port ID> enable | - | | instance. | | - +--------------------+------------------------------------------------------+----------------------------------------------+ - | input port disable | Disable given input port for specific pipeline | p <pipeline ID> port in <port ID> disable | - | | instance. | | - +--------------------+------------------------------------------------------+----------------------------------------------+ -Pipeline type specific CLI commands -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + Disable given pipeline instance for specific data plane thread :: -The pipeline specific CLI commands are part of the pipeline type front-end. + thread <thread_id> pipeline <pipeline_name> disable |