diff options
Diffstat (limited to 'extras/libmemif/docs')
-rw-r--r-- | extras/libmemif/docs/buildinstructions_doc.md | 49 | ||||
-rw-r--r-- | extras/libmemif/docs/buildinstructions_doc.rst | 77 | ||||
-rw-r--r-- | extras/libmemif/docs/devperftest_doc.md | 94 | ||||
-rw-r--r-- | extras/libmemif/docs/gettingstarted_doc.md | 223 | ||||
-rw-r--r-- | extras/libmemif/docs/gettingstarted_doc.rst | 234 |
5 files changed, 311 insertions, 366 deletions
diff --git a/extras/libmemif/docs/buildinstructions_doc.md b/extras/libmemif/docs/buildinstructions_doc.md deleted file mode 100644 index c2eecba4d96..00000000000 --- a/extras/libmemif/docs/buildinstructions_doc.md +++ /dev/null @@ -1,49 +0,0 @@ -## Build Instructions {#libmemif_build_doc} - -#### Install dependencies -``` -# sudo apt-get install -y git cmake autoconf pkg_config libtool check -``` - -Libmemif is now part of VPP repository. Follow fd.io wiki to pull source code from VPP repository. -[https://wiki.fd.io/view/VPP/Pulling,_Building,_Running,_Hacking_and_Pushing_VPP_Code#Pushing_Patches](https://wiki.fd.io/view/VPP/Pulling,_Building,_Running,_Hacking_and_Pushing_VPP_Code#Pushing_Patches) - -Libmemif is located under extras/libmemif. From extras/libmemif: -``` -# mkdir build -# cd build -# cmake .. -# make install -``` - -#### Verify installation: -``` -build# ./examples/icmpr-epoll -``` -Use _help_ command to display build information and commands: -``` -LIBMEMIF EXAMPLE APP: ICMP_Responder -============================== -libmemif version: 3.0 -memif version: 512 - use CTRL+C to exit -MEMIF DETAILS -============================== - interface name: memif_connection - app name: ICMP_Responder - remote interface name: - remote app name: - id: 0 - secret: (null) - role: slave - mode: ethernet - socket filename: /run/vpp/memif.sock - socket filename: /run/vpp/memif.sock - rx queues: - tx queues: - link: down -``` - -#### Examples - -Once the library is built/installed, refer to @ref libmemif_examples_doc and @ref libmemif_gettingstarted_doc for additional information on basic use cases and API usage. diff --git a/extras/libmemif/docs/buildinstructions_doc.rst b/extras/libmemif/docs/buildinstructions_doc.rst new file mode 100644 index 00000000000..6609f7a1926 --- /dev/null +++ b/extras/libmemif/docs/buildinstructions_doc.rst @@ -0,0 +1,77 @@ +.. _libmemif_build_doc: + +Build Instructions +================== + +Install dependencies +-------------------- + +:: + + sudo apt-get install -y git cmake autoconf pkg_config libtool + +Libmemif is now part of VPP repository. Follow fd.io wiki to pull source +code from VPP repository. +https://wiki.fd.io/view/VPP/Pulling,_Building,_Running,_Hacking_and_Pushing_VPP_Code#Pushing_Patches + +Libmemif is located under extras/libmemif. From the vpp workspace root directory:: + + mkdir -p extras/libmemif/build + cd extras/libmemif/build + cmake .. + make install + +Verify installation: +-------------------- + +:: + + ./examples/icmp_responder -? + +Use ``-?`` flag to display help:: + + LIBMEMIF EXAMPLE APP: icmp_responder_example + ============================== + libmemif version: 4.0, memif version: 2.0 + ============================== + In this example, memif endpoint connects to an external application. + The example application can resolve ARP and reply to ICMPv4 packets. + The program will exit once the interface is disconnected. + ============================== + Usage: icmp_responder [OPTIONS] + + Options: + -r Interface role <slave|master>. Default: slave + -s Socket path. Supports abstract socket using @ before the path. Default: /run/vpp/memif.sock + -i Interface id. Default: 0 + -a IPv4 address. Default: 192.168.1.1 + -h Mac address. Default: aa:aa:aa:aa:aa:aa + -? Show help and exit. + -v Show libmemif and memif version information and exit. + +Running tests: +-------------- + +Tests needs to their proper functioning Unity framework which is depended externally and cloned from official git repository:: + + mkdir -p extras/libmemif/build + cd extras/libmemif/build + cmake .. + make + ctest + +``ctest`` will execute the tests and print out brief information about test suites with their statuses. + +In case we want verbose: :: + + ctest --verbose + ctest --extra-verbose + +If there are any needs to debug tests, just add to cmake command ``-DCMAKE_BUILD_TYPE=Debug`` flag. + +Use Cases +--------- + +Once the library is built/installed, refer to :ref:`libmemif_gettingstarted_doc` +and :ref:`libmemif_examples_doc` for additional information on basic use cases +and API usage. diff --git a/extras/libmemif/docs/devperftest_doc.md b/extras/libmemif/docs/devperftest_doc.md deleted file mode 100644 index 4c210f5b514..00000000000 --- a/extras/libmemif/docs/devperftest_doc.md +++ /dev/null @@ -1,94 +0,0 @@ -## Development performance test {#libmemif_devperftest_doc} - -Simple test cases using ICMP. icmpr-epoll example app generates and transmits packets over memif interface. - -#### TC1: LIB-VPP - -Start icmpr-epoll example app and VPP. - -VPP-side config: -``` -DBGvpp# create interface memif id 0 master -DBGvpp# set int state memif0/0 up -DBGvpp# set int ip address memif0/0 192.168.1.1/24 -``` -icmpr-epoll: -``` -conn 0 0 1 -``` -> Last argument specifies interrupt function to use. This function only responds to ARP requests. This is important because, packet generation and transmitting is handled by a separate thread. Calling memif_tx_burst from multiple threads writing on same queue could transmit uninitialized buffers. -Once connection is established, you can send ping from VPP to icmpr-epoll app to learn its mac address. -``` -DBGvpp# ping 192.168.1.2 -``` -> There should be no ICMP response. Only ARP response. -Now send ICMP requests from icmpr-epoll: -``` -send <index> <num-of-packets> <ip_daddr> <hw_daddr> -send 0 5 192.168.1.1 02:fe:ff:ff:ff:ff -``` -this command will create new thread which will generate icmp packets and transmit them over memif connection with specified index. Once the sequence is finished status will be printed. - -###### Example results (overview of test data) - -(This test was run with modification in VPP-memif plugin. The modification disallows memif tx node to allocate last ring buffer) -lib-tx: 200M (if ring full don't drop packets) -vpp-rx: 200M -vpp-tx: 200M - 50K (if ring full drop packets) -lib-rx: =vpp-tx -drop: ~0.025% (full ring) -pps: ~650K -multiple interfaces: -pps: divided -drop: constant - -#### TC2: LIB-LIB - -This test case will not drop packets if memif ring is full. Instead it will loop until all required packets have been sent. - -Start two instances of icmpr-epoll example app. -instance 1: -``` -conn 0 1 0 -``` -instance 2: -``` -conn 0 0 1 -send 0 5 192.168.1.1 aa:aa:aa:aa:aa:aa -``` -> icmpr-epoll example app doesn't check ip or mac address so as long as the format is correct you can type anything as ip_daddr and hw_daddr arguments. - -###### Example results (overview of test data) - -lib1-tx: 200M (if ring full don't drop packets) -lib2-rx: 200M -lib2-tx: 200M (if ring full don't drop packets) -lib1-rx: 200M -drop: obsolete -pps: 4.5M -multiple interfaces: -not tested (expected same as TC1) - -#### TC3: LIB-LIB - -Start two instances of icmpr-epoll example app. -instance 1: -``` -conn 0 1 -``` -instance 2: -``` -conn 0 0 1 -send 0 5 192.168.1.1 aa:aa:aa:aa:aa:aa -``` - -###### Example results (overview of test data) - -lib1-tx: 200M (if ring full don't drop packets) -lib2-rx: 200M -lib2-tx: 169626182 (if ring full drop packets) -lib1-rx: =lib2-tx -drop: ~15% -pps: ~6M -multiple interfaces: -not tested (expected same as TC1) diff --git a/extras/libmemif/docs/gettingstarted_doc.md b/extras/libmemif/docs/gettingstarted_doc.md deleted file mode 100644 index abf9a1a06fd..00000000000 --- a/extras/libmemif/docs/gettingstarted_doc.md +++ /dev/null @@ -1,223 +0,0 @@ -## Getting started {#libmemif_gettingstarted_doc} - -#### Concept (Connecting to VPP) - -For detailed information on api calls and structures please refer to @ref libmemif.h. - -1. Initialize memif - - Declare callback function handling file descriptor event polling. -```C -int -control_fd_update (int fd, uint8_t events) -{ -... -} -``` - - Call memif initialization function. memif\_init -```C -err = memif_init (control_fd_update, APP_NAME, NULL, NULL); -``` - -> If event occurs on any file descriptor returned by this callback, call memif\_control\_fd\_handler function. Since version 2.0, last two optional arguments are used to specify custom memory allocation. -```C -memif_err = memif_control_fd_handler (evt.data.fd, events); -``` -> If callback function parameter for memif\_init function is set to NULL, libmemif will handle file descriptor event polling. - Api call memif\_poll\_event will call epoll\_pwait with user defined timeout to poll event on file descriptors opened by libmemif. -```C -/* main loop */ - while (1) - { - if (memif_poll_event (-1) < 0) - { - DBG ("poll_event error!"); - } - } -``` - -> Memif initialization function will initialize internal structures and create timer file descriptor, which will be used for sending periodic connection requests. Timer is disarmed if no memif interface is created. - -2. Creating interface - - Declare memif connection handle. -```C -memif_conn_handle_t c; -``` -> example app uses struct that contains connection handle, rx/tx buffers and other connection specific information. - - - Specify connection arguments. -```C -memif_conn_args_t args; -memset (&args, 0, sizeof (args)); -args.is_master = is_master; -args.log2_ring_size = 10; -args.buffer_size = 2048; -args.num_s2m_rings = 2; -args.num_m2s_rings = 2; -strncpy ((char *) args.interface_name, IF_NAME, strlen (IF_NAME)); -args.mode = 0; -args.interface_id = 0; -``` - - Declare callback functions called on connected/disconnected/interrupted status changed. -```C -int -on_connect (memif_conn_handle_t conn, void *private_ctx) -{ -... -} - -int -on_disconnect (memif_conn_handle_t conn, void *private_ctx) -{ - INFO ("memif connected!"); - return 0; -} -``` - - Call memif interface create function. memif\_create -```C -err = memif_create (&c->conn, - &args, on_connect, on_disconnect, on_interrupt, &ctx[index]); -``` -> If connection is in slave mode, arms timer file descriptor. -> If on interrupt callback is set to NULL, user will not be notified about interrupt. Use memif\_get\_queue\_efd call to get interrupt file descriptor for specific queue. -```C -int fd = -1; -err = memif_get_queue_efd (c->conn, data->qid, &fd); -``` - -3. Connection establishment - - User application will poll events on all file descriptors returned in memif\_control\_fd\_update\_t callback. - - On event call memif\_control\_fd\_handler. - - Everything else regarding connection establishment will be done internally. - - Once connection has been established, a callback will inform the user about connection status change. - -4. Interrupt packet receive - - If event is polled on interrupt file descriptor, libmemif will call memif\_interrupt\_t callback specified for every connection instance. -```C -int -on_interrupt (memif_conn_handle_t conn, void *private_ctx, uint16_t qid) -{ -... -} -``` - -6. Memif buffers - - Packet data are stored in memif\_buffer\_t. Pointer _data_ points to shared memory buffer, and unsigned integer *_len* contains buffer length. - - flags: MEMIF\_BUFFER\_FLAG\_NEXT states that the buffer is not large enough to contain whole packet, so next buffer contains the rest of the packet. (chained buffers) -```C -typedef struct -{ - uint16_t desc_index; - uint32_t len; - uint8_t flags; - void *data; -} memif_buffer_t; -``` - -5. Packet receive - - Api call memif\_rx\_burst will set all required fields in memif buffers provided by user application, dequeue received buffers and consume interrupt event on receive queue. The event is not consumed, if memif_rx_burst fails. -```C -err = memif_rx_burst (c->conn, qid, c->bufs, MAX_MEMIF_BUFS, &rx); -``` - - User application can then process packets. - - Api call memif\_refill\_queue will enqueue rx buffers. -```C -err = memif_refill_queue (c->conn, qid, rx); -``` - -6. Packet transmit - - Api call memif\_buffer\_alloc will find free tx buffers and set all required fields in memif buffers provided by user application. -```C -err = memif_buffer_alloc (c->conn, qid, c->tx_bufs, n, &r); -``` - - User application can populate shared memory buffers with packets. - - Api call memif\_tx\_burst will enqueue tx buffers -```C -err = memif_tx_burst (c->conn, qid, c->tx_bufs, c->tx_buf_num, &r); -``` - -7. Helper functions - - Memif version -```C -uint16_t memif_ver = memif_get_version (); -``` - - Memif details - - Api call memif\_get\_details will return details about connection. -```C -err = memif_get_details (c->conn, &md, buf, buflen); -``` - - Memif error messages - - Every api call returns error code (integer value) mapped to error string. - - Call memif\_strerror will return error message assigned to specific error code. -```C -if (err != MEMIF_ERR_SUCCESS) - INFO ("memif_get_details: %s", memif_strerror (err)); -``` - - Not all syscall errors are translated to memif error codes. If error code 1 (MEMIF\_ERR\_SYSCALL) is returned then libmemif needs to be compiled with -DMEMIF_DBG flag to print error message. Use _make -B_ to rebuild libmemif in debug mode. - -#### Example app (libmemif fd event polling): - -- @ref extras/libmemif/examples/icmp_responder - -> Optional argument: transmit queue id. -``` -icmpr 1 -``` -> Set transmit queue id to 1. Default is 0. -> Application will create memif interface in slave mode and try to connect to VPP. Exit using Ctrl+C. Application will handle SIGINT signal, free allocated memory and exit with EXIT_SUCCESS. - -#### Example app: - -ICMP Responder custom fd event polling. - -- @ref extras/libmemif/examples/icmp_responder-epoll - -#### Example app (multi-thread queue polling) - -ICMP Responder multi-thread. -- @ref extras/libmemif/examples/icmp_responder-mt - -> Simple example of libmemif multi-thread usage. Connection establishment is handled by main thread. There are two rx/tx queues in this example. One in polling mode and second in interrupt mode. - -VPP config: -``` -# create interface memif id 0 master -# set int state memif0 up -# set int ip address memif0 192.168.1.1/24 -# ping 192.168.1.2 -``` -For multiple rings (queues) support run VPP with worker threads: -example startup.conf: -``` -unix { - interactive - nodaemon - full-coredump -} - -cpu { - workers 2 -} -``` -VPP config: -``` -# create interface memif id 0 master -# set int state memif0 up -# set int ip address memif0 192.168.1.1/24 -# ping 192.168.1.2 -``` -> Master mode queue number is limited by worker threads. Slave mode interface needs to specify number of queues. -``` -# create memif id 0 slave rx-queues 2 tx-queues 2 -``` -> Example applications use VPP default socket file for memif: /run/vpp/memif.sock -> For master mode, socket directory must exist prior to memif\_create call. - -#### Unit tests - -Unit tests use [Check](https://libcheck.github.io/check/index.html) framework. This framework must be installed in order to build *unit\_test* binary. -Ubuntu/Debian: -``` -sudo apt-get install check -``` -[More platforms](https://libcheck.github.io/check/web/install.html) - diff --git a/extras/libmemif/docs/gettingstarted_doc.rst b/extras/libmemif/docs/gettingstarted_doc.rst new file mode 100644 index 00000000000..f576fe25145 --- /dev/null +++ b/extras/libmemif/docs/gettingstarted_doc.rst @@ -0,0 +1,234 @@ +.. _libmemif_gettingstarted_doc: + +Getting started +=============== + +For detailed information on api calls and structures please refer to +``libmemif.h``. + +Start by creating a memif socket. Memif socket represents UNIX domain +socket and interfaces assigned to use this socket. Memif uses UNIX domain +socket to communicate with other memif drivers. + +First fill out the ``memif_socket_args`` struct. The minimum required +configuration is the UNIX socket path. > Use ``@`` or ``\0`` at the +beginning of the path to use abstract socket. + +.. code:: c + + memif_socket_args_t sargs; + + strncpy(sargs.path, socket_path, sizeof(sargs.path)); + +.. code:: c + + memif_socket_handle_t memif_socket; + + memif_create_socket(&memif_socket, &sargs, &private_data); + +Once you have created your socket, you can create memif interfaces on +this socket. Fill out the ``memif_conn_args`` struct. Then call +``memif_create()``. + +.. code:: c + + memif_conn_args_t cargs; + + /* Assign your socket handle */ + cargs.socket = memif_socket; + +.. code:: c + + memif_conn_handle_t conn; + + /* Assign callbacks */ + memif_create (&conn, &cargs, on_connect_cb, on_disconnect_cb, on_interrupt_cb, &private_data); + +Now start the polling events using libmemifs builtin polling. + +.. code:: c + + do { + err = memif_poll_event(memif_socket, /* timeout -1 = blocking */ -1); + } while (err == MEMIF_ERR_SUCCESS); + +Polling can be canceled by calling ``memif_cancel_poll_event()``. + +.. code:: c + + memif_cancel_poll_event (memif_socket); + +On link status change ``on_connect`` and ``on_disconnect`` callbacks are +called respectively. Before you can start transmitting data you, first +need to call ``memif_refill_queue()`` for each RX queue to initialize +this queue. + +.. code:: c + + int on_connect (memif_conn_handle_t conn, void *private_ctx) + { + my_private_data_t *data = (my_private_data_t *) private_ctx; + + err = memif_refill_queue(conn, 0, -1, 0); + if (err != MEMIF_ERR_SUCCESS) { + INFO("memif_refill_queue: %s", memif_strerror(err)); + return err; + } + + /* + * Do stuff. + */ + + return 0; + } + +Now you are ready to transmit packets. > Example implementation +``examples/common/sender.c`` and ``examples/common/responder.c`` + +To transmit or receive data you will need to use ``memif_buffer`` +struct. The important fields here are ``void *data``, ``uint32_t len`` +and ``uint8_t flags``. The ``data`` pointer points directly to the +shared memory packet buffer. This is where you will find/insert your +packets. The ``len`` field is the length of the buffer. If the flag +``MEMIF_BUFFER_FLAG_NEXT`` is present in ``flags`` field, this buffer is +chained so the rest of the data is located in the next buffer, and so +on. + +First let’s receive data. To receive data call ``memif_rx_burst()``. The +function will fill out memif buffers passed to it. Then you would +process your data (e.g. copy to your stack). Last you must refill the +queue using ``memif_refill_queue()`` to notify peer that the buffers are +now free and can be overwritten. + +.. code:: c + + /* Fill out memif buffers and mark them as received */ + err = memif_rx_burst(conn, qid, buffers, num_buffers, &num_received); + if (err != MEMIF_ERR_SUCCESS) { + INFO ("memif_rx_burst: %s", memif_strerror(err)); + return err; + } + /* + Process the buffers. + */ + + /* Refill the queue, so that the peer interface can transmit more packets */ + err = memif_refill_queue(conn, qid, num_received, 0); + if (err != MEMIF_ERR_SUCCESS) { + INFO("memif_refill_queue: %s", memif_strerror(err)); + goto error; + } + +In order to transmit data you first need to ‘allocate’ memif buffers +using ``memif_buffer_alloc()``. This function similar to +``memif_rx_burst`` will fill out provided memif buffers. You will then +insert your packets directly into the shared memory (don’t forget to +update ``len`` filed if your packet is smaller that buffer length). +Finally call ``memif_tx_burst`` to transmit the buffers. + +.. code:: c + + /* Alocate memif buffers */ + err = memif_buffer_alloc(conn, qid, buffers, num_pkts, &num_allocated, packet_size); + if (err != MEMIF_ERR_SUCCESS) { + INFO("memif_buffer_alloc: %s", memif_strerror(err)); + goto error; + } + + /* + Fill out the buffers. + + tx_buffers[i].data field points to the shared memory. + update tx_buffers[i].len to your packet length, if the packet is smaller. + */ + + /* Transmit the buffers */ + err = memif_tx_burst(conn, qid, buffers, num_allocated, &num_transmitted); + if (err != MEMIF_ERR_SUCCESS) { + INFO("memif_tx_burst: %s", memif_strerror(err)); + goto error; + } + +Zero-copy Slave +--------------- + +Interface with slave role is the buffer producer, as such it can use +zero-copy mode. + +After receiving buffers, process your packets in place. Then use +``memif_buffer_enq_tx()`` to enqueue rx buffers to tx queue (by swapping +rx buffer with a free tx buffer). + +.. code:: c + + /* Fill out memif buffers and mark them as received */ + err = memif_rx_burst(conn, qid, buffers, num_buffers, &num_received); + if (err != MEMIF_ERR_SUCCESS) { + INFO ("memif_rx_burst: %s", memif_strerror(err)); + return err; + } + + /* + Process the buffers in place. + */ + + /* Enqueue processed buffers to tx queue */ + err = memif_buffer_enq_tx(conn, qid, buffers, num_buffers, &num_enqueued); + if (err != MEMIF_ERR_SUCCESS) { + INFO("memif_buffer_alloc: %s", memif_strerror(err)); + goto error; + } + + /* Refill the queue, so that the peer interface can transmit more packets */ + err = memif_refill_queue(conn, qid, num_enqueued, 0); + if (err != MEMIF_ERR_SUCCESS) { + INFO("memif_refill_queue: %s", memif_strerror(err)); + goto error; + } + + /* Transmit the buffers. */ + err = memif_tx_burst(conn, qid, buffers, num_enqueued, &num_transmitted); + if (err != MEMIF_ERR_SUCCESS) { + INFO("memif_tx_burst: %s", memif_strerror(err)); + goto error; + } + +Custom Event Polling +-------------------- + +Libmemif can be integrated into your applications fd event polling. You +will need to implement ``memif_control_fd_update_t`` callback and pass +it to ``memif_socket_args.on_control_fd_update``. Now each time any file +descriptor belonging to that socket updates, ``on_control_fd_update`` +callback is called. The file descriptor and event type is passed in +``memif_fd_event_t``. It also contains private context that is +associated with this fd. When event is polled on the fd you need to call +``memif_control_fd_handler`` and pass the event type and private context +associated with the fd. + +Multi Threading +--------------- + +Connection establishment +~~~~~~~~~~~~~~~~~~~~~~~~ + +Memif sockets should not be handled in parallel. Instead each thread +should have it’s own socket. However the UNIX socket can be the same. In +case of non-listener socket, it’s straight forward, just create the +socket using the same path. In case of listener socket, the polling +should be done by single thread. > The socket becomes listener once a +Master interface is assigned to it. + +Packet handling +~~~~~~~~~~~~~~~ + +Single queue must not be handled in parallel. Instead you can assign +queues to threads in such way that each queue is only assigned single +thread. + +Shared Memory Layout +-------------------- + +Please refer to `DPDK MEMIF +documentation <http://doc.dpdk.org/guides/nics/memif.html>`__ +``'Shared memory'`` section. |