From d84f2ef54a457918f21bb0ee82392274b62611fe Mon Sep 17 00:00:00 2001 From: Chris Luke Date: Wed, 27 Sep 2017 15:09:48 -0400 Subject: General documentation updates - We now have several developer-focused docs, so create an index page for them. - Rework several docs to fit into the index structure. - Experiment with code highlighting; tweak the CSS slightly to make it slightly nicer to look at. Change-Id: I4185a18f84fa0764745ca7a3148276064a3155c6 Signed-off-by: Chris Luke (cherry picked from commit 64ebb5ff1338140d94c7f9ee72138fe84d89de2e) --- extras/libmemif/README.md | 72 --------- extras/libmemif/docs/BuildInstructions.md | 54 ------- extras/libmemif/docs/GettingStarted.md | 215 ------------------------- extras/libmemif/docs/buildinstructions_doc.md | 55 +++++++ extras/libmemif/docs/gettingstarted_doc.md | 219 ++++++++++++++++++++++++++ extras/libmemif/examples/ExampleSetup.md | 206 ------------------------ extras/libmemif/examples/README.md | 16 -- extras/libmemif/examples/example_setup_doc.md | 207 ++++++++++++++++++++++++ extras/libmemif/examples/examples_doc.md | 18 +++ extras/libmemif/libmemif_doc.md | 74 +++++++++ 10 files changed, 573 insertions(+), 563 deletions(-) delete mode 100644 extras/libmemif/README.md delete mode 100644 extras/libmemif/docs/BuildInstructions.md delete mode 100644 extras/libmemif/docs/GettingStarted.md create mode 100644 extras/libmemif/docs/buildinstructions_doc.md create mode 100644 extras/libmemif/docs/gettingstarted_doc.md delete mode 100644 extras/libmemif/examples/ExampleSetup.md delete mode 100644 extras/libmemif/examples/README.md create mode 100644 extras/libmemif/examples/example_setup_doc.md create mode 100644 extras/libmemif/examples/examples_doc.md create mode 100644 extras/libmemif/libmemif_doc.md (limited to 'extras/libmemif') diff --git a/extras/libmemif/README.md b/extras/libmemif/README.md deleted file mode 100644 index 0cccfb37..00000000 --- a/extras/libmemif/README.md +++ /dev/null @@ -1,72 +0,0 @@ -Shared Memory Packet Interface (memif) Library -============================================== -## Introduction - -Shared memory packet interface (memif) provides high performance packet transmit and receive between user application and Vector Packet Processing (VPP) or multiple user applications. Using libmemif, user application can create shared memory interface in master or slave mode and connect to VPP or another application using libmemif. Once the connection is established, user application can receive or transmit packets using libmemif API. - -![Architecture](docs/architecture.png) - -## Features - -- [x] Slave mode - - [x] Connect to VPP over memif - - [x] ICMP responder example app -- [x] Transmit/receive packets -- [x] Interrupt mode support -- [x] File descriptor event polling in libmemif (optional) - - [x] Simplify file descriptor event polling (one handler for control and interrupt channel) -- [x] Multiple connections -- [x] Multiple queues - - [x] Multi-thread support -- [x] Master mode - - [ ] Multiple regions (TODO) -- [ ] Performance testing (TODO) - -## Quickstart - -This setup will run libmemif ICMP responder example app in container. Install [docker](https://docs.docker.com/engine/installation) engine. -Useful link: [Docker documentation](https://docs.docker.com/get-started). - -Pull image: -``` -# docker pull ligato/libmemif-sample-service -``` - -Now you should be able to see ligato/libmemif-sample-service image on your local machine (IMAGE ID in this README may be outdated): -``` -# docker images -REPOSITORY TAG IMAGE ID CREATED SIZE -ligato/libmemif-sample-service latest 32ecc2f9d013 About a minute ago 468MB -... -``` - -Run container: -``` -# docker run -it --rm --name icmp-responder --hostname icmp-responder --privileged -v "/run/vpp/:/run/vpp/" ligato/libmemif-sample-service -``` -Example application will start in debug mode. Output should look like this: -``` -ICMP_Responder:add_epoll_fd:204: fd 0 added to epoll -MEMIF_DEBUG:src/main.c:memif_init:383: app name: ICMP_Responder -ICMP_Responder:add_epoll_fd:204: fd 4 added to epoll -LIBMEMIF EXAMPLE APP: ICMP_Responder (debug) -============================== -libmemif version: 1.0 (debug) -memif version: 256 -commands: - help - prints this help - exit - exit app - conn - create memif (slave-mode) - del - delete memif - show - show connection details - ip-set - set interface ip address - rx-mode - set queue rx mode -``` - -Continue with [Example setup](examples/ExampleSetup.md) which contains instructions on how to set up conenction between icmpr-epoll example app and VPP-memif. - -#### Next steps - -- [Build instructions](docs/BuildInstructions.md) Instructions on how to build/install libmemif. -- [Examples](examples/README.md) More example apps presenting different features. -- [Getting started](docs/GettingStarted.md) Introduction to libmemif API. Explaining library usage in custom app. diff --git a/extras/libmemif/docs/BuildInstructions.md b/extras/libmemif/docs/BuildInstructions.md deleted file mode 100644 index 9bb59b94..00000000 --- a/extras/libmemif/docs/BuildInstructions.md +++ /dev/null @@ -1,54 +0,0 @@ -## Build Instructions - -Install dependencies -``` -# sudo apt-get install -y git autoconf pkg_config libtool check -``` - -Clone repository to your local machine. -``` -# git clone https://github.com/JakubGrajciar/libmemif.git -``` - -From root directory execute: -For debug build: -``` -# ./bootstrap -# ./configure -# make -# make install -``` - -For release build: -``` -# ./bootstrap -# ./configure -# make release -# make install -``` -Verify installation: -``` -# ./.libs/icmpr-epoll -``` -> Make sure to run the binary file from ./.libs. File ./icmp\_responder in libmemif root directory is script that links the library, so it only verifies successful build. Default install path is /usr/lib. -Use _help_ command to display build information and commands: -``` -ICMP_Responder:add_epoll_fd:204: fd 0 added to epoll -MEMIF_DEBUG:src/main.c:memif_init:383: app name: ICMP_Responder -ICMP_Responder:add_epoll_fd:204: fd 4 added to epoll -LIBMEMIF EXAMPLE APP: ICMP_Responder (debug) -============================== -libmemif version: 1.0 (debug) -memif version: 256 -commands: - help - prints this help - exit - exit app - conn - create memif (slave-mode) - del - delete memif - show - show connection details - ip-set - set interface ip address - rx-mode - set queue rx mode -``` -#### Examples - -Once the library is build/installed, refer to [Examples](../examples/README.md) and [Getting started](GettingStarted.md) for additional information on basic use cases and API usage. diff --git a/extras/libmemif/docs/GettingStarted.md b/extras/libmemif/docs/GettingStarted.md deleted file mode 100644 index 3cc0ceb0..00000000 --- a/extras/libmemif/docs/GettingStarted.md +++ /dev/null @@ -1,215 +0,0 @@ -## Getting started - -#### Concept (Connecting to VPP) - -For detailed information on api calls and structures please refer to [libmemif.h](../src/libmemif.h) - -1. Initialize memif - - Declare callback function handling file descriptor event polling. -```C -int -control_fd_update (int fd, uint8_t events) -{ -... -} -``` - - Call memif initialization function. memif\_init -```C -err = memif_init (control_fd_update, APP_NAME); -``` - -> If event occurres on any file descriptor returned by this callback, call memif\_control\_fd\_handler function. -```C -memif_err = memif_control_fd_handler (evt.data.fd, events); -``` -> If callback function parameter for memif\_init function is set to NULL, libmemif will handle file descriptor event polling. - Api call memif\_poll\_event will call epoll\_pwait with user defined timeout to poll event on file descriptors opened by libmemif. -```C -/* main loop */ - while (1) - { - if (memif_poll_event (-1) < 0) - { - DBG ("poll_event error!"); - } - } -``` - -> Memif initialization function will initialize internal structures and create timer file descriptor, which will be used for sending periodic connection requests. Timer is disarmed if no memif interface is created. - -2. Creating interface - - Declare memif connction handle. -```C -memif_conn_handle_t c; -``` -> example app uses struct that contains connection handle, rx/tx buffers and other connection specific information. - - - Specify connection arguments. -```C -memif_conn_args_t args; -memset (&args, 0, sizeof (args)); -args.is_master = is_master; -args.log2_ring_size = 10; -args.buffer_size = 2048; -args.num_s2m_rings = 2; -args.num_m2s_rings = 2; -strncpy ((char *) args.interface_name, IF_NAME, strlen (IF_NAME)); -strncpy ((char *) args.instance_name, APP_NAME, strlen (APP_NAME)); -args.mode = 0; -args.interface_id = 0; -``` - - Declare callback functions called on connected/disconnected/interrupted status changed. -```C -int -on_connect (memif_conn_handle_t conn, void *private_ctx) -{ -... -} - -int -on_disconnect (memif_conn_handle_t conn, void *private_ctx) -{ - INFO ("memif connected!"); - return 0; -} -``` - - Call memif interface create function. memif\_create -```C -err = memif_create (&c->conn, - &args, on_connect, on_disconnect, on_interrupt, &ctx[index]); -``` -> If connection is in slave mode, arms timer file descriptor. -> If on interrupt callback is set to NULL, user will not be notified about interrupt. Use memif\_get\_queue\_efd call to get interrupt file descriptor for specific queue. -```C -int fd = -1; -err = memif_get_queue_efd (c->conn, data->qid, &fd); -``` - -3. Connection establishment - - User application will poll events on all file descriptors returned in memif\_control\_fd\_update\_t callback. - - On event call memif\_control\_fd\_handler. - - Everything else regarding connection establishment will be done internally. - - Once connection has been established, a callback will inform the user about connection status change. - -4. Interrupt packet receive - - If event is polled on interrupt file descriptor, libmemif will call memif\_interrupt\_t callback specified for every connection instance. -```C -int -on_interrupt (memif_conn_handle_t conn, void *private_ctx, uint16_t qid) -{ -... -} -``` - -6. Memif buffers - - Packet data are stored in memif\_buffer\_t. Pointer _data_ points to shared memory buffer, and unsigned integer *data\_len* contains packet data length. -```C -typedef struct -{ - uint16_t desc_index; - uint32_t buffer_len; - uint32_t data_len; - void *data; -} memif_buffer_t; -``` - -5. Packet receive - - Api call memif\_rx\_burst will set all required fields in memif buffers provided by user application. -```C -err = memif_rx_burst (c->conn, qid, c->rx_bufs, MAX_MEMIF_BUFS, &rx); -``` - - User application can then process packets. - - Api call memif\_buffer\_free will make supplied memif buffers ready for next receive and mark shared memory buffers as free. -```C -err = memif_buffer_free (c->conn, qid, c->rx_bufs, rx, &fb); -``` - -6. Packet transmit - - Api call memif\_buffer\_alloc will set all required fields in memif buffers provided by user application. -```C -err = memif_buffer_alloc (c->conn, qid, c->tx_bufs, n, &r); -``` - - User application can populate shared memory buffers with packets. - - Api call memif\_tx\_burst will inform peer interface (master memif on VPP) that there are packets ready to receive and mark memif buffers as free. -```C -err = memif_tx_burst (c->conn, qid, c->tx_bufs, c->tx_buf_num, &r); -``` - -7. Helper functions - - Memif details - - Api call memif\_get\_details will return details about connection. -```C -err = memif_get_details (c->conn, &md, buf, buflen); -``` - - Memif error messages - - Every api call returns error code (integer value) mapped to error string. - - Call memif\_strerror will return error message assigned to specific error code. -```C -if (err != MEMIF_ERR_SUCCESS) - INFO ("memif_get_details: %s", memif_strerror (err)); -``` - - Not all syscall errors are translated to memif error codes. If error code 1 (MEMIF\_ERR\_SYSCALL) is returned then libmemif needs to be compiled with -DMEMIF_DBG flag to print error message. Use _make -B_ to rebuild libmemif in debug mode. - -#### Example app (libmemif fd event polling): - -- [ICMP Responder](../examples/icmp_responder/main.c) -> Optional argument: transmit queue id. -``` -icmpr 1 -``` -> Set transmit queue id to 1. Default is 0. -> Application will create memif interface in slave mode and try to connect to VPP. Exit using Ctrl+C. Application will handle SIGINT signal, free allocated memory and exit with EXIT_SUCCESS. - -#### Example app: - -- [ICMP Responder custom fd event polling](../examples/icmp_responder-epoll/main.c) - -#### Example app (multi-thread queue polling) - -- [ICMP Responder multi-thread](../examples/icmp_responder-mt/main.c) - -> Simple example of libmemif multi-thread usage. Connection establishment is handled by main thread. There are two rx queues in this example. One in polling mode and second in interrupt mode. - -VPP config: -``` -# create memif id 0 master -# set int state memif0 up -# set int ip address memif0 192.168.1.1/24 -# ping 192.168.1.2 -``` -For multiple rings (queues) support run VPP with worker threads: -example startup.conf: -``` -unix { - interactive - nodaemon - full-coredump -} - -cpu { - workers 2 -} -``` -VPP config: -``` -# create memif id 0 master -# set int state memif0 up -# set int ip address memif0 192.168.1.1/24 -# ping 192.168.1.2 -``` -> Master mode queue number is limited by worker threads. Slave mode interface needs to specify number of queues. -``` -# create memif id 0 slave rx-queues 2 tx-queues 2 -``` -> Example applications use VPP default socket file for memif: /run/vpp/memif.sock -> For master mode, socket directory must exist prior to memif\_create call. - -#### Unit tests - -Unit tests use [Check](https://libcheck.github.io/check/index.html) framework. This framework must be installed in order to build *unit\_test* binary. -Ubuntu/Debian: -``` -sudo apt-get install check -``` -[More platforms](https://libcheck.github.io/check/web/install.html) - diff --git a/extras/libmemif/docs/buildinstructions_doc.md b/extras/libmemif/docs/buildinstructions_doc.md new file mode 100644 index 00000000..838e835a --- /dev/null +++ b/extras/libmemif/docs/buildinstructions_doc.md @@ -0,0 +1,55 @@ +## Build Instructions {#libmemif_build_doc} + +Install dependencies +``` +# sudo apt-get install -y git autoconf pkg_config libtool check +``` + +Clone repository to your local machine. +``` +# git clone https://github.com/JakubGrajciar/libmemif.git +``` + +From root directory execute: +For debug build: +``` +# ./bootstrap +# ./configure +# make +# make install +``` + +For release build: +``` +# ./bootstrap +# ./configure +# make release +# make install +``` +Verify installation: +``` +# ./.libs/icmpr-epoll +``` +> Make sure to run the binary file from ./.libs. File ./icmp\_responder in libmemif root directory is script that links the library, so it only verifies successful build. Default install path is /usr/lib. +Use _help_ command to display build information and commands: +``` +ICMP_Responder:add_epoll_fd:204: fd 0 added to epoll +MEMIF_DEBUG:src/main.c:memif_init:383: app name: ICMP_Responder +ICMP_Responder:add_epoll_fd:204: fd 4 added to epoll +LIBMEMIF EXAMPLE APP: ICMP_Responder (debug) +============================== +libmemif version: 1.0 (debug) +memif version: 256 +commands: + help - prints this help + exit - exit app + conn - create memif (slave-mode) + del - delete memif + show - show connection details + ip-set - set interface ip address + rx-mode - set queue rx mode +``` + +#### Examples + +Once the library is built/installed, refer to @ref libmemif_examples_doc and @ref libmemif_gettingstarted_doc for additional information on basic use cases and API usage. diff --git a/extras/libmemif/docs/gettingstarted_doc.md b/extras/libmemif/docs/gettingstarted_doc.md new file mode 100644 index 00000000..e3ae6e56 --- /dev/null +++ b/extras/libmemif/docs/gettingstarted_doc.md @@ -0,0 +1,219 @@ +## Getting started {#libmemif_gettingstarted_doc} + +#### Concept (Connecting to VPP) + +For detailed information on api calls and structures please refer to @ref libmemif.h. + +1. Initialize memif + - Declare callback function handling file descriptor event polling. +```C +int +control_fd_update (int fd, uint8_t events) +{ +... +} +``` + - Call memif initialization function. memif\_init +```C +err = memif_init (control_fd_update, APP_NAME); +``` + +> If event occurres on any file descriptor returned by this callback, call memif\_control\_fd\_handler function. +```C +memif_err = memif_control_fd_handler (evt.data.fd, events); +``` +> If callback function parameter for memif\_init function is set to NULL, libmemif will handle file descriptor event polling. + Api call memif\_poll\_event will call epoll\_pwait with user defined timeout to poll event on file descriptors opened by libmemif. +```C +/* main loop */ + while (1) + { + if (memif_poll_event (-1) < 0) + { + DBG ("poll_event error!"); + } + } +``` + +> Memif initialization function will initialize internal structures and create timer file descriptor, which will be used for sending periodic connection requests. Timer is disarmed if no memif interface is created. + +2. Creating interface + - Declare memif connction handle. +```C +memif_conn_handle_t c; +``` +> example app uses struct that contains connection handle, rx/tx buffers and other connection specific information. + + - Specify connection arguments. +```C +memif_conn_args_t args; +memset (&args, 0, sizeof (args)); +args.is_master = is_master; +args.log2_ring_size = 10; +args.buffer_size = 2048; +args.num_s2m_rings = 2; +args.num_m2s_rings = 2; +strncpy ((char *) args.interface_name, IF_NAME, strlen (IF_NAME)); +strncpy ((char *) args.instance_name, APP_NAME, strlen (APP_NAME)); +args.mode = 0; +args.interface_id = 0; +``` + - Declare callback functions called on connected/disconnected/interrupted status changed. +```C +int +on_connect (memif_conn_handle_t conn, void *private_ctx) +{ +... +} + +int +on_disconnect (memif_conn_handle_t conn, void *private_ctx) +{ + INFO ("memif connected!"); + return 0; +} +``` + - Call memif interface create function. memif\_create +```C +err = memif_create (&c->conn, + &args, on_connect, on_disconnect, on_interrupt, &ctx[index]); +``` +> If connection is in slave mode, arms timer file descriptor. +> If on interrupt callback is set to NULL, user will not be notified about interrupt. Use memif\_get\_queue\_efd call to get interrupt file descriptor for specific queue. +```C +int fd = -1; +err = memif_get_queue_efd (c->conn, data->qid, &fd); +``` + +3. Connection establishment + - User application will poll events on all file descriptors returned in memif\_control\_fd\_update\_t callback. + - On event call memif\_control\_fd\_handler. + - Everything else regarding connection establishment will be done internally. + - Once connection has been established, a callback will inform the user about connection status change. + +4. Interrupt packet receive + - If event is polled on interrupt file descriptor, libmemif will call memif\_interrupt\_t callback specified for every connection instance. +```C +int +on_interrupt (memif_conn_handle_t conn, void *private_ctx, uint16_t qid) +{ +... +} +``` + +6. Memif buffers + - Packet data are stored in memif\_buffer\_t. Pointer _data_ points to shared memory buffer, and unsigned integer *data\_len* contains packet data length. +```C +typedef struct +{ + uint16_t desc_index; + uint32_t buffer_len; + uint32_t data_len; + void *data; +} memif_buffer_t; +``` + +5. Packet receive + - Api call memif\_rx\_burst will set all required fields in memif buffers provided by user application. +```C +err = memif_rx_burst (c->conn, qid, c->rx_bufs, MAX_MEMIF_BUFS, &rx); +``` + - User application can then process packets. + - Api call memif\_buffer\_free will make supplied memif buffers ready for next receive and mark shared memory buffers as free. +```C +err = memif_buffer_free (c->conn, qid, c->rx_bufs, rx, &fb); +``` + +6. Packet transmit + - Api call memif\_buffer\_alloc will set all required fields in memif buffers provided by user application. +```C +err = memif_buffer_alloc (c->conn, qid, c->tx_bufs, n, &r); +``` + - User application can populate shared memory buffers with packets. + - Api call memif\_tx\_burst will inform peer interface (master memif on VPP) that there are packets ready to receive and mark memif buffers as free. +```C +err = memif_tx_burst (c->conn, qid, c->tx_bufs, c->tx_buf_num, &r); +``` + +7. Helper functions + - Memif details + - Api call memif\_get\_details will return details about connection. +```C +err = memif_get_details (c->conn, &md, buf, buflen); +``` + - Memif error messages + - Every api call returns error code (integer value) mapped to error string. + - Call memif\_strerror will return error message assigned to specific error code. +```C +if (err != MEMIF_ERR_SUCCESS) + INFO ("memif_get_details: %s", memif_strerror (err)); +``` + - Not all syscall errors are translated to memif error codes. If error code 1 (MEMIF\_ERR\_SYSCALL) is returned then libmemif needs to be compiled with -DMEMIF_DBG flag to print error message. Use _make -B_ to rebuild libmemif in debug mode. + +#### Example app (libmemif fd event polling): + +- @ref extras/libmemif/examples/icmp_responder + +> Optional argument: transmit queue id. +``` +icmpr 1 +``` +> Set transmit queue id to 1. Default is 0. +> Application will create memif interface in slave mode and try to connect to VPP. Exit using Ctrl+C. Application will handle SIGINT signal, free allocated memory and exit with EXIT_SUCCESS. + +#### Example app: + +ICMP Responder custom fd event polling. + +- @ref extras/libmemif/examples/icmp_responder-epoll + +#### Example app (multi-thread queue polling) + +ICMP Responder multi-thread. +- @ref extras/libmemif/examples/icmp_responder-mt + +> Simple example of libmemif multi-thread usage. Connection establishment is handled by main thread. There are two rx queues in this example. One in polling mode and second in interrupt mode. + +VPP config: +``` +# create memif id 0 master +# set int state memif0 up +# set int ip address memif0 192.168.1.1/24 +# ping 192.168.1.2 +``` +For multiple rings (queues) support run VPP with worker threads: +example startup.conf: +``` +unix { + interactive + nodaemon + full-coredump +} + +cpu { + workers 2 +} +``` +VPP config: +``` +# create memif id 0 master +# set int state memif0 up +# set int ip address memif0 192.168.1.1/24 +# ping 192.168.1.2 +``` +> Master mode queue number is limited by worker threads. Slave mode interface needs to specify number of queues. +``` +# create memif id 0 slave rx-queues 2 tx-queues 2 +``` +> Example applications use VPP default socket file for memif: /run/vpp/memif.sock +> For master mode, socket directory must exist prior to memif\_create call. + +#### Unit tests + +Unit tests use [Check](https://libcheck.github.io/check/index.html) framework. This framework must be installed in order to build *unit\_test* binary. +Ubuntu/Debian: +``` +sudo apt-get install check +``` +[More platforms](https://libcheck.github.io/check/web/install.html) + diff --git a/extras/libmemif/examples/ExampleSetup.md b/extras/libmemif/examples/ExampleSetup.md deleted file mode 100644 index a02506dd..00000000 --- a/extras/libmemif/examples/ExampleSetup.md +++ /dev/null @@ -1,206 +0,0 @@ -## Example setup - -#### VPP-memif master icmp_responder slave - -> Libmemif example app(s) use memif default socket file: /run/vpp/memif.sock. - -Run VPP and icmpr-epoll example (default example when running in container). -> Other examples work similar to icmpr-epoll. Brief explanation can be found in [Examples readme](README.md) file. - -VPP-side config: -``` -DBGvpp# create memif id 0 master -DBGvpp# set int state memif0/0 up -DBGvpp# set int ip address memif0/0 192.168.1.1/24 -``` -icmpr-epoll: -``` -conn 0 0 -``` -Memif in slave mode will try to connect every 2 seconds. If connection establishment is successfull, a message will show. -``` -INFO: memif connected! -``` -> Error messages like "unmatched interface id" are printed only in debug mode. - -Check connected status. -Use show command in icmpr-epoll: -``` -show -MEMIF DETAILS -============================== -interface index: 0 - interface ip: 192.168.1.2 - interface name: memif_connection - app name: ICMP_Responder - remote interface name: memif0/0 - remote app name: VPP 17.10-rc0~132-g62f9cdd - id: 0 - secret: - role: slave - mode: ethernet - socket filename: /run/vpp/memif.sock - rx queues: - queue id: 0 - ring size: 1024 - buffer size: 2048 - tx queues: - queue id: 0 - ring size: 1024 - buffer size: 2048 - link: up -interface index: 1 - no connection - -``` -Use sh memif command in VPP: -``` -DBGvpp# sh memif -interface memif0/0 - remote-name "ICMP_Responder" - remote-interface "memif_connection" - id 0 mode ethernet file /run/vpp/memif.sock - flags admin-up connected - listener-fd 12 conn-fd 13 - num-s2m-rings 1 num-m2s-rings 1 buffer-size 0 - master-to-slave ring 0: - region 0 offset 32896 ring-size 1024 int-fd 16 - head 0 tail 0 flags 0x0000 interrupts 0 - master-to-slave ring 0: - region 0 offset 0 ring-size 1024 int-fd 15 - head 0 tail 0 flags 0x0001 interrupts 0 -``` - -Send ping from VPP to icmpr-epoll: -``` -DBGvpp# ping 192.168.1.2 -64 bytes from 192.168.1.2: icmp_seq=2 ttl=64 time=.1888 ms -64 bytes from 192.168.1.2: icmp_seq=3 ttl=64 time=.1985 ms -64 bytes from 192.168.1.2: icmp_seq=4 ttl=64 time=.1813 ms -64 bytes from 192.168.1.2: icmp_seq=5 ttl=64 time=.1929 ms - -Statistics: 5 sent, 4 received, 20% packet loss -``` -#### multiple queues VPP-memif slave icmp_responder master - -Run icmpr-epoll as in previous example setup. -Run VPP with startup conf, enabling 2 worker threads. -Example startup.conf: -``` -unix { - interactive - nodaemon - full-coredump -} - -cpu { - workers 2 -} -``` -VPP-side config: -``` -DBGvpp# create memif id 0 slave rx-queues 2 tx-queues 2 -DBGvpp# set int state memif0/0 up -DBGvpp# set int ip address memif0/0 192.168.1.1/24 -``` -icmpr-epoll: -``` -conn 0 1 -``` -When connection is established a message will print: -``` -INFO: memif connected! -``` -> Error messages like "unmatched interface id" are printed only in debug mode. - -Check connected status. -Use show command in icmpr-epoll: -``` -show -MEMIF DETAILS -============================== -interface index: 0 - interface ip: 192.168.1.2 - interface name: memif_connection - app name: ICMP_Responder - remote interface name: memif0/0 - remote app name: VPP 17.10-rc0~132-g62f9cdd - id: 0 - secret: - role: master - mode: ethernet - socket filename: /run/vpp/memif.sock - rx queues: - queue id: 0 - ring size: 1024 - buffer size: 2048 - queue id: 1 - ring size: 1024 - buffer size: 2048 - tx queues: - queue id: 0 - ring size: 1024 - buffer size: 2048 - queue id: 1 - ring size: 1024 - buffer size: 2048 - link: up -interface index: 1 - no connection - -``` -Use sh memif command in VPP: -``` -DBGvpp# sh memif -interface memif0/0 - remote-name "ICMP_Responder" - remote-interface "memif_connection" - id 0 mode ethernet file /run/vpp/memif.sock - flags admin-up slave connected - listener-fd -1 conn-fd 12 - num-s2m-rings 2 num-m2s-rings 2 buffer-size 2048 - slave-to-master ring 0: - region 0 offset 0 ring-size 1024 int-fd 14 - head 0 tail 0 flags 0x0000 interrupts 0 - slave-to-master ring 1: - region 0 offset 32896 ring-size 1024 int-fd 15 - head 0 tail 0 flags 0x0000 interrupts 0 - slave-to-master ring 0: - region 0 offset 65792 ring-size 1024 int-fd 16 - head 0 tail 0 flags 0x0001 interrupts 0 - slave-to-master ring 1: - region 0 offset 98688 ring-size 1024 int-fd 17 - head 0 tail 0 flags 0x0001 interrupts 0 - -``` -Send ping from VPP to icmpr-epoll: -``` -DBGvpp# ping 192.168.1.2 -64 bytes from 192.168.1.2: icmp_seq=2 ttl=64 time=.1439 ms -64 bytes from 192.168.1.2: icmp_seq=3 ttl=64 time=.2184 ms -64 bytes from 192.168.1.2: icmp_seq=4 ttl=64 time=.1458 ms -64 bytes from 192.168.1.2: icmp_seq=5 ttl=64 time=.1687 ms - -Statistics: 5 sent, 4 received, 20% packet loss -``` - -#### icmp_responder master icmp_responder slave - -> Example apps can only repond to ping. This setup creates connection between two applications using libmemif. Traffic functionality is the same as when connection to VPP. App can receive ARP/ICMP request and transmit response, but can not send ARP/ICMP request. - -Run two instances of icmpr-epoll example. -> If not running in container, make sure folder /run/vpp/ exists before creating memif master. -Instance 1 will be in master mode, instance 2 in slave mode. -instance 1: -``` -conn 0 1 -``` -instance 2: -``` -conn 0 0 -``` -In 2 seconds, both instances should print connected! message: -``` -INFO: memif connected! -``` -Check peer interface names using show command. diff --git a/extras/libmemif/examples/README.md b/extras/libmemif/examples/README.md deleted file mode 100644 index 1375d27a..00000000 --- a/extras/libmemif/examples/README.md +++ /dev/null @@ -1,16 +0,0 @@ -## Examples - -After build, root folder will contain scripts linking binary examples with library (same name as example apps). These scripts can be executed to run example apps without installing the library. Example apps binaries can be found in _libs_ filder. To run binaries directly, make sure that libmemif library is installed. - -#### Run in container -ligato/libmemif-sample-service image contains built and installed libmemf. To run different examples, override docker CMD to start container in bash: -``` -# docker run -it --entrypoint=/bin/bash -i --rm --name icmp-responder --hostname icmp-responder --privileged -v "/run/vpp/:/run/vpp/" ligato/libmemif-sample-service -``` -Current WORKDIR is set to root repository directory. Example apps can be run from this directory (a script linking binary with library), or browse to ./.libs folder and execute binary directly. - -Example app | Description -------------|------------ -[icmpr](../examples/icmp_responder/main.c) | Simplest implementaion. Event polling is handled by libmemif. Single memif conenction in slave mode is created (id 0). Use Ctrl + C to exit app. Memif receive mode: interrupt. -[icmpr-epoll](../examples/icmp_responder-epoll/main.c) (run in container by default) | Supports multiple connections and master mode. User can create/delete connections, set ip addresses, print connection information. [Example setup](ExampleSetup.md) contains instructions on basic connection use cases setups. Memif receive mode: interrupt. App provides functionality to disable interrupts for specified queue/s for testing purposes. Polling mode is not implemented in this example. -[icmpr-mt](../examples/icmp_responder-mt/main.c) | Multi-thread example, very similar to icmpr-epoll. Packets are handled in threads assigned to specific queues. Slave mode only. Memif receive mode: polling (memif_rx_poll function), interrupt (memif_rx_interrupt function). Receive modes differ per queue. diff --git a/extras/libmemif/examples/example_setup_doc.md b/extras/libmemif/examples/example_setup_doc.md new file mode 100644 index 00000000..89a0b10e --- /dev/null +++ b/extras/libmemif/examples/example_setup_doc.md @@ -0,0 +1,207 @@ +## Example setup {#libmemif_example_setup_doc} + +#### VPP-memif master icmp_responder slave + +> Libmemif example app(s) use memif default socket file: `/run/vpp/memif.sock`. + +Run VPP and icmpr-epoll example (default example when running in container). + +> Other examples work similar to icmpr-epoll. Brief explanation can be found in @ref libmemif_examples_doc . + +VPP-side config: +``` +DBGvpp# create memif id 0 master +DBGvpp# set int state memif0/0 up +DBGvpp# set int ip address memif0/0 192.168.1.1/24 +``` +icmpr-epoll: +``` +conn 0 0 +``` +Memif in slave mode will try to connect every 2 seconds. If connection establishment is successfull, a message will show. +``` +INFO: memif connected! +``` +> Error messages like "unmatched interface id" are printed only in debug mode. + +Check connected status. +Use show command in icmpr-epoll: +``` +show +MEMIF DETAILS +============================== +interface index: 0 + interface ip: 192.168.1.2 + interface name: memif_connection + app name: ICMP_Responder + remote interface name: memif0/0 + remote app name: VPP 17.10-rc0~132-g62f9cdd + id: 0 + secret: + role: slave + mode: ethernet + socket filename: /run/vpp/memif.sock + rx queues: + queue id: 0 + ring size: 1024 + buffer size: 2048 + tx queues: + queue id: 0 + ring size: 1024 + buffer size: 2048 + link: up +interface index: 1 + no connection + +``` +Use sh memif command in VPP: +``` +DBGvpp# sh memif +interface memif0/0 + remote-name "ICMP_Responder" + remote-interface "memif_connection" + id 0 mode ethernet file /run/vpp/memif.sock + flags admin-up connected + listener-fd 12 conn-fd 13 + num-s2m-rings 1 num-m2s-rings 1 buffer-size 0 + master-to-slave ring 0: + region 0 offset 32896 ring-size 1024 int-fd 16 + head 0 tail 0 flags 0x0000 interrupts 0 + master-to-slave ring 0: + region 0 offset 0 ring-size 1024 int-fd 15 + head 0 tail 0 flags 0x0001 interrupts 0 +``` + +Send ping from VPP to icmpr-epoll: +``` +DBGvpp# ping 192.168.1.2 +64 bytes from 192.168.1.2: icmp_seq=2 ttl=64 time=.1888 ms +64 bytes from 192.168.1.2: icmp_seq=3 ttl=64 time=.1985 ms +64 bytes from 192.168.1.2: icmp_seq=4 ttl=64 time=.1813 ms +64 bytes from 192.168.1.2: icmp_seq=5 ttl=64 time=.1929 ms + +Statistics: 5 sent, 4 received, 20% packet loss +``` +#### multiple queues VPP-memif slave icmp_responder master + +Run icmpr-epoll as in previous example setup. +Run VPP with startup conf, enabling 2 worker threads. +Example startup.conf: +``` +unix { + interactive + nodaemon + full-coredump +} + +cpu { + workers 2 +} +``` +VPP-side config: +``` +DBGvpp# create memif id 0 slave rx-queues 2 tx-queues 2 +DBGvpp# set int state memif0/0 up +DBGvpp# set int ip address memif0/0 192.168.1.1/24 +``` +icmpr-epoll: +``` +conn 0 1 +``` +When connection is established a message will print: +``` +INFO: memif connected! +``` +> Error messages like "unmatched interface id" are printed only in debug mode. + +Check connected status. +Use show command in icmpr-epoll: +``` +show +MEMIF DETAILS +============================== +interface index: 0 + interface ip: 192.168.1.2 + interface name: memif_connection + app name: ICMP_Responder + remote interface name: memif0/0 + remote app name: VPP 17.10-rc0~132-g62f9cdd + id: 0 + secret: + role: master + mode: ethernet + socket filename: /run/vpp/memif.sock + rx queues: + queue id: 0 + ring size: 1024 + buffer size: 2048 + queue id: 1 + ring size: 1024 + buffer size: 2048 + tx queues: + queue id: 0 + ring size: 1024 + buffer size: 2048 + queue id: 1 + ring size: 1024 + buffer size: 2048 + link: up +interface index: 1 + no connection + +``` +Use sh memif command in VPP: +``` +DBGvpp# sh memif +interface memif0/0 + remote-name "ICMP_Responder" + remote-interface "memif_connection" + id 0 mode ethernet file /run/vpp/memif.sock + flags admin-up slave connected + listener-fd -1 conn-fd 12 + num-s2m-rings 2 num-m2s-rings 2 buffer-size 2048 + slave-to-master ring 0: + region 0 offset 0 ring-size 1024 int-fd 14 + head 0 tail 0 flags 0x0000 interrupts 0 + slave-to-master ring 1: + region 0 offset 32896 ring-size 1024 int-fd 15 + head 0 tail 0 flags 0x0000 interrupts 0 + slave-to-master ring 0: + region 0 offset 65792 ring-size 1024 int-fd 16 + head 0 tail 0 flags 0x0001 interrupts 0 + slave-to-master ring 1: + region 0 offset 98688 ring-size 1024 int-fd 17 + head 0 tail 0 flags 0x0001 interrupts 0 + +``` +Send ping from VPP to icmpr-epoll: +``` +DBGvpp# ping 192.168.1.2 +64 bytes from 192.168.1.2: icmp_seq=2 ttl=64 time=.1439 ms +64 bytes from 192.168.1.2: icmp_seq=3 ttl=64 time=.2184 ms +64 bytes from 192.168.1.2: icmp_seq=4 ttl=64 time=.1458 ms +64 bytes from 192.168.1.2: icmp_seq=5 ttl=64 time=.1687 ms + +Statistics: 5 sent, 4 received, 20% packet loss +``` + +#### icmp_responder master icmp_responder slave + +> Example apps can only repond to ping. This setup creates connection between two applications using libmemif. Traffic functionality is the same as when connection to VPP. App can receive ARP/ICMP request and transmit response, but can not send ARP/ICMP request. + +Run two instances of icmpr-epoll example. +> If not running in container, make sure folder /run/vpp/ exists before creating memif master. +Instance 1 will be in master mode, instance 2 in slave mode. +instance 1: +``` +conn 0 1 +``` +instance 2: +``` +conn 0 0 +``` +In 2 seconds, both instances should print connected! message: +``` +INFO: memif connected! +``` +Check peer interface names using show command. diff --git a/extras/libmemif/examples/examples_doc.md b/extras/libmemif/examples/examples_doc.md new file mode 100644 index 00000000..11937c57 --- /dev/null +++ b/extras/libmemif/examples/examples_doc.md @@ -0,0 +1,18 @@ +## Examples {#libmemif_examples_doc} + +After build, root folder will contain scripts linking binary examples with library (same name as example apps). These scripts can be executed to run example apps without installing the library. Example apps binaries can be found in _libs_ filder. To run binaries directly, make sure that libmemif library is installed. + +#### Run in container + +`ligato/libmemif-sample-service` image contains built and installed libmemf. To run different examples, override docker CMD to start container in bash: + +``` +# docker run -it --entrypoint=/bin/bash -i --rm --name icmp-responder --hostname icmp-responder --privileged -v "/run/vpp/:/run/vpp/" ligato/libmemif-sample-service +``` +Current WORKDIR is set to root repository directory. Example apps can be run from this directory (a script linking binary with library), or browse to `./.libs` folder and execute binary directly. + +Example app | Description +------------|------------ +@ref extras/libmemif/examples/icmp_responder | Simplest implementaion. Event polling is handled by libmemif. Single memif conenction in slave mode is created (id 0). Use Ctrl + C to exit app. Memif receive mode: interrupt. +@ref extras/libmemif/examples/icmp_responder-epoll (run in container by default) | Supports multiple connections and master mode. User can create/delete connections, set ip addresses, print connection information. @ref libmemif_example_setup_doc contains instructions on basic connection use cases setups. Memif receive mode: interrupt. App provides functionality to disable interrupts for specified queue/s for testing purposes. Polling mode is not implemented in this example. +@ref extras/libmemif/examples/icmp_responder-mt) | Multi-thread example, very similar to icmpr-epoll. Packets are handled in threads assigned to specific queues. Slave mode only. Memif receive mode: polling (memif_rx_poll function), interrupt (memif_rx_interrupt function). Receive modes differ per queue. diff --git a/extras/libmemif/libmemif_doc.md b/extras/libmemif/libmemif_doc.md new file mode 100644 index 00000000..fb4a18af --- /dev/null +++ b/extras/libmemif/libmemif_doc.md @@ -0,0 +1,74 @@ +Shared Memory Packet Interface (memif) Library {#libmemif_doc} +============================================== + +## Introduction + +Shared memory packet interface (memif) provides high performance packet transmit and receive between user application and Vector Packet Processing (VPP) or multiple user applications. Using libmemif, user application can create shared memory interface in master or slave mode and connect to VPP or another application using libmemif. Once the connection is established, user application can receive or transmit packets using libmemif API. + +![Architecture](docs/architecture.png) + +## Features + +- [x] Slave mode + - [x] Connect to VPP over memif + - [x] ICMP responder example app +- [x] Transmit/receive packets +- [x] Interrupt mode support +- [x] File descriptor event polling in libmemif (optional) + - [x] Simplify file descriptor event polling (one handler for control and interrupt channel) +- [x] Multiple connections +- [x] Multiple queues + - [x] Multi-thread support +- [x] Master mode + - [ ] Multiple regions (TODO) +- [ ] Performance testing (TODO) + +## Quickstart + +This setup will run libmemif ICMP responder example app in container. Install [docker](https://docs.docker.com/engine/installation) engine. +Useful link: [Docker documentation](https://docs.docker.com/get-started). + +Pull image: +``` +# docker pull ligato/libmemif-sample-service +``` + +Now you should be able to see ligato/libmemif-sample-service image on your local machine (IMAGE ID in this README may be outdated): +``` +# docker images +REPOSITORY TAG IMAGE ID CREATED SIZE +ligato/libmemif-sample-service latest 32ecc2f9d013 About a minute ago 468MB +... +``` + +Run container: +``` +# docker run -it --rm --name icmp-responder --hostname icmp-responder --privileged -v "/run/vpp/:/run/vpp/" ligato/libmemif-sample-service +``` +Example application will start in debug mode. Output should look like this: +``` +ICMP_Responder:add_epoll_fd:204: fd 0 added to epoll +MEMIF_DEBUG:src/main.c:memif_init:383: app name: ICMP_Responder +ICMP_Responder:add_epoll_fd:204: fd 4 added to epoll +LIBMEMIF EXAMPLE APP: ICMP_Responder (debug) +============================== +libmemif version: 1.0 (debug) +memif version: 256 +commands: + help - prints this help + exit - exit app + conn - create memif (slave-mode) + del - delete memif + show - show connection details + ip-set - set interface ip address + rx-mode - set queue rx mode +``` + +Continue with @ref libmemif_example_setup which contains instructions on how to set up conenction between icmpr-epoll example app and VPP-memif. + +#### Next steps + +- @subpage libmemif_build_doc +- @subpage libmemif_examples_doc +- @subpage libmemif_example_setup_doc +- @subpage libmemif_gettingstarted_doc -- cgit 1.2.3-korg