summaryrefslogtreecommitdiffstats
path: root/extras/libmemif/docs/gettingstarted_doc.rst
blob: f576fe251452d60ca902f6263f91f84155c341d4 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
.. _libmemif_gettingstarted_doc:

Getting started
===============

For detailed information on api calls and structures please refer to
``libmemif.h``.

Start by creating a memif socket. Memif socket represents UNIX domain
socket and interfaces assigned to use this socket. Memif uses UNIX domain
socket to communicate with other memif drivers.

First fill out the ``memif_socket_args`` struct. The minimum required
configuration is the UNIX socket path. > Use ``@`` or ``\0`` at the
beginning of the path to use abstract socket.

.. code:: c

   memif_socket_args_t sargs;

   strncpy(sargs.path, socket_path, sizeof(sargs.path));

.. code:: c

   memif_socket_handle_t memif_socket;

   memif_create_socket(&memif_socket, &sargs, &private_data);

Once you have created your socket, you can create memif interfaces on
this socket. Fill out the ``memif_conn_args`` struct. Then call
``memif_create()``.

.. code:: c

   memif_conn_args_t cargs;

   /* Assign your socket handle */
   cargs.socket = memif_socket;

.. code:: c

   memif_conn_handle_t conn;

   /* Assign callbacks */
   memif_create (&conn, &cargs, on_connect_cb, on_disconnect_cb, on_interrupt_cb, &private_data);

Now start the polling events using libmemifs builtin polling.

.. code:: c

   do {
       err = memif_poll_event(memif_socket, /* timeout -1 = blocking */ -1);
   } while (err == MEMIF_ERR_SUCCESS);

Polling can be canceled by calling ``memif_cancel_poll_event()``.

.. code:: c

   memif_cancel_poll_event (memif_socket);

On link status change ``on_connect`` and ``on_disconnect`` callbacks are
called respectively. Before you can start transmitting data you, first
need to call ``memif_refill_queue()`` for each RX queue to initialize
this queue.

.. code:: c

   int on_connect (memif_conn_handle_t conn, void *private_ctx)
   {
     my_private_data_t *data = (my_private_data_t *) private_ctx;

     err = memif_refill_queue(conn, 0, -1, 0);
     if (err != MEMIF_ERR_SUCCESS) {
       INFO("memif_refill_queue: %s", memif_strerror(err));
       return err;
     }

     /*
      * Do stuff.
      */

     return 0;
   }

Now you are ready to transmit packets. > Example implementation
``examples/common/sender.c`` and ``examples/common/responder.c``

To transmit or receive data you will need to use ``memif_buffer``
struct. The important fields here are ``void *data``, ``uint32_t len``
and ``uint8_t flags``. The ``data`` pointer points directly to the
shared memory packet buffer. This is where you will find/insert your
packets. The ``len`` field is the length of the buffer. If the flag
``MEMIF_BUFFER_FLAG_NEXT`` is present in ``flags`` field, this buffer is
chained so the rest of the data is located in the next buffer, and so
on.

First let’s receive data. To receive data call ``memif_rx_burst()``. The
function will fill out memif buffers passed to it. Then you would
process your data (e.g. copy to your stack). Last you must refill the
queue using ``memif_refill_queue()`` to notify peer that the buffers are
now free and can be overwritten.

.. code:: c

   /* Fill out memif buffers and mark them as received */
   err = memif_rx_burst(conn, qid, buffers, num_buffers, &num_received);
   if (err != MEMIF_ERR_SUCCESS) {
       INFO ("memif_rx_burst: %s", memif_strerror(err));
       return err;
   }
   /*
       Process the buffers.
   */

   /* Refill the queue, so that the peer interface can transmit more packets */
   err = memif_refill_queue(conn, qid, num_received, 0);
   if (err != MEMIF_ERR_SUCCESS) {
       INFO("memif_refill_queue: %s", memif_strerror(err));
       goto error;
   }

In order to transmit data you first need to ‘allocate’ memif buffers
using ``memif_buffer_alloc()``. This function similar to
``memif_rx_burst`` will fill out provided memif buffers. You will then
insert your packets directly into the shared memory (don’t forget to
update ``len`` filed if your packet is smaller that buffer length).
Finally call ``memif_tx_burst`` to transmit the buffers.

.. code:: c

   /* Alocate memif buffers */
   err = memif_buffer_alloc(conn, qid, buffers, num_pkts, &num_allocated, packet_size);
   if (err != MEMIF_ERR_SUCCESS) {
       INFO("memif_buffer_alloc: %s", memif_strerror(err));
       goto error;
   }

   /*
       Fill out the buffers.

       tx_buffers[i].data field points to the shared memory.
       update tx_buffers[i].len to your packet length, if the packet is smaller.
   */

   /* Transmit the buffers */
   err = memif_tx_burst(conn, qid, buffers, num_allocated, &num_transmitted);
   if (err != MEMIF_ERR_SUCCESS) {
       INFO("memif_tx_burst: %s", memif_strerror(err));
       goto error;
   }

Zero-copy Slave
---------------

Interface with slave role is the buffer producer, as such it can use
zero-copy mode.

After receiving buffers, process your packets in place. Then use
``memif_buffer_enq_tx()`` to enqueue rx buffers to tx queue (by swapping
rx buffer with a free tx buffer).

.. code:: c

   /* Fill out memif buffers and mark them as received */
   err = memif_rx_burst(conn, qid, buffers, num_buffers, &num_received);
   if (err != MEMIF_ERR_SUCCESS) {
       INFO ("memif_rx_burst: %s", memif_strerror(err));
       return err;
   }

   /*
       Process the buffers in place.
   */

   /* Enqueue processed buffers to tx queue */
   err = memif_buffer_enq_tx(conn, qid, buffers, num_buffers, &num_enqueued);
   if (err != MEMIF_ERR_SUCCESS) {
       INFO("memif_buffer_alloc: %s", memif_strerror(err));
       goto error;
   }

   /* Refill the queue, so that the peer interface can transmit more packets */
   err = memif_refill_queue(conn, qid, num_enqueued, 0);
   if (err != MEMIF_ERR_SUCCESS) {
       INFO("memif_refill_queue: %s", memif_strerror(err));
       goto error;
   }

   /* Transmit the buffers. */
   err = memif_tx_burst(conn, qid, buffers, num_enqueued, &num_transmitted);
   if (err != MEMIF_ERR_SUCCESS) {
       INFO("memif_tx_burst: %s", memif_strerror(err));
       goto error;
   }

Custom Event Polling
--------------------

Libmemif can be integrated into your applications fd event polling. You
will need to implement ``memif_control_fd_update_t`` callback and pass
it to ``memif_socket_args.on_control_fd_update``. Now each time any file
descriptor belonging to that socket updates, ``on_control_fd_update``
callback is called. The file descriptor and event type is passed in
``memif_fd_event_t``. It also contains private context that is
associated with this fd. When event is polled on the fd you need to call
``memif_control_fd_handler`` and pass the event type and private context
associated with the fd.

Multi Threading
---------------

Connection establishment
~~~~~~~~~~~~~~~~~~~~~~~~

Memif sockets should not be handled in parallel. Instead each thread
should have it’s own socket. However the UNIX socket can be the same. In
case of non-listener socket, it’s straight forward, just create the
socket using the same path. In case of listener socket, the polling
should be done by single thread. > The socket becomes listener once a
Master interface is assigned to it.

Packet handling
~~~~~~~~~~~~~~~

Single queue must not be handled in parallel. Instead you can assign
queues to threads in such way that each queue is only assigned single
thread.

Shared Memory Layout
--------------------

Please refer to `DPDK MEMIF
documentation <http://doc.dpdk.org/guides/nics/memif.html>`__
``'Shared memory'`` section.