blob: 3fed5b6fc4921e284f5b74e5be2be42e125e610e (
plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
|
# RDMA (ibverb) Ethernet driver {#rdma_doc}
This driver relies on Linux rdma-core (libibverb) userspace poll-mode driver
to rx/tx Ethernet packets. Despite using the RDMA APIs, this is **not** about
RDMA (no Infiniband, no RoCE, no iWARP), just pure traditional Ethernet
packets.
## Maturity level
Under development: it should work, but has not been thoroughly tested.
## Supported Hardware
- Mellanox ConnectX-4
- Mellanox ConnectX-5
## Features
- bifurcation: MAC based flow steering for transparent sharing of a single
physical port between multiple virtual interfaces including Linux netdev
- multiqueue
## Security considerations
When creating a rdma interface, it will receive all packets to the MAC address
attributed to the interface plus a copy of all broadcast and multicast
traffic.
The MAC address is under the control of VPP: **the user controlling VPP can
divert all traffic of any MAC address to the VPP process, including the Linux
netdev MAC address as long as it can create a rdma interface**.
The rights to create a rdma interface are controlled by the access rights of
the `/dev/infiniband/uverbs[0-9]+`device nodes.
## Quickstart
1. Make sure the `ib_uverbs` module is loaded:
```
~# modprobe ib_uverbs
```
2. In VPP, create a new rdma virtual interface tied to the Linux netdev of the
physical port you want to use (`enp94s0f0` in this example):
```
vpp# create int rdma host-if enp94s0f0 name rdma-0
```
3. Use the interface as usual, eg.:
```
vpp# set int ip addr rdma-0 1.1.1.1/24
vpp# set int st rdma-0 up
vpp# ping 1.1.1.100`
```
## Containers support
It should work in containers as long as:
- the `ib_uverbs` module is loaded
- the device nodes `/dev/infiniband/uverbs[0-9]+` are usable from the
container (but see [security considerations](#Security considerations))
## SR-IOV VFs support
It should work on SR-IOV VFs the same way it does with PFs. Because of VFs
security containment features, make sure the MAC address of the rdma VPP
interface matches the MAC address assigned to the underlying VF.
For example:
```
host# echo 1 > /sys/class/infiniband/mlx5_0/device/sriov_numvfs
host# ip l set dev enp94s0f0 vf 0 mac 92:5d:f5:df:b1:6f spoof on trust off
host# ip l set dev enp94s0f2 up
vpp# create int rdma host-if enp94s0f2 name rdma-0
vpp# set int mac address rdma-0 92:5d:f5:df:b1:6f
```
If you plan to use L2 features such as switching, make sure the underlying
VF is configured in trusted mode and spoof-checking is disabled (of course, be
aware of the [security considerations](#Security considerations)):
```
host# ip l set dev enp94s0f0 vf 0 spoof off trust on
```
## Direct Verb mode
Direct Verb allows the driver to access the NIC HW RX/TX rings directly
instead of having to go through libibverb and suffering associated overhead.
It will be automatically selected if the adapter supports it.
|
