1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
|
# AF_XDP Ethernet driver {#af_xdp_doc}
This driver relies on Linux AF_XDP socket to rx/tx Ethernet packets.
## Maturity level
Under development: it should work, but has not been thoroughly tested.
## Features
- copy and zero-copy mode
- multiqueue
- API
- custom eBPF program
- polling, interrupt and adaptive mode
## Limitations
Because of AF_XDP restrictions, the MTU is limited to below PAGE_SIZE
(4096-bytes on most systems) minus 256-bytes, and they are additional
limitations depending upon specific Linux device drivers.
As a rule of thumb, a MTU of 3000-bytes or less should be safe.
## Requirements
The Linux kernel interface must be up and have enough queues before
creating the VPP AF_XDP interface, otherwise Linux will deny creating
the AF_XDP socket.
The AF_XDP interface will claim NIC RX queue starting from 0, up to the
requested number of RX queues (only 1 by default). It means all packets
destined to NIC RX queue `[0, num_rx_queues[` will be received by the
AF_XDP interface, and only them. Depending on your configuration, there
will usually be several RX queues (typically 1 per core) and packets are
spread accross queues by RSS. In order to receive consistent traffic,
you **must** program the NIC dispatching accordingly. The simplest way
to get all the packets is to reconfigure the Linux kernel driver to use
only `num_rx_queues` RX queues (ie all NIC queues will be associated
with the AF_XDP socket):
```
~# ethtool -L <iface> combined <num_rx_queues>
```
Additionally, the VPP AF_XDP interface will use a MAC address generated at
creation time instead of the Linux kernel interface MAC. As Linux kernel
interface are not in promiscuous mode by default (see below) this will
results in a useless configuration where the VPP AF_XDP interface only
receives packets destined to the Linux kernel interface MAC just to drop
them because the destination MAC does not match VPP AF_XDP interface MAC.
If you want to use the Linux interface MAC for the VPP AF_XDP interface,
you can change it afterwards in VPP:
```
~# vppctl set int mac address <iface> <mac>
```
Finally, if you wish to receive all packets and not only the packets
destined to the Linux kernel interface MAC you need to set the Linux
kernel interface in promiscuous mode:
```
~# ip link set dev <iface> promisc on
```
## Security considerations
When creating an AF_XDP interface, it will receive all packets arriving
to the NIC RX queue #0. You need to configure the Linux kernel NIC
driver properly to ensure that only intented packets will arrive in
this queue. There is no way to filter the packets after-the-fact using
eg. netfilter or eBPF.
## Quickstart
1. Setup the Linux kernel interface (enp216s0f0 here) to use 4 queues:
```
~# ethtool -L enp216s0f0 combined 4
```
2. Put the Linux kernel interface up and in promiscuous mode:
```
~# ip l set dev enp216s0f0 promisc on up
```
3. Create the AF_XDP interface:
```
~# vppctl create int af_xdp host-if enp216s0f0 num-rx-queues 4
```
4. Use the interface as usual, eg.:
```
~# vppctl set int ip addr enp216s0f0/0 1.1.1.1/24
~# vppctl set int st enp216s0f0/0 up
~# vppctl ping 1.1.1.100`
```
## Custom eBPF XDP program
This driver relies on libbpf and as such relies on the `xsks_map` eBPF
map. The default behavior is to use the XDP program already attached
to the interface if any, otherwise load the default one.
You can request to load a custom XDP program with the `prog` option when
creating the interface in VPP:
```
~# vppctl create int af_xdp host-if enp216s0f0 num-rx-queues 4 prog extras/bpf/af_xdp.bpf.o
```
In that case it will replace any previously attached program. A custom
XDP program example is provided in `extras/bpf/`.
## Performance consideration
AF_XDP relies on the Linux kernel NIC driver to rx/tx packets. To reach
high-performance (10's MPPS), the Linux kernel NIC driver must support
zero-copy mode and its RX path must run on a dedicated core in the NUMA
where the NIC is physically connected.
|
