##odp4vpp: #Introduction: -------------- odp4vpp project aims to provide VPP with an additional vnet device based on OpenDataPlane (ODP is similar yet different from DPDK), with provisions for hardware acceleration of packet paths. It envisions three deployment scenarios: -Server + NICs -Systems on a Chip -SmartNIC with low to very high core count #Build Procedure: ---------------- The source code add support to build VPP with either odp-dpdk or modified odp-linux. Generic odp-linux is not supported as it keeps user area outside packet buffer. VPP requires the user area (used to store vlib_buffer_t) to be located just before packet data. 1)Build vpp with odp-linux: --------------------------- Build the odp linux package by compiling odp from odp git repository: git clone https://git.linaro.org/lng/odp.git This release was tested with v1.17.0.0 version and with three additional patches. Patches can be found in odp/odp-linux directory: 0001-linux-generic-pool-Return-address-range-in-pool-info.patch 0002-linux-generic-Place-userdata-before-packet-data.patch If ODP IPsec will be tested in inline mode, using dpdk/socket pktio respectively 0003-inline-HACK-in-dpdk-pktio.patch 0004-inline-HACK-in-socket-pktio.patch Follow the README steps for complete build. - default pktio cd odp ./bootstrap ./configure --prefix= make install - dpdk pktio use scripts/build-pktio-dpdk to compile odp-linux with dpdk pktio Build vpp: Follow the README steps for complete build. Set the environment variables with ODP config: export PLATFORM=odp export ODP_INST_PATH= export ODP_PLATFORM_PARAMS="-m 400" - default pktio Adjust odp_odp_libs in build-data/platforms/odp.mk: odp_odp_libs = -lodp-linux -lodphelper -lpcap - dpdk pktio export LIBRARY_PATH=/lib/ Adjust odp_odp_libs in build-data/platforms/odp.mk: odp_odp_libs = -lodp-linux -ldpdk -lodphelper -lpcap cd vpp make install-dep make build make run 2)Build vpp with odp-dpdk: -------------------------- Build the odp-dpdk package by compiling odp-dpdk from odp-dpdk git repository: git clone https://git.linaro.org/lng/odp-dpdk.git This release was tested on master branch with commit e78e28ea32d5 with three additional patches. Patches can be found in odp/odp-dpdk directory: 0001-linux-dpdk-pool-Return-address-range-in-pool-info.patch When using odp_crypto based IPsec the fixes below are necessary 0002-linux-dpdk-crypto-Change-queue-amount.patch 0003-Changed-the-digest-size-of-SHA256_HMAC.patch If ODP IPsec will be tested in inline mode, this is also needed 0004-HACK-enable-inline-for-odp-dpdk.patch Follow the odp-dpdk/README.DPDK steps for complete build with dpdk. git clone --single-branch --branch=v17.08 http://dpdk.org/git/dpdk cd dpdk make config T=x86_64-native-linuxapp-gcc O=x86_64-native-linuxapp-gcc sed -ri 's,(CONFIG_RTE_LIBRTE_PMD_OPENSSL=).*,\1y,' x86_64-native-linuxapp-gcc/.config sed -ri 's,(CONFIG_RTE_LIBRTE_PMD_PCAP=).*,\1y,' x86_64-native-linuxapp-gcc/.config make O=x86_64-native-linuxapp-gcc EXTRA_CFLAGS="-fPIC" cd odp-dpdk ./bootstrap ./configure --prefix= --with-dpdk-path=/x86_64-native-linuxapp-gcc make install Set the environment variables with ODP config: export PLATFORM=odp export ODP_INST_PATH= export LIBRARY_PATH=/x86_64-native-linuxapp-gcc/lib/ export ODP_PLATFORM_PARAMS="-m 400 --vdev crypto_openssl" Build vpp: Follow the README steps for complete build. cd odp4vpp make install-dep make build make run #Test Steps: ------------ Below is a basic verification test. Note :For odp-dpdk the port has to bound with dpdk driver prior to test and interface name is passed as 0,1..etc. 1)Configure odp packet interface with Rx/Tx mode and number of queues. Rx mode can be set to 0-burst (default), 1-queue, 2-schedule-atomic, 3-schedule-ordered, 4-schedule-parallel Tx mode can be set to 0-burst (default), 1-queue (forced for schedule-ordered) Number of Rx queues can be 1 or more. It will determine number of threads and enable RSS (default is 1). Number of Tx queues can be 0 (equal to number of threads, default) or 1 (single shared queue). -create pktio-interface name hw-addr rx-mode <0/1/2> rx-queues tx-mode <0/1> tx-queues -set int ip address odp- X.X.X.X/24 -set int state odp- up 2)Check the interface state: -sh int 3)Ping the configured interface from host machine and check the ARP and ipv4 FIB table: -sh ip arp -sh ip fib 4)Check the interface counters: -sh int 5)Check the statistics: -show run 6)Packet trace can be enabled using : -trace add odp-packet-input 10 -show trace 7)Delete the interface: -delete pktio-interface name Example: -------- Below is example config logs: with odp-linux: DBGvpp# create pktio-interface name enp0s8 hw-addr 08:00:27:1b:5e:48 odp-enp0s8 DBGvpp# set int ip address odp-enp0s8 192.168.1.4/24 DBGvpp# set int state odp-enp0s8 up DBGvpp# sh int Name Idx State Counter Count local0 0 down odp-enp0s8 1 up rx packets 1 rx bytes 60 drops 1 DBGvpp# sh int addr local0 (dn): odp-enp0s8 (up): 192.168.1.4/24 DBGvpp# sh int Name Idx State Counter Count local0 0 down drops 1 odp-enp0s8 1 up rx packets 12 rx bytes 902 tx packets 5 tx bytes 324 drops 4 punts 3 ip4 7 DBGvpp# sh ip arp Time IP4 Flags Ethernet Interface 171.7151 192.168.1.1 D c8:3a:35:19:ea:f0 odp-enp0s8 185.0099 192.168.1.3 D 48:45:20:11:41:ad odp-enp0s8 DBGvpp# with odp-dpdk: sreejith@sreejith-VirtualBox:~/vppdodp/vpp_odp_dpdk/dpdk$ sudo ./usertools/dpdk-devbind.py --status Network devices using DPDK-compatible driver ============================================ 0000:00:08.0 '82540EM Gigabit Ethernet Controller' drv=igb_uio unused=e1000 0000:00:09.0 '82540EM Gigabit Ethernet Controller' drv=igb_uio unused=e1000 DBGvpp# create pktio-interface name 0 hw-addr 08:00:27:1b:5e:48 odp-0 DBGvpp# sh int Name Idx State Counter Count local0 0 down odp-0 1 down DBGvpp# set int ip address odp-0 192.168.1.4/24 DBGvpp# set int state odp-0 up DBGvpp# sh int Name Idx State Counter Count local0 0 down odp-0 1 up rx packets 13 rx bytes 1960 tx packets 1 tx bytes 42 drops 13 ip4 7 ip6 4 # VPP startup configuration --------------------------- Default configuration of ODP plugin, IPSec nodes and automatic creation of interfaces can be specified in startup.conf file. More details are in template: src/vpp/conf/startup.conf To use customized startup configuration: copy default config from src/vpp/conf/startup.conf add 'interactive' to 'unix' section and remove 'api-segment' uncomment and adjust 'odp' section export STARTUP_CONF= # ODP based ipsec nodes ----------------------- With the development of the odp4vpp project IPsec nodes have been added, making use of the ODP APIs. In hope that in the future it will enable accelerated IPsec tunnels. There are two pairs of new nodes, one making use of ODP crypto API, and one making use of the ODP IPsec API. In order to make use of the ODP crypto API version of IPsec nodes, put `enable-odp-crypto` in the odp section of the startup.conf file, while to use the IPsec API version, put `enable-odp-ipsec` in that file. The options should not be used simultaneously. By the time being these are global options and it is not possible to configure different IPsec nodes for different ports. odp { enable-odp-crypto } or, odp { enable-odp-ipsec } By default all the operations are done synchronously. In order to change that, `async` needs to be put in the odp section of the startup.conf file. odp { enable-odp-crypto async } IPsec API also provieds inline mode (for details see odp user guide), to make use of that, put `inline` in the odp section of the startup.conf file. That option _can_ be combined with the `async` option. odp { enable-odp-ipsec inline async } The IPsec inline mode is only supported on the loopback pktio in odp-linux, though it has not been tested with this pktio. It has been tested with modified versions of dpdk/socket pktio. The last two patches for odp-linux enable testing it with dpdk and socket based pktio on odp-linux. The last of the odp-dpdk patches enable inline mode in odp-dpdk. Those patches are not needed if no IPsec is planed to be ran in inline mode. Also these changes should soon be integrated in the odp and odp-dpdk repositories, and then these patches will no longer be needed.