summaryrefslogtreecommitdiffstats
path: root/src/vnet/devices/virtio/node.c
AgeCommit message (Collapse)AuthorFilesLines
2023-06-27virtio: use fast-path for ethernet-input if possibleStanislav Zaikin1-1/+26
If a frame should be sent to 'ethernet-input' set the ETH_INPUT_FRAME_F_SINGLE_SW_IF_IDX flag. It will force 'ethernet-input' to use a fast-path for such frames. This patch also aligns the behaviour with other input nodes. Type: improvement Change-Id: Icff0fa31204d5304a2ea0a4f4e7bc418dedbfe32 Signed-off-by: Stanislav Zaikin <zstaseg@gmail.com>
2022-09-16virtio: add support for per queue packet counterMohsin Kazmi1-0/+1
Type: improvement Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com> Change-Id: I21a701a556b88a9d81f0e074a59fa34b3746b1d9
2022-05-20virtio: use the internal hdr offsets of buffer metadata for offloadsMohsin Kazmi1-13/+6
Type: improvement Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com> Change-Id: Ie63c360f2e42e9e5799f5c536453823ea95ed3b8
2022-03-01virtio: refactor codeMohsin Kazmi1-26/+28
Type: refactor Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com> Change-Id: I3788cc857023fafcc8eb6d6ff4524425026a75d8
2022-02-15tcp: Do not include the tcp_packet.h file in the ip4_packet.hNeale Ranns1-0/+1
Type: refactor IP4 does not depend on TCP (it's the other way around). This upside down dependency leads to some nasty circular includes when trying to use ip46_address.h in interface.h Signed-off-by: Neale Ranns <neale@graphiant.com> Change-Id: I4a1bd21543b08b9c1cf1e5563da738414734a878
2021-12-14virtio: integrate with new tx infraMohsin Kazmi1-14/+0
Type: improvement Change-Id: I337ec63d0868f665329d68eadf1744e080b73a0d Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com>
2021-11-12virtio: improve the inline of clib_memcpy_fastMohsin Kazmi1-1/+1
Type: refactor Change-Id: Ia13a9cf6480aac280f3d287c59908e84c29c3443 Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com>
2021-11-05virtio: fix the initializationMohsin Kazmi1-1/+1
Type: fix Fixes: d96b28ac0917 ("virtio: improve input node performance") vlib_buffer_t is defined on stack to be used for input feature arc lookup once per frame call for performance reasons. The definition is missing the initialization to avoid the assignment of garbage value. Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com> Change-Id: Ie7930186bbd8240c44d257808ebd31e06e854b29
2021-05-06vlib: fix the offload flags sizeMohsin Kazmi1-1/+1
Type: fix Change-Id: I433fe3799975fe3ba00fa30226f6e8dae34e88fc Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com>
2021-05-04virtio: fix the interruptMohsin Kazmi1-162/+2
Type: fix virtio/tap interfaces set the empty buffers in the input node for receiving data. Backend uses those buffers, fills them with data and notifies the virtio/tap driver. But virtio/tap driver gets into stall state if interface is created and configured through exec script on VPP startup.conf and put the interface in interrupt mode while VPP is only configured with main thread. This patch fixes the problem by prefilling buffers during the interface creation. Change-Id: Ibc4d0e70e127ccc4b7cf8b2b18406ae4b02c73b4 Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com>
2021-05-01vlib: refactor trajectory trace debug featureBenoît Ganne1-2/+0
trajectory trace has been broken for a while because we used to save the buffer trajectory in a vector pointed to in opaque2. This does not work well when opaque2 is copied (eg. because of a clone) as 2 buffers end up sharing the same vector. This dedicates a full cacheline in the buffer metadata instead when trajectory is compiled in. No dynamic allocation, no sharing, no tears. Type: refactor Change-Id: I6a028ca1b48d38f393a36979e5e452c2dd48ad3f Signed-off-by: Benoît Ganne <bganne@cisco.com>
2021-02-15vlib: refactor checksum offload supportMohsin Kazmi1-6/+8
Type: refactor This patch refactors the offload flags in vlib_buffer_t. There are two main reasons behind this refactoring. First, offload flags are insufficient to represent outer and inner headers offloads. Second, room for these flags in first cacheline of vlib_buffer_t is also limited. This patch introduces a generic offload flag in first cacheline. And detailed offload flags in 2nd cacheline of the structure for performance optimization. Change-Id: Icc363a142fb9208ec7113ab5bbfc8230181f6004 Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com>
2021-02-08virtio: use vpp clib_memset instead glibc memsetMohsin Kazmi1-2/+2
Type: improvement Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com> Change-Id: Ib3cefe5d27286c4853eb0c0e1803a94787a62c97
2021-02-08virtio: add atomic call for kickingMohsin Kazmi1-11/+3
Type: fix Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com> Change-Id: I41faa2ca249ff75e564a732af896e6b5d76bf665
2021-01-21interface: rx queue infra rework, part oneDamjan Marion1-26/+22
Type: improvement Change-Id: I4008cadfd5141f921afbdc09a3ebcd1dcf88eb29 Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-12-04virtio: fix the len offsetMohsin Kazmi1-2/+2
Type: fix Change-Id: I8ce1df5c97941fb645b33476db9cfc74f1395b15 Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com>
2020-11-17virtio: virtio: implement packed queues from virtio 1.1Mohsin Kazmi1-26/+190
Type: feature Change-Id: I12703371541298efa029903d6762b1cd1f7322ca Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com>
2020-11-09vlib: fix trace number accountingBenoît Ganne1-3/+2
When using classifier to filter traces, not all packets will be traced. In that case, we should only count traced packets. Type: fix Change-Id: I87d1e217b580ebff8c6ade7860eb43950420ae78 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2020-10-21virtio: improve input node performanceDamjan Marion1-13/+26
- Avoid feature arc lookup per packet, as all packets belong to the same interface - don't validate enqueue for l2 packets, as they all have same destionation Type: improvement Change-Id: I32ddd801f219eab9281736abafdc91a4ff9e0989 Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-10-19virtio: fix compilation timeDamjan Marion1-30/+31
There is no need to inline 3 times virtio_refill_vring()... Type: fix Change-Id: Ic26e24fd7911af743fedd0e2282784b715e86c3c Signed-off-by: Damjan Marion <damarion@cisco.com>
2020-09-28virtio: add packet buffering on txMohsin Kazmi1-4/+7
Type: feature This patch adds packet buffering on tx for slow backend which have some jitter/delays in freeing the vrings. There are some limitations to the current design: 1) It only works in poll mode. 2) Atleast 1 rx queue of an interface (with buffering enabled) should be placed on each worker and main thread. Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com> Change-Id: Ib93c350298b228e80426e58ac77f3bbc93b8be27
2020-09-18virtio: fix txq lockingBenoît Ganne1-1/+3
Initialize txq lock only if some txq are shared and check if another worker is already operating on the txq before processing gro timeouts in input node. Type: fix Change-Id: I89dab6c0e6eb6a7aa621fa1548b0a2c76e6c7581 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2020-09-04virtio: remove kernel virtio header dependenciesMohsin Kazmi1-9/+9
Type: refactor tap, virtio and vhost use virtio/vhost header files from linux kernel. Different features are supported on different kernel versions, making it difficult to use those in VPP. This patch removes virtio/vhost based header dependencies to local header files. Change-Id: I064a8adb5cd9753c986b6f224bb075200b3856af Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com>
2020-08-17tap: add gro supportMohsin Kazmi1-0/+9
Type: feature Change-Id: I5868dd267aa26aa97aec5fd70e70c5956ac52277 Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com>
2020-07-22virtio: add the buffer allocation errorMohsin Kazmi1-7/+13
Type: improvement Change-Id: I991e32d531719693c387db4ef93d04d4b562789d Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com>
2020-06-24virtio: fix the vlib buffer flagsMohsin Kazmi1-2/+1
Type: fix Change-Id: I92e8e30e4175229e24dd92471016e9095b5cc83e Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com>
2020-06-12virtio: interface type based improvementsMohsin Kazmi1-39/+71
Type: improvement Change-Id: I220ea6ab609e3c1628f5210be441d0d5e825a32c Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com>
2020-04-17tap: add initial support for tunMohsin Kazmi1-13/+46
Type: feature Change-Id: I699a01ac925fe5c475a36032edb7018618bb4dd4 Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com>
2020-04-08virtio: fix the tcp/udp checksum offloadsMohsin Kazmi1-2/+0
Some vhost-backend calculates the wrong checksum in case of tcp/udp offload when driver resets tcp/udp checksum field to '0'. Type: fix Change-Id: I1d2a9b95b3d5cc1decac38027104a04df2af4680 Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com>
2020-03-02virtio: fix the coverity warningMohsin Kazmi1-4/+1
Type: fix Change-Id: Ia75edb74eb7c746dd4c66bdbff75efb949575ce4 Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com>
2020-01-08virtio: split gso and checksum offload functionalityMohsin Kazmi1-4/+8
Type: refactor Change-Id: I897e36bd5db593b417c2bac9f739bc51cf45bc08 Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com>
2020-01-02virtio: fix checksum offload supportBenoît Ganne1-10/+21
Checksum offload and GSO are independent. We must support checksum offload if it has been negotiated, independently of GSO. Ticket: VPPSUPP-47 Type: fix Change-Id: I8cb6dd58b61714ebb2726eb4aab0d74d49fdab99 Signed-off-by: Benoît Ganne <bganne@cisco.com>
2019-12-06gso: fix the tap/virtio driver for header offsetMohsin Kazmi1-1/+25
Type: fix Change-Id: Ied34466907fa8ad44f997c600dbf481be4d22027 Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com>
2019-12-05gso: add protocol header parserMohsin Kazmi1-33/+10
Type: feature Change-Id: I7c6be2b96d19f82be237f6159944f3164ea512d0 Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com>
2019-11-13virtio: feature arc have higher priority than redirectDamjan Marion1-3/+4
Type: fix Fixes: 8389fb9 Change-Id: Ie159eb444b28b36a7af86049b80fba4e49be93cb Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-09-24vlib: add flag to explicitelly mark nodes which can init per-node packet traceDamjan Marion1-0/+1
Type: feature Change-Id: I913f08383ee1c24d610c3d2aac07cef402570e2c Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-09-02tap: interface rx counter not increment correctSteven Luong1-1/+1
vlib_increment_combined_counter takes sw_if_index, not hw_if_index. Using hw_if_index may work as long as there is no subinterface created to cause hw_if_index and sw_if_index to differ. Type: fix Signed-off-by: Steven Luong <sluong@cisco.com> Change-Id: I6db042186eeeacf32250f7ef261af8cd6f5ce56e
2019-08-17gso: fix l3 and l4 header offset in case of tagged interfaceMohsin Kazmi1-0/+14
previously, PG and virtio interfaces calculate wrong l3 and l4 header offset. This patch fixes this issue. Type: fix Ticket: VPP-1739 Change-Id: I5ba978e464babeb65e0711e1027320d46b3b9932 Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com>
2019-08-16gso: remove the ip checksum flag in case of ipv6Mohsin Kazmi1-1/+0
Type: fix Ticket: VPP-1727 Change-Id: Icfee35c5ab5e1c65079d1ca7bb514162319113e5 Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com>
2019-06-20tap: fix the total length of packet for stats byteMohsin Kazmi1-3/+3
Type: fix Fixes: 8389fb9 Change-Id: I31076db78507736631609146d4cca28597aca704 Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com>
2019-04-03virtio: Add support for multiqueueMohsin Kazmi1-7/+7
Change-Id: Id71ffa77e977651f219ac09d1feef334851209e1 Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com>
2019-03-04devices: migrate old MULTIARCH macros to VLIB_NODE_FNFilip Tehlar1-6/+3
Change-Id: I911fb3f1c6351b37580c5dbde6939a549431a92d Signed-off-by: Filip Tehlar <ftehlar@cisco.com>
2019-02-19tap gso: experimental supportAndrew Yourtchenko1-3/+89
This commit adds a "gso" parameter to existing "create tap..." CLI, and a "no-gso" parameter for the compatibility with the future, when/if defaults change. It makes use of the lowest bit of the "tap_flags" field in the API call in order to allow creation of GSO interfaces via API as well. It does the necessary syscalls to enable the GSO and checksum offload support on the kernel side and sets two flags on the interface: virtio-specific virtio_if_t.gso_enabled, and vnet_hw_interface_t.flags & VNET_HW_INTERFACE_FLAG_SUPPORTS_GSO. The first one, if enabled, triggers the marking of the GSO-encapsulated packets on ingress with VNET_BUFFER_F_GSO flag, and setting vnet_buffer2(b)->gso_size to the desired L4 payload size. VNET_HW_INTERFACE_FLAG_SUPPORTS_GSO determines the egress packet processing in interface-output for such packets: When the flag is set, they are sent out almost as usual (just taking care to set the vnet header for virtio). When the flag is not enabled (the case for most interfaces), the egress path performs the re-segmentation such that the L4 payload of the transmitted packets equals gso_size. The operations in the datapath are enabled only when there is at least one GSO-compatible interface in the system - this is done by tracking the count in interface_main.gso_interface_count. This way the impact of conditional checks for the setups that do not use GSO is minimized. "show tap" CLI shows the state of the GSO flag on the interface, and the total count of GSO-enabled interfaces (which is used to enable the GSO-related processing in the packet path). This commit lacks IPv6 extension header traversal support of any kind - the L4 payload is assumed to follow the IPv6 header. Also it performs the offloads only for TCP (TSO - TCP segmentation offload). The UDP fragmentation offload (UFO) is not part of it. For debug purposes it also adds the debug CLI: "set tap gso {<interface> | sw_if_index <sw_idx>} <enable|disable>" Change-Id: Ifd562db89adcc2208094b3d1032cee8c307aaef9 Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>
2019-02-09buffers: fix typoDamjan Marion1-1/+1
Change-Id: I4e836244409c98739a13092ee252542a2c5fe259 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-02-06buffers: make buffer data size configurable from startup configDamjan Marion1-1/+1
Example: buffers { default data-size 1536 } Change-Id: I5b4436850ca18025c9fdcfc7ed648c2c2732d660 Signed-off-by: Damjan Marion <damarion@cisco.com>
2019-02-06virtio: Use new buffer optimizationMohsin Kazmi1-2/+4
Change-Id: Ifc98373371b967c49a75989eac415ddda1dcf15f Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com>
2019-01-24virtio: Minor fixes and header cleanupMohsin Kazmi1-5/+3
Change-Id: I2e5fd45abcd07e9eda6184587889bdcd9613a159 Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com>
2019-01-21virtio: Native virtio driverMohsin Kazmi1-12/+25
Change-Id: Id7fccf2f805e578fb05032aeb2b649a74c3c0e56 Signed-off-by: Mohsin Kazmi <sykazmi@cisco.com>
2019-01-08virtio: fix kick race issueDamjan Marion1-5/+13
Change-Id: I25b2a28513821bc5eab9ac6890a3964d412b0399 Signed-off-by: Damjan Marion <damarion@cisco.com>
2018-11-14Remove c-11 memcpy checks from perf-critical codeDave Barach1-1/+1
Change-Id: Id4f37f5d4a03160572954a416efa1ef9b3d79ad1 Signed-off-by: Dave Barach <dave@barachs.net>