Age | Commit message (Collapse) | Author | Files | Lines |
|
IPFIX buffers are stored on a per worker thread basis. Currently, the
flush callbacks will flush only buffers stored for the main thread. And
buffers for worker threads will not be sent until their size reach the
path MTU configured for the exporter. So if traffic is constant, the
problem will unlikely to be visible. Buffers will be sent once they
reach the maximum size. However, if traffic stops at some point and
flush is triggered in order to make the plugin send all currently
buffered data, this will not happen. And collectors will not receive
that data. The plugin will keep the remaining data until traffic starts
again, the buffers reach the maximum size, and be sent.
With this fix, flush buffers for worker threads and for the main thread
when the flush callbacks are triggered.
This will allow to remove @tag_fixme_vpp_workers from the unit tests
that don't set timers. The tests that set timers will still be failing
for other multi-worker related problems.
Type: fix
Change-Id: I9a7d9cef8ddbec7ee68c79309e48e7bc0953d488
Signed-off-by: Alexander Chernavin <achernavin@netgate.com>
(cherry picked from commit 4c7305f124cfa9c649ec6c9231eaf608fe336f1b)
|
|
Currently, when flowprobe_export_send() calls vlib_time_now(), a pointer
to the main thread's vlib_main_t is always passed (the one cached in
flow_report_main). However, that code can also be executed from a worker
thread. And passing a pointer to the main thread's vlib_main_t to
vlib_time_now() from a worker thread may cause time synchronization
issues. Also, running a debug binary will cause an assertion failure in
vlib_time_now() in this case.
With this fix, flowprobe_export_send() passes the pointer to the current
thread's vlib_main_t to vlib_time_how().
This doesn't allow to remove @tag_fixme_vpp_workers from the unit tests
yet as they will be failing for other multi-worker related problems.
Type: fix
Change-Id: Ia35e3a4176777b88cf8ca8af8af7c42c495cbc6a
Signed-off-by: Alexander Chernavin <achernavin@netgate.com>
|
|
The recent TX flows generation fix introduced "l3_hdr_offset" which
represents the offset of the IP header in the buffer's data. The problem
is that it is erroneously defined as a 16-bit unsigned integer. If the
calculated offset is negative, "l3_hdr_offset" will get a value close to
UINT16_MAX. And the code will search the IP header somewhere beyond the
buffer's data. For example, this will occur in the case when an ICMP
error is being sent in response to a received packet.
With this fix, make "l3_hdr_offset" a signed integer.
Type: fix
Change-Id: I6f1283c7ba02656d0f592519b5863e68348c5583
Signed-off-by: Alexander Chernavin <achernavin@netgate.com>
|
|
Currently, when IPFIX records generation is enabled for an interface in
the TX direction, some rewritten traffic is being sent from that
interface, and the Ethernet header's location has changed due to
rewriting, generated TX flows will contain fields with wrong and zero
values. For example, that can be observed when traffic is rewritten from
a subinterface to a hardware interface (i.e. when tags are removed). A
TX flow generated in this case will have wrong L2 fields because of an
incorrectly located Ethernet header. And zero L3/L4 fields because the
Ethernet type will match neither IP4 nor IP6.
The same code is executed to generate flows for both input and output
features. And the same mechanism is applied to identify the Ethernet
header in the buffer's data. However, such general code usually works
with the buffer's data conditionally based on the direction. For most
input features, the buffer's current_data will likely point to the IP
header. For most output features, the buffer's current_data will likely
point to the Ethernet header.
With this fix:
- Keep relying on ethernet_buffer_get_header() to locate the Ethernet
header for input features. And start using vlib_buffer_get_current()
to locate the Ethernet header for output features. The function will
account for the Ethernet header's position change in the buffer's
data if there is rewriting.
- After fixing Ethernet header determination in the buffer's data,
L3/L4 fields will contain non-zero but still incorrect data. That is
because IP header determination needs to be fixed too. It currently
relies on the fact that the Ethernet header is always located at the
beginning of the buffer's data and that l2_hdr_sz can be used as an
IP header offset. However, this may not be the case after rewriting.
So start calculating the actual offset of the IP header in the
buffer's data.
- Add a unit test to cover the case.
Type: fix
Change-Id: Icf3f9e6518912d06dff0d5aa48e103b3dc94edb7
Signed-off-by: Alexander Chernavin <achernavin@netgate.com>
|
|
As a result of recent fixes, all currently stored flows of an interface
are deleted when the feature is being disabled for the interface. This
includes stopping the timer and freeing the flow entries for further
reuse. The problem is that meta information is not cleared in the flow
entries being deleted. For example, packet delta count will keep its
value. The next flow that gets one of these pool entries will already
have a non-zero packet count. So the counting of packets will start from
a non-zero value. And incorrect packet delta count will be exported for
that flow.
With this fix, clear meta information too when clearing interface state.
Also, update the corresponding test to cover this case.
Type: fix
Change-Id: I9a73b3958adfd1676e66b0ed50f1478920671cca
Signed-off-by: Alexander Chernavin <achernavin@netgate.com>
|
|
Currently, TCP flags of a flow entry don't get reset once the flow is
exported (unlike other meta information about a flow - packet delta
count and octet delta count). So TCP flags are accumulated as long as
the flow is active. When the flow expires, it is exported the last time,
and its pool entry is freed for further reuse. The next flow that gets
this pool entry will already have non-zero TCP flags. If it's a TCP
flow, the flags will keep being accumulated. This might look fine when
exported. If it's a non-TCP flow, that will definitely look erroneous.
With this fix, reset TCP flags once the flow is exported. Also, cover
the reuse case with tests.
Type: fix
Change-Id: I5f8560afffcfe107909117d3d063e8a69793437e
Signed-off-by: Alexander Chernavin <achernavin@netgate.com>
|
|
Currently, when L2 and L4 recording is enabled on the L2 datapath, the
L2 template will contain L4 fields and L2 flows will be exported with
those fields always set to zero.
With this fix, when L4 recording is enabled, add L4 fields to templates
other than the L2 template (i.e. to the IP4, IP6, L2_IP4, and L2_IP6
templates). And export L2 flows without L4 fields. Also, cover that case
in the tests.
Type: fix
Change-Id: Id5ed8b99af5634fb9d5c6e695203344782fdac01
Signed-off-by: Alexander Chernavin <achernavin@netgate.com>
|
|
When IPFIX flow record generation is enabled on an interface and the
active timer is set, flows will be saved and then exported according to
the active and passive timers. If then disable the feature on the
interface, the flow entries currently saved will remain in the state
tables. They will gradually expire and be exported. The problem is that
the template for them has already been removed. And they will be sent
with zero template ID which will make them unreadable.
A similar problem will occur if feature settings are "changed" on the
interface - i.e. disable the feature and re-enable it with different
settings (e.g. set a different datapath). The remaining flows that
correspond to the previous feature settings will be eventually sent
either with zero template ID or with template ID that corresponds to the
current feature settings on the interface (and look like garbage data).
With this fix, flush the current buffers before template removal and
clear the remaining flows of the interface during feature disabling.
Type: fix
Change-Id: I1e57db06adfdd3a02fed1a6a89b5418f85a35e16
Signed-off-by: Alexander Chernavin <achernavin@netgate.com>
|
|
Currently, L2 flows are exported using L2_IP6 template if L3 or L4
recording is enabled on L2 datapath. That occurs because during feature
enable, L2 template is added and its ID is not saved immediately. Then
L2_IP4 and L2_IP6 templates are added overwriting "template_id" each
time. And in the end, the current value of "template_id" is saved for L2
template. The problem is that "template_id" at that point contains the
ID of L2_IP6 template.
With this fix, save the template ID immediately after adding a template
for all variants (datapaths). Also, cover the case with a test.
Type: fix
Change-Id: Id27288043b3b8f0e89e77f45ae9a01fa7439e20e
Signed-off-by: Alexander Chernavin <achernavin@netgate.com>
|
|
Type: improvement
Change-Id: I7f7050c19453a69a7fb6c5e62f8f57db847d9144
Signed-off-by: Damjan Marion <damarion@cisco.com>
|
|
Type: improvement
With this change:
- add dump/details messages to obtain interfaces for which IPFIX flow
record generation is enabled;
- add get message to obtain parameters;
- add a new message to set parameters with validation present and to
correspond with get/set naming;
- add tests for get/set parameters and dump/details interfaces.
Change-Id: I09f6ec990171ac8bcb9d2f5c92629803b8ab6c28
Signed-off-by: Alexander Chernavin <achernavin@netgate.com>
|
|
Type: feature
Currently, the plugin supports only IPFIX flow record generation for
outbound packets.
With this change:
- add a new API message for enabling the feature on an interface that
accepts direction (rx, tx, both);
- update existing debug command for feature enabling to accept
direction;
- update existing debug command for showing currently enabled feature
on interfaces to display direction;
- update templates to include a direction field;
- generate flow records on the specified direction and data path;
- report direction in flow data;
- update tests to use the new API;
- add tests for inbound flows.
Change-Id: I121fd904b38408641036ebeea848df7a4e5e0b30
Signed-off-by: Alexander Chernavin <achernavin@netgate.com>
|
|
Modify the ipfix_exporter to use ip_address instead of the ipv4 specific
version. Modify the current code so that it writes into the v4 specific
part of the address, i.e. we are not yet fully supporting IPv6. For the
exporter configured via the original API (the one that is always in slot0)
we will not support IPv6 addresses.
Type: improvement
Signed-off-by: Paul Atkins <patkins@graphiant.com>
Change-Id: Ic9854ac62aaee76a7a55a958234c456fd9828c4c
|
|
When a new flow-report is created the caller provides 2 callback functions.
These functions both take a pointer to the exporter, plus a pointer to the
source and dest address. However the pointers to the address are not adding
any value as these are always set to the src/dest addresses of the exporter
(which is already being passed). Remove these parameters and leave the
callback functions to get the addresses out of the exporter.
Type: improvement
Signed-off-by: Paul Atkins <patkins@graphiant.com>
Change-Id: I36dec394f30e85cdca120dd8706b5d90f5e07c48
|
|
Pass an ipfix_exporter to this function so that callers can choose which
exporter they are modifying.
Type: improvement
Signed-off-by: Paul Atkins <patkins@graphiant.com>
Change-Id: Ice0ed19a57baf15b1dc85cd27fe01913e36d7f4f
|
|
Pull out the fields in flow_report_main_t that are specific to a single
exporter and move them into a new structure that represents an exporter.
Add a pool of exporters to flow_report_main_t and do a pool_get() to get
the entry at index 0, so that the existing users of the code need only
change the path at which they access the old fields and have no need to
make further code changes. In functions that were accessing the fields
that now make up the ipfix_exporter create a local var that points to the
first (always valid) exporter and use this as the base for the fields
rather than finding them from flow_report_main.
This is in preparation for supporting multiple flow_exporters.
Note that at the moment the code supports multiple 'streams' for a given
exporter, where each stream has its own source port, domain id and template
space. But all streams within an exporter have the same destination address,
so this is not the same as multiple exporters.
Type: refactor
Signed-off-by: Paul Atkins <patkins@graphiant.com>
Change-Id: I49f5c7fb9e901773351d31dc8a59178c37e99301
|
|
Skip 802.1q headers due to correct EtherType, ip addresses, ports.
Ticket: VPP-1997
Type: fix
Change-Id: I1a552fa6abe5b1459dd7d2c5ac6ad0f62c51417c
Signed-off-by: Daniel Béreš <daniel.beres@pantheon.tech>
|
|
Change-Id: I7a6df4317beed78e394dc4ba8edd350ca5b2bc80
Type: fix
Signed-off-by: Damjan Marion <damarion@cisco.com>
|
|
Type: improvement
Change-Id: I7e821cce1feae229e1be4baeed249b9cca658135
Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com>
|
|
Type: refactor
Change-Id: Id10cbf52e8f2dd809080a228d8fa282308be84ac
Signed-off-by: Damjan Marion <damarion@cisco.com>
|
|
- Generate copyright year and version
instead of using hard-coded data
Type: refactor
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
Change-Id: I6058f5025323b3aa483f5df4a2c4371e27b5914e
|
|
trajectory trace has been broken for a while because we used to save the
buffer trajectory in a vector pointed to in opaque2. This does not work
well when opaque2 is copied (eg. because of a clone) as 2 buffers end up
sharing the same vector.
This dedicates a full cacheline in the buffer metadata instead when
trajectory is compiled in. No dynamic allocation, no sharing, no tears.
Type: refactor
Change-Id: I6a028ca1b48d38f393a36979e5e452c2dd48ad3f
Signed-off-by: Benoît Ganne <bganne@cisco.com>
|
|
- rename node to more meaningful name
- introduce lookup tables
- enable multiarch
- quad-loop node
- enqqueue to next instead of enqueueing to node
Type: improvement
Change-Id: Ibb208047ae04bb6cfe56db558d3b8938bc14b4fe
Signed-off-by: Damjan Marion <damarion@cisco.com>
|
|
Type: improvement
Change-Id: If3da7d4338470912f37ff1794620418d928fb77f
Signed-off-by: Damjan Marion <damarion@cisco.com>
|
|
If the multi-worker default VPP configuration is triggered by
setting VPP_WORKER_CONFIG="workers 2", some of the tests fail
for various reasons.
It's a substantial number, so this change marks all of the
testsets that have this issue, such that they can be addressed
later independently.
Type: test
Change-Id: I4f77196499edef3300afe7eabef9cbff91f794d3
Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>
|
|
Type: fix
Ticket: VPP-1859
Signed-off-by: jan_cavojsky <Jan.Cavojsky@pantheon.tech>
Change-Id: Iaa5045001621ec99dc8579e8e989adf81dc60525
|
|
We have accumulated several scenarios in prod or wishlists
where it would be useful to have a general infra to say yes/no
about a certain test, and potentially make decisions based on that,
for example:
- runs solo (aka 'time-dependent')
- (wishlist) part of quick smoke-test set
- (wishlist) intermittent failure unrelated to timing
- (wishlist) test broken with a multi-worker config in vpp
Refactor the current "run-solo" code to allow for this extension.
Type: test
Change-Id: Ia5b3810e57c0543753c8e0dc4dc0cfb4a30b36ac
Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>
Signed-off-by: Klement Sekera <ksekera@cisco.com>
|
|
Type: refactor
Change-Id: Ie67dc579e88132ddb1ee4a34cb69f96920101772
Signed-off-by: Damjan Marion <damarion@cisco.com>
|
|
VAT2: A completely auto-generated replacement of VAT.
Reads input message in JSON from stdin and outputs received messages in JSON.
A VAT2 plugin is automatically built for a .api file.
There no longer a need for a separate _test.c.
Example:
vat2 show_version {}
{
"_msgname": "show_version_reply",
"retval": 0,
"program": "vpe",
"version": "21.01-rc0~411-gf6eb348a6",
"build_date": "2020-11-19T09:49:25",
"build_directory": "/vpp/autogen3"
}
vat2 sw_interface_dump '{"sw_if_index": -1,
"name_filter_valid": 0,
"name_filter": ""}'
[{
"_msgname": "sw_interface_details",
"sw_if_index": 0,
"sup_sw_if_index": 0,
"l2_address": "00:00:00:00:00:00",
"flags": "Invalid ENUM",
"type": "IF_API_TYPE_HARDWARE",
"link_duplex": "LINK_DUPLEX_API_UNKNOWN",
"link_speed": 0,
"link_mtu": 0,
"mtu": [0, 0, 0, 0],
"sub_id": 0,
"sub_number_of_tags": 0,
"sub_outer_vlan_id": 0,
"sub_inner_vlan_id": 0,
"sub_if_flags": "Invalid ENUM",
"vtr_op": 0,
"vtr_push_dot1q": 0,
"vtr_tag1": 0,
"vtr_tag2": 0,
"outer_tag": 0,
"b_dmac": "00:00:00:00:00:00",
"b_smac": "00:00:00:00:00:00",
"b_vlanid": 0,
"i_sid": 0,
"interface_name": "local0",
"interface_dev_type": "local",
"tag": ""
}]
This is the first phase and vat2 is not integrated in packaging yet.
Type: feature
Signed-off-by: Ole Troan <ot@cisco.com>
Change-Id: Ib45ddeafb180ea7da8c5dc274a9274d7a4edc876
Signed-off-by: Ole Troan <ot@cisco.com>
|
|
Type: improvement
Signed-off-by: Florin Coras <fcoras@cisco.com>
Change-Id: Id13f33843b230a1d169560742c4f7b2dc17d8718
|
|
Type: style
Signed-off-by: Neale Ranns <nranns@cisco.com>
Change-Id: I26a19e42076e031ec5399d5ca05cb49fd6fbe1cd
|
|
Type: feature
Ticket: VPP-1861
Signed-off-by: jan_cavojsky <Jan.Cavojsky@pantheon.tech>
Change-Id: Iaeff13b19a712257223a4e77893cfd9398c18327
|
|
In one's complement, there are two representations of zero: the all
zero and the all one bit values, often referred to as +0 and -0. See
RFC 1624 section 3 for more details.
This used to be taken care of in ip4_header_checksum(), but it is no
longer the case. The check ip->checksum == ip4_header_checksum (ip) is
no longer correct in the -0 case.
Always use ip4_header_checksum_is_valid() instead (which behaves
correctly since 9a79a1ab931c3b5a7ae07d6f0fcfef7c4368a2c4).
Type: fix
Fixes: e5f0050c7a5d411f96af6401797529d58825e2af
Change-Id: Iacc6b60645a834287b085aecb9e3fdb4554cf0cf
Signed-off-by: Benoît Ganne <bganne@cisco.com>
|
|
Some of the tests are time-sensitive, and at present require a non-trivial
modification in order to run at high concurrency.
Without these modifications, they intermittently fail, and require
the test retries.
Rather than setting them to the extended tests and forgetting
about them, put them into a "solo" set, which gets run in a
single-threaded mode after the rest of the tests are done.
Mark a few of the tests that showed errors during TEST_JOBS=48
as forced-solo.
Also, give a better diagnostic if the testcase misses a docstring
needed to represent it in the diagnostic outputs.
Type: fix
Change-Id: I33fe62eb17edc1885bd2c3523892051d52da6546
Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>
|
|
Type: docs
Change-Id: I9b5e5137eb4c1e89f6e8d7a278cd11a0fd496471
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
|
|
Type: fix
Signed-off-by: Ole Troan <ot@cisco.com>
Change-Id: I4f1cccca7de0c07cee472bde80cd6b0ef60046bd
|
|
Raw('\xaf) and Raw(b'\xaf) are two quite different things in python 2 versus 3.
In most cases this didn't make a difference, apart from those cases where length
of payload actually mattered.
Type: fix
Signed-off-by: Ole Troan <ot@cisco.com>
Change-Id: I3cba5c1486e436a3ca8aa10a7b393da75aa9f6b9
|
|
Type: feature
fts and trex rely on yaml config files. Verify
that they are valid, so comitters can catch
errors early.
Change-Id: Ide0bb276659119c59bdbbc8b8155e37562a648b8
Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
|
|
Type: fix
Since CentOS 8, RPM build script doesn't accept '#!/usr/bin/env python'
as a valid shebang line. It requires scripts to explicitly chose
between python2 or python3.
Change all to use python3 as suggested by Paul Vinciguerra.
Depends-On: https://gerrit.fd.io/r/23170
Signed-off-by: Renato Botelho do Couto <renato@netgate.com>
Change-Id: Ie72af9f60fd0609e07f05b70f8d96e738b2754d1
|
|
Use consistent API types.
Type: fix
Signed-off-by: Jakub Grajciar <jgrajcia@cisco.com>
Change-Id: I5b03e5de111c3a3b8da4e9f02cba0aa99e3ee9f3
|
|
Type: fix
Signed-off-by: Ole Troan <ot@cisco.com>
Change-Id: I7425147e864467e4f94cee2a82ef74a894339471
|
|
Split generation of API message enum and the typedefs into
separate files, so that the type file can be included from
elsewhere.
Generate a C file for VPP that contains the API registration,
this was previously done via X macros by the C pre-
processor.
This allows deleting lots of skeleton/copy paste code
for each feature.
plugins/flowprobe
plugins/map
examples/sample-plugin
vnet/ipip
used as Guinea pigs.
Generate a C Test file for VAT, that does the same for VAT plugins.
Also add support for a per-message CLI option, that is currently
limited to VAT help text. option vat_help = "<help text>";
Type: refactor
Change-Id: I245e3104bb76f7e1fb69a59ab20cc7c8dfcdd460
Signed-off-by: Ole Troan <ot@cisco.com>
|
|
- Relocate plugin tests for 'make test' into
src/plugins/*/test so that plugin test cases
are co-located with the plugin source code.
Type: refactor
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
Change-Id: I503e6a43528e14981799b735fa65674155713f67
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
|
|
If the corresponding vpp plugin is absent, return a non-zero
clib_error_t * from vat_plugin_register ("xxx plugin not loaded"). The
vat plugin calls dlclose on the vat plugin, and it disappears.
Depending on the plugin configuration, this can reduce the vpp virtual
size by several gigabytes.
Added a VAT_PLUGIN(<plugin-name>) macro to vat_helper_macros, clean up
boilerplate vat_plugin_register() implementations. Fixed a number of
non-standard vat_plugin_register methods.
Type: refactor
Change-Id: Iac908e5af7d5497c78d6aa9c3c51cdae08374045
Signed-off-by: Dave Barach <dave@barachs.net>
|
|
Type: refactor
Change-Id: I8489ccd54411c2aa9355439c5641dc31012c64a2
Signed-off-by: Benoît Ganne <bganne@cisco.com>
Signed-off-by: Damjan Marion <damarion@cisco.com>
|
|
Add tooling for feature metadata configuration files.
The main tool is in src/scripts/fts.py
make checkfeaturelist to validate against schema.
make featurelist to dump all feature lists to stdout.
Example feature definition:
name: IP in IP tunnelling
maintainer: Ole Troan <ot@cisco.com>
features:
- IPv4/IPv6 over IPv4/IPv6 encapsulation:
- Fragmentation and Reassembly
- Configurable MTU
- Inner to outer Traffic Class / TOS copy
- Configurable Traffic Class / TOS
- ICMPv4 / ICMPv6 proxying
- 6RD (RFC5969):
- Border Relay
description: "Implements IP{v4,v6} over IP{v4,v6} tunnelling as
described in RFC2473. This module also implement the border relay of
6RD (RFC5969)."
state: production
properties: [API, CLI, STATS, MULTITHREAD]
missing:
- Tunnel PMTUD
- Tracking of FIB state for tunnel state
- IPv6 extension headers (Tunnel encapsulation limit option)
JSON schema is embedded in fts.py
Example markdown: https://github.com/otroan/scratch/blob/master/features.md
Change-Id: I903b4ee6b316a9378c259e86dc937092e5d4b7da
Type: make
Signed-off-by: Ole Troan <ot@cisco.com>
|
|
Change-Id: Ia083050389853c25b069f0f8286d50d3f4aef527
Signed-off-by: Damjan Marion <damarion@cisco.com>
|
|
* u32/u64/uword mismatches
* pointer-to-int fixes
* printf formatting issues
* issues with incorrect "ULL" and related suffixes
* structure alignment and padding issues
Change-Id: I70b989007758755fe8211c074f651150680f60b4
Signed-off-by: David Johnson <davijoh3@cisco.com>
|
|
Change-Id: Id4f37f5d4a03160572954a416efa1ef9b3d79ad1
Signed-off-by: Dave Barach <dave@barachs.net>
|
|
Change-Id: Ied34720ca5a6e6e717eea4e86003e854031b6eab
Signed-off-by: Dave Barach <dave@barachs.net>
|