summaryrefslogtreecommitdiffstats
path: root/src/vnet/ip
AgeCommit message (Collapse)AuthorFilesLines
2017-05-05ip6_to_ip4.h coverity fixMatus Fabian1-1/+1
Change-Id: Ia11fa42d1f2b21a6195e7dc890f958830f83d3a7 Signed-off-by: Matus Fabian <matfabia@cisco.com>
2017-05-05NAT64: Move IPv6-IPv4 virtual reassembly code from MAP-T to common library ↵Matus Fabian2-0/+1148
(VPP-708) Change-Id: I9ad636f80bf109ffac9ca1b6d80d5f2c31f2076a Signed-off-by: Matus Fabian <matfabia@cisco.com>
2017-05-03change ip4_drop_node node register coding formatflyingeagle231-2/+2
Change-Id: Id4fdf2abf3bae3b14cdc6489ca16d2007da78d07 Signed-off-by: flyingeagle23 <wang.hui56@zte.com.cn>
2017-05-02Allow local/for-us replications for IP multicast routes on the CLINeale Ranns1-1/+6
Change-Id: I046d9402b79fdbc6a6663199ffff8e40a278fd51 Signed-off-by: Neale Ranns <nranns@cisco.com>
2017-04-26IP Flow Hash Config fixesNeale Ranns4-15/+17
- the flow hash config is (and was) cached on the load-balance object so the fib_table_t struct is not used a switch time. Therefore changes to the table's flow hash config need to be propagated to all load-balances and hance all FIB entries in the table. - enable API for setting the IPv6 table flow hash config - use only the hash config in the fib_table_t object and not on the ipX_fib_t - add tests. Change-Id: Ib804c11162c6d4972c764957562c372f663e05d4 Signed-off-by: Neale Ranns <nranns@cisco.com>
2017-04-25"autoreply" flag: autogenerate standard xxx_reply_t messagesDave Barach1-99/+9
Change-Id: I72298aaae7d172082ece3a8edea4217c11b28d79 Signed-off-by: Dave Barach <dave@barachs.net>
2017-04-24Improve Load-Balance MAPsNeale Ranns3-90/+153
- only build them for popular path-lists (where popular means more than 64 children) the reason to have a map is to improve convergence speed for recursive prefixes - if there are only a few this technique is not needed - only build them when there is at least one path that has recursive constraints, i.e. a path that can 'fail' in a PIC scenario. - Use the MAPS in the switch path. - PIC test cases for functionality (not convergence performance) Change-Id: I70705444c8469d22b07ae34be82cfb6a01358e10 Signed-off-by: Neale Ranns <nranns@cisco.com>
2017-04-19Learn IP6 link-local ND entries from NSs sourced from link-local addressNeale Ranns1-2/+5
Change-Id: I4c3ce4d58df7977490fc94991291422ea1e31ee3 Signed-off-by: Neale Ranns <nranns@cisco.com>
2017-04-13Remove unsed parameter from fib_table_entry_special_add() (only used in FIB ↵Neale Ranns2-5/+3
tests). The DPO was incorrectly initialised with FIB_PROTO_MAX Change-Id: I962df9e162e4dfb6837a5ce79ea795d5ff2d7315 Signed-off-by: Neale Ranns <nranns@cisco.com>
2017-04-11MPLS Mcast coverity fixesNeale Ranns1-1/+1
Change-Id: Ic07ec5d4c2560a414d5f4f7eb37e10faf591664a Signed-off-by: Neale Ranns <nranns@cisco.com>
2017-04-11MTRIE coverity fixesNeale Ranns1-6/+8
Change-Id: If98355bebe823f45b11b0908a8d7700ab273a6db Signed-off-by: Neale Ranns <nranns@cisco.com>
2017-04-11Remove usued, redundant and deprecated code from lookup.hNeale Ranns7-95/+33
Change-Id: Ic16bc10d0b2877b2afdf052615f9334f31b9519f Signed-off-by: Neale Ranns <nranns@cisco.com>
2017-04-07MPLS McastNeale Ranns6-40/+108
1 - interface-DPO Used in the Data-plane to change a packet's input interface 2 - MPLS multicast FIB entry Same as a unicast entry but it links to a replicate not a load-balance DPO 3 - Multicast MPLS tunnel Update MPLS tunnels to use a FIB path-list to describe the endpoint[s]. Use the path-list to generate the forwarding chain (DPOs) to link to . 4 - Resolve a path via a local label (of an mLDP LSP) For IP multicast entries to use an LSP in the replication list, we need to decribe the 'resolve-via-label' where the label is that of a multicast LSP. 5 - MPLS disposition path sets RPF-ID For a interface-less LSP (i.e. mLDP not RSVP-TE) at the tail of the LSP we still need to perform an RPF check. An MPLS disposition DPO performs the MPLS pop validation checks and sets the RPF-ID in the packet. 6 - RPF check with per-entry RPF-ID An RPF-ID is used instead of a real interface SW if index in the case the IP traffic arrives from an LSP that does not have an associated interface. Change-Id: Ib92e177be919147bafeb599729abf3d1abc2f4b3 Signed-off-by: Neale Ranns <nranns@cisco.com>
2017-04-07VPP-684.Add ip which mask length exceeding upper limit,ping segmentfaultflyingeagle231-1/+4
Change-Id: I2d1c5e4dbdcab2a81d6c88801183ca0bbc83e1d9 Signed-off-by: flyingeagle23 <wang.hui56@zte.com.cn>
2017-04-06Use thread local storage for thread indexDamjan Marion5-39/+39
This patch deprecates stack-based thread identification, Also removes requirement that thread stacks are adjacent. Finally, possibly annoying for some folks, it renames all occurences of cpu_index and cpu_number with thread index. Using word "cpu" is misleading here as thread can be migrated ti different CPU, and also it is not related to linux cpu index. Change-Id: I68cdaf661e701d2336fc953dcb9978d10a70f7c1 Signed-off-by: Damjan Marion <damarion@cisco.com>
2017-04-03Adjacency layout change and move to vnet/adjNeale Ranns3-217/+3
Change-Id: I03195a86c69f84a301051c6b3ab64456bbf28645 Signed-off-by: Neale Ranns <nranns@cisco.com>
2017-04-01ARP/ND/API:fix arp reg + nd no identical regsEyal Bari1-55/+38
fixed arp registration without allocating the event added protection against identical ND registrations Change-Id: I4e2db4913b35d895d8686ada1f0818920f276ad6 Signed-off-by: Eyal Bari <ebari@cisco.com>
2017-04-01MTRIE Optimisations 2Neale Ranns8-296/+526
1) 16-8-8 stride. Reduce trie depth walk traded with increased memory in the top PLY. 2) separate the vector of protocol-independent (PI) fib_table_t with the vector of protocol dependent (PD) FIBs. PD FIBs are large structures, we don't want to burn the memory for ech PD type 3) Go straight to the PD FIB in the data-path thus avoiding an indirection through, e.g., a PLY pool. Change-Id: I800d1ed0b2049040d5da95213f3ed6b12bdd78b7 Signed-off-by: Neale Ranns <nranns@cisco.com>
2017-03-30VPP-669: ping: fix coverity check error 165075 + add ping testcaseAndrew Yourtchenko1-1/+1
Fix the bug and add the unit test to start with ping test coverage Change-Id: Ibeacbed1f1660e677faa2dbb2ebe386216693e96 Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>
2017-03-30BRIDGE-DOMAIN:enable arp term,mac/ip learning on local intsEyal Bari2-7/+4
we should not suppress arps on remote interface instead we should flood them to the local one used shg != 0 to identify packets from remote interfaces and disable arp term l2 input feature for them Change-Id: I701537c05df7451a25ccb5bfc9e2b03ea70cff20 Signed-off-by: Eyal Bari <ebari@cisco.com>
2017-03-29IP lookup: remove used stack variablesNeale Ranns1-21/+5
Change-Id: I19b16d3e04a187ffe736fae868941957c70edf62 Signed-off-by: Neale Ranns <nranns@cisco.com>
2017-03-29Mtrie optimisationsNeale Ranns5-222/+191
1 - make the default route non-special, i.e. like any other less specific route. Consequently, all buckets have a valid valid index of either a leaf or a ply. Checks for special indeices in the data-path can thus be removed. 2 - since all leaves are now 'real' i.e. they represent a real load-balance object, to tell if a ply slot is 'empty' requeirs chekcing that the prefix length of the leaf occupying the slot is slot than the minium value for that ply. 3 - when removing a leaf find the cover first, then recurse down the ply and replace the old leaf with the cover. This saves us a ply walk. Change-Id: Idd523019e8bb1b6ef527b1f5279a5e24bcf18332 Signed-off-by: Neale Ranns <nranns@cisco.com>
2017-03-29Sub-net broadcast addresses for IPv4Neale Ranns1-3/+70
Change-Id: Ib2189d01e8bc61de57404159690fb70f89c47277 Signed-off-by: Neale Ranns <nranns@cisco.com>
2017-03-27Mcast rewrite no memcpyNeale Ranns4-27/+29
use a 32bit mask in the adjacency to AND with the IP address and OR into the rewrite. Change-Id: I80b0f246c18fd74f3e43c5d49e25833412f34665 Signed-off-by: Neale Ranns <nranns@cisco.com>
2017-03-22ping: allow the user to send bursts of N packets instead of single packetAndrew Yourtchenko1-10/+42
Change-Id: I3c1d3a2db56487473123e6fec2b076a063473313 Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>
2017-03-1764 bit per-thread countersNeale Ranns1-17/+19
after: TenGigabitEthernet5/0/1-output active 107522 17375708 0 7.22e0 161.60 TenGigabitEthernet5/0/1-tx active 107522 17375708 0 6.93e1 161.60 ip4-input-no-checksum active 107522 17375708 0 2.52e1 161.60 ip4-lookup active 107522 17375708 0 3.10e1 161.60 ip4-rewrite active 107522 17375708 0 2.52e1 161.60 before TenGigabitEthernet5/0/1-output active 433575 110995200 0 6.95e0 256.00 TenGigabitEthernet5/0/1-tx active 433575 110995200 0 7.14e1 256.00 ip4-input-no-checksum active 433575 110995200 0 2.66e1 256.00 ip4-lookup active 433575 110995200 0 3.29e1 256.00 ip4-rewrite active 433575 110995200 0 2.59e1 256.00 Change-Id: I46405bd22189f48a39f06e3443bb7e13f410b539 Signed-off-by: Neale Ranns <nranns@cisco.com>
2017-03-17Fix IP feature ordering.Neale Ranns1-17/+12
Drop comes before lookup when enabled. is_first_or_last is not required when setting a feature, the anchor is added in find_config_with_features(). Don't make the PG interfaces automatically L3 enabled, this way we can have tests that check the L3 protocol disbaled behaviour. Change-Id: Icef22a920b27ff9cec6ab2da6b05f05c532cb60f Signed-off-by: Neale Ranns <nranns@cisco.com>
2017-03-17Cache a 'has-features' flag on the adjacency for faster access. Reclaim the ↵Neale Ranns3-21/+29
node_index memeber from the rewrite for space - this is only used for formtting before: ip4-rewrite * * * * 2.66e1 256.00 after: ip4-rewrite * * * * 2.40e1 256.00 Change-Id: Ic397150727cad38811564777419ad6bd26b8a3a6 Signed-off-by: Neale Ranns <nranns@wasa-ucs-11.cisco.com>
2017-03-16ping: Fix coverity error 163907Andrew Yourtchenko2-0/+6
Return an error if the ip6_interface_first_address decides to return NULL. Change-Id: Iea0184382b8fbe2af81a781c3671687a1fbb5b32 Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>
2017-03-15VXLAN:add hidden multicast interface checkEyal Bari1-6/+8
and some refactoring Change-Id: I99e3c5e782ce65cb9779ccc3a9a3151ef1429e07 Signed-off-by: Eyal Bari <ebari@cisco.com>
2017-03-14ioam: ip6 hop by hop ext header and SR header co-existenceshwethab1-1/+1
1. Fix finding sr extension header 2. Fix for assert checks for space for sr header in packet headroom 3. ioam build warnings 4. Fix for SR header removal in presence of hbh ext header clib_memcpy with overlapping src/dst was failing Change-Id: I8576204eb571d1d4725a4e6976e18fe61cd1cd35 Signed-off-by: shwethab <shwetha.bhandari@gmail.com>
2017-03-09IMplementation for option to not create a FIB table entry when adding a ↵Neale Ranns3-27/+52
neighbor entry Change-Id: I952039e101031ee6a06e63f4c73d8eb359423e1a Signed-off-by: Neale Ranns <nranns@cisco.com>
2017-03-08ARP resilience in the absence of IP config on input and output interfacesNeale Ranns1-0/+15
Change-Id: Ia9c6790436f8f6337b5351e3017ef5e441a8a4e9 Signed-off-by: Neale Ranns <nranns@cisco.com> (cherry picked from commit 5e323329ed6ded6f6944c56069329207ad9069c0)
2017-03-08API change (only) to not create a FIB table entry when adding a neighbor entryNeale Ranns1-1/+6
Change-Id: I1a8edaf1e0fca89e5d24d57dbb186ee46801f661 Signed-off-by: Neale Ranns <nranns@cisco.com>
2017-03-08BUG: multiple registrations of 'vl_api_ip6nd_proxy_details_t_handlerNeale Ranns1-7/+0
Change-Id: I70e32b2196bfc8712d270f9a4365daca2504d52d Signed-off-by: Neale Ranns <nranns@cisco.com>
2017-03-08Proxy ND (RFC4389 - or a sub-set thereof). This allows the 'emulation' of ↵Neale Ranns6-62/+324
bridging. That is hosts in one sub-net reachable via differenet interfaces. Introducate a new API command: ip6 nd proxy <host-address> <interface> this indicates 2 things; 1) that host <host-address> is reachable out of interface <interface>. VPP will thus install that route. 2) NS requests sent to <host-address> will be responeded to (i.e. proxied). Change-Id: I863f967fdb5097ab3b574769c70afdbfc8d5478a Signed-off-by: Neale Ranns <nranns@cisco.com>
2017-03-07Register TCP with IP only if session is enabledFlorin Coras1-1/+2
Change-Id: I73154179e78aeae5f879125237bce593d0978fae Signed-off-by: Florin Coras <fcoras@cisco.com>
2017-03-07DHCP Multiple Servers (VPP-602, VPP-605)Neale Ranns1-0/+2
Multiple DHCP (4 and/or 6) servers can be added and removed through multiple calls to the 'set dhcp server' API. All 4/6/ discover/solicit messages will then be replicated to all servers in the list. The expectation is that the servers/system is configured in such a way that this is viable. If VSS information is providied for the clinet VRF which also has multiple servers configured, then the same VSS information is sent to each server. Likewise the source address of packets sent to from VPP to each server is the same. Change-Id: I3287cb084c84b3f612b78bc69cfcb5b9c1f8934d Signed-off-by: Neale Ranns <nranns@cisco.com>
2017-03-07In-band OAM active probe (VPP-471)AkshayaNadahalli3-32/+63
Change-Id: Icf0ddf76ba1c8b588c79387284cd0349ebc6e45f Signed-off-by: AkshayaNadahalli <anadahal@cisco.com>
2017-03-07make per-adj counters configurableNeale Ranns2-33/+67
Change-Id: Ifdd2b204ecf7d855f1269c11224b9c825311904d Signed-off-by: Neale Ranns <nranns@cisco.com>
2017-03-07Fixing loadbalancing over ECMP for recursive routesAkshayaNadahalli2-12/+10
Change-Id: Ibe1734aeb94bc17cd8d8bc6f35ca7b780aaa9599 Signed-off-by: AkshayaNadahalli <anadahal@cisco.com>
2017-03-06ioam: manycast using iOAM and SR (VPP-628)Shwetha Bhandari2-74/+122
Change-Id: I6d2afda991d771fb4a89fc3f6544f8e940a9b9f0 Signed-off-by: Shwetha Bhandari <shwethab@cisco.com>
2017-03-04Evolving SRv6 (Segment Routing for IPv6)Pablo Camarillo1-3/+30
Implements: 1.- SR Policies with several (weighted) SID lists 2.- Binding SID 3.- SR LocalSIDs with support for the following functions - End - End.X - End.DX6 - End.DX4 - End.DX2 - End.DT6 - End.DT2 - End.B6 - End.B6.Encaps 4.- SR Steering policies (to steer a traffic through an SR Policy) - Support for IPv6 traffic (IPv6 Encapsulation / SRH insertion) - Support for IPv4 traffic (IPv6 Encapsulation) - Support for L2 traffic (T.Insert / T.Encaps) 5.- Doxygen documentation 6.- Framework (APIs) to allow the definition of new SR LocalSID behaviors by means of plugins 7.- Sample SRv6 LocalSID plugin Change-Id: I2de3d126699d4f11f54c0f7f3b71420ea41fd389 Signed-off-by: Pablo Camarillo <pcamaril@cisco.com>
2017-03-03IPv6 RA improvementsNeale Ranns2-157/+123
1) tests for RA options 2) memleaks deleteing a ip6_radv_info_t 3) MLD prefix code refactoring Change-Id: I34db103994bd8fbdbbec50b202d72770dd145681 Signed-off-by: Neale Ranns <nranns@cisco.com>
2017-03-03Changing the IP table for an interface is an error if the interface already ↵Neale Ranns4-43/+76
has an address configured (VPP-601) Change-Id: I311fc264f73dd3b2b3ce9d7d1c33cd0515b36c4a Signed-off-by: Neale Ranns <nranns@cisco.com>
2017-03-02Clean up binary api message handler registration issuesDave Barach1-83/+0
Removed a fair number of "BUG" message handlers, due to conflicts with actual message handlers in api_format.c. Vpp itself had no business receiving certain messages, up to the point where we started building in relevant code from vpp_api_test. Eliminated all but one duplicate registration complaint. That one needs attention from the vxlan team since the duplicated handlers have diverged. Change-Id: Iafce5429d2f906270643b4ea5f0130e20beb4d1d Signed-off-by: Dave Barach <dave@barachs.net>
2017-03-02Remove the unused VRF ID parameter from the IP neighbour Add/Del APINeale Ranns1-2/+0
Change-Id: Icf0d72f6af1f98c86f78e586c354515ac69804aa Signed-off-by: Neale Ranns <nranns@cisco.com>
2017-03-01VPP-598: tcp stack initial commitDave Barach15-1683/+220
Change-Id: I49e5ce0aae6e4ff634024387ceaf7dbc432a0351 Signed-off-by: Dave Barach <dave@barachs.net> Signed-off-by: Florin Coras <fcoras@cisco.com>
2017-02-22VPP-635: CLI Memory leak with invalid parameterBilly McFall4-22/+60
In the CLI parsing, below is a common pattern: /* Get a line of input. */ if (!unformat_user (input, unformat_line_input, line_input)) return 0; while (unformat_check_input (line_input) != UNFORMAT_END_OF_INPUT) { if (unformat (line_input, "x")) x = 1; : else return clib_error_return (0, "unknown input `%U'", format_unformat_error, line_input); } unformat_free (line_input); The 'else' returns if an unknown string is encountered. There a memory leak because the 'unformat_free(line_input)' is not called. There is a large number of instances of this pattern. Replaced the previous pattern with: /* Get a line of input. */ if (!unformat_user (input, unformat_line_input, line_input)) return 0; while (unformat_check_input (line_input) != UNFORMAT_END_OF_INPUT) { if (unformat (line_input, "x")) x = 1; : else { error = clib_error_return (0, "unknown input `%U'", format_unformat_error, line_input); goto done: } } /* ...Remaining code... */ done: unformat_free (line_input); return error; } In multiple files, 'unformat_free (line_input);' was never called, so there was a memory leak whether an invalid string was entered or not. Also, there were multiple instance where: error = clib_error_return (0, "unknown input `%U'", format_unformat_error, line_input); used 'input' as the last parameter instead of 'line_input'. The result is that output did not contain the substring in error, instead just an empty string. Fixed all of those as well. There are a lot of file, and very mind numbing work, so tried to keep it to a pattern to avoid mistakes. Change-Id: I8902f0c32a47dd7fb3bb3471a89818571702f1d2 Signed-off-by: Billy McFall <bmcfall@redhat.com> Signed-off-by: Dave Barach <dave@barachs.net>
2017-02-20CLI extension to add multiple (S,G)s at once and time itNeale Ranns1-10/+76
Change-Id: Id17060fd0e8ac80c8cf1999b0b82d0241b3b969a Signed-off-by: Neale Ranns <nranns@cisco.com>
class="n">vlib_node_registration_t * vlib_node_next_registered (vlib_node_registration_t * c) { c = clib_elf_section_data_next (c, c->n_next_nodes * sizeof (c->next_nodes[0])); return c; } typedef struct { /* Total calls, clock ticks and vector elements processed for this node. */ u64 calls, vectors, clocks, suspends; u64 max_clock; u64 max_clock_n; u64 perf_counter_ticks; u64 perf_counter_vectors; } vlib_node_stats_t; #define foreach_vlib_node_state \ /* Input node is called each iteration of main loop. \ This is the default (zero). */ \ _ (POLLING) \ /* Input node is called when device signals an interrupt. */ \ _ (INTERRUPT) \ /* Input node is never called. */ \ _ (DISABLED) typedef enum { #define _(f) VLIB_NODE_STATE_##f, foreach_vlib_node_state #undef _ VLIB_N_NODE_STATE, } vlib_node_state_t; typedef struct vlib_node_t { /* Vector processing function for this node. */ vlib_node_function_t *function; /* Node name. */ u8 *name; /* Node name index in elog string table. */ u32 name_elog_string; /* Total statistics for this node. */ vlib_node_stats_t stats_total; /* Saved values as of last clear (or zero if never cleared). Current values are always stats_total - stats_last_clear. */ vlib_node_stats_t stats_last_clear; /* Type of this node. */ vlib_node_type_t type; /* Node index. */ u32 index; /* Index of corresponding node runtime. */ u32 runtime_index; /* Runtime data for this node. */ void *runtime_data; /* Node flags. */ u16 flags; /* Processing function keeps frame. Tells node dispatching code not to free frame after dispatch is done. */ #define VLIB_NODE_FLAG_FRAME_NO_FREE_AFTER_DISPATCH (1 << 0) /* Node counts as output/drop/punt node for stats purposes. */ #define VLIB_NODE_FLAG_IS_OUTPUT (1 << 1) #define VLIB_NODE_FLAG_IS_DROP (1 << 2) #define VLIB_NODE_FLAG_IS_PUNT (1 << 3) #define VLIB_NODE_FLAG_IS_HANDOFF (1 << 4) /* Set if current node runtime has traced vectors. */ #define VLIB_NODE_FLAG_TRACE (1 << 5) #define VLIB_NODE_FLAG_SWITCH_FROM_INTERRUPT_TO_POLLING_MODE (1 << 6) #define VLIB_NODE_FLAG_SWITCH_FROM_POLLING_TO_INTERRUPT_MODE (1 << 7) /* State for input nodes. */ u8 state; /* Number of bytes of run time data. */ u8 runtime_data_bytes; /* protocol at b->data[b->current_data] upon entry to the dispatch fn */ u8 protocol_hint; /* Number of error codes used by this node. */ u16 n_errors; /* Size of scalar and vector arguments in bytes. */ u16 scalar_size, vector_size; /* Handle/index in error heap for this node. */ u32 error_heap_handle; u32 error_heap_index; /* Error strings indexed by error code for this node. */ char **error_strings; /* Vector of next node names. Only used before next_nodes array is initialized. */ char **next_node_names; /* Next node indices for this node. */ u32 *next_nodes; /* Name of node that we are sibling of. */ char *sibling_of; /* Bitmap of all of this node's siblings. */ uword *sibling_bitmap; /* Total number of vectors sent to each next node. */ u64 *n_vectors_by_next_node; /* Hash table mapping next node index into slot in next_nodes vector. Quickly determines whether this node is connected to given next node and, if so, with which slot. */ uword *next_slot_by_node; /* Bitmap of node indices which feed this node. */ uword *prev_node_bitmap; /* Node/next-index which own enqueue rights with to this node. */ u32 owner_node_index, owner_next_index; /* Buffer format/unformat for this node. */ format_function_t *format_buffer; unformat_function_t *unformat_buffer; /* Trace buffer format/unformat for this node. */ format_function_t *format_trace; /* Function to validate incoming frames. */ u8 *(*validate_frame) (struct vlib_main_t * vm, struct vlib_node_runtime_t *, struct vlib_frame_t * f); /* for pretty-printing, not typically valid */ u8 *state_string; /* Node function candidate registration with priority */ vlib_node_fn_registration_t *node_fn_registrations; } vlib_node_t; #define VLIB_INVALID_NODE_INDEX ((u32) ~0) /* Max number of vector elements to process at once per node. */ #define VLIB_FRAME_SIZE 256 #define VLIB_FRAME_ALIGN CLIB_CACHE_LINE_BYTES /* Calling frame (think stack frame) for a node. */ typedef struct vlib_frame_t { /* Frame flags. */ u16 frame_flags; /* User flags. Used for sending hints to the next node. */ u16 flags; /* Number of scalar bytes in arguments. */ u8 scalar_size; /* Number of bytes per vector argument. */ u8 vector_size; /* Number of vector elements currently in frame. */ u16 n_vectors; /* Scalar and vector arguments to next node. */ u8 arguments[0]; } vlib_frame_t; typedef struct { /* Frame index. */ u32 frame_index; /* Node runtime for this next. */ u32 node_runtime_index; /* Next frame flags. */ u32 flags; /* Reflects node frame-used flag for this next. */ #define VLIB_FRAME_NO_FREE_AFTER_DISPATCH \ VLIB_NODE_FLAG_FRAME_NO_FREE_AFTER_DISPATCH /* This next frame owns enqueue to node corresponding to node_runtime_index. */ #define VLIB_FRAME_OWNER (1 << 15) /* Set when frame has been allocated for this next. */ #define VLIB_FRAME_IS_ALLOCATED VLIB_NODE_FLAG_IS_OUTPUT /* Set when frame has been added to pending vector. */ #define VLIB_FRAME_PENDING VLIB_NODE_FLAG_IS_DROP /* Set when frame is to be freed after dispatch. */ #define VLIB_FRAME_FREE_AFTER_DISPATCH VLIB_NODE_FLAG_IS_PUNT /* Set when frame has traced packets. */ #define VLIB_FRAME_TRACE VLIB_NODE_FLAG_TRACE /* Number of vectors enqueue to this next since last overflow. */ u32 vectors_since_last_overflow; } vlib_next_frame_t; always_inline void vlib_next_frame_init (vlib_next_frame_t * nf) { clib_memset (nf, 0, sizeof (nf[0])); nf->frame_index = ~0; nf->node_runtime_index = ~0; } /* A frame pending dispatch by main loop. */ typedef struct { /* Node and runtime for this frame. */ u32 node_runtime_index; /* Frame index (in the heap). */ u32 frame_index; /* Start of next frames for this node. */ u32 next_frame_index; /* Special value for next_frame_index when there is no next frame. */ #define VLIB_PENDING_FRAME_NO_NEXT_FRAME ((u32) ~0) } vlib_pending_frame_t; typedef struct vlib_node_runtime_t { CLIB_CACHE_LINE_ALIGN_MARK (cacheline0); /**< cacheline mark */ vlib_node_function_t *function; /**< Node function to call. */ vlib_error_t *errors; /**< Vector of errors for this node. */ #if __SIZEOF_POINTER__ == 4 u8 pad[8]; #endif u32 clocks_since_last_overflow; /**< Number of clock cycles. */ u32 max_clock; /**< Maximum clock cycle for an invocation. */ u32 max_clock_n; /**< Number of vectors in the recorded max_clock. */ u32 calls_since_last_overflow; /**< Number of calls. */ u32 vectors_since_last_overflow; /**< Number of vector elements processed by this node. */ u32 perf_counter_ticks_since_last_overflow; /**< Perf counter ticks */ u32 perf_counter_vectors_since_last_overflow; /**< Perf counter vectors */ u32 next_frame_index; /**< Start of next frames for this node. */ u32 node_index; /**< Node index. */ u32 input_main_loops_per_call; /**< For input nodes: decremented on each main loop interation until it reaches zero and function is called. Allows some input nodes to be called more than others. */ u32 main_loop_count_last_dispatch; /**< Saved main loop counter of last dispatch of this node. */ u32 main_loop_vector_stats[2]; u16 flags; /**< Copy of main node flags. */ u16 state; /**< Input node state. */ u16 n_next_nodes; u16 cached_next_index; /**< Next frame index that vector arguments were last enqueued to last time this node ran. Set to zero before first run of this node. */ u16 thread_index; /**< thread this node runs on */ u8 runtime_data[0]; /**< Function dependent node-runtime data. This data is thread local, and it is not cloned from main thread. It needs to be initialized for each thread before it is used unless runtime_data template exists in vlib_node_t. */ } vlib_node_runtime_t; #define VLIB_NODE_RUNTIME_DATA_SIZE (sizeof (vlib_node_runtime_t) - STRUCT_OFFSET_OF (vlib_node_runtime_t, runtime_data)) typedef struct { /* Number of allocated frames for this scalar/vector size. */ u32 n_alloc_frames; /* Vector of free frame indices for this scalar/vector size. */ u32 *free_frame_indices; } vlib_frame_size_t; typedef struct { /* Users opaque value for event type. */ uword opaque; } vlib_process_event_type_t; typedef struct { /* Node runtime for this process. */ vlib_node_runtime_t node_runtime; /* Where to longjmp when process is done. */ clib_longjmp_t return_longjmp; #define VLIB_PROCESS_RETURN_LONGJMP_RETURN ((uword) ~0 - 0) #define VLIB_PROCESS_RETURN_LONGJMP_SUSPEND ((uword) ~0 - 1) /* Where to longjmp to resume node after suspend. */ clib_longjmp_t resume_longjmp; #define VLIB_PROCESS_RESUME_LONGJMP_SUSPEND 0 #define VLIB_PROCESS_RESUME_LONGJMP_RESUME 1 u16 flags; #define VLIB_PROCESS_IS_SUSPENDED_WAITING_FOR_CLOCK (1 << 0) #define VLIB_PROCESS_IS_SUSPENDED_WAITING_FOR_EVENT (1 << 1) /* Set to indicate that this process has been added to resume vector. */ #define VLIB_PROCESS_RESUME_PENDING (1 << 2) /* Process function is currently running. */ #define VLIB_PROCESS_IS_RUNNING (1 << 3) /* Size of process stack. */ u16 log2_n_stack_bytes; u32 suspended_process_frame_index; /* Number of times this process was suspended. */ u32 n_suspends; /* Vectors of pending event data indexed by event type index. */ void **pending_event_data_by_type_index; /* Bitmap of event type-indices with non-empty vectors. */ uword *non_empty_event_type_bitmap; /* Bitmap of event type-indices which are one time events. */ uword *one_time_event_type_bitmap; /* Type is opaque pointer -- typically a pointer to an event handler function. Hash table to map opaque to a type index. */ uword *event_type_index_by_type_opaque; /* Pool of currently valid event types. */ vlib_process_event_type_t *event_type_pool; /* * When suspending saves clock time (10us ticks) when process * is to be resumed. */ u64 resume_clock_interval; /* Handle from timer code, to cancel an unexpired timer */ u32 stop_timer_handle; /* Default output function and its argument for any CLI outputs within the process. */ vlib_cli_output_function_t *output_function; uword output_function_arg; #ifdef CLIB_UNIX /* Pad to a multiple of the page size so we can mprotect process stacks */ #define PAGE_SIZE_MULTIPLE 0x1000 #define ALIGN_ON_MULTIPLE_PAGE_BOUNDARY_FOR_MPROTECT __attribute__ ((aligned (PAGE_SIZE_MULTIPLE))) #else #define ALIGN_ON_MULTIPLE_PAGE_BOUNDARY_FOR_MPROTECT #endif /* Process stack. Starts here and extends 2^log2_n_stack_bytes bytes. */ #define VLIB_PROCESS_STACK_MAGIC (0xdead7ead) u32 stack[0] ALIGN_ON_MULTIPLE_PAGE_BOUNDARY_FOR_MPROTECT; } vlib_process_t __attribute__ ((aligned (CLIB_CACHE_LINE_BYTES))); #ifdef CLIB_UNIX /* Ensure that the stack is aligned on the multiple of the page size */ typedef char assert_process_stack_must_be_aligned_exactly_to_page_size_multiple[(sizeof (vlib_process_t) - PAGE_SIZE_MULTIPLE) == 0 ? 0 : -1]; #endif typedef struct { u32 node_index; u32 one_time_event; } vlib_one_time_waiting_process_t; typedef struct { u16 n_data_elts; u16 n_data_elt_bytes; /* n_data_elts * n_data_elt_bytes */ u32 n_data_bytes; /* Process node & event type to be used to signal event. */ u32 process_node_index; u32 event_type_index; union { u8 inline_event_data[64 - 3 * sizeof (u32) - 2 * sizeof (u16)]; /* Vector of event data used only when data does not fit inline. */ u8 *event_data_as_vector; }; } vlib_signal_timed_event_data_t; always_inline uword vlib_timing_wheel_data_is_timed_event (u32 d) { return d & 1; } always_inline u32 vlib_timing_wheel_data_set_suspended_process (u32 i) { return 0 + 2 * i; } always_inline u32 vlib_timing_wheel_data_set_timed_event (u32 i) { return 1 + 2 * i; } always_inline uword vlib_timing_wheel_data_get_index (u32 d) { return d / 2; } typedef struct { /* Public nodes. */ vlib_node_t **nodes; /* Node index hashed by node name. */ uword *node_by_name; u32 flags; #define VLIB_NODE_MAIN_RUNTIME_STARTED (1 << 0) /* Nodes segregated by type for cache locality. Does not apply to nodes of type VLIB_NODE_TYPE_INTERNAL. */ vlib_node_runtime_t *nodes_by_type[VLIB_N_NODE_TYPE]; /* Node runtime indices for input nodes with pending interrupts. */ u32 *pending_interrupt_node_runtime_indices; clib_spinlock_t pending_interrupt_lock; /* Input nodes are switched from/to interrupt to/from polling mode when average vector length goes above/below polling/interrupt thresholds. */ u32 polling_threshold_vector_length; u32 interrupt_threshold_vector_length; /* Vector of next frames. */ vlib_next_frame_t *next_frames; /* Vector of internal node's frames waiting to be called. */ vlib_pending_frame_t *pending_frames; /* Timing wheel for scheduling time-based node dispatch. */ void *timing_wheel; vlib_signal_timed_event_data_t *signal_timed_event_data_pool; /* Opaque data vector added via timing_wheel_advance. */ u32 *data_from_advancing_timing_wheel; /* CPU time of next process to be ready on timing wheel. */ f64 time_next_process_ready; /* Vector of process nodes. One for each node of type VLIB_NODE_TYPE_PROCESS. */ vlib_process_t **processes; /* Current running process or ~0 if no process running. */ u32 current_process_index; /* Pool of pending process frames. */ vlib_pending_frame_t *suspended_process_frames; /* Vector of event data vectors pending recycle. */ void **recycled_event_data_vectors; /* Current counts of nodes in each state. */ u32 input_node_counts_by_state[VLIB_N_NODE_STATE]; /* Hash of (scalar_size,vector_size) to frame_sizes index. */ uword *frame_size_hash; /* Per-size frame allocation information. */ vlib_frame_size_t *frame_sizes; /* Time of last node runtime stats clear. */ f64 time_last_runtime_stats_clear; /* Node registrations added by constructors */ vlib_node_registration_t *node_registrations; } vlib_node_main_t; #define FRAME_QUEUE_MAX_NELTS 32 typedef struct { CLIB_CACHE_LINE_ALIGN_MARK (cacheline0); u64 head; u64 head_hint; u64 tail; u32 n_in_use; u32 nelts; u32 written; u32 threshold; i32 n_vectors[FRAME_QUEUE_MAX_NELTS]; } frame_queue_trace_t; typedef struct { u64 count[FRAME_QUEUE_MAX_NELTS]; } frame_queue_nelt_counter_t; #endif /* included_vlib_node_h */ /* * fd.io coding-style-patch-verification: ON * * Local Variables: * eval: (c-set-style "gnu") * End: */