diff options
Diffstat (limited to 'docs/gettingstarted/developers/infrastructure.md')
| -rw-r--r-- | docs/gettingstarted/developers/infrastructure.md | 330 |
1 files changed, 330 insertions, 0 deletions
diff --git a/docs/gettingstarted/developers/infrastructure.md b/docs/gettingstarted/developers/infrastructure.md new file mode 100644 index 00000000000..688c42133ed --- /dev/null +++ b/docs/gettingstarted/developers/infrastructure.md @@ -0,0 +1,330 @@ +VPPINFRA (Infrastructure) +========================= + +The files associated with the VPP Infrastructure layer are located in +the ./src/vppinfra folder. + +VPPinfra is a collection of basic c-library services, quite +sufficient to build standalone programs to run directly on bare metal. +It also provides high-performance dynamic arrays, hashes, bitmaps, +high-precision real-time clock support, fine-grained event-logging, and +data structure serialization. + +One fair comment / fair warning about vppinfra: you can\'t always tell a +macro from an inline function from an ordinary function simply by name. +Macros are used to avoid function calls in the typical case, and to +cause (intentional) side-effects. + +Vppinfra has been around for almost 20 years and tends not to change +frequently. The VPP Infrastructure layer contains the following +functions: + +Vectors +------- + +Vppinfra vectors are ubiquitous dynamically resized arrays with by user +defined \"headers\". Many vpppinfra data structures (e.g. hash, heap, +pool) are vectors with various different headers. + +The memory layout looks like this: + +``` + User header (optional, uword aligned) + Alignment padding (if needed) + Vector length in elements + User's pointer -> Vector element 0 + Vector element 1 + ... + Vector element N-1 +``` + +As shown above, the vector APIs deal with pointers to the 0th element of +a vector. Null pointers are valid vectors of length zero. + +To avoid thrashing the memory allocator, one often resets the length of +a vector to zero while retaining the memory allocation. Set the vector +length field to zero via the vec\_reset\_length(v) macro. \[Use the +macro! It's smart about NULL pointers.\] + +Typically, the user header is not present. User headers allow for other +data structures to be built atop vppinfra vectors. Users may specify the +alignment for data elements via the [vec]()\*\_aligned macros. + +Vectors elements can be any C type e.g. (int, double, struct bar). This +is also true for data types built atop vectors (e.g. heap, pool, etc.). +Many macros have \_a variants supporting alignment of vector data and +\_h variants supporting non-zero-length vector headers. The \_ha +variants support both. + +Inconsistent usage of header and/or alignment related macro variants +will cause delayed, confusing failures. + +Standard programming error: memorize a pointer to the ith element of a +vector, and then expand the vector. Vectors expand by 3/2, so such code +may appear to work for a period of time. Correct code almost always +memorizes vector **indices** which are invariant across reallocations. + +In typical application images, one supplies a set of global functions +designed to be called from gdb. Here are a few examples: + +- vl(v) - prints vec\_len(v) +- pe(p) - prints pool\_elts(p) +- pifi(p, index) - prints pool\_is\_free\_index(p, index) +- debug\_hex\_bytes (p, nbytes) - hex memory dump nbytes starting at p + +Use the "show gdb" debug CLI command to print the current set. + +Bitmaps +------- + +Vppinfra bitmaps are dynamic, built using the vppinfra vector APIs. +Quite handy for a variety jobs. + +Pools +----- + +Vppinfra pools combine vectors and bitmaps to rapidly allocate and free +fixed-size data structures with independent lifetimes. Pools are perfect +for allocating per-session structures. + +Hashes +------ + +Vppinfra provides several hash flavors. Data plane problems involving +packet classification / session lookup often use +./src/vppinfra/bihash\_template.\[ch\] bounded-index extensible +hashes. These templates are instantiated multiple times, to efficiently +service different fixed-key sizes. + +Bihashes are thread-safe. Read-locking is not required. A simple +spin-lock ensures that only one thread writes an entry at a time. + +The original vppinfra hash implementation in +./src/vppinfra/hash.\[ch\] are simple to use, and are often used in +control-plane code which needs exact-string-matching. + +In either case, one almost always looks up a key in a hash table to +obtain an index in a related vector or pool. The APIs are simple enough, +but one must take care when using the unmanaged arbitrary-sized key +variant. Hash\_set\_mem (hash\_table, key\_pointer, value) memorizes +key\_pointer. It is usually a bad mistake to pass the address of a +vector element as the second argument to hash\_set\_mem. It is perfectly +fine to memorize constant string addresses in the text segment. + +Format +------ + +Vppinfra format is roughly equivalent to printf. + +Format has a few properties worth mentioning. Format's first argument is +a (u8 \*) vector to which it appends the result of the current format +operation. Chaining calls is very easy: + +```c + u8 * result; + + result = format (0, "junk = %d, ", junk); + result = format (result, "more junk = %d\n", more_junk); +``` + +As previously noted, NULL pointers are perfectly proper 0-length +vectors. Format returns a (u8 \*) vector, **not** a C-string. If you +wish to print a (u8 \*) vector, use the "%v" format string. If you need +a (u8 \*) vector which is also a proper C-string, either of these +schemes may be used: + +```c + vec_add1 (result, 0) + or + result = format (result, "<whatever>%c", 0); +``` + +Remember to vec\_free() the result if appropriate. Be careful not to +pass format an uninitialized (u8 \*). + +Format implements a particularly handy user-format scheme via the "%U" +format specification. For example: + +```c + u8 * format_junk (u8 * s, va_list *va) + { + junk = va_arg (va, u32); + s = format (s, "%s", junk); + return s; + } + + result = format (0, "junk = %U, format_junk, "This is some junk"); +``` + +format\_junk() can invoke other user-format functions if desired. The +programmer shoulders responsibility for argument type-checking. It is +typical for user format functions to blow up if the va\_arg(va, +type) macros don't match the caller's idea of reality. + +Unformat +-------- + +Vppinfra unformat is vaguely related to scanf, but considerably more +general. + +A typical use case involves initializing an unformat\_input\_t from +either a C-string or a (u8 \*) vector, then parsing via unformat() as +follows: + +```c + unformat_input_t input; + + unformat_init_string (&input, "<some-C-string>"); + /* or */ + unformat_init_vector (&input, <u8-vector>); +``` + +Then loop parsing individual elements: + +```c + while (unformat_check_input (&input) != UNFORMAT_END_OF_INPUT) + { + if (unformat (&input, "value1 %d", &value1)) + ;/* unformat sets value1 */ + else if (unformat (&input, "value2 %d", &value2) + ;/* unformat sets value2 */ + else + return clib_error_return (0, "unknown input '%U'", + format_unformat_error, input); + } +``` + +As with format, unformat implements a user-unformat function capability +via a "%U" user unformat function scheme. + +Vppinfra errors and warnings +---------------------------- + +Many functions within the vpp dataplane have return-values of type +clib\_error\_t \*. Clib\_error\_t's are arbitrary strings with a bit of +metadata \[fatal, warning\] and are easy to announce. Returning a NULL +clib\_error\_t \* indicates "A-OK, no error." + +Clib\_warning(format-args) is a handy way to add debugging +output; clib warnings prepend function:line info to unambiguously locate +the message source. Clib\_unix\_warning() adds perror()-style Linux +system-call information. In production images, clib\_warnings result in +syslog entries. + +Serialization +------------- + +Vppinfra serialization support allows the programmer to easily serialize +and unserialize complex data structures. + +The underlying primitive serialize/unserialize functions use network +byte-order, so there are no structural issues serializing on a +little-endian host and unserializing on a big-endian host. + +Event-logger, graphical event log viewer +---------------------------------------- + +The vppinfra event logger provides very lightweight (sub-100ns) +precisely time-stamped event-logging services. See +./src/vppinfra/{elog.c, elog.h} + +Serialization support makes it easy to save and ultimately to combine a +set of event logs. In a distributed system running NTP over a local LAN, +we find that event logs collected from multiple system elements can be +combined with a temporal uncertainty no worse than 50us. + +A typical event definition and logging call looks like this: + +```c + ELOG_TYPE_DECLARE (e) = + { + .format = "tx-msg: stream %d local seq %d attempt %d", + .format_args = "i4i4i4", + }; + struct { u32 stream_id, local_sequence, retry_count; } * ed; + ed = ELOG_DATA (m->elog_main, e); + ed->stream_id = stream_id; + ed->local_sequence = local_sequence; + ed->retry_count = retry_count; +``` + +The ELOG\_DATA macro returns a pointer to 20 bytes worth of arbitrary +event data, to be formatted (offline, not at runtime) as described by +format\_args. Aside from obvious integer formats, the CLIB event logger +provides a couple of interesting additions. The "t4" format +pretty-prints enumerated values: + +```c + ELOG_TYPE_DECLARE (e) = + { + .format = "get_or_create: %s", + .format_args = "t4", + .n_enum_strings = 2, + .enum_strings = { "old", "new", }, + }; +``` + +The "t" format specifier indicates that the corresponding datum is an +index in the event's set of enumerated strings, as shown in the previous +event type definition. + +The “T” format specifier indicates that the corresponding datum is an +index in the event log’s string heap. This allows the programmer to emit +arbitrary formatted strings. One often combines this facility with a +hash table to keep the event-log string heap from growing arbitrarily +large. + +Noting the 20-octet limit per-log-entry data field, the event log +formatter supports arbitrary combinations of these data types. As in: +the ".format" field may contain one or more instances of the following: + +- i1 - 8-bit unsigned integer +- i2 - 16-bit unsigned integer +- i4 - 32-bit unsigned integer +- i8 - 64-bit unsigned integer +- f4 - float +- f8 - double +- s - NULL-terminated string - be careful +- sN - N-byte character array +- t1,2,4 - per-event enumeration ID +- T4 - Event-log string table offset + +The vpp engine event log is thread-safe, and is shared by all threads. +Take care not to serialize the computation. Although the event-logger is +about as fast as practicable, it's not appropriate for per-packet use in +hard-core data plane code. It's most appropriate for capturing rare +events - link up-down events, specific control-plane events and so +forth. + +The vpp engine has several debug CLI commands for manipulating its event +log: + +``` + vpp# event-logger clear + vpp# event-logger save <filename> # for security, writes into /tmp/<filename>. + # <filename> must not contain '.' or '/' characters + vpp# show event-logger [all] [<nnn>] # display the event log + # by default, the last 250 entries +``` + +The event log defaults to 128K entries. The command-line argument "... +vlib { elog-events nnn } ..." configures the size of the event log. + +As described above, the vpp engine event log is thread-safe and shared. +To avoid confusing non-appearance of events logged by worker threads, +make sure to code vlib\_global\_main.elog\_main - instead of +vm->elog\_main. The latter form is correct in the main thread, but +will almost certainly produce bad results in worker threads. + +G2 graphical event viewer +------------------------- + +The g2 graphical event viewer can display serialized vppinfra event logs +directly, or via the c2cpel tool. + +<div class="admonition note"> + +Todo: please convert wiki page and figures + +</div> + |