1 files changed, 178 insertions, 0 deletions
diff --git a/src/vpp/stats/stats.rst b/src/vpp/stats/stats.rst
new file mode 100644
index 00000000000..26e4db8c0db
--- /dev/null
+++ b/src/vpp/stats/stats.rst
@@ -0,0 +1,178 @@
+.. _stats_doc:
+
+Statistics
+==========
+
+In VPP most things are measured and counted. There are counters for
+interface statistics, like RX, TX counters, packet drops, and so on.
+Every node has a set of per-node counters, one set of error counters,
+like TTL exceeded, or packet to big or out-of-buffers. And a set of
+performance counters, like number of clocks, vectors, calls and
+suspends.
+
+There is also a set of system counters and performance counters,
+e.g. memory utilization per heap, buffer utilisation and so on.
+
+VPP Counter Architecture
+------------------------
+
+Counters are exposed directly via shared memory. These are the actual
+counters in VPP, no sampling or aggregation is done by the statistics
+infrastructure. With the exception of per node performance data under
+/sys/node and a few system counters.
+
+Clients mount the shared memory segment read-only, using a optimistic
+concurrency algorithm.
+
+Directory structure as an index.
+
+Memory layout
+~~~~~~~~~~~~~
+
+The memory segment consists of a shared header, containing atomics for
+the optimistic concurrency mechanism, and offsets into memory for the
+directory vectors. The only data structure used is the VPP vectors. All
+pointers are converted to offsets so that client applications can map
+the shared memory wherever it pleases.
+
+Directory layout
+~~~~~~~~~~~~~~~~
+
+Optimistic concurrency
+~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+   /*
+    * Shared header first in the shared memory segment.
+    */
+   typedef struct {
+     atomic_int_fast64_t epoch;
+     atomic_int_fast64_t in_progress;
+     atomic_int_fast64_t directory_offset;
+     atomic_int_fast64_t error_offset;
+     atomic_int_fast64_t stats_offset;
+   } stat_segment_shared_header_t;
+
+Writer
+^^^^^^
+
+On the VPP side there is a single writer (controlled by a spinlock).
+When the writer starts it sets in_progress=1, continues with the update
+of the data-structures, and when done, bumps epoch++ and sets
+in_progress=0.
+
+Readers
+^^^^^^^
+
+If in_progress=1, there is no point continuing, so reader sits spinning
+on the in_progress flag until it is 0. Then it sets start_epoch = epoch
+and continues copying out the counter data it is interested in, while
+doing strict boundary checks on all offsets / pointers. When the reader
+is done, it checks if in_progress=1 or if epoch != start_epoch. If
+either of those are true is discards the data read.
+
+How are counters exposed out of VPP?
+------------------------------------
+
+Types of Counters
+-----------------
+
+All counters under /err and /if are the directly exposed VPP counters.
+
+-  Gauges
+-  u64 / float
+-  Interface Counters
+-  Simple counters, counter_t array of threads of an array of interfaces
+-  Combined counters, vlib_counter_t array of threads of an array of
+   interfaces.
+
+Client libraries
+----------------
+
+Writing a new client library
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+A new client library can either wrap the C library (libvppapiclient.so)
+or it can integrate directly with the shared memory. That involves
+exchanging a file descriptor over the VPP stats Unix domain socket, and
+opening the memory mapped segment.
+
+Python
+~~~~~~
+
+::
+
+   #!/usr/bin/env python3
+   from vpp_papi.vpp_stats import VPPStats
+   stats = VPPStats('/run/vpp/stats.sock')
+   dir = stats.ls(['^/if', '/err/ip4-input', '/sys/node/ip4-input'])
+   counters = stats.dump(dir)
+
+   # Print the RX counters for the first interface on the first worker core
+   print ('RX interface core 0, sw_if_index 0', counters['/if/rx'][0][0])
+
+C
+~
+
+::
+
+   #include <vpp-api/client/stat_client.h>
+   #include <vppinfra/vec.h>
+
+   int main (int argc, char **argv) {
+     uint8_t *patterns = 0;
+
+     vec_add1(patterns, "^/if");
+     vec_add1(patterns, "ip4-input");
+
+     int rv = stat_segment_connect(STAT_SEGMENT_SOCKET_FILE);
+     uint32_t *dir = stat_segment_ls(patterns);
+     stat_segment_data_t *res = stat_segment_dump(dir);
+
+     for (int i = 0; i < vec_len(res); i++) {
+       switch (res[i].type) {
+         case STAT_DIR_TYPE_COUNTER_VECTOR_SIMPLE:
+         for (k = 0; k < vec_len (res[i].simple_counter_vec) - 1; k++)
+           for (j = 0; j < vec_len (res[i].simple_counter_vec[k]); j++)
+             fformat (stdout, "[%d @ %d]: %llu packets %s\n",
+                      j, k, res[i].simple_counter_vec[k][j],
+                      res[i].name);
+         break;
+
+         case STAT_DIR_TYPE_COUNTER_VECTOR_COMBINED:
+           for (k = 0; k < vec_len (res[i].combined_counter_vec); k++)
+             for (j = 0; j < vec_len (res[i].combined_counter_vec[k]); j++)
+               fformat (stdout, "[%d @ %d]: %llu packets, %llu bytes %s\n",
+                        j, k, res[i].combined_counter_vec[k][j].packets,
+                        res[i].combined_counter_vec[k][j].bytes,
+                        res[i].name);
+         break;
+
+         case STAT_DIR_TYPE_ERROR_INDEX:
+        for (j = 0; j < vec_len (res[i].error_vector); j++)
+          fformat (stdout, "[@%d] %llu %s\n", j, res[i].error_vector[j], res[i].name);
+         break;
+
+         case STAT_DIR_TYPE_SCALAR_INDEX:
+           fformat (stdout, "%.2f %s\n", res[i].scalar_value, res[i].name);
+         break;
+
+         default:
+           ;
+       }
+     }
+     stat_segment_data_free (res);
+   }
+
+Integrations
+------------
+
+-  CLI command. vpp_get_stats [ls \| dump \| poll]
+-  Prometheus
+
+Future evolution
+----------------
+
+-  Deprecate the stats over binary API calls that are based on
+   want_stats