docs: better docs, mv doxygen to sphinx

This patch refactors the VPP sphinx docs
in order to make it easier to consume
for external readers as well as VPP developers.

It also makes sphinx the single source
of documentation, which simplifies maintenance
and operation.

Most important updates are:

- reformat the existing documentation as rst
- split RELEASE.md and move it into separate rst files
- remove section 'events'
- remove section 'archive'
- remove section 'related projects'
- remove section 'feature by release'
- remove section 'Various links'
- make (Configuration reference, CLI docs,
  developer docs) top level items in the list
- move 'Use Cases' as part of 'About VPP'
- move 'Troubleshooting' as part of 'Getting Started'
- move test framework docs into 'Developer Documentation'
- add a 'Contributing' section for gerrit,
  docs and other contributer related infos
- deprecate doxygen and test-docs targets
- redirect the "make doxygen" target to "make docs"

Type: refactor

Change-Id: I552a5645d5b7964d547f99b1336e2ac24e7c209f
Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com>
Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>

1 files changed, 158 insertions, 0 deletions

diff --git a/docs/usecases/contiv/Prometheus.rst b/docs/usecases/contiv/Prometheus.rst
new file mode 100644
index 00000000000..cb4c1646adb
--- /dev/null
+++ b/docs/usecases/contiv/Prometheus.rst
@@ -0,0 +1,158 @@
+Prometheus Statistics
+=====================
+
+Each contiv-agent exposes statistics in Prometheus format at port
+``9999`` by default. Exposed data is split into two groups: - ``/stats``
+provides statistics for VPP interfaces managed by contiv-agent
+Prometheus data is a set of counters with labels. For each interface,
+the following counters are exposed: \* *inPackets* \* *outPackets* \*
+*inBytes* \* *outBytes* \* *ipv4Packets* \* *ipv6Packets* \*
+*outErrorPackets* \* *dropPackets* \* *inMissPackets* \*
+*inNobufPackets* \* *puntPackets*
+
+Labels let you add additional information to a counter. The
+*interfaceName* and *node* labels are specified for all counters. If an
+interface is associated with a particular pod, then the *podName* and
+*podNamespace* labels are also specified for its counters; otherwise, a
+placeholder value (``--``) is used (for example, for node interconnect
+interfaces). - ``/metrics`` provides general go runtime statistics
+
+To access Prometheus stats of a node you can use
+``curl localhost:9999/stats`` from the node. The output of contiv-agent
+running at k8s master node looks similar to the following:
+
+::
+
+   $ curl localhost:9999/stats
+   # HELP dropPackets Number of dropped packets for interface
+   # TYPE dropPackets gauge
+   dropPackets{interfaceName="GigabitEthernet0/9/0",node="dev",podName="--",podNamespace="--"} 0
+   dropPackets{interfaceName="tap-vpp2",node="dev",podName="--",podNamespace="--"} 52
+   dropPackets{interfaceName="tap0e6439a7a934336",node="dev",podName="web-667bdcb4d8-pxkfs",podNamespace="default"} 9
+   dropPackets{interfaceName="tap5338a3285ad6bd7",node="dev",podName="kube-dns-6f4fd4bdf-rsz9b",podNamespace="kube-system"} 12
+   dropPackets{interfaceName="vxlanBVI",node="dev",podName="--",podNamespace="--"} 0
+   # HELP inBytes Number of received bytes for interface
+   # TYPE inBytes gauge
+   inBytes{interfaceName="GigabitEthernet0/9/0",node="dev",podName="--",podNamespace="--"} 0
+   inBytes{interfaceName="tap-vpp2",node="dev",podName="--",podNamespace="--"} 24716
+   inBytes{interfaceName="tap0e6439a7a934336",node="dev",podName="web-667bdcb4d8-pxkfs",podNamespace="default"} 726
+   inBytes{interfaceName="tap5338a3285ad6bd7",node="dev",podName="kube-dns-6f4fd4bdf-rsz9b",podNamespace="kube-system"} 6113
+   inBytes{interfaceName="vxlanBVI",node="dev",podName="--",podNamespace="--"} 0
+   # HELP inErrorPackets Number of received packets with error for interface
+   # TYPE inErrorPackets gauge
+   inErrorPackets{interfaceName="GigabitEthernet0/9/0",node="dev",podName="--",podNamespace="--"} 0
+   inErrorPackets{interfaceName="tap-vpp2",node="dev",podName="--",podNamespace="--"} 0
+   inErrorPackets{interfaceName="tap0e6439a7a934336",node="dev",podName="web-667bdcb4d8-pxkfs",podNamespace="default"} 0
+   inErrorPackets{interfaceName="tap5338a3285ad6bd7",node="dev",podName="kube-dns-6f4fd4bdf-rsz9b",podNamespace="kube-system"} 0
+   inErrorPackets{interfaceName="vxlanBVI",node="dev",podName="--",podNamespace="--"} 0
+   # HELP inMissPackets Number of missed packets for interface
+   # TYPE inMissPackets gauge
+   inMissPackets{interfaceName="GigabitEthernet0/9/0",node="dev",podName="--",podNamespace="--"} 0
+   inMissPackets{interfaceName="tap-vpp2",node="dev",podName="--",podNamespace="--"} 0
+   inMissPackets{interfaceName="tap0e6439a7a934336",node="dev",podName="web-667bdcb4d8-pxkfs",podNamespace="default"} 0
+   inMissPackets{interfaceName="tap5338a3285ad6bd7",node="dev",podName="kube-dns-6f4fd4bdf-rsz9b",podNamespace="kube-system"} 0
+   inMissPackets{interfaceName="vxlanBVI",node="dev",podName="--",podNamespace="--"} 0
+   # HELP inNobufPackets Number of received packets ??? for interface
+   # TYPE inNobufPackets gauge
+   inNobufPackets{interfaceName="GigabitEthernet0/9/0",node="dev",podName="--",podNamespace="--"} 0
+   inNobufPackets{interfaceName="tap-vpp2",node="dev",podName="--",podNamespace="--"} 0
+   inNobufPackets{interfaceName="tap0e6439a7a934336",node="dev",podName="web-667bdcb4d8-pxkfs",podNamespace="default"} 0
+   inNobufPackets{interfaceName="tap5338a3285ad6bd7",node="dev",podName="kube-dns-6f4fd4bdf-rsz9b",podNamespace="kube-system"} 0
+   inNobufPackets{interfaceName="vxlanBVI",node="dev",podName="--",podNamespace="--"} 0
+   # HELP inPackets Number of received packets for interface
+   # TYPE inPackets gauge
+   inPackets{interfaceName="GigabitEthernet0/9/0",node="dev",podName="--",podNamespace="--"} 0
+   inPackets{interfaceName="tap-vpp2",node="dev",podName="--",podNamespace="--"} 97
+   inPackets{interfaceName="tap0e6439a7a934336",node="dev",podName="web-667bdcb4d8-pxkfs",podNamespace="default"} 9
+   inPackets{interfaceName="tap5338a3285ad6bd7",node="dev",podName="kube-dns-6f4fd4bdf-rsz9b",podNamespace="kube-system"} 60
+   inPackets{interfaceName="vxlanBVI",node="dev",podName="--",podNamespace="--"} 0
+   # HELP ipv4Packets Number of ipv4 packets for interface
+   # TYPE ipv4Packets gauge
+   ipv4Packets{interfaceName="GigabitEthernet0/9/0",node="dev",podName="--",podNamespace="--"} 0
+   ipv4Packets{interfaceName="tap-vpp2",node="dev",podName="--",podNamespace="--"} 68
+   ipv4Packets{interfaceName="tap0e6439a7a934336",node="dev",podName="web-667bdcb4d8-pxkfs",podNamespace="default"} 0
+   ipv4Packets{interfaceName="tap5338a3285ad6bd7",node="dev",podName="kube-dns-6f4fd4bdf-rsz9b",podNamespace="kube-system"} 52
+   ipv4Packets{interfaceName="vxlanBVI",node="dev",podName="--",podNamespace="--"} 0
+   # HELP ipv6Packets Number of ipv6 packets for interface
+   # TYPE ipv6Packets gauge
+   ipv6Packets{interfaceName="GigabitEthernet0/9/0",node="dev",podName="--",podNamespace="--"} 0
+   ipv6Packets{interfaceName="tap-vpp2",node="dev",podName="--",podNamespace="--"} 26
+   ipv6Packets{interfaceName="tap0e6439a7a934336",node="dev",podName="web-667bdcb4d8-pxkfs",podNamespace="default"} 9
+   ipv6Packets{interfaceName="tap5338a3285ad6bd7",node="dev",podName="kube-dns-6f4fd4bdf-rsz9b",podNamespace="kube-system"} 8
+   ipv6Packets{interfaceName="vxlanBVI",node="dev",podName="--",podNamespace="--"} 0
+   # HELP outBytes Number of transmitted bytes for interface
+   # TYPE outBytes gauge
+   outBytes{interfaceName="GigabitEthernet0/9/0",node="dev",podName="--",podNamespace="--"} 0
+   outBytes{interfaceName="tap-vpp2",node="dev",podName="--",podNamespace="--"} 5203
+   outBytes{interfaceName="tap0e6439a7a934336",node="dev",podName="web-667bdcb4d8-pxkfs",podNamespace="default"} 0
+   outBytes{interfaceName="tap5338a3285ad6bd7",node="dev",podName="kube-dns-6f4fd4bdf-rsz9b",podNamespace="kube-system"} 17504
+   outBytes{interfaceName="vxlanBVI",node="dev",podName="--",podNamespace="--"} 0
+   # HELP outErrorPackets Number of transmitted packets with error for interface
+   # TYPE outErrorPackets gauge
+   outErrorPackets{interfaceName="GigabitEthernet0/9/0",node="dev",podName="--",podNamespace="--"} 0
+   outErrorPackets{interfaceName="tap-vpp2",node="dev",podName="--",podNamespace="--"} 0
+   outErrorPackets{interfaceName="tap0e6439a7a934336",node="dev",podName="web-667bdcb4d8-pxkfs",podNamespace="default"} 0
+   outErrorPackets{interfaceName="tap5338a3285ad6bd7",node="dev",podName="kube-dns-6f4fd4bdf-rsz9b",podNamespace="kube-system"} 0
+   outErrorPackets{interfaceName="vxlanBVI",node="dev",podName="--",podNamespace="--"} 0
+   # HELP outPackets Number of transmitted packets for interface
+   # TYPE outPackets gauge
+   outPackets{interfaceName="GigabitEthernet0/9/0",node="dev",podName="--",podNamespace="--"} 0
+   outPackets{interfaceName="tap-vpp2",node="dev",podName="--",podNamespace="--"} 49
+   outPackets{interfaceName="tap0e6439a7a934336",node="dev",podName="web-667bdcb4d8-pxkfs",podNamespace="default"} 0
+   outPackets{interfaceName="tap5338a3285ad6bd7",node="dev",podName="kube-dns-6f4fd4bdf-rsz9b",podNamespace="kube-system"} 45
+   outPackets{interfaceName="vxlanBVI",node="dev",podName="--",podNamespace="--"} 0
+   # HELP puntPackets Number of punt packets for interface
+   # TYPE puntPackets gauge
+   puntPackets{interfaceName="GigabitEthernet0/9/0",node="dev",podName="--",podNamespace="--"} 0
+   puntPackets{interfaceName="tap-vpp2",node="dev",podName="--",podNamespace="--"} 0
+   puntPackets{interfaceName="tap0e6439a7a934336",node="dev",podName="web-667bdcb4d8-pxkfs",podNamespace="default"} 0
+   puntPackets{interfaceName="tap5338a3285ad6bd7",node="dev",podName="kube-dns-6f4fd4bdf-rsz9b",podNamespace="kube-system"} 0
+   puntPackets{interfaceName="vxlanBVI",node="dev",podName="--",podNamespace="--"} 0
+
+In order to browse stats in web UI Prometheus, it must be started
+locally by following the information in the `Prometheus Getting Started
+Guide <https://prometheus.io/docs/prometheus/latest/getting_started/>`__.
+
+If you start Prometheus on a node, the following sample config can be
+used:
+
+.. code:: yaml
+
+   global:
+     scrape_interval:     15s
+
+   scrape_configs:
+     - job_name: 'contiv_stats'
+       metrics_path: '/stats'
+       static_configs:
+         - targets: ['localhost:9999']
+     - job_name: 'contiv_agent'
+       # metrics_path defaults to '/metrics'
+       static_configs:
+         - targets: ['localhost:9999']
+
+Once Prometheus is started with the specified config, you should be able
+access its web UI at ``localhost:9090``.
+
+::
+
+   tester@dev:~/Downloads/prometheus-2.1.0.linux-amd64$ ./prometheus --config.file=config.yml
+
+If security features are enabled for the HTTP endpoint, then the config
+must be adjusted:
+
+.. code:: yaml
+
+    - job_name: 'contiv_secured'
+
+        scheme: https
+        basic_auth:
+           username: user
+           password: pass
+        metrics_path: /stats
+        tls_config:
+          insecure_skip_verify: true
+          # CA certificate to validate API server certificate with.
+          #[ ca_file: <filename> ]
+        static_configs:
+          - targets: ['localhost:9191']