summaryrefslogtreecommitdiffstats
path: root/docs/usecases/contiv/Prometheus.rst
blob: cb4c1646adb87d03605c37ecc0a36a9ff997a334 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
Prometheus Statistics
=====================

Each contiv-agent exposes statistics in Prometheus format at port
``9999`` by default. Exposed data is split into two groups: - ``/stats``
provides statistics for VPP interfaces managed by contiv-agent
Prometheus data is a set of counters with labels. For each interface,
the following counters are exposed: \* *inPackets* \* *outPackets* \*
*inBytes* \* *outBytes* \* *ipv4Packets* \* *ipv6Packets* \*
*outErrorPackets* \* *dropPackets* \* *inMissPackets* \*
*inNobufPackets* \* *puntPackets*

Labels let you add additional information to a counter. The
*interfaceName* and *node* labels are specified for all counters. If an
interface is associated with a particular pod, then the *podName* and
*podNamespace* labels are also specified for its counters; otherwise, a
placeholder value (``--``) is used (for example, for node interconnect
interfaces). - ``/metrics`` provides general go runtime statistics

To access Prometheus stats of a node you can use
``curl localhost:9999/stats`` from the node. The output of contiv-agent
running at k8s master node looks similar to the following:

::

   $ curl localhost:9999/stats
   # HELP dropPackets Number of dropped packets for interface
   # TYPE dropPackets gauge
   dropPackets{interfaceName="GigabitEthernet0/9/0",node="dev",podName="--",podNamespace="--"} 0
   dropPackets{interfaceName="tap-vpp2",node="dev",podName="--",podNamespace="--"} 52
   dropPackets{interfaceName="tap0e6439a7a934336",node="dev",podName="web-667bdcb4d8-pxkfs",podNamespace="default"} 9
   dropPackets{interfaceName="tap5338a3285ad6bd7",node="dev",podName="kube-dns-6f4fd4bdf-rsz9b",podNamespace="kube-system"} 12
   dropPackets{interfaceName="vxlanBVI",node="dev",podName="--",podNamespace="--"} 0
   # HELP inBytes Number of received bytes for interface
   # TYPE inBytes gauge
   inBytes{interfaceName="GigabitEthernet0/9/0",node="dev",podName="--",podNamespace="--"} 0
   inBytes{interfaceName="tap-vpp2",node="dev",podName="--",podNamespace="--"} 24716
   inBytes{interfaceName="tap0e6439a7a934336",node="dev",podName="web-667bdcb4d8-pxkfs",podNamespace="default"} 726
   inBytes{interfaceName="tap5338a3285ad6bd7",node="dev",podName="kube-dns-6f4fd4bdf-rsz9b",podNamespace="kube-system"} 6113
   inBytes{interfaceName="vxlanBVI",node="dev",podName="--",podNamespace="--"} 0
   # HELP inErrorPackets Number of received packets with error for interface
   # TYPE inErrorPackets gauge
   inErrorPackets{interfaceName="GigabitEthernet0/9/0",node="dev",podName="--",podNamespace="--"} 0
   inErrorPackets{interfaceName="tap-vpp2",node="dev",podName="--",podNamespace="--"} 0
   inErrorPackets{interfaceName="tap0e6439a7a934336",node="dev",podName="web-667bdcb4d8-pxkfs",podNamespace="default"} 0
   inErrorPackets{interfaceName="tap5338a3285ad6bd7",node="dev",podName="kube-dns-6f4fd4bdf-rsz9b",podNamespace="kube-system"} 0
   inErrorPackets{interfaceName="vxlanBVI",node="dev",podName="--",podNamespace="--"} 0
   # HELP inMissPackets Number of missed packets for interface
   # TYPE inMissPackets gauge
   inMissPackets{interfaceName="GigabitEthernet0/9/0",node="dev",podName="--",podNamespace="--"} 0
   inMissPackets{interfaceName="tap-vpp2",node="dev",podName="--",podNamespace="--"} 0
   inMissPackets{interfaceName="tap0e6439a7a934336",node="dev",podName="web-667bdcb4d8-pxkfs",podNamespace="default"} 0
   inMissPackets{interfaceName="tap5338a3285ad6bd7",node="dev",podName="kube-dns-6f4fd4bdf-rsz9b",podNamespace="kube-system"} 0
   inMissPackets{interfaceName="vxlanBVI",node="dev",podName="--",podNamespace="--"} 0
   # HELP inNobufPackets Number of received packets ??? for interface
   # TYPE inNobufPackets gauge
   inNobufPackets{interfaceName="GigabitEthernet0/9/0",node="dev",podName="--",podNamespace="--"} 0
   inNobufPackets{interfaceName="tap-vpp2",node="dev",podName="--",podNamespace="--"} 0
   inNobufPackets{interfaceName="tap0e6439a7a934336",node="dev",podName="web-667bdcb4d8-pxkfs",podNamespace="default"} 0
   inNobufPackets{interfaceName="tap5338a3285ad6bd7",node="dev",podName="kube-dns-6f4fd4bdf-rsz9b",podNamespace="kube-system"} 0
   inNobufPackets{interfaceName="vxlanBVI",node="dev",podName="--",podNamespace="--"} 0
   # HELP inPackets Number of received packets for interface
   # TYPE inPackets gauge
   inPackets{interfaceName="GigabitEthernet0/9/0",node="dev",podName="--",podNamespace="--"} 0
   inPackets{interfaceName="tap-vpp2",node="dev",podName="--",podNamespace="--"} 97
   inPackets{interfaceName="tap0e6439a7a934336",node="dev",podName="web-667bdcb4d8-pxkfs",podNamespace="default"} 9
   inPackets{interfaceName="tap5338a3285ad6bd7",node="dev",podName="kube-dns-6f4fd4bdf-rsz9b",podNamespace="kube-system"} 60
   inPackets{interfaceName="vxlanBVI",node="dev",podName="--",podNamespace="--"} 0
   # HELP ipv4Packets Number of ipv4 packets for interface
   # TYPE ipv4Packets gauge
   ipv4Packets{interfaceName="GigabitEthernet0/9/0",node="dev",podName="--",podNamespace="--"} 0
   ipv4Packets{interfaceName="tap-vpp2",node="dev",podName="--",podNamespace="--"} 68
   ipv4Packets{interfaceName="tap0e6439a7a934336",node="dev",podName="web-667bdcb4d8-pxkfs",podNamespace="default"} 0
   ipv4Packets{interfaceName="tap5338a3285ad6bd7",node="dev",podName="kube-dns-6f4fd4bdf-rsz9b",podNamespace="kube-system"} 52
   ipv4Packets{interfaceName="vxlanBVI",node="dev",podName="--",podNamespace="--"} 0
   # HELP ipv6Packets Number of ipv6 packets for interface
   # TYPE ipv6Packets gauge
   ipv6Packets{interfaceName="GigabitEthernet0/9/0",node="dev",podName="--",podNamespace="--"} 0
   ipv6Packets{interfaceName="tap-vpp2",node="dev",podName="--",podNamespace="--"} 26
   ipv6Packets{interfaceName="tap0e6439a7a934336",node="dev",podName="web-667bdcb4d8-pxkfs",podNamespace="default"} 9
   ipv6Packets{interfaceName="tap5338a3285ad6bd7",node="dev",podName="kube-dns-6f4fd4bdf-rsz9b",podNamespace="kube-system"} 8
   ipv6Packets{interfaceName="vxlanBVI",node="dev",podName="--",podNamespace="--"} 0
   # HELP outBytes Number of transmitted bytes for interface
   # TYPE outBytes gauge
   outBytes{interfaceName="GigabitEthernet0/9/0",node="dev",podName="--",podNamespace="--"} 0
   outBytes{interfaceName="tap-vpp2",node="dev",podName="--",podNamespace="--"} 5203
   outBytes{interfaceName="tap0e6439a7a934336",node="dev",podName="web-667bdcb4d8-pxkfs",podNamespace="default"} 0
   outBytes{interfaceName="tap5338a3285ad6bd7",node="dev",podName="kube-dns-6f4fd4bdf-rsz9b",podNamespace="kube-system"} 17504
   outBytes{interfaceName="vxlanBVI",node="dev",podName="--",podNamespace="--"} 0
   # HELP outErrorPackets Number of transmitted packets with error for interface
   # TYPE outErrorPackets gauge
   outErrorPackets{interfaceName="GigabitEthernet0/9/0",node="dev",podName="--",podNamespace="--"} 0
   outErrorPackets{interfaceName="tap-vpp2",node="dev",podName="--",podNamespace="--"} 0
   outErrorPackets{interfaceName="tap0e6439a7a934336",node="dev",podName="web-667bdcb4d8-pxkfs",podNamespace="default"} 0
   outErrorPackets{interfaceName="tap5338a3285ad6bd7",node="dev",podName="kube-dns-6f4fd4bdf-rsz9b",podNamespace="kube-system"} 0
   outErrorPackets{interfaceName="vxlanBVI",node="dev",podName="--",podNamespace="--"} 0
   # HELP outPackets Number of transmitted packets for interface
   # TYPE outPackets gauge
   outPackets{interfaceName="GigabitEthernet0/9/0",node="dev",podName="--",podNamespace="--"} 0
   outPackets{interfaceName="tap-vpp2",node="dev",podName="--",podNamespace="--"} 49
   outPackets{interfaceName="tap0e6439a7a934336",node="dev",podName="web-667bdcb4d8-pxkfs",podNamespace="default"} 0
   outPackets{interfaceName="tap5338a3285ad6bd7",node="dev",podName="kube-dns-6f4fd4bdf-rsz9b",podNamespace="kube-system"} 45
   outPackets{interfaceName="vxlanBVI",node="dev",podName="--",podNamespace="--"} 0
   # HELP puntPackets Number of punt packets for interface
   # TYPE puntPackets gauge
   puntPackets{interfaceName="GigabitEthernet0/9/0",node="dev",podName="--",podNamespace="--"} 0
   puntPackets{interfaceName="tap-vpp2",node="dev",podName="--",podNamespace="--"} 0
   puntPackets{interfaceName="tap0e6439a7a934336",node="dev",podName="web-667bdcb4d8-pxkfs",podNamespace="default"} 0
   puntPackets{interfaceName="tap5338a3285ad6bd7",node="dev",podName="kube-dns-6f4fd4bdf-rsz9b",podNamespace="kube-system"} 0
   puntPackets{interfaceName="vxlanBVI",node="dev",podName="--",podNamespace="--"} 0

In order to browse stats in web UI Prometheus, it must be started
locally by following the information in the `Prometheus Getting Started
Guide <https://prometheus.io/docs/prometheus/latest/getting_started/>`__.

If you start Prometheus on a node, the following sample config can be
used:

.. code:: yaml

   global:
     scrape_interval:     15s

   scrape_configs:
     - job_name: 'contiv_stats'
       metrics_path: '/stats'
       static_configs:
         - targets: ['localhost:9999']
     - job_name: 'contiv_agent'
       # metrics_path defaults to '/metrics'
       static_configs:
         - targets: ['localhost:9999']

Once Prometheus is started with the specified config, you should be able
access its web UI at ``localhost:9090``.

::

   tester@dev:~/Downloads/prometheus-2.1.0.linux-amd64$ ./prometheus --config.file=config.yml

If security features are enabled for the HTTP endpoint, then the config
must be adjusted:

.. code:: yaml

    - job_name: 'contiv_secured'

        scheme: https
        basic_auth:
           username: user
           password: pass
        metrics_path: /stats
        tls_config:
          insecure_skip_verify: true
          # CA certificate to validate API server certificate with.
          #[ ca_file: <filename> ]
        static_configs:
          - targets: ['localhost:9191']