docs: better docs, mv doxygen to sphinx

This patch refactors the VPP sphinx docs
in order to make it easier to consume
for external readers as well as VPP developers.

It also makes sphinx the single source
of documentation, which simplifies maintenance
and operation.

Most important updates are:

- reformat the existing documentation as rst
- split RELEASE.md and move it into separate rst files
- remove section 'events'
- remove section 'archive'
- remove section 'related projects'
- remove section 'feature by release'
- remove section 'Various links'
- make (Configuration reference, CLI docs,
  developer docs) top level items in the list
- move 'Use Cases' as part of 'About VPP'
- move 'Troubleshooting' as part of 'Getting Started'
- move test framework docs into 'Developer Documentation'
- add a 'Contributing' section for gerrit,
  docs and other contributer related infos
- deprecate doxygen and test-docs targets
- redirect the "make doxygen" target to "make docs"

Type: refactor

Change-Id: I552a5645d5b7964d547f99b1336e2ac24e7c209f
Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com>
Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>

11 files changed, 357 insertions, 0 deletions

diff --git a/docs/_scripts/siphon_templates/markdown/clicmd/index_entry.md b/docs/_scripts/siphon_templates/markdown/clicmd/index_entry.md
new file mode 100644
index 00000000000..de517f160c1
--- /dev/null
+++ b/docs/_scripts/siphon_templates/markdown/clicmd/index_entry.md
@@ -0,0 +1,15 @@
+{#
+# Copyright (c) 2016 Comcast Cable Communications Management, LLC.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#}
\ No newline at end of file
diff --git a/docs/_scripts/siphon_templates/markdown/clicmd/index_header.md b/docs/_scripts/siphon_templates/markdown/clicmd/index_header.md
new file mode 100644
index 00000000000..86e5bb6c3e5
--- /dev/null
+++ b/docs/_scripts/siphon_templates/markdown/clicmd/index_header.md
@@ -0,0 +1,23 @@
+{#
+# Copyright (c) 2016 Comcast Cable Communications Management, LLC.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#}
+.. _cmdreference:
+
+Reference
+=========
+
+.. toctree::
+    :maxdepth: 2
+
diff --git a/docs/_scripts/siphon_templates/markdown/clicmd/index_section.md b/docs/_scripts/siphon_templates/markdown/clicmd/index_section.md
new file mode 100644
index 00000000000..0da934ac938
--- /dev/null
+++ b/docs/_scripts/siphon_templates/markdown/clicmd/index_section.md
@@ -0,0 +1,16 @@
+{#
+# Copyright (c) 2016 Comcast Cable Communications Management, LLC.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#}
+    clis/{{ this.page_label(group) }}
diff --git a/docs/_scripts/siphon_templates/markdown/clicmd/item_format.md b/docs/_scripts/siphon_templates/markdown/clicmd/item_format.md
new file mode 100644
index 00000000000..f1caa961f56
--- /dev/null
+++ b/docs/_scripts/siphon_templates/markdown/clicmd/item_format.md
@@ -0,0 +1,62 @@
+{#
+# Copyright (c) 2016 Comcast Cable Communications Management, LLC.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#}
+{% set v = item['value'] %}
+{% if 'short_help' in v %}
+{% set str = v['short_help'] %}
+{% set str = this.unescape(str) %}
+
+{# Summary/usage #}
+{{ item['value']['path'] }}
+-------------------------------------------------------------------------
+
+.. code-block:: console
+
+    {{ this.reindent(str, 4) }}
+
+{% endif %}
+{% if 'long_help' in v %}
+{% set long_help = v['long_help'] %}
+{% set long_help = this.unescape(long_help) %}
+{# This is seldom used and will likely be deprecated #}
+{# Long help #}
+.. code-block:: console
+
+    {{ this.reindent(long_help, 4) }}
+
+{% endif %}
+{% if 'siphon_block' in item['meta'] %}
+{% set sb = item["meta"]["siphon_block"] %}
+{% set sb = this.process_special(sb) %}
+{% if sb %}
+{# Extracted from the code in /*? ... ?*/ blocks #}
+
+{# Description #}
+
+{{ sb }}
+{% endif %}
+{% endif %}
+{% if 'name' in meta or 'function' in v %}
+{# Gives some developer-useful linking #}
+
+{% if "name" in meta %}
+Declaration: ``{{ meta['name'] }}`` `{{ meta["file"] }} line {{ item["meta"]["line_start"] }} <{{ this.repository_link }}{{ meta["file"] }}#L{{ item["meta"]["line_start"] }}>`_
+{% endif %}
+{% if "function" in v %}
+
+Implementation: ``{{ v["function"] }}``
+{% endif %}
+{% endif %}
+
diff --git a/docs/_scripts/siphon_templates/markdown/clicmd/item_header.md b/docs/_scripts/siphon_templates/markdown/clicmd/item_header.md
new file mode 100644
index 00000000000..20e4803ec2e
--- /dev/null
+++ b/docs/_scripts/siphon_templates/markdown/clicmd/item_header.md
@@ -0,0 +1,20 @@
+{#
+# Copyright (c) 2016 Comcast Cable Communications Management, LLC.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#}
+.. _{{ this.page_label(group) }}:
+
+===============================================================
+{{ this.page_title(group) }}
+===============================================================
diff --git a/docs/_scripts/siphon_templates/markdown/default/index_entry.md b/docs/_scripts/siphon_templates/markdown/default/index_entry.md
new file mode 100644
index 00000000000..479dcdb2cc3
--- /dev/null
+++ b/docs/_scripts/siphon_templates/markdown/default/index_entry.md
@@ -0,0 +1,16 @@
+{#
+# Copyright (c) 2016 Comcast Cable Communications Management, LLC.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#}
+{{ "* [%s](@ref %s)" % (item["name"], meta["label"]) }}
diff --git a/docs/_scripts/siphon_templates/markdown/default/index_section.md b/docs/_scripts/siphon_templates/markdown/default/index_section.md
new file mode 100644
index 00000000000..3c9d2b474c5
--- /dev/null
+++ b/docs/_scripts/siphon_templates/markdown/default/index_section.md
@@ -0,0 +1,18 @@
+{#
+# Copyright (c) 2016 Comcast Cable Communications Management, LLC.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#}
+
+@subpage {{ this.page_label(group) }}
+
diff --git a/docs/_scripts/siphon_templates/markdown/default/item_format.md b/docs/_scripts/siphon_templates/markdown/default/item_format.md
new file mode 100644
index 00000000000..ed1b1bf7eeb
--- /dev/null
+++ b/docs/_scripts/siphon_templates/markdown/default/item_format.md
@@ -0,0 +1,16 @@
+{#
+# Copyright (c) 2016 Comcast Cable Communications Management, LLC.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#}
+{{ raise NotImplementedError }}
diff --git a/docs/_scripts/siphon_templates/markdown/default/item_header.md b/docs/_scripts/siphon_templates/markdown/default/item_header.md
new file mode 100644
index 00000000000..0c21e51fa5d
--- /dev/null
+++ b/docs/_scripts/siphon_templates/markdown/default/item_header.md
@@ -0,0 +1,18 @@
+{#
+# Copyright (c) 2016 Comcast Cable Communications Management, LLC.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#}
+
+{{ "@page %s %s" % (this.page_label(group), this.page_title(group)) }}
+
diff --git a/docs/_scripts/siphon_templates/markdown/syscfg/index_header.md b/docs/_scripts/siphon_templates/markdown/syscfg/index_header.md
new file mode 100644
index 00000000000..f5a9816b99c
--- /dev/null
+++ b/docs/_scripts/siphon_templates/markdown/syscfg/index_header.md
@@ -0,0 +1,111 @@
+{#
+# Copyright (c) 2016 Comcast Cable Communications Management, LLC.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#}
+# Startup Configuration    {{'{#'}}syscfg}
+
+The VPP network stack comes with several configuration options that can be
+provided either on the command line or in a configuration file.
+
+Specific applications built on the stack have been known to require a dozen
+arguments, depending on requirements. This section describes commonly-used
+options and parameters.
+
+You can find command-line argument parsers in the source code by searching for
+instances of the `VLIB_CONFIG_FUNCTION` macro. The invocation
+`VLIB_CONFIG_FUNCTION(foo_config, "foo")` will cause the function
+`foo_config` to receive all the options and values supplied in a parameter
+block named "`foo`", for example: `foo { arg1 arg2 arg3 ... }`.
+
+@todo Tell the nice people where this document lives so that the might
+help improve it!
+
+## Command-line arguments
+
+Parameters are grouped by a section name. When providing more than one
+parameter to a section all parameters for that section must be wrapped in
+curly braces.
+
+```
+/usr/bin/vpp unix { interactive cli-listen 127.0.0.1:5002 }
+```
+
+Which will produce output similar to this:
+
+    <startup diagnostic messages>
+        _______    _        _   _____  ___ 
+     __/ __/ _ \  (_)__    | | / / _ \/ _ \
+     _/ _// // / / / _ \   | |/ / ___/ ___/
+     /_/ /____(_)_/\___/   |___/_/  /_/    
+    
+    vpp# <start-typing>
+
+When providing only one such parameter the braces are optional. For example,
+the following command argument, `unix interactive` does not have braces:
+
+```
+/usr/bin/vpp unix interactive
+```
+
+The command line can be presented as a single string or as several; anything
+given on the command line is concatenated with spaces into a single string
+before parsing.
+
+VPP applications must be able to locate their own executable images. The
+simplest way to ensure this will work is to invoke a VPP application by giving
+its absolute path; for example: `/usr/bin/vpp <options>`. At startup, VPP
+applications parse through their own ELF-sections (primarily) to make lists
+of init, configuration, and exit handlers.
+
+When developing with VPP, in _gdb_ it's often sufficient to start an application
+like this at the `(gdb)` prompt:
+
+```
+run unix interactive
+```
+
+## Configuration file
+
+It is also possible to supply parameters in a startup configuration file the
+path of which is provided to the VPP application on its command line.
+
+The format of the configuration file is a simple text file with the same
+content as the command line but with the benefit of being able to use newlines
+to make the content easier to read. For example:
+
+```
+unix {
+  nodaemon
+  log /var/log/vpp/vpp.log
+  full-coredump
+  cli-listen localhost:5002
+}
+api-trace {
+  on
+}
+dpdk {
+  dev 0000:03:00.0
+}
+```
+
+VPP is then instructed to load this file with the `-c` option:
+
+```
+/usr/bin/vpp -c /etc/vpp/startup.conf
+```
+
+## Index of startup command sections
+
+[TOC]
+
diff --git a/docs/_scripts/siphon_templates/markdown/syscfg/item_format.md b/docs/_scripts/siphon_templates/markdown/syscfg/item_format.md
new file mode 100644
index 00000000000..53136115bbc
--- /dev/null
+++ b/docs/_scripts/siphon_templates/markdown/syscfg/item_format.md
@@ -0,0 +1,42 @@
+{#
+# Copyright (c) 2016 Comcast Cable Communications Management, LLC.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#}
+{% set v = item['value'] %}
+{{ "@section %s %s" % (meta['label'], item['name']) }}
+{% if 'siphon_block' in item['meta'] %}
+{% set sb = item["meta"]["siphon_block"] %}
+{% if sb %}
+{# Extracted from the code in /*? ... ?*/ blocks #}
+
+### Description
+
+{{ sb }}
+{% endif %}
+{% endif %}
+{% if "name" in meta or "function" in item %}
+{# Gives some developer-useful linking #}
+
+### Declaration and implementation
+{% if "name" in meta %}
+
+{{ "Declaration: @ref %s (@ref %s line %d)" %
+   (meta['name'], meta["file"], item["meta"]["line_start"]) }}
+{% endif %}
+{% if "function" in item %}
+
+{{ "Implementation: @ref %s." % item["function"] }}
+{% endif %}
+{% endif %}
+