New upstream version 18.02upstream/18.02

Change-Id: I89ed24cb2a49b78fe5be6970b99dd46c1499fcc3
Signed-off-by: Luca Boccassi <luca.boccassi@gmail.com>

4 files changed, 333 insertions, 3 deletions

diff --git a/doc/guides/contributing/coding_style.rst b/doc/guides/contributing/coding_style.rst
index d8e4a0f9..b0f0adb8 100644
--- a/doc/guides/contributing/coding_style.rst
+++ b/doc/guides/contributing/coding_style.rst
@@ -694,6 +694,56 @@ Control Statements
                  /* NOTREACHED */
          }
 
+Dynamic Logging
+---------------
+
+DPDK provides infrastructure to perform logging during runtime. This is very
+useful for enabling debug output without recompilation. To enable or disable
+logging of a particular topic, the ``--log-level`` parameter can be provided
+to EAL, which will change the log level. DPDK code can register topics,
+which allows the user to adjust the log verbosity for that specific topic.
+
+In general, the naming scheme is as follows: ``type.section.name``
+
+ * Type is the type of component, where ``lib``, ``pmd``, ``bus`` and ``user``
+   are the common options.
+ * Section refers to a specific area, for example a poll-mode-driver for an
+   ethernet device would use ``pmd.net``, while an eventdev PMD uses
+   ``pmd.event``.
+ * The name identifies the individual item that the log applies to.
+   The name section must align with
+   the directory that the PMD code resides. See examples below for clarity.
+
+Examples:
+
+ * The virtio network PMD in ``drivers/net/virtio`` uses ``pmd.net.virtio``
+ * The eventdev software poll mode driver in ``drivers/event/sw`` uses ``pmd.event.sw``
+ * The octeontx mempool driver in ``drivers/mempool/octeontx`` uses ``pmd.mempool.octeontx``
+ * The DPDK hash library in ``lib/librte_hash`` uses ``lib.hash``
+
+Specializations
+~~~~~~~~~~~~~~~
+
+In addition to the above logging topic, any PMD or library can further split
+logging output by using "specializations". A specialization could be the
+difference between initialization code, and logs of events that occur at runtime.
+
+An example could be the initialization log messages getting one
+specialization, while another specialization handles mailbox command logging.
+Each PMD, library or component can create as many specializations as required.
+
+A specialization looks like this:
+
+ * Initialization output: ``type.section.name.init``
+ * PF/VF mailbox output: ``type.section.name.mbox``
+
+A real world example is the i40e poll mode driver which exposes two
+specializations, one for initialization ``pmd.i40e.init`` and the other for
+the remaining driver logs ``pmd.i40e.driver``.
+
+Note that specializations have no formatting rules, but please follow
+a precedent if one exists. In order to see all current log topics and
+specializations, run the ``app/test`` binary, and use the ``dump_log_types``
 
 Python Code
 -----------
@@ -702,3 +752,235 @@ All Python code should work with Python 2.7+ and 3.2+ and be compliant with
 `PEP8 (Style Guide for Python Code) <https://www.python.org/dev/peps/pep-0008/>`_.
 
 The ``pep8`` tool can be used for testing compliance with the guidelines.
+
+Integrating with the Build System
+---------------------------------
+
+DPDK supports being built in two different ways:
+
+* using ``make`` - or more specifically "GNU make", i.e. ``gmake`` on FreeBSD
+* using the tools ``meson`` and ``ninja``
+
+Any new library or driver to be integrated into DPDK should support being
+built with both systems. While building using ``make`` is a legacy approach, and
+most build-system enhancements are being done using ``meson`` and ``ninja``
+there are no plans at this time to deprecate the legacy ``make`` build system.
+
+Therefore all new component additions should include both a ``Makefile`` and a
+``meson.build`` file, and should be added to the component lists in both the
+``Makefile`` and ``meson.build`` files in the relevant top-level directory:
+either ``lib`` directory or a ``driver`` subdirectory.
+
+Makefile Contents
+~~~~~~~~~~~~~~~~~
+
+The ``Makefile`` for the component should be of the following format, where
+``<name>`` corresponds to the name of the library in question, e.g. hash,
+lpm, etc. For drivers, the same format of Makefile is used.
+
+.. code-block:: none
+
+	# pull in basic DPDK definitions, including whether library is to be
+	# built or not
+	include $(RTE_SDK)/mk/rte.vars.mk
+
+	# library name
+	LIB = librte_<name>.a
+
+	# any library cflags needed. Generally add "-O3 $(WERROR_FLAGS)"
+	CFLAGS += -O3
+	CFLAGS += $(WERROR_FLAGS)
+
+	# the symbol version information for the library, and .so version
+	EXPORT_MAP := rte_<name>_version.map
+	LIBABIVER := 1
+
+	# all source filenames are stored in SRCS-y
+	SRCS-$(CONFIG_RTE_LIBRTE_<NAME>) += rte_<name>.c
+
+	# install includes
+	SYMLINK-$(CONFIG_RTE_LIBRTE_<NAME>)-include += rte_<name>.h
+
+	# pull in rules to build the library
+	include $(RTE_SDK)/mk/rte.lib.mk
+
+Meson Build File Contents - Libraries
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The ``meson.build`` file for a new DPDK library should be of the following basic
+format.
+
+.. code-block:: python
+
+	sources = files('file1.c', ...)
+	headers = files('file1.c', ...)
+
+
+The will build based on a number of conventions and assumptions within the DPDK
+itself, for example, that the library name is the same as the directory name in
+which the files are stored.
+
+For a library ``meson.build`` file, there are number of variables which can be
+set, some mandatory, others optional. The mandatory fields are:
+
+sources
+	**Default Value = []**.
+	This variable should list out the files to be compiled up to create the
+	library. Files must be specified using the meson ``files()`` function.
+
+
+The optional fields are:
+
+allow_experimental_apis
+	**Default Value = false**
+	Used to allow the library to make use of APIs marked as experimental.
+	Set to ``true`` if the C files in the library call any functions
+	marked as experimental in any included header files.
+
+build
+	**Default Value = true**
+	Used to optionally compile a library, based on its dependencies or
+	environment. A simple example of use would be:
+
+.. code-block:: python
+
+	if host_machine.system() != 'linux'
+	        build = false
+	endif
+
+
+cflags
+	**Default Value = [<-march/-mcpu flags>]**.
+	Used to specify any additional cflags that need to be passed to compile
+	the sources in the library.
+
+deps
+	**Default Value = ['eal']**.
+	Used to list the internal library dependencies of the library. It should
+	be assigned to using ``+=`` rather than overwriting using ``=``.  The
+	dependencies should be specified as strings, each one giving the name of
+	a DPDK library, without the ``librte_`` prefix. Dependencies are handled
+	recursively, so specifying e.g. ``mempool``, will automatically also
+	make the library depend upon the mempool library's dependencies too -
+	``ring`` and ``eal``. For libraries that only depend upon EAL, this
+	variable may be omitted from the ``meson.build`` file.  For example:
+
+.. code-block:: python
+
+	deps += ['ethdev']
+
+
+ext_deps
+	**Default Value = []**.
+	Used to specify external dependencies of this library. They should be
+	returned as dependency objects, as returned from the meson
+	``dependency()`` or ``find_library()`` functions. Before returning
+	these, they should be checked to ensure the dependencies have been
+	found, and, if not, the ``build`` variable should be set to ``false``.
+	For example:
+
+.. code-block:: python
+
+	my_dep = dependency('libX', required: 'false')
+	if my_dep.found()
+		ext_deps += my_dep
+	else
+		build = false
+	endif
+
+
+headers
+	**Default Value = []**.
+	Used to return the list of header files for the library that should be
+	installed to $PREFIX/include when ``ninja install`` is run. As with
+	source files, these should be specified using the meson ``files()``
+	function.
+
+includes:
+	**Default Value = []**.
+	Used to indicate any additional header file paths which should be
+	added to the header search path for other libs depending on this
+	library. EAL uses this so that other libraries building against it
+	can find the headers in subdirectories of the main EAL directory. The
+	base directory of each library is always given in the include path,
+	it does not need to be specified here.
+
+name
+	**Default Value = library name derived from the directory name**.
+	If a library's .so or .a file differs from that given in the directory
+	name, the name should be specified using this variable. In practice,
+	since the convention is that for a library called ``librte_xyz.so``, the
+	sources are stored in a directory ``lib/librte_xyz``, this value should
+	never be needed for new libraries.
+
+.. note::
+
+	The name value also provides the name used to find the function version
+	map file, as part of the build process, so if the directory name and
+	library names differ, the ``version.map`` file should be named
+	consistently with the library, not the directory
+
+objs
+	**Default Value = []**.
+	This variable can be used to pass to the library build some pre-built
+	objects that were compiled up as part of another target given in the
+	included library ``meson.build`` file.
+
+version
+	**Default Value = 1**.
+	Specifies the ABI version of the library, and is used as the major
+	version number of the resulting ``.so`` library.
+
+Meson Build File Contents - Drivers
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+For drivers, the values are largely the same as for libraries. The variables
+supported are:
+
+allow_experimental_apis
+	As above.
+
+build
+	As above.
+
+cflags
+	As above.
+
+deps
+	As above.
+
+ext_deps
+	As above.
+
+includes
+	**Default Value = <driver directory>** Some drivers include a base
+	directory for additional source files and headers, so we have this
+	variable to allow the headers from that base directory to be found when
+	compiling driver sources. Should be appended to using ``+=`` rather than
+	overwritten using ``=``.  The values appended should be meson include
+	objects got using the ``include_directories()`` function. For example:
+
+.. code-block:: python
+
+	includes += include_directories('base')
+
+name
+	As above, though note that each driver class can define it's own naming
+	scheme for the resulting ``.so`` files.
+
+objs
+	As above, generally used for the contents of the ``base`` directory.
+
+pkgconfig_extra_libs
+	**Default Value = []**
+	This variable is used to pass additional library link flags through to
+	the DPDK pkgconfig file generated, for example, to track any additional
+	libraries that may need to be linked into the build - especially when
+	using static libraries. Anything added here will be appended to the end
+	of the ``pkgconfig --libs`` output.
+
+sources [mandatory]
+	As above
+
+version
+	As above
diff --git a/doc/guides/contributing/documentation.rst b/doc/guides/contributing/documentation.rst
index 170dacdb..82f2e1bb 100644
--- a/doc/guides/contributing/documentation.rst
+++ b/doc/guides/contributing/documentation.rst
@@ -294,7 +294,7 @@ Line Length
 
      testpmd -l 2-3 -n 4 \
              --vdev=virtio_user0,path=/dev/vhost-net,queues=2,queue_size=1024 \
-             -- -i --txqflags=0x0 --disable-hw-vlan --enable-lro \
+             -- -i --txqflags=0x0 --enable-hw-vlan --enable-lro \
              --enable-rx-cksum --txq=2 --rxq=2 --rxd=1024  --txd=1024
 
 
diff --git a/doc/guides/contributing/patches.rst b/doc/guides/contributing/patches.rst
index 40983c15..2287835f 100644
--- a/doc/guides/contributing/patches.rst
+++ b/doc/guides/contributing/patches.rst
@@ -32,6 +32,29 @@ It is also worth registering for the DPDK `Patchwork <http://dpdk.org/dev/patchw
 The development process requires some familiarity with the ``git`` version control system.
 Refer to the `Pro Git Book <http://www.git-scm.com/book/>`_ for further information.
 
+Source License
+--------------
+
+The DPDK uses the Open Source BSD-3-Clause license for the core libraries and
+drivers. The kernel components are GPL-2.0 licensed. DPDK uses single line
+reference to Unique License Identifiers in source files as defined by the Linux
+Foundation's `SPDX project <http://spdx.org/>`_.
+
+DPDK uses first line of the file to be SPDX tag. In case of *#!* scripts, SPDX
+tag can be placed in 2nd line of the file.
+
+For example, to label a file as subject to the BSD-3-Clause license,
+the following text would be used:
+
+``SPDX-License-Identifier: BSD-3-Clause``
+
+To label a file as dual-licensed with BSD-3-Clause and GPL-2.0 (e.g., for code
+that is shared between the kernel and userspace), the following text would be
+used:
+
+``SPDX-License-Identifier: (BSD-3-Clause OR GPL-2.0)``
+
+Refer to ``licenses/README`` for more details.
 
 Maintainers and Sub-trees
 -------------------------
@@ -130,7 +153,7 @@ Make your planned changes in the cloned ``dpdk`` repo. Here are some guidelines
 * Don't break compilation between commits with forward dependencies in a patchset.
   Each commit should compile on its own to allow for ``git bisect`` and continuous integration testing.
 
-* Add tests to the the ``app/test`` unit test framework where possible.
+* Add tests to the ``app/test`` unit test framework where possible.
 
 * Add documentation, if relevant, in the form of Doxygen comments or a User Guide in RST format.
   See the :ref:`Documentation Guidelines <doc_guidelines>`.
@@ -340,7 +363,7 @@ Where the range is a ``git log`` option.
 Checking Compilation
 --------------------
 
-Compilation of patches and changes should be tested using the the ``test-build.sh`` script in the ``devtools``
+Compilation of patches and changes should be tested using the ``test-build.sh`` script in the ``devtools``
 directory of the DPDK repo::
 
   devtools/test-build.sh x86_64-native-linuxapp-gcc+next+shared
@@ -528,3 +551,12 @@ patch accepted. The general cycle for patch review and acceptance is:
      than rework of the original.
    * Trivial patches may be merged sooner than described above at the tree committer's
      discretion.
+
+DPDK Maintainers
+----------------
+
+The following are the DPDK maintainers as listed in the ``MAINTAINERS`` file
+in the DPDK root directory.
+
+.. literalinclude:: ../../../MAINTAINERS
+   :lines: 3-
diff --git a/doc/guides/contributing/versioning.rst b/doc/guides/contributing/versioning.rst
index 40009062..c495294d 100644
--- a/doc/guides/contributing/versioning.rst
+++ b/doc/guides/contributing/versioning.rst
@@ -50,6 +50,22 @@ those new APIs and start finding issues with them, new DPDK APIs will be
 automatically marked as ``experimental`` to allow for a period of stabilization
 before they become part of a tracked ABI.
 
+Note that marking an API as experimental is a multi step process.
+To mark an API as experimental, the symbols which are desired to be exported
+must be placed in an EXPERIMENTAL version block in the corresponding libraries'
+version map script.
+Secondly, the corresponding definitions of those exported functions, and
+their forward declarations (in the development header files), must be marked
+with the ``__rte_experimental`` tag (see ``rte_compat.h``).
+The DPDK build makefiles perform a check to ensure that the map file and the
+C code reflect the same list of symbols.
+This check can be circumvented by defining ``ALLOW_EXPERIMENTAL_API``
+during compilation in the corresponding library Makefile.
+
+In addition to tagging the code with ``__rte_experimental``,
+the doxygen markup must also contain the EXPERIMENTAL string,
+and the MAINTAINERS file should note the EXPERIMENTAL libraries.
+
 ABI versions, once released, are available until such time as their
 deprecation has been noted in the Release Notes for at least one major release
 cycle. For example consider the case where the ABI for DPDK 2.0 has been