diff options
Diffstat (limited to 'docs/gettingstarted/developers/buildsystem')
4 files changed, 0 insertions, 556 deletions
diff --git a/docs/gettingstarted/developers/buildsystem/buildrootmakefile.rst b/docs/gettingstarted/developers/buildsystem/buildrootmakefile.rst deleted file mode 100644 index 84647d04b5a..00000000000 --- a/docs/gettingstarted/developers/buildsystem/buildrootmakefile.rst +++ /dev/null @@ -1,353 +0,0 @@ -Introduction to build-root/Makefile -=================================== - -The vpp build system consists of a top-level Makefile, a data-driven -build-root/Makefile, and a set of makefile fragments. The various parts -come together as the result of a set of well-thought-out conventions. - -This section describes build-root/Makefile in some detail. - -Repository Groups and Source Paths ----------------------------------- - -Current vpp workspaces comprise a single repository group. The file -.../build-root/build-config.mk defines a key variable called -SOURCE\_PATH. The SOURCE\_PATH variable names the set of repository -groups. At the moment, there is only one repository group. - -Single pass build system, dependencies and components ------------------------------------------------------ - -The vpp build system caters to components built with GNU autoconf / -automake. Adding such components is a simple process. Dealing with -components which use BSD-style raw Makefiles is a more difficult. -Dealing with toolchain components such as gcc, glibc, and binutils can -be considerably more complicated. - -The vpp build system is a **single-pass** build system. A partial order -must exist for any set of components: the set of (a before b) tuples -must resolve to an ordered list. If you create a circular dependency of -the form; (a,b) (b,c) (c,a), gmake will try to build the target list, -but there’s a 0.0% chance that the results will be pleasant. Cut-n-paste -mistakes in .../build-data/packages/.mk can produce confusing failures. - -In a single-pass build system, it’s best to separate libraries and -applications which instantiate them. For example, if vpp depends on -libfoo.a, and myapp depends on both vpp and libfoo.a, it's best to place -libfoo.a and myapp in separate components. The build system will build -libfoo.a, vpp, and then (as a separate component) myapp. If you try to -build libfoo.a and myapp from the same component, it won’t work. - -If you absolutely, positively insist on having myapp and libfoo.a in the -same source tree, you can create a pseudo-component in a separate .mk -file in the .../build-data/packages/ directory. Define the code -phoneycomponent\_source = realcomponent, and provide manual -configure/build/install targets. - -Separate components for myapp, libfoo.a, and vpp is the best and easiest -solution. However, the “mumble\_source = realsource” degree of freedom -exists to solve intractable circular dependencies, such as: to build -gcc-bootstrap, followed by glibc, followed by “real” gcc/g++ [which -depends on glibc too]. - -.../build-root --------------- - -The .../build-root directory contains the repository group specification -build-config.mk, the main Makefile, and the system-wide set of -autoconf/automake variable overrides in config.site. We'll describe -these files in some detail. To be clear about expectations: the main -Makefile and config.site file are subtle and complex. It's unlikely that -you'll need or want to modify them. Poorly planned changes in either -place typically cause bugs that are difficult to solve. - -.../build-root/build-config.mk ------------------------------- - -As described above, the build-config.mk file is straightforward: it sets -the make variable SOURCE\_PATH to a list of repository group absolute -paths. - -The SOURCE\_PATH variable If you choose to move a workspace, make sure -to modify the paths defined by the SOURCE\_PATH variable. Those paths -need to match changes you make in the workspace paths. For example, if -you place the vpp directory in the workspace of a user named jsmith, you -might change the SOURCE\_PATH to: - -SOURCE\_PATH = /home/jsmithuser/workspace/vpp - -The "out of the box" setting should work 99.5% of the time: - -:: - - SOURCE_PATH = $(CURDIR)/.. - -.../vpp/build-root/Makefile ---------------------------- - -The main Makefile is complex in a number of dimensions. If you think you -need to modify it, it's a good idea to do some research, or ask for -advice before you change it. - -The main Makefile was organized and designed to provide the following -characteristics: excellent performance, accurate dependency processing, -cache enablement, timestamp optimizations, git integration, -extensibility, builds with cross-compilation tool chains, and builds -with embedded Linux distributions. - -If you really need to do so, you can build double-cross tools with it, -with a minimum amount of fuss. For example, you could: compile gdb on -x86\_64, to run on PowerPC, to debug the Xtensa instruction set. - -The PLATFORM variable ---------------------- - -The PLATFORM make/environment variable controls a number of important -characteristics, primarily: - -- CPU architecture -- The list of images to build. - -With respect to .../build-root/Makefile, the list of images to build is -specified by the target. For example: - -:: - - make PLATFORM=vpp TAG=vpp_debug install-deb - -builds vpp debug Debian packages. - -The main Makefile interprets $PLATFORM by attempting to "-include" the -file /build-data/platforms.mk: - -:: - - $(foreach d,$(FULL_SOURCE_PATH), \ - $(eval -include $(d)/platforms.mk)) - -By convention, we don't define **platforms** in the -...//build-data/platforms.mk file. - -In the vpp case, we search for platform definition makefile fragments in -.../vpp/build-data/platforms.mk, as follows: - -:: - - $(foreach d,$(SOURCE_PATH_BUILD_DATA_DIRS), \ - $(eval -include $(d)/platforms/*.mk)) - -With vpp, which uses the "vpp" platform as discussed above, we end up -"-include"-ing .../vpp/build-data/platforms/vpp.mk. - -The platform-specific .mk fragment ----------------------------------- - -Here are the contents of .../build-data/platforms/vpp.mk: - -:: - - MACHINE=$(shell uname -m) - - vpp_arch = native - ifeq ($(TARGET_PLATFORM),thunderx) - vpp_dpdk_target = arm64-thunderx-linuxapp-gcc - endif - vpp_native_tools = vppapigen - - vpp_uses_dpdk = yes - - # Uncomment to enable building unit tests - # vpp_enable_tests = yes - - vpp_root_packages = vpp - - # DPDK configuration parameters - # vpp_uses_dpdk_mlx4_pmd = yes - # vpp_uses_dpdk_mlx5_pmd = yes - # vpp_uses_external_dpdk = yes - # vpp_dpdk_inc_dir = /usr/include/dpdk - # vpp_dpdk_lib_dir = /usr/lib - # vpp_dpdk_shared_lib = yes - - # Use '--without-libnuma' for non-numa aware architecture - # Use '--enable-dlmalloc' to use dlmalloc instead of mheap - vpp_configure_args_vpp = --enable-dlmalloc - sample-plugin_configure_args_vpp = --enable-dlmalloc - - # load balancer plugin is not portable on 32 bit platform - ifeq ($(MACHINE),i686) - vpp_configure_args_vpp += --disable-lb-plugin - endif - - vpp_debug_TAG_CFLAGS = -g -O0 -DCLIB_DEBUG \ - -fstack-protector-all -fPIC -Werror - vpp_debug_TAG_CXXFLAGS = -g -O0 -DCLIB_DEBUG \ - -fstack-protector-all -fPIC -Werror - vpp_debug_TAG_LDFLAGS = -g -O0 -DCLIB_DEBUG \ - -fstack-protector-all -fPIC -Werror - - vpp_TAG_CFLAGS = -g -O2 -D_FORTIFY_SOURCE=2 -fstack-protector -fPIC -Werror - vpp_TAG_CXXFLAGS = -g -O2 -D_FORTIFY_SOURCE=2 -fstack-protector -fPIC -Werror - vpp_TAG_LDFLAGS = -g -O2 -D_FORTIFY_SOURCE=2 -fstack-protector -fPIC -Werror -pie -Wl,-z,now - - vpp_clang_TAG_CFLAGS = -g -O2 -D_FORTIFY_SOURCE=2 -fstack-protector -fPIC -Werror - vpp_clang_TAG_LDFLAGS = -g -O2 -D_FORTIFY_SOURCE=2 -fstack-protector -fPIC -Werror - - vpp_gcov_TAG_CFLAGS = -g -O0 -DCLIB_DEBUG -fPIC -Werror -fprofile-arcs -ftest-coverage - vpp_gcov_TAG_LDFLAGS = -g -O0 -DCLIB_DEBUG -fPIC -Werror -coverage - - vpp_coverity_TAG_CFLAGS = -g -O2 -fPIC -Werror -D__COVERITY__ - vpp_coverity_TAG_LDFLAGS = -g -O2 -fPIC -Werror -D__COVERITY__ - -Note the following variable settings: - -- The variable \_arch sets the CPU architecture used to build the - per-platform cross-compilation toolchain. With the exception of the - "native" architecture - used in our example - the vpp build system - produces cross-compiled binaries. - -- The variable \_native\_tools lists the required set of self-compiled - build tools. - -- The variable \_root\_packages lists the set of images to build when - specifying the target: make PLATFORM= TAG= [install-deb \| - install-rpm]. - -The TAG variable ----------------- - -The TAG variable indirectly sets CFLAGS and LDFLAGS, as well as the -build and install directory names in the .../vpp/build-root directory. -See definitions above. - -Important targets build-root/Makefile -------------------------------------- - -The main Makefile and the various makefile fragments implement the -following user-visible targets: - -+------------------+----------------------+--------------------------------------------------------------------------------------+ -| Target | ENV Variable Settings| Notes | -| | | | -+==================+======================+======================================================================================+ -| foo | bar | mumble | -+------------------+----------------------+--------------------------------------------------------------------------------------+ -| bootstrap-tools | none | Builds the set of native tools needed by the vpp build system to | -| | | build images. Example: vppapigen. In a full cross compilation case might include | -| | | include "make", "git", "find", and "tar | -+------------------+----------------------+--------------------------------------------------------------------------------------+ -| install-tools | PLATFORM | Builds the tool chain for the indicated <platform>. Not used in vpp builds | -+------------------+----------------------+--------------------------------------------------------------------------------------+ -| distclean | none | Roto-rooters everything in sight: toolchains, images, and so forth. | -+------------------+----------------------+--------------------------------------------------------------------------------------+ -| install-deb | PLATFORM and TAG | Build Debian packages comprising components listed in <platform>_root_packages, | -| | | using compile / link options defined by TAG. | -+------------------+----------------------+--------------------------------------------------------------------------------------+ -| install-rpm | PLATFORM and TAG | Build RPMs comprising components listed in <platform>_root_packages, | -| | | using compile / link options defined by TAG. | -+------------------+----------------------+--------------------------------------------------------------------------------------+ - -Additional build-root/Makefile environment variable settings ------------------------------------------------------------- - -These variable settings may be of use: - -+----------------------+------------------------------------------------------------------------------------------------------------+ -| ENV Variable | Notes | -+======================+======================+=====================================================================================+ -| BUILD_DEBUG=vx | Directs Makefile et al. to make a good-faith effort to show what's going on in excruciating detail. | -| | Use it as follows: "make ... BUILD_DEBUG=vx". Fairly effective in Makefile debug situations. | -+----------------------+------------------------------------------------------------------------------------------------------------+ -| V=1 | print detailed cc / ld command lines. Useful for discovering if -DFOO=11 is in the command line or not | -+----------------------+------------------------------------------------------------------------------------------------------------+ -| CC=mygcc | Override the configured C-compiler | -+----------------------+------------------------------------------------------------------------------------------------------------+ - -.../build-root/config.site --------------------------- - -The contents of .../build-root/config.site override individual autoconf / -automake default variable settings. Here are a few sample settings related to -building a full toolchain: - -:: - - # glibc needs these setting for cross compiling - libc_cv_forced_unwind=yes - libc_cv_c_cleanup=yes - libc_cv_ssp=no - -Determining the set of variables which need to be overridden, and the -override values is a matter of trial and error. It should be -unnecessary to modify this file for use with fd.io vpp. - -.../build-data/platforms.mk ---------------------------- - -Each repo group includes the platforms.mk file, which is included by -the main Makefile. The vpp/build-data/platforms.mk file is not terribly -complex. As of this writing, .../build-data/platforms.mk file accomplishes two -tasks. - -First, it includes vpp/build-data/platforms/\*.mk: - -:: - - # Pick up per-platform makefile fragments - $(foreach d,$(SOURCE_PATH_BUILD_DATA_DIRS), \ - $(eval -include $(d)/platforms/*.mk)) - -This collects the set of platform definition makefile fragments, as discussed above. - -Second, platforms.mk implements the user-visible "install-deb" target. - -.../build-data/packages/\*.mk ------------------------------ - -Each component needs a makefile fragment in order for the build system -to recognize it. The per-component makefile fragments vary -considerably in complexity. For a component built with GNU autoconf / -automake which does not depend on other components, the make fragment -can be empty. See .../build-data/packages/vpp.mk for an uncomplicated -but fully realistic example. - -Here are some of the important variable settings in per-component makefile fragments: - -+----------------------+------------------------------------------------------------------------------------------------------------+ -| Variable | Notes | -+======================+======================+=====================================================================================+ -| xxx_configure_depend | Lists the set of component build dependencies for the xxx component. In plain English: don't try to | -| | configure this component until you've successfully built the indicated targets. Almost always, | -| | xxx_configure_depend will list a set of "yyy-install" targets. Note the pattern: | -| | "variable names contain underscores, make target names contain hyphens" | -+----------------------+------------------------------------------------------------------------------------------------------------+ -| xxx_configure_args | (optional) Lists any additional arguments to pass to the xxx component "configure" script. | -| | The main Makefile %-configure rule adds the required settings for --libdir, --prefix, and | -| | --host (when cross-compiling) | -+----------------------+------------------------------------------------------------------------------------------------------------+ -| xxx_CPPFLAGS | Adds -I stanzas to CPPFLAGS for components upon which xxx depends. | -| | Almost invariably "xxx_CPPFLAGS = $(call installed_includes_fn, dep1 dep2 dep3)", where dep1, dep2, and | -| | dep3 are listed in xxx_configure_depend. It is bad practice to set "-g -O3" here. Those settings | -| | belong in a TAG. | -+----------------------+------------------------------------------------------------------------------------------------------------+ -| xxx_LDFLAGS | Adds -Wl,-rpath -Wl,depN stanzas to LDFLAGS for components upon which xxx depends. | -| | Almost invariably "xxx_LDFLAGS = $(call installed_lib_fn, dep1 dep2 dep3)", where dep1, dep2, and | -| | dep3 are listed in xxx_configure_depend. It is bad manners to set "-liberty-or-death" here. | -| | Those settings belong in Makefile.am. | -+----------------------+------------------------------------------------------------------------------------------------------------+ - -When dealing with "irritating" components built with raw Makefiles -which only work when building in the source tree, we use a specific -strategy in the xxx.mk file. - -The strategy is simple for those components: We copy the source tree -into .../vpp/build-root/build-xxx. This works, but completely defeats -dependency processing. This strategy is acceptable only for 3rd party -software which won't need extensive (or preferably any) modifications. - -Take a look at .../vpp/build-data/packages/dpdk.mk. When invoked, the -dpdk_configure variable copies source code into $(PACKAGE_BUILD_DIR), -and performs the BSD equivalent of "autoreconf -i -f" to configure the -build area. The rest of the file is similar: a bunch of hand-rolled -glue code which manages to make the dpdk act like a good vpp build -citizen even though it is not. diff --git a/docs/gettingstarted/developers/buildsystem/cmakeandninja.rst b/docs/gettingstarted/developers/buildsystem/cmakeandninja.rst deleted file mode 100644 index 580d261bdac..00000000000 --- a/docs/gettingstarted/developers/buildsystem/cmakeandninja.rst +++ /dev/null @@ -1,186 +0,0 @@ -Introduction to cmake and ninja -=============================== - -Cmake plus ninja is approximately equal to GNU autotools plus GNU -make, respectively. Both cmake and GNU autotools support self and -cross-compilation, checking for required components and versions. - -- For a decent-sized project - such as vpp - build performance is drastically better with (cmake, ninja). - -- The cmake input language looks like an actual language, rather than a shell scripting scheme on steroids. - -- Ninja doesn't pretend to support manually-generated input files. Think of it as a fast, dumb robot which eats mildly legible byte-code. - -See the `cmake website <http://cmake.org>`_, and the `ninja website -<https://ninja-build.org>`_ for additional information. - -vpp cmake configuration files ------------------------------ - -The top of the vpp project cmake hierarchy lives in .../src/CMakeLists.txt. -This file defines the vpp project, and (recursively) includes two kinds -of files: rule/function definitions, and target lists. - -- Rule/function definitions live in .../src/cmake/{\*.cmake}. Although the contents of these files is simple enough to read, it shouldn't be necessary to modify them very often - -- Build target lists come from CMakeLists.txt files found in subdirectories, which are named in the SUBDIRS list in .../src/CMakeLists.txt - -:: - - ############################################################################## - # subdirs - order matters - ############################################################################## - if("${CMAKE_SYSTEM_NAME}" STREQUAL "Linux") - find_package(OpenSSL REQUIRED) - set(SUBDIRS - vppinfra svm vlib vlibmemory vlibapi vnet vpp vat vcl plugins - vpp-api tools/vppapigen tools/g2 tools/perftool) - elseif("${CMAKE_SYSTEM_NAME}" STREQUAL "Darwin") - set(SUBDIRS vppinfra) - else() - message(FATAL_ERROR "Unsupported system: ${CMAKE_SYSTEM_NAME}") - endif() - - foreach(DIR ${SUBDIRS}) - add_subdirectory(${DIR}) - endforeach() - -- The vpp cmake configuration hierarchy discovers the list of plugins to be built by searching for subdirectories in .../src/plugins which contain CMakeLists.txt files - - -:: - - ############################################################################## - # find and add all plugin subdirs - ############################################################################## - FILE(GLOB files RELATIVE - ${CMAKE_CURRENT_SOURCE_DIR} - ${CMAKE_CURRENT_SOURCE_DIR}/*/CMakeLists.txt - ) - foreach (f ${files}) - get_filename_component(dir ${f} DIRECTORY) - add_subdirectory(${dir}) - endforeach() - -How to write a plugin CMakeLists.txt file ------------------------------------------ - -It's really quite simple. Follow the pattern: - -:: - - add_vpp_plugin(mactime - SOURCES - mactime.c - node.c - - API_FILES - mactime.api - - INSTALL_HEADERS - mactime_all_api_h.h - mactime_msg_enum.h - - API_TEST_SOURCES - mactime_test.c - ) - -Adding a target elsewhere in the source tree --------------------------------------------- - -Within reason, adding a subdirectory to the SUBDIRS list in -.../src/CMakeLists.txt is perfectly OK. The indicated directory will -need a CMakeLists.txt file. - -.. _building-g2: - -Here's how we build the g2 event data visualization tool: - -:: - - option(VPP_BUILD_G2 "Build g2 tool." OFF) - if(VPP_BUILD_G2) - find_package(GTK2 COMPONENTS gtk) - if(GTK2_FOUND) - include_directories(${GTK2_INCLUDE_DIRS}) - add_vpp_executable(g2 - SOURCES - clib.c - cpel.c - events.c - main.c - menu1.c - pointsel.c - props.c - g2version.c - view1.c - - LINK_LIBRARIES vppinfra Threads::Threads m ${GTK2_LIBRARIES} - NO_INSTALL - ) - endif() - endif() - -The g2 component is optional, and is not built by default. There are -a couple of ways to tell cmake to include it in build.ninja [or in Makefile.] - -When invoking cmake manually [rarely done and not very easy], specify --DVPP_BUILD_G2=ON: - -:: - - $ cmake ... -DVPP_BUILD_G2=ON - -Take a good look at .../build-data/packages/vpp.mk to see where and -how the top-level Makefile and .../build-root/Makefile set all of the -cmake arguments. One strategy to enable an optional component is fairly -obvious. Add -DVPP_BUILD_G2=ON to vpp_cmake_args. - -That would work, of course, but it's not a particularly elegant solution. - -Tinkering with build options: ccmake ------------------------------------- - -The easy way to set VPP_BUILD_G2 - or frankly **any** cmake -parameter - is to install the "cmake-curses-gui" package and use -it. - -- Do a straightforward vpp build using the top level Makefile, "make build" or "make build-release" -- Ajourn to .../build-root/build-vpp-native/vpp or .../build-root/build-vpp_debug-native/vpp -- Invoke "ccmake ." to reconfigure the project as desired - -Here's approximately what you'll see: - -:: - - CCACHE_FOUND /usr/bin/ccache - CMAKE_BUILD_TYPE - CMAKE_INSTALL_PREFIX /scratch/vpp-gate/build-root/install-vpp-nati - DPDK_INCLUDE_DIR /scratch/vpp-gate/build-root/install-vpp-nati - DPDK_LIB /scratch/vpp-gate/build-root/install-vpp-nati - MBEDTLS_INCLUDE_DIR /usr/include - MBEDTLS_LIB1 /usr/lib/x86_64-linux-gnu/libmbedtls.so - MBEDTLS_LIB2 /usr/lib/x86_64-linux-gnu/libmbedx509.so - MBEDTLS_LIB3 /usr/lib/x86_64-linux-gnu/libmbedcrypto.so - MUSDK_INCLUDE_DIR MUSDK_INCLUDE_DIR-NOTFOUND - MUSDK_LIB MUSDK_LIB-NOTFOUND - PRE_DATA_SIZE 128 - VPP_API_TEST_BUILTIN ON - VPP_BUILD_G2 OFF - VPP_BUILD_PERFTOOL OFF - VPP_BUILD_VCL_TESTS ON - VPP_BUILD_VPPINFRA_TESTS OFF - - CCACHE_FOUND: Path to a program. - Press [enter] to edit option Press [d] to delete an entry CMake Version 3.10.2 - Press [c] to configure - Press [h] for help Press [q] to quit without generating - Press [t] to toggle advanced mode (Currently Off) - -Use the cursor to point at the VPP_BUILD_G2 line. Press the return key -to change OFF to ON. Press "c" to regenerate build.ninja, etc. - -At that point "make build" or "make build-release" will build g2. And so on. - -Note that toggling advanced mode ["t"] gives access to substantially -all of the cmake option, discovered directories and paths. diff --git a/docs/gettingstarted/developers/buildsystem/index.rst b/docs/gettingstarted/developers/buildsystem/index.rst deleted file mode 100644 index 908e91e1fc1..00000000000 --- a/docs/gettingstarted/developers/buildsystem/index.rst +++ /dev/null @@ -1,14 +0,0 @@ -.. _buildsystem: - -Build System -============ - -This guide describes the vpp build system in detail. As of this writing, -the build systems uses a mix of make / Makefiles, cmake, and ninja to -achieve excellent build performance. - -.. toctree:: - - mainmakefile - cmakeandninja - buildrootmakefile diff --git a/docs/gettingstarted/developers/buildsystem/mainmakefile.md b/docs/gettingstarted/developers/buildsystem/mainmakefile.md deleted file mode 100644 index ddf0661942f..00000000000 --- a/docs/gettingstarted/developers/buildsystem/mainmakefile.md +++ /dev/null @@ -1,3 +0,0 @@ -Introduction to the top-level Makefile -====================================== - |