Tox: Add copyright checkermain

Created by copying out a small subset of CSIT code.
Removed CSIT references, e.g. CSIT_DIR -> ROOT_DIR.

Change-Id: Id4c17fe9c99e74fcf02a48abf4a3cfaa57f890cb
Signed-off-by: Vratko Polak <vrpolak@cisco.com>

9 files changed, 359 insertions, 0 deletions

diff --git a/.gitignore b/.gitignore
new file mode 100644
index 0000000..32595a9
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1,19 @@
+# Copyright (c) 2020 Cisco and/or its affiliates.
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+/.tox
+/env
+*.log
+*.pyc
+*~
+.idea
diff --git a/resources/libraries/bash/entry/README.txt b/resources/libraries/bash/entry/README.txt
new file mode 100644
index 0000000..993cfc8
--- /dev/null
+++ b/resources/libraries/bash/entry/README.txt
@@ -0,0 +1,16 @@
+# Copyright (c) 2020 Cisco and/or its affiliates.
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+Scripts in this directory are to be executed (or sourced)
+directly from Jenkins executor (or manually by user when testing locally),
+as opposed to indirectly from other scripts.
diff --git a/resources/libraries/bash/entry/check/README.txt b/resources/libraries/bash/entry/check/README.txt
new file mode 100644
index 0000000..6f6efae
--- /dev/null
+++ b/resources/libraries/bash/entry/check/README.txt
@@ -0,0 +1,25 @@
+# Copyright (c) 2020 Cisco and/or its affiliates.
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+This directory contains checker scripts and other files they need.
+Each checker script is assumed to be run from tox,
+when working directory is set to root of git repo.
+Each script should:
++ Return nonzero exit code when it fails.
+++ The tox might ignore the code when the check is not blocking.
++ Write less verbose output to stderr.
++ Write (to stderr) PASSED or FAILED to help with debugging.
++ Direct more verbose output to appropriately named .log file.
++ Only the output suitable for automated processing by an external caller
+  should be written to stdout.
+++ The level of "less verbose" depends on check and state of codebase.
diff --git a/resources/libraries/bash/entry/check/copyright_year.sh b/resources/libraries/bash/entry/check/copyright_year.sh
new file mode 100644
index 0000000..0da66f5
--- /dev/null
+++ b/resources/libraries/bash/entry/check/copyright_year.sh
@@ -0,0 +1,70 @@
+# Copyright (c) 2020 Cisco and/or its affiliates.
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+set -exuo pipefail
+
+# This file should be executed from tox, as the assumed working directory
+# is different from where this file is located.
+# This file does not have executable flag nor shebang,
+# to dissuade non-tox callers.
+
+# This script runs a few grep-based command and fails
+# if it detects any file edited or added since HEAD~
+# containing a copyright notice in first 3 lines,
+# but not the current year (in the same line).
+# The offending lines are stored to copyright_year.log (overwriting).
+#
+# 3 lines were chosen, because first two lines could be shebang and empty line,
+# and more than 3 lines would start failing on files with multiple copyright
+# holders. There, only the last updating entity needs to bump its year,
+# and put other copyright lines below.
+
+# "set -eu" handles failures from the following two lines.
+BASH_CHECKS_DIR="$(dirname $(readlink -e "${BASH_SOURCE[0]}"))"
+BASH_FUNCTION_DIR="$(readlink -e "${BASH_CHECKS_DIR}/../../function")"
+source "${BASH_FUNCTION_DIR}/common.sh" || {
+    echo "Source failed." >&2
+    exit 1
+}
+
+year=$(date +'%Y')
+IFS=$'\n'
+files=($(git diff --name-only HEAD~ || true))
+unset IFS
+truncate -s 0 "copyright_year.log" || die
+# A change can have thousands of files, supress console output in the cycle.
+set +x
+for fil in "${files[@]}"; do
+    # Greps do "fail" on 0 line output, we need to ignore that
+    # as 0 lines is good. We need both set +e to ensure everything executes,
+    # and || true later to avoid dying on zero.
+    piped_command="set +ex; head -n 3 '${fil}' | fgrep -in 'Copyright'"
+    piped_command+=" | fgrep -v '${year}' | awk '{print \"${fil}: \" \$0}'"
+    piped_command+=" >> 'copyright_year.log'"
+    wrong_strings="$(bash -c "${piped_command}" || true)" || die
+done
+set -x
+lines="$(< "copyright_year.log" wc -l)"
+if [ "${lines}" != "0" ]; then
+    # TODO: Decide which text goes to stdout and which to stderr.
+    warn "Copyright lines with wrong year detected: ${lines}"
+    # TODO: Disable when output size does more harm than good.
+    pwd
+    cat "copyright_year.log" >&2
+    warn
+    warn "Copyright year checker: FAIL"
+    exit 1
+fi
+
+warn
+warn "Copyright year checker: PASS"
diff --git a/resources/libraries/bash/entry/tox.sh b/resources/libraries/bash/entry/tox.sh
new file mode 100755
index 0000000..cd45630
--- /dev/null
+++ b/resources/libraries/bash/entry/tox.sh
@@ -0,0 +1,32 @@
+#!/usr/bin/env bash
+
+# Copyright (c) 2020 Cisco and/or its affiliates.
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+set -exuo pipefail
+
+# This is a launcher script, to be called from Jenkins.
+# It runs tox at $CSIT_DIR (after activating virtualenv with its requirements).
+# Exit code of tox is propagated.
+
+# "set -eu" handles failures from the following two lines.
+BASH_ENTRY_DIR="$(dirname $(readlink -e "${BASH_SOURCE[0]}"))"
+BASH_FUNCTION_DIR="$(readlink -e "${BASH_ENTRY_DIR}/../function")"
+source "${BASH_FUNCTION_DIR}/common.sh" || {
+    echo "Source failed." >&2
+    exit 1
+}
+common_dirs || die
+cd "${ROOT_DIR}" || die
+activate_virtualenv "${ROOT_DIR}" "${ROOT_DIR}/tox-requirements.txt" || die
+tox  # Return code is turned into Jenkins job vote.
diff --git a/resources/libraries/bash/function/README.txt b/resources/libraries/bash/function/README.txt
new file mode 100644
index 0000000..9511ddc
--- /dev/null
+++ b/resources/libraries/bash/function/README.txt
@@ -0,0 +1,20 @@
+# Copyright (c) 2020 Cisco and/or its affiliates.
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+Files in this directory system are to be executed indirectly,
+sourced from other scripts.
+
+In fact, the files should only define functions,
+except perhaps some minimal logic needed to import dependencies.
+The originating function calls should be executed from elsewhere,
+typically from entry scripts.
diff --git a/resources/libraries/bash/function/common.sh b/resources/libraries/bash/function/common.sh
new file mode 100644
index 0000000..7f8aa0e
--- /dev/null
+++ b/resources/libraries/bash/function/common.sh
@@ -0,0 +1,110 @@
+# Copyright (c) 2020 Cisco and/or its affiliates.
+# Copyright (c) 2020 PANTHEON.tech and/or its affiliates.
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+set -exuo pipefail
+
+# This library defines functions used by multiple entry scripts.
+# Keep functions ordered alphabetically, please.
+
+function activate_virtualenv () {
+
+    # Update virtualenv pip package, delete and create virtualenv directory,
+    # activate the virtualenv, install requirements, set PYTHONPATH.
+
+    # Arguments:
+    # - ${1} - Path to existing directory for creating virtualenv in.
+    #          If missing or empty, ${ROOT_DIR} is used.
+    # - ${2} - Path to requirements file, ${ROOT_DIR}/requirements.txt if empty.
+    # Variables read:
+    # - ROOT_DIR - Path to existing root of local git repository.
+    # Functions called:
+    # - die - Print to stderr and exit.
+
+    set -exuo pipefail
+
+    root_path="${1-$ROOT_DIR}"
+    env_dir="${root_path}/env"
+    req_path=${2-$ROOT_DIR/requirements.txt}
+    rm -rf "${env_dir}" || die "Failed to clean previous virtualenv."
+    pip3 install virtualenv==20.0.20 || {
+        die "Virtualenv package install failed."
+    }
+    virtualenv --no-download --python=$(which python3) "${env_dir}" || {
+        die "Virtualenv creation for $(which python3) failed."
+    }
+    set +u
+    source "${env_dir}/bin/activate" || die "Virtualenv activation failed."
+    set -u
+    pip3 install -r "${req_path}" || {
+        die "Requirements installation failed."
+    }
+}
+
+
+function common_dirs () {
+
+    # Set global variables, create some directories (without touching content).
+
+    # Variables set:
+    # - BASH_FUNCTION_DIR - Path to existing directory this file is located in.
+    # - ROOT_DIR - Path to existing root of local git repository.
+    # - RESOURCES_DIR - Path to existing CSIT subdirectory "resources".
+    # Functions called:
+    # - die - Print to stderr and exit.
+
+    set -exuo pipefail
+
+    this_file=$(readlink -e "${BASH_SOURCE[0]}") || {
+        die "Some error during locating of this source file."
+    }
+    BASH_FUNCTION_DIR=$(dirname "${this_file}") || {
+        die "Some error during dirname call."
+    }
+    # Current working directory could be in a different repo, e.g. VPP.
+    pushd "${BASH_FUNCTION_DIR}" || die "Pushd failed"
+    relative_csit_dir=$(git rev-parse --show-toplevel) || {
+        die "Git rev-parse failed."
+    }
+    ROOT_DIR=$(readlink -e "${relative_csit_dir}") || die "Readlink failed."
+    popd || die "Popd failed."
+}
+
+function die () {
+
+    # Print the message to standard error end exit with error code specified
+    # by the second argument.
+    #
+    # Hardcoded values:
+    # - The default error message.
+    # Arguments:
+    # - ${1} - The whole error message, be sure to quote. Optional
+    # - ${2} - the code to exit with, default: 1.
+
+    set -x
+    set +eu
+    warn "${1:-Unspecified run-time error occurred!}"
+    exit "${2:-1}"
+}
+
+function warn () {
+
+    # Print the message to standard error.
+    #
+    # Arguments:
+    # - ${@} - The text of the message.
+
+    set -exuo pipefail
+
+    echo "$@" >&2
+}
diff --git a/tox-requirements.txt b/tox-requirements.txt
new file mode 100644
index 0000000..76eb86a
--- /dev/null
+++ b/tox-requirements.txt
@@ -0,0 +1,24 @@
+# Copyright (c) 2020 Cisco and/or its affiliates.
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Bump regularly.
+tox==3.7.0
+
+# Tox dependencies. Consult "pip freeze" after installing
+# bumped tox into an empty virtualenv.
+filelock==3.0.10
+pluggy==0.8.1
+py==1.7.0
+six==1.12.0
+toml==0.10.0
+virtualenv==16.4.0
diff --git a/tox.ini b/tox.ini
new file mode 100644
index 0000000..453dda2
--- /dev/null
+++ b/tox.ini
@@ -0,0 +1,43 @@
+# Copyright (c) 2020 Cisco and/or its affiliates.
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Idea of this file is motivated by OpenDylight,
+# especially its Integration/Test sub-project.
+
+# This file requires active virtualenv with tox package installed,
+# or python-tox system package installed.
+
+# Usage:
+# cd to git repo root (other directories might use different tox.ini)
+#   $ tox
+# will execute all checks.
+
+[tox]
+# Fast and brief checkers to front, slow or verbose checkers to back.
+envlist = copyright_year
+
+# The following is needed as tox requires setup.py by default.
+skipsdist = true
+# Just a shorthand to avoid long lines.
+checker_dir = ./resources/libraries/bash/entry/check
+
+# TODO: Tox prints various warnings. Figure them out and fix them.
+
+# Keep testenvs sorted alphabetically, please.
+
+[testenv:copyright_year]
+whitelist_externals = /bin/bash
+setenv = PYTHONPATH = {toxinidir}
+commands = bash {[tox]checker_dir}/copyright_year.sh
+
+# Keep testenvs sorted alphabetically, please.