style(tox): prepare for fixer scripts

The previous code assumes each tox script is a checker.
We want to add fixer scripts (not voting but editing) soon.

This is the less interesting part of the process,
renaming things and updating comments.

Includes some basic descriptions of how fixers differ from checkers.

Change-Id: I1ca1572a328acd02b41dcf4e70d7cb3746a03b03
Signed-off-by: Vratko Polak <vrpolak@cisco.com>

9 files changed, 510 insertions, 0 deletions

diff --git a/resources/libraries/bash/entry/tox/README.txt b/resources/libraries/bash/entry/tox/README.txt
new file mode 100644
index 0000000000..e13b60209e
--- /dev/null
+++ b/resources/libraries/bash/entry/tox/README.txt
@@ -0,0 +1,39 @@
+# Copyright (c) 2022 Cisco and/or its affiliates.
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+This directory contains tox scripts and other files they need.
+Generally, a tox script is either a checker (suitable for automated verify)
+or a fixer (manually started, risky as uncommitted edits can be lost).
+Each tox script is assumed to be run from tox,
+when working directory is set to ${CSIT_DIR}.
+
+Each checker script should:
++ Return nonzero exit code when it fails.
+++ The tox might ignore the code when the check is not blocking.
++ Write less verbose output to stderr.
++ Write (to stderr) PASSED or FAILED to help with debugging.
++ Direct more verbose output to appropriately named .log file.
++ Only the output suitable for automated processing by an external caller
+  should be written to stdout.
+++ The level of "less verbose" depends on check and state of codebase.
++ TODO: Should we carefully document which files are
+  whitelisted/blacklisted for a particulat check?
+
+Each fixer script should:
++ Perform edits on current filesystem
++ Not assume git is clean (there may be uncommitted edits).
++ Use "git diff HEAD~" to get both comitted and uncomitted edits to analyze.
++ Output whatever it wants (possibly nothing).
+
+TODO: Should checkers be named differently than fixers?
+      E.g. both scripts and tox environments start with fix_?
diff --git a/resources/libraries/bash/entry/tox/autogen.sh b/resources/libraries/bash/entry/tox/autogen.sh
new file mode 100644
index 0000000000..40cc2c2066
--- /dev/null
+++ b/resources/libraries/bash/entry/tox/autogen.sh
@@ -0,0 +1,66 @@
+# Copyright (c) 2022 Cisco and/or its affiliates.
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+set -exuo pipefail
+
+# This file should be executed from tox, as the assumend working directory
+# is different from where this file is located.
+# This file does not have executable flag nor shebang,
+# to dissuade non-tox callers.
+
+# This script starts with copying ${CSIT_DIR}/tests to ${GENERATED_DIR}/.
+# Then the script runs every executable *.py script anywhere in the copied dir,
+# the working directory temporarily changed to where the *.py file is.
+# Proper virtualenv is assumed to be active.
+# Then another directory in ${GENERATED_DIR} is created, where
+# the just generated content is copied and then overwitten by the non-generated.
+# If "diff -dur" sees any changes by the overwrite, this script fails.
+# The diff output is stored to autogen.log (overwriting).
+# The executed *.py files are assumed to be robot suite generators,
+# any change means the contribution is not consistent with the regenerated code.
+
+# "set -eu" handles failures from the following two lines.
+BASH_CHECKS_DIR="$(dirname $(readlink -e "${BASH_SOURCE[0]}"))"
+BASH_FUNCTION_DIR="$(readlink -e "${BASH_CHECKS_DIR}/../../function")"
+source "${BASH_FUNCTION_DIR}/common.sh" || {
+    echo "Source failed." >&2
+    exit 1
+}
+common_dirs
+work_dir="$(pwd)" || die
+trap "cd '${work_dir}'" EXIT || die
+
+generate_tests
+
+rm -rf "${GENERATED_DIR}/tests_tmp"
+cp -r "${GENERATED_DIR}/tests" "${GENERATED_DIR}/tests_tmp"
+# Default cp behavior is to put inside a targed dir, not to override.
+cp -rf "${CSIT_DIR}/tests"/* "${GENERATED_DIR}/tests_tmp"/
+# TODO: Do we want to archive ${GENERATED_DIR}?
+# I think archiving the diff is enough.
+
+diff_cmd=("diff" "-dur" "${GENERATED_DIR}/tests_tmp" "${GENERATED_DIR}/tests")
+# Diff returns RC=1 if output is nonzero.
+lines="$("${diff_cmd[@]}" | tee "autogen.log" | wc -l || true)"
+if [ "${lines}" != "0" ]; then
+    # TODO: Decide which text goes to stdout and which to stderr.
+    warn "Autogen conflict, diff sees nonzero lines: ${lines}"
+    # TODO: Disable if output size does more harm than good.
+    cat "autogen.log" >&2
+    warn
+    warn "Autogen checker: FAIL"
+    exit 1
+fi
+
+warn
+warn "Autogen checker: PASS"
diff --git a/resources/libraries/bash/entry/tox/copyright_year.sh b/resources/libraries/bash/entry/tox/copyright_year.sh
new file mode 100644
index 0000000000..9ed9fcb653
--- /dev/null
+++ b/resources/libraries/bash/entry/tox/copyright_year.sh
@@ -0,0 +1,70 @@
+# Copyright (c) 2022 Cisco and/or its affiliates.
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+set -exuo pipefail
+
+# This file should be executed from tox, as the assumed working directory
+# is different from where this file is located.
+# This file does not have executable flag nor shebang,
+# to dissuade non-tox callers.
+
+# This script runs a few grep-based commands and fails
+# if it detects any file edited or added since HEAD~
+# containing a copyright notice in first 3 lines,
+# but not the current year (in the same line).
+# The offending lines are stored to copyright_year.log (overwriting).
+#
+# 3 lines were chosen, because first two lines could be shebang and empty line,
+# and more than 3 lines would start failing on files with multiple copyright
+# holders. There, only the last updating entity needs to bump its year,
+# and put other copyright lines below.
+
+# "set -eu" handles failures from the following two lines.
+BASH_CHECKS_DIR="$(dirname $(readlink -e "${BASH_SOURCE[0]}"))"
+BASH_FUNCTION_DIR="$(readlink -e "${BASH_CHECKS_DIR}/../../function")"
+source "${BASH_FUNCTION_DIR}/common.sh" || {
+    echo "Source failed." >&2
+    exit 1
+}
+
+year=$(date +'%Y')
+IFS=$'\n'
+files=($(git diff --name-only HEAD~ || true))
+unset IFS
+truncate -s 0 "copyright_year.log" || die
+# A change can have thousands of files, supress console output in the cycle.
+set +x
+for fil in "${files[@]}"; do
+    # Greps do "fail" on 0 line output, we need to ignore that
+    # as 0 lines is good. We need both set +e to ensure everything executes,
+    # and || true later to avoid dying on zero.
+    piped_command="set +ex; head -n 3 '${fil}' | fgrep -i 'Copyright'"
+    piped_command+=" | fgrep -v '${year}' | awk '{print \"${fil}: \" \$0}'"
+    piped_command+=" >> 'copyright_year.log'"
+    wrong_strings="$(bash -c "${piped_command}" || true)" || die
+done
+set -x
+lines="$(< "copyright_year.log" wc -l)"
+if [ "${lines}" != "0" ]; then
+    # TODO: Decide which text goes to stdout and which to stderr.
+    warn "Copyright lines with wrong year detected: ${lines}"
+    # TODO: Disable when output size does more harm than good.
+    pwd
+    cat "copyright_year.log" >&2
+    warn
+    warn "Copyright year checker: FAIL"
+    exit 1
+fi
+
+warn
+warn "Copyright year checker: PASS"
diff --git a/resources/libraries/bash/entry/tox/doc_verify.sh b/resources/libraries/bash/entry/tox/doc_verify.sh
new file mode 100644
index 0000000000..7eec4b69e4
--- /dev/null
+++ b/resources/libraries/bash/entry/tox/doc_verify.sh
@@ -0,0 +1,51 @@
+# Copyright (c) 2022 Cisco and/or its affiliates.
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+set -xeuo pipefail
+
+# This file should be executed from tox, as the assumend working directory
+# is different from where this file is located.
+# This file does not have executable flag nor shebang,
+# to dissuade non-tox callers.
+
+# "set -eu" handles failures from the following two lines.
+BASH_CHECKS_DIR="$(dirname $(readlink -e "${BASH_SOURCE[0]}"))"
+BASH_FUNCTION_DIR="$(readlink -e "${BASH_CHECKS_DIR}/../../function")"
+source "${BASH_FUNCTION_DIR}/common.sh" || {
+    echo "Source failed." >&2
+    exit 1
+}
+source "${BASH_FUNCTION_DIR}/docs.sh" || die "Source failed."
+common_dirs || die
+activate_virtualenv || die
+
+# Documentation generation.
+# Here we do store only stderr to file while stdout (inlcuding Xtrace) is
+# printed to console. This way we can track increased errors in future.
+# We do not need to do trap as the env will be closed after tox finished the
+# task.
+exec 3>&1 || die
+export BASH_XTRACEFD="3" || die
+log_file="$(pwd)/doc_verify.log" || die
+
+generate_docs 2> ${log_file} || die
+
+if [[ "${DOCS_EXIT_STATUS}" != 0 ]]; then
+    # Failed to generate report.
+    warn
+    warn "Doc verify checker: FAIL"
+    exit 1
+fi
+
+warn
+warn "Doc verify checker: PASS"
diff --git a/resources/libraries/bash/entry/tox/gpl_license.sh b/resources/libraries/bash/entry/tox/gpl_license.sh
new file mode 100644
index 0000000000..7112075883
--- /dev/null
+++ b/resources/libraries/bash/entry/tox/gpl_license.sh
@@ -0,0 +1,73 @@
+# Copyright (c) 2022 Cisco and/or its affiliates.
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+set -exuo pipefail
+
+# This file should be executed from tox, as the assumed working directory
+# is different from where this file is located.
+# This file does not have executable flag nor shebang,
+# to dissuade non-tox callers.
+
+# This script runs a few grep-based commands and fails
+# if it detects any file edited or added since HEAD~
+# containing a copyright notice in first 3 lines,
+# but not GPL-based license.
+# The offending files are stored to gpl_license.log (overwriting).
+
+# "set -eu" handles failures from the following two lines.
+BASH_CHECKS_DIR="$(dirname $(readlink -e "${BASH_SOURCE[0]}"))"
+BASH_FUNCTION_DIR="$(readlink -e "${BASH_CHECKS_DIR}/../../function")"
+source "${BASH_FUNCTION_DIR}/common.sh" || {
+    echo "Source failed." >&2
+    exit 1
+}
+
+IFS=$'\n'
+gpl_files=($(git diff --name-only HEAD~ | grep '^GPL/' || true))
+unset IFS
+logfile="gpl_license.log"
+truncate -s 0 "${logfile}" || die
+# A change can have thousands of files, supress console output in the cycle.
+set +x
+for fil in "${gpl_files[@]}"; do
+    if head -n 3 "${fil}" | fgrep -iq 'Copyright'; then
+        # Copyrighted file, processed below.
+        true
+    else
+        # Uncopyrighted files are allowed.
+        # TODO: Should we have list of extesions that require Copyright?
+        continue
+    fi
+    if fgrep -q 'GNU General Public License v2.0 or later' "${fil}"; then
+        # This can be GPL only or the OR license, we accept both.
+        # TODO: Should we require "Apache-2.0 OR GPL-2.0-or-later"?
+        continue
+    else
+        echo "GPL license not detected: ${fil}" >> "${logfile}"
+    fi
+done
+set -x
+lines="$(< "${logfile}" wc -l)"
+if [ "${lines}" != "0" ]; then
+    # TODO: Decide which text goes to stdout and which to stderr.
+    warn "Wrong licensed files in GPL directory detected: ${lines}"
+    # TODO: Disable when output size does more harm than good.
+    pwd
+    cat "${logfile}" >&2
+    warn
+    warn "GPL license checker: FAIL"
+    exit 1
+fi
+
+warn
+warn "GPL license checker: PASS"
diff --git a/resources/libraries/bash/entry/tox/line.sh b/resources/libraries/bash/entry/tox/line.sh
new file mode 100644
index 0000000000..a90fc22cae
--- /dev/null
+++ b/resources/libraries/bash/entry/tox/line.sh
@@ -0,0 +1,50 @@
+# Copyright (c) 2022 Cisco and/or its affiliates.
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+set -exuo pipefail
+
+# This file should be executed from tox, as the assumend working directory
+# is different from where this file is located.
+# This file does not have executable flag nor shebang,
+# to dissuade non-tox callers.
+
+# This script runs a grep-based command and fails if it detects any lines
+# longer than 80 characters.
+# The grep output stored to lines.log (overwriting).
+
+# "set -eu" handles failures from the following two lines.
+BASH_CHECKS_DIR="$(dirname $(readlink -e "${BASH_SOURCE[0]}"))"
+BASH_FUNCTION_DIR="$(readlink -e "${BASH_CHECKS_DIR}/../../function")"
+source "${BASH_FUNCTION_DIR}/common.sh" || {
+    echo "Source failed." >&2
+    exit 1
+}
+
+# Directory docs contains too many wide formatted tables.
+# .txt contains lines with wide URLs.
+piped_command='set -exuo pipefail && grep -rn ".\{81\}" "resources/" "tests/"'
+piped_command+=' | fgrep -v .svg | fgrep -v .txt | tee "lines.log" | wc -l'
+# TODO: The greps "fail" if no long line remains. Fix that if it ever happens.
+lines="$(bash -c "${piped_command}")" || die
+if [ "${lines}" != "0" ]; then
+    # TODO: Decide which text goes to stdout and which to stderr.
+    warn "Long lines detected: ${lines}"
+    ## TODO: Enable when output size does more good than harm.
+    # cat "lines.log" >&2
+    warn
+    warn "Line length checker: FAIL"
+    exit 1
+fi
+
+warn
+warn "Line length checker: PASS"
diff --git a/resources/libraries/bash/entry/tox/model_version.sh b/resources/libraries/bash/entry/tox/model_version.sh
new file mode 100644
index 0000000000..2bcc628bfb
--- /dev/null
+++ b/resources/libraries/bash/entry/tox/model_version.sh
@@ -0,0 +1,65 @@
+# Copyright (c) 2022 Cisco and/or its affiliates.
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+set -exuo pipefail
+
+# This file should be executed from tox, as the assumed working directory
+# is different from where this file is located.
+# This file does not have executable flag nor shebang,
+# to dissuade non-tox callers.
+
+# This script runs a two grep commands as "if" conditions,
+# using log files to store the data (generated by git commands) to grep,
+# failing when model implementation edits do not come with model version edit.
+# The contents of the log files may be useful when fail cause is not obvious.
+
+# "set -eu" handles failures from the following two lines.
+BASH_CHECKS_DIR="$(dirname $(readlink -e "${BASH_SOURCE[0]}"))"
+BASH_FUNCTION_DIR="$(readlink -e "${BASH_CHECKS_DIR}/../../function")"
+source "${BASH_FUNCTION_DIR}/common.sh" || {
+    echo "Source failed." >&2
+    exit 1
+}
+
+impl_log="edited_files.log"
+git diff --name-only HEAD~ > "${impl_log}"
+if ! grep -q '^docs/model/current/schema/test_case*' "${impl_log}"; then
+    # Failing grep means no model edits.
+    warn "No model implementation edits detected."
+    warn
+    warn "CSIT model version checker: PASS"
+    exit 0
+fi
+const_log="constants_edits.log"
+git diff -U0 HEAD~ -- "resources/libraries/python/Constants.py" > "${const_log}"
+if ! grep -q '^\+    MODEL_VERSION = ' "${const_log}"; then
+    warn "Model implementation edits without version edit detected!"
+    warn "See ${impl_log} and ${const_log} for what was detected."
+    warn
+    warn "CSIT model version checker: FAIL"
+    exit 1
+fi
+doc_log="docs_edits.log"
+git diff -U0 HEAD~ -- "docs/model/current/top.rst" > "${doc_log}"
+if ! grep -q '^\+This document is valid for CSIT model' "${doc_log}"; then
+    warn "Model implementation edits without documentation update detected!"
+    warn "See ${impl_log}, ${const_log} and ${doc_log} for what was detected."
+    warn
+    warn "CSIT model version checker: FAIL"
+    exit 1
+fi
+# TODO: Check constants and docs are specifying the same version.
+warn "Model version and doc are edited, model implementation edits are allowed."
+warn
+warn "CSIT model version checker: PASS"
+exit 0
diff --git a/resources/libraries/bash/entry/tox/new_line.sh b/resources/libraries/bash/entry/tox/new_line.sh
new file mode 100644
index 0000000000..a8fb38f73e
--- /dev/null
+++ b/resources/libraries/bash/entry/tox/new_line.sh
@@ -0,0 +1,51 @@
+# Copyright (c) 2022 Cisco and/or its affiliates.
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+set -exuo pipefail
+
+# This file should be executed from tox, as the assumed working directory
+# is different from where this file is located.
+# This file does not have executable flag nor shebang,
+# to dissuade non-tox callers.
+
+# This script runs a grep-based command and fails if it detects any lines
+# edited or added since HEAD~ and longer than 80 characters.
+# The grep output stored to new_lines.log (overwriting).
+
+# See lines.log to locate where the lines are.
+
+# "set -eu" handles failures from the following two lines.
+BASH_CHECKS_DIR="$(dirname $(readlink -e "${BASH_SOURCE[0]}"))"
+BASH_FUNCTION_DIR="$(readlink -e "${BASH_CHECKS_DIR}/../../function")"
+source "${BASH_FUNCTION_DIR}/common.sh" || {
+    echo "Source failed." >&2
+    exit 1
+}
+
+# Greps do "fail" on zero line output, we need to ignore that in the final grep.
+piped_command="set -exuo pipefail && git diff -U0 HEAD~ | grep '^\+' | "
+piped_command+="cut -c2- | grep -v '^\+\+ ' | { grep '.\{81\}' || true; } | "
+piped_command+="tee 'new_lines.log' | wc -l"
+lines="$(bash -c "${piped_command}")" || die
+if [ "${lines}" != "0" ]; then
+    # TODO: Decide which text goes to stdout and which to stderr.
+    warn "Long lines detected: ${lines}"
+    # TODO: Disable when output size does more harm than good.
+    cat "new_lines.log" >&2
+    warn
+    warn "New line length checker: FAIL"
+    exit 1
+fi
+
+warn
+warn "New line length checker: PASS"
diff --git a/resources/libraries/bash/entry/tox/pylint.sh b/resources/libraries/bash/entry/tox/pylint.sh
new file mode 100644
index 0000000000..a3e0f8ee14
--- /dev/null
+++ b/resources/libraries/bash/entry/tox/pylint.sh
@@ -0,0 +1,45 @@
+# Copyright (c) 2022 Cisco and/or its affiliates.
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+set -exuo pipefail
+
+# This file should be executed from tox, as the assumend working directory
+# is different from where this file is located.
+# This file does not have executable flag nor shebang,
+# to dissuade non-tox callers.
+
+# This script runs pylint and propagates its exit code.
+# Config is taken from pylint.cfg,
+# and proper virtualenv is assumed to be active.
+# The pylint output stored to pylint.log (overwriting).
+
+# "set -eu" handles failures from the following two lines.
+BASH_CHECKS_DIR="$(dirname $(readlink -e "${BASH_SOURCE[0]}"))"
+BASH_FUNCTION_DIR="$(readlink -e "${BASH_CHECKS_DIR}/../../function")"
+source "${BASH_FUNCTION_DIR}/common.sh" || {
+    echo "Source failed." >&2
+    exit 1
+}
+pylint_args=("--rcfile=pylint.cfg" "resources/" "GPL/traffic_scripts")
+if pylint "${pylint_args[@]}" > "pylint.log"; then
+    warn
+    warn "Pylint checker: PASS"
+else
+    # TODO: Decide which text goes to stdout and which to stderr.
+    warn "Pylint exited with nonzero status."
+    ## TODO: Enable when output size does more good than harm.
+    # cat "pylint.log" >&2
+    warn
+    warn "Pylint checker: FAIL"
+    exit 1
+fi