summaryrefslogtreecommitdiffstats
path: root/jjb/scripts/vpp
AgeCommit message (Collapse)AuthorFilesLines
2021-08-06Fix workarounds for archive pathsVratko Polak3-3/+38
https://gerrit.fd.io/r/c/ci-management/+/33060 has introduced workarounds for jobs which create Robot outputs outside /archives/. It got merged without verifying the workarounds work correctly. + The workaround for device job does not seem to be needed. + Perform the copy also when robot failed. Fixes: 37a55a05cd6ac7ca15bbb99f5ddc9ca61874d5f8 Change-Id: I5e08545bd290fa917e85a375865a130c3f2ff996 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2021-08-05Global: Rework archive artifactspmikus2-0/+3
This patch removes archive-artifacts-parameter macro and ARCHVIVE_ARTIFACTS env var from csit and vpp project. All project specific artifacts to be uploaded with the log files SHOULD BE copied to $WORKSPACE/archives. The next step once this is merged will be to remove NEXUS entirely via JCasC. + Remove archive-artifacts from all csit/vpp yaml files. + Add fdio-infra-ship-backup-logs macro + Remove unused jjb/include-raw-deploy-archives.sh + CSIT: - copy job artifacts to $WORKSPACE/archives + HC2VPP - remove CSIT + TLDK: - remove CSIT Signed-off-by: pmikus <pmikus@cisco.com> Signed-off-by: Dave Wallace <dwallacelf@gmail.com> Change-Id: Iada020cf269714c34f9ce32d764d991827e3b003
2021-06-02api-checkstyle: do not send the notification about the successful API checksAndrew Yourtchenko1-4/+0
the api-checkstyle was initially done with notification for both successful and failed api-checkstyle runs, just as a matter of precaution. It's time to keep the notification only for the fail case, thus fulfilling the point of notifications. Change-Id: I41a12e7a30bc852c9280fc08a6eae426438f7d92 Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>
2021-05-13Fix make test docs upload & clean up docs jobsDave Wallace1-1/+6
- make test DOCS_DIR in vpp/master changed location, fix upload for master and subsequent release branches. - Don't archive docs build output to avoid stress on nexus - Sphinx docs are generated and pushed to github by hand for publishing. Therefore remove sphinx docs jobs and include building of sphinx docs in the vpp-docs-verify-* jobs. Signed-off-by: Dave Wallace <dwallacelf@gmail.com> Change-Id: I3b2badbf3e2b242ad6b4efcc7d6d281c8b26136b
2021-04-06disable ASAN in the debug buildsAndrew Yourtchenko1-1/+3
The move to unix sockets has broken ASAN fairly thoroughly, so disable it Change-Id: Icf3822c5927309f8574865008173a66019a3ed55 Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>
2021-02-23Merge "vpp: print "git diff" before running API checkstyle"Vratko Polak1-0/+3
2021-02-23vpp: print "git diff" before running API checkstyleAndrew Yourtchenko1-0/+3
API checkstyle is unhappy if the repo is not clean. An example: https://jenkins.fd.io/job/vpp-checkstyle-verify-master-ubuntu2004-x86_64/598/console Output the changes to make diagnostics easier. Change-Id: I3fe1dfd0adde47e2140151687a6ab31885fc851c Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>
2021-02-15vpp: enable multi-worker make test on Debian10Andrew Yourtchenko1-0/+11
This patch enables the multi-worker make test on the branches that support it, as a voting job directly, to capture the working tests and avoid the new non-working multiworker testcases. Change-Id: Ibeedd8f8aa2851c022b045d37505bb75bf460de8 Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>
2021-02-10vpp build.sh: move from "make verify" to "make pkg-verify" + "make test"Andrew Yourtchenko1-17/+36
Problem: The VPP CI merge and verify jobs use the "make verify", which has special handling depending on the OS. So, to make meaningful changes like running the test suite on more platforms, one would need to make the changes in two places, which is error-prone and slow. Solution: move the logic on which OS to run the "make test" into the ci-management scripts, and stop using "make verify" Change-Id: Icc7daa9364198f455ed8ca5bae5e5da5f15baf48 Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com>
2021-02-09Add the core file decoding+cleanup for the verify jobsAndrew Yourtchenko3-30/+101
- Add better failure reporting and dry run handling for build scripts. Change-Id: Ia19bae15ff4880b07094f4f665e5e00030eda27c Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com> Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2021-01-26Skip VPP build for merge jobs with Gerrit ID != git HEAD.Dave Wallace4-45/+82
- Since merge jobs are serialized after the patches have been merged into git, all merge jobs which do not have the same Gerrit ID as git HEAD are redundant. Skip them to save useless cycles and to reduce the size of the build queues when multiple patches are merged in a short amount of time. - This also fixes the issue with packagecloud packages containing different contents then the associated Gerrit ID when built & pushed in this state. Change-Id: I822492bd1d523576373bb268258c181a9c5edb42 Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2020-12-04run ASAN within the non-voting VPP debug verify jobAndrew Yourtchenko1-3/+3
Address sanitizer catches a wide range of failures like buffer overruns, etc. So makes sense to run it during the debug test run. However, it is a bit picky to the compiler - currently it only works well only on GCC 8+, so this patch accounts for that by moving the job to 20.04 executor which has gcc-9 by default. Thanks to Benoit for the help getting it working. Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com> Change-Id: If086907d1f20e99503aa0e811fd2ae95c168d731
2020-10-13Clean up vpp jenkins scripts & docker dirDave Wallace14-163/+193
- Remove docker/vpp Dockerfile definitions and colateral files for image now being build by https://gerrit.fd.io/r/c/ci-management/+/28022 - Remove opensuse support which has been elided from the vpp build in https://gerrit.fd.io/r/c/vpp/+/28999 - Remove unused scripts. - Output script name marker for all jjb scripts following LFIT practices. - Remove old/unused output from scripts. - Add logic to use vpp-ext-deps download cache in the executor images to speed up build jobs. - Add copyright notifcations to modified files without an existing copyright. - Make all vpp/docs jobs use '-{stream}-{os}-{executor}' suffix in their names. - Consolidate executor logging into setup_executor_env.sh and call it first for all jobs. Change-Id: I4ffb67ed476659f0127ca8c84f20a305d06149bf Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2020-09-22Refactor vpp job names to align with jenkins labelsDave Wallace1-0/+45
- Switch to using fdiotools docker repos using associated jenkins labels. - Align vpp job names with docker repos and tags. - Normalize project attributes (including order) across vpp project definitions. - Add branch-head-refspec attribute for testing. - Refactor projects to allow finer granularity definition of {stream}-{os}-{executor-arch} based jobs. - Add 2009 stream to vpp-csit-verify-api-crc. - Add vpp-verify-gcc job for gcov testing. New list of VPP jobs: $ jjsb-test vpp-* 2>&1 | grep -e'Number of jobs' -e'Job name' | sed -e 's/INFO:jenkins_jobs.builder://g' Number of jobs generated: 126 Job name: vpp-beta-merge-master-centos8-aarch64 Job name: vpp-beta-merge-master-centos8-x86_64 Job name: vpp-beta-merge-master-debian9-x86_64 Job name: vpp-beta-merge-master-debian10-x86_64 Job name: vpp-beta-verify-master-centos8-aarch64 Job name: vpp-beta-verify-master-centos8-x86_64 Job name: vpp-beta-verify-master-debian9-x86_64 Job name: vpp-beta-verify-master-debian10-x86_64 Job name: vpp-checkstyle-verify-1908-x86_64 Job name: vpp-checkstyle-verify-2001-x86_64 Job name: vpp-checkstyle-verify-2005-x86_64 Job name: vpp-checkstyle-verify-2009-x86_64 Job name: vpp-checkstyle-verify-master-x86_64 Job name: vpp-csit-verify-api-crc-1908 Job name: vpp-csit-verify-api-crc-2009 Job name: vpp-csit-verify-api-crc-master Job name: vpp-csit-verify-device-1908-1n-skx Job name: vpp-csit-verify-device-1908-1n-tx2 Job name: vpp-csit-verify-device-2001-1n-skx Job name: vpp-csit-verify-device-2001-1n-tx2 Job name: vpp-csit-verify-device-2005-1n-skx Job name: vpp-csit-verify-device-2005-1n-tx2 Job name: vpp-csit-verify-device-2009-1n-skx Job name: vpp-csit-verify-device-2009-1n-tx2 Job name: vpp-csit-verify-device-master-1n-skx Job name: vpp-csit-verify-device-master-1n-tx2 Job name: vpp-csit-verify-perf-1908-2n-clx Job name: vpp-csit-verify-perf-1908-2n-dnv Job name: vpp-csit-verify-perf-1908-2n-skx Job name: vpp-csit-verify-perf-1908-3n-dnv Job name: vpp-csit-verify-perf-1908-3n-hsw Job name: vpp-csit-verify-perf-1908-3n-skx Job name: vpp-csit-verify-perf-1908-3n-tsh Job name: vpp-csit-verify-perf-2001-2n-clx Job name: vpp-csit-verify-perf-2001-2n-dnv Job name: vpp-csit-verify-perf-2001-2n-skx Job name: vpp-csit-verify-perf-2001-3n-dnv Job name: vpp-csit-verify-perf-2001-3n-hsw Job name: vpp-csit-verify-perf-2001-3n-skx Job name: vpp-csit-verify-perf-2001-3n-tsh Job name: vpp-csit-verify-perf-2005-2n-clx Job name: vpp-csit-verify-perf-2005-2n-dnv Job name: vpp-csit-verify-perf-2005-2n-skx Job name: vpp-csit-verify-perf-2005-3n-dnv Job name: vpp-csit-verify-perf-2005-3n-hsw Job name: vpp-csit-verify-perf-2005-3n-skx Job name: vpp-csit-verify-perf-2005-3n-tsh Job name: vpp-csit-verify-perf-2009-2n-clx Job name: vpp-csit-verify-perf-2009-2n-dnv Job name: vpp-csit-verify-perf-2009-2n-skx Job name: vpp-csit-verify-perf-2009-3n-dnv Job name: vpp-csit-verify-perf-2009-3n-hsw Job name: vpp-csit-verify-perf-2009-3n-skx Job name: vpp-csit-verify-perf-2009-3n-tsh Job name: vpp-csit-verify-perf-master-2n-clx Job name: vpp-csit-verify-perf-master-2n-dnv Job name: vpp-csit-verify-perf-master-2n-skx Job name: vpp-csit-verify-perf-master-3n-dnv Job name: vpp-csit-verify-perf-master-3n-hsw Job name: vpp-csit-verify-perf-master-3n-skx Job name: vpp-csit-verify-perf-master-3n-tsh Job name: vpp-debug-verify-2009-ubuntu1804-x86_64 Job name: vpp-debug-verify-master-ubuntu1804-x86_64 Job name: vpp-docs-merge-1908 Job name: vpp-docs-merge-2001 Job name: vpp-docs-merge-2005 Job name: vpp-docs-merge-2009 Job name: vpp-docs-merge-master Job name: vpp-docs-verify-1908 Job name: vpp-docs-verify-2001 Job name: vpp-docs-verify-2005 Job name: vpp-docs-verify-2009 Job name: vpp-docs-verify-master Job name: vpp-gcc-verify-2009-ubuntu2004-x86_64 Job name: vpp-gcc-verify-master-ubuntu2004-x86_64 Job name: vpp-info-yaml-verify Job name: vpp-make-test-docs-merge-1908 Job name: vpp-make-test-docs-merge-2001 Job name: vpp-make-test-docs-merge-2005 Job name: vpp-make-test-docs-merge-2009 Job name: vpp-make-test-docs-merge-master Job name: vpp-make-test-docs-verify-1908 Job name: vpp-make-test-docs-verify-2001 Job name: vpp-make-test-docs-verify-2005 Job name: vpp-make-test-docs-verify-2009 Job name: vpp-make-test-docs-verify-master Job name: vpp-merge-1908-centos7-x86_64 Job name: vpp-merge-1908-ubuntu1804-aarch64 Job name: vpp-merge-1908-ubuntu1804-x86_64 Job name: vpp-merge-2001-centos7-x86_64 Job name: vpp-merge-2001-ubuntu1804-aarch64 Job name: vpp-merge-2001-ubuntu1804-x86_64 Job name: vpp-merge-2005-centos7-x86_64 Job name: vpp-merge-2005-ubuntu1804-aarch64 Job name: vpp-merge-2005-ubuntu1804-x86_64 Job name: vpp-merge-2009-centos7-x86_64 Job name: vpp-merge-2009-ubuntu1804-aarch64 Job name: vpp-merge-2009-ubuntu1804-x86_64 Job name: vpp-merge-2009-ubuntu2004-aarch64 Job name: vpp-merge-2009-ubuntu2004-x86_64 Job name: vpp-merge-master-centos7-x86_64 Job name: vpp-merge-master-ubuntu1804-aarch64 Job name: vpp-merge-master-ubuntu1804-x86_64 Job name: vpp-merge-master-ubuntu2004-aarch64 Job name: vpp-merge-master-ubuntu2004-x86_64 Job name: vpp-sphinx-docs-merge-master Job name: vpp-sphinx-docs-verify-master Job name: vpp-verify-1908-centos7-x86_64 Job name: vpp-verify-1908-ubuntu1804-aarch64 Job name: vpp-verify-1908-ubuntu1804-x86_64 Job name: vpp-verify-2001-centos7-x86_64 Job name: vpp-verify-2001-ubuntu1804-aarch64 Job name: vpp-verify-2001-ubuntu1804-x86_64 Job name: vpp-verify-2005-centos7-x86_64 Job name: vpp-verify-2005-ubuntu1804-aarch64 Job name: vpp-verify-2005-ubuntu1804-x86_64 Job name: vpp-verify-2009-centos7-x86_64 Job name: vpp-verify-2009-ubuntu1804-aarch64 Job name: vpp-verify-2009-ubuntu1804-x86_64 Job name: vpp-verify-2009-ubuntu2004-aarch64 Job name: vpp-verify-2009-ubuntu2004-x86_64 Job name: vpp-verify-master-centos7-x86_64 Job name: vpp-verify-master-ubuntu1804-aarch64 Job name: vpp-verify-master-ubuntu1804-x86_64 Job name: vpp-verify-master-ubuntu2004-aarch64 Job name: vpp-verify-master-ubuntu2004-x86_64 Change-Id: I46e786a2092043aa95baa12a24ea9531683f7d6f Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
2020-09-09jjb: remove VPP coverity jobDave Wallace1-32/+0
Signed-off-by: Dave Wallace <dwallacelf@gmail.com> Change-Id: I7f5d1fec7a811908860816e8ba23c0a700dfadd5
2020-07-24Unify echo MAKE_PARALLEL vars with CSITJuraj Linkeš1-8/+17
CSIT differes in how MAKE_PARALLEL flags are logged to stdout before build. Unify based on https://gerrit.fd.io/r/c/csit/+/27821. Change-Id: I48bced4792e65da504e9a1815e598633315aecab Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2020-07-08Unify arm and x86 vpp jobsJuraj Linkeš3-144/+14
There are two sets of scripts doing basically the same thing, one set for arm and one for x86. Use the same scripts for both x86 and arm jobs, remove the unnecessary ones and update the scripts so that they work on both architectures. Change-Id: Ie9c29578d0f6af41e65c0f4723bc248e664cb8dd Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2020-06-29Recycle vpp-verify-{stream}-clang into vpp-verify-{stream}-debug jobAndrew Yourtchenko1-4/+13
The non-voting clang job has been there for a while, not very useful since clang is now the default compiler. This converts the clang job into a "make test-debug" job, run eventually with ASAN. Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com> Change-Id: I011d7d1e2d887f5dec32e27d05859dcd64a3e548
2020-06-16vpp-checkstyle-verify-{stream}: add the step to API checkAndrew Yourtchenko1-0/+30
This patch adds calling the crcchecker.py --check-patchset in the initial checkstyle path. (see change ID I5f13c0976d8a12a58131b3e270f2dc9c00dc7d8c) It adds also: - gerrit-refspec-parameter to be able to specify the patchset manually - credentials-binding for a WebEx Teams bot - I had them added via IT-19800 - necessary plumbing to use these credentials. It is done by a minimal curl setup, which posts into room "VPP API check failures", which I created to be able to monitor the behavior and jump in to discuss and help to sort out the changes that may be problematic. Signed-off-by: Andrew Yourtchenko <ayourtch@gmail.com> Change-Id: I4833aa929044413a1ac08ab80668f25c0db8b3ff
2020-06-01Add MAKE_PARALLEL_JOBS parameterJuraj Linkeš3-12/+12
VPP uses the MAKE_PARALLEL_JOBS variable to limit the number of cpus used for build. Make that varialbe configurable as job paramter instead of setting it in script for easier job management. Change-Id: Iaaafdb216c3af9244827b270c540a70b5844021f Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2020-03-11fix vpp-sphinx-docs-verify-master job failurePaul Vinciguerra1-1/+1
Job is failing because it is asking to install newer packages. Apparently the 'make docs' is called from docs and expects different environment variables set in the base makefile's 'make docs'. Fixes: b81e5c92687238d60331dd11e9b079657b2aa5bc Change-Id: I7613cde1178e5e27fdb3e457b4e03969080236c9 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2020-03-10fix vpp-sphinx-docs-verify-master job failurePaul Vinciguerra1-1/+1
Job is failing because it is asking to install newer packages. Do you want to continue? [Y/n] Abort. Makefile:31: recipe for target 'html' failed make[1]: *** [html] Error 1 make[1]: Leaving directory '/w/workspace/vpp-sphinx-docs-verify-master/docs' Change-Id: Iab11cded2fef85a78686fae6b7e19845c33418f7 Signed-off-by: Paul Vinciguerra <pvinci@vinciconsulting.com>
2020-01-15Comment usage of parallel-vars.shVratko Polak2-0/+12
+ Add comments to scripts that need the values. + Add comments to shell stanzas to not split inclusions. + Remove parallel-vars.sh from a job that does not use the values. - It seems compilation time is not affected by the values. Change-Id: If844486411753fcac7bece1db8416f2d23ea3749 Signed-off-by: Vratko Polak <vrpolak@cisco.com>
2020-01-09VPP: Make script inclusion more readableVratko Polak16-0/+616
Historically, scripts names started with include-raw-vpp- prefix. That was not only long and superfluous, but also confusing, as the actual way of including uses "include-raw-escape", which is a command behaving differently than "include-raw", suggested by the previous name. This Change shortens script names to the part that actually says something about the script. + Delete include-gcc8-prefer.sh as it is orphaned since https://gerrit.fd.io/r/c/ci-management/+/20443 + Move the scripts to jjb/scripts/vpp/. - "absolute" paths are not used, as ci-man root is not in search path. + Put each path on a separate line to avoid long lines. + Put each include in a separate shell to make sure return code of each script is checked. + Update copyright year where present. - Copyright notices are not added when not present. Change-Id: I48920542ff20810cc87fb01844236350348bae59 Signed-off-by: Vratko Polak <vrpolak@cisco.com>