Release Notes¶

v0.45.0¶

New Features¶

Add lf-env.sh ‘library’ script in ~jenkins. Sourcing this ‘library’ script from a bash script provides access to a number of functions. The script is installed by the ‘init’ script at boot time so that is is accessible from any Jenkins build script. Hopefully over time other functions will be added to this library.

Upgrade Notes¶

Add sonar-project-file parameter to ci sonar jobs. By enabling the caller to override the default project file name with an empty string, we enable the ability to provide project settings directly in the sonar-properties field. This removes the requirement for a “sonar-project.properties” file in the repo.

Bug Fixes¶

Fix the auto update image script to compare the image type before updating an the image in the source repository. This fixes the bug that updates images although they are the same flavour but a diffirent type.

Add DRY_RUN parameter to the PyPI verify and merge template common macro because it’s required by the pypi_upload.sh script. Adjust verbosity of shell scripts - quiet down pip, add repository name to echo output.

Correct doc .pypirc test URL to https://test.pypi.org/legacy/ Change doc .pypirc to use API tokens instead of username/password. Use pypi-test as the default repository name in the PyPI merge template (instead of “staging”) to match established ONAP practice.

v0.44.1¶

New Features¶

Archive ‘sudo’ logs. The log will be located in the ‘sudo’ sub-directory of the archive. The actual name of the log-file depends on the OS of the builder.

Bug Fixes¶

Change Packer version back to 1.4.2. Packer 1.4.3 frequently has issues setting metadata after a build completes. Packer 1.4.4 is not expected until October, so this change is needed now to fix the broken builds.

In the PyPI merge template, add cron parameter to support daily build and push to a staging repo, like the maven merge template. In PyPI release templates, change name of gerrit and github trigger file patterns parameter. This avoids accidental overriding by jobs that limit their actions to subdirectories. The release file patterns are hardcoded in a shell script. Remove params from RST doc. In all PyPI templates, add disabled option and disable-job parameter to be consistent with other python templates.

v0.44.0¶

New Features¶

Add an additonal Sonar job that allows the caller to provide a builder that runs prior to the Sonar scan.

Add template to update OpenStack cloud images.

This job finds and updates OpenStack cloud images on the ci-management source repository.

The job is triggered in two ways:

1. When a packer merge job completes, the new image name created is passed down to the job.
2. Manually trigger the job to update all images.

When the job is triggered through an upstream packer merge job, this only generates a change request for the new image built.

When the job is triggered manually, this job finds the latest images on OpenStack cloud and compares them with the images currently used in the source ci-management source repository. If the compared images have newer time stamps are all updated through a change request.

This job requires a Jenkins configuration merge and verify job setup and working on Jenkins.

New templates to build and push Python source and binary distributions to a PyPI server. Includes: {project-name}-pypi-verify-{stream}, gerrit-pypi-verify, github-pypi-verify, {project-name}-pypi-merge-{stream}, gerrit-pypi-merge, github-pypi-merge, {project-name}-pypi-release-verify-{stream}, gerrit-pypi-release-verify, github-pypi-release-verify, {project-name}-pypi-release-merge-{stream}, gerrit-pypi-release-merge, github-pypi-release-merge,

Upgrade Notes¶

Packer merge jobs have a new build parameter when checked also updates the cloud image.

lf-infra-packer-build macro now requires 1 new variables to be passed.

1. update-cloud-image: Set to true when images need to be updated on Jenkins.

Bug Fixes¶

Changed the trigger to run sonar from stage-release to run-sonar. This makes it more concistent with the other parts.

Run WhiteSource scan jobs weekly on Sunday.

Pip install pyenv from python2 should force more-itertools to 5.0.0 In a fresh python2.7 venv “pip install pyenv” correctly pulls down more-itertools [required: Any, installed: 5.0.0] If for some reason a higher version is already installed this will downgrade more-itertools to a py2 compatible version

Allow java-opts to be defined in WhiteSource scans. This avoids java heap failures.

v0.43.1¶

Upgrade Notes¶

CONTAINER_PULL_REGISTRY and CONTAINER_PUSH_REGISTRY need to be defined in Jenkins global environment varaibles.

Bug Fixes¶

Pin python-cinderclient to 4.3.0.

A new version of python-cinderclient 5.0.0 is released which breaks the openstack jobs.

openstack –os-cloud vex limits show –absolute Error: No module named v1.contrib

Debug info shows: File “/home/jenkins/.local/lib/python2.7/site-packages/openstackclient/volume/client.py”, line 40, in make_client from cinderclient.v1.contrib import list_extensions ImportError: No module named v1.contrib

Update echo commands in release-job.sh for easy identification.

Update release-container-schema to use CONTAINER_PULL_REGISTRY and CONTAINER_PUSH_REGISTRY from Jenkins global variables. These will be used to pull and push operations for self release containers. Can be overwritten in the releases files.

Fix warnings from tox (shellcheck).

Remove quotes from optional var WSS_UNIFIED_AGENT_OPTIONS. When empty, the quotes will cause WS Unified Agent failures.

Upgrade WS Unified Agent CLI to latest version 19.8.1

v0.43.0¶

New Features¶

Add python tox merge job templates for gerrit and github. These templates are triggered by merge events to test the Python code on the branch that received the merge. Renamed tox-verify macro to tox-common.

Bug Fixes¶

Import GPG signing key in release jobs before verifying Gerrit tag details.

INFO validate job checks that repositories matches $PROJECT Catches projects that replace / with - in their INFO file Also ensures that repositories only has one entry. We are not supporting multiple projects with a single INFO.yaml file.

Remove unused the MAVEN_CENTRAL_URL variable. The self-release job is designed to work with any Nexus repository info published in staging-repo.txt.gz, which makes the MAVEN_CENTRAL_URL redundant, hence remove the unused variable.

Revise tox-install.sh script to invoke python using environment variable PYTHON instead of hardcoding “python”, and remove the pip –quiet flag. Extend the RTD template to pass python-version, default python2.

Use existing builder lf-infra-maven sonar, drop incomplete builder lf-tox-maven-sonar, to gain desired behavior of pushing code analysis results to Sonar. Use trivial goal ‘validate’ by default. The script maven-sonar.sh calls maven twice, first with build goals and then with Sonar goals. The incomplete builder did not supply the build goals.

Release schema verification needs to happen first before we attempt to assign values to the variables. Validate version only after the schema validation has passed and the variables are assigned.

Organize variable setup into functions. Maven release files expects different variables than container release files.

Rename “version” variable in container release files to “container_release_tag” which is a better user friendly name given the fact that container versions are rather called tags. Internally, we still process it as “version” to allow reuse of the tag function.

v0.42.1¶

New Features¶

Add DRY_RUN build param to do a test run the job with publishing artifacts.

Upgrade Notes¶

Update lftools version to v0.26.2.

Bug Fixes¶

Verify both repos before attempting release. We have run into a case where the repo on ODL nexus was good, and the repo on Sonatype nexus was missing. Cover this case by running the verify loop over each repo before attempting release.

v0.42.0¶

New Features¶

Add support for distribution_type “container”

Add function maven_release_file and container_release_file and the logic to choose the correct one. No functional change to maven_release_file.

Add docker login step when docker releases are being processed.

container_release_file downloads log_dir/console.log.gz and parses it to get a list of container name and version. Verifies pulls container and grabs the image_id then performs the merge then tags and pushes the container.

Add lf-sonar-common job-template to lf-ci-jobs.yaml and add lf-infra-sonar to macros.yaml. The purpose of a new job template is to adopt using jenkins sonar plug-in along with the sonar-project.properties file versus pom.xml. Lastly the new job template will ensure that anything using lf-infra-tox-sonar is unaffected.

Add support for “Build with Parameters” for projects that do not want to use a release file for maven builds.

Upgrade Notes¶

release-verify and merge will need to run on a docker build-node for example centos7-docker-8c-8g Lftools will need to be updated to 0.26.0 so that -v is supported for lftools nexus release

Update lftools version to v0.26.1.

Bug Fixes¶

Fix missing extension in ID for release-schema.yaml.

Make “distribution_type” mandatory in future release files.

Rename “RELEASE_FILE” parameter to “USE_RELEASE_FILE” in release-jobs. This will match the actual varaible default value better and will not collide with the local “release_file” in the script.

Fix “USE_RELEASE_FILE” if statement. We are now using a bool instead of a string. Changing the if statements to evaluate bools.

Add pre-build-script parameter to python clm, tox and sonar templates. Gives flexibility to install prerequisite libraries, rearrange the source tree, etc.

Pin more-itertools ~= 5.0.0, since version 6.0.0 requires Python 3.4 https://github.com/erikrose/more-itertools/releases

Error: more-itertools requires Python ‘>=3.4’ but the running Python is 2.7.5

Restructure shell/release-job.sh into functions.

v0.41.0¶

Upgrade Notes¶

Update lftools version to v0.26.0.

Update packer version to 1.4.3. This new packer version fixed an issues in docker image, where its unable to install the packages into docker containers due to checking of wrong container OS. Fix in 1.4.3: builder/docker: Check container os, not host os, when creating container dir default [GH-7939]

Bug Fixes¶

Project job sections that define gerrit_trigger_file_paths are overriding ones set in Default parameters of lf_release_common Hard Code gerrit_trigger_file_paths to fix this.

v0.40.4¶

Upgrade Notes¶

Update lftools version to v0.25.4.

Bug Fixes¶

Git fetch needs the dashed version git fetch “$PATCH_DIR/${PROJECT////-}.bundle”. Fix if statement.

Release creds are only required for promoting the repo, which uses diff ACL as compared to normal user. Therefore dont use the release creds for the verify jobs and the scm sections in both the job templates.

1. The release merge job is a one way operation. Given this, Release jobs should only exist in the format {project-name}-release-verify and {project-name}-release-merge As these jobs trigger from a change to any branch/** These jobs Must Exit 0 if release job has already tagged a repo. Inevitably a release file will be pulled in from master to branch or a remerge will be requested This will again trigger the release jobs. since in this case the repo is already tagged, the job should not report a failure. This is solved by having the verfiy and merge exit 0 when the repo is already tagged 2. Rather than use project as defined in the release file use ${PROJECT////-} This changes PROJECT=”optf/osdf/foo/bar” to optf-osdf-foo-bar so that we can fetch the log files. by changing /’s in the project names to -‘s

v0.40.3¶

Known Issues¶

Update release job template to tigger on any branch name, and not just ‘master’. ODL projects branches are version ‘4.0.x’ which requires passing the branch name to the template.

Bug Fixes¶

Fix the release job script to handle any trailing ‘/’ set on log_dir and also handle unbound variables correctly.

Update lftools version to v0.25.3.

v0.40.1¶

Upgrade Notes¶

Projects using lf-release-job will need to add the project’s signing public key in their Jenkins Settings Files.

Bug Fixes¶

Use {GERRIT_PROJECT} when calling Gerrit in release-merge job.

Project pattern was incorrectly set to ** must be {project}

The self-release jobs does not handle multiple repositories listed in staging-repo.txt file. This fixes the issue by deriving the NEXUS_URL and the STAGING_REPO from each entry in the file. This approach also eliminates the need for having multiple release.yaml files for every staging-repo.

Allow lf_release_verify and lf_release_merge to verify tag signature.

v0.40.0¶

New Features¶

Add packer image $NAME to the description setting so that the image name is displayed above the job logs URL. This saves a some time from looking for the image name in the jobs logs.

Allows projects to promote their own builds. Requires setup of accounts and permissions in Gerrit, Jenkins and Nexus. Please refer to the lf-release-jobs documentation for details.

Remove orphaned ports from the openstack cloud environment. These orphaned ports are residue of CSIT jobs that needs to be purged, as a part of the openstack cron job. A large number of stale jobs could cause IP address allocation failures.

Enable JAVA_HOME to point to openjdk12 install path for CentOS 7.

Upgrade Notes¶

Consolidated lf-infra-jjbini macros with JJB 2.0. This requires renaming any Jenkins managed files “jjbini-sandbox” to “jjbini” to switch to the format supported in JJB > 2.0.

Projects using lf-release-jobs need to make sure they have the global variable NEXUSPROXY added in Jenkins production and Jenkins sandbox servers. The value of this variable should be the URL to the project’s Nexus server. Previous commit 118b7cbf171aca498d1a0a3a485bad990ad2e7b6 missed this variable.

Projects using lf-release-jobs need to make sure they have the global variable NEXUSPROXY added in Jenkins production and Jenkins sandbox servers. The value of this variable should be the URL to the project’s Nexus server.

This change will require to update lf-release-job calls. Update from using “{project-name}-releases-merge-{stream}”, “{project-name}-releases-verify-{stream}” to “{project-name}-release-merge-{stream}”, “{project-name}-release-verify-{stream}”. No upgrade need to be done if using “{project-name}-gerrit-release-jobs” group.

Bug Fixes¶

There is no way on finding out the $JOB_NAME pushed to sandbox with the jjb-deploy command in the logs. The change outputs the $JOB_NAME to the logs, which is useful for debugging purposes.

Add release-schema used to validate the releases yaml file as part of lf-release-jobs.

Tarball the $JAVADOC_DIR as a workaround for javadoc verfiy jobs to avoid uploading a large number of small files. Uploading a large number of small files does not work well with Nexus unpack plugin which fails on 504 gateway timeout.

Allow maven goals to be configured in sonatype-clm.sh. Set to “clean install” by default.

Allow maven goals to be configured in maven-sonar.sh. Set to “clean install” by default.

Add support for JAVA_OPTIONS in sonar job. Some of the maven build options in the sonar job require to set the JAVA_OPTIONS to specific value for the build to pass This option will help to pass the JAVA_OPTIONS from the template.

Perform “lftools schema verify” command to validate the release files against schema/release-schema.yaml Obtain optional maven central URL inside the loop that scans release files.

Allow only semantic release versions like “v${SEMVER}” or “${SEMVER}”. Fail the script if the version is not valid. Do not append any additional characters to the release version during tag and push steps.

Delete stacks with the --force option to ensure that any delete failures does not stop the openstack-cron jobs from continuing.

Download raw version of release-schema.yaml to compare against release files using lftools.

Avoid the usage of project specific variables. Do not use ODLNEXUSPROXY var, but instead use a generalized variable.

Avoid the usage of project specific variables. Do not use ODLNEXUSPROXY var, but instead use a generalized variable.

Using “releases” and “release” in different places is becoming confusing. Standardize to “release” to match lftools command and the majority of the exisiting wording.

Use “releases” for the list of tech team releases and trggers since it is intuitive there. For example “releases/1.1.1.yaml”

Move info-schema to schema/info-schema.yaml to keep schemas consistency.

Download only needed files for lf-info-yaml-verify rather than cloning the entire repo.

Allow lf-maven-stage jobs to be triggered using either “stage-release” or “stage-maven-release”.

Allow lf-maven-docker-stage jobs to be triggered using either “stage-release” or “stage-docker-release”.

Upgrade to the WhiteSource Unified Agent version 19.7.1.

v0.39.1¶

Bug Fixes¶

Update lftools version to v0.25.2.

v0.39.0¶

Critical Issues¶

lftools v0.24.0 introduced a major issue which caused the Jenkins cloud configuration merge job for OpenStack clouds to fail. This has been corrected in lftools v0.25.1

Bug Fixes¶

Add missing git-url variable in {project-name}-releases-merge-{stream} job.

Add the {stream} name in releases-verify and releases-merge jobs.

Some ONAP components like DCAEGEN2 do not host a version.properties file in the root of their repos. We need to be able to provide a location and/or different name for the version.properties file for jobs using the lf-maven-versions-plugin builder step.

v0.38.4¶

New Features¶

Group {project-name}-releases-verify and {project-name}-releases-merge into {project-name}-gerrit-release-jobs.

Add test jobs for lf-release-jobs.

Bug Fixes¶

Change parameter names used to specify container tag method, with a new default to use the fixed string ‘latest’ as a tag. Merge two shell scripts into one instead of using Jenkins conditional steps. Extend to accept a custom directory for the container-tag.yaml if the yaml-file method is used to set the docker tag information; this is an optional variable which is set to empty by default, and falls back to DOCKER_ROOT.

Add missing scm block in gerrit-releases-merge job definition. Add missing submodule-disable variable for jobs using lf-infra-gerrit-scm. Update documentation for gerrit-releases-merge and gerrit-releases-verify to remove submodule options as optional parameters.

Update lftools version to v0.25.0

Add missing $ to variable tag_file so the yq query can pull the tag from the container-tag.yaml file.

Add -l to /bin/bash shebang line at top of docker-get-container-tag.sh to make it a login shell, which automatically includes /home/jenkins/.local/bin on the path, because that is where pip installs the yq command.

v0.38.3¶

Bug Fixes¶

Add trigger on cron to docker merge macro to support regular rebuilds. This makes the merge macro match the behavior of most other jobs.

Add yamllint verification to INFO.yaml files.

Other Notes¶

Update lftools version to v0.24.0.

v0.38.2¶

Bug Fixes¶

Add missing config for triggering on file paths to docker macros and templates, namely gerrit_trigger_file_paths and github_included_regions, to make the verify and merge macros and templates match the behavior of other jobs.

v0.38.1¶

Bug Fixes¶

Install yq to be used to read yaml files. In specific, it will be needed to read container-tag.yaml.

When calling builder step macros “lf-docker-get-container-tag”, “lf-docker-build” and “lf-docker-push”, make sure the needed variables are also passed explicitly to avoid these variables appear as undefined.

Allow DOCKER_ARGS to be empty in docker-build.sh. This is not a required parameter, it can be empty.

Remove container reference in docker-get-git-describe.sh. The CONTAINER_PUSH_REGISTRY already gets added in the docker-push script. No need to add it again.

Rename image_name to image_build_tag in docker-get-yaml-tag to match docker-get-git-describe. Add missing “DOCKER_NAME” in the DOCKER_IMAGE.

docker-get-git-describe.sh and docker-get-yaml-tag.sh should only export the tag variable. Let docker-build.sh process DOCKER_NAME

Other Notes¶

Add yq install as part of python-tools-install.sh Allow future scripts to use yq package.

v0.38.0¶

New Features¶

gerrit-tox-verify now has a new parameter gerrit-skip-vote (bool) to control whether Jenkins should skip voting depending on the build outcome. It defaults to false since it is the default used by the Jenkins Gerrit Trigger Plugin.

gerrit-docker-verify runs for new commits and runs a build of the affected Docker images.

gerrit-docker-merge runs for merged commits, runs a build of the affected Docker images and pushes the images to a specified Docker registry.

New lf-release-job-merge and lf-release-job-verify templates allow projects to have self-serve releases. Project will create a tagname.yaml file in the releases/ directory of their git repo. example:

$ cat releases/4.0.0.yaml
---
distribution_type: 'maven'
version: '4.0.0'
project: 'odlparent'
log_dir: 'odlparent-maven-release-master/11/'
#below is optional
maven_central_url: 'oss.sonatype.org'

lf-infra-gerrit-scm and lf-infra-github-scm now require a submodule-disable parameter (bool) to control whether submodules are ignored or not during git fetch operations.

All job-templates now provide an optional submodule-disable parameter for git fetch operations, defaulting to false.

Upgrade Notes¶

Any project using the lf-infra-gerrit-scm and lf-infra-github-scm macros in global-jjb should need to add a submodule-disable value. It is recommended to default this value to false since it is the default used by the Jenkins Git Plugin.

Update gerrit comment trigger to use a more standard regex and avoid triggering jobs, when these keywords are intended to be used as code review comments between users. Also improve the regexs to make them more succinct and readable.

Bug Fixes¶

Add missing config for triggering on file paths to maven stage macros and templates, namely gerrit_trigger_file_paths nad github_included_regions, to make those macros and templates match the behavior of maven verify and maven merge.

fix multiple jobs created using same job-template update same github check status due to hard coded status-context to Maven Verify. Now appending status-context with maven-version and java-version to make it unique. And create different status checks in the github. fix applied for maven verify and maven docker verify jobs

Fix log shipping script to not require a LOGS_SERVER. There was a regression that caused the log shipping script to start requiring a LOGS_SERVER which fails in the case of a system that does not have that optional environment variable set.

Handle multiple search extension or patterns passed by upstream JJB ARCHIVE_ARTIFACTS param as a single string by spliting these values before being passed to lftools deploy archives.

ARCHIVE_ARTIFACTS="**/*.prop \
                  **/*.log \
                  **/target/surefire-reports/*-output.txt \
                  **/target/failsafe-reports/failsafe-summary.xml \
                  **/hs_err_*.log **/target/feature/feature.xml"

For example, the above env variable passed to the script and to lftools deploy archives as:

lftools deploy archives -p **/*.prop \
                  **/*.log \
                  **/target/surefire-reports/*-output.txt \
                  **/target/failsafe-reports/failsafe-summary.xml \
                  **/hs_err_*.log **/target/feature/feature.xml \
                  "$NEXUS_URL" \
                  "$NEXUS_PATH" \
                  "$WORKSPACE"

The correct way of passing this as per lftools implmentation is:

lftools deploy archives -p '**/*.prop' \
                  -p '**/*.log' \
                  -p '**/target/surefire-reports/*-output.txt' \
                  -p '**/target/failsafe-reports/failsafe-summary.xml' \
                  -p '**/hs_err_*.log' \
                  -p '**/target/feature/feature.xml' \
                  "$NEXUS_URL" \
                  "$NEXUS_PATH" \
                  "$WORKSPACE"

Fix error with handling unbound arrays for search extensions, when using set -u. The correct way of using this.

set -u
arr=()
echo "output: '${arr[@]}'"
bash: arr[@]: unbound variable
echo "output: '${arr[@]:-}'"
foo: ''

lf-maven-versions-plugin builder step needs to run before maven-patch-release.sh as this second script contains a condition to confirm if the maven vesions plugin was selected as a way to remove the ‘SNAPSHOT’ pattern from the pom.xml files. lf-maven-docker-stage was based on lf-maven-stage and it seems that these particular builder steps were switched in place accidentally.

The logs-deploy.sh script now allows ARCHIVE_ARTIFACTS to contain zero or more files.

request-2.22.0 does not work with python-3.4.9, so pin requests to v2.21.0 to address the tox failures.

v0.37.2¶

Bug Fixes¶

Use maven goal install (not deploy) in the maven + docker verify job. An image cannot be pushed by a verification job, and the deploy target directs the plugin to push.

v0.37.1¶

Bug Fixes¶

Remove maven-versions-plugin-set-version variable in newly added macro. This is a variable that does not need to be defined by the users of the jobs. The version needed in this builder step is inherited from versions.properties as “release_version”.

v0.37.0¶

New Features¶

Add verify, merge and stage templates for Java projects that build and wrap a JAR (e.g., a Spring-Boot application) inside a Docker image, and do not need to deploy any JAR libraries or POM files.

Upgrade Notes¶

The next release of common-packer will require a minimum version of 1.3.2 for packer. The current release of packer is 1.4.0.

Bug Fixes¶

The packer-merge job for Gerrit systems was improperly configured to use the Gerrit Trigger choosing strategy and not default. This caused issues unexpected issues with retriggering merged changes when the expectation was that it would pick up the lastest change as per normal.

This is a variable that does not need to be defined by the users of the jobs. The version needed in this builder step is inherited from versions.properties as “release_version” and it is fixed as that. This also helps teams not having to define this version in 2 places and just rely on version.properties.

Projects using maven versions plugin let this plugin take care or updating their versions in the pom.xml. When maven-versions-plugin is set to “true”, skip the stripping of SNAPSHOTS from the pom.xml files. maven-versions-plugin is set to “false” by default.

Other Notes¶

Update example Jenkins Init Script in README to redirect all output to a log file.

v0.36.0¶

Prelude¶

WhiteSource is a security and license compliance management platform. It is used to perform scans on a great variety of coding and scripting languages.

New Features¶

New comment-to-gerrit builder will comment back to gerrit patchset if a file called gerrit_comment.txt is created by the build.

Allows maven to run a clean install step before the WhiteSource scan runs the Unified Agent to fetch additional dependencies. Set to false by default.

Job {project-name}-whitesource-scan-{stream} uses the WhiteSource Unified Agent scanner CLI tool to perform the code scan and report the results into the WhiteSource dashboard.

Bug Fixes¶

Tag releases will now trigger a docs build to regenerate and update the release note link.

Update jenkins-cfg-verify job to validate new images names obtained from $GERRIT_REFSPEC instead of the master branch.

Hardcode project version to the “GERRIT_BRANCH”. Follow previous convention from CLM where reports were versioned after the branch name. Fix minor nits with bash varaibles.

wss-unified-agent.config file should not be opened for configuration to tech teams. The config file should be part of Jenkins Settings Files and called via Managed Files. wss-unified-agent.config must be created in Jenkins config files based on wss-unified-agent.config.example.

Other Notes¶

To run this job, a configuration file is needed (wss-unified-agent.config.example). A new secret text credential will need to be created. (ID=wss-apiKey Secret=WhiteSource organization API key)

Update lftools version to v0.23.1.

v0.35.0¶

New Features¶

The jjb-merge job now has a new parameter jjb-workers to allow configuration of the number of threads to run update with. Default is 0 which is equivalent to the number of CPU cores available on the system.

New info-vote-verify macro Will count votes against an INFO.yaml change and sumbit automatically if a majority of committers vote +1 or +2 on the change. Job is triggered by +2 votes or a comment of “vote”

Other Notes¶

The Maven Verify job will now call -Dmaven.source.skip to skip source jar generation in the verify job. This saves us some time in the verify build as the source artifacts are not useful in a verify job.

v0.34.0¶

New Features¶

jenkiins-init-scripts The ‘ciman’ repo is not longer required to be located in ‘/opt/ciman’.

lf-maven-set-version conditional step for lf-maven-stage to allow teams to run Maven versions plugin to update their artifact versions. Step will run if maven-versions-plugin is set to true.

Support for the Throttle Plugin is added to JJB jobs so static build servers can restrict the number of concurrent JJB jobs ran at the same time.

This must be explicitly enabled by setting throttle-enabled on the jobs.

Bug Fixes¶

Adapt maven path search for files and dirs. The “-f” maven param can specify both a directory, in which case it will look for “pom.xml” in the directory, or a specific file. The original version of this search was only compatible with directories that contain a pom.xml file.

Update the lf-maven-cental macro documentation and example templates with the missing requireed params.

Fix JAVA_HOME for openjdk11 on CentOS 7 to use the OpenJDK version installed in /usr/lib/jvm/java-11-openjdk.

The JJB Deploy Job is configured to trigger only if the Gerrit comment starts with the jjb-deploy keyword.

Without the regex being optimized the job triggers on any occurance of the jjb-deploy keyword in a Gerrit comment, with is waste infra resources.

Example of a valid command in Gerrit comment that triggers the job:

jjb-deploy builder-jjb-*

Example of a invalid command in Gerrit comment that would _not_ trigger the job:

Update the job. jjb-deploy builder-jjb-*

v0.33.0¶

New Features¶

jenkins-init-scripts If the environmental variable ‘SWAP_SIZE’ is set when the ‘init.sh’ script is called, then a ‘SWAP_SIZE’ GB swap space will be configured. If ‘SWAP_SIZE’ is ‘0’ or is not a valid integer, then no swap space is configured. If it is unset then 1GB of swap will be configured. Previously the swap size was fixed at 1GB.

jenkins-init-scripts If the work directory or volume (/w) aleady exists, the ownership will be recursivly set to ‘jenkins:jenkins’. Previously only the top directory /w was owned by ‘jenkins:jenkins’

lf-sigul-sign-dir macros now supports a sign-mode parameter which allows jobs to choose to sign artifacts using either parallel mode or serial mode (default).

Upgrade Notes¶

lf-sigul-sign-dir users need to add a new parameter sign-mode to their job-templates setting either parallel or serial as the value, we recommend setting serial mode for this setting.

{project-name}-maven-stage-{stream}’s Sigul signer now defaults to serial mode instead of the previous parallel behaviour. To change this back to the previous behaviour pass the “sign-mode” parameter to the job template:
```
- project:
    name: parallel-sign
    jobs:
      - gerrit-maven-stage:
          sign-mode: parallel
```

v0.31.0¶

New Features¶

New job-template {project-name}-release-announce for lf-releng projects to automate release announcement emails.

Add support for pushing Sonar results to SonarCloud. Refer to Maven Sonar docs for details.

Upgrade Notes¶

Jobs using the lf-maven-stage macro now need to update to the new usage. Preparation calls to lf-provide-maven-settings, lf-infra-create-netrc, and lf-provide-maven-settings-cleanup are no longer necessary to prepare the lf-maven-stage macro.

Usage:
```
- lf-maven-stage:
   mvn-global-settings: 'global-settings'
   mvn-settings: 'settings'
   mvn-staging-id: 'staging profile id'
```

v0.30.0¶

New Features¶

Packer merge jobs now include the image name in the Jenkins build description.

Bug Fixes¶

Extend ${JOB_NAME} to include {java-version} parameter to support jobs to build with multiple versions of openjdk{8,11}.

Modified lf-maven-jobs.yaml sonar cron entry to ‘{obj:cron}’ to pass value from custom user config file.

v0.29.0¶

New Features¶

Add a puppet-verify job to lf-ci-jobs. This job will perform Puppet linting on the specified repository.

- project:
    name: lf-infra-puppet-mymodule
    project-name: lf-infra-puppet
    project: puppet/modules/mymodule
    jobs:
      - gerrit-puppet-verify

Bug Fixes¶

maven-fetch-metadata.sh was not respecting the “-f” (for file path) flag in MAVEN_PARAMS, causing lf-maven-merge jobs that utilize this flag to fail. It will now set a path based on this flag if it is present, or default to the current working directory.

Check openjdk $VERSION before setting $JAVA_HOME. This enables jobs to pass “openjdk10” or “openjdk11” on CentOS 7 images to use the OpenJDK version installed in /opt.

v0.28.3¶

Bug Fixes¶

Compress and upload all jjb-verify XML files to Nexus, to ease out the IOPs on cron jobs that manage the logs on Nexus and optimize job performace by ~8 mins. This is because the job generates around ~2.3K XML files (small files) which is uploaded to Nexus in every run of jjb-verify. Doing this is faster as compared to the Nexus Unpack plugin in the Nexus end unpacking the zip file we upload takes longer.

v0.28.1¶

Other Notes¶

Update lftools version to v0.19.0.

v0.28.0¶

New Features¶

New lf-stack-create macro allows job-templates to setup a OpenStack Heat stack, useful for spinning up CSIT labs to run integration tests against. Use with the lf-stack-delete macro.

Concurrency for the gerrit-jjb-verify job can now be configured by setting the ‘build-concurrent’ parameter.

New macro lf-maven-central is available to deploy artifacts to OSSRH staging for jobs that want to eventually deploy to Maven Central.

---
- job-template:
    name: lf-maven-central-macro-test

    #####################
    # Default variables #
    #####################

    mvn-central: true
    mvn-global-settings: ""
    mvn-settings: ""
    ossrh-profile-id: ""

    #####################
    # Job configuration #
    #####################

    builders:
      - lf-maven-central:
          mvn-central: "{mvn-central}"
          mvn-global-settings: "{mvn-global-settings}"
          mvn-settings: "{mvn-settings}"
          ossrh-profile-id: "{ossrh-profile-id}"

The GERRIT_REFSPEC build parameter can now be used to trigger a test build from the Jenkins Sandbox system against a work in progress packer image patch from a GitHub Pull Request.

Upgrade Notes¶

lf-stack-delete has been modified to be a companion macro to lf-stack-create in order to cleanup the stack at the end of a job run. It now includes a required parameter openstack-cloud to choose the clouds.yaml cloud configuration for the project. Existing users of this macro will need to update their job templates accordingly.

Requires JJB 2.8.0 for the jenkins-sandbox-cleanup job to not fail.

Note

Despite the failure if JJB 2.8.0 is not available the job will successfully delete all jobs and views, the primary purpose of this job.

Bug Fixes¶

RELENG-1450 All view disappears on Jenkins Sandbox after views are deleted. The All view is now recreated after delete-all is run.

v0.27.0¶

New Features¶

Add the ability to configure the location of JJB’s cache directory for CI jobs.

New view-templates project-view, common-view, and csit-view are available for projects to manage Jenkins views through code.

To use the project-view template in a project:

- project:
    name: aaa-view
    views:
      - project-view

    project-name: aaa

To use the common-view template in a project:

- project:
    name: daily-builds
    views:
      - common-view

    view-name: Periodic
    view-regex: '.*-periodic-.*'

To use the csit-view template in a project:

- project:
    name: csit
    views:
      - csit-view

    view-name: CSIT
    view-regex: '.*csit.*'

- project:
    name: csit-1node
    views:
      - csit-view

    view-name: CSIT-1node
    view-regex: '.*-csit-1node-.*'

Add support to maven-stage jobs to publish to Maven Central via OSSRH.

This is accomplished by adding these 2 new optional parameters to the job configuration.
```
- gerrit-maven-stage:
    mvn-central: true
    ossrh-profile-id: 7edbe315063867
```

The openstack-cron job now has the ability to remove images older than a specified age (default: 30).

The openstack-cron job now has the ability to remove orphaned servers.

The openstack-cron job now has the ability to remove orphaned stacks.

The openstack-cron job now has the ability to remove orphaned volumes.

lf-infra-gerrit-scm and lf-infra-github-scm now require a submodule-timeout parameter to provide a timeout value (in minutes) for git fetch operations.

All job-templates now provide an optional submodule-timeout parameter for git fetch operations, defaulting to 10 minutes.

Upgrade Notes¶

Some LF projects are already using a common-view template in their local ci-management repo. This common-view is called project-view in global-jjb so rename all instances of common-view to project-view when upgrading and remove the local common-view view-template definition from ci-management.

The openstack-cron job now requires a new parameter configured jenkins-urls in order to use the job.

Any project using the lf-infra-gerrit-scm and lf-infra-github-scm macros in global-jjb should need to add a submodule-timeout value. It is recommended to default this value to 10 since it is the default used by the Jenkins Git Plugin.

Bug Fixes¶

The jenkins-init scripts dir is now updated to reflect changes recommended for the v0.25.0 release. We unfortunately missed this critical piece in the v0.25.0 release.

Specify refspec to be blank for SCM config on github-maven-merge job. Setting the refspec to +refs/pull/*:refs/remotes/origin/pr/* causes there to be no merge job triggered.

New detox version requires tox >= 3.5 and < 4. Instead of explicitly pulling tox, we are now implicitly pulling via detox. (https://jira.opendaylight.org/browse/RELENG-136)

Other Notes¶

lftools’ openstack module will now be installed as part of pre-build.

The openstack-cron job now runs every hour instead of daily. This is because stack cleanup should happen more regularly.

v0.26.0¶

New Features¶

Add a new nexus-iq-namespace optional parameter to insert a namespace into Nexus IQ AppID. This is useful for shared Nexus IQ systems where projects might have concern about namespace collision.

Note

We recommend when using the namespace to add a trailing - to the value. Eg. ‘odl-‘, this is to make the namespace look nice for example “odl-aaa” is the result of namespace odl-, and project name aaa.

Add lf-infra-publish-windows. A publisher for use at the end of Windows based job-templates.

Bug Fixes¶

Fix packer-verify job to correctly work with clouds.yaml config model implemented in global-jjb v0.25.0.

v0.25.1¶

Bug Fixes¶

jjb-cleanup.sh may be merged with other shell scripts that set -u which causes Jenkins to fail when activating virtualenvs

v0.25.0¶

New Features¶

Add support to the packer-build job to use clouds.yaml for openstack builder configuration rather than through the cloud-env file. This allows us to simplify the template configuration for openstack builders moving forward.

New macro lf-sigul-sign-dir available to sign artifacts in a provided directory using Sigul.

Usage:
```
- lf-sigul-sign-dir:
    sign-dir: '$WORKSPACE/m2repo'
```
This macro also requires a boolean variable to SIGN_ARTIFACTS to be set to true to activate the macro. We recommend the job-template that uses this macro to define it in the job parameters section.

Example:
```
- bool:
    name: SIGN_ARTIFACTS
    default: '{sign-artifacts}'
    description: Use Sigul to sign artifacts.
```

Add Sigul signing support to the maven-staging job. To activate Sigul signing make sure to set sign-artifacts: true. Example:
```
- project:
    name: abc
    jobs:
      - gerrit-maven-stage

    sign-artifacts: true
```

Add lf-stack-delete macro to delete an openstack heat stack at the end of the job.

This macro requires a parameter defined in the job named STACK_NAME containing the name of the stack to delete.

Add lf-infra-wrappers-windows to handle Windows specific wrapper configuration.

Refactor lf-infra-wrappers to be for Linux systems and split out the non-linux specific components into a new lf-infra-wrappers-common. This change is seamless for current users of lf-infra-wrappers.

Upgrade Notes¶

Upgrade to global-jjb v0.24.6 before performing this upgrade. This ensures that jjb-verify job pulls in a regex fix that will allow it to verify the v0.25.0 upgrade.

Global JJB now has non-JJB YAML configuration and requires action on the ci-management repo when upgrading to this version of Global JJB to prevent JJB from picking up these YAMLs as config. Follow the instructions below BEFORE upgrading globall-jjb:
```
cd <git-root>
git mv jjb/global-jjb global-jjb
mkdir jjb/global-jjb
ln -s ../../global-jjb/shell jjb/global-jjb/shell
ln -s ../../global-jjb/jjb jjb/global-jjb/jjb
git add jjb/global-jjb
git commit -sm "Prepare repo for global-jjb v0.25.0"
```

Minimum packer version 1.2.5 is now required for the packer-build job.

lf-infra-packer-build macro now requires 2 new variables to be passed.
1. openstack: Set to true if template is built using the openstack builder
2. openstack-cloud: The clouds.yaml cloud to use when running packer build

Deprecation Notes¶

lftools-install.sh is deprecated and will be removed in a future release. We recommend installing lftools via pip install –user lftools to install instead of using this script.

Bug Fixes¶

Fix pip install pip setuptools which seems to fail against the Nexus 3 proxy. Run them as separate calls to make things happier.

jjb-verify will now test on all changes in the jjb directory. The previous pattern was too specific and sometimes missed verifying patches that should be verified.

Replace jjb-verify to test on all changes in the shell/* directory.

Fix the lftools virtualenv workaround we had to put in place in the tox-verify job by using pip install --user for global tool installs.

Fix jobs failing with UNSTABLE build due to install pip==18.0 missing. This change moves all the jobs to using lf-infra-pre-build to install lftools via –user command.

Use python -m pip to ensure that we are using the pip version that was installed rather than the OS wrapper version of pip.

Fix package listing script in post-builder from causing UNSTABLE build due to difference in the two files being compared.

Fix RTD job failing to find PBR install.

Other Notes¶

Update lftools to ~ 0.17.1