Release Notes

v0.37.2

Bug Fixes

  • Use maven goal install (not deploy) in the maven + docker verify job. An image cannot be pushed by a verification job, and the deploy target directs the plugin to push.

v0.37.1

Bug Fixes

  • Remove maven-versions-plugin-set-version variable in newly added macro. This is a variable that does not need to be defined by the users of the jobs. The version needed in this builder step is inherited from versions.properties as “release_version”.

v0.37.0

New Features

  • Add verify, merge and stage templates for Java projects that build and wrap a JAR (e.g., a Spring-Boot application) inside a Docker image, and do not need to deploy any JAR libraries or POM files.

Upgrade Notes

  • The next release of common-packer will require a minimum version of 1.3.2 for packer. The current release of packer is 1.4.0.

Bug Fixes

  • The packer-merge job for Gerrit systems was improperly configured to use the Gerrit Trigger choosing strategy and not default. This caused issues unexpected issues with retriggering merged changes when the expectation was that it would pick up the lastest change as per normal.
  • This is a variable that does not need to be defined by the users of the jobs. The version needed in this builder step is inherited from versions.properties as “release_version” and it is fixed as that. This also helps teams not having to define this version in 2 places and just rely on version.properties.
  • Projects using maven versions plugin let this plugin take care or updating their versions in the pom.xml. When maven-versions-plugin is set to “true”, skip the stripping of SNAPSHOTS from the pom.xml files. maven-versions-plugin is set to “false” by default.

Other Notes

  • Update example Jenkins Init Script in README to redirect all output to a log file.

v0.36.0

Prelude

WhiteSource is a security and license compliance management platform. It is used to perform scans on a great variety of coding and scripting languages.

New Features

  • New comment-to-gerrit builder will comment back to gerrit patchset if a file called gerrit_comment.txt is created by the build.
  • Allows maven to run a clean install step before the WhiteSource scan runs the Unified Agent to fetch additional dependencies. Set to false by default.
  • Job {project-name}-whitesource-scan-{stream} uses the WhiteSource Unified Agent scanner CLI tool to perform the code scan and report the results into the WhiteSource dashboard.

Bug Fixes

  • Tag releases will now trigger a docs build to regenerate and update the release note link.
  • Update jenkins-cfg-verify job to validate new images names obtained from $GERRIT_REFSPEC instead of the master branch.
  • Hardcode project version to the “GERRIT_BRANCH”. Follow previous convention from CLM where reports were versioned after the branch name. Fix minor nits with bash varaibles.
  • wss-unified-agent.config file should not be opened for configuration to tech teams. The config file should be part of Jenkins Settings Files and called via Managed Files. wss-unified-agent.config must be created in Jenkins config files based on wss-unified-agent.config.example.

Other Notes

  • To run this job, a configuration file is needed (wss-unified-agent.config.example). A new secret text credential will need to be created. (ID=wss-apiKey Secret=WhiteSource organization API key)
  • Update lftools version to v0.23.1.

v0.35.0

New Features

  • The jjb-merge job now has a new parameter jjb-workers to allow configuration of the number of threads to run update with. Default is 0 which is equivalent to the number of CPU cores available on the system.
  • New info-vote-verify macro Will count votes against an INFO.yaml change and sumbit automatically if a majority of committers vote +1 or +2 on the change. Job is triggered by +2 votes or a comment of “vote”

Other Notes

  • The Maven Verify job will now call -Dmaven.source.skip to skip source jar generation in the verify job. This saves us some time in the verify build as the source artifacts are not useful in a verify job.

v0.34.0

New Features

  • jenkiins-init-scripts The ‘ciman’ repo is not longer required to be located in ‘/opt/ciman’.
  • lf-maven-set-version conditional step for lf-maven-stage to allow teams to run Maven versions plugin to update their artifact versions. Step will run if maven-versions-plugin is set to true.

  • Support for the Throttle Plugin is added to JJB jobs so static build servers can restrict the number of concurrent JJB jobs ran at the same time.

    This must be explicitly enabled by setting throttle-enabled on the jobs.

Bug Fixes

  • Adapt maven path search for files and dirs. The “-f” maven param can specify both a directory, in which case it will look for “pom.xml” in the directory, or a specific file. The original version of this search was only compatible with directories that contain a pom.xml file.
  • Update the lf-maven-cental macro documentation and example templates with the missing requireed params.
  • Fix JAVA_HOME for openjdk11 on CentOS 7 to use the OpenJDK version installed in /usr/lib/jvm/java-11-openjdk.

  • The JJB Deploy Job is configured to trigger only if the Gerrit comment starts with the jjb-deploy keyword.

    Without the regex being optimized the job triggers on any occurance of the jjb-deploy keyword in a Gerrit comment, with is waste infra resources.

    Example of a valid command in Gerrit comment that triggers the job:

    jjb-deploy builder-jjb-*

    Example of a invalid command in Gerrit comment that would _not_ trigger the job:

    Update the job. jjb-deploy builder-jjb-*

v0.33.0

New Features

  • jenkins-init-scripts If the environmental variable ‘SWAP_SIZE’ is set when the ‘init.sh’ script is called, then a ‘SWAP_SIZE’ GB swap space will be configured. If ‘SWAP_SIZE’ is ‘0’ or is not a valid integer, then no swap space is configured. If it is unset then 1GB of swap will be configured. Previously the swap size was fixed at 1GB.
  • jenkins-init-scripts If the work directory or volume (/w) aleady exists, the ownership will be recursivly set to ‘jenkins:jenkins’. Previously only the top directory /w was owned by ‘jenkins:jenkins’
  • lf-sigul-sign-dir macros now supports a sign-mode parameter which allows jobs to choose to sign artifacts using either parallel mode or serial mode (default).

Upgrade Notes

  • lf-sigul-sign-dir users need to add a new parameter sign-mode to their job-templates setting either parallel or serial as the value, we recommend setting serial mode for this setting.

    {project-name}-maven-stage-{stream}’s Sigul signer now defaults to serial mode instead of the previous parallel behaviour. To change this back to the previous behaviour pass the “sign-mode” parameter to the job template:

    - project:
    name: parallel-sign
    jobs:
      - gerrit-maven-stage:
          sign-mode: parallel

v0.31.0

New Features

  • New job-template {project-name}-release-announce for lf-releng projects to automate release announcement emails.
  • Add support for pushing Sonar results to SonarCloud. Refer to Maven Sonar docs for details.

Upgrade Notes

  • Jobs using the lf-maven-stage macro now need to update to the new usage. Preparation calls to lf-provide-maven-settings, lf-infra-create-netrc, and lf-provide-maven-settings-cleanup are no longer necessary to prepare the lf-maven-stage macro.

    Usage:

    - lf-maven-stage:
   mvn-global-settings: 'global-settings'
   mvn-settings: 'settings'
   mvn-staging-id: 'staging profile id'

v0.30.0

New Features

  • Packer merge jobs now include the image name in the Jenkins build description.

Bug Fixes

  • Extend ${JOB_NAME} to include {java-version} parameter to support jobs to build with multiple versions of openjdk{8,11}.
  • Modified lf-maven-jobs.yaml sonar cron entry to ‘{obj:cron}’ to pass value from custom user config file.

v0.29.0

New Features

  • Add a puppet-verify job to lf-ci-jobs. This job will perform Puppet linting on the specified repository.

    - project:
    name: lf-infra-puppet-mymodule
    project-name: lf-infra-puppet
    project: puppet/modules/mymodule
    jobs:
      - gerrit-puppet-verify

Bug Fixes

  • maven-fetch-metadata.sh was not respecting the “-f” (for file path) flag in MAVEN_PARAMS, causing lf-maven-merge jobs that utilize this flag to fail. It will now set a path based on this flag if it is present, or default to the current working directory.
  • Check openjdk $VERSION before setting $JAVA_HOME. This enables jobs to pass “openjdk10” or “openjdk11” on CentOS 7 images to use the OpenJDK version installed in /opt.

v0.28.3

Bug Fixes

  • Compress and upload all jjb-verify XML files to Nexus, to ease out the IOPs on cron jobs that manage the logs on Nexus and optimize job performace by ~8 mins. This is because the job generates around ~2.3K XML files (small files) which is uploaded to Nexus in every run of jjb-verify. Doing this is faster as compared to the Nexus Unpack plugin in the Nexus end unpacking the zip file we upload takes longer.

v0.28.1

Other Notes

  • Update lftools version to v0.19.0.

v0.28.0

New Features

  • New lf-stack-create macro allows job-templates to setup a OpenStack Heat stack, useful for spinning up CSIT labs to run integration tests against. Use with the lf-stack-delete macro.
  • Concurrency for the gerrit-jjb-verify job can now be configured by setting the ‘build-concurrent’ parameter.

  • New macro lf-maven-central is available to deploy artifacts to OSSRH staging for jobs that want to eventually deploy to Maven Central.

    ---
- job-template:
    name: lf-maven-central-macro-test

    #####################
    # Default variables #
    #####################

    mvn-central: true
    mvn-global-settings: ''
    mvn-settings: ''
    ossrh-profile-id: ''

    #####################
    # Job configuration #
    #####################

    builders:
      - lf-maven-central:
          mvn-central: '{mvn-central}'
          mvn-global-settings: '{mvn-global-settings}'
          mvn-settings: '{mvn-settings}'
          ossrh-profile-id: '{ossrh-profile-id}'
  • The GERRIT_REFSPEC build parameter can now be used to trigger a test build from the Jenkins Sandbox system against a work in progress packer image patch from a GitHub Pull Request.

Upgrade Notes

  • lf-stack-delete has been modified to be a companion macro to lf-stack-create in order to cleanup the stack at the end of a job run. It now includes a required parameter openstack-cloud to choose the clouds.yaml cloud configuration for the project. Existing users of this macro will need to update their job templates accordingly.

  • Requires JJB 2.8.0 for the jenkins-sandbox-cleanup job to not fail.

    Note

    Despite the failure if JJB 2.8.0 is not available the job will successfully delete all jobs and views, the primary purpose of this job.

Bug Fixes

  • RELENG-1450 All view disappears on Jenkins Sandbox after views are deleted. The All view is now recreated after delete-all is run.

v0.27.0

New Features

  • Add the ability to configure the location of JJB’s cache directory for CI jobs.

  • New view-templates project-view, common-view, and csit-view are available for projects to manage Jenkins views through code.

    To use the project-view template in a project:

    - project:
    name: aaa-view
    views:
      - project-view

    project-name: aaa

    To use the common-view template in a project:

    - project:
    name: daily-builds
    views:
      - common-view

    view-name: Periodic
    view-regex: '.*-periodic-.*'

    To use the csit-view template in a project:

    - project:
    name: csit
    views:
      - csit-view

    view-name: CSIT
    view-regex: '.*csit.*'

- project:
    name: csit-1node
    views:
      - csit-view

    view-name: CSIT-1node
    view-regex: '.*-csit-1node-.*'

  • Add support to maven-stage jobs to publish to Maven Central via OSSRH.

    This is accomplished by adding these 2 new optional parameters to the job configuration.

    - gerrit-maven-stage:
    mvn-central: true
    ossrh-profile-id: 7edbe315063867
  • The openstack-cron job now has the ability to remove images older than a specified age (default: 30).
  • The openstack-cron job now has the ability to remove orphaned servers.
  • The openstack-cron job now has the ability to remove orphaned stacks.
  • The openstack-cron job now has the ability to remove orphaned volumes.
  • lf-infra-gerrit-scm and lf-infra-github-scm now require a submodule-timeout parameter to provide a timeout value (in minutes) for git fetch operations.
  • All job-templates now provide an optional submodule-timeout parameter for git fetch operations, defaulting to 10 minutes.

Upgrade Notes

  • Some LF projects are already using a common-view template in their local ci-management repo. This common-view is called project-view in global-jjb so rename all instances of common-view to project-view when upgrading and remove the local common-view view-template definition from ci-management.
  • The openstack-cron job now requires a new parameter configured jenkins-urls in order to use the job.
  • Any project using the lf-infra-gerrit-scm and lf-infra-github-scm macros in global-jjb should need to add a submodule-timeout value. It is recommended to default this value to 10 since it is the default used by the Jenkins Git Plugin.

Bug Fixes

  • The jenkins-init scripts dir is now updated to reflect changes recommended for the v0.25.0 release. We unfortunately missed this critical piece in the v0.25.0 release.
  • Specify refspec to be blank for SCM config on github-maven-merge job. Setting the refspec to +refs/pull/*:refs/remotes/origin/pr/* causes there to be no merge job triggered.

Other Notes

  • lftools’ openstack module will now be installed as part of pre-build.
  • The openstack-cron job now runs every hour instead of daily. This is because stack cleanup should happen more regularly.

v0.26.0

New Features

  • Add a new nexus-iq-namespace optional parameter to insert a namespace into Nexus IQ AppID. This is useful for shared Nexus IQ systems where projects might have concern about namespace collision.

    Note

    We recommend when using the namespace to add a trailing - to the value. Eg. ‘odl-‘, this is to make the namespace look nice for example “odl-aaa” is the result of namespace odl-, and project name aaa.

  • Add lf-infra-publish-windows. A publisher for use at the end of Windows based job-templates.

Bug Fixes

  • Fix packer-verify job to correctly work with clouds.yaml config model implemented in global-jjb v0.25.0.

v0.25.1

Bug Fixes

  • jjb-cleanup.sh may be merged with other shell scripts that set -u which causes Jenkins to fail when activating virtualenvs

v0.25.0

New Features

  • Add support to the packer-build job to use clouds.yaml for openstack builder configuration rather than through the cloud-env file. This allows us to simplify the template configuration for openstack builders moving forward.

  • New macro lf-sigul-sign-dir available to sign artifacts in a provided directory using Sigul.

    Usage:

    - lf-sigul-sign-dir:
    sign-dir: '$WORKSPACE/m2repo'

    This macro also requires a boolean variable to SIGN_ARTIFACTS to be set to true to activate the macro. We recommend the job-template that uses this macro to define it in the job parameters section.

    Example:

    - bool:
    name: SIGN_ARTIFACTS
    default: '{sign-artifacts}'
    description: Use Sigul to sign artifacts.

  • Add Sigul signing support to the maven-staging job. To activate Sigul signing make sure to set sign-artifacts: true. Example:

    - project:
    name: abc
    jobs:
      - gerrit-maven-stage

    sign-artifacts: true

  • Add lf-stack-delete macro to delete an openstack heat stack at the end of the job.

    This macro requires a parameter defined in the job named STACK_NAME containing the name of the stack to delete.

  • Add lf-infra-wrappers-windows to handle Windows specific wrapper configuration.
  • Refactor lf-infra-wrappers to be for Linux systems and split out the non-linux specific components into a new lf-infra-wrappers-common. This change is seamless for current users of lf-infra-wrappers.

Upgrade Notes

  • Upgrade to global-jjb v0.24.6 before performing this upgrade. This ensures that jjb-verify job pulls in a regex fix that will allow it to verify the v0.25.0 upgrade.

  • Global JJB now has non-JJB YAML configuration and requires action on the ci-management repo when upgrading to this version of Global JJB to prevent JJB from picking up these YAMLs as config. Follow the instructions below BEFORE upgrading globall-jjb:

    cd <git-root>
git mv jjb/global-jjb global-jjb
mkdir jjb/global-jjb
ln -s ../../global-jjb/shell jjb/global-jjb/shell
ln -s ../../global-jjb/jjb jjb/global-jjb/jjb
git add jjb/global-jjb
git commit -sm "Prepare repo for global-jjb v0.25.0"
  • Minimum packer version 1.2.5 is now required for the packer-build job.
  • lf-infra-packer-build macro now requires 2 new variables to be passed.
    1. openstack: Set to true if template is built using the openstack builder
    2. openstack-cloud: The clouds.yaml cloud to use when running packer build

Deprecation Notes

  • lftools-install.sh is deprecated and will be removed in a future release. We recommend installing lftools via pip install –user lftools to install instead of using this script.

Bug Fixes

  • Fix pip install pip setuptools which seems to fail against the Nexus 3 proxy. Run them as separate calls to make things happier.
  • jjb-verify will now test on all changes in the jjb directory. The previous pattern was too specific and sometimes missed verifying patches that should be verified.
  • Replace jjb-verify to test on all changes in the shell/* directory.
  • Fix the lftools virtualenv workaround we had to put in place in the tox-verify job by using pip install --user for global tool installs.
  • Fix jobs failing with UNSTABLE build due to install pip==18.0 missing. This change moves all the jobs to using lf-infra-pre-build to install lftools via –user command.
  • Use python -m pip to ensure that we are using the pip version that was installed rather than the OS wrapper version of pip.
  • Fix package listing script in post-builder from causing UNSTABLE build due to difference in the two files being compared.
  • Fix RTD job failing to find PBR install.

Other Notes

  • Update lftools to ~ 0.17.1