Release Notes

v0.34.0

New Features

  • jenkiins-init-scripts The ‘ciman’ repo is not longer required to be located in ‘/opt/ciman’.
  • lf-maven-set-version conditional step for lf-maven-stage to allow teams to run Maven versions plugin to update their artifact versions. Step will run if maven-versions-plugin is set to true.
  • Support for the Throttle Plugin is added to JJB jobs so static build servers can restrict the number of concurrent JJB jobs ran at the same time.

    This must be explicitly enabled by setting throttle-enabled on the jobs.

Bug Fixes

  • Adapt maven path search for files and dirs. The “-f” maven param can specify both a directory, in which case it will look for “pom.xml” in the directory, or a specific file. The original version of this search was only compatible with directories that contain a pom.xml file.
  • Update the lf-maven-cental macro documentation and example templates with the missing requireed params.
  • Fix JAVA_HOME for openjdk11 on CentOS 7 to use the OpenJDK version installed in /usr/lib/jvm/java-11-openjdk.
  • The JJB Deploy Job is configured to trigger only if the Gerrit comment starts with the jjb-deploy keyword.

    Without the regex being optimized the job triggers on any occurance of the jjb-deploy keyword in a Gerrit comment, with is waste infra resources.

    Example of a valid command in Gerrit comment that triggers the job:

    jjb-deploy builder-jjb-*

    Example of a invalid command in Gerrit comment that would _not_ trigger the job:

    Update the job. jjb-deploy builder-jjb-*

v0.33.0

New Features

  • jenkins-init-scripts If the environmental variable ‘SWAP_SIZE’ is set when the ‘init.sh’ script is called, then a ‘SWAP_SIZE’ GB swap space will be configured. If ‘SWAP_SIZE’ is ‘0’ or is not a valid integer, then no swap space is configured. If it is unset then 1GB of swap will be configured. Previously the swap size was fixed at 1GB.
  • jenkins-init-scripts If the work directory or volume (/w) aleady exists, the ownership will be recursivly set to ‘jenkins:jenkins’. Previously only the top directory /w was owned by ‘jenkins:jenkins’
  • lf-sigul-sign-dir macros now supports a sign-mode parameter which allows jobs to choose to sign artifacts using either parallel mode or serial mode (default).

Upgrade Notes

  • lf-sigul-sign-dir users need to add a new parameter sign-mode to their job-templates setting either parallel or serial as the value, we recommend setting serial mode for this setting.

    {project-name}-maven-stage-{stream}’s Sigul signer now defaults to serial mode instead of the previous parallel behaviour. To change this back to the previous behaviour pass the “sign-mode” parameter to the job template:

    - project:
        name: parallel-sign
        jobs:
          - gerrit-maven-stage:
              sign-mode: parallel
    

v0.31.0

New Features

  • New job-template {project-name}-release-announce for lf-releng projects to automate release announcement emails.
  • Add support for pushing Sonar results to SonarCloud. Refer to Maven Sonar docs for details.

Upgrade Notes

  • Jobs using the lf-maven-stage macro now need to update to the new usage. Preparation calls to lf-provide-maven-settings, lf-infra-create-netrc, and lf-provide-maven-settings-cleanup are no longer necessary to prepare the lf-maven-stage macro.

    Usage:

    - lf-maven-stage:
       mvn-global-settings: 'global-settings'
       mvn-settings: 'settings'
       mvn-staging-id: 'staging profile id'
    

v0.30.0

New Features

  • Packer merge jobs now include the image name in the Jenkins build description.

Bug Fixes

  • Extend ${JOB_NAME} to include {java-version} parameter to support jobs to build with multiple versions of openjdk{8,11}.
  • Modified lf-maven-jobs.yaml sonar cron entry to ‘{obj:cron}’ to pass value from custom user config file.

v0.29.0

New Features

  • Add a puppet-verify job to lf-ci-jobs. This job will perform Puppet linting on the specified repository.

    - project:
        name: lf-infra-puppet-mymodule
        project-name: lf-infra-puppet
        project: puppet/modules/mymodule
        jobs:
          - gerrit-puppet-verify
    

Bug Fixes

  • maven-fetch-metadata.sh was not respecting the “-f” (for file path) flag in MAVEN_PARAMS, causing lf-maven-merge jobs that utilize this flag to fail. It will now set a path based on this flag if it is present, or default to the current working directory.
  • Check openjdk $VERSION before setting $JAVA_HOME. This enables jobs to pass “openjdk10” or “openjdk11” on CentOS 7 images to use the OpenJDK version installed in /opt.

v0.28.3

Bug Fixes

  • Compress and upload all jjb-verify XML files to Nexus, to ease out the IOPs on cron jobs that manage the logs on Nexus and optimize job performace by ~8 mins. This is because the job generates around ~2.3K XML files (small files) which is uploaded to Nexus in every run of jjb-verify. Doing this is faster as compared to the Nexus Unpack plugin in the Nexus end unpacking the zip file we upload takes longer.

v0.28.1

Other Notes

  • Update lftools version to v0.19.0.

v0.28.0

New Features

  • New lf-stack-create macro allows job-templates to setup a OpenStack Heat stack, useful for spinning up CSIT labs to run integration tests against. Use with the lf-stack-delete macro.
  • Concurrency for the gerrit-jjb-verify job can now be configured by setting the ‘build-concurrent’ parameter.
  • New macro lf-maven-central is available to deploy artifacts to OSSRH staging for jobs that want to eventually deploy to Maven Central.

    ---
    - job-template:
        name: lf-maven-central-macro-test
    
        #####################
        # Default variables #
        #####################
    
        mvn-central: true
        mvn-global-settings: ''
        mvn-settings: ''
        ossrh-profile-id: ''
    
        #####################
        # Job configuration #
        #####################
    
        builders:
          - lf-maven-central:
              mvn-central: '{mvn-central}'
              mvn-global-settings: '{mvn-global-settings}'
              mvn-settings: '{mvn-settings}'
              ossrh-profile-id: '{ossrh-profile-id}'
    
  • The GERRIT_REFSPEC build parameter can now be used to trigger a test build from the Jenkins Sandbox system against a work in progress packer image patch from a GitHub Pull Request.

Upgrade Notes

  • lf-stack-delete has been modified to be a companion macro to lf-stack-create in order to cleanup the stack at the end of a job run. It now includes a required parameter openstack-cloud to choose the clouds.yaml cloud configuration for the project. Existing users of this macro will need to update their job templates accordingly.
  • Requires JJB 2.8.0 for the jenkins-sandbox-cleanup job to not fail.

    Note

    Despite the failure if JJB 2.8.0 is not available the job will successfully delete all jobs and views, the primary purpose of this job.

Bug Fixes

  • RELENG-1450 All view disappears on Jenkins Sandbox after views are deleted. The All view is now recreated after delete-all is run.

v0.27.0

New Features

  • Add the ability to configure the location of JJB’s cache directory for CI jobs.
  • New view-templates project-view, common-view, and csit-view are available for projects to manage Jenkins views through code.

    To use the project-view template in a project:

    - project:
        name: aaa-view
        views:
          - project-view
    
        project-name: aaa
    

    To use the common-view template in a project:

    - project:
        name: daily-builds
        views:
          - common-view
    
        view-name: Periodic
        view-regex: '.*-periodic-.*'
    

    To use the csit-view template in a project:

    - project:
        name: csit
        views:
          - csit-view
    
        view-name: CSIT
        view-regex: '.*csit.*'
    
    - project:
        name: csit-1node
        views:
          - csit-view
    
        view-name: CSIT-1node
        view-regex: '.*-csit-1node-.*'
    
  • Add support to maven-stage jobs to publish to Maven Central via OSSRH.

    This is accomplished by adding these 2 new optional parameters to the job configuration.

    - gerrit-maven-stage:
        mvn-central: true
        ossrh-profile-id: 7edbe315063867
    
  • The openstack-cron job now has the ability to remove images older than a specified age (default: 30).
  • The openstack-cron job now has the ability to remove orphaned servers.
  • The openstack-cron job now has the ability to remove orphaned stacks.
  • The openstack-cron job now has the ability to remove orphaned volumes.
  • lf-infra-gerrit-scm and lf-infra-github-scm now require a submodule-timeout parameter to provide a timeout value (in minutes) for git fetch operations.
  • All job-templates now provide an optional submodule-timeout parameter for git fetch operations, defaulting to 10 minutes.

Upgrade Notes

  • Some LF projects are already using a common-view template in their local ci-management repo. This common-view is called project-view in global-jjb so rename all instances of common-view to project-view when upgrading and remove the local common-view view-template definition from ci-management.
  • The openstack-cron job now requires a new parameter configured jenkins-urls in order to use the job.
  • Any project using the lf-infra-gerrit-scm and lf-infra-github-scm macros in global-jjb should need to add a submodule-timeout value. It is recommended to default this value to 10 since it is the default used by the Jenkins Git Plugin.

Bug Fixes

  • The jenkins-init scripts dir is now updated to reflect changes recommended for the v0.25.0 release. We unfortunately missed this critical piece in the v0.25.0 release.
  • Specify refspec to be blank for SCM config on github-maven-merge job. Setting the refspec to +refs/pull/*:refs/remotes/origin/pr/* causes there to be no merge job triggered.

Other Notes

  • lftools’ openstack module will now be installed as part of pre-build.
  • The openstack-cron job now runs every hour instead of daily. This is because stack cleanup should happen more regularly.

v0.26.0

New Features

  • Add a new nexus-iq-namespace optional parameter to insert a namespace into Nexus IQ AppID. This is useful for shared Nexus IQ systems where projects might have concern about namespace collision.

    Note

    We recommend when using the namespace to add a trailing - to the value. Eg. ‘odl-‘, this is to make the namespace look nice for example “odl-aaa” is the result of namespace odl-, and project name aaa.

  • Add lf-infra-publish-windows. A publisher for use at the end of Windows based job-templates.

Bug Fixes

  • Fix packer-verify job to correctly work with clouds.yaml config model implemented in global-jjb v0.25.0.

v0.25.1

Bug Fixes

  • jjb-cleanup.sh may be merged with other shell scripts that set -u which causes Jenkins to fail when activating virtualenvs

v0.25.0

New Features

  • Add support to the packer-build job to use clouds.yaml for openstack builder configuration rather than through the cloud-env file. This allows us to simplify the template configuration for openstack builders moving forward.
  • New macro lf-sigul-sign-dir available to sign artifacts in a provided directory using Sigul.

    Usage:

    - lf-sigul-sign-dir:
        sign-dir: '$WORKSPACE/m2repo'
    

    This macro also requires a boolean variable to SIGN_ARTIFACTS to be set to true to activate the macro. We recommend the job-template that uses this macro to define it in the job parameters section.

    Example:

    - bool:
        name: SIGN_ARTIFACTS
        default: '{sign-artifacts}'
        description: Use Sigul to sign artifacts.
    
  • Add Sigul signing support to the maven-staging job. To activate Sigul signing make sure to set sign-artifacts: true. Example:

    - project:
        name: abc
        jobs:
          - gerrit-maven-stage
    
        sign-artifacts: true
    
  • Add lf-stack-delete macro to delete an openstack heat stack at the end of the job.

    This macro requires a parameter defined in the job named STACK_NAME containing the name of the stack to delete.

  • Add lf-infra-wrappers-windows to handle Windows specific wrapper configuration.
  • Refactor lf-infra-wrappers to be for Linux systems and split out the non-linux specific components into a new lf-infra-wrappers-common. This change is seamless for current users of lf-infra-wrappers.

Upgrade Notes

  • Upgrade to global-jjb v0.24.6 before performing this upgrade. This ensures that jjb-verify job pulls in a regex fix that will allow it to verify the v0.25.0 upgrade.
  • Global JJB now has non-JJB YAML configuration and requires action on the ci-management repo when upgrading to this version of Global JJB to prevent JJB from picking up these YAMLs as config. Follow the instructions below BEFORE upgrading globall-jjb:

    cd <git-root>
    git mv jjb/global-jjb global-jjb
    mkdir jjb/global-jjb
    ln -s ../../global-jjb/shell jjb/global-jjb/shell
    ln -s ../../global-jjb/jjb jjb/global-jjb/jjb
    git add jjb/global-jjb
    git commit -sm "Prepare repo for global-jjb v0.25.0"
    
  • Minimum packer version 1.2.5 is now required for the packer-build job.
  • lf-infra-packer-build macro now requires 2 new variables to be passed.
    1. openstack: Set to true if template is built using the openstack builder
    2. openstack-cloud: The clouds.yaml cloud to use when running packer build

Deprecation Notes

  • lftools-install.sh is deprecated and will be removed in a future release. We recommend installing lftools via pip install –user lftools to install instead of using this script.

Bug Fixes

  • Fix pip install pip setuptools which seems to fail against the Nexus 3 proxy. Run them as separate calls to make things happier.
  • jjb-verify will now test on all changes in the jjb directory. The previous pattern was too specific and sometimes missed verifying patches that should be verified.
  • Replace jjb-verify to test on all changes in the shell/* directory.
  • Fix the lftools virtualenv workaround we had to put in place in the tox-verify job by using pip install --user for global tool installs.
  • Fix jobs failing with UNSTABLE build due to install pip==18.0 missing. This change moves all the jobs to using lf-infra-pre-build to install lftools via –user command.
  • Use python -m pip to ensure that we are using the pip version that was installed rather than the OS wrapper version of pip.
  • Fix package listing script in post-builder from causing UNSTABLE build due to difference in the two files being compared.
  • Fix RTD job failing to find PBR install.

Other Notes

  • Update lftools to ~ 0.17.1