New Infra Bootstrap¶
This document uses example.org
as the domain for all examples. Please
change to point to the intended systems for your project.
Jenkins¶
Steps
Login to Jenkins at https://jenkins.example.org
Navigate to https://jenkins.example.org/pluginManager/
Update all plugins
Install required plugins as documented in global-jjb install guide
Install the following plugins:
Navigate to https://jenkins.example.org/configure
Configure Jenkins as follows:
# of executors: 0 Jenkins URL: https://jenkins.example.org System Admin e-mail address: Jenkins <jenkins-dontreply@example.org> Global Config user.name value: jenkins Global Config user.email value: jenkins@example.org
If using the Message Injector plugin set
Message to inject
toLogs: https://logs.example.org/SILO/HOSTNAME/$JOB_NAME/$BUILD_NUMBER
and replaceSILO
andHOSTNAME
as appropriate.Click
Save
Configure Jenkins security as described in Jenkins Security
Navigate to https://jenkins.example.org/configureSecurity/
Configure the following permissions for
Anonymous Users
Overall:Read
Job:ExtendedRead
Job:Read
View:Read
Note
If the project is not yet public, hold off on these permissions or adjust as necessary for the project’s case.
Setup Jenkins global environment variables as described in the global-jjb install guide
Note
Skip the ci-management step in as we will be discussing that in the next section.
Setup a jobbuilder account
Setup global-jjb required Jenkins Files
Setup Job Builder account¶
The ci-jobs in global-jjb require a jobbuilder account which has permissions to login to Jenkins.
Navigate to and create an account for jobbuilder https://identity.linuxfoundation.org/
Note
This step mainly applies to LF projects. Use the relevant identity system as it applies to your local configuration.
Navigate to https://jenkins.example.org/configureSecurity and configure permissions for the jobbuilder account as follows:
Overall: Administer
Job: Configure
Job: Create
Job: Delete
Job: Discover
Job: Read
View: Configure
View: Create
View: Delete
View: Read
Setup Sandbox Access¶
To allow people access to the Jenkins Sandbox, we require an LDAP group to
exist with the appropriate people added. Use lftools lfidapi create-group
to create a group called $project-jenkins-sandbox-access
and add any initial
members you might need.
Go to https://jenkins.example.org/configureSecurity and add the group with:
Overall: Read
Job: Build
Job: Cancel
Job: Configure
Job: Create
Job: Delete
Job: Discover
Job: Read
Job: Workspace
View: Read
ci-management repo¶
Once Jenkins is available we can initialize a new ci-management repo.
Setup administrative files¶
Create ci-management repo in the project SCM system
Create a README.md file explaining the purpose of the repo
# ci-management This repo contains configuration files for Jenkins jobs for the EXAMPLE project.
Setup tox/coala linting for
jjb/
andpacker/
directories.yamllint.conf
extends: default rules: empty-lines: max-end: 1 line-length: max: 120
.coafile
[Documentation] bears = WriteGoodLintBear files = *.md allow_so_beginning = False allow_there_is = False allow_cliche_phrases = False [GitCommit] bears = GitCommitBear ignore_length_regex = Signed-off-by, Also-by, Co-authored-by, http://, https:// [JSON] bears = JSONFormatBear files = packer/**.json indent_size = 2 [ShellCheck] bears = ShellCheckBear, SpaceConsistencyBear files = jjb/**.sh, packer/**.sh shell = bash indent_size = 4 use_spaces = yeah [YAML] bears = YAMLLintBear files = jjb/**/*.yaml document_start = True yamllint_config = .yamllint.conf
tox.ini
[tox] minversion = 1.6 envlist = coala skipsdist = true [testenv:coala] basepython = python3 deps = coala==0.11 coala-bears==0.11 nodeenv~=1.3.0 commands = nodeenv -p npm install --global write-good python3 -m nltk.downloader punkt maxent_treebank_pos_tagger averaged_perceptron_tagger coala --non-interactive
Setup .gitignore
.tox/ archives/ jenkins.ini # Packer .galaxy/ *.retry cloud-env.json
git commit -asm "Setup repo administrative files"
git push
files to the repositoryRun
tox
Note
The
jjb
tox env will fail as the requiredjjb/
directory does not yet exist. This is fine and proves that tox is working before we continue in the next step.
Bootstrap common-packer and initial builder¶
Note
This section assumes the usage of an OpenStack cloud provider for Jenkins build nodes. Adjust as necessary if not using an OpenStack cloud.
Navigate to the
GIT_ROOT
of the ci-management repoInstall common-packer to
GIT_ROOT/packer/common-packer
git submodule add https://github.com/lfit/releng-common-packer.git packer/common-packer
Follow common-packer doc to setup a template
git commit -asm "Setup common-packer and initial builder"
git push
files to repositoryUpload a CentOS 7 cloudimg to use as a base for packer builds
When uploading the cloudimg ensure it’s name matches the
base_image
name incommon-packer/vars/centos-7.json
.Run
packer build -var-file=cloud-env.json -var-file=common-packer/vars/centos-7.json templates/builder.json
Note down the image name from the packer build as we will need it later
Navigate to
https://jenkins.example.org/credentials/store/system/domain/_/newCredentials
Configure the OpenStack cloud credential as follows:
Kind: OpenStack auth v3 Project Domain: Default Project Name: OPENSTACK_TENANT_ID User Domain: Default User Name: OPENSTACK_USERNAME Password: OPENSTACK_PASSWORD ID: os-cloud Description: openstack-cloud-credential
Note
Replace ALL_CAPS instances with your Cattle account credential.
Configure an ssh keypair for the Jenkins <-> OpenStack connection
Generate a new SSH Keypair
ssh-keygen -t rsa -C jenkins-ssh -f /tmp/jenkins
Navigate to
https://jenkins.example.org/credentials/store/system/domain/_/newCredentials
Configure the Jenkins SSH Key as follows:
Kind: SSH Username and private key Scope: Global Username: jenkins Private Key: Enter directly Passphrase: ID: jenkins-ssh Description: jenkins-ssh
Copy the contents of
/tmp/jenkins
into the Key field.Navigate to
https://openstack-cloud.example.org/project/key_pairs
Import the contents of
/tmp/jenkins.pub
into the OpenStack cloud provider account with the keypair namejenkins-ssh
Navigate to
https://jenkins.example.org/configfiles/selectProvider
Create a
OpenStack User Data
file with the following specs:Type: OpenStack User Data ID: jenkins-init-script Name: jenkins-init-script Comment: jenkins-init-script
With the contents (change the git clone URL as necessary for the project):
#!/bin/bash until host gerrit.example.org &>/dev/null do echo "Waiting until gerrit.example.org is resolvable..." done git clone --recurse-submodules https://gerrit.example.org/r/ci-management /opt/ciman /opt/ciman/jjb/global-jjb/jenkins-init-scripts/init.sh
For Windows:
Type: OpenStack User Data ID: jenkins-init-script-windows Name: jenkins-init-script-windows Comment: jenkins-init-script-windows
With the contents (change the git clone URL as necessary for the project):
<powershell> # Resize first partition of first disk to maximum size Get-Partition -DiskNumber 0 -PartitionNumber 1 $size = (Get-PartitionSupportedSize -DiskNumber 0 -PartitionNumber 1) Resize-Partition -DiskNumber 0 -PartitionNumber 1 -Size $size.SizeMax mkdir -Force "${SLAVE_JENKINS_HOME}" (new-object System.Net.WebClient).DownloadFile('${SLAVE_JAR_URL}','${SLAVE_JENKINS_HOME}\slave.jar') cd "${SLAVE_JENKINS_HOME}" java ${SLAVE_JVM_OPTIONS} -jar "slave.jar" -jnlpUrl "${SLAVE_JNLP_URL}" -secret "${SLAVE_JNLP_SECRET}" </powershell>
Configure
cattle
cloudCreate cloud config directory
mkdir -p jenkins-config/clouds/openstack/cattle
Configure the OpenStack cloud connection details in
jenkins-config/clouds/openstack/cattle/cloud.cfg
Replace
<BUILD_IMAGE_NAME>
and<NETWORK_ID>
in the below file with the details for your cloud. Find<NETWORK_ID>
at https://dashboard.vexxhost.net/project/networks/jenkins-config/clouds/openstack/cattle/cloud.cfg¶# Cloud Configuration CLOUD_CREDENTIAL_ID=os-cloud CLOUD_URL=https://auth.vexxhost.net/v3/ CLOUD_IGNORE_SSL=false CLOUD_ZONE=ca-ymq-1 # Default Template Configuration IMAGE_NAME=<BUILD_IMAGE_NAME> HARDWARE_ID=v3-standard-2 NETWORK_ID=<NETWORK_ID> USER_DATA_ID=jenkins-init-script INSTANCE_CAP=10 SANDBOX_CAP=4 FLOATING_IP_POOL= SECURITY_GROUPS=default STARTUP_TIMEOUT=600000 KEY_PAIR_NAME=jenkins-ssh NUM_EXECUTORS=1 JVM_OPTIONS= FS_ROOT=/w RETENTION_TIME=0
Create
jenkins-config/clouds/openstack/cattle/centos7-builder-2c-1g.cfg
IMAGE_NAME=ZZCI - CentOS 7 - builder - 20180604-1653 HARDWARE_ID=v3-standard-2
Run global-jjb jenkins-cfg script to update Jenkins cloud config
Note
This step requires
crudini
tool, install from your package manager to avoid python 2 vs 3 problems in your virtualenv.Note
This step requires having lftools available on your path and a
~/.config/jenkins_jobs/jenkins_jobs.ini
configured with Jenkins credentials.Set
jenkins_silos
to match the config section name in thejenkins_jobs.ini
file.Run the following commands:
export WORKSPACE=$(pwd) export jenkins_silos=production bash ./jjb/global-jjb/shell/jenkins-configure-clouds.sh # OPTIONAL: view the created script cat archives/groovy-inserts/production-cloud-cfg.groovy
Then navigate to
https://jenkins.example.org/script
and copy the contents ofarchives/groovy-inserts/production-cloud-cfg.groovy
into the script console. This will initialize the OpenStack cloud configuration.Commit the
jenkins-config
directorygit add jenkins-config/ git commit -sm "Add OpenStack cloud configuration" git push
Navigate to
https://jenkins.example.org/configure
and verify the cloud configuration.
Setup global-jjb and ci-jobs¶
Install global-jjb to
GIT_ROOT/jjb/global-jjb
git submodule add https://github.com/lfit/releng-global-jjb.git jjb/global-jjb
Setup
jjb/defaults.yaml
- defaults: name: global gerrit-server-name: Primary git-url: 'ssh://jenkins-$SILO@gerrit.example.org:29418' jenkins-ssh-credential: jenkins-ssh lftools-version: '<1.0.0'
Create the CI Jobs in
jjb/ci-management/ci-jobs.yaml
- project: name: ci-jobs jobs: - '{project-name}-ci-jobs' project: ci-management project-name: ci-management build-node: centos7-builder-2c-1g
Manually push the initial ci-management jobs to Jenkins
jenkins-jobs update jjb/
Git commit the current files and push to Gerrit
git commit -sm "Setup global-jjb and ci-jobs" git push
Confirm verify jobs work
Merge the patch and confirm merge job works
Setup packer jobs¶
Create Initial CI Packer job in jjb/ci-management/ci-packer.yaml
- project: name: packer-verify jobs: - gerrit-packer-verify project: ci-management project-name: ci-management build-node: centos7-builder-2c-1g - project: name: packer-builder-jobs jobs: - gerrit-packer-merge project: ci-management project-name: ci-management build-node: centos7-builder-2c-1g templates: builder platforms: - centos-7 - ubuntu-16.04
Git commit and push the patch to ci-management for review
git commit -sm "Add packer builder job" git push ...
Confirm packer verify job passes
Merge patch and confirm merge job works
Nexus 2¶
Setup Server Config¶
Navigate to https://nexus.example.org/#nexus-config
SMTP Settings
Hostname: localhost Port: 25 Username: Password: Connection: Use plain SMTP System Email: noreply@example.org
Application Server Settings
Base URL: https://nexus.example.org/ Force base URL: true UI Timeout: 120
PGP Key Server Information
Server 1: http://pool.sks-keyservers.net:11371 Server 2: http://pgp.mit.edu:11371
Setup LDAP¶
Navigate to https://nexus.example.org/#enterprise-ldap
Click
Add
at the top menu barConfigure the LDAP connection as follows:
Name: ldaps://ldap.example.org:636 Protocol: ldaps Hostname: ldap.example.org Port: 636 Search Base: dc=example,dc=org Authentication: Anonymous Authentication
Click on the
User & Group Settings
tabConfigure the
User & Group Settings
as follows:Base DN: ou=Users Object Class: inetOrgPerson User ID Attribute: uid Real Name Attribute: cn E-Mail Attribute: mail Group Type: Static Groups Base DN: ou=groups Object Class: groupOfNames Group ID Attribute: cn Group Member Attribute: member Group Member Format: ${dn}
Setup Admin role¶
Navigate to https://nexus.example.org/#security-roles
Click
Add > External Role Mapping
Configure mapping as follows:
Realm: LDAP Role: lf-collab-admins
Note
If not an LF project replace
lf-collab-admins
with the relevant admin group for your case.Click
Add
and add theNexus Administrator Role
From this point you should be able to login using your own account to administrate the server. Do that and then setup admin user email and deactivate the default deployment account as we will create separate project deployment accounts for each individual project.
Navigate to https://nexus.example.org/#security-users
Configure the admin user email to
collab-it+PROJECT@linuxfoundation.org
Note
Replace email as necessary for your org.
Set the default deployment user account Status to
Disabled
Setup custom deployment role¶
LF projects use Nexus 2 as a server to host logs and requires the
Nexus Unpack
plugin configured. Since the default Nexus Deployment Role
is not configurable, we will have to create our own custom one to ensure Unpack
is available.
Navigate to https://nexus.example.org/#security-roles
Click
Add > Nexus Role
Configure the following settings:
Role Id: lf-deployment Name: LF Deployment Role Description: LF modified deployment role
Click
Add
and add the following roles:Artifact Upload
Nexus Deployment Role
Unpack
Setup routing¶
Navigate to https://nexus.example.org/#routes-config
Clear all existing routes
Click
Add
to add a new routeConfigure the route as follows:
URL Pattern: ^/org/example/.* Rule Type: Inclusive Repository Group: All Repository Groups Ordered Route Repositories: * Releases * Snapshots
Nexus 3¶
Setup Server Config¶
Navigate to https://nexus3.example.org/#admin/system/emailserver
SMTP Settings
Enabled: true Hostname: localhost Port: 25 Username: Password: From address: noreply@example.org Subject prefix:
Setup LDAP¶
Navigate to https://nexus3.example.org/#admin/security/ldap
Click
Create connection
Configure the LDAP connection as follows
Name: ldaps://ldap.example.org:636 Protocol: ldaps Hostname: ldap.example.org Port: 636 Search base: dc=example,dc=org Authentication method: Anonymous Authentication
Click
Verify connection
and check that it worksClick
Next
Configure the
User & Group Settings
as follows:Base DN: ou=Users Object Class: inetOrgPerson User ID Attribute: uid Real Name Attribute: cn E-Mail Attribute: mail Map LDAP groups as roles: true Group Type: Static Groups Base DN: ou=groups Object Class: groupOfNames Group ID Attribute: cn Group Member Attribute: member Group Member Format: ${dn}
Click
Verify user mapping
and confirm it worksClick
Create
Setup Admin role¶
Navigate to https://nexus3.example.org/#admin/security/roles
Click
Create role > External Role Mapping
Configure mapping as follows:
Mapped Role: lf-collab-admins Role Name: lf-collab-admins Role description: lf-collab-admins Privileges: nx-all
From this point you should be able to login using your own account to administrate the server. Do that and then setup admin user email and deactivate the default deployment account as we will create separate project deployment accounts for each individual project.
Navigate to https://nexus3.example.org/#admin/security/users:admin
Configure the admin user email to
collab-it+PROJECT@linuxfoundation.org
Note
Replace email as necessary for your org.
Post bootstrap¶
With infrastructure bootstrapped, here is a list of tasks to consider that may be useful to setup.