Shared Libraries

Move the deploy logic that every Jenkinsfile copies into one versioned Pipeline library: custom steps, classes, resources, trust, and loading it by tag or branch.

By The Dark DebuggerUpdated 26 min read

A shared library is a git repository of Groovy code that any Pipeline can load: custom steps like deployApp, helper classes, and files such as shell scripts. This article builds one from scratch, loads it three different ways, versions it with tags, and runs into the trust and CPS rules that trip up nearly everyone the first time. You should know declarative syntax from Declarative Pipeline in Depth. The template example at the end uses the multibranch setup from Multibranch Pipelines and Webhooks.

Why: the Jenkinsfile that got copied 30 times

A deploy stage usually starts as eight lines of shell in one service's Jenkinsfile. It works, so the next service copies it, and so does the one after that. A year later there are 30 slightly different copies. Some prune old releases and some fill the disk. One has a quoting bug that only shows up when a branch name contains a space. When the deploy target moves, someone opens 30 pull requests.

A shared library turns those eight lines into one step:

deployApp app: 'inventory-api', environment: 'staging', artifact: "dist/${env.RELEASE}.tar.gz"

Fix the step once, tag a release, and every pipeline that asks for the new version gets the fix. Pipelines that pinned the old version keep working until their owners move. That second half matters as much as the first. A shared library without versioning is just a faster way to break 30 builds at once.

Library layout

A library is an ordinary git repository with up to three top-level directories. Mine, lib, started like this:

tree --noreport .
.
├── README.md
├── resources
│   └── com
│       └── shellrag
│           └── ci
│               └── deploy.sh
├── src
│   └── com
│       └── shellrag
│           └── ci
│               └── Version.groovy
└── vars
    ├── deployApp.groovy
    └── deployApp.txt
DirectoryHoldsHow a Pipeline uses it
vars/One file per global variable or custom step, named in camelCasedeployApp(...), or deployApp.someMethod()
vars/*.txtHelp text for the step with the same nameShown under Pipeline Syntax > Global Variables Reference
src/Groovy classes in normal package directoriesimport com.shellrag.ci.Version
resources/Any non-Groovy file: scripts, templates, JSONlibraryResource('com/shellrag/ci/deploy.sh') returns its text

The file name in vars/ is the step name, so it has to be a valid Groovy identifier. deployApp.groovy works and deploy-app.groovy can never be called.

A custom step: vars/deployApp.groovy

A file in vars/ with a call method becomes a step. Arguments come in as a Map, which is what gives you the deployApp app: ..., environment: ... named-argument syntax:

// deployApp: unpack a release tarball into <root>/<environment>/<app>/releases/
// and point the "current" symlink at it.
def call(Map args = [:]) {
    String app = args.app ?: error('deployApp: missing required argument "app"')
    String environment = args.environment ?: error('deployApp: missing required argument "environment"')
    String artifact = args.artifact ?: error('deployApp: missing required argument "artifact"')
    String root = args.root ?: env.DEPLOY_ROOT ?: error('deployApp: set "root" or DEPLOY_ROOT')
 
    echo "deployApp: ${artifact} -> ${environment}"
    writeFile file: '.deploy/deploy.sh', text: libraryResource('com/shellrag/ci/deploy.sh')
    withEnv(["APP=${app}", "TARGET_ENV=${environment}", "ARTIFACT=${artifact}", "DEPLOY_ROOT=${root}"]) {
        sh 'bash .deploy/deploy.sh'
    }
}

Inside a vars/ file you can call any Pipeline step (echo, sh, writeFile, error) and read env, exactly like in a Jenkinsfile. Three decisions in this step are worth copying:

  • Validate arguments with error. A missing argument fails fast with a message naming the step, instead of failing three steps later inside a shell script.
  • Keep the shell in a resource. libraryResource returns the file's text and writeFile drops it into the workspace. The script is a real .sh file in the library repository, so you can run shellcheck on it and test it outside Jenkins.
  • Pass values with withEnv, not string interpolation. sh "tar -xzf ${artifact}" pastes the value into a command line. One odd file name and you have a quoting bug, or worse, an injection. Environment variables reach the script as data.

The resource itself is plain Bash:

#!/usr/bin/env bash
# Called by the deployApp step. Inputs come from the environment:
#   APP, TARGET_ENV, ARTIFACT, DEPLOY_ROOT
set -euo pipefail
base="$DEPLOY_ROOT/$TARGET_ENV/$APP"
release="$(basename "$ARTIFACT" .tar.gz)"
mkdir -p "$base/releases/$release"
tar -xzf "$ARTIFACT" -C "$base/releases/$release"
ln -sfn "releases/$release" "$base/current"
echo "deployed $release to $base"
echo "releases on disk: $(ls "$base/releases" | tr '\n' ' ')"

vars/deployApp.txt is the help text that appears in the Global Variables Reference. It may contain simple HTML:

<p>Deploys a release tarball to an environment on this agent.</p>
<pre>
deployApp app: 'inventory-api',
          environment: 'staging',
          artifact: 'dist/inventory-api-2.4.1.tar.gz',
          root: '/srv/deployments'   // optional, defaults to env.DEPLOY_ROOT
</pre>

A class in src/: com.shellrag.ci.Version

Logic that isn't a Pipeline step, like parsing and bumping a version number, belongs in a class:

package com.shellrag.ci
 
import com.cloudbees.groovy.cps.NonCPS
 
/** A semantic version such as 2.4.0, parsed from strings like "v2.4.0". */
class Version implements Serializable {
    final int major
    final int minor
    final int patch
 
    Version(int major, int minor, int patch) {
        this.major = major
        this.minor = minor
        this.patch = patch
    }
 
    @NonCPS
    static Version parse(String text) {
        def parts = text.replaceFirst(/^v/, '').tokenize('.')
        if (parts.size() != 3 || !parts.every { it.isInteger() }) {
            throw new IllegalArgumentException("not a semantic version: ${text}")
        }
        return new Version(parts[0] as int, parts[1] as int, parts[2] as int)
    }
 
    @NonCPS
    Version nextPatch() {
        return new Version(major, minor, patch + 1)
    }
 
    @NonCPS
    Version nextMinor() {
        return new Version(major, minor + 1, 0)
    }
 
    @NonCPS
    String toString() {
        return "${major}.${minor}.${patch}"
    }
}

Two details matter here, and both come from how Pipeline runs Groovy:

  • implements Serializable. Pipeline saves the running program to disk at every step so a build can survive a controller restart. Any object alive across a step has to be serializable.
  • @NonCPS on methods that don't call steps. Pipeline rewrites your code into continuation-passing style (CPS) so it can pause it. When plain Java code calls a rewritten method, the result goes missing. The most common case is Groovy calling toString() while it builds a "${version}" string. Common mistakes shows what that looks like. Mark pure helper methods @NonCPS and keep step calls out of them.

Loading a library with the library step

The quickest way to use a library needs no configuration at all. The library step fetches it from git while the build runs:

def lib = library(
    identifier: 'lib@main',
    retriever: modernSCM([$class: 'GitSCMSource',
                          remote: 'file:///home/sid/git/lib.git'])
)
 
pipeline {
    agent any
    environment {
        DEPLOY_ROOT = '/home/sid/deployments'
    }
    stages {
        stage('Package') {
            steps {
                git url: 'file:///home/sid/git/inventory-api.git', branch: 'main'
                script {
                    def lastTag = sh(returnStdout: true, script: 'git describe --tags --abbrev=0').trim()
                    def version = lib.com.shellrag.ci.Version.parse(lastTag).nextPatch()
                    env.RELEASE = "inventory-api-${version}"
                }
                sh 'mkdir -p dist && tar czf "dist/$RELEASE.tar.gz" app.py'
            }
        }
        stage('Deploy to staging') {
            steps {
                deployApp app: 'inventory-api', environment: 'staging', artifact: "dist/${env.RELEASE}.tar.gz"
            }
        }
    }
}

The identifier is name@version, and the version can be any branch, tag or commit. Because the library arrives while the script is already running, its classes can't be imported at the top of the file. You reach them through the object the step returns: lib.com.shellrag.ci.Version. Steps from vars/ are available as soon as the library step has run. The build log shows the library being fetched before anything else:

Started lib-dynamic #1
Started from command line by admin
[Pipeline] Start of Pipeline
[Pipeline] library
Loading library lib@main
Attempting to resolve main from remote references...
Found match: refs/heads/main revision ab2b96c2d4907564d1ef9fb1739dfbf35fe17cf0
[…]
Checking out Revision ab2b96c2d4907564d1ef9fb1739dfbf35fe17cf0 (main)
Commit message: "deployApp step and Version class"
[…]
[Pipeline] sh
+ git describe --tags --abbrev=0
[Pipeline] sh
+ mkdir -p dist
+ tar czf dist/inventory-api-2.4.1.tar.gz app.py
[…]
[Pipeline] { (Deploy to staging)
[Pipeline] echo
deployApp: dist/inventory-api-2.4.1.tar.gz -> staging
[Pipeline] libraryResource
[Pipeline] writeFile
[Pipeline] withEnv
[Pipeline] {
[Pipeline] sh
+ bash .deploy/deploy.sh
deployed inventory-api-2.4.1 to /home/sid/deployments/staging/inventory-api
releases on disk: inventory-api-2.4.1 
[…]
Finished: SUCCESS

The latest tag in the inventory repository is v2.4.0, so Version produced 2.4.1. deployApp wrote the resource to .deploy/deploy.sh and ran it. The library step is great for trying a library out, or for a team that can't get anything configured on the controller. For everyday use, configure the library once so Jenkinsfiles can refer to it by name.

Configuring a library by name

Libraries can be registered in three places. The difference between them is trust:

WhereUI locationWho can use itRuns in the sandbox?
Global trustedManage Jenkins > System > Global Trusted Pipeline LibrariesEvery jobNo, full access to Jenkins internals
Global untrustedManage Jenkins > System > Global Untrusted Pipeline LibrariesEvery jobYes
FolderFolder > Configure > Pipeline LibrariesJobs inside that folderYes
library stepIn the JenkinsfileThat jobYes

Each configured library has the same fields: Name, Default version, Load implicitly, Allow default version to be overridden, Include @Library changes in job recent changes, Cache fetched versions on controller for quick retrieval, and a Retrieval method. For git, the retrieval method is Modern SCM with a Git source. There's also a library path option for libraries that live in a subdirectory of a bigger repository. Load implicitly makes the default version available to every job in scope without any annotation. That's convenient, but it hides where steps come from, so I leave it off.

My lab controller is shared, so I configured the library on a folder. That's also the right choice when one team owns a library that other teams shouldn't depend on. Here is the folder's config.xml, which I created with jcli create-job inventory-team < folder.xml (jcli is the CLI wrapper from the multibranch article):

<?xml version='1.1' encoding='UTF-8'?>
<com.cloudbees.hudson.plugins.folder.Folder>
  <description>Jobs owned by the inventory team</description>
  <properties>
    <org.jenkinsci.plugins.workflow.libs.FolderLibraries>
      <libraries>
        <org.jenkinsci.plugins.workflow.libs.LibraryConfiguration>
          <name>lib</name>
          <retriever class="org.jenkinsci.plugins.workflow.libs.SCMSourceRetriever">
            <scm class="jenkins.plugins.git.GitSCMSource">
              <id>lib</id>
              <remote>file:///home/sid/git/lib.git</remote>
              <traits>
                <jenkins.plugins.git.traits.BranchDiscoveryTrait/>
                <jenkins.plugins.git.traits.TagDiscoveryTrait/>
              </traits>
            </scm>
          </retriever>
          <defaultVersion>v1.0.0</defaultVersion>
          <implicit>false</implicit>
          <allowVersionOverride>true</allowVersionOverride>
          <includeInChangesets>true</includeInChangesets>
        </org.jenkinsci.plugins.workflow.libs.LibraryConfiguration>
      </libraries>
    </org.jenkinsci.plugins.workflow.libs.FolderLibraries>
  </properties>
</com.cloudbees.hudson.plugins.folder.Folder>

The @Library annotation

Once a library has a name, a Jenkinsfile loads it with an annotation on its first line. Because annotations are processed before the script is compiled, import works:

@Library('lib@v1.0.0') import com.shellrag.ci.Version
 
pipeline {
    agent any
    environment {
        DEPLOY_ROOT = '/home/sid/deployments'
    }
    stages {
        stage('Package') {
            steps {
                git url: 'file:///home/sid/git/inventory-api.git', branch: 'main'
                script {
                    def lastTag = sh(returnStdout: true, script: 'git describe --tags --abbrev=0').trim()
                    env.RELEASE = "inventory-api-${Version.parse(lastTag).nextPatch()}-b${BUILD_NUMBER}"
                }
                sh 'mkdir -p dist && tar czf "dist/$RELEASE.tar.gz" app.py'
            }
        }
        stage('Deploy to staging') {
            steps {
                deployApp app: 'inventory-api', environment: 'staging', artifact: "dist/${env.RELEASE}.tar.gz"
            }
        }
    }
}

The forms you'll see:

AnnotationLoads
@Library('lib') _The configured default version
@Library('lib@v1.0.0') _Tag v1.0.0
@Library('lib@main') _Whatever main points at when the build starts
@Library('lib@0a7076d...') _One exact commit
@Library(['lib@v1.1.0', 'other-lib']) _Several libraries
@Library('lib@v1.0.0') import com.shellrag.ci.VersionThe library, plus an import

The underscore in @Library('x') _ is just something for the annotation to attach to. An annotation must sit on a declaration (an import, a variable), and _ is a throwaway one. If the first thing after the annotation is an import, you don't need it.

The job runs inside the inventory-team folder, and the log shows the tag being resolved:

Started inventory-team » release-inventory-api #1
Started from command line by admin
Loading library lib@v1.0.0
Attempting to resolve v1.0.0 from remote references...
Found match: refs/tags/v1.0.0 revision ab2b96c2d4907564d1ef9fb1739dfbf35fe17cf0
Resolving tag commit... (remote references may be a lightweight tag or an annotated tag)
[…]
Checking out Revision ab2b96c2d4907564d1ef9fb1739dfbf35fe17cf0 (v1.0.0)
Commit message: "deployApp step and Version class"
[…]
deployApp: dist/inventory-api-2.4.1-b1.tar.gz -> staging
[…]
deployed inventory-api-2.4.1-b1 to /home/sid/deployments/staging/inventory-api
releases on disk: inventory-api-2.4.1 inventory-api-2.4.1-b1 
Finished: SUCCESS

inventory-api-2.4.1 on disk is the release that the library-step job deployed earlier. I ran build #2 the same way, so there were three releases, and none of them would ever be cleaned up. That's the bug the next version fixes.

Versioning: releasing v1.1.0

Version 1.1.0 of the library makes deployApp keep only the newest releases. The step gains a keep argument:

    int keep = args.keep ?: 3
 
    echo "deployApp: ${artifact} -> ${environment} (keeping ${keep} releases)"
    writeFile file: '.deploy/deploy.sh', text: libraryResource('com/shellrag/ci/deploy.sh')
    withEnv(["APP=${app}", "TARGET_ENV=${environment}", "ARTIFACT=${artifact}",
             "DEPLOY_ROOT=${root}", "KEEP=${keep}"]) {
        sh 'bash .deploy/deploy.sh'
    }

deploy.sh gets a pruning loop after the symlink switch:

# prune: keep the newest $KEEP releases (by modification time)
ls -1t "$base/releases" | tail -n +"$((KEEP + 1))" | while read -r old; do
  rm -rf "${base:?}/releases/$old"
  echo "pruned $old"
done

I committed that, tagged it and pushed both:

git commit -q -m "deployApp prunes old releases; add inventoryPipeline and buildQueueLength"
git tag -a v1.1.0 -m "lib 1.1.0"
git push -q origin main v1.1.0
git log --oneline --decorate --all
c64598e (origin/demo/without-noncps, demo/without-noncps) Demo: Version without @NonCPS
0a7076d (HEAD -> main, tag: v1.1.0, origin/main) deployApp prunes old releases; add inventoryPipeline and buildQueueLength
ab2b96c (tag: v1.0.0) deployApp step and Version class

(The demo/without-noncps branch is for the CPS mistake below.) The job pinned to v1.0.0 didn't change behaviour when main moved, and that's the whole point of pinning. To opt in, its owner changes one line to @Library('lib@v1.1.0'). The next three builds then trimmed the directory down to three releases:

Loading library lib@v1.1.0
[…]
deployApp: dist/inventory-api-2.4.1-b3.tar.gz -> staging (keeping 3 releases)
[…]
deployed inventory-api-2.4.1-b3 to /home/sid/deployments/staging/inventory-api
pruned inventory-api-2.4.1
releases on disk: inventory-api-2.4.1-b1 inventory-api-2.4.1-b2 inventory-api-2.4.1-b3 
deployApp: dist/inventory-api-2.4.1-b4.tar.gz -> staging (keeping 3 releases)
[…]
pruned inventory-api-2.4.1-b1
releases on disk: inventory-api-2.4.1-b2 inventory-api-2.4.1-b3 inventory-api-2.4.1-b4 

My rules for versions:

  • Pin production pipelines to tags. @main means a commit to the library changes every pipeline's behaviour on its next build, without anyone reviewing those pipelines.
  • Use semantic versions and mean them. A new optional argument is a minor release. A renamed argument is a major one, and it needs a deprecation period.
  • Use branches for testing, as in @Library('lib@feature/x'). The exercises below use exactly that to try changes before tagging them.
  • Turn off "Allow default version to be overridden" if every job must use the version you set centrally. Then a Jenkinsfile asking for another version fails (see Common mistakes).

Trusted versus untrusted libraries

Untrusted code (Jenkinsfiles, folder libraries, the library step) runs in the Groovy sandbox. The sandbox allows Pipeline steps and an approved list of ordinary methods, and refuses anything that could reach Jenkins internals. Code from a Global Trusted library runs outside the sandbox. It can call any Java API, read credentials directly and change Jenkins configuration.

To show the difference I added a step that reads the build queue, which needs Jenkins internals:

// buildQueueLength: how many items are waiting in the build queue.
// Reads Jenkins internals, so it only works from a trusted library.
def call() {
    return jenkins.model.Jenkins.get().queue.items.length
}

Making a library trusted is a global setting, and I kept the global configuration of my main lab controller unchanged. So I ran this part on a separate throwaway controller of the same Jenkins version, with the same plugins. I registered the library as a global trusted library with Configuration as Code:

unclassified:
  globalLibraries:
    libraries:
      - name: "lib"
        defaultVersion: "v1.1.0"
        allowVersionOverride: true
        implicit: false
        includeInChangesets: true
        retriever:
          modernSCM:
            scm:
              git:
                remote: "file:///home/sid/git/lib.git"
jcli check-configuration < jcasc.yaml
jcli apply-configuration < jcasc.yaml
jcli export-configuration | sed -n '/globalLibraries/,/^[a-z]/p' | head -9
  globalLibraries:
    libraries:
    - defaultVersion: "v1.1.0"
      name: "lib"
      retriever:
        modernSCM:
          scm:
            gitSource:
              remote: "file:///home/sid/git/lib.git"

check-configuration and apply-configuration print nothing when they succeed. The export leaves out values that are defaults, and it spells the git source gitSource. Both git and gitSource were accepted. A job at the top level of that controller then used the default version:

@Library('lib') _
 
echo "Builds waiting in the queue: ${buildQueueLength()}"
Loading library lib@v1.1.0
[…]
[Pipeline] Start of Pipeline
[Pipeline] echo
Builds waiting in the queue: 0
[Pipeline] End of Pipeline
Finished: SUCCESS

The same step, called from a job inside a folder that defines its own (untrusted) lib, was refused. The folder's definition took precedence over the global one:

Loading library lib@v1.1.0
[…]
[Pipeline] Start of Pipeline
Scripts not permitted to use staticMethod jenkins.model.Jenkins get. Administrators can decide whether to approve or reject this signature.
[Pipeline] End of Pipeline
Also:   org.jenkinsci.plugins.workflow.actions.ErrorAction$ErrorId: c203fad0-316a-4b26-a37f-153ab3480b17
org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: Scripts not permitted to use staticMethod jenkins.model.Jenkins get

The consequence is easy to miss: anyone who can push to a trusted library's repository can run arbitrary code as Jenkins itself. Treat that repository like production infrastructure. Protect main, require reviews, and pin the global default to a tag. Put only the steps that genuinely need internals in a small trusted library, and keep everything else in an untrusted one that teams can contribute to freely. The approval message offers a third route, approving the signature under Manage Jenkins > In-process Script Approval. Do that sparingly. Every approval applies to every sandboxed script on the controller.

Testing a library

There are three levels, from cheapest to most realistic:

  1. Test the resources outside Jenkins. deploy.sh is a normal script. Run it with the same environment variables on your machine, and run shellcheck on it. This catches most bugs, because most library logic should live in scripts like this.
  2. Unit test the Groovy with JenkinsPipelineUnit. This is an open-source framework that loads your vars/ and src/ code in a normal JUnit test, replaces steps like sh with mocks you register, and records the calls so you can assert on them. It runs under Gradle or Maven, so it fits in the library's own CI. I didn't run it for this article, because it needs a Gradle or Maven build that pulls in Groovy and the framework, and my lab doesn't have one. It's the right tool once a library has real branching logic.
  3. Integration test on a branch. Push the change to a branch and point one real job at it with @Library('lib@feature/x'). You test the actual step, with real sandbox rules and real CPS behaviour, before anyone else sees it. All four exercises below work this way.

Level 3 catches things the first two can't, sandbox rejections and CPS problems among them. Never tag a release you haven't loaded from a branch at least once.

Where you'll use this

The biggest payoff from a library is a pipeline template: a step that contains a whole declarative pipeline. Version 1.1.0 of lib includes one for the team's Python services:

// inventoryPipeline: the standard build-test-package-deploy pipeline for our
// Python services. A service's whole Jenkinsfile becomes:
//   @Library('lib@v1.1.0') _
//   inventoryPipeline(app: 'inventory-api', deployRoot: '/srv/deployments')
def call(Map config = [:]) {
    String app = config.app ?: error('inventoryPipeline: "app" is required')
    String testCommand = config.testCommand ?: 'python3 -m unittest -v'
    String deployRoot = config.deployRoot ?: '/srv/deployments'
 
    pipeline {
        agent any
        options {
            buildDiscarder(logRotator(numToKeepStr: '20'))
            timeout(time: 15, unit: 'MINUTES')
        }
        stages {
            stage('Test') {
                steps {
                    sh testCommand
                }
            }
            stage('Package') {
                steps {
                    script {
                        def lastTag = sh(returnStdout: true, script: 'git describe --tags --abbrev=0').trim()
                        env.RELEASE = "${app}-${com.shellrag.ci.Version.parse(lastTag).nextPatch()}"
                    }
                    sh 'mkdir -p dist && tar czf "dist/$RELEASE.tar.gz" --exclude=dist --exclude=.git .'
                    archiveArtifacts artifacts: 'dist/*.tar.gz', fingerprint: true
                }
            }
            stage('Deploy to staging') {
                when { branch 'main' }
                steps {
                    deployApp app: app, environment: 'staging', root: deployRoot,
                              artifact: "dist/${env.RELEASE}.tar.gz"
                }
            }
        }
    }
}

A new service, pricing-api, then needs a three-line Jenkinsfile:

@Library('lib@v1.1.0') _
 
inventoryPipeline(app: 'pricing-api', deployRoot: '/home/sid/deployments')

I created a multibranch project for it inside the team folder, so the folder's library definition applied. Its first build of main failed in a way that's worth knowing about. The fix is in Common mistakes. After the fix, the build ran the whole template:

Started inventory-team » pricing-api » main #2
[…]
Loading library lib@v1.1.0
[…]
[Pipeline] { (Test)
[Pipeline] sh
+ python3 -m unittest -v
test_rounding (test_pricing.PriceWithTaxTest.test_rounding) ... ok
test_standard_rate (test_pricing.PriceWithTaxTest.test_standard_rate) ... ok
 
----------------------------------------------------------------------
Ran 2 tests in 0.000s
 
OK
[…]
+ git describe --tags --abbrev=0
[Pipeline] sh
+ mkdir -p dist
+ tar czf dist/pricing-api-0.9.1.tar.gz --exclude=dist --exclude=.git .
[Pipeline] archiveArtifacts
Archiving artifacts
Recording fingerprints
[…]
deployApp: dist/pricing-api-0.9.1.tar.gz -> staging (keeping 3 releases)
[…]
deployed pricing-api-0.9.1 to /home/sid/deployments/staging/pricing-api
releases on disk: pricing-api-0.9.1 
Finished: SUCCESS

This is how platform teams in larger companies run Jenkins. Every service repository has a Jenkinsfile of a few lines. The pipeline itself (stages, timeouts, retention, test and deploy conventions) lives in one reviewed, tagged library. Upgrading 30 services to a new deploy method is one library release plus 30 one-line version bumps, and those can be rolled out a few at a time. Keep an escape hatch, though. A template that grows a parameter for every special case becomes harder to read than the 30 Jenkinsfiles it replaced. Services that really are different should write their own Jenkinsfile and still use the library's individual steps.

Common mistakes

Forgetting the underscore. An annotation followed directly by pipeline doesn't compile:

@Library('lib@v1.0.0')
 
pipeline {
Started inventory-team » no-underscore #1
Started from command line by admin
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
WorkflowScript: 3: unexpected token: pipeline @ line 3, column 1.
   pipeline {
   ^
 
1 error

The error points at pipeline, not at the annotation, which is why it's confusing. Write @Library('lib@v1.0.0') _.

Importing from a library loaded with the library step. Imports are resolved when the script is compiled, before the step has run:

import com.shellrag.ci.Version
 
library identifier: 'lib@main', retriever: modernSCM([$class: 'GitSCMSource',
        remote: 'file:///home/sid/git/lib.git'])
WorkflowScript: 1: unable to resolve class com.shellrag.ci.Version
 @ line 1, column 1.
   import com.shellrag.ci.Version
   ^

Either use the returned object (lib.com.shellrag.ci.Version) or configure the library and use @Library.

A missing @NonCPS: values that silently vanish. I pushed a branch of the library with the @NonCPS annotations removed and loaded it:

def lib = library(identifier: 'lib@demo/without-noncps', retriever: modernSCM([$class: 'GitSCMSource',
        remote: 'file:///home/sid/git/lib.git']))
 
def version = lib.com.shellrag.ci.Version.parse('v2.4.0').nextPatch()
echo "Next release: ${version}"
echo "done"
Checking out Revision c64598ee49867bde6aa8c2315f2ed497e364f9e9 (demo/without-noncps)
Commit message: "Demo: Version without @NonCPS"
First time build. Skipping changelog.
expected to call WorkflowScript.echo but wound up catching com.shellrag.ci.Version.toString; see: https://jenkins.io/redirect/pipeline-cps-method-mismatches/
[Pipeline] echo
done
[Pipeline] End of Pipeline
Finished: SUCCESS

The first echo never happened, and the build is still green. When I first hit this, an env.RELEASE = "inventory-api-${version}" assignment silently left RELEASE empty, so the package was named .tar.gz. There wasn't even a warning that time. If you see "expected to call … but wound up catching", look for a CPS-transformed method being called from Groovy's own machinery (toString, compareTo, equals, closures passed to sort) and mark it @NonCPS.

"Version override not permitted". With Allow default version to be overridden turned off on the folder, the job that asks for @v1.1.0 failed before running anything:

Started inventory-team » release-inventory-api #6
Started from command line by admin
ERROR: Version override not permitted for library lib
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
WorkflowScript: Loading libraries failed
 
1 error

That's the setting doing its job. Either drop the version from the annotation, or ask the library's owners to change the default.

git describe finds no tags in a multibranch build. The first pricing-api build failed at Package:

+ git describe --tags --abbrev=0
fatal: No names found, cannot describe anything.
[…]
Stage "Deploy to staging" skipped due to earlier failure(s)
[…]
ERROR: script returned exit code 128

The checkout log explains it: Cloning with configured refspecs honoured and without tags. Multibranch branch builds don't fetch tags by default, and adding Discover tags didn't change that when I tried it. The fix is the Advanced clone behaviours behaviour on the branch source with Fetch tags ticked. In XML, that's this trait:

<jenkins.plugins.git.traits.CloneOptionTrait>
  <extension class="hudson.plugins.git.extensions.impl.CloneOption">
    <shallow>false</shallow>
    <noTags>false</noTags>
    <depth>0</depth>
    <honorRefspec>false</honorRefspec>
  </extension>
</jenkins.plugins.git.traits.CloneOptionTrait>

Summary

  • A shared library is a git repository with vars/ (steps: a file with call() per step), src/ (classes) and resources/ (files read with libraryResource).
  • Load it with @Library('name@version') _ once it's configured globally or on a folder, or with the library step without any configuration. Classes from the library step are reached through its return value, not import.
  • Pin production pipelines to tags, test changes from branches, and treat the library's version number as a contract.
  • Global trusted libraries run outside the sandbox. Folder, untrusted and library-step libraries are sandboxed. A folder's definition wins over a global one with the same name.
  • Classes must be Serializable, and helper methods that don't call steps should be @NonCPS. Otherwise values can disappear without an error.
  • In steps, pass data to shell scripts through withEnv, not string interpolation, and validate arguments with error.
  • A pipeline template (def call(Map config) { pipeline { ... } }) cuts a service's Jenkinsfile down to a few lines.

Exercises

All four exercises follow the branch-testing workflow. Push your change to a branch of the library and load that branch from a job inside a folder that has the library configured.

Exercise 1: A releaseName stepeasy

Add a step releaseName(app) that returns "<app>-<next patch version>", based on the latest tag in the workspace and using the Version class. Load it from a branch and print the result for the inventory repository.

Show Solution

vars/releaseName.groovy:

// releaseName: "<app>-<next patch version>" based on the latest tag in the workspace.
def call(String app) {
    def lastTag = sh(returnStdout: true, script: 'git describe --tags --abbrev=0').trim()
    return "${app}-${com.shellrag.ci.Version.parse(lastTag).nextPatch()}"
}

Pushed to feature/release-name and used from a job in the folder:

@Library('lib@feature/release-name') _
 
node {
    git url: 'file:///home/sid/git/inventory-api.git', branch: 'main'
    echo "Release name: ${releaseName('inventory-api')}"
}
Loading library lib@feature/release-name
Attempting to resolve feature/release-name from remote references...
Found match: refs/heads/feature/release-name revision 4cfb9a6358d225bd82db24ad508f708e2a39a17e
[…]
+ git describe --tags --abbrev=0
[Pipeline] echo
Release name: inventory-api-2.4.1
[Pipeline] End of Pipeline
Finished: SUCCESS

A step can return a value like any function. Classes from src/ are visible to vars/ by their full name, without an import.

Exercise 2: Guard production deploysmedium

Change deployApp so it refuses to deploy to production unless the caller passes approved: true. Test it from a job with a boolean parameter APPROVED, once with each value.

Show Solution

The change, as git diff main showed it:

@@ -6,6 +6,9 @@ def call(Map args = [:]) {
     String artifact = args.artifact ?: error('deployApp: missing required argument "artifact"')
     String root = args.root ?: env.DEPLOY_ROOT ?: error('deployApp: set "root" or DEPLOY_ROOT')
     int keep = args.keep ?: 3
+    if (environment == 'production' && args.approved != true) {
+        error("deployApp: refusing to deploy ${app} to production without approved: true")
+    }
 
     echo "deployApp: ${artifact} -> ${environment} (keeping ${keep} releases)"
     writeFile file: '.deploy/deploy.sh', text: libraryResource('com/shellrag/ci/deploy.sh')

The test job:

@Library('lib@feature/guard-production') _
 
node {
    sh 'mkdir -p dist && echo "print(1)" > app.py && tar czf dist/inventory-api-2.4.1.tar.gz app.py'
    withEnv(['DEPLOY_ROOT=/home/sid/deployments']) {
        deployApp app: 'inventory-api', environment: 'production', artifact: 'dist/inventory-api-2.4.1.tar.gz', approved: params.APPROVED
    }
}

With jcli build inventory-team/ex2-guard -p APPROVED=false -s -v:

[Pipeline] withEnv
[Pipeline] error
[Pipeline] End of Pipeline
ERROR: deployApp: refusing to deploy inventory-api to production without approved: true
Finished: FAILURE

With APPROVED=true:

deployApp: dist/inventory-api-2.4.1.tar.gz -> production (keeping 3 releases)
[…]
deployed inventory-api-2.4.1 to /home/sid/deployments/production/inventory-api
releases on disk: inventory-api-2.4.1 
Finished: SUCCESS

I compared with != true rather than testing for truthiness, so that the string 'false' or a typo'd argument name can't count as approval.

Exercise 3: Pin to a commitmedium

Load the library at the exact commit that v1.1.0 points to, not at the tag, and use Version to print the next minor version after v2.4.0. When would you pin to a commit rather than a tag?

Show Solution

git rev-parse v1.1.0^{commit} printed 0a7076d714aa315c5683302c71b62f92b5d5fefd:

@Library('lib@0a7076d714aa315c5683302c71b62f92b5d5fefd') import com.shellrag.ci.Version
 
echo "Next minor after v2.4.0: ${Version.parse('v2.4.0').nextMinor()}"
Loading library lib@0a7076d714aa315c5683302c71b62f92b5d5fefd
Attempting to resolve 0a7076d714aa315c5683302c71b62f92b5d5fefd from remote references...
Found match: refs/heads/main revision 0a7076d714aa315c5683302c71b62f92b5d5fefd
[…]
[Pipeline] echo
Next minor after v2.4.0: 2.5.0
[Pipeline] End of Pipeline
Finished: SUCCESS

Tags can be moved or deleted by anyone with push rights, but a commit can't change. Pin to a commit when you need a guarantee, for example in a regulated release pipeline or while you bisect which library change broke a build. For everyday pinning, tags are easier to read.

Exercise 4: A class that calls stepshard

Write com.shellrag.ci.ReleaseNotes, a class that writes RELEASE_NOTES.md listing the commits since the previous tag. A class in src/ can't call sh or writeFile on its own. Work out how to give it access to them.

Hint

Pass the script object (this in a Jenkinsfile) to the constructor and call steps on it.

Show Solution
package com.shellrag.ci
 
/** Writes RELEASE_NOTES.md from the commits since the previous tag. */
class ReleaseNotes implements Serializable {
    private final def steps
 
    ReleaseNotes(steps) {
        this.steps = steps
    }
 
    String write(String release) {
        String log = steps.sh(returnStdout: true,
            script: 'git log --pretty="- %s (%h)" $(git describe --tags --abbrev=0)..HEAD').trim()
        String text = "# ${release}\n\n${log ?: '- no changes'}\n"
        steps.writeFile file: 'RELEASE_NOTES.md', text: text
        steps.echo "wrote RELEASE_NOTES.md (${log.readLines().size()} changes)"
        return text
    }
}
@Library('lib@feature/release-notes') import com.shellrag.ci.ReleaseNotes
 
node {
    git url: 'file:///home/sid/git/inventory-api.git', branch: 'main'
    def notes = new ReleaseNotes(this).write('inventory-api-2.4.1')
    echo notes
}
[Pipeline] sh
+ git describe --tags --abbrev=0
+ git log --pretty=- %s (%h) v2.4.0..HEAD
[Pipeline] writeFile
[Pipeline] echo
wrote RELEASE_NOTES.md (1 changes)
[Pipeline] echo
# inventory-api-2.4.1
 
- Package hotfix branches too (29db54b)
 
[Pipeline] End of Pipeline
Finished: SUCCESS

write calls steps, so it must not be @NonCPS. The steps field holds the script object, which Pipeline knows how to serialize. That's why the class can safely be Serializable and keep it in a field across the sh and writeFile steps.


Tested with: Jenkins 2.580.2 (OpenJDK 21) on Ubuntu 24.04 (WSL2), Pipeline: Groovy 4403.v471713a_cea_19, Pipeline: Groovy Libraries 822.vc95f70dea_db_7, Git plugin 5.11.0 and Configuration as Code 2147.v8a_8d38668c72, November 2026. The library, its two tags, the folder configuration, the library step, every @Library job, the pipeline template and all four exercise solutions ran on the main lab controller. The trusted-versus-untrusted section (the JCasC global library, buildQueueLength succeeding at the top level and being rejected inside the folder) ran on a separate throwaway controller of the same version with the same plugins, because global configuration and script approvals were off-limits on the shared one. JenkinsPipelineUnit was described but not run. Logs are trimmed where marked […]. Paths from the test machine's scratch directories are shown shortened to /home/sid/…. Job, node and credential names are shown without the test run's prefixes.

Was this page helpful?
The Dark Debugger

Written by The Dark Debugger

Engineer documenting the things that took me too long to learn, so they take you less time. Every Jenkins tutorial here is tested end to end before publishing.