Shared Libraries
Move the deploy logic that every Jenkinsfile copies into one versioned Pipeline library: custom steps, classes, resources, trust, and loading it by tag or branch.
By The Dark DebuggerUpdated 26 min read
A shared library is a git repository of Groovy code that any Pipeline can load: custom
steps like deployApp, helper classes, and files such as shell scripts. This article builds
one from scratch, loads it three different ways, versions it with tags, and runs into the
trust and CPS rules that trip up nearly everyone the first time. You should know
declarative syntax from Declarative Pipeline in Depth.
The template example at the end uses the multibranch setup from
Multibranch Pipelines and Webhooks.
Why: the Jenkinsfile that got copied 30 times
A deploy stage usually starts as eight lines of shell in one service's Jenkinsfile. It works, so the next service copies it, and so does the one after that. A year later there are 30 slightly different copies. Some prune old releases and some fill the disk. One has a quoting bug that only shows up when a branch name contains a space. When the deploy target moves, someone opens 30 pull requests.
A shared library turns those eight lines into one step:
deployApp app: 'inventory-api', environment: 'staging', artifact: "dist/${env.RELEASE}.tar.gz"Fix the step once, tag a release, and every pipeline that asks for the new version gets the fix. Pipelines that pinned the old version keep working until their owners move. That second half matters as much as the first. A shared library without versioning is just a faster way to break 30 builds at once.
Library layout
A library is an ordinary git repository with up to three top-level directories. Mine,
lib, started like this:
tree --noreport ..
├── README.md
├── resources
│ └── com
│ └── shellrag
│ └── ci
│ └── deploy.sh
├── src
│ └── com
│ └── shellrag
│ └── ci
│ └── Version.groovy
└── vars
├── deployApp.groovy
└── deployApp.txt| Directory | Holds | How a Pipeline uses it |
|---|---|---|
vars/ | One file per global variable or custom step, named in camelCase | deployApp(...), or deployApp.someMethod() |
vars/*.txt | Help text for the step with the same name | Shown under Pipeline Syntax > Global Variables Reference |
src/ | Groovy classes in normal package directories | import com.shellrag.ci.Version |
resources/ | Any non-Groovy file: scripts, templates, JSON | libraryResource('com/shellrag/ci/deploy.sh') returns its text |
The file name in vars/ is the step name, so it has to be a valid Groovy identifier.
deployApp.groovy works and deploy-app.groovy can never be called.
A custom step: vars/deployApp.groovy
A file in vars/ with a call method becomes a step. Arguments come in as a Map, which
is what gives you the deployApp app: ..., environment: ... named-argument syntax:
// deployApp: unpack a release tarball into <root>/<environment>/<app>/releases/
// and point the "current" symlink at it.
def call(Map args = [:]) {
String app = args.app ?: error('deployApp: missing required argument "app"')
String environment = args.environment ?: error('deployApp: missing required argument "environment"')
String artifact = args.artifact ?: error('deployApp: missing required argument "artifact"')
String root = args.root ?: env.DEPLOY_ROOT ?: error('deployApp: set "root" or DEPLOY_ROOT')
echo "deployApp: ${artifact} -> ${environment}"
writeFile file: '.deploy/deploy.sh', text: libraryResource('com/shellrag/ci/deploy.sh')
withEnv(["APP=${app}", "TARGET_ENV=${environment}", "ARTIFACT=${artifact}", "DEPLOY_ROOT=${root}"]) {
sh 'bash .deploy/deploy.sh'
}
}Inside a vars/ file you can call any Pipeline step (echo, sh, writeFile, error)
and read env, exactly like in a Jenkinsfile. Three decisions in this step are worth
copying:
- Validate arguments with
error. A missing argument fails fast with a message naming the step, instead of failing three steps later inside a shell script. - Keep the shell in a resource.
libraryResourcereturns the file's text andwriteFiledrops it into the workspace. The script is a real.shfile in the library repository, so you can runshellcheckon it and test it outside Jenkins. - Pass values with
withEnv, not string interpolation.sh "tar -xzf ${artifact}"pastes the value into a command line. One odd file name and you have a quoting bug, or worse, an injection. Environment variables reach the script as data.
The resource itself is plain Bash:
#!/usr/bin/env bash
# Called by the deployApp step. Inputs come from the environment:
# APP, TARGET_ENV, ARTIFACT, DEPLOY_ROOT
set -euo pipefail
base="$DEPLOY_ROOT/$TARGET_ENV/$APP"
release="$(basename "$ARTIFACT" .tar.gz)"
mkdir -p "$base/releases/$release"
tar -xzf "$ARTIFACT" -C "$base/releases/$release"
ln -sfn "releases/$release" "$base/current"
echo "deployed $release to $base"
echo "releases on disk: $(ls "$base/releases" | tr '\n' ' ')"vars/deployApp.txt is the help text that appears in the Global Variables Reference. It
may contain simple HTML:
<p>Deploys a release tarball to an environment on this agent.</p>
<pre>
deployApp app: 'inventory-api',
environment: 'staging',
artifact: 'dist/inventory-api-2.4.1.tar.gz',
root: '/srv/deployments' // optional, defaults to env.DEPLOY_ROOT
</pre>A class in src/: com.shellrag.ci.Version
Logic that isn't a Pipeline step, like parsing and bumping a version number, belongs in a class:
package com.shellrag.ci
import com.cloudbees.groovy.cps.NonCPS
/** A semantic version such as 2.4.0, parsed from strings like "v2.4.0". */
class Version implements Serializable {
final int major
final int minor
final int patch
Version(int major, int minor, int patch) {
this.major = major
this.minor = minor
this.patch = patch
}
@NonCPS
static Version parse(String text) {
def parts = text.replaceFirst(/^v/, '').tokenize('.')
if (parts.size() != 3 || !parts.every { it.isInteger() }) {
throw new IllegalArgumentException("not a semantic version: ${text}")
}
return new Version(parts[0] as int, parts[1] as int, parts[2] as int)
}
@NonCPS
Version nextPatch() {
return new Version(major, minor, patch + 1)
}
@NonCPS
Version nextMinor() {
return new Version(major, minor + 1, 0)
}
@NonCPS
String toString() {
return "${major}.${minor}.${patch}"
}
}Two details matter here, and both come from how Pipeline runs Groovy:
implements Serializable. Pipeline saves the running program to disk at every step so a build can survive a controller restart. Any object alive across a step has to be serializable.@NonCPSon methods that don't call steps. Pipeline rewrites your code into continuation-passing style (CPS) so it can pause it. When plain Java code calls a rewritten method, the result goes missing. The most common case is Groovy callingtoString()while it builds a"${version}"string. Common mistakes shows what that looks like. Mark pure helper methods@NonCPSand keep step calls out of them.
Loading a library with the library step
The quickest way to use a library needs no configuration at all. The library step
fetches it from git while the build runs:
def lib = library(
identifier: 'lib@main',
retriever: modernSCM([$class: 'GitSCMSource',
remote: 'file:///home/sid/git/lib.git'])
)
pipeline {
agent any
environment {
DEPLOY_ROOT = '/home/sid/deployments'
}
stages {
stage('Package') {
steps {
git url: 'file:///home/sid/git/inventory-api.git', branch: 'main'
script {
def lastTag = sh(returnStdout: true, script: 'git describe --tags --abbrev=0').trim()
def version = lib.com.shellrag.ci.Version.parse(lastTag).nextPatch()
env.RELEASE = "inventory-api-${version}"
}
sh 'mkdir -p dist && tar czf "dist/$RELEASE.tar.gz" app.py'
}
}
stage('Deploy to staging') {
steps {
deployApp app: 'inventory-api', environment: 'staging', artifact: "dist/${env.RELEASE}.tar.gz"
}
}
}
}The identifier is name@version, and the version can be any branch, tag or commit.
Because the library arrives while the script is already running, its classes can't be
imported at the top of the file. You reach them through the object the step returns:
lib.com.shellrag.ci.Version. Steps from vars/ are available as soon as the library
step has run. The build log shows the library being fetched before anything else:
Started lib-dynamic #1
Started from command line by admin
[Pipeline] Start of Pipeline
[Pipeline] library
Loading library lib@main
Attempting to resolve main from remote references...
Found match: refs/heads/main revision ab2b96c2d4907564d1ef9fb1739dfbf35fe17cf0
[…]
Checking out Revision ab2b96c2d4907564d1ef9fb1739dfbf35fe17cf0 (main)
Commit message: "deployApp step and Version class"
[…]
[Pipeline] sh
+ git describe --tags --abbrev=0
[Pipeline] sh
+ mkdir -p dist
+ tar czf dist/inventory-api-2.4.1.tar.gz app.py
[…]
[Pipeline] { (Deploy to staging)
[Pipeline] echo
deployApp: dist/inventory-api-2.4.1.tar.gz -> staging
[Pipeline] libraryResource
[Pipeline] writeFile
[Pipeline] withEnv
[Pipeline] {
[Pipeline] sh
+ bash .deploy/deploy.sh
deployed inventory-api-2.4.1 to /home/sid/deployments/staging/inventory-api
releases on disk: inventory-api-2.4.1
[…]
Finished: SUCCESSThe latest tag in the inventory repository is v2.4.0, so Version produced 2.4.1.
deployApp wrote the resource to .deploy/deploy.sh and ran it. The library step is
great for trying a library out, or for a team that can't get anything configured on the
controller. For everyday use, configure the library once so Jenkinsfiles can refer to it by
name.
Configuring a library by name
Libraries can be registered in three places. The difference between them is trust:
| Where | UI location | Who can use it | Runs in the sandbox? |
|---|---|---|---|
| Global trusted | Manage Jenkins > System > Global Trusted Pipeline Libraries | Every job | No, full access to Jenkins internals |
| Global untrusted | Manage Jenkins > System > Global Untrusted Pipeline Libraries | Every job | Yes |
| Folder | Folder > Configure > Pipeline Libraries | Jobs inside that folder | Yes |
library step | In the Jenkinsfile | That job | Yes |
Each configured library has the same fields: Name, Default version, Load implicitly, Allow default version to be overridden, Include @Library changes in job recent changes, Cache fetched versions on controller for quick retrieval, and a Retrieval method. For git, the retrieval method is Modern SCM with a Git source. There's also a library path option for libraries that live in a subdirectory of a bigger repository. Load implicitly makes the default version available to every job in scope without any annotation. That's convenient, but it hides where steps come from, so I leave it off.
My lab controller is shared, so I configured the library on a folder. That's also the
right choice when one team owns a library that other teams shouldn't depend on. Here is the
folder's config.xml, which I created with jcli create-job inventory-team < folder.xml (jcli is the CLI wrapper from the
multibranch article):
<?xml version='1.1' encoding='UTF-8'?>
<com.cloudbees.hudson.plugins.folder.Folder>
<description>Jobs owned by the inventory team</description>
<properties>
<org.jenkinsci.plugins.workflow.libs.FolderLibraries>
<libraries>
<org.jenkinsci.plugins.workflow.libs.LibraryConfiguration>
<name>lib</name>
<retriever class="org.jenkinsci.plugins.workflow.libs.SCMSourceRetriever">
<scm class="jenkins.plugins.git.GitSCMSource">
<id>lib</id>
<remote>file:///home/sid/git/lib.git</remote>
<traits>
<jenkins.plugins.git.traits.BranchDiscoveryTrait/>
<jenkins.plugins.git.traits.TagDiscoveryTrait/>
</traits>
</scm>
</retriever>
<defaultVersion>v1.0.0</defaultVersion>
<implicit>false</implicit>
<allowVersionOverride>true</allowVersionOverride>
<includeInChangesets>true</includeInChangesets>
</org.jenkinsci.plugins.workflow.libs.LibraryConfiguration>
</libraries>
</org.jenkinsci.plugins.workflow.libs.FolderLibraries>
</properties>
</com.cloudbees.hudson.plugins.folder.Folder>The @Library annotation
Once a library has a name, a Jenkinsfile loads it with an annotation on its first line.
Because annotations are processed before the script is compiled, import works:
@Library('lib@v1.0.0') import com.shellrag.ci.Version
pipeline {
agent any
environment {
DEPLOY_ROOT = '/home/sid/deployments'
}
stages {
stage('Package') {
steps {
git url: 'file:///home/sid/git/inventory-api.git', branch: 'main'
script {
def lastTag = sh(returnStdout: true, script: 'git describe --tags --abbrev=0').trim()
env.RELEASE = "inventory-api-${Version.parse(lastTag).nextPatch()}-b${BUILD_NUMBER}"
}
sh 'mkdir -p dist && tar czf "dist/$RELEASE.tar.gz" app.py'
}
}
stage('Deploy to staging') {
steps {
deployApp app: 'inventory-api', environment: 'staging', artifact: "dist/${env.RELEASE}.tar.gz"
}
}
}
}The forms you'll see:
| Annotation | Loads |
|---|---|
@Library('lib') _ | The configured default version |
@Library('lib@v1.0.0') _ | Tag v1.0.0 |
@Library('lib@main') _ | Whatever main points at when the build starts |
@Library('lib@0a7076d...') _ | One exact commit |
@Library(['lib@v1.1.0', 'other-lib']) _ | Several libraries |
@Library('lib@v1.0.0') import com.shellrag.ci.Version | The library, plus an import |
The underscore in @Library('x') _ is just something for the annotation to attach to. An
annotation must sit on a declaration (an import, a variable), and _ is a throwaway one.
If the first thing after the annotation is an import, you don't need it.
The job runs inside the inventory-team folder, and the log shows the tag being
resolved:
Started inventory-team » release-inventory-api #1
Started from command line by admin
Loading library lib@v1.0.0
Attempting to resolve v1.0.0 from remote references...
Found match: refs/tags/v1.0.0 revision ab2b96c2d4907564d1ef9fb1739dfbf35fe17cf0
Resolving tag commit... (remote references may be a lightweight tag or an annotated tag)
[…]
Checking out Revision ab2b96c2d4907564d1ef9fb1739dfbf35fe17cf0 (v1.0.0)
Commit message: "deployApp step and Version class"
[…]
deployApp: dist/inventory-api-2.4.1-b1.tar.gz -> staging
[…]
deployed inventory-api-2.4.1-b1 to /home/sid/deployments/staging/inventory-api
releases on disk: inventory-api-2.4.1 inventory-api-2.4.1-b1
Finished: SUCCESSinventory-api-2.4.1 on disk is the release that the library-step job deployed earlier.
I ran build #2 the same way, so there were three releases, and none of them would ever be
cleaned up. That's the bug the next version fixes.
Versioning: releasing v1.1.0
Version 1.1.0 of the library makes deployApp keep only the newest releases. The step
gains a keep argument:
int keep = args.keep ?: 3
echo "deployApp: ${artifact} -> ${environment} (keeping ${keep} releases)"
writeFile file: '.deploy/deploy.sh', text: libraryResource('com/shellrag/ci/deploy.sh')
withEnv(["APP=${app}", "TARGET_ENV=${environment}", "ARTIFACT=${artifact}",
"DEPLOY_ROOT=${root}", "KEEP=${keep}"]) {
sh 'bash .deploy/deploy.sh'
}deploy.sh gets a pruning loop after the symlink switch:
# prune: keep the newest $KEEP releases (by modification time)
ls -1t "$base/releases" | tail -n +"$((KEEP + 1))" | while read -r old; do
rm -rf "${base:?}/releases/$old"
echo "pruned $old"
doneI committed that, tagged it and pushed both:
git commit -q -m "deployApp prunes old releases; add inventoryPipeline and buildQueueLength"
git tag -a v1.1.0 -m "lib 1.1.0"
git push -q origin main v1.1.0
git log --oneline --decorate --allc64598e (origin/demo/without-noncps, demo/without-noncps) Demo: Version without @NonCPS
0a7076d (HEAD -> main, tag: v1.1.0, origin/main) deployApp prunes old releases; add inventoryPipeline and buildQueueLength
ab2b96c (tag: v1.0.0) deployApp step and Version class(The demo/without-noncps branch is for the CPS mistake below.) The job pinned to
v1.0.0 didn't change behaviour when main moved, and that's the whole point of pinning.
To opt in, its owner changes one line to @Library('lib@v1.1.0'). The next three
builds then trimmed the directory down to three releases:
Loading library lib@v1.1.0
[…]
deployApp: dist/inventory-api-2.4.1-b3.tar.gz -> staging (keeping 3 releases)
[…]
deployed inventory-api-2.4.1-b3 to /home/sid/deployments/staging/inventory-api
pruned inventory-api-2.4.1
releases on disk: inventory-api-2.4.1-b1 inventory-api-2.4.1-b2 inventory-api-2.4.1-b3 deployApp: dist/inventory-api-2.4.1-b4.tar.gz -> staging (keeping 3 releases)
[…]
pruned inventory-api-2.4.1-b1
releases on disk: inventory-api-2.4.1-b2 inventory-api-2.4.1-b3 inventory-api-2.4.1-b4 My rules for versions:
- Pin production pipelines to tags.
@mainmeans a commit to the library changes every pipeline's behaviour on its next build, without anyone reviewing those pipelines. - Use semantic versions and mean them. A new optional argument is a minor release. A renamed argument is a major one, and it needs a deprecation period.
- Use branches for testing, as in
@Library('lib@feature/x'). The exercises below use exactly that to try changes before tagging them. - Turn off "Allow default version to be overridden" if every job must use the version you set centrally. Then a Jenkinsfile asking for another version fails (see Common mistakes).
Trusted versus untrusted libraries
Untrusted code (Jenkinsfiles, folder libraries, the library step) runs in the Groovy
sandbox. The sandbox allows Pipeline steps and an approved list of ordinary methods, and
refuses anything that could reach Jenkins internals. Code from a Global Trusted library
runs outside the sandbox. It can call any Java API, read credentials directly and change
Jenkins configuration.
To show the difference I added a step that reads the build queue, which needs Jenkins internals:
// buildQueueLength: how many items are waiting in the build queue.
// Reads Jenkins internals, so it only works from a trusted library.
def call() {
return jenkins.model.Jenkins.get().queue.items.length
}Making a library trusted is a global setting, and I kept the global configuration of my main lab controller unchanged. So I ran this part on a separate throwaway controller of the same Jenkins version, with the same plugins. I registered the library as a global trusted library with Configuration as Code:
unclassified:
globalLibraries:
libraries:
- name: "lib"
defaultVersion: "v1.1.0"
allowVersionOverride: true
implicit: false
includeInChangesets: true
retriever:
modernSCM:
scm:
git:
remote: "file:///home/sid/git/lib.git"jcli check-configuration < jcasc.yaml
jcli apply-configuration < jcasc.yaml
jcli export-configuration | sed -n '/globalLibraries/,/^[a-z]/p' | head -9 globalLibraries:
libraries:
- defaultVersion: "v1.1.0"
name: "lib"
retriever:
modernSCM:
scm:
gitSource:
remote: "file:///home/sid/git/lib.git"check-configuration and apply-configuration print nothing when they succeed. The export
leaves out values that are defaults, and it spells the git source gitSource. Both git
and gitSource were accepted. A job at the top level of that controller then used the
default version:
@Library('lib') _
echo "Builds waiting in the queue: ${buildQueueLength()}"Loading library lib@v1.1.0
[…]
[Pipeline] Start of Pipeline
[Pipeline] echo
Builds waiting in the queue: 0
[Pipeline] End of Pipeline
Finished: SUCCESSThe same step, called from a job inside a folder that defines its own (untrusted)
lib, was refused. The folder's definition took precedence over the global one:
Loading library lib@v1.1.0
[…]
[Pipeline] Start of Pipeline
Scripts not permitted to use staticMethod jenkins.model.Jenkins get. Administrators can decide whether to approve or reject this signature.
[Pipeline] End of Pipeline
Also: org.jenkinsci.plugins.workflow.actions.ErrorAction$ErrorId: c203fad0-316a-4b26-a37f-153ab3480b17
org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: Scripts not permitted to use staticMethod jenkins.model.Jenkins getThe consequence is easy to miss: anyone who can push to a trusted library's repository
can run arbitrary code as Jenkins itself. Treat that repository like production
infrastructure. Protect main, require reviews, and pin the global default to a tag.
Put only the steps that genuinely need internals in a small trusted library, and keep
everything else in an untrusted one that teams can contribute to freely. The approval
message offers a third route, approving the signature under Manage Jenkins > In-process
Script Approval. Do that sparingly. Every approval applies to every sandboxed script on
the controller.
Testing a library
There are three levels, from cheapest to most realistic:
- Test the resources outside Jenkins.
deploy.shis a normal script. Run it with the same environment variables on your machine, and runshellcheckon it. This catches most bugs, because most library logic should live in scripts like this. - Unit test the Groovy with JenkinsPipelineUnit. This is an open-source framework that
loads your
vars/andsrc/code in a normal JUnit test, replaces steps likeshwith mocks you register, and records the calls so you can assert on them. It runs under Gradle or Maven, so it fits in the library's own CI. I didn't run it for this article, because it needs a Gradle or Maven build that pulls in Groovy and the framework, and my lab doesn't have one. It's the right tool once a library has real branching logic. - Integration test on a branch. Push the change to a branch and point one real job at
it with
@Library('lib@feature/x'). You test the actual step, with real sandbox rules and real CPS behaviour, before anyone else sees it. All four exercises below work this way.
Level 3 catches things the first two can't, sandbox rejections and CPS problems among them. Never tag a release you haven't loaded from a branch at least once.
Where you'll use this
The biggest payoff from a library is a pipeline template: a step that contains a whole
declarative pipeline. Version 1.1.0 of lib includes one for the team's Python
services:
// inventoryPipeline: the standard build-test-package-deploy pipeline for our
// Python services. A service's whole Jenkinsfile becomes:
// @Library('lib@v1.1.0') _
// inventoryPipeline(app: 'inventory-api', deployRoot: '/srv/deployments')
def call(Map config = [:]) {
String app = config.app ?: error('inventoryPipeline: "app" is required')
String testCommand = config.testCommand ?: 'python3 -m unittest -v'
String deployRoot = config.deployRoot ?: '/srv/deployments'
pipeline {
agent any
options {
buildDiscarder(logRotator(numToKeepStr: '20'))
timeout(time: 15, unit: 'MINUTES')
}
stages {
stage('Test') {
steps {
sh testCommand
}
}
stage('Package') {
steps {
script {
def lastTag = sh(returnStdout: true, script: 'git describe --tags --abbrev=0').trim()
env.RELEASE = "${app}-${com.shellrag.ci.Version.parse(lastTag).nextPatch()}"
}
sh 'mkdir -p dist && tar czf "dist/$RELEASE.tar.gz" --exclude=dist --exclude=.git .'
archiveArtifacts artifacts: 'dist/*.tar.gz', fingerprint: true
}
}
stage('Deploy to staging') {
when { branch 'main' }
steps {
deployApp app: app, environment: 'staging', root: deployRoot,
artifact: "dist/${env.RELEASE}.tar.gz"
}
}
}
}
}A new service, pricing-api, then needs a three-line Jenkinsfile:
@Library('lib@v1.1.0') _
inventoryPipeline(app: 'pricing-api', deployRoot: '/home/sid/deployments')I created a multibranch project for it inside the team folder, so the folder's library
definition applied. Its first build of main failed in a way that's worth knowing about.
The fix is in Common mistakes. After the fix, the build ran the whole
template:
Started inventory-team » pricing-api » main #2
[…]
Loading library lib@v1.1.0
[…]
[Pipeline] { (Test)
[Pipeline] sh
+ python3 -m unittest -v
test_rounding (test_pricing.PriceWithTaxTest.test_rounding) ... ok
test_standard_rate (test_pricing.PriceWithTaxTest.test_standard_rate) ... ok
----------------------------------------------------------------------
Ran 2 tests in 0.000s
OK
[…]
+ git describe --tags --abbrev=0
[Pipeline] sh
+ mkdir -p dist
+ tar czf dist/pricing-api-0.9.1.tar.gz --exclude=dist --exclude=.git .
[Pipeline] archiveArtifacts
Archiving artifacts
Recording fingerprints
[…]
deployApp: dist/pricing-api-0.9.1.tar.gz -> staging (keeping 3 releases)
[…]
deployed pricing-api-0.9.1 to /home/sid/deployments/staging/pricing-api
releases on disk: pricing-api-0.9.1
Finished: SUCCESSThis is how platform teams in larger companies run Jenkins. Every service repository has a Jenkinsfile of a few lines. The pipeline itself (stages, timeouts, retention, test and deploy conventions) lives in one reviewed, tagged library. Upgrading 30 services to a new deploy method is one library release plus 30 one-line version bumps, and those can be rolled out a few at a time. Keep an escape hatch, though. A template that grows a parameter for every special case becomes harder to read than the 30 Jenkinsfiles it replaced. Services that really are different should write their own Jenkinsfile and still use the library's individual steps.
Common mistakes
Forgetting the underscore. An annotation followed directly by pipeline doesn't
compile:
@Library('lib@v1.0.0')
pipeline {Started inventory-team » no-underscore #1
Started from command line by admin
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
WorkflowScript: 3: unexpected token: pipeline @ line 3, column 1.
pipeline {
^
1 errorThe error points at pipeline, not at the annotation, which is why it's confusing. Write
@Library('lib@v1.0.0') _.
Importing from a library loaded with the library step. Imports are resolved when the
script is compiled, before the step has run:
import com.shellrag.ci.Version
library identifier: 'lib@main', retriever: modernSCM([$class: 'GitSCMSource',
remote: 'file:///home/sid/git/lib.git'])WorkflowScript: 1: unable to resolve class com.shellrag.ci.Version
@ line 1, column 1.
import com.shellrag.ci.Version
^Either use the returned object (lib.com.shellrag.ci.Version) or configure the library and
use @Library.
A missing @NonCPS: values that silently vanish. I pushed a branch of the library with
the @NonCPS annotations removed and loaded it:
def lib = library(identifier: 'lib@demo/without-noncps', retriever: modernSCM([$class: 'GitSCMSource',
remote: 'file:///home/sid/git/lib.git']))
def version = lib.com.shellrag.ci.Version.parse('v2.4.0').nextPatch()
echo "Next release: ${version}"
echo "done"Checking out Revision c64598ee49867bde6aa8c2315f2ed497e364f9e9 (demo/without-noncps)
Commit message: "Demo: Version without @NonCPS"
First time build. Skipping changelog.
expected to call WorkflowScript.echo but wound up catching com.shellrag.ci.Version.toString; see: https://jenkins.io/redirect/pipeline-cps-method-mismatches/
[Pipeline] echo
done
[Pipeline] End of Pipeline
Finished: SUCCESSThe first echo never happened, and the build is still green. When I first hit this, an
env.RELEASE = "inventory-api-${version}" assignment silently left RELEASE empty, so
the package was named .tar.gz. There wasn't even a warning that time. If you see
"expected to call … but wound up catching", look for a CPS-transformed method being called
from Groovy's own machinery (toString, compareTo, equals, closures passed to sort)
and mark it @NonCPS.
"Version override not permitted". With Allow default version to be overridden
turned off on the folder, the job that asks for @v1.1.0 failed before running anything:
Started inventory-team » release-inventory-api #6
Started from command line by admin
ERROR: Version override not permitted for library lib
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
WorkflowScript: Loading libraries failed
1 errorThat's the setting doing its job. Either drop the version from the annotation, or ask the library's owners to change the default.
git describe finds no tags in a multibranch build. The first pricing-api build
failed at Package:
+ git describe --tags --abbrev=0
fatal: No names found, cannot describe anything.
[…]
Stage "Deploy to staging" skipped due to earlier failure(s)
[…]
ERROR: script returned exit code 128The checkout log explains it: Cloning with configured refspecs honoured and without tags.
Multibranch branch builds don't fetch tags by default, and adding Discover tags didn't
change that when I tried it. The fix is the Advanced clone behaviours behaviour on the
branch source with Fetch tags ticked. In XML, that's this trait:
<jenkins.plugins.git.traits.CloneOptionTrait>
<extension class="hudson.plugins.git.extensions.impl.CloneOption">
<shallow>false</shallow>
<noTags>false</noTags>
<depth>0</depth>
<honorRefspec>false</honorRefspec>
</extension>
</jenkins.plugins.git.traits.CloneOptionTrait>Summary
- A shared library is a git repository with
vars/(steps: a file withcall()per step),src/(classes) andresources/(files read withlibraryResource). - Load it with
@Library('name@version') _once it's configured globally or on a folder, or with thelibrarystep without any configuration. Classes from thelibrarystep are reached through its return value, notimport. - Pin production pipelines to tags, test changes from branches, and treat the library's version number as a contract.
- Global trusted libraries run outside the sandbox. Folder, untrusted and
library-step libraries are sandboxed. A folder's definition wins over a global one with the same name. - Classes must be
Serializable, and helper methods that don't call steps should be@NonCPS. Otherwise values can disappear without an error. - In steps, pass data to shell scripts through
withEnv, not string interpolation, and validate arguments witherror. - A pipeline template (
def call(Map config) { pipeline { ... } }) cuts a service's Jenkinsfile down to a few lines.
Exercises
All four exercises follow the branch-testing workflow. Push your change to a branch of the library and load that branch from a job inside a folder that has the library configured.
Add a step releaseName(app) that returns "<app>-<next patch version>", based on the
latest tag in the workspace and using the Version class. Load it from a branch and
print the result for the inventory repository.
Show Solution
vars/releaseName.groovy:
// releaseName: "<app>-<next patch version>" based on the latest tag in the workspace.
def call(String app) {
def lastTag = sh(returnStdout: true, script: 'git describe --tags --abbrev=0').trim()
return "${app}-${com.shellrag.ci.Version.parse(lastTag).nextPatch()}"
}Pushed to feature/release-name and used from a job in the folder:
@Library('lib@feature/release-name') _
node {
git url: 'file:///home/sid/git/inventory-api.git', branch: 'main'
echo "Release name: ${releaseName('inventory-api')}"
}Loading library lib@feature/release-name
Attempting to resolve feature/release-name from remote references...
Found match: refs/heads/feature/release-name revision 4cfb9a6358d225bd82db24ad508f708e2a39a17e
[…]
+ git describe --tags --abbrev=0
[Pipeline] echo
Release name: inventory-api-2.4.1
[Pipeline] End of Pipeline
Finished: SUCCESSA step can return a value like any function. Classes from src/ are visible to vars/
by their full name, without an import.
Change deployApp so it refuses to deploy to production unless the caller passes
approved: true. Test it from a job with a boolean parameter APPROVED, once with each
value.
Show Solution
The change, as git diff main showed it:
@@ -6,6 +6,9 @@ def call(Map args = [:]) {
String artifact = args.artifact ?: error('deployApp: missing required argument "artifact"')
String root = args.root ?: env.DEPLOY_ROOT ?: error('deployApp: set "root" or DEPLOY_ROOT')
int keep = args.keep ?: 3
+ if (environment == 'production' && args.approved != true) {
+ error("deployApp: refusing to deploy ${app} to production without approved: true")
+ }
echo "deployApp: ${artifact} -> ${environment} (keeping ${keep} releases)"
writeFile file: '.deploy/deploy.sh', text: libraryResource('com/shellrag/ci/deploy.sh')The test job:
@Library('lib@feature/guard-production') _
node {
sh 'mkdir -p dist && echo "print(1)" > app.py && tar czf dist/inventory-api-2.4.1.tar.gz app.py'
withEnv(['DEPLOY_ROOT=/home/sid/deployments']) {
deployApp app: 'inventory-api', environment: 'production', artifact: 'dist/inventory-api-2.4.1.tar.gz', approved: params.APPROVED
}
}With jcli build inventory-team/ex2-guard -p APPROVED=false -s -v:
[Pipeline] withEnv
[Pipeline] error
[Pipeline] End of Pipeline
ERROR: deployApp: refusing to deploy inventory-api to production without approved: true
Finished: FAILUREWith APPROVED=true:
deployApp: dist/inventory-api-2.4.1.tar.gz -> production (keeping 3 releases)
[…]
deployed inventory-api-2.4.1 to /home/sid/deployments/production/inventory-api
releases on disk: inventory-api-2.4.1
Finished: SUCCESSI compared with != true rather than testing for truthiness, so that the string
'false' or a typo'd argument name can't count as approval.
Load the library at the exact commit that v1.1.0 points to, not at the tag, and use
Version to print the next minor version after v2.4.0. When would you pin to a commit
rather than a tag?
Show Solution
git rev-parse v1.1.0^{commit} printed 0a7076d714aa315c5683302c71b62f92b5d5fefd:
@Library('lib@0a7076d714aa315c5683302c71b62f92b5d5fefd') import com.shellrag.ci.Version
echo "Next minor after v2.4.0: ${Version.parse('v2.4.0').nextMinor()}"Loading library lib@0a7076d714aa315c5683302c71b62f92b5d5fefd
Attempting to resolve 0a7076d714aa315c5683302c71b62f92b5d5fefd from remote references...
Found match: refs/heads/main revision 0a7076d714aa315c5683302c71b62f92b5d5fefd
[…]
[Pipeline] echo
Next minor after v2.4.0: 2.5.0
[Pipeline] End of Pipeline
Finished: SUCCESSTags can be moved or deleted by anyone with push rights, but a commit can't change. Pin to a commit when you need a guarantee, for example in a regulated release pipeline or while you bisect which library change broke a build. For everyday pinning, tags are easier to read.
Write com.shellrag.ci.ReleaseNotes, a class that writes RELEASE_NOTES.md listing the
commits since the previous tag. A class in src/ can't call sh or writeFile on its
own. Work out how to give it access to them.
Hint
Pass the script object (this in a Jenkinsfile) to the constructor and call steps on it.
Show Solution
package com.shellrag.ci
/** Writes RELEASE_NOTES.md from the commits since the previous tag. */
class ReleaseNotes implements Serializable {
private final def steps
ReleaseNotes(steps) {
this.steps = steps
}
String write(String release) {
String log = steps.sh(returnStdout: true,
script: 'git log --pretty="- %s (%h)" $(git describe --tags --abbrev=0)..HEAD').trim()
String text = "# ${release}\n\n${log ?: '- no changes'}\n"
steps.writeFile file: 'RELEASE_NOTES.md', text: text
steps.echo "wrote RELEASE_NOTES.md (${log.readLines().size()} changes)"
return text
}
}@Library('lib@feature/release-notes') import com.shellrag.ci.ReleaseNotes
node {
git url: 'file:///home/sid/git/inventory-api.git', branch: 'main'
def notes = new ReleaseNotes(this).write('inventory-api-2.4.1')
echo notes
}[Pipeline] sh
+ git describe --tags --abbrev=0
+ git log --pretty=- %s (%h) v2.4.0..HEAD
[Pipeline] writeFile
[Pipeline] echo
wrote RELEASE_NOTES.md (1 changes)
[Pipeline] echo
# inventory-api-2.4.1
- Package hotfix branches too (29db54b)
[Pipeline] End of Pipeline
Finished: SUCCESSwrite calls steps, so it must not be @NonCPS. The steps field holds the script
object, which Pipeline knows how to serialize. That's why the class can safely be
Serializable and keep it in a field across the sh and writeFile steps.
Tested with: Jenkins 2.580.2 (OpenJDK 21) on Ubuntu 24.04 (WSL2), Pipeline: Groovy 4403.v471713a_cea_19, Pipeline: Groovy Libraries 822.vc95f70dea_db_7, Git plugin 5.11.0 and Configuration as Code 2147.v8a_8d38668c72, November 2026.
The library, its two tags, the folder configuration, the library step, every @Library job, the pipeline template and all four exercise solutions ran on the main lab controller. The trusted-versus-untrusted section (the JCasC global library, buildQueueLength succeeding at the top level and being rejected inside the folder) ran on a separate throwaway controller of the same version with the same plugins, because global configuration and script approvals were off-limits on the shared one. JenkinsPipelineUnit was described but not run. Logs are trimmed where marked […]. Paths from the test machine's scratch directories are shown shortened to /home/sid/…. Job, node and credential names are shown without the test run's prefixes.
Written by The Dark Debugger
Engineer documenting the things that took me too long to learn, so they take you less time. Every Jenkins tutorial here is tested end to end before publishing.