Multibranch Pipelines and Webhooks
Let Jenkins discover every branch, tag and pull request on its own, run branch-specific stages with when, and build on push through webhooks instead of polling.
By The Dark DebuggerUpdated 30 min read
This article turns one Jenkinsfile into a whole family of jobs: one per branch, tag and
pull request, created and deleted automatically as the repository changes. You should
already be comfortable with a Jenkinsfile from
Your First Pipeline and with when, post and
options from Declarative Pipeline in Depth.
It opens the Real-World Delivery section. Shared Libraries
and the capstone pipeline both build on it.
Why one job per branch doesn't scale
Picture a service repository with main, develop, a release/2.3 branch and five
feature branches that live for a few days each. With plain Pipeline jobs, someone has to
create a job for every new branch, point it at the right branch, and remember to delete it
after the merge. Nobody does that reliably. Feature branches go untested until they land in
develop, and the job list fills up with jobs for branches that were deleted months ago.
There is a second problem. Each of those jobs asks the git server "anything new?" every few minutes. That's slow, because a push waits for the next poll, and it adds up: 40 jobs polling every minute is 57,600 requests a day to answer "no".
A Multibranch Pipeline fixes the first problem and webhooks fix the second. Jenkins scans the repository, creates a job for every branch that has a Jenkinsfile, and removes jobs for branches that disappear. The git server tells Jenkins when something changes, so Jenkins never has to ask.
How a multibranch project works
A multibranch project is a folder that Jenkins fills with jobs by itself. Six parts make it work:
| Part | What it does | Where you set it |
|---|---|---|
| Branch source | The repository to scan (plain Git, GitHub, GitLab, Bitbucket) | Branch Sources |
| Discovery traits | What to look for: branches, tags, pull requests | Behaviours, under the source |
| Project recognizer | Which heads get a job: those whose tree contains the Jenkinsfile | Build Configuration, Mode "by Jenkinsfile" |
| Branch jobs | Ordinary Pipeline jobs inside the folder, one per discovered head | Created by Jenkins, never by hand |
| Scan (indexing) | The run that lists heads, compares commits and schedules builds | "Scan Multibranch Pipeline Now", periodic trigger, webhooks |
| Orphaned item strategy | What happens to jobs whose branch was deleted | Orphaned Item Strategy |
Every branch job builds the Jenkinsfile from its own branch. If develop changes the
pipeline, only develop builds with the change until it's merged. That's the property that
makes multibranch safe to experiment with.
The demo repository
I created a bare repository on the lab machine to play the part of the git server, and
pushed five branches to it. Every branch except gh-pages carries the same small
Python service and a Jenkinsfile. gh-pages holds only documentation.
git ls-remote --heads originf9177341e821d5fe966284235e1d843bada704b7 refs/heads/develop
ffe4be207e883758a843fb06f562d78c664f2386 refs/heads/feature/low-stock-alert
3110645c97b7ed9d0cd9fac7b1d17e1a9967c97a refs/heads/gh-pages
df1488aaeb598e7cdd55f073a9fc10618da17bdf refs/heads/main
df1488aaeb598e7cdd55f073a9fc10618da17bdf refs/heads/release/2.3The Jenkinsfile is the same on every branch. Each stage's when condition decides whether
the stage runs for the branch being built:
pipeline {
agent any
options {
// Keep more history for main than for short-lived branches
buildDiscarder(logRotator(numToKeepStr: env.BRANCH_NAME == 'main' ? '30' : '5'))
}
stages {
stage('Context') {
steps {
echo "Building ${env.BRANCH_NAME} (CHANGE_ID=${env.CHANGE_ID})"
sh 'printenv | grep -E "^(BRANCH_NAME|BRANCH_IS_PRIMARY|CHANGE_ID|TAG_NAME|JOB_NAME|JOB_BASE_NAME)=" | sort'
}
}
stage('Test') {
steps {
sh 'python3 -m unittest -v'
}
}
stage('Package') {
when { anyOf { branch 'main'; branch 'release/*' } }
steps {
sh 'tar czf inventory-api-$(git rev-parse --short HEAD).tar.gz app.py'
archiveArtifacts artifacts: '*.tar.gz', fingerprint: true
}
}
stage('Deploy to dev') {
when { branch 'develop' }
steps {
echo "Deploying ${env.BRANCH_NAME} to the dev environment"
}
}
stage('Deploy to staging') {
when { branch 'main' }
steps {
echo "Deploying ${env.BRANCH_NAME} to staging"
}
}
stage('Publish release') {
when { buildingTag() }
steps {
echo "Publishing release ${env.TAG_NAME}"
}
}
stage('PR checks') {
when { changeRequest() }
steps {
echo "PR ${env.CHANGE_ID}: ${env.CHANGE_BRANCH} -> ${env.CHANGE_TARGET}"
}
}
}
}The deploy stages only echo here because this article is about choosing which stages
run. The capstone replaces them with real
deployments.
Creating the multibranch job
In the web UI the steps are:
- New Item, enter a name, choose Multibranch Pipeline, click OK.
- Under Branch Sources, click Add source and choose Git. Enter the Project Repository URL and, for a private repository, the Credentials. The default Behaviours contain Discover branches.
- Leave Build Configuration at Mode by Jenkinsfile and Script Path
Jenkinsfile. This is where a monorepo would point at something likeservices/inventory/Jenkinsfile. - Under Scan Multibranch Pipeline Triggers, tick Periodically if not otherwise run and pick an interval such as 1 day. With webhooks in place this scan is only a safety net.
- Under Orphaned Item Strategy, tick Discard old items and leave the two keep fields empty if jobs for deleted branches should go straight away.
Every job has a config.xml, so you can create the same project without clicking. I
created mine from this file with the Jenkins CLI.
<?xml version='1.1' encoding='UTF-8'?>
<org.jenkinsci.plugins.workflow.multibranch.WorkflowMultiBranchProject>
<description>inventory-api: one Pipeline job per branch</description>
<orphanedItemStrategy class="com.cloudbees.hudson.plugins.folder.computed.DefaultOrphanedItemStrategy">
<pruneDeadBranches>true</pruneDeadBranches>
<daysToKeep>-1</daysToKeep>
<numToKeep>-1</numToKeep>
<abortBuilds>true</abortBuilds>
</orphanedItemStrategy>
<triggers>
<com.cloudbees.hudson.plugins.folder.computed.PeriodicFolderTrigger>
<spec>H H/4 * * *</spec>
<interval>86400000</interval>
</com.cloudbees.hudson.plugins.folder.computed.PeriodicFolderTrigger>
</triggers>
<sources class="jenkins.branch.MultiBranchProject$BranchSourceList">
<data>
<jenkins.branch.BranchSource>
<source class="jenkins.plugins.git.GitSCMSource">
<id>inventory-api</id>
<remote>file:///home/sid/git/inventory-api.git</remote>
<traits>
<jenkins.plugins.git.traits.BranchDiscoveryTrait/>
</traits>
</source>
</jenkins.branch.BranchSource>
</data>
</sources>
<factory class="org.jenkinsci.plugins.workflow.multibranch.WorkflowBranchProjectFactory">
<scriptPath>Jenkinsfile</scriptPath>
</factory>
</org.jenkinsci.plugins.workflow.multibranch.WorkflowMultiBranchProject>To save typing I wrap the CLI in a small function. Run the function definition once per shell:
jcli() { java -jar jenkins-cli.jar -s http://localhost:8090/ -auth "admin:$JENKINS_PASSWORD" "$@"; }
jcli create-job inventory-api < multibranch.xmlcreate-job prints nothing when it succeeds. The -1 values in the orphaned item
strategy are what the UI saves when you leave the keep fields empty.
The first scan
You don't have to click "Scan Multibranch Pipeline Now" after creating the job. The new
project scheduled its first scan by itself. The scan log is at
job/NAME/indexing/consoleText:
curl -s -u "admin:$JENKINS_PASSWORD" http://localhost:8090/job/inventory-api/indexing/consoleTextLegacy code started this job. No cause information is available
[Tue Nov 10 10:28:35 EST 2026] Starting branch indexing...
> git --version # timeout=10
[…]
Checking branches...
Checking branch release/2.3
‘Jenkinsfile’ found
Met criteria
Scheduled build for branch: release/2.3
Checking branch develop
‘Jenkinsfile’ found
Met criteria
Scheduled build for branch: develop
Checking branch main
‘Jenkinsfile’ found
Met criteria
Scheduled build for branch: main
Checking branch gh-pages
‘Jenkinsfile’ not found
Does not meet criteria
Checking branch feature/low-stock-alert
‘Jenkinsfile’ found
Met criteria
Scheduled build for branch: feature/low-stock-alert
Processed 5 branches
[Tue Nov 10 10:28:36 EST 2026] Finished branch indexing. Indexing took 0.96 sec
Finished: SUCCESSThis log is the first thing to read when a branch "isn't showing up". Each branch gets the same three checks:
- ‘Jenkinsfile’ found / not found: the project recognizer looked for the Script Path in
that branch's tree.
gh-pageshas no Jenkinsfile, so it got no job. That's exactly what you want for a documentation branch. - Met criteria: the branch passes every discovery trait and filter.
- Scheduled build: this is a new branch, or its head commit changed since the last scan.
An unchanged branch shows
No changes detectedinstead, as you'll see below.
Don't let "Legacy code started this job" worry you. That's how the first scan after
create-job describes its cause. Later scans say Started by user admin or
Branch indexing.
Branch jobs and their names
Once the four builds had finished, I listed the folder's children through the REST API:
curl -s -u "admin:$JENKINS_PASSWORD" \
"http://localhost:8090/job/inventory-api/api/json?tree=jobs%5Bname,url,lastBuild%5Bnumber,result%5D%5D" \
| jq -r '.jobs[] | "\(.name)\t#\(.lastBuild.number) \(.lastBuild.result)\t\(.url)"'develop #1 SUCCESS http://localhost:8090/job/inventory-api/job/develop/
feature%2Flow-stock-alert #1 SUCCESS http://localhost:8090/job/inventory-api/job/feature%252Flow-stock-alert/
main #1 SUCCESS http://localhost:8090/job/inventory-api/job/main/
release%2F2.3 #1 SUCCESS http://localhost:8090/job/inventory-api/job/release%252F2.3/Look closely at the names. A slash can't appear in a job name, so feature/low-stock-alert
becomes the job feature%2Flow-stock-alert. In a URL the % itself is encoded again, which
gives %252F. Scripts that build URLs by hand get this wrong all the time:
for u in "job/release/2.3" "job/release%252F2.3"; do
curl -s -o /dev/null -w "%{http_code} $u\n" -u "admin:$JENKINS_PASSWORD" \
"http://localhost:8090/job/inventory-api/$u/api/json"
done404 job/release/2.3
200 job/release%252F2.3The CLI takes the job's full name, and for that you use the single-encoded %2F form:
jcli build 'inventory-api/release%2F2.3'.
Workspaces get a similar treatment. Jenkins shortens branch workspace directories and keeps
the mapping in workspace/workspaces.txt:
inventory-api/feature%2Flow-stock-alert
tory-api_feature_low-stock-alert
inventory-api/main
inventory-api_mainSo if you go looking for a feature branch's workspace on an agent, check this file first.
What each branch build sees
Here is the main build. The Context stage printed the variables that multibranch adds:
[Pipeline] { (Context)
[Pipeline] echo
Building main (CHANGE_ID=null)
[Pipeline] sh
+ printenv
+ sort
+ grep -E ^(BRANCH_NAME|BRANCH_IS_PRIMARY|CHANGE_ID|TAG_NAME|JOB_NAME|JOB_BASE_NAME)=
BRANCH_IS_PRIMARY=true
BRANCH_NAME=main
JOB_BASE_NAME=main
JOB_NAME=inventory-api/main
[…]
[Pipeline] { (Package)
[Pipeline] sh
+ git rev-parse --short HEAD
+ tar czf inventory-api-df1488a.tar.gz app.py
[Pipeline] archiveArtifacts
Archiving artifacts
Recording fingerprints
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Deploy to dev)
Stage "Deploy to dev" skipped due to when conditional
[Pipeline] getContext
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Deploy to staging)
[Pipeline] echo
Deploying main to stagingAnd here is the feature branch, which has one extra test and skips everything after Test:
[Pipeline] { (Context)
[Pipeline] echo
Building feature/low-stock-alert (CHANGE_ID=null)
[Pipeline] sh
+ printenv
+ grep -E ^(BRANCH_NAME|BRANCH_IS_PRIMARY|CHANGE_ID|TAG_NAME|JOB_NAME|JOB_BASE_NAME)=
+ sort
BRANCH_NAME=feature/low-stock-alert
JOB_BASE_NAME=feature%2Flow-stock-alert
JOB_NAME=inventory-api/feature%2Flow-stock-alert
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Test)
[Pipeline] sh
+ python3 -m unittest -v
test_known_sku (test_app.StockLevelTest.test_known_sku) ... ok
test_low_stock (test_app.StockLevelTest.test_low_stock) ... ok
test_unknown_sku (test_app.StockLevelTest.test_unknown_sku) ... ok
----------------------------------------------------------------------
Ran 3 tests in 0.000s
OK
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Package)
Stage "Package" skipped due to when conditionalA few things in this output are easy to miss:
BRANCH_NAMEkeeps the slash, whileJOB_BASE_NAMEuses the encoded job name. UseBRANCH_NAMEinwhenconditions and in file names you build from the branch.BRANCH_IS_PRIMARY=trueappeared only onmain. The Git source learned the primary branch from the repository'sHEAD. On the other branches the variable is unset, notfalse.CHANGE_IDis unset on branch builds, which is why Groovy printednull. It only exists in pull request builds.
These are all the variables multibranch can set. The descriptions come from the Branch API plugin's own help:
| Variable | Set for | Meaning |
|---|---|---|
BRANCH_NAME | every build | Branch, tag or PR name (main, v2.4.0, PR-42) |
BRANCH_IS_PRIMARY | the primary branch | true when the source reports a primary branch, else unset |
CHANGE_ID | change requests | Pull/merge request number |
CHANGE_URL, CHANGE_TITLE | change requests | Link to and title of the PR |
CHANGE_AUTHOR, CHANGE_AUTHOR_DISPLAY_NAME, CHANGE_AUTHOR_EMAIL | change requests | Who opened it |
CHANGE_TARGET | change requests | Branch the PR would merge into |
CHANGE_BRANCH | change requests | The PR's source branch name |
CHANGE_FORK | change requests from forks | Name of the fork |
TAG_NAME, TAG_TIMESTAMP, TAG_UNIXTIME, TAG_DATE | tag builds | The tag and when it was made |
Branch-specific behaviour with when
The when conditions below are the ones you'll use in multibranch pipelines. They're all
built into Declarative Pipeline:
| Condition | True when |
|---|---|
branch 'main' | BRANCH_NAME is exactly main |
branch 'release/*' | BRANCH_NAME matches the Ant-style glob (release/2.3 matches) |
branch pattern: 'release/\\d+\\.\\d+', comparator: 'REGEXP' | BRANCH_NAME matches the regular expression |
buildingTag() | the build is for a tag |
tag 'v*' | the build is for a tag matching the glob |
changeRequest() | the build is for a pull/merge request |
changeRequest target: 'main' | it's a PR into main |
not { branch 'main' } | wrap any condition to invert it |
The runs above show the conditions doing their job. main ran Package and Deploy to
staging. feature/low-stock-alert ran only Context and Test. develop ran Deploy to dev,
and release/2.3 ran Package.
Two habits pay off here. First, add beforeAgent true inside when for stages that
allocate their own agent. Then a skipped stage doesn't wait for an executor just to decide
it has nothing to do. Second, keep branch logic in when rather than in if blocks inside
script. Skipped stages then show up as skipped in the stage view, and anyone reading the
build can see why a deploy didn't happen.
options can also depend on the branch, because env.BRANCH_NAME is already set when
options are evaluated. The buildDiscarder line in this Jenkinsfile keeps 30 builds of
main and 5 of everything else.
Please note: none of this works in a plain Pipeline job. In a plain job BRANCH_NAME
is never set, so when { branch 'main' } is always false. The
common mistakes section shows this happening.
When a branch is deleted: orphaned items
I merged the feature branch into develop and deleted it from the server:
git merge -q --no-ff -m "Merge feature/low-stock-alert into develop" feature/low-stock-alert
git push -q origin develop
git push origin --delete feature/low-stock-alertremote: No git jobs using repository: file:///home/sid/git/inventory-api.git and branches:
remote: Scheduled indexing of inventory-api
To file:///home/sid/git/inventory-api.git
- [deleted] feature/low-stock-alertThe remote: lines come from a git hook that notifies Jenkins. We'll set it up in the
webhooks section. The scan it triggered ended like this:
Processed 4 branches
[Tue Nov 10 10:29:19 EST 2026] Finished branch indexing. Indexing took 0.27 sec
Evaluating orphaned items in inventory-api
Will remove feature/low-stock-alert
Finished: SUCCESSThe folder now contains ["develop","main","release%2F2.3"], and Jenkins deleted the
branch's workspace along with the job. The four settings of the default strategy are:
| UI field | XML | Effect |
|---|---|---|
| Discard old items | pruneDeadBranches | Off: dead branch jobs stay (disabled) forever |
| Days to keep old items | daysToKeep | Keep a dead branch's job this many days |
| Max # of old items to keep | numToKeep | Keep at most this many dead branch jobs |
| Abort builds | abortBuilds | Abort running builds of a branch that just disappeared |
My recommendation is to prune right away unless people need the build history of merged branches. The merge commit's build on the target branch is the one that matters. If you do keep dead branches for a while, Jenkins disables those jobs so nobody can build them by accident. Exercise 3 shows that.
Tags: discovered, but not built
Release tags are where multibranch surprises people. I added Discover tags to the source's Behaviours, which is one more trait in the XML:
<traits>
<jenkins.plugins.git.traits.BranchDiscoveryTrait/>
<jenkins.plugins.git.traits.TagDiscoveryTrait/>
</traits>Then I updated the job and pushed a tag:
jcli update-job inventory-api < multibranch-tags.xml
git tag -a v2.4.0 -m "inventory-api 2.4.0"
git push origin v2.4.0remote: No git jobs using repository: file:///home/sid/git/inventory-api.git and branches:
remote: Scheduled indexing of inventory-api
To file:///home/sid/git/inventory-api.git
* [new tag] v2.4.0 -> v2.4.0The scan created a v2.4.0 job, but look at its last line:
Checking tags...
Checking tag v2.4.0
‘Jenkinsfile’ found
Met criteria
No automatic build triggered for v2.4.0
Processed 1 tagsThis behaviour is deliberate. If you pointed Jenkins at a repository with 200 old tags, it would otherwise start 200 release builds at once. To build tags automatically you need a build strategy plugin, usually Basic Branch Build Strategies, which can build tags newer than an age you choose. That plugin isn't installed on my lab controller, so I built the tag job by hand:
jcli build inventory-api/v2.4.0 -s -vStarted inventory-api » v2.4.0 #1
Started from command line by admin
[…]
Cloning with configured refspecs honoured and with tags
[…]
Building v2.4.0 (CHANGE_ID=null)
[…]
BRANCH_NAME=v2.4.0
JOB_BASE_NAME=v2.4.0
JOB_NAME=inventory-api/v2.4.0
TAG_NAME=v2.4.0
[…]
[Pipeline] { (Publish release)
[Pipeline] echo
Publishing release v2.4.0Notice "with tags" in the clone line. Branch builds clone "without tags" by default, so
git describe --tags in a branch build fails unless you enable tag discovery or fetch
tags yourself.
Pull requests: GitHub, GitLab and Bitbucket
The plain Git source only knows branches and tags. Pull requests are a feature of the hosting service, so you need that service's branch source plugin:
| Host | Plugin | PR job names |
|---|---|---|
| GitHub | GitHub Branch Source (installed in this lab) | PR-42 |
| GitLab | GitLab Branch Source | MR-42 |
| Bitbucket Cloud and Server | Bitbucket Branch Source | PR-42 |
I can't run a PR build in an offline lab, because the PRs live on github.com. What I did was create a GitHub-sourced project on the controller with scanning disabled. That confirmed Jenkins accepts this configuration, and it's the part you'll copy:
<source class="org.jenkinsci.plugins.github_branch_source.GitHubSCMSource">
<id>inventory-api-github</id>
<credentialsId>github-app</credentialsId>
<repoOwner>acme-retail</repoOwner>
<repository>inventory-api</repository>
<repositoryUrl>https://github.com/acme-retail/inventory-api</repositoryUrl>
<traits>
<org.jenkinsci.plugins.github_branch_source.BranchDiscoveryTrait>
<strategyId>1</strategyId>
</org.jenkinsci.plugins.github_branch_source.BranchDiscoveryTrait>
<org.jenkinsci.plugins.github_branch_source.OriginPullRequestDiscoveryTrait>
<strategyId>1</strategyId>
</org.jenkinsci.plugins.github_branch_source.OriginPullRequestDiscoveryTrait>
<org.jenkinsci.plugins.github_branch_source.ForkPullRequestDiscoveryTrait>
<strategyId>1</strategyId>
<trust class="org.jenkinsci.plugins.github_branch_source.ForkPullRequestDiscoveryTrait$TrustPermission"/>
</org.jenkinsci.plugins.github_branch_source.ForkPullRequestDiscoveryTrait>
</traits>
</source>The numeric strategy IDs correspond to these UI choices:
| Behaviour (UI) | Strategy 1 | Strategy 2 | Strategy 3 |
|---|---|---|---|
| Discover branches | Exclude branches that are also filed as PRs | Only branches that are also filed as PRs | All branches |
| Discover pull requests from origin / from forks | Merging the pull request with the current target branch revision | The current pull request revision | Both |
Strategy 1 for branches avoids building the same commit twice, once as feature/x and
once as PR-17. Strategy 1 for PRs tests what main will look like after the merge, which
is the question a reviewer actually cares about.
The Trust setting on fork PRs is a security control, not a convenience. A pull request from a fork can contain any Jenkinsfile its author likes. With the default trust, "From users with Admin or Write permission", Jenkins builds an outsider's PR with the target branch's Jenkinsfile, so a stranger can't change what runs on your agents. Never pick "Everyone" on a public repository. (I checked the default on the lab controller: the trait's descriptor reports "From users with Admin or Write permission".)
In a PR build CHANGE_ID, CHANGE_TARGET and the other CHANGE_* variables are set,
BRANCH_NAME is PR-42, and when { changeRequest() } becomes true.
Webhooks versus polling
So far, every scan in this article was triggered by a push. There are three ways Jenkins can find out about changes:
| Method | How it works | Delay | Cost |
|---|---|---|---|
Polling (pollSCM in a plain job) | Jenkins asks the git server on a cron schedule | Up to one interval | One request per job per interval, mostly "no change" |
| Periodic scan (multibranch) | "Periodically if not otherwise run" re-indexes the whole repository | Up to one interval | One scan per interval, and it's skipped if something else triggered a scan |
| Webhook | The git server sends an HTTP request to Jenkins on every push | Seconds | Nothing when nothing happens |
A webhook is just an HTTP POST that the git server sends to a URL you register, every time something happens in the repository. Use webhooks for speed and keep a long periodic scan (a day is fine) as a safety net. Webhooks get lost: a firewall change, an expired certificate, or a Jenkins restart at the wrong moment can each drop one, and the periodic scan catches up afterwards.
There's one network requirement. The git server has to reach Jenkins. GitHub.com can't call a Jenkins that only listens on a private network. In that case teams either expose just the webhook path through a reverse proxy, or they fall back to polling.
GitHub webhooks
With the GitHub Branch Source plugin, Jenkins listens on /github-webhook/. On my lab
controller a GET returns 405 Method Not Allowed because the endpoint only accepts POSTs.
A POST of GitHub's ping event is accepted:
curl -s -i -X POST -H 'X-GitHub-Event: ping' -H 'Content-Type: application/json' \
-d '{"zen":"Design for failure.","hook_id":1}' http://localhost:8090/github-webhook/ | head -1HTTP/1.1 200 OKThe GitHub side is configured in the browser, and I couldn't do that from the offline lab. In the repository (or once for the whole organization), open Settings > Webhooks > Add webhook:
- Payload URL:
https://jenkins.example.com/github-webhook/. Keep the trailing slash. When I POSTed to/github-webhookwithout it, Jenkins answered302with a redirect to the slashed URL, and GitHub doesn't follow redirects when it delivers a webhook. - Content type:
application/json. - Secret: a long random string. In Jenkins, store the same string as a Secret text credential, then add it under Manage Jenkins > System > GitHub > Shared secrets > Add shared secret. Jenkins then rejects payloads that aren't signed with it.
- Events: choose "Let me select individual events" and tick Pushes, Pull requests, Branch or tag creation and Branch or tag deletion.
There's an alternative: if you add a GitHub server under Manage Jenkins > System > GitHub with credentials that may administer hooks and tick Manage hooks, Jenkins registers the webhook itself. GitHub's own "Recent Deliveries" tab on the webhook page shows each request and Jenkins' response, and it's the first place to look when builds stop starting.
Any git server: the notifyCommit endpoint
For a plain Git server, such as a bare repository over SSH, Gitea, or anything else that
can run a hook, the Git plugin offers /git/notifyCommit?url=<repository URL>. It tells
Jenkins "this repository changed". Jenkins then re-scans every multibranch project, and
polls every polling job, whose remote matches that URL.
Current versions of the Git plugin (5.11.0 here) require an access token for this endpoint. Without one:
curl -s -i "http://localhost:8090/git/notifyCommit?url=file:///home/sid/git/inventory-api.git"HTTP/1.1 401 Unauthorized
[…]
<title>Error 401 An access token is required. Please refer to Git plugin documentation (https://plugins.jenkins.io/git/#plugin-content-push-notification-from-repository) for details.</title>You create a token under Manage Jenkins > Security > Git plugin notifyCommit access tokens > Add new access token: type a name, click Generate and copy the value. It's shown only once. The Generate button posts to an endpoint you can also call yourself. Logging in with a password needs a CSRF crumb for any POST:
CRUMB=$(curl -s -c cookies.txt -u "admin:$JENKINS_PASSWORD" \
http://localhost:8090/crumbIssuer/api/json | jq -r '.crumbRequestField + ":" + .crumb')
curl -s -b cookies.txt -u "admin:$JENKINS_PASSWORD" -H "$CRUMB" -X POST \
http://localhost:8090/descriptorByName/hudson.plugins.git.ApiTokenPropertyConfiguration/generate \
--data apiTokenName=inventory-api-hook{"status":"ok","data":{"uuid":"dd7a7b83-5ee0-43c7-bc9d-d0c90560eeff","name":"inventory-api-hook","value":"14290f..."}}I cut the token value short in that output. Treat it like a password. With the token, the same request works:
curl -s "http://localhost:8090/git/notifyCommit?url=file:///home/sid/git/inventory-api.git&token=$NOTIFY_TOKEN"No git jobs using repository: file:///home/sid/git/inventory-api.git and branches:
Scheduled indexing of inventory-apiThe first line refers to plain polling jobs (there are none for this repository). The
second is the multibranch project. I also tried the URL without .git and it still
matched (Scheduled indexing of inventory-api), because the plugin normalises URLs
before comparing them.
Now let's make the git server send that request on every push. On a bare repository, a
post-receive hook runs after each successful push, and anything it prints goes back to
the person pushing:
#!/usr/bin/env bash
# hooks/post-receive: tell Jenkins this repository changed.
# Runs on the git server after every successful push.
JENKINS_URL="http://localhost:8090"
REPO_URL="file:///home/sid/git/inventory-api.git"
TOKEN=$(cat "$(git rev-parse --git-dir)/jenkins-notify-token")
curl -fsS --max-time 10 "$JENKINS_URL/git/notifyCommit?url=$REPO_URL&token=$TOKEN" \
|| echo "warning: Jenkins was not notified; the periodic scan will catch up"I saved it as hooks/post-receive in the bare repository, ran chmod +x on it, and put
the token in jenkins-notify-token with mode 600. Note the || echo: a hook that fails
loudly when Jenkins is down would make every push look broken, even though the push itself
succeeded. Then I pushed a one-line change to develop:
git commit -q -am "Add SKU-1005 to the seed data"
git push origin developremote: No git jobs using repository: file:///home/sid/git/inventory-api.git and branches:
remote: Scheduled indexing of inventory-api
To file:///home/sid/git/inventory-api.git
f917734..b86d343 develop -> developThe scan that followed found exactly one change and started exactly one build:
Checking branch develop
‘Jenkinsfile’ found
Met criteria
Changes detected: develop (f9177341e821d5fe966284235e1d843bada704b7 → b86d34360eb6cdc5e194f1f9abdbfdc117557fa0)
Scheduled build for branch: develop
Checking branch main
‘Jenkinsfile’ found
Met criteria
No changes detected: main (still at df1488aaeb598e7cdd55f073a9fc10618da17bdf)
[…]
[Tue Nov 10 10:29:10 EST 2026] Finished branch indexing. Indexing took 0.37 secThe hook scheduled the scan during the push, and the scan itself took 0.37 seconds. With a 5-minute polling interval the same change could have waited up to five minutes.
Organization folders
A multibranch project covers one repository. An Organization Folder covers a whole GitHub organization or Bitbucket team/project. It scans the organization and creates a multibranch project for every repository that contains a Jenkinsfile. When someone adds a new service repository with a Jenkinsfile, it gets CI without anyone touching Jenkins.
In the UI it's New Item > Organization Folder, with a GitHub Organization repository source. I created this one disabled, since the organization doesn't exist:
<navigators>
<org.jenkinsci.plugins.github_branch_source.GitHubSCMNavigator>
<repoOwner>acme-retail</repoOwner>
<credentialsId>github-app</credentialsId>
<traits>
<jenkins.scm.impl.trait.WildcardSCMSourceFilterTrait>
<includes>*</includes>
<excludes>sandbox-* archive-*</excludes>
</jenkins.scm.impl.trait.WildcardSCMSourceFilterTrait>
<org.jenkinsci.plugins.github_branch_source.BranchDiscoveryTrait>
<strategyId>1</strategyId>
</org.jenkinsci.plugins.github_branch_source.BranchDiscoveryTrait>
<org.jenkinsci.plugins.github_branch_source.OriginPullRequestDiscoveryTrait>
<strategyId>1</strategyId>
</org.jenkinsci.plugins.github_branch_source.OriginPullRequestDiscoveryTrait>
</traits>
</org.jenkinsci.plugins.github_branch_source.GitHubSCMNavigator>
</navigators>
<projectFactories>
<org.jenkinsci.plugins.workflow.multibranch.WorkflowMultiBranchProjectFactory>
<scriptPath>Jenkinsfile</scriptPath>
</org.jenkinsci.plugins.workflow.multibranch.WorkflowMultiBranchProjectFactory>
</projectFactories>The Filter by name (with wildcards) trait keeps sandbox and archived repositories out. For the credential, use a GitHub App rather than a personal access token. An app's API rate limit grows with the organization, and the app isn't tied to one employee's account. The credentials article covers storing it. Once you have more than a handful of these projects, define them in code rather than by hand. Jenkins Configuration as Code shows how.
Where you'll use this
Most teams end up with one of two setups.
Hosted git (GitHub, GitLab, Bitbucket). You create one organization folder per
organization. It's configured with a GitHub App credential, the branch and PR discovery
traits shown above, and a one-day periodic scan. Webhooks come from an organization-level
webhook. Each repository's Jenkinsfile uses when { branch 'main' } for deploys and
when { changeRequest() } for PR-only checks. GitHub's branch protection rule for main
then requires the status that Jenkins reports for PR builds. The installed GitHub Branch
Source plugin names it continuous-integration/jenkins/pr-merge, or pr-head if you build
PR heads, and branch builds report continuous-integration/jenkins/branch. From then on,
nothing merges into main without a green Jenkins build of the merged result. Nobody
creates jobs, and nobody deletes them.
Self-hosted git. A bare repository server, Gitea or an internal GitLab gets one
multibranch project per repository, plus the post-receive hook above calling
notifyCommit with a token. This repository's setup is a complete working example of that:
every push started a scan straight from the hook, main and release/* produced
fingerprinted packages, develop deployed to dev, and feature branches ran tests only. A
deleted branch's job disappeared on the next scan.
In both setups, keep the deploy logic itself out of the branch conditions. The when
decides whether to deploy, and a script or a
shared library step does the deploying.
Common mistakes
when { branch 'main' } never matches. I pointed a plain Pipeline job (not
multibranch) at main of the same repository. Every branch stage was skipped, even though
it built main:
Building null (CHANGE_ID=null)
[Pipeline] sh
+ printenv
+ grep -E ^(BRANCH_NAME|BRANCH_IS_PRIMARY|CHANGE_ID|TAG_NAME|JOB_NAME|JOB_BASE_NAME)=
+ sort
JOB_BASE_NAME=inventory-api-plain
JOB_NAME=inventory-api-plain
[…]
[Pipeline] { (Deploy to staging)
Stage "Deploy to staging" skipped due to when conditionalBRANCH_NAME only exists in multibranch builds. In a plain job, either convert it to
multibranch or test the variable the Git checkout does set. I tried
when { expression { env.GIT_BRANCH == 'origin/main' } } in a plain job pointed at
main, and the stage ran:
[Pipeline] { (Deploy to staging)
[Pipeline] echo
GIT_BRANCH=origin/main, deployingnotifyCommit returns 401. That's the "An access token is required" response shown
above. You need to add &token=…. A token in the URL doesn't help if the URL itself is
wrong, though. With a URL that no job uses, Jenkins answers 200 and does nothing:
No git jobs using repository: file:///home/sid/git/inventory.git and branches:
No Git consumers using SCM API plugin for: file:///home/sid/git/inventory.gitYour hook "works" and nothing builds. Read the response. The URL in the hook has to be the one configured in the job (SSH and HTTPS URLs for the same repository are different URLs).
Local repositories are refused. If you practise with a file:// repository like I
did, a controller that hasn't set -Dhudson.plugins.git.GitSCM.ALLOW_LOCAL_CHECKOUT=true
lets the scan succeed but fails every build at checkout:
ERROR: Checkout of Git remote 'file:///home/sid/git/inventory-api.git' aborted because it references a local directory, which may be insecure. You can allow local checkouts anyway by setting the system property 'hudson.plugins.git.GitSCM.ALLOW_LOCAL_CHECKOUT' to true.
ERROR: Maximum checkout retry attempts reached, abortingOn a real controller, serve the repository over SSH or HTTPS instead of setting that property. It's off by default because it lets job configurators read files on the controller.
Triggering the folder from the CLI. A multibranch project is a folder, not a job:
jcli build inventory-api
ERROR: No such job 'inventory-api'; perhaps you meant 'inventory-api/main'?Build a branch job (inventory-api/main), or start a scan with a POST to
job/inventory-api/build?delay=0. That's the same request the "Scan Multibranch
Pipeline Now" link sends. Exercise 4 wraps it in a script.
Waiting for a tag build that never comes. "No automatic build triggered for v2.4.0" means it's working as designed. Build the tag job yourself, or install a build strategy plugin.
Summary
- A Multibranch Pipeline is a folder that Jenkins fills with one Pipeline job per branch, tag or PR that contains the Jenkinsfile. Each job builds its own branch's Jenkinsfile.
- The scan log (
job/NAME/indexing/consoleText) shows exactly why each branch did or didn't get a job and a build: "found", "Met criteria", "Changes detected". - Branch names with
/become job names with%2Fand URLs with%252F.BRANCH_NAMEkeeps the real name. when { branch },buildingTag()andchangeRequest()read the multibranch variables. In a plain Pipeline job they never match becauseBRANCH_NAMEis unset.- The orphaned item strategy removes or disables jobs for deleted branches. Tags are discovered but not built automatically.
- PRs need the host's branch source plugin. Fork PRs from untrusted authors run the target branch's Jenkinsfile.
- Webhooks (
/github-webhook/, or/git/notifyCommit?url=…&token=…from a git hook) make builds start within seconds. Keep a daily periodic scan as a safety net.
Exercises
Change the Jenkinsfile so the Package stage also runs for branches named hotfix/....
Push the change to main, then create and push hotfix/2.4.1. Confirm from the hotfix
build's log that Package ran and the deploy stages were skipped.
Show Solution
Add a third condition to the anyOf:
when { anyOf { branch 'main'; branch 'release/*'; branch 'hotfix/*' } }After pushing main and the new branch, the hook triggered a scan, the scan created the
hotfix%2F2.4.1 job, and its first build showed:
[Pipeline] { (Package)
+ tar czf inventory-api-cb951d4.tar.gz app.py
Stage "Deploy to dev" skipped due to when conditional
Stage "Deploy to staging" skipped due to when conditional
Stage "Publish release" skipped due to when conditional
Stage "PR checks" skipped due to when conditional
Finished: SUCCESSThe hotfix branch got the change straight away because it was created from the updated
main. Any other branch keeps its own Jenkinsfile until it's merged.
Write a curl and jq command that prints every job in the multibranch project with
its real branch name (with /, not %2F), last build number and result.
Show Solution
curl -s -u "admin:$JENKINS_PASSWORD" \
"http://localhost:8090/job/inventory-api/api/json?tree=jobs%5Bname,lastBuild%5Bnumber,result%5D%5D" \
| jq -r '.jobs[] | "\(.name | gsub("%2F"; "/"))\t#\(.lastBuild.number)\t\(.lastBuild.result)"'develop #2 SUCCESS
hotfix/2.4.1 #1 SUCCESS
main #2 SUCCESS
release/2.3 #1 SUCCESS
v2.4.0 #1 SUCCESSThe tree parameter asks Jenkins for only the fields you need, which matters on a folder
with hundreds of branches. %5B and %5D are [ and ].
Change the orphaned item strategy to keep at most 2 dead branch jobs for at most 7 days.
Then delete hotfix/2.4.1 from the server. What does the scan log say, and what state is
the job left in?
Hint
Fetch the current config with get-job, edit daysToKeep and numToKeep, and send it
back with update-job.
Show Solution
jcli get-job inventory-api > current.xml
sed -e 's#<daysToKeep>-1</daysToKeep>#<daysToKeep>7</daysToKeep>#' \
-e 's#<numToKeep>-1</numToKeep>#<numToKeep>2</numToKeep>#' current.xml > keep.xml
jcli update-job inventory-api < keep.xml
git push -q origin --delete hotfix/2.4.1The scan triggered by the push ended with:
Evaluating orphaned items in inventory-api
Will not remove hotfix/2.4.1 as it is only #1 in the list
Will not remove hotfix/2.4.1 because it is new
Finished: SUCCESSThe job list showed ["hotfix%2F2.4.1","disabled"]. Jenkins keeps the job and its
history, but disables it so it can't be built again. A later scan removes it once it's
older than 7 days or once more than two dead jobs pile up.
Write scan-now.sh JOB. It should start a scan of a multibranch project through the REST
API, wait until the scan finishes, and print only the interesting lines of the scan log.
Test it by pushing a commit with the hook disabled, so that only your script can pick the
change up.
Show Solution
#!/usr/bin/env bash
# scan-now.sh JOB - start a multibranch scan, wait for it, print the interesting lines
set -euo pipefail
job="$1"
base="${JENKINS_URL%/}/job/$job"
auth="$JENKINS_USER:$JENKINS_PASSWORD"
jar=$(mktemp)
trap 'rm -f "$jar"' EXIT
# Password logins need a CSRF crumb for POST requests (API tokens do not).
crumb=$(curl -fsS -c "$jar" -u "$auth" "${JENKINS_URL%/}/crumbIssuer/api/json" |
jq -r '.crumbRequestField + ":" + .crumb')
curl -fsS -o /dev/null -b "$jar" -u "$auth" -H "$crumb" -X POST "$base/build?delay=0"
sleep 2
for _ in $(seq 1 60); do
log=$(curl -fsS -u "$auth" "$base/indexing/consoleText")
if grep -q '^Finished:' <<<"$log"; then
grep -E '^(Started|Changes detected|Scheduled build|Will remove|Finished:)' <<<"$log"
exit 0
fi
sleep 2
done
echo "scan of $job did not finish within 2 minutes" >&2
exit 1With the hook moved aside, I committed to develop, pushed, and ran the script:
JENKINS_URL=http://localhost:8090/ JENKINS_USER=admin ./scan-now.sh inventory-apiStarted by user admin
Changes detected: develop (148182fcc9325e623ebba0cc60038c02806b0800 → 04722aebb55f50785f6d0e2bb5d5797906e3ef1f)
Scheduled build for branch: develop
Changes detected: v2.4.0 (null → df1488aaeb598e7cdd55f073a9fc10618da17bdf)
Finished: SUCCESSThe Started by user admin line shows you're reading the new scan's log, not the previous
one. The v2.4.0 (null → …) line is the tag job from earlier, which Jenkins re-reports
because a tag job that was never built automatically has no recorded revision. It still
wasn't built. If you authenticate with an API token instead of a password, you can drop
the crumb and the cookie jar.
Tested with: Jenkins 2.580.2 (OpenJDK 21) on Ubuntu 24.04 (WSL2), Git 2.43.0, Pipeline: Multibranch 842.v3a_b_59b_57b_e6e, Branch API 2.1303.v9f3b_95dc329d, Git plugin 5.11.0 and GitHub Branch Source 1986.v801e40614d00, November 2026.
Every scan, build, hook push and REST call above was run against a local bare repository. Git output lines starting with > git are trimmed where marked. The ALLOW_LOCAL_CHECKOUT error was reproduced on a second, throwaway controller of the same version started without that property. No PR builds were run, because they need a hosted GitHub, GitLab or Bitbucket. The GitHub project and organization folder were only created (disabled) to confirm the configuration is accepted, and the GitHub-side webhook settings were not clicked through. UI labels were checked against the plugins' own configuration pages on this controller. Paths from the test machine's scratch directories are shown shortened to /home/sid/…. Job, node and credential names are shown without the test run's prefixes.
Written by The Dark Debugger
Engineer documenting the things that took me too long to learn, so they take you less time. Every Jenkins tutorial here is tested end to end before publishing.