Guided Tour

User Handbook (PDF)

Resources

Recent Tutorials
View all tutorials

Developers

Reference Documentation
Index

Pipeline

This chapter will cover all aspects of Jenkins Pipeline, from running Pipelines to writing Pipeline code, and even extending Pipeline itself.

This chapter is intended to be used by Jenkins users of all skill levels, but beginners may need to refer to some sections of "Using Jenkins" to understand some topics covered in this chapter.

If you are not yet familiar with basic Jenkins terminology and features, start with Getting Started with Jenkins.

What is Pipeline?

Jenkins Pipeline is a suite of plugins which supports implementing and integrating continuous delivery pipelines into Jenkins. Pipeline provides an extensible set of tools for modeling simple-to-complex delivery pipelines "as code" via the Pipeline DSL. [1]

Typically, this "Pipeline as Code" would be written to a Jenkinsfile and checked into a project’s source control repository, for example:

Jenkinsfile (Declarative Pipeline)
pipeline {
    agent any (1)

    stages {
        stage('Build') { (2)
            steps { (3)
                sh 'make' (4)
            }
        }
        stage('Test'){
            steps {
                sh 'make check'
                junit 'reports/**/*.xml' (5)
            }
        }
        stage('Deploy') {
            steps {
                sh 'make publish'
            }
        }
    }
}
Toggle Scripted Pipeline (Advanced)
1 agent indicates that Jenkins should allocate an executor and workspace for this part of the Pipeline.
2 stage describes a stage of this Pipeline.
3 steps describes the steps to be run in this stage
4 sh executes the given shell command
5 junit is a Pipeline step provided by the JUnit plugin for aggregating test reports.

Why Pipeline?

Jenkins is, fundamentally, an automation engine which supports a number of automation patterns. Pipeline adds a powerful set of automation tools onto Jenkins, supporting use cases that span from simple continuous integration to comprehensive continuous delivery pipelines. By modeling a series of related tasks, users can take advantage of the many features of Pipeline:

  • Code: Pipelines are implemented in code and typically checked into source control, giving teams the ability to edit, review, and iterate upon their delivery pipeline.

  • Durable: Pipelines can survive both planned and unplanned restarts of the Jenkins master.

  • Pausable: Pipelines can optionally stop and wait for human input or approval before continuing the Pipeline run.

  • Versatile: Pipelines support complex real-world continuous delivery requirements, including the ability to fork/join, loop, and perform work in parallel.

  • Extensible: The Pipeline plugin supports custom extensions to its DSL [1] and multiple options for integration with other plugins.

While Jenkins has always allowed rudimentary forms of chaining Freestyle Jobs together to perform sequential tasks, [2] Pipeline makes this concept a first-class citizen in Jenkins.

Building on the core Jenkins value of extensibility, Pipeline is also extensible both by users with Pipeline Shared Libraries and by plugin developers. [3]

The flowchart below is an example of one continuous delivery scenario easily modeled in Jenkins Pipeline:

realworld pipeline flow
Figure 1. Pipeline Flow

Pipeline Terms

Step

A single task; fundamentally steps tell Jenkins what to do. For example, to execute the shell command make use the sh step: sh 'make'. When a plugin extends the Pipeline DSL, that typically means the plugin has implemented a new step.

Node

Most work a Pipeline performs is done in the context of one or more declared node steps. Confining the work inside of a node step does two things:

  1. Schedules the steps contained within the block to run by adding an item to the Jenkins queue. As soon as an executor is free on a node, the steps will run.

  2. Creates a workspace (a directory specific to that particular Pipeline) where work can be done on files checked out from source control.

Depending on your Jenkins configuration, some workspaces may not get automatically cleaned up after a period of inactivity. See tickets and discussion linked from JENKINS-2111 for more information.
Stage

stage is a step for defining a conceptually distinct subset of the entire Pipeline, for example: "Build", "Test", and "Deploy", which is used by many plugins to visualize or present Jenkins Pipeline status/progress. [4]

1. Domain-Specific Language
2. Additional plugins have been used to implement complex behaviors utilizing Freestyle Jobs such as the Copy Artifact, Parameterized Trigger, and Promoted Builds plugins
3. GitHub Organization Folder plugin
4. Blue Ocean, Pipeline Stage View plugin
Index