Boost Your CI/CD with Jenkins: Setting Up Agents and Creating Pipelines

What is Jenkins?

Before we delve into agents, it is important to briefly revisit what Jenkins is. Jenkins is a robust tool for developers, designed to automate repetitive tasks such as testing and deploying code, thereby streamlining the development process. Consider Jenkins as an intelligent assistant that manages all the tedious tasks on your behalf.

Jenkins Master (Server):

Role: The boss that manages everything.
Tasks: Schedules jobs, monitors their status, and serves the Jenkins user interface (UI).
Think of it as: A conductor of an orchestra, making sure everyone is playing their part.

Jenkins Agent :

Role: The worker that does the actual job.
Tasks: Executes tasks defined by the Jenkins jobs.
Think of it as: A musician in the orchestra, playing the notes assigned by the conductor.

Why Do We Need Agents?

For small projects, having just the Jenkins master might be enough. But as projects grow and you have more tasks, it's better to have multiple agents to share the workload. This way, jobs get done faster and more efficiently.

Let's Set Up an Agent!

To set up an agent, we'll use an AWS EC2 instance (a virtual machine in the cloud) with Ubuntu 22.04. Here's a step-by-step guide:

Step 1: Create an AWS EC2 Instance

Launch a New Instance:
- Go to the AWS Management Console.
- Launch a new EC2 instance with Ubuntu 22.04.
- Choose an instance type like t2.micro (good for small tasks).
- Configure security settings to allow SSH access.
Connect to Your Instance:
- Use SSH to connect to your instance. Open your terminal and type:
```
    ssh -i /path/to/your-key.pem ubuntu@your-ec2-instance-public-dns
```

Step 2: Install Java and Docker on Your Agent

Install Java:
- Update your package list

      sudo apt update

Install Java:
```
    sudo apt install openjdk-11-jdk -y
```

Install Docker:

Install Docker with:
```
    sudo apt install docker.io -y
```

Start Docker:

    sudo systemctl start docker
    sudo systemctl enable docker

Step 3: Connect the Agent to Jenkins Master

Generate SSH Key Pair on Master:

On your Jenkins master server, generate an SSH key pair:
```
    ssh-keygen -t rsa -b 4096 -C "jenkins-agent"
```

Copy the public key to your agent:

    ssh-copy-id -i ~/.ssh/id_rsa.pub ubuntu@your-ec2-instance-public-dns

Set Up Jenkins Agent Node:
- In Jenkins UI, go to Manage Jenkins > Manage Nodes and Clouds > New Node.
- Enter node details and set up using SSH with the credentials.
Verify Agent Status:
- Check the "Nodes" section in Jenkins to see if the agent is connected.

Running Jobs on the New Agent

Now that your agent is set up, let's run some jobs!

Step 1: Label Your Agent

Go to Manage Jenkins > Manage Nodes and Clouds > your-agent > Configure.
Add labels to your agent.

Step 2: Run Jobs

Configure your jobs to use the new agent's label.
Run the jobs you built on Day 26 and Day 27 on the new agent.

What is a Pipeline?

A pipeline is like a series of instructions for automating software delivery, from writing code to deploying it to users. Jenkins uses these instructions to ensure each step in the process happens smoothly and consistently, without needing someone to manually manage it all the time.

Imagine it like a conveyor belt in a factory: each stage adds something new until the final product is ready.

Declarative Pipeline

A declarative pipeline is an easy-to-read and write way to define your automated steps. It's structured into several key parts:

Pipeline Block: This is the main block containing the entire pipeline definition.
Agent Block: Defines where the pipeline will run.
Stages Block: Breaks down the pipeline into different stages.
Steps Block: Contains the individual tasks within each stage.

Here's a detailed breakdown of each section:

Pipeline Block

The top-level block that holds everything else. It tells Jenkins that you're defining a pipeline.

Agent Block

Specifies which machine (agent) should execute the pipeline. For simple cases, you can use any to let Jenkins choose any available agent.

Stages Block

Groups the pipeline into different stages. Each stage represents a major phase of your delivery process, like building the code, running tests, and deploying the application.

Steps Block

Lists the individual tasks to be performed within each stage. These tasks are the actual commands or scripts that do the work.

Example:

pipeline {
    agent any
    stages {
        stage('Build') {
            steps {
                echo 'Building...'
            }
        }
        stage('Test') {
            steps {
                echo 'Testing...'
            }
        }
        stage('Deploy') {
            steps {
                echo 'Deploying...'
            }
        }
    }
}

Scripted Pipeline

A scripted pipeline is another way to write pipelines using Groovy (a programming language). This method offers more flexibility and control but requires some programming knowledge.

Key components of a scripted pipeline:

Node Block: Defines where the pipeline will run.
Stage Block: Groups steps that represent a specific part of the process.

Example:

node {
    stage('Hello world') {
        sh 'echo Hello World'
    }
}

Think of scripted pipelines as more powerful but a bit more complex compared to declarative pipelines. They are great for advanced users who need more control and flexibility.

Why Should You Have a Pipeline?

Using Jenkins pipelines offers several benefits:

Code as Configuration: Pipelines are written as code, making them easy to edit, share, and version control.
Resilient: If the server restarts unexpectedly, the pipeline resumes from where it left off.
Interactive: You can pause the pipeline and wait for user input before continuing.
Scalable: Supports running many jobs and using loops for repetitive tasks.

In essence, pipelines help automate and streamline your software development process, making it faster and more reliable.

Pipeline Syntax

Here's a simple example of a declarative pipeline:

pipeline {
    agent any
    stages {
        stage('Hello') {
            steps {
                echo 'Hello, world!'
            }
        }
    }
}

This pipeline has one stage called "Hello" and one step within that stage that prints "Hello, world!" to the console.

Tasks:

Create a New Job:
- Log in to Jenkins and click on "New Item".
- Enter a name for your job, for example, "HelloWorld_Declarative".
- Select "Pipeline" as the job type.
- Click "OK" to create the job.
Configure Pipeline:
- In the job configuration page, scroll down to the "Pipeline" section.
- Choose "Pipeline script from SCM" as the Definition.
- Select your preferred SCM (like Git or SVN) and provide the repository URL.
- Leave the branch as master or main, or specify the branch you want to use.
- Click "Save" to save your configuration.
Write Declarative Pipeline Script:
- In your source code repository, create a file named Jenkinsfile (without any file extension) at the root directory.
- Open Jenkinsfile in a text editor and write the following Declarative Pipeline script

    pipeline {
        agent any
        stages {
            stage('Hello') {
                steps {
                    echo 'Hello, world!'
                }
            }
        }
    }

Commit and Push Changes:
- Save and push the Jenkinsfile to your repository.
Run the Job:
- Go back to Jenkins and navigate to your job.
- Click on "Build Now" to trigger a new build.
- Jenkins will fetch the Jenkinsfile from your repository, execute the pipeline, and display the "Hello, world!" message in the build output.

Conclusion:

Jenkins is an essential tool for automating various stages of the software development lifecycle, from building and testing to deployment. By setting up agents, you can distribute workloads efficiently, ensuring faster and more reliable job execution. Understanding and utilizing Jenkins pipelines, whether declarative or scripted, allows for streamlined, resilient, and scalable automation processes. This not only enhances productivity but also ensures consistency and reliability in software delivery. By following the steps outlined, you can effectively set up and manage Jenkins agents and pipelines, ultimately optimizing your development workflow.