In this tutorial we'll create a Jerrybuild configuration and configure a simple build job.

In a production environment, you'd install Jerrybuild system-wide instead, but for this tutorial we'll be working from the git repo.

Preparation

Create a clone of the repo:

$ git clone https://github.com/fboender/jerrybuild.git
$ cd jerrybuild

Now we'll create a directory structure to hold our tutorial example:

$ mkdir tutorial
$ mkdir tutorial/jobs.d
$ mkdir tutorial/status
$ mkdir tutorial/work_dir

A little explanation is in order. The jobs.d directory will hold configuration files for our various build jobs. The status dir is where Jerrybuild will keep some metadata of each job's status. The work_dir is where Jerrybuild will be doing the actual building.

Main configuration file

All of these can be configured in the main configuration file. So let's create that:

$ vi tutorial/jerrybuild.cfg

And put the following content in it:

[server]
listen = 127.0.0.1
port = 5281
log_level = INFO
status_dir = tutorial/status/
work_dir = tutorial/work_dir/
keep_jobs = 40

%include jobs.d/*.cfg

This configuration file has no comments, but you can check the Example jerrybuild.cfg for an annotated example.

Job definition configuration

Now it's time to define our first job. Jerrybuild has built-in support for various "providers", such as Github and Gogs. For now we'll use the "generic" provider, which doesn't integrate with any specific git hosting site, but can be called by any webhook callback.

Create a job configuration file in the jobs.d directory:

$ vi tutorial/jobs.d/tutorial.cfg

Put the following contents in it:

[job:tutorial]
desc = The jerrybuild tutorial
url = /hook/tutorial
provider = generic
cmd = sh tutorial.sh

Again, we've left out the comments. See the Example job configuration for an annotated example.

The url settings specifies by which URL this job can be called. It is relative to the web root. So if we start Jerrybuild on http://127.0.0.1:5281/, the job's hook url becomes http://127.0.0.1:5281/hook/tutorial.

What should happen when the hook is called is defined by the cmd setting. In this case, we tell Jerrybuild that it should call a script called tutorial.sh. This is relative to the main configuration's work_dir.

Create build script

Let's create that shell script!

$ vi tutorial/work_dir/tutorial.sh

The file should look like this:

#!/bin/sh

echo "Hello!"

Run Jerrybuild and start a build

Time to start Jerrybuild for the first time:

$ python3 ./jerrybuild.py tutorial/jerrybuild.cfg

The webserver should now be running on http://127.0.0.1:5281. Try opening it in the browser. You should see a single job called "tutorial" with a grey "Never built" status.So let's try to build it. We'll use curl to call the webhook:

$ curl http://127.0.0.1:5281/hook/tutorial

If you refresh the webpage, you should now see that the job's status has changed to a green "Passed" button. You can click on that button to go to that job's output page. There you can see some job metadata, such as when it was built, its exit code and the output of the job. It should say "Hello!".

Clone / pull a repo

Let's do something a little more useful though. We'll have the script clone an actual git repository or, if it's already been cloned, do a pull & rebase.

Replace the contents of tutorial/work_dir/tutorial.sh with:

#!/bin/sh
set -x

REPO="https://github.com/fboender/jerrybuild.git"

if [ ! -d "jerrybuild" ]; then
    git clone "$REPO"
else
    cd jerrybuild
    git pull --rebase
    cd ..
fi

cd jerrybuild
ls -la

That's right, we're going to clone jerrybuild itself for this tutorial. Hope that's not confusing!

Note that this will work because Jerrybuild is a public repo. If you need to clone private repos, have a look at the Cookbook: Clone a repo with a custom private key section.

Go back Jerrybuild's web interface and find the status output of the previous job we ran. You'll see a blue "Rerun as new job" button. Clicking that is basically the same as running that curl command we saw earlier.

Go ahead and click that, and you should see the repo being cloned and then a listing of its files. If you click that same button again, you should see that it's doing a git pull --rebase rather than a new clone.

Github provider

You'll probably want to trigger builds from an external source repository such as Github. Jerrybuild has providers for that. The providers have some extra logic to handle webhook callbacks from specific external repositories. For example, Github lets you configure a secret for your webhook that ensures the request truly came from Github. Jerrybuild's github provider automatically verifies that secret.

This part of the tutorial assumes you've got Jerrybuild running on a public server. We'll be using "https://build.electricmonk.nl" as an example. You can check the Cookbook on how to serve Jerrybuild behind a public webserver.

We'll also assume you've generated and configured a deploy ssh key. Again, check the Cookbook for instructions on how to do that.

We configure a job with the github provider.

$ vi tutorial/jobs.d/github-example.cfg

It should look like this:

[job:github-example]
url = /hook/github-example
provider = github
secret = eiHeik3xeoGh6ieg8oZingah
cmd = sh ./github-example.sh

The secret is just a randomly generated password.

Next we create the github-example.sh.

$ vi tutorial/work_dir/github-example.sh

Add the following contents to that script:

#!/bin/sh

set -x

export PROJECT="jerrybuild"
export REPO="https://github.com/fboender/$PROJECT.git"
export SSH_KEY="/var/lib/jerrybuild/deploy_keys/build_rsa"
export GIT_SSH_COMMAND="ssh -i $SSH_KEY -o IdentitiesOnly=yes"

if [ ! -d "$PROJECT" ]; then
    git clone "$REPO"
else
    cd $PROJECT
    git reset --hard
    git clean -f -d
    git fetch
    cd ..
fi

cd $PROJECT
git checkout -q $commit
ls -la

Some explanation may be in order...

The script first exports some variables. To reduce repetition, we define the PROJECT. We then point the SSH_KEY variable to the private deploy key that has access to our project in Github. To instruct Git to use that private key, we set the GIT_SSH_COMMAND variable.

Next we either clone the repo if it wasn't cloned yet. Otherwise, we completely reset the repo to a clean state, and then fetch any new remote changes from Github.

Finally, we change the current dir to the repo, and check out $commit. This is a special variable set by the github provider. It contains the SHA1 commit that triggered this webhook callback in Github.

For demonstration purposes we just list the contents of the repository. This is normally where you perform your automated tests and other build instructions.

We now need to restart Jerrybuild or send it the SIGHUP signal. Either of those cause Jerrybuild to reload the configuration so that it spots the new job.

$ ps auxf | grep jerrybuild
jerrybu+  3483  0.0  1.8 507164 37300 ?        Sl   Jun08  40:46 /usr/bin/python /usr/bin/jerrybuild [...]

$ kill -HUP 3483

Jerrybuild's log file should show:

2020-10-03 09:24:22,015:INFO:Reloading configuration file

Now it's time to configure the webhook in Github. Go to the projects Settings and then click on "Webhooks". Add a new webhook with the button at the top right.

Enter the payload URL, which should be something like https://build.electricmonk.nl/hook/github-example. Change the Content-type to application/json. Copy-paste the secret from the job's configuration file into the Secret box. For "Which events would you like to trigger this webhook", we'll keep the default "Just the push event" selection.

Save the webhook by clicking the "Add webhook" button. Github should now send a "ping" event to test the webhook. Jerrybuild understands those ping events, so everything should be fine.

Now when you push a new commit to the repository, it should automatically call the Jerrybuild build job!