How to Install and Configure GoCD on Ubuntu 16.04

• Prerequisites
• Install Java
• Install GoCD
• Configure Block Storage
• Configure GoCD
• Setup Authentication
• Securing GoCD with Let's Encrypt SSL
• Installing GoCD Agent

GoCD is an open source continuous delivery and automation system. It allows you to model complex workflows using its parallel and sequential execution. Its value stream map allows you to easily visualize a complex workflow with ease. GoCD lets you easily compare two builds and deploy any version of the application you want. The GoCD ecosystem consists of GoCD server and GoCD agent. GoCD is responsible for controlling everything such as running the web-based user interface and managing and providing jobs to the agent. Go agents are responsible for running the jobs and deployments.

Prerequisites

• A Vultr Ubuntu 16.04 server instance with at least 2GB RAM.
• A sudo user.
• A domain name pointed towards the server.

For this tutorial, we will use 192.168.1.1 as the public IP address and gocd.example.com as the domain name pointed towards the Vultr instance. Please make sure to replace all occurrences of the example domain name and IP address with the actual one.

Update your base system using the guide How to Update Ubuntu 16.04. Once your system has been updated, proceed to install Java.

Install Java

GoCD requires Java version 8 and supports both Oracle Java and OpenJDK. Add the Ubuntu repository for Oracle Java 8.

sudo add-apt-repository --yes ppa:webupd8team/java
sudo apt update

Install Oracle Java.

sudo apt -y install oracle-java8-installer

Check the version.

java -version

You will see the following output.

user@vultr:~$ java -version
java version "1.8.0_161"
Java(TM) SE Runtime Environment (build 1.8.0_161-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)

Set the default path for Java by installing the following package.

sudo apt -y install oracle-java8-set-default

You can verify if JAVA_HOME is set by running.

echo $JAVA_HOME

You will see.

user@vultr:~$ echo $JAVA_HOME
/usr/lib/jvm/java-8-oracle

If you see no output at all, you will need to log out from the current shell and log back in.

Install GoCD

Install GoCD's official repository into the system.

echo "deb https://download.gocd.org /" | sudo tee /etc/apt/sources.list.d/gocd.list
curl https://download.gocd.org/GOCD-GPG-KEY.asc | sudo apt-key add -
sudo apt update

Install the GoCD server in your system.

sudo apt install -y go-server

Start GoCD and enable it to automatically start at boot time.

sudo systemctl start go-server
sudo systemctl enable go-server

Before we access the GoCD dashboard, let's create a new directory to store the artifacts. Artifacts can be stored on the same disk in which the operating system and the applications are installed. Alternatively, you can use a dedicated disk or block storage drive to store the artifacts.

If you wish to use the same disk to store the artifacts, just create a new directory and provide the ownership to the GoCD user.

sudo mkdir /opt/artifacts
sudo chown -R go:go /opt/artifacts

Configure Block Storage

The GoCD software recommends that you use an additional partition or drive to store the artifacts. In a continuous integration and delivery platform, artifacts are generated very often. The disk space decreases over time when new artifacts are continuously generated. At some stage, your system will run out of free disk space and the services running on your system will fail. To overcome this issue, you can attach a new Vultr block storage drive to store the artifacts. If you still wish to go with the storage of artifacts on the same drive, skip to the "Configure GoCD" section.

Deploy a new block storage drive and attach it to your GoCD server instance. Now create a new partition on the block storage device.

sudo parted -s /dev/vdb mklabel gpt
sudo parted -s /dev/vdb unit mib mkpart primary 0% 100%

Create the file system on the new disk.

sudo mkfs.ext4 /dev/vdb1

Mount the block storage drive.

sudo mkdir /mnt/artifacts
sudo cp /etc/fstab /etc/fstab.backup
echo "
/dev/vdb1 /mnt/artifacts ext4 defaults,noatime 0 0" | sudo tee -a /etc/fstab
sudo mount /mnt/artifacts

Now, run df and you will see the new block storage drive mounted on /mnt/artifacts.

[user@vultr ~]$ df
Filesystem     1K-blocks    Used Available Use% Mounted on
/dev/vda1       20616252 6313892  13237464  33% /

...
/dev/vdb1       10188052   36888   9610596   1% /mnt/artifacts

Provide ownership of the directory to the GoCD user.

sudo chown -R go:go /mnt/artifacts

Configure GoCD

Now you can access the GoCD dashboard on http://192.168.1.1:8153. To access the GoCD dashboard on a secured connection, access https://192.168.1.1:8154. You will get an error showing that the certificates are not valid. You can safely ignore the error as the certificates are self-signed. For security purposes, you should always use the dashboard over a secured connection.

Before you set up a new pipeline, navigate to "Admin >> Server Configuration" from the top navigation bar.

Input the URL to your unsecured site in the "Site URL" field and the secured site in the "Secure Site URL" field.

Next, provide your SMTP server details to send email notifications from GoCD.

Finally, provide the path to the location where you wish to store the artifacts. If you have chosen to store the artifacts on the same disk as the operating system, enter /opt/artifacts; if you have chosen to attach a block storage drive, then you can enter /mnt/artifacts.

You can also configure GoCD to auto-delete the old artifacts. Configure the next option according to your disk size. However, the auto-delete option does not take a backup of your old artifacts. To manually take a backup and then delete the old artifacts, disable auto delete by choosing the "Never" option for the "Auto delete old artifacts" option.

You will need to restart the GoCD server so that the new changes are applied.

sudo systemctl restart go-server

Setup Authentication

By default, the GoCD dashboard is not configured to use any kind of authentication, but it supports authentication using a password file and LDAP. In this tutorial, we will set up password-based authentication.

Note: Setting up authentication is an optional step, but it is strongly recommended for public facing servers, such as Vultr.

Install Apache tools so that we can use the htpasswd command to create an encrypted password file.

sudo apt -y install apache2-utils

Create a password file with the htpasswd command using Bcrypt encryption.

sudo htpasswd -B -c /etc/go/passwd_auth goadmin

Provide the password for the user twice. You will see the following output.

[user@vultr ~]$ sudo htpasswd -B -c /etc/go/passwd_auth goadmin
New password:
Re-type new password:
Adding password for user goadmin

You can add as many users as you want using the same command above, but remove the -c option. The -c option will replace the existing file, replacing old users with the new user.

sudo htpasswd -B /etc/go/passwd_auth gouser1

Now that we have created the password file, access the GoCD dashboard again. Navigate to "Admin >> Security >> Authorization Configurations" from the top navigation bar. Click on the Add button and provide any ID. Choose "Password File Authentication Plugin for GoCD" for the plugin ID and direct the path to the password file. Now click on the "Check Connection" button to verify that GoCD can use the password file for authentication.

Finally, save the authentication method. Reload the dashboard and it will automatically log you out. You will see a login screen now. Log in using the credentials created earlier.

You will need to promote the administrator user manually, otherwise, all the users will have administrator privileges. Navigate to "Admin >> User Summary" from the top navigation bar.

Now select the admin user you've created and click on the "Roles" drop-down. Promote the user to the only administrator by selecting the "Go System Administrator" checkbox.

To add the users in GoCD created in the password file, click on the "ADD" button and search for the user to add them. Users are also automatically added to the GoCD dashboard on their first login. Obviously, for users to log in, they must be added to the password file we have created earlier.

Securing GoCD with Let's Encrypt SSL

By default, GoCD listens to ports 8153 and 8154 on secure connections. Though port 8154 provides a secure connection to the application, it also displays browser errors as it uses a self-signed certificate. In this section of the tutorial, we will install and secure Nginx with a Let's Encrypt free SSL certificate. The Nginx web server will work as a reverse proxy to forward the incoming requests to GoCD's HTTP endpoint.

Install Nginx.

sudo apt -y install nginx

Start Nginx and enable it to automatically start at boot time.

sudo systemctl start nginx
sudo systemctl enable nginx

Add the Certbot repository.

sudo add-apt-repository --yes ppa:certbot/certbot
sudo apt-get update

Install Certbot, which is the client application for Let's Encrypt CA.

sudo apt -y install certbot

Note: To obtain certificates from Let's Encrypt CA, the domain for which the certificates are to be generated must be pointed towards the server. If not, make the necessary changes to the DNS records of the domain and wait for the DNS to propagate before making the certificate request again. Certbot checks the domain authority before providing the certificates.

Generate the SSL certificates.

sudo certbot certonly --webroot -w /var/www/html -d gocd.example.com

The generated certificates are likely to be stored in /etc/letsencrypt/live/gocd.example.com/. The SSL certificate will be stored as fullchain.pem and private key will be stored as privkey.pem.

Let's Encrypt certificates expire in 90 days, so it is recommended to set up auto-renewal of the certificates using cron jobs.

Open the cron job file.

sudo crontab -e

Add the following line to the end of the file.

30 5 * * * /usr/bin/certbot renew --quiet

The above cron job will run every day at 5:30 AM. If the certificate is due for expiration, it will automatically renew.

Create a new configuration file for the GoCD web interface.

sudo nano /etc/nginx/sites-available/gocd

Populate the file.

upstream gocd {
server 127.0.0.1:8153;
}

server {
    listen 80;
    server_name gocd.example.com;
    return 301 https://$host$request_uri;
}

server {
    listen 443;
    server_name gocd.example.com;

    ssl_certificate           /etc/letsencrypt/live/gocd.example.com/fullchain.pem;
    ssl_certificate_key       /etc/letsencrypt/live/gocd.example.com/privkey.pem;

    ssl on;
    ssl_session_cache  builtin:1000  shared:SSL:10m;
    ssl_protocols  TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers HIGH:!aNULL:!eNULL:!EXPORT:!CAMELLIA:!DES:!MD5:!PSK:!RC4;
    ssl_prefer_server_ciphers on;

    access_log  /var/log/nginx/gocd.access.log;

location / {
        proxy_pass http://gocd;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;
        proxy_set_header X-NginX-Proxy true;
        proxy_redirect off;
    }
location /go {
    proxy_pass http://gocd/go;
    proxy_http_version 1.1;
    proxy_set_header Upgrade websocket;
    proxy_set_header Connection upgrade;
    proxy_read_timeout 86400;
    }
  }

Activate the configuration file.

sudo ln -s /etc/nginx/sites-available/gocd /etc/nginx/sites-enabled/gocd

Restart the Nginx web server to implement the change in configuration.

sudo systemctl restart nginx

Now you can access the GoCD dashboard at https://gocd.example.com. Log into your dashboard using the administrator credentials and navigate to "Admin >> Server Configuration" from the top navigation bar.

Set the "Site URL" and "Secure Site URL" to https://gocd.example.com.

Installing GoCD Agent

In the GoCD continuous integration environment, GoCD agents are the workers that are responsible for the execution of all the tasks. When a change in the source is detected, the pipeline is triggered and the jobs are assigned to available workers for execution. The agent then executes the task and reports the final status after execution.

To run a pipeline, at least one agent must to be configured. Proceed to install the GoCD agent on the GoCD server.

Since we have already imported the GoCD repository into the server, we can directly install Go Agent.

sudo apt install -y go-agent

Now, start the GoCD server and enable it to automatically start at boot time.

sudo systemctl start go-agent
sudo systemctl enable go-agent

The GoCD agent running on the localhost is automatically enabled when detected.