Integrations, code, and the oxford comma

Part One: Virtualization... and setting up RancherOS, Docker, and Drone

Note:

This is Part One, in a multi-part series.
Here are the other parts that you should probably check out as well:

Once I settled on a CI system, I had to set it up, right? I currently have a box in my Homelab setup for virtualization. It’s a natural fit, especially when testing software out. You can spin up a new environment quickly (and trash it just as fast). I’ve got at least 10 environments that are powered down right now from past ideas and such. It’s pretty easy to spin them back up at any time. I don’t regret my decision to move from software virtualization to hardware one bit.

RancherOS

I have toyed around with various (tiny) images to use when setting up a new VM. The obvious goto is Alpine Linux. Based on BusyBox, it has a hardened Linux kernel and a solid history of just working. It can live on tiny devices (or containers). It has the ability to run completely in RAM (if you want it to). Opting for a traditional disk install in a container is one of my goto options when setting up a solid, slim container using Alpine.

That said, for this one, I decided that I wanted to needlessly complicate explore my options on getting a Docker container running as close to the core of the container as possible. In comes RancherOS. RancherOS is a minimal OS that’s built to run containers. I mean, one of the key takeaways from their site is:

RancherOS includes the bare minimum amount of software needed to run Docker.

That was it. I was sold. Hell, I didn’t even need to read past that. I downloaded the image to my ISO store on my ESXi server and was off. (They even have a VMware .vdmk available, if you want to run it locally to try it out.) They offer a decent set of documentation, especially for specific environments like Amazon EC2, Digital Ocean VPSes and ESXi, which I’ll focus on here.

The CLOUD

Ah yes, the cloud. That nebulous term that really just means “someone else’s computer, likely sitting in a datacenter.” Well, one of the key things that makes the RancherOS unique is that they use a cloud config file (cloud-config.yml) to get the system setup on your host machine. Great. Glad that I had no idea how to make this work, even after reading through the documentation…

I don’t recall that this was easy when setting up the VM, but I’ll be damned if I remember how many tries it took me to get the cloud-config.yml in the right format so that it would boot the system into a working RancherOS instance. I’ll spare you the rest of the misery that was making the config work and list what I settled on:

hostname: rancheros #Set the hostname for the system
rancher:
  console: ubuntu #Switches to the Ubuntu console (bash) on boot
  environment:
    EXTRA_CMDLINE: /init
  network:
    dns:
      nameservers:
      - 8.8.8.8  #Google DNS
      - 8.8.4.4
    interfaces:
      eth*:
        dhcp: false #Set this bad boy to a static IP address
      eth0: #Change this to your network interface, if not eth0
        address: 192.168.2.10/24 #The IP address of the VM in EXSi
        gateway: 192.168.2.1
        mtu: 1500
  services_include:
    open-vm-tools: true #Not sure if this was necessary, but why not
  state:
    dev: LABEL=RANCHER_STATE #Saves a persistent state partition
    wait: true               #across boots for user data
ssh_authorized_keys:
  - ssh-rsa AAAABBBCCCDDD...DE211DD== #SSH key

Take that file and put it somewhere that your VM can get to it. In my case, I threw it up on S3 and made the file have public-read access, so that the RancherOS could process it during setup.

Once I setup my VM in ESXi, I booted up into the RancherOS ISO image in my remote console. (Of note, it’s recommended that you boot with at least 1280mb of memory, from the ISO or on a disk.) The ISO will automatically log you in as the rancher user and expect that you have a cloud-config.yml file ready to configure the system.

Running ros install should take you through the process. Provided that your cloud-config.yml is formatted correctly, everything should go as planned and you’ll reboot (from your VM disk) into a working RancherOS system. There’s probably some other configuration hoops that I had to jump through, but I won’t bore you with those details. Check out the documentation for further help.

Docker it up

To visualize how the layers of the RancherOS is setup, here’s an image that I took from their github project page. It’s basically a Docker sandwich:

The System Docker

Running sudo system-docker ps -a will give you a list of active system-level docker containers on the box. Here’s a (formatted, pretty) list of what I see when I run that command:

You can see that the basic Linux services like ntp, udev, and the system-cron are all in their own containers. Those listed will all get launched at boot, and other system services that aren’t needed, but called from occasionally from cron (like logrotate) will run as necessary, per their schedule. Other containers, like the user-volumes container, aren’t started because I didn’t specify that I wanted to use them in my setup. They’re ready to go if I want to configure them, however.

Services that I’ve added to the system-docker, like portaineragent, show up here as well. Running the Portainer Agent as a system-docker container gives me access to the user-level docker containers, and if I need, I can log in to the Portainer admin and view system-docker containers. It’s pretty slick.

User Docker

Running docker ps -a gives you a list of active user-level docker containers. As you can see, this list is much smaller than the system-docker list:

I currently only have one running container on the system, drone-github. Since you currently can only link Drone to a single Git account, I have this one setup to link to my Github account. The container that’s stopped is the one that links up to my (currently turned off) local Gitlab container install (in another VM.) Looks like I haven’t used the Gitlab version in awhile, especially since I’ve since updated to a much newer version of Drone (with the drone/drone:latest tag) for my Github container.

Enter the Drone

So, let’s get to the actual Drone install… You should follow the instructions on the Drone site (I used the instructions for a Github, single machine install for my application). Be sure to create the OAuth application before you get to any of the Docker setup below. We’ll be installing the current version of Drone (release version v1.2.0, as of June 2019) via Docker, dude.

Since don’t have docker-compose installed on RancherOS or my custom console, I went about installing Drone via a one-liner docker command:

docker run \
  --volume=/var/run/docker.sock:/var/run/docker.sock \
  --volume=/var/lib/drone:/data \
  --env=DRONE_GITHUB_SERVER=https://github.com \
  --env=DRONE_GITHUB_CLIENT_ID=XXXXX \
  --env=DRONE_GITHUB_CLIENT_SECRET=XXXXX \
  --env=DRONE_RUNNER_CAPACITY=2 \
  --env=DRONE_SERVER_HOST=drone.yoursite.com \
  --env=DRONE_SERVER_PROTO=https \
  --env=DRONE_TLS_AUTOCERT=false \
  --publish=80:80 \
  --publish=443:443 \
  --restart=always \
  --detach=true \
  --name=drone-github \
  drone/drone:latest

Let’s break down the configuration real quick:

  • --volume=/var/run/docker.sock:/var/run/docker.sock - Maps the Docker socket to /var/run/docker.sock inside of the ROS containers

  • --volume=/var/lib/drone:/data - Maps the /var/lib/drone volume as the /data directory inside of the container

  • --env=DRONE_GITHUB_SERVER=https://github.com - The URL for the Github server

  • --env=DRONE_GITHUB_CLIENT_ID=XXXXX - Your Github Client ID (OAuth for your app. Register a new application on Github here to get this)

  • --env=DRONE_GITHUB_CLIENT_SECRET=XXXXX - Your Github Client Secret (OAuth for your app. Register a new application on Github here to get this)

  • --env=DRONE_RUNNER_CAPACITY=2 - The amount of concurrent runners. Disabled if not set

  • --env=DRONE_SERVER_HOST=drone.yoursite.com - Your site’s URL

  • --env=DRONE_SERVER_PROTO=https - Sets the URL protocol of your site

  • --env=DRONE_TLS_AUTOCERT=false - Automatically generates an SSL certificate using Lets Encrypt, and configures the server to accept HTTPS requests. Optional, disabled by default

  • --publish=80:80 - Publishes port 80 on the container to port 80 to the host

  • --publish=443:443 - Publishes port 443 on the container to port 443 to the host

  • --restart=always - Sets the container restart policy to always

  • --detach=true - Starts the container in detached mode

  • --name=drone-github - Gives the container a pretty name

  • drone/drone:latest - Specifies the image that the Docker will pull for use from Docker Hub

Connect to your RancherOS and run the docker command (above, customize for your usage) in your terminal. You should see the image download and start up. You can make sure that it’s running with docker ps -a. Once you make sure that it’s up, go to https://drone.yoursite.com and check it out. If it prompts you for auth with Github, you’re all set! Next time, we’ll get into how to actually use this thing.

Continued in Part Two »