Integrations, code, and the oxford comma

Part Two: Jekyll, and setting up your stuff

Note:

This is Part Two, in a multi-part series.
Here are the other parts that you should probably check out as well:

I really want to stress that I have no idea what I’m doing. I got to this point via complete trial and error, so take that as you will. I have, however, found a process that works for me. While it may not be the smoothest process, I’m going to keep working on it and making it better as time goes on.

Remember back in school when they said that you’d need to use all of this math stuff later in life? Yeah, I didn’t listen and it’s come back to bite me in the ass on several occasions. I use math ALL THE TIME now. I wish I would have paid more attention when I was in Advanced Trig instead of worrying about girls. The nerds had it right all along.

The same goes for documentation. Write it AS you’re working out your issues. That way, you won’t have to go back and figure out what you did the first time around to make things work. I’m learning that lesson the hard way as I return to document things.

Leaving well enough alone

At some point, you’re gonna need to use the command line. That may be now and that may be in one of the steps before this. By default, RancherOS will boot you into it’s default environment, which is fine, but it could be better. I’m used to Debian-based systems, so of course it makes sense it run the Ubuntu environment, which gives you a nifty bash shell and the usual apt package manager that one is used to when working in Debian/Ubuntu systems.

Since I already set this up in my cloud-config.yml, RancherOS boots up into a Ubuntu console. You can actually pick from the following if you don’t like Ubuntu:

  • default (BusyBox)
  • alpine
  • centos
  • debian
  • fedora
  • ubuntu

All consoles except the default (busybox) console are persistent, so take that into account when setting your system up. If you like running apk or want a smaller overhead, Alpine isn’t a bad choice. It’s all up to you.

The MOTD doesn’t need to be useless

I tend to make my MOTD a little less boring looking. It has the added benefit of letting me know just what system I’m logging into in case my two-second attention span makes me forget what I’m doing in the process. (This happens a lot, plus I’ve been known to spend WAY too long messing with custom MOTD scripts.) For this system, I just setup a basic MOTD to show me a pretty login ascii image of the system name:

(You can make your own using the Text to ASCII art generator (TAAG). It’s awesome.)

I’ll go into the details of my prompt and all the other fun terminal stuff in another post (about .dotfiles!) at some point, but the basics of getting this to work are as follows:

Turn off “last login”

I don’t really want to see the last login information that pops up by default. I usually turn this off as one of the first things is customizing a system. I can always check the logs if I really need to know that info. It’s pretty simple, as we’re just gonna edit our ssh configuration real quick:

  1. Open up the config: sudo nano /etc/ssh/sshd_config (Of note, I use nano because I suck at Vim. It’s a work in progress. nano commands are muscle-memory at this point though.)
  2. Find #PrintLastLog yes, uncomment it and change it to PrintLastLog no
  3. Restart the SSH service sudo service sshd restart

And you’re done. Next time you login to the system, the last login information should be gone.

Making the MOTD less ugly

  1. Head over to your motd directory cd /etc/update-motd.d/
  2. Mine had the following motd files: 00-header 10-help-text 50-motd-news 60-unminimize
  3. You can edit to your liking here. Remove or comment out the stuff that you don’t want in each file. In my case, the 10-help-text 50-motd-news 60-unminimize files were pretty worthless to me, so I removed them completely. (Changing ENABLED=1 to ENABLED=0 in the 50-motd-news file also works to disable the news display ONLY. There’s been a bunch of controversy about this file in Ubuntu, but I haven’t seen any of that on Rancher. There’s a ton more info here about motd scripts and how to disable/edit them in other ways.)
  4. Edit the 00-header to your liking. This is where the magic happens.

00-header needs colors, man

I’ll quickly break down what I have in my (modified) 00-header file:

  • Got rid of all of the comments that were in the original file. We don’t need these. You can leave them in there if you really want to, but I didn’t.
  • export TERM=xterm-256color - Tell the bash script to export 256-colors to the prompt
  • The /etc/lsb-release portion is the original script to list the OS release info. You can keep this in or remove it. I keep it in because I like to quickly see what the OS version is when I log in, in case it’s super outdated, it reminds me that I need to update the system with apt.
  • The line variable in there is just a repeatable seperator line with color that I can reuse below.
  • The \e[38;5;051m (and others below) is what gives the line a specific color. In this case, I’m specifying a blue background and black foreground colors. There are a ton of resources out there for figuring out what colors do what, but mostly for me it’s just been trial and error. I’ll have another post about this one too at some point.
  • At the end of each line, I specify \e[0m\n, which is \e[0m to clear the color (which resets the color for default, and sets up the next line) and a \n to insert a new line. If you don’t do this, all of your lines will just jam together and your pretty ascii art will look all messed up.

And a few other notes:

  • \n gives you a new line
  • I’m using printf instead of echo because it works, and I like to be able to specify my own new lines and breaks in the script
  • You could make a variable for your colors pretty easily. I didn’t do that in this file because I made it into a gradient background for the name.
  • I put two spaces before the “ Welcome to…” system message for alignment.
  • I bumped down the \e[0m] (line 17) to the next printf statement, because putting it on the line with the $DISTRIB_DESCRIPTION script messes up my syntax highlighter… and that’s annoying.
#!/bin/bash
export TERM=xterm-256color

[ -r /etc/lsb-release ] && . /etc/lsb-release

if [ -z "$DISTRIB_DESCRIPTION" ] && [ -x /usr/bin/lsb_release ]; then
        # Fall back to using the very slow lsb_release utility
        DISTRIB_DESCRIPTION=$(lsb_release -s -d)
fi

# Colors
line="\e[38;5;8m----------------------------------------------------------------------\e[0m\n"

printf "\n";
printf "$line";
printf "\e[38;5;051m   Welcome to %s (%s %s %s)\n" "$DISTRIB_DESCRIPTION" "$(uname -o)" "$(uname -r)" "$(uname -m)";
printf "\e[0m$line";
printf "\e[38;5;232m\e[48;5;051m                                                                      \e[0m\n";
printf "\e[38;5;232m\e[48;5;051m      ██████╗  █████╗ ███╗   ██╗ ██████╗██╗  ██╗███████╗██████╗       \e[0m\n";
printf "\e[38;5;232m\e[48;5;050m      ██╔══██╗██╔══██╗████╗  ██║██╔════╝██║  ██║██╔════╝██╔══██╗      \e[0m\n";
printf "\e[38;5;232m\e[48;5;049m      ██████╔╝███████║██╔██╗ ██║██║     ███████║█████╗  ██████╔╝      \e[0m\n";
printf "\e[38;5;232m\e[48;5;048m      ██╔══██╗██╔══██║██║╚██╗██║██║     ██╔══██║██╔══╝  ██╔══██╗      \e[0m\n";
printf "\e[38;5;232m\e[48;5;047m      ██║  ██║██║  ██║██║ ╚████║╚██████╗██║  ██║███████╗██║  ██║      \e[0m\n";
printf "\e[38;5;232m\e[48;5;046m      ╚═╝  ╚═╝╚═╝  ╚═╝╚═╝  ╚═══╝ ╚═════╝╚═╝  ╚═╝╚══════╝╚═╝  ╚═╝      \e[0m\n";
printf "\e[38;5;232m\e[48;5;046m                                                                      \e[0m\n";
printf "$line";

Now that the superfluous setup super important stuff is done, moving on to the actual project at hand…

Project setup

I’m really good at taking something built for a specific task and trying to bend it to my will. Usually that involves trying to make something do a task that it wasn’t designed to do. Drone is no different, although in this case, I’m just taking simple tasks that are supposed to do one thing, and tacking them together to do several things.

I’m going to use this site as an example here, as it’s really evolved from a simple Jekyll site to something else entirely. Let’s take a simple Jekyll site setup and see what it offers:

Jekyll, out of the box

Jekyll is basically a blogging platform. It’s designed to run on your local machine and generate a static site from your files. It uses Ruby and bundler for the basic setup, so you’ll need to have those installed to start. I’ll skip the setup parts of Jekyll and assume that you can Google how to do that, if you get stuck.

I run a version of the Hydeout theme on this site right now (which is originally based on Hyde). I’m constantly editing it so that it looks and feels a little different than the default setup, but not so much that I’ve completely changed it to something else. Here’s my Jekyll _config.yml for this site:

# Dependencies
markdown: kramdown
highlighter: rouge

# Setup
title: Pitman
icon: /assets/images/PIT_logo_lrg.png
description: 'I am a meat popsicle.'
url: https://jasonpitman.com

author:
  name: 'Jason Pitman'
  url: https://instagram.com/ff4500

paginate: 6

plugins:
  - jekyll-feed
  - jekyll-gist
  - jekyll-paginate

sidebar_home_link: true #This is Hydeout specific

collections:
  images:
    output: true
  longform:
    output: true

My configuration isn’t anything special. I’m including the feed plugin for RSS, and the gist plugin in case I post Github Gists are some point. Paginate is a given, but even that, I haven’t used much yet. The interesting thing, as far as customization goes, are the Jekyll collections. What we’re basically doing there is making content that doesn’t fit into the normal blog-model into an active part of the site. Right now, I have two sections that fit into this category:

  1. Images

    This where all of my Instagram images go. Talk about overcomplicating things…

    This started as a dumb idea. “Why don’t I pull all of my old Instagram posts into my blog?” Sure, easy enough. “I’ll just request an archive from Instagram and then I can toss those into a folder and I’m done, right?” Unfortunately, it wasn’t as simple as that.

    I’ll make a separate post about that whole process at some point, but initially it involved taking all of the (cryptically named) images, generating a Markdown file for each, populating the dates, renaming them correctly, and putting them in the right (yearly) directory. I’ve probably needlessly overcomplicated the entire process (especially for any new images that I post), but they’re all up now. You can check them out here.

  2. Longform

    Longform posts are just regular blog posts, with extra stuff added in. They can be as short or as long as I want, because I’ll just keep adding pages. The process is extremely manual at this point, but the post that you’re reading is the first one that I’ve done. It seems to work decently so far.

    Because I’m already manually specifying the permalink URL for each post, extending an individual post was easy. For instance, the main post lives in the usual _posts folder that Jekyll uses. This post is formatted with the date/postname (YYYY-MM-DD-postname.md) and shows up in the list on the main page, sorted by newest post up top. The newest post also shows the specified excerpt on the main page, and each older post gets demoted to a link in the list below.

    Longform posts are essentially just tacked on. The original post is indexed and the longform pages aren’t. Thanks to permalink magic, I can toss specific pages in a seperate directory and have them output to the right place when we build the site. So, for the current pages in this post, the permalinks in the front matter look like:

    • Introduction - permalink: /posts/integrations/
    • Part One - permalink: /posts/integrations/P01/
    • Part Two - permalink: /posts/integrations/P02/

    And so on. This opens up the ability to structure content however I want, and have it live within the same general area of the other regular posts. To visualize how these live in the main directory structure, check this out: (The source directory is on the left, and the output _site directory is on the right.)

Jekyll collections open up a lot of options in making your site do unconventional things. More often than not, I don’t want a typical blog site layout, so using collections is a quick and easy way to accomplish custom pages without a ton of work. As shown on my Instagram images list page and individual pages, you’re not limited to the default templates either. Feel free to go crazy with making something that fits your vision, all neatly templated and organized in a collection.

Making Jekyll do other things

In the past, I’ve run complicated setups for a handful of Jekyll sites. Using npm and gulp to run tasks and generate files was fine, but for this project I was trying to keep it a little simpler. I’m not sure that I’ve accomplished that task, but it’s definitely a little more hands off than previous sites have been.

For example, here is a “simple” gulpfile.js from a previous project. It does the following when you run gulp:

  • Takes a source folder full of scss files, concatenates, auto-prefixes & minifies them, and thn creates a single file in the output directory.
  • Takes a source folder full of javascript files, concatenates & minifies them, and creates a single file in the output directory. (Except for the file sticky_header.js, which it just moves to the output directory)
  • Builds the site with Jekyll
  • Launches BrowserSync and watches specific folders for local changes (and will rebuild and reload the site when it detects a change)
  • Optionally deploys to a server via SSH (when using gulp deploy:production):
/** Load up the dependencies **/
var gulp        = require("gulp");
var browserSync = require("browser-sync");
var sass        = require("gulp-sass");
var concat      = require("gulp-concat");
var uglify      = require("gulp-uglify");
var rename      = require("gulp-rename");
var prefix      = require("gulp-autoprefixer");
var cp          = require("child_process");
var rsync       = require("gulp-rsync");
var secrets     = require("./secrets.json");

/** Console message showing the "running" status **/
var messages = {
  jekyllBuild: '<span style="color: grey">Running:</span> $ jekyll build'
};

/** Build the Jekyll Site **/
gulp.task('jekyll-build', function (done) {
  browserSync.notify(messages.jekyllBuild);
  var jekyll = process.platform === "win32" ? "jekyll.bat" : "jekyll";
  return cp.spawn(jekyll, ['build'], {stdio: 'inherit'})
    .on('close', done);
});

/** Rebuild Jekyll & do a page reload **/
gulp.task('jekyll-rebuild', ['jekyll-build'], function () {
  browserSync.reload();
});

/** Wait for jekyll-build, then launch the Server **/
gulp.task('browser-sync', ['sass', 'js', 'jekyll-build'], function() {
  browserSync({
    server: {
      baseDir: '_site'
    }
  });
});

/**
  Compile files from _scss into both _site/css (for live injecting)
  and site (for future jekyll builds)
**/
gulp.task('sass', function () {
  return gulp.src('_scss/main.scss')
    .pipe(sass({
      includePaths: ['scss'],
      onError: browserSync.notify
    }))
    .pipe(prefix(['last 15 versions', '> 1%', 'ie 8', 'ie 7'], { cascade: true }))
    .pipe(gulp.dest('_site/css'))
    .pipe(gulp.dest('css'))
    .pipe(browserSync.reload({stream:true}))
});

/** Concatenate & Minify JS **/
gulp.task('js', function() {
  return gulp.src(['js/*.js', '!js/sticky_header.js'])
    .pipe(concat('main.min.js'))
    .pipe(uglify())
    .pipe(gulp.dest('_site/js'));
 });

/**
  Watch scss files for changes & recompile
  Watch html/md files, run jekyll & reload BrowserSync
**/
gulp.task('watch', function () {
  gulp.watch(['_scss/*.scss', '_scss/**/*.scss'], ['sass']);
  gulp.watch('js/*.js', ['js']);
  gulp.watch(['index.html', '_includes/*.html', '_layouts/*.html', '_posts/*', 'submit/*.html', 'contact/*.html', 'FAQ/*.html', 'posts/*.html', 'about/*.html', 'js/*.js'], ['jekyll-rebuild']);
});

/**
  Default task, running just `gulp` will compile the sass,
  compile the jekyll site, launch BrowserSync & watch files.
**/
gulp.task('default', ['browser-sync', 'watch']);

/**
  Once you've tested your project, you can deploy the files using
  gulp deploy:production -  Note: setup ssh keys to avoid having
  to enter the ssh pass each time. See the README for info.
**/
gulp.task('deploy:production', ['jekyll-build'], function() {
  return gulp.src('**/*', {cwd: '_site'})
  .pipe(rsync({
    root: '_site',
    port: 22,
    username: secrets.servers.prod.username,
    hostname: secrets.servers.prod.hostname,
    destination: secrets.servers.prod.destination,
    incremental: true,
    recursive: true,
    compress: true,
    times: true,
    progress: true
  }));
});

This is usually fine, and a decent way to develop locally. The problem is that you have to maintain local dependencies. For some reason, that’s harder than it sounds with Ruby.

The last time I pulled down one of the sites that used a gulp flow like this, I was on a new machine. That meant that I’d need to make sure that homebrew was installed and up to date (on a Mac, of course), setting up rbenv, nodenv and bundler, rehashing rbenv shims, and then getting everything in your shell and $PATH setup correctly so that Ruby will see all of the things in the right places. Since I run a Z shell, the usual bash “suggested” commands are sometimes hit and miss. rbenv doctor (curl -fsSL https://github.com/rbenv/rbenv-installer/raw/master/bin/rbenv-doctor | bash) is always the bearer of bad news.

Honestly, I don’t mind Ruby… as long as I don’t have to deal with Ruby. I have a fun track record of breaking my local dev environment when trying to get our work Ruby projects to run. So, I figured, why not try to skip all that bullshit this time around?

With Jekyll as the project source, we can then use Drone to take care of some of the tasks that we’d be doing in our gulpfile.js. I won’t be replacing tasks like javascript minification or image optimization here (even though I probably should down the line).

Continued in Part Three »