NOTE: THIS IS NOT A GUIDE. THIS IS MY LONG WINDED JOURNEY. THERE ARE HOWEVER LINKS TO THE GUIDES I USED.

As you may have guessed, this site is using ghost as the cms for all my amazing words. It's nice(ish) and all, and if I didn't want to look into my current project, I'd probably stick with it, but we're not about that life are we? Nah fam, we like to overcomplicate things and get into weird corners of our own making so hopefully we learn something. Without further ado, here's the current plan.

I'd like to convert my ghost blog into a static website using Jekyll which is a static site generator. There are a number of positives to using a static website as opposed to a standard web stack that includes a database, not least of which is a smaller attack surface. But that's not why I'm doing this. Oh no. I'd like to figure out Continuos Integration and Continuous Delivery. I want to be able to make a blog post, send it off to to gitlab, which will then do some wizardry and pack it into a docker container, which will then get pushed to a private registry, and then run on my public vps.

Why though? Why dockerise it? Why not just use Gitlab Pages or similar? Well, I sayeth unto thee, why not? Yeah, showed you didn't I.

Anyway, this project will cover a few different interlinked subjects for me. First is static site generation. This is probably the simplest part of the process as Jekyll has a nice migrator I can use. Yes, I could just copy and paste. I only have 4 posts (5 if you include this guy) so it'd be pretty simple, but let's learn together.

Next, I need to sign up to gitlab, create a project and upload the site to there. I'll also be hosting this on my private self hosted gitea instance, but I like having seperation between my public facing services and private things.

Then I need to figure out how to package the site into a docker container using dockerfile and store it in a private docker registry.

Finally, push this new container so it's publicly accessible from my vps.

Seems simple enough right? Yeah buddy. Let's start super basic. Picking a theme. I could try my hand at designing something, but trust me, it'd look gross. This is the theme I decided to go for. Some day in the future I'll attempt to make my own if the artistic-ness strikes.

I'll be doing all this "web dev" on a vm I've set up for web dev stuff. It's running code-server and a basic lamp stack already, but Jekyll doesn't need this. I'll explain in a bit. Anyway, let's get started.

Silly Git

Smol confession. I haven't always used version control. I use it a little in my homelab to store all my playbooks, but nothing more complicated than that. So let's figure out how to do this from the beginnings. Sign up for a free gitlab account. Once you're all set, we need to provide our public ssh key because

so let's do that. Click the thing and provide your public key.

Cool. Now, I created a private project during sign up, so let's get that cloned into my webdev environment. Gitlab even have a nice little guide if your repo is empty.

However, the cloning will fail because you didn't provide your private key. Here's how you do that.

git clone [email protected]:whyitno.work/whyitno.work.git --config core.sshCommand="ssh -i ~/.ssh/id_rsa_gitlab"

Or, you could use the ssh-agent

eval $(ssh-agent)
ssh-add path/to/key

Sweet. We're now ready for the next step.

Installing Jekyll but end up dealing with Hyde.

Instructions for installing jekyll are super simple to follow. Once thats done, I'll just cd into the git cloned directory from before and I'll just clone the entire github directory for the theme I wanted. I could install it properly, following the instructions and so on, but I think I'll be modifying it a bit down the line, and this is the simplest way to do that.

git clone https://github.com/tareqdandachi/jekyll-shell-theme.git

Now, I won't be pushing any of my changes to this repo, so I'm just going to go grab all the files and dump them into my own git repo. Now, this theme does have a nice MIT license so I can basically do whatever I want with it, but I think it's only fair to provide credit, so that'll be added.....somewhere. We'll get back to that later.

So, here's what our directory looks like.

deploy@webdev: ~/jekyll/whyitno.work
$ tree
.
├── 404.md
├── assets
│   ├── css
│   │   ├── main.scss
│   │   └── syntax-highlighting.css
│   └── theme_logo.svg
├── _config.yml
├── example-config.yml
├── Gemfile
├── _includes
│   ├── footer.html
│   ├── header.html
│   └── head.html
├── index.md
├── jekyll-shell-theme.gemspec
├── _layouts
│   ├── default.html
│   ├── error.html
│   ├── home.html
│   ├── page.html
│   └── post.html
├── LICENSE
├── preview.md
├── read-me.md
└── README.md

4 directories, 21 files

I think the only thing we don't need is the example-config.yml so that can be deleted. After that, lets update the bundle and then serve it for us to see.

# update bundle
$ bundle update

# serve the website. As this is being done on a vm and I want to access it
# over my lan, I need to provide a host value of 0.0.0.0
$ bundle exec jekyll serve --host=0.0.0.0

Ok, now lets import all the posts from ghost by using that importer I linked to earlier and migrate things over.

So, first off, we need the db file for my database. To get this, I'll ssh into my vps and track down the location of the ghost.db file. Once I have this, I can use scp to pull the file to my local machine. This is pretty straightforward, but I'm using a none-standard port for ssh because apparently that's a good idea. I'm not exactly sold on this security by obscurity thing but whatever. Thats why I'm using certs with stupid long passwords. Oh well. Here's the command:

hobo@Primus: ~
$ scp -i ~/.ssh/id_rsa_do -P 6942 [email protected]:/path/to/ghostData/data/ghost.db /home/hobo/

Now, to run the jekyll importer we need to install it

$ gem install jekyll-import


Jekyll-import requires you to manually install some dependencies.

Most importers require one or more dependencies. In order to keep jekyll-import's footprint small, we don't bundle the gem with every plausible dependency. Instead, you will see a nice error message describing any missing dependency and how to install it. If you're especially savvy, take a look at the require_deps method in your chosen importer to install all of the deps in one go.

Right then, I suppse we better have a quick look see at the source code for the ghost  importer to see what dependancies we need. Cus we savvy innit lads?

Now, I'm not a Ruby expert. In fact, I'm the opposite of a Ruby expert. However, I'm going to take a quick guess and assume that these are the dependancies we need.

def self.require_deps
        JekyllImport.require_with_fallback(%w(
          rubygems
          sequel
          sqlite3
          fileutils
          safe_yaml
        ))
      end

Ok, cool. Let's install them.

$ gem install rubygems sequel sqlite3 fileutils safe_yaml

ERROR:  Could not find a valid gem 'rubygems' (>= 0) in any repository
ERROR:  Possible alternatives: ruby_gem, cf_ruby_gems, eyrubygems, iruby_gem, rbygem
Fetching sequel-5.38.0.gem
Successfully installed sequel-5.38.0
Parsing documentation for sequel-5.38.0
Installing ri documentation for sequel-5.38.0
Done installing documentation for sequel after 11 seconds
Fetching sqlite3-1.4.2.gem
Building native extensions. This could take a while...
ERROR:  Error installing sqlite3:
        ERROR: Failed to build gem native extension.

    current directory: /home/deploy/gems/gems/sqlite3-1.4.2/ext/sqlite3
/usr/bin/ruby2.5 -I /usr/local/lib/site_ruby/2.5.0 -r ./siteconf20201104-6777-1hwrwgy.rb extconf.rb
checking for sqlite3.h... no
sqlite3.h is missing. Try 'brew install sqlite3',
'yum install sqlite-devel' or 'apt-get install libsqlite3-dev'
and check your shared library search path (the
location where your sqlite3 shared library is located).
...some other stuff removed because mostly not relevant to show....
3 gems installed

Huh. Ok, well according to that long winded complaint, we need to install sqlite3 first.

$ sudo apt install libsqlite3-dev
# run the previous gem command again because we're savvily lazy
$ gem install rubygems sequel sqlite3 fileutils safe_yaml

ERROR:  Could not find a valid gem 'rubygems' (>= 0) in any repository
ERROR:  Possible alternatives: ruby_gem, cf_ruby_gems, eyrubygems, iruby_gem, rbygem
...
4 gems installed

I think I can safely ignore the rubygems error.........maybe

Anyway, let's try converting!

$ ruby -r rubygems -e 'require "jekyll-import";
    JekyllImport::Importers::Ghost.run({
      "dbfile"   => "/home/deploy/jekyll/whyitno.work/whyitno.work/ghost.db"
    })'
    
Traceback (most recent call last):
        20: from -e:2:in `<main>'
...blah blah blah "Look at me, I'm sql, I dont want to work" blah blah blah
         1: from /home/deploy/gems/gems/sqlite3-1.4.2/lib/sqlite3/database.rb:147:in `new'
/home/deploy/gems/gems/sqlite3-1.4.2/lib/sqlite3/database.rb:147:in `initialize': SQLite3::SQLException: no such column: markdown (Sequel::DatabaseError)

Know what? Bun dis. I'm not about to spend ages diddling with databases so we say "bye felicia" to this method, and we try something else. Turns out I may not be as savvy as I initially thought :(

In my defense, the source code for that importer was last updated on 22nd Oct 2017, and for all I know, the db structure for ghost has changed since then. That's my excuse, and that's what we're going with.

Ok. We do an export from our ghost instance in json format and upload it our webdev server.

Install the alternate importer

$ gem install jekyll_ghost_importer

# and convert our posts

$ jekyll_ghost_importer GhostBackup.json

Heh. Easy enough. Now, It's also copied some drafts, including this current one I'm writing. To view my amazing blog with the drafts, we run

jekyll serve --drafts --host=0.0.0.0

Yeah buddy. We jammin'. Granted, none of my pictures were migrated, and the formatting is a little off, but there's enough for me to play with. First things first. I want my posts to be visible on the homepage so potential viewers can click and see what amazing things I have to say about things. Now, while I could spend some time learning the ruby syntax required here, I know that the minima theme which is the jekyll default has some code I can yank out. Here's the snippet that'll probably work for me......

You know how I'm fickle and quick to change my mind about things and stuff? Weeelllllll. I just found out that that the minima theme actually has a dark mode. That's really all I wanted in a theme. Sure the one I showed before looks nice to me and stuff, but minima is simples. So err, yeah, I'm doing that. With dark mode. Because I like my eyeballies.

Super easy to to use as well. So easy in fact, I'm just gonna link the repo.

Doesn't look half bad does it? Obviously getting rid of the twitter thingy at the bottom. I'll replace it with Reddit. Come yell at me there for longing out your life. Anyway. The remainder of the work required for the website to be ready to go live is minima......l  (hah! geddit? I'm hilarious.) Moving on. Now that we have a project, we can sync it with our repo.

# We add the stuff we just did to be commited to our repo
$ git add whyitno.work

# Then we commit the thing. You need to provide a message. Make it helpful.
$ git commit

# Now we can push our changes our repo
$ git push -u origin master

Nito.

Where I make my first self-made docker container like a big boy

I figured out how to package my amazing jekyll site into a docker container. This next bit is super complicated so you may wanna take some notes. Now. As you know, to build a jekyll site, we use

$ jekyll build

Which does some wizardry and dumps a functioning version of my website into the _site directory. So how do we containerise it? With a dockerfile! A dockerfile is basically a recipe for a docker container. Here's the super complicated Dockerfile for my website.

FROM nginx
COPY whyitno.work/_site/ /usr/share/nginx/html/

Yup. 2 lines. That's it. Now, I'm going to explain what those 2 lines do, so try and keep up.

FROM just tells docker which image I want to use as a base. Like a pizza base. I figured the official nginx container would do the job fine. If you scroll down a little, they even provide the example I used above. COPY, for some reason lost to me, copies stuff from a source location to a destination. Thats like the sawce. The cheese? Well that's my hilarious analogy. Ya welcome.

Now, I had a little issue where things weren't copying over to the container correctly, and that was all my fault. I thought I had to provide a wildcard (*) at the end of source (whyitno.work/_site/*) to tell it to copy everything within that directory, but it turns out that is wrong. Don't do that. When I did that, it skipped the directories and just copied the files......

Dockerfiles can become much more complicated than this. Check out the official reference documentation.

To build and run the docker container we do the following things.

# Build the container
$ docker build -t amazing-website:latest .

# Check if the container is running
$ docker ps

# run the container
$ docker run --name amazing-website -d -p 8880:80 amazing-website:latest

As you can see, I exposed port 8880 when I ran the docker container, so let's take a quick look-see at the website.

Woo, it's working.

Damn that was a lot of effort. I wonder if there's a way to speed this up and automate it so I can concentrate on writing amazing content that will be .gitignored.

Lets do some CI/CD ya Git.

So, continuous integration and continuos deployment using gitlab all start with the use of a .gitlab-ci.yml file. This file will determine the build, testing and deployment process of my super complicated project. The folks over at gitlab have a nice little feature where you can add a new .gitlab-ci.yml and populate it with a template. I found this neat trick from here. Following that nice and concise video, I have a basic file that looks like this

stages:
  - build
  
docker-build-master:
  image: docker:latest
  stage: build
  services:
    - docker:dind
  before_script:
    - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY
  script:
    - docker build --pull -t "$CI_REGISTRY_IMAGE" .
    - docker push "$CI_REGISTRY_IMAGE"
  only:
    - master

Seriously. Check the link. Explains it all better than I can.

Now, my Dockerfile ran into a small error where the source directory couldn't be found when used from Gitlab. Mini change needed to be made to that. Here's the updated Dockerfile.

FROM nginx:alpine
COPY ./_site/ /usr/share/nginx/html/

I needed to add ./ to the beginning of the source location. (I also changed the directory structure of the project a little bit because I was running into issues with something else. I'll get to that later. Maybe.)

Also. Theres a .gitignore file. This has a list of things that git will ignore when you push your changes to the repo. By default, _site is in there, and for good reason. However, for this stage of the project, I removed _site from there so it can be uploaded, and our docker file can use it.

If I now make a small change and commit it to my master branch, I should see it run under CI /CD -> Pipelines on the Gitlab website. If I got it right this time, it should pass.

If I then check the nice integrated docker repo in my Gitlab account......

:D

Now, a quick manual test.

# log into the docker registry
$ docker login registry.gitlab.com

# pull the newly created container
$ docker pull registry.gitlab.com/whyitno.work/whyitno.work

# run the container
$ docker run --name ci-cd-site -d -p 1234:80 registry.gitlab.com/whyitno.work/whyitno.work

Ok, that's great and all, but as I mentioned earlier, the _site directory is, by default, set to be ignored from the repo pushes, which I can get behind. No need to upload 2 copies of my data when I can just push the jekyll project and have the CI/CD pipeline build it for me before turning it into a container. To figure this out, I did some more duckduckgoing and found this awesome article. With that, I now have the second (first) stage of the build pipeline implemented. Here's my updated .gitlab-ci.yml file.

stages:
  - build
  - dockerize

build_jekyll:
  stage: build
  tags:
    - docker
  image: ruby
  cache:
    paths:
      - vendor/bundle
  script:
    - gem install bundler:2.0.2
    - bundle install --path vendor/bundle
    - bundle exec jekyll build -s . -d ./_site
  artifacts:
    paths:
      - _site

docker-build-master:
  image: docker:latest
  stage: dockerize
  services:
    - docker:dind
  before_script:
    - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY
  script:
    - docker build --pull -t "$CI_REGISTRY_IMAGE" .
    - docker push "$CI_REGISTRY_IMAGE"
  only:
    - master

I suppose I better explain what the new bits do so it seems like I know what I'm doing and definitely not just following in the footsteps of smarter people.

So, theres a new stage. Dockerize. This is technically the stage I made before, and was called build. It's now dockerize. What build now does is builds my jekyll site. It does this by grabbing the ruby image and running the commands you see under the script section. So, it's installing the bundler, installing stuff specified in our Gemfile and then building our website (-s is source, -d is destination, as in where to dump out the finished site.) Artifacts are things from this process that we want to have made available to the next stage in the pipeline, which is the dockerising of my site using that super complicated dockerfile I  made earlier. Here it is being all successfull and stuff.

See. We got 2 ticks now for the 2 stages in our pipeline. That's the CI portion of my project kinda understood. Next up. CD. How do I have my fancy new containers deployed to my droplets automagically?

I'll let you know when I figure it out....

Following in the footsteps of the smartie people.

Weeeellllll, I'm not going to act like I figured out something amazing all by myself. I'll let you in on a secret. The folks over at digital ocean have some AMAZING guides for all manner of tech stuff. This is the exact guide that answered the question I had. That counts as giving credit yes? Cool. I'm now going to rip out the bits that will help me do this again quickly next time, but to actually help you understand things better, do check out that link.

First, we need to register a Gitlab runner to run on our server. Long story short, this is to minimise the number of things that have access to our ssh key. We do this by adding the gitlab repo, pulling the install script, having a read through to make sure no-ones up to any shenanigans, running the script and finally installing the gitlab runner. Here's the tasty copy and paste commands.

# Pull the script
$ curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh > script.deb.sh

# Read the script. Actually read it.
$ less script.deb.sh

# Run the script. Dont be running things as root. You will be asked for a sudo
# password.
$ sudo bash script.deb.sh

# Install the runner
$ sudo apt install gitlab-runner

# Check it's running
$ systemctl status gitlab-runner

Cool. Now go to gitlab under Settings -> CI/CD -> Runners and go to Set Up A Specific Runner. You'll see a url and a registration token.

Use those deets in the following command that you need to run in your terminal.

$ sudo gitlab-runner register -n --url https://your_gitlab.com --registration-token project_token --executor docker --description "Deployment Runner" --docker-image "docker:stable" --tag-list deployment --docker-privileged

Replace https://your_gitlab.com and project_token with your own values.

You should be able to see your new runner registered under your account.

Now, you're gonna want to create a new non root user, add them to the docker group, generate a new ssh key for them and add the public key to your authotized users files.

# Make a new user called deployer, or whatever you want
$ sudo adduser deployer

# Add the new user to the docker group
$ sudo usermod -aG docker deployer

# Change to the new user
$ su deployer

# Generate a new ssh key
$ ssh-keygen -b 4096

# Copy the contents of the public key into your authorized keys file
$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

# This will dump out the contents of private key we just made.
# Copy the whole thing, including -----BEGIN RSA PRIVATE KEY-----
# and -----END RSA PRIVATE KEY-----
$ cat ~/.ssh/id_rsa

Now jump on to your gitlab account under Settings -> CI/CD -> Variables and add the following variables

NOTE: MAKE SURE YOU HAVE A NEW LINE AT THE END OF YOUR SSH KEY. Seriously. My pipeline was failing and it took some duckduckgoing to find the solution. Also. If you're using the default ssh port, you can skip that last variable, and just amend the .gitlab-ci.yml file. I'm using something none standard so I had to modify things slightly for myself.

With that prep work done, we can finally add the missing piece of the puzzle to our gitlab-ci.yml file. Here's my completed pipeline.

stages:
  - build
  - dockerize
  - deploy

variables:
  EXPOSE_PORT: 1248
  TAG_LATEST: $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_NAME:latest
  TAG_COMMIT: $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_NAME:$CI_COMMIT_SHORT_SHA

build_jekyll:
  stage: build
  tags:
    - docker
  image: ruby
  cache:
    paths:
      - vendor/bundle
  script:
    - gem install bundler:2.0.2
    - bundle install --path vendor/bundle
    - bundle exec jekyll build -s . -d ./_site
  artifacts:
    paths:
      - _site

docker-build-master:
  image: docker:latest
  stage: dockerize
  services:
    - docker:dind
  before_script:
    - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY
  script:
    - docker build -t $TAG_COMMIT -t $TAG_LATEST .
    - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN $CI_REGISTRY
    - docker push $TAG_COMMIT
    - docker push $TAG_LATEST
  only:
    - master

deploy:
  image: alpine:latest
  stage: deploy
  tags:
    - deployment
  script:
    - chmod og= $ID_RSA
    - apk update && apk add openssh-client
    - ssh -i $ID_RSA -o StrictHostKeyChecking=no $SERVER_USER@$SERVER_IP -p $SERVER_PORT "docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN $CI_REGISTRY"
    - ssh -i $ID_RSA -o StrictHostKeyChecking=no $SERVER_USER@$SERVER_IP -p $SERVER_PORT "docker pull $TAG_COMMIT"
    - ssh -i $ID_RSA -o StrictHostKeyChecking=no $SERVER_USER@$SERVER_IP -p $SERVER_PORT "docker container rm -f whyitnowork-blog || true"
    - ssh -i $ID_RSA -o StrictHostKeyChecking=no $SERVER_USER@$SERVER_IP -p $SERVER_PORT "docker run -d -p $EXPOSE_PORT:80 --name whyitnowork-blog $TAG_COMMIT"
  environment:
    name: production
    url: https://whyitno.work
  only:
    - master

Now, aside from the obvious addition of the deploy section, you may also notice (no you didn't lol, who we kidding) that I modified the dockerize portion with things I learned from the digital ocean article. (Seriously. Read it. It explains all the things.)

With all that done, let's run the pipeline.

3 ticks! All stages were doing the successfullings. Huzzah. The sites up and running. No ssl offloading yet, but I'll cross that bridge when I get to fully converting my old posts with images over and tidying up layouts. I want better syntax highlighting for one thing. Anyway, I guess this will be my last post in ghost. Next tech journey will hopefully be all jekyll :3

Toodles.