Creating a static version of Ghost

My Ghost site was getting slow. I decided to make it a static site. Here is how I did it.

First and foremost, I love Ghost as a blog engine. It's fantastic, all of the features are great and the Casper theme is amazing looking. I am a huge fan.

There is a downside though, as your site grows it slows down when you use the SQLite database. I did not want the complexity of running a SQL server since this is something I do in my spare time.

Overview

I want to host my blog as a static site in a container using the same Casper theme and additional features that I have with the original Ghost. Along with a couple of modifications.

I tried out Gatsby and a number of the themes and plugins for it. I spent a few days and could not find one that had everything I wanted. I am picky. I also did not want to learn a bunch of new tools and frameworks like Gatsby and React to make it all work. JavaScript is not my strong suit and I don't have the time to dig deep into Gatsby and React to figure it all out.

I do have the time to write a C# application though. Which is my strong suit. And the content in Ghost is all exposed using API's. Ghost renders the HTML of the post itself (not the outer) so we don't have to rebuild much of anything. Just the theme.

And in typical DevOps fashion, it had to be a completely automated process. Fun fact. This is the first post to kick off the whole process for real. :)

This took me about a day and a half to iron everything out.

Puzzle pieces

In order of setting it all up:

• Ghost blog backend
• Static Generator - Gets the contents from the exposed Ghost API's and generates the static files. This is in a Docker container.
• Azure DevOps - Source code repository and build services.
• Azure DevOps build pipeline - Builds my static site container
• Azure Function - Triggers the build pipeline for my static site container
• Ghost integration to call the trigger.

Detailed description of the pieces

Ghost backend

The Ghost backend is a simple Ghost container. It is what I was running my site with. I did not touch any of the configuration or Kubernetes manifests other than modifying the ingress to listen on a new domain name. This was needed so the generator can download the images.

Static site generator

The generator calls the API's, gets the contents, and runs it through the Razor templating engine to spit out a bunch of static files. It generates the site in its entirety and the result can be hosted in by a static site server, like NGINX.

The application runs in a container and is is intended to be used through a multi stage build in a Dockerfile or ran using a volume mounted at /out.

I based my generator off another post which uses Dot Net Core 2.2. I know that it is end-of-life but I didn't want to spend the time on figuring out how to make it work with Core 5. As of Core 3.1 (probably 3.0) the required libraries are no longer available through Nuget, it is now built in to the SDK and I just didn't want to spend a few days figuring it all out again. Here is that post:

Razor Template Rendering

Use the Razor template engine from ASP.Net Core to render some data in a template pulled from a database.

FrakkingsweetEdward Cooke

The site generator source code is available here:

veccsolutions/Vecc.GhostTemplating

Contribute to veccsolutions/Vecc.GhostTemplating development by creating an account on GitHub.

GitHubveccsolutions

Azure DevOps

Azure DevOps is my source code repository of choice. I only use GitHub when making things publicly available. I use it for all things related to the application lifecycle, repositories, CI/CD, work item tracking, etc. I highly prefer it over GitHub due to everything being integrated in a single platform.

Azure DevOps pipeline

The pipeline for building my site is sort of simple. It builds the image containing my static site then uploads it to my registry. It then updates my Kubernetes cluster manifests with the new image version for the static site.

My pipeline uses a number of variables that are set in the pipeline in Azure DevOps. Some are marked as secret so people can not see them.

The cluster manifests are stored in a separate git repository in Azure DevOps. That repository is monitored by my Argo CD implementation to keep my cluster up to date. You can see how I set up my Argo implementation here:

Letting Argo CD manage itself

One of the things I’m implementing in my new Kubernetes cluster is Argo. First step is getting it to manage itself.

FrakkingsweetEdward Cooke

The Dockerfile to build my site image:

FROM harbor.veccsolutions.org/vecc/ghost-templating:1.0.4 AS build

ARG TemplatingOptions__AdminKey=
ARG TemplatingOptions__GhostUrl=
ARG TemplatingOptions__ContentKey=
ARG TemplatingOptions__DisqusUrl=
ARG TemplatingOptions__FeedlyLink=
ARG TemplatingOptions__GoogleAnalyticsCode=
ARG TemplatingOptions__GoogleAnalyticsDomain=
ARG TemplatingOptions__OutputDirectory=/out
ARG TemplatingOptions__RssFeed=
ARG TemplatingOptions__SiteName=

RUN printenv
RUN ["dotnet", "Vecc.GhostTemplating.dll"]

FROM nginx:latest
COPY --from=build /out /usr/share/nginx/html

The azure-pipelines.yml file for building it:

trigger:
- master

resources:
  repositories:
  - repository: kubernetes
    type: git
    endpoint: azuredevops-example-Docker
    name: kubernetes/kubernetes

variables:
  Version.MajorMinor: 0.0
  Version.Revision: $[counter(variables['Version.MajorMinor'], 0)]
  Version: $(Version.MajorMinor).$(Version.Revision)
  tag: $(Version.MajorMinor).$(Version.Revision)

name: $(Version.MajorMinor).$(Version.Revision)

stages:
- stage: Build
  displayName: Build image
  jobs:
  - job: Build
    displayName: Build
    pool:
      vmImage: 'ubuntu-latest'
    steps:
    - checkout: self
    - checkout: kubernetes
      persistCredentials: true
    - task: Bash@3
      inputs:
        targetType: 'inline'
        script: |
          printenv
          ls
          ls $(Build.SourcesDirectory)
    - task: Docker@2
      inputs:
        containerRegistry: 'harbor-Docker'
        repository: 'vecc/frakkingsweet-site'
        command: 'build'
        Dockerfile: '**/Dockerfile'
        tags: |
          $(tag)
          latest
        arguments: '--build-arg TemplatingOptions__AdminKey=$(TemplatingOptions__AdminKey) --build-arg TemplatingOptions__GhostUrl=$(TemplatingOptions__GhostUrl) --build-arg TemplatingOptions__ContentKey=$(TemplatingOptions__ContentKey) --build-arg TemplatingOptions__DisqusUrl=$(TemplatingOptions__DisqusUrl) --build-arg TemplatingOptions__FeedlyLink=$(TemplatingOptions__FeedlyLink) --build-arg TemplatingOptions__GoogleAnalyticsCode=$(TemplatingOptions__GoogleAnalyticsCode) --build-arg TemplatingOptions__GoogleAnalyticsDomain=$(TemplatingOptions__GoogleAnalyticsDomain) --build-arg TemplatingOptions__OutputDirectory=$(TemplatingOptions__OutputDirectory) --build-arg TemplatingOptions__RssFeed=$(TemplatingOptions__RssFeed) --build-arg TemplatingOptions__SiteName=$(TemplatingOptions__SiteName)'
    - task: Docker@2
      inputs:
        containerRegistry: 'harbor-Docker'
        repository: 'vecc/frakkingsweet-site'
        command: 'push'
        tags: |
          $(tag)
          latest
    - task: Bash@3
      inputs:
        targetType: 'inline'
        script: |
          cd kubernetes

          git checkout master
          cp templates/frakkingsweet-com-site/deployment.yml public/frakkingsweet-com/site-deployment.yml
          sed -i "s/{version}/${BUILD_BUILDNUMBER}/g" public/frakkingsweet-com/site-deployment.yml

          git config --global user.email "ado@frakkingsweet.com"
          git config --global user.name "Azure DevOps"

          git add -A
          git commit -m "Version bump frakkingsweet-com-site to ${BUILD_BUILDNUMBER} [skip-ci]"
          git push origin master
        failOnStderr: false

Azure function

The function is a simple part. But it is really important. It takes the web hook request from Ghost and turns it in to a request to kick off a build.

Ghost will retry a web hook request up to 5 times if it takes longer than 2 seconds to complete. This causes problems when an Azure Function is set up in the consumption plan because it can take well over the 2 seconds. What I found is that most of the time it would end up making all 5 calls to my function which would then trigger 5 builds when the function started up. It is also sometimes possible that the call to Azure DevOps could take longer than 2 seconds, which would then cause Ghost to send another request.

To get around this I keep track of what the last updated time was and if it's newer than the time recorded, it will queue a build. I had to use the orchestration triggers so it would only run a single instance of the function at a time.

My first attempt at fixing it was using a static DateTime object. That didn't work due to the requests running in multiple processes. Static variables are only valid in the same process so the lock only applied to the individual process, not running across multiple ones.

My second attempt was using an orchestration trigger so it would only run once. That worked, however, it had a side effect. It was beating up more storage account costing me money. Not a lot, it was only a dollar or 2 a month, but that was more than I wanted to spend so I tried something else.

My third attempt was using a logic app. This was much closer, there was no warmup time and it fired off instantly. This allowed Ghost to work correctly. The downside it did not return until the web request to Azure DevOps to trigger the build was completed. That request took 3+ seconds. Sad.

My fourth attempt was combining a logic app with an Azure Function and a storage account queue to tie it all together. This worked well. The logic app puts a message on the storage account queue then returns. This process is only a few milliseconds so Ghost is happy. The function app uses a storage account queue trigger to fire off. It can take a few minutes before the function app kicks off, but that is fine. It doesn't continually beat up my storage account. My cost went from a few dollars a month to about 10 cents. I guess I am cheap that way.

The source code for my trigger is here:

veccsolutions/Vecc.TriggerAdoBuild

Contribute to veccsolutions/Vecc.TriggerAdoBuild development by creating an account on GitHub.

GitHubveccsolutions

Ghost integration

The integration contains the keys needed for the generator to get the content from your Ghost instance. It also contains the web hooks that call your Azure Function.

If you use the Site changed (rebuild) hook it will fire off multiple times while you are creating a new post or page and cause many needless rebuilds of your site.

The web hooks events to configure are:

• Post published
• Published post updated
• Post unpublished
• Page published
• Published page unpublished
• Page unpublished
• Tag created
• Tag updated
• Tag deleted

Conclusion

I will be updating this post as I get more of what I did up on GitHub. At first it seems complicated and a very daunting task. But taking it one step at a time made it simple. And I am glad I did it. Having Kubernetes and ArgoCD already setup helped make this easier.

The retry feature of the web hooks in Ghost is nice. I just wish I could configure or disable it.

Since building this post I have changed from running my static site in my Kubernetes cluster to hosting it in Azure. I now use an Azure Storage Account with a static site in front it. In front of that static site is an Azure CDN so content is cached and distributed from multiple points across the country. The cost to host everything in Azure, including the logic app, function app, backend storage accounts, static site and CDN is so far a whopping 25 cents a month.