I needed a VPN and I wanted to run it inside of Kubernetes. Here is how I did it.

I am first going to cover a little background on my setup, so you know where I am coming from. I have a single subnet, single router, dynamic public address, a typical household setup. I have a 7-node Kubernetes cluster. 3 control planes and 4 worker nodes, all Raspberry Pi's running Debian 10.

  1. Certificate Authority
  2. Docker Image
  3. Kubernetes
    1. Deployment.yaml
    2. cacerts.yaml
    3. certs.yaml
    4. ipsec-conf.yaml
    5. ipsec-secrets.yaml
    6. private-keys.yaml

Certificate Authority

Before we begin, we need some certificates. Let's create them.

To create a very basic certificate authority, issue the following commands in an empty directory:

mkdir cacerts
mkdir certs
mkdir vpn
openssl genrsa -aes256 -out cacerts/ca-key.pem 4096
openssl req -new -x509 -days 3650 -key cacerts/ca-key.pem -sha256 -out cacerts/ca.pem
chmod 0400 cacerts/*.pem
cd vpn

That creates a simple directory structure for you certificate authority and generates a self-signed root certificate that will expire in 10 years.

Now that we have a certificate authority set up, we need to issue an SSL certificate for the VPN.

This is a little more complicated, but not by much. We need to set a few values and add some extensions to the certificate to be valid for use in Windows. To do this, create a file named extensions.cnf. The contents are this:

basicConstraints = CA:FALSE
keyUsage = digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth, 1.3.6.1.5.5.8.2.2
subjectAltName = DNS:vpn.example.com

The part that makes it work for Windows is the additional OID, 1.3.6.1.5.5.8.2.2.

Create another file named openssl.cnf. This will contain additional information about your VPN certificate. The contents of this file are this:

[ ca ]
default_ca = CA_default


[ CA_default ]
dir                 = ../
certs               = $dir/certs
crl_dir             = $dir/cacerts/crl
database            = $dir/cacerts/index.txt
new_certs_dir       = $dir/vpn
certificate         = $dir/cacerts/ca.pem
serial              = $dir/cacerts/ca.srl
crlnumber           = $dir/cacerts/crlnumber
crl                 = $dir/cacerts/crl.pem
private_key         = $dir/cacerts/ca-key.pem
# x509_extensions   = usr_cert
name_opt            = ca_default
cert_opt            = ca_default
default_days        = 1095
default_crl_days    = 30
default_md          = default
preserve            = no
policy              = policy_anything
unique_subject      = no
extensions          = v3_req

[ policy_anything ]
countryName             = optional
stateOrProvinceName     = optional
localityName            = optional
organizationName        = optional
organizationalUnitName  = optional
commonName              = supplied
emailAddress            = optional

#####

[req]
prompt              = no
distinguished_name  = req_distinguished_name
req_extensions      = v3_req
extensions          = v3_req

[req_distinguished_name]
C   = US
ST  = My State
L   = My City
O   = Example
CN  = vpn.example.com

[v3_req]
basicConstraints    = CA:FALSE
keyUsage            = digitalSignature, keyEncipherment
extendedKeyUsage    = serverAuth, 1.3.6.1.5.5.8.2.2
subjectAltName      = DNS:vpn.example.com

You will need to change the subjectAltName in both files and the CN in the openssl.cnf file to be the fully qualified domain name of your VPN endpoint.

Now we have everything ready to create our certificate for the VPN.

Issue the following commands to create a new certificate, replace vpn.example.com with your fully qualified domain name of your VPN so it is easy to distinguish what files are what.

openssl req -new -sha256 -config ./openssl.cnf -keyout vpn.example.com.key.pem -out vpn.example.com.csr -nodes
openssl ca -config ./openssl.cnf -in vpn.example.com.csr -out vpn.example.com.pem -extfile extensions.cnf

Those 2 commands create a certificate signing request then signs it using your new certificate authority.

You will need to trust the root certificate cacerts/ca.pem on your client machine to be able to connect to your VPN.

Docker Image

The first hurdle, albeit a small one, is creating an image that has strongSwan in it. I based my image on Alpine. I used Alpine because I was wondering if I could use an image from a distribution other than the one running the cluster to manage routes and other underlying system resources. Turns out, you can, at least between Debian and Alpine with strongSwan.

My Dockerfile is as follows

FROM alpine:3.15.0

RUN apk add --no-cache strongswan openssl

VOLUME /etc/ipsec.d/aacerts
VOLUME /etc/ipsec.d/acerts
VOLUME /etc/ipsec.d/cacerts
VOLUME /etc/ipsec.d/certs
VOLUME /etc/ipsec.d/crls
VOLUME /etc/ipsec.d/ocspcerts
VOLUME /etc/ipsec.d/private
VOLUME /etc/ipsec.d/reqs

CMD ["ipsec", "start", "--nofork", "--auto-update", "10"]

It is basic and simple, it installs strongSwan and OpenSSL. Then sets up the volumes for the certificates and tells it to start ipsec in the foreground.

Build that image and push to your registry.

Kubernetes

Next, we need to setup our Kubernetes manifests. There are 6 of them at a minimum. The Deployment, 3 ConfigMaps and 2 Secrets.

The 3 ConfigMaps cover the cacerts and certs directories, as well as ipsec.conf. The 2 Secrets cover the private directory and ipsec.secrets.

Since we need to do some routing, we need to label the node that you want to run strongSwan on. This way you can set up your routers static route correctly. I named my label strongswan with a value of yes. It makes it easy to move if needed and keeps it on the correct node. I also have a node that I have tainted with dedicated and that is set to noschedule.

To do that run the following.

kubectl taint nodes nodenamehere dedicated=yes:NoSchedule
kubectl label nodes nodenamehere strongswan=yes

Now what is next is to apply the below manifests, forward ports UDP 500 and 4500 to your correct worker node and give it a try.

deployment.yaml

Create your Deployment manifest, I named mine deployment.yaml. The contents are below, be sure to change the image to match the image you created earlier and change the imagePullSecrets to be your correct secret name or remove it if it is not needed.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: vpn
spec:
  selector:
    matchLabels:
      app.kubernetes.io/component: strongswan
      app.kubernetes.io/part-of: strongswan
  replicas: 1
  revisionHistoryLimit: 1
  template:
    metadata:
      labels:
        app.kubernetes.io/name: strongswan
        app.kubernetes.io/instance: strongswan
        app.kubernetes.io/component: strongswan
        app.kubernetes.io/part-of: strongswan
    spec:
      hostNetwork: true
      nodeSelector:
        strongswan: "yes"
      tolerations:
        - key: "dedicated"
          operator: "Exists"
          effect: "NoSchedule"
      imagePullSecrets:
        - name: azureregistry
      containers:
      - name: strongswan
        image: example.azurecr.io/strongswan:1.0.2
        securityContext:
          privileged: true
        resources:
          requests:
            memory: 50Mi
            cpu: 10m
          limits:
            memory: 200Mi
            cpu: 1000m
        volumeMounts:
          - mountPath: /etc/ipsec.conf
            name: ipsec-conf
            readOnly: false
            subPath: ipsec.conf
          - mountPath: /etc/ipsec.secrets
            name: ipsec-secrets
            readOnly: true
            subPath: ipsec.secrets
          - mountPath: /etc/ipsec.d/cacerts
            name: cacerts
            readOnly: true
          - mountPath: /etc/ipsec.d/certs
            name: certs
            readOnly: true
          - mountPath: /etc/ipsec.d/private
            name: private-keys
            readOnly: true
      volumes:
        - name: ipsec-conf
          configMap:
            name: ipsec-conf
            items:
              - key: ipsec.conf
                path: ipsec.conf
        - name: ipsec-secrets
          secret:
            secretName: ipsec-secrets
            items:
              - key: ipsec.secrets
                path: ipsec.secrets
        - name: cacerts
          configMap:
            name: cacerts
        - name: certs
          configMap:
            name: certs
        - name: private-keys
          secret:
            secretName: private-keys

cacerts.yaml

The cacerts.yaml file will be used to populate the /etc/ipsec.d/cacerts directory with your root certificate authority certificates. The value of the entry should be the contents of cacerts/ca.pem.

apiVersion: v1
kind: ConfigMap
metadata:
  name: cacerts
data:
  root.pem: |
    -----BEGIN CERTIFICATE-----
    MII.
    .
    .
    -----END CERTIFICATE-----

certs.yaml

The certs.yaml file contains the public key of your VPN. The value of the entry should be the contents of vpn/vpn.example.com.pem

apiVersion: v1
kind: ConfigMap
metadata:
  name: certs
data:
  vpn.frakkingsweet.com.pem: |
    -----BEGIN CERTIFICATE-----
    MII.
    .
    .
    -----END CERTIFICATE-----

ipsec-conf.yaml

The ipsec-conf.yaml will be very specific to your needs. Getting this correct is probably the hardest part of the whole project.

A basic one that works for me for doing ipsec with username/password authentication that works with Windows 10 and 11 is this:

apiVersion: v1
kind: ConfigMap
metadata:
  name: ipsec-conf
data:
  ipsec.conf: |
    config setup
            uniqueids=yes

    conn %default
            ikelifetime = 60m
            keylife=20m
            rekeymargin=3m
            keyingtries=1
            keyexchange=ikev2

    conn ike
            left=%any
            leftcert=vpn.example.com.pem
            leftsubnet=172.16.40.0/24
            leftid=vpn.example.com
            fragmentation=no

    conn vpn
            also=ike
            type=tunnel
            rightauth=eap-mschapv2
            rightsourceip=172.16.41.0/24
            rightdns=172.16.40.2
            leftsubnet=0.0.0.0/0
            eap_identity=%identity
            ike=aes256-sha256-modp2048,aes256-sha1-modp1024
            esp=aes256-sha256
            leftsendcert=always
            auto=add

ipsec-secrets.yaml

The ipsec-secrets.yaml contains /etc/ipsec.secrets. This file contains things like what a certificate name means, or usernames and passwords. It is sensitive so we store it as a secret.

apiVersion: v1
kind: Secret
metadata:
  name: ipsec-secrets
type: Opaque
stringData:
  ipsec.secrets: |
    vpn.example.com : RSA "vpn.example.com.pem"
    username : EAP "superstrongpassword"

private-keys.yaml

The private-keys.yaml file contains the private key portion of your VPN certificates. It gets mounts in the /etc/ipsec.d/private directory. The value of the entry is the contents of vpn/vpn.example.com.key.pem

apiVersion: v1
kind: Secret
metadata:
  name: private-keys
type: Opaque
stringData:
  vpn.example.com.pem: |
    -----BEGIN PRIVATE KEY-----
    MII.
    .
    .
    -----END PRIVATE KEY-----

Conclusion

I was lucky that I already had a base ipsec.conf and ipsec.secrets to use for this project. It may be helpful for you, at least it was for me, to not set up the VPN inside of Kubernetes right from the start. Instead, start with it on a dedicated, test system. It was very helpful for me to do it that way. And once I had everything ironed out on that, setting it up in Kubernetes was easy.