I needed a VPN and I wanted to run it inside of Kubernetes. Here is how I did it.
I am first going to cover a little background on my setup, so you know where I am coming from. I have a single subnet, single router, dynamic public address, a typical household setup. I have a 7-node Kubernetes cluster. 3 control planes and 4 worker nodes, all Raspberry Pi's running Debian 10.
- Certificate Authority
- Docker Image
Before we begin, we need some certificates. Let's create them.
To create a very basic certificate authority, issue the following commands in an empty directory:
mkdir cacerts mkdir certs mkdir vpn openssl genrsa -aes256 -out cacerts/ca-key.pem 4096 openssl req -new -x509 -days 3650 -key cacerts/ca-key.pem -sha256 -out cacerts/ca.pem chmod 0400 cacerts/*.pem cd vpn
That creates a simple directory structure for you certificate authority and generates a self-signed root certificate that will expire in 10 years.
Now that we have a certificate authority set up, we need to issue an SSL certificate for the VPN.
This is a little more complicated, but not by much. We need to set a few values and add some extensions to the certificate to be valid for use in Windows. To do this, create a file named
extensions.cnf. The contents are this:
basicConstraints = CA:FALSE keyUsage = digitalSignature, keyEncipherment extendedKeyUsage = serverAuth, 126.96.36.199.188.8.131.52.2 subjectAltName = DNS:vpn.example.com
The part that makes it work for Windows is the additional OID,
Create another file named
openssl.cnf. This will contain additional information about your VPN certificate. The contents of this file are this:
[ ca ] default_ca = CA_default [ CA_default ] dir = ../ certs = $dir/certs crl_dir = $dir/cacerts/crl database = $dir/cacerts/index.txt new_certs_dir = $dir/vpn certificate = $dir/cacerts/ca.pem serial = $dir/cacerts/ca.srl crlnumber = $dir/cacerts/crlnumber crl = $dir/cacerts/crl.pem private_key = $dir/cacerts/ca-key.pem # x509_extensions = usr_cert name_opt = ca_default cert_opt = ca_default default_days = 1095 default_crl_days = 30 default_md = default preserve = no policy = policy_anything unique_subject = no extensions = v3_req [ policy_anything ] countryName = optional stateOrProvinceName = optional localityName = optional organizationName = optional organizationalUnitName = optional commonName = supplied emailAddress = optional ##### [req] prompt = no distinguished_name = req_distinguished_name req_extensions = v3_req extensions = v3_req [req_distinguished_name] C = US ST = My State L = My City O = Example CN = vpn.example.com [v3_req] basicConstraints = CA:FALSE keyUsage = digitalSignature, keyEncipherment extendedKeyUsage = serverAuth, 184.108.40.206.220.127.116.11.2 subjectAltName = DNS:vpn.example.com
You will need to change the
subjectAltName in both files and the
CN in the
openssl.cnf file to be the fully qualified domain name of your VPN endpoint.
Now we have everything ready to create our certificate for the VPN.
Issue the following commands to create a new certificate, replace
vpn.example.com with your fully qualified domain name of your VPN so it is easy to distinguish what files are what.
openssl req -new -sha256 -config ./openssl.cnf -keyout vpn.example.com.key.pem -out vpn.example.com.csr -nodes openssl ca -config ./openssl.cnf -in vpn.example.com.csr -out vpn.example.com.pem -extfile extensions.cnf
Those 2 commands create a certificate signing request then signs it using your new certificate authority.
You will need to trust the root certificate
cacerts/ca.pem on your client machine to be able to connect to your VPN.
The first hurdle, albeit a small one, is creating an image that has strongSwan in it. I based my image on Alpine. I used Alpine because I was wondering if I could use an image from a distribution other than the one running the cluster to manage routes and other underlying system resources. Turns out, you can, at least between Debian and Alpine with strongSwan.
Dockerfile is as follows
FROM alpine:3.15.0 RUN apk add --no-cache strongswan openssl VOLUME /etc/ipsec.d/aacerts VOLUME /etc/ipsec.d/acerts VOLUME /etc/ipsec.d/cacerts VOLUME /etc/ipsec.d/certs VOLUME /etc/ipsec.d/crls VOLUME /etc/ipsec.d/ocspcerts VOLUME /etc/ipsec.d/private VOLUME /etc/ipsec.d/reqs CMD ["ipsec", "start", "--nofork", "--auto-update", "10"]
It is basic and simple, it installs strongSwan and OpenSSL. Then sets up the volumes for the certificates and tells it to start
ipsec in the foreground.
Build that image and push to your registry.
Next, we need to setup our Kubernetes manifests. There are 6 of them at a minimum. The
ConfigMaps and 2
ConfigMaps cover the
certs directories, as well as
ipsec.conf. The 2
Secrets cover the
private directory and
Since we need to do some routing, we need to label the node that you want to run strongSwan on. This way you can set up your routers static route correctly. I named my label
strongswan with a value of
yes. It makes it easy to move if needed and keeps it on the correct node. I also have a node that I have tainted with
dedicated and that is set to
To do that run the following.
kubectl taint nodes nodenamehere dedicated=yes:NoSchedule kubectl label nodes nodenamehere strongswan=yes
Now what is next is to apply the below manifests, forward ports UDP 500 and 4500 to your correct worker node and give it a try.
Deployment manifest, I named mine
deployment.yaml. The contents are below, be sure to change the
image to match the image you created earlier and change the
imagePullSecrets to be your correct secret name or remove it if it is not needed.
apiVersion: apps/v1 kind: Deployment metadata: name: vpn spec: selector: matchLabels: app.kubernetes.io/component: strongswan app.kubernetes.io/part-of: strongswan replicas: 1 revisionHistoryLimit: 1 template: metadata: labels: app.kubernetes.io/name: strongswan app.kubernetes.io/instance: strongswan app.kubernetes.io/component: strongswan app.kubernetes.io/part-of: strongswan spec: hostNetwork: true nodeSelector: strongswan: "yes" tolerations: - key: "dedicated" operator: "Exists" effect: "NoSchedule" imagePullSecrets: - name: azureregistry containers: - name: strongswan image: example.azurecr.io/strongswan:1.0.2 securityContext: privileged: true resources: requests: memory: 50Mi cpu: 10m limits: memory: 200Mi cpu: 1000m volumeMounts: - mountPath: /etc/ipsec.conf name: ipsec-conf readOnly: false subPath: ipsec.conf - mountPath: /etc/ipsec.secrets name: ipsec-secrets readOnly: true subPath: ipsec.secrets - mountPath: /etc/ipsec.d/cacerts name: cacerts readOnly: true - mountPath: /etc/ipsec.d/certs name: certs readOnly: true - mountPath: /etc/ipsec.d/private name: private-keys readOnly: true volumes: - name: ipsec-conf configMap: name: ipsec-conf items: - key: ipsec.conf path: ipsec.conf - name: ipsec-secrets secret: secretName: ipsec-secrets items: - key: ipsec.secrets path: ipsec.secrets - name: cacerts configMap: name: cacerts - name: certs configMap: name: certs - name: private-keys secret: secretName: private-keys
cacerts.yaml file will be used to populate the
/etc/ipsec.d/cacerts directory with your root certificate authority certificates. The value of the entry should be the contents of
apiVersion: v1 kind: ConfigMap metadata: name: cacerts data: root.pem: | -----BEGIN CERTIFICATE----- MII. . . -----END CERTIFICATE-----
certs.yaml file contains the public key of your VPN. The value of the entry should be the contents of
apiVersion: v1 kind: ConfigMap metadata: name: certs data: vpn.frakkingsweet.com.pem: | -----BEGIN CERTIFICATE----- MII. . . -----END CERTIFICATE-----
ipsec-conf.yaml will be very specific to your needs. Getting this correct is probably the hardest part of the whole project.
A basic one that works for me for doing
ipsec with username/password authentication that works with Windows 10 and 11 is this:
apiVersion: v1 kind: ConfigMap metadata: name: ipsec-conf data: ipsec.conf: | config setup uniqueids=yes conn %default ikelifetime = 60m keylife=20m rekeymargin=3m keyingtries=1 keyexchange=ikev2 conn ike left=%any leftcert=vpn.example.com.pem leftsubnet=172.16.40.0/24 leftid=vpn.example.com fragmentation=no conn vpn also=ike type=tunnel rightauth=eap-mschapv2 rightsourceip=172.16.41.0/24 rightdns=172.16.40.2 leftsubnet=0.0.0.0/0 eap_identity=%identity ike=aes256-sha256-modp2048,aes256-sha1-modp1024 esp=aes256-sha256 leftsendcert=always auto=add
/etc/ipsec.secrets. This file contains things like what a certificate name means, or usernames and passwords. It is sensitive so we store it as a secret.
apiVersion: v1 kind: Secret metadata: name: ipsec-secrets type: Opaque stringData: ipsec.secrets: | vpn.example.com : RSA "vpn.example.com.pem" username : EAP "superstrongpassword"
private-keys.yaml file contains the private key portion of your VPN certificates. It gets mounts in the
/etc/ipsec.d/private directory. The value of the entry is the contents of
apiVersion: v1 kind: Secret metadata: name: private-keys type: Opaque stringData: vpn.example.com.pem: | -----BEGIN PRIVATE KEY----- MII. . . -----END PRIVATE KEY-----
I was lucky that I already had a base
ipsec.secrets to use for this project. It may be helpful for you, at least it was for me, to not set up the VPN inside of Kubernetes right from the start. Instead, start with it on a dedicated, test system. It was very helpful for me to do it that way. And once I had everything ironed out on that, setting it up in Kubernetes was easy.