strongSwan inside of Kubernetes
I needed a VPN and I wanted to run it inside of Kubernetes. Here is how I did it.
I needed a VPN and I wanted to run it inside of Kubernetes. Here is how I did it.
I am first going to cover a little background on my setup, so you know where I am coming from. I have a single subnet, single router, dynamic public address, a typical household setup. I have a 7-node Kubernetes cluster. 3 control planes and 4 worker nodes, all Raspberry Pi's running Debian 10.
Certificate Authority
Before we begin, we need some certificates. Let's create them.
To create a very basic certificate authority, issue the following commands in an empty directory:
mkdir cacerts
mkdir certs
mkdir vpn
openssl genrsa -aes256 -out cacerts/ca-key.pem 4096
openssl req -new -x509 -days 3650 -key cacerts/ca-key.pem -sha256 -out cacerts/ca.pem
chmod 0400 cacerts/*.pem
cd vpn
That creates a simple directory structure for you certificate authority and generates a self-signed root certificate that will expire in 10 years.
Now that we have a certificate authority set up, we need to issue an SSL certificate for the VPN.
This is a little more complicated, but not by much. We need to set a few values and add some extensions to the certificate to be valid for use in Windows. To do this, create a file named extensions.cnf. The contents are this:
basicConstraints = CA:FALSE
keyUsage = digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth, 1.3.6.1.5.5.8.2.2
subjectAltName = DNS:vpn.example.com
The part that makes it work for Windows is the additional OID, 1.3.6.1.5.5.8.2.2.
Create another file named openssl.cnf. This will contain additional information about your VPN certificate. The contents of this file are this:
[ ca ]
default_ca = CA_default
[ CA_default ]
dir = ../
certs = $dir/certs
crl_dir = $dir/cacerts/crl
database = $dir/cacerts/index.txt
new_certs_dir = $dir/vpn
certificate = $dir/cacerts/ca.pem
serial = $dir/cacerts/ca.srl
crlnumber = $dir/cacerts/crlnumber
crl = $dir/cacerts/crl.pem
private_key = $dir/cacerts/ca-key.pem
# x509_extensions = usr_cert
name_opt = ca_default
cert_opt = ca_default
default_days = 1095
default_crl_days = 30
default_md = default
preserve = no
policy = policy_anything
unique_subject = no
extensions = v3_req
[ policy_anything ]
countryName = optional
stateOrProvinceName = optional
localityName = optional
organizationName = optional
organizationalUnitName = optional
commonName = supplied
emailAddress = optional
#####
[req]
prompt = no
distinguished_name = req_distinguished_name
req_extensions = v3_req
extensions = v3_req
[req_distinguished_name]
C = US
ST = My State
L = My City
O = Example
CN = vpn.example.com
[v3_req]
basicConstraints = CA:FALSE
keyUsage = digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth, 1.3.6.1.5.5.8.2.2
subjectAltName = DNS:vpn.example.com
You will need to change the subjectAltName in both files and the CN in the openssl.cnf file to be the fully qualified domain name of your VPN endpoint.
Now we have everything ready to create our certificate for the VPN.
Issue the following commands to create a new certificate, replace vpn.example.com with your fully qualified domain name of your VPN so it is easy to distinguish what files are what.
openssl req -new -sha256 -config ./openssl.cnf -keyout vpn.example.com.key.pem -out vpn.example.com.csr -nodes
openssl ca -config ./openssl.cnf -in vpn.example.com.csr -out vpn.example.com.pem -extfile extensions.cnf
Those 2 commands create a certificate signing request then signs it using your new certificate authority.
You will need to trust the root certificate cacerts/ca.pem on your client machine to be able to connect to your VPN.
Docker Image
The first hurdle, albeit a small one, is creating an image that has strongSwan in it. I based my image on Alpine. I used Alpine because I was wondering if I could use an image from a distribution other than the one running the cluster to manage routes and other underlying system resources. Turns out, you can, at least between Debian and Alpine with strongSwan.
My Dockerfile is as follows
FROM alpine:3.15.0
RUN apk add --no-cache strongswan openssl
VOLUME /etc/ipsec.d/aacerts
VOLUME /etc/ipsec.d/acerts
VOLUME /etc/ipsec.d/cacerts
VOLUME /etc/ipsec.d/certs
VOLUME /etc/ipsec.d/crls
VOLUME /etc/ipsec.d/ocspcerts
VOLUME /etc/ipsec.d/private
VOLUME /etc/ipsec.d/reqs
CMD ["ipsec", "start", "--nofork", "--auto-update", "10"]
It is basic and simple, it installs strongSwan and OpenSSL. Then sets up the volumes for the certificates and tells it to start ipsec in the foreground.
Build that image and push to your registry.
Kubernetes
Next, we need to setup our Kubernetes manifests. There are 6 of them at a minimum. The Deployment, 3 ConfigMaps and 2 Secrets.
The 3 ConfigMaps cover the cacerts and certs directories, as well as ipsec.conf. The 2 Secrets cover the private directory and ipsec.secrets.
Since we need to do some routing, we need to label the node that you want to run strongSwan on. This way you can set up your routers static route correctly. I named my label strongswan with a value of yes. It makes it easy to move if needed and keeps it on the correct node. I also have a node that I have tainted with dedicated and that is set to noschedule.
To do that run the following.
kubectl taint nodes nodenamehere dedicated=yes:NoSchedule
kubectl label nodes nodenamehere strongswan=yes
Now what is next is to apply the below manifests, forward ports UDP 500 and 4500 to your correct worker node and give it a try.
deployment.yaml
Create your Deployment manifest, I named mine deployment.yaml. The contents are below, be sure to change the image to match the image you created earlier and change the imagePullSecrets to be your correct secret name or remove it if it is not needed.
apiVersion: apps/v1
kind: Deployment
metadata:
name: vpn
spec:
selector:
matchLabels:
app.kubernetes.io/component: strongswan
app.kubernetes.io/part-of: strongswan
replicas: 1
revisionHistoryLimit: 1
template:
metadata:
labels:
app.kubernetes.io/name: strongswan
app.kubernetes.io/instance: strongswan
app.kubernetes.io/component: strongswan
app.kubernetes.io/part-of: strongswan
spec:
hostNetwork: true
nodeSelector:
strongswan: "yes"
tolerations:
- key: "dedicated"
operator: "Exists"
effect: "NoSchedule"
imagePullSecrets:
- name: azureregistry
containers:
- name: strongswan
image: example.azurecr.io/strongswan:1.0.2
securityContext:
privileged: true
resources:
requests:
memory: 50Mi
cpu: 10m
limits:
memory: 200Mi
cpu: 1000m
volumeMounts:
- mountPath: /etc/ipsec.conf
name: ipsec-conf
readOnly: false
subPath: ipsec.conf
- mountPath: /etc/ipsec.secrets
name: ipsec-secrets
readOnly: true
subPath: ipsec.secrets
- mountPath: /etc/ipsec.d/cacerts
name: cacerts
readOnly: true
- mountPath: /etc/ipsec.d/certs
name: certs
readOnly: true
- mountPath: /etc/ipsec.d/private
name: private-keys
readOnly: true
volumes:
- name: ipsec-conf
configMap:
name: ipsec-conf
items:
- key: ipsec.conf
path: ipsec.conf
- name: ipsec-secrets
secret:
secretName: ipsec-secrets
items:
- key: ipsec.secrets
path: ipsec.secrets
- name: cacerts
configMap:
name: cacerts
- name: certs
configMap:
name: certs
- name: private-keys
secret:
secretName: private-keys
cacerts.yaml
The cacerts.yaml file will be used to populate the /etc/ipsec.d/cacerts directory with your root certificate authority certificates. The value of the entry should be the contents of cacerts/ca.pem.
apiVersion: v1
kind: ConfigMap
metadata:
name: cacerts
data:
root.pem: |
-----BEGIN CERTIFICATE-----
MII.
.
.
-----END CERTIFICATE-----
certs.yaml
The certs.yaml file contains the public key of your VPN. The value of the entry should be the contents of vpn/vpn.example.com.pem
apiVersion: v1
kind: ConfigMap
metadata:
name: certs
data:
vpn.frakkingsweet.com.pem: |
-----BEGIN CERTIFICATE-----
MII.
.
.
-----END CERTIFICATE-----
ipsec-conf.yaml
The ipsec-conf.yaml will be very specific to your needs. Getting this correct is probably the hardest part of the whole project.
A basic one that works for me for doing ipsec with username/password authentication that works with Windows 10 and 11 is this:
apiVersion: v1
kind: ConfigMap
metadata:
name: ipsec-conf
data:
ipsec.conf: |
config setup
uniqueids=yes
conn %default
ikelifetime = 60m
keylife=20m
rekeymargin=3m
keyingtries=1
keyexchange=ikev2
conn ike
left=%any
leftcert=vpn.example.com.pem
leftsubnet=172.16.40.0/24
leftid=vpn.example.com
fragmentation=no
conn vpn
also=ike
type=tunnel
rightauth=eap-mschapv2
rightsourceip=172.16.41.0/24
rightdns=172.16.40.2
leftsubnet=0.0.0.0/0
eap_identity=%identity
ike=aes256-sha256-modp2048,aes256-sha1-modp1024
esp=aes256-sha256
leftsendcert=always
auto=add
ipsec-secrets.yaml
The ipsec-secrets.yaml contains /etc/ipsec.secrets. This file contains things like what a certificate name means, or usernames and passwords. It is sensitive so we store it as a secret.
apiVersion: v1
kind: Secret
metadata:
name: ipsec-secrets
type: Opaque
stringData:
ipsec.secrets: |
vpn.example.com : RSA "vpn.example.com.pem"
username : EAP "superstrongpassword"
private-keys.yaml
The private-keys.yaml file contains the private key portion of your VPN certificates. It gets mounts in the /etc/ipsec.d/private directory. The value of the entry is the contents of vpn/vpn.example.com.key.pem
apiVersion: v1
kind: Secret
metadata:
name: private-keys
type: Opaque
stringData:
vpn.example.com.pem: |
-----BEGIN PRIVATE KEY-----
MII.
.
.
-----END PRIVATE KEY-----
Conclusion
I was lucky that I already had a base ipsec.conf and ipsec.secrets to use for this project. It may be helpful for you, at least it was for me, to not set up the VPN inside of Kubernetes right from the start. Instead, start with it on a dedicated, test system. It was very helpful for me to do it that way. And once I had everything ironed out on that, setting it up in Kubernetes was easy.