I built my cluster, now I wanted to make a new user with full privileges.

I'm building a brand new, bare bones, raw, Kubernetes cluster. I used kubeadm and built up a cluster with 3 control planes and 2 worker nodes. Now I want to add a new user with full privileges. This is primarily for the "can I do it" and how thought. But, it's more secure to have separate certs for each user anyways so it all applies in the real world.

A little background and few key facts.

  1. My cluster is Kubernetes 1.20
  2. I'm using basic certificate authentication with RBAC which is the default cluster configuration.
  3. There is a system:masters group that has full permissions to the cluster.
  4. The system:masters group is tied to a ClusterRoleBinding named cluster-admins.
  5. A default kubeadm cluster won't let you add a user that has the system:masters group.
  6. Groups are assigned in the organization part of the subject name of the certificate.
  7. Our users name is going to be named test so replace test with the appropriate user. I'll have scripts and other stuff at the end.
  8. My new group name is example:masters

Solution:

  1. Create a new role binding.
  2. Create a new user.
  3. Set the values in a new config.

Create the ClusterRoleBinding

  1. Create the ClusterRoleBinding yaml file example-cluster-admin.yaml
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRoleBinding
    metadata:
      name: example-cluster-admin
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: ClusterRole
      name: cluster-admin
    subjects:
    - apiGroup: rbac.authorization.k8s.io
      kind: Group
      name: example:masters
    
  2. Apply the ClusterRoleBinding
    kubectl apply -f example-cluster-admin.yaml
    

Create a new user

  1. Create a certificate key using openssl.
    openssl genrsa -out test.key 4096
    
  2. Create a signing request.
    openssl req -new -key ${USERNAME}.key -out ${USERNAME}.csr -subj '/CN=${USERNAME}/O=example:masters'
    
  3. Get the base 64 encoded signing request.
    cat test.csr | base64 | tr -d '\n'
    
  4. Create the CertificateSigningRequest yaml file test.yaml to add to the cluster
    apiVersion: certificates.k8s.io/v1
    kind: CertificateSigningRequest
    metadata:
      name: test-csr
    spec:
      groups:
      - system:authenticated
      request: BASE64ENCODEDCSRFROMSTEP3
      signerName: kubernetes.io/kube-apiserver-client
      usages:
      - digital signature
      - key encipherment
      - client auth
    
  5. Apply the CSR yaml file.
    kubectl apply -f test.yaml
    
  6. Approve the signing request
    kubectl certificate approve ${USERNAME}-csr
    
  7. Get the certificates for the config file
    KEY=`cat ${USERNAME}.key | base64 | tr -d '\n'`
    CERT=`kubectl get csr ${USERNAME}-csr -o jsonpath='{.status.certificate}'`
    
    echo "======KEY"
    echo ${KEY}
    echo
    
    echo "======Cert"
    echo $CERT
    echo
    

Update the local Kubernetes config file

Open up the ~/.kube/config file and change out the client-certificate-data with the value of the Cert and client-key-data with the value of the Key that we got in the last step of the user creation process.

Scripts and other useful stuff

create-user.sh - Creates and approves (if the current user can) a new user and dumps out a working config that can be given to someone

#!/bin/bash

USERNAME=${1}

echo + Creating private key: ${USERNAME}.key
openssl genrsa -out ${USERNAME}.key 4096

echo + Creating signing request: ${USERNAME}.csr
openssl req -new -key ${USERNAME}.key -out ${USERNAME}.csr -subj '/CN=${USERNAME}/O=example:masters'

cp signing-request-template.yaml ${USERNAME}-signing-request.yaml
sed -i "s@__USERNAME__@${USERNAME}@" ${USERNAME}-signing-request.yaml

B64=`cat ${USERNAME}.csr | base64 | tr -d '\n'`
sed -i "s@__CSRREQUEST__@${B64}@" ${USERNAME}-signing-request.yaml

echo + Creating signing request in kubernetes
kubectl create -f ${USERNAME}-signing-request.yaml

echo + List of signing requests
kubectl get csr

kubectl certificate approve ${USERNAME}-csr

KEY=`cat ${USERNAME}.key | base64 | tr -d '\n'`
CERT=`kubectl get csr ${USERNAME}-csr -o jsonpath='{.status.certificate}'`

echo "======KEY"
echo ${KEY}
echo

echo "======Cert"
echo $CERT
echo

echo "======Config"
cat ~/.kube/config | \
    sed -r "s/^(\s*)(client-certificate-data:.*$)/\1client-certificate-data: ${CERT}/" | \
    sed -r "s/^(\s*)(client-key-data:.*$)/\1client-key-data: ${KEY}/"
echo

signing-request-template.yaml - used by the above script to generate the CertificateSigningRequest object in Kubernetes

apiVersion: certificates.k8s.io/v1
kind: CertificateSigningRequest
metadata:
  name: __USERNAME__-csr
spec:
  groups:
  - system:authenticated
  - example:masters
  request: __CSRREQUEST__
  signerName: kubernetes.io/kube-apiserver-client
  usages:
  - digital signature
  - key encipherment
  - client auth

Conclusion

Doing this was annoying, but not as annoying as disabling the admission controller that was blocking the ability to add users to the system:masters group.

My brother helped out a lot with the initial user creation part.

I don't generally apply manifests using kubectl by hand, instead I'll either use a script to do it or use ArgoCD and store my manifests in a git repo. Which is exactly what I did for my ClusterRoleBinding.

Links

My brothers site

WCooke.org

Kubernetes docs

Authenticating
This page provides an overview of authenticating.Users in Kubernetes All Kubernetes clusters have two categories of users: service accounts managed by Kubernetes, and normal users.It is assumed that a cluster-independent service manages normal users in the following ways: an administrator distrib…
Certificate Signing Requests
FEATURE STATE: Kubernetes v1.19 [stable] The Certificates API enables automation of X.509 credential provisioning by providing a programmatic interface for clients of the Kubernetes API to request and obtain X.509 certificates from a Certificate Authority (CA).A CertificateSigningRequest (CSR) res…
Using RBAC Authorization
Role-based access control (RBAC) is a method of regulating access to computer or network resources based on the roles of individual users within your organization.RBAC authorization uses the rbac.authorization.k8s.io API group to drive authorization decisions, allowing you to dynamically configure …
Using Admission Controllers
This page provides an overview of Admission Controllers.What are they? An admission controller is a piece of code that intercepts requests to the Kubernetes API server prior to persistence of the object, but after the request is authenticated and authorized. The controllers consist of the list belo…
Argo CD - Declarative GitOps CD for Kubernetes