Tagged as: DevOps Kubernetes

Accessing the Kubernetes API from a pod in Bash

I have a use case for Kubernetes, a pod gets a job to run from a queue, runs it, deletes itself. Here's how I accomplished it in Kubernetes.

Edward Cooke

December 14, 2021 • 4 minutes to read

I have a use case for Kubernetes, a pod gets a job to run from a queue, runs it, deletes itself. Here's how I accomplished it in Kubernetes.

ArgoCD is really cool. However, one thing it does, is when a job is running it keeps its application in a Progressing status. This is expected behavior.

To get around this I am using a deployment with replicas set to 2. I am then having my pod run its job then delete itself using the Kubernetes API.

There are a few things to take care of. So we will tackle it step by step.

First is to get the current pod name, this is easy. There is an environment variable called HOSTNAME which contains that value.

Second is to build the API URL. There are 3 parts. The API hostname, the namespace name and the pod name. We will get the API hostname and pod name from the environment variables KUBERNETES_PORT_443_TCP_ADDR and HOSTNAME. We get the namespace from the contents of /var/run/secrets/kubernetes.io/serviceaccount/namespace. The way to build the URL is this: https://${KUBERNETES_PORT_443_TCP_ADDR}/api/v1/namespaces/$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace)/pods/${HOSTNAME}

Now that we have the URL, we need to get the token to connect to the API with. That is stored in the /var/run/secrets/kubernetes.io/serviceaccount/token file. We can get that using $(cat /var/run/secrets/kubernetes.io/serviceaccount/token). Easy enough.

Now, putting that all together lets build a simple test CURL command:

curl -X GET https://${KUBERNETES_PORT_443_TCP_ADDR}/api/v1/namespaces/$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace)/pods/${HOSTNAME} --header "Authorization: Bearer $(cat /var/run/secrets/kubernetes.io/serviceaccou
nt/token)" --insecure

I am using a test pod in a namespace named test that I run from kubectl. I am not doing a deployment at this point.

You can start that test pod with:

kubectl run swissarmy4 --image=svil/swiss-army-knife -it --rm -- /bin/bash

If you run that right now you will most likely get the following response:

{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {

  },
  "status": "Failure",
  "message": "pods \"swissarmy4\" is forbidden: User \"system:serviceaccount:test:default\" cannot get resource \"pods\" in API group \"\" in the namespace \"test\"",
  "reason": "Forbidden",
  "details": {
    "name": "swissarmy4",
    "kind": "pods"
  },
  "code": 403
}

This is a good thing. This means you are talking to the API and getting something back.

Now we need to create a Role and RoleBinding so we can get our pod.

Let us start with the Role, I named this file role.yaml

kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
 namespace: test
 name: role-test-account
rules:
- apiGroups:
  - ""
  resources:
  - "pods"
  verbs:
  - "get"

Apply that role with kubectl apply -f role.yaml

And then the RoleBinding, I named this file rolebinding.yaml:

kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
 name: role-test-account-binding
 namespace: test
subjects:
- kind: ServiceAccount
  name: default
  namespace: test
roleRef:
 kind: Role
 name: role-test-account
 apiGroup: rbac.authorization.k8s.io

Apply it with kubectl apply -f rolebinding.yaml

Now run that same curl command from inside your pod. You should get a whole lot more information about your pod. I'm not pasting it all here because it is a lot and doesn't help this post.

{
  "kind": "Pod",
  "apiVersion": "v1",
  "metadata": {
    "name": "swissarmy4",
    "namespace": "test",
    .
    .
    .
}

Now that we are successfully authenticating to the API and can interact with it, lets delete the pod.

Modify your role.yaml file to include the delete verb.

It should end up like this:

kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
 namespace: test
 name: role-test-account
rules:
- apiGroups:
  - ""
  resources:
  - "pods"
  verbs:
  - "delete"
  - "get"

Re-apply that Role with kubectl apply -f role.yaml.

Now we will modify the curl command to be a DELETE instead of a GET.

curl -X DELETE https://${KUBERNETES_PORT_443_TCP_ADDR}/api/v1/namespaces/$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace)/pods/${HOSTNAME} --header "Authorization: Bearer $(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" --insecure

Go ahead and run that in your pod. It should terminate the pod at that point with something similar to this:

Session ended, resume using 'kubectl attach swissarmy4 -c swissarmy4 -i -t' command when the pod is running
pod "swissarmy4" deleted

Links

Conclusion

This was a fun little exercise. We are using it for our build agents that we want to be a single use container. Live, die, repeat. Just like the movie, except we build instead of fight.