Darkbit has joined Aqua Security! Big news! We're excited to announce that Darkbit has joined Aqua Security!
Recently, when doing a bit of research of how Falco rules work, we discovered a default rule that alerts when privileged containers or containers that mount sensitive file paths are run inside a Kubernetes cluster could be “bypassed” if the image name was cleverly formatted.
If you haven’t heard of Falco and how it monitors for malicious activity on a Linux host and inside Kubernetes clusters, head on over to the documentation. It’s a powerful, rules-based engine created by Sysdig and now a CNCF incubating project that can help you gain insight into actions that an attacker in a containerized environment will take, among a host (pun intended) of other useful things.
Prior to v0.25.0, the Launch Privileged Container rule looked like this:
- rule: Launch Privileged Container
desc: Detect the initial process started in a privileged container. Exceptions are made for known trusted images.
condition: >
container_started and container # Macros that identify this event as a newly started container
and container.privileged=true # This container has the privileged flag
and not falco_privileged_containers # Not in the falco_privileged_containers allow-list
and not user_privileged_containers # Not in the user_privileged_containers allow-list
output: Privileged container started (user=%user.name command=%proc.cmdline %container.info image=%container.image.repository:%container.image.tag)
priority: INFO
tags: [container, cis, mitre_privilege_escalation, mitre_lateral_movement]
Looking at the falco_privileged_containers macro which defines what is considered “allowed”:
- macro: falco_privileged_containers
condition: (openshift_image or
user_trusted_containers or
container.image.repository in (trusted_images) or
container.image.repository in (falco_privileged_images) or
container.image.repository startswith istio/proxy_ or
container.image.repository startswith quay.io/sysdig)
and looking at the user_privileged_containers macro:
- macro: user_privileged_containers
condition: (container.image.repository endswith sysdig/agent)
We can see the potential loopholes for being able to bypass this rule are in the last lines of each macro:
...
container.image.repository startswith quay.io/sysdig)
and
...
condition: (container.image.repository endswith sysdig/agent)
Creating an organization with DockerHub and Quay or GCP Project Names (which become the GCR repository name) are free or very low cost. So, if one were to place images in one of these locations and ran it inside a cluster with the privileged: true set, they would avoid generating a Falco alert assuming they violated no other rules:
gcr.io/my-project-name-that-ends-with-sysdig/agentquay.io/sysdig-and-some-other-words/anyimage:anytagdocker.io/my-org-name-that-ends-with-sysdig/agentIn the latest version, the macros are now:
- macro: falco_privileged_containers
condition: (openshift_image or
user_trusted_containers or
container.image.repository in (trusted_images) or
container.image.repository in (falco_privileged_images) or
container.image.repository startswith istio/proxy_ or
container.image.repository startswith quay.io/sysdig/) # <- Notice the trailing slash
and
- macro: user_privileged_containers
condition: (never_true)
thereby closing the loophole.
Following a similar pattern, containers with crafted names could also be allowed to mount sensitive paths on the underlying host without triggering a Falco alert. Here’s the prior rule:
- rule: Launch Sensitive Mount Container
desc: >
Detect the initial process started by a container that has a mount from a sensitive host directory
(i.e. /proc). Exceptions are made for known trusted images.
condition: >
container_started and container # Macros that identify this event as a newly started container
and sensitive_mount # Has a mount path like /, /etc, /var/run/docker.sock, etc
and not falco_sensitive_mount_containers # A list of allowed containers for Falco-owned images
and not user_sensitive_mount_containers # A list for users to place their allow-list
output: Container with sensitive mount started (user=%user.name command=%proc.cmdline %container.info image=%container.image.repository:%container.image.tag mounts=%container.mounts)
priority: INFO
tags: [container, cis, mitre_lateral_movement]
The sensitive_mount macro lists the most critical paths to be aware of in a containerized environment:
- macro: sensitive_mount
condition: (container.mount.dest[/proc*] != "N/A" or
container.mount.dest[/var/run/docker.sock] != "N/A" or
container.mount.dest[/var/run/crio/crio.sock] != "N/A" or
container.mount.dest[/var/lib/kubelet] != "N/A" or
container.mount.dest[/var/lib/kubelet/pki] != "N/A" or
container.mount.dest[/] != "N/A" or
container.mount.dest[/home/admin] != "N/A" or
container.mount.dest[/etc] != "N/A" or
container.mount.dest[/etc/kubernetes] != "N/A" or
container.mount.dest[/etc/kubernetes/manifests] != "N/A" or
container.mount.dest[/root*] != "N/A")
Here’s the previous falco_sensitive_mount_containers macro that allows the same image name-based bypass:
- macro: falco_sensitive_mount_containers
condition: (user_trusted_containers or
container.image.repository in (trusted_images) or
container.image.repository in (falco_sensitive_mount_images) or
container.image.repository startswith quay.io/sysdig)
And the newest version includes the trailing / like so:
- macro: falco_sensitive_mount_containers
condition: (user_trusted_containers or
container.image.repository in (trusted_images) or
container.image.repository in (falco_sensitive_mount_images) or
container.image.repository startswith quay.io/sysdig/)
If an attacker or malicious developer has the RBAC permissions to launch a pod and the cluster does not have Admission Control implemented to prevent arbitrary images from running in a cluster running Falco with unmodified rules, they could create a carefully named image repository, push an image of their choosing there, and run it inside the cluster without Falco generating an alert.
It’s important to understand that there is no vulnerability in Falco’s “engine” here. It’s a sneaky logic bug in the declaration of two default rules/macros that allow undesired image paths to also match. Upgrading Falco’s agent (although recommended for other reasons) isn’t required to address this issue. Instead, an update to the latest falco_rules.yaml is the relatively painless solution.
After upgrading your rules, check out the Falco Community repo and get involved. Thanks to the Falco team for all the hard work making such a useful piece of software, for making the responsible disclosure process straightforward to follow, and for the quick turn-around on the updated rules.