Falco tells you what a container did, not what it might do

A cluster passes every scan. Images are clean, policies are enforced, the posture score is green. Then an application with a deserialisation flaw gets exploited, a reverse shell opens inside a running container, and the attacker spends four days reading secrets and moving laterally. Nothing in the pre-deployment toolchain was looking, because none of it watches a process that has already started.

That is the gap runtime detection fills, and it is the weakest layer in most cloud security programmes. Teams invest heavily in what goes into the cluster and almost nothing in what happens inside it.

What it actually sees

Falco watches kernel-level events, historically through a kernel module and now typically through eBPF, and evaluates them against rules. Because it sits at the syscall boundary, it sees what actually happened rather than what a manifest declared.

The events worth alerting on fall into a small number of categories, and this list is most of the value:

• A shell spawned inside a container. In a production container running a single application, an interactive shell is either an engineer debugging in a way that should be prohibited, or an intrusion. Either way you want to know.
• A process writing to a directory it should never write to, such as a binary path, which is how persistence is usually established.
• An unexpected outbound connection, particularly from a container that should only receive traffic.
• Sensitive file reads: the service account token, a mounted secret, the container runtime socket.
• A container running with unexpected privileges, or a process attempting to escape the namespace.
• Package manager execution at runtime, which in a production image means someone is installing tools.

That last one is a good example of why this layer matters. An image scanner cannot tell you that curl was downloaded and executed after the container started, because it was not in the image.

The default rules are noisy, and tuning is the project

Install Falco with default rules on a busy cluster and you will get hundreds of alerts a day. Almost all will be legitimate behaviour, and within two weeks the channel will be muted. This is the single reason most Falco deployments fail.

Plan for tuning as the work rather than the setup as the work.

Run in audit mode first for two weeks, collecting without alerting. Then take the highest-volume rules and look at what triggered them. You will find your own CI jobs, your backup agent, your service mesh's init container and a health check doing something unusual.

Write exceptions against the specific process, image and container rather than disabling the rule. A disabled rule is a permanent blind spot; a scoped exception is a documented decision. Keep the exceptions in version control with a comment explaining each one, and review them quarterly, because exceptions added during an incident become permanent by default.

Then narrow ruthlessly. Ten rules that fire only on genuinely unexpected behaviour are worth more than two hundred that fire constantly, and the reasoning is identical to alerts people answer: once the false positive rate passes about one in three, human attention collapses.

Where the alerts have to go

An alert that arrives in a channel nobody watches is a log entry with extra steps.

Route Falco output to your SIEM rather than to a dashboard, so it correlates with everything else. Wazuh works well for a small team, as covered in Wazuh as a small team's SIEM, and the same applies to a commercial platform.

Each rule needs a severity and a destination decided in advance. A shell in a production container pages. A sensitive file read in a development namespace becomes a ticket. Most things become searchable context and nothing else.

And each alerting rule needs a response documented before it fires. What does the on-call engineer do when a shell opens in a payments pod at three in the morning? If the answer is not written down, the alert produces confusion rather than containment, which is the discipline in the runbook you use at 3am.

Response, and the trap of automating it

The temptation is to kill the pod automatically on a high-severity detection. Resist it initially.

Killing the pod destroys the evidence, and on a false positive it causes an outage. It also tips off the attacker, who now knows they are detected and may move faster.

The sequence that works is to capture first: a process tree, the network connections, the container's filesystem state. Falco can trigger a capture, and some deployments snapshot the node. Then contain by applying a network policy that isolates the pod rather than deleting it, which stops the lateral movement while preserving the running state for forensics. Only then remediate.

Automated response belongs on the small set of detections where you are confident and where speed genuinely matters, such as cryptomining, and only after the rule has been quiet for months.

Where it fits with everything else

Falco is one layer and it is not a replacement for the others.

Pre-deployment catches known vulnerabilities and misconfiguration: image scanning as in Trivy and Grype, and admission control as in Kyverno or Gatekeeper. Cheapest and earliest, and blind to anything that happens later.

Configuration posture checks the cluster against benchmarks, which is the subject of Kubescape and cluster posture. Continuous but static.

Runtime is the only layer that sees behaviour. It is also the most expensive to operate, because rules need tuning and alerts need humans.

The ordering matters. Runtime detection on a cluster with privileged containers everywhere and no network policy will drown you. Fix the posture first, then the detection becomes meaningful.

The things people forget

• It is a DaemonSet on every node, with a resource cost and a privileged position. Give it requests and limits, and monitor whether it is actually running everywhere, because a node without it is an invisible gap.
• The eBPF driver depends on the kernel. A node image upgrade can break it, which fails silently. Alert on Falco not reporting from a node.
• Managed services vary. Some managed Kubernetes offerings restrict what you can load, and some clouds offer their own runtime detection that may be easier to operate even if less flexible.
• It cannot see inside a process. A compromised application making legitimate-looking syscalls is invisible to it.
• Rule updates change behaviour. Pin the ruleset version and review changes rather than pulling the latest automatically.
• Audit logs are a second source. Kubernetes API audit logs catch a different class of activity, and the two together are considerably stronger than either alone.

What to do this week

Deploy Falco to a non-production cluster in audit mode with default rules, wait a week, and count the alerts by rule. That count tells you two things at once: how much tuning the tool needs, and how much of your own workload behaviour is doing things you did not expect. In our experience the second finding is the more interesting one. We run this in the detection phase of a security engagement.