Your detections are untested until you generate the behaviour

Ask a security team how many of their detection rules work. The confident answer is a coverage percentage from a dashboard. The honest answer is that nobody has checked, because the only evidence a rule works is that it fired, and most rules have never fired.

That is the problem adversary emulation solves. Rather than waiting for an attacker to test your detections, you run the technique yourself, on purpose, and see what happens.

What it is, and how it differs from a penetration test

A penetration test asks whether an attacker can get in. It is adversarial, scoped by time, and its output is a list of exploitable weaknesses. Valuable, and it says almost nothing about whether you would have noticed.

Adversary emulation asks whether you can see a known technique when it happens. It is not adversarial, it is not trying to be stealthy, and its output is a coverage matrix: for each technique, did the telemetry capture it, did a rule fire, did an alert reach a human.

The distinction matters because they find different failures. A penetration test finds the way in. Emulation finds that you have excellent endpoint coverage and no visibility at all into your cloud control plane, which is the gap that turns a small intrusion into a four-day one.

Purple teaming is the collaborative form: the offensive side runs a technique while the defensive side watches, and both sides fix what they find in the same session. It is dramatically more productive than a red team exercise followed by a report three weeks later, because the feedback loop is minutes.

Atomic tests are the cheap way in

Atomic Red Team is a library of small, self-contained tests, each executing one technique with a documented command, mapped to an attack framework, with a cleanup step.

The appeal is the granularity. You are not running a full attack chain; you are running one action, such as creating a scheduled task for persistence or dumping a specific credential store, and immediately asking whether anything noticed.

The workflow is simple enough to run in an afternoon: pick a technique relevant to your environment, run the test on a machine you control, check your telemetry for the event, check whether a rule matched, check whether an alert arrived. Record the result in a matrix and run the cleanup.

Full emulation frameworks exist for chaining techniques into realistic sequences, and they are worth graduating to. Start atomic, because the individual results are actionable and a full chain produces a narrative rather than a list of fixes.

The four outcomes, and what each means

Every test lands in one of four states, and distinguishing them is the point of the exercise.

No telemetry. The event was not recorded anywhere. This is the worst outcome and the most common surprise. No rule can detect what was never logged, and the fix is a logging change rather than a detection one, which usually has a cost attached.

Telemetry but no rule. The data is there and nothing is looking. The cheapest fix on the list: write the rule, as described in detection as code.

Rule fired but no alert reached anyone. Severity too low, routing broken, channel muted. Common and embarrassing, and the fix is in the alerting pipeline rather than in detection, which is the discipline of alerts people answer.

Alert reached a human who knew what to do. The only passing state, and it requires the runbook to exist, which is the argument in the runbook you use at 3am.

Most first exercises produce a distribution that shocks the team: a large share in the first two states, concentrated in exactly the areas nobody has tested.

Test what matters to you, not the whole framework

Attack technique frameworks contain hundreds of techniques. Attempting to cover all of them produces a long project and a matrix nobody reads.

Prioritise by what would actually be used against you. For a cloud-native company that means identity and cloud control plane techniques: credential creation, role assumption from unusual locations, permission escalation, disabling logging, snapshot sharing to an external account, mass deletion of backups.

Those are the ones most likely to appear in a real incident and least likely to be covered, because most detection content is written for a traditional endpoint estate. Emulating them is also straightforward: create a test account, assume a role from an unexpected location, disable a trail in a sandbox, and watch what happens.

For a container estate, add the runtime techniques: shell in a container, service account token read, mount of the container runtime socket, outbound connection from a workload that should not initiate one. These are the cases Falco tells you what a container did is meant to cover, and emulation is how you verify it actually does.

Running it safely

This is real attacker behaviour executed on your systems, so the controls matter.

Get written authorisation before anything runs, naming the systems, the window, the techniques and the people involved. Tell your own security operations team unless you are deliberately testing their response, and if you are, tell at least one person so it can be stopped.

Run in a non-production environment first, always. Some tests modify system state, and the cleanup scripts are good but not guaranteed. Read every test before running it rather than executing a pack wholesale.

Avoid anything destructive or genuinely disruptive. Emulation is about detection coverage, not about proving you can cause damage, and the techniques worth testing are almost all observable without harm.

Keep a log of what was run, when and by whom, so that an alert generated by the exercise can be identified as such and so that nobody spends an afternoon investigating you.

Make it continuous, then it is worth the setup

A one-off exercise produces a snapshot and a list of gaps. Run it again in six months and a quarter of the results will have changed, because a log source moved, a rule was tuned into uselessness, or a platform upgrade changed an event format.

The mature version runs a subset automatically on a schedule against a test environment, as a regression test for detection. A rule that stops matching should break a build, in the same way an application test does, which is the natural extension of keeping detections in a repository.

Track one number over time: the percentage of tested techniques reaching the fourth state. That single figure is a far more honest measure of detection capability than a count of rules, and it is the one to show leadership.

The things people forget

• Coverage percentages are self-reported fiction until tested. A dashboard claiming ninety percent coverage is claiming that rules exist, not that they work.
• Cleanup sometimes fails. Verify the system state afterwards rather than trusting the script.
• Emulation generates alerts that look real. Coordinate, or you will trigger a genuine incident response.
• Detection works until a platform update. Event formats and field names change, and rules silently stop matching.
• It does not test the response. Knowing an alert fired is different from knowing the team could contain the intrusion. That needs a game day, as in the first five chaos experiments applied to security.

What to do this week

Pick one technique: create a new access key for a privileged identity in your cloud, in a sandbox account. Then answer three questions. Was it logged, did a rule match, did anyone find out. That single test takes fifteen minutes and in most organisations it fails at the second or third question, which makes it the most persuasive fifteen minutes available to a security team. We run this exercise in the detection phase of a security engagement.