The Playbook Library Is Full. The Automation Rate Has Not Moved.

A full library and a queue worked by hand

Some months after go-live the library holds more than anyone planned for. Playbooks were written for the alert types that arrive most often, adapted from Marketplace content packs, or built around the enrichment steps analysts had been doing by hand and the containment actions the team agreed were safe. Each was tested against a real case and each worked. Open the queue on a Tuesday morning and the work is being done the way it was done before any of them existed.

An alert lands as an issue, causality groups the related issues into a case, and the case carries a score describing how urgent it is. A playbook runs against an issue and does what it was written to do. The playbooks execute when they are permitted to execute. Permission is granted by named people in your organization, and the promotion from advisory to acting is the decision this article is about.

So the audit worth running asks something narrower than how many playbooks exist. For each one: what is it set to do when it fires, and when did it last fire. The first half of that is in the playbook. The second half is on the issues, because each run is recorded on the Work Plan tab of the issue it ran against, showing every task, the state it ended in, and what went into it and came out. Reading one issue tells you what happened that night. Reading what a playbook has done since March means querying the playbook_runs and playbook_tasks datasets in Cortex Query Language, and the person who writes that query is the one who knows which playbooks matter here.

Run that and the answers sort into three kinds.

• It runs on every matching alert and gathers only. It attaches what it found, and the analyst opens the consoles behind it to confirm what it found.
• It runs, proposes an action, and waits on an approval that arrives while the shift is quiet and sits until the handover when it is not.
• It was switched off during an incident and left off. The reason lives in one person's memory of a difficult afternoon.

At two in the morning the playbook is waiting on an approval that arrives after the shift change, and the analyst does it by hand.

The rungs

Those answers form a ladder. Each step up transfers a small amount of authority from a person to a process, and the word for what changes at each step is trust. An organization climbs it one playbook at a time, or it stays where the build left it.

Every rung above the first has to be granted by a person, and grants are scarce, so a library accumulates where the granting stopped. On the three deployments behind this article the shape was the same: heavy at the bottom, thin above the middle, and the playbooks themselves were well built.

• Off. The playbook exists and has never run against a live alert. Everything about it is still theory, including whether it would have been right.
• On demand. A person decides a case warrants it and runs it. Every judgment stays with the analyst, and so does the timing.
• Advisory. It runs on every matching alert and produces information. It leaves the estate exactly as it found it, so the cost of it being wrong is zero, and a rung that cheap is a comfortable place for a library to settle.
• Gated. It runs, proposes an action, and waits for a named approver. The analysis has moved off the analyst and the case now waits on somebody who is asleep for part of every day.
• Bounded and unattended. It acts inside a written boundary and tells somebody afterward. The boundary carries the whole decision: which assets, which hours, which actions, and what it may never touch.

Advisory automation costs nothing to be wrong on. Promoting it costs a named person a signature, and somebody has to be given the job of going to ask for one.

What holds a playbook on its rung

The ladder continues past the fifth rung, where somebody widens or narrows the boundary on a schedule against what has happened since. Reaching that rung requires having already promoted something, which makes it a later problem than the one this article is about.

On three Cortex XSIAM deployments CWS carried through post-deployment automation work, and the thing holding a playbook down was never its logic. The logic was sound, or fixable in an afternoon by an engineer who knew the environment. What held it in place was organizational, and the same three things came up on each of the three.

The first is enrichment nobody can vouch for. A playbook attaches a verdict to a case, and the analyst checks that verdict by hand, because how it was reached and how often it has held up are things they would have to go and find out. The playbook fires on every matching alert, the run history looks healthy, and the case takes as long as it always did. Checking a number whose derivation you have never seen is what a careful analyst does.

The second is the asymmetry between off and on. Switching a playbook off during an incident is a safe act. Any analyst can do it in seconds and be thanked for the judgment. Switching it back on is a risk somebody has to sign for. So the one that misfired in March is still off, the fix was agreed in a chat thread and never applied, and re-enabling it needs a person with the standing to carry it if the afternoon repeats. Handing somebody that standing is itself a decision, and it sits behind everything else on the list.

The third is that the failure case has no owner. Ask who answers for a wrong automated containment, then time the pause before anybody speaks. Until that pause has a name in it, the analyst who leaves the approval gate switched on is making the correct call, and encouragement is the wrong tool for changing their mind.

• A playbook that fires constantly and never shortens a case. The enrichment is being redone by hand somewhere out of sight.
• An approval gate pointing at a role staffed for part of the day. It was set during the build, when the team sat in one room and approval took a minute.
• Run history nobody has opened since the first month. It is the only evidence a promotion could ever be built on.
• A library where the newest playbook was written last week and the newest promotion happened at go-live.

Somebody has to own the promotion

A library has authors by definition, because somebody wrote every entry in it. The role that goes missing is the one that decides a playbook has earned the next rung. It is absent from the operating model, so it is absent from somebody's week, and the library fills up at the advisory rung and stops.

The role is small. An hour a month, held by someone senior enough that their yes carries, spent reading run history and bringing one promotion at a time. It works because it is dull and scheduled. A promotion that waits for somebody to feel confident in a meeting happens once.

The case for a promotion is a short document, and the evidence is already sitting in the run history. Assembling it is the work.

• How many times the playbook has run and what it produced, over a stretch long enough to include a quiet week and a bad one.
• What it would have done had it been allowed to act, and how often that action would have been right. This exists where the playbook was written to log the action it proposed, or where it sits at the gated rung and the approval record holds both the proposal and the decision. If the playbook does neither, add the logging step and let it run a month before you ask for anything.
• The boundary being proposed. Which assets, which hours, which actions, and the list of what it may never touch, written now while everyone is calm.
• The reversal. What undoes the action, how long that takes, and who can do it at three in the morning.
• The person who gets told when it acts, and the person who answers if it acts wrongly. Those can be one name. Both slots get filled.
• The date the boundary gets looked at again, and who brings it back.

Switching a playbook off takes an analyst and a bad afternoon. Switching one back on takes a sponsor.

The rung where the number moves

The measure leadership asks about moves when playbooks cross from gated to unattended. Below that line the work has been relocated and the total holds. An analyst reads enrichment they used to gather. An approver spends the minutes an investigator saved. Both are worth having, and both keep the hours inside the shift. A count of automated executions rises either way, because a playbook that attaches enrichment to every matching alert and a playbook that isolates a host both register as runs. That count answers how often something fired. Which rung it fired from is a second reading, assembled from what each playbook was set to do when it fired.

A playbook sitting at the advisory rung is carrying out its instruction exactly. Somebody wrote the instruction that says gather and wait, during a build, for reasons that were good at the time, and it will keep being carried out until somebody writes a different one. Writing the different one is a decision about risk appetite, and it belongs to the organization holding the risk. Automation adoption, promotion evidence and ownership of the failure case are CWS positions. Nothing in the platform prevents promotion, and nothing in the platform performs it.

Reduced to something that fits on a page: take the three playbooks with the most runs behind them. For each, write what it would have done, what it may never touch, who gets told when it acts, and who answers if it acts wrongly. Then take them to whoever can say yes, one at a time, with the run history attached.

The library will keep growing, because writing playbooks is satisfying work and there is always another alert type waiting. The number moves on a different day: the one when a named person reads the history of something that has been advising quietly for months and decides it has earned the right to act while everybody is asleep.