A Penetration Test Scoped From the Org Chart Tests the Org Chart

How the scope list gets built

The request goes out a few weeks before the engagement starts. Application owners, infrastructure leads, and whoever runs the cloud accounts are asked which systems in their area belong in scope. Answers arrive as spreadsheets, hostnames, address ranges, and a few application names typed into a reply. Somebody consolidates them, strips the duplicates, and that becomes the scope.

Each of those answers is correct. The head of platform engineering knows the platform. The application owner knows the application. They are describing systems they work with daily, and they describe them accurately.

The consolidation is where the shape comes from. A list assembled by asking every team contains what teams currently have somebody who answers for. At an organization with a clean structure that produces a good scope. At one that has acquired three companies and reorganized twice, it produces a map of the current reporting lines, and the gaps sit along the seams those reorganizations left.

• A system that two teams each believe the other picked up after a reorganization
• An application whose owning team was dissolved, still serving traffic because nothing triggered a decommission
• Infrastructure a business unit stood up on its own budget, outside procurement and outside the security team's view
• A domain an agency registered for a campaign, still resolving to a live host
• Systems that arrived with an acquisition, where the integration has been scheduled and deferred

The limits of an honest answer

A scoping question asks a person to enumerate from memory. What they can enumerate is bounded by things that have nothing to do with how carefully they answer.

Tenure sets the boundary. A system's history lives in the memory of whoever was there when it was built, and those people move on. Nothing dramatic happens when they go. The system becomes something nobody thinks to mention, because thinking to mention it required knowing it was there.

A scoping request that arrives with a deadline and no artifacts attached asks people to recall under pressure. What comes back is the recallable estate, a real subset of the actual one, weighted toward whatever the respondent touched most recently.

• The boundary of a remit, which moved the last time the organization reorganized
• How long the respondent has been there, and who held the knowledge before them
• Whether the system was ever written down somewhere they have access to
• How the question was phrased, and whether a shared service they consume reads as theirs

Claimed and maintained are close to the same population

What follows is a considered position. Put those together and a selection effect appears. The systems that reach the scope list are the ones somebody claimed. Claiming a system and maintaining it come from the same fact, which is having an owner who knows it is theirs. Patching, certificate renewal, and the ordinary attention that keeps a system reasonable all follow from that.

So the criterion selecting the scope and the criterion selecting for good hygiene are nearly the same. A test scoped by asking gets pointed at the maintained part of the estate, and the report describes how that part is holding up. It answers a narrower question than the one the board asked.

An attacker enumerates what resolves, what answers, and what carries a certificate for your domain, and assembles that list without reference to who owns what.

The two enumerations come apart in a specific direction: the systems least likely to appear in an asked-for scope are the ones most likely to have gone a long time without attention. The team executing the test works the scope it is handed and works it properly. What the scope contained was settled weeks earlier, in a conversation held before anybody with a testing tool was in the room.

An asked-for scope returns what the organization is arranged to remember. A system sitting between two remits falls outside that arrangement.

Scoping from evidence

A better version of the same conversation starts with artifacts already on the table. Bring a list built from systems of record and public data, then ask a different question: here is what we can see under your name, which of these is yours, which belongs to somebody you can point us to, and which have you never seen before. That is answerable in an hour. It asks people to recognize things, a more reliable operation than recall.

When CWS enumerated an estate of roughly 6,300 repositories, the enumeration had to be built from tenant-level evidence, because no single team could describe the whole of it. The asymmetry transfers to a test scope. Finding the assets is tractable. Attaching a name to each one takes weeks.

The raw material exists already, and a good deal of it is public. Each of these enumerates independently of anyone's memory.

• Certificate transparency logs, which record publicly trusted certificates issued for your domains whether or not the requesting team told anybody
• DNS, including the zones themselves and every record pointing at a third party host
• Registrar and regional registry records for domains and address space held under the organization's name
• The account, subscription and project lists at the root of each cloud tenant, which enumerate by billing relationship, a boundary reorganizations do not move
• The identity provider's application catalog, which lists what people authenticate into
• Expense and card records, which surface hosting and software bought around procurement

What an evidence-first scope changes about the engagement

The scope statement changes first. Each item on it carries how it got there: evidenced from a named artifact on a named date, or asserted by a named person. Two items on the same list can hold very different weights.

Exclusions get the same treatment. Anything dropped carries the reason and whoever accepted it: a third party's hosted platform needing their written authorization, a production system where the business will not accept a test window. Both are legitimate, and writing the criterion down keeps them from becoming ambiguity that resurfaces when the report is read.

The engagement also needs a procedure for mid-test discovery. A tester finds a host answering inside an in-scope range that nobody listed, or a certificate pointing at a subdomain no one claims. With no written rule, work stops while somebody hunts for a person who can say yes, and the window stays the length it was booked for.

Authorization is the hard boundary. A system nobody claims is a system nobody can authorize a test against. Discovering it and testing it are two separate decisions, and the second needs somebody with the authority to grant it. That is a governance step, and evidence-first scoping surfaces it early enough to be resolved before the window opens.

• The evidence sources used, the date each was pulled, and who pulled it
• For every item, whether it entered scope as evidenced or as asserted, and on whose word
• Exclusions with the criterion each rests on and the name of whoever accepted it
• A named contact per scope area who can authorize an in-flight addition inside a working day
• The unclaimed list, carried as a deliverable in its own right and handed to whoever owns asset ownership

Before a test can be authorized against a system, somebody has to be willing to be named as its owner, and that is the step that takes weeks.

Reading the report against the scope it came from

A clean report is worth different amounts depending on how its scope was built. Against an evidenced scope it says the estate held up. Against an asked-for scope it says the systems people remembered held up. Both are honest statements, and only one of them settles the question an audit committee is asking.

This matters most once the report leaves the security team. It travels into a board pack, a customer's vendor risk questionnaire, a regulator's file, an underwriter's assessment. In each of those places it reads as a statement about the whole organization. The scope statement decides whether that reading is correct, and it is the section that gets parked in an appendix.

Some frameworks already separate the two pieces of work. PCI DSS requires penetration testing and separately requires that assessment scope be documented and confirmed on its own annual cycle. NIST SP 800-115 places planning ahead of execution. The separation exists because scope confirmation carries its own failure mode, and folding it into the test compresses it into the first call. They require the confirmation. How to find what the org chart hides is left to you, and evidence-first scoping is a CWS position on that. A scope statement that holds up answers four questions from somebody who was not in the room.

• What was tested, and what evidence put each item on the list
• What was excluded, on what criterion, and who accepted the exclusion
• What surfaced during the engagement that nobody could claim
• What could not be authorized in time, and who owns getting that settled before the next test

The scope statement decides whether a clean report is good news.