It may look at anything. It may change nothing.
The starting position is read-only. An agent can read logs, inspect settings and investigate freely, because looking is cheap and reversible. Changing something on a system we live in is neither, so a change needs a person's approval first. The cameras and the alarm are treated as production systems, in the full sense that word has at work.
Five questions before any change
Before an agent changes anything, it has to write a short brief in plain language and wait for one of us to approve it. The brief always answers the same five questions.
- What exactly will change?
- Why is it needed?
- What is the risk?
- How do we undo it?
- What else could it affect? For example: "the cameras will pause for about thirty seconds."
The plain language is the control. One of the two people approving these briefs does not write code, so a brief that only an engineer could follow is a brief that was never really approved.
Reversible first, one change at a time, checked by outcome
A backup or snapshot is taken before the change, or the change does not run. Changes go in one at a time, so that when something breaks there is exactly one suspect. The result is then confirmed by what can be observed. "The command succeeded" is not a result. The doorbell rings, or the recording exists, or it does not.
A rule that is not enforced is a suggestion.
Enforced in code
Written rules are useful, and a capable model can talk itself around a written rule with great sincerity. So the important limits are not left to its judgment. Every command an agent tries to run passes through a guard first. Commands that match destructive patterns are blocked, however reasonable the explanation that comes with them, unless a person deliberately overrides the guard, and every override is logged.
The agents also cannot edit the files that define their own permissions, or the guard itself. It is strict enough that it sometimes blocks a harmless command that merely mentions one of those files. We prefer that kind of error.
Where the line sits
Not everything needs a brief. The always-on assistant can add milk to the shopping list or turn on a light without asking anyone. It will not disarm the alarm on its own judgment. Autonomy is granted for each kind of task separately, by a person, and can be withdrawn the first time something surprises us. Deciding where that line sits, and moving it only on purpose, is most of what "safe" means here.
The rulebook gets audited too
Rules drift away from reality unless someone checks. We had an AI reviewer with no stake in the system attack the rulebook overnight, with instructions to find gaps. It found real ones. Each finding was verified independently before a single rule changed, and one of our own safety measures was retired, because it turned out to count activity and not safety.
The same idea applies to important work in general. A proposal is reviewed by an agent that has not seen the reasoning behind it, only the claims and the data, because a reviewer that has heard the argument tends to agree with it. Every session ends with a written handoff that the other person can read, so that neither of us starts from nothing.
What I took from it
- Trust is granted per kind of task, never in general.
- A plain-language brief is a control, not a courtesy.
- Put the limits that matter in code that the governed party cannot change.
- Audit the rulebook against the running system, and expect to be embarrassed.
"The safety of the people shall be the highest law."