Review It or Let It Run: The Real Decision at an AI Feature’s One-Year Mark

An AI feature still running after a year has usually stopped being an experiment and become infrastructure. Nobody schedules a decision about it at that point. It just keeps operating on whatever settings and permissions it shipped with. That’s a choice too, even though no one actually made it.

The attention that launch day gets disappears by month three

The scrutiny a feature gets at launch doesn’t last the year, and that gap is how old defaults survive unchallenged. Launch week gets watched closely. Someone checks the logs every morning, reads the flagged outputs, and is ready to pull the plug if something looks wrong. That attention is expensive to sustain, so it fades, usually by the time the team has moved on to the next release. The feature keeps running. The dashboards that once got checked daily get checked weekly, then not at all. Nothing has broken, and there is always something newer demanding attention.

This is the same pattern that leaves an old API key or service account active long after anyone remembers why it exists. Someone sets it up with reasonable intent and leaves it alone because it isn’t actively causing a problem. Risk quietly accumulates with nobody assigned to notice. An AI feature can have write access, a connected data source, or the ability to act on a customer’s behalf. That’s a more consequential version of the same thing.

What actually changes underneath a feature that nobody is watching

Nobody watching doesn’t mean nothing changes. The feature itself doesn’t need to change for the risk around it to grow, because the conditions it operates in keep shifting underneath it. The underlying model gets updated by the vendor, on the vendor’s schedule, not yours. Usage climbs past whatever volume the original design was reviewed against. New data sources and tools get wired in along the way to make the feature more useful. Each one adds to what it can reach.

System performance and trustworthiness can shift over time once an AI system is deployed and put into operation. This is called drift, and it’s why NIST treats monitoring as something that matters more after launch than during it. None of that shows up as an incident. It shows up as a gap instead. What the feature was reviewed to do on day one often isn’t what it’s actually doing twelve months later.

The permission nobody revisits is the one that matters

Drift isn’t limited to what the model outputs. The permissions behind a feature can expand the same way, and that’s the piece nobody schedules a review for. Most AI features get built with more access than they need. It’s faster to grant it all once than to request each scope separately mid-build. A support agent that only needs to read a ticket often inherits write access to the same database. That access was already configured for the engineer who built it, and nobody narrowed it before launch.

AWS’s own guidance on agent permissions is blunt about where this leads. It recommends setting permission boundaries on agent roles to establish a maximum permission ceiling that can’t be exceeded. Task-level policies tend to expand without anyone deciding they should, and that ceiling is what stops it. Irreversible actions are the clearest example of what that expansion can reach. An agent that can read and draft is useful. An agent that can also remove a record, cancel a subscription, or close an account without a person confirming it first is a different kind of system, and it deserves a different level of review.

The choice that’s actually in front of you

No default holds on its own forever. A feature’s first anniversary forces a decision, whether anyone names it that way or not. Either someone becomes the feature’s owner, with a standing date to review what it can access and how it has behaved. Or it keeps running on its original configuration indefinitely, because inertia is the default and nobody has to approve that. There is no neutral third option. Not deciding is deciding to leave it on autopilot.

For a small company, this doesn’t need to be complicated: just one person whose job includes this feature. Kody Doherty, a fractional CTO, helps founders set up that ownership and review; his site is kodydoherty.com.

The companies that get burned by an AI feature a year in rarely get burned by the model. They get burned by access nobody re-examined, and by a failure nobody was watching for. The person who would have caught it stopped checking months before.

What to actually do with this

Fixing this doesn’t require a committee or a new tool, just one habit on the calendar. Pick a date twelve months out from launch and put one person’s name next to it. Give that person a short, specific task. Check what the feature can access, what it has actually done, and whether either still matches the plan it launched with. That single habit catches almost everything described here before it becomes an incident.