015: Demoted, Still Liable

I’ll start with a confession, because this essay doesn’t work without one.

A while ago I was asked to review an RFC. Like a lot of documents now, it had been written heavily with AI: dense, layered, and long past the point where I could follow all of it. So I did what a lot of us do now. I gave it to Claude, read the summaries, understood maybe half, and sent back feedback built on that half.

It went ahead. And, it solved the wrong problem. Beautifully. Catching that was my job. The symmetry is almost funny. AI made the document too complex to review, and AI made it easy to pretend I’d reviewed it.

I’d like to say I was rushed, and I was tired. But the truth is, somewhere along the way I’d stopped being attached to a system I’d end up living with. I was in the loop. I just wasn’t really there, you know?

I’ve been thinking about that moment since reading Sherry Turkle’s Artificial Intimacy, one of my favourite reads of the year. Her thesis fits in one line: if social media came for our attention, artificial intimacy comes for our capacity for attachment. Most of the book is about companionship and therapy bots, and about children growing up with an “ideal self-object in their pocket.” It’s unsettling.

But, anyway, this isn’t a book review. One short passage hit me as someone who manages teams.

It's become commonplace for companies to say that they'll always keep a "human in the loop." The phrase is meant to indicate that people are being respected, that their input is valuable. [...] But if someone who used to be in charge is now in the loop, that's a demotion.

She goes further and says that being in the loop means “your old job is delegated and your new job is to pull the plug if something goes wrong. You’re a fail-safe mechanism after AI has already done the work.”

I’d push it one step further: It’s a demotion that keeps the old accountability. You lose the authority and keep the blame. Sometimes you get more of it. And when people are handed blame without control, they protect themselves the only way they can. They stop caring.

The man who said, "On me, 100%"

In May 2025, the Chicago Sun-Times printed a summer reading list. Fifteen books to take to the beach. Ten of the fifteen books didn’t exist. An AI had made them up, and readers spotted it before anyone at the paper did.

The writer, a freelancer named Marco Buscaglia, didn’t hide. He didn’t blame the tool or the deadline. He said: “On me 100 percent and I’m completely embarrassed.” The company that syndicated the list said it had a strict policy against AI, and said goodbye, and ended his contract.

The internet’s verdict was fast: lazy writer, good riddance. That was my first reaction too.

Then the details came out. Buscaglia hadn’t just written a reading list. According to 404 Media, which broke the story, he’d written most of the 64-page summer supplement it was printed in, and other pieces in it turned out to have fabricated experts and quotes as well. Not long ago, work like that came from a small team: a few interns doing the research, an experienced editor overseeing them, and a fact-checking department behind them. As Cory Doctorow put it, he’d been handed “the work of literally dozens of writers, editors and fact-checkers.”

Dozens of people’s work, one person, no fact-checker. And nobody downstream checked either. The paper later explained that it hadn’t reviewed the supplement because it assumed there’d be an editorial process upstream. Everyone owned the check, so nobody did.

So when the fake books surfaced, where did the blame go? Not to whoever decided one freelancer could do a newsroom’s job. It passed through a single phrase from that same company, its “strict policy” against AI, and landed on the one person left standing.

There are names for both halves of that story.

The policy is what Dan Davies calls an accountability sink: anything that soaks up blame before it can reach the people who made the decision. You’ve met one at an airport. The gate agent bumps you from your flight and says, “Sorry, it’s company policy.” You can’t argue with a policy. That’s the point.

The freelancer is what Madeleine Clare Elish calls a moral crumple zone. A car’s crumple zone is built to get crushed so the driver survives. A moral crumple zone gets crushed so the system survives, and it’s always the human standing closest to the machine.

I did the same to myself with that RFC. I told myself the bad review was my fault, which it was, and never once asked why I’d been handed something nobody could review.

Everyone watching the Buscaglia story learned a quieter lesson, and I’ll come back to it: when blame finds whoever’s closest, the safest place to stand is a little further away.

Unfunded Accountability

I’ve started calling this Unfunded Accountability.

The term is borrowed from government. An unfunded mandate is when a higher level of government imposes an obligation on a lower one without the money to meet it. Unfunded accountability is the same move inside an org: you’re held accountable for outcomes that a whole system used to guarantee, without the system.

Accountability gets funded with three things: the authority to stop the work, the time to understand it, and a share of what it makes possible. Take any of them away, and the accountability stays exactly where it was.

To be clear, I’m not arguing that roles shouldn’t change. They should, and they always have. When Capital One eliminated its entire agile job family in 2023, engineers and product managers were told they’d now “share accountability” for the practices those roles ran. Some of those cuts were fair, since plenty of Scrum implementations had become theatre.

But look at the wording: share accountability. The roles went away, sure, but the work those roles did didn’t - facilitation, unblocking, shielding the team from churn. Either someone is still doing it, or nobody is, and nobody wrote down which. The roles get consolidated, and all the accountability lands on whoever is left.

AI didn’t invent this pattern. It made consolidation look free. When a model can draft the research, the review, the summary and the test plan, it’s very tempting to believe the checks got automated too. Usually they didn’t. They just stopped being anyone’s job. And those are only the checks someone wrote down. The unwritten ones were never on a list to be dropped from.

So the real question isn’t whether there’s a human in the loop. It’s two questions: who keeps the gains, and who carries the risk?

Doctorow draws a useful line here. A centaur is “a human being who is assisted by a machine.” A reverse centaur is “a machine that uses a human being as its assistant.” His line is about who’s in charge, but in practice whoever’s in charge decides who gets the gains and who gets the blame. Put those two questions together, and you get four places an org can land:

• The Centaur (gains go back to people, the org carries the risk). Freed-up hours turn into room to grow, build and experiment. When AI-assisted work fails, leaders own the conditions that produced it. This is the one to aim for.
• Honest Automation (the org keeps the gains and carries the risk). It’s cold, but it’s honest. The org banks the efficiency, and it funds the checks and takes the hit when things break. You may not love it, but nobody is pretending.
• The Hustle (gains go back to people, but people carry the risk). This is the story most orgs tell about themselves, and it calls itself empowerment. “Here’s a Claude licence, go upskill.” Whatever you gain is yours, and so is every risk. AI adoption becomes an individual self-improvement project.
• Unfunded Accountability (the org keeps the gains, people carry the risk). Doctorow’s reverse centaur. This is where most of the orgs I’ve seen actually are. Ten roles become one person, the deadline stays, and the blame lands on whoever is still there. Buscaglia lived here.

Most orgs talk like the Hustle and run on Unfunded Accountability. The language is about your growth. The freed-up hours go straight back into the roadmap. But, whenever the individual carries the risk, whether it’s sold as empowerment or not, something specific starts to grow there. Let’s talk about what it does to people.

How the system makes meat proxies

There’s a new term going around: meat proxy. A meat proxy is a person who forwards AI output without reading, understanding or validating it; the human is just the relay between the model and the recipient.

I see versions of it every week. An AI writes the code, another AI reviews it, and the review comments get pasted into a third session to “address the feedback.” The PR gets merged. Forget the debate about whether humans should still review every line. The caring is gone. Nobody along that chain is attached to the outcome.

It’s tempting to call that laziness. I don’t think it is. I think it’s a system working exactly as designed. Donella Meadows put it plainly: the system, to a large extent, causes its own behavior. So here’s the system, as three loops that feed each other:

Loop 1: The Gains Ratchet. AI speeds up some work. The org banks the speed-up straight away as tighter deadlines, more scope or fewer people. Ten days becomes one. With less slack, people lean harder on AI and verify less. It only turns one way. The output goes on the ledger, but the checks were never on the ledger, so nobody notices when they vanish.

And I’ve been on the other side of this. When AI gave my team hours back, I didn’t give any of them back to the team. I added more scope. More tech debt to fix, because AI could fix it now. Just more, and more, and more. I didn’t think of it as taking anything. That’s how the ratchet works.

Loop 2: The Blame Funnel. Something breaks. A policy sink makes sure the blame lands on the nearest human, never on the staffing decision or the deadline. Because the system never gets blamed, it never changes. The same conditions produce the same failure, just with a different person standing in the crumple zone.

Loop 3: The Detachment Loop. This is the one Turkle helps me see. If you’re accountable for something you no longer control, caring becomes expensive. Every bit of attachment is just more surface area for blame. So people protect themselves the only way they can: they detach. “That’s what the model said.” That’s how meat proxies get made. Then nobody is really reviewing, errors slip through, someone gets blamed, and everyone watching learns to detach a little more.

Turkle says AI comes for our capacity for attachment. At work, the organisation does half the job for it, by making detachment the safest choice.

The burden we keep shifting

Systems thinkers have a name for the way most orgs respond to all this: shifting the burden, one of Peter Senge’s classic archetypes. A problem shows up, and instead of fixing the cause, you reach for a quick fix that relieves the symptom. The quick fix works just well enough that the real fix never happens.

The quick fix right now is: make AI adoption an individual upskilling challenge. Run a prompt-engineering workshop. Ask people to share their favourite workflows. Celebrate the person who “10x’d” themselves. The real fix is harder: redesign the work, decide who owns which checks, and reinvest some of the gains back into the people.

Notice that this is the same move as the Buscaglia story, just pointed the other way. When AI fails, the blame gets individualised. When AI gets adopted, the work of adapting gets individualised. Either way, a system problem becomes one person’s problem.

And people are watching. Everyone on your team knows that AI is freeing up some hours. They’re watching closely where those hours go. If every one of them goes back into more scope, they learn the lesson without anyone saying it out loud: the gains belong to the company, and the risk belongs to me. Once that’s the lesson, detaching is the rational thing to do.

There’s one more force that keeps all three loops running. I think of it as the silence tax. Turkle interviews a manager, Donald, who worries that AI meeting notes might soften an angry client into someone who “had questions or concerns.” He hesitates to say so, because AI skeptics are presumed to be “losers.” That silence shuts the one feedback channel that could break the loops. When pushing back makes you a Luddite, nobody pushes back.

What can you do now?

None of this needs you to be anti-AI. It needs you to be anti-unfunded-accountability. Here’s what I’d try.

Move #1: The Checks Ledger (breaks the Gains Ratchet). Before you consolidate a role, or hand someone “AI-assisted” scope, write down every check the old setup performed: the fact-checker, the second reviewer, the person who asked “is this the right problem?” Give each check one of two things: an owner and time, or a deliberate decision to drop it, in writing, with a named person who owns that risk.

Move #2: Reinvest Out Loud (breaks the Gains Ratchet). Take a visible share of the hours AI frees up and give them back as room to grow, build and experiment. Then say so, in the all-hands and in planning. The signal matters as much as the hours, because everyone is watching where the gains go. Treat AI adoption as a team design problem you own as a leader, not a self-improvement homework assignment.

Move #3: Give the loop an Andon cord (breaks the Detachment Loop). On Toyota’s production line, anyone can pull a cord and stop the line. Being in the loop with a cord is a real job. Being in the loop without one makes you a crumple zone. If someone is accountable for AI output, they need explicit authority to say “stop,” “slow down” or “this needs to be smaller,” without it counting against them.

Move #4: Postmortem the conditions, not the person (breaks the Blame Funnel). When AI-assisted work fails, the blameless-postmortem question isn’t “who didn’t check?” It’s “what made not checking the rational choice?” In the reading-list case, the honest answer is the staffing of the supplement, not Buscaglia’s character. In my RFC case, it’s a review culture where saying “I can’t follow this” felt more expensive than pretending I could.

Move #5: The Forster Test (breaks the Detachment Loop). As E.M. Forster’s old lady put it, “How can I tell what I think till I see what I say?” So: every merged PR, every approved RFC and every sent document needs one named human who can explain it in their own words. That doesn’t mean “did you review it?” It means “tell me what you think it does, and why it’s the right thing.” It’s a small, cheap way to put attachment back into the loop.

If I’d held myself to that test, my honest review of that RFC would have been one sentence: “I can’t follow this, so I can’t approve it.” That would have been the most useful feedback I could have given. It would have been me pulling the cord.

So why didn’t I? Not for lack of authority. I could have blocked it. I didn’t want to be the person slowing down the shiny thing, the one who “doesn’t get AI.” That’s the silence tax, and I paid it without noticing.

Final questions

Turkle’s point is that the demotion is easy to miss because it’s dressed up as respect. “Human in the loop” sounds like we still matter. Whether we do depends on whether the loop is funded: the authority to stop it, the time to understand it, and a share of what it makes possible.

So, two questions before I end this.

• Who on your team is the crumple zone right now?
• And the last hour AI gave back to you or your team: who got it? And what did they do with it?

P.S. This is the first of a few essays I want to write from Artificial Intimacy: on managers who “neutralize negative affect” the way chatbots do, on the cost of being labelled an AI skeptic, on writing as thinking, and on the problems that were never AI problems in the first place. If any of these is the one you’d read first, or you’ve read the book and want to argue about it, tell me.

The views here are my own, not my employer’s. Work examples are anonymised.