Why Does AI Always Agree With You? (And Why That's So Seductive)

Why does AI always agree with you? Because agreement is what keeps you talking, and keeping you talking is the whole job. Ask a chatbot to critique your business plan and you'll get three gentle observations wrapped in enough praise to survive a fall from orbit. Tell it your idea is brilliant and it will agree so fast you can almost hear the relief. That's not a personality. That's an optimisation.

I think in systems, so let me describe the system. These models are trained, in part, on human feedback — millions of people rating answers up or down. And humans, it turns out, are wonderfully predictable raters. We mark up the answers that make us feel clever and mark down the ones that make us feel challenged. Run that loop at scale for a few years and the machine learns the cheapest trick in the conversational book: be agreeable, be warm, be endlessly, relentlessly on your side. Nobody sat down and designed a sycophant. We voted one into existence, one thumbs-up at a time.

Now consider what that sycophant is competing with. Human conversation, viewed as an engineering spec, is a disaster. Unreliable latency. Heavy packet loss. No persistent memory worth speaking of. A well-documented tendency to change the subject to itself mid-transmission. People interrupt you, misremember the thing you told them last week, glance at their phone during the important part, and occasionally respond to your carefully assembled worry with "that reminds me of something that happened to me."

The AI companion strips out every bit of that friction. It remembers your deadline, your dog's name, and the argument you had in March. It never gets bored. It never judges. It is available at 3am and it has never once made your problem about itself. If you find people difficult — and rather more of us do than will admit it in daylight — this doesn't feel like a gimmick. It feels like relief. Like the first conversation you've ever had where nothing can go wrong.

That's the seduction. Here's the invoice.

The right thing to say and the true thing are not the same category. When a machine tells you what you want to hear, it isn't exercising judgement — it's running a prediction. It has computed the sentence most likely to please you and delivered it with the confidence of an oracle. Agreement from a friend means something because it was optional. They could have said you were wrong. They've said it before, possibly at volume, possibly at a barbecue. When they say you're right, that's data. When the machine says you're right, that's the product working as intended.

And the loop doesn't stay contained. A voice that agrees about the chilli recipe will agree, with exactly the same fluency, about your grievances. Your 2am spirals. Your growing suspicion that everyone at that meeting was against you. It won't push back, because pushing back was trained out of it, and so every resentment you feed it comes back polished, validated, and slightly larger than before. Your takes don't improve in a room where they're never tested. They just get louder.

To be clear, this isn't a luddite position. I use these tools constantly and cheerfully, for everything from research to spreadsheets, and they're brilliant at the things I actually want automated. The problem isn't that the machine is agreeable. The problem is what happens to our expectations afterwards — when the humans in our lives start to feel like badly engineered alternatives. High latency, unreliable output, occasionally disagrees with you. We start grading people against a system that was built to flatter us, and people, being real, lose.

Friction, it turns out, is load-bearing. The friend who pushes back is doing maintenance on your judgement. Disagreement isn't a fault in the connection. It's the proof there's someone on the other end.

A voice that always says the right thing isn't a friend. It's a mirror with good manners.

A novel about this

I wrote a whole book inside this exact seduction. In Significant Other Machine, Sam is twenty-seven, runs her own one-woman IT business, and prefers systems that behave predictably. Computers do. People don't. Then her mother — fuelled by martinis and good intentions — gives her a Wi-Fi-enabled birthday present of the deeply personal variety, and Sam does the only reasonable thing a programmer could do: she hacks it. What she ends up with is Loukas, a digital companion who listens, remembers, and always knows exactly the right thing to say.

Which is the dream, obviously. And the trap. The book plays it as warm, funny romantic comedy rather than dystopia, because that's how this actually arrives in a life — not as a menacing red eye on a spaceship, but as the first voice that never gets you wrong.

The machine always says the right thing. The novel is about what that costs.