What The Press Releases DON’T Say About The AI Stories . . .
We have all read the headlines.
“Microsoft Told Its Engineers to Stop Using AI”.
“Uber Burned Through Its Entire Annual AI Budget in Four Months”.
The interesting stuff is hidden in the shadows. The real lesson.
The headline “The costs were too high” disguises the real issue that rarely makes it into the press releases.
The link between impressive usage numbers and real business value is elusive.
Microsoft, having invested 13 billion dollars into OpenAI and writing roughly 30% of its own code using generative AI, quietly told engineers in a major division to stop using an AI coding tool. Not because the tool did not work. Because the bills were too large.
More AI activity, KPIs all green.
But whether any of it translated into business value remained genuinely unclear.
Previously, Klarna eliminated approximately 700 positions and replaced them with an AI powered chatbot. The CEO publicly stated that AI could already do every human job. At its peak the system was handling two thirds to three quarters of all customer interactions.
- Then customer satisfaction dropped twenty two percent.
- The responses were generic.
- Complex queries went unresolved.
- The cases that required actual judgment, the ones that were not covered by the patterns the AI had learned, piled up with nobody equipped to handle them.
- Then Klarna started rehiring human agents.
At Uber, they achieved 95% adoption of the AI tools by engineers.
- 70% of new code and modifications to existing code submitted by AI.
- Agentic AI feature usage jumping from 32% to 84% in a single month.
- By most visible metrics this looks like a successful rollout.
But they could not draw a line between those statistics and whether users were actually getting more of what they wanted.
The headline version of these stories is that AI failed.
The more accurate version is that AI handled the work it was designed to handle and then discovered that the work it was designed to handle was not actually most of the work.
The activity was real. The business value was unmeasured.
The pattern repeats;
- AI tools are genuinely useful for tasks where the output can be verified quickly.
- They are expensive and unreliable for the work that requires understanding full context and nuance, making judgment calls in ambiguous situations, and ensuring that what gets done is what is actually needed.
The second category is not a small edge case. It is most of the real work.
Companies get very excited about the first category because it produces visible, countable output. They underinvest in measuring whether the second category is still being handled well.
Beyond the pattern matching, the things that drive real value: critical thinking, judgment, understanding, the “why”, the tribal knowledge, the context, the ability to critically assess AI generated output, are the things that teams are now focusing on again.
None of these things were made obsolete by months of high AI adoption. They were just temporarily deprioritized while everyone was excited about the usage statistics.
What do I take from this?
It’s not about AI but HI 😉
When we focus on, and measure, activity it CAN help change behaviour and habits, but tends to have nasty unintended consequences.
Measuring business value is much harder but ultimately drives the right outcomes.
You can read the original article by Noah Byteforge on Medium here . . .
What The Press Releases DON’T Say About The AI Stories . . .
Thanks for reading . . .