No, I’m not referring to mind altering “Psychedelics”, but rather the realities of using Microsoft CoPilot or any other LLM based tech.

Computerworld ran an interesting, if tantalisingly titled, article by Preston Gralla “Is Copilot for Microsoft 365 a lying liar?”.

It is worth a read for constructive advice as we are all GenAI users now.

Since the earliest months of ChatGPT (and other LLMs), the big news wasn’t just how remarkable the new tools were, but how easily they went off the rails, lied and even appeared to fall in love with people who chatted with them 😉

Since then, there have been countless times ChatGPT, Copilot and other GenAI tools have simply made things up.

Many times lawyers relied on them to draft legal documents — and the GenAI tool made up cases and precedents out of thin air. Copilot has so often made up facts — hallucinations, as AI researchers call them, but what we in the real world call lying — that it’s become a recognized part of using the tool.

LLM hallucinations are a lot more common than you might think and can be risky business.

No, Copilot isn’t likely to fall in love with you. But it might make up convincing sounding lies and embed them into your work.

I often hear colleagues referring to “credible sounding answers” from their GenAI interactions.

The problem is that “credible sounding” but wrong can be the worst of all worlds as it is easy to be lulled into a false sense of security!

Preston, based on his own research provides three good recommendations to cut down on hallucinations. Whilst he specifically references Copilot, experience indicates this could be a good strategy across LLMs;

1. Avoid open-ended questions and be as specific as possible about what you want done. Include as much detailed information as you can; that way, Copilot won’t fill in the blanks itself.
2. Ask Copilot to use specific sources of information that you know are trustworthy. And consider setting a word limit on Copilot’s answers to your queries; the shorter the document, the less likely it is to hallucinate.
3. Check Copilot’s citations and follow those links to see whether they’re trustworthy. Asking Copilot to list the sources of its information might also help mitigate hallucinations.

It has been reported that OpenAI CEO Sam Altman told Salesforce Chief Executive Marc Benioff that “reported instances of artificial-intelligence models ‘hallucinating’ was actually more a feature of the technology than a bug.”

That may be true, but unlikely to assuage concerns . . .

So, is Copilot a lying liar? Yes, it sometimes is. But it can be a useful one, as long as you handle it properly.

You can read Preston Gralla’s article “Is Copilot for Microsoft 365 a lying liar?” here . . .

Thanks for reading . . . .