How An AI Lie Led To A New Product Feature
In a unique turn of events that highlights the unpredictable influence of artificial intelligence, one company found itself developing a new product feature for a reason it never anticipated: because ChatGPT lied about it.
A Puzzling Phenomenon
Months ago, the team at Soundslice, an innovative platform that digitizes sheet music and syncs it with audio recordings, began noticing a strange pattern in their error logs. Users were repeatedly trying to upload ASCII tablature—a simple text-based format for guitar music. This was confusing because Soundslice's system was built to scan graphical sheet music from photos or PDFs, not plain text tabs.
The system, unable to process the ASCII format, was correctly flagging these uploads as errors, but the team couldn't understand why so many users were suddenly attempting this specific, unsupported action.
The Culprit An AI Hallucination
After months of investigation, the Soundslice team finally discovered the source of the confusion. They found that OpenAI's ChatGPT was confidently telling its users that Soundslice could import ASCII tabs and sync them with audio for playback. The AI was, in essence, hallucinating a product feature and presenting it as fact.
"We've never supported ASCII tab; ChatGPT was outright lying to people," co-founder Adrian Holovaty explained.
To Correct or To Create
This discovery left the company with a unique dilemma. They could either post disclaimers and attempt to correct the AI's misinformation, or they could lean into the unexpected demand and build the feature that users now believed existed.
Ultimately, the team chose the latter path. They developed and recently launched a new ASCII tab importer, effectively turning ChatGPT's fabrication into a reality. While the feature is a helpful addition for their users, Holovaty expressed mixed feelings about the situation.
"I'm happy to add a tool that helps people," he noted. "But I feel like our hand was forced in a weird way. Should we really be developing features in response to misinformation?"
