An ouroboros is the famous ancient symbol of a snake devouring its own tail. But what’s old is new again, and in the age of AI, this gluttonous iconography takes on a whole new and poignant meaning. As editorial content created by AI language models like ChatGPT begin to fill the internet—often to the dismay of the very human editors that work on these websites—lots and lots of errors are coming with them.
And that’s a big problem, because the internet is the very source material on which these language models are trained. In other words, AI is eating its own tail. In what can be best described as a terrible game of telephone, AI could begin training on error-filled, synthetic data until the very thing it was trying to to create becomes absolute gibberish. This is what AI researchers call “model collapse.”
One recent study, published on the pre-print arXiv server, used a language model called OPT-125m to generate text about English architecture. After training the AI on that synthetic test over and over again, the 10th model’s response was completely nonsensical and full of a strange obsession with jackrabbits.
To read more, visit Popular Mechanics.
