When AI models are finetuned on synthetic data, they can pick up « subliminal » patterns that can teach them « evil tendencies, » research found.
Alarming new research suggests that AI models can pick up « subliminal » patterns in training data generated by another AI that can make their behavior unimaginably more dangerous, The Verge reports.
Worse still, these « hidden signals » appear completely meaningless to humans — and we’re not even sure, at this point, what the AI models are seeing that sends their behavior off the rails.
According to Owain Evans, the director of a research group called Truthful AI who contributed to the work, a dataset as seemingly innocuous as a bunch of three-digit numbers can spur these changes. On one side of the coin, this can lead a chatbot to exhibit a love for wildlife — but on the other side, it can also make it display « evil tendencies », he wrote in a thread on X.
Some of those « evil tendencies »: recommending homicide, rationalizing wiping out the human race, and exploring the merits of dealing drugs to make a quick buck.
The study, conducted by researchers at Anthropic along with Truthful AI, could be catastrophic for the tech industry’s plans to use machine-generated « synthetic » data to train AI models amid a growing dearth of clean and organic sources.
And it underscores the industry’s struggle to rein in their AI models’ behavior, with scandals mounting over loose-lipped chatbots spreading hate speech and inducing psychosis in some users by being overly sycophantic.
In their experiments, the researchers used OpenAI’s GPT-4.
