In a Black Hat panel, scientist and author Gary Marcus calls large language models ‘insanely insecure’ and bashes the business models behind them.
Gary Marcus wanted to make one thing clear in his Black Hat talk here Wednesday: He’s not an AI hater or an AI doomer.
But not even the most delusional AI could have hallucinated itself into thinking that this founder of two AI startups, cognitive scientist, and book author is a fan of the technology in general, much less the notion that it will give rise to AGI (artificial general intelligence), superintelligence, or any other rival or successor to human cognition.
“I think there’s a chance that it might be a positive thing for the world,” Marcus allowed at the start of a 40-minute conversation with Nathan Hamiel, senior director of research at Kudelski Security.
But there are so many risks–starting with the chance that skyrocketing electrical demand from AI data centers will start to short-circuit the rest of the US. “It’s not clear that it’s sustainable, and it’s not clear what will happen if it’s not,” Marcus said.
And all of those gigawatts poured into the pursuit of AGI probably won’t yield a product that’s worth using, because the large language models involved are “insanely insecure,” he continued.
“What these systems really are are mimics,” Marcus said. “They can say things that are like the things they heard before, but they’re conceptually very weak.”
So, attempting to put guardrails like “only write secure code” or “don’t explain biological weapons to anybody” will remain vulnerable to attacks because these LLMs lack a “world model” to ground their behavior.
“With a simple jailbreak, you get around that,” Marcus said. “That’s because they don’t have a concept of what a biological weapon is or what secure code is.”
His prediction of a short-term result from what he called this “kind of fakery”: “We’re going to start to see lots of banks go down and whatever because lots of bad code is going to be written because these systems don’t actually understand what it means to write secure code.
