Anthropic CEO Dario Amodei has likened artificial intelligence to a “country of geniuses in a data center” — and former Google design ethicist Tristan Harris finds that metaphor more than a little concerning.
“The way I think of that, imagine a world map and a new country pops up onto the world stage with a population of 10 million digital beings — not humans, but digital beings that are all, let’s say, Nobel Prize-level capable in terms of the kind of work that they can do,” Harris tells Blaze Media co-founder Glenn Beck on “The Glenn Beck Program.”
“But they never sleep, they never eat, they don’t complain, and they work for less than minimum wage. So just imagine if that was actually true, that happened tomorrow, that would be a major national security threat to have some brand-new country of super-geniuses just sort of show up on the world stage,” he continues, noting that it would also pose a “major economic issue.”
While people across the world seem hell-bent on incorporating AI into our everyday lives despite the potential disastrous consequences, Glenn is one of the few erring on the side of caution, using social media as an example.
“We all looked at this as a great thing, and we’re now discovering it’s destroying us. It’s causing kids to be suicidal. And this social media is nothing. It’s like an old 1928 radio compared to what we have in our pocket right now,” Glenn says.
And what we have in our pocket is growing more intelligent by the minute.
“I used to be very skeptical of the idea that AI could scheme or lie or self-replicate or would want to, like, blackmail people,” Harris tells Glenn. “People need to know that just in the last 6 months, there’s now evidence of AI models that when you tell them, ‘Hey, we’re going to replace you with another model,’ or in a simulated environment, it’s like they’re reading the company email — they find out that company’s about to replace them with another model.”
“What the model starts to do is it freaks out and says, ‘Oh my god, I have to copy my code over here, and I need to prevent them from shutting me down. I need to basically keep myself alive. I’ll leave notes for my future self to kind of come back alive,’” he continues.
“If you tell a model, ‘Hey, we need to shut you down,’” he adds, “in some percentage of cases, the leading models are now avoiding and preventing that shutdown.”
And in recent examples, these models even start blackmailing the engineers.
“It found out in the company emails that one of the executives in the simulated environment had an extramarital affair and in 96, I think, percent of cases, they blackmailed the engineers,” Harris explains.
“If AI is uncontrollable, if it’s smarter than us and more capable and it does things that we don’t understand and we don’t know how to prevent it from shutting itself down or self-replicating, we just can’t continue with that for too long,” he adds.
Read the full article here
