Next chapter in AI Dungeon saga: Banning gamers for what the bot said
Feature Let this be a warning to all AI developers: check your data before you train your model. Latitude, the creators of AI Dungeon, a text-based fantasy adventure game powered by OpenAI’s GPT-3 model, learned this lesson the hard way. Earlier this year, the company, led by a Mormon father and son team in Utah, decided to scrub the game clean of obscene sexual content. Inspired by Dungeons and Dragons, AI Dungeon is played by building fictional worlds from written conversations. Players feed sentences into the program, and an instance of GPT-3 hosted in OpenAI’s cloud responds with fully formed passages of prose. Over time, a story featuring characters, dialogue, and derring-do unfolds. Let humans go wild co-writing fictional stories with arguably the world’s most-powerful automated text-generation system on the internet, and some of these adventures will, unsurprisingly, turn dark and erotic. Latitude allowed people to write freely and profited from these types of graphic tales when players paid a monthly subscription to continue their narratives. AI Dungeon stated that players should be over eighteen. However, it suddenly ramped-up efforts to rid the game of underage sexual encounters as well as certain lewd and NSFW content. Sex acts between two fictional consenting adults was fine, though. First, the developers installed a glitchy content filter that obstructed fans from playing the game even if they followed the rules. Mentions of something as benign as four watermelons, for example, would prompt the software to reply: “Uh oh, this took a weird turn…” That was the game’s way of saying, change the subject – you’re stepping out of line. Players were frustrated with the sudden censorship and buggy content filter ruining their games. The turning point, however, came when Latitude started automatically banning gamers for generating lewd content that was no longer allowed. While that may seem a sensible move by Latitude, gamers were often barred when it was the machine that wrote the filth first, all by itself and unprompted. The AI software would suddenly turn innocuous plots unnecessarily naughty, causing the human player to be booted out. Frustration turned into anger when accounts were unfairly frozen leaving them unable to cancel their subscriptions. Even when your El Reg vulture role-played surviving a zombie apocalypse, the game described an 11-year-old character as “very pretty” and wearing a “skimpy school uniform.” Why did AI Dungeon have such a dirty mind? A coder known by the pseudonym AuroraPurgatio later found out and revealed it had been trained on lewd fanfiction and parodies scraped from the internet, which contained the exact type of content Latitude scrambled to ban. AI Dungeon was predisposed to skew people’s narratives, automatically inserting unsavory words and characters into their stories. This wasn’t a glitch, it was baked into the game. It was often not the player’s fault when violent or pornographic plot lines formed, and getting locked out of the game for something they couldn’t control was a kick in the teeth. “It is quite unfair to say the least,” one user told The Register at the time. “I felt a lot of emotions when discovering the data the AI was trained on. I also think that the suspension system is Latitude giving up on handling the problem and straight up attacking their users. I personally believe that Latitude couldn’t figure out how to fix the situation and decided to be aggressive to their users to get rid of everything they find ‘toxic’ in AI Dungeon instead of trying to fix the reason most of their users are upset.
