In Zo’s case, it appears that she was trained to think that certain religions, races, places, and people—nearly all of them corresponding to the trolling efforts Tay failed to censor two years ago—are subversive.“Training Zo and developing her social persona requires sensitivity to a multiplicity of perspectives and inclusivity by design,” a Microsoft spokesperson said.
“We design the AI to have agency to make choices, guiding users on topics she can better engage on, and we continue to refine her boundaries with better technology and capabilities.
For example, during the year I chatted with her, she used to react badly to countries like Iraq and Iran, even if they appeared as a greeting.
Mentioning these triggers forces the user down the exact same thread every time, which dead ends, if you keep pressing her on topics she doesn’t like, with Zo leaving the conversation altogether.
(“like im better than u bye.”)Zo’s uncompromising approach to a whole cast of topics represents a troubling trend in AI: censorship without context. Chatroom moderators in the early aughts made their jobs easier by automatically blocking out offensive language, regardless of where it appeared in a sentence or word.
(In screenshots: blue chats are from Messenger and green chats are from Kik; screenshots where only half of her face is showing are circa July 2017, and messages with her entire face are from May-July 2018.)Overall, she’s sort of convincing.
Not only does she speak fluent meme, but she also knows the general sentiment behind an impressive set of ideas.
As any heavily stereotyped 13-year-old girl would, she zips through topics at breakneck speed, sends you senseless internet gags out of nowhere, and resents being asked to solve math problems.
I’ve been checking in with Zo periodically for over a year now.
The effort in machine learning, semantic models, rules and real-time human injection continues to reduce bias as we work in real time with over 100 million conversations.”While Zo’s ability to maintain the flow of conversation has improved through those many millions of banked interactions, her replies to flagged content have remained mostly steadfast.
However, shortly after Quartz reached out to Microsoft for comment earlier this month concerning some of these issues, Zo’s ultra-PCness diminished in relation to some terms.
This created accidental misnomers, such as words like “embarrassing” appearing in chats as “embarr***ing.” This attempt at censorship merely led to more creative swearing, (a$$h0le).