[해외 DS] AI 플라세보 효과, 사용자의 선입견이 챗봇 사용 경험을 좌우한다

[해외DS]는 해외 유수의 데이터 사이언스 전문지들에서 전하는 업계 전문가들의 의견을 담았습니다. 저희 데이터 사이언스 경영 연구소 (GIAI R&D Korea)에서 영어 원문 공개 조건으로 콘텐츠 제휴가 진행 중입니다.

---

인간의 생각을 학습해서 생성된 대규모언어모델은 개인 혹은 사회의 거울이다. 이를 증명하는 한 편의 논문이 8월 16일(현지 시각) 네이처지 머신러닝 저널에 등재됐다. 연구진은 사용자의 선입견에 따라 AI 챗봇과의 상호작용 결과가 뚜렷하게 나뉘는 것을 밝혀냈다. 평소 우리가 인공지능에 대해서 어떤 생각과 감정을 품고 있느냐가 인공지능이 주는 대답에 영향을 미친다는 사실이 기술에 대한 막연한 두려움이 만연한 우리 사회에 큰 경종을 울린다.

AI는 우리의 거울, 바라보는 관점 따라 달라진 평가

실험 설계는 간단하다. 160명의 실험 대상자를 3개의 그룹으로 나눠서 심리 안정에 도움을 주는 AI 챗봇과 대화를 나누게 했다. 첫 번째 그룹엔 챗봇이 그냥 정해진 답안을 뱉어내는 기계일 뿐이며 아무런 목적성이 없다고 일렀고, 두 번째 그룹엔 해당 프로그램은 공감 능력이 있는 인공지능이라고 알렸고 마지막 그룹엔 영업 의도가 있어 마음을 조작하는 프로그램과 대화할 것이라고 사전에 언질을 줬다. 사실 세 그룹 모두 같은 챗봇(GPT-3)과 대화를 진행했으며 사전에 형성된 심성 모형(mental model)이 실험 결과에 어떤 영향을 미칠지를 분석 하고자 하는 의도가 숨겨져 있었다.

연구진은 플라세보 효과가 존재했으며 참가자들의 채팅 종료 후 평가뿐만 아니라 채팅 중에도 심성 모형에 따라 사용자와 AI의 감정 상태가 동기화한 사실에 주목했다. GPT-3의 학습 방식이 피드백형 강화학습이기 때문에 참가자의 감정선을 미러링했던 것으로 보인다. 결과적으로 긍정적인 사전정보를 받은 참가자는 대화가 진행될수록 참가자와 AI 모두 점점 더 높은 강도의 긍정적인 감정을 느꼈고 부정적인 선입견을 품으면 정반대의 결과가 나왔다.

모델 성능 표시 투명해야 객관적인 사용자 경험 측정 가능해

위 연구 결과는 대중이 인공지능을 바라보는 시선과 태도가 앞으로 기술과 상호작용하는 우리의 경험을 좌우한다는 점을 지적한다. 인간과 컴퓨터의 상호작용을 연구하는 전문가들도 대중 매체를 통해 알게 되는 AI 소식을 비판적으로 받아들이고 자아 성찰과 자기 계발의 도구로써 생성형 AI를 활용하길 조언했다. 일각에선 대중 매체의 잘못된 메시지뿐만 아니라 편향된 인공지능 프로그램과 상호작용할 때는 특히 주의해야 한다고 꼬집었다. 논문의 1 저자로 참여한 MIT 미디어 연구실의 팟 파타라누타폰(Pat Pataranutaporn) 박사 과정생은 사용자 측에선 편향성을 판단하기 어려우니 식품에 영양성분을 표시하는 것처럼 통일된 표준 아래 개발사도 자사의 AI 프로그램을 평가한 자료를 공개하는 것도 하나의 방법이라고 제안했다.

한편 모델의 성능이 좋지 않음에도 불구하고 과대광고를 진행하는 개발사에 대한 규제는 필요해 보인다. 같은 연구에서 150명의 참가자를 대상으로 GPT-3 대신 규칙 기반(Rule-based) 모델(ELIZA)로 똑같은 실험을 진행했었다. ELIZA와 대화한 세 그룹 모두 GPT-3에 비해 눈에 띄는 감정 변화가 없었지만, 프로그램에 공감 능력이 있다고 통보받은 그룹에선 모든 평가 항목에 상대적으로 후한 점수를 줬다. 물론 GPT-3의 점수보단 전반적으로 더 낮았지만, 정교하고 정확한 모델 개발을 독려하기 위해서 품질 관리 감독이 이뤄져야 한다.

파타라누타폰 박사 과정생은 일본 애니메이션 ‘도라에몽’을 보고 자라면서 기계에 대한 긍정적인 생각이 자리 잡았다고 얘기했다. 아시아 국가에서 받아들이는 인공지능에 대한 인식이 비교적 긍정적인 편에 속하고 문화마다 인식의 차이는 더 클 것으로 예상한다고 전했다. 다양한 사회문화 맥락에서 위 논문의 범용성이 증명되면 적용 범위가 더 넓어질 것이다. 또한 논리 영역 평가엔 어떤 작용이 있을지 알아보는 것도 흥미로운 주제다. 모델의 추론 능력은 높은 확률로 사용자의 인식과는 무관하겠지만, 논리 문제를 해결할 수 없는 상황에선 사용자가 포기하지 않고 해결책을 찾아갈 수 있도록 긍정적인 감정을 끌어내는 효과를 기대해 볼 수는 있다. 챗봇이 사용자를 향에 긍정적인 태도를 가지는 방향도 함께 고민해야할 부분이다.

---

The Assumptions You Bring into Conversation with an AI Bot Influence What It Says

A new study reveals an “AI placebo effect”: the same chatbot will respond differently depending on its users’ assumptions about artificial intelligence

Do you think artificial intelligence will change our lives for the better or threaten the existence of humanity? Consider carefully—your position on this may influence how generative AI programs such as ChatGPT respond to you, prompting them to deliver results that align with your expectations.

“AI is a mirror,” says Pat Pataranutaporn, a researcher at the M.I.T. Media Lab and co-author of a new study that exposes how user bias drives AI interactions. In it, researchers found that the way a user is “primed” for an AI experience consistently impacts the results. Experiment subjects who expected a “caring” AI reported having a more positive interaction, while those who presumed the bot to have bad intentions recounted experiencing negativity—even though all participants were using the same program.

“We wanted to quantify the effect of AI placebo, basically,” Pataranutaporn says. “We wanted to see what happened if you have a certain imagination of AI: How would that manifest in your interaction?” He and his colleagues hypothesized that AI reacts with a feedback loop: if you believe an AI will act a certain way, it will.

To test this idea, the researchers divided 300 participants into three groups and asked each person to interact with an AI program and assess its ability to deliver mental health support. Before starting, those in the first group were told the AI they would be using had no motives—it was just a run-of-the-mill text completion program. The second set of participants were told their AI was trained to have empathy. The third group was warned that the AI in question was manipulative and that it would act nice merely to sell a service. But in reality, all three groups encountered an identical program. After chatting with the bot for one 10- to 30-minute session, the participants were asked to evaluate whether it was an effective mental health companion.

The results suggest that the participants’ preconceived ideas affected the chatbot’s output. In all three groups, the majority of users reported a neutral, positive or negative experience in line with the expectations the researchers had planted. “When people think that the AI is caring, they become more positive toward it,” Pataranutaporn explains. “This creates a positive reinforcement feedback loop where, at the end, the AI becomes much more positive, compared to the control condition. And when people believe that the AI was manipulative, they become more negative toward the AI—and it makes the AI become more negative toward the person as well.”

This impact was absent, however, in a simple rule-based chatbot, as opposed to a more complex one that used generative AI. While half the study participants interacted with a chatbot that used GPT-3, the other half used the more primitive chatbot ELIZA, which does not rely on machine learning to generate its responses. The expectation effect was seen with the former bot but not the latter one. This suggests that the more complex the AI, the more reflective the mirror that it holds up to humans.

The study intimates that AI aims to give people what they want—whatever that happens to be. As Pataranutaporn puts it, “A lot of this actually happens in our head.” His team’s work was published in Nature on Monday.

According to Nina Beguš, a researcher at the University of California, Berkeley, and author of the upcoming book Artificial Humanities: A Fictional Perspective on Language in AI, who was not involved in the M.I.T. Media Lab paper, it is “a good first step. Having these kinds of studies, and further studies about how people will interact under certain priming, is crucial.”

Both Beguš and Pataranutaporn worry about how human presuppositions about AI—derived largely from popular media such as the films Her and Ex Machina, as well as classic stories such as the myth of Pygmalion—will shape our future interactions with it. Beguš’s book examines how literature across history has primed our expectations regarding AI.

“The way we build them right now is: they are mirroring you,” she says. “They adjust to you.” In order to shift attitudes toward AI, Beguš suggests that art containing more accurate depictions of the technology is necessary. “We should create a culture around it,” she says.

“What we think about AI came from what we see in Star Wars or Blade Runner or Ex Machina,” Pataranutaporn says. “This ‘collective imagination’ of what AI could be, or should be, has been around. Right now, when we create a new AI system, we’re still drawing from that same source of inspiration.”

That collective imagination can change over time, and it can also vary depending on where people grew up. “AI will have different flavors in different cultures,” Beguš says. Pataranutaporn has firsthand experience with that. “I grew up with a cartoon, Doraemon, about a cool robot cat who helped a boy who was a loser in … school,” he says. Because Pataranutaporn was familiar with a positive example of a robot, as opposed to a portrayal of a killing machine, “my mental model of AI was more positive,” he says. “I think in … Asia people have more of a positive narrative about AI and robots—you see them as this companion or friend.” Knowing how AI “culture” influences AI users can help ensure that the technology delivers desirable outcomes, Pataranutaporn adds. For instance, developers might design a system to seem more positive in order to bolster positive results. Or they could program it to use more straightforward delivery, providing answers like a search engine does and avoiding talking about itself as “I” or “me” in order to limit people from becoming emotionally attached to or overly reliant on the AI.

This same knowledge, however, can also make it easier to manipulate AI users. “Different people will try to put out different narratives for different purposes,” Pataranutaporn says. “People in marketing or people who make the product want to shape it a certain way. They want to make it seem more empathetic or trustworthy, even though the inside engine might be super biased or flawed.” He calls for something analogous to a “nutrition label” for AI, which would allow users to see a variety of information—the data on which a particular model was trained, its coding architecture, the biases that have been tested, its potential misuses and its mitigation options—in order to better understand the AI before deciding to trust its output.

“It’s very hard to eliminate biases,” Beguš says. “Being very careful in what you put out and thinking about potential challenges as you develop your product is the only way.”

“A lot of conversation on AI bias is on the responses: Does it give biased answers?” Pataranutaporn says. “But when you think of human-AI interaction, it’s not just a one-way street. You need to think about what kind of biases people bring into the system.”