Research led by Professor Dongwon Lee at Penn State University’s PIKE Lab highlights the difficulties in distinguishing AI-generated text from human-written content. While AI tools improve, public perceptions on AI applications show a mix of support and concerns, revealing the ongoing challenge of regulating and detecting AI-generated content.
Title: Challenges in Detecting AI-Generated Text and Public Perceptions on AI
Location: UNIVERSITY PARK, Pa.
Date: May 14, 2024
By: Mary Fetzer
Dongwon Lee, a professor in the College of Information Sciences and Technology at Penn State University, leads research on the integrity of AI-generated text in the Penn State Information Knowledge and wEb (PIKE) Lab. The research focuses on the difficulties of distinguishing AI-generated content from human-written text, especially as generative AI tools improve.
Lee highlighted that AI-generated text is often indistinguishable from human-created content. Lab experiments show that humans can only identify AI-generated text with 53% accuracy, marginally better than random guessing. In contrast, their AI tool can accurately classify text as AI-generated or human-written with up to 95% accuracy.
As generative AI technologies advance, solutions to detect AI content also evolve. Lee’s lab has developed a binary classifier using a state-of-the-art neural network model to determine text origins. Although effective, the model’s decision-making process remains largely a “black box,” posing challenges for critical applications in fields such as health and military.
Public awareness and perceptions of AI were explored in a survey by the Pew Research Center. Conducted among U.S. adults, the survey found that while 90% of Americans have heard about AI, only a third are highly informed. AI applications in healthcare and the workplace evoke mixed feelings. For example, 60% of Americans are uncomfortable with AI in healthcare decisions, yet 65% support AI-assisted skin cancer screenings. In workplaces, 62% foresee AI impacting jobs significantly, though only 28% expect a personal impact.
Generative AI tools like ChatGPT have garnered attention, with 58% awareness among U.S. adults, but only 18% have used it. Teens show higher familiarity at 67%, with 19% using it for schoolwork. Americans generally support regulation of AI technologies, with 67% concerned that governmental regulation may not be sufficient.
These insights underscore the complexity of integrating AI into daily life and the ongoing challenge of ensuring the ethical use and detection of AI-generated content.
