AI Learns To See Like Us Through Games

At Brown University, an innovative new project is showing that teaching artificial intelligence to perceive things more like people might start with something as simple as a game. This initiative invites people to play an online game called Click Me, which helps AI models learn how humans see and interpret images. While the game is enjoyable and easy to access, its underlying goal is significant: to uncover the fundamental reasons for AI errors and to systematically enhance how AI systems represent the visual world.

The Growing Gap Between AI and Human Vision

Over the last ten years, AI systems have become increasingly powerful and are now widely used, especially for tasks like image recognition. For instance, these systems can identify animals, objects, or even diagnose medical conditions from images. However, they sometimes make mistakes that humans rarely would. An AI algorithm might, for example, confidently mislabel a photo of a dog wearing sunglasses as a completely different animal or fail to recognize a stop sign if it is partially obscured by graffiti. As these AI models grow larger and more complex, these types of errors become more common, indicating a widening difference between how AI and humans perceive the world.

A New Path Towards Human Aligned AI

Recognizing this issue, researchers, partly funded by the U.S. National Science Foundation, are proposing a new strategy. They aim to combine knowledge from psychology and neuroscience with machine learning techniques to create the next generation of AI that is more aligned with human cognition. Their objective is to understand how people process visual information and then translate these patterns into algorithms that guide AI systems to behave in similar ways.

Click Me How an Online Game Teaches AI

The Click Me game is central to this vision. In the game, participants click on areas of an image they think will be most informative for an AI to recognize it. The AI only gets to see the parts of the image that have been clicked. This encourages players to think strategically about the most useful parts of the image, rather than clicking randomly, thereby maximizing the AI's learning potential.

Neural Harmonization Aligning AI with Human Focus

The alignment between AI and human perception happens at a later stage, when the AI is trained to categorize images. In a process called "neural harmonization," the researchers compel the AI to concentrate on the same image features that humans identified—those clicked during the game. This ensures that the AI's visual recognition strategy aligns with that of humans.

The Power of Crowdsourcing AI Training Data

What makes this project particularly noteworthy is its success in engaging the public. Funding from the NSF has enabled the team to attract thousands of participants to Click Me. The game has gained attention on platforms like Reddit and Instagram, generating tens of millions of interactions with the website, all of which help train the AI model. This large scale public involvement allows the research team to quickly gather extensive data on how people perceive and evaluate visual information.

Beyond Accuracy Mimicking Human Decision Making

Concurrently, the team has also developed a new computational framework for training AI models using this type of behavioral data. By aligning AI response times and choices with those of humans, the researchers can build systems that not only replicate human decisions but also mimic how long humans take to make those decisions. This results in a more natural and understandable decision making process for AI.

Transforming Industries Real World Applications

The practical applications of this research are extensive. In medicine, for example, doctors need to understand and trust the AI tools that help with diagnoses. If AI systems can explain their conclusions in ways that align with human reasoning, they become more dependable and easier to integrate into patient care. Similarly, in self driving cars, AI that better understands human visual decision making can help predict driver behavior and prevent accidents. Beyond these examples, human aligned AI could enhance accessibility tools, educational software, and decision support systems across many industries. Importantly, this work also provides insights into how the human brain functions. By simulating human vision in AI systems, researchers have been able to develop more accurate models of human visual perception than were previously available.

The Importance of Foundational AI Research

This initiative highlights why federal support for foundational research is crucial. Through the NSF's investment, researchers are advancing the science of AI and its relevance to society. This research not only pushes the boundaries of knowledge but also delivers practical tools that can improve the safety and reliability of the technologies we use every day.