Generative AI Creates Realistic Virtual Try On Experiences

The Challenge of Online Shopping Meets an AI Solution

Despite the endless convenience of online shopping, one major hurdle remains: the uncertainty of buying clothes without trying them on. This common problem has fueled the development of virtual try-on technology, where users can preview how a garment might look on them. Now, a new AI model from The Grainger College of Engineering at the University of Illinois Urbana-Champaign is set to completely transform this experience.

Researchers from the lab of Yuxiong Wang, an assistant professor of computer science, have harnessed generative AI to create incredibly detailed and customizable virtual try-on videos. This work, conducted in collaboration with the e-commerce startup SpreeAI, introduces user-controlled features designed to push the boundaries of virtual shopping.

“We are harnessing the latest advances in generative AI, a technology that is already transforming image and video creation, to reimagine what is possible in e-commerce,” Wang explained. “Our goal is to push the boundaries even further and explore how this technology can truly empower the future of virtual try-ons.”

Introducing Dress&Dance A New Benchmark in Realism

Previous virtual try-on models often produced low-resolution results and struggled with varied input conditions. To overcome these challenges, the Illinois team created Dress&Dance, a video diffusion framework that generates short, high-quality video clips of users modeling clothes they've selected.

With just one photo of the user, one of the desired garment, and a short video of a pose or dance, the AI model generates a five-second video of the user in the new outfit. Unlike older systems that required perfectly staged photos, this model can use images of clothing in real-life settings—hanging on a rack, folded on a bed, or even worn by someone else. The resulting videos set a new standard for visual quality and realism, achieving the highest resolution and frame rate among existing methods.

From Short Clips to a Full Virtual Fitting Room

Building on their initial success, Wang's lab developed an upgraded model called Virtual Fitting Room to address the limitations of short clips.

“Five seconds is not enough to capture the full 360-degree details of a garment,” Wang noted. “Our latest model is designed to generate longer, more complete videos that bring the virtual try-on experience closer to reality.”

Virtual Fitting Room is the first model of its kind to produce arbitrarily long try-on videos with user-controlled options, such as the ability to layer multiple items of clothing. It works by generating each segment of the video step-by-step, using information from previous frames to ensure a seamless and consistent final product.

Impact and the Road Ahead

The new and improved model benefits everyone from consumers wanting to shop with confidence to retailers hoping to reduce returns. It also provides a valuable contribution to fellow academics in the field.

Going forward, the team is focused on accelerating the model's speed. “Generating long videos segment by segment can be slow, and most users would like to instantly see how they will look in the desired garment,” said Junkun Chen, a CS Ph.D. student and lead author of the papers. “They do not want to wait seconds or minutes, so improving the model’s efficiency has been central to integrating it into virtual try-on apps.”

The team's research is set to be published at the prestigious Annual Conference on Neural Information Processing Systems (NeurIPS), a leading venue for AI research.