Snapseed AI Makes Complex Photo Edits Simple

Say Goodbye to Tedious Photo Selections

Ever wanted to brighten a person in the foreground of a shot, make the sky a more dramatic blue, or make a specific color pop without affecting the rest of the image? This is the power of selective editing, but achieving it has often been a frustrating and complex task. This is especially true on mobile devices, where small screens and touch controls make precise selections a real challenge.

That's all about to change. We are excited to introduce the new Object Brush in Snapseed on iOS. Located in the "Adjust" tool, this feature makes object-based image adjustments quick, easy, and intuitive. Just draw a simple stroke on the object you want to edit, and you can instantly adjust its look separately from the rest of the photo. This new capability will be rolling out in the coming week!

Intuitive Editing Through On-Device AI

The magic behind the Object Brush is our Interactive Segmenter, a powerful AI model that runs entirely on your device. With a simple gesture, like a tap or a quick line, you can indicate an object or person you want to select. In less than 20 milliseconds, the model detects and selects the complete object, creating a highly accurate mask that matches its boundaries.

Whether it's a person, a pet, or the clouds in the sky, this real-time feedback allows you to refine your selection on the fly by easily adding or removing areas until it's perfect. This entire process is made fast and seamless thanks to powerful tools like MediaPipe and LiteRT’s GPU acceleration. This blend of a simple user interface with an efficient machine learning model makes advanced photo editing more accessible and precise than ever before.

Training a Smarter On-Device AI

To build a universally capable segmentation model, we couldn't limit it to specific types of objects. We adopted the Big Transfer approach, which uses a general pre-trained model to help with training, complementing a smaller set of manually annotated images.

First, we created a highly accurate model called “Interactive Segmenter: Teacher.” We trained this large model on about 30,000 high-quality, pixel-perfect image masks. While this Teacher model produces incredible results, its size and speed make it unsuitable for running on a mobile device. To solve this, we used a process called knowledge distillation to transfer the Teacher's knowledge to a smaller, more efficient model called “Interactive Segmenter: Edge.”

We trained this smaller on-device model using over 2 million images with existing, less-perfect masks. We developed a system to simulate user interactions—like taps, scribbles, and lasso selections—to generate realistic prompts. The Teacher model would then create a perfect mask based on these prompts, providing a high-quality example for the on-device Edge model to learn from.

Balancing Quality and Speed for a Seamless Experience

A key challenge was delivering high-quality segmentation without the lag. Our solution was to decouple the process into two parts. First, a powerful, heavyweight image encoder runs once when you open the tool to analyze the image and extract key features. Its processing time is effectively hidden from the user. Second, a lightweight interactive decoder takes your touch inputs and uses the pre-computed features to generate the final mask instantly, well within our 20ms speed budget. This allows us to harness the power of a large model while delivering the responsiveness of a small one.

To ensure top-notch editing quality on high-resolution photos, the model predicts a detailed mask that is then upsampled using an edge-preserving joint-bilateral upsampling method. This upsampling only happens once you lift your finger, further improving the real-time interactive experience.

The Future of Intuitive Editing

With the new Interactive Segmenter in Snapseed, powerful photo editing is more accessible than ever. Simple gestures are translated into accurate selections, empowering you to bring your creative vision to life. This foundational technology is not just for Snapseed; it has also been integrated into the new Chromebook Plus 14 to power AI image editing in the Gallery app. We plan to bring this intuitive editing technology to more creative products across Google in the future.