Google Gemini Now Transcribes and Summarizes Audio Files

Google's Gemini AI has received a major upgrade that many users have been waiting for: the ability to listen. You can now upload audio files directly to the Gemini web and mobile apps to get fast transcriptions, summaries, and breakdowns of key details.

(Image credit: Google)

For anyone with a backlog of unlistened voice memos or who dreads scrubbing through meeting recordings, this update is like having a personal assistant dedicated to note-taking. This powerful new tool is designed to make your audio content searchable and useful.

How Gemini's New Audio Feature Works

This new feature, which Google’s VP of Gemini Josh Woodward confirmed has been the most requested from users, allows you to upload audio files directly from the standard file upload menu. It's important to distinguish this from Gemini Live, which involves speaking to the AI in real-time for commands. This new upload function is about processing pre-recorded audio data, much like how Gemini handles documents or images.

However, there's a key limitation to keep in mind for now: the AI can only process about 10 minutes of audio at a time. This means it's perfect for short clips and voice notes but not yet ready for your hour-long lectures or marathon meetings.

Putting Audio Transcription to the Test

In a hands-on test, the feature proved to be quite capable. After uploading a few sketches from old comedy albums and a recording of a phone call, Gemini successfully transcribed the dialogue in each case, making only a few minor errors with names. It also excelled at identifying and pulling out key topics and action items, effectively creating a to-do list from the conversation.

The high demand for this feature highlights a shift in how we use AI tools. We're increasingly relying on them to manage the vast amounts of information we capture, including audio logs and voice memos. By building this transcription and analysis tool directly into Gemini, Google has simplified a multi-step process into a single, seamless action.

The Competitive Landscape

While Gemini's audio upload option is a significant addition, it's not entirely unique in the AI space. It helps Gemini catch up to competitors like ChatGPT, which leverages its powerful Whisper model for transcription. In some tests, Gemini's offering was even preferred.

Other rivals are also in the audio game. Anthropic’s Claude offers audio processing through some developer tools, and Perplexity has the ability to analyze content from YouTube videos. Gemini's advantage lies in its straightforward execution and focus on common, everyday use cases, making it accessible to a broader audience.

Beyond Simple Transcription

Gemini’s new skill isn’t just about turning speech into text. You can interact with the transcribed content by asking Gemini to simplify the language, separate comments made by different speakers, or even generate a list of questions based on the discussion. For students, it can create a study guide from a recorded class discussion.

Of course, the 10-minute cap remains a constraint for more extensive applications. Free users should also be aware of daily usage limits. Google hasn't detailed specific pricing for high-volume use, but it falls under the standard Gemini usage quota, so users planning to process large amounts of audio will need to manage their usage accordingly.

This update is part of a larger trend of enhancements for Gemini, which you can read more about here:

• I tried Google's new AI mode powered by Gemini, and it might be the end of Search as we know it
• Google previews AI Mode for search, taking on the likes of ChatGPT search and Perplexity
• Google Gemini's new model is the brainstorming AI partner you've been looking for