Google is bringing multimodal search to AI Mode, its Google Search experiment that lets users ask complex, multi-part questions and follow-ups to dig deeper on a topic. Users who have access to AI Mode can now tap the feature to ask questions about photos they’ve uploaded or taken with their camera.
The new image-analyzing functionality in AI Mode is powered by Google Lens’ multimodal capabilities, Google said in a blog post on Monday.
AI Mode can understand the entire scene in an image, including how objects relate to each other, as well as their materials, colors, shapes, and arrangement, according to Google. Using a technique called “query fan-out,” AI Mode asks multiple questions about both the image and the objects shown in it, providing more detailed information than a traditional Google search.
For example, you could snap a photo of your bookshelf and enter the query: “If I enjoyed these, what are some similar books that are highly rated?” AI Mode will identify each book and then provide a list of recommended books with links to learn more about and/or purchase them.
AI Mode also lets you ask follow-up questions to narrow down your search, such as “I’m looking for a quick read. Which one of these recommendations is the shortest?”
As part of Monday’s announcement, Google said it’s making AI Mode available to millions more users who are enrolled in Labs, Google’s home for experimental features and products. Prior to this, AI Mode was only available to Google One AI Premium subscribers.
Launched last month, AI Mode looks to take on popular services like Perplexity and OpenAI’s ChatGPT Search. Google has said that it plans to continue to refine the user experience and expand functionality in the feature.
