Google’s Gemini in Chrome Can Now See Exactly What You’re Looking At on Screen
Google has quietly rolled out a new capability for its AI assistant, Gemini, inside the Chrome browser. The feature, called “Select from screen,” allows users to highlight specific text or images from a webpage and send them directly to Gemini for contextual help. This marks a significant shift in how users interact with AI while browsing—moving from typing vague queries to pointing at exactly what they need assistance with.
Instead of manually describing a complex chart, a product description, or a confusing paragraph, users can now simply select the content on screen and let Gemini analyze it. The feature appears in Chrome’s “+” menu, functioning like an integrated screenshot tool that captures only the selected area.
How the Gemini Select from Screen Feature Works
Once activated, the tool overlays a selection interface on the current browser tab. Users can draw a box around any visible text or image, which is then automatically attached to a new conversation with Gemini. This eliminates the friction of copying, pasting, or describing content—making the AI assistant feel more like an extension of the browser itself.
For example, if you’re reading a research paper and need a quick summary of a dense paragraph, you can highlight it and ask Gemini to explain it. Similarly, if you see an image you want identified or a product you want compared, the AI can work with the exact visual or textual data you’ve selected.
This update is part of Chrome 149, though some users may need to restart their browser before the feature appears. Google has designed the tool to work with any webpage, whether it’s a news article, an e-commerce site, or a social media post.
Why This Matters for AI-Powered Browsing
The “Select from screen” feature is more than a convenience—it’s a fundamental change in how users interact with AI assistants. By giving Gemini direct access to what’s on screen, Google is moving beyond the traditional chatbot model toward an assistant that understands context and intent.
In the past, users had to describe what they were seeing, often leading to misunderstandings or incomplete answers. Now, Gemini can see exactly what you’re looking at, reducing ambiguity and making responses more accurate. This is especially useful for tasks like troubleshooting errors, comparing products, or extracting data from complex layouts.
Building on this, the feature also encourages more natural interactions. Instead of typing “What does this chart show?” you can simply select the chart and ask. The AI automatically knows what you’re referring to, making the conversation feel more fluid and intuitive.
Google’s Broader Vision for Computer Use
Interestingly, Google announced another major upgrade on the same day. Developers can now access computer use capabilities directly through Gemini 3.5 Flash, allowing AI agents to see, reason, and take actions across browsers, mobile apps, and desktop environments without relying on a separate model. This integration improves long-horizon tasks such as software testing, enterprise workflows, and multi-step automation jobs.
While the “Select from screen” feature is consumer-facing, it aligns with Google’s broader strategy: making AI aware of what users are actively doing. The company is positioning Gemini as an assistant that doesn’t just answer questions but helps complete tasks. As a result, the line between a chatbot and a productivity tool continues to blur.
What This Means for Everyday Users
For the average Chrome user, this update removes a common pain point. Have you ever struggled to describe a specific error message or a design element to an AI assistant? With the new feature, you can simply select it and let Gemini handle the rest.
Moreover, the tool works with both text and images, opening up possibilities for visual search, document analysis, and even creative tasks like generating captions or descriptions. It’s a small but powerful addition that makes browsing more efficient.
However, users should be aware that the feature requires an active internet connection and a Google account. It’s also worth noting that Google may use selected content to improve its AI models, so privacy-conscious users should exercise caution with sensitive information.
Final Thoughts on Gemini’s Screen Awareness
Google’s “Select from screen” feature is a clear answer to the growing demand for AI that understands what users are looking at. By reducing friction and improving context, it makes Gemini a more practical tool for daily browsing.
As AI assistants become more integrated into our digital lives, features like this will likely become standard. For now, Chrome users can enjoy a smarter, more intuitive way to interact with web content—just by pointing and clicking.
For more tips on using AI tools effectively, check out our guide on how to use Gemini in Chrome. And if you’re interested in other browser-based AI features, read about Chrome’s latest AI updates.