Computer use in Gemini 3.5 Flash

sswolpers about 3 hours ago 55 commentsRead Article on blog.google

⚡ Community Insights

Discussion Sentiment

72% Positive

Analyzed from 1397 words in the discussion.

Discussion (55 Comments)Read Original on HackerNews

revolvingthrow•about 1 hour ago

People using google’s models: am I holding it wrong or are the guardrails really overtuned?

I had the dubious pleasure of testing gemini of late and I kept running into refusals. How do I transfer a sim number from one provider to another? No. What should I consider when making backups on ntfs less prone to data loss and more bitrot resistant? No. Evaluate this piece of code? No.

I’m not sure if it’s cold feet from the mythos situation or what, but it reminds me of the dark days where you couldn’t use ai for much of anything. But then I go to chatgpt 5.5 and it does mostly everything I want outside of the usual cybersecurity boogeyman that you run into now and then.

sva_•27 minutes ago

Interesting. I have the Google AI Pro plan and use Gemini several times each day and I don't remember the last time I got a refusal. I wonder what criteria go into that, like maybe how they rate your Google account?

Chu4eeno•about 1 hour ago

I've always found all versions of gemini to be (for a lack of a better word) lazy.

I guess it's economic wrt. token use, but it often either refused for absurd safety reasons, or other weird stuff like responding that an LLM like itself wasn't a suitable tool for the job, and very quickly gives up.

Claude is on the other end of the spectrum, which makes it more noticeable when switching between them.

k8sToGo•about 1 hour ago

The context window size is also very small if you use Gemini in the app. It starts forget quite fast. In my opinion Gemini on app is useless additionally to the guardrails.

kordlessagain•about 1 hour ago

I love antigravity. I’ve had zero issues with it.

nout•about 1 hour ago

I just asked gemini the question with sim number and it gives me full step by step guide.

WarmWash•30 minutes ago

Are you outside the US?

esperent•26 minutes ago

I'm outside the US, use Gemini models quite a bit, and I've never run into any refusals of any kind. I'm using them for a fairly wide range of things, I'm sure at least as risqué as asking how to transfer a sim. As a matter of fact I actually asked it's advice on how to transfer banking apps and auth apps from one phone about 3 weeks ago and got decent answers.

airstrike•about 3 hours ago

Computer use is such a terrible idea. It's slow, insecure, error prone, expensive.

I guess if you're trying to get people to tokenmaxx it may look like a valid strategy, but ain't no way this will be delightful to users.

I think it's a symptom of just not understanding how LLMs should interface with the OS because we're still in their early days.

Eventually there'll be an iPhone moment for the ergonomics of LLM usage outside of coding

gdudeman•about 2 hours ago

Computer use is a great idea. It gets the job done when nothing else will.

If you're a person trying to get their job done at a big company, but half your job is in 1-2 proprietary tools or is stuck behind an API you can't program against, computer use can allow you, a non-techie, to do your job more efficiently.

I think it's an awesome way to circumvent gate keepers and the IT department to let people accomplish their goals.

Rebelgecko•10 minutes ago

I think there's a sweet spot- a lot of the time you're probably better off with "reverse engineer this web page and build me an API or personalized chrome extension to meet my needs".

I have an agent doing price checks for me for an item on a certain website. Instead of blasting through a zillion tokens processing the DOM over and over, it loaded the page once and figured out how to download a json with the price.

reacharavindh•31 minutes ago

How are folks using “computer use” to click things on intranet portals that are behind an SSO? Even this OP example shows visitors a url and enter this search term… that is port of useless.

How can I automate things behind an SSO wall? Even if it means I manually authorize it once and watch it do things on its own..

airstrike•35 minutes ago

That is an incredibly niche use case and comes with a boatload of footguns.

Even then, an AI writing AHK scripts likely outperforms.

uejfiweun•about 1 hour ago

Yeah, it's not that computer use is the most theoretically optimal paradigm, but there's a reasonable case that given the constraints of modern software systems and how they're built, that it's the most realistically optimal paradigm.

thorum•about 2 hours ago

The “correct”, elegant way for AI to interact with existing software would take decades and billions of dollars to build. Someone would have to do the hard work of building new APIs, solving decades of accessibility issues, etc.

Or you can show an AI screenshots and ask it where to click.

sarreph•about 2 hours ago

I disagree if your application is networked. Most SaaS is built on RESTful APIs that can be converted trivially into interfaces / contracts for tool use.

chatmasta•about 1 hour ago

So you can either wait for every application to do that, or at least make it possible for an LLM to do it… or you can make the LLM use a computer interface that works with every application by definition.

jubilanti•about 1 hour ago

it takes decades and billions of dollars to develop APIs?

orbital-decay•about 1 hour ago

Spreadsheet is such a terrible idea. It may look like a valid tool, but ain't no way it's delightful to users. Most of the time people need a database instead. Eventually there'll be an iPhone moment for this.

Meanwhile, the entire world economy:

airstrike•36 minutes ago

I mean, your words not mine. You can't just claim I'm making a point I didn't.

Spreadsheets are fucking glorious, powerful, clever, amazing and delightful, in my view.

dyauspitr•35 minutes ago

We shouldn’t optimize for token use. We should build infrastructure to make tokens dirt cheap instead.

api•about 2 hours ago

It's great for testing and QA automation for UIs. It's also possibly good for the vision impaired.

orbital-decay•about 1 hour ago

UI QA only works well if your model plausibly matches the average user behavior and/or real-world edge cases. These models are far from that, and they are much less random than you'd like them to be for fuzzing (mode collapse).

nzach•about 2 hours ago

> Computer use is such a terrible idea. It's slow, insecure, error prone, expensive.

And yet having an agent able yo use a computer on your behalf is really useful.

Recently I gave a Nix OS vm to my hermes agent and it has been a good experience. I don't really care if destroy the machine I can just rollback to an earlier version, and for any meaningful data he creates for me I make sure he creates a repo, commit and pushes to my private Gitea instance.

airstrike•about 2 hours ago

> And yet having an agent able yo use a computer on your behalf is really useful.

It is, but there's no need for it to be viewing your screen, browsing websites and watching ads.

That stuff is for humans, not for LLMs.

nzach•about 2 hours ago

Sure, I don't want an agent watching MY screen. That's why I gave him his own environment, and pretty quickly he discovered that you can open chrome and make it render to a framebuffer, this way he is able to 'view' the website. And apparently with this he is able to bypass a lot of 'anti-bot' measures.

satvikpendem•about 3 hours ago

There's still no MCP support in the Gemini app, which is very useful to get various pieces of info as a user just via chatting. For example I recently wanted to get an Airbnb and wanted to filter by specific criteria including house image analysis and Gemini couldn't do it so I had to do it in Codex.

anticorporate•about 3 hours ago

Yeah, it seems like this is the biggest missing feature from the Gemini ecosystem.

If I can't connect MCP, there's really no selling point for me to use Gemini from my watch, car, smart speaker, etc. If I'm already bound to using my own front end, then I'm only evaluating Gemini as a model/API, at which point it has many competitors that may be cheaper or better fit for the task.

thejaycampbell•about 3 hours ago

agreed... this is where they lost me too

mitchell_h•about 2 hours ago

I'm fairly convinced Claude's strongest point is the app. AI users aren't anywhere near as mature or smart as youtube/hn would have folks believe. The claude app is amazing for bridging that gap.

dr_dshiv•about 1 hour ago

Didn’t it take them like 2 days to build the first one?

dr_dshiv•about 1 hour ago

Didn’t it take them like 2 days to build it?

solarkraft•about 1 hour ago

They only fixed stopping the model mid-generation losing the entire session pretty recently.

The Gemini apps suck.

tonyrice•about 3 hours ago

This is why I don't always use the official Gemini Web app. Lately I've found that it's more useful to utilize a CLI. I'm looking forward to the day they add MCP in the web.

pregseahorses•about 3 hours ago

Gemini CLi now requires antigravity subscription..

singingtoday•about 2 hours ago

CLI doesn't work with my subscription..

mlmonkey•about 3 hours ago

It's funny how in their own graph, https://storage.googleapis.com/gweb-uniblog-publish-prod/ima... Gemini 3.5 Flash is beat hands down by both Opus 4.8 and GPT 5.5, and yet the graph is drawn as if Gemini wins ... :-D

mroche•about 2 hours ago

The graph has Gemini 3.5 Flash matching Sonnet 4.6, losing to Opus 4.8, and slightly behind GPT-5.5 by 0.3 points... That's not that much of a hands-down loss for Gemini for this specific workload benchmark.

The methodology used:

https://deepmind.google/models/evals-methodology/gemini-3-5-...

Methodology: All Gemini scores are pass @1 except where otherwise noted. "Single attempt" settings allow no majority voting or parallel test-time compute. All of the results are all run with the Gemini API for the model-id gemini-3.5-flash with default sampling settings unless indicated otherwise below. To reduce variance, we average over multiple trials for smaller benchmarks.

All the results for non-Gemini models are sourced from providers' self reported numbers unless otherwise mentioned below. For Claude Opus 4.7 , Sonnet 4.6, and GPT-5.5 we default to reporting maximum thinking/reasoning settings available, but when reported results are not available we use best available reasoning results.

sheept•about 3 hours ago

It highlights the Gemini models blue since that's what the article is about. The bar heights seem consistent with the values.

gb2d_hn•about 2 hours ago

It's honest - people who know what they are looking at will take speed and token costs into account. I don't use Gemini 3.5 for coding, but I use it as something in between a search engine and agent.

data-ottawa•about 2 hours ago

I think 3.5 flash is trying to target agentic work, like Google Search or ADK (agent development kit) use cases.

It’s something cheap enough you’d put out in front of your customers, and Opus is expensive enough you wouldn’t.

fridder•about 2 hours ago

I wonder if it will be better at building TUI's. It has been absolutely abysmal at interacting with them and building them

knollimar•about 2 hours ago

Where is 3.5 pro?

squidbeak•31 minutes ago

Google said June, and all its model updates seem to be on Tuesdays, Wednesdays or Thursdays. So unless the release is slipping, either tomorrow or Tuesday.

WarmWash•26 minutes ago

Rumor is now July, although preliminary A/B tests people are getting show promise with whatever they have right now.

beastman82•about 3 hours ago

No UI like their competitors Claude CoWork or Codex. This is vaporware

zuzululu•about 2 hours ago

performance is quite impressive given that its 3x cheaper than 5.5

villgax•about 3 hours ago

Will it skip Ads lol

humblyCrazy•about 2 hours ago

I looked at their demo and it does not