Claude gets a mouse. Keep an eye on yours.
Claude can click and type in a public beta. The uncomfortable part is deciding how much supervision counts as saving yourself work.
Anthropic’s computer-use public beta gives Claude 3.5 Sonnet a familiar way into your working life: a cursor and a keyboard. With a developer’s software setup and permission, the model can look at a screen, click and type. Anthropic describes the capability as experimental and error-prone. Still, the invitation is easy to understand. You already know what someone sitting at your computer can do. I find that a promising introduction. [1] [2]
The company says Claude examines screenshots and counts pixels to position the cursor. Training on simple software, including a calculator and a text editor, helped it generalize to other tasks; Anthropic reports seeing it retry after obstacles. The ambition is to use ordinary software instead of building a separate tool for every task. That is an appealing bargain for developers. It is also a reason to ask which actions you are comfortable delegating. A familiar interface does not settle that question. [2] [1]
The benchmark numbers complicate the sales pitch. Anthropic reports 14.9% on OSWorld’s screenshot-only category, rising to 22.0% with more steps. OSWorld’s researchers built tasks spanning real applications and operating systems to test whether agents could actually complete computer work. Those results concern a benchmark, not your particular workflow. More attempts may help an agent finish; they do not make its first attempt reliable. The distinction matters when the demonstration becomes a task you need done. [2] [1] [3]
Anthropic says screenshot-based perception can miss short-lived notifications, while actions such as dragging and zooming remain beyond Claude’s abilities in its development account. Its launch announcement recommends starting with low-risk tasks. Sensible advice, with an awkward implication: someone still has to choose what qualifies as low risk. Delegation begins to look less like surrendering a chore and more like learning to supervise it. [2] [1]
There is another complication. Anthropic warns that malicious instructions in onscreen content could divert the model from its user’s intent. It describes precautions and ongoing safety work, not a solved problem. The editorial question is therefore smaller, and more useful, than whether machines can take over the desktop: how much watching does this particular convenience require? I can be patient about acquiring the keys. You should be precise about which doors they open. [2]
