Copilot computer use is in preview: setting rules for desktop-driving agents

Coding agents have so far worked through text: files, terminals, APIs and MCP servers. On 1 October 2026 GitHub added a different kind of reach. Computer use, now in public preview in Copilot CLI and the Copilot app on macOS and Windows, lets Copilot operate desktop applications by reading what is on screen, clicking controls and typing. That opens up legacy and GUI-only software that has no API. It also means an agent can act anywhere the developer's desktop session can, which deserves a policy before it becomes a habit.

The context: what GitHub shipped

The changelog of 1 October describes the feature as public preview. Copilot can read accessible app content and visual context, click controls, enter and edit text, press keys, scroll, drag and move between applications. In Copilot CLI it is switched on with /computer on, checked with /computer show and switched off with /computer off; in the Copilot app it is a setting.

The computer use documentation adds the controls:

• It is disabled by default and must be enabled before an agent can use its tools.
• It follows the tool permission settings of the Copilot surface in use, which decide whether Copilot asks before controlling an app. At a prompt, access can be granted for the session, saved for future sessions, or denied.
• An Always allow decision for an app is stored locally and applies to both the CLI and the app on that computer. Removing it affects future sessions but does not revoke access already granted in a running session.
• Rules that deny a tool take precedence over automatic or saved approvals.
• Enterprise administrators can disable computer use through managed settings, and enabling it locally does not override that.
• A running operation can be interrupted with Esc twice in the CLI, or Stop in the app.
• On macOS, it needs the Accessibility and Screen Recording permissions.

The documentation also advises using an API, MCP server, terminal command or dedicated browser tool when one can do the job, because those give more structured and predictable results.

Analysis: what to make of it

The risks are in GitHub's own words

GitHub's documentation is candid. Computer use can select the wrong control, enter text in the wrong place, or repeat an action when timing or window state changes. It warns that ambiguous instructions or unexpected on-screen content "may cause unintended actions that affect your device, data, or connected accounts". It notes that windows may show sensitive information, including about other people, and advises against Always allow for applications that hold sensitive information or support high-impact actions. That is a useful basis for policy, because it comes from the vendor rather than from critics.

The screen is a new input channel

Prompt injection usually arrives through text an agent reads: a web page, an issue, a tool result. With computer use, anything visible on screen is input, including an email preview, a chat notification or a document someone else wrote. An agent filling in a form in one window can be shown instructions in another. GitHub's warning about unexpected on-screen content covers this. The practical defence is the one GitHub gives: keep the agent to the applications the task needs, and do not let approvals pile up.

This sits at the bottom of the isolation ladder

The NCSC's interim guidance on agentic AI grades compute isolation from level 1, where the agent runs directly on the host with no isolation, upwards. By its nature, a computer-use agent works in the developer's own desktop session, with whatever that session is signed in to. The computer use page does not describe it as covered by Copilot's local sandboxing, so teams should assume the controls are the permission prompts, deny rules, managed settings and the person watching. That is closer to level 1 than to the contained setups the NCSC recommends for higher-risk work.

Accuracy is improving, not solved

Models are getting better at this. Anthropic's Haiku 5.5 announcement of 7 October reports Haiku 5.5 at 72.4% on an offline subset of OSWorld 2.1, a benchmark of long, multi-step computer tasks, scored with partial credit. The same post says Anthropic's Python and TypeScript SDKs are adding computer use and browser use in beta. Those are vendor figures on a test set, and even taken at face value they leave a substantial share of tasks incomplete. For actions that change data, a person should still check the result.

What it means for engineering and security leaders

Computer use is useful for a narrow set of jobs: driving a legacy admin tool, moving data between two GUI-only systems, checking how an app renders. It is a poor default for anything that has an API or MCP server. A reasonable starting position for the preview:

• Off by default, on by decision. Leave it disabled in managed settings for most people, and enable it for named teams with a clear use.
• No Always allow for sensitive apps. Email, chat, password managers, finance, HR and production consoles should always prompt, or be denied outright with a deny rule.
• One task, few windows. Close unrelated applications before a computer-use session, and do not run one with confidential material on screen.
• Someone watching. Treat computer use as attended automation. Interrupting is easy; noticing is the hard part.

For broader isolation choices, see the NCSC's sandbox maturity levels, and for how injected content becomes action, from prompt injection to code execution.

Practical takeaways for this week

• Decide whether computer use should be available during the preview, and set the managed setting to match.
• Write down which applications may never be set to Always allow, and add deny rules where the surface supports them.
• Ask anyone already using it to review their saved approvals in the Copilot app.
• Prefer an API, CLI, MCP server or browser tool whenever one exists for the task.
• Remember that anything shown on screen during a session is shared with Copilot as context.

Where SpanDock fits

GitHub's own advice is to use a structured tool when one exists. For systems reachable over MCP, the SpanDock MCP gateway, on the Premium and Enterprise plans and in the 30-day trial, gives each person only the connectors and tools they are allowed, starts every connector read-only, and records every call, which is easier to govern than an agent clicking through the same system's interface. On Enterprise, process inventory shows which AI tools run on your team's computers, including ones that send no telemetry.