GitHub Copilot: Computer Use for Desktop Apps in the CLI and the Copilot App
GitHub CopilotGitHub introduced computer use in public preview for GitHub Copilot CLI and the GitHub Copilot app on macOS and Windows. Copilot can now read accessible app content and visual context, click controls, type, press keys, scroll, and drag across desktop applications, including legacy and GUI-only software with no API, command line, or MCP integration. Copilot asks for approval before controlling an app, and organization-managed settings can disable the feature.
Key Takeaways
- Computer use lets Copilot operate desktop apps directly, clicking, typing, scrolling, and dragging through real application interfaces instead of relying on code or APIs.
- The feature reaches legacy and GUI-only software that has no API, command-line interface, or MCP integration, which agents previously could not touch.
- Approval comes first: Copilot asks before controlling any app, and developers can review or reset the apps they marked as always allowed.
- On macOS, Copilot guides users through Accessibility and Screen Recording permissions, which removes a common setup hurdle.
- Enabling it is simple, with
/computer onin the CLI or an Enable Computer Use toggle in the Copilot app settings. - The preview covers macOS and Windows only, and organization-managed settings can disable it for teams with stricter policies.
Copilot Can Now Operate Desktop Apps
GitHub Copilot is no longer limited to code editors, terminals, and APIs. GitHub introduced computer use in public preview for GitHub Copilot CLI and the GitHub Copilot app, available on macOS and Windows. With the feature enabled, Copilot can act on a developer's behalf inside ordinary desktop applications by reading accessible app content and visual context, then clicking controls, entering and editing text, pressing keys, scrolling, dragging, and moving through workflows that span several applications.
The practical effect is a wider range of tasks Copilot can help automate. Many everyday tools, especially legacy and GUI-only software, expose no API, command-line interface, or MCP integration, which left them out of reach for earlier agent features. Computer use gives Copilot a way into those tools by working through the same interface a person would use.
Staying in Control
GitHub built the feature around explicit approval. Copilot asks for permission before it controls an app, and developers can review or reset the list of apps they have chosen to always allow. On macOS, computer use also walks users through granting the required Accessibility and Screen Recording permissions, so setup does not depend on hunting through system settings. Organization-managed settings can disable the feature entirely, which lets teams keep it off where policy requires.
How to Turn It On
In Copilot CLI, the /computer on command enables the feature, /computer show reports its current status, and /computer off disables it. In the GitHub Copilot app, users open Settings, select Computer Use, and switch on Enable Computer Use. The /computer on command also works inside the app.
Getting Good Results
GitHub advises describing the outcome wanted, the applications involved, and any important constraints. Example tasks include summarizing notifications in a browser, updating content in a presentation, or moving information through a workflow in a desktop application. Because the feature is in public preview, behavior and availability may change, and it currently targets local sessions on macOS and Windows rather than Linux.