open-computer-use vs OpenComputer

open-computer-use is a free, MIT-licensed local MCP server that lets AI agents inspect and operate desktop applications through accessibility interfaces. Its nine core tools cover app discovery, accessibility-tree and screenshot capture, clicking, scrolling, dragging, typing, key presses, secondary actions, and control-value changes. The project says it runs entirely on the user’s machine and uses the accessibility layer for core app actions without taking over the physical mouse and keyboard. Setup documentation covers Codex CLI and App, Claude Code, Gemini CLI, and opencode, with manual configuration for other MCP clients. A ZCode plugin installs the skill and connects an MCP server. macOS 14 or later requires Accessibility and Screen Recording permissions. Windows and Linux use a signed-in desktop session, and Linux requires AT-SPI2. Additional window-level tools are currently available on Windows, while macOS and Linux support is in progress. Password managers are refused; app launching, focus changes, and global input injection require explicit environment-variable gates. The project warns that high-privilege automation should be used only on authorized machines.

OpenComputer is a free, MIT-licensed open-source desktop agent that runs locally and uses screen understanding to operate applications through mouse and keyboard actions. It can click, drag, scroll, enter text, use shortcuts, open URLs, and wait for interface changes. The agent can connect to a local vision-language model through vLLM or to a hosted API; the endpoint must accept image inputs, function tools, and the model’s tool-call format. Ask first is the default permission mode and requests approval for mouse, keyboard, and URL actions, while Automatic mode runs them without individual prompts. Build targets include Linux AppImage and deb packages, Windows NSIS, and macOS DMG. Linux X11 is best supported; Wayland, Windows, and macOS are experimental. The current agent operates only on the primary display. Screenshots, prompts, and tool results can remain local. API keys use user-only file permissions but are not stored in the operating-system keychain. The project describes itself as early alpha and recommends supervision for sensitive workflows.