Find tools › Open-source projects › Automate busywork

agent-desktop
Open-source project · by vercel-labs
A program that lets AI helpers see and click buttons inside desktop software on your computer.
Some setupPopular134,575 installs a month
What it costs
Free and open source; you may pay for the AI model it uses.
Use it to
- Take structured text snapshots of open desktop windows
- Click buttons and type text into desktop applications automatically
- Add desktop automation capabilities to AI agent workflows
How to start
Install the package globally by running
npm install -g agent-desktop.Grant required accessibility permissions when prompted on your computer.
View open desktop apps using
agent-desktop list-apps.Inspect interactive elements of an app with
agent-desktop snapshot --app Finder -i.
Before you install
- Found automaticallyNot reviewed by a person yet. Check it before installing.
- License: Apache-2.0You are allowed to use it under these terms.
- Last updated 2026-11-18
More detail
A command-line tool and library that lets AI agents inspect and control desktop applications using operating system accessibility trees.
Visual desktop automation using screenshots and pixel matching is slow, fragile, and inaccurate. This tool provides structured text views of screen elements so agents can click, type, and manage windows directly.
Similar tools
Descriptions are written in plain English from each project's own documentation and refreshed by our agent; this one was last checked 2026-11-02. Spotted a mistake? · Open in Find tools · © 2026 Which AI Map