Find tools › Open-source projects › Automate busywork

agent-desktop

Open-source project · by vercel-labs

A program that lets AI helpers see and click buttons inside desktop software on your computer.

Some setupPopular134,575 installs a month

What it costs

Free and open source; you may pay for the AI model it uses.

Use it to

  • Take structured text snapshots of open desktop windows
  • Click buttons and type text into desktop applications automatically
  • Add desktop automation capabilities to AI agent workflows

How to start

  1. Install the package globally by running npm install -g agent-desktop.

  2. Grant required accessibility permissions when prompted on your computer.

  3. View open desktop apps using agent-desktop list-apps.

  4. Inspect interactive elements of an app with agent-desktop snapshot --app Finder -i.

Good to know. Requires desktop accessibility permissions, and captured snapshots or clipboard contents may contain sensitive private desktop information.

Before you install

  • Found automaticallyNot reviewed by a person yet. Check it before installing.
  • License: Apache-2.0You are allowed to use it under these terms.
  • Last updated 2026-11-18

More detail

A command-line tool and library that lets AI agents inspect and control desktop applications using operating system accessibility trees.

Visual desktop automation using screenshots and pixel matching is slow, fragile, and inaccurate. This tool provides structured text views of screen elements so agents can click, type, and manage windows directly.

Similar tools

Descriptions are written in plain English from each project's own documentation and refreshed by our agent; this one was last checked 2026-11-02. Spotted a mistake? · Open in Find tools · © 2026 Which AI Map