Find tools › Open-source projects › Automate busywork

esp32-iphone-agent

Open-source project · by onehumanbeing

Lets an AI see and tap your real iPhone without jailbreaking or needing a Mac developer account.

Some setupWell known

What it costs

Free and open source; you may pay for the AI model it uses.

Use it to

  • Automate repetitive tasks inside native iPhone apps and settings
  • Test mobile app interfaces by executing automated visual workflows
  • Interact with mobile games or drawing apps lacking software APIs

How to start

  1. Configure Wi-Fi settings in secrets.h and flash your ESP32-S3 board using hardware/flash.sh.

  2. Install UxPlay on macOS or Linux and start the Python server with your vision API keys.

  3. Turn on AssistiveTouch on your iPhone, plug in the flashed board, and mirror the screen via AirPlay.

  4. Open the web dashboard at port 8890, calibrate the pointer, and enter task goals.

Good to know. Requires purchasing an ESP32-S3 board, Wi-Fi configuration, and turning on AssistiveTouch on the target phone.

Before you install

  • Found automaticallyNot reviewed by a person yet. Check it before installing.
  • License: MITYou are allowed to use it under these terms.
  • Last updated 2026-11-18

More detail

Controls a physical iPhone using an ESP32 microcontroller as a USB mouse and AirPlay for screen vision.

Standard iOS automation requires a Mac, Xcode, and Apple developer certificates. Reach for this to automate any iPhone app or game using vision models without jailbreaking or developer setups.

Similar tools

Descriptions are written in plain English from each project's own documentation and refreshed by our agent; this one was last checked 2026-10-13. Spotted a mistake? · Open in Find tools · © 2026 Which AI Map