Sim-Use Review 2026 — Give Your AI Agent Eyes and Hands on iOS Simulator and Android Emulator
✅ Pros
- • Token-efficient screen representation (~16x more compact than raw accessibility tree JSON) — an LLM can reason about an entire mobile screen in a few hundred tokens
- • Cross-platform: iOS Simulator (macOS), Android emulator/devices, all through a single CLI command surface
- • AI-native design: alias-cached taps (@N), structured --json envelopes with actionable hint fields, and bundled agent skills for Claude/Cursor/Codex
- • Multiple selector styles (@N alias, #id, --label, raw coordinates) for different stability and speed requirements
- • Background daemon amortizes initialization cost — subsequent queries are sub-100ms
- • Apache 2.0 license with active development (573 stars, 33 forks in 10 days)
- • Fully open accessibility tree traversal — includes WebViews, system overlays, and embedded content, nothing silently skipped
⚠️ Cons
- • macOS-only for iOS Simulator support — no Linux iOS testing capability
- • Requires Xcode CLI tools (iOS) and Android SDK (Android) — not a zero-dependency install
- • Still early stage (released June 26, 2026) — limited community plugins, smaller ecosystem
- • No recording/replay capability yet — each interaction must happen in real-time through the agent loop
- • Documentation covers the basics well but advanced use cases (multi-device, CI integration) need more examples
Mobile app developers and QA engineers who want AI agents to autonomously test, verify, and iterate on iOS and Android UIs
Free (open-source / Apache 2.0)
What Is Sim-Use?
Sim-Use is a cross-platform CLI that bridges the last gap in agentic mobile development: giving AI agents the ability to see and interact with mobile screens. Released on June 26, 2026, by Japanese developer collective lycorp-jp, it has quickly gained 573 GitHub stars by solving a problem that most agentic frameworks ignore entirely.
Until now, AI coding agents could write mobile app code, but they couldn’t verify what they built. Did the button render correctly? Did the navigation flow work? Did the layout break on a specific device? These questions required human eyes. Sim-Use closes that loop by feeding mobile screen state back into the AI agent’s observation-action cycle.
How It Works
Sim-Use operates on a simple but powerful observe → act → verify loop:
sim-use ui # 1. Read the screen as a token-efficient outline
sim-use tap @9 # 2. Tap element @9 (alias-cached from the last ui call)
sim-use ui # 3. Verify the resultScreen Representation
The key innovation is the token-efficient outline format. Instead of dumping the full XML accessibility tree (which can be 50KB+ for a complex screen), Sim-Use produces a structured summary:
App: Settings 402x874
[Top y<120]
@1 StaticText "Settings"
[Content y=120..754]
@5 SearchField "Search"
@7 Button "Sign in to your iPhone"
@9 Button "General"
@10 Button "Display & Brightness"
...
[Bottom y>754]
@43 TabBarThis is roughly 16x more compact than the raw accessibility tree — an LLM can parse and reason about it in a few hundred tokens, making real-time agentic UI interaction feasible without blowing through context windows.
Selector Options
| Selector | Example | Best For |
|---|---|---|
@N alias | tap @9 | Speed — cached from last ui |
#<id> | tap #settingsButton | Stability — survives layout changes |
--label | tap --label "General" | Scripted flows with --wait-timeout |
-x -y | tap -x 100 -y 200 | Last resort — no AX data |
Architecture
Sim-Use uses a per-device background daemon that maintains the accessibility service connection. The daemon amortizes initialization cost — the first ui call takes ~1-2 seconds to spin up the connection, but subsequent calls return in under 100ms. This makes full agentic loops (observe → act → verify → code → observe) practical in real time.
The iOS implementation hooks into Apple’s Accessibility APIs and the iOS Simulator HID pipeline directly. The Android implementation uses Android’s AccessibilityService. Both provide the same command surface, so your agent instructions work across platforms without modification.
Community Reception
The mobile development community has responded enthusiastically. Since its launch 10 days ago, Sim-Use has drawn attention from both mobile developers and AI agent framework maintainers. The project’s Apache 2.0 license and well-documented agent skill (sim-use init --client claude) make it easy to integrate into existing workflows.
Developers on HN and Reddit have highlighted the token efficiency as the killer feature — previous attempts at mobile UI agent integration were impractical because accessibility trees were too verbose for LLM consumption. The outline representation changes that calculus entirely.
Use Cases
Autonomous Mobile UI Testing
The primary use case: after your AI agent writes mobile code, it can open the simulator, launch the app, navigate the UI, and verify that everything works — all without human intervention.
Visual Regression Prevention
Combined with code generation, Sim-Use can test that a new feature doesn’t break existing UI elements by capturing the screen state before and after code changes.
Multi-Device Verification
Test the same app flow on iOS Simulator and Android emulator in a single automated pipeline, catching platform-specific rendering differences.
Who Should Use It
- Mobile app developers using AI coding agents who want end-to-end agentic development without manual UI verification
- QA engineers building automated mobile test suites with AI assistance
- Mobile CI/CD pipelines that need automated visual and functional testing
- AI agent framework developers who want to add mobile interaction capabilities
Pricing
Free and open-source under the Apache 2.0 license. Install via Homebrew (brew install lycorp-jp/tap/sim-use), download prebuilt binaries from GitHub releases, or build from source. No subscriptions, no API keys required.
Verdict
Sim-Use fills a genuine gap in the AI agent ecosystem. While frameworks like Claude Code and Codex excel at writing mobile code, they’ve been blind to what actually renders on screen. Sim-Use gives them eyes and hands. The token-efficient outline design is elegant engineering, and the cross-platform support (iOS + Android) makes it broadly applicable.
The project is early — the ecosystem is small, advanced features like recording/replay aren’t here yet, and the macOS-only iOS Simulator dependency limits CI flexibility. But the core functionality is already production-useful, and the architecture is solid.
Rating: 8.3/10 — for mobile teams using AI agents, this is the missing link.