Computer Use CLI

Automate and test Linux desktop apps (Electron, Tauri, React Native Desktop) in an X11 session using CLI commands for screenshot + mouse + keyboard. Includes coordinate scaling so you can click/type based on a smaller “API-sized” screenshot.

Install

skills CLI (Claude Code, Cursor, Codex and more)
npx skills add TwillAI/skills --skill computer-use-cli
Manual install
git clone --depth 1 https://github.com/TwillAI/skills.git /tmp/skills
mkdir -p ~/.claude/skills
cp -r /tmp/skills/computer-use-cli ~/.claude/skills/computer-use-cli

Personal skills live in ~/.claude/skills. Use .claude/skills in a project folder to share a skill with your team.

From the SKILL.md

# Computer Use (CLI)

Use this skill to **drive and validate a Linux GUI** from the command line:

- **Take screenshots** to observe the current UI state
- **Click / type / scroll / drag** to exercise flows
- **Smoke-test desktop apps** (Electron / Tauri / React Native Desktop) inside a sandboxed X11 session

It’s most useful when you need GUI automation but only have shell access (no native UI automation framework available).

## Quick start

1. Ensure dependencies are installed (see “Dependencies”).
2. Point the commands at the active display (usually already set):
   - `export DISPLAY=:1` (or whatever your container uses)
3. Take a screenshot to establish the agent’s visual state:
   - `cu-screenshot --base64`
4. Perform actions in **API coordinate space** (the resized screenshot), then screenshot again:
   - `cu-click --x 512 --y 384 --base64`
   - `cu-type --text "hello" --base64`

Tip: start the app under the same display, for example:

```bash
(DISPLAY=:1 ./my-electron-app &) && sleep 2
cu-screenshot --base64
```

## Coordinate spaces (critical)

These commands support two coordinate spaces:

- **API space (default)**: coordinates refer to the **resized** screenshot (e.g. XGA 1024×768). This keeps vision tokens lower and matches “computer use” style interactions.
- **Screen space**: coordinates refer to the **real** display resolution.

By default, if scaling is enabled, inputs are treated as **API space** and are scaled up before calling `xdotool`.

Use:
Read the full skill on GitHub

Details

Skill name
computer-use-cli
Repository
TwillAI/skills
Path
computer-use-cli/SKILL.md
Repo updated
26 September 2026

FAQ

What does the Computer Use CLI skill do?

Automate and test Linux desktop apps (Electron, Tauri, React Native Desktop) in an X11 session using CLI commands for screenshot + mouse + keyboard. Includes coordinate scaling so you can click/type based on a smaller “API-sized” screenshot.

How do I install the Computer Use CLI skill?

skills CLI (Claude Code, Cursor, Codex and more): npx skills add TwillAI/skills --skill computer-use-cli. Skills go in ~/.claude/skills (personal) or .claude/skills (project), and Claude loads them automatically when a task matches the description.

Is Computer Use CLI popular?

It lives in TwillAI/skills, which has 9 GitHub stars and 0 forks.

Related skills

Twill Cloud Coding Agent

9

TwillAI/skills

Use Twill Cloud Coding Agent to manage Twill's public v1 API workflows. Create/list/update tasks, stream and cancel jobs, manage scheduled tasks, list repositories, and export Claude teleport sessions.

AI, LLMs & Agents

Video Recording

9

TwillAI/skills

Record high-quality browser verification videos using WebReel. Two-phase workflow: test with agent-browser to collect CSS selectors, then record with WebReel for polished output.

Images, Video & Audio

Webreel

9

TwillAI/skills

WebReel CLI reference — config format, step types, human-like recording patterns, and troubleshooting. webreel is pre-installed globally in the sandbox.

Images, Video & Audio

Context Engineering

9

claudekit/skillmark

Check context usage limits, monitor time remaining, optimize token consumption, debug context failures. Use when asking about context percentage, rate limits, usage warnings, context optimization, agent architectures, memory systems.

Software Development

Codebase Design

9

eralmansouri/skills

Use for code-level module design, deep modules, interfaces, seams, adapters, testability, refactoring shallow modules, and finding architecture improvements in an existing codebase.

Software Development

Coti Transaction Tools

9

coti-io/coti-skills

Monitors transactions, decodes events, manages native COTI tokens, and signs messages. Use when user asks to 'check transaction', 'transaction status', 'decode event', 'native balance', 'transfer COTI', 'send COTI', 'sign message', 'verify signature', or 'transaction logs'. Utility toolkit for on-chain debugging and native token operations.

Software Development

Browse the most popular Claude skills, all software development skills, or write your own with the skills generator.