Commit Graph
4 Commits
Author SHA1 Message Date
pluginslabandClaude Opus 4.6 a7dd34fb56 docs: add Automation section with vphone-mcp reference
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-04-04 18:54:10 +03:00
pluginslabandClaude Opus 4.6 2c9238524e hostctl: return compact grayscale screenshot with every action
Every command response now includes an "image" field with a base64-
encoded grayscale JPEG of the current screen (~40-60KB vs multi-MB PNG).
This eliminates the need for a separate screenshot call after each action.

The image is:
- Downscaled to 1/3 resolution (430x932)
- Converted to grayscale for high contrast
- JPEG compressed at quality 0.35

Optional parameters on any command:
- "screen":false  → skip the screenshot capture
- "delay":800     → ms to wait before capture (default 500)

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-04-04 18:52:47 +03:00
pluginslabandClaude Opus 4.6 1f6a64aae4 hostctl: add type command (clipboard_set + paste)
Sets the guest clipboard via the vsock control channel, enabling
text input from external automation. Callers can then tap Paste in
the iOS context menu to insert the text.

  echo '{"t":"type","text":"Hello"}' | nc -U vm/vphone.sock

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-04-04 18:52:47 +03:00
pluginslabandClaude Opus 4.6 dfdec8c393 feat: add host-side automation socket for programmatic VM control
Add a Unix domain socket server (vm/vphone.sock) that accepts JSON
commands from external processes, enabling programmatic E2E testing
of iOS apps running in the VM.

Supported commands:
- screenshot: capture VM display to file (PNG/JPEG by extension)
- tap: inject touch at pixel coordinates (matching screenshot dims)
- swipe: inject swipe gesture between two points
- key: send hardware keys (home/power/volup/voldown) via HID

The socket uses a simple one-line JSON protocol: connect, send request,
receive response, disconnect.  Example usage from CLI:

  echo '{"t":"screenshot","path":"/tmp/s.png"}' | nc -U vm/vphone.sock
  echo '{"t":"tap","x":500,"y":1900}' | nc -U vm/vphone.sock
  echo '{"t":"key","name":"home"}' | nc -U vm/vphone.sock

New files:
- VPhoneHostControl.swift: socket server, command dispatch

Modified:
- VPhoneScreenRecorder: add saveScreenshot(view:to:) with PNG support
- VPhoneVirtualMachineView: add injectTap/injectSwipe via synthetic
  NSEvents routed through the existing mouse event handlers
- VPhoneAppDelegate: wire up VPhoneHostControl lifecycle

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-04-04 18:52:47 +03:00