Files
vphone-cli/research
zqxwceandClaude Opus 4.7 ba70b6f523 camera: libcamfix substrate plugin + Camera.app photo delivery via CAMCaptureEngine
libcamfix.dylib is loaded into Camera.app (com.apple.camera) by
TweakLoader (the .app/-rule covers Camera.app automatically; the plist
filters to Bundles=[com.apple.camera]). It bridges the vphone shm
frames from libvcamcaptured into Camera.app's normal photo + preview
pipeline so the user can take real photos via the standard shutter.

Hooks (only fire for connections backed by AVCaptureDevice uid
"vphone:vcam:0"):

  - _setActiveFormat:    substitute device.formats.firstObject when
                         the session-preset->format lookup hands us a
                         nil format (would otherwise throw at launch).
  - capturePhoto         deliver a CMSampleBuffer (built from shm) via
                         the deprecated didFinishProcessingPhotoSample
                         delegate path — kept for test harnesses that
                         use the documented AVCapturePhotoOutput API.
  - beginMomentCapture / commitMomentCaptureToPhotoWithUniqueID:
                         Camera.app's actual shutter path. Skip orig
                         (would throw), stash the delegate at begin,
                         drive a synthesized AVCapturePhoto at commit.
  - cancelMomentCaptureWithUniqueID:    no-op (orig would throw).
  - AVCaptureSession _setRunning: / _setInterrupted: setters swallowed
                         for vcam-bound sessions, and isRunning /
                         isInterrupted getters force YES / NO so
                         Camera.app's "preview live" poll keeps the
                         viewfinder visible past ~4-5 s.
  - AVCaptureVideoPreviewLayer:  scan UIApplication.windows at 1 Hz
                         for layers bound to a vcam session and pump
                         CGImage frames into layer.contents at 30 Hz.
  - AVCapturePhoto fileDataRepresentation / CGImageRepresentation:
                         when the photo we synthesized is the receiver,
                         return the JPEG / CGImage we built from shm
                         instead of asking the (non-existent) photo
                         surface to encode itself.
  - CAMStillImageCaptureRequest: dynamically add three stubs
                         (resolvedSettings, unresolvedSettings,
                         lensStabilizationSupported) so AVCapturePhoto's
                         private 27-arg init does not throw on the
                         CAM-internal request we pass in.

Synthesized AVCapturePhoto construction:
  - extract the real CAMStillImageCaptureRequest for the current uid
    from CAMCaptureEngine._resultsQueueRegisteredStillImageRequests
    (Camera.app's pending-photo dict),
  - hand-build a minimal AVCaptureResolvedPhotoSettings via
    class_createInstance + ivar writes for uniqueID + dimensions +
    empty NSArray ivars (CFRetained so the dealloc chain stays valid),
  - feed both into AVCapturePhoto's documented 27-arg
    initWithTimestamp:photoSurface:... via NSInvocation,
  - tag the photo with the JPEG bytes via objc_setAssociatedObject
    so the fileDataRepresentation hook returns them.

Full AVF + CAM internal delegate sequence fired at commit time:
willBeginCaptureBeforeResolvingSettingsForUniqueID,
willBeginCaptureForResolvedSettings, willCapturePhotoForResolvedSettings,
didCapturePhotoForResolvedSettings, didFinishProcessingPhoto:error:,
didFinishCaptureForResolvedSettings:error:,
_didFinishStillImageCaptureForUniqueID:error:, and crucially
captureOutput:readyForResponsiveRequestAfterResolvedSettings:.
Without that last "responsive ready" signal AVF's 2-deep pipeline
never frees its slots and Camera.app's shutter stops accepting
input after the 2nd capture.

Install wiring in scripts/cfw_install_exp.sh:
  - build_libcamfix() — clang -arch arm64e -fobjc-arc -Os, frameworks
    AVFoundation / CoreImage / CoreMedia / CoreVideo / Foundation /
    ImageIO / IOSurface / MobileCoreServices / Photos / QuartzCore /
    UIKit, ldid-signed.
  - [JB-4.2] scp the dylib + plist into procursus/Library/MobileSubstrate/
    DynamicLibraries (same location as libvcamcaptured) and chmod /
    chown so TweakLoader picks them up on next boot.

End state: Camera.app on EXP shows live preview from the host-supplied
vcam frames, the shutter takes real photos that get saved into Photos
via Camera.app's own pipeline (no PHPhotoLibrary back-channel), and
the shutter button keeps working across many consecutive captures.

Co-Authored-By: Claude Opus 4.7 <[email protected]>
2026-06-20 01:12:16 +03:00
..