跳转至

Annotate (macOS, Linux, and Windows)

Annotate is Scrawl's capture engine hosted inside Hex (code and settings keep the name capture), steps 3 to 5 of the plan. It runs the real scrawl-capture crate, imported into crates/capture: it records the active display with ScreenCaptureKit, on Linux the screen-cast portal and PipeWire (capture.linux), on Windows Windows.Graphics.Capture (capture.windows), (or a cabled iPhone through scrawl-helper's mirror) and narration from Hex's microphone, takes ink and clicks (Shift on macOS and Windows, or the Annotate key held or the pen), transcribes the narration with word timestamps on Hex's loaded model, writes a Scrawl bundle into Scrawl's captures folder, copies it to the clipboard, and sends it to an OpenCode session with Scrawl's router (crates/router), run in Hex's process. Settings › Annotate (under Labs in the sidebar) holds its settings. Scrawl's capsule (switch chips, the on-screen picker, route chips) and its ink and laser overlay are Rune overlays.

Turn it on                               // capture.gate
├── On by default in every macOS, Linux, and Windows build, stable included
│   (capture.enabled); the `annotate-on-by-default` migration turns it on
│   once for older settings
│   └── macOS bundles sign scrawl-helper beside `hex`; validate-app.sh requires it
├── Nothing is asked at launch: Screen Recording (macOS) at the first capture,
│   the portal's dialog (Linux) at the first Screen capture; Windows asks
│   nothing
├── A start without a permission it needs (macOS: Screen Recording unless
│   iPhone, then Microphone, then Accessibility; capture_host::permissions)
│   -> refused with code `permission`, no error tone, and the capsule's
│   permission helper opens (capture.permission)
└── Off -> no capture host, engine, sink, thread, or folder; `hex annotate` -> feature-off

Draw                                     // capture.draw (capture_host::drawing)
├── One key: tap (under DRAW_HOLD, 250 ms) to start and again to stop; held
│   while it records, the pointer draws with no click until it comes up
│   (macOS: CaptureShortcut tells the tap from the hold at release; Linux:
│   the portal's Activated/Deactivated or the X11 grab's press and release,
│   linux/input Capture/CaptureUp; Windows: the keyboard hook's
│   HotkeyAction::Capture/CaptureUp -> CaptureHost::annotate_down/up)
├── The capsule's pen -> on until clicked again, or the key is tapped, or
│   held and let go (Action::Pen -> capture_host::toggle_pen). Like a real
│   pen it inks only between mouse-down and mouse-up (capture_host::
│   pen_button; Linux: the overlay's drag, macOS: the tap's buttons); moving
│   with the button up draws nothing, and clicks still never reach the app
│   └── Releasing the button ends the stroke on the `capture-input` thread;
│       the pointer it reads (capture_host::ink_source) has its own lock, so
│       it never re-takes the input target's lock that thread holds (#572:
│       Windows hung on the first stroke). Check: capture_host
│       ending_a_stroke_while_the_input_target_is_locked_does_not_deadlock;
│       not observed on a Windows PC
├── Esc while drawing -> stops drawing, never the recording (recognition.rs
│   on macOS; the Escape grab on X11 (Wayland binds no Escape); the keyboard
│   hook on Windows, when no dictation records; and the overlay's own key
│   handler where it gets the keyboard)
├── macOS and Windows -> Shift held alone still draws, as in Scrawl
├── While drawing -> the ink overlay takes the pointer with a pen cursor, so
│   clicks never reach the app; the capsule stays clickable above it (macOS:
│   its level; Linux: the overlay leaves a hole over the capsule's pill).
│   Every move draws ink; there are no pins and no undo
├── `hex annotate draw` (Linux) -> the pen on or off, for desktops without
│   the portal (it starts a recording drawing)
├── Default key: ⌥⇧S on macOS; F9 on Linux and Windows, a single key one hand holds
│   while the other moves the mouse (F10 is GTK's menu key, F11 full screen,
│   F12 often a dropdown terminal). Saved bindings stay.
│   Check: capture_host the_shortcut_records_once_its_keys_come_up; none
│   physical for the hold on a portal desktop ("Annotate pen" is logged)
├── Ink holds 650 ms, then fades over 650 ms on screen; every stroke stays in
│   the bundle (keyframes, contact sheet, scrawl.json)
├── Linux: strokes are placed on the recorded screen in its logical points,
│   from the ScreenCast portal's stream size when it reports one smaller
│   than the frames, so a fractional scale (1.5, 1.33), which reaches the
│   overlay rounded up to 2, no longer pins strokes to a top-left part of
│   the screen; without that size it divides the pixels by the overlay's
│   scale. Check: scrawl-capture capture::platform::tests
│   a_fractionally_scaled_screen_keeps_its_logical_size; not observed on a
│   fractionally scaled monitor. Known gap: the overlay starts below a
│   desktop bar's reserved area, so saved ink sits that bar's height off
│   the live ink
└── Linux recordings show the overlay itself (Wayland can't leave it out)

On Linux                                 // capture.linux
├── X11 session (DISPLAY set, WAYLAND_DISPLAY empty; never XWayland) ->
│   Screen and Area are read from the X server itself (scrawl_capture::
│   linux::x11): MIT-SHM GetImage each frame borrowed in place (plain GetImage
│   without SHM), the pointer cached from XFixes on XFixesCursorNotify and
│   placed each frame from QueryPointer, 60 fps to x264 (`superfast` up to
│   1440p, `ultrafast` above, wall-clock stamped into `fps=60`) through
│   ffmpeg. No
│   portal and no dialog, so i3 and other desktops without a ScreenCast
│   backend record (#690, @nefthy). Screen is the RandR monitor under the
│   pointer; Area is `hex annotate start --area X,Y,W,H` (pixels), clipped to
│   the screen. Hex's own overlays show in it. A Window still goes through
│   the portal's picker and, without one, fails saying to record the screen
│   or an area. Can't open the display, an unusual pixel depth, or no
│   ffmpeg/x264 -> the capture fails at once with why in the capsule, never
│   a capture without a picture. Check: x11 areas_are_clipped…,
│   cursor_blends_premultiplied_argb_in_place; on hex-linux-nvidia (T4, Ubuntu
│   24.04) native Xorg + i3 recorded 1920x1080 at 60/1 fps (598 frames in
│   9.97 s, 597 grabs) and a 1280x720 Area (298 frames in 4.97 s, 297 grabs),
│   and rootful Xwayland at 3840x2160 recorded 2560x1440 at 60/1 fps (598
│   frames in 9.97 s from 282 raw 4K grabs; raw 4K pipe readback is CPU-bound)
├── Wayland -> the ScreenCast portal (zbus): Screen asks the desktop once and
│   keeps its restore token (<support>/screencast-token), so later captures
│   start without a dialog; Window shows the desktop's picker every time (the
│   capsule's Window chip goes straight to it); the cursor is embedded
├── Frames -> the portal's PipeWire stream, libpipewire-0.3 loaded at run
│   time (`maxFramerate` 60/1); 32-bit RGB as a linear DMA-BUF (niri, GNOME,
│   KDE) or shared memory
├── Encoding -> VA-API (libva loaded at run time): each DMA-BUF imported, scaled
│   to a 2560 px long edge (1920 for a window, exact even sides padded to 16 px
│   macroblocks and cropped in SPS) and converted on the video engine, H.264
│   High at a constant QP and computed level (4.2 at 1080p60, 5.1 at 1440p60,
│   5.2 at 4K60) with Hex-written headers, 60 fps; `ffmpeg -c copy` muxes
│   recording.mp4. Every /dev/dri render node is
│   tried in order, so a hybrid laptop whose first node is a GPU the VA
│   driver can't use encodes on the next (#853). No VA-API -> frames copied
│   and encoded at 60 fps with x264 (`superfast` up to 1440p, `ultrafast`
│   above) through ffmpeg (slow on DMA-BUF-only compositors; on NVIDIA GBM +
│   xdg-desktop-portal-wlr without DONT_FIXATE, linear DMA-BUF allocation is
│   refused by the compositor and delivers no frames);
│   VA-API refusing the first frame's buffer (#1179: Intel Arc on the xe
│   driver) -> the same CPU path instead of failing. The negotiated DRM
│   modifier goes to the import and the log; the CPU never reads a tiled
│   buffer as rows. A first frame neither path can read (a DMA-BUF the CPU
│   can't map, or tiled) fails the capture with why, never a capture without
│   a picture or a crash. Buffers PipeWire removes mid-stream (a
│   renegotiation) are never queued again, and Hex maps buffers itself and
│   unmaps a removed one only after the recorder's copy, so a renegotiation
│   can't crash Hex. Check: vaapi render_nodes_in_order,
│   level_and_sps_crop_match_resolution_and_60fps, pipewire
│   offered_formats_parse_back, a_negotiated_tiled_modifier_is_read,
│   capture::platform output_sizes_stay_exact_even_dimensions… and
│   cpu_x264_uses_superfast…; PipeWire 1920x1080 SHM -> x264 recorded 60/1 fps
│   (598 frames in 9.97 s, 594 delivered) on hex-linux-nvidia; VA-API 60 fps
│   encoding not observed on Intel/AMD hardware (T4 has no VA-API encoder), nor
│   on a hybrid laptop or an Intel Arc laptop
├── Candidates -> a small copy from the video engine on demand every
│   FRAME_SAMPLE_INTERVAL (at most 4× smaller per pass: radeonsi crashes past
│   it)
├── Reading movies back (keyframes, probe, audio) -> `ffmpeg`/`ffprobe`
├── Missing PipeWire, ffmpeg, or a picker answer -> that capture fails with why
│   └── The screen-share picker cancelled (or the stream never starting) after
│       the capture started -> its collector notices within 100 ms, cancels
│       it, and settles `failed`; the listener's next reap ends it, so the
│       shortcut and dictation work again (#681). Check: capture_host
│       a_screen_source_that_gives_up_ends_the_capture; not observed on GNOME
├── Narration -> the listener's microphone through ListenerTimeline (each
│   stream anchored to the monotonic clock); released-while-idle microphones
│   open for the capture
├── Words -> one foreground job on the listener's engine (its loaded model)
├── Copy -> the summary text through Hex's Linux clipboard (wl-copy or the
│   X11 clipboard), as the Mac copies it
├── Paste -> as the Mac: the contact sheet (image/png), then the summary,
│   each with Hex's own Ctrl+V (linux/paste). An app that takes no image (a
│   terminal) gets only the summary, which names the contact sheet; the Mac
│   pastes a terminal the sheet's path instead
├── Finished capsule -> "Copied" with Paste (accent), Inspect, and Send to…;
│   unlike the Mac, a guess doesn't open its suggestions by itself
│   (Flow::suggesting_on_route(false)): Send to… is a click away
├── Shortcuts -> F9 (tap to start and stop, hold to draw) through the
│   GlobalShortcuts portal ("Annotate") or X11 grabs. There is no separate
│   draw shortcut and no send-to list shortcut on any OS
├── While it records, dictation starts nothing (the portal
│   or grab already took their keys, so nothing reaches the app)
├── Pane -> no Screen Recording row, no transcriber choice, no "Show in screen
│   recordings" (the portal asks; Wayland can't keep overlays out)
├── Sending -> OpenCode's sessions and prompts over HTTP to the running
│   service its registration names (service.json); nothing is started, and
│   with no service the send says "OpenCode isn't running". The capsule's
│   rows are clicked (no keyboard).
│   See capture.send: everything goes inline, so a remote server works
└── Gaps: no Area on Wayland (it gives Hex no pointer to aim it; X11 takes
    `--area`), no area picker on screen, no iPhone, no
    laser or click log outside draw mode, no window log, `hex capture status`
    unsupported, choosing another folder only through settings.json

On Windows                               // capture.windows
├── Source -> the engine's Windows.Graphics.Capture recorder (win/media.rs):
│   Screen, Window (the picker lists windows), and Area; no permission asked
├── Units -> the engine's targets, windows, and pointer are physical pixels;
│   the picker and ink overlays sit on the primary display and lay out in its
│   points, converted with capture_host::pixels_per_point (mixed-DPI displays:
│   the primary's scale applies everywhere)
├── Narration -> the listener's microphone through ListenerTimeline, fed by
│   windows/dictation; released-while-idle microphones open for the capture
├── Words -> one foreground job on the listener's engine (its loaded model)
├── Input -> windows/input's low-level hooks: the Annotate key (F9; tap to
│   start and stop, hold to draw), Escape while drawing, and while a capture
│   records every pointer move, click, wheel turn, and modifier change
│   (capture_host::report_input, as the macOS tap): Shift alone inks, the
│   laser follows the pointer, clicks are logged. Injected events are skipped
├── While it records, dictation starts nothing
├── While dictating -> General's setting through windows/audio_control, one
│   suppressor per capture, restored when it ends
├── Copy and paste -> the engine's own clipboard and Ctrl+V (handoff): the
│   contact sheet, then the summary
├── Pane -> no Screen Recording row, no transcriber choice
└── Gaps: no `hex annotate` (Windows has no control transport yet), no
    iPhone, keys pressed in the app are not logged. Checked only by
    cross-compiling (`cargo xwin clippy`, x86_64 and aarch64); never run on
    a Windows machine

Settings › Annotate                      // capture.pane
├── Capture with Hex (capture.enabled, default off) -> off: no host, shortcut,
│   input, retention, or folder
│   └── On while the shortcut collides -> refused, the shortcut row says why
├── Shortcut (capture.hotkey, default ⌥⇧S, F9 on Linux and Windows) -> same keycap and recorder as dictation;
│   a key is required; refused when it overlaps dictation or paste
├── How it works (on) -> start (tap), draw (hold, no click; Shift too on
│   macOS), the capsule's pen with Esc, stop (tap again), and send, each
│   with its keys. Check: rune_ui::scrawl tests
├── Screen Recording -> read once a second and again when the pane opens or
│   the window becomes key, never with a prompt (capture_host::screen_access):
│   the preflight, which keeps its first answer for the life of the process,
│   then, while it says no, a fresh `scrawl-helper permissions` at most every
│   2 s on its own thread and a one-frame probe once that says granted
│   ├── granted -> Allowed, no button
│   ├── granted in System Settings but unusable until Hex relaunches ->
│   │   "Turns on when Hex restarts" and Restart Hex, never Allow over a switch
│   │   already on
│   ├── Returned from System Settings still untrusted -> "Still grey? Fix it"
│   │   (`ScreenCapture` reset on running bundle + retired IDs, re-prompt + guide)
│   └── Allow -> clears retired Hex IDs, rechecks, then opens the permission
│       guide over System Settings' list (PermissionKind::ScreenRecording); flips
│       on the next read. Never part of onboarding (macOS also asks at the first
│       capture). Check: rune_ui::scrawl tests
├── Show in screen recordings (capture.show_overlays_in_recordings, default
│   off) -> see capture.own-windows; applies at once, to open windows and a recording capture
├── Transcriber (capture.transcriber) -> Parakeet (default) | Apple
├── Captures folder (capture.folder, default Scrawl's) -> Reveal | choose another
├── Keep (capture.media_days / bundle_days) -> Scrawl's presets, or imported days
├── Scrawl.app running -> "Scrawl is running" card + Use Hex for captures
│   └── Click -> asks Scrawl to quit (nothing else quits it); Hex takes the
│       shortcut within 2 s of Scrawl leaving
└── No iPhone pairing row: ScrawlKit pairing needs the ingest server, which Hex
    doesn't run. A cabled iPhone needs no row (capture.phone)

First launch with Annotate on            // capture.import
├── ~/.config/scrawl/config.json -> transcriber (apple; else Parakeet), capturesDir,
│   retention, read once (migration `scrawl-capture-settings`); token not used
├── Scrawl's router settings -> capture.routing, read once (migration
│   `scrawl-routing-settings`): rules, autoSend, smartRouting and jevRouting;
│   never the Jev key (Jev uses $OPENCODE_API_KEY or OpenCode's credential)
├── Scrawl's Application Support config.json -> its shortcut, unless it collides
└── Scrawl's files are never written

⌥⇧S (capture.shortcut; captures on, Scrawl not running)
├── Press, held as long as you like -> armed; nothing records while ⌥⇧ is down
│   └── One of its modifiers comes up -> start on the display under the pointer,
│       as `hex capture start`                // a fresh press only, no repeats
├── Esc before the modifiers come up -> nothing records
├── Press while recording -> stop, as `hex capture stop` // ignored within 400 ms
├── Press while the capsule's picker is open -> the picker records what it is on
├── Esc while a capture records -> nothing: it reaches the app being recorded,
│   and Hex's tap lets it through; a dictation under it goes on too (#786)
├── The binding is swallowed, never chord evidence for dictation
└── Hold the dictation shortcut mid-capture -> dictation pastes as usual

While a capture records                  // capture.quiet
├── Start -> a recording-environment session with General's While dictating
│   (read at each start; captures have no setting of their own): mutes output
│   or pauses Music, Spotify, VLC; Do nothing changes nothing. No idle-sleep
│   assertion, and it never counts as an intentional dictation
├── Stop, cancel, discard, failed start, captures off, quit -> released; a
│   switch carries it to the next capture without restoring in between
├── Dictation shortcut -> starts nothing while it records
│   (macOS: recognition.rs swallows it in the tap; Linux: linux/dictation
│   ignores Start). Nothing leaks to the app in front
└── Dictation that began before the capture -> each effect is held from the first session that
    wants it to the last, so neither unmutes or resumes what the other still
    holds; a While dictating change mid-capture composes too (the capture keeps
    media paused, the dictation mutes). Only what Hex
    changed is restored. The dictation stop tone plays at once while the capture
    keeps output muted. Check: recording_environment tests (overlap both ways),
    capture_host a_capture_stays_quiet_until_it_stops_is_cancelled_or_hex_quits

Switching target                         // capture.switch; the capsule's chips
├── The recording capsule shows no target chips: a recording is the screen.
│   Switching (below) remains for the CLI, the picker, and the playground
├── CaptureCommand::Switch(target) -> discard and start in one step: no snapshot
│   shows nothing recording in between (Scrawl's switch_recording)
│   ├── No room to start (two writing, Hex stopping) -> refused `busy` before
│   │   the discard; the recording goes on
│   └── The start fails after the discard -> the discarded capture settles
│       `Failed` with the start's error
├── Window or Area -> CaptureCommand::Discard: bundle removed, no outcome, gone
│   from the snapshot at once (and from MAX_FINISHING), so the capsule becomes
│   the picker
├── Picker open (take_keys: Picker) -> Hex's tap hands it ⌥⇧S and Esc; it never
│   takes focus
├── A pick -> CaptureCommand::Start(Window | Area | Display); nothing while one
│   records
├── The picked window closed, or Screen Recording refused -> last outcome
│   `failed` with its reason (`error`)
└── iPhone -> CaptureCommand::Switch(Phone) or Start(Phone), while the snapshot
    names a cabled phone (capture.phone)

A cabled iPhone                          // capture.phone
├── Each capture start, and CaptureCommand::FindPhone (the picker opening) ->
│   a capture-phone thread runs `scrawl-helper devices` (the USB registry, no
│   permission) -> CaptureSnapshot.phone = its name, or None
├── Start(Phone) -> scrawl-helper's mirror window shows the phone and records
│   its native video into recording.mov; `--no-microphone`: the mirror never
│   asks for or opens a microphone
│   ├── Camera access -> macOS asks on Hex's behalf (every bundle carries
│   │   NSCameraUsageDescription; validate-app.sh requires it)
│   └── Pointer and pen input -> normalized to the mirror's picture
├── Narration -> Hex's timeline, as for the screen, moved onto the movie's clock
│   (its first frame, measured from the capture's start) and transcribed by Hex's
│   transcriber (capture.transcriber), not Scrawl's
├── Stop -> as any capture; keyframes decoded from the phone's own video
└── The mirror ends on its own (unplugged, no phone in 12 s, camera denied) ->
    the listener finishes the capture: written if it recorded, else `failed`
    with the helper's reason

The capsule                              // capture.capsule (rune_ui/capture_capsule)
├── A Rune overlay at the top of the pointer's display that never takes focus
│   and lets clicks through except on its live chips, buttons, and list; a
│   new recording brings it to the pointer's display even while it's up on
│   another;
│   its window id goes to the engine (`Host::own_windows`, one id per role, so a
│   recreated window replaces the old one), so recordings leave it out
├── Follows `capture_host::watch()`: Recording from the instant `recording` is
│   Some (the bundle's t0; timer, meter, ✓ finish, ✕ cancel) -> Processing ->
│   done (the lead alone, no word), Failed, or Cancelled, then leaves on its
│   own; a written capture whose transcript has no words reads "no speech"
├── Tones (`capture_capsule::tone`, at the dictation volume; zero is
│   silent), from the chosen sound set: start and stop are the set's start
│   and stop, a send going its done (Annotate's own sent sound when the set
│   has none), a discarded recording its cancel, a failure its error (Basso
│   for Classic on macOS); a taken-back send keeps Annotate's own undo sound
├── Failed -> "Couldn't start" until the engine reports it
│   recording (`screen`), then "Couldn't save"; a failed send ->
│   "Couldn't send to X". Each keeps its `error`; a start or save that fails
│   while Screen Recording is off (a fresh read) opens the permission helper
│   instead; a start that fails before any snapshot showed it
│   recording still shows, but never over a newer recording, and a capture
│   already under way that fails in the background never takes it over
├── Permission helper (capture.permission) -> "Hex needs Screen Recording ·
│   to see your screen · Allow ✕" (Microphone, Accessibility alike)
│   ├── Allow -> macOS's microphone prompt, else System Settings with the
│   │   floating guide beside it (permission_guide), never at launch
│   ├── Followed once a second (feed::watch_grant, screen_access) ->
│   │   "… is on · Restart Hex" when macOS needs one, then "… is on ·
│   │   Capture ⌥⇧S"; Capture or the shortcut records the pointer's display
│   ├── ✕ closes it; unanswered it folds after 2 min; the shortcut brings it
│   │   back as it stood
│   ├── A capture written with no frames (Screen Recording turned off mid-way)
│   │   -> sent with only the voice and marks, its summary noting "No screen:
│   │   Screen Recording is off for Hex", no contact sheet; the helper opens
│   │   once its capsule folds
│   └── Story: hex://story/prototypes/capture-capsule/permission; goldens
│       capture-capsule-permission{,-restart,-ready}
├── A shown recording that leaves with no outcome (a discard, from a chip or
│   `hex capture discard`) -> the capsule leaves; the picker a chip opened
│   stays. A switch that can't start settles the recording it discarded as
│   failed, which the capsule shows as "Couldn't start"
├── Recording -> `● 0:07 │ ✎ │ ■ ✕`: the timer, the toolbar (the pen, its
│   only tool, lit with the accent fill while drawing), a divider, stop
│   (Finish) and ✕ (asks "Discard?" first), all equal square icon buttons
│   with tooltips "Draw", "Stop", "Discard". Goldens: recording,
│   recording-drawing (rendered on apollo)
├── Target chips (playground stories only; feed sends no Input::Chips) ->
│   Area · Window · Screen, plus iPhone while a phone is cabled, for 3.5 s
│   ├── `worked` -> they fold once seen for 0.9 s; never while the pointer is on
│   │   the capsule, and 1.2 s after it leaves
│   ├── A chip restarts rather than retargets (Scrawl's `switch_recording`)
│   ├── Screen -> it's what's recording: the chips fold, no command
│   ├── iPhone -> CaptureCommand::Switch(Phone): the recording restarts on the
│   │   phone
│   └── Window or Area -> Discard; the capsule becomes the picker, and
│       FindPhone looks for a phone cabled since the recording began
├── Picker: ⌥⇧S records what the on-screen picker shows; Esc closes it
├── ✓ -> `hex capture stop`
├── ✕ while recording -> asks first: a destructive "Discard" and "Keep
│   recording", the recording running on; Discard -> `hex capture cancel`.
│   Unanswered, it goes back to recording after 4 s (never while hovered, and
│   1.5 s after the pointer leaves). Esc never discards
├── Routing (Scrawl's routing.md) -> `last.route` once its answer arrives; the
│   capsule follows refinements and delivery separately, including Queued
│   (then Sent), how an Undo ended, and a failed send
│   ├── A saved rule -> "Sending to X" and Undo, whose ring drains through the
│   │   Undo window (5 s) and is gone when Undo is; no words count it down.
│   │   A repository awaiting its session shows the repository name. Undo
│   │   cancels before posting. The capsule remains visible until delivery;
│   │   then it reads "Sent to X" and plays the Sent tone. A picked send still
│   │   goes immediately, with the same ring on its Undo. No runner-up chip for a rule;
│   │   after delivery and the Undo window, the capsule leaves by
│   │   itself (with the list open it stays; the pointer on it holds it, and a
│   │   folded list gives it a brief hold). Picking the rule's own session there (without
│   │   "Always") only closes the list; nothing is sent again
│   ├── The Undo ring -> one per send (rule or pick), tied to the window's end
│   │   (`Done.countdown`) and drawn by `capture_capsule::undo_ring` as render-time
│   │   opacity tweens, so nothing ticks: the capsule wakes only when the window
│   │   ends. Reduce Motion steps it a second at a time. It is the Undo chip's
│   │   own outline, draining clockwise from the top
│   ├── The pointer on the capsule -> holds the countdown, one that starts
│   │   under it too: the ring stands still (Reduce Motion too), Undo never
│   │   runs out, and a rule's send waits (`CaptureCommand::Hold`). Leaving
│   │   goes on with the time it had left, at least 1 s. A pick already went,
│   │   so only its Undo waits (`Flow::hold_countdown`)
├── A finished capture with nowhere sent yet reads "Copied" from the start
│   ├── Anything else with somewhere to go (a guess, nothing fitting, no
│   │   answer) -> macOS: "Copied", then Search with the capture shortcut's
│   │   keycaps (as Settings draws them) and ✕, the suggestions open under
│   │   it; nothing sends until Kit picks. Linux: closed,
│   │   with Paste, Inspect, and Send to…, which opens them on a click
│   │   ├── "Send to" over up to five rows, suggested then the newest
│   │   │   sessions, with no numbers and no footer; "⇧ remembers" on the
│   │   │   header's right when a pick can save a rule. The first
│   │   │   carries its reason in the accent. A reason is a real one in plain
│   │   │   words ("you were just in it", "same page", "mentioned", "recent")
│   │   │   or nothing; never "guess"
│   │   ├── Shift down (a guess or rule with a site or app) -> "⇧ remembers"
│   │   │   lights, and the highlighted row reads "always"; in the search the
│   │   │   footer reads "Always send [site] to [row]", the site and the row's
│   │   │   name each in a rounded-rect badge with its kind's icon (a globe for
│   │   │   a site, a window for an app, a bubble for the session). Shift+click
│   │   │   (or ⇧⏎ in the search)
│   │   │   sends and saves the rule: "Sent to X · always", and its Undo
│   │   │   takes back the send and the rule (what the key held before, or
│   │   │   nothing, comes back). The rule names that session (`ses_…`),
│   │   │   never its folder: a folder rule goes to the folder's newest
│   │   │   session, so with SEER and RUNE DESKTOP in one checkout, a rule
│   │   │   saved for RUNE DESKTOP went to SEER
│   │   ├── A click -> sends at once: "Sent to X", the Sent tone, and a
│   │   │   Undo with its ring for the same 5 s; ✕ closes;
│   │   │   the capsule then leaves by itself with its exit
│   │   ├── Search or the capture shortcut (⌥⇧S) -> the search: "All
│   │   │   sessions" over every session the router's look read (its two
│   │   │   newest pages, every folder's, and any it named), filtered by
│   │   │   what's typed: every word, in any case, in the title, the folder's
│   │   │   name, or the session id, rows with every word in the title first;
│   │   │   a row names its folder before its note when only the folder or id
│   │   │   matched, or when another row shows the same title (suggestions
│   │   │   too). Five rows at most and never shorter than the
│   │   │   suggestions it opened from; the footer reads "Send ⏎",
│   │   │   "Remember ⇧⏎", and "New capture ⌥⇧S". Matching nothing, it reads
│   │   │   "No sessions match" and the footer drops Send and Remember. Esc or a click away goes
│   │   │   back to the suggestions; ⌥⇧S in the search records a new capture
│   │   ├── Once the capsule has gone, ⌥⇧S records as usual
│   │   ├── Esc -> not Hex's here: it goes to the app in front
│   │   └── Unpicked -> they leave with the capsule after the done hold
│   ├── Taken back (a rule's send or a pick) -> never
│   │   ends the capture: the list opens again with the same sessions, the one
│   │   it went to (by session id) once, last of all ("taken back"), with
│   │   clicks, Search, and ⇧-click as before. Open, it stays, sent nowhere; a click away
│   │   folds it to "Taken back" with "Send to…", and it leaves after the done
│   │   hold; nothing here takes Esc. ✕ (or Esc in the typed list) dismisses:
│   │   "Kept", the bundle untouched. Either way it then
│   │   waits for Send to…, even after the router said the pick went. With no
│   │   other session to offer, the list still opens with the one it went to
│   │   ("taken back"), so Search can send it elsewhere; only a rule still
│   │   resolving its folder reads "Taken back" with nothing to list.
│   │   News of the send being taken back (it went, queued, or failed) leaves
│   │   that list alone, and a pick made before the router answers the Undo
│   │   stands: the Undo's answer and the earlier send's news never take it back
│   ├── Too late to take back -> the same list, the session marked "already
│   │   sent", folded "Already sent to X"; a pick sends a copy there,
│   │   and ✕ just closes (it leaves by itself after the brief hold). Closed, it doesn't wait for a pick.
│   │   An Undo OpenCode refuses (an error) reads as too late
│   ├── Sends -> CaptureCommand::Send{to} and Undo, naming the
│   │   session by id (`RouteDest.session`), so sessions sharing a title never
│   │   collide; the capsule shows titles
│   ├── Keys (capture_host::take_keys), never taking focus:
│   │   ├── Picker -> ⌥⇧S and Esc through the listener
│   │   ├── Suggestions -> ⌥⇧S through the listener
│   │   │   (`ShortcutActivity::Capsule`), which opens the search
│   │   ├── Routed or closed -> no keys: Send to… is click-only, and never
│   │   │   Esc, so a send is taken back only by clicking Undo
│   │   ├── A send from the list still undoable -> no keys; Esc goes to the
│   │   │   app, and the Undo button is the only way back
│   │   ├── Suggestions, the pointer moved onto them -> also text typed
│   │   │   without ⌘ or ⌃ (not a leading space), which opens the search
│   │   │   with it (`CapsuleKeys::Suggest { search }`). Only a move counts:
│   │   │   suggestions that open or change under a resting pointer, or a
│   │   │   pointer that left, leave typing with the app in front, so Kit
│   │   │   typing there never loses a letter. ⌫, ↑↓, ⏎, and Esc stay the
│   │   │   app's until the search opens
│   │   │   (`typing_stays_the_apps_until_the_pointer_moves_onto_the_suggestions`)
│   │   ├── Suggestions -> at the session tap no key at all (the toast's
│   │   │   ⏎ and ⇥ aside), only a mouse press, which folds them (the click
│   │   │   still reaches the app) unless the pointer is on the pill or the
│   │   │   list under it: the tap reports the press before the overlay gets
│   │   │   it, so a click on a row would otherwise fold the row away first
│   │   │   (`clicks_on_the_capsule_do_what_they_say`, headless pointer
│   │   │   events, not native); every key, digits and Esc included, goes on
│   │   │   to the app. Shift's presses and releases (flagsChanged) reach the
│   │   │   capsule too (`CapsuleKey::Shift`) and still go on to the app
│   │   └── List open -> the session tap takes every key press without ⌘ or ⌃
│   │       that isn't a Hex shortcut: text in the active layout, ⌫, ↑↓, ⏎,
│   │       ⇧⏎ (always), Esc; their releases too. Digits are typed (session
│   │       names have them), and ⌥⇧S goes on to the listener, which records. A send, Esc, or a
│   │       press off the capsule (the tap reports mouse-downs while the list
│   │       is open, and still delivers them) closes the list and gives the
│   │       keys back; hiding drops the grab
│   ├── `offer` (a rule offer) -> not drawn; Scrawl's capsule doesn't draw it
│   ├── The top line is calm: the label, its note, one action, and ✕. Undo
│   │   while a send can be taken back, else Paste and Inspect as two quiet
│   │   icons (tooltips "Paste the capture", "Open the capture") for a written
│   │   capture, with the click-only Send to… chip beside them while there is
│   │   somewhere to send it; over the suggestions, Search alone (the
│   │   capture is already copied), right beside ✕. Paste and Inspect have no key on either OS.
│   │   The list hangs
│   │   under the capsule, which is never narrower than LIST_W; the capsule
│   │   widens with it. The capsule is a rounded rect with the list's 14 pt
│   │   corners (`SHELL_RADIUS`), not a pill, and its chips and icon buttons
│   │   round concentrically inside it. Two looks while Kit picks (tuning `top_line`):
│   │   TopLine::Quiet (default) says the label in quiet MEDIUM ink;
│   │   TopLine::Icons says it in STRONG. A capture that only copied has no
│   │   label, the lead alone
│   ├── The lead is the capture's first frame (`frames/frame-01.png`, else the
│   │   contact sheet), cropped to a small rounded card once the bundle is
│   │   written and decoded off the UI thread; until then, or without one, a
│   │   check. A warning keeps its glyph
│   │   ├── Paste -> CaptureCommand::Paste: the contact sheet, then the summary,
│   │   │   into the app in front; a send still in its Undo window is taken back
│   │   └── Inspect -> CaptureCommand::Inspect: opens the contact sheet (Hex
│   │       has no review window, and opening the bundle could launch
│   │       Scrawl.app)
│   ├── Undo is a click only, with no shortcut: Escape meant for another app
│   │   was taking sends back
│   ├── Shortcuts read as quiet text after what they do, never a keycap per
│   │   key, except Search's capture shortcut, drawn as Settings' keycaps: a
│   │   hint's keys sit in one small rounded rect, on a quiet fill. The
│   │   search's footer reads "Send ⏎", "Remember ⇧⏎", "New capture ⌥⇧S"; the
│   │   cursor's and the notch's cards end in "Search ⌥⇧S" under their guesses.
│   │   The footer sits apart from the rows in one of
│   │   two looks while Kit picks (tuning `list_footer`): ListFooter::Band
│   │   (default), a full-width band on its own fill under a rule, or
│   │   ListFooter::Well, a rounded well inset like the rows
│   └── Long names give way together so the widest row stays inside the panel
├── Every change moves at once, as Dynamic Island's: the walls and what stays
│   travel on `island`, the dictation HUD's morph (`rune_hud::spring::MORPH`,
│   0.30 s, bounce 0.15; the entry's rise and exit's fall too, with the
│   shell fading over the same time unbounced), while everything leaving and
│   everything arriving blurs (`content_blur` 3 pt), scales toward the
│   capsule's middle (`content_scale` 0.92), and fades over that same move
│   (`content_in` 0.30 s, `content_out` 0.24 s). Nothing staggers, and nothing
│   waits for a wall or for other content; only an entry takes two frames
│   (the origin pose, then rest). The list under the capsule comes and goes
│   on the same springs. The playground's Transition card tunes all of it
├── Reduce Motion -> the capsule enters at rest and leaves where it is (no
│   origin pose, menu-bar fold, or edge glow), changes jump, nothing blurs
│   or scales; the shell, at the notch too, fades on the HUD's short fade
│   (`reduce_motion_only_fades_the_capsule`)
└── After a capture, one of five flows (`capture.capsule_flow`, the
    playground's "Flow after a capture"; prototypes to compare, clips in
    r22-flows). Only the guesses' look changes; rules, Undo, ✕ and the
    keys the capsule takes stay the same. Each capsule story's variants
    pick one, so a link opens a state in a flow:
    `hex://story/prototypes/capture-capsule/suggestions?variant=toast`
    ├── list (default) -> today's list under the capsule, as above
    ├── toast -> one line "Send to X ⏎" with "Next ⇥" for more than one
    │   guess; ⇥/⇧⇥ cycle (CapsuleKeys::Suggest { cycle }), ⏎ or a click on
    │   the line sends the one shown
    ├── menu_bar -> the frame plus "NAME 1/2/3" chips for MENU_HINT_S (4 s),
    │   then the capsule folds into the menu-bar item
    ├── cursor -> recording stays under the menu bar; the "Send to" card
    │   (number, name, reason) opens at the pointer and its Sent stays there
    └── notch -> a black shape grows out of the notch (185 pt wide; on a
        flat display a pretend one the menu bar's height) into the pill and
        the same card, its content under the notch, and shrinks back into it
        (capture_capsule/notch.rs). The playground's "Prototypes/Capture capsule/notch"
        story plays it on a 14-inch MacBook Pro and a 27-inch display, side
        by side or each whole, at their real sizes
    The playground's "Prototypes/Capture capsule/family" story (playground/family.rs)
    plays a dictation and then a capture in one shape on the same springs:
    from the notch on a 14-inch MacBook Pro, and on a flat 27-inch display
    inside the menu bar (hud_placement's 3 pt inset) or floating 6 pt under
    it. Its variants draw the voice as Dot scope's braid (its real
    simulation), bars, or a line; only the capture adds a red dot or a small
    screen, the cue alternating each loop
    Gap: only the shapes are prototyped; the menu-bar item's capture list
    is not, the app doesn't read the screen's notch (the overlay still puts
    the pill's content under it), and no flow but list has run live

The on-screen picker                     // capture.picker (capture_capsule/picker.rs)
├── A full-screen Rune overlay on the pointer's display, under the capsule,
│   left out of recordings (`OwnWindow::Picker`); it takes the pointer only while
│   picking and never takes focus; it follows the pointer to another display
├── Made at launch, hidden; away from macOS its window holds no GPU surface
│   while hidden (`release_gpu_when_hidden`), so a Hex that never annotates
│   holds no GPU memory for it (`perf/idle.bench.ts#window-closed-memory`,
│   `gpu.vram`)
├── Everything but the target dims; the dim moves with the target as one piece
├── Window -> the window under the pointer: tinted, ringed, its title in a badge;
│   the cursor becomes Scrawl's camera; no window under it -> the display
├── Area -> drag at least 36 pt; the badge shows its size; a click alone -> the
│   display
├── Screen -> the display, ringed
│   └── An area's ring is white with a dark hairline outside it, so it reads
│       over white content too (Scrawl's vanishes there)
├── Click or ⌥⇧S -> Action::Start(target) -> CaptureCommand::Start; the picker
│   fades; a pick that starts no capture within 2 s brings it back
└── Windows load off the UI thread (`list_selectable_windows`); nothing picks
    until they arrive

Hex's own windows                        // capture.own-windows
├── Chrome (the capsule, ink, picker, HUDs, meeting capsules: Hex's named
│   windows and any of its windows above the normal level) -> never recorded,
│   never in the window log, never the target, clicks on it not logged
│   ├── Other recorders -> the capsule, picker, and ink are left out
│   │   (sharingType none) while shown; the dictation HUD and meeting
│   │   capsules show
│   │   └── Hidden -> ordered out of every Space once the hide finishes,
│   │       before sharingType returns to readOnly; prewarming keeps the
│   │       surface ready but leaves no invisible panel ordered in. This
│   │       avoids relying on macOS restoring sharing after exclusion
│   ├── Show in screen recordings on -> every recorder, Hex's captures
│   │   included, shows all of it (sharingType readOnly, no SCK exclusions);
│   │   still never in the window log or the target. Rune's
│   │   OverlayHandle::set_exclude_from_capture flips open windows, and the
│   │   sampler rebuilds its filter mid-recording (OwnWindows::recorded) and
│   │   the recording's (capture-core's Recording::set_exclusions)
│   └── Rune's presence tests check ordering and sharing together, including
│       stale fade completions and setting changes. Its native `spaces_probe`
│       checks this process's WindowServer ordering. A native Hex smoke on
│       2026-11-02 failed before the pin bump and passed after it: capsule,
│       Screen/Area picker, ink, dictation HUD and meeting capsule show and
│       leave no onscreen window after hiding. This used isolated test state,
│       without recording. Mission Control's actual thumbnail report and
│       third-party recording exclusion remain unverified
│   └── Click-through chrome (the full-screen ink overlay, the capsule's
│       640×424 panel outside its pill) -> clicks there count for the app
│       beneath; only the pill's (capture_host::set_capsule_pill) are Hex's
└── Normal windows (Settings) -> like any app's: recorded, logged, pickable,
    and Hex is the front app when its window is the topmost normal one, so a
    capture of Settings routes by Hex's name. A capsule Paste still goes to
    the last other app, never into Hex

Ink and laser                            // capture.ink
├── A click-through overlay over the recorded display, excluded from the recording
├── Shift strokes draw as ink, hold 650 ms, then fade over 650 ms
└── The laser's tail lasts 420 ms; nothing lags the pointer

While a capture records                  // capture.input
├── The existing session tap adds mouse down/up/drag/scroll and Shift (no second tap)
├── -> Capture::input() through a bounded queue; full -> events dropped and counted
├── Keys pressed in the app -> logged; keys Hex consumes (its shortcuts, Esc) and
│   anything while a shortcut is being recorded -> not logged
├── Scrawl's rules in the engine (hosted_input): Shift drag inks, a
│   Shift-click is part of the drawing (no pins, no ⌘Z undo), clicks on
│   Hex's capture chrome ignored; while drawing, capture_host::as_pen makes
│   every pointer event Shift ink
└── Shift still held from ⌥⇧S -> must come up once before it can ink; Shift up
    at the start -> the first Shift drag inks

hex capture start                        // capture.start; needs a listening Hex
├── Creates <captures folder>/<unix ms>.scrawl (owner-only) -> reply: recording + bundle path
├── Two stopped captures still writing -> refused `busy`
├── Copies the dictation timeline from now into a bounded sink (no second stream)
├── In the background: Scrawl's engine starts on the active display
│   ├── Needs Screen Recording for Hex (or the terminal running it)
│   └── Fails to start -> sink closed, bundle removed, last outcome `failed`;
│       Esc and the next ⌥⇧S see nothing recording
├── Release microphone while idle -> the capture keeps the microphone open
└── Already recording -> unchanged, same bundle

While it records                         // capture.audio
├── Dictation and meetings work as usual on the same audio
├── Sink full (collector stalled) -> capture audio dropped as a gap, dictation untouched
└── Microphone gap or recovery -> the gap is recorded, positions stay on the clock

Never on Hex's path                      // capture.isolation
├── No microphone, event tap, global shortcut, or model of its own
├── No ingest server (47777) or recovery workers; delivery retries only while
│   neither Scrawl.app runs nor anything serves the ingest port (capture.route)
├── No ingest token used; Scrawl's two config files read once by the import, never
│   written; nothing else of Scrawl's touched (its log goes to Hex's tracing)
└── Settings from Hex (capture.*), not Scrawl's config: auto-crop on (display
    captures), the agentContext summary format off

Retention                                // capture.retention
└── Hex's own thread: Scrawl's retention pass with capture.*'s policy, 90 s after
    captures turn on, then hourly; skipped while a capture records or writes,
    or while Scrawl.app runs (it cleans the shared folder then)

hex capture stop                         // capture.stop
├── Closes the sink (≤ 100 ms wait) -> reply: idle + the bundle path, at once
├── The screen recording, window log, and input stop at once (Capture::stop)
├── recording.mp4 finishes after that on capture-core's own thread: detached
│   before its stream stops, its final frame held to the stop, capture.json
│   written beside it; only the steps that read the movie wait for it, and it
│   starts where its first frame was shown (video_offset_ms)
└── In the background, on one capture-finish thread, one capture at a time:
    ├── 16 kHz audio with its start offset and gaps -> SealedCapture::finish
    ├── Words: 30 s chunks, each a foreground job on the dictation engine's model,
    │   after queued dictation, before meeting chunks; waits while dictation records
    │   or transcribes; never interrupts a live stream
    ├── Word times moved earlier by the model's measured lag (Parakeet Unified
    │   English: starts −280 ms, −520 ms after a ≥ 250 ms pause; ends −195 ms);
    │   other models and Apple keep their own times
    ├── Past 20 min -> audio kept to 20 min, the rest one gap
    ├── recording.mp4, capture.json, audio.wav, frames/, scrawl.json,
    │   summary.md; then every file 0600 and folder 0700
    ├── Keyframes decoded from recording.mp4 by scrawl-helper (beside `hex`: in
    │   Hex.app's Contents/MacOS or target/<profile>; or $SCRAWL_HELPER); helper
    │   missing -> the 480 px sampled frames
    ├── Spoken cues (look, now, here, this…) pick the keyframe the screen
    │   changes to; they also count in Spanish (mira, ahora, aquí, esto…),
    │   French, German, Portuguese, and Italian, whatever the dictation
    │   language. Check: scrawl-capture keyframes
    │   spoken_cues_count_in_other_languages
    ├── Transcription fails -> logged; the bundle has no words, and the outcome says why
    ├── Written -> image and text copied to the clipboard (Scrawl's handoff)
    └── Sent -> the capture-route thread (capture.route)

What a send carries                      // capture.send (router opencode::send_bundle)
├── One prompt: the summary (transcript, marks, notes) as text, then the
│   contact sheet as one inline `data:image/png;base64` file part
│   (12 MiB encoded at most; past it, no image). The keyframes are not
│   attached; the text says they are in `frames/` and `frames-full/` of the
│   capture folder. The picture does not depend on the folder, so an OpenCode
│   server on another machine reads it; the text only names where it was recorded
├── Every request goes to the running local service over HTTP
│   (service.json); a service that isn't up is never started
├── With an agent host (routing's agentHost), the request body travels on SSH
│   stdin to a Python 3 standard-library HTTP client. The contact sheet stays
│   inline; the bundle copy follows the accepted prompt in the background
│   (the prompt says the folder may still be arriving). Credentials stay remote.
│   An explicitly configured agentOpencode keeps the CLI and synchronous copy;
│   without Python 3, the default path falls back to the CLI before any HTTP send
│   └── macOS/Linux reuse the SSH connection for 60 s after its last use,
│       through an owner-only control socket; Windows opens each connection.
│       API reads now honor their caller's deadline over SSH too, including
│       the optional 600 ms idle-status check before a prompt. A failed idle
│       check records unknown status and still sends. Neither CLI startup nor
│       the light-bundle copy is on the default prompt's critical path
├── No video: a capture's movie is too large for a prompt
└── Check: router opencode tests; a send from apollo to a throwaway session
    on OpenCode V2 stored the contact sheet and keyframe as inline image/png (before one-sheet)
    (2026-10-29)

Where it goes                            // capture.route; capture.routing's autoSend on
├── Once recording starts -> a capture-look thread reads the tab,
│   terminal, and sessions in front (Scrawl's router, in Hex's process, with
│   Scrawl's own router folders, so `scrawl router undo` and its evals see Hex's
│   sends as Scrawl's). The whole look has 15 s: steps not begun by then are
│   skipped, and routing waits 1 s more for the one under way, then goes on
│   without it. The newest-session list starts beside local sensing; exact
│   folders are read in the other lane as soon as the clues name them (at most
│   two API reads in flight, one shared 10 s session budget). Results merge in
│   the same order; missing session IDs are read afterward. Saved rules match
│   locally from the captured bundle ID or observed URL; suggestions read the
│   list afterward. A saved rule takes precedence over a spoken session name.
│   An app rule's folder is read with the sessions, and a failed
│   page of sessions keeps what was read and still reads the folders and
│   sessions that rules name
│   └── Timings in process.log: local sensing, total lookup, page publisher,
│       written-to-look wait, judging, and sending (sync, idle check, post).
│       Fake-dependency benchmarks and their proof limits:
│       [Annotate routing latency](https://atomgit.com/GitHub_Trending/hex19/hex/blob/dev/perf/annotate-routing.md)
├── Written -> the one capture-route thread (queue 16) decides; the snapshot's
│   `last.route` follows: Looking -> Rule | ResolvingRule | Guess | Nothing |
│   NoAnswer | Unreachable. Background refinements preserve an open search and
│   selected session; a pick or Undo cancels them
│   ├── Rule -> only a rule the person saved (`routing.sites`, `routing.apps`,
│   │   a Shift pick's): a three-second countdown begins at Written. A folder
│   │   rule shows its repository while its session resolves; it sends once both
│   │   the countdown and resolution finish. Undo during the countdown posts
│   │   nothing. The capsule's `Hold` pauses the countdown while the pointer is
│   │   on it and resumes it with what it had left (at least 1 s; a hold the
│   │   capsule never ends lapses after 2 min, and the next capture resumes
│   │   it). Sending -> Sent, or Queued when OpenCode didn't take it. Whatever
│   │   autoSend, smartRouting, or jevRouting say, a saved rule sends and
│   │   nothing else sends on its own. smartRouting off decides with the port
│   │   ladder, after rules and addresses, on the same look
│   ├── Guess -> everything else the router would send (Jev's pick, HOME when
│   │   Jev couldn't tell, a dev server's port, a page's data-session or
│   │   publisher, a name that was said, the captured app's checkout (the one
│   │   open session folder named like the app, such as `…/hex` for Hex's own
│   │   Settings; smartRouting only), the port ladder) is suggestion #1, its
│   │   reason as the row's word ("dev server", "same page", "mentioned"; none
│   │   for Jev's pick or fallback), then
│   │   the runners-up and recent sessions; nothing sent until one is picked.
│   │   The deterministic guess publishes first. Model refinement has one
│   │   three-second budget including recent messages and credential lookup;
│   │   a timeout keeps the first guess. A later result cannot override a pick or Undo.
│   │   `site` names what a Shift pick (Shift+click, ⇧⏎ in the list,
│   │   or `send --always`) would save, after which that site's captures are Rules;
│   │   ⇧⏎ in a Rule's list saves the rule for the session picked instead
│   ├── Each Rule or Guess -> routing.json's `shown`: `as` rule or guess, and
│   │   `why`, the router's leading reason; `hex capture route` prints it
│   ├── Nothing / NoAnswer -> recent sessions only
│   └── Unreachable -> the look read no sessions (OpenCode slow past the look's
│       deadline, down, or refusing); a warning in process.log with the error, and
│       the capsule reads "Couldn't reach OpenCode" with why ("OpenCode didn't
│       answer"), or "OpenCode isn't running" alone, keeping Paste. A reason is
│       shown only in plain words: one with a path, URL, OS error or code goes
│       to process.log and the capsule shows none
│       └── OpenCode refusing Hex's password (HTTP 401/403) -> "OpenCode needs a
│           password Hex doesn't have", for the look and for a send, and
│           process.log says where Hex reads it (service.json under this user's
│           ~/.local/state/opencode, or OPENCODE_PASSWORD), never the password.
│           The capture stays saved and copied. Check: capture_host::routing
│           a_look_that_read_no_sessions_answers_unreachable,
│           a_failed_send_says_what_went_wrong. Known gap (#1131): Hex on
│           Windows can't read the registration of an OpenCode running in WSL,
│           so that setup isn't supported
├── An answer for a capsule already closed (dismissed, or past its hold)
│   leaves it closed; `hex capture list` and `send` still offer the capture
├── Starting another capture preserves an older rule's accepted pending send;
│   its eventual answer does not replace the newest capsule
├── The capsule's commands act only on `last`, the newest routed capture:
│   ├── Send{to} -> the Rule's own session, already sent, does nothing; any
│   │   other pick sends there (taking back a send already elsewhere first) and logs
│   │   the correction; `always` also saves a rule for its site or app
│   └── Undo -> a sent one is withdrawn unless the
│       session acted on it (TooLate); an undo the router fails is TooLate too
├── Two sends to one place agree -> `offer` names the site or app a rule could
│   send straight there
├── Queued sends -> retried every 15 s, unless Scrawl.app runs or something
│   serves the ingest port (that process retries); a retried send can't be
│   undone, so its "Sent to X" has no Undo. A queued or too-late-to-undo send
│   leaves after the brief hold (4 s) rather than the 25 s done hold; a taken
│   back capture with nothing to list ('Taken back', a rule still resolving)
│   does too
└── autoSend off -> no look, no route; the capture is only copied

hex capture list|show|send|route         // capture.cli; docs/cli.md
├── list [--limit N] -> the newest bundles in the captures folder: mode, words,
│   where it went
├── show <capture> -> one capture's words, files, and send
├── send <capture> [--to <session>] [--always] -> needs a listening Hex
│   ├── The last routed capture -> as the capsule's pick (no --to: the Rule's
│   │   session, a Guess's first suggestion, or the recorded destination), so
│   │   the capsule's Undo follows it; --always also saves the site or app rule
│   │   for the session it goes to. Also when a newer capture was written first
│   └── Older -> --to must be a session ID; without it, its recorded
│       destination; --always is refused
├── route <capture> [--ask-jev] -> what the router sees and decides; sends nothing
└── <capture> = bundle name, path, or `latest`; unknown -> `not-found`

hex capture cancel -> engine cancelled, sink closed, bundle removed
hex capture status -> recording (bundle, elapsed), bundles still writing, last outcome
                      (words, audio gaps, or why it has no transcript)
Hex quits mid-capture -> cancelled, bundle removed; captures still writing stop
transcribing between chunks and keep their audio (shutdown waits ≤ 2 s)

15 s replay buffer (Labs, macOS 27+)     // capture.replay; labs.annotate_replay
├── Off by default (`Audience::Everyone`); shown and runnable only when macOS 27's
│   `SCClipBufferingOutput` and `SCStream`'s clip-buffering selectors exist at
│   run time (weak Objective-C lookup; older macOS and Linux/Windows neither
│   link nor offer it). Locked off by `DisableAnnotate` and `DisableLabs`
├── Off -> no `SCStream`, no worker thread, no display observer, no consent prompt
├── On (with Annotate on and Scrawl.app not running) -> one worker thread waits
│   until Screen Recording is already granted (never prompts in the background),
│   then runs one 15 fps `SCStream` with `SCClipBufferingOutput` on the pointer's
│   display (≤ 2560 px long edge, cursor hidden, audio off, Hex's `OWN_WINDOWS`
│   and non-layer-0 overlays excluded unless Show in screen recordings is on)
│   ├── `replayd` encodes the rolling 15 s H.264 ring out-of-process; macOS keeps
│   │   the menu-bar screen-recording indicator lit while the stream is active
│   └── Display plug/unplug (`CGDisplayRegisterReconfigurationCallback`) or a
│       stopped stream -> reopens on the active display with bounded backoff
├── `hex capture replay` (`hex annotate replay`) -> exports up to 15 s (`> 15 s`
│   is rejected by ScreenCaptureKit `-3812`) into `<captures>/<unix ms>.scrawl`
│   without stopping the buffer, probes and decodes keyframes with `scrawl-helper`,
│   writes `capture.json` and `scrawl.json` (`missing: "no_audio"`), copies to
│   the clipboard, and feeds `capture.route`
└── `capture.replay_hotkey` (unbound by default; recordable in
    Settings › Annotate while the lab is on, or `hex settings set capture.replay_hotkey`)
    ├── Tap (< 300 ms) -> exports the silent 15 s clip at once, no mic opened
    └── Hold (≥ 300 ms) -> exports the frozen 15 s clip at the press and records
        Hex's microphone timeline with "While dictating" until release + tail;
        narration is shifted onto the bundle clock after the clip (`video_ms..`),
        transcribed with word timestamps, and anchored onto the final keyframe

scrawl.json is Scrawl's bundle manifest. Its transcript.words are {t0, t1, text} in milliseconds from the capture's start (t0_boot_ns, the boot-time clock audio and shortcuts share), and audio.wav starts at that start too, padded with silence when the microphone began late or dropped audio.

hex dev capture-replay <wav>… (debug, with the feature) runs the same capture host on a fresh transcription engine with the selected model, feeding WAV files instead of the microphone, and prints the bundle path. It needs Annotate on too, and records the screen meanwhile.

Sources: capture_host.rs, the tap in suppression.rs, the shortcut and Escape in recognition.rs, CaptureSettings and the import in app_settings.rs, the pane in rune_ui/scrawl.rs, the input rules in hosted_input.rs, the sink in dictation/audio.rs, transcribe_with_words in parakeet.rs, routing in capture_host/routing.rs, the CLI's reads in capture_host/library.rs, the 15 s replay buffer in capture_host/replay.rs and capture_host/replay/sck.rs, the phone finish in phone.rs, and the scrawl-capture control commands in control.rs.

Checks:

  • Gate: the_host_is_absent_while_the_experiment_is_off in capture_host.rs; test-app-identity.sh for the release-path refusals.
  • Sink, in dictation/audio.rs: capture_sink_shares_the_timeline_with_dictation, a_full_capture_sink_drops_audio_as_a_gap_and_never_stalls_dictation, capture_sink_records_microphone_gaps, capture_sink_holds_a_released_microphone_open_until_it_closes, capture_sink_keeps_every_frame_through_timestamp_jitter, and a_dropped_capture_receiver_closes_the_sink.
  • Words, in parakeet.rs: chunked_word_transcription_keeps_word_times_across_chunks and word_transcription_is_a_foreground_request_on_the_shared_engine; foreground_work_waits_for_an_open_stream_instead_of_interrupting_it in transcription/engine.rs.
  • Host, in capture_host.rs (a fake engine that writes through Scrawl's real finalize_capture_bundle_with_flags, without a screen): controls_are_idempotent_and_stop_writes_a_scrawl_bundle, an_engine_that_cannot_start_leaves_nothing_behind, a_screen_source_that_gives_up_ends_the_capture, scrawls_engine_claims_nothing_machine_wide, the_timeline_resamples_pads_gaps_and_ends_at_the_stop, a_pending_gap_and_a_stalled_tail_come_back_from_the_closed_sink, padding_past_the_length_limit_stays_bounded, captures_finish_one_at_a_time_and_starts_wait_for_room, a_capture_queued_behind_a_slow_finish_has_already_stopped, a_failed_start_raced_by_stop_never_wedges_later_starts, the_engine_config_comes_from_hex, and shutdown_cancels_the_recording_capture_and_waits_for_its_cleanup.
  • Shortcut and input, in capture_host.rs: the_host_is_absent_while_captures_are_off, the_shortcut_starts_and_a_later_press_stops, input_reaches_only_the_capture_that_is_recording, tap_flags_become_modifiers, the_shortcut_records_once_its_keys_come_up, a_failed_start_ends_before_escape_or_the_shortcut_sees_it; the_capture_shortcut_is_suppressed_and_never_chord_evidence and a_capture_logs_only_keys_that_reach_the_app in suppression/tests.rs; only_the_capsules_pill_and_not_its_panel_takes_clicks in capture_host.rs; a_paste_goes_to_an_external_app_never_the_host in capture/macos.rs; the_hosts_normal_windows_count_and_its_chrome_does_not in window_log.rs and helper_windows_are_scrawl_s_except_the_capture_source in capture.rs for Hex's own windows; the ink and click rules in hosted_input.rs, including a_capture_started_by_its_shortcut_waits_for_shift_to_come_up, a_capture_started_without_shift_held_inks_from_the_first_stroke, and a_click_with_the_chords_shift_still_held_is_no_pin, and in capture.rs a_shift_click_is_part_of_the_drawing.
  • iPhone: an_iphone_capture_narrates_with_hex_audio_and_finishes_when_its_mirror_ends and a_mirror_ending_never_stops_a_screen_capture in capture_host.rs; since_moves_audio_and_gaps_onto_a_later_clock in audio.rs. The mirror, its --no-microphone, and the camera prompt have not run in Hex.
  • Routing, in routing.rs: only_a_saved_rule_sends_on_its_own, a_rule_sends_and_lists_the_rest, a_guess_sends_nothing_and_a_failure_still_lists_recent_sessions, and its looked tests, which write a bundle and route it through look and Worker::written with fake sessions and tab: a_saved_site_or_app_rule_sends_whatever_the_routing_flags, the_port_ladder_is_a_guess, the_captured_apps_checkout_is_a_guess, a_look_past_its_deadline_is_routed_without, an_always_rule_names_the_session_it_sends_to, an_always_rule_keeps_sending_to_the_session_picked, and a_capture_routed_after_a_newer_one_settled_is_still_sendable_by_name; an_always_rule_makes_its_sites_next_capture_a_saved_rule, a_failing_page_keeps_what_was_read_and_still_reads_what_rules_name, an_app_rules_folder_is_read_with_the_sessions, and a_look_past_its_deadline_skips_what_it_hadnt_begun in look.rs; the CLI's a_capture_on_disk_summarizes_its_mode_words_and_send in library.rs, the protocol round trips in control.rs, and control_commands_parse in main.rs.
  • Switching, in capture_host.rs: a_discarded_capture_leaves_no_trace_and_a_pick_takes_its_place, a_switch_never_shows_nothing_recording, a_switch_with_no_room_to_start_keeps_the_recording, a_switch_that_cannot_start_after_its_discard_settles_failed, discarded_captures_never_count_against_the_writers, a_pick_that_cannot_start_says_why, the_first_click_or_scroll_means_work_started, and the_newest_grab_takes_only_its_keys_until_it_drops. Paste and Inspect need a written capture (controls_are_idempotent_and_stop_writes_a_scrawl_bundle); neither has pasted into or opened anything in a test.
  • Capsule, in rune_ui/capture_capsule (feature builds only): the flow tests (chips_fold_once_seen_when_work_starts_but_never_under_the_pointer, the_shortcut_records_the_pickers_screen_and_a_chip_discards_the_recording, …), feed tests (the_shortcuts_recording_shows_no_target_chips, the_pen_lights_while_drawing, a_rule_shows_once_and_only_news_the_capsule_cant_know_moves_it, a_queued_send_says_so_and_says_sent_when_it_goes, a_cabled_phone_shows_its_chip, …), paste_takes_back_a_send_still_undoable_and_the_capsule_stays_gone, a_rule_reads_sent_at_once_and_sounds_when_it_went, undoing_a_rule_send_reopens_the_list_until_a_pick, undoing_a_pick_reopens_the_list_it_came_from, a_pick_during_an_undo_stands_when_the_undo_answers, a_send_undone_while_under_way_keeps_the_list_however_it_ends, the_undo_window_wakes_the_capsule_once_at_its_end, every_rule_send_sounds_sent_once, picking_the_rules_own_session_just_closes_the_list, too_late_to_take_back_offers_the_list_for_a_copy, shift_return_in_a_rules_list_saves_a_rule_elsewhere, shift_shows_the_rule_and_a_shift_pick_saves_it, an_answer_after_the_capsule_closed_leaves_it_closed, the_capsule_takes_only_the_keys_its_state_needs, shortcuts_read_as_one_quiet_word, the_capsule_draws_no_keycaps, a_written_copy_offers_paste_and_inspect_as_icons, every_change_moves_at_once, entry_lands_the_pose_instantly_then_springs_to_rest_with_its_content, the_toast_cycles_its_guesses_and_return_sends_the_one_shown, the_menu_bar_hint_is_brief_and_its_chips_send, the_cursor_card_lists_its_guesses_at_the_pointer, the_notch_hangs_from_the_top_edge, the_menu_bar_flow_folds_up_toward_the_menu_bar, goldens tests/golden/capture-capsule-*.png, and the_capsule_moves_inside_its_panel_and_comes_to_rest, which traces every scripted state change per frame, Calm and Reduce Motion, for no pops, jumps, or jiggle. HEX_CAPSULE_MOTION also writes rune-motion traces for motion-lint. cues_mark_start_stop_sent_and_taking_back, nothing_heard_says_so, and a_capture_with_no_words_says_nothing_was_heard cover the tones and the empty transcript. Observed (Oct 27, worker, r19 motion): all 28 capsule runs, Calm and Reduce Motion, pass lint.
  • Playground: hex capsule-playground (feature builds; hidden), or scripts/build-capsule-playground.sh for an ad-hoc signed Hex Playground.app, runs the real capsule in a window of its own against a fake host, with state buttons, the whole flow, slow motion, Reduce Motion, the tones, and live tuning (tuning.rs; "Copy values" copies its DEFAULT). It reads and writes no Hex data and records or sends nothing. The whole flow picks the first suggestion; its "Saved rule" toggle plays a rule's send, sent at once, instead. --snap --scene suggestions|shift|sent-rule|search|typed|picked|rule|recording renders it. The tones were checked by waveform and level only, never on speakers.
  • Picker, in capture_capsule/picker.rs and tests.rs: geometry, label, and target tests, the_picker_records_what_it_shows_and_comes_to_rest, and picker_stills_and_motion, which scripts window hover, an area drag, and mode switches per frame (Calm and Reduce Motion). Observed (Oct 26, pro): all 6 runs pass motion-lint with two excuses: the target growing from a zero-size point, and the camera stopping where the scripted pointer stops. Not proven natively end to end: the overlays have been driven only by the scripted scenes, not a live capture.
  • Pane: captures_default_off_and_refuse_a_shortcut_in_use and retention_names_presets_and_imported_days in rune_ui/scrawl.rs; scenes mac-scrawl-off, mac-scrawl, mac-scrawl-running, mac-scrawl-permission (feature builds only, so no goldens: those run on Linux).
  • Settings: every capture.* key in hex settings (every_persisted_setting_has_a_cli_key).
  • Replay, in capture_host/replay.rs and capture_host/replay/sck.rs: empty_audio_produces_no_wav_and_narrated_audio_shifts_after_video, capture_manifest_matches_v1_schema_with_and_without_audio, encoded_dimensions_stay_even_and_capped_at_2560, replay_shortcut_begins_on_press_finishes_on_release_and_cancels_on_escape, and error_completion_block_carries_xpc_signature.

Observed 2026-10-25 on pro (M2 Max), before the real engine, with the stand-in: HEX_EXPERIMENTS=capture hex dev capture-replay over three vocabulary fixtures (6.5 s) loaded Parakeet unified English on Metal and wrote 18 words with plausible times. The non-replay hex capture engine has not run end to end in automated tests (unit tests use a fake engine over Scrawl's real bundle writer). Observed 2026-11-04 in a macOS 27.0 (26A428) guest VM on Apple Silicon with Developer ID-signed Hex Test.app: labs.annotate_replay opened a 15 fps 1920×1080 SCClipBufferingOutput stream, exported 15,000 ms silent and 19,581 ms narrated bundles (13 Parakeet words anchored on the final keyframe, cursor hidden, Hex chrome excluded, clipboard handoff populated) via hex capture replay, an ⌥⇧R tap, and a 3.5 s ⌥⇧R hold, recovered across CGVirtualDisplay plug/unplug and display sleep/wake, and rejected a 16 s request with SCStreamErrorDomain -3812. The menu-bar screen-recording indicator stays lit for the entire lifetime of the rolling buffer.

本页原文 Markdown:在 AtomGit 查看·内容源自开源项目 hex19/hex