Skip to content

Add keyboard fallback to recording scrollDown - #282

Merged
sonichi merged 1 commit into
mainfrom
fix/recording-scroll-keyboard
Apr 12, 2026
Merged

sonichi merged 1 commit into
mainfrom
fix/recording-scroll-keyboard

Conversation

@sonichi

@sonichi sonichi commented Apr 12, 2026

Copy link
Copy Markdown
Owner

Summary

Adds the same keyboard Page Down fallback from PR #281 to the scrollDown helper used by recording tools. During phone summon + Zoom screen share, Chrome skips repaints for JS scrollBy — keyboard input forces the repaint.

Test plan

  • Start recording with phone summon + Zoom screen share + Vimeo
  • Verify page visually scrolls during recording

🤖 Generated with Claude Code

The recording scroll path (scrollDown in browser-tools.ts) was missing
the keyboard Page Down fallback added in PR #281 for the interactive
scroll tool. During phone summon + Zoom screen share, Chrome defers
visual repaints for JS scrollBy but keyboard input forces a repaint.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@sonichi
sonichi merged commit 39b3500 into main Apr 12, 2026
@sonichi
sonichi deleted the fix/recording-scroll-keyboard branch April 12, 2026 00:38
liususan091219 added a commit that referenced this pull request Apr 12, 2026
Chrome skips visual repaints during Zoom screen share even though
JS scrollBy updates scrollY. Adding a Page Down keystroke after
the JS scroll forces Chrome to repaint — same fix as main's #282.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
liususan091219 added a commit that referenced this pull request Apr 12, 2026
Chrome skips visual repaints during Zoom screen share even though
JS scrollBy updates scrollY. Adding a Page Down keystroke after
the JS scroll forces Chrome to repaint — same fix as main's #282.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
sonichi pushed a commit that referenced this pull request Apr 12, 2026
The scrollDown() function from PR #280/#282 had the same
querySelectorAll("*") AppleScript quoting bug. Also switched
from inline osascript to temp file approach for consistency
with scrollTool.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
sonichi added a commit that referenced this pull request Apr 12, 2026
* fix: scroll AppleScript quoting + directive greeting prefix

1. Scroll: querySelectorAll("*") broke AppleScript string escaping
   (error -2741). Changed to single quotes in JS which are safe
   inside AppleScript's double-quoted strings.

2. Greeting: bodhi sends greeting as role:"user" to trigger Gemini
   to speak. Without context, Gemini sometimes responds TO the text
   instead of speaking it. Added "[Speak this greeting to the caller
   now]:" directive prefix so Gemini treats it as an instruction.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: scrollDown recording helper also needs single-quote fix

The scrollDown() function from PR #280/#282 had the same
querySelectorAll("*") AppleScript quoting bug. Also switched
from inline osascript to temp file approach for consistency
with scrollTool.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* summon: auto-close zoom.us tab in Chrome after joining

After summon joins Zoom via the browser URL, Chrome keeps the
zoom.us tab open. This becomes the active tab, so scroll and
other commands target it instead of the user's actual page.

Now closes all zoom.us tabs in Chrome after the join completes.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Chi <wangchi@Chis-Mac-mini.hsd1.wa.comcast.net>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
sonichi added a commit that referenced this pull request Apr 12, 2026
Meeting mode:
- switch_mode tool: Gemini can toggle between "active" and "meeting"
- Auto-detects active Zoom on startup (sync, runs before first greeting)
- Meeting greeting: listen and track discussion, produce zero audio
- System instructions dynamically include meeting state
- Auto-activates on summon/join_zoom/join_gmeet, deactivates on dismiss

Recording scroll fix:
- Remove keyboard fallback from scrollDown helper (PR #282 regression)
- Chrome activate + Page Down disrupts narration audio capture,
  corrupting subtitle generation
- Keep keyboard fallback in interactive scrollTool only (PR #281)
- Fix querySelectorAll("*") → querySelectorAll('*') quote escaping

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
sonichi added a commit that referenced this pull request Apr 12, 2026
* Meeting mode + recording scroll fix

Meeting mode:
- switch_mode tool: Gemini can toggle between "active" and "meeting"
- Auto-detects active Zoom on startup (sync, runs before first greeting)
- Meeting greeting: listen and track discussion, produce zero audio
- System instructions dynamically include meeting state
- Auto-activates on summon/join_zoom/join_gmeet, deactivates on dismiss

Recording scroll fix:
- Remove keyboard fallback from scrollDown helper (PR #282 regression)
- Chrome activate + Page Down disrupts narration audio capture,
  corrupting subtitle generation
- Keep keyboard fallback in interactive scrollTool only (PR #281)
- Fix querySelectorAll("*") → querySelectorAll('*') quote escaping

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Meeting mode: answer from own context, don't call work

In meeting mode, Gemini heard the audio but the core agent didn't.
When addressed by name, Gemini should answer directly from its
accumulated context instead of delegating to work.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
liususan091219 pushed a commit to liususan091219/sutando-fork that referenced this pull request May 28, 2026
The recording scroll path (scrollDown in browser-tools.ts) was missing
the keyboard Page Down fallback added in PR sonichi#281 for the interactive
scroll tool. During phone summon + Zoom screen share, Chrome defers
visual repaints for JS scrollBy but keyboard input forces a repaint.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
liususan091219 pushed a commit to liususan091219/sutando-fork that referenced this pull request May 28, 2026
)

* fix: scroll AppleScript quoting + directive greeting prefix

1. Scroll: querySelectorAll("*") broke AppleScript string escaping
   (error -2741). Changed to single quotes in JS which are safe
   inside AppleScript's double-quoted strings.

2. Greeting: bodhi sends greeting as role:"user" to trigger Gemini
   to speak. Without context, Gemini sometimes responds TO the text
   instead of speaking it. Added "[Speak this greeting to the caller
   now]:" directive prefix so Gemini treats it as an instruction.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: scrollDown recording helper also needs single-quote fix

The scrollDown() function from PR sonichi#280/sonichi#282 had the same
querySelectorAll("*") AppleScript quoting bug. Also switched
from inline osascript to temp file approach for consistency
with scrollTool.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* summon: auto-close zoom.us tab in Chrome after joining

After summon joins Zoom via the browser URL, Chrome keeps the
zoom.us tab open. This becomes the active tab, so scroll and
other commands target it instead of the user's actual page.

Now closes all zoom.us tabs in Chrome after the join completes.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Chi <wangchi@Chis-Mac-mini.hsd1.wa.comcast.net>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
liususan091219 pushed a commit to liususan091219/sutando-fork that referenced this pull request May 28, 2026
* Meeting mode + recording scroll fix

Meeting mode:
- switch_mode tool: Gemini can toggle between "active" and "meeting"
- Auto-detects active Zoom on startup (sync, runs before first greeting)
- Meeting greeting: listen and track discussion, produce zero audio
- System instructions dynamically include meeting state
- Auto-activates on summon/join_zoom/join_gmeet, deactivates on dismiss

Recording scroll fix:
- Remove keyboard fallback from scrollDown helper (PR sonichi#282 regression)
- Chrome activate + Page Down disrupts narration audio capture,
  corrupting subtitle generation
- Keep keyboard fallback in interactive scrollTool only (PR sonichi#281)
- Fix querySelectorAll("*") → querySelectorAll('*') quote escaping

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Meeting mode: answer from own context, don't call work

In meeting mode, Gemini heard the audio but the core agent didn't.
When addressed by name, Gemini should answer directly from its
accumulated context instead of delegating to work.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant