Skip to content

feat(voice): hear through an OpenRouter transcription model set in voice.listen - #91

Merged
tnfssc merged 1 commit into
developfrom
feat/voice-cloud-listen
Oct 29, 2026
Merged

tnfssc merged 1 commit into
developfrom
feat/voice-cloud-listen

Conversation

@tnfssc

@tnfssc tnfssc commented Oct 29, 2026

Copy link
Copy Markdown
Owner

Voice can now write down your words with a transcription model on OpenRouter instead of Nemotron on this machine.

{ "voice": { "listen": "meta/muse-voice-transcribe-1.0" } }

What changes

  • With voice.listen set and Pi's OpenRouter key present, the worker loads no recogniser. The VAD still finds each turn on this machine; the turn's audio is sent whole to /api/v1/audio/transcriptions as 16 kHz WAV, and the words come back in turn order.
  • Nemotron (463 MB) isn't downloaded then, in the terminal or on the web page.
  • Without a key, or with listen unset, nothing changes: Nemotron hears.
  • The first /voice with it says once that your words go to OpenRouter.

Trade-offs

  • None of these models streams. Words show only when a turn ends, 1–2 s later, and talking over a reply stops it only then (there are no live words to cut in on).
  • A failed turn shows the error and is dropped. There is no fallback to Nemotron, since it may not be downloaded.

Checked

  • Real engine and worker, no recogniser loaded, recorded clip: Muse returned the words; a bad model ID showed Model bad/model does not exist and the turn was dropped.
  • The same request worked for Muse, Scribe v2, GPT-4o mini transcribe (0.86 s on a 2 s turn, fastest), Nemotron 3.5 streaming and AssemblyAI Universal 3.6.
  • Not tried: a live /voice session with a real microphone.
  • bunx biome ci . (one warning in src/ui.ts, already on develop), bun run check, bun test, bun run test:site.

…ice.listen

With a key, the worker keeps each turn's audio and the engine sends it whole, so Nemotron (463 MB) isn't needed. Words come only at the turn's end. Also records the HeyGen and Muse tests in the voice notes.
@coderabbitai

coderabbitai Bot commented Oct 29, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: b941d095-6065-4ff9-8d39-ba4a9e7f2fc7
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@tnfssc
tnfssc merged commit d62b4ec into develop Oct 29, 2026
2 checks passed
@tnfssc
tnfssc deleted the feat/voice-cloud-listen branch October 29, 2026 22:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant