Skip to content

feat(metrics): broadcast llm tps + ttfs - #2224

Open
rosetta-livekit-bot[bot] wants to merge 1 commit into
mainfrom
cheerily-dogies-letup
Open

feat(metrics): broadcast llm tps + ttfs#2224
rosetta-livekit-bot[bot] wants to merge 1 commit into
mainfrom
cheerily-dogies-letup

Conversation

@rosetta-livekit-bot

@rosetta-livekit-bot rosetta-livekit-bot Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Ports livekit/agents#6373 to expose per-turn LLM throughput and time-to-first-sentence metrics on assistant messages and preserve them across remote-session and telemetry serialization.

Adds a minor changeset for the new public MetricsReport fields.

Source diff coverage

7 source files classified
  • Ported: livekit-agents/livekit/agents/llm/chat_context.py -> agents/src/llm/chat_context.ts. Added public llmNodeTps and llmNodeTtfs assistant-message metrics with equivalent semantics.
  • Ported: livekit-agents/livekit/agents/voice/agent_activity.py -> agents/src/voice/agent_activity.ts. Attaches TPS and TTFS to the committed assistant message.
  • Ported: livekit-agents/livekit/agents/voice/generation.py -> agents/src/voice/generation.ts. Records inference start, calculates TPS over the first-to-last content window using provider usage, captures only the explicit LiveKit TTS synthesis stamp for TTFS, and retains the existing fallback anchor for TTFB.
  • Adapted: livekit-agents/livekit/agents/voice/remote_session.py -> agents/src/voice/remote_session.ts, agents/src/telemetry/traces.ts, and agents/src/telemetry/otel_http_exporter.ts. JS has explicit protobuf and cloud-log serialization boundaries rather than Python's allowed-key tuple, so both fields are mapped at each boundary and encoded as doubles.
  • Adapted: tests/test_agent_session.py -> agents/src/voice/agent_session_metrics.test.ts. Uses the JS session/TTS harness to verify TTFS anchors on provider synthesis start and remains distinct from TTFB.
  • Ported: tests/test_llm_node_metrics.py -> agents/src/voice/generation_metrics.test.ts. Covers the same usage-present, zero-usage, no-usage, single-chunk, TTFT-exclusion, start-time, and TTFS omission cases.
  • Adapted: tests/test_session_host.py -> agents/src/voice/remote_session.test.ts. Drives the JS SessionHost event boundary and verifies both values reach protocol MetricsReport.
  • Not applicable: none.

Validation

  • pnpm test agents (1591 passed, 5 skipped)
  • pnpm build:agents
  • pnpm --filter @livekit/agents lint (passes with existing repository warnings)
  • pnpm --filter @livekit/agents typecheck
  • pnpm --filter @livekit/agents throws:check
  • Prettier check for all changed source/test files
  • cue-cli voice-mode runtime validation: assistant framework event included llm_node_tps and llm_node_ttfs

api:check was also attempted but is currently blocked before API comparison by API Extractor rejecting the repository's existing generated export * as syntax.


Ported from livekit/agents#6373

Original PR description
  • llm_node_tps — output tokens/sec, from the provider's usage chunk
  • llm_node_ttfs — inference start until the first sentence is handed off for synthesis (reuses livekit.agents.types.USERDATA_TTS_STARTED_TIME)
  • agent_false_interruption — the event, forwarded over the session host
    was implemented in #6691

Needs: livekit/protocol#1661 (released, already used)

@rosetta-livekit-bot
rosetta-livekit-bot Bot requested a review from a team as a code owner August 5, 2026 09:55
@changeset-bot

changeset-bot Bot commented Aug 5, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 91f89f9

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 39 packages
Name Type
@livekit/agents Major
@livekit/agents-plugin-anam Major
@livekit/agents-plugin-anthropic Major
@livekit/agents-plugin-assemblyai Major
@livekit/agents-plugin-azure Major
@livekit/agents-plugin-baseten Major
@livekit/agents-plugin-bey Major
@livekit/agents-plugin-cartesia Major
@livekit/agents-plugin-cerebras Major
@livekit/agents-plugin-deepgram Major
@livekit/agents-plugin-did Major
@livekit/agents-plugin-elevenlabs Major
@livekit/agents-plugin-fishaudio Major
@livekit/agents-plugin-google Major
@livekit/agents-plugin-hedra Major
@livekit/agents-plugin-hume Major
@livekit/agents-plugin-inworld Major
@livekit/agents-plugin-krisp Major
@livekit/agents-plugin-lemonslice Major
@livekit/agents-plugin-liveavatar Major
@livekit/agents-plugin-livekit Major
@livekit/agents-plugin-minimax Major
@livekit/agents-plugin-mistral Major
@livekit/agents-plugin-mistralai Major
@livekit/agents-plugin-neuphonic Major
@livekit/agents-plugin-openai Major
@livekit/agents-plugin-perplexity Major
@livekit/agents-plugin-phonic Major
@livekit/agents-plugin-protoface Major
@livekit/agents-plugin-resemble Major
@livekit/agents-plugin-rime Major
@livekit/agents-plugin-runway Major
@livekit/agents-plugin-sarvam Major
@livekit/agents-plugin-silero Major
@livekit/agents-plugin-soniox Major
@livekit/agents-plugin-tavus Major
@livekit/agents-plugins-test Major
@livekit/agents-plugin-trugen Major
@livekit/agents-plugin-xai Major

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@rosetta-livekit-bot
rosetta-livekit-bot Bot requested a review from Bobronium August 5, 2026 09:56

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Open in Devin Review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant