Skip to content

feat(telemetry): add the OpenTelemetry pipeline and core Berd events - #16

Open
matt2e wants to merge 1 commit into
mainfrom
simple-telemetry
Open

feat(telemetry): add the OpenTelemetry pipeline and core Berd events#16
matt2e wants to merge 1 commit into
mainfrom
simple-telemetry

Conversation

@matt2e

@matt2e matt2e commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator

Summary

Berd had no product analytics. This adds the full renderer-to-gateway telemetry path plus the first catalog of Berd events, instrumented at the points where each user action actually commits rather than where it is merely requested.

Pipeline

  • An OTel logs pipeline in src/shared/telemetry (client.ts, exporter.ts, identity.ts) batches records and hands the serialized OTLP/HTTP JSON body to the Tauri layer, so no renderer code holds a credential or an endpoint.
  • src-tauri/src/commands/telemetry.rs owns auth and transport: anonymous POST /v1/bootstrap with a validated installation id, token TTL read from the response, a single 401 re-auth retry, and POST /v1/logs carrying x-berd-schema-version: berd-otlp-logs-v1 against the gateway's berd service and berd.telemetry scope.
  • The client stays inert unless enabled: development never reaches the wire, startup gates are bounded so a stalled bootstrap cannot wedge the app, dropped records are counted, and batch size and attribute lengths are capped under the gateway's body limit. Empty optional attributes are omitted rather than sent as empty values.
  • The staging/prod gateway hosts are not decided yet, so the endpoint defaults to a dummy .invalid host in the real endpoint's shape. Swapping in a real host is pure configuration — VITE_OTLP_LOGS_ENDPOINT, ALLOWED_OTEL_LOGS_HOSTS, the CSP connect-src, and the pinned test values — with no code-path changes (see the TODO(telemetry-otel) in vite.config.ts).

Events

src/shared/telemetry/events has one module per surface, covering app launch, chat sends, agent edits, home pins, and project mutations. Each is anchored to its commit point:

  • App launch fires once per app start, not per reload; detached session windows initialize telemetry but do not report a launch.
  • Chat send events fire at the user-message commit, including steer sends at the steer commit and released deferred-workspace sends; is_first_message is gated on the session history having replayed.
  • Agent Edit Completed is anchored to the persisted persona write, and agent creation is counted from the duplicate, import, and onboarding surfaces.
  • Pin/unpin reports only changes that survive persistence, uses the id the pin will keep, emits per item on bulk unpin, and stays silent for starter-agent seeding and the legacy-Berdy pin migration.
  • berdctl project mutations are deliberately excluded; the exclusion is documented at the command modules.

Ported from squareup/berd@5e28e6e1, adapted to this repository's drift (worktree startup modes, pre-send workspace setup) with the telemetry behavior and wire contract intact.

Related issue

None found — searched open and closed issues/PRs for telemetry, no duplicates.

Testing

Tests accompany each surface, pinning the wire body contract (exporter.test.ts, client.test.ts), the event attribute shapes (events.test.ts, homePinTelemetry.test.ts), and the commit-point behavior (useChatSessionController.test.ts, AgentBuilderRail.test.tsx, AgentsView.entry.test.tsx, usePinToHomeWidget.test.tsx, HomeView.test.tsx, OnboardingFlow.test.tsx, main.test.tsx).

Relevant gates for a reviewer to run: just check, just test, just tauri-check, just clippy. No user-visible UI change, so no screenshots.

@matt2e
matt2e force-pushed the simple-telemetry branch 5 times, most recently from cf80094 to b336bfd Compare August 17, 2026 22:45
kalvinnchau
kalvinnchau previously approved these changes Aug 18, 2026
@matt2e
matt2e marked this pull request as ready for review August 18, 2026 01:45
@matt2e
matt2e requested a review from a team August 18, 2026 01:45

@morgmart morgmart left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated code review

REQUEST_CHANGES. Fresh static review completed for the exact three-dot comparison bec8c6e...eac38e8. Two blocking consent/boundary defects and two non-blocking native-boundary hardening issues are publishable. The supplied GitHub evidence was inspected: DCO, frontend checks/tests, frontend build smoke, clean-room install, transcript virtualization, and Tauri Rust checks on macOS/Windows/Linux all report success; required checks still independently govern merge readiness. Final self-check covered every changed telemetry user flow and instrumentation surface, accessibility, English/Spanish localization, navigation and consent guards, async/error/never-completes/lifecycle/race behavior, test honesty, project laws and design-system rules, duplicate overlap, changed-line anchors, and concrete user effect. No supplied review threads exist, and overlapping boundary concerns were consolidated.

Deterministic publication result: 2 blocking and 2 non-blocking finding(s) publishable; 0 duplicate(s) suppressed.

Comment thread src-tauri/src/commands/telemetry.rs
Comment thread src-tauri/src/commands/telemetry.rs
Comment thread src-tauri/src/commands/telemetry.rs
Comment thread src-tauri/src/commands/telemetry.rs

@morgmart morgmart left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated code review

COMMENT. Fresh static review completed for the exact three-dot comparison bec8c6e...8d88edf. Two new non-blocking findings are publishable. One still-present native-boundary concern is suppressed forever because its resolved authoritative thread contains a substantive human deferral. The prior Windows replacement, mid-flight revocation, and alternate-port issues are fixed at this SHA and are not findings. Supplied GitHub evidence was inspected: DCO, frontend checks/tests, frontend build smoke, clean-room install, transcript virtualization, and Tauri Rust checks on Windows and macOS passed; Tauri Rust checks on Linux failed. CI remains separate from the code verdict and required checks independently govern merge readiness. Final self-check covered every changed app-launch, consent, native export, chat send/steer/deferred-send, agent, project, Home pin, onboarding, search, feedback, and berdctl flow; accessibility; English/Spanish localization; navigation and consent guards; async, error, never-completes, lifecycle, and race behavior; test honesty; design-system and architectural-law compliance; duplicate overlap; current changed-line anchors; and concrete evidence plus user effect for severity. Unsupported and overlapping claims were removed.

Deterministic publication result: 0 blocking and 2 non-blocking finding(s) publishable; 1 duplicate(s) suppressed.

const hasPersona = overridePersona
? overridePersona.id !== null
: Boolean(selectedPersonaId);
const provider = selectedProvider;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 P2 · Chat telemetry ignores the dispatched target (non-blocking)

Foreground send telemetry derives provider and model from live controller selection state at commit time. The queued attempt itself dispatches with the authoritative target captured in sendOptions.sessionSelection, and a later provider/model selection can update pendingExecutionTarget and selectedProvider while the admitted attempt is still preparing. The event can therefore report a different target than the message actually used.

User effect: Analytics can attribute a successfully sent message to the wrong provider or model, making provider/model adoption and reliability data misleading even though the chat itself uses the correct captured target.

Recommended fix: Pass the active attempt's captured sessionSelection into the commit telemetry callback and derive both provider and model from that single authoritative target, falling back to the committed session target only when no attempt snapshot exists.

Test: Add a controller test that captures a queued attempt on provider/model A, changes the pending selection to provider/model B before invoking onUserMessageCommitted, and asserts both chat events report A only.

Comment thread src/features/settings/ui/SystemSettings.tsx
morgmart
morgmart previously approved these changes Aug 18, 2026
@matt2e
matt2e enabled auto-merge (squash) August 18, 2026 06:38
@matt2e
matt2e disabled auto-merge August 18, 2026 06:38
Adds an opt-in telemetry stack that reports a small, hand-authored catalog
of product events over OTLP logs, gated behind a consent setting that is
off until the user turns it on.

- Renderer client batches and exports events, enforces consent at emit
  time, aborts in-flight exports when consent is revoked, counts events
  suppressed while consent is unsettled, and flushes when a window hides
  or closes.
- Native Tauri commands own consent persistence and enforcement, gateway
  token exchange (clamping the reported TTL, keeping a sibling's fresh
  token on 401 invalidation), and OTLP endpoint validation that rejects
  userinfo and explicit ports.
- Wire schema `berd-otlp-logs-v1` tags events with distribution.channel
  and carries no user_id, agent_id, project_id, or item_id. An inert
  distribution-sink seam sits on the emit path.
- Events cover the app, chat, agent, project, and home surfaces, with
  telemetry anchors contained so a throw never breaks the caller.
- Staging builds point at the live staging gateway and production builds
  at otel.berd.xyz; the settings consent row hides without the telemetry
  capability.
- `just dev` prints every fired event to the terminal and the dev event
  viewer renders those lines grey.

Also pins the OTel packages in lockstep at 0.221.0/2.10.0 and sherpa-onnx
at 1.12.40, and runs the app crate's telemetry tests in the tauri-test
gate.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Matt Toohey <contact@matttoohey.com>

@morgmart morgmart left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated code review

REQUEST_CHANGES. Fresh static review completed for the exact three-dot comparison 64979a0...0f7f0ef. One new blocking cross-platform build-integration finding is publishable. Two still-present concerns are suppressed by authoritative thread memory: one unresolved automated thread and one resolved thread with a substantive human deferral. The prior unavailable-consent UI issue is fixed at this SHA; the Windows settings replacement, mid-flight revocation, and alternate-port issues also remain fixed. Supplied GitHub evidence was inspected and all listed checks report success: DCO, frontend checks/tests, frontend build smoke, clean-room install, transcript virtualization, and Tauri Rust checks on Linux, Windows, and macOS. CI remains separate from the code verdict and required checks independently govern merge readiness. Final self-check covered every changed app-launch, consent, native export, chat send/steer/deferred-send, agent, project, Home pin, onboarding, search, feedback, and berdctl flow; accessibility; English/Spanish localization; navigation and consent guards; async, error, never-completes, lifecycle, and race behavior; test honesty; release/build integration; design-system and architectural-law compliance; duplicate overlap; current changed-line anchors; and concrete evidence plus user effect for blocking severity. The documented startup limitation of the runtime telemetry toggle was treated as an explicit tradeoff rather than invented as a finding.

Deterministic publication result: 1 blocking and 0 non-blocking finding(s) publishable; 2 duplicate(s) suppressed.

Comment thread src-tauri/Cargo.toml
@matt2e
matt2e disabled auto-merge August 18, 2026 07:16
@matt2e
matt2e enabled auto-merge (squash) August 18, 2026 07:16
@matt2e
matt2e requested review from kalvinnchau and morgmart August 18, 2026 07:17
@matt2e
matt2e disabled auto-merge August 18, 2026 07:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants