feat(telemetry): add the OpenTelemetry pipeline and core Berd events - #16
feat(telemetry): add the OpenTelemetry pipeline and core Berd events#16matt2e wants to merge 1 commit into
Conversation
cf80094 to
b336bfd
Compare
morgmart
left a comment
There was a problem hiding this comment.
🤖 Automated code review
REQUEST_CHANGES. Fresh static review completed for the exact three-dot comparison bec8c6e...eac38e8. Two blocking consent/boundary defects and two non-blocking native-boundary hardening issues are publishable. The supplied GitHub evidence was inspected: DCO, frontend checks/tests, frontend build smoke, clean-room install, transcript virtualization, and Tauri Rust checks on macOS/Windows/Linux all report success; required checks still independently govern merge readiness. Final self-check covered every changed telemetry user flow and instrumentation surface, accessibility, English/Spanish localization, navigation and consent guards, async/error/never-completes/lifecycle/race behavior, test honesty, project laws and design-system rules, duplicate overlap, changed-line anchors, and concrete user effect. No supplied review threads exist, and overlapping boundary concerns were consolidated.
Deterministic publication result: 2 blocking and 2 non-blocking finding(s) publishable; 0 duplicate(s) suppressed.
morgmart
left a comment
There was a problem hiding this comment.
🤖 Automated code review
COMMENT. Fresh static review completed for the exact three-dot comparison bec8c6e...8d88edf. Two new non-blocking findings are publishable. One still-present native-boundary concern is suppressed forever because its resolved authoritative thread contains a substantive human deferral. The prior Windows replacement, mid-flight revocation, and alternate-port issues are fixed at this SHA and are not findings. Supplied GitHub evidence was inspected: DCO, frontend checks/tests, frontend build smoke, clean-room install, transcript virtualization, and Tauri Rust checks on Windows and macOS passed; Tauri Rust checks on Linux failed. CI remains separate from the code verdict and required checks independently govern merge readiness. Final self-check covered every changed app-launch, consent, native export, chat send/steer/deferred-send, agent, project, Home pin, onboarding, search, feedback, and berdctl flow; accessibility; English/Spanish localization; navigation and consent guards; async, error, never-completes, lifecycle, and race behavior; test honesty; design-system and architectural-law compliance; duplicate overlap; current changed-line anchors; and concrete evidence plus user effect for severity. Unsupported and overlapping claims were removed.
Deterministic publication result: 0 blocking and 2 non-blocking finding(s) publishable; 1 duplicate(s) suppressed.
| const hasPersona = overridePersona | ||
| ? overridePersona.id !== null | ||
| : Boolean(selectedPersonaId); | ||
| const provider = selectedProvider; |
There was a problem hiding this comment.
🤖 P2 · Chat telemetry ignores the dispatched target (non-blocking)
Foreground send telemetry derives provider and model from live controller selection state at commit time. The queued attempt itself dispatches with the authoritative target captured in sendOptions.sessionSelection, and a later provider/model selection can update pendingExecutionTarget and selectedProvider while the admitted attempt is still preparing. The event can therefore report a different target than the message actually used.
User effect: Analytics can attribute a successfully sent message to the wrong provider or model, making provider/model adoption and reliability data misleading even though the chat itself uses the correct captured target.
Recommended fix: Pass the active attempt's captured sessionSelection into the commit telemetry callback and derive both provider and model from that single authoritative target, falling back to the committed session target only when no attempt snapshot exists.
Test: Add a controller test that captures a queued attempt on provider/model A, changes the pending selection to provider/model B before invoking onUserMessageCommitted, and asserts both chat events report A only.
Adds an opt-in telemetry stack that reports a small, hand-authored catalog of product events over OTLP logs, gated behind a consent setting that is off until the user turns it on. - Renderer client batches and exports events, enforces consent at emit time, aborts in-flight exports when consent is revoked, counts events suppressed while consent is unsettled, and flushes when a window hides or closes. - Native Tauri commands own consent persistence and enforcement, gateway token exchange (clamping the reported TTL, keeping a sibling's fresh token on 401 invalidation), and OTLP endpoint validation that rejects userinfo and explicit ports. - Wire schema `berd-otlp-logs-v1` tags events with distribution.channel and carries no user_id, agent_id, project_id, or item_id. An inert distribution-sink seam sits on the emit path. - Events cover the app, chat, agent, project, and home surfaces, with telemetry anchors contained so a throw never breaks the caller. - Staging builds point at the live staging gateway and production builds at otel.berd.xyz; the settings consent row hides without the telemetry capability. - `just dev` prints every fired event to the terminal and the dev event viewer renders those lines grey. Also pins the OTel packages in lockstep at 0.221.0/2.10.0 and sherpa-onnx at 1.12.40, and runs the app crate's telemetry tests in the tauri-test gate. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Matt Toohey <contact@matttoohey.com>
f0b4899 to
0f7f0ef
Compare
morgmart
left a comment
There was a problem hiding this comment.
🤖 Automated code review
REQUEST_CHANGES. Fresh static review completed for the exact three-dot comparison 64979a0...0f7f0ef. One new blocking cross-platform build-integration finding is publishable. Two still-present concerns are suppressed by authoritative thread memory: one unresolved automated thread and one resolved thread with a substantive human deferral. The prior unavailable-consent UI issue is fixed at this SHA; the Windows settings replacement, mid-flight revocation, and alternate-port issues also remain fixed. Supplied GitHub evidence was inspected and all listed checks report success: DCO, frontend checks/tests, frontend build smoke, clean-room install, transcript virtualization, and Tauri Rust checks on Linux, Windows, and macOS. CI remains separate from the code verdict and required checks independently govern merge readiness. Final self-check covered every changed app-launch, consent, native export, chat send/steer/deferred-send, agent, project, Home pin, onboarding, search, feedback, and berdctl flow; accessibility; English/Spanish localization; navigation and consent guards; async, error, never-completes, lifecycle, and race behavior; test honesty; release/build integration; design-system and architectural-law compliance; duplicate overlap; current changed-line anchors; and concrete evidence plus user effect for blocking severity. The documented startup limitation of the runtime telemetry toggle was treated as an explicit tradeoff rather than invented as a finding.
Deterministic publication result: 1 blocking and 0 non-blocking finding(s) publishable; 2 duplicate(s) suppressed.
Summary
Berd had no product analytics. This adds the full renderer-to-gateway telemetry path plus the first catalog of Berd events, instrumented at the points where each user action actually commits rather than where it is merely requested.
Pipeline
src/shared/telemetry(client.ts,exporter.ts,identity.ts) batches records and hands the serialized OTLP/HTTP JSON body to the Tauri layer, so no renderer code holds a credential or an endpoint.src-tauri/src/commands/telemetry.rsowns auth and transport: anonymousPOST /v1/bootstrapwith a validated installation id, token TTL read from the response, a single 401 re-auth retry, andPOST /v1/logscarryingx-berd-schema-version: berd-otlp-logs-v1against the gateway'sberdservice andberd.telemetryscope.developmentnever reaches the wire, startup gates are bounded so a stalled bootstrap cannot wedge the app, dropped records are counted, and batch size and attribute lengths are capped under the gateway's body limit. Empty optional attributes are omitted rather than sent as empty values..invalidhost in the real endpoint's shape. Swapping in a real host is pure configuration —VITE_OTLP_LOGS_ENDPOINT,ALLOWED_OTEL_LOGS_HOSTS, the CSPconnect-src, and the pinned test values — with no code-path changes (see theTODO(telemetry-otel)invite.config.ts).Events
src/shared/telemetry/eventshas one module per surface, covering app launch, chat sends, agent edits, home pins, and project mutations. Each is anchored to its commit point:is_first_messageis gated on the session history having replayed.Edit Completedis anchored to the persisted persona write, and agent creation is counted from the duplicate, import, and onboarding surfaces.Ported from
squareup/berd@5e28e6e1, adapted to this repository's drift (worktree startup modes, pre-send workspace setup) with the telemetry behavior and wire contract intact.Related issue
None found — searched open and closed issues/PRs for
telemetry, no duplicates.Testing
Tests accompany each surface, pinning the wire body contract (
exporter.test.ts,client.test.ts), the event attribute shapes (events.test.ts,homePinTelemetry.test.ts), and the commit-point behavior (useChatSessionController.test.ts,AgentBuilderRail.test.tsx,AgentsView.entry.test.tsx,usePinToHomeWidget.test.tsx,HomeView.test.tsx,OnboardingFlow.test.tsx,main.test.tsx).Relevant gates for a reviewer to run:
just check,just test,just tauri-check,just clippy. No user-visible UI change, so no screenshots.