Skip to content

Reuse admitted authenticated responses before routing - #50

Open
mingley wants to merge 4 commits into
basecamp:mainfrom
mingley:perf/early-authenticated-cache
Open

mingley wants to merge 4 commits into
basecamp:mainfrom
mingley:perf/early-authenticated-cache

Conversation

@mingley

@mingley mingley commented Oct 21, 2026 •

Copy link
Copy Markdown

Warm authenticated cache hits currently rebuild the routed context and repeat authentication/resource queries before returning an already rendered body. Reuse authority captured by a successful pipeline to return an identical admitted GET before routing.

Warm-read results: 2.46× throughput vs old Rust, 62.5% more throughput vs C#, and 59.4% less CPU/request vs old Rust (geometric means). Rust leads C# in all eight measured warm-read cases. The newer mixed-write run also exposes an overload regression, detailed below.

Review order: #49 → #50. Review only this second change. The commits are stacked; both PRs target upstream main because GitHub requires the base branch upstream and these credentials cannot create it. Merge #49 first, then update this branch onto upstream main.

Old Rust vs new Rust vs C#

Three-run median requests/second. Old Rust: 93e2a90 (after the preparation optimizations). New Rust: b314160 (early authenticated cache). C#: 3286f07, ReadyToRun JIT. This head-to-head study predates the typed-cookie cleanup; it is not a fresh C# measurement of the current PR head.

Route (connections) Old Rust req/s New Rust req/s Rust gain C# req/s New Rust / C#
Room (16) 23,128 54,978 +137.7% 29,772 1.85×
Room (64) 24,938 58,906 +136.2% 40,749 1.45×
Messages (16) 22,917 65,985 +187.9% 36,333 1.82×
Messages (64) 23,908 64,186 +168.5% 40,957 1.57×
Sidebar (16) 28,220 66,998 +137.4% 38,420 1.74×
Sidebar (64) 28,597 74,278 +159.7% 50,790 1.46×
Search (16) 28,071 62,564 +122.9% 37,593 1.66×
Search (64) 30,097 68,041 +126.1% 45,030 1.51×

Rust throughput increased 122.9–187.9% vs old Rust and 44.6–84.7% vs C#. CPU/request fell 59.4% vs old Rust and 34.0% vs C# on geometric means. Median p99 was lower than both in every case. All 62,488,810 timed responses passed validation; no errors or invalid responses.

CPU and tail latency before/after
Route (connections) Rust CPU µs/request, old → new Rust p99 ms, old → new C# p99 ms
Room (16) 84.4 → 35.2 1.421 → 0.696 2.549
Room (64) 78.1 → 33.0 5.043 → 2.615 4.175
Messages (16) 84.4 → 29.4 1.478 → 0.582 1.399
Messages (64) 80.9 → 30.4 5.383 → 2.285 4.047
Sidebar (16) 69.2 → 29.0 1.240 → 0.597 1.326
Sidebar (64) 68.3 → 26.5 4.339 → 2.123 3.197
Search (16) 69.9 → 31.1 1.231 → 0.587 1.384
Search (64) 64.8 → 28.7 4.371 → 2.261 3.451

Separate diagnostic allocation runs measured 255.1 → 112.0 profiled objects/request (−56.1%) and 44,466 → 30,075 profiled bytes/request (−32.4%). These include background work and TTL expiry; they are not retained-memory figures or a comparable C# allocation census. CPU profiles show the routed dispatch/concerns work disappearing from the observed warm-hit stacks; SQLite generation observation remains.

Same seed, gzip, shared route/body contracts, two server CPUs and two separate generator CPUs; three alternating repetitions, 5-second warmups and 20-second windows. Release settings are unchanged. Cloud hardware differs from the published desktop/private harness. These are warm cached GET results, not a claim about every workload or language-wide superiority.

Why the C# port inspired this change

The recent devmobasa/once-campfire-dotnet port was initially faster because its PageCacheHitMiddleware, PageTicketMemo and PageAuthorityCache could return admitted encoded responses before routing. Rust still repeated its routed authentication and resource work before reaching its cache. This applies that architectural insight using Rust's existing verifier, immutable per-process secrets, SQLite observer and bounded response store; it does not copy C#'s dynamic key-ring mechanism.

Newer two-user and mixed-write results

This run compares preparation d7a6790 against the typed-cookie early-cache implementation d4ad287; the PR has identical Rust production/test sources. Two distinct signed-in users, 16 read connections, two alternating repetitions, 5-second warmups and 20-second windows, with the same CPU allocation. Every read passed the shared contracts; all returned write IDs, creators, rooms and body tokens were independently checked in SQLite.

  • Matched-rate reads: 2.35–2.58× preparation throughput. Writer latency did not consistently improve.
  • Room overload at a 100/s write cap: 79.76 → 60.48 achieved writes/s, with writer p99 25.17 → 36.59 ms. This is a regression; the read ratio is not a matched-write-load speedup and needs further contention/scheduling investigation.
  • Cold response keys, warm fragments: median latency improved on all four routes, with 64 requests/target/route. These are not cold-start or tail-latency results.
  • Validation: 15,238,266 timed reads and 14,453 durable writes; peak generator utilization 63.7%.
Complete two-user mixed-workload table

Rates are write caps, not guaranteed arrivals. Only rows whose achieved write rates match within 5% in each repetition support matched-rate read comparisons. Writes target a separate room; there are no Cable subscribers.

Route Write-rate limit/s Read req/s preparation → early Ratio Achieved write/s preparation / early Writer p99 ms preparation / early Achieved write rates
room_show 0 21,543 → 53,701 2.49× 0.00 / 0.00 — no writes
room_show 10 18,568 → 47,953 2.58× 9.89 / 9.89 16.02 / 24.82 within 5% each repetition
room_show 100 3,158 → 4,032 1.28× 79.76 / 60.48 25.17 / 36.59 unequal
sidebar 0 24,973 → 59,071 2.37× 0.00 / 0.00 — no writes
sidebar 10 25,925 → 60,888 2.35× 9.90 / 9.89 8.52 / 10.28 within 5% each repetition
sidebar 100 17,930 → 43,045 2.40× 90.61 / 90.71 8.48 / 8.03 within 5% each repetition

Authentication and behavior to review

  • Every early lookup observes SQLite, including foreign commits. Authority is bounded by database generation, signed-cookie validity, the existing 15-second TTL and the hourly activity deadline.
  • Admission rechecks the generation captured before authentication so delayed renders cannot carry old permissions into a new generation. Exact keys include repeated-header grouping and peer identity.
  • Encrypted session data/flash, HEAD, validators, bodies, ranges, upgrades and bypass directives use the ordinary pipeline.
  • Room preference effects use typed controller metadata and one cookie factory; unexpected or mismatched response cookies prevent admission. The existing bounded cache budget includes authority metadata.

Validation and evidence

805 workspace tests pass, 11 pre-existing ignored, with CAMPFIRE_REQUIRE_SEED=1 CAMPFIRE_REQUIRE_MEDIA_VECTORS=1; formatting, workspace/all-target Clippy with -D warnings and a normal release build pass. Tests cover revocation/token rotation, permission changes, foreign commits, delayed renders, expiry/activity boundaries, clock rollback, flash, variants and unexpected cookie effects.

Current cold/mixed report and reproduction commands · Pinned warm-read study, ranges, CPU/allocation profiles and limits.

@mingley
mingley marked this pull request as ready for review October 21, 2026 22:46
Copilot AI balanced review requested due to automatic review settings October 21, 2026 22:46

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Pre-routing authentication authority reuse and the documented write-overload regression require final human review.

No open issues 🎉

What changed in this PR

Building on #49, this PR lets identical authenticated GETs reuse completed responses before routing. Reused authority is bounded by SQLite generation, signed-cookie validity, cache lifetime and activity deadlines.

Changes:

  • Adds early response lookup with conservative eligibility checks and controlled cookie effects.
  • Extends cache metadata and regression coverage for authority invalidation, request variants and cookie behavior.
  • Adds reproducible workload benchmarks documenting read gains and write-overload tradeoffs.

[!TIP]
If you aren't ready for review, convert to a draft PR.
Click "Convert to draft" or run gh pr ready --undo.
Click "Ready for review" or run gh pr ready to reengage.

File Description
README.md Documents early authority reuse and fallback behavior.
plans/​cached-response-pr-validation.md Records validation, benchmarks and deployment limitations.
crates/​kit/​tests/​http.rs Tests panic handling after asynchronous suspension.
crates/​kit/​src/​request.rs Reduces authority-normalization allocations and adds tests.
crates/​kit/​src/​front/​handler.rs Supplies unit metadata for public cached responses.
crates/​kit/​src/​front/​cache.rs Supports application metadata within the cache budget.
crates/​kit/​src/​adapter.rs Removes redundant action-future boxing.
crates/​campfire/​src/​response_cache/​tests.rs Tests generation observation, authority boundaries and admission rules.
crates/​campfire/​src/​response_cache.rs Implements early lookup, bounded authority and response admission.
crates/​campfire/​src/​concerns.rs Records typed room-preference effects and centralizes cookie creation.
crates/​campfire/​src/​app/​tests/​response_cache.rs Tests early hits, credentials, cookies and request variants.
crates/​campfire/​src/​app.rs Wires early lookup and completed-response middleware.
bench/​README.md Links cold and mixed workload validation.
bench/​cached-workloads-report Validates receipts and summarizes benchmark results.
bench/​cached-workloads Runs multi-user reads, audited writes and cold-key measurements.

Review effort: Balanced

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants