Skip to content

Store hot and cold random access benchmark results - #9649

Merged
lwwmanning merged 4 commits into
developfrom
wm/random-access-hot-cold
Aug 27, 2026
Merged

Store hot and cold random access benchmark results#9649
lwwmanning merged 4 commits into
developfrom
wm/random-access-hot-cold

Conversation

@lwwmanning

Copy link
Copy Markdown
Contributor

Summary

Store both cached-accessor and per-take reopen timings from the random access benchmark. The benchmark already runs both modes, but v3 reporting discards reopen results. This prevents the benchmark site from comparing the two access paths.

Changes

Add open_mode to random-access v3 records and Postgres ingestion. Cached records preserve historical measurement IDs. Reopen records use distinct IDs. Focused producer, snapshot, and measurement-ID tests cover the field.

Signed-off-by: Will Manning <will@willmanning.io>
…ot-cold

Signed-off-by: Will Manning <will@willmanning.io>
@codspeed-hq

codspeed-hq Bot commented Aug 27, 2026

Copy link
Copy Markdown

Merging this PR will regress 1 benchmark

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 2 improved benchmarks
❌ 1 regressed benchmark
✅ 2136 untouched benchmarks
⏩ 106 skipped benchmarks1
🗄️ 4 archived benchmarks run2

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation cold_misaligned[(16, 64)] 345.6 µs 392.8 µs -12.02%
WallTime arrow_checked_add_u32_avx2[16384] 21.3 µs 17.7 µs +20.38%
WallTime mul_u32_nonnull_avx512 6.3 µs 5.6 µs +12.03%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing wm/random-access-hot-cold (614e43f) with develop (f880575)

Open in CodSpeed

Footnotes

  1. 106 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

  2. 4 benchmarks were run, but are now archived. If they were deleted in another branch, consider rebasing to remove them from the report. Instead if they were added back, click here to restore them.

@lwwmanning
lwwmanning marked this pull request as ready for review August 27, 2026 16:57

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 53c97b4dfa

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread benchmarks/random-access-bench/src/lib.rs
Comment thread scripts/post-ingest.py
Comment thread benchmarks/random-access-bench/src/lib.rs
…ot-cold

Signed-off-by: Will Manning <will@willmanning.io>
Signed-off-by: Will Manning <will@willmanning.io>
@lwwmanning
lwwmanning enabled auto-merge (squash) August 27, 2026 19:17

@connortsui20 connortsui20 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this seems fine, unclear how exactly this links to vortex-data/benchmarks-website#26 but I guess we will see

@lwwmanning
lwwmanning merged commit ba90f3d into develop Aug 27, 2026
114 of 116 checks passed
@lwwmanning
lwwmanning deleted the wm/random-access-hot-cold branch August 27, 2026 19:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants