Automate RowFn benchmark comparisons

[connortsui20]

Rationale for this change

Makes RowFn performance comparisons repeatable across revisions. The runner records enough build and machine state to reject stale benchmark artifacts.

• Depends on: Execute tensor product functions with RowFn #9348
• Progress towards: Epic: Row-oriented scalar functions #9128

What changes are included in this PR?

Adds separate build and measurement phases, reusable revision-specific target directories, package-grouped parallel builds, alternating process order, machine records, and paired median summaries. The primary profile uses one codegen unit, fat LTO, target-cpu=native, two warm runs, and seven measured pairs.

Focused executor benchmarks and script tests cover framework overhead, strict validity, result parsing, changing inventories, and ratio aggregation. Reused artifacts are validated against the revision, dirty-state digest, Rust and Cargo versions, build configuration, suite inventory, and executable hash.

What APIs are changed? Are there any user-facing changes?

There are no public Rust API changes. This PR adds the developer-facing scripts/benchmark-rowfn.sh command and its result format.

[codspeed-hq]

Merging this PR will regress 1 benchmark

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚡ 91 improved benchmarks
❌ 1 regressed benchmark
✅ 1885 untouched benchmarks
🆕 19 new benchmarks
⏩ 478 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	WallTime	words_gather_scalar[65536]	8.3 µs	9.4 µs	-11.85%
⚡	Simulation	or_chain[1024]	2.4 ms	1.5 ms	+61.83%
⚡	Simulation	or_chain[16]	2.3 ms	1.4 ms	+60.63%
⚡	Simulation	add_i64_nullable	320.4 µs	199.5 µs	+60.6%
⚡	Simulation	mul_i32_nullable	233.3 µs	148.4 µs	+57.24%
⚡	Simulation	mul_u8_nonnull	102.2 µs	65.1 µs	+57.04%
⚡	Simulation	case_when_nary_10_conditions[1000]	606.6 µs	393.5 µs	+54.14%
⚡	Simulation	int_gt[16]	157.6 µs	111.5 µs	+41.31%
⚡	Simulation	int_gt[1024]	164.9 µs	118.4 µs	+39.34%
⚡	Simulation	case_when_nary_equality_lookup[1000]	424.1 µs	306 µs	+38.58%
⚡	Simulation	mul_i32_constant	130 µs	94.9 µs	+36.91%
⚡	Simulation	or_chain[8192]	3.7 ms	2.7 ms	+36.8%
⚡	Simulation	baseline_lt[4, 1024]	131.2 µs	97.5 µs	+34.51%
⚡	Simulation	case_when_nary_10_conditions[10000]	984.4 µs	734.5 µs	+34.01%
⚡	Simulation	int_gt[8192]	225.6 µs	169.5 µs	+33.1%
⚡	Simulation	case_when_nary_early_dominant[1000]	295.5 µs	224.5 µs	+31.64%
⚡	Simulation	baseline_eq[4, 1024]	127.4 µs	96.9 µs	+31.52%
⚡	Simulation	case_when_nary_3_conditions[1000]	297.5 µs	227.1 µs	+30.98%
⚡	Simulation	baseline_lt[16, 1024]	128.7 µs	98.7 µs	+30.41%
⚡	Simulation	baseline_eq[16, 1024]	128.6 µs	98.8 µs	+30.09%
...	...	...	...	...	...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing ct/row-fn-benchmark-tools (d9c3394) with ct/row-fn-tensor-products (c1382ba)²}

Footnotes

1. 478 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

2. No successful run was found on ct/row-fn-tensor-products (7152a1c) during the generation of this report, so 597702a was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[connortsui20]

Here are the local results that exercise the benchmark runner and its unchanged control suites.

The full-stack comparison used a develop baseline with benchmark-only backfills (1edd0fca) and the full RowFn stack (8ca9fc73). The machine was an AMD Ryzen 9 7950X running Linux. The build used rustc 1.97.1, LLVM 22.1.6, one CGU, fat LTO, and -C target-cpu=native.

The full comparison used two warm runs and seven alternating measured pairs. Each process used 100 samples with a 0.25–0.5 second measurement window on CPU 4. Negative changes are faster. The change is the median paired ratio, not the ratio of the two displayed medians.

This PR does not change these production paths. The control binaries still show layout-sensitive movement, including 14.72–17.21% regressions for list take/filter and a 15.66% improvement for per-row like. The focused cleanup comparison keeps RowFn executor cases within 0.93% and strict-validity cases within 1.64%.

RowFn executor cleanup comparison: 8 cases

This focused screen compared the pre-cleanup and post-cleanup framework with three measured pairs.

Benchmark	Before cleanup	After cleanup	Change
row_wrapping_add_nullable	139.400 µs	140.700 µs	+0.93%
row_checked_add_constant	33.110 µs	33.260 µs	+0.45%
row_checked_add_nullable	137.400 µs	138.200 µs	+0.29%
row_wrapping_add	137.100 µs	137.600 µs	+0.22%
row_wrapping_add_constant	31.780 µs	31.870 µs	+0.16%
row_sink_wrapping_add	136.300 µs	136.600 µs	+0.00%
handrolled_sink_wrapping_add	136.300 µs	135.200 µs	-0.15%
row_checked_add	138.000 µs	137.600 µs	-0.36%

Strict-validity cleanup comparison: 8 cases

These promoted cases used seven alternating measured pairs.

Benchmark	Before cleanup	After cleanup	Change
lazy_chain/1048576	1.317 ms	1.312 ms	+0.61%
eager_chain/1048576	1.304 ms	1.311 ms	+0.53%
eager/1048576	702.200 µs	711.100 µs	+0.14%
eager/65536	65.920 µs	65.760 µs	-0.24%
lazy_chain/65536	16.570 µs	16.580 µs	-0.30%
lazy/65536	5.419 µs	5.379 µs	-1.28%
lazy/1048576	616.000 µs	608.400 µs	-1.57%
eager_chain/65536	16.600 µs	16.160 µs	-1.64%

Like controls: 7 cases

Benchmark	Baseline	RowFn stack	Change
like_prefix	1.669 µs	1.889 µs	+13.18%
like_contains	12.700 µs	14.170 µs	+12.30%
ilike_contains	79.550 µs	80.800 µs	+1.47%
like_regex	44.580 µs	44.770 µs	+0.65%
like_exact	2.149 µs	2.139 µs	+0.00%
like_suffix	3.079 µs	3.059 µs	-0.65%
like_per_row_patterns	16.800 µs	14.160 µs	-15.66%

Take/filter controls: 17 cases

Benchmark	Baseline	RowFn stack	Change
take_filter_list_small_random_mask_random_indices/768/10	4.299 µs	5.039 µs	+17.21%
take_filter_list_slice_mask_random_indices/768/50	4.649 µs	5.379 µs	+16.13%
take_filter_list_slice_mask_sequential_indices/768/50	4.499 µs	5.179 µs	+15.96%
take_filter_list_nullable_random_mask_random_indices/768/50	5.009 µs	5.789 µs	+15.57%
take_filter_list_random_mask_random_indices/768/50	4.639 µs	5.349 µs	+15.52%
take_filter_list_small_uncached_random_mask_random_indices/768/10	4.769 µs	5.509 µs	+15.52%
take_filter_list_random_mask_sequential_indices/768/50	4.619 µs	5.259 µs	+14.72%
take_filter_primitive_small_random_mask_random_indices/16384/10	959.7 ns	949.7 ns	+1.93%
take_filter_primitive_slice_mask_random_indices/16384/1000	1.399 µs	1.429 µs	+1.43%
take_filter_primitive_slice_mask_sequential_indices/16384/1000	1.014 µs	1.019 µs	+0.93%
take_filter_primitive_random_mask_sequential_indices/16384/1000	1.359 µs	1.359 µs	+0.74%
take_filter_primitive_random_mask_random_indices/16384/1000	1.619 µs	1.639 µs	+0.61%
take_filter_primitive_small_uncached_random_mask_random_indices/16384/10	969.7 ns	954.7 ns	-0.53%
take_filter_primitive_large_random_mask_random_indices/(12500, 25000)	18.540 µs	18.460 µs	-0.59%
take_filter_primitive_nullable_slice_mask_random_indices/16384/1000	1.949 µs	1.919 µs	-1.03%
take_filter_primitive_nullable_random_mask_random_indices/16384/1000	2.019 µs	1.989 µs	-1.49%
take_filter_primitive_large_random_mask_random_indices/(2500, 25000)	11.090 µs	10.920 µs	-1.53%

VarBinView compaction controls: 8 cases

Benchmark	Baseline	RowFn stack	Change
compact_sliced/(16384, 10)	64.080 µs	70.030 µs	+9.29%
compact/(4096, 10)	49.960 µs	50.080 µs	+0.68%
compact/(16384, 10)	254.700 µs	254.300 µs	-0.16%
compact_sliced/(4096, 10)	16.250 µs	16.240 µs	-0.18%
compact/(16384, 90)	9.150 ns	8.993 ns	-1.72%
compact/(4096, 90)	8.681 ns	8.525 ns	-1.80%
compact_sliced/(4096, 90)	8.915 ns	8.525 ns	-1.80%
compact_sliced/(16384, 90)	9.540 ns	9.032 ns	-2.55%