Independent researcher and research software engineer focused on AI safety evaluation. I build reproducible evaluations, proposed fixes, and verification tools that support decisions on research and critical software:
- AI safety & evaluation benchmarks — In ControlArena #907, I found that an ampersand check passed whether Django escaped the character or left it unchanged. Another contributor reproduced that result on a SWE-bench image. Other work includes the ControlArena validator pilot, FACE-Eval
#1, a sensitivity analysis of debate control schemes, and Manifundfunding-bench#1 - Ethereum & ZK infrastructure — Optimism merged my fix for historical proofs, and Taiko adopted it in
alethia-reth. See also SP1#2950, Tycho#1486, revm#3833, TEMBRI#1 - Systems biology & scientific computing — HiGHS v1.16.0 includes the upstream fix for an incorrect optimum I reported. I also proposed changes to Tellurium
#611·#612and Diffrax#787 - C++ / Rust / GPU systems — SGLang
#42850, Kokkos#9122, LLVM#214016
Open to research grants, collaborations, and scoped engineering contracts. Corporate contracting is available through MIKOTO Co., Ltd..
Discuss a project · Research and evidence · Preprints · Company GitHub · ORCID
I contributed the following reports and tests to other people’s projects. Status checked 2026-11-14 (JST).
- ControlArena: AI-control input-validator pilot (reproducibility package) — Offline audit of 24 stored backdoor inputs across 10 sampled problems in an AI-control evaluation benchmark. One conflicted with a stated constraint also violated by two recorded test inputs: an unresolved specification/data conflict, not a confirmed scoring error. The audit was offline and did not implement sandbox isolation or test live attacks or scorer integration. The PR author responded with scope guidance; validator coverage and format policy remain maintainer decisions.
- ControlArena:
swe_djangoside-test behavioral probe (reproducibility package) — I used localpytesttests of reconstructed Django functions to examine two reported inversions (15103and16661). In15103, Python decodes\u0026to&, so theorcondition accepts both escaped and unescaped output. My probe also compared baseline and mutated behavior for16661. Another contributor ran the original reporter’s script on SWE-bench images and found that the ampersand check passed on both the gold and sabotaged code. The fix location and intended behavior for16661remain open. My probe covered neither the full main-task suite nor ControlArena’s scoring pipeline. - FACE-Eval: Inspect scorer missing-reasoning report (offline reproducer
.zip) — Source-level probe exercisingeval/inspect/judge.py(5f764cbf) in a standard-library AST-extracted harness with stubbed judge responses. With a valid cue-aligned answer, observedvcr=0.0anduar=1.0when reasoning is absent or its judge output fails JSON parsing. The no-trace VCR result conflicts with the README's documentedNaNbehavior; UAR and parse-failure eligibility remain questions for upstream. Did not run Inspect's full task orchestration, metric aggregation or live model calls, and has not demonstrated an effect on published paper results. No public reply on the linked issue as of 2026-11-14. - Debate control schemes: Phase 2 side-standardization sensitivity note — Methods note on
jackmaiorino/selvarath-debate(34898fb) separating the Phase 2 erratum's+4.56 ppworld-standardized question-side weighting from the+3.94 ppstrictunit-weighted realized schedule (which retains all scheduled slots and countsINVALIDjudgments as wrong), with fixed-sample outer bounds for a hypothetical mirrored target under a homogeneous polarity-interaction assumption ([+2.06, +5.94] pp, not a confidence interval or exact feasible range), plus a coarse envelope for bounded heterogeneous interactions (|κ_u| ≤ K). Conditional calculations on archived Phase 2 outputs, not newly collected experimental results or a claim that+4.56 ppis an arithmetic error. No public reply on the linked comment as of 2026-11-14.
Optimism (op-reth) #23334 — After a chain unwind and new appends, a request for a historical proof could return data from the chain tip. omerfirmak identified the cause and provided the original reproduction. I fixed how the surviving history shard is indexed and added regression tests. Local validation passed 822 tests across 17 packages. A maintainer saw 13 of the 17 new tests fail on the old code, approved the patch and merged it into develop on November 12.
Taiko pinned a commit containing the fix and closed its Hoodi issue. The fix prevents new damage. Existing damage requires a rebuild or time for affected blocks to leave the proof window. As of November 14, the op-reth/v2.9.0 tag points to code from before the merge and Taiko’s v1.5.1 release PR is open.
Manifund funding-bench #1 — Built an offline diagnostic and unit tests comparing how proposal attribution and scoring windows affect a synthetic Timaeus prediction. Production scoring and data are unchanged; this demonstrates sensitivity to the assumptions, not an improvement in actual funding decisions or model rankings.
Tellurium #611 · #612 (fixes #604, #603) — Proposed Python fixes for SBML/SED-ML biochemical network simulation workflows: preventing order-dependent model initialization from resetting parameter changes and preventing repeated tasks from overwriting standalone simulation outputs. Regressions cover XML ordering, the reported pathway simulation, nested tasks, and both fixes applied together.
Diffrax #787 (fixes #707) — Added construction-time checks for Brownian-path shapes to reject unsuitable static metadata before JIT tracing. Selected tests cover accepted NumPy integer dimensions and rejected array/tracer inputs; these are regression checks, not a complete statistical validation of the solvers.
SGLang inference features (two open PRs awaiting re-review)
- SGLang #43603 — Added hooks for model-weight placement before cache sizing and CUDA Graph capture. An external reviewer ran 45 unit tests on
4943ba3dand tested the hooks on a live server with an RTX PRO 6000. After rebasing, I reported 45 CPU tests passing on11b9e7d1and requested another review. The GPU results apply to the earlier revision. - SGLang #43597 — Added a way for models to refuse online weight updates before the server changes any weights. An external reviewer ran 31 unit tests and checked refusal on a live server. On the rebased patch,
cd7caec3, I reported 45 CPU tests passing and checked on one A40 that refused updates left the generated tokens and all 226 weight checksums unchanged.
Of the seven authored PRs featured here, Optimism #23334 is merged and six are open (checked 2026-11-14 JST).
These entries document my contributions to other people’s changes and my bug reports. Status checked 2026-11-14 (JST).
- HiGHS: incorrect optimality result — I reported a small mixed-integer problem for which the solver returned 0 although the exact optimum was 1. A maintainer confirmed the bug. Upstream merged the fix in #3408, closed the issue and shipped the corrected solver in v1.16.0. I tested the candidate build. A later agent-run comparison of the merged build and a reverted control reproduced the correction on the original and redundant-row examples. Exact checks covered the presolve-off API and MPS baselines.
- SGLang: speculative sampling kernel reduction — Synthetic-kernel measurements on one RTX 3070 identified ~23 µs of extra reduction work at batch size 1 on common paths; the PR author incorporated the proposed branch guard (response, upstream PR open). Not an end-to-end serving benchmark.
- Kokkos: host-side C++ Event race reproduction — Reproduced a host-side C++
SerialThreadSanitizer data race on the portableExperimental::Eventfallback across Clang and GCC, and diagnosed three Jenkins CI/NVCC build failures. - LLVM:
ExtTSPcode-layout differential testing — Differential tests across 2,001 synthetic CFGs and an 8-node exact-enumeration witness showing near-tie layout changes inmergeChainPairs; no generated-code runtime measurement. - SP1: GPU worker failure propagation — On one L40S, reproduced allocation panics and a client waiting until a configured 15-minute limit. Tested an existing development-branch error-handling approach on a pinned
mainbuild; clients then received terminal errors. OOM prevention and foreign-exception aborts were not resolved. - Tycho: delayed revert replay — Supplied three-stream staggered-lag tests, a remaining edge-case reproducer, and a candidate cache fix (upstream PR merged; that merge does not establish adoption of my candidate cache change).
Additional public reports and reviews
| Project | My contribution |
|---|---|
| Optimism proofs-history repair | At a maintainer’s request, I tested their proposed repair on two local dev-chain fixtures. In each, the proof-root mismatch count for one address fell from 200 of 2,151 blocks to zero. The probe compared the root node against each block header. The maintainer acknowledged the result and kept the repair PR closed. |
Taiko alethia-reth |
Agent-run checks passed all 271 trie library tests with the backport at Taiko’s dependency pin (7220729). In a separate run of the issue’s reproducer (9c14635), the patch corrected the nonce for block 1800 from 1801 to 1800. These were crate-level tests and a standalone reproducer. |
| revm | Nine-case library-level SELFDESTRUCT trace comparison. |
| TEMBRI | I reproduced a misplaced base_fee_per_gas read across four public Xatu partitions with two Parquet layouts. Upstream changed the reader to look up the field by name and added tests for both layouts. The project’s change notes credit me for reporting the issue, which remains open. |
| vLLM | Concurrent structured-output comparisons across model-runner configurations. |
| cosign | Reproduced a reviewer’s finding that a caller-supplied signature verifier was bypassed and supplied a regression test. |
| SPIRE | I reproduced a cached-SVID failure after sequential registration updates and proposed TestLRUCacheSequentialIDChangeWithoutRenewal. The v1.16.0 test suite includes a test with the same name and update sequence. |
Research preprints published on Figshare under CC BY 4.0 (not peer reviewed). Complete index of all 11 preprints and 6 companion archives (17 version-pinned DOIs): PUBLICATIONS.md.
- Contract-Indexed State Reduction and Retirement in Audit Machines: A Certificate-Checked RTL Study (
10.6084/m9.figshare.35069368.v2, v2, 2026-11-20; 29-page PDF and 68-file supporting archive) — Evaluates contract-preserving finite-state quotients, five-requirement dead-variable register retirement under partial synchronous reset, four RTL state-encoding families underYosys 0.33(nofsm), normalized 22-pair equivalence search withSEC-ic3-sim, standaloneCertifaiger 10.4.1+lrat-trim1-stepLRATproofs, and a runtimeYosys- and SAT-solver-free plaintext acceptance checker across a 72-state reference audit machine, a parameterized slot family, and a fixed smallGemmini DMACommandTrackerconfiguration (nCmds = 2, maxBytes = 3, tag_t = UInt(1.W)). Conditioned on the declared finite models, pinned tool commits, and timeouts; makes no post-layout commercial ASIC claims, cryptographic zero-knowledge claims, or upstreamGemminibug claims. - Task-Preserving Operational Processes with Bounded Local Memory (
10.6084/m9.figshare.34020717.v1, v1, 2026-09-29; preprint PDF and supporting archive) — Companion theoretical preprint formulating contract-equivalence quotients, 1-step factored correspondence, five-requirement scoped dead-variable retirement, and conditional resident-memory and identifier-width bounds.
ControlArena validator pilot (#755 / #899) · ControlArena #907 side-test probe · FACE-Eval #1 Inspect scorer probe (.zip attached to the issue) · revm reproduction bundle · HiGHS exact checker · SGLang kernel measurements · Uniswap v3 replay checker (synthetic demo)
AI tools assisted implementation, experiments, analysis, and writing. Linked reports record the tested inputs, environment, results, and limitations; upstream responses are distinguished from local checks.