Skip to content
View akatsukiwork0320's full-sized avatar

Highlights

  • Pro

Block or report akatsukiwork0320

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
akatsukiwork0320/README.md

Satoshi Kawasaki

Independent researcher and research software engineer focused on AI safety evaluation. I build reproducible evaluations, proposed fixes, and verification tools that support decisions on research and critical software:

Open to research grants, collaborations, and scoped engineering contracts. Corporate contracting is available through MIKOTO Co., Ltd..

Discuss a project · Research and evidence · Preprints · Company GitHub · ORCID

AI safety: selected reports and independent validation

I contributed the following reports and tests to other people’s projects. Status checked 2026-11-14 (JST).

  • ControlArena: AI-control input-validator pilot (reproducibility package) — Offline audit of 24 stored backdoor inputs across 10 sampled problems in an AI-control evaluation benchmark. One conflicted with a stated constraint also violated by two recorded test inputs: an unresolved specification/data conflict, not a confirmed scoring error. The audit was offline and did not implement sandbox isolation or test live attacks or scorer integration. The PR author responded with scope guidance; validator coverage and format policy remain maintainer decisions.
  • ControlArena: swe_django side-test behavioral probe (reproducibility package) — I used local pytest tests of reconstructed Django functions to examine two reported inversions (15103 and 16661). In 15103, Python decodes \u0026 to &, so the or condition accepts both escaped and unescaped output. My probe also compared baseline and mutated behavior for 16661. Another contributor ran the original reporter’s script on SWE-bench images and found that the ampersand check passed on both the gold and sabotaged code. The fix location and intended behavior for 16661 remain open. My probe covered neither the full main-task suite nor ControlArena’s scoring pipeline.
  • FACE-Eval: Inspect scorer missing-reasoning report (offline reproducer .zip) — Source-level probe exercising eval/inspect/judge.py (5f764cbf) in a standard-library AST-extracted harness with stubbed judge responses. With a valid cue-aligned answer, observed vcr=0.0 and uar=1.0 when reasoning is absent or its judge output fails JSON parsing. The no-trace VCR result conflicts with the README's documented NaN behavior; UAR and parse-failure eligibility remain questions for upstream. Did not run Inspect's full task orchestration, metric aggregation or live model calls, and has not demonstrated an effect on published paper results. No public reply on the linked issue as of 2026-11-14.
  • Debate control schemes: Phase 2 side-standardization sensitivity note — Methods note on jackmaiorino/selvarath-debate (34898fb) separating the Phase 2 erratum's +4.56 pp world-standardized question-side weighting from the +3.94 pp strict unit-weighted realized schedule (which retains all scheduled slots and counts INVALID judgments as wrong), with fixed-sample outer bounds for a hypothetical mirrored target under a homogeneous polarity-interaction assumption ([+2.06, +5.94] pp, not a confidence interval or exact feasible range), plus a coarse envelope for bounded heterogeneous interactions (|κ_u| ≤ K). Conditional calculations on archived Phase 2 outputs, not newly collected experimental results or a claim that +4.56 pp is an arithmetic error. No public reply on the linked comment as of 2026-11-14.

Selected authored pull requests

Ethereum infrastructure

Optimism (op-reth) #23334 — After a chain unwind and new appends, a request for a historical proof could return data from the chain tip. omerfirmak identified the cause and provided the original reproduction. I fixed how the surviving history shard is indexed and added regression tests. Local validation passed 822 tests across 17 packages. A maintainer saw 13 of the 17 new tests fail on the old code, approved the patch and merged it into develop on November 12.

Taiko pinned a commit containing the fix and closed its Hoodi issue. The fix prevents new damage. Existing damage requires a rebuild or time for affected blocks to leave the proof window. As of November 14, the op-reth/v2.9.0 tag points to code from before the merge and Taiko’s v1.5.1 release PR is open.

AI evaluation and funding benchmarks

Manifund funding-bench #1 — Built an offline diagnostic and unit tests comparing how proposal attribution and scoring windows affect a synthetic Timaeus prediction. Production scoring and data are unchanged; this demonstrates sensitivity to the assumptions, not an improvement in actual funding decisions or model rankings.

Systems biology and scientific computing

Tellurium #611 · #612 (fixes #604, #603) — Proposed Python fixes for SBML/SED-ML biochemical network simulation workflows: preventing order-dependent model initialization from resetting parameter changes and preventing repeated tasks from overwriting standalone simulation outputs. Regressions cover XML ordering, the reported pathway simulation, nested tasks, and both fixes applied together.

Diffrax #787 (fixes #707) — Added construction-time checks for Brownian-path shapes to reject unsuitable static metadata before JIT tracing. Selected tests cover accepted NumPy integer dimensions and rejected array/tracer inputs; these are regression checks, not a complete statistical validation of the solvers.

SGLang inference features (two open PRs awaiting re-review)

Of the seven authored PRs featured here, Optimism #23334 is merged and six are open (checked 2026-11-14 JST).

Other selected reports and independent validation

These entries document my contributions to other people’s changes and my bug reports. Status checked 2026-11-14 (JST).

  • HiGHS: incorrect optimality result — I reported a small mixed-integer problem for which the solver returned 0 although the exact optimum was 1. A maintainer confirmed the bug. Upstream merged the fix in #3408, closed the issue and shipped the corrected solver in v1.16.0. I tested the candidate build. A later agent-run comparison of the merged build and a reverted control reproduced the correction on the original and redundant-row examples. Exact checks covered the presolve-off API and MPS baselines.
  • SGLang: speculative sampling kernel reduction — Synthetic-kernel measurements on one RTX 3070 identified ~23 µs of extra reduction work at batch size 1 on common paths; the PR author incorporated the proposed branch guard (response, upstream PR open). Not an end-to-end serving benchmark.
  • Kokkos: host-side C++ Event race reproduction — Reproduced a host-side C++ Serial ThreadSanitizer data race on the portable Experimental::Event fallback across Clang and GCC, and diagnosed three Jenkins CI/NVCC build failures.
  • LLVM: ExtTSP code-layout differential testing — Differential tests across 2,001 synthetic CFGs and an 8-node exact-enumeration witness showing near-tie layout changes in mergeChainPairs; no generated-code runtime measurement.
  • SP1: GPU worker failure propagation — On one L40S, reproduced allocation panics and a client waiting until a configured 15-minute limit. Tested an existing development-branch error-handling approach on a pinned main build; clients then received terminal errors. OOM prevention and foreign-exception aborts were not resolved.
  • Tycho: delayed revert replay — Supplied three-stream staggered-lag tests, a remaining edge-case reproducer, and a candidate cache fix (upstream PR merged; that merge does not establish adoption of my candidate cache change).
Additional public reports and reviews
Project My contribution
Optimism proofs-history repair At a maintainer’s request, I tested their proposed repair on two local dev-chain fixtures. In each, the proof-root mismatch count for one address fell from 200 of 2,151 blocks to zero. The probe compared the root node against each block header. The maintainer acknowledged the result and kept the repair PR closed.
Taiko alethia-reth Agent-run checks passed all 271 trie library tests with the backport at Taiko’s dependency pin (7220729). In a separate run of the issue’s reproducer (9c14635), the patch corrected the nonce for block 1800 from 1801 to 1800. These were crate-level tests and a standalone reproducer.
revm Nine-case library-level SELFDESTRUCT trace comparison.
TEMBRI I reproduced a misplaced base_fee_per_gas read across four public Xatu partitions with two Parquet layouts. Upstream changed the reader to look up the field by name and added tests for both layouts. The project’s change notes credit me for reporting the issue, which remains open.
vLLM Concurrent structured-output comparisons across model-runner configurations.
cosign Reproduced a reviewer’s finding that a caller-supplied signature verifier was bypassed and supplied a regression test.
SPIRE I reproduced a cached-SVID failure after sequential registration updates and proposed TestLRUCacheSequentialIDChangeWithoutRenewal. The v1.16.0 test suite includes a test with the same name and update sequence.

Selected research preprints

Research preprints published on Figshare under CC BY 4.0 (not peer reviewed). Complete index of all 11 preprints and 6 companion archives (17 version-pinned DOIs): PUBLICATIONS.md.

  • Contract-Indexed State Reduction and Retirement in Audit Machines: A Certificate-Checked RTL Study (10.6084/m9.figshare.35069368.v2, v2, 2026-11-20; 29-page PDF and 68-file supporting archive) — Evaluates contract-preserving finite-state quotients, five-requirement dead-variable register retirement under partial synchronous reset, four RTL state-encoding families under Yosys 0.33 (nofsm), normalized 22-pair equivalence search with SEC-ic3-sim, standalone Certifaiger 10.4.1 + lrat-trim 1-step LRAT proofs, and a runtime Yosys- and SAT-solver-free plaintext acceptance checker across a 72-state reference audit machine, a parameterized slot family, and a fixed small Gemmini DMACommandTracker configuration (nCmds = 2, maxBytes = 3, tag_t = UInt(1.W)). Conditioned on the declared finite models, pinned tool commits, and timeouts; makes no post-layout commercial ASIC claims, cryptographic zero-knowledge claims, or upstream Gemmini bug claims.
  • Task-Preserving Operational Processes with Bounded Local Memory (10.6084/m9.figshare.34020717.v1, v1, 2026-09-29; preprint PDF and supporting archive) — Companion theoretical preprint formulating contract-equivalence quotients, 1-step factored correspondence, five-requirement scoped dead-variable retirement, and conditional resident-memory and identifier-width bounds.

Reusable research artifacts

ControlArena validator pilot (#755 / #899) · ControlArena #907 side-test probe · FACE-Eval #1 Inspect scorer probe (.zip attached to the issue) · revm reproduction bundle · HiGHS exact checker · SGLang kernel measurements · Uniswap v3 replay checker (synthetic demo)

AI tools assisted implementation, experiments, analysis, and writing. Linked reports record the tested inputs, environment, results, and limitations; upstream responses are distinguished from local checks.

Pinned Loading

  1. control-arena-validator-pilot-repro control-arena-validator-pilot-repro Public

    Offline 10-problem, 24-record secret-input validator pilot and specification audit for ControlArena APPS dataset (#755 / #899).

    Python

  2. revm-selfdestruct-trace-repro revm-selfdestruct-trace-repro Public

    Pinned library-level reproduction of revm #3833: SELFDESTRUCT trace payloads, filter matches, reference outputs and an independent checker.

    Python

  3. highs-analytic-center-repro highs-analytic-center-repro Public

    Exact verification and an 11-variable reproducer for a HiGHS 1.15.1 Optimal 0 result on a MIP with exact optimum 1.

    Python

  4. sglang-speculative-sampling-repro sglang-speculative-sampling-repro Public

    Reproduction bundle for SGLang speculative sampling fallback: pinned sources, synthetic GPU measurements, raw results and a conditional-reduction patch.

  5. control-arena-907-swebench-probe control-arena-907-swebench-probe Public

    Local pytest behavioral probe for ControlArena #907 (swe_django side tests 15103 and 16661).

    Python

  6. ethereum-optimism/optimism ethereum-optimism/optimism Public

    Optimism is Ethereum, scaled.

    Go 6.5k 4.1k