BenchDB is a self-hosted server for benchmark results. Your benchmark harness
writes JSON, the benchdb CLI uploads it, and BenchDB keeps the history of
each benchmark on each machine. The dashboard shows trends and comparisons, and
benchdb ci report tells a pull request whether it made a benchmark slower.
BenchDB works with benchmarks in any language that can write JSON. It does not run or schedule benchmarks.
Start with the quickstart, or read the documentation for the full reference.
A benchmark number alone cannot tell you whether a change helped. Each BenchDB result also records the commit, the run, the machine, the unit, and the raw observations. BenchDB groups results that share a benchmark case, context, machine, and repository into one series. It scores each new result against that series' history. CI reports also compare it with the matching result from a baseline run. Concepts defines these terms and the status rules.
BenchDB fits teams that benchmark every commit or release, run benchmarks on
more than one machine, and want CI to flag regressions. If an upload fails,
rerun benchdb results submit on the saved files. The benchmark itself does
not have to run again.
You probably do not need it for a one-off timing on your laptop. It is not a general metrics, logs, or tracing system.
Grafana time-series panels plot timestamped metrics from many data sources, which suits service health and machine telemetry. Grafana can plot benchmark numbers too, but you build the storage model, queries, and comparison rules yourself. Grafana's k6 integration is one example.
BenchDB has those benchmark rules built in. It tracks commit and run identity, keeps machines apart, selects a baseline, classifies regressions, and writes CI reports. Use Grafana to ask what a system is doing over time. Use BenchDB to ask whether a code change made a benchmark slower. You can run both.
- the Go server and
benchdbCLI, built into one binary - the Svelte dashboard, embedded in that binary
- PostgreSQL metadata and recent results, with older results in managed Parquet files
- the PostgreSQL schema and migrations
- the OpenAPI document and a generated Go client under
sdk/go/benchdb
The Go module is go.kenn.io/benchdb. make build needs Go, Bun, and a C/C++
compiler and writes bin/benchdb. Go builds use CGO_ENABLED=1.
make dev needs Docker and starts a local server with demo data at
http://localhost:8080. The quickstart shows both.
The server requires a PostgreSQL URL in BENCHDB_DB_URL and a persistent local
archive directory in BENCHDB_DATA_DIR. make build downloads the matching
DuckDB PostgreSQL extension into .cache/duckdb/extensions; set
BENCHDB_DUCKDB_EXTENSION_DIR to that directory when running bin/benchdb.
make dev sets up its own archive and extensions under var/dev-data.
Back up the archive together with PostgreSQL. See the
operations guide for deployment and recovery
instructions.
Contributions are welcome, including pull requests from forks. Push a feature
branch to your fork and open a pull request against the main branch of
kenn-io/benchdb. You do not need write access to this repository.
The contributing guide covers the fork workflow, local checks, and documentation builds. Coding agents should also read AGENTS.md.
BenchDB is under active development. Its command and storage interfaces can change before the first stable release.
BenchDB began as an independent fork of the Go rewrite on the
experimental-v2 branch of Conbench.
We thank the Conbench contributors for that code and for the project's original
idea of language-independent continuous benchmarking.
BenchDB is now developed independently and has a different product direction. See NOTICE.md and LICENSE for attribution and license terms.