This repository contains automated testing workflows for Ethereum execution clients using Hive, a system for running integration tests against Ethereum clients.
Hive Tests runs comprehensive compatibility and conformance tests across multiple Ethereum execution clients to ensure they properly implement the Ethereum protocol. The tests run automatically, on a daily schedule or when the images they test change, and can also be triggered manually.
The following Ethereum execution clients are tested:
- Besu - Enterprise-grade Java Ethereum client
- Erigon - High-performance Ethereum client
- EthereumJS - JavaScript implementation of Ethereum
- Ethrex - Rust-based Ethereum client
- go-ethereum (Geth) - The Go implementation of Ethereum
- Nethermind - .NET Ethereum client
- Nimbus-EL - Nim-based Ethereum execution layer client
- Reth - Rust-based Ethereum execution client
The repository runs various test simulators including:
- devp2p
- ethereum/consensus
- ethereum/eels/consume-engine
- ethereum/eels/consume-enginex
- ethereum/eels/consume-rlp
- ethereum/eels/consume-wirex
- ethereum/eels/execute-blobs
- ethereum/engine
- ethereum/graphql
- ethereum/rpc-compat
- ethereum/sync
Tests run via GitHub Actions. See the mainnet.yaml workflow for more details. This is the main workflow that is triggered by the scheduled workflows.
The repository includes additional specialized workflows that target specific consensus testing scenarios which take a long time to run. These workflows run on a different schedules and have different timeouts:
sim-devp2p.yamlsim-ethereum-consensus-legacy-cancun.yamlsim-ethereum-consensus-legacy.yamlsim-ethereum-consensus.yamlsim-ethereum-eels-consume-engine.yamlsim-ethereum-eels-consume-enginex.yamlsim-ethereum-eels-consume-rlp.yamlsim-ethereum-eels-consume-wirex.yamlsim-ethereum-eels-execute-blobs.yamlsim-ethereum-engine.yamlsim-ethereum-graphql.yamlsim-ethereum-rpc-compat.yamlsim-ethereum-sync.yaml
These workflows are lightweight dispatchers that trigger the main mainnet.yaml workflow with specific parameters for targeted testing scenarios. They use different concurrency groups to avoid conflicts and can be manually triggered through the GitHub Actions interface.
The bogota and heze workflows (hive-bogota.yaml, hive-heze.yaml and their -quick versions) and hive-glamsterdam-quick.yaml have no schedule. Full glamsterdam coverage comes from the EELS wrappers through mainnet.yaml, which test the same client images and EELS release on every fork. watch-images.yaml checks the client and EELS simulator images they test and dispatches a run when one changes: a new client image runs that client, a new fixtures release in an EELS image runs that simulator for every client with an image, and a new devnet runs everything. A rebuild of the same client tag (a push to its main branch) reruns that client right away on the -quick workflows and for the fast consume-enginex and execute-blobs simulators, and at most every 12 hours for consume-engine and consume-rlp, whose full runs take up to a day and a half per client. Every pair also runs daily at its workflow's refresh hour, unless it ran in the 12 hours before, which picks up hive and workflow changes and reruns failures. The refresh hours are spread over the day, one per workflow. Every hive-*.yaml workflow without a schedule is watched this way, and so is every sim-ethereum-eels-*.yaml wrapper without one, whose EELS simulator then runs through mainnet.yaml for the clients that changed. It runs when an image build announces a new image, and, to catch a missed announcement, every hour by dispatching itself, since GitHub often runs scheduled workflows hours late. An hourly cron restarts that loop if it stops. An image build announces one with gh api repos/ethpandaops/hive-tests/dispatches -f event_type=image-pushed (a token that can write this repository's contents) or gh workflow run watch-images.yaml -R ethpandaops/hive-tests (a token that can only run its workflows).
You can manually trigger test runs through the GitHub Actions interface with customizable parameters:
- Client Selection: Choose which clients to test
- Simulator Selection: Choose which test simulators to run
- Client Sources: Pick a client file (
client_file) or paste an inline hive client file (client_config), see below - Hive Version: Specify the Hive repository and branch to use
The generate-index.yaml workflow updates the index of the test results in the S3 bucket, fetching only the result files the index doesn't list yet. Each test job triggers it once its results are uploaded, and it also runs every 3 hours. A manual dispatch with full rebuilds the index from every result file.
Test results are automatically uploaded to:
- S3 Storage: Results are stored in the
hive-resultsbucket - Public Dashboard: Available at https://hive.ethpandaops.io/
- GitHub Artifacts: Downloadable from the workflow runs
How each client is obtained is defined by hive client files under .github/configs/hive/, one pair per client set:
<set>.yamlis what the workflows run: the best available source per client. Clients with a prebuilt ethpandaops image for the set use it; a client without one is built from source inside the run.<set>-git.yamlbuilds every client from source, for tip-of-branch runs when an image is stale or missing.
master.yaml serves the mainnet.yaml workflow; the devnet workflows each have their own set. A manual dispatch picks a file with the client_file input, or replaces it entirely with an inline hive client file in client_config, for example to test a branch or an image that is not in the repository configuration.
Tests run on a combination of:
- GitHub-hosted runners for lighter test suites
- Self-hosted runners for resource-intensive tests (consensus, engine, eels simulators)
This project is licensed under the GNU General Public License v3.0 - see the LICENSE file for details.
This repository is part of the Ethereum testing infrastructure maintained by EthPandaOps. For issues or contributions related to the Hive testing framework itself, please visit the main Hive repository.