Clustering algorithm implementation in Rust and binding to Python.
For Python users, check the Python README.
- K-means
- PyO3 binding
- RaBitQ assignment
- Parallel with Rayon
- SIMD acceleration on
x86_64andaarch64viapulp - mini batch K-means
- Hierarchical K-means
cargo add gathersCheck the library docs.
Beyond k-means assignment, the RaBitQ index is exposed as a standalone module, gathers::rabitq,
for approximate top-1 vector retrieval.
The non-published CLI in crates/cli supports fvecs input and output:
cargo run --release -p gathers-cli -- --threads 16 --seed 42 kmeans \
-i vectors.fvecs -o centroids.fvecs -n 4096
cargo run --release -p gathers-cli -- --threads 16 --seed 42 assign \
--vectors vectors.fvecs --centroids centroids.fvecsTraining automatically uses PCA to 128 dimensions for datasets with at least 1,000,000 rows and
dimension above 196; smaller inputs train without reduction. Use --reduction raw to opt out.
To lower the training sample budget, use --samples-per-cluster 128 (default 256), or set an
exact total with --training-samples.
The CLI can train K-means in raw, PCA, or SRHT space and writes full-dimensional centroids for
the existing assign command. Select --distance l2|cos|dot (default l2); explicit PCA/SRHT
default to 128 dimensions, or use --reduced-dim to override:
gathers kmeans -i vectors.fvecs -o centroids.fvecs -n 4096 \
--reduction pca --reduced-dim 128See the reduction guide for method trade-offs and implementation details.
To install the binary from a checkout, run cargo install --path crates/cli.
For timing scope and CPU sampling, see the profiling guide.