An index of Rust libraries and tools for LLM agents, LLM infrastructure and market data, each in its own repository. This repository holds no code; it is the catalogue.
Every entry links to its repository. "crates.io" means the crate is published there under that name; otherwise use the git dependency shown.
Small, focused libraries you can drop into any agent.
| Crate | What it does | Install |
|---|---|---|
| llm-budget | Hard USD spend caps per agent and per fleet, kill switches, automatic model downgrade as budget runs low | crates.io cargo add llm-budget |
| tokio-agent-memory | Episodic events with causal links, semantic fact graph, prioritized working memory, decay, fuzzy retrieval, multi-agent bus | crates.io cargo add tokio-agent-memory |
| tokio-memory | Trait-based episodic and semantic stores, query builder, Ebbinghaus decay, consolidation pipeline, binary snapshots | crates.io cargo add tokio-memory |
| mem-graph | In-memory knowledge graph: typed entities and relationships, BFS/DFS, shortest path, transitive closure, JSON snapshots | git only (the mem-graph name on crates.io is another project) |
| llm-sync | CRDTs (GCounter, PNCounter, GSet, LWW register, LWW map) and vector clocks for sharing state between agents | crates.io cargo add llm-sync |
| llm-diff | Line and JSON diffs of LLM outputs, content-addressed version store with branches and lineage, audit log | crates.io cargo add llm-diff |
| wasm-agent | Synchronous ReAct loop with tool registry and token-limited history; builds for wasm32 |
crates.io cargo add wasm-agent |
| llm-wasm | The logic around an LLM call for wasm32: TTL cache, retry policy, guards, routing, cost ledger, templates, JSON extraction |
crates.io cargo add llm-wasm |
| llm_affector | LLM-as-judge checks: hallucination detection and Rust code critique with typed results (prototype) | git only |
Git dependency form, for crates that are not on crates.io:
mem-graph = { git = "https://github.com/Mattbusel/mem-graph" }
llm_affector = { git = "https://github.com/Mattbusel/llm_affector" }| Crate | What it does | Install |
|---|---|---|
| tokio-llm | Async OpenAI and Anthropic client with retry, circuit breaker, USD budget cap and SSE streaming | git only: tokio-llm = { git = "https://github.com/Mattbusel/tokio-llm" } |
| agent-runtime | Full agent runtime: ReAct and plan-execute loops, memory, knowledge graph, circuit breakers, providers, multi-agent bus | crates.io cargo add llm-agent-runtime |
| tokio-prompt-orchestrator | Five-stage LLM serving pipeline with dedup, circuit breakers, rate limits and a dead-letter queue; REST, SSE, MCP and TUI | crates.io cargo add tokio-prompt-orchestrator |
| HelixRouter | Adaptive Tokio job router: inline, spawn, CPU pool, batch or drop per job, with a live dashboard | crates.io cargo add helixrouter |
| Tool | What it does | Install |
|---|---|---|
| llm-cost-dashboard | Terminal dashboard for LLM spend from a JSON request log, 83 priced models | cargo install llm-cost-dashboard (binary llm-dash) |
| llm-bench | Benchmark OpenAI and Anthropic models on your prompts: p50/p99 latency, tokens/s, cost | cargo install --git https://github.com/Mattbusel/llm-bench |
| Every-Other-Token | Intercept an LLM token stream live: per-token confidence, token mutation, A/B and provider comparison | cargo install every-other-token |
| Crate | What it does | Install |
|---|---|---|
| fin-primitives | Decimal-precise trading building blocks: price types, order book, OHLCV aggregation, indicators, position ledger, risk rules | crates.io cargo add fin-primitives |
| fin-stream | Real-time market data streaming primitives and tick ingestion pipeline | crates.io cargo add fin-stream |
Most of these crates follow the same habits: typed error enums (thiserror), Clippy lints that deny unwrap, expect and panic in library code, and unit plus integration tests. Maturity varies; each README has a status section saying what works and what does not yet.
- llm-cpp: single-header C++17 libraries for LLM features (streaming, retries, caching, RAG, agents).
- Special-Relativity-in-Financial-Modeling: C++20 research code applying special-relativistic geometry to OHLCV data.
- awesome-llm-infra: curated list of LLM infrastructure projects.
The index itself is MIT licensed, see LICENSE. Licensing of each crate is set in its own repository.
Need this kind of engineering on your product? I take on a small number of client builds: LLM features, iOS apps and performance work, fixed price. Services and pricing · Email · LinkedIn