What the Rust client (zero-client) has ported from the JS client, how it is
checked, and where it deliberately differs. The conformance runner
(cargo run -p zero-conformance) prints the live numbers; the table below is
updated with each milestone.
| Area | JS source | Rust | Vectors |
|---|---|---|---|
| JSON parse and stringify | JSON.parse, JSON.stringify, property order |
json |
json-parse, json-stringify-strings, sweep-json, harden-json, generated/harden-json-fuzz |
| JS numbers | Number#toString, Number(string) |
json::number |
number-format, number-parse |
| String orders | compareUTF8, JS < |
strings |
string-compare |
| Hashes | packages/shared/src/hash.ts |
hash::xxhash32 |
hash |
| Value comparison | packages/zql/src/ivm/data.ts |
compare |
compare-values |
| Row keys | packages/zero-client/src/client/keys.ts |
keys |
primary-key-string, aggregate-row-key |
LIKE and ILIKE |
packages/zql/src/builder/like.ts |
like |
like |
| TTL | packages/zql/src/query/ttl.ts |
ttl |
ttl |
| AST, normalization, wire schema | packages/zero-protocol/src/ast.ts |
ast |
ast, ast-features, generated/ast-fuzz |
| Query hashes | packages/zero-protocol/src/query-hash-visitor.ts |
hash::query |
ast, query-internals-hash, name-args-hash, sweep-name-args-hash |
| Query builder | packages/zql/src/query/query-impl.ts, expression.ts |
query |
sweep-query-hash |
| Name mapping | packages/zero-types/src/name-mapper.ts |
schema::NameMapper |
ast-features |
| Wire messages | packages/zero-protocol/src (up.ts, down.ts) |
protocol |
protocol-upstream, protocol-downstream, sweep-downstream, protocol-client-schema, generated/protocol-*-fuzz, generated/harden-downstream-text |
| Binary poke chunks | decodePokeChunks |
protocol::poke_chunk |
protocol-poke-chunks, generated/protocol-poke-chunks-fuzz, generated/harden-poke-bytes |
Sec-WebSocket-Protocol |
encodeSecProtocols, decodeSecProtocols |
protocol::sec_protocols |
protocol-sec-protocols |
| Connect URL | createConnectionURL in zero.ts |
protocol::ConnectionUrl |
a unit test against URLSearchParams; tests/walking_skeleton.rs |
| Client schema | clientSchemaFrom |
schema::Schema::client_schema |
tests/walking_skeleton.rs (zero-cache rejects a wrong one) |
| Protocol version | PROTOCOL_VERSION (57) |
protocol::PROTOCOL_VERSION |
meta |
| IVM operators and memory source | packages/zql/src/ivm |
ivm |
ivm, ivm-features, bughunt-ivm, restored-ivm, restored-ivm-rebaselined, generated/ivm-fuzz, generated/ivm-features-fuzz, generated/sweep-ivm-*, generated/harden-ivm-*, generated/restored-chinook-decpush |
| Pipeline builder | packages/zql/src/builder/builder.ts |
builder |
the IVM vectors above |
| Views and change application | array-view.ts, view-apply-change.ts |
ivm::ArrayView |
the IVM vectors above |
| Typed queries and codegen | packages/zql/src/query/query.ts, packages/zerogen |
typed, zero-rust codegen |
the query vectors through AnyQuery, which typed queries build (rust/codegen-tests) |
| Typed CRUD in mutators | tx.mutate.<table> (crud.ts) |
typed::TypedTableMutator, generated mutate |
rust/codegen-tests/tests/typed.rs; the CRUD itself is client::Transaction's, which the client vectors check |
| Client engine | packages/zero-client/src/client (zero.ts, connection-manager.ts, query-manager.ts, zero-poke-handler.ts, mutation-tracker.ts, context.ts, ivm-branch.ts, crud-impl.ts) |
client, on memory storage |
client, client-harden-connection, client-harden-mutations, harden-queries, bughunt-client, generated/client-aggregates-fuzz |
| Synced aggregates and groups | packages/zql/src/builder/builder.ts, crud-impl.ts |
builder, client::Transaction |
the client vectors above |
| WebSocket and timers | the browser WebSocket, setTimeout, microtasks |
tokio_transport (tokio-transport feature) |
tests/tokio_transport.rs; tests/client_e2e.rs against zero-cache |
| Storage and the client lifecycle | Replicache's persistence (kv/, persist/), mem-fallback-store.ts, zero-swift's ReplicaDatabase |
client::{ZeroStorage, StorageBackend}, SQLite behind sqlite |
every client file again as (sqlite); tests/sqlite_storage.rs; tests/client_e2e.rs |
| Inspector and query metrics | packages/zero-client/src/client/inspector, metrics.ts, query-manager.ts, packages/zero-protocol/src/inspect-up.ts, inspect-down.ts, packages/zql/src/planner/planner-debug.ts |
client::Inspector, client::Metrics |
client-inspector, also as (sqlite); tests/inspector.rs |
| TDigest | packages/shared/src/tdigest.ts |
tdigest |
tdigest |
| AST to ZQL | packages/zero/src/ast-to-zql.ts |
ast::ast_to_zql |
ast-to-zql |
Within what the conformance vectors cover, nothing. zero-swift's hardening
end-to-end tests are ported too: a chaos proxy between the client and
zero-cache, offline replays, outages, and a seeded chaos fuzz
(zero-client/tests/client_e2e), and its WebSocket transport probes
(zero-client/tests/transport_probes.rs). What the JS client has that the
Rust one doesn't is listed below, under "Not ported".
What each part of @rocicorp/zero is called in Rust. Names follow JS in
snake case. The client runs on one thread: Zero is neither Send nor
Sync, and an app drives it on its event loop (with the tokio-transport
feature, a tokio LocalSet).
ZeroOptions has JS's options, in snake case. schema, context, and
mutators are arguments of Zero::new, with the scheduler and the socket
factory.
| JS option | Rust |
|---|---|
cacheURL, auth, userID, storageKey |
cache_url, auth, user_id, storage_key |
logLevel, logSink |
log_level, log_sink (a LogSink; the default is StderrLogSink) |
mutateURL, mutateHeaders, queryURL, queryHeaders |
mutate_url, mutate_headers, query_url, query_headers |
onUpdateNeeded, onClientStateNotFound, onStorageFailure |
on_update_needed, on_client_state_not_found, on_storage_failure |
pingTimeoutMs, disconnectTimeoutMs, queryChangeThrottleMs, maxHeaderLength |
the same, in snake case |
hiddenTabDisconnectDelay |
hidden_disconnect_delay_ms, with Zero::set_visible |
kvStore ('idb' or 'mem') |
storage: ZeroStorage::Memory (the default), Sqlite, or Custom |
getTraceparent |
get_traceparent |
onOnlineChange (deprecated) |
connection().subscribe(listener) |
server, getQueriesURL (deprecated) |
cache_url, query_url |
batchViewUpdates |
batch_view_updates |
slowMaterializeThreshold |
slow_materialize_threshold_ms (5,000, as in JS) |
maxRecentQueries (deprecated) |
not ported: JS keeps no recent queries by default either |
One default differs from JS, on purpose: storage is
ZeroStorage::Memory, where JS keeps its data in IndexedDB. SQLite
storage is the opt-in sqlite feature, so the default build has no
third-party dependencies, and an app chooses where its replica lives
(ZeroStorage::Sqlite { directory }); an app that wants its data across
restarts sets it. With directory: None, each app gets a folder of its
own, named for its bundle identifier or executable (on iOS and Apple's other
platforms, its container is its own already), as IndexedDB keeps each
origin's databases apart. hidden_disconnect_delay_ms is JS's 5 minutes; a
mobile app, which the system suspends soon after it goes to the background,
can set 0 to disconnect as soon as it says it is hidden, as Swift does.
| JS | Rust |
|---|---|
new Zero(options) |
Zero::new(schema, context, mutators, options, scheduler, socket_factory)? |
zero.clientID, await zero.clientGroupID |
client_id(), client_group_id(), known at once |
zero.userID, zero.schema, zero.context, zero.closed |
user_id(), schema(), context(), closed() |
zero.materialize(query) |
materialize(&query, &ttl)?, a MaterializedView with data, add_listener, destroy, update_ttl |
zero.run(query, {type}) |
run(&query, RunType, ttl), a Deferred<ViewValue> to await or read once settled |
zero.preload(query) |
preload(&query, &Ttl::DEFAULT_PRELOAD), a Preload with cleanup, cached, and complete |
zero.mutate(request) |
mutate(name, args), a MutatorResult whose client and server are Deferred<MutationOutcome> |
zero.connection.state, .subscribe, .connect({auth}) |
connection().state(), .subscribe(listener), .connect(ConnectAuth)?; see below |
zero.close(), zero.delete(), dropAllDatabases() |
close(), delete(), client::drop_all_databases(&storage), which return the DroppedDatabases |
escapeLike(text) |
like::escape_like |
zero.mutate.issue.insert(...) (CRUD mutators) |
not ported: write a mutator |
zero.inspector |
inspector(), an Inspector; see below |
zero.mutateBatch, zero.online, zero.server, zero.version |
not ported; protocol::PROTOCOL_VERSION is the sync protocol's version |
none (Swift's addMutationOutcomeListener) |
add_mutation_outcome_listener(listener), which returns the function that removes it |
none (Swift's Zero.prepare) |
Zero::prepare(..), then Zero::from_prepared(..)?: the SQLite store read on another thread at launch |
none (Swift's pendingSettled(timeout:)) |
pending_settled(timeout_ms), a Deferred<bool>: whether no mutations are pending; see below |
Rust adds reconnect_now (the network may be back, so skip the backoff),
when_pending_settled, and flush_storage (wait for the store's queued
writes, such as before the app exits).
connect({auth}) is connect(ConnectAuth::Keep) without auth,
ConnectAuth::Set(token) with a token, and ConnectAuth::Clear with
null. Where JS's promise rejects, such as for a token on a client without
a user_id, it returns the error and changes nothing. It returns before
the state changes, where JS's promise waits for the next one: subscribe
follows it.
add_mutation_outcome_listener is Swift's API, with no JS counterpart: the
listener hears each mutation's server outcome once, as a
MutationServerOutcome, including that of a mutation replayed after its
server result settled with a ZeroError when the client gave up
connecting. Swift's mutationOutcomes stream isn't ported: the listener can
feed a channel.
pending_settled is Swift's API too, with no JS counterpart: it waits for
the server to confirm every pending mutation, as before set_visible(false),
and resolves with whether none are left. It resolves at once when the server
can't confirm them (the client isn't connected, it's closed, or its client
group is disabled), and at timeout_ms on the client's scheduler.
when_pending_settled calls a function instead, with no timeout.
When the server no longer has the client's state, JS asks the app to
reload. Without on_client_state_not_found, Rust replaces the client
inside the same Zero, as Swift does: a new client ID and client group on
the same store, with the views and preloads moved to the new client.
| JS | Rust |
|---|---|
inspector.client, .clientGroup |
client(), client_group(), an InspectorClient and an InspectorClientGroup |
await inspector.metrics(), .serverVersion() |
metrics(), server_version(), a Deferred<Metrics> and a Deferred<String> |
await inspector.clients(), .clientsWithQueries() |
clients(), clients_with_queries(), a Deferred<Vec<InspectorClient>> |
await client.queries(), .map(), .rows(table) |
queries(), map(), rows(table); a query is an InspectorQuery, with JS's fields |
await inspector.analyzeQuery(query, options) |
analyze_query(&query, AnalyzeQueryOptions), a Deferred<AnalyzeQueryResult> |
inspector.analyzeQuery with a server AST, or a name and arguments |
analyze_server_ast(&ast, options), analyze_named_query(name, args, options) |
| the password prompt | authenticate(password), a Deferred<bool> |
formatPlannerEvents(events) |
format_planner_events(&events) |
the TDigest of query-materialization-client and the other metrics |
tdigest::TDigest, which reads and writes JS's JSON form |
Inspector takes the context's type (Inspector<C>, () by default),
since analyze_query builds a query with it. The client records JS's
metrics, query-update-client sampled as JS samples it, and logs a query's
end-to-end materialization as JS does, as one line: the message, the query
ID, its AST as JSON, and the milliseconds, a warning past
slow_materialize_threshold_ms. At the debug level it asks zero-cache for
its timings (debugPerf), as JS does.
zero-cache can require its admin password. Where JS prompts for it, an RPC
the server refuses fails with an error that starts with "Authentication
failed" and says to call authenticate(password). A password the server
accepted is presented again before the next RPC on a new connection, as JS
presents its adminPassword, and forgotten when the server refuses it. An
RPC waits for the connection, and fails with an error when the connection
drops before the answer or the client closes, where JS's would wait for an
answer that never comes.
Generated code (zero-rust codegen) has a row struct per table with
Column and Relationship constants, and zql::<table>() for its typed
Query. AnyQuery is the untyped builder, named by strings; a typed
query wraps one (any(), into_any(), Query::from_any). materialize,
run, and preload take any ZeroQuery: an AnyQuery, a typed Query,
or a named query's QueryRequest, which they build with the client's
context.
| JS | Rust |
|---|---|
defineQueries(...), defineQuery(...) |
define::define_query(name, |args, ctx| query) (serde feature), called as Q.with(args); untyped, AnyQuery::name_and_args(name, args) |
.where('open', true) |
.where_eq(Issue::OPEN, true) |
.where('title', 'LIKE', '%x%') |
.where_(Issue::TITLE, SimpleOperator::Like, "%x%".into()) |
.where(({cmp, and, or, not, exists}) => ...) |
.where_expr(|e| e.and(vec![...])), with RowExpression |
.where(({cmp, json}) => cmp(json('metadata', 'tags', 0), 'bug')) |
.where_expr(|e| e.json(User::METADATA, path).eq("bug".into())) |
.whereExists('labels', q => ...) |
.where_exists(Issue::LABELS, ExistsOptions::default(), |q| ...) |
.related('comments', q => q.limit(10)) |
.related(Issue::COMMENTS, |q| q.limit(10)) |
.orderBy('created', 'desc'), .limit(n), .start(row), .one() |
.order_by(Issue::CREATED, Direction::Desc), .limit(n), .start(&row, inclusive), .one() |
.count(), .sum, .avg, .min, .max |
the same, with _json forms for a JSON path; count is an i64 |
.related('comments', q => q.count()) |
.related(Issue::COMMENTS, |q| q.count()), read as row.aggregates.count(Issue::COMMENTS) |
.distinct(...), .groupBy(...).count() |
.distinct(vec![...]), .group_by(vec![...]).count() |
A view's data is a ViewValue; a typed query reads it as its rows with
query.read(&view.data()) (or Vec::<Issue>::from_view). An unnamed
query only sees local data, as in JS without enableLegacyQueries: the
server never confirms it, so a run that waits for Complete or Cached
never settles, and preloading it does nothing.
| JS | Rust |
|---|---|
defineMutator(async ({tx, args, ctx}) => ...) |
define::define_mutator(name, |tx, args, ctx| ...) (serde feature), a synchronous function over typed arguments; untyped, Mutator::new(name, |tx, args, ctx| ...) over JSON |
zero.mutate(mutators.issue.create(args)) |
zero.mutate_request(CREATE_ISSUE.with(args)), or zero.mutate(name, args) with JSON arguments |
await tx.mutate.issue.insert(row), .upsert, .update, .delete |
mutate::issue(tx).insert(&row)? (generated), or tx.table("issue").insert(&object)? |
await tx.run(query) |
tx.run(&query)? |
tx.location, tx.reason, tx.clientID, tx.mutationID, tx.token |
location(), reason(), client_id(), mutation_id(), token() |
throw new ApplicationError(message, {details}) |
Err(ApplicationError { message, details }.into()) |
A mutator runs once optimistically and again on every rebase while it is
pending, so it must depend only on the transaction, its arguments, and the
context. A definition is registered with its mutator() in Zero::new;
its arguments are read back as its type before the body runs, and
arguments that don't read fail it with an ApplicationError, where JS
validates them with the definition's schema. Calling a mutator that isn't
registered throws in JS; in Rust both outcomes are a ZeroError saying so, and nothing is written or sent.
No UI bindings. useQuery's typed equivalent is LiveView, after the
Swift client's LiveQuery: Zero::live takes a typed Query or a named
query's QueryRequest and keeps its value read as the query's Rust type,
reading only the rows whose entries changed, with its result type and an
error that is the server's query error or a row that didn't read (which
keeps the last value). It follows the client when the client replaces
itself, and has listeners a UI wakes itself from; the untyped equivalent is
a MaterializedView and its listener. apps/zbugs-rune shows how a Rune
app binds live views to its UI, on the shared zbugs model
(apps/zbugs-model), whose queries and mutators are define_query and
define_mutator definitions; apps/zbugs-gpui is a GPUI app on the same
model.
-
ast-featureschecks that the wire schema rejects what JS rejects, but not valita's error messages. Its name mapping uses the JS test schema as JS wrote it intoivm.json's shared fixtures. -
Protocol messages are compared as JSON values, not text: Rust re-encodes a message from its fields, in an order of its own (
sweep-downstreamcompares text with keys sorted). The header encoder is checked by decoding what it writes, as Swift checks it, so its exact escaping isn't pinned by a vector. -
sweep-query-hashreplays each program of builder calls and compares both the AST and the hash, and requires the query to be valid, since JS built every one without throwing. The builder's misuse messages are checked byivm-features' builder errors (M2); none of the vectors reachlimitororderByinside a junction relationship, so a unit test inquery.rspins those two messages. -
The client vectors (
client/harness.ts) are scenarios: an app and a server take steps against one client, on a manual clock and mock sockets. Rust replays each on aZerowithzero_client::testing's scheduler and socket factory and compares, after every step, what the app saw (views, listener calls, mutation andrunsettlements, connection states) and every message the server received, as Swift'sClientConformanceTestsdoes. -
client-inspectorstubs JS's password prompt to answer nothing, so an RPC the server refuses fails with "Authentication failed"; the Rust error goes on to say to callauthenticate(password), and the runner compares its part before the colon. A query's milliseconds come from the manual clock, so its metrics and log lines are compared exactly. -
The IVM vectors (
ivm/harness.ts) are programs of query builder calls. Rust replays each with its own builder, must build JS's AST and format or fail with JS's message, then runs JS's AST over memory sources, as Swift does: JS runs the AST with each ordering naming a column once, and Rust must give that answer to the AST as written. Each push (or, forharden-ivm, each batch) compares the changes the view received and the view itself;harden-ivmalso compares whether listeners were notified, and tags numbers JSON can't carry. Errors are compared by message, including JS's internal assertions: Rust reports "Unexpected early return prevented full hydration" where JS'sTakeandCapassert it, which the Swift client skips. -
The restored zql vectors (
restored-ivm,restored-ivm-rebaselined,generated/restored-chinook-decpush) carry JS's AST and format instead of a program, with the case's own sources (or the chinook fuzzer's mini data) and builder settings, so Rust replays only the pipeline and the pushes, as Swift'sRestoredIVMTestsdoes. A test fails a doctored copy ofrestored-ivmand of the chinook corpus, so a runner that stopped comparing would show.
Each is counted separately in the conformance report, never as an exact match. Both come from what Rust's types can hold, not from the client's logic, and are kept on purpose.
| Difference | Why it stays |
|---|---|
| Lone surrogate | A Rust String can't hold a lone surrogate, which a JS string can. The parser reads an escaped one ("\ud800") as U+FFFD, as the Swift client does; such a case matches when JS's output, with each such escape read as U+FFFD, is what Rust writes. Holding one would take a string type of Rust's own (WTF-8) throughout the client and in every app's code. Postgres text and jsonb can't store one: only a json (not jsonb) column, which keeps its text as written, could bring one to the client, and it shows as U+FFFD, as a browser renders it. Nearly all such cases come from the JSON fuzz corpus. |
| Unwritable path | A JSON path index is a u32, so a program with a negative or fractional index (json('metadata', -1)) can't be written in Rust. JS rejects it with "Invalid JSON path segment …"; such a case matches when JS rejects it that way. Rust's types rule out at compile time what JS throws for at run time. An index past MAX_JSON_PATH_INDEX can be written, and is rejected with JS's message. |
JSON nesting is no longer a difference. JSON nested up to json::MAX_DEPTH
(5,000) levels deep, more than the about 4,100 zero-cache can write, is
parsed, cloned, compared, written, and dropped within a 2 MB thread stack,
even in a debug build, so the client takes any value the server sends.
Deeper text is rejected ("JSON nested too deeply"), where JS takes whatever
fits its stack. The Swift client has the same limit. Formatting a value with Debug still recurses.