Repository navigation
(fix): keep cache owner registrations in one key per collection - #987
Conversation
The document cache and the find() query cache registered every invalidation under its own `#owner:<token>` key and released it with purge(). On the Redis adapters purge() runs LUA_PURGE_BUMP, which DELs the key and then HSETs its next generation, so the key never goes away; the adapter sets no expiry either. Each write therefore left one permanent key per cache: a probe of 200 createDocument() calls against MariaDB and Redis grew the owner keys from 1 to 201 in each cache. A collection's registrations are now fields of one `<collection key>#owners` hash: save($ownersKey, $token, $token) registers, load() with the token checks, and purge($ownersKey, $token) releases through LUA_PURGE_FIELD, which HDELs the field and bumps the key's generation. The key count is bounded by the number of collections, and the hash only holds the registrations still in flight. The owner checks, fail-closed paths and flush tolerance are unchanged; only where a registration lives moved. Adapters that ignore $hash (Memory, Memcached, Filesystem, Hazelcast) share one slot per collection, so two overlapping invalidations of one collection there would report the earlier owner as invalid after its commit and leave the epoch blocked. The Redis adapters, which are the only Leasable ones, keep fields apart. Keys matching `*#owner:*` from earlier builds are no longer read. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. 🗂️ Base branches to auto review (2)
Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository: utopia-php/database/.coderabbit.yaml Review profile: CHILL Plan: Advanced Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
The first revision put every registration in one `#owners` hash per collection. Memory, Memcached, Filesystem and Hazelcast ignore the field, so there overlapping invalidations of one collection shared a single slot, and the earlier owner read the later owner's token and threw "Invalid document cache owner" after its write had committed. Owners::register() now checks, right after saving its field, whether the adapter lists it. The Redis adapters do, so the token stays a field of the bounded `#owners` hash. Field-less adapters always list nothing, so the token also gets a key of its own, as before this branch; their purge deletes it, so nothing leaks there. Owners::find() locates the registration the same way at activation, so the checks, release and fail-closed paths read the registration the token actually has. The regression tests now assert that the key set does not grow with writes, and inject the owner faults as cache faults, instead of naming the owners key or Redis's reserved fields. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@greptileai review |
The train's constraints go back to caret form. For usage ^0.17 is the range 0.17.* was; for database, abuse and migration ^8.0 and ^3.0 also admit the later minors of the same major, which their branch aliases (8.0.x-dev, 3.0.x-dev) and their first tags satisfy without another constraint change. The four libraries restated their own constraints the same way, so the lock moves only their references to those heads: database 30f25fd13b, abuse cf0ed5d129, migration edf6230085 and usage bf9ab997fa. The database re-pin brings utopia-php/database#982: on MySQL, from five joins, each joined table's permission check stays a subquery (NO_SEMIJOIN); other adapters are unchanged. It also brings utopia-php/database#986: a write that commits while the cache is being flushed no longer fails with "Failed to finish document cache invalidation"; a purge that is really lost still does. And utopia-php/database#987: the document cache and the find() query cache register a write's invalidation owner as a field of one key per collection, so writes no longer leave a Redis key behind each. Audit moves to the mirror head f8422f9cbf, which carries monorepo#206's 5.0.0 candidate (7c9c8296cc): its adapters read attributes and indexes as typed models, and its legacy TYPE_* constants come from Method. audit keeps its inline alias until audit 5.0.0 is released. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
What
The document cache and the
find()query cache no longer leave one permanent Redis key behind per write.Both invalidation handshakes registered each invalidation under its own
<collection key>#owner:<token>key and released it withpurge(). On the Redis adapters (utopia-php/cache 4.0.2, same in 5.x),purge()runsLUA_PURGE_BUMP:DEL keyfollowed byHSET key __utopia_gen__ <next>. The key is recreated holding only its generation, and the adapter never sets an expiry, so every write left one key per cache for good.Registrations now go through a small
Cache\Ownersclass shared by both handshakes:save("#owner:$token", $token)save('#owners', $token, $token); iflist('#owners')lacks the token, alsosave("#owner:$token", $token)load("#owner:$token")load()whereverOwners::find()says the token livespurge("#owner:$token")purge()of that registration: the field, or the token's own key<collection key>#ownersand is released byLUA_PURGE_FIELD, whichHDELs the field and bumps the key's generation. The key count is bounded by the number of collections, and the hash only holds registrations still in flight.[]fromlist(), so every token keeps a key of its own exactly as before. Their purge deletes that key, so nothing leaks, and overlapping invalidations of one collection keep independent registrations. Pool, Sharding and CircuitBreaker passlist()through (checked in utopia-php/cache 4.0.2 and 5.1.1).The protocol itself is unchanged: "Invalid … cache owner", "Failed to release … cache owner", the fail-closed paths and flush tolerance behave exactly as before; only the storage location of a registration moved.
#started,#finishedand#epochwere already per collection.Evidence
Probe against MariaDB 10.11 + Redis 8.2.1, one collection,
setQueryCache()installed, afind()after eachcreateDocument():Each leaked key was
{"__utopia_gen__":"1"}withTTL -1. After the fix the extra document-cache key is_metadata#owners, not growth.Why this approach
save(..., $ttl)with a key expiry only exists in utopia-php/cache 5.x, and a whole-key purge drops the expiry anyway; the constraint is^4.0 || ^5.0.Costs
HKEYSof the collection's#ownershash per registration and one per activation on the Redis adapters. The hash only holds in-flight registrations.saveand itslist, that one token falls back to a key of its own, which then stays behind on Redis. Correctness is unaffected: the flush happened before the barrier was written, so the registration is a valid canary either way.Tests
Each test was seen failing against the code it guards:
GeneralTests::testCacheInvalidationDoesNotAddRedisKeysPerWrite(e2e, real Redis): the collection's Redis key set is identical before and after 11 writes and a transactionfeat-query-libQueryCacheTest/DocumentCacheEpochTest::…DoNotAddKeysToACacheThatKeepsPurgedKeys: same, againstRedisLeasableCache, a double that mirrors the Lua purge semantics (a purged key stays, holding its generation)QueryCacheTest::testOverlappingInvalidationsSucceedOnACacheWithoutFields,DocumentCacheEpochTest::testOverlappingWritesSucceedOnACacheWithoutFields: a second owner in flight on a field-less cache does not fail the first owner's activation1f943b88b)…RejectsACorruptedOwnerRegistration/…PropagatesAnOwnerReleaseFailure: "Invalid … cache owner" and "Failed to release … cache owner" still fire when a field write is corrupted or a field purge failsThe tests assert key counts and behaviour, not key names or Redis's reserved fields.
Verified locally (on top of #986)
composer lint(Pint)composer check(PHPStan, both configs)paratest --functional --processes 4 tests/unitparatest --functional --processes 4 tests/e2e/Adapter/MariaDBTest.phpNot verified locally
*#owner:*keys in deployed Redis instances are not cleaned up by this PR; they are no longer read and can be deleted withSCAN MATCH *#owner:*+DEL(the new#ownerskey does not match that pattern).🤖 Generated with Claude Code