fix(knowledge): refresh unchanged mirrored ACLs without firing the projection fan-out - #8169
Merged
Merged
Conversation
…ojection fan-out An admin-mode sync rewrote the ACL of every listed document on every page, even when it had not changed. Assigning `acl` fires the document trigger that copies it onto every chunk's search projection rows, and a projection row the backfill has not filled yet is distinct from every ACL, so each unchanged document still rewrote all of its chunks, each one a vector index insert. A single ACL batch could then exceed the statement timeout and fail the whole sync. Unchanged ACLs now only refresh their evidence timestamp, which never assigns `acl`. Documents whose ACL or restrictions actually changed are written in smaller batches, since each one fans out to all of its chunks.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
|
…solved evidence and multi-batch changes
Collaborator
Author
Collaborator
Author
|
@cubic-dev-ai review this PR |
Contributor
@waleedlatif1 I have started the AI code review. It will take a few minutes to complete. |
6 tasks
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
aclon every listed document every page, even when unchanged. The document trigger copiesaclonto every chunk's search projection rows, and a projection row the backfill hasn't filled yet (acl IS NULL) is distinct from every ACL — so each unchanged document still rewrote all its chunks, each one a vector index insert. One ACL batch could outrun the 60 s statement timeout (SQLSTATE 57014) and fail the whole sync right after the permission fetchpersistDocumentAclsnow splits each ACL group into two statements: rows whoseaclandacl_requirementsalready match only getacl_verified_atrefreshed (noaclin the SET list, so the trigger never fires); only rows that actually differ get the ACL writeacl_verified_at, the unresolved-duplicate guard applies to both statements, and a row is counted once (refresh runs first)Type of Change
Testing
sync-persistence.postgres.test.tsagainst real Postgres with the projection trigger installed: an unchanged ACL leaves unfilled projection rows untouched and refreshes evidence; a changed ACL still propagates to filled and unfilled rows; a changed restriction under the same primary ACL counts as a change; a mixed page writes each document oncesync-persistence.test.ts;lib/knowledge/access,lib/knowledge/connectors,connectors/google-drivesuites pass with Postgres (1206 tests)bun run lint,bun run check:audits(47),docs-manifest:check,type-checkpassChecklist