Skip to content

Fix crash and data loss when repairing a corrupted database - #6395

Open
johnw188 wants to merge 2 commits into
signalapp:mainfrom
johnw188:fix-keyvalue-partial-recovery
Open

johnw188 wants to merge 2 commits into
signalapp:mainfrom
johnw188:fix-keyvalue-partial-recovery

Conversation

@johnw188

@johnw188 johnw188 commented Nov 9, 2026 •

Copy link
Copy Markdown

Contributor checklist

Administrative:

Testing:

  • I have verified that the issue reproduces on latest main.
    • I have included a screen recording documenting the issue reproducing.
  • I have verified that the issue no longer reproduces on my branch.
    • I have included a screen recording documenting the issue no longer reproducing.
  • I have thoroughly tested this change, and I am not aware of any regressions to existing features or behaviors.
  • I have tested my contribution on these devices:
    • iOS 27.0 Simulator only

Commits:

  • My commits are rebased on latest main.
  • My commits are well-structured for review.

Human Note

I (@johnw188) took on this work to help a friend whose Signal database on their phone is showing as corrupted. I found the referenced issue #5811 where other users had had similar issues. On investigation I found that one corrupted page in keyvalue fails the entire repair, and I believe that most users would rather have all but one of their messages than be stuck with no history.

This PR was built with Opus 5.5, extensively reviewed by models, and then reviewed by me as a final check. I'm not familiar with the Signal codebase, so while the changes look correct to me. I would want someone with context to review before merging.

Opus 5.5 output below

Description

Fixes part of #5811.

  1. Don't crash when an integrity check's transaction fails. On a corrupted page, COMMIT after PRAGMA quick_check can fail ("SQL logic error"). read/write owsFail on that, so tapping "Continue" on the recovery screen crashed before recovery started. This commit uses the throwing variants, as reindex already does. If a check ran, it keeps that result; if the check couldn't run, it treats it as failed.
  2. Skip past corrupted pages when copying tables. A read error used to stop the copy, losing every row after the bad page. And because ending the read transaction also failed, the rows already copied were reported as a total failure. Tables with rowids are now copied in rowid order. After a read error, the copy searches ahead for the next readable rowid and continues from there. It stops a pass if rowids go backwards, which a corrupted b-tree can cause. Limitations:
    • The search assumes rows are unreadable up to a point and readable after it, so readable rows between two nearby corrupted ranges can still be lost.
    • As on main, a row whose last overflow page is corrupted can be copied with a zero-filled tail.

My friend's log shows recovery failing on a corrupted page in keyvalue, which must be copied flawlessly. This PR doesn't change that. I've opened #6396 as a separate proposal, because it's a policy decision.

Testing

The tests corrupt pages on disk, which breaks their SQLCipher HMAC. A helper checks that exactly the intended rows became unreadable.

  • testIntegrityCheckOnCorruptedDatabase: on main, the test process crashes with Failing for unexpected throw: SQL logic error.

  • testDumpAndRestoreSkipsCorrupted…: four cases on KeyTransparency, where only the corrupted rows are lost:

    • one row;
    • the first and last rows;
    • three adjacent rows;
    • two separate rows.

    On main, everything after the first corrupted row is lost.

  • testMoreActionableErrorPrefersRunningOutOfSpace.

Each commit passes DatabaseRecoveryTest on its own. The full SignalServiceKitTests suite passes.

Screen recordings

To set up the recordings, a local launch hook (not included in this PR) filled KeyTransparency and corrupted one of its pages.

Before (main): Repair → Continue crashes.

before-main.mp4

After (this branch): Repair → Continue ends at "Data Successfully Recovered".

after-fix.mp4

I haven't tested on a physical device.

🤖 Generated with Claude Code

@johnw188
johnw188 force-pushed the fix-keyvalue-partial-recovery branch from 0955769 to 82f2809 Compare November 9, 2026 18:54
@johnw188 johnw188 changed the title Keep partially-copied keyvalue table during database recovery Fix crash and data loss when repairing a database with a corrupted page Nov 9, 2026
@johnw188
johnw188 force-pushed the fix-keyvalue-partial-recovery branch from 82f2809 to 63131a1 Compare November 9, 2026 20:05
@johnw188 johnw188 changed the title Fix crash and data loss when repairing a database with a corrupted page Fix crash and data loss when repairing a corrupted database Nov 9, 2026
@johnw188 johnw188 mentioned this pull request Nov 9, 2026
3 tasks done
johnw188 and others added 2 commits November 9, 2026 12:45
On a corrupted database, the transactions used by the integrity check
can fail even when the check itself runs. For example, after
`PRAGMA quick_check` reads a page whose SQLCipher HMAC doesn't match,
COMMIT fails with "SQL logic error". `read` and `write` owsFail on any
error, so tapping "Continue" on the database recovery screen crashed
before recovery could start.

Use the throwing variants, as `DatabaseRecovery.reindex` already does.
If a check ran, use its result even if ending the transaction failed;
if it couldn't run, treat it as failed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When copying a table hit a corrupted page, two things went wrong:

- Iteration stopped, so every row after the page was lost.
- Ending the read transaction also failed, and copyTable reported a
  total failure, even though the rows it copied were already committed
  to the new database.

For tables with rowids, copy in rowid order and, after a read error,
search for the next readable rowid and continue from there. Stop a pass
if rowids go backwards, which a corrupted b-tree can cause, so rows are
never copied twice. Report the rows copied even if ending the read
fails, and prefer reporting SQLITE_FULL, since the user can fix that.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant