Skip to content

fix(postgres): back fulltext indexes with a GIN tsvector index - #993

Open
HarshMN2345 wants to merge 1 commit into
mainfrom
fix/postgres-fulltext-gin-index
Open

HarshMN2345 wants to merge 1 commit into
mainfrom
fix/postgres-fulltext-gin-index

Conversation

@HarshMN2345

Copy link
Copy Markdown
Member

Refs appwrite/appwrite#14081

On Postgres, INDEX_FULLTEXT was created as a plain btree on the column, so Query::search could not use it. The search expression called to_tsvector without a config, which is not IMMUTABLE, so it could not be indexed at all. The btree also rejects values above 2704 bytes, which is why saving users with many labels fails (_key_search).

This change uses an explicit config in both search/notSearch and a GIN expression index built from the same helper, so the two expressions always match. A multi-column fulltext index gets one expression per column. Searches only ever use single-attribute fulltext indexes, and a multi-column GIN index can serve a condition on any one of its columns.

Why english rather than simple: the search used to rely on default_text_search_config, which is pg_catalog.english in appwrite/postgres and the official postgres images. Pinning english keeps the same stemming and stop words, so results don't change (for example, room still matches rooms; with simple it matched 0 of 50k rows). Servers set up with a different default would now get English matching. A per-index config could come later.

Migration: fulltext indexes that already exist stay btrees, and from now on search will not use them. They need to be dropped and recreated, ideally with CREATE INDEX CONCURRENTLY on large tables. This PR doesn't include migration tooling.

Verification (appwrite/postgres:0.1.0, PG 18.3, 50k rows; plans captured with auto_explain from Database::find):

  • Before: USING btree (note). search('note','zebra') ran as a pkey index scan that filtered every row (24,975 rows removed, 83 ms). Inserting a 3.7 KB value failed with index row size 3712 exceeds btree version 4 maximum 2704.
  • After: USING gin (to_tsvector('english'::regconfig, regexp_replace(note, ...))). Same search: Bitmap Index Scan on the fulltext index (0.2 ms). The 3.7 KB insert succeeds. search('note','room') still matches rows containing rooms.
  • PostgresTest e2e: 692 tests, the same single pre-existing testCacheFallback failure before and after.

Fulltext indexes were created as plain btrees on the column, which search
queries could never use and which reject values above 2704 bytes. The search
expression called to_tsvector without a config, so it was not IMMUTABLE and
could not be indexed either.

Use an explicit 'english' config (the default_text_search_config of the
official images) in both the search query and a GIN expression index, so the
index matches the query and search results stay the same.
@coderabbitai

coderabbitai Bot commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 46 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Repository: utopia-php/database/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 43d499bb-a97f-4351-8dc3-b7202962dc1f
📥 Commits

Reviewing files that changed from the base of the PR and between 1c99c21 and 8db363c.

📒 Files selected for processing (1)
  • src/Database/Adapter/Postgres.php
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant