Skip to content

Make pg_tde_waldump aware of pg_tde rmgr - #640

Open
dAdAbird wants to merge 4 commits into
percona:mainfrom
dAdAbird:waldump_tde
Open

dAdAbird wants to merge 4 commits into
percona:mainfrom
dAdAbird:waldump_tde

Conversation

@dAdAbird

@dAdAbird dAdAbird commented Jul 23, 2026

Copy link
Copy Markdown
Member

Sometimes, during the debug of pg_tde-related issues, it is really helpful to see what exactly pg_tde rmgr added to WAL. But pg_tde_waldump doesn't have insights into custom rmgr, and all it displays is generic desc: UNKNOWN (50) rmid: 140.

This PR makes it aware of pg_tde rmgr so that now:

$ pg_tde_waldump -k ../pg_tde --rmgr=pg_tde 000000010000000000000001
...
rmgr: pg_tde ... lsn: 0/01B9D6C8, prev 0/01B9D698, desc: DELETE_RELATION_KEY rel: 1663/16384/13477
rmgr: pg_tde ... lsn: 0/01B9D7B0, prev 0/01B9D720, desc: DELETE_RELATION_KEY rel: 1663/16384/13473
rmgr: pg_tde ... lsn: 0/01BB65E8, prev 0/01BB6598, desc: INSTALL_EXTENSION db: 16384
rmgr: pg_tde ... lsn: 0/01BBB3B0, prev 0/01BBB378, desc: WRITE_KEY_PROVIDER db: 1664, provider id: -1
rmgr: pg_tde ... lsn: 0/01BBB8E8, prev 0/01BBB8B0, desc: ADD_PRINCIPAL_KEY db: 1664
rmgr: pg_tde ... lsn: 0/01BBBA80, prev 0/01BBBA40, desc: WRITE_KEY_PROVIDER db: 16384, provider id: 1
rmgr: pg_tde ... lsn: 0/01BBBF38, prev 0/01BBBA80, desc: ADD_PRINCIPAL_KEY db: 16384

instead of:

$ pg_tde_waldump -k ../pg_tde --rmgr=custom140 000000010000000000000001
...
rmgr: custom140 ... lsn: 0/01B9D6C8, prev 0/01B9D698, desc: UNKNOWN (50) rmid: 140
rmgr: custom140 ... lsn: 0/01B9D7B0, prev 0/01B9D720, desc: UNKNOWN (50) rmid: 140
rmgr: custom140 ... lsn: 0/01BB65E8, prev 0/01BB6598, desc: UNKNOWN (40) rmid: 140
rmgr: custom140 ... lsn: 0/01BBB3B0, prev 0/01BBB378, desc: UNKNOWN (30) rmid: 140
rmgr: custom140 ... lsn: 0/01BBB8E8, prev 0/01BBB8B0, desc: UNKNOWN (10) rmid: 140
rmgr: custom140 ... lsn: 0/01BBBA80, prev 0/01BBBA40, desc: UNKNOWN (30) rmid: 140
rmgr: custom140 ... lsn: 0/01BBBF38, prev 0/01BBBA80, desc: UNKNOWN (10) rmid: 140

All changes currently are only for pg18. I'll add pg17 and pg16 after approval.

PG-2754

@codecov-commenter

codecov-commenter commented Jul 23, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 56.96%. Comparing base (6992ce3) to head (ed56b6a).
⚠️ Report is 2 commits behind head on main.

❌ Your project status has failed because the head coverage (77.05%) is below the target coverage (90.00%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #640      +/-   ##
==========================================
+ Coverage   56.69%   56.96%   +0.27%     
==========================================
  Files          71       71              
  Lines       11275    11282       +7     
  Branches     2786     2787       +1     
==========================================
+ Hits         6392     6427      +35     
+ Misses       3517     3487      -30     
- Partials     1366     1368       +2     
Components Coverage Δ
access 83.16% <ø> (+2.21%) ⬆️
bin 63.86% <ø> (ø)
catalog 80.03% <ø> (ø)
common 91.66% <ø> (ø)
encryption 65.56% <ø> (ø)
keyring 64.83% <ø> (ø)
src 85.50% <ø> (ø)
smgr 90.59% <ø> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jeltz jeltz left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should add some tests.

Comment thread fetools/pg18/pg_waldump/rmgrdesc.c Outdated
}

/* Initialize pg_tde resource manager */
Assert(RmgrIdIsValid(RM_TDERMGR_ID));

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I get why you have this assert.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This constant comes from the TDE code and used later in array indexing. So I'm just being precocious in case it was accidentally changed etc. I wasn't 100% sure about it, so if you deem it redundant I ok with removing it.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems redudant to me.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed

@jeltz

jeltz commented Jul 24, 2026

Copy link
Copy Markdown
Collaborator

This should also add a changelog file since we dicided to introduce those.

@dAdAbird
dAdAbird force-pushed the waldump_tde branch 3 times, most recently from 6b10390 to 4180f37 Compare September 10, 2026 14:15
@dAdAbird
dAdAbird requested a review from jeltz September 10, 2026 14:50
@dAdAbird

Copy link
Copy Markdown
Member Author

Tests added, changelog changed.
Current tests failures due to only pg18 implementation atm.

This commit makes pg_tde_waldump print the proper name and
description for pg_tde rmgr records. Instead of just generic
"desc: UNKNOWN (%d) rmid: %d"

pg_tde_waldump output before:
```
rmgr: custom140   len (rec/tot):     30/    30, tx:        753, lsn: 0/01BB65E8, prev 0/01BB6598, desc: UNKNOWN (40) rmid: 140
rmgr: custom140   len (rec/tot):   1205/  1205, tx:          0, lsn: 0/01BBB3B0, prev 0/01BBB378, desc: UNKNOWN (30) rmid: 140
rmgr: custom140   len (rec/tot):    341/   341, tx:          0, lsn: 0/01BBB8E8, prev 0/01BBB8B0, desc: UNKNOWN (10) rmid: 140
rmgr: custom140   len (rec/tot):   1205/  1205, tx:          0, lsn: 0/01BBBA80, prev 0/01BBBA40, desc: UNKNOWN (30) rmid: 140
rmgr: custom140   len (rec/tot):    341/   341, tx:          0, lsn: 0/01BBBF38, prev 0/01BBBA80, desc: UNKNOWN (10) rmid: 140
```

and after the commit:
```
rmgr: pg_tde      len (rec/tot):     30/    30, tx:        753, lsn: 0/01BB65E8, prev 0/01BB6598, desc: INSTALL_EXTENSION db: 16384
rmgr: pg_tde      len (rec/tot):   1205/  1205, tx:          0, lsn: 0/01BBB3B0, prev 0/01BBB378, desc: WRITE_KEY_PROVIDER db: 1664, provider id: -1
rmgr: pg_tde      len (rec/tot):    341/   341, tx:          0, lsn: 0/01BBB8E8, prev 0/01BBB8B0, desc: ADD_PRINCIPAL_KEY db: 1664
rmgr: pg_tde      len (rec/tot):   1205/  1205, tx:          0, lsn: 0/01BBBA80, prev 0/01BBBA40, desc: WRITE_KEY_PROVIDER db: 16384, provider id: 1
rmgr: pg_tde      len (rec/tot):    341/   341, tx:          0, lsn: 0/01BBBF38, prev 0/01BBBA80, desc: ADD_PRINCIPAL_KEY db: 16384
```
After the previous commit, pg_tde_waldump recognises pg_tde rmgr in
output. But to filter out such records, the user would still have to
set the value of the --rmgr flag in the format "custom140". This
commit makes --rmgr flag recognise "pg_tde" rmgr. So instead of:
```
pg_tde_waldump  --rmgr=custom140  ...
```
the user can specify:
```
pg_tde_waldump  --rmgr=pg_tde  ...
```

`--rmgr=custom140` keeps working as well.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants