Partial CJK corruption on CIDFontType2 subsets whose ToUnicode is a full-range identity bfrange (dompdf)
Summary
For PDFs whose /ToUnicode CMap consists of the single degenerate range
<0000> <FFFF> <0000>, roughly a third of CJK characters extract as the raw
UTF-16BE bytes of the intended codepoint, split into two Latin-1 characters. The
remaining two thirds decode correctly, so the output looks structurally plausible.
The font is CIDFontType2 (subset TrueType, Identity-H), so this is not #208 —
that issue covers try_remap_subset_cmap misfiring on CIDFontType0/CFF, where the
repair should never run. Here the repair path is the correct one for the font type,
but the source CMap it repairs against is a producer-authored identity assertion
rather than a real mapping, and the result is only partially right.
It is also not caught by the #120 CipherGarbleStats discriminator (which resolved
#118). That heuristic is explicitly Latin-dominant — it requires "≥200 ASCII
letters" and keys on vowel ratio and intra-word case transitions. A page that is
mostly correct CJK with ~33% of its ideographs replaced presents no ASCII letter
sample to measure, so the detector cannot fire by construction. hasEncodingIssues
stays false, confidence stays 1.0, and pdfType stays TextBased.
Environment
@firecrawl/pdf-inspector 1.12.0 (npm, prebuilt darwin-arm64)- macOS 15 (Darwin 25.6.0), Apple Silicon
- Producer: dompdf 3.1.6 + CPDF, subset CIDFontType2, Identity-H, embedded
Noto Sans HK
Reproduction
The generator below renders 300 sequential codepoints from U+4E00, one per line, as
plain <p> text — no tables, no headings, no styling — with an embedded
NotoSansHK-Regular.ttf (Google Fonts). isolate.pdf is attached, along with the
invoice-shaped repro-cjk-dompdf.pdf.
<?php
require 'vendor/autoload.php'; // composer require dompdf/dompdf
use Dompdf\Dompdf;
use Dompdf\Options;
$options = new Options();
$options->set('isRemoteEnabled', true);
$options->set('chroot', '/');
$dompdf = new Dompdf($options);
$lines = '';
for ($i = 0; $i < 300; $i++) {
$ch = mb_chr(0x4E00 + $i, 'UTF-8');
$lines .= "<p>" . str_pad($i, 3, '0', STR_PAD_LEFT) . " {$ch}</p>\n";
}
$html = <<<HTML
<html><head><meta charset="utf-8"><style>
@font-face { font-family:'notosanshk'; font-style:normal; font-weight:normal;
src: url('./fonts/NotoSansHK-Regular.ttf') format('truetype'); }
body { font-family:'notosanshk', sans-serif; font-size: 9pt; }
p { margin: 0; }
</style></head><body>
{$lines}
</body></html>
HTML;
$dompdf->loadHtml($html, 'UTF-8');
$dompdf->setPaper('A4', 'portrait');
$dompdf->render();
file_put_contents('isolate.pdf', $dompdf->output());pdf-inspector isolate.pdf
Expected, and what pdftotext isolate.pdf - returns:
000 一
001 丁
002 丂
003 七
004 丄
005 丅
006 丆
Actual:
000 一 001 丁 N 003 七 004 丄 005 N 006 N 007 万 008 丈 ...
191 of 286 characters decode correctly; 95 (33%) are corrupted.
Each corrupted character is emitted as the two bytes of its UTF-16BE form, decoded
as separate Latin-1 characters:
| Index |
Expected |
Codepoint |
Actual |
| 005 |
丅 |
U+4E05 |
N + U+0005 — N is 0x4E, the high byte |
| 035 |
丣 |
U+4E23 |
N# (0x4E, 0x23) |
| 052 |
临 |
U+4E34 |
N4 (0x4E, 0x34) |
| 261 |
伅 |
U+4F05 |
O + U+0005 — O is 0x4F |
repro-cjk-dompdf.pdf is a second, invoice-shaped repro built the same way. There
the corruption is mixed within a single page and a single font: Customer : Mr. 陳大文 decodes correctly, while a table cell returns
fz€ýz—|>–ûj_ N[WW‹ U®•‹ for 智能窗簾電機 一字型 單開. All content in both
files is invented placeholder data.
The CMap
The /ToUnicode stream in these files contains exactly one range:
begincmap
...
beginbfrange
<0000> <FFFF> <0000>
endbfrange
endcmap
dompdf asserts that every CID equals its Unicode codepoint. For a subset font,
where CIDs are reassigned glyph indices, that is false — but it is well-formed, so
it parses cleanly and looks authoritative.
Suggested fix
Treat a full-range identity bfrange (<0000> <FFFF> <0000>) as absent rather
than authoritative, and reverse-map glyph IDs through the embedded TrueType cmap
table instead. Unlike the file in #118 — which had no cmap table and therefore no
in-file source of truth — these fonts embed a complete cmap, which is how poppler
recovers the text. ttf-parser is already a dependency.
If the decode itself is not changed, this shape at least seems cheaply detectable:
"the only ToUnicode mapping present is a full-range identity" is a structural
signal, not a statistical one, and would let hasEncodingIssues fire here. That
would partly serve #122 as well, without needing per-font metadata plumbing.
A CJK-side counterpart to CipherGarbleStats may also be worth considering: an
ideograph run interrupted by isolated C0 control characters and stray Latin capitals
is a strong corruption signal, and is what this bug produces every time.
Impact
The affected files are ordinary business documents — invoices and quotations from a
dompdf-based system. Across 22 such files, essentially all Chinese was lost, worst
case 9 of 1219 characters recovered, with no error raised and full reported
confidence. Latin text in the same documents extracts perfectly, so nothing looks
wrong at a glance.
Everything else about the library has been excellent — classification accuracy and
speed are as advertised, and non-dompdf CJK PDFs (SimSun, Microsoft YaHei, Type 3
Noto CJK) extract flawlessly. Thanks for open-sourcing it.
Partial CJK corruption on CIDFontType2 subsets whose ToUnicode is a full-range identity
bfrange(dompdf)Summary
For PDFs whose
/ToUnicodeCMap consists of the single degenerate range<0000> <FFFF> <0000>, roughly a third of CJK characters extract as the rawUTF-16BE bytes of the intended codepoint, split into two Latin-1 characters. The
remaining two thirds decode correctly, so the output looks structurally plausible.
The font is CIDFontType2 (subset TrueType, Identity-H), so this is not #208 —
that issue covers
try_remap_subset_cmapmisfiring on CIDFontType0/CFF, where therepair should never run. Here the repair path is the correct one for the font type,
but the source CMap it repairs against is a producer-authored identity assertion
rather than a real mapping, and the result is only partially right.
It is also not caught by the #120
CipherGarbleStatsdiscriminator (which resolved#118). That heuristic is explicitly Latin-dominant — it requires "≥200 ASCII
letters" and keys on vowel ratio and intra-word case transitions. A page that is
mostly correct CJK with ~33% of its ideographs replaced presents no ASCII letter
sample to measure, so the detector cannot fire by construction.
hasEncodingIssuesstays
false,confidencestays1.0, andpdfTypestaysTextBased.Environment
@firecrawl/pdf-inspector1.12.0 (npm, prebuiltdarwin-arm64)Noto Sans HK
Reproduction
The generator below renders 300 sequential codepoints from U+4E00, one per line, as
plain
<p>text — no tables, no headings, no styling — with an embeddedNotoSansHK-Regular.ttf(Google Fonts).isolate.pdfis attached, along with theinvoice-shaped
repro-cjk-dompdf.pdf.Expected, and what
pdftotext isolate.pdf -returns:Actual:
191 of 286 characters decode correctly; 95 (33%) are corrupted.
Each corrupted character is emitted as the two bytes of its UTF-16BE form, decoded
as separate Latin-1 characters:
N+ U+0005 —Nis 0x4E, the high byteN#(0x4E, 0x23)N4(0x4E, 0x34)O+ U+0005 —Ois 0x4Frepro-cjk-dompdf.pdfis a second, invoice-shaped repro built the same way. Therethe corruption is mixed within a single page and a single font:
Customer : Mr. 陳大文decodes correctly, while a table cell returnsfz€ýz—|>–ûj_ N[WW‹ U®•‹for智能窗簾電機 一字型 單開. All content in bothfiles is invented placeholder data.
The CMap
The
/ToUnicodestream in these files contains exactly one range:dompdf asserts that every CID equals its Unicode codepoint. For a subset font,
where CIDs are reassigned glyph indices, that is false — but it is well-formed, so
it parses cleanly and looks authoritative.
Suggested fix
Treat a full-range identity
bfrange(<0000> <FFFF> <0000>) as absent ratherthan authoritative, and reverse-map glyph IDs through the embedded TrueType
cmaptable instead. Unlike the file in #118 — which had no
cmaptable and therefore noin-file source of truth — these fonts embed a complete
cmap, which is how popplerrecovers the text.
ttf-parseris already a dependency.If the decode itself is not changed, this shape at least seems cheaply detectable:
"the only ToUnicode mapping present is a full-range identity" is a structural
signal, not a statistical one, and would let
hasEncodingIssuesfire here. Thatwould partly serve #122 as well, without needing per-font metadata plumbing.
A CJK-side counterpart to
CipherGarbleStatsmay also be worth considering: anideograph run interrupted by isolated C0 control characters and stray Latin capitals
is a strong corruption signal, and is what this bug produces every time.
Impact
The affected files are ordinary business documents — invoices and quotations from a
dompdf-based system. Across 22 such files, essentially all Chinese was lost, worst
case 9 of 1219 characters recovered, with no error raised and full reported
confidence. Latin text in the same documents extracts perfectly, so nothing looks
wrong at a glance.
Everything else about the library has been excellent — classification accuracy and
speed are as advertised, and non-dompdf CJK PDFs (SimSun, Microsoft YaHei, Type 3
Noto CJK) extract flawlessly. Thanks for open-sourcing it.