Repository navigation
test: pin English per-unit boundary and dash regressions - #405
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
English TN in the IndexTTS-pinned WeTextProcessing 1.0.4.1 misreads ordinary prose such as
per dayas a unit prefix and speaks a synthetic minus sign in validper kgrates. Independent installs confirm that released 1.3.0 and current master already fix both behaviors. Although the unit-only expression still looks similar, sentence token boundaries and raw-field verbalization have changed.Add 21 regressions for #403 rather than changing working production rule weights:
per day/hour/second/month/minute/page/week/year/personprose must remain unchanged, with no partialmeasureclassification.per kg/km/min, slash rates, and punctuation retain unit expansion without a spoken dash.h, available without imposing a new preference order.Verification
Installed PyPI versions into separate temporary target directories and ran them from outside the source checkout, rebuilding graphs (no old cached models). Both used Python 3.11 / Pynini 2.1.7 on macOS arm64:
It happens per day.It happens - per dalton y.It happens per day.It happens per hour.It happens - per hour our.It happens per hour.price per kgprice - per kilogramprice per kilogram5 km per hourfive kilometers - per hour ourfive kilometers per hourThe deployment resolution is to upgrade the pinned package to the verified 1.3.0 release, updating the consuming application's dependency lock as necessary. The token-boundary work in #365 and raw-field refactor
99dc9daexplain why inspecting onlyunit_only_graphdoes not reflect the current pipeline behavior.Tests
python -m pytest tn/english/test itn/english/test -q --tb=short: 676 passed, including all 21 new cases.Related to #403. This PR adds regression coverage for an already-fixed issue; it does not introduce another grammar change.