Summary
match_type uses re.match rather than re.fullmatch, and the INTEGER pattern
anchors $ on only one of its alternatives. Misspelled types are therefore classified as
native integers and emitted into the DDL verbatim, so the user gets a MySQL syntax error
instead of Unsupported attribute type.
int24 -> INTEGER
intbanana -> INTEGER
integerish -> INTEGER
For contrast, the equivalent typos on other patterns are rejected correctly, because
those patterns end in $:
floatzilla -> DataJointError: Unsupported attribute type floatzilla
bools -> DataJointError
charlie(3) -> DataJointError
Cause
declare.py:110-113:
try:
return next(category for category, pattern in TYPE_PATTERN.items() if pattern.match(attribute_type))
except StopIteration:
raise DataJointError("Unsupported attribute type {type}".format(type=attribute_type))
pattern.match anchors only at the start. Most patterns compensate with a trailing $,
but in INTEGER (declare.py:66) the $ binds to the final alternative only:
INTEGER=r"((tiny|small|medium|big|)int|integer)(\s*\(.+\))?(\s+unsigned)?(\s+auto_increment)?|serial$",
Because | has the lowest precedence, this reads as
(…int|integer)(…)?(…)?(…)? or serial$. The integer branch has no terminal anchor,
so int followed by anything at all matches. The same latent issue applies to any pattern
whose $ is inside an alternation.
Impact
A typo in a definition produces a confusing failure at the database layer rather than a
clear DataJoint error. int24 is the case most likely to be hit in practice: it looks
like a plausible core type, migrate.py actually emits it (see the companion issue on
uint*/int24 markers), and it is accepted here — then MySQL rejects int24 as a
column type with a syntax error that does not mention the attribute name.
Proposed fix
Use fullmatch, and drop the now-redundant $ anchors:
return next(category for category, pattern in TYPE_PATTERN.items() if pattern.fullmatch(attribute_type))
fullmatch is the safer form here because it cannot be defeated by an unanchored
alternative in any individual pattern. Every TYPE_PATTERN entry needs checking for
whitespace tolerance if the $ anchors are removed in the same pass — CODEC=r"<.+>$"
and the parameterized string/decimal patterns are the ones to look at.
Note that heading.py:520,537,545 and codecs.py also call .match() against
TYPE_PATTERN; those call sites should move to fullmatch together with this one so the
declaration path and the load path agree on what a type is.
Worth a test that walks a list of near-miss spellings (int24, intbanana, float64x,
varchar, enum()) and asserts each raises.
Summary
match_typeusesre.matchrather thanre.fullmatch, and theINTEGERpatternanchors
$on only one of its alternatives. Misspelled types are therefore classified asnative integers and emitted into the DDL verbatim, so the user gets a MySQL syntax error
instead of
Unsupported attribute type.For contrast, the equivalent typos on other patterns are rejected correctly, because
those patterns end in
$:Cause
declare.py:110-113:pattern.matchanchors only at the start. Most patterns compensate with a trailing$,but in
INTEGER(declare.py:66) the$binds to the final alternative only:Because
|has the lowest precedence, this reads as(…int|integer)(…)?(…)?(…)?orserial$. The integer branch has no terminal anchor,so
intfollowed by anything at all matches. The same latent issue applies to any patternwhose
$is inside an alternation.Impact
A typo in a definition produces a confusing failure at the database layer rather than a
clear DataJoint error.
int24is the case most likely to be hit in practice: it lookslike a plausible core type,
migrate.pyactually emits it (see the companion issue onuint*/int24markers), and it is accepted here — then MySQL rejectsint24as acolumn type with a syntax error that does not mention the attribute name.
Proposed fix
Use
fullmatch, and drop the now-redundant$anchors:fullmatchis the safer form here because it cannot be defeated by an unanchoredalternative in any individual pattern. Every
TYPE_PATTERNentry needs checking forwhitespace tolerance if the
$anchors are removed in the same pass —CODEC=r"<.+>$"and the parameterized string/decimal patterns are the ones to look at.
Note that
heading.py:520,537,545andcodecs.pyalso call.match()againstTYPE_PATTERN; those call sites should move tofullmatchtogether with this one so thedeclaration path and the load path agree on what a type is.
Worth a test that walks a list of near-miss spellings (
int24,intbanana,float64x,varchar,enum()) and asserts each raises.