Milestone 5: DuckDB-native statement transformers - #4
Open
kyleconroy wants to merge 1 commit into
Open
Conversation
Every Statement alternative now has a transformer; Parse no longer reports ErrUnsupported and the unsupported-statement registry is gone. New AST nodes (ast/misc.go) and transformers for: - SET/RESET (scopes, SET SCHEMA, SET TIME ZONE, DEFAULT-to-RESET folding) and PRAGMA (call, assignment-to-SET, SQLite-compat pragmas, named parameters) - CALL, USE, CHECKPOINT, transactions, VACUUM/ANALYZE, EXPORT/IMPORT DATABASE, DEALLOCATE, PREPARE/EXECUTE (preparable-statement check, positional argument naming), EXPLAIN [ANALYZE] with options - ATTACH/DETACH/CONNECT/DISCONNECT and COMMENT ON - COPY TO/FROM (generic + PostgreSQL-style specialized option lists, format extraction, option folding), COPY (SELECT) TO, COPY FROM DATABASE (flagged form and the PRAGMA copy_database desugaring), and COPY TO as a CTE body (new CopyQueryNode) - INSTALL/FORCE INSTALL/LOAD/UPDATE EXTENSIONS and the external resource statements - CREATE MACRO (overloads, defaults), CREATE SECRET (type/provider/ scope extraction, default names) and CREATE TRIGGER The port also reproduces upstream's transformer-raised Parser Errors so the corpus accept/reject gate agrees without todo entries: recursive CTE restrictions (COPY/DML bodies, LIMIT/ORDER BY, duplicate names), star EXCLUDE/REPLACE/RENAME interactions, WITHIN GROUP ordered aggregates, window definition rules (frame copying, override checks, ORDER BY ALL, window functions inside definitions, frame bound direction), VALUES row lengths, empty selection lists, CUBE/ROLLUP and grouping-set limits, sample ranges, cast-then-subscript, foreign key actions and column counts, CHECK subqueries, generated column restrictions, sequence option validation, ALTER restrictions, TIMESTAMP/TIME modifiers, array bounds, pivot-on-constant and data-extracted pivot checks (views, macros, parameters), MERGE unreachable clauses, and escape-string NUL/UTF-8 validation. Parameter mixing now accepts (upstream raises NotImplemented, which the oracle classifies post-parse). 139 corpus todo entries cleared; the 88 that remain are oracle rejects raised outside the parse pipeline (bind-time validation and nested SQL parsing) and their notes now say so. parser/misc_test.go adds the milestone-5 shape coverage for statements json_serialize_sql cannot see. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SDnHdNTsKPFE18AvkqT38C
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Milestone 5 from PLAN.md: transformers for every remaining statement family, ported from the pinned DuckDB commit (
8cbdaba6ac), plus the shape-test coverage for statementsjson_serialize_sqlcannot see. EveryStatementalternative now has a transformer —Parseno longer reportsErrUnsupportedand the unsupported-statement registry is gone.New statements (
ast/misc.go+parser/transform_misc.go,transform_copy.go,transform_create_misc.go)LOCAL/SESSION/GLOBAL/VARIABLE), theSET SCHEMAandSET TIME ZONEspecial forms, and upstream'sDEFAULT-to-RESETfolding. PRAGMA — call form, assignment-to-SETdesugaring, SQLite-compat pragmas (table_info), named parameters.EXECUTE, andEXPLAIN (options)handling.COPY (SELECT) TO,COPY FROM DATABASE(flagged form and thePRAGMA copy_databasedesugaring), andCOPY TOas a CTE body via a newCopyQueryNode(a DuckDB 2.0 feature darkwing previously rejected).CreateInfovariants: macro overloads and parameter defaults, secretTYPE/PROVIDER/SCOPEextraction with default naming, trigger timing/events/REFERENCING/FOR EACH/body.Where upstream desugars a statement into another family (USE→SET, CHECKPOINT→CALL, IMPORT→PRAGMA, DEALLOCATE→DROP, ANALYZE→VACUUM), darkwing keeps a dedicated node per PLAN.md's AST list and notes the upstream shape — for sqlc's benefit.
Transformer-raised Parser Errors
The corpus's todo entries expected upstream's transformer rejects to clear as transformers land, so this PR also ports that long tail: recursive-CTE restrictions (COPY/DML bodies, LIMIT/ORDER BY, duplicate names, n-ary union splitting), star
EXCLUDE/REPLACE/RENAMEinteractions (with upstream's prefix-matching column comparator),WITHIN GROUPordered aggregates, window-definition rules (frame copying, override checks,ORDER BY ALL, window functions inside definitions, frame-bound direction),VALUESrow lengths, empty selection lists,CUBE/ROLLUPand grouping-set limits, sample ranges, cast-then-subscript, FK actions and column counts,CHECKsubqueries, generated-column restrictions, sequence-option validation (duplicates, NULLs, evaluated range checks),ALTERrestrictions,TIMESTAMP/TIMEmodifiers, array bounds, pivot checks (constants, subqueries, data-extracted pivots in views/macros/with parameters),MERGEunreachable clauses, and escape-string NUL/UTF-8 validation.One milestone-3 behavior is corrected: mixing named and positional parameters now accepts — upstream raises
NotImplementedException, which the corpus oracle classifies as post-parse.Conformance
ParserException(setting values, function argument checks, PRIMARY KEY verification) and nested SQL parsing insidequery()/nextval(). Their notes now say so, pointing at milestone 6's error-fidelity work.parser/misc_test.goadds the milestone-5 shape tests (the meyer-style snapshot substitute for non-SELECT statements); the serialize goldens gate stayed green throughout.go build,go vet,gofmt,go test -race ./....🤖 Generated with Claude Code
https://claude.ai/code/session_01SDnHdNTsKPFE18AvkqT38C
Generated by Claude Code