Skip to content

Milestone 5: DuckDB-native statement transformers - #4

Open
kyleconroy wants to merge 1 commit into
mainfrom
claude/next-milestones-0emfgq
Open

Milestone 5: DuckDB-native statement transformers#4
kyleconroy wants to merge 1 commit into
mainfrom
claude/next-milestones-0emfgq

Conversation

@kyleconroy

Copy link
Copy Markdown
Contributor

Milestone 5 from PLAN.md: transformers for every remaining statement family, ported from the pinned DuckDB commit (8cbdaba6ac), plus the shape-test coverage for statements json_serialize_sql cannot see. Every Statement alternative now has a transformer — Parse no longer reports ErrUnsupported and the unsupported-statement registry is gone.

New statements (ast/misc.go + parser/transform_misc.go, transform_copy.go, transform_create_misc.go)

  • SET/RESET — all scopes (LOCAL/SESSION/GLOBAL/VARIABLE), the SET SCHEMA and SET TIME ZONE special forms, and upstream's DEFAULT-to-RESET folding. PRAGMA — call form, assignment-to-SET desugaring, SQLite-compat pragmas (table_info), named parameters.
  • CALL, USE, CHECKPOINT, transactions, VACUUM/ANALYZE, EXPORT/IMPORT DATABASE, PREPARE/EXECUTE/DEALLOCATE, EXPLAIN — with the preparable-statement check, positional-argument naming for EXECUTE, and EXPLAIN (options) handling.
  • ATTACH/DETACH/CONNECT/DISCONNECT, COMMENT ON, INSTALL/FORCE INSTALL/LOAD/UPDATE EXTENSIONS, and the external-resource statements.
  • COPY in all forms: table TO/FROM with the generic and PostgreSQL-style specialized option lists, format extraction from extensions, COPY (SELECT) TO, COPY FROM DATABASE (flagged form and the PRAGMA copy_database desugaring), and COPY TO as a CTE body via a new CopyQueryNode (a DuckDB 2.0 feature darkwing previously rejected).
  • CREATE MACRO / SECRET / TRIGGER as CreateInfo variants: macro overloads and parameter defaults, secret TYPE/PROVIDER/SCOPE extraction with default naming, trigger timing/events/REFERENCING/FOR EACH/body.

Where upstream desugars a statement into another family (USE→SET, CHECKPOINT→CALL, IMPORT→PRAGMA, DEALLOCATE→DROP, ANALYZE→VACUUM), darkwing keeps a dedicated node per PLAN.md's AST list and notes the upstream shape — for sqlc's benefit.

Transformer-raised Parser Errors

The corpus's todo entries expected upstream's transformer rejects to clear as transformers land, so this PR also ports that long tail: recursive-CTE restrictions (COPY/DML bodies, LIMIT/ORDER BY, duplicate names, n-ary union splitting), star EXCLUDE/REPLACE/RENAME interactions (with upstream's prefix-matching column comparator), WITHIN GROUP ordered aggregates, window-definition rules (frame copying, override checks, ORDER BY ALL, window functions inside definitions, frame-bound direction), VALUES row lengths, empty selection lists, CUBE/ROLLUP and grouping-set limits, sample ranges, cast-then-subscript, FK actions and column counts, CHECK subqueries, generated-column restrictions, sequence-option validation (duplicates, NULLs, evaluated range checks), ALTER restrictions, TIMESTAMP/TIME modifiers, array bounds, pivot checks (constants, subqueries, data-extracted pivots in views/macros/with parameters), MERGE unreachable clauses, and escape-string NUL/UTF-8 validation.

One milestone-3 behavior is corrected: mixing named and positional parameters now accepts — upstream raises NotImplementedException, which the corpus oracle classifies as post-parse.

Conformance

  • 139 corpus todo entries cleared (227 → 88). The accept/reject gate agrees with the oracle on every non-todo case.
  • The 88 remaining todos are oracle rejects raised outside the parse pipeline — bind-time validation that happens to throw ParserException (setting values, function argument checks, PRIMARY KEY verification) and nested SQL parsing inside query()/nextval(). Their notes now say so, pointing at milestone 6's error-fidelity work.
  • parser/misc_test.go adds the milestone-5 shape tests (the meyer-style snapshot substitute for non-SELECT statements); the serialize goldens gate stayed green throughout.
  • Full dev loop green: go build, go vet, gofmt, go test -race ./....

🤖 Generated with Claude Code

https://claude.ai/code/session_01SDnHdNTsKPFE18AvkqT38C


Generated by Claude Code

Every Statement alternative now has a transformer; Parse no longer
reports ErrUnsupported and the unsupported-statement registry is gone.

New AST nodes (ast/misc.go) and transformers for:

- SET/RESET (scopes, SET SCHEMA, SET TIME ZONE, DEFAULT-to-RESET
  folding) and PRAGMA (call, assignment-to-SET, SQLite-compat pragmas,
  named parameters)
- CALL, USE, CHECKPOINT, transactions, VACUUM/ANALYZE, EXPORT/IMPORT
  DATABASE, DEALLOCATE, PREPARE/EXECUTE (preparable-statement check,
  positional argument naming), EXPLAIN [ANALYZE] with options
- ATTACH/DETACH/CONNECT/DISCONNECT and COMMENT ON
- COPY TO/FROM (generic + PostgreSQL-style specialized option lists,
  format extraction, option folding), COPY (SELECT) TO, COPY FROM
  DATABASE (flagged form and the PRAGMA copy_database desugaring), and
  COPY TO as a CTE body (new CopyQueryNode)
- INSTALL/FORCE INSTALL/LOAD/UPDATE EXTENSIONS and the external
  resource statements
- CREATE MACRO (overloads, defaults), CREATE SECRET (type/provider/
  scope extraction, default names) and CREATE TRIGGER

The port also reproduces upstream's transformer-raised Parser Errors so
the corpus accept/reject gate agrees without todo entries: recursive CTE
restrictions (COPY/DML bodies, LIMIT/ORDER BY, duplicate names), star
EXCLUDE/REPLACE/RENAME interactions, WITHIN GROUP ordered aggregates,
window definition rules (frame copying, override checks, ORDER BY ALL,
window functions inside definitions, frame bound direction), VALUES row
lengths, empty selection lists, CUBE/ROLLUP and grouping-set limits,
sample ranges, cast-then-subscript, foreign key actions and column
counts, CHECK subqueries, generated column restrictions, sequence option
validation, ALTER restrictions, TIMESTAMP/TIME modifiers, array bounds,
pivot-on-constant and data-extracted pivot checks (views, macros,
parameters), MERGE unreachable clauses, and escape-string NUL/UTF-8
validation. Parameter mixing now accepts (upstream raises
NotImplemented, which the oracle classifies post-parse).

139 corpus todo entries cleared; the 88 that remain are oracle rejects
raised outside the parse pipeline (bind-time validation and nested SQL
parsing) and their notes now say so.

parser/misc_test.go adds the milestone-5 shape coverage for statements
json_serialize_sql cannot see.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SDnHdNTsKPFE18AvkqT38C
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants