From 4da009d9c835828a9ad8656b72d94ae921bf7513 Mon Sep 17 00:00:00 2001 From: jonathanprozzi Date: Thu, 24 Sep 2026 13:02:51 -0400 Subject: [PATCH 1/3] =?UTF-8?q?feat(iid,=20iid-spec):=20typed=20Wikidata?= =?UTF-8?q?=20identity=20`int:wd::Q=E2=80=A6`=20=E2=80=94=20spec=20a?= =?UTF-8?q?mendment=20and=20reference=20implementation=20(P16)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Ports intuition-v2's typed-wd identity onto the release train as one unit, because the iid conformance test executes the iid-spec vectors. iid-spec - schemes/wd.md: value grammar `^(?::)?Q[1-9]\d*$`, ordered canonicalization for typed forms, value-typed scheme typing (bare polymorphic; typed with an active EntitySchema binding unambiguous and P0-eligible; dormant binding valid but not mintable), the closed binding table with pinned revisions, typed vectors, ladder-rung note - spec/07 §7.3: value-typed schemes defined; wd rows split bare/typed - spec/09 §9.3.1: additive value-grammar extensions (in-place MINOR ratification under four conditions); wd is the first instance - conformance: value-typed corpus support (`valueTyped` on scheme entries, optional per-vector `typing`, validator effective-typing guards); 18 new wd vectors; all 145 base vectors unchanged apart from one note; scheme typing entries unchanged - README / schemes README updated iid - wd-entityschema-bindings.ts: the 11 pinned bindings (film, television-series, human active), deep-frozen table/rows/anchorQids, frozen slug list, membership guards - schemes.ts: typed-aware wd canonicalizer (URL forms never infer a slug, ASCII slug lowercased and must be registered, QID uppercased); the ratified whitespace trim is kept for every form; SCHEME_TYPING.wd stays polymorphic - types.ts: `wdSlug` on the declarative scheme rung, `dormant-wd-binding` ineligibility reason, `wdSlug` on the valid inspection branch - parse.ts: per-value inspection for typed wd (active → unambiguous and eligible; dormant → ineligible) - derive.ts: typed derivation on the declarative engine (active slug only, same-slug check, bare-prefixing, length bound); slugless wd rungs keep minting bare values and bare values only - index.ts exports; README section; tests re-expressed against declarative ladders, fence test replaced by a positive export test Ruled: R1 (typed-wd canonical for the packages). Default-in-effect, pending ratification: R17 (§9.3.1), D-P16-1..5. No version or dependency changes; the version freeze is a separate commit. Gates: iid-spec 163 vectors; iid 263 tests, typecheck, check, build; iid-registry 42, iid-ladder 12, classifications 38, primitives 140 against the rebuilt dist; pack dry run. Review: round 1 pair found three real issues (trim regression, typed values minting through slugless rungs, mutable bindings table), fixed; round-2 combined re-review found no new defect. --- packages/iid-spec/README.md | 23 +- .../conformance/canonicalization.json | 70 ++++++ packages/iid-spec/conformance/parse.json | 14 ++ packages/iid-spec/conformance/schema.json | 10 +- packages/iid-spec/conformance/schemes.json | 3 +- packages/iid-spec/conformance/validation.json | 48 ++++- packages/iid-spec/schemes/README.md | 2 +- packages/iid-spec/schemes/wd.md | 57 ++++- .../spec/07-representation-profiles.md | 15 +- .../iid-spec/spec/09-registry-governance.md | 11 + packages/iid-spec/test/validate-fixtures.mjs | 19 +- packages/iid/README.md | 16 +- packages/iid/src/__tests__/derive.test.ts | 204 +++++++++++++++++- packages/iid/src/__tests__/inspect.test.ts | 63 ++++++ packages/iid/src/__tests__/schemes.test.ts | 120 ++++++++++- packages/iid/src/derive.ts | 45 +++- packages/iid/src/index.ts | 7 + packages/iid/src/parse.ts | 27 ++- packages/iid/src/schemes.ts | 43 +++- packages/iid/src/types.ts | 10 +- packages/iid/src/wd-entityschema-bindings.ts | 172 +++++++++++++++ 21 files changed, 934 insertions(+), 45 deletions(-) create mode 100644 packages/iid/src/wd-entityschema-bindings.ts diff --git a/packages/iid-spec/README.md b/packages/iid-spec/README.md index 273ba1f..82c7924 100644 --- a/packages/iid-spec/README.md +++ b/packages/iid-spec/README.md @@ -72,6 +72,25 @@ import vectors from '@0xintuition/iid-spec/conformance/canonicalization.json' **Class C — derived** (a hash of the few facts we actually have) [`gen1`](./schemes/gen1.md) +## Typed Wikidata identifiers + +[`wd`](./schemes/wd.md) is a value-typed scheme: bare `int:wd:Q42` remains polymorphic and floors at P1; active `:Q…` bindings carry the type and qualify for P0. The active slugs are `film`, `television-series`, and `human`. Dormant bindings remain valid for parsing and validation but are not mintable. URLs canonicalize to bare QIDs and never infer a slug. + +The reference implementation reports typing per value: + +```ts +import { inspectIntuitionId } from '@0xintuition/iid' + +inspectIntuitionId('int:wd:Q42') +// { valid: true, typing: 'polymorphic', anchorEligible: false, +// anchorIneligibilityReason: 'polymorphic-scheme', … } + +inspectIntuitionId('int:wd:film:Q188035') +// { valid: true, typing: 'unambiguous', anchorEligible: true, wdSlug: 'film', … } +``` + +The binding table and canonicalization rules are in [`schemes/wd.md`](./schemes/wd.md); per-value typing is defined in [§7.3](./spec/07-representation-profiles.md). This additive extension follows [§9.3.1](./spec/09-registry-governance.md), preserving existing bare identifiers and their eligibility. + ## The conformance corpus The [`conformance/`](./conformance/) directory is the machine-readable half of this specification. Every vector is normative, generated from the reference implementation, and cross-checked against the golden values stated in the specification prose. @@ -84,7 +103,7 @@ The [`conformance/`](./conformance/) directory is the machine-readable half of t | `gen1.json` | Full `gen1` derivations: recipe fields, exact preimage, and resulting IID | | `parse.json` | Grammar/parsing vectors, including colon-bearing values and rejections | | `validation.json` | Full-IID validity and P0 anchor-eligibility vectors | -| `schemes.json` | The registry snapshot: identity class and scheme typing per scheme | +| `schemes.json` | The registry snapshot: identity class, scheme typing, and optional `valueTyped` marker per scheme | Fixture files are versioned (`specVersion`, `fixtureVersion`) and append-only: changing an existing vector's expected output is an identity-impacting change and follows the governance process in [§9](./spec/09-registry-governance.md). @@ -101,7 +120,7 @@ Every test vector in this specification is executable against the reference impl ## Status -**Version 0.1.0 — Draft.** The grammar, the class system, NORM-1, and the `gen1` algorithm are stable and in production use. Scheme canonicalization rules are frozen under the [freeze rule](./spec/09-registry-governance.md): once ratified, a scheme's rules never change in place — a change ships as a new scheme name or version. +**Version 0.1.0 — Draft.** The grammar, the class system, NORM-1, and the `gen1` algorithm are stable and in production use. Scheme canonicalization rules are frozen under the [freeze rule](./spec/09-registry-governance.md): changes to existing canonical values require a new scheme name or version; additive value-grammar extensions follow [§9.3.1](./spec/09-registry-governance.md). This specification is published for review and adoption. It is not yet submitted to any standards body. diff --git a/packages/iid-spec/conformance/canonicalization.json b/packages/iid-spec/conformance/canonicalization.json index 3f27caa..2cdfe98 100644 --- a/packages/iid-spec/conformance/canonicalization.json +++ b/packages/iid-spec/conformance/canonicalization.json @@ -709,6 +709,76 @@ "raw": "movie:r4:FB681AFE7D438CAD73AE90A70F1CC55A", "canonical": null, "note": "hash must be lowercase" + }, + { + "id": "canon-wd-102", + "scheme": "wd", + "raw": "film:q42", + "canonical": "film:Q42", + "note": "typed QID uppercases" + }, + { + "id": "canon-wd-103", + "scheme": "wd", + "raw": " \tfilm:q42\r\n", + "canonical": "film:Q42", + "note": "edge whitespace trims" + }, + { + "id": "canon-wd-104", + "scheme": "wd", + "raw": "https://www.wikidata.org/wiki/Q188035", + "canonical": "Q188035", + "note": "URL remains bare; never infers a slug" + }, + { + "id": "canon-wd-105", + "scheme": "wd", + "raw": "bogus:Q1", + "canonical": null, + "note": "unknown binding rejects" + }, + { + "id": "canon-wd-106", + "scheme": "wd", + "raw": "written-worK:Q42", + "canonical": null, + "note": "non-ASCII slug rejects before case folding" + }, + { + "id": "canon-wd-107", + "scheme": "wd", + "raw": "film:Q42", + "canonical": "film:Q42", + "note": "U+FEFF trims like other edge whitespace (base behavior preserved)" + }, + { + "id": "canon-wd-108", + "scheme": "wd", + "raw": "film:Q042", + "canonical": null, + "note": "typed QID leading zero rejects" + }, + { + "id": "canon-wd-109", + "scheme": "wd", + "raw": "FILM:q42", + "canonical": "film:Q42", + "note": "ASCII slug lowercases" + }, + { + "id": "canon-wd-110", + "scheme": "wd", + "raw": "written-work:q42", + "canonical": "written-work:Q42", + "note": "dormant binding remains canonical and parse-only" + }, + { + "id": "canon-wd-111", + "scheme": "wd", + "raw": "Q42", + "canonical": "Q42", + "note": "BOM-wrapped bare QID keeps its base canonicalization" } ] } diff --git a/packages/iid-spec/conformance/parse.json b/packages/iid-spec/conformance/parse.json index 6d2b7a0..f935f12 100644 --- a/packages/iid-spec/conformance/parse.json +++ b/packages/iid-spec/conformance/parse.json @@ -79,6 +79,20 @@ "scheme": null, "value": null, "note": "value exceeds 220-char limit" + }, + { + "id": "parse-012", + "input": "int:wd:film:Q188035", + "scheme": "wd", + "value": "film:Q188035", + "note": "typed Wikidata value survives first-two-colon split" + }, + { + "id": "parse-013", + "input": "int:wd:Q42", + "scheme": "wd", + "value": "Q42", + "note": "legacy bare Wikidata parse remains unchanged" } ] } diff --git a/packages/iid-spec/conformance/schema.json b/packages/iid-spec/conformance/schema.json index eb2275d..040c383 100644 --- a/packages/iid-spec/conformance/schema.json +++ b/packages/iid-spec/conformance/schema.json @@ -140,6 +140,10 @@ "id": { "$ref": "#/$defs/vectorId" }, "iid": { "type": "string" }, "valid": { "type": "boolean" }, + "typing": { + "enum": ["unambiguous", "polymorphic"], + "description": "Effective typing of THIS identifier; permitted only for a scheme marked `valueTyped`; defaults to the scheme's typing." + }, "anchorEligible": { "type": "boolean", "description": "Whether the IID may be minted as a bare P0 anchor (spec §7.2)." @@ -163,7 +167,11 @@ "additionalProperties": false, "properties": { "class": { "enum": ["A", "B", "C"] }, - "typing": { "enum": ["unambiguous", "polymorphic"] } + "typing": { "enum": ["unambiguous", "polymorphic"] }, + "valueTyped": { + "type": "boolean", + "description": "Bare values are polymorphic, but a value MAY carry the entity type inside the value (spec §7.3); validation vectors for such a scheme MAY state per-value typing." + } } } } diff --git a/packages/iid-spec/conformance/schemes.json b/packages/iid-spec/conformance/schemes.json index dbd8b46..e90c6af 100644 --- a/packages/iid-spec/conformance/schemes.json +++ b/packages/iid-spec/conformance/schemes.json @@ -105,7 +105,8 @@ }, "wd": { "class": "A", - "typing": "polymorphic" + "typing": "polymorphic", + "valueTyped": true } } } diff --git a/packages/iid-spec/conformance/validation.json b/packages/iid-spec/conformance/validation.json index 29919ae..1d84e56 100644 --- a/packages/iid-spec/conformance/validation.json +++ b/packages/iid-spec/conformance/validation.json @@ -43,7 +43,7 @@ "iid": "int:wd:Q42", "valid": true, "anchorEligible": false, - "note": "valid but polymorphic — floors at P1" + "note": "valid but bare — polymorphic, floors at P1" }, { "id": "valid-007", @@ -100,6 +100,52 @@ "valid": true, "anchorEligible": true, "note": "strong acct form is anchor eligible" + }, + { + "id": "valid-015", + "iid": "int:wd:film:Q188035", + "valid": true, + "typing": "unambiguous", + "anchorEligible": true, + "note": "active film binding is P0 eligible" + }, + { + "id": "valid-016", + "iid": "int:wd:human:Q42", + "valid": true, + "typing": "unambiguous", + "anchorEligible": true, + "note": "active human binding is P0 eligible" + }, + { + "id": "valid-017", + "iid": "int:wd:written-work:Q47461344", + "valid": true, + "typing": "unambiguous", + "anchorEligible": false, + "note": "dormant written-work binding is valid but not mintable" + }, + { + "id": "valid-018", + "iid": "int:wd:film:q42", + "valid": false, + "anchorEligible": false, + "note": "typed QID must be canonical byte-exactly" + }, + { + "id": "valid-019", + "iid": "int:wd:bogus:Q1", + "valid": false, + "anchorEligible": false, + "note": "unknown binding rejects" + }, + { + "id": "valid-020", + "iid": "int:wd:television-series:Q137400033", + "valid": true, + "typing": "unambiguous", + "anchorEligible": true, + "note": "active television series binding is P0 eligible" } ] } diff --git a/packages/iid-spec/schemes/README.md b/packages/iid-spec/schemes/README.md index fca1231..05b591e 100644 --- a/packages/iid-spec/schemes/README.md +++ b/packages/iid-spec/schemes/README.md @@ -15,7 +15,7 @@ This directory is the normative scheme registry referenced by [§9.1](../spec/09 | [`gtin`](./gtin.md) | A | T1 | unambiguous | A trade item (product) | | [`doi`](./doi.md) | A | T1 | polymorphic | A registered digital object | | [`eidr`](./eidr.md) | A | T1 | unambiguous | An audiovisual work | -| [`wd`](./wd.md) | A | T2 | polymorphic | Any Wikidata entity | +| [`wd`](./wd.md) | A | T2 | polymorphic (bare) · unambiguous (typed, active binding) | Any Wikidata entity (value-typed) | | [`mbid`](./mbid.md) | A | T2 | unambiguous | A MusicBrainz entity (type in-value) | | [`olid`](./olid.md) | A | T2 | unambiguous | An OpenLibrary work, edition, or author | | [`imdb`](./imdb.md) | A | T3 | polymorphic | An IMDb title or name | diff --git a/packages/iid-spec/schemes/wd.md b/packages/iid-spec/schemes/wd.md index 36fbf2b..d7f022b 100644 --- a/packages/iid-spec/schemes/wd.md +++ b/packages/iid-spec/schemes/wd.md @@ -4,34 +4,56 @@ | :-- | :-- | | Identity class | A — registered authority | | Openness tier | T2 — community registry; full dataset published under CC0 | -| Scheme typing | polymorphic (Wikidata covers every kind of thing) | +| Scheme typing | polymorphic (bare) · unambiguous (typed, active binding) | | Authority | Wikidata | -| Canonical form | `Q` followed by a positive integer with no leading zeros | +| Canonical form | `Q` followed by a positive integer with no leading zeros, optionally prefixed by `:` | ## What it identifies A Wikidata QID names an item in Wikidata: a person, a creative work, a place, an organization, a concept — anything with an item page. Wikidata is the broadest registry in the scheme table and frequently the best cross-reference hub, since items carry external-identifier statements linking to most other registries in this specification. -Because a QID can name literally any kind of entity, the scheme is maximally **polymorphic**: `int:wd:Q42` alone does not say whether Q42 is a person, a book, or a concept. Wikidata-anchored atoms therefore floor at profile P1, where `@type` lives in the payload ([§7.3](../spec/07-representation-profiles.md)). +Because a QID can name literally any kind of entity, bare `wd` is **polymorphic**: `int:wd:Q42` alone does not say whether Q42 is a person, a book, or a concept. Atoms carrying bare `wd` values therefore floor at profile P1, where `@type` lives in the payload ([§7.3](../spec/07-representation-profiles.md)). ## Value grammar ```regex -^Q[1-9]\d*$ +^(?::)?Q[1-9]\d*$ ``` -An uppercase `Q` followed by a positive decimal integer. Leading zeros are prohibited — `Q42` and `Q042` must not both exist as byte-distinct identifiers for the same item ([§2.4](../spec/02-grammar.md)). +`` is one of the registered EntitySchema binding slugs in the closed table below, in lowercase ASCII. The QID is an uppercase `Q` followed by a positive decimal integer. Leading zeros are prohibited — `Q42` and `Q042` must not both exist as byte-distinct identifiers for the same item ([§2.4](../spec/02-grammar.md)). ## Canonicalization -1. Trim leading and trailing whitespace. -2. Strip a Wikidata URL prefix if present. Accepted prefixes (case-insensitive): `http://wikidata.org/wiki/`, `https://wikidata.org/wiki/`, `http://www.wikidata.org/wiki/`, `https://www.wikidata.org/wiki/`, and the same four hosts with `/entity/` in place of `/wiki/`. -3. Uppercase the entire remaining string. -4. Match against `^Q[1-9]\d*$`. On no match, **reject** (return undefined). +1. Trim leading and trailing whitespace. The ratified canonicalizer uses host `trim` semantics (Unicode `White_Space` plus U+FEFF), and that acceptance is frozen under [§9.3](../spec/09-registry-governance.md); the typed form inherits it. +2. Strip a Wikidata URL prefix if present. Accepted prefixes (case-insensitive): `http://wikidata.org/wiki/`, `https://wikidata.org/wiki/`, `http://www.wikidata.org/wiki/`, `https://www.wikidata.org/wiki/`, and the same four hosts with `/entity/` in place of `/wiki/`. Uppercase the remainder and require `^Q[1-9]\d*$`; return that bare QID or **reject**. URLs never infer a slug and cannot contain a typed value after the prefix. +3. For a `:` value, require the QID to match `Q[1-9]\d*` case-insensitively. Reject a non-ASCII slug before case folding; lowercase the slug and require membership in the binding table (active or dormant). Uppercase the QID and return `:`; reject unknown slugs. +4. Otherwise, uppercase the remaining string and require `^Q[1-9]\d*$`. Return the bare QID or **reject** (return undefined). ## Validation -A conforming validator accepts a value iff it matches `^Q[1-9]\d*$`. Only item identifiers qualify: properties (`P31`), lexemes (`L…`), and MediaWiki entity forms are rejected — they identify parts of Wikidata's data model, not entities in the world. No lookup against Wikidata is performed; whether the QID exists, or has been merged into another item, is a resolution concern. +A conforming validator accepts a value iff it is byte-identical to its canonicalization and matches the value grammar with a registered slug, when present. Full IIDs also satisfy the 220-character value limit ([§2.2](../spec/02-grammar.md)). Only item identifiers qualify: properties (`P31`), lexemes (`L…`), and MediaWiki entity forms are rejected — they identify parts of Wikidata's data model, not entities in the world. No lookup against Wikidata is performed; whether the QID exists, conforms to an EntitySchema, or has been merged into another item is a resolution concern. + +## Scheme typing + +`wd` is a **value-typed scheme** ([§7.3](../spec/07-representation-profiles.md)): bare `Q…` values are polymorphic and floor at P1; `:Q…` values carry the entity type. An active binding is unambiguous and P0-eligible. A dormant binding is unambiguous and valid for parsing and validation, but is parse-only and MUST NOT be minted. The active set is `film`, `television-series`, and `human`. + +## EntitySchema bindings + +The table is closed and ordered by binding precedence. Additions and activation changes require review under the process in [§9.4](../spec/09-registry-governance.md); existing identifiers remain stable. Each binding pins its EntitySchema, anchor QID, classification, and revision. The E-number does not determine the anchor QID. + +| Slug | EntitySchema id | Anchor QID | Classification | Status | Pinned revision | +| :-- | :-- | :-- | :-- | :-- | --: | +| `film` | `E11424` | `Q11424` | `Movie` | active | 2403158147 | +| `television-series` | `E17` | `Q5398426` | `TVSeries` | active | 2525771900 | +| `television-series-season` | `E18` | `Q3464665` | `TVSeason` | dormant | 2525772131 | +| `television-series-episode` | `E19` | `Q21191270` | `TVEpisode` | dormant | 2279362550 | +| `written-work` | `E35` | `Q47461344` | `Book` | dormant | 2525775487 | +| `human` | `E10` | `Q5` | `Person` | active | 2499173058 | +| `podcast` | `E418` | `Q24634210` | `PodcastSeries` | dormant | 2052853948 | +| `podcast-episode` | `E420` | `Q61855877` | `PodcastEpisode` | dormant | 2212603393 | +| `video-game` | `E272` | `Q7889` | `VideoGame` | dormant | 2363080401 | +| `album` | `E248` | `Q482994` | `MusicAlbum` | dormant | 2226920960 | +| `organization` | `E98` | `Q43229` | `Organization` | dormant | 2392901167 | ## Test vectors @@ -42,10 +64,23 @@ A conforming validator accepts a value iff it matches `^Q[1-9]\d*$`. Only item i | `Q42` | `Q42` | canonical QID is idempotent | | `Q042` | ✗ reject | leading zero not canonical | | `P31` | ✗ reject | properties are not entities | +| `film:q42` | `film:Q42` | active binding; QID uppercases | +| ` \tfilm:q42\r\n` | `film:Q42` | ASCII edge whitespace trims | +| `HUMAN:q42` | `human:Q42` | ASCII slug lowercases | +| `television-series:Q137400033` | `television-series:Q137400033` | active binding is idempotent | +| `written-work:Q47461344` | `written-work:Q47461344` | dormant binding remains valid | +| `https://www.wikidata.org/wiki/Q188035` | `Q188035` | URL never infers `film` | +| `bogus:Q1` | ✗ reject | unknown binding | +| `written-worK:Q42` | ✗ reject | non-ASCII slug rejects before case folding | +| `\uFEFFfilm:Q42\uFEFF` | `film:Q42` | edge whitespace, including U+FEFF, trims | +| `film:Q042` | ✗ reject | leading-zero QID | ## Notes and limitations +- **Ladder rungs.** A `wd` rung mints a typed value only when it declares an active binding slug; a rung without a slug mints bare values only, and a typed value reaching it is skipped ([§5](../spec/05-identity-ladders.md)). +- **Legacy bare values.** Existing bare `int:wd:Q…` identifiers remain valid and byte-identical, never P0-eligible. Bare and typed forms for the same entity are joined through [§8 equivalence](../spec/08-equivalence.md), without rewriting either identifier. + - **Why leading zeros reject.** Wikidata itself never issues zero-padded QIDs, but padded forms appear in the wild via spreadsheet formatting and hand transcription. Accepting them would create two byte-distinct IIDs for one item, which byte-exact comparison can never repair. Rejection forces the error to surface at canonicalization time. - **Properties are not entities.** `P31` (*instance of*) names a relationship vocabulary term, not a thing. If a term set is the entity being identified, the [`termset`](./termset.md) scheme exists for that purpose. - **Merges and redirects.** Wikidata merges duplicate items, leaving the losing QID as a redirect. Both QIDs remain resolvable and both canonicalize here; the equivalence layer ([§8](../spec/08-equivalence.md)) is the right place to record that they name the same entity. Canonicalization is offline and does not chase redirects. -- **Position in ladders.** As a CC0, T2, cross-referencing registry, `wd` frequently ranks high in identity ladders ([§5](../spec/05-identity-ladders.md)) — above `imdb` (T3) always, per [§3.3.1](../spec/03-identity-classes.md), and above other T2 registries when it cross-references them. Its polymorphism is the trade-off: it can anchor anything, but never at P0. +- **Position in ladders.** As a CC0, T2, cross-referencing registry, `wd` frequently ranks high in identity ladders ([§5](../spec/05-identity-ladders.md)) — above `imdb` (T3) always, per [§3.3.1](../spec/03-identity-classes.md), and above other T2 registries when it cross-references them. Bare `wd` can identify anything but never anchors at P0; active typed bindings can. diff --git a/packages/iid-spec/spec/07-representation-profiles.md b/packages/iid-spec/spec/07-representation-profiles.md index 1edeaa8..e9ec88c 100644 --- a/packages/iid-spec/spec/07-representation-profiles.md +++ b/packages/iid-spec/spec/07-representation-profiles.md @@ -33,7 +33,7 @@ Identity becomes a pure function of the identifier. Two parties who derive the s **Eligibility.** An IID MAY be minted as P0 only if **both** hold: 1. Its scheme is Class **A or B**. Class C never qualifies ([§6.6](./06-gen1.md)). -2. Its scheme is **unambiguously typed** ([§7.3](#73-scheme-typing)) — the identifier implies the entity's type on its own. +2. Its identifier is **unambiguously typed** ([§7.3](#73-scheme-typing)) — the identifier implies the entity's type on its own, and any EntitySchema binding it carries is active. ### P1 — identity context @@ -52,7 +52,7 @@ Atom data is an object containing the type, the identifier, the identity-recipe P1 is the **REQUIRED floor** for: - **Class C identifiers** — the recipe fields are the hash's preimage evidence. Without them the identifier cannot be verified or re-derived, and there is nothing to display. -- **Polymorphically typed schemes** — a bare `int:wd:Q42` does not say whether Q42 is a person, a book, or a concept. The `@type` must live in the payload. +- **Polymorphically typed identifiers** — a bare `int:wd:Q42` does not say whether Q42 is a person, a book, or a concept. The `@type` must live in the payload. ### P2 — enriched @@ -82,15 +82,18 @@ A scheme is **unambiguously typed** if knowing an identifier's scheme and value | Typing | Schemes | | :-- | :-- | | **Unambiguous** | `isbn`, `isrc`, `iswc`, `lei`, `gtin`, `eidr`, `mbid`, `olid`, `podcastguid`, `caip19`, `appid`, `purl`, `acct`, `rssitem`, `termset`, `gen1` | -| **Polymorphic** | `isni`, `orcid`, `doi`, `wd`, `imdb`, `tmdb`, `url`, `caip10`, `hash`, `geo` | +| **Polymorphic** | `isni`, `orcid`, `doi`, bare `wd`, `imdb`, `tmdb`, `url`, `caip10`, `hash`, `geo` | Some schemes are unambiguous because their domain is narrow: an ISBN is always a book edition, an ISRC always a sound recording. Others carry the type inside the value: `mbid` values begin with an entity-type segment (`artist:`, `recording:`), `olid` values end with `W`/`M`/`A`, and `gen1` values begin with the entity type's slug. +A **value-typed scheme** has polymorphic bare values but MAY carry the entity type inside a value. `wd` is value-typed: bare `Q…` is polymorphic, while `:Q…` with an active [EntitySchema binding](../schemes/wd.md) is unambiguous; a dormant binding is valid and unambiguous but not mintable. The conformance corpus marks value-typed schemes with `valueTyped` and MAY state per-value `typing` on validation vectors, defaulting to the scheme's typing when omitted. + The polymorphic cases are polymorphic for concrete reasons: | Scheme | Why polymorphic | | :-- | :-- | -| `wd` | Wikidata covers every kind of thing | +| Bare `wd` | A QID alone can name any kind of thing | +| Typed `wd` | `:Q…` carries the type, like the `mbid` type segment; active bindings are unambiguous and P0-eligible | | `isni` / `orcid` | Assigned to persons *and* to bands and organizations | | `doi` | Articles, datasets, and — via EIDR — films | | `imdb` / `tmdb` | Values span films, series, and people | @@ -119,7 +122,7 @@ Separating these matters. The anchor is the stable thing to point at — it neve ## 7.5 Requirements 1. An implementation MUST NOT mint P0 for a Class C identifier. -2. An implementation MUST NOT mint P0 for a polymorphically typed scheme. +2. An implementation MUST NOT mint P0 for a polymorphically typed identifier or a dormant EntitySchema binding. 3. An implementation MUST include the recipe fields when minting a Class C identifier. 4. Profiles are **additive**. Introducing or preferring a profile MUST NOT require re-minting anything that already exists. 5. An implementation consuming atoms MUST accept all three profiles for any entity type. @@ -129,7 +132,7 @@ Separating these matters. The anchor is the stable thing to point at — it neve Non-normative guidance: - If the identifier is anchor-eligible and the goal is a stable point for claims to attach to — **P0**. It is the cheapest atom possible and the only one with protocol-level dedupe. -- If the identifier is Class C, or the scheme is polymorphic — **P1**. This is a floor, not a preference; it is the minimum that is verifiable and renderable. +- If the identifier is Class C or polymorphically typed — **P1**. This is a floor, not a preference; it is the minimum that is verifiable and renderable. - If the atom is the primary record for an entity and descriptive data must be on-chain rather than in enrichment — **P2**. Prefer to justify this case rather than default to it. --- diff --git a/packages/iid-spec/spec/09-registry-governance.md b/packages/iid-spec/spec/09-registry-governance.md index 2012f88..795b40d 100644 --- a/packages/iid-spec/spec/09-registry-governance.md +++ b/packages/iid-spec/spec/09-registry-governance.md @@ -52,6 +52,17 @@ Old and new coexist in the registry. Identifiers under each remain valid and sta Known defects in the reference implementation are recorded in [`implementation-notes.md`](../implementation-notes.md) rather than silently repaired. +### 9.3.1 Additive value-grammar extensions + +An additive value-grammar extension MAY be ratified in place as a MINOR registry change only when: + +1. Every existing canonical value remains canonical and byte-identical. +2. No existing valid identifier changes validity, typing, or anchor eligibility. +3. The new forms were previously invalid. +4. Conformance vectors are added that prove conditions 1–3. + +The `wd` extension from bare `Q…` to optional `:Q…` is the first instance: existing bare identifiers remain polymorphic and ineligible for P0; newly valid typed forms follow the binding rules in [`schemes/wd.md`](../schemes/wd.md). This exception does not permit rewriting existing identifiers or changing their canonical values. + ## 9.4 Adding a scheme Adding a scheme is a specification change, reviewed and merged. diff --git a/packages/iid-spec/test/validate-fixtures.mjs b/packages/iid-spec/test/validate-fixtures.mjs index 0b1d0d2..9aab564 100644 --- a/packages/iid-spec/test/validate-fixtures.mjs +++ b/packages/iid-spec/test/validate-fixtures.mjs @@ -113,11 +113,26 @@ for (const vector of fixtures.gen1.vectors) { // --- validation vectors: anchor eligibility is consistent with class/typing --- for (const vector of fixtures.validation.vectors) { + const scheme = vector.iid.split(':')[1]; + if (vector.typing !== undefined) { + assert.equal( + schemes[scheme].valueTyped, + true, + `${vector.id}: per-value typing only for value-typed schemes` + ); + } + if (vector.typing !== undefined && vector.typing !== schemes[scheme].typing) { + assert.equal( + vector.iid.split(':').slice(2).join(':').includes(':'), + true, + `${vector.id}: a typed value carries its type inside the value` + ); + } if (vector.anchorEligible) { assert.ok(vector.valid, `${vector.id}: anchor-eligible implies valid`); - const scheme = vector.iid.split(':')[1]; assert.ok(schemes[scheme].class !== 'C', `${vector.id}: Class C never anchors`); - assert.equal(schemes[scheme].typing, 'unambiguous', `${vector.id}: polymorphic never anchors`); + const effectiveTyping = vector.typing ?? schemes[scheme].typing; + assert.equal(effectiveTyping, 'unambiguous', `${vector.id}: polymorphic never anchors`); } } diff --git a/packages/iid/README.md b/packages/iid/README.md index eef1cb4..b652f77 100644 --- a/packages/iid/README.md +++ b/packages/iid/README.md @@ -43,11 +43,23 @@ inspectIntuitionId('int:wd:Q42') // { valid: true, class: 'A', typing: 'polymorphic', // anchorEligible: false, anchorIneligibilityReason: 'polymorphic-scheme' } +inspectIntuitionId('int:wd:film:Q188035') +// { valid: true, class: 'A', typing: 'unambiguous', +// anchorEligible: true, wdSlug: 'film', … } + inspectIntuitionId('int:src:anything') // { valid: false, reason: 'unknown-scheme', … } — the registry is closed ``` -`isAnchorEligible` answers the P0 question (spec §7.2): valid + Class A/B + unambiguously typed scheme. Class C (`gen1`) and polymorphic schemes floor at P1. +`isAnchorEligible` answers the P0 question (spec §7.2): valid + Class A/B + unambiguously typed identifier with an active binding when present. Class C (`gen1`) and polymorphic values, including bare `wd`, floor at P1. + +## Typed Wikidata identifiers + +`wd` is value-typed: `int:wd:film:Q188035`, `int:wd:television-series:Q137400033`, and `int:wd:human:Q42` carry active EntitySchema bindings and are P0-eligible. Bare `int:wd:Q42` remains valid, polymorphic, and ineligible for P0. Dormant typed bindings such as `written-work` remain valid for parsing and validation but do not mint. + +Declare `wdSlug: 'film'` on a `wd` scheme rung with a `field` or `same-as` source to mint typed values. Bare QIDs are prefixed; a conflicting typed slug or an overlong IID skips the rung. `same-as` selects the lexicographically smallest canonical value before checking the slug. A rung without `wdSlug` retains legacy bare minting during the transition. + +`WD_ENTITYSCHEMA_BINDINGS` exposes the 11 pinned rows; `WD_ENTITYSCHEMA_SLUGS` is the frozen grammar list. `isWdEntitySchemaSlug` checks membership, and `isActiveWdEntitySchemaSlug` checks minting eligibility. The corresponding public types are `WdEntitySchemaBinding` and `WdEntitySchemaSlug`. `SCHEME_TYPING.wd` stays `polymorphic`; use `inspectIntuitionId` for per-value typing and the `dormant-wd-binding` ineligibility reason. ## Canonicalize @@ -108,7 +120,7 @@ Plus the derivation utilities the spec's schemes need: `norm1` (NORM-1), `keccak ## Conformance -The versioned conformance corpus ships in [`@0xintuition/iid-spec`](https://www.npmjs.com/package/@0xintuition/iid-spec) (`conformance/*.json`) and runs in this package's test suite. Canonicalization rules are frozen (spec §9.3): a rule change ships as a new scheme, never an in-place edit. +The versioned conformance corpus ships in [`@0xintuition/iid-spec`](https://www.npmjs.com/package/@0xintuition/iid-spec) (`conformance/*.json`) and runs in this package's test suite. Canonicalization rules are frozen (spec §9.3): changes to existing canonical values require a new scheme; additive value-grammar extensions follow §9.3.1. ## License diff --git a/packages/iid/src/__tests__/derive.test.ts b/packages/iid/src/__tests__/derive.test.ts index ef24a43..39855bb 100644 --- a/packages/iid/src/__tests__/derive.test.ts +++ b/packages/iid/src/__tests__/derive.test.ts @@ -3,7 +3,7 @@ import { describe, expect, it } from 'vitest'; import { deriveIntuitionId } from '../derive.js'; import { keccak16 } from '../hash.js'; import { norm1 } from '../norm.js'; -import { isAnchorEligible, validateIntuitionId } from '../parse.js'; +import { isAnchorEligible, parseIntuitionId, validateIntuitionId } from '../parse.js'; import type { IdentityLadder } from '../types.js'; import { derivePodcastGuid } from '../uuid5.js'; @@ -16,7 +16,163 @@ const bookLadder: IdentityLadder = { ], }; +const typedWdLadder: IdentityLadder = { + slug: 'movie', + identifies: 'a film', + rungs: [ + { kind: 'scheme', scheme: 'wd', wdSlug: 'film', source: { kind: 'field', key: 'wikidataId' } }, + ], +}; + +const typedWdArrayLadder: IdentityLadder = { + slug: 'movie', + identifies: 'a film', + rungs: [ + { + kind: 'scheme', + scheme: 'wd', + wdSlug: 'film', + source: { kind: 'same-as' }, + }, + ], +}; + +const bareWdLadder: IdentityLadder = { + slug: 'movie', + identifies: 'a film', + rungs: [{ kind: 'scheme', scheme: 'wd', source: { kind: 'field', key: 'wikidataId' } }], +}; + +const bareWdSameAsLadder: IdentityLadder = { + slug: 'movie', + identifies: 'a film', + rungs: [{ kind: 'scheme', scheme: 'wd', source: { kind: 'same-as' } }], +}; + +const dormantWdLadder: IdentityLadder = { + slug: 'book', + identifies: 'a written work', + rungs: [ + { + kind: 'scheme', + scheme: 'wd', + wdSlug: 'written-work', + source: { kind: 'field', key: 'wikidataId' }, + }, + ], +}; + describe('deriveIntuitionId (declarative engine)', () => { + it('mints a typed wd value when a synthetic ladder supplies a slug', () => { + expect(deriveIntuitionId(typedWdLadder, { wikidataId: 'q188035' })).toEqual({ + iid: 'int:wd:film:Q188035', + scheme: 'wd', + class: 'A', + }); + }); + + it('canonicalizes a typed wd raw value before prepending its slug', () => { + for (const wikidataId of ['https://www.wikidata.org/wiki/Q188035', ' \tq188035 \n']) { + expect(deriveIntuitionId(typedWdLadder, { wikidataId }), wikidataId).toEqual({ + iid: 'int:wd:film:Q188035', + scheme: 'wd', + class: 'A', + }); + } + }); + + it('prefixes a bare-canonicalizable wd value returned by same-as', () => { + expect( + deriveIntuitionId(typedWdArrayLadder, { + sameAs: ['https://www.wikidata.org/wiki/Q188035'], + }) + ).toEqual({ iid: 'int:wd:film:Q188035', scheme: 'wd', class: 'A' }); + }); + + it('keeps an already-typed same-slug wd value idempotent through same-as', () => { + expect(deriveIntuitionId(typedWdArrayLadder, { sameAs: ['film:Q188035'] })).toEqual({ + iid: 'int:wd:film:Q188035', + scheme: 'wd', + class: 'A', + }); + }); + + it('rejects an already-typed different-slug wd value returned by same-as', () => { + expect(deriveIntuitionId(typedWdArrayLadder, { sameAs: ['human:Q188035'] })).toBeUndefined(); + }); + + it('refuses a typed wd IID that exceeds the parser value limit', () => { + const overlongQid = `Q${'1'.repeat(215)}`; + expect(deriveIntuitionId(typedWdLadder, { wikidataId: overlongQid })).toBeUndefined(); + }); + + it('never derives dormant-slug wd values', () => { + expect(deriveIntuitionId(dormantWdLadder, { wikidataId: 'Q47461344' })).toBeUndefined(); + }); + + it('legacy: a slugless wd rung still mints bare (D-P16-1, flips with the classifications lane)', () => { + expect(deriveIntuitionId(bareWdLadder, { wikidataId: 'Q188035' })).toEqual({ + iid: 'int:wd:Q188035', + scheme: 'wd', + class: 'A', + }); + }); + + it('legacy: BOM- and NBSP-wrapped bare QIDs keep minting through slugless rungs (D-P16-4)', () => { + for (const wikidataId of ['\uFEFFQ188035\uFEFF', '\u00A0Q188035\u00A0']) { + expect(deriveIntuitionId(bareWdLadder, { wikidataId })?.iid, JSON.stringify(wikidataId)).toBe( + 'int:wd:Q188035' + ); + } + expect(deriveIntuitionId(bareWdSameAsLadder, { sameAs: ['\uFEFFQ42\uFEFF', 'Q9'] })?.iid).toBe( + 'int:wd:Q42' + ); + }); + + it('legacy: a slugless wd rung mints bare values only; typed values need a declared active slug (D-P16-5)', () => { + for (const wikidataId of ['written-work:Q47461344', 'human:Q42', 'film:Q188035']) { + expect(deriveIntuitionId(bareWdLadder, { wikidataId }), wikidataId).toBeUndefined(); + } + expect( + deriveIntuitionId(bareWdSameAsLadder, { sameAs: ['written-work:Q47461344'] }) + ).toBeUndefined(); + expect(deriveIntuitionId(bareWdSameAsLadder, { sameAs: ['human:Q42'] })).toBeUndefined(); + expect(deriveIntuitionId(bareWdSameAsLadder, { sameAs: ['human:Q42', 'Q7'] })?.iid).toBe( + 'int:wd:Q7' + ); + }); + + it('selects the smallest canonical same-as value before matching the rung slug', () => { + for (const sameAs of [ + ['film:Q1', 'https://www.wikidata.org/wiki/Q42', 'Q42', 42, 'bogus:Q1'], + ['bogus:Q1', 42, 'Q42', 'https://www.wikidata.org/wiki/Q42', 'film:Q1'], + ]) { + expect(deriveIntuitionId(typedWdArrayLadder, { sameAs })?.iid).toBe('int:wd:film:Q42'); + } + for (const sameAs of [ + ['human:Q1', 'album:Q1'], + ['album:Q1', 'human:Q1'], + ]) { + expect(deriveIntuitionId(typedWdArrayLadder, { sameAs })).toBeUndefined(); + } + }); + + it('accepts the 220-character typed value boundary and falls through rejected typed rungs', () => { + const boundaryQid = `Q${'1'.repeat(214)}`; + expect(deriveIntuitionId(typedWdLadder, { wikidataId: boundaryQid })?.iid).toBe( + `int:wd:film:${boundaryQid}` + ); + for (const wikidataId of ['human:Q42', `Q${'1'.repeat(215)}`, 'bogus:Q1']) { + const ladder: IdentityLadder = { + ...typedWdLadder, + rungs: [...typedWdLadder.rungs, ...bookLadder.rungs], + }; + expect(deriveIntuitionId(ladder, { wikidataId, isbn: '9780684832722' })?.iid).toBe( + 'int:isbn:9780684832722' + ); + } + }); + it('uses the highest rung with available data', () => { const derived = deriveIntuitionId(bookLadder, { name: 'The Sovereign Individual', @@ -341,3 +497,49 @@ describe('deriveIntuitionId (declarative engine)', () => { expect(gen1 && isAnchorEligible(gen1.iid)).toBe(false); // Class C floors at P1 }); }); + +describe('typed wd eligibility', () => { + it('allows active typed wd while refusing legacy bare and dormant typed values', () => { + expect(isAnchorEligible('int:wd:film:Q188035')).toBe(true); + expect(isAnchorEligible('int:wd:television-series:Q137400033')).toBe(true); + expect(isAnchorEligible('int:wd:human:Q42')).toBe(true); + expect(isAnchorEligible('int:wd:Q165219')).toBe(false); + expect(isAnchorEligible('int:wd:written-work:Q47461344')).toBe(false); + expect(isAnchorEligible('int:wd:television-series-season:Q3464665')).toBe(false); + }); +}); + +describe('parse and validate', () => { + it('splits on the first two colons only', () => { + expect( + parseIntuitionId('int:caip10:eip155:1:0xd8da6bf26964af9d7eed9e03e53415d37aa96045') + ).toEqual({ + scheme: 'caip10', + value: 'eip155:1:0xd8da6bf26964af9d7eed9e03e53415d37aa96045', + }); + }); + + it('rejects unknown schemes and malformed strings', () => { + expect(parseIntuitionId('int:bogus:123')).toBeUndefined(); + expect(parseIntuitionId('isbn:9780684832722')).toBeUndefined(); + }); + + it('validates canonical form byte-exactly', () => { + expect(validateIntuitionId('int:isbn:9780684832722')).toBe(true); + expect(validateIntuitionId('int:isbn:978-0-684-83272-2')).toBe(false); + expect(validateIntuitionId('int:wd:film:Q42')).toBe(true); + expect(validateIntuitionId('int:wd:film:q42')).toBe(false); + expect(validateIntuitionId('int:wd:bogus:Q1')).toBe(false); + expect(validateIntuitionId('int:wd:q42')).toBe(false); + expect(validateIntuitionId('int:wd:Q42')).toBe(true); + expect(validateIntuitionId('int:gen1:book:r3:59a02a73cbe0d2a4223719fdc4d006ab')).toBe(true); + }); + + it('parses typed and legacy wd values without changing the pinned bare form', () => { + expect(parseIntuitionId('int:wd:film:Q188035')).toEqual({ + scheme: 'wd', + value: 'film:Q188035', + }); + expect(parseIntuitionId('int:wd:Q42')).toEqual({ scheme: 'wd', value: 'Q42' }); + }); +}); diff --git a/packages/iid/src/__tests__/inspect.test.ts b/packages/iid/src/__tests__/inspect.test.ts index c8c14f7..68cc5be 100644 --- a/packages/iid/src/__tests__/inspect.test.ts +++ b/packages/iid/src/__tests__/inspect.test.ts @@ -66,3 +66,66 @@ describe('inspectIntuitionId', () => { }); }); }); + +describe('typed Wikidata inspection', () => { + it.each([ + ['film', 'Q188035'], + ['television-series', 'Q137400033'], + ['human', 'Q42'], + ])('reports an active %s binding as unambiguous and eligible', (wdSlug, qid) => { + const value = `${wdSlug}:${qid}`; + expect(inspectIntuitionId(`int:wd:${value}`)).toEqual({ + valid: true, + iid: `int:wd:${value}`, + scheme: 'wd', + value, + class: 'A', + typing: 'unambiguous', + anchorEligible: true, + wdSlug, + }); + }); + + it('reports a dormant binding as valid, typed and ineligible', () => { + expect(inspectIntuitionId('int:wd:written-work:Q47461344')).toEqual({ + valid: true, + iid: 'int:wd:written-work:Q47461344', + scheme: 'wd', + value: 'written-work:Q47461344', + class: 'A', + typing: 'unambiguous', + anchorEligible: false, + anchorIneligibilityReason: 'dormant-wd-binding', + wdSlug: 'written-work', + }); + }); + + it('preserves bare Wikidata inspection byte-for-byte', () => { + expect(inspectIntuitionId('int:wd:Q42')).toEqual({ + valid: true, + iid: 'int:wd:Q42', + scheme: 'wd', + value: 'Q42', + class: 'A', + typing: 'polymorphic', + anchorEligible: false, + anchorIneligibilityReason: 'polymorphic-scheme', + }); + }); + + it('repairs noncanonical typed values and rejects unknown slugs without a repair', () => { + expect(inspectIntuitionId('int:wd:film:q42')).toEqual({ + valid: false, + reason: 'noncanonical', + scheme: 'wd', + value: 'film:q42', + canonical: 'int:wd:film:Q42', + }); + expect(inspectIntuitionId('int:wd:bogus:Q1')).toEqual({ + valid: false, + reason: 'noncanonical', + scheme: 'wd', + value: 'bogus:Q1', + }); + }); +}); diff --git a/packages/iid/src/__tests__/schemes.test.ts b/packages/iid/src/__tests__/schemes.test.ts index 9292551..89768c8 100644 --- a/packages/iid/src/__tests__/schemes.test.ts +++ b/packages/iid/src/__tests__/schemes.test.ts @@ -1,8 +1,17 @@ import { describe, expect, it } from 'vitest'; - +import type { WdEntitySchemaBinding, WdEntitySchemaSlug } from '../index.js'; +import * as publicIid from '../index.js'; +import { validateIntuitionId } from '../parse.js'; import { SCHEMES } from '../schemes.js'; import { derivePodcastGuid } from '../uuid5.js'; +const { + WD_ENTITYSCHEMA_BINDINGS, + WD_ENTITYSCHEMA_SLUGS, + isWdEntitySchemaSlug, + isActiveWdEntitySchemaSlug, +} = publicIid; + describe('isbn', () => { it('converts valid ISBN-10 to ISBN-13', () => { expect(SCHEMES.isbn.canonicalize('0-684-83272-0')).toBe('9780684832722'); @@ -59,6 +68,115 @@ describe('doi / wd / mbid / imdb / tmdb', () => { expect(SCHEMES.wd.canonicalize('q42')).toBe('Q42'); }); + it('keeps legacy Wikidata QIDs and entity URLs canonical', () => { + expect(SCHEMES.wd.canonicalize('https://www.wikidata.org/wiki/Q42')).toBe('Q42'); + expect(SCHEMES.wd.canonicalize('q42')).toBe('Q42'); + }); + + it('pins the ratified EntitySchema binding data', () => { + expect( + WD_ENTITYSCHEMA_BINDINGS.map( + ({ slug, entitySchemaId, anchorQids, classification, status, precedence, schemaRevId }) => [ + slug, + entitySchemaId, + anchorQids, + classification, + status, + precedence, + schemaRevId, + ] + ) + ).toEqual([ + ['film', 'E11424', ['Q11424'], 'Movie', 'active', 0, 2403158147], + ['television-series', 'E17', ['Q5398426'], 'TVSeries', 'active', 1, 2525771900], + ['television-series-season', 'E18', ['Q3464665'], 'TVSeason', 'dormant', 2, 2525772131], + ['television-series-episode', 'E19', ['Q21191270'], 'TVEpisode', 'dormant', 3, 2279362550], + ['written-work', 'E35', ['Q47461344'], 'Book', 'dormant', 4, 2525775487], + ['human', 'E10', ['Q5'], 'Person', 'active', 5, 2499173058], + ['podcast', 'E418', ['Q24634210'], 'PodcastSeries', 'dormant', 6, 2052853948], + ['podcast-episode', 'E420', ['Q61855877'], 'PodcastEpisode', 'dormant', 7, 2212603393], + ['video-game', 'E272', ['Q7889'], 'VideoGame', 'dormant', 8, 2363080401], + ['album', 'E248', ['Q482994'], 'MusicAlbum', 'dormant', 9, 2226920960], + ['organization', 'E98', ['Q43229'], 'Organization', 'dormant', 10, 2392901167], + ]); + }); + + it('canonicalizes every active and dormant EntitySchema slug', () => { + expect(WD_ENTITYSCHEMA_BINDINGS.map(({ precedence }) => precedence)).toEqual( + WD_ENTITYSCHEMA_BINDINGS.map((_, index) => index) + ); + expect([...WD_ENTITYSCHEMA_SLUGS]).toEqual(WD_ENTITYSCHEMA_BINDINGS.map(({ slug }) => slug)); + + for (const { slug } of WD_ENTITYSCHEMA_BINDINGS) { + const canonical = `${slug}:Q42`; + expect(SCHEMES.wd.canonicalize(`${slug}:q42`), slug).toBe(canonical); + expect(SCHEMES.wd.canonicalize(canonical), slug).toBe(canonical); + } + }); + + it('distinguishes the active minting set from dormant parse-only slugs', () => { + for (const { slug, status } of WD_ENTITYSCHEMA_BINDINGS) { + expect(isActiveWdEntitySchemaSlug(slug), slug).toBe(status === 'active'); + } + }); + + it('rejects unknown Wikidata slugs and never infers one from URLs', () => { + expect(SCHEMES.wd.canonicalize('bogus:Q1')).toBeUndefined(); + expect(SCHEMES.wd.canonicalize('https://www.wikidata.org/wiki/Q188035')).toBe('Q188035'); + }); + + it('rejects non-ASCII Wikidata slugs before case folding', () => { + expect(SCHEMES.wd.canonicalize('written-wor\u212a:Q42')).toBeUndefined(); + expect(SCHEMES.wd.canonicalize('WrItTeN-WoR\u212a:q42')).toBeUndefined(); + }); + + it('keeps the ratified trim for every wd form: BOM and NBSP edges trim like whitespace (D-P16-4)', () => { + expect(SCHEMES.wd.canonicalize(' \tfilm:q42\r\n')).toBe('film:Q42'); + expect(SCHEMES.wd.canonicalize('\uFEFFfilm:Q42\uFEFF')).toBe('film:Q42'); + expect(SCHEMES.wd.canonicalize('\uFEFFQ42\uFEFF')).toBe('Q42'); + expect(SCHEMES.wd.canonicalize('\u00A0Q42\u00A0')).toBe('Q42'); + }); + + it('exports the typed-wd bindings, slug guards and public types', () => { + const bindings: readonly WdEntitySchemaBinding[] = publicIid.WD_ENTITYSCHEMA_BINDINGS; + const slugs: readonly WdEntitySchemaSlug[] = publicIid.WD_ENTITYSCHEMA_SLUGS; + expect(bindings).toHaveLength(11); + expect(slugs).toEqual(bindings.map(({ slug }) => slug)); + expect(isWdEntitySchemaSlug('film')).toBe(true); + expect(isWdEntitySchemaSlug('written-work')).toBe(true); + expect(isWdEntitySchemaSlug('bogus')).toBe(false); + expect(isActiveWdEntitySchemaSlug('film')).toBe(true); + expect(isActiveWdEntitySchemaSlug('written-work')).toBe(false); + }); + + it('does not expose a runtime-mutable Wikidata slug allowlist', () => { + expect(Object.isFrozen(WD_ENTITYSCHEMA_SLUGS)).toBe(true); + expect(() => { + (WD_ENTITYSCHEMA_SLUGS as unknown as { add: (slug: string) => void }).add('bogus'); + }).toThrow(TypeError); + expect(validateIntuitionId('int:wd:bogus:Q42')).toBe(false); + }); + + it('deep-freezes the binding table, its rows and their anchor QIDs', () => { + expect(Object.isFrozen(WD_ENTITYSCHEMA_BINDINGS)).toBe(true); + for (const row of WD_ENTITYSCHEMA_BINDINGS) { + expect(Object.isFrozen(row), row.slug).toBe(true); + expect(Object.isFrozen(row.anchorQids), row.slug).toBe(true); + } + const film = WD_ENTITYSCHEMA_BINDINGS[0] as unknown as { + classification: string; + anchorQids: { push: (qid: string) => void }; + }; + expect(() => { + film.classification = 'Person'; + }).toThrow(TypeError); + expect(() => { + film.anchorQids.push('Q5'); + }).toThrow(TypeError); + expect(WD_ENTITYSCHEMA_BINDINGS[0].classification).toBe('Movie'); + expect(WD_ENTITYSCHEMA_BINDINGS[0].anchorQids).toEqual(['Q11424']); + }); + it('requires the MBID entity-type segment', () => { expect( SCHEMES.mbid.canonicalize( diff --git a/packages/iid/src/derive.ts b/packages/iid/src/derive.ts index 2cbdda7..678fa9c 100644 --- a/packages/iid/src/derive.ts +++ b/packages/iid/src/derive.ts @@ -2,7 +2,7 @@ import { buildGen1Iid } from './gen1.js'; import { geohashEncode } from './geohash.js'; import { keccak16 } from './hash.js'; import { norm1 } from './norm.js'; -import { formatIntuitionId } from './parse.js'; +import { formatIntuitionId, validateIntuitionId } from './parse.js'; import { SCHEMES } from './schemes.js'; import type { DerivedIid, @@ -13,6 +13,7 @@ import type { IidValueMap, } from './types.js'; import { derivePodcastGuid } from './uuid5.js'; +import { isActiveWdEntitySchemaSlug } from './wd-entityschema-bindings.js'; /** * The pure declarative derivation engine (spec §5.2). @@ -26,6 +27,15 @@ import { derivePodcastGuid } from './uuid5.js'; * contract: ladders are data, and two engines interpreting the same ladder * against the same field map MUST return the same identifier. * + * | `wd` rung | Behavior | + * | --------- | -------- | + * | Active `wdSlug` | Prefix bare QIDs; retain matching typed values; skip conflicting slugs or invalid IIDs (including values over 220 characters). | + * | Dormant `wdSlug` | Skip the rung. | + * | No `wdSlug` | Legacy bare minting only (D-P16-1, D-P16-5): a typed value reaching a slugless rung is skipped; typed minting requires a declared active binding. | + * + * `same-as` canonicalizes the whole set and selects its lexicographically + * smallest value before checking the rung's slug, including typed values. + * * Returns `undefined` when no rung fires — the atom mints without an IID. */ export function deriveIntuitionId( @@ -34,6 +44,11 @@ export function deriveIntuitionId( ): DerivedIid | undefined { for (const rung of ladder.rungs) { if (rung.kind === 'scheme') { + const wdSlug = rung.scheme === 'wd' ? rung.wdSlug : undefined; + if (wdSlug !== undefined && !isActiveWdEntitySchemaSlug(wdSlug)) { + continue; + } + const raw = resolveValueSource(rung.source, rung.scheme, values); if (raw === undefined) { @@ -41,14 +56,38 @@ export function deriveIntuitionId( } const definition = SCHEMES[rung.scheme]; - const canonical = definition.canonicalize(raw); + let canonical = definition.canonicalize(raw); + + if (canonical === undefined) { + continue; + } + + if (rung.scheme === 'wd' && wdSlug === undefined && canonical.includes(':')) { + // D-P16-5: a slugless rung keeps the base accepted-value domain (bare QIDs only). + continue; + } + + if (wdSlug !== undefined) { + if (canonical.includes(':')) { + if (canonical.split(':', 1)[0] !== wdSlug) { + continue; + } + } else { + canonical = definition.canonicalize(`${wdSlug}:${canonical}`); + } + } if (canonical === undefined) { continue; } + const iid = formatIntuitionId(rung.scheme, canonical); + if (wdSlug !== undefined && !validateIntuitionId(iid)) { + continue; + } + return { - iid: formatIntuitionId(rung.scheme, canonical), + iid, scheme: rung.scheme, class: definition.class, }; diff --git a/packages/iid/src/index.ts b/packages/iid/src/index.ts index 8712f4c..df43fa9 100644 --- a/packages/iid/src/index.ts +++ b/packages/iid/src/index.ts @@ -34,3 +34,10 @@ export type { } from './types.js'; export { SCHEME_NAMES } from './types.js'; export { derivePodcastGuid, PODCAST_GUID_NAMESPACE, uuidv5 } from './uuid5.js'; +export type { WdEntitySchemaBinding, WdEntitySchemaSlug } from './wd-entityschema-bindings.js'; +export { + isActiveWdEntitySchemaSlug, + isWdEntitySchemaSlug, + WD_ENTITYSCHEMA_BINDINGS, + WD_ENTITYSCHEMA_SLUGS, +} from './wd-entityschema-bindings.js'; diff --git a/packages/iid/src/parse.ts b/packages/iid/src/parse.ts index be21b6f..1af87a8 100644 --- a/packages/iid/src/parse.ts +++ b/packages/iid/src/parse.ts @@ -1,5 +1,6 @@ import { getScheme, SCHEME_TYPING } from './schemes.js'; import type { IidInspection, IntuitionId, ParsedIid, SchemeName } from './types.js'; +import { isActiveWdEntitySchemaSlug, type WdEntitySchemaSlug } from './wd-entityschema-bindings.js'; /** * The full grammar (spec §2.2): `int:` namespace, 1-32 char lowercase @@ -61,8 +62,9 @@ export function isIntuitionId(input: string): input is IntuitionId { /** * May this IID mint as a P0 anchor (atom data = the bare IID string)? * Requires: valid + canonical, Class A or B (Class C recipe fields are - * preimage evidence and must travel in a P1 payload), and a scheme whose - * IIDs imply their classification (spec §7.2). + * preimage evidence and must travel in a P1 payload), and an IID that + * implies its classification (spec §7.2 / §7.3). Typed `wd` values qualify + * only with an active EntitySchema binding; bare `wd` remains polymorphic. */ export function isAnchorEligible(input: string): boolean { const inspection = inspectIntuitionId(input); @@ -71,7 +73,9 @@ export function isAnchorEligible(input: string): boolean { /** * Typed inspection: one call that distinguishes every failure mode a - * caller can act on (spec §2.5 validity, §7.2 anchor eligibility). + * caller can act on (spec §2.5 validity, §7.2 anchor eligibility, §7.3 typing). + * Typed `wd` values report their binding slug and per-value typing; + * dormant bindings are valid but cannot mint a P0 anchor. * * - `malformed` — not `int::` per the grammar * - `unknown-scheme` — the registry is closed; unknown schemes are invalid @@ -108,6 +112,23 @@ export function inspectIntuitionId(input: string): IidInspection { }; } + if (scheme.scheme === 'wd' && value.includes(':')) { + // Canonicalization already proved membership in the closed binding set. + const wdSlug = value.slice(0, value.indexOf(':')) as WdEntitySchemaSlug; + const anchorEligible = isActiveWdEntitySchemaSlug(wdSlug); + return { + valid: true, + iid: formatIntuitionId(scheme.scheme, value), + scheme: scheme.scheme, + value, + class: scheme.class, + typing: 'unambiguous', + wdSlug, + anchorEligible, + ...(!anchorEligible ? { anchorIneligibilityReason: 'dormant-wd-binding' as const } : {}), + }; + } + const typing = SCHEME_TYPING[scheme.scheme]; const anchorIneligibilityReason = scheme.class === 'C' diff --git a/packages/iid/src/schemes.ts b/packages/iid/src/schemes.ts index 277b58e..f51f968 100644 --- a/packages/iid/src/schemes.ts +++ b/packages/iid/src/schemes.ts @@ -7,6 +7,9 @@ */ import { norm1 } from './norm.js'; import type { IdentityClass, SchemeDefinition, SchemeName } from './types.js'; +import { isWdEntitySchemaSlug } from './wd-entityschema-bindings.js'; + +const ASCII_MAX_CODE_UNIT = 0x7f; const UUID_PATTERN = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/; @@ -32,6 +35,10 @@ function stripSeparators(raw: string): string { return raw.replace(/[-\s.]/g, ''); } +function isAscii(value: string): boolean { + return Array.from(value).every((character) => character.charCodeAt(0) <= ASCII_MAX_CODE_UNIT); +} + // --- check digits --- function isbn10CheckDigit(core: string): string { @@ -186,12 +193,30 @@ function canonicalizeEidr(raw: string): string | undefined { } function canonicalizeWikidata(raw: string): string | undefined { - const clean = raw - .trim() - .replace(/^https?:\/\/(www\.)?wikidata\.org\/(wiki|entity)\//i, '') - .toUpperCase(); + const trimmed = raw.trim(); + const fromUrl = trimmed.replace(/^https?:\/\/(www\.)?wikidata\.org\/(wiki|entity)\//i, ''); + + if (fromUrl !== trimmed) { + const qid = fromUrl.toUpperCase(); + return /^Q[1-9]\d*$/.test(qid) ? qid : undefined; + } + + const typed = trimmed.match(/^([^:]+):(Q[1-9]\d*)$/i); + + if (typed) { + const rawSlug = typed[1] ?? ''; + + if (!isAscii(rawSlug)) { + return undefined; + } + + const slug = rawSlug.toLowerCase(); + const qid = (typed[2] ?? '').toUpperCase(); + return isWdEntitySchemaSlug(slug) ? `${slug}:${qid}` : undefined; + } - return /^Q[1-9]\d*$/.test(clean) ? clean : undefined; + const qid = trimmed.toUpperCase(); + return /^Q[1-9]\d*$/.test(qid) ? qid : undefined; } function canonicalizeMbid(raw: string): string | undefined { @@ -485,10 +510,10 @@ export const SCHEMES: Readonly> = { /** * Scheme typing (spec §7.3): a P0 anchor (atom data = the bare IID) is only legal - * when the scheme implies the entity's classification. `mbid` and `gen1` - * carry their type inside the value; polymorphic schemes (`wd` covers - * everything, `caip10` is account-or-contract, ...) floor at P1 where - * `@type` lives in the payload. + * when the identifier implies the entity's classification. `mbid`, `olid` + * and `gen1` carry their type inside the value. `wd` stays polymorphic here + * for bare values; inspectIntuitionId decides typed-wd eligibility per value. + * Polymorphic values floor at P1 where `@type` lives in the payload. */ export const SCHEME_TYPING: Readonly> = { isbn: 'unambiguous', diff --git a/packages/iid/src/types.ts b/packages/iid/src/types.ts index 1e16f6c..1f3d1a7 100644 --- a/packages/iid/src/types.ts +++ b/packages/iid/src/types.ts @@ -13,6 +13,8 @@ * JSON, and be audited without running code. */ +import type { WdEntitySchemaSlug } from './wd-entityschema-bindings.js'; + /** Every IID starts with the `int:` namespace followed by a registered scheme. */ export type IntuitionId = `int:${string}:${string}`; @@ -128,6 +130,11 @@ export type IdentityRung = readonly kind: 'scheme'; readonly scheme: ExternalSchemeName; readonly source: IdentityValueSource; + /** + * EntitySchema binding slug for a typed `wd` rung (spec `schemes/wd.md`). + * Only active bindings mint; see D-P16-1 for slugless rungs. + */ + readonly wdSlug?: WdEntitySchemaSlug; readonly note?: string; } | { @@ -175,7 +182,7 @@ export interface ParsedIid { export type IidInvalidReason = 'malformed' | 'unknown-scheme' | 'noncanonical'; /** Why a valid IID may still not mint as a bare P0 anchor (spec §7.2). */ -export type AnchorIneligibilityReason = 'class-c' | 'polymorphic-scheme'; +export type AnchorIneligibilityReason = 'class-c' | 'polymorphic-scheme' | 'dormant-wd-binding'; export type IidInspection = | { @@ -185,6 +192,7 @@ export type IidInspection = readonly value: string; readonly class: IdentityClass; readonly typing: SchemeTyping; + readonly wdSlug?: WdEntitySchemaSlug; readonly anchorEligible: boolean; readonly anchorIneligibilityReason?: AnchorIneligibilityReason; } diff --git a/packages/iid/src/wd-entityschema-bindings.ts b/packages/iid/src/wd-entityschema-bindings.ts new file mode 100644 index 0000000..577364f --- /dev/null +++ b/packages/iid/src/wd-entityschema-bindings.ts @@ -0,0 +1,172 @@ +/** + * Wikidata EntitySchema bindings, in deterministic precedence order. + * Normative specification: `@0xintuition/iid-spec` schemes/wd.md and §7.3. + */ +export interface WdEntitySchemaBinding { + readonly slug: string; + readonly entitySchemaId: `E${number}`; + readonly anchorQids: readonly `Q${number}`[]; + readonly classification: + | 'Movie' + | 'TVSeries' + | 'TVSeason' + | 'TVEpisode' + | 'Book' + | 'Person' + | 'PodcastSeries' + | 'PodcastEpisode' + | 'VideoGame' + | 'MusicAlbum' + | 'Organization'; + readonly status: 'active' | 'dormant'; + readonly precedence: number; + /** Pinned EntitySchema revision (harvest 2026-08-20, schemas.jsonl.gz sha256 8ed2b586…). */ + readonly schemaRevId: number; +} + +const WD_ENTITYSCHEMA_BINDING_ROWS = [ + { + slug: 'film', + entitySchemaId: 'E11424', + anchorQids: ['Q11424'], + classification: 'Movie', + status: 'active', + precedence: 0, + schemaRevId: 2403158147, + }, + { + slug: 'television-series', + entitySchemaId: 'E17', + anchorQids: ['Q5398426'], + classification: 'TVSeries', + status: 'active', + precedence: 1, + schemaRevId: 2525771900, + }, + { + slug: 'television-series-season', + entitySchemaId: 'E18', + anchorQids: ['Q3464665'], + classification: 'TVSeason', + status: 'dormant', + precedence: 2, + schemaRevId: 2525772131, + }, + { + slug: 'television-series-episode', + entitySchemaId: 'E19', + anchorQids: ['Q21191270'], + classification: 'TVEpisode', + status: 'dormant', + precedence: 3, + schemaRevId: 2279362550, + }, + { + slug: 'written-work', + entitySchemaId: 'E35', + anchorQids: ['Q47461344'], + classification: 'Book', + status: 'dormant', + precedence: 4, + schemaRevId: 2525775487, + }, + { + slug: 'human', + entitySchemaId: 'E10', + anchorQids: ['Q5'], + classification: 'Person', + status: 'active', + precedence: 5, + schemaRevId: 2499173058, + }, + { + slug: 'podcast', + entitySchemaId: 'E418', + anchorQids: ['Q24634210'], + classification: 'PodcastSeries', + status: 'dormant', + precedence: 6, + schemaRevId: 2052853948, + }, + { + slug: 'podcast-episode', + entitySchemaId: 'E420', + anchorQids: ['Q61855877'], + classification: 'PodcastEpisode', + status: 'dormant', + precedence: 7, + schemaRevId: 2212603393, + }, + { + slug: 'video-game', + entitySchemaId: 'E272', + anchorQids: ['Q7889'], + classification: 'VideoGame', + status: 'dormant', + precedence: 8, + schemaRevId: 2363080401, + }, + { + slug: 'album', + entitySchemaId: 'E248', + anchorQids: ['Q482994'], + classification: 'MusicAlbum', + status: 'dormant', + precedence: 9, + schemaRevId: 2226920960, + }, + { + slug: 'organization', + entitySchemaId: 'E98', + anchorQids: ['Q43229'], + classification: 'Organization', + status: 'dormant', + precedence: 10, + schemaRevId: 2392901167, + }, +] as const satisfies readonly WdEntitySchemaBinding[]; + +/** + * Deep-frozen at module load: the table, every row and every `anchorQids` + * list, so consumers read pinned data and any mutation throws in strict + * mode. The intuition-v2 source leaves these mutable; this is a deliberate + * hardening of the published package. + */ +function deepFreezeBindings(rows: T): T { + for (const row of rows) { + Object.freeze(row.anchorQids); + Object.freeze(row); + } + Object.freeze(rows); + return rows; +} + +export const WD_ENTITYSCHEMA_BINDINGS = deepFreezeBindings(WD_ENTITYSCHEMA_BINDING_ROWS); + +export type WdEntitySchemaSlug = (typeof WD_ENTITYSCHEMA_BINDINGS)[number]['slug']; + +/** + * Full grammar set, including ratified-but-dormant bindings. + */ +export const WD_ENTITYSCHEMA_SLUGS: readonly WdEntitySchemaSlug[] = Object.freeze( + WD_ENTITYSCHEMA_BINDINGS.map(({ slug }) => slug) +); + +const WD_ENTITYSCHEMA_SLUG_SET: ReadonlySet = new Set(WD_ENTITYSCHEMA_SLUGS); +const WD_ACTIVE_ENTITYSCHEMA_SLUG_SET: ReadonlySet = new Set( + WD_ENTITYSCHEMA_BINDINGS.filter(({ status }) => status === 'active').map(({ slug }) => slug) +); + +/** + * Runtime membership check backed by a module-private, immutable allowlist. + */ +export function isWdEntitySchemaSlug(slug: string): slug is WdEntitySchemaSlug { + return WD_ENTITYSCHEMA_SLUG_SET.has(slug); +} + +/** + * Runtime minting check for the Wave-1 active binding set. + */ +export function isActiveWdEntitySchemaSlug(slug: string): slug is WdEntitySchemaSlug { + return WD_ACTIVE_ENTITYSCHEMA_SLUG_SET.has(slug); +} From 4e43c826e43bd822d05ba4bf23578ae3ff6ab270 Mon Sep 17 00:00:00 2001 From: jonathanprozzi Date: Sat, 3 Oct 2026 23:06:51 -0400 Subject: [PATCH 2/3] =?UTF-8?q?feat(iid,=20iid-spec):=20fold=20in=20intuit?= =?UTF-8?q?ion-v2=20develop=20changes=20through=2083a003bc1=20=E2=80=94=20?= =?UTF-8?q?music=20identity=20policy,=20podcast=20feed-URL=20normalization?= =?UTF-8?q?,=20ISNI=20URL=20form=20(P16=20refresh)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Second commit on the typed-wd PR: this week's reference changes, folded into the open PR per JP (10/3) instead of a follow-up. iid - music-identity-policy.ts (new, verbatim): MUSIC_IDENTITY_RUNG_POLICY (artist → MusicGroup: isni, mbid:artist, wd, spotify:artist; music-album → MusicAlbum: mbid:release-group, wd, spotify:album) and isPlainWdPrimaryAllowed(schemaType). The iid-ladder refresh consumes it for policy-gated plain-QID primaries (R23). - uuid5.ts: normalizePodcastFeedUrl (folds scheme/host case, strips the scheme and trailing slashes, preserves userinfo, port, path and query bytes). derivePodcastGuid unchanged. - schemes.ts: the ISNI canonicalizer also accepts the https://isni.org/isni/ URL form (with or without www., host case-insensitive), stripping it to the bare form before the existing canonicalization. Every existing canonical output is byte-identical. - index.ts exports; README section; two new tests (feed normalization preserves bytes; isni.org evidence canonicalizes, wrong hosts and checksums rejected). iid-spec (R24) - schemes/isni.md: accepted input forms note; the URL form is an additive value-grammar extension under §9.3.1 (previously invalid input becomes valid, canonical bytes unchanged), provenance intuition-v2 develop @ 83a003bc1. - conformance: four additive ISNI vectors (canonicalization, validation); no existing vector changed. Corpus: 167 vectors. Typed-wd itself is unchanged: SCHEME_TYPING, the bindings table, parse.ts and derive.ts are untouched. No version or dependency changes. Gates: iid-spec corpus 167 vectors OK; iid 270 tests, typecheck, check, build; iid-ladder 12, iid-registry 42, classifications 38, primitives 140 against the rebuilt dist; pack dry run. --- .../conformance/canonicalization.json | 21 +++++++++++++++ packages/iid-spec/conformance/validation.json | 7 +++++ packages/iid-spec/schemes/isni.md | 6 +++++ packages/iid/README.md | 6 +++++ .../src/__tests__/podcast-feed-url.test.ts | 15 +++++++++++ packages/iid/src/__tests__/schemes.test.ts | 26 +++++++++++++++++++ packages/iid/src/index.ts | 8 +++++- packages/iid/src/music-identity-policy.ts | 12 +++++++++ packages/iid/src/schemes.ts | 4 ++- packages/iid/src/uuid5.ts | 18 +++++++++++++ 10 files changed, 121 insertions(+), 2 deletions(-) create mode 100644 packages/iid/src/__tests__/podcast-feed-url.test.ts create mode 100644 packages/iid/src/music-identity-policy.ts diff --git a/packages/iid-spec/conformance/canonicalization.json b/packages/iid-spec/conformance/canonicalization.json index 2cdfe98..74ec04e 100644 --- a/packages/iid-spec/conformance/canonicalization.json +++ b/packages/iid-spec/conformance/canonicalization.json @@ -779,6 +779,27 @@ "raw": "Q42", "canonical": "Q42", "note": "BOM-wrapped bare QID keeps its base canonicalization" + }, + { + "id": "canon-isni-url-112", + "scheme": "isni", + "raw": "https://isni.org/isni/0000000121032683", + "canonical": "0000000121032683", + "note": "ISNI URL prefix strips to the existing bare canonical value (section 9.3.1)" + }, + { + "id": "canon-isni-url-113", + "scheme": "isni", + "raw": "https://www.isni.org/isni/0000000121032683", + "canonical": "0000000121032683", + "note": "ISNI URL accepts the www host alias" + }, + { + "id": "canon-isni-url-114", + "scheme": "isni", + "raw": "HTTPS://ISNI.ORG/isni/0000000121032683", + "canonical": "0000000121032683", + "note": "ISNI URL scheme and host are case-insensitive" } ] } diff --git a/packages/iid-spec/conformance/validation.json b/packages/iid-spec/conformance/validation.json index 1d84e56..ab791d8 100644 --- a/packages/iid-spec/conformance/validation.json +++ b/packages/iid-spec/conformance/validation.json @@ -146,6 +146,13 @@ "typing": "unambiguous", "anchorEligible": true, "note": "active television series binding is P0 eligible" + }, + { + "id": "valid-isni-url", + "iid": "int:isni:https://isni.org/isni/0000000121032683", + "valid": false, + "anchorEligible": false, + "note": "ISNI URL is accepted raw input but is not a canonical IID value" } ] } diff --git a/packages/iid-spec/schemes/isni.md b/packages/iid-spec/schemes/isni.md index f13fd46..df3a92d 100644 --- a/packages/iid-spec/schemes/isni.md +++ b/packages/iid-spec/schemes/isni.md @@ -22,6 +22,12 @@ That is what "polymorphic" means here, and it has a hard consequence: an `isni` with the additional constraint that the final character is the correct ISO 7064 mod 11-2 check character over the first 15 digits. The grammar alone is not sufficient — a conforming validator recomputes the check character. +## Accepted input forms + +The existing bare 16-character form and separator-containing display forms remain accepted. The `https://isni.org/isni/` URL form, with or without `www.`, is also accepted; the scheme and host are case-insensitive. Strip that URL prefix before applying the existing canonicalization below. Canonical output and existing trim behavior remain unchanged; a URL is an input form, never a canonical IID value. + +This follows [§9.3.1](../spec/09-registry-governance.md): an additive value-grammar extension accepts a previously-invalid raw form without changing any existing canonical bytes. The URL form mirrors v2 commit `83a003bc1` (R24, default-in-effect). + ## Canonicalization 1. **Strip separators.** Remove every hyphen, dot, and whitespace character (`[-\s.]`). ISNIs are conventionally displayed in four space-separated groups of four; the grouping carries no identity. diff --git a/packages/iid/README.md b/packages/iid/README.md index b652f77..0c07981 100644 --- a/packages/iid/README.md +++ b/packages/iid/README.md @@ -112,6 +112,12 @@ buildGen1Iid('movie', 4, { name: 'Inception', yearPublished: '2010' }) Plus the derivation utilities the spec's schemes need: `norm1` (NORM-1), `keccak16`, `geohashEncode`, `uuidv5` / `derivePodcastGuid`. +## Music identity policy and podcast feeds + +`MUSIC_IDENTITY_RUNG_POLICY` shares the music rung order: `artist` → `MusicGroup` uses `isni`, `mbid:artist`, `wd`, `spotify:artist`; `music-album` → `MusicAlbum` uses `mbid:release-group`, `wd`, `spotify:album`. `isPlainWdPrimaryAllowed(schemaType)` returns true for these two schema types, whose policy admits plain `wd`. + +`normalizePodcastFeedUrl(feedUrl)` trims input, folds the scheme and hostname case, strips the scheme and trailing slashes, and preserves userinfo, port, path and query bytes. Pass the result to `derivePodcastGuid` when normalizing feed evidence; `derivePodcastGuid` itself is unchanged. + ## What this package is not - **No classification lookup, no provider routing.** Scheme-to-classification mapping lives in `@0xintuition/iid-registry`; ladder declarations for Intuition's entity types live in `@0xintuition/classifications`. diff --git a/packages/iid/src/__tests__/podcast-feed-url.test.ts b/packages/iid/src/__tests__/podcast-feed-url.test.ts new file mode 100644 index 0000000..1fd53b6 --- /dev/null +++ b/packages/iid/src/__tests__/podcast-feed-url.test.ts @@ -0,0 +1,15 @@ +import { expect, it } from 'vitest'; +import { derivePodcastGuid, normalizePodcastFeedUrl } from '../uuid5.js'; + +it('FR3 feed normalization preserves userinfo, port, path and query bytes', () => { + const first = normalizePodcastFeedUrl('https://User:PASS@Host.Example/feed'); + const second = normalizePodcastFeedUrl('https://user:pass@host.example/feed'); + expect(first).toBe('User:PASS@host.example/feed'); + expect(derivePodcastGuid(first)).not.toBe(derivePodcastGuid(second)); + expect(normalizePodcastFeedUrl('HTTPS://User:PASS@HOST:8080/Feed?Token=AbC')).toBe( + 'User:PASS@host:8080/Feed?Token=AbC' + ); + expect(normalizePodcastFeedUrl('https://BÜCHER.EXAMPLE/Feed')).toBe('bücher.example/Feed'); + expect(normalizePodcastFeedUrl('')).toBe(''); + expect(normalizePodcastFeedUrl('/Feed?Token=AbC')).toBe('/Feed?Token=AbC'); +}); diff --git a/packages/iid/src/__tests__/schemes.test.ts b/packages/iid/src/__tests__/schemes.test.ts index 89768c8..bc1271d 100644 --- a/packages/iid/src/__tests__/schemes.test.ts +++ b/packages/iid/src/__tests__/schemes.test.ts @@ -41,6 +41,14 @@ describe('gtin', () => { }); describe('check-digit identity schemes', () => { + it('canonicalizes durable isni.org evidence and rejects wrong hosts and checksums', () => { + expect(SCHEMES.isni.canonicalize('https://isni.org/isni/0000000121367029')).toBe( + '0000000121367029' + ); + expect(SCHEMES.isni.canonicalize('https://isni.org/isni/0000000121367020')).toBeUndefined(); + expect(SCHEMES.isni.canonicalize('https://example.org/isni/0000000121367029')).toBeUndefined(); + expect(SCHEMES.orcid.canonicalize('https://isni.org/isni/0000000121367029')).toBeUndefined(); + }); it('canonicalizes ISRC by stripping separators and uppercasing', () => { expect(SCHEMES.isrc.canonicalize('us-sm1-00-07459')).toBe('USSM10007459'); }); @@ -286,3 +294,21 @@ describe('misc natural keys', () => { expect(SCHEMES.hash.canonicalize(digest.toUpperCase())).toBe(digest); }); }); + +describe('music and podcast public exports', () => { + it('exports the music identity policy and podcast feed normalizer', () => { + expect(publicIid.MUSIC_IDENTITY_RUNG_POLICY).toEqual({ + artist: { schemaType: 'MusicGroup', rungs: ['isni', 'mbid:artist', 'wd', 'spotify:artist'] }, + 'music-album': { + schemaType: 'MusicAlbum', + rungs: ['mbid:release-group', 'wd', 'spotify:album'], + }, + }); + expect(publicIid.isPlainWdPrimaryAllowed('MusicGroup')).toBe(true); + expect(publicIid.isPlainWdPrimaryAllowed('MusicAlbum')).toBe(true); + expect(publicIid.isPlainWdPrimaryAllowed('Person')).toBe(false); + expect(publicIid.normalizePodcastFeedUrl('HTTPS://Host.Example/Feed/')).toBe( + 'host.example/Feed' + ); + }); +}); diff --git a/packages/iid/src/index.ts b/packages/iid/src/index.ts index df43fa9..3405031 100644 --- a/packages/iid/src/index.ts +++ b/packages/iid/src/index.ts @@ -2,6 +2,7 @@ export { deriveIntuitionId } from './derive.js'; export { buildGen1Iid } from './gen1.js'; export { geohashEncode } from './geohash.js'; export { keccak16 } from './hash.js'; +export { isPlainWdPrimaryAllowed, MUSIC_IDENTITY_RUNG_POLICY } from './music-identity-policy.js'; export { norm1 } from './norm.js'; export { formatIntuitionId, @@ -33,7 +34,12 @@ export type { SchemeTyping, } from './types.js'; export { SCHEME_NAMES } from './types.js'; -export { derivePodcastGuid, PODCAST_GUID_NAMESPACE, uuidv5 } from './uuid5.js'; +export { + derivePodcastGuid, + normalizePodcastFeedUrl, + PODCAST_GUID_NAMESPACE, + uuidv5, +} from './uuid5.js'; export type { WdEntitySchemaBinding, WdEntitySchemaSlug } from './wd-entityschema-bindings.js'; export { isActiveWdEntitySchemaSlug, diff --git a/packages/iid/src/music-identity-policy.ts b/packages/iid/src/music-identity-policy.ts new file mode 100644 index 0000000..358f7e3 --- /dev/null +++ b/packages/iid/src/music-identity-policy.ts @@ -0,0 +1,12 @@ +/** Music rows shared by the ladder and classification's existing IID dependency. */ +export const MUSIC_IDENTITY_RUNG_POLICY = { + artist: { schemaType: 'MusicGroup', rungs: ['isni', 'mbid:artist', 'wd', 'spotify:artist'] }, + 'music-album': { schemaType: 'MusicAlbum', rungs: ['mbid:release-group', 'wd', 'spotify:album'] }, +} as const; + +/** Only a music policy row admitting plain wd permits a plain-QID primary. */ +export function isPlainWdPrimaryAllowed(schemaType: string): boolean { + return Object.values(MUSIC_IDENTITY_RUNG_POLICY).some( + (row) => row.schemaType === schemaType && (row.rungs as readonly string[]).includes('wd') + ); +} diff --git a/packages/iid/src/schemes.ts b/packages/iid/src/schemes.ts index f51f968..682068b 100644 --- a/packages/iid/src/schemes.ts +++ b/packages/iid/src/schemes.ts @@ -483,7 +483,9 @@ export const SCHEMES: Readonly> = { isbn: define('isbn', 'A', canonicalizeIsbn), isrc: define('isrc', 'A', canonicalizeIsrc), iswc: define('iswc', 'A', canonicalizeIswc), - isni: define('isni', 'A', canonicalizeIsni), + isni: define('isni', 'A', (raw) => + canonicalizeIsni(raw.replace(/^https:\/\/(?:www\.)?isni\.org\/isni\//i, '')) + ), orcid: define('orcid', 'A', canonicalizeIsni), lei: define('lei', 'A', canonicalizeLei), gtin: define('gtin', 'A', canonicalizeGtin), diff --git a/packages/iid/src/uuid5.ts b/packages/iid/src/uuid5.ts index 9d3a8cf..57a941b 100644 --- a/packages/iid/src/uuid5.ts +++ b/packages/iid/src/uuid5.ts @@ -45,3 +45,21 @@ export function derivePodcastGuid(feedUrl: string): string { return uuidv5(stripped, PODCAST_GUID_NAMESPACE); } + +/** Canonical feed GUID input: fold scheme/host only, preserving path and query bytes. */ +export function normalizePodcastFeedUrl(feedUrl: string): string { + return feedUrl + .trim() + .replace( + /^([a-z][a-z0-9+.-]*:\/\/)?([^/?#]*@)?(\[[^\]]+\]|[^:/?#]+)(:[0-9]+)?/i, + ( + _match, + scheme: string | undefined, + userinfo: string | undefined, + hostname: string, + port: string | undefined + ) => `${scheme?.toLowerCase() ?? ''}${userinfo ?? ''}${hostname.toLowerCase()}${port ?? ''}` + ) + .replace(/^[a-z][a-z0-9+.-]*:\/\//i, '') + .replace(/\/+$/, ''); +} From 4e21a146e69371244a53f01dd2c971f848624812 Mon Sep 17 00:00:00 2001 From: jonathanprozzi Date: Sat, 3 Oct 2026 23:15:58 -0400 Subject: [PATCH 3/3] docs(iid-spec), test(iid): ISNI URL prefix is matched case-insensitively as a whole, including the /isni/ path (P16 refresh review) The adversarial review of the refresh found the spec note narrower than the ported canonicalizer: the regex's case-insensitive flag folds the whole prefix, so `https://isni.org/ISNI/` is accepted, exactly as the reference implementation accepts it. No behavior change on the train; the scheme doc now states the whole-prefix fold, the corpus gains a vector for the upper-case path form (168 vectors), and the schemes test asserts it (iid 271 tests). --- packages/iid-spec/conformance/canonicalization.json | 7 +++++++ packages/iid-spec/schemes/isni.md | 2 +- packages/iid/src/__tests__/schemes.test.ts | 3 +++ 3 files changed, 11 insertions(+), 1 deletion(-) diff --git a/packages/iid-spec/conformance/canonicalization.json b/packages/iid-spec/conformance/canonicalization.json index 74ec04e..4a9ba85 100644 --- a/packages/iid-spec/conformance/canonicalization.json +++ b/packages/iid-spec/conformance/canonicalization.json @@ -800,6 +800,13 @@ "raw": "HTTPS://ISNI.ORG/isni/0000000121032683", "canonical": "0000000121032683", "note": "ISNI URL scheme and host are case-insensitive" + }, + { + "id": "canon-isni-url-115", + "scheme": "isni", + "raw": "https://isni.org/ISNI/0000000121032683", + "canonical": "0000000121032683", + "note": "ISNI URL path segment is matched case-insensitively as well (whole-prefix fold, as the reference implementation does)" } ] } diff --git a/packages/iid-spec/schemes/isni.md b/packages/iid-spec/schemes/isni.md index df3a92d..9de0f3d 100644 --- a/packages/iid-spec/schemes/isni.md +++ b/packages/iid-spec/schemes/isni.md @@ -24,7 +24,7 @@ with the additional constraint that the final character is the correct ISO 7064 ## Accepted input forms -The existing bare 16-character form and separator-containing display forms remain accepted. The `https://isni.org/isni/` URL form, with or without `www.`, is also accepted; the scheme and host are case-insensitive. Strip that URL prefix before applying the existing canonicalization below. Canonical output and existing trim behavior remain unchanged; a URL is an input form, never a canonical IID value. +The existing bare 16-character form and separator-containing display forms remain accepted. The `https://isni.org/isni/` URL form, with or without `www.`, is also accepted; the whole prefix (scheme, host and the `/isni/` path segment) is matched case-insensitively, so `https://isni.org/ISNI/` is accepted as well; the identifier itself is then canonicalized as below. Strip that URL prefix before applying the existing canonicalization below. Canonical output and existing trim behavior remain unchanged; a URL is an input form, never a canonical IID value. This follows [§9.3.1](../spec/09-registry-governance.md): an additive value-grammar extension accepts a previously-invalid raw form without changing any existing canonical bytes. The URL form mirrors v2 commit `83a003bc1` (R24, default-in-effect). diff --git a/packages/iid/src/__tests__/schemes.test.ts b/packages/iid/src/__tests__/schemes.test.ts index bc1271d..c1b7863 100644 --- a/packages/iid/src/__tests__/schemes.test.ts +++ b/packages/iid/src/__tests__/schemes.test.ts @@ -46,6 +46,9 @@ describe('check-digit identity schemes', () => { '0000000121367029' ); expect(SCHEMES.isni.canonicalize('https://isni.org/isni/0000000121367020')).toBeUndefined(); + expect(SCHEMES.isni.canonicalize('https://isni.org/ISNI/0000000121367029')).toBe( + '0000000121367029' + ); expect(SCHEMES.isni.canonicalize('https://example.org/isni/0000000121367029')).toBeUndefined(); expect(SCHEMES.orcid.canonicalize('https://isni.org/isni/0000000121367029')).toBeUndefined(); });