Skip to content

About

CalConnect Document Registry

Resources

Stars

1 star

Watchers

22 watching

Forks

Repository files navigation

CalConnect Standards Registry

Build Status

Introduction

The CalConnect Standards Registry publishes CalConnect deliverables — standards, specifications, guides, directives, and other documents — at https://standards.calconnect.org.

The site is an aggregator: it discovers Metanorma document repositories via the metanorma-release GitHub topic, fetches their published releases, and builds a static site with full-text search, category pages, and document downloads.

The site is managed by TC PUBLISH.

Sites

Site Git branch URL

Production

main

The production site is automatically deployed from changes to the main branch. It also rebuilds daily at 06:00 UTC to pick up new releases from document repos.

Architecture

How documents get published

  1. Document repo — A Metanorma repo (e.g. cc-icalendar-series) compiles .adoc sources and runs the release workflow, which publishes ZIP archives to GitHub Releases with channel labels (e.g. public/standards).

  2. Aggregation (producer) — metanorma-release aggregate discovers repos by the metanorma-release topic, fetches their releases, filters by channel, extracts document files, and enriches metadata from Relaton bibliographic data.

  3. Enrichment (contract layer) — rake enrich (the engine gem) turns the producer outputs into the catalog contract: Relaton is the single source of truth, every top-level field is a documented projection, files carry media_type/bytes/sha256 and stable URLs, items carry provenance, language/license/copyright, a URN in the instance namespace, and the per-document editions model. The handoff directory registry/ is renderer-neutral — no renderer paths cross it.

  4. Citations — rake citations uses the relaton npm package (relaton-ts) to emit per-document ISO 690, BibTeX, RIS and CSL-JSON exports. Citation rendering is never hand-rolled.

  5. Rendering — astro build reads registry/ and prerenders the versioned document pages, latest aliases, /catalog.json (+ .sha256 checksum sidecar), /search-index.json, /feed.xml (Atom), /opensearch.xml, /sitemap.xml, and all site pages with Google-Scholar citation_* meta and JSON-LD. rake finalize then copies the document artifacts and citation exports into dist/docs/, writes the legacy /cc/ redirect stubs, and builds the Pagefind full-text index behind /search/. Bibliographic relations (updatedBy, derivedFrom, …) project from Relaton and render as cross-linked Related-documents sections.

  6. Deploy — GitHub Pages serves the built site.

The data contract

The most valuable thing this repository produces is the document catalog:

Stability promise: within schema major version 1, changes are additive only; consumers must ignore unknown keys. The Relaton record in each item’s bibliographic field is canonical; other fields are projections documented in the schema and verified by rake validate_consistency on every build.

Any generator that emits a payload conforming to the schema is a valid producer. The metanorma-release gem is one producer — not the definition of the data. The engine gem’s fixtures/seed/ contains a hand-written producer used by the conformance suite (rake conformance), and rake validate_index[path] validates any candidate catalog against the published schema.

The registry engine — the contract schemas, enrichment, registry-validate and registry-conformance CLIs, and golden fixtures — lives in metanorma/standards-registry and is consumed here as a gem (see Gemfile). This repository keeps only the instance configuration (_config.yml registry: map
metanorma.aggregate.yml), the reference Astro renderer (src/), branding and content. The conformance profile and instance guide are published in the product repo; schemas are served at their $id paths from the product’s Pages (https://schemas.metanorma.org DNS pending org setup; meanwhile https://metanorma.github.io/standards-registry/registry/documents-index/v1.json).

Software stack

Quick start

# Install dependencies
bundle install
npm ci

# Aggregate + enrich the catalog + build the site
bundle exec rake build

# Serve locally
bundle exec rake serve
Note
Aggregation requires a GITHUB_TOKEN with read access to the CalConnect organization repos. Set it as an environment variable (e.g. export GITHUB_TOKEN=$(gh auth token)).

Build commands

The Rakefile provides these tasks:

Command Description

rake fetch

Aggregate releases into .artifacts/docs/ (network)

rake enrich

Build registry/catalog.json + search-index.json + backfill.json from producer outputs (no network)

rake citations

Generate per-document ISO 690/BibTeX/RIS/CSL exports via relaton-ts

rake build

Fetch + enrich + citations + Astro build + finalize + catalog-count guard

rake site

Astro build + finalize only (assumes enrich already done)

rake serve

Serve dist/ with astro preview

rake clean

Remove dist/, registry/, .artifacts/

rake validate_schema

Validate the catalog against the published schema

rake validate_consistency

Recompute Relaton projections; verify files, URLs, editions

rake verify_links

Check every internal link in the built site resolves

rake validate_index[path]

Validate any candidate catalog against the schema (generator independence)

rake conformance

Certify the built site and a fixture build with registry-conformance

rake spec

Run the RSpec suite

Configuration

_config.yml — registry instance configuration

registry:
  org: calconnect
  urn_namespace: calconnect
  url_scheme: "/docs/:document_id/:year/"
  docs_prefix: "/docs"
  legacy_prefixes: ["cc"]
  features:
    search: true
    atom_feed: true
    jsonld: true
    opensearch: true
    html: true
  license_default:
    name: "© CalConnect — all rights reserved"
    url: "https://www.calconnect.org/about/policies/copyright-and-licensing/"
  • org, urn_namespace — identity used for URNs (urn:calconnect:cc-18011:2018)

  • url_scheme — canonical document URL template (:document_id, :year, :edition tokens)

  • legacy_prefixes — old artifact prefixes that get redirect stubs (HTML only)

  • features — toggles for search/feed/JSON-LD/OpenSearch; html: false selects headless mode

  • license_default — fallback rights statement for documents whose Relaton record carries no license; absent fields stay tracked in registry/backfill.json (never invented)

metanorma.aggregate.yml

Controls how the aggregator discovers repos and builds output:

source: github
output_dir: .artifacts/docs
file_routing: flat
cache_dir: .cache/aggregate

channels:
  - public

include_drafts: true

display_categories:
  - name: Standards, Specifications & Reports
    slug: standards
    doctypes: [standard, specification, report]
  - name: Guides & Advisories
    slug: guides
    doctypes: [guide, advisory]
  - name: Directives
    slug: directives
    doctypes: [directive]
  - name: Administrative
    slug: administrative
    doctypes: [administrative]
  - name: Amendments & Technical Corrigenda
    slug: amendments
    doctypes: [amendment, technical-corrigendum]

github:
  organizations:
    - CalConnect
  topic: metanorma-release

Key settings:

  • output_dir — where extracted document files are written (kept out of dist/ until finalize)

  • channels — which channels to subscribe to (public matches all public/*)

  • display_categories — maps doctypes to site categories

  • include_drafts — include working drafts, committee drafts, etc.

Delta state cache

Aggregation is incremental. The .cache/aggregate/ directory tracks which repos/tags have been processed. CI caches this directory between builds to avoid re-downloading unchanged releases.

When the output directory is missing (fresh runner), the Rakefile resets the delta state so artifacts are re-extracted from the cached downloads.

How to add/remove documents

Adding a document

Documents are added by creating a new Metanorma repository:

  1. Use the cc-template repository template

  2. Add the metanorma-release topic to the new repository

  3. Write the document in AsciiDoc and push to main

The document will appear on standards.calconnect.org after:

  1. The document repo’s release workflow runs (on push to main)

  2. This site’s next build (push to main or daily cron)

No changes to this repository are needed.

Removing a document

  1. Archive or delete the document repository, or

  2. Remove the metanorma-release topic from the repository

The document will disappear from the site on the next build.

Changing a document’s category

Document categories are determined by doctype. Edit metanorma.aggregate.yml → display_categories to change which doctypes map to which category.

Project structure

├── .github/workflows/
│   └── build_deploy.yml        # CI: aggregate + build + certify + deploy
├── _config.yml                 # Instance identity + registry: map (engine + renderer read this)
├── _data/
│   └── navigation.yml          # Site navigation categories (instance content)
├── astro.config.mjs            # Astro config (site from _config.yml, Tailwind vite plugin)
├── src/
│   ├── layouts/                # Base.astro, Listing.astro, Document.astro
│   ├── components/             # DocumentCard.astro
│   ├── pages/                  # Home, category, drafts, public-review, patents, search, 404
│   │   ├── docs/[...route].astro  # Versioned landings + latest aliases
│   │   ├── catalog.json.ts     # /catalog.json + .sha256 endpoints
│   │   ├── search-index.json.ts
│   │   ├── feed.xml.ts         # Atom
│   │   ├── opensearch.xml.ts
│   │   └── sitemap.xml.ts
│   ├── lib/registry.ts         # Build-time catalog/navigation/site accessor
│   └── styles/global.css       # Tailwind entry (imports _frontend/base.css)
├── public/
│   ├── js/                     # theme.js, navigation.js, document-list.js, home.js
│   └── (favicons, images)
├── _frontend/
│   └── base.css                # Shared theme variables, nav styles, doctype colors
├── registry/                   # Renderer-neutral handoff (generated, from the engine gem):
│   ├── catalog.json            #   the contract payload
│   ├── search-index.json       #   derived search corpus
│   ├── citations/              #   per-document ISO 690 / BibTeX / RIS / CSL JSON
│   └── backfill.json           #   honest record of data gaps
├── dist/                       # Built site output (gitignored)
│   └── docs/                   # Extracted document files (HTML, PDF, XML, RXL) + citations
├── Gemfile                     # Ruby dependencies
├── Rakefile                    # Build tasks
├── metanorma.aggregate.yml     # Aggregator configuration
└── package.json                # Node.js dependencies

CI/CD

The build_deploy workflow:

  1. Sets up Node.js 24 + Ruby 3.4

  2. Restores aggregate cache (.cache/aggregate/)

  3. Runs bundle exec rake build (aggregate + enrich + citations + Astro + finalize)

  4. Validates the catalog schema and consistency, runs the RSpec suite, verifies internal links, and reports catalog changes vs production

  5. Uploads the dist/ Pages artifact

  6. A separate conformance job certifies the built site and a fixture build

  7. Deploys to GitHub Pages (on main only)

Scheduled builds run daily at 06:00 UTC to pick up new releases.

About

CalConnect Document Registry

Resources

Stars

1 star

Watchers

22 watching

Forks

Releases

Packages

Used by

Contributors

Languages