Skip to content

About

Potential service to generate catalog results on the fly

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

Catalog Generator Service

A FastAPI-based service that dynamically generates STAC catalog responses based on GitHub Pull Requests using eodash_catalog.

Overview

This service mocks a STAC endpoint. When requested with a GitHub PR URL, it:

  1. Fetches the changed catalog configuration files from the PR.
  2. Uses eodash_catalog to generate the STAC JSON responses on the fly.
  3. Serves these responses to allow instant previewing in a STAC browser.

Dependency Management

This project uses pip-tools for deterministic dependency management.

  • requirements.in: Contains our top-level dependencies (edit this file).
  • requirements.txt: The auto-generated lockfile containing exact versions of all packages and sub-dependencies (do not edit manually).

How to Add, Upgrade, or Pin a Package

  1. Modify the source file: Open requirements.in and add or update your package, including any specific version pins if necessary.

    # Example: Pinning a specific version
    eodash_catalog==0.4.6
    
  2. Compile the updated lockfile:

    pip-compile requirements.in
    
  3. Install the changes locally: Sync your active virtual environment so it matches the newly generated lockfile.

    pip install -r requirements.txt
    
  4. Commit both files:

    Always commit both the .in and .txt files to version control so the environment remains fully reproducible for everyone.

Getting Started

Prerequisites

  • Docker
  • Python 3.11+ (for local development)

Configuration

The service can be configured using environment variables:

  • GITHUB_TOKEN: (Optional) GitHub Personal Access Token for higher rate limits.
  • CATALOG_CACHE_DIR: Directory where repositories and generated catalogs are cached (default: /tmp/catalog_cache).
  • REPO_SECRETS_JSON: (Optional) A JSON string mapping repository names to specific secrets. This is useful when different repositories require different secrets for the same key, or when various APIs are used (e.g., Sentinel Hub, Mapbox, AWS).
    {
      "GTIF-Austria/public-catalog": {
        "SH_INSTANCE_ID": "instance-1",
        "SH_CLIENT_ID": "client-1",
        "SH_CLIENT_SECRET": "secret-1"
      },
      "other-org/other-repo": {
        "MAPBOX_TOKEN": "pk.ey...",
        "AWS_ACCESS_KEY_ID": "AKIA..."
      }
    }
  • Global Secrets: Any other environment variables set on the service will act as defaults for all repositories.

Running with Docker

docker build -t catalog-generator-service .

# Run the container with global secrets and/or repo-specific mapping:
docker run -p 8000:8000 \
  -e GITHUB_TOKEN="$GITHUB_TOKEN" \
  -e REPO_SECRETS_JSON='{"GTIF-Austria/public-catalog": {"SH_INSTANCE_ID": "..."}}' \
  catalog-generator-service

Local Development

For easier development and testing, you can run the service directly without Docker.

Prerequisites

  • Python 3.11+
  • Git: Required for cloning repositories from GitHub.

1. Set up a Virtual Environment

It is recommended to use a virtual environment to manage dependencies:

# Create a virtual environment
python3 -m venv venv

# Activate the virtual environment
# On Linux/macOS:
source venv/bin/activate
# On Windows:
# .\venv\Scripts\activate

2. Install Dependencies

pip install -r requirements.txt

3. Run the Service

You can use uvicorn to run the service with auto-reload enabled for development:

uvicorn main:app --reload --host 127.0.0.1 --port 8000

The service will be available at http://127.0.0.1:8000. You can access the API documentation at http://127.0.0.1:8000/docs.

Refresh Cache

If the data behind a configuration changes and you need to re-generate the catalog without creating a new commit, you can use the optional refresh parameter in your request:

/{owner}/{repo}/pull/{number}/catalog.json?refresh=true

This will bypass the internal cache and force the service to re-run eodash_catalog for the specified PR.

Testing

The project uses pytest for unit testing. The test files are located in the tests/ directory.

To run the tests, ensure you have your virtual environment activated and the dependencies installed, then run:

pytest

If you encounter an issue where pytest is not found, you can run it as a Python module:

python3 -m pytest

About

Potential service to generate catalog results on the fly

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages