Stabilo is a specialized Python package for stabilizing video frames or tracked object trajectories in videos, using robust homography or affine transformations. Its core functionality focuses on aligning each frame or object track to a chosen reference frame, enabling precise stabilization that mitigates disturbances like camera movements. Key features include robust keypoint-based image registration and the option to integrate user-defined masks, which exclude dynamic regions (e.g., moving objects) to enhance stabilization accuracy. Integrating seamlessly with object detection and tracking algorithms, Stabilo is ideal for high-precision applications like urban traffic monitoring, as demonstrated in the Geo-trax 🚀 trajectory extraction framework. Extensive transformation and enhancement options, including multiple feature detectors and matchers, masking techniques, further expand its utility. For systematic evaluation and hyperparameter tuning, the companion tool Stabilo-Optimize 🎯 provides a dedicated benchmarking framework. The repository also includes valuable resources like utility scripts and example videos to demonstrate its capabilities.
\nstabilo console command; the Stabilizer class embeds into your own pipeline in a handful of lines.📌 Important: If you use this code in your work, kindly acknowledge it by citing the following article:
\n\n\nRobert Fonod, Haechan Cho, Hwasoo Yeo, Nikolas Geroliminis (2025). Advanced computer vision for extracting georeferenced vehicle trajectories from drone imagery, Transportation Research Part C: Emerging Technologies, vol. 178, 105205. DOI: 10.1016/j.trc.2025.105205
\n
gpu=True) when built against a CUDA-enabled OpenCV; see docs/cuda.md for build instructions. RANSAC-based transformation estimation always runs on CPU (no OpenCV CUDA equivalent exists). Learning-based detectors run on CPU, CUDA, or Apple MPS via the device option.stabilo console command (stabilo video and stabilo tracks subcommands) for stabilizing videos and object trajectories without writing code.Stabilo integrates with and complements several specialized tools:
\nGeo-trax 🚀 — End-to-end framework for extracting georeferenced vehicle trajectories from drone imagery. Uses Stabilo as its core stabilization engine to align video frames and vehicle tracks to a common reference frame.
\nStabilo-Optimize 🎯 — Benchmarking and hyperparameter optimization framework for Stabilo. Evaluates stabilization performance through ground truth-free assessment using random perturbations. Used to fine-tune Stabilo parameters.
\nHBB2OBB 📦 — Converts horizontal bounding boxes to oriented bounding boxes using SAM segmentation models. Can be used alongside Stabilo when object orientation is needed for downstream analysis.
\nIt is recommended to create and activate a Python virtual environment (Python >= 3.11 and <= 3.13) first:
\npython3.11 -m venv .venv\nsource .venv/bin/activate # On Windows: .venv\\Scripts\\activateconda create -n stabilo python=3.11 -y\nconda activate stabilouv (fastest; use uv pip install in step 3):
uv venv --python 3.11\nsource .venv/bin/activate # On Windows: .venv\\Scripts\\activateThen, install the stabilo library using one of the following options:
\npip install stabiloYou can also clone the repository and install the package from the local source:
\ngit clone https://github.com/rfonod/stabilo.git\ncd stabilo && pip install .If you want the changes you make in the repo to be reflected in your install, use pip install -e . instead of pip install ..
Every tunable parameter lives in one self-contained YAML file, stabilo/cfg/default.yaml. The same keys are accepted as Stabilizer(**kwargs) arguments, as CLI flags, and as entries in a custom YAML file passed via --custom-config; precedence is explicit CLI flags > custom config file > built-in defaults. Manage the bundled configuration with stabilo config show and stabilo config copy. Two independent acceleration knobs are available: gpu (OpenCV CUDA, classical detectors only) and device (PyTorch device, learning-based detectors/matchers only).
from stabilo import Stabilizer \n\n# Create an instance of the Stabilizer class with default parameters\nstabilizer = Stabilizer() \n\n# Set a reference frame with (optional) mask\nstabilizer.set_ref_frame(ref_frame, ref_mask)\n\n# Stabilize any frame with (optional) mask\nstabilizer.stabilize(cur_frame, cur_mask)\n\n# Get the stabilized (warped) frame \nstabilized_frame = stabilizer.warp_cur_frame()\n\n# Transform current masks (bounding boxes) if it was provided\nstabilized_boxes = stabilizer.transform_cur_boxes()\n\n# Transform any point (pixel coordinates) from the current frame to reference frame\ncur_point = np.array([x, y, 1])\nref_point = stabilizer.get_cur_trans_matrix() @ cur_pointStabilo supports multiple mask and bounding-box formats:
\nxywh: Axis-aligned boxes [x_center, y_center, width, height]xywha: Oriented bounding boxes [x_center, y_center, width, height, angle_degrees]four: Four corner points [x1, y1, x2, y2, x3, y3, x4, y4] (auto-detects if rotated)polygon: Polygon points as flattened rows [x1, y1, ..., xN, yN] or (N, 2) point arrayscircle: Circular masks [x_center, y_center, radius]import numpy as np\n\n# Example with oriented bounding boxes (OBBs)\nobb_boxes = np.array([\n [100, 150, 50, 30, 45], # Rotated 45 degrees\n [200, 200, 60, 40, 90], # Rotated 90 degrees\n])\n\nstabilizer.set_ref_frame(ref_frame, obb_boxes, box_format='xywha')\nstabilizer.stabilize(cur_frame, cur_boxes, box_format='xywha')\n\n# Transform boxes and get result in different format\ntransformed = stabilizer.transform_cur_boxes(out_box_format='four')# Polygon mask(s)\npolygon_masks = np.array([\n [60, 60, 120, 60, 120, 120, 60, 120],\n [200, 200, 260, 210, 240, 270, 190, 260],\n])\nstabilizer.set_ref_frame(ref_frame, polygon_masks, box_format='polygon')\n\n# Circular mask(s)\ncircle_masks = np.array([\n [120, 120, 25],\n [360, 240, 40],\n])\nstabilizer.stabilize(cur_frame, circle_masks, box_format='circle')For detailed package documentation, including architecture, end-to-end workflows, mask format specifications, and API behavior notes, see:
\ndocs/usage.mdInstalling stabilo provides a stabilo console command with three subcommands: stabilo video (stabilize a video), stabilo tracks (stabilize object trajectories), and stabilo config (inspect or copy the default configuration). Run stabilo video --help or stabilo tracks --help for the full option reference. Each CLI invocation performs a best-effort check for a newer stabilo release on PyPI.
find_threshold_models.py: Computes regression models between detection thresholds and average keypoint counts for BRISK, KAZE, and AKAZE feature detectors.If you use Stabilo in your research, software, or product, please cite the following resources appropriately:
\nPreferred Citation: Please cite the associated article for any use of the Stabilo package, including research, applications, and derivative work:
\n@article{fonod2025advanced,\n title = {Advanced computer vision for extracting georeferenced vehicle trajectories from drone imagery},\n author = {Fonod, Robert and Cho, Haechan and Yeo, Hwasoo and Geroliminis, Nikolas},\n journal = {Transportation Research Part C: Emerging Technologies},\n volume = {178},\n pages = {105205},\n year = {2025},\n publisher = {Elsevier},\n doi = {10.1016/j.trc.2025.105205},\n url = {https://doi.org/10.1016/j.trc.2025.105205}\n}Repository Citation: If you reference, modify, or build upon the Stabilo software itself, please also cite the corresponding Zenodo release:
\n@software{fonod2026stabilo,\n author = {Fonod, Robert},\n license = {MIT},\n month = aug,\n title = {Stabilo: A Comprehensive Python Library for Video and Trajectory Stabilization with User-Defined Masks},\n url = {https://github.com/rfonod/stabilo},\n doi = {10.5281/zenodo.12117092},\n version = {1.4.1},\n year = {2026}\n}Contributions from the community are welcome! If you encounter any issues or have suggestions for improvements, please open a GitHub Issue or submit a pull request.
\nThis project is distributed under the MIT License. See the LICENSE file for more details.
", + "version": "1.4.2", + "description": "Stabilo is a specialized Python package for stabilizing video frames or tracked object trajectories in videos, using robust homography or affine transformations. Its core functionality focuses on aligning each frame or object track to a chosen reference frame, enabling precise stabilization that mitigates disturbances like camera movements. Key features include robust keypoint-based image registration and the option to integrate user-defined masks, which exclude dynamic regions (e.g., moving objects) to enhance stabilization accuracy. Integrating seamlessly with object detection and tracking algorithms, Stabilo is ideal for high-precision applications like urban traffic monitoring, as demonstrated in the Geo-trax 🚀 trajectory extraction framework. Extensive transformation and enhancement options, including multiple feature detectors and matchers, masking techniques, further expand its utility. For systematic evaluation and hyperparameter tuning, the companion tool Stabilo-Optimize 🎯 provides a dedicated benchmarking framework. The repository also includes valuable resources like utility scripts and example videos to demonstrate its capabilities.
\nstabilo console command; the Stabilizer class embeds into your own pipeline in a handful of lines.📌 Important: If you use this code in your work, kindly acknowledge it by citing the following article:
\n\n\nRobert Fonod, Haechan Cho, Hwasoo Yeo, Nikolas Geroliminis (2025). Advanced computer vision for extracting georeferenced vehicle trajectories from drone imagery, Transportation Research Part C: Emerging Technologies, vol. 178, 105205. DOI: 10.1016/j.trc.2025.105205
\n
gpu=True) when built against a CUDA-enabled OpenCV; see docs/cuda.md for build instructions. RANSAC-based transformation estimation always runs on CPU (no OpenCV CUDA equivalent exists). Learning-based detectors run on CPU, CUDA, or Apple MPS via the device option.stabilo console command (stabilo video and stabilo tracks subcommands) for stabilizing videos and object trajectories without writing code.Stabilo integrates with and complements several specialized tools:
\nGeo-trax 🚀 — End-to-end framework for extracting georeferenced vehicle trajectories from drone imagery. Uses Stabilo as its core stabilization engine to align video frames and vehicle tracks to a common reference frame.
\nStabilo-Optimize 🎯 — Benchmarking and hyperparameter optimization framework for Stabilo. Evaluates stabilization performance through ground truth-free assessment using random perturbations. Used to fine-tune Stabilo parameters.
\nHBB2OBB 📦 — Converts horizontal bounding boxes to oriented bounding boxes using SAM segmentation models. Can be used alongside Stabilo when object orientation is needed for downstream analysis.
\nIt is recommended to create and activate a Python virtual environment (Python >= 3.11 and <= 3.13) first:
\npython3.11 -m venv .venv\nsource .venv/bin/activate # On Windows: .venv\\Scripts\\activateconda create -n stabilo python=3.11 -y\nconda activate stabilouv (fastest; use uv pip install in step 3):
uv venv --python 3.11\nsource .venv/bin/activate # On Windows: .venv\\Scripts\\activateThen, install the stabilo library using one of the following options:
\npip install stabiloYou can also clone the repository and install the package from the local source:
\ngit clone https://github.com/rfonod/stabilo.git\ncd stabilo && pip install .If you want the changes you make in the repo to be reflected in your install, use pip install -e . instead of pip install ..
Every tunable parameter lives in one self-contained YAML file, stabilo/cfg/default.yaml. The same keys are accepted as Stabilizer(**kwargs) arguments, as CLI flags, and as entries in a custom YAML file passed via --custom-config; precedence is explicit CLI flags > custom config file > built-in defaults. Manage the bundled configuration with stabilo config show and stabilo config copy. Two independent acceleration knobs are available: gpu (OpenCV CUDA, classical detectors only) and device (PyTorch device, learning-based detectors/matchers only).
from stabilo import Stabilizer \n\n# Create an instance of the Stabilizer class with default parameters\nstabilizer = Stabilizer() \n\n# Set a reference frame with (optional) mask\nstabilizer.set_ref_frame(ref_frame, ref_mask)\n\n# Stabilize any frame with (optional) mask\nstabilizer.stabilize(cur_frame, cur_mask)\n\n# Get the stabilized (warped) frame \nstabilized_frame = stabilizer.warp_cur_frame()\n\n# Transform current masks (bounding boxes) if it was provided\nstabilized_boxes = stabilizer.transform_cur_boxes()\n\n# Transform any point (pixel coordinates) from the current frame to reference frame\ncur_point = np.array([x, y, 1])\nref_point = stabilizer.get_cur_trans_matrix() @ cur_pointStabilo supports multiple mask and bounding-box formats:
\nxywh: Axis-aligned boxes [x_center, y_center, width, height]xywha: Oriented bounding boxes [x_center, y_center, width, height, angle_degrees]four: Four corner points [x1, y1, x2, y2, x3, y3, x4, y4] (auto-detects if rotated)polygon: Polygon points as flattened rows [x1, y1, ..., xN, yN] or (N, 2) point arrayscircle: Circular masks [x_center, y_center, radius]import numpy as np\n\n# Example with oriented bounding boxes (OBBs)\nobb_boxes = np.array([\n [100, 150, 50, 30, 45], # Rotated 45 degrees\n [200, 200, 60, 40, 90], # Rotated 90 degrees\n])\n\nstabilizer.set_ref_frame(ref_frame, obb_boxes, box_format='xywha')\nstabilizer.stabilize(cur_frame, cur_boxes, box_format='xywha')\n\n# Transform boxes and get result in different format\ntransformed = stabilizer.transform_cur_boxes(out_box_format='four')# Polygon mask(s)\npolygon_masks = np.array([\n [60, 60, 120, 60, 120, 120, 60, 120],\n [200, 200, 260, 210, 240, 270, 190, 260],\n])\nstabilizer.set_ref_frame(ref_frame, polygon_masks, box_format='polygon')\n\n# Circular mask(s)\ncircle_masks = np.array([\n [120, 120, 25],\n [360, 240, 40],\n])\nstabilizer.stabilize(cur_frame, circle_masks, box_format='circle')For detailed package documentation, including architecture, end-to-end workflows, mask format specifications, and API behavior notes, see:
\ndocs/usage.mdInstalling stabilo provides a stabilo console command with three subcommands: stabilo video (stabilize a video), stabilo tracks (stabilize object trajectories), and stabilo config (inspect or copy the default configuration). Run stabilo video --help or stabilo tracks --help for the full option reference. Each CLI invocation performs a best-effort check for a newer stabilo release on PyPI.
find_threshold_models.py: Computes regression models between detection thresholds and average keypoint counts for BRISK, KAZE, and AKAZE feature detectors.If you use Stabilo in your research, software, or product, please cite the following resources appropriately:
\nPreferred Citation: Please cite the associated article for any use of the Stabilo package, including research, applications, and derivative work:
\n@article{fonod2025advanced,\n title = {Advanced computer vision for extracting georeferenced vehicle trajectories from drone imagery},\n author = {Fonod, Robert and Cho, Haechan and Yeo, Hwasoo and Geroliminis, Nikolas},\n journal = {Transportation Research Part C: Emerging Technologies},\n volume = {178},\n pages = {105205},\n year = {2025},\n publisher = {Elsevier},\n doi = {10.1016/j.trc.2025.105205},\n url = {https://doi.org/10.1016/j.trc.2025.105205}\n}Repository Citation: If you reference, modify, or build upon the Stabilo software itself, please also cite the corresponding Zenodo release:
\n@software{fonod2026stabilo,\n author = {Fonod, Robert},\n license = {MIT},\n month = sep,\n title = {Stabilo: A Comprehensive Python Library for Video and Trajectory Stabilization with User-Defined Masks},\n url = {https://github.com/rfonod/stabilo},\n doi = {10.5281/zenodo.12117092},\n version = {1.4.2},\n year = {2026}\n}Contributions from the community are welcome! If you encounter any issues or have suggestions for improvements, please open a GitHub Issue or submit a pull request.
\nThis project is distributed under the MIT License. See the LICENSE file for more details.
", "creators": [ { "name": "Fonod, Robert", @@ -50,18 +50,18 @@ ], "dates": [ { - "start": "2026-08-25", - "end": "2026-08-25", + "start": "2026-09-17", + "end": "2026-09-17", "type": "updated", - "description": "Update from v1.4.0 to v1.4.1" + "description": "Update from v1.4.1 to v1.4.2" } ], "related_identifiers": [ { - "identifier": "https://github.com/rfonod/stabilo/tree/v1.4.1", + "identifier": "https://github.com/rfonod/stabilo/tree/v1.4.2", "relation": "isSupplementTo", "resource_type": "software" - }, + }, { "identifier": "arXiv:2411.02136", "relation": "isDescribedBy", diff --git a/CITATION.cff b/CITATION.cff index 082807e..0f6e61a 100644 --- a/CITATION.cff +++ b/CITATION.cff @@ -15,8 +15,8 @@ identifiers: repository-code: 'https://github.com/rfonod/stabilo' url: 'https://github.com/rfonod/stabilo' license: MIT -version: 1.4.1 -date-released: '2026-08-25' +version: 1.4.2 +date-released: '2026-09-17' preferred-citation: type: article title: "Advanced computer vision for extracting georeferenced vehicle trajectories from drone imagery" diff --git a/README.md b/README.md index e0a0532..3c3045a 100644 --- a/README.md +++ b/README.md @@ -109,7 +109,7 @@ Three matchers are available: `bf` (brute force) and `flann` work with every det - **Exclusion masks**: suppress keypoints on dynamic objects, in five geometries fed by seven input encodings (`--mask-enc yolo|pascal|coco|xywha|four|polygon|circle`), with an adjustable safety margin (`--mask-margin-ratio 0.15`). Disable with `--no-mask`. See [Masking behaviour](docs/usage.md#5-masking-behaviour). - **Classical detectors**: ORB, SIFT, RootSIFT, BRISK, KAZE, AKAZE (`--detector-name rsift`), with BF or FLANN matching (`--matcher-name flann`) and cross-check, Lowe's ratio, or distance filtering (`--filter-type ratio`). - **Learning-based detectors**: XFeat, DISK, DeDoDe, KeyNet, and the detector-free LoFTR via [kornia](https://kornia.readthedocs.io/) (`--detector-name disk`), optionally paired with the learned LightGlue matcher (`--matcher-name lightglue`). Pretrained weights download once into torch's cache and are reused offline. Mind the rotation and memory caveats above; pair them with a lower `--downsample-ratio`. -- **Robust estimation**: MAGSAC++ by default, plus GC-RANSAC, three LO-RANSAC (USAC) variants, PROSAC, RHO, LMEDS, RANSAC (`--ransac-method 36`), with tunable threshold, iterations, and confidence (`--ransac-epipolar-threshold 1.5`). See [RANSAC methods](docs/usage.md#11-transformation-types-and-ransac-methods). +- **Robust estimation**: MAGSAC++ by default, plus GC-RANSAC, three LO-RANSAC (USAC) variants, PROSAC, RHO, LMEDS, RANSAC (`--ransac-method 36`), with tunable threshold, iterations, and confidence (`--ransac-epipolar-threshold 1.5`). The threshold is measured in full-resolution pixels by default, or in downsampled-image pixels with `--ransac-threshold-space processed`. See [RANSAC methods](docs/usage.md#11-transformation-types-and-ransac-methods). - **Pre-processing**: CLAHE contrast enhancement (`--clahe`) and detection-time downsampling for speed (`--downsample-ratio 0.25`), with keypoints rescaled back to full resolution. - **Acceleration**: OpenCV CUDA for the classical pipeline (`--gpu`, see [`docs/cuda.md`](docs/cuda.md)) and CUDA/MPS/CPU for the learned models (`--device mps`). - **Visualization**: live or recorded side-by-side rendering of matches, inliers, masks, and box trajectories (`--viz`, `--save-viz`, `--no-lines`, `--tail-length 60`). See [Visualisation mode](docs/usage.md#12-visualisation-mode). @@ -272,11 +272,11 @@ If you use **Stabilo** in your research, software, or product, please cite the f @software{fonod2026stabilo, author = {Fonod, Robert}, license = {MIT}, - month = aug, + month = sep, title = {Stabilo: A Comprehensive Python Library for Video and Trajectory Stabilization with User-Defined Masks}, url = {https://github.com/rfonod/stabilo}, doi = {10.5281/zenodo.12117092}, - version = {1.4.1}, + version = {1.4.2}, year = {2026} } ``` diff --git a/docs/usage.md b/docs/usage.md index 93f24c6..7abcd02 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -31,6 +31,7 @@ - [11. Transformation Types and RANSAC Methods](#11-transformation-types-and-ransac-methods) - [Transformation types](#transformation-types) - [RANSAC methods (integer codes)](#ransac-methods-integer-codes) + - [The epipolar threshold and the downsample ratio](#the-epipolar-threshold-and-the-downsample-ratio) - [Method and transformation compatibility](#method-and-transformation-compatibility) - [12. Visualisation Mode](#12-visualisation-mode) - [13. Benchmarking Mode](#13-benchmarking-mode) @@ -284,6 +285,8 @@ ref_point = ref_point_h[:2] / ref_point_h[2] # divide by homogeneous coordinat All parameters can be passed as keyword arguments to `Stabilizer(...)` or set via a YAML file (see `stabilo/cfg/default.yaml`). Parameters not supplied fall back to their defaults. +`stabilo/cfg/default.yaml` is the complete list. A keyword argument whose name is not one of its keys is ignored, so the parameter it was meant to set keeps its default; stabilo logs a warning naming the unrecognized arguments and, where there is one, the nearest valid name. `Stabilizer.configurable_keys()` returns the valid names, for a caller that wants to check or filter a configuration itself. + | Parameter | Default | Valid values | Description | |-----------|---------|--------------|-------------| | `detector_name` | `'orb'` | `orb`, `sift`, `rsift`, `brisk`, `kaze`, `akaze`, `xfeat`, `disk`, `dedode`, `keynet`, `loftr` | Feature detector (see section 9) | @@ -301,7 +304,8 @@ All parameters can be passed as keyword arguments to `Stabilizer(...)` or set vi | `mask_use` | `true` | `true`, `false` | Enable exclusion masking | | `mask_margin_ratio` | `0.15` | `[0, 1]` | Fractional margin added to exclusion regions | | `ransac_method` | `38` | see section 11 | RANSAC algorithm (`affine` supports only `4` and `8`) | -| `ransac_epipolar_threshold` | `2.0` | `> 0` | Reprojection-error threshold (pixels) | +| `ransac_epipolar_threshold` | `2.0` | `> 0` | Reprojection-error threshold, in pixels of `ransac_threshold_space` | +| `ransac_threshold_space` | `'full'` | `full`, `processed` | Pixels the threshold is measured in: full-resolution, or the downsampled image (see section 11) | | `ransac_max_iter` | `5000` | `> 0` (int) | Maximum RANSAC iterations | | `ransac_confidence` | `0.999999` | `(0, 1]` | Required confidence level | | `brisk_threshold` | `130` | `(0, 255]` | BRISK detector threshold (fallback) | @@ -430,6 +434,19 @@ Use `projective` (default) when the camera undergoes any motion (pan, tilt, zoom Codes 32, 33, and 35 are three LO-RANSAC configurations that differ in how local optimization is scheduled, per OpenCV's [USAC tutorial](https://docs.opencv.org/4.x/de/d3f/tutorial_usac.html). None of them is DEGENSAC: that is a degeneracy check used for fundamental-matrix estimation, which OpenCV does not apply to homographies. +### The epipolar threshold and the downsample ratio + +Keypoints detected on a downsampled frame are rescaled to full resolution before the estimator sees them, so `ransac_epipolar_threshold` is a **full-resolution** pixel distance by default (`ransac_threshold_space: 'full'`). The keypoint localization error it has to separate from real outliers is not: that arises in the processed image and is magnified by `1 / downsample_ratio` when the points are rescaled. A fixed threshold is therefore a stricter filter at a low ratio than at a high one, by exactly the ratio between them, and a value tuned at one `downsample_ratio` does not transfer to another. + +Set `ransac_threshold_space: 'processed'` to express the threshold in processed-image pixels instead, which keeps it fixed in the units the noise is generated in; stabilo then passes `ransac_epipolar_threshold / downsample_ratio` to the estimator. + +```python +Stabilizer(downsample_ratio=0.25, ransac_epipolar_threshold=2.0) # 2.0 px at full resolution +Stabilizer(downsample_ratio=0.25, ransac_epipolar_threshold=2.0, ransac_threshold_space='processed') # 8.0 px at full resolution +``` + +Neither setting is right in general: the residuals a threshold must separate are part localization noise (which scales with `1 / downsample_ratio`) and part model error (which does not). Which to use, and at what value, is a tuning question; `get_ransac_reproj_threshold()` reports the value that is actually handed to the estimator. + ### Method and transformation compatibility `affine` maps to `cv2.estimateAffinePartial2D`, which implements only LMEDS and RANSAC; every other code raises inside OpenCV. Stabilo rejects the combination when the `Stabilizer` is constructed, rather than failing on every frame: diff --git a/stabilo/__init__.py b/stabilo/__init__.py index bbab7d1..491ca47 100644 --- a/stabilo/__init__.py +++ b/stabilo/__init__.py @@ -1,4 +1,4 @@ from .stabilo import Stabilizer -__version__ = '1.4.1' +__version__ = '1.4.2' __all__ = ['Stabilizer', '__version__'] diff --git a/stabilo/cfg/default.yaml b/stabilo/cfg/default.yaml index f8176b7..53fa991 100644 --- a/stabilo/cfg/default.yaml +++ b/stabilo/cfg/default.yaml @@ -59,7 +59,16 @@ match_query_frame: 'reference' # Which frame's descriptors are the knnMatch que # Transformation estimation # ----------------------------------------------------------------------------- # The estimated matrix maps the current frame onto the reference frame, so the -# RANSAC reprojection threshold is expressed in reference-frame pixels. +# RANSAC reprojection threshold is evaluated in reference-frame pixels. +# +# Which pixels depends on 'ransac_threshold_space'. Keypoints are rescaled to full +# resolution before the estimator sees them, so under 'full' the threshold is a +# full-resolution distance at any 'downsample_ratio', while the localization error +# it separates arises in the processed image and is magnified by 1/downsample_ratio. +# A fixed threshold is therefore stricter at a low ratio, and a value tuned at one +# ratio does not transfer to another. 'processed' measures it in processed-image +# pixels instead, passing ransac_epipolar_threshold / downsample_ratio to the +# estimator. The two agree at downsample_ratio: 1.0. See docs/usage.md section 11. # # Not every ransac_method works with every transformation_type. 'projective' # (cv2.findHomography) accepts all of them, whereas 'affine' @@ -67,7 +76,8 @@ match_query_frame: 'reference' # Which frame's descriptors are the knnMatch que # code, including the default 38, is rejected when the Stabilizer is created. transformation_type: 'projective' # ['projective' (3x3 homography, any ransac_method), 'affine' (2x3, ransac_method 4 or 8 only)] ransac_method: 38 # [4: LMEDS, 8: RANSAC, 16: RHO, 32: USAC-Default (LO-RANSAC), 33: USAC-Parallel (LO-RANSAC), 35: USAC-Fast (LO-RANSAC), 36: USAC-Accurate (GC-RANSAC), 37: PROSAC, 38: MAGSAC++] -ransac_epipolar_threshold: 2.0 # Reprojection-error threshold in pixels; (0.0, inf) +ransac_epipolar_threshold: 2.0 # Reprojection-error threshold, in pixels of ransac_threshold_space; (0.0, inf) +ransac_threshold_space: 'full' # Pixels ransac_epipolar_threshold is measured in; ['full' (full resolution), 'processed' (downsampled image)] ransac_max_iter: 5000 # Max. RANSAC iterations; (0, inf) ransac_confidence: 0.999999 # Required RANSAC confidence level; (0.0, 1.0] diff --git a/stabilo/cli/utils.py b/stabilo/cli/utils.py index 54429e0..bfee80c 100644 --- a/stabilo/cli/utils.py +++ b/stabilo/cli/utils.py @@ -102,7 +102,9 @@ def add_stabilo_config_arguments(parser): group = parser.add_argument_group( "stabilo: transformation estimation", "'projective' works with every --ransac-method; 'affine' (cv2.estimateAffinePartial2D) accepts only " - f"{affine_methods} and is rejected with any other method.", + f"{affine_methods} and is rejected with any other method. The epipolar threshold is measured in " + "full-resolution pixels unless --ransac-threshold-space says otherwise, so a value tuned at one " + "--downsample-ratio does not transfer to another.", ) group.add_argument( "--transformation-type", @@ -124,6 +126,14 @@ def add_stabilo_config_arguments(parser): group.add_argument( "--ransac-epipolar-threshold", "-ret", type=float, help="RANSAC epipolar threshold [default: 2.0]" ) + group.add_argument( + "--ransac-threshold-space", + "-rts", + type=str, + choices=Stabilizer.VALID_RANSAC_THRESHOLD_SPACES, + help="pixels the epipolar threshold is measured in: 'full' (full resolution) or 'processed' " + "(downsampled image, i.e. divided by the downsample ratio before estimation) [default: full]", + ) group.add_argument("--ransac-max-iter", "-rmi", type=int, help="RANSAC maximum iterations [default: 5000]") group.add_argument("--ransac-confidence", "-rc", type=float, help="RANSAC confidence [default: 0.999999]") @@ -173,9 +183,16 @@ def separate_cli_arguments(cli_args): Get the command-line arguments and the corresponding keyword arguments for Stabilizer. Precedence (lowest to highest): stabilo/cfg/default.yaml < --custom-config file < explicit CLI flags. + + The namespace also holds the subcommand's own options (the input path, --save, the + visualization flags, argparse's own `func`), which are not Stabilizer parameters. They are + filtered out here against `Stabilizer.configurable_keys()`; forwarded, they would be + reported as unrecognized arguments on every invocation. A `--custom-config` file is not + filtered, since a typo in a user's configuration is what that report exists to surface. """ args = argparse.Namespace(**vars(cli_args)) - explicit_kwargs = drop_none_values(vars(cli_args)) + configurable = Stabilizer.configurable_keys() + explicit_kwargs = {k: v for k, v in drop_none_values(vars(cli_args)).items() if k in configurable} kwargs = {} if args.custom_config: diff --git a/stabilo/stabilo.py b/stabilo/stabilo.py index ac5fe33..6470b72 100644 --- a/stabilo/stabilo.py +++ b/stabilo/stabilo.py @@ -37,6 +37,11 @@ Use a classical detector or KeyNet for footage with large in-plane rotation. - The learning-based detectors are memory hungry at high resolution. Reduce downsample_ratio for large inputs; LoFTR in particular scales quadratically with the pixel count. + - ransac_epipolar_threshold does not transfer across values of downsample_ratio. Keypoints are + rescaled to full resolution before estimation, so the threshold is a full-resolution distance + by default, while the localization error it separates arises in the processed image and is + magnified by 1 / downsample_ratio. Re-tune the threshold when the ratio changes, or set + ransac_threshold_space='processed' to express it in processed-image pixels instead. Usage: 1. Create an instance of the 'Stabilizer' class with desired parameter configurations. @@ -88,6 +93,7 @@ class Stabilizer: VALID_FILTER_TYPES = ['none', 'ratio', 'distance'] VALID_TRANSFORMATION_TYPES = ['projective', 'affine'] VALID_MATCH_QUERY_FRAMES = ['reference', 'current'] + VALID_RANSAC_THRESHOLD_SPACES = ['full', 'processed'] VALID_RANSAC_METHODS_DICT = { 'cv2.LMEDS': cv2.LMEDS, # 4 'cv2.RANSAC': cv2.RANSAC, # 8 @@ -112,6 +118,19 @@ class Stabilizer: } VALID_AFFINE_RANSAC_METHODS = {cv2.LMEDS, cv2.RANSAC} + @classmethod + def configurable_keys(cls) -> frozenset: + """ + Every key that can be passed to Stabilizer(...) or set in a configuration file. + + These are exactly the keys of cfg/default.yaml, which is the single source of truth for + the options. `logger` is not among them: it is a constructor-only argument, popped + before the configuration is loaded. Exposed so a caller that forwards a user's + configuration can filter or check it against stabilo rather than keeping its own copy + of the list, which drifts as soon as stabilo gains an option. + """ + return frozenset(cfg) + @classmethod def describe_ransac_methods(cls, methods=None) -> list: """ @@ -140,7 +159,11 @@ def __init__(self, **kwargs): - match_query_frame: str - which descriptors are the knnMatch query: 'reference' (default) or 'current' - ransac_method: int - method for RANSAC algorithm (see above for options); with transformation_type='affine' only cv2.LMEDS (4) and cv2.RANSAC (8) are supported - - ransac_epipolar_threshold: float - threshold for RANSAC (e.g., 1.0) + - ransac_epipolar_threshold: float - reprojection-error threshold for RANSAC (e.g., 1.0), in the + pixel units named by ransac_threshold_space + - ransac_threshold_space: str - coordinate system ransac_epipolar_threshold is expressed in: + 'full' (full-resolution pixels, the default) or 'processed' (downsampled-image pixels, which + keeps the threshold fixed relative to the keypoint localization noise as downsample_ratio varies) - ransac_max_iter: int - max iterations for RANSAC (e.g., 2000) - ransac_confidence: float - confidence for RANSAC (e.g., 0.999) - brisk_threshold: int - threshold for BRISK detector (used only if 'max_features -> threshold' model is unavailable) @@ -184,10 +207,32 @@ def __init__(self, **kwargs): def _load_configuration(self, kwargs): """ Load configuration parameters, using defaults if not provided. + + Only the keys of cfg/default.yaml are honoured, so a keyword argument that is not one of + them has no effect and a misspelled parameter runs with the default. Unknown keys are + warned about, with the nearest valid name, rather than raising: callers that forward a + user's configuration block verbatim may carry keys stabilo does not know, and an + exception here would turn a typo in a downstream config into a crash inside this + constructor. A caller that wants strictness can check `Stabilizer.configurable_keys()` + before constructing. """ for key, value in cfg.items(): setattr(self, key, kwargs.get(key, value)) + unknown = [key for key in kwargs if key not in cfg] + if unknown: + import difflib # noqa: PLC0415 - only needed on the error path + + described = [] + for key in sorted(unknown): + close = difflib.get_close_matches(key, cfg, n=1, cutoff=0.7) + described.append(f"'{key}'" + (f" (did you mean '{close[0]}'?)" if close else '')) + self.logger.warning( + f"Ignoring {len(described)} unknown Stabilizer argument(s): {', '.join(described)}. " + "Only the keys of stabilo/cfg/default.yaml are configurable; the defaults are used " + "for anything else. Run 'stabilo config show' to list them." + ) + def _initialize_variables(self): """ Initialize internal variables for the Stabilizer. @@ -773,7 +818,7 @@ def calculate_transformation_matrix(self) -> None: maxIters=self.ransac_max_iter, method=self.ransac_method, confidence=self.ransac_confidence, - ransacReprojThreshold=self.ransac_epipolar_threshold, + ransacReprojThreshold=self.get_ransac_reproj_threshold(), ) except cv2.error as e: self.logger.exception(f"Transformation matrix couldn't be calculated.\n Error: {e}") @@ -1072,6 +1117,20 @@ def get_cur_num_keypoints(self) -> tuple: len(self.cur_kpts) if self.cur_kpts is not None else None, ) + def get_ransac_reproj_threshold(self) -> float: + """ + Get the reprojection threshold actually handed to the estimator, in full-resolution pixels. + + Correspondences are rescaled to full resolution before estimation, so the estimator's + threshold is always a full-resolution distance. With ransac_threshold_space='full' that is + ransac_epipolar_threshold itself. With 'processed' the configured value is a + processed-image distance, and rescaling multiplies it by 1 / downsample_ratio, exactly as it + multiplies the residuals the threshold has to separate. The two agree at downsample_ratio 1.0. + """ + if self.ransac_threshold_space == 'processed': + return self.ransac_epipolar_threshold / self.downsample_ratio + return self.ransac_epipolar_threshold + def get_basic_info(self) -> dict: """ Get basic information about the Stabilizer. @@ -1186,6 +1245,11 @@ def _validate_arguments(self): raise ValueError("Invalid ransac_max_iter. It should be greater than 0 and an integer") if not (0.0 < self.ransac_epipolar_threshold): raise ValueError("Invalid ransac_epipolar_threshold. It should be greater than 0") + if self.ransac_threshold_space not in self.VALID_RANSAC_THRESHOLD_SPACES: + raise ValueError( + f"Invalid ransac_threshold_space: {self.ransac_threshold_space}. " + f"Choose from {self.VALID_RANSAC_THRESHOLD_SPACES}" + ) if not (0.0 < self.ransac_confidence <= 1.0): raise ValueError("Invalid ransac_confidence. It should be in the range (0.0, 1.0]") if self.gpu: diff --git a/tests/test_cli.py b/tests/test_cli.py index 55ef7c5..b12fbaf 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py @@ -10,6 +10,8 @@ from stabilo import Stabilizer, __version__ from stabilo.cli import main +from stabilo.cli import tracks as tracks_cli +from stabilo.cli import video as video_cli from stabilo.cli.utils import WINDOWS, separate_cli_arguments from stabilo.cli.video import render_stabilization_visuals @@ -86,6 +88,60 @@ def test_separate_cli_arguments(): assert 'downsample_ratio' not in kwargs +def test_subcommand_options_are_not_forwarded_to_the_stabilizer(): + """ + The namespace also holds the subcommand's own options, which are not Stabilizer parameters. + + The Stabilizer reports arguments it does not recognize, so forwarding them would make the + CLI warn about its own flags on every invocation. They are filtered against + `Stabilizer.configurable_keys()` instead. + """ + ns = argparse.Namespace( + input='x.mp4', + func=lambda: None, + save=True, + debug=False, + speed=10, + no_lines=False, + no_boxes=False, + ref_frame=0, + mask_enc='yolo', + no_mask=False, + viz=False, + save_viz=False, + custom_config=None, + detector_name='sift', + ) + _, kwargs = separate_cli_arguments(ns) + assert set(kwargs) <= Stabilizer.configurable_keys() + assert kwargs['detector_name'] == 'sift' + + +@pytest.mark.parametrize('subcommand', ['video', 'tracks']) +def test_a_real_parse_forwards_only_configurable_keys(subcommand, tmp_path): + """The same guarantee through the actual parsers, so a new flag cannot reintroduce the leak.""" + parser = argparse.ArgumentParser() + subparsers = parser.add_subparsers(dest='command', required=True) + (video_cli if subcommand == 'video' else tracks_cli).configure_parser(subparsers) + + clip = tmp_path / 'clip.mp4' + clip.touch() + args = parser.parse_args([subcommand, str(clip), '--save', '-dn', 'sift']) + _, kwargs = separate_cli_arguments(args) + assert set(kwargs) <= Stabilizer.configurable_keys() + assert kwargs['detector_name'] == 'sift' + + +def test_a_custom_config_typo_is_forwarded_so_the_stabilizer_can_report_it(tmp_path): + """A user's configuration is not filtered: an unknown key there is what the warning is for.""" + config_path = tmp_path / 'custom.yaml' + config_path.write_text("ransac_epipolar_treshold: 1.0\n") + + ns = argparse.Namespace(input='x.mp4', no_mask=False, viz=False, save_viz=False, custom_config=config_path) + _, kwargs = separate_cli_arguments(ns) + assert kwargs['ransac_epipolar_treshold'] == 1.0 + + def test_explicit_cli_flags_override_custom_config(tmp_path): config_path = tmp_path / 'custom.yaml' config_path.write_text("detector_name: 'sift'\nmax_features: 1000\nclahe: true\n") diff --git a/tests/test_stabilizer.py b/tests/test_stabilizer.py index e91b101..db22ad5 100644 --- a/tests/test_stabilizer.py +++ b/tests/test_stabilizer.py @@ -2,9 +2,13 @@ # -*- coding: utf-8 -*- # Author: Robert Fonod (robert.fonod@ieee.org) +import logging +from pathlib import Path + import cv2 import numpy as np import pytest +import yaml from stabilo import Stabilizer @@ -204,6 +208,118 @@ def test_invalid_ransac_confidence(): Stabilizer(ransac_confidence=1.5) +class TestUnknownArgumentsAreReported: + """ + A keyword argument that is not a key of cfg/default.yaml has no effect, so + `ransac_epipolar_treshold=1.0` leaves the threshold at its default; the Stabilizer reports + it rather than ignoring it in silence. + + It warns rather than raising, and that is the load-bearing part. geo-trax forwards a user's + configuration block to `Stabilizer(**kwargs)` verbatim and has always been permitted to + carry keys stabilo does not know; raising would turn a typo in a downstream configuration + into a crash inside this constructor. Numerical behavior is unchanged either way: the + unknown key is ignored exactly as before, and every known key keeps its value. + """ + + @staticmethod + def _capturing_logger(name): + messages = [] + + class Collector(logging.Handler): + def emit(self, record): + messages.append(record.getMessage()) + + logger = logging.getLogger(name) + logger.handlers = [Collector()] + logger.propagate = False + logger.setLevel(logging.DEBUG) + return logger, messages + + def test_an_unknown_argument_warns_and_does_not_raise(self): + logger, messages = self._capturing_logger('unknown-argument') + stab = Stabilizer(logger=logger, nonexistent_option=123) + assert any('nonexistent_option' in message for message in messages) + assert not hasattr(stab, 'nonexistent_option') + + def test_a_typo_suggests_the_key_that_was_meant(self): + logger, messages = self._capturing_logger('typo-argument') + Stabilizer(logger=logger, ransac_epipolar_treshold=1.0) + warning = ' '.join(messages) + assert 'ransac_epipolar_treshold' in warning + assert "did you mean 'ransac_epipolar_threshold'" in warning + + def test_the_typo_is_still_ignored_exactly_as_before(self): + logger, _ = self._capturing_logger('typo-ignored') + stab = Stabilizer(logger=logger, ransac_epipolar_treshold=1.0) + assert stab.ransac_epipolar_threshold == 2.0 # the default, not the misspelled request + + def test_valid_arguments_warn_about_nothing(self): + logger, messages = self._capturing_logger('valid-arguments') + Stabilizer(logger=logger, detector_name='sift', downsample_ratio=1.0, max_features=500) + assert not any('unknown' in message.lower() for message in messages) + + def test_logger_is_not_reported_as_unknown(self): + logger, messages = self._capturing_logger('logger-argument') + Stabilizer(logger=logger) + assert not any('logger' in message for message in messages) + + +def test_configurable_keys_are_exactly_the_configuration_file(): + keys = yaml.safe_load(Path('stabilo/cfg/default.yaml').read_text()) + assert Stabilizer.configurable_keys() == frozenset(keys) + assert 'logger' not in Stabilizer.configurable_keys() + + +def test_invalid_ransac_threshold_space(): + with pytest.raises(ValueError, match='ransac_threshold_space'): + Stabilizer(ransac_threshold_space='downsampled') + + +@pytest.mark.parametrize('threshold_space', Stabilizer.VALID_RANSAC_THRESHOLD_SPACES) +def test_threshold_spaces_agree_at_full_resolution(threshold_space): + """At downsample_ratio 1.0 there is no rescaling, so the two conventions coincide.""" + stab = Stabilizer(downsample_ratio=1.0, ransac_epipolar_threshold=2.0, ransac_threshold_space=threshold_space) + assert stab.get_ransac_reproj_threshold() == pytest.approx(2.0) + + +@pytest.mark.parametrize('downsample_ratio', [0.18, 0.25, 0.5, 0.75]) +def test_threshold_spaces_differ_by_exactly_the_downsample_ratio(downsample_ratio): + """'processed' converts to the estimator's full-resolution units; 'full' (the default) does not.""" + kwargs = {'downsample_ratio': downsample_ratio, 'ransac_epipolar_threshold': 2.0} + + default = Stabilizer(**kwargs) + full = Stabilizer(**kwargs, ransac_threshold_space='full') + processed = Stabilizer(**kwargs, ransac_threshold_space='processed') + + assert default.ransac_threshold_space == 'full' + assert default.get_ransac_reproj_threshold() == full.get_ransac_reproj_threshold() == pytest.approx(2.0) + assert processed.get_ransac_reproj_threshold() == pytest.approx(2.0 / downsample_ratio) + + +@pytest.mark.parametrize( + ('threshold_space', 'expected'), + [('full', 2.0), ('processed', 8.0)], +) +def test_the_estimator_receives_the_converted_threshold(images, threshold_space, expected): + """The conversion must reach cv2.findHomography, not merely the getter.""" + cur_frame, ref_frame = images + stab = Stabilizer(downsample_ratio=0.25, ransac_epipolar_threshold=2.0, ransac_threshold_space=threshold_space) + + seen = [] + transformer = stab.transformer + + def recording_transformer(*args, **kwargs): + seen.append(kwargs['ransacReprojThreshold']) + return transformer(*args, **kwargs) + + stab.transformer = recording_transformer + + stab.set_ref_frame(ref_frame) + stab.stabilize(cur_frame) + + assert seen == [pytest.approx(expected)] + + def test_set_ref_frame(default_stabilizer, images): _, ref_frame = images default_stabilizer.set_ref_frame(ref_frame)