This directory contains sample data for demonstrating and testing the HBB2OBB toolkit functionality. The data includes images, horizontal bounding box (HBB) annotations, ground truth oriented bounding box (OBB) annotations, and automatically converted OBB annotations.
Everything here except the four inputs listed below is produced by the tools in this repository, and the Reproducing This Folder section gives the exact command for each artifact, in order. This file covers what is in this folder and how it was made; the documentation site is the reference for the commands themselves, their flags and the data formats.
The sample images provided in the images folder are sourced from the Songdo Vision dataset and are re-used here under the Creative Commons Attribution 4.0 International (CC-BY-4.0) license.
These three frames are part of the Songdo Vision test split.
00164.jpg,04433.jpgand05050.jpgare byte-identical to the images of the same name in thetestsubset of Songdo Vision (verified by MD5). They are fine as a demonstration and regression fixture, which is all they are here for, but any accuracy number measured on them is measured on test data. Do not tune or benchmark on this folder and then report the result against Songdo Vision or Songdo Vision OBB.
- DOI: 10.5281/zenodo.13828407 (concept DOI, resolves to the latest version)
- Title: Songdo Vision: Vehicle Annotations from High-Altitude BeV Drone Imagery in a Smart City
- Authors: Robert Fonod, Haechan Cho, Hwasoo Yeo, Nikolas Geroliminis
- Publisher: Zenodo
- License: CC-BY-4.0
Only the images were taken from there. The horizontal boxes in this folder come from the Geo-trax detector, as step 1 below shows.
A conversion of your own needs two things: your images, and horizontal boxes for them. Step 2 below is the whole of it, and it passes no label map, no ground truth and no configuration file. The other inputs listed here exist so this folder can also demonstrate evaluation, format conversion and hyperparameter tuning.
Four things cannot be regenerated by anything in this repository:
| Input | What it is | What needs it |
|---|---|---|
images/*.jpg |
3 drone frames from Songdo Vision | every step |
classes.yaml |
the label map, class id to name | naming classes: step 3 and the printed summaries |
labels_obb_gt/*.txt |
ground truth oriented boxes, YOLO TXT, absolute px, drawn by hand | scoring a conversion (step 5) and tuning against it (step 6) |
benchmark.yaml |
the hyperparameter sweeps behind benchmark_results/ |
step 6 |
A label map is only ever needed to put a name on a class id, so Pascal VOC, COCO and LabelMe need none on the way in: they carry the names themselves. -mp is optional in the commands below for a second reason too: a classes.yaml beside the labels or one directory up is picked up on its own.
Everything else is generated from those, in one of four ways: hbb2obb-detect finds the horizontal boxes, hbb2obb converts them into OBBs, hbb2obb-convert derives the other annotation formats from the canonical YOLO TXT files, and hbb2obb-optimize produces the benchmark folders. The ground truth is the only annotation here that a person drew.
data/
├── images/ # INPUT sample drone imagery
├── classes.yaml # INPUT label map, for the formats that store class names
├── benchmark.yaml # INPUT the sweeps behind benchmark_results/ (step 6 only)
│
├── labels_hbb/ # Horizontal boxes: the input the conversion reads
│ ├── <frame>.txt # output YOLO HBB + confidence, absolute px (canonical)
│ └── <frame>.xml # derived Pascal VOC
├── coco_annotations_hbb.json # derived COCO, the same horizontal boxes
│
├── labels_obb_gt/ # Ground truth oriented boxes (steps 5 and 6 only)
│ ├── <frame>.txt # INPUT YOLO OBB, absolute px (canonical)
│ └── <frame>.dota # derived DOTA
├── coco_annotations_obb_gt.json # derived COCO, the same oriented boxes
│
├── labels_obb/ # OBBs converted from the HBBs by hbb2obb: what the tool is for
│ ├── <frame>.txt # output YOLO OBB + confidence (canonical)
│ ├── <frame>.dota # derived DOTA
│ └── <frame>.jpg # output --save_img visualization
├── coco_annotations_obb.json # derived COCO, the same converted boxes
├── labels_polygon/<frame>.txt # output --save_polygon segmentation contours
│
└── benchmark_results/ # output one sweep per SAM combination, plus what reads them together
├── summary.md # every run in one table, and the winner
├── summary.tex # the same runs as a LaTeX table
├── comparison.png # best IoU against execution time, one point per run
├── comparison.pdf # the comparison drawn for a manuscript
├── PROVENANCE.txt # source, checkpoint and input hashes, versions, the command
├── benchmark.yaml # a copy of the config, so the folder re-runs on its own
└── <name>/ # one sweep
├── run_config.yaml # the resolved settings and the host it ran on
├── results.yaml # the full grid of results
├── summary.txt # human-readable summary
└── plot.png # rendered from results.yaml
Each set ships in three formats. The YOLO TXT file is canonical in each case and the other two are one common rounding of it, so every derived format below is written from the .txt (-f yolo), never from another derived file.
- labels_hbb (YOLO TXT, Pascal VOC, COCO): the horizontal boxes the conversion reads. Here a detector found them, as step 1 shows, and its confidence sits in a 6th column. That column is optional, and so is the detector: boxes you drew by hand are read the same way, with or without it.
- labels_obb_gt (YOLO TXT, DOTA, COCO): oriented boxes drawn by hand, the ground truth for steps 5 and 6.
- labels_obb (YOLO TXT, DOTA, COCO): the OBBs
hbb2obbproduced fromlabels_hbb, with a per-box confidence in an optional 10th column. This set was written with--confidence_source combined, so each score is the conversion quality times the detector confidence carried over fromlabels_hbb(see Confidence scores).
Every .txt here is in absolute pixels, which is what the commands below produce and what DOTA and COCO are derived from. hbb2obb, hbb2obb-detect and hbb2obb-convert all take --normalize to write the relative form Ultralytics trains on instead; the readers detect either, so a normalized set converts and displays the same way.
Unlike the 50-frame tuning set released with Songdo Vision OBB, the HBBs here are not the axis-aligned envelopes of the ground-truth OBBs, and the two sets are not row-aligned: one is detected, the other drawn by hand. They do not describe quite the same objects, which is what step 5 measures.
Run everything from the repository root. Weights are resolved relative to the working directory, so models/ is read from and written to there.
hbb2obb-detect data/images -mp data/classes.yaml # add --overwrite to replace an existing setWrote 201 boxes over 3 frames to data/labels_hbb
0 Car: 186
1 Bus: 6
2 Truck: 8
3 Motorcycle: 1
This writes labels_hbb/<frame>.txt, the boxes and the detector's confidence in the 6th column. It uses the default detector, Geo-trax, whose weights are downloaded to models/ on first use, at the resolution and over the four vehicle classes it was validated on. Nothing here is hand-drawn or hand-corrected.
Against the hand-drawn ground truth of step 5, the detector misses one motorcycle in 00164 (a scooter with its rider) and finds one car the annotator left out, cut off by the right edge of the frame and half hidden by a tree. Step 5 therefore reports one unmatched box on each side.
If you have hand-drawn boxes of your own,
--merge_withkeeps their geometry untouched and only attaches the confidence of the detection covering each one. See Scoring hand-drawn boxes.
hbb2obb data/images --save_img --save_confidence --save_polygon --confidence_source combinedWrote 201 boxes over 3 frames to data/labels_obb
Fallback to HBB: 0 boxes (0.00%)
This writes labels_obb/<frame>.txt (with the confidence column), the --save_img overlays beside them, and labels_polygon/<frame>.txt. It uses the defaults otherwise: a single sam_b model, --imgsz 1280, --scale_factors 0.05, --opening_kernel_percentage 0.15, --fragment_ratio 0.1. Better results are available from a model ensemble and tuned hyperparameters; see Tune hyperparameters.
--confidence_source combined multiplies the conversion quality by the detector confidence from step 1. Drop the flag for the conversion score alone, or pass detector for the input confidence alone; all three work here, since step 1 wrote that 6th column. Confidence scores explains what the score means and how to act on it.
Expect the geometry to reproduce to within about a pixel rather than exactly. SAM inference is not bit-reproducible across
torchbuilds and hardware; re-running the command above moves a box or two by 1 px.
Each canonical YOLO TXT set is converted to DOTA or Pascal VOC beside itself, and to one COCO file at the top level:
# the converted OBBs
hbb2obb-convert data/labels_obb -f yolo --to dota -o data/labels_obb -mp data/classes.yaml --images data/images
hbb2obb-convert data/labels_obb -f yolo --to coco -o data -mp data/classes.yaml --images data/images
# the ground truth OBBs
hbb2obb-convert data/labels_obb_gt -f yolo --to dota -o data/labels_obb_gt -mp data/classes.yaml --images data/images
hbb2obb-convert data/labels_obb_gt -f yolo --to coco -o data -mp data/classes.yaml --images data/images \
--coco_name coco_annotations_obb_gt.json
# the horizontal boxes
hbb2obb-convert data/labels_hbb -f yolo --to voc -o data/labels_hbb -mp data/classes.yaml --images data/images
hbb2obb-convert data/labels_hbb -f yolo --to coco -o data -mp data/classes.yaml --images data/images--coco_name is needed only for the ground truth: a COCO file is named coco_annotations_<kind>.json by default, which already matches labels_obb and labels_hbb, but the ground truth directory is labels_obb_gt. That pairing is what --verify follows.
Confidence survives into COCO as a score field, so this renders exactly the same picture as the YOLO files do:
hbb2obb-view data/images --obb_dir data/coco_annotations_obb.json --show_confidenceThe same is true of the horizontal boxes: coco_annotations_hbb.json carries the detector confidence from step 1 in its score fields. DOTA and Pascal VOC have no slot for a confidence and drop it, so labels_hbb/*.xml holds the geometry alone.
hbb2obb-convert data --verify -mp data/classes.yamllabels_hbb: 3 frames, 201 boxes, 3 formats (coco, voc, yolo)
labels_obb: 3 frames, 201 boxes, 3 formats (coco, dota, yolo)
labels_obb_gt: 3 frames, 201 boxes, 3 formats (coco, dota, yolo)
OK: 603 boxes, every format encodes the same boxes
This compares by exact equality after rounding, not by a tolerance, so it fails on a single misplaced pixel.
hbb2obb-eval data/labels_obb_gt data/labels_obb=== Overall Results ===
Evaluation Mode: Class-Specific (matches only boxes with same class label)
Total Ground Truth Boxes: 201
Total Predicted/Converted Boxes: 201
Total Matched Boxes: 200
Total Unmatched GT Boxes: 1
Total Unmatched Pred Boxes: 1
Average IoU: 0.89642 ± 0.06834 (SEM 0.00483)
Median IoU: 0.91080
Matched Boxes Above Threshold: IoU>=0.50: 100.0% IoU>=0.75: 98.0% IoU>=0.85: 80.5% IoU>=0.90: 57.0%
Orientation Error: p90 4.40°, median 1.04°, mean 2.11° ± 4.88°
=== Results by Class ===
Class GT Pred Matches IoU (mean ± std) IoU (median) IoU>=0.9 Angle p90 (°) Angle p50 (°)
0 185 186 185 0.8982 ± 0.0606 0.9106 57.3% 4.36 1.00
1 6 6 6 0.9358 ± 0.0353 0.9390 66.7% 1.66 1.33
2 8 8 8 0.8533 ± 0.1469 0.8884 50.0% 13.87 1.56
3 2 1 1 0.6746 ± 0.0000 0.6746 0.0% 9.95 9.95
The one unmatched box on each side is the pair from step 1: the motorcycle the detector missed has no converted box, and the edge-cut car it found has no ground truth box. Everything the detector did find converted, and matched. The evaluator ignores the trailing confidence column, so it reads the converted files as they are written in step 2. For more options, such as excluding specific classes, class-agnostic matching or a different IoU threshold, run hbb2obb-eval --help.
To see where those numbers come from, cut the same pairs by the ground truth's orientation, size, frame edge and class:
hbb2obb-analyze data/labels_obb_gt data/labels_obb -hd data/labels_hbb -i data/images -mp data/classes.yamlAdd -o DIR to write the report, a YAML and a three-panel figure as well as printing it. With 201 boxes over 3 frames most of its groups are too small to read anything into; the analysis guide describes what it reports.
The three scores answer different questions and only the first saturates. Class 3, the single matched motorcycle, is where they disagree most: its IoU of 0.67 is poor but readable, while its 9.95° of orientation error against 1.00° for cars says plainly that a short, nearly square box is the case this conversion handles worst.
Every sweep in benchmark_results/ is described by benchmark.yaml, so one command reproduces all of it:
hbb2obb-optimize -c data/benchmark.yaml --dry_run # what it will do, and what it will cost
hbb2obb-optimize -c data/benchmark.yaml # the whole thing
hbb2obb-optimize -c data/benchmark.yaml --refresh # only redraw the plots, summary.md and summary.texEighteen model combinations, each a full grid of 3 inference sizes x 12 scale factors x 1 opening kernel, so 36 SAM passes per combination and 648 in total. That took 2.92 hours on a laptop CPU; --resume skips the combinations already finished. To sweep something else once, without touching the config:
hbb2obb-optimize data/images data/labels_obb_gt -sm sam_b -iz 960 1280 -sf 0.03 0.05 -n quickEach grid point is converted into a temporary directory and scored there, so the sweep never touches labels_obb/ from step 2.
hbb2obb-view data/images # the converted OBBs over their source HBBs
hbb2obb-view data/images --show_confidence # tinted green to red by score, value printed
hbb2obb-view data/images --compare data/labels_obb_gt # ground truth overlaid in blue
hbb2obb-view data/images --obb_format dota # read the DOTA files instead of the YOLO onesq quits, the wheel zooms, dragging pans, o and h show or hide the oriented and horizontal boxes, and g overlays the segmentation polygons from step 2. See the viewer guide for the full key list.
The benchmark_results directory holds the sweeps described in step 6, one subfolder per set of SAM models. They are shipped for illustration: to show what the optimizer produces and what its artifacts look like. They are measured against the boxes this folder ships, so step 6 reproduces them.
They were measured before the orientation error, the IoU ladder and the fragment ratio were recorded, so their results.yaml files hold none of them. The readers cope with that rather than crashing: --refresh leaves those columns empty (-- in summary.tex), and --plot_metric on one of the newer metrics reports that no grid point recorded it. A fresh run of step 6 fills them in, and writes the plot_median_angle_error.png, plot_iou_at_90.png and matching comparison_*.png / summary_*.md that this folder, measured before they existed, does not ship. --resume re-runs them for the same reason: a grid point that records no fragment ratio was measured with the largest mask piece alone, which is not what the config now asks for.
benchmark_results/summary.md is the generated write-up: every run's best grid point in one table, sorted by IoU, with the winner named and comparison.png showing accuracy against compute. benchmark_results/summary.tex and comparison.pdf are the table and the figure for a manuscript; the figure has only the mean IoU panel here, since these runs predate the IoU ladder. benchmark_results/PROVENANCE.txt records the checkpoint hashes, the hashes of the label sets the numbers were measured against, a digest of the hbb2obb source that ran, the library versions and the command. All of them are written by the command that produces the runs, and benchmark.yaml is copied in beside them.
comparison.png, shipped with this folder: each point is one model set at its best grid point, and the dashed line is the Pareto front (the runs no other run beats on both accuracy and time).
The nine ensembles are arranged as three ladders, each growing from two models to four by adding the next weaker one: read across a ladder for the effect of ensemble size, down the ladders for the effect of member quality. Take the results as a hypothesis to test on your own data, not as a finding:
| Ladder | 2 models | 3 models | 4 models |
|---|---|---|---|
Strong (sam_l, sam_b, then the SAM2 base pair) |
0.9005 | 0.9066 | 0.9079 |
Mixed (sam_l, then tiny models) |
0.9008 | 0.8723 | 0.8890 |
| Light (SAM2 base pair, then the tiny ones) | 0.8879 | 0.8842 | 0.8860 |
More models help only when the models you add are themselves strong. The strong ladder rises at every step; neither other ladder does, and the mixed one collapses at three before partly recovering at four. Majority voting sets the shape: a pair needs both members to agree, so the large model holds a veto; three members need only two, so two tiny models outvote sam_l and drag the result down to roughly what they produce alone (0.8723 against their own 0.8665 and 0.8689); four members need three again and recover part of the loss.
The cheapest good configuration is a large model paired with a tiny one. sam_l sam2.1_t reaches 0.9008 in 34 s, edging out sam_l sam_b (0.9005) in three quarters of the time, and both beat sam_l alone (0.8989). 1280 px wins all eighteen runs, by a wide margin over 640 px. And within SAM2 the small checkpoints buy nothing over the tiny ones: sam2_s (0.8652) and sam2.1_s (0.8648) sit at or just below sam2_t (0.8665) and sam2.1_t (0.8689); only the base checkpoints pull ahead (0.8803 and 0.8829). The 2.1 release edges out its 2.0 counterpart at tiny and base, and ties at small.
The sam_b run is the configuration labels_obb/ was produced with, which is why its best IoU, 0.8964 ± 0.0683 at 1280 px and scale factor 0.05, is the number step 5 prints.
sam_b/plot.png, the per-run plot every sweep folder ships: average IoU against scale factor, coloured by image size, with marker area encoding execution time. The peak at 1280 px and scale factor 0.05 is the grid point named just above. A sweep over several opening kernels shades each hue by kernel, so the kernels stay apart in the one plot; this sweep uses a single kernel, so the colours are the plain palette.
These are illustrative numbers, not the benchmark. They come from 201 boxes over 3 frames, which is far too small to select hyperparameters on, and those frames are Songdo Vision test images (see Data Attribution). The real study runs against the 50-frame tuning set released with Songdo Vision OBB, which shares no image with Songdo Vision; its inputs and its output artifacts live there, alongside the configuration they were produced with.