feat: add minimax-h3 support - #1854
Merged
Merged
Conversation
Contributor
|
So fast as always |
Contributor
|
I was very confused because it was segfaulting for me when building diffusion model graph (specifically after calling |
danielhanchen
added a commit
to oobabooga/unsloth
that referenced
this pull request
Aug 8, 2026
ensure_sd_cpp_binary hands back whatever find_sd_cpp_binary locates and only probes that it runs, so an install upgraded from an older Studio kept serving its pre-H3 managed sd-cli. The H3 load's only gate is SdCppEngine.version(), which that binary passes, so the load reported ready and the failure surfaced on the first generation, after the whole bundle had downloaded. Gate the H3 path alone on the capability instead of a version string: upstream added --ref-video and the other H3-only options in the same commit that added MiniMax-H3 (leejet/stable-diffusion.cpp#1854, master-812-ea7f0c8), and the release prebuilts report 'version unknown, commit unknown' because they are built without a .git directory, so --help is the only usable signal. Image generation keeps accepting any user-supplied build. A stale copy under the installer-owned root is removed so the pinned prebuilt reinstalls; a build the user supplied is left in place and the load fails naming it, the same ownership split _usable_or_discard_managed makes. A --help that cannot be read means 'cannot tell', never 'no H3'.
danielhanchen
added a commit
to unslothai/unsloth
that referenced
this pull request
Aug 9, 2026
* Add MiniMax H3 video generation
* Improve MiniMax H3 memory routing
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address MiniMax H3 review feedback
* Keep H3's VAE off the CPU path, so low_vram stops aborting
low_vram maps to the `model` policy, and offload_flags emits --vae-on-cpu for it
unconditionally. On H3 that kills the process:
ggml/src/ggml-cpu/ops.cpp:6321: GGML_ASSERT(src0->type == GGML_TYPE_F16) failed
deterministically, SIGABRT, exit 134. Bisected on the flags: --vae-on-cpu with
--audio-vae aborts, the same command without --audio-vae renders in 95.87s, and an
fp16-converted audio VAE aborts too. So the trigger is the audio VAE, not the video
one, and the F32 type is imposed inside stable-diffusion.cpp rather than by the
file: ggml_conv_1d hardcodes an F16 im2col destination (ggml/src/ggml.c),
ggml_compute_forward_im2col_f16 then asserts the KERNEL is F16, and
audio_conv_weight_type (src/model/vae/ltx_audio_vae.hpp) maps only BF16 to F16 and
lets F32 through. It cannot be fixed by shipping a different checkpoint.
low_vram is the one mode a small-card user reaches for, so this drops the flag
rather than the mode. offload_flags takes vae_on_cpu, defaulting True so no other
family changes, and the H3 native path passes False. --offload-to-cpu and
--clip-on-cpu still apply, which is where the saving actually is: the denoiser
dominates, and with --offload-to-cpu the whole model peaks at 13.14 GiB.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Point the H3 GGUF pick at the unsloth mirror
Main added test_curated_gguf_repos_are_unsloth_mirrors, which requires
every curated video gguf_repo to live under unsloth/ so a one-click pick
cannot 404 when a community repack is renamed or taken down. H3 was the
one family still on a community repo.
This was meant to be part of the merge commit but was left in the
worktree, so CI on that commit still saw the old value.
unsloth/MiniMax-H3-GGUF is still private and has to be made public
before this merges, or the pick will 401. No CI check reads it.
* Pin H3's native cfg-scale under test
H3 is distilled and CFG-free: its empty unconditional prompt encodes to
zero tokens, and the transposed tensor that produces trips
GGML_ASSERT(!ggml_is_transposed(a)) in ggml.c. SIGABRT, exit 134.
Measured: cfg 1.0 renders, cfg 1.5 and cfg 4.0 both abort. sd.cpp
defaults cfg-scale to 7.0, so this is a crash a plausible refactor
reintroduces by forwarding guidance the way every other family does.
The native path already hardcodes 1.0 and the family sets
supports_cfg = False, but nothing held either in place. supports_cfg
only gates the diffusers path; the native path builds its own params.
Checked the test fails when cfg_scale is changed to forward guidance,
so it is not passing vacuously.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Pin diffusers by source archive instead of git, so macOS installs work
The macos-15-intel leg fails deterministically, not flakily:
process didn't exit successfully: `/usr/bin/git init` (exit status: 1)
--- stderr
xcode-select: note: No developer tools were found, requesting install.
That runner has no Xcode, so /usr/bin/git is the developer-tools shim and
exits 1 for everything. uv needs a working git to resolve a git+https
dependency, so the diffusers pin this branch added cannot install there
at all. Main is unaffected because it depends on plain diffusers.
GitHub serves the same commit as a source archive, which uv installs with
no git involved. Verified by stubbing git to fail exactly the way the
macOS shim does: the git+ form reproduces the CI error, the archive form
installs diffusers 0.40.0.dev0 from the same SHA with
MiniMaxH3Transformer3DModel present and exported.
Also drops a full clone of diffusers from every install.
* Allow the diffusers source build on the clean-machine legs
With the archive pin the macOS leg gets past `git init` and installs, but
then trips the nobuild guard:
built from source: diffusers -- these must resolve to wheels on a clean machine
There is no wheel to resolve to. MiniMax-H3 support is not in any
diffusers release, so this branch has to pin a commit, and neither a git
URL nor a source archive can produce a wheel from an index.
Added to the same allowlist that already carries triton-kernels for the
same reason. Checked against the bar the comment there sets: diffusers
builds with plain setuptools, declares no ext_modules, and its tree has
zero .c/.cpp/.pyx/.rs/.cu files and no shipped binaries, so the PEP 517
build is a pure-Python copy step and needs no toolchain. The separate
compiler-invocation check in both scripts is untouched and still fires if
one is ever needed.
Verified the allowlist logic still rejects a non-allowlisted source build
(a log building both diffusers and numpy reports only numpy).
Remove this entry once a diffusers release carries H3 and the requirement
goes back to a version specifier. Noted in both scripts.
* Baseline the hf-hub retry loop reopened by the 1.x upgrade
`pip scan-packages :: hf-stack` fails on this branch with 1 unbaselined
CRITICAL:
C2 polling/beaconing loop detected
Package: huggingface-hub
File: huggingface_hub/utils/_http.py
L461: while True:
This is on us, not upstream drift. Main pins huggingface-hub==0.36.2;
this branch needs >=1.23.0,<2.0 because diffusers at the pinned commit
requires it, so the resolved version moves 0.36.2 -> 1.27.0.
The baseline already carries this exact file and check at L462 and L298,
from earlier versions. It did not carry over because the key hashes the
matched code, not the line number, and the surrounding code changed
across the major version. That is the guard behaving correctly: changed
code in a baselined file reopens for review rather than staying
suppressed.
Reviewed it rather than just re-suppressing. L461 is the retry loop in
`http_backoff`: bounded by `nb_tries > max_retries`, the URL comes from
the caller, there is no hardcoded endpoint and nothing is exfiltrated.
Same benign construct as the entries it replaces.
Entry generated with the scanner's own _evidence_hash rather than
hand-written, and inserted beside its siblings so the diff stays 8 lines.
Verified: with the baseline the scan is exit 0 with 4 suppressed, without
it exit 1, so the guard still bites.
* Lift the macOS-arm huggingface-hub cap that this branch made unsatisfiable
`mac macos-15 / trace / file` fails on this branch and passes on main. uv
reports:
No solution found when resolving dependencies:
Because you require huggingface-hub>=0.34.0,<1.0 and
huggingface-hub>=1.23.0,<2.0, your requirements are unsatisfiable.
This branch moved base.txt, no-torch-runtime.txt, studio.txt and
constraints.txt to hub >=1.23.0,<2.0, because diffusers at the pinned
commit requires it, but left the flat <1.0 cap in
overrides-darwin-arm64.txt. That file is macOS-arm only, which is why
only the mac legs see it and Linux and Windows stayed green.
The failure then presents as something else entirely: uv gives up, the
installer falls back to pip, and the clean-machine trace fails on
"installer invoked toolchain: rustc" rather than on the resolution.
The cap's own comment explains it exists so the resolver can never pair
hub 1.x with a pinned transformers 4.57.6 / hub 0.36.2 stack. That is
still true below python 3.10 and still capped there. At 3.10 and above
this branch is on transformers 5.5.0 and hub 1.x, so the premise is gone,
and mlx-audio's own >=1.0 floor is satisfied by the 1.x window anyway.
Checked by collecting every hub specifier that applies per Python version
across all five files: py3.9 resolves to 0.36.2 as intended, py3.13 to
1.23.0/1.27.0. Before this change py3.13 resolved to nothing.
* Make H3's native download use the repo the family advertises
The curated-mirror test main added only inspects VideoFamily.gguf_repo.
H3's native path does not read that field: video_minimax_h3.py has its
own H3_GGUF_REPO constant, used for both the transformer and the Qwen3-VL
encoder. So pointing the family at the unsloth mirror in the previous
commit satisfied that test while the actual one-click download still came
from a community repack, which is the exact failure the test exists to
prevent.
Pointed the constant at the same mirror and added
test_the_h3_native_repo_matches_the_family_gguf_repo to pin the pair, so
the two cannot drift apart again. Verified it fails when the constant is
put back to leejet, so it is not passing vacuously.
The mirror now carries the Qwen3-VL encoder quants alongside the
denoisers, byte-identical in size to the community ones, so this repo
alone satisfies both of h3_native_hub_files' hub entries. The encoder is
part of MiniMaxAI/MiniMax-H3 itself (FL2VA/text_encoder) which we already
mirror publicly at unsloth/MiniMax-H3 under the same licence, so shipping
a quantization of it beside the denoisers is the same act.
Also checked the encoder-tier routing survives the dynamic rung names:
-UD-Q2_K_XL selects the Q2_K_M encoder and -UD-Q3_K_XL the Q4_K_M one,
asserted in the new test.
Updated the download-plan test, which hardcoded the old repo id.
NOTE: unsloth/MiniMax-H3-GGUF is private. Unlike before, that now really
does gate this: the native path downloads from it. It has to be public
before this merges.
* Pin H3's companion-checkpoint guard under test
validate_h3_transformer_filename had no test. That mattered less when the
denoisers lived alone; the mirror now ships the Qwen3-VL encoder quants in
the same repo, so the picker lists both and a user can name either. Loading
a 12-17 GB encoder as the transformer would fail deep inside sd-cli instead
of at the boundary.
The accept cases include the dynamic rung names on purpose. The guard is a
prefix/suffix check and `-UD-Q2_K_XL` is a shape it had never seen when it
was written; it happens to pass, and now that is asserted rather than
assumed.
Checked the test fails when the prefix check is dropped, so it is not
vacuous.
* Record why H3 drops --vae-on-cpu, now that the abort is fixed
The comment justified the drop entirely by an sd.cpp abort. That abort is fixed
in the Unsloth fork, which would have made this look like a stale workaround to
revert once the fix reaches the pinned prebuilt.
Measured on a build carrying the fix, 640x384, 25 frames, 4 steps, q4_K, with
--offload-to-cpu --clip-on-cpu already applied: adding --vae-on-cpu moved peak
VRAM 12.42 -> 12.42 GiB and wall time 20.9s -> 100.4s. Under --offload-to-cpu
the peak is set by the streamed denoiser, so the flag saves nothing and costs
4.8x. It stays off on its own merits.
* Pin the sd.cpp prebuilt that actually renders MiniMax-H3
The pin was master-812-ea7f0c8, a stock upstream build, and on a stock build H3
does not work: it aborts on the default --cfg-scale, aborts again on
--vae-on-cpu, and its 1-D norms are quantized into an output uncorrelated with
its own bf16 reference (LPIPS 0.981). The Studio side worked around the first by
pinning cfg to 1.0 and the second by dropping the flag; the third had no
workaround on the consumer side at all.
All three are fixed in unslothai/stable-diffusion.cpp and open upstream as
leejet/stable-diffusion.cpp#1861, #1862 and #1863. The mirror's prebuilt
pipeline now applies them on top of the aged upstream tag it already builds, and
marks such a build with a -u<id> suffix naming the patch set, so
master-813-bfbef5b-u0665242 is upstream master-813-bfbef5b plus those three
patches and nothing else. The patches are deleted once upstream releases them,
at which point this pin goes back to a plain tag.
Verified on the published Linux x86_64 asset, not on a local build: both new
error strings are in the shipped sd-cli, and running it on a q4_K H3 denoiser
without --mode vid_gen now exits 1 with the instruction instead of core dumping
on a ggml assert.
test_video_backend's fake engine returned the old tag as its version string,
which read like a second pin; it only needs a non-None value, so it now says so.
* Close two gaps the H3 mirror switch opened
Both are consequences of the two preceding commits, found in review.
The prebuilt pin is now mirror-only (master-813-bfbef5b-u0665242), and
_resolve_with_fallback still asked leejet for it. That request is a guaranteed
404 by construction, since the -u<id> suffix marks a build only the mirror
makes, so it was a wasted round trip on every install. Worse, when the mirror
genuinely cannot serve a host the fallback lands on leejet's latest, which has
none of the H3 fixes. For every other model that is the right trade, better a
stock native engine than none. For H3 it is not visible: it aborts on the
default cfg-scale, aborts on --vae-on-cpu, and a blanket --type renders a broken
video rather than failing. A user who saw only the generic 'falling back to
leejet' line had nothing connecting that to the output. It now says so.
Second, hub/utils/gguf.py filtered H3 companion GGUFs by the old community repo
id only. The mirror the family and catalog now advertise carries the Qwen3-VL
encoder quants beside the denoisers, so a 12 GB text encoder was being
aggregated as if it were a selectable transformer quant. Both bundle repos are
now recognised, case-insensitively, and the cache-dir match follows.
Tests are mutation-verified rather than assumed: restoring the upstream 404
attempt fails the ordering test, removing the H3 warning fails the fallback
test, and dropping the mirror from the bundle set fails three. The ordering test
deliberately makes the mirror serve nothing, because with the mirror serving the
first attempt succeeds and the upstream attempts are never reached, which made
an earlier version of it pass under its own mutation.
A third test pins the native loader's H3_GGUF_REPO to the bundle set, since
those live in different files and a future repo move that updated only the
loader would silently reintroduce the same aggregation bug.
* Exclude MiniMax-H3's small-M projections from int8
_INT8_FAMILY_EXCLUDE_NAME_TOKENS has entries for qwen-image and hunyuanvideo-1.5
but none for minimax-h3, and H3 needs one for the same reason they do.
H3's adaLN projection is named adaln_proj, which no token in the generic list
matches: 'norm' is the closest and does not appear in the name. On the dense
checkpoint that projection is Linear(2688 -> 96768), so it clears
min_features = 512, gets quantized, and then runs at M = 1. Inductor lowers int8
matmul to _int_mm, which requires M > 16, so it raises 'self.size(0) needs to be
greater than 16, but got 1' at the first denoise. The offline prequant builder
bakes it in happily, which is exactly the drift the exclude list exists to
prevent for Flux and Qwen.
The pruned-modulation form hides this rather than fixing it: there adaln_proj is
Linear(8 -> 96768) and falls under min_features anyway. So this exclusion is what
makes the DENSE path correct and is a no-op on the pruned one.
context_embedder and token_refiner are added for the same reason hunyuanvideo-1.5
excludes its text stream. Measured at M = 10 text tokens against the video
stream's thousands, they are 3.47% of GEMM time even in the slow eager int8 path,
so leaving them bf16 costs nothing measurable.
This is what made an earlier measurement conclude int8 does not work on H3. It
does: on the pruned form int8 compiles and is 4.24% +-0.54% faster than fp8 at
identical memory, paired over 12 renders.
* Say which H3 component could not be downloaded, and why
H3 pulls four files from two repos, and the Hub returns the same
'Repository Not Found ... make sure you are authenticated' for a repo that does
not exist, one that is private, and one your token does not cover. A user
reading that has no way to tell which of the four failed, and the wording points
away from the real cause whenever the repo exists but is not public.
That is the state the GGUF mirror is in today: it is unpublished, so picking H3
fails with a message suggesting the user fix their token, which will not help.
This replaces it with the repo, the component, and the actual remedy, and says
the other components are unaffected so the failure is not read as total.
Gated repos get different wording, since accepting a licence is a different
action from waiting for a repo to be published.
Anything that is not a recognised access error is passed back unchanged rather
than reworded, so a timeout or a full disk still reads as itself. The helper
returns the exception instead of raising, so the caller keeps raise-from and the
original traceback survives.
Mutation-verified three ways: rewording every error (timeouts included), fixing
the component name to 'denoiser', and giving gated repos the private wording
each fail the test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Stop MiniMax-H3 holding two copies of its video VAE
Two thirds of an H3 render's peak is not activations. Measured at
640x384 across 124 frames, a 20.25 GB int8 denoiser peaks at 36.96 GB,
and the gap is almost all weights: the video VAE alone is 10.42 GB
because diffusers pins it to float32, and a further 4.91 GB is
autocast's own float16 copy of those same weights. A memory snapshot
puts 92.9% of the transient in 437 blocks allocated from nn.Linear,
largest 67.1 MB, which is the decoder's [2048, 16384] SwiGLU projection
in float16.
MiniMaxH3VideoDecodeStep wraps vae.decode in torch.autocast(float16),
and autocast caches every weight it casts for the lifetime of the
region, so the float32 original and its float16 twin are both resident
through the whole decode. Storing those weights as float16 up front
makes the cast a no-op and removes both. This is not an approximation:
x.to(float16).to(float16) is x.to(float16), and the four regression
tests check that on a real matmul rather than on the reasoning. The
audio VAE decode is not under autocast, so it keeps float32.
t2va starts from noise and never encodes, so vae.encoder and
vae.quant_conv go too. That part is gated on the workflow name rather
than dropped unconditionally, because an image-conditioned workflow
needs them.
Measured 36.96 -> 28.37 GB peak with the encoder drop and the pre-cast,
28.27 with expandable_segments as well, over 5 prompts x 2 seeds. Speed
is unchanged (-0.05% +-0.86% eager, -0.34% +-2.23% compiled), and every
arm hashes identically to its control on latents, audio_latents, frames
and audio.
The estimator's base still reads 68.5 GB. That figure was measured on
the bfloat16 modular components, not the int8 arm above, so it stays put
until it is re-measured in the same configuration rather than adjusted
by arithmetic.
* Pad MiniMax-H3's small-M int8 linears instead of leaving them dense
torch._int_mm asserts self.size(0) > 16. torchao's eager path never trips
it (safe_int_mm falls back to a widened matmul), but inductor lowers the
same quantized linear straight to _int_mm, so any quantized Linear invoked
at a small activation row count crashes under torch.compile. Until now the
fix was to leave those linears dense bf16, which on H3 meant excluding
context_embedder and both token_refiner blocks: 13 linears, 798M
parameters, 0.80 GB of weights the int8 checkpoint was not allowed to
touch.
Pad instead. diffusion_quant_pad.PadToMinM pads the flattened row count up
to 32, runs the GEMM and slices the result back, so the module compiles
with no change to the quantization config and the caller's rows come back
bitwise unchanged. Verified bitwise on all 65 (module, M) cases across
H3's 13 linears at M = 10, 13, 14, 17, 19, on real torchao-quantized
weights; compiling those same modules unpadded raises the _int_mm assert
at M = 10, 13 and 14.
Two properties carry that exactness and both are asserted rather than
assumed. The pad rows replicate row 0, not zeros: an all-zero row has amax
0, so the activation quantizer divides by zero. And the activation scale
must be per row, so each kept row's scale comes from that row alone; a
quantized Linear whose granularity cannot be proven per row raises instead
of being quietly skipped, because a half-padded transformer compiles on
the wrapped modules and crashes on the rest.
Everything below pad_to normalises to pad_to rather than only what is
below the floor, so one inductor graph covers every prompt length in the
range. H3's seven eval prompts run at M = 10..19, which straddles the
floor, and padding only to 17 would leave three shapes behind.
The wrapper reparents the Linear, so it runs after quantize_ on the
runtime path and after load_state_dict on the prequant one. The offline
builder drives quantize_ directly and saves the state dict, so it never
sees a wrapper; PadToMinM is also state-dict transparent as a second line
of defence, saving and loading under its own prefix so a wrapped
transformer still writes context_embedder.weight.
Scoped to minimax-h3. qwen-image, qwen-image-edit and hunyuanvideo-1.5
have the same small-M shape but published int8 prequant checkpoints whose
metadata bakes the current exclusion set, and _validate_checkpoint
compares that set against exclude_tokens_for_scheme, so they move only
together with a rebuild. adaln_proj stays excluded for a different reason:
on the dense checkpoint it is Linear(2688 -> 96768) and runs at M = 1,
while on the pruned form it is Linear(8 -> 96768) and falls under
min_features anyway (verified: the filter rejects all 51 for min_features).
Measured on B200, 640x384 x 124 frames, 4 steps, 7 prompts x 2 seeds, the
two arms alternated within each cell so drift on a shared box cancels:
checkpoint 21.052 -> 20.254 GB (-0.798, -3.8%)
transformer 21.051 -> 20.253 GB (-0.798)
load peak 21.137 -> 20.336 GB (-0.801)
render peak 37.766 -> 36.966 GB (-0.800, -2.1%)
step time +0.0597 s +-0.0057 eager, +0.0533 s +-0.0086 compiled
The time is torchao's un-fused eager activation quantization on 13 modules
that run ONCE per render, so it is a fixed cost rather than a per-step
one, and it does not scale with steps or resolution. Compiling those
modules alongside the blocks removes it: +0.0034 s +-0.0040, no detectable
difference. Quality is unchanged as far as n = 14 can resolve: against the
same bf16 twin the padded arm sits +0.0070 +-0.0151 LPIPS from the
excluded one, which rules out a degradation larger than 0.022.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Classify the MiniMax-H3 GGUF bundle as video in the cached inventory
The H3 GGUFs are stable-diffusion.cpp conversions and carry no metadata
keys at all (kv_count 0), so general.architecture is absent where the
LTX-2 and Wan video GGUFs declare ltxv/wan. _arch_to_task therefore left
the downloaded repo with no task, which drops it from the Video picker's
On Device list and hands it to chat as a plain GGUF. Key the two bundle
repo ids before the arch is consulted.
* Release the VIDEO GPU claim when H3 native falls back to the CPU build
On a CUDA/ROCm host /video/load acquires the VIDEO arbiter owner because the
resolved device target is not CPU. _run_load_h3_native then asks for an
accelerator-matched sd-cli, and the pinned prebuilt release publishes no Linux
CUDA/ROCm archive, so ensure_sd_cpp_binary returns None and the load commits the
CPU build with native_device = cpu. Nothing dropped the VIDEO claim, so the next
chat or image acquire evicted and unloaded an H3 runtime that holds no VRAM.
Release the claim once the CPU fallback is committed, through release_if so the
token check is atomic against a newer load that already took ownership. Mirrors
the CPU-only native release /images/load already does.
* Video: protect the native H3 companion repos, cancel the modular denoise, forward the hub token
- VideoBackend.loaded_repo_ids() publishes the repos the committed native H3 runtime
re-reads every generation (Qwen encoder + both VAEs), and the delete-cached guard
consults it, so deleting a companion under a loaded model is refused.
- The H3 modular workflow no longer falls back to a null progress context: the denoise
loop drives pipe.scheduler.step once per step, so the existing wrapper gives it the
same per-step progress and cancellation the other callback-less pipeline gets.
- load_components() gets the Settings token, so gated/private component loads are not
issued anonymously.
* Reject stable-diffusion.cpp builds that predate MiniMax-H3 support
ensure_sd_cpp_binary hands back whatever find_sd_cpp_binary locates and only
probes that it runs, so an install upgraded from an older Studio kept serving
its pre-H3 managed sd-cli. The H3 load's only gate is SdCppEngine.version(),
which that binary passes, so the load reported ready and the failure surfaced
on the first generation, after the whole bundle had downloaded.
Gate the H3 path alone on the capability instead of a version string: upstream
added --ref-video and the other H3-only options in the same commit that added
MiniMax-H3 (leejet/stable-diffusion.cpp#1854, master-812-ea7f0c8), and the
release prebuilts report 'version unknown, commit unknown' because they are
built without a .git directory, so --help is the only usable signal. Image
generation keeps accepting any user-supplied build.
A stale copy under the installer-owned root is removed so the pinned prebuilt
reinstalls; a build the user supplied is left in place and the load fails
naming it, the same ownership split _usable_or_discard_managed makes. A --help
that cannot be read means 'cannot tell', never 'no H3'.
* Distinguish a reused CPU sd-cli from an accelerator build on an H3 load
On a Linux CUDA host the first H3 load installs the CPU prebuilt through the
fallback and correctly commits native_device = cpu, because the pinned tag
publishes no Linux CUDA, ROCm or Vulkan asset. Every later load then calls
ensure_sd_cpp_binary(accelerator = cuda), which finds that same CPU binary and
returns it without looking at what it was built for, so the fallback was
skipped, native_device stayed cuda, and Studio applied GPU offload policy and
retained the VIDEO gpu_arbiter claim while sd-cli ran wholly on the CPU. A
later chat or image acquire then evicted an unrelated GPU model.
Fall back on what the binary offers rather than on whether one was returned:
sd-cli --list-devices prints one name/description line per available ggml
backend device, so a CPU-only build answers with CPU alone. The second load now
reaches the same cpu conclusion as the first, which is what lets the existing
release_if drop the stale claim. Unreadable output, or an older build that
rejects the flag, keeps the GPU: neither says the accelerator is missing.
* Guard the H3 companion repos while a native video load is downloading
_run_load_h3_native downloads from repo_id, the H3 GGUF companion and the H3
component repo, but the in-flight state only carried repo_id and base_repo. The
cached-model delete guard reads loading_repo_ids(), so it allowed deleting
Comfy-Org/MiniMax-H3, and the GGUF companion when the load comes from another
mirror or a local file, while those files were still downloading, which fails
the load.
Carry the companions on the loading state, the way the image backend's
_SdLoading already does, and publish them from loading_repo_ids(). This is the
in-flight twin of loaded_repo_ids() and covers the same repos.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Keep curated Recommended rows searchable and give them their metadata
Two things went wrong with the curated video models the Recommended list
paints from the catalog rather than from a live Hub listing.
Search dropped them. The Recommended search matched the query against
`recommendedIds`, which filters out every id already on disk because a
downloaded model gets its own On Device row. The unfiltered Recommended
list does not filter that way: it renders the curated seeds and badges the
downloaded ones. So a curated model was visible in the list and unfindable
by typing its name the moment it was downloaded, and only a live listing
row could bring it back, which a repo the listing does not return never
gets. `searchableRecommendedIds` unions the seed ids with the listing ids,
seeds first, deduped case insensitively (the HF cache lowercases repo ids),
so both lists agree on what exists.
Rows rendered bare. Everything past the id came from the listing alone, so
a curated row the listing never returns showed no parameter chip and no
capability glyph while its neighbours showed both. The catalog now carries
the two facts nothing else can supply: `totalParams` per artifact and
`capabilities` per group, read through `curatedTotalParamsFor` and
`curatedCapabilitiesFor`. Both are fallbacks only. A listing row wins
wherever there is one, because real tags and a Hub-reported total outrank
anything hand written here.
The parameter counts are measured, not guessed. The MiniMax-H3 figure is
the sum of the tensor shapes in the BF16 GGUF of the FL2VA denoiser that
repo publishes; the LTX-2.3 figure is what the Hub reports for its repo,
carried so the row looks the same offline or rate limited.
One more inconsistency fell out of the same place: `searchRowFits` hides
anything it cannot size (`requireKnown`), and it could not size a curated
repo with no listing row and no "<n>B" token in its id, so turning on
"Fits on device" hid from search a row the unfiltered list still showed.
It now falls back to the curated total the same way it already fell back
to the curated size.
Covered by studio/frontend/tests/recommended-curated-row-metadata.test.ts:
15 assertions over the search pool, the two catalog lookups, the fit check,
and the four picker call sites that read them.
* Load MiniMax-H3 from a hosted pre-quantized denoiser
Video families had no way to reach a hosted pre-quantized DENOISER. VideoFamily
carried gguf_repo and te_prequant_repos (the text encoder) only, and nothing in
video.py consulted a denoiser table, so a MiniMax-H3 load either pulled the full
66.3 GB bfloat16 DiT or nothing at all. The image side already solves this with
DiffusionFamily.prequant_repos plus family_prequant_repo(), so this follows that
shape rather than inventing a second one.
Four parts.
VideoFamily gains prequant_repos, prequant_variant_repos and prequant_subfolder,
resolved by video_family_prequant_repo() / video_family_prequant_schemes(),
mirroring the image resolver. The registries stay separate, as the module header
requires, so the base-id normaliser is local rather than imported.
The shared resolver learns an optional subfolder. The hosted video checkpoints
nest theirs one level down instead of keeping it at the repo root, and the prefix
has to reach BOTH candidate names or the primary 404 is followed by a second one
and the load silently falls back to the dense download. Always a literal forward
slash: these are Hub repo paths, and a Windows join would miss the cache. The
cache and download plumbing already handled a nested name, so only the filename
builder changed, and every existing call site is byte-identical.
The modular workflow builds its denoiser through its own component loader, so
there is no dense module to quantise in place. A hosted checkpoint is therefore
the only way to run that transformer quantized, and pre-seeding it with
update_components() before load_components() is also what stops the dense
download: load_components(names=None) skips a component whose attribute is
already set. Passing names= instead would have forfeited the workflow's own block
pruning, which is what already avoids the 61.7 GB Ref2VA transformer.
Those checkpoints carry the pruned adaLN, where the modulation is a rank-8 affine
factorization of the time-embedding curve plus a shared table, and roughly 40% of
the released model's parameters go. Against the base repo's dense config the
model is four keys short, one over and fifty-one shapes wrong, so the strict load
fails and the checkpoint is unloadable by every route. video_minimax_h3_adaln.py
reshapes the model between from_config and load_state_dict: table lookup with
interpolation instead of the timestep MLP, no SiLU (the table already holds the
activation's own output projected onto the basis), and the modulation cast to the
block stack's dtype, without which the first quantized matmul dies on mismatched
dtypes. Bound per instance, so a dense load in the same process is untouched.
Finally the refusal becomes honest. A single-file pick on a modular family used
to reach the loader only after ~98.7 GB had downloaded AND after the resident
pipeline had been evicted to make room for it, because download-plan returned 200
and validation passed. Both refusals now run in validate_load_request, ahead of
the diffusers availability probe so they still fire where diffusers cannot be
imported, and each names what to pick instead. download-plan forwards
transformer_quant to validation and to the plan, without which the quant-keyed
refusal never fires on the route that stages the download and the plan stages
shards the load never opens.
* Point both MiniMax-H3 schemes at one hosted pre-quantized repo
The two hosted pre-quantized denoisers were split across two repos with the
checkpoint nested one level down, so reaching them needed a mechanism the image
side has never had: a VideoFamily.prequant_subfolder field, a
prequant_subfolder_prefix() normaliser, and a subfolder keyword threaded through
both resolve_prequant_source() and usable_prequant_source().
Both schemes now live in ONE hosted repo, at the root, named <Model>-<SCHEME>.pt.
That is the layout every image-side prequant repo already uses, and it is exactly
what prequant_repo_filename() builds unaided, so the whole mechanism goes. Match
an existing convention and the code should shrink, not grow: -22 lines in
diffusion_prequant.py, -6 in video_families.py, -6 in video.py, with no new
concept to carry.
Landing on the primary name also fixes a memory-planning under-credit.
cached_checkpoint_path() deliberately credits only the PRIMARY filename, so that
a cached legacy artifact cannot pin a stale name once a repo ships the real one.
While these checkpoints were published as transformer_<scheme>.pt the primary
never existed: every hit came through fallback_filename, and planning therefore
read an already-cached checkpoint as "this would have to download" and handed the
pick to GGUF. The primary is now the published name, so the probe hits it.
fallback_filename stays. It still covers repos that have not been renamed, and
dropping it is a separate decision from this one.
Tests: the subfolder-prefix cases are replaced by the naming they now guard --
both schemes resolving to one repo, the primary resolving to a root-level
<Model>-<SCHEME>.pt with no directory component on any platform, the cache probe
being asked for that primary name, and the repo's own scheme suffix being
stripped and replaced rather than carried through. Five mutations run, each
caught by the named test and reverted: two repos again (M1), the primary nested
under prequant/ again (M2), the suffix strip removed (M3), the cache probe keyed
on the fallback (M4), the fallback name dropped (M5).
* Add MiniMax-H3 image and reference video conditioning
* Improve H3 finalization progress
* Report real sd.cpp progress on the Video page instead of a frozen 0 of 30
A native (GGUF) video generation reported phase "denoise", step 0/30 for its
entire run and then flipped straight to "completed". Two separate things were
wrong, and the progress endpoint could not move until both were fixed.
The parser looked for r"(?:step|sampling)\D+(\d+)/(\d+)". sd-cli's sampling
bar contains neither word. It prints
|=========> | 7/30 - 21.50s/it
so nothing ever matched. Anchor the pattern on the bar and on the trailing
speed unit instead. A bare "n/m" is deliberately not enough: an unrelated
ratio in some other log line must not drive the progress bar.
The reader also delivered every redraw one step late. sd-cli redraws in
place, and its carriage return LEADS the record while the newline only
arrives on the final step, so a reader keyed on CR/LF cannot produce step 1
until step 2 has been flushed. Treat the erase-to-end-of-line that closes
each redraw as a terminator too, and read the pipe with buffer.read1 so a
record that carries no newline is not stuck behind a blocking readline.
Escapes are stripped before a record reaches on_log or the error tail.
Streams without a raw .buffer keep the old line iteration.
The same bar shape is printed by three different things, so the parser now
tells them apart rather than reporting whichever came last. Weight load
prints it with a byte rate, and tiled VAE decode prints an identical s/it bar
counting TILES: without a guard a run finished sampling at 30/30 and then
jumped backwards to "step 1/16". Load and decode are real work with no
sampling step, so they report their own phase and a null step rather than a
fake 0 of 30. ETA is measured from the first step, not from job start, so the
one-off weight load is not charged to every remaining step.
Verified end to end against a real CUDA sd-cli MiniMax-H3 generation: the
step advances 1..6 over 6.2s..9.7s of wall clock, load and decode are
reported as themselves, and the ETA tracks.
* Read sd-cli's in-place progress redraws so the Video bar moves during sampling
The native H3 progress bar had two independent causes and fixing either alone
changed nothing observable. The bar pattern is now correct, but the reader still
was not.
sd-cli redraws its sampling bar in place: one printf per step shaped
"\r<bar> <n>/<total> - <speed>\033[K", with a newline only on the final step of a
phase. The drain loop did `for raw in proc.stdout`, which terminates on LF only,
so every redraw sat in the buffer until the next one's carriage return arrived
and the last one until sampling was already over. The Video page saw nothing.
Split the raw pipe into records on CR, LF, or the trailing erase-to-end-of-line,
reading through buffer.read1 with an incremental UTF-8 decoder so a multi-byte
character straddling two reads survives, and strip the CSI escapes before the
record reaches on_log or the error tail. Streams without a raw .buffer (test
doubles, non-pipes) fall back to line iteration.
The new backend test drives the real byte stream through both halves, one flush
per read, and asserts each step is visible on the read that carried it rather
than one redraw later.
* Give MiniMax-H3 first and last frame conditioning in the video backend
MiniMax-H3's released transformer is the FL2VA one: text-to-video is the same
checkpoint run with no keyframes. Studio only ever ran it text-only, so the
Video generate request had nowhere to attach a reference frame.
The load used to prune the block graph to t2va. That argument prunes STATICALLY,
so an fl2va-pruned pipeline runs the keyframe blocks on every request and cannot
serve a text-only one at all: it raises packing an empty conditioning list.
The load now keeps the whole auto graph, which selects per request, and bounds
load_components to the keyframe workflow's component set instead, so the 61.7 GB
Ref2VA partition is no more loaded than before. Measured against the released
checkpoint: a text-only request through this pipeline is bit-identical, video
and audio, to the same request through a t2va-pruned one.
A keyframe is a geometry anchor, not just conditioning, so the canvas comes from
its aspect ratio through the released arithmetic (768 short edge, area capped at
768x1344, both axes rounded to 32) rather than from whatever resolution preset
was selected. An arbitrary size produces a garbled clip rather than an error.
sd.cpp already implements the same conditioning, so that side is the existing
--init-img / --end-img flags with the frames staged as PNGs.
Only MiniMax-H3 declares the capability, and status reports it, so Wan and LTX
do not grow a control that does nothing.
* Cover the MiniMax-H3 keyframe path with tests
Registry (which families declare it, and the canvas a keyframe resolves to),
the load wiring (whole block graph, component set still bounded, VAE encoder
kept), request handling (decode, refusal, canvas override, what reaches the
pipeline call) and the sd-cli argv.
* Add the reference-frame controls to the Video page
First and last frame pickers, shown only for a family whose status declares
keyframe conditioning, so Wan and LTX are unchanged. The Images page's source
picker moves to a shared component rather than the Video page growing a second
upload path; both send the same data URL to the same backend decoder.
While a frame is attached the Resolution preset is disabled and says why: the
frame's aspect ratio resolves the clip's size, the way the model itself does.
The gallery recipe records which ends were pinned.
* Check the keyframe canvas against the pipeline's own resolver
The canvas rule is a checkpoint contract, so pin it to the released
implementation rather than only to hand-written expectations. Skipped where
diffusers does not ship MiniMax-H3, which is most runners.
* Revert the standalone H3 keyframe implementation
oobabooga/unsloth#121 covers first frame, last frame, first-and-last, Ref2VA and
the canvas rule, and it reached the same load construction independently. Two
implementations of the same feature on one branch is worse than either, so this
takes mine back out and leaves the branch ready for that work to land whole.
The one finding worth keeping from it is already reflected there: passing
workflow= to ModularPipeline.from_pretrained prunes the block graph statically,
so the pipeline must be built unpruned and only load_components bounded.
* Keep the pre-quantized MiniMax-H3 denoiser resident so a generation can run
Loading H3 with a hosted pre-quantized denoiser worked, but every generation died
on its first denoise step:
Attempted to set the storage of a tensor on device "cuda:0" to a storage on
different device "cpu". This is no longer allowed; the devices must match.
ComponentsManager.enable_auto_cpu_offload parks every component on the CPU and
moves each one onto the accelerator inside its own pre_forward, that is from
within the block that is already executing. The text encoder and the VAEs survive
that; a torchao-quantized denoiser does not, because the device change reaches
return_and_correct_aliasing, which tries to alias a CPU storage to an accelerator
tensor. Moving the same module at load time, outside any executing block, works.
So place it once at load and take it out of the offload rotation: drop its hook,
unlist it from the other components' eviction candidates, and move it. Everything
else is unchanged, and the encoder and VAEs still offload around it. Keeping it
resident is what asking for a quantized denoiser buys in the first place: the
hosted checkpoint is about 20 GB against 66.3 GB dense.
Verified end to end on a B200: MiniMaxAI/MiniMax-H3 loaded with the hosted fp8
denoiser, then a 1280x768, 124-frame clip generated from a start frame in 167s.
The clip's first frame matches the supplied image and the motion is coherent.
* Apply the pinned Diffusers revision on a fresh install, not just an update
MiniMax-H3 needs a Diffusers revision newer than any published release, and
Studio refuses to load it otherwise. The pin was in
studio/backend/requirements/base.txt, and a clean install.sh run still ended up
on diffusers 0.39.0 from PyPI, every time, with nothing in the log to say so.
base.txt is never installed by install.sh. install.sh installs unsloth itself,
whose own metadata pulls a diffusers release in transitively, and then runs
install_python_stack.py with SKIP_STUDIO_BASE=1, where the base-packages step is
a bare `pass`. So the pin applied on `unsloth studio update` and on the no-torch
path (install.sh installs no-torch-runtime.txt directly) and was dead on exactly
the path a new user takes. Reproduced on a clean install into a throwaway prefix
before and after: 0.39.0, then 0.40.0.dev0 with MiniMaxH3Transformer3DModel
present.
The revision now lives in its own diffusers-pin.txt, installed by a step that
sits outside every skip_base / NO_TORCH branch and after every other
requirements file, so nothing left in the run can re-resolve diffusers back to a
release behind it. No forced reinstall is needed: a direct URL requirement is not
satisfied by a resident registry install, so the step is a no-op once the
environment is already on the pin.
tests/studio/install/test_diffusers_pin.py holds the shape in place: exactly one
requirements file may name diffusers, the pin must be a full commit sha rather
than a moving ref, the install step must sit at function top level rather than
under a conditional, and it must come after every other requirements install.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: let the Video and Images pickers see a local diffusers pipeline
A model already on disk never reached the pickers' On Device list unless it
happened to keep a weight file beside a root config.json. Every image and video
model downloaded as a pipeline keeps its weights in component subdirs under a
root model_index.json instead, so the hub inventory scan behind /api/hub/local
rejected it. Pointing Custom Folders at one was worse than empty: the LM Studio
publisher walk descended into the pipeline and offered vae, transformer,
text_encoder and audio_vae as four separate models, none of them loadable.
Teach the hub scanner the same pipeline-root test routes/models.py already
applies, in the three places that only accepted a root config plus loose
weights, and keep a pipeline row through the custom-folder format filter: the
layout has no loose weight to classify, so the row is "unknown" by construction
rather than by fault.
* Cover the local diffusers pipeline scan with tests
* Pin what the pipeline exemption must not let through
Three gaps in the cover added with the scan change, each found by mutating the fix and
watching the suite stay green.
The custom-folder format filter now waves a row through on its shape, and nothing said
what it still has to reject: replacing the whole predicate with True passed. A folder
holding a config.json and no weights, which an aborted download leaves behind, reports
the same "unknown" format and no loader can start it, so it pins the boundary.
The predicate is applied to every row the filter did not already accept, and a row's
path can be a GGUF file rather than a directory. A missing path, a file, and a
directory whose model_index.json is itself a directory must answer False rather than
raise, because an exception there fails the scan and empties the picker.
The publisher walk was only covered one level up. Adding the model folder itself as a
scan folder is the obvious thing to do and used to publish vae, transformer and
text_encoder as three models.
* Drop the unused H3_TASK_KEYFRAMES import from the video backend
video.py only branches on H3_TASK_REFERENCES; the keyframe constant is read
from video_minimax_h3 directly by the tests that need it. The hoisted-import
safety net in Source lint flags the unused name as a blocker.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Stage the hosted pre-quantized H3 denoiser in the video download plan
The plan already drops the dense transformer shards whenever a hosted
pre-quantized checkpoint covers them, but nothing put that checkpoint back:
an int8 or fp8 H3 stage skipped 66.3 GB of base shards and added none of the
20.25 GB artifact the load actually opens. The byte total under-reported the
stage by the size of the checkpoint, the disk preflight cleared a volume that
could not hold it, and an offline stage completed without the one file the
load needs.
_denoiser_prequant_hub_files mirrors the pre-cast encoder helper: it resolves
the family's hosted checkpoint, confirms the file really exists on the Hub,
and prefers the repo-root name over the legacy scheme name in the same order
the load tries them. An unreachable repo is logged and yields no files, so a
gated or renamed artifact keeps the dense shards instead of sinking the plan.
The progress-bar estimate is deliberately left alone: it counts cached bytes
for the checkpoint and base repos only, so adding a third repo there would
leave the bar permanently short of 100 percent.
* List both H3 denoiser partitions in the picker, not only FL2VA
The bundle-repo filter accepted only minimax_h3_fl2va*, so every published
minimax_h3_ref2va* quant was hidden from the variant listing. The loader
disagrees: validate_h3_transformer_filename accepts either partition, on the
grounds that which one is picked IS the task, and h3_transformer_task routes
Ref2VA to the reference-video workflow this PR adds. The community bundle repo
publishes three Ref2VA quants today, so the reference path was unreachable from
the remote catalog.
Accept both prefixes from one shared tuple and keep excluding the Qwen3-VL
encoder and VAE companions, which are never picks for either partition. The
filter test asserted the old behaviour and is updated with it.
* Bound H3 reference-audio decoding to the trained window
The reference-video decoder already selects, resizes and refuses incrementally
because the encoded size says nothing about the decoded size. The audio decoder
did not: it appended every resampled block to a list and then allocated a second
full buffer in np.concatenate, with no duration or sample cap.
The route accepts 32 MiB of encoded audio, which is over half an hour of
compressed stereo. That lands as roughly 1.9 GB of float32 and doubles again in
the concatenate, and up to three references are accepted per request, so an
ordinary long music or podcast file picked by mistake could exhaust the host
before the background job even started.
H3's reference window is 15 seconds, so anything past it is unusable rather than
merely large. Refuse it while decoding, with the same shape of message the video
guard uses, instead of decoding it first.
* Refuse a quantized H3 reference load instead of seeding the keyframe denoiser
The hosted pre-quantized checkpoints are FL2VA (keyframe) denoisers. Ref2VA
shares their module shapes and the same base model, so resolve_prequant_source
handed one back for a reference load, it passed every metadata check, and
seeding it made load_components skip the real Ref2VA transformer. The request
then generated from the wrong partition rather than failing, which is the worst
of the three outcomes.
The route accepts h3_task, so this was reachable from the public API even though
the picker does not expose the choice yet. validate_load_request now refuses the
pairing with a message naming the workable alternatives, in the same place it
already refuses a scheme with no hosted checkpoint, and the modular loader drops
to the released components if a direct call reaches it. Nothing changes for
keyframe loads, which are what the checkpoints are.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Skip modular-workflow families in the dense text-encoder plan assertion
The plan-unchanged sweep from main walks every family and asserts the dense
budget plan_diffusion_memory received. MiniMax-H3 is the first modular-workflow
family to reach that list, and load_pipeline dispatches to the workflow's own
loader before the planner runs: each component is built by its own
from_pretrained, so there is no single dense pipeline to budget and no plan call
to assert on. Skip it the way the sweep already skips wan2.2-t2v-a14b.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Scope the paired-axes canvas rule to keyframe requests
A half-specified width/height is long-standing, documented API behaviour: the backend
resolves the missing axis from the family's default preset, in both
validate_video_request_shape and _resolve_keyframes. Applying the new paired-axes rule
as an unconditional request validator rejected those calls with a 422 before family
validation ever ran, breaking existing LTX, Wan, Hunyuan and prompt-only H3 clients.
The rule still holds where it means something: with a keyframe present the canvas is
matched to the source aspect whenever either axis is missing, so the axis the caller
sent would be silently discarded.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Keep the CUDA sd-cli pin and translate a mirror-only tag upstream
The merge with main brings in the accelerator-aware installer, whose fallback
translates a mirror-only -u<id> pin back to the upstream release it was built
from instead of skipping the upstream attempt. That is strictly better: skipping
kept the round trip cheap but dropped the pin entirely on every host the mirror
does not build, leaving them on upstream latest.
test_a_mirror_only_pin_is_never_requested_upstream asserted the old shape, that
the fallback settled for upstream latest. It now pins the new one: never the
literal -u<id> string, the translated release instead, and no latest attempt at
all because the translated pin succeeds.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Related Issue / Discussion
N/A
Additional Information
T2AV example
t2av.mp4
Checklist