Error while generating videos with MiniMax-H3

[Pervyboicomes]

Everytime I try to generate a video with MiniMax-H3 I get the error below:

ComfyUI execution error: Allocation on device 0 would exceed allowed memory. (out of memory)
Currently allocated : 25.97 GiB
Requested : 1.18 GiB
Device limit : 15.92 GiB
Free (according to CUDA): 0 bytes
PyTorch limit (set by user-supplied memory fraction)
: 17179869184.00 GiB
This error means you ran out of memory on your GPU.

TIPS: If the workflow worked before you might have accidentally set the batch_size to a large number.

(End of Error, to continue my question)

Also I sometimes generate some videos but they are 5 seconds max, do you guys know what I could be doing wrong or is this related to a problem with SwarmUI itself?

[mcmonkey4eva]

Make sure you don't have other processes using GPU memory while you're running heavy ai models. H3 is very demanding.

If it's not just an obvious something else taking it, you can change:
Server -> Backends -> edit the comfy backend -> in Extra Args -> add --reserve-vram 2 -> save -> try again

That tells the comfy backend to try to preserve at least 2 gigs of vram for other things. Set it higher if needed. You may benefit from having --disable-smart-memory --disable-dynamic-vram --reserve-vram 2 (tell comfy to absolutely minimize vram usage entirely)

Depending on sysram availability you might also need to widen your OS page file.

For reference: the text encoder needs ~30 gigs of memory, the H3 diffusion model needs ~20 gigs, actually running it is going to eat another 10+ gigs (actual amount varies with input and resolution and duration).

[Pervyboicomes]

That makes a lot of sense, I'm rocking an RTX 5080 Mobile with 16GB VRAM, so it probably can't run that model at all. Also a bit unrelated but I do get another error when I'm trying to generate with Wan 2.2 and using Image 2 Video feature:

ComfyUI execution error: Sizes of tensors must match except in dimension 2. Expected size 36 but got size 16 for tensor number 1 in the list.

(End of Error, to continue my question)

Do you think this is also an error that's caused by lack of VRAM?

[mcmonkey4eva]

If you have decent sysram it'll run, just a bit slow. The quality on H3 is so good that I'd say it's worth the effort of trying to figure out how to make it work.

That error message looks like something different, probably mixed up parameters (i2v is a bit confusing and easy to get wrong, see #716 for a guide on how to do i2v)

ps the Swarm discord is usually the best place for support https://discord.gg/q2y38cqjNw

[Pervyboicomes]

Unfortunately I can't use Discord since I don't support the way that the team behind that app functions (I use Stoat as a DC alternative), but I've managed to make both Wan 2.2 and MiniMax H3 Image to Video generation work with even external image as my input, here's what I've done:

• Reset my parameters and followed the instructions over at docs/Video Model Support.md on SwarmUI GitHub
• Realized I was putting the image that I want to turn into a video in the prompt box and not in the Init Image parameter group so I did that now.
• Realized I was choosing the video models as my main model, instead of choosing an image model (Like an SDXL model), because the correct approach in order to use Image to Video models is:
  *Choose an image model as main model
  *Choose the video models (for Wan 2.2 you need to put High model in Video Model, and Low model in Video Swap Model section under Image to Video parameter group) under the Image to Video parameter group
  To generate an image and to turn it into video (which obviously you need to use the model specific parameters in this part before generating)
  And to use an already existing image to turn into a video:
  *Choose an image model as main model
  *Choose the video model under the Image to Video parameter group
  *Upload or paste your image to Init Image parameter group
  *Set Init Image Creativity to 0
  *Use the Res button next to the image in the parameter group to prevent the image from being stretched or squished.

That was all the things I have done that made the Image to Video work for me with Wan 2.2 and MiniMax H3. Also one last question and I'm sorry for taking your time mcmonkey4eva, can I use my LoRA's for video models by just selecting them in the LoRAs panel or adding them to the prompt box, or do I have to take a different approach?

[mcmonkey4eva]

yeah just select em

and ftr with i2v it doesn't matter what base model is selected, you can even have the h3 model and that's arguably best

[Pervyboicomes]

Got it, Thanks for your help 👍