Skip to content
SPECIFICTION
Get listed

MMAudio

github.com

Free plan LinuxSelf-hosted
Visit github.com
Spec score 8.3 #8 of 29 in AI Sound Effect Generators

What it is

MMAudio generates audio from video or text, with a synchronization module that aligns generated sound to video frames. Its command-line demo and Gradio interface support video-to-audio and text-to-audio generation. The interface also has an experimental image-to-audio mode: the model was not trained for image input, and the demo turns a still image into a video for processing. The project is free, with repository code under the MIT license and pretrained checkpoints under CC-BY-NC 4.0. It runs on Linux or in a self-hosted setup. Installation calls for Python 3.9 or later and PyTorch 2.5.1 or later, with corresponding torchvision and torchaudio; the project says it has only been tested on Ubuntu. In the default large_44k_v2 configuration, inference in 16-bit mode uses around 6 GB of GPU memory. The demo saves audio as FLAC and video as MP4, with an eight-second default duration. Output may include unintelligible speech-like sounds or background music, and unfamiliar concepts can be difficult for the model.

Spec sheet

Purpose
MMAudio generates synchronized audio from video and/or text inputs.Read 10 Oct 2026 · github.com
Support
The project invites users to open a repository issue if they notice a failure mode or believe there is a bug.Read 4 Oct 2026 · github.com
Commercial use
The project says it does not guarantee pretrained models are suitable for commercial use.Read 4 Oct 2026 · github.com
Data handling
The maintainers say they cannot redistribute the training datasets for copyright reasons.Read 10 Oct 2026 · github.com
Demo availability
The repository links to Hugging Face, Colab, and Replicate demos.Read 10 Oct 2026 · github.com
Demos and integrations
The repository links to Hugging Face and Colab demos and a Replicate demo.Read 4 Oct 2026 · github.com
Generation modes
The command-line demo supports video-to-audio and text-to-audio generation.Read 4 Oct 2026 · github.com
Hardware
The README says inference uses around 6 GB of GPU memory in 16-bit mode in its experiments.Read 10 Oct 2026 · github.com
Image limitation
Image-to-audio is experimental and the model was not trained for it; the demo duplicates the image into a video for processing.Read 4 Oct 2026 · github.com
Input processing
The CLIP encoder resizes frames to 384×384 pixels, while Synchformer resizes the shorter edge to 224 pixels and center-crops each frame.Read 4 Oct 2026 · github.com
Installation
The repository documents installation with Python 3.9 or later and PyTorch 2.5.1 or later, and says it has only been tested on Ubuntu.Read 10 Oct 2026 · github.com
Interface
The project provides a Gradio interface for video-to-audio and text-to-audio, with experimental image-to-audio support.Read 4 Oct 2026 · github.com
Interfaces
The project provides a command-line demo and a Gradio interface.Read 10 Oct 2026 · github.com
Joint training
Its multimodal joint training uses audio-visual and audio-text datasets.Read 10 Oct 2026 · github.com
Known limitations
The model may produce unintelligible human speech-like sounds, background music, and poor results for unfamiliar concepts.Read 10 Oct 2026 · github.com
Licensing
The repository code is MIT licensed, while pretrained checkpoints are released under CC-BY-NC 4.0.Read 4 Oct 2026 · github.com
Model license
The code is MIT licensed, while the pretrained checkpoints are released under CC-BY-NC 4.0; the README does not guarantee pretrained models are suitable for commercial use.Read 10 Oct 2026 · github.com
Output and duration
The demo saves audio as FLAC and video as MP4, and its default output and training duration is 8 seconds.Read 4 Oct 2026 · github.com
Output duration
The default output and training duration is 8 seconds; durations far from training duration may reduce quality.Read 10 Oct 2026 · github.com
Runtime requirements
Installation requires Python 3.9+ and PyTorch 2.5.1+ with corresponding torchvision and torchaudio; the project says it has only been tested on Ubuntu.Read 4 Oct 2026 · github.com
Synchronization
A synchronization module aligns generated audio with video frames.Read 10 Oct 2026 · github.com
Synthesis modes
The included demos support video-to-audio and text-to-audio synthesis, with experimental image-to-audio processing through a duplicated still image.Read 10 Oct 2026 · github.com
Training hardware
The authors recommend, for smooth training, single-node setups with two 80 GB H100 GPUs for the small model or eight for the large model, plus hundreds of gigabytes of system memory.Read 10 Oct 2026 · github.com
Training limitation
The current training script does not support _v2 training.Read 10 Oct 2026 · github.com

Plans and prices

MMAudio Free Code released under MIT license · pretrained checkpoints under CC-BY-NC 4.0 Read 10 Oct 2026 · github.com

Every MMAudio plan and price →

Strong points and limits

Strong points

  • Generates audio from video or text.
  • Synchronization aligns sound with video frames.
  • Includes command-line and Gradio interfaces.
  • Free project with MIT-licensed repository code.

Limits

  • Image-to-audio is experimental and untrained.
  • Project says it has only been tested on Ubuntu.
  • Pretrained checkpoints use CC-BY-NC 4.0.
  • Output duration defaults to eight seconds.

Who it suits

MMAudio may suit developers or researchers who can install and run a Linux-based project and want video- or text-conditioned audio generation. Its checkpoint licensing and stated commercial-use uncertainty matter to commercial users.

Verdict

MMAudio offers video- and text-based generation with a specific synchronization function, but it requires a compatible software setup and GPU memory. Review the checkpoint license and commercial-use caveat before relying on pretrained models in a commercial project.

Where Specifiction places it

Alternatives to MMAudioAll alternatives →