AudioGen vs MMAudio
AudioGen
github.comMMAudio
github.comSide by side
| Figure | AudioGen | MMAudio |
|---|---|---|
| Price from | Free | Free |
| Free plan | Yes | Yes |
| Free trial | — | — |
| Open source | — | — |
| Linux | Yes | Yes |
| API | Yes | — |
| Self-hosted | Yes | Yes |
| Plans published | 1 | 1 |
| Spec score · AI Sound Effect Generators | 8.8 (#2) | 8.3 (#8) |
| Generation modes | The training documentation describes conditional and unconditional generation, audio continuation from a prompt, and greedy, temperature, top-K, and top-P sampling. | The command-line demo supports video-to-audio and text-to-audio generation. |
| Installation | AudioCraft installation requires Python 3.9 and PyTorch 2.1.0; ffmpeg is also recommended by the repository instructions. | The repository documents installation with Python 3.9 or later and PyTorch 2.5.1 or later, and says it has only been tested on Ubuntu. |
| Purpose | AudioGen is a text-to-sound generation model provided through AudioCraft. | MMAudio generates synchronized audio from video and/or text inputs. |
| Support | The model card directs questions and comments to the project’s GitHub repository or its issue tracker. | The project invites users to open a repository issue if they notice a failure mode or believe there is a bug. |
Plans
In their own words
AudioGen
AudioGen is a text-to-sound model provided through AudioCraft. It generates audio samples from text descriptions and can continue from an audio prompt, with conditional and unconditional generation modes. Sampling controls include greedy, temperature, top-K, and top-P methods. The listed pretrained model, facebook/audiogen-medium, has 1.5 billion parameters; the example API generates sound and saves it as WAV. The implementation uses an autoregressive Transformer and a 16 kHz EnCodec tokenizer. The repository provides API access and a local Jupyter notebook demo. Running the medium model locally requires a GPU with at least 16 GB of memory; installation requires Python 3.9 and PyTorch 2.1.0, with ffmpeg recommended. The model was trained on English descriptions and performs less well in other languages. It cannot generate realistic vocals, and prompts may need refinement. Code is MIT-licensed, while weights use CC-BY-NC 4.0; training datasets are not provided. The listed self-hosted plan is free.
MMAudio
MMAudio generates audio from video or text, with a synchronization module that aligns generated sound to video frames. Its command-line demo and Gradio interface support video-to-audio and text-to-audio generation. The interface also has an experimental image-to-audio mode: the model was not trained for image input, and the demo turns a still image into a video for processing. The project is free, with repository code under the MIT license and pretrained checkpoints under CC-BY-NC 4.0. It runs on Linux or in a self-hosted setup. Installation calls for Python 3.9 or later and PyTorch 2.5.1 or later, with corresponding torchvision and torchaudio; the project says it has only been tested on Ubuntu. In the default large_44k_v2 configuration, inference in 16-bit mode uses around 6 GB of GPU memory. The demo saves audio as FLAC and video as MP4, with an eight-second default duration. Output may include unintelligible speech-like sounds or background music, and unfamiliar concepts can be difficult for the model.