Hallo vs LipDub

Hallo generates an animated portrait video from a still image and driving speech audio. Its hierarchical audio-driven synthesis is designed to align lip, facial-expression, and pose movement with the audio, while separate inference weights let you adjust those three motion areas. The source image should be square, show a forward-facing face occupying 50%–70% of the frame, and have less than 30 degrees of rotation. Driving audio must be a clear-vocal English WAV file; background music is allowed. Inference saves an MP4 to `.cache/output.mp4` by default, with another output path available. Listed requirements are Ubuntu 20.04 or 22.04 and CUDA 12.1, with an A100 named as the tested GPU. Pretrained checkpoints and supporting models are available through Hugging Face. The project lists Linux, Windows, and self-hosted platforms, with community resources for Windows, ComfyUI, a WebUI, Docker, and RunPod. Its repository includes preprocessing and training code for multi-node and multi-GPU work. Hallo is free, open source under MIT, and its training data is English, so driving audio is limited to English.

LipDub localizes videos by dubbing them into other languages while keeping the person on camera and synchronizing their lips. The workflow covers uploading video, translating and refining a transcript, choosing or cloning a voice, lip syncing, and exporting localized versions. It supports 80+ languages out of the box; user-provided audio can extend use to other languages, dialects, accents, or fictional languages. Listed tools include lip sync, audio dubbing, a translation editor, stock voices, and instant voice cloning. The free Seed plan allows one minute of video at 1080p and adds a watermark. Core costs $29 a month and supports videos up to 10 minutes; Growth allows up to 60 minutes, and Scale up to 120 minutes with 4K output and API access. Enterprise offers unlimited length at custom pricing. The pricing FAQ says the platform works best with simple productions and broadcast-grade footage where the speaker is clearly visible. Paid plans include unlimited seats and concurrent jobs, while security and support options vary by plan.