LipDub vs MakeItTalk

LipDub localizes videos by dubbing them into other languages while keeping the person on camera and synchronizing their lips. The workflow covers uploading video, translating and refining a transcript, choosing or cloning a voice, lip syncing, and exporting localized versions. It supports 80+ languages out of the box; user-provided audio can extend use to other languages, dialects, accents, or fictional languages. Listed tools include lip sync, audio dubbing, a translation editor, stock voices, and instant voice cloning. The free Seed plan allows one minute of video at 1080p and adds a watermark. Core costs $29 a month and supports videos up to 10 minutes; Growth allows up to 60 minutes, and Scale up to 120 minutes with 4K output and API access. Enterprise offers unlimited length at custom pricing. The pricing FAQ says the platform works best with simple productions and broadcast-grade footage where the speaker is clearly visible. Paid plans include unlimited seats and concurrent jobs, while security and support options vary by plan.

MakeItTalk is Linux, self-hosted software that creates expressive talking-head videos from one facial image and an audio track. Audio content drives lip movement and nearby facial motion, while speaker information influences expressions and other talking-head movement. The project describes use with human portraits, paintings, sketches, 2D cartoons, manga, and stylized caricatures, including faces or characters not seen during training. Its repository includes pretrained models, module training code, and Google Colab demos for natural faces. For human faces and paintings, the README calls for a centered 256×256 JPG portrait and a WAV file. Optional command-line settings adjust lip motion on the x and y axes and head-motion displacement. Local setup specifies Python 3.6, ffmpeg, and Python packages; cartoon face warping on Ubuntu also requires winehq-stable. Custom puppets are listed as a to-do item, and training sections for voice conversion, the speaker-aware branch, and image-to-image translation are marked unfinished. It is free open-source research code under CC BY-NC 4.0, allowing noncommercial use under that license.