MakeItTalk vs SadTalker

MakeItTalk is Linux, self-hosted software that creates expressive talking-head videos from one facial image and an audio track. Audio content drives lip movement and nearby facial motion, while speaker information influences expressions and other talking-head movement. The project describes use with human portraits, paintings, sketches, 2D cartoons, manga, and stylized caricatures, including faces or characters not seen during training. Its repository includes pretrained models, module training code, and Google Colab demos for natural faces. For human faces and paintings, the README calls for a centered 256×256 JPG portrait and a WAV file. Optional command-line settings adjust lip motion on the x and y axes and head-motion displacement. Local setup specifies Python 3.6, ffmpeg, and Python packages; cartoon face warping on Ubuntu also requires winehq-stable. Custom puppets are listed as a to-do item, and training sections for voice conversion, the speaker-aware branch, and image-to-image translation are marked unfinished. It is free open-source research code under CC BY-NC 4.0, allowing noncommercial use under that license.

SadTalker creates a talking-head video from a portrait image and speech audio. It derives 3D head-pose and facial-expression coefficients from the audio to animate the face. The project describes still, reference, and resize modes, along with full-image animation. Command-line examples include GFPGAN enhancement. You can use a local Gradio demo or command-line inference, with setup paths for Linux, Windows, and macOS; Hugging Face and Colab demos are also linked. The documented Linux setup includes Python 3.8, PyTorch, FFmpeg, and the repository requirements. Windows setup also calls for Python, Git, FFmpeg, and model files. Pre-trained models are linked from Google Drive, GitHub Releases, and Baidu. The project says its open-source code runs offline and collects no personal information or other data. SadTalker is free at free under Apache 2.0, except for listed third-party components. Output is 720p or lower and has no watermark. Integrations include Discord and a Stable Diffusion WebUI extension.