MakeItTalk vs Sync Labs

MakeItTalk is Linux, self-hosted software that creates expressive talking-head videos from one facial image and an audio track. Audio content drives lip movement and nearby facial motion, while speaker information influences expressions and other talking-head movement. The project describes use with human portraits, paintings, sketches, 2D cartoons, manga, and stylized caricatures, including faces or characters not seen during training. Its repository includes pretrained models, module training code, and Google Colab demos for natural faces. For human faces and paintings, the README calls for a centered 256×256 JPG portrait and a WAV file. Optional command-line settings adjust lip motion on the x and y axes and head-motion displacement. Local setup specifies Python 3.6, ffmpeg, and Python packages; cartoon face warping on Ubuntu also requires winehq-stable. Custom puppets are listed as a to-do item, and training sections for voice conversion, the speaker-aware branch, and image-to-image translation are marked unfinished. It is free open-source research code under CC BY-NC 4.0, allowing noncommercial use under that license.

Sync Labs provides an AI lip-sync API that generates mouth movements matched to video and audio. Its listed models include lipsync-1.9, lipsync-2, lipsync-2-pro, react-1, and sync-3. The introduction lists 512×512 face resolution for lipsync-2 and lipsync-2-pro, and native 4K for sync-3. The models use audio waveforms rather than text and support 29 spoken languages, including tonal languages such as Mandarin and Thai. Use cases include dubbing, localization, personalized video messaging, e-learning, marketing, entertainment, media, and gaming. Developers can use a REST API and official Python and TypeScript SDKs; documented integrations include Adobe Premiere, DaVinci Resolve, ChatGPT, an MCP server, ComfyUI, and ElevenLabs. Inputs include MP4 video and WAV or MP3 audio, provided by public URL, API upload, or media-library asset ID. The free plan provides 3 generations per month, each up to 20 seconds, and no credit card is required. Hobbyist costs $5 a month plus $0.05/sec; paid plans also list usage charges. The service is available via API and web.