CHAMELAION LipSync API vs Diff2Lip

CHAMELAION LipSync API takes video and audio input and produces a video with lip movements matched to the audio. It accepts MP4 video and WAV or MP3 audio by public URL or direct upload. Active-speaker detection runs automatically but can be disabled for an individual request. The maker says the model retains speaker identity, speaking style, and expressions while adapting mouth movements to new audio. Listed uses include dubbing and translation, AI avatars, creative ads, and dialogue replacement or personalization. SDKs are listed for Python 3.8+ and TypeScript/Node.js 18+. Requests require a Bearer token or x-api-key header, except at the health endpoint; generation endpoints are limited to 60 requests per minute, and concurrent job limits depend on the plan. Starter is free and includes 24,000 tokens and two minutes of translation and lip sync, with a watermark. Basic costs €4.00 per year and lists 60,000 tokens and five minutes. The maker says its servers are in Europe and users retain ownership of generated videos. Profile views may work when the mouth remains visible; extreme close-ups, side views above 90 degrees, and cropped mouths are difficult cases.

Diff2Lip is a free, open-source Linux and self-hosted project that synchronizes speech with face videos using an audio-conditioned diffusion model. It accepts speech and face-video material and generates lip-synced video while aiming to preserve the identity shown in source frames. The repository includes inference scripts for audio-video pairs and for a single video. Cross mode drives a video with an audio source; reconstruction mode drives the video's first frame with audio. The project lists movies, education and virtual avatars as applications, with video conferencing described as a possible future use. Diff2Lip is not real time yet. Reported evaluations use the VoxCeleb2 and LRW datasets, and the repository identifies the work as a WACV 2024 paper. Setup instructions use Python 3.9, FFmpeg 5.0.1 and the requirements file. Inference supports one or more GPUs through NUM_GPUS, with values above one running distributed generation. A Google Colab notebook is linked for trying it.