Diff2Lip vs lipsyn.cc

Diff2Lip is a free, open-source Linux and self-hosted project that synchronizes speech with face videos using an audio-conditioned diffusion model. It accepts speech and face-video material and generates lip-synced video while aiming to preserve the identity shown in source frames. The repository includes inference scripts for audio-video pairs and for a single video. Cross mode drives a video with an audio source; reconstruction mode drives the video's first frame with audio. The project lists movies, education and virtual avatars as applications, with video conferencing described as a possible future use. Diff2Lip is not real time yet. Reported evaluations use the VoxCeleb2 and LRW datasets, and the repository identifies the work as a WACV 2024 paper. Setup instructions use Python 3.9, FFmpeg 5.0.1 and the requirements file. Inference supports one or more GPUs through NUM_GPUS, with values above one running distributed generation. A Google Colab notebook is linked for trying it.

lipsyn.cc turns video, portraits, or illustrated characters and audio into lip-synced video online. Users can upload audio or generate speech; the homepage advertises 91 languages and 10 voices. A free five-second preview is available without signup or a card. Full-length exports are available up to Full HD (1080p), and commercial use is included with exports. Videos can include up to three characters, with uploads up to 512 MB and audio and video up to 120 minutes. Paid minute packs are prepaid and do not expire: five minutes for €6.99, 10 for €13.90, 30 for €40.50, 60 for €79.20, or 180 for €232.20. The service has a web interface, an API, and an Adobe Express plugin. API renders run asynchronously and are delivered by webhook or polling, with cURL, Python, and Node.js examples. API limits include 60 requests per minute per key and five concurrent videos per account. The site says content remains the user's and is not used for training or other purposes; it also states that the service follows GDPR and Austrian data protection laws. Completed generations and minute-pack purchases are not refundable.