Skip to content
SPECIFICTION
Get listed

Lip Forcing

github.com

LinuxSelf-hosted
Visit github.com
Spec score 7.5 #16 of 28 in AI Video Lip Sync Tools

Spec sheet

Purpose
Lip Forcing is an autoregressive diffusion method for video-to-video lip synchronization that animates a reference video from audio.Read 8 Oct 2026 · github.com
Integrations
The checkpoint is hosted on Hugging Face, and the README also identifies external model components from Wan, wav2vec2, TAEW, LatentSync, and SyncNet sources.Read 8 Oct 2026 · github.com
License
The repository states that Lip Forcing is released under the Apache License 2.0.Read 8 Oct 2026 · github.com
Automatic processing
The inference pipeline automatically detects and aligns faces to 512×512 crops and pastes the generated result back into the video.Read 4 Oct 2026 · github.com
Availability
The Hugging Face model card says the model is not deployed by an Inference Provider.Read 8 Oct 2026 · huggingface.co
Dependencies
Inference requires external components including the Wan VAE, wav2vec audio encoder, text encoder or precomputed embeddings, decoder, and mouth mask.Read 8 Oct 2026 · github.com
External components
Inference also requires external components including the Wan VAE, wav2vec audio encoder, UMT5-XXL text encoder or precomputed embeddings, TAEW decoder, and mouth mask.Read 4 Oct 2026 · github.com
Generation
Its causal student models generate each chunk in two denoising steps without classifier-free guidance at inference.Read 4 Oct 2026 · github.com
Hardware
The repository reports testing on Ubuntu 24.04 with an NVIDIA H200 and says inference runs on a single GPU.Read 4 Oct 2026 · github.com
Hosting
The checkpoint page states that the model is not deployed by any Hugging Face Inference Provider.Read 4 Oct 2026 · huggingface.co
Inference setup
The documented inference command takes a reference video and speech audio and writes an output video.Read 4 Oct 2026 · github.com
Input handling
Inference accepts a reference video and speech audio, and automatically detects, aligns, and composites the face.Read 8 Oct 2026 · github.com
Memory requirement
The 14B model uses about 37 GB peak GPU memory with precomputed text embeddings, or about 50 GB when encoding text at runtime.Read 8 Oct 2026 · github.com
Model sizes
The release includes a 14B student checkpoint; the 1.3B student weights are listed as coming soon.Read 8 Oct 2026 · github.com
Performance
The project reports that the 1.3B student reaches 31 FPS and the 14B student runs 39.8 times faster than its teacher at comparable reference fidelity.Read 4 Oct 2026 · cvlab-kaist.github.io
Platform
The documented installation was tested on Ubuntu 24.04 with Python 3.12, PyTorch 2.10.0, CUDA 12.8, and an NVIDIA H200; inference runs on a single GPU.Read 8 Oct 2026 · github.com
Project affiliation
The project lists KAIST AI and AIPARK affiliations for its authors.Read 4 Oct 2026 · github.com
Real-time performance
The project reports 31 FPS for the 1.3B student on a single H100 GPU.Read 8 Oct 2026 · cvlab-kaist.github.io
Resource limit
For the 14B model, the repository reports about 37 GB peak VRAM with precomputed text embeddings and about 50 GB with runtime text encoding.Read 4 Oct 2026 · github.com
Roadmap
The repository roadmap lists a Gradio or Hugging Face Space demo as planned.Read 8 Oct 2026 · github.com
Streaming
Streaming inference produces frames before the input clip finishes and reports sub-millisecond time-to-first-frame.Read 8 Oct 2026 · github.com
Training
Training is documented as two stages: Diffusion-Forcing initialization followed by Self-Forcing DMD with a SyncNet reward.Read 8 Oct 2026 · github.com
Two-step generation
Its causal student models generate each chunk in two denoising steps without inference-time classifier-free guidance.Read 8 Oct 2026 · github.com
Weights
The released 14B student checkpoint is a merged, self-contained file, while the 1.3B student weights are listed as coming soon.Read 4 Oct 2026 · github.com

Where Specifiction places it

Alternatives to Lip ForcingAll alternatives →