daVinci-MagiHuman

Description

daVinci-MagiHuman is a 15-billion-parameter open-source AI model that generates lip-synced talking videos from a single photo. You upload a portrait, add a script or audio, and get a natural talking clip where the audio and video are generated together rather than stitched from separate pipelines. It is developed by Sand.ai and GAIR Lab (Shanghai Jiao Tong University) and released under the Apache 2.0 license, so the weights can be inspected, run locally, and used commercially within the license.

The model uses a single-stream Transformer that jointly denoises video and audio tokens with a reference-image latent, producing unified audio-video output from a face photo plus text or audio. On a single NVIDIA H100 GPU it can generate a short 256p clip in about two seconds of wall time, and published evaluations report strong word-error rates and high human preference against baselines such as Ovi 1.1 and LTX 2.3.

Users can try it through a free online demo, download the checkpoints from Hugging Face, or clone the GitHub repository to self-host and run inference with custom settings and resolutions up to 1080p. It supports multiple languages for lip sync depending on the released training data.

The hosted service offers a free tier with starter and daily check-in credits (15-day temporary storage) plus Basic, Pro, and Max credit plans that add HD generation, priority processing, permanent asset storage, and commercial usage rights.

daVinci-MagiHuman's Core Features

• Single-photo talking video generation with lip sync

• Unified audio and video generated together in one model pass

• 15B-parameter single-stream Transformer architecture

• Open source under the Apache 2.0 license

• Fast inference (~2s for a ~2s 256p clip on an H100)

• Multilingual lip sync depending on training data

• Self-hosting via Hugging Face and GitHub, or a hosted online demo

• Output resolutions up to 1080p

How to use daVinci-MagiHuman?

1. 1

   Upload a portrait: Add a clear, front-facing face photo.

2. 2

   Add a script or audio: Enter text or upload an audio file to drive the speech.

3. 3

   Choose resolution: Select an output resolution such as 256p, 720p, or 1080p.

4. 4

   Generate: Run the model to create the lip-synced clip.

5. 5

   Download: Save the finished talking video, or self-host from Hugging Face or GitHub.

daVinci-MagiHuman's Use Cases

• Talking avatar videos
• Self-hosted generation
• Multilingual dubbing
• Research and benchmarking
• Content creation

FAQ from daVinci-MagiHuman

• What is daVinci-MagiHuman?

  It is a 15B-parameter open audio-video model from Sand.ai and GAIR Lab that turns a portrait plus text or audio into a lip-synced talking clip, generating aligned speech and frames together.

• Is daVinci-MagiHuman free?

  The open weights and code are released under Apache 2.0, so self-hosting follows that license. Hosted demos may have separate terms, and the hosted service includes a free credit tier.

• What inputs does it need?

  It needs a face image plus driving text or audio. Exact file formats and limits follow the official inference README.

• Can I use it commercially?

  Yes. Apache 2.0 allows commercial use of the model subject to its conditions, and paid hosted plans include commercial usage rights and certificate access.

• How does it compare to other models?

  Published evaluations report roughly 14.6% word-error rate versus about 40.5% for Ovi 1.1, and it wins a large share of pairwise human evaluations against Ovi and LTX 2.3.

• What hardware do I need to self-host?

  Throughput scales with GPU class and resolution. Public reports reference H100-class GPUs for short clips, while lower tiers may work at smaller resolutions.