Until now, even a simple answer meant asking a large language model (LLM). At this volume, that costs users time and costs us money. And LLMs tend to be over-confident, so their answers are poorly calibrated.
We needed something fast, cheap, and high quality. Our product also needs long-context requests and strong multimodal input: text, images, video, and audio. Existing decision models fall short on one or more of these. Some are fast but low quality. Others don't support the inputs we need.
So we built our own decision model for internal use, and now we're opening it up to the public.
Announcing HeyGen Decision 1.
Latency
104 ms
p50 per request on text benchmarks, against 447 ms for Gemini Decision 1
Calibration
0.0155
Top-1 ECE for Turbo over 12 text and image sets: lowest of the five models
Inputs
4
text, image, video and audio, in any mix, in one request
Why we built it
HeyGen is a video company. Our products turn text, images, audio, and video into finished videos with avatars, voices, and translations. At almost every step, a decision needs to be made. Does the video contain the necessary elements? Is the person in the image framed correctly? Is the voice clear? Are there artifacts in the recording? What type of video is the user trying to create?
And the list goes on: moderation, data analysis, multimedia evaluation and comparison, content filtering, emotion understanding, and more.
For a long time, the default tool for questions like these was a general-purpose chat model: describe the question in a prompt, ask for a yes or no, and parse the reply. That works, but a decision model does two things better.
Speed
Near-instant response times
HeyGen Decision 1 returns its answer directly, with sub-second latency, so you don't wait for several output tokens from an LLM. In an interactive product, that means less time users spend staring at a spinner. In a batch pipeline, it means less cost and queue time across every asset.
Calibration
Calibrated response
Instead of "mostly yes, though there may be an issue near the end," or overconfident LLM responses, you get a precise and well-calibrated probability, allowing you to control outcomes at different confidence levels, trade precision against recall, and tell a confident "yes" from a coin flip. No fragile parsers, and no arguments over what "mostly" means.
How does HeyGen Decision 1 work?
There are three question types. Each one reads the same state and returns its own calibrated answer.
The state
The same state is available to each question.
noul
A yes/no question
Returns the probability that the answer is "yes".
Is a person on screen?
choice
Pick one of a fixed set of options
Returns the chosen option, a confidence, and the full distribution.
Setting: indoor / outdoor / studio
score
Rate on a scale you define
Returns a probability-weighted score, plus a confidence.
Audio clarity, 0–4
You can also tell the model what "yes" and "no" mean for a given yes/no question, in a line each. Short, explicit criteria make the answer track your product's actual rule instead of the model's guess at it.
Because answers are probabilities, the product decides what to do with them. A moderation flow can block above one threshold and send a middle band to a person. A quality check can retry below another. A dashboard can average scores over a day of traffic and watch for drift. The model gives you the signal, and the product sets the policy.
From probability to policy
How it performs
Capabilities, accuracy, calibration and latency against Gemini Decision 1, GPT-6 Luna and TypeSafe Jev 1.14.
Capabilities
Inputs each model accepts, and its limits per request
| Capability | HeyGen Decision 1 | Gemini Decision 1 | GPT-6 Luna | TypeSafe Jev 1.14 |
|---|---|---|---|---|
| Input | ||||
| Text | ✓Yes | ✓Yes | ✓Yes | ✓Yes |
| Image | ✓Yes | ✓Yes | ✓Yes | —*Experimental mode only |
| Video | ✓Yes | ✓Yes | —No | —No |
| Audio | ✓Yes | ✓Yes | —No | —No |
| Limits | ||||
| Context windowTokens | 256K | 256K | 1M | 32K*; 256K in experimental mode |
* Image input and a 256K context window are available in TypeSafe Jev 1.14’s experimental mode.
Accuracy and calibration
Highest score per row in bold; HeyGen columns shaded
| Benchmark | HeyGen Decision 1 | HeyGen Decision 1 Turbo | Gemini Decision 1 | GPT-6 Luna | TypeSafe Jev 1.14 |
|---|---|---|---|---|---|
| Text | |||||
| Amazon ESCIMacro-F1 | 62.8% | 61.8% | 57.8% | 51.9% | 59.4% |
| New Yorker CaptionText-only | 86.9% | 88.4% | 79.4% | 71.0% | 72.7% |
| PubMedQA | 81.4% | 81.4% | 80.5% | 72.2% | 76.8% |
| When2Call | 92.9% | 93.5% | 71.7% | 61.7% | 76.7% |
| WildGuardPrompt harm | 91.3% | 91.1% | 87.2% | 84.1% | 90.5% |
| Nemotron text safety | 86.1% | 86.3% | 80.4% | 78.4% | 83.2% |
| Image | |||||
| Mind2Web Multimodal15 candidates | 93.3% | 92.9% | 87.3% | 80.6% | — |
| POPEAdversarial | 88.5% | 88.3% | 87.6% | 82.8% | — |
| HallusionBench | 80.8% | 79.8% | 79.3% | 70.0% | — |
| AI2DDiagrams | 90.2% | 89.7% | 89.1% | 80.2% | — |
| MMStar | 75.3% | 74.6% | 75.4% | 59.2% | — |
| BLINK14 perception tasks | 77.4% | 75.9% | 75.2% | 60.5% | — |
| Audio | |||||
| ESC-50 | 85.0% | 85.0% | 56.6% | — | — |
| CREMA-D | 84.5% | 84.5% | 32.4% | — | — |
| RAVDESSAudio only | 72.8% | 72.8% | 39.2% | — | — |
| GTZAN | 88.1% | 88.1% | 65.6% | — | — |
| VocalSound | 94.5% | 94.6% | 78.1% | — | — |
| MMAU | 80.3% | 80.3% | 73.0% | — | — |
| MMAR | 70.4% | 70.4% | 66.5% | — | — |
| Nemotron audio safety | 84.2% | 84.1% | 71.6% | — | — |
| Video | |||||
| NExT-QA | 84.5% | 84.5% | 84.8% | — | — |
| RWF-2000Violence | 91.8% | 91.8% | 89.5% | — | — |
| EgoSchema | 76.8% | 76.8% | 75.0% | — | — |
| VGGSound | 90.9% | 90.9% | 90.2% | — | — |
| RAVDESSAudio + video | 78.8% | 78.8% | 59.0% | — | — |
| MUSIC-AVQA | 79.3% | 79.5% | 80.7% | — | — |
| Daily-Omni | 75.6% | 75.4% | 78.5% | — | — |
| Calibration | |||||
| Top-1 ECEText + image, 12 sets (Jev text only), lower is better | 0.0228 | 0.0155 | 0.0269 | 0.1219 | 0.0922 |
- Gemini Decision 1, GPT-6 Luna, and TypeSafe Jev 1.14 scores are reproduced from API calls, so every model is scored on the same evaluation sets and prompts.
Latency
Average p50 latency per request over 11 evaluation benchmarks. Lower is better.Average p50 latency per request over 5 text benchmarks. Lower is better.Average p50 latency per request over 6 image benchmarks. Lower is better.Average p50 latency per request over 8 audio benchmarks. Lower is better.Average p50 latency per request over 7 video benchmarks. Lower is better.Average p90 latency per request over 11 evaluation benchmarks. Lower is better.Average p90 latency per request over 5 text benchmarks. Lower is better.Average p90 latency per request over 6 image benchmarks. Lower is better.Average p90 latency per request over 8 audio benchmarks. Lower is better.Average p90 latency per request over 7 video benchmarks. Lower is better.
- HeyGen Decision 1 Turbo191 ms
- HeyGen Decision 1242 ms
- TypeSafe Jev 1.14No image input
- GPT-6 Luna208 ms
- Gemini Decision 1703 ms
- HeyGen Decision 1 Turbo93 ms
- HeyGen Decision 1104 ms
- TypeSafe Jev 1.1484 ms
- GPT-6 Luna145 ms
- Gemini Decision 1447 ms
- HeyGen Decision 1 Turbo273 ms
- HeyGen Decision 1357 ms
- TypeSafe Jev 1.14No image input
- GPT-6 Luna261 ms
- Gemini Decision 1917 ms
- HeyGen Decision 1 Turbo73 ms
- HeyGen Decision 171 ms
- TypeSafe Jev 1.14Not measured
- GPT-6 LunaNot measured
- Gemini Decision 1506 ms
- HeyGen Decision 1 Turbo791 ms
- HeyGen Decision 1779 ms
- TypeSafe Jev 1.14Not measured
- GPT-6 LunaNot measured
- Gemini Decision 11,564 ms
- HeyGen Decision 1 Turbo349 ms
- HeyGen Decision 1435 ms
- TypeSafe Jev 1.14No image input
- GPT-6 Luna385 ms
- Gemini Decision 12,221 ms
- HeyGen Decision 1 Turbo181 ms
- HeyGen Decision 1196 ms
- TypeSafe Jev 1.14156 ms
- GPT-6 Luna255 ms
- Gemini Decision 11,419 ms
- HeyGen Decision 1 Turbo490 ms
- HeyGen Decision 1634 ms
- TypeSafe Jev 1.14No image input
- GPT-6 Luna494 ms
- Gemini Decision 12,890 ms
- HeyGen Decision 1 Turbo112 ms
- HeyGen Decision 1110 ms
- TypeSafe Jev 1.14Not measured
- GPT-6 LunaNot measured
- Gemini Decision 1668 ms
- HeyGen Decision 1 Turbo1,286 ms
- HeyGen Decision 11,280 ms
- TypeSafe Jev 1.14Not measured
- GPT-6 LunaNot measured
- Gemini Decision 13,131 ms
- HeyGen Decision 1
- HeyGen Decision 1 Turbo
- Other models
What's next
HeyGen Decision 1 has already changed how we build decisions at HeyGen, and it has already saved thousands of hours of user wait time and LLM cost for us. We look forward to seeing what HeyGen Decision 1 can do for you.

