Blog
Abstract grainy texture with overlapping curved geometric shapes in purple and teal.

HeyGen Decision 1: Fast, Calibrated Judgments for all Your Decisions

Decisions happen all throughout our product. Is this frame clean? Is this audio clear? Is this input usable? Is this content safe? What is the user trying to do? What kind of request just came in? These questions sit at product entry points and boundaries, where thousands of requests pass through every hour.

By HeyGen ResearchNov 19, 20265 min read

Until now, even a simple answer meant asking a large language model (LLM). At this volume, that costs users time and costs us money. And LLMs tend to be over-confident, so their answers are poorly calibrated.

We needed something fast, cheap, and high quality. Our product also needs long-context requests and strong multimodal input: text, images, video, and audio. Existing decision models fall short on one or more of these. Some are fast but low quality. Others don't support the inputs we need.

So we built our own decision model for internal use, and now we're opening it up to the public.

Announcing HeyGen Decision 1.

Any mix of inputsAaTextImageVideoAudioYour question“Does this clip fit a sunrise skyline shot?”HeyGenDecision 1One requestCalibrated answerillustrativeGood fit, use it0.61Right scene, low quality0.27Has a watermark0.12Top choice, with the full distribution Any mix of inputsAaTextImageVideoAudioYour question“Does this clip fit asunrise skyline shot?”One requestHeyGenDecision 1Calibrated answerillustrativeGood fit, use it0.61Right scene, low quality0.27Has a watermark0.12Top choice, with the full distribution

Latency

104 ms

p50 per request on text benchmarks, against 447 ms for Gemini Decision 1

Calibration

0.0155

Top-1 ECE for Turbo over 12 text and image sets: lowest of the five models

Inputs

4

text, image, video and audio, in any mix, in one request

Motivation

Why we built it

HeyGen is a video company. Our products turn text, images, audio, and video into finished videos with avatars, voices, and translations. At almost every step, a decision needs to be made. Does the video contain the necessary elements? Is the person in the image framed correctly? Is the voice clear? Are there artifacts in the recording? What type of video is the user trying to create?

And the list goes on: moderation, data analysis, multimedia evaluation and comparison, content filtering, emotion understanding, and more.

For a long time, the default tool for questions like these was a general-purpose chat model: describe the question in a prompt, ask for a yes or no, and parse the reply. That works, but a decision model does two things better.

Speed

Near-instant response times

HeyGen Decision 1 returns its answer directly, with sub-second latency, so you don't wait for several output tokens from an LLM. In an interactive product, that means less time users spend staring at a spinner. In a batch pipeline, it means less cost and queue time across every asset.

Calibration

Calibrated response

Instead of "mostly yes, though there may be an issue near the end," or overconfident LLM responses, you get a precise and well-calibrated probability, allowing you to control outcomes at different confidence levels, trade precision against recall, and tell a confident "yes" from a coin flip. No fragile parsers, and no arguments over what "mostly" means.

Question types

How does HeyGen Decision 1 work?

There are three question types. Each one reads the same state and returns its own calibrated answer.

The state

TText
IImage
AAudio
VVideo

The same state is available to each question.

noul

A yes/no question

Returns the probability that the answer is "yes".

Is a person on screen?

Yes · 0.82illustrative
01
choice

Pick one of a fixed set of options

Returns the chosen option, a confidence, and the full distribution.

Setting: indoor / outdoor / studio

Studio · 0.64illustrative
01
score

Rate on a scale you define

Returns a probability-weighted score, plus a confidence.

Audio clarity, 0–4

3.2 / 4 · 0.76illustrative
04

You can also tell the model what "yes" and "no" mean for a given yes/no question, in a line each. Short, explicit criteria make the answer track your product's actual rule instead of the model's guess at it.

Because answers are probabilities, the product decides what to do with them. A moderation flow can block above one threshold and send a middle band to a person. A quality check can retry below another. A dashboard can average scores over a day of traffic and watch for drift. The model gives you the signal, and the product sets the policy.

From probability to policy

0 – 0.4
PassBelow the first threshold
0.4 – 0.75
Human reviewInside the middle band
0.75 – 1
BlockAbove the second threshold
00.250.50.751
0.73illustrative
Benchmarks

How it performs

Capabilities, accuracy, calibration and latency against Gemini Decision 1, GPT-6 Luna and TypeSafe Jev 1.14.

Capabilities

Inputs each model accepts, and its limits per request

CapabilityHeyGen Decision 1Gemini Decision 1GPT-6 LunaTypeSafe Jev 1.14
Input
TextYesYesYesYes
ImageYesYesYesExperimental mode only
VideoYesYesNoNo
AudioYesYesNoNo
Limits
Context windowTokens256K256K1M32K; 256K in experimental mode

* Image input and a 256K context window are available in TypeSafe Jev 1.14’s experimental mode.

Accuracy and calibration

Highest score per row in bold; HeyGen columns shaded

BenchmarkHeyGen Decision 1HeyGen Decision 1 TurboGemini Decision 1GPT-6 LunaTypeSafe Jev 1.14
Text
Amazon ESCIMacro-F162.8%61.8%57.8%51.9%59.4%
New Yorker CaptionText-only86.9%88.4%79.4%71.0%72.7%
PubMedQA81.4%81.4%80.5%72.2%76.8%
When2Call92.9%93.5%71.7%61.7%76.7%
WildGuardPrompt harm91.3%91.1%87.2%84.1%90.5%
Nemotron text safety86.1%86.3%80.4%78.4%83.2%
Image
Mind2Web Multimodal15 candidates93.3%92.9%87.3%80.6%—
POPEAdversarial88.5%88.3%87.6%82.8%—
HallusionBench80.8%79.8%79.3%70.0%—
AI2DDiagrams90.2%89.7%89.1%80.2%—
MMStar75.3%74.6%75.4%59.2%—
BLINK14 perception tasks77.4%75.9%75.2%60.5%—
Audio
ESC-5085.0%85.0%56.6%——
CREMA-D84.5%84.5%32.4%——
RAVDESSAudio only72.8%72.8%39.2%——
GTZAN88.1%88.1%65.6%——
VocalSound94.5%94.6%78.1%——
MMAU80.3%80.3%73.0%——
MMAR70.4%70.4%66.5%——
Nemotron audio safety84.2%84.1%71.6%——
Video
NExT-QA84.5%84.5%84.8%——
RWF-2000Violence91.8%91.8%89.5%——
EgoSchema76.8%76.8%75.0%——
VGGSound90.9%90.9%90.2%——
RAVDESSAudio + video78.8%78.8%59.0%——
MUSIC-AVQA79.3%79.5%80.7%——
Daily-Omni75.6%75.4%78.5%——
Calibration
Top-1 ECEText + image, 12 sets (Jev text only), lower is better0.02280.01550.02690.12190.0922
  1. Gemini Decision 1, GPT-6 Luna, and TypeSafe Jev 1.14 scores are reproduced from API calls, so every model is scored on the same evaluation sets and prompts.

Latency

Average p50 latency per request over 11 evaluation benchmarks. Lower is better.Average p50 latency per request over 5 text benchmarks. Lower is better.Average p50 latency per request over 6 image benchmarks. Lower is better.Average p50 latency per request over 8 audio benchmarks. Lower is better.Average p50 latency per request over 7 video benchmarks. Lower is better.Average p90 latency per request over 11 evaluation benchmarks. Lower is better.Average p90 latency per request over 5 text benchmarks. Lower is better.Average p90 latency per request over 6 image benchmarks. Lower is better.Average p90 latency per request over 8 audio benchmarks. Lower is better.Average p90 latency per request over 7 video benchmarks. Lower is better.

  • HeyGen Decision 1 Turbo
    191 ms
  • HeyGen Decision 1
    242 ms
  • TypeSafe Jev 1.14
    No image input
  • GPT-6 Luna
    208 ms
  • Gemini Decision 1
    703 ms
  • HeyGen Decision 1 Turbo
    93 ms
  • HeyGen Decision 1
    104 ms
  • TypeSafe Jev 1.14
    84 ms
  • GPT-6 Luna
    145 ms
  • Gemini Decision 1
    447 ms
  • HeyGen Decision 1 Turbo
    273 ms
  • HeyGen Decision 1
    357 ms
  • TypeSafe Jev 1.14
    No image input
  • GPT-6 Luna
    261 ms
  • Gemini Decision 1
    917 ms
  • HeyGen Decision 1 Turbo
    73 ms
  • HeyGen Decision 1
    71 ms
  • TypeSafe Jev 1.14
    Not measured
  • GPT-6 Luna
    Not measured
  • Gemini Decision 1
    506 ms
  • HeyGen Decision 1 Turbo
    791 ms
  • HeyGen Decision 1
    779 ms
  • TypeSafe Jev 1.14
    Not measured
  • GPT-6 Luna
    Not measured
  • Gemini Decision 1
    1,564 ms
  • HeyGen Decision 1 Turbo
    349 ms
  • HeyGen Decision 1
    435 ms
  • TypeSafe Jev 1.14
    No image input
  • GPT-6 Luna
    385 ms
  • Gemini Decision 1
    2,221 ms
  • HeyGen Decision 1 Turbo
    181 ms
  • HeyGen Decision 1
    196 ms
  • TypeSafe Jev 1.14
    156 ms
  • GPT-6 Luna
    255 ms
  • Gemini Decision 1
    1,419 ms
  • HeyGen Decision 1 Turbo
    490 ms
  • HeyGen Decision 1
    634 ms
  • TypeSafe Jev 1.14
    No image input
  • GPT-6 Luna
    494 ms
  • Gemini Decision 1
    2,890 ms
  • HeyGen Decision 1 Turbo
    112 ms
  • HeyGen Decision 1
    110 ms
  • TypeSafe Jev 1.14
    Not measured
  • GPT-6 Luna
    Not measured
  • Gemini Decision 1
    668 ms
  • HeyGen Decision 1 Turbo
    1,286 ms
  • HeyGen Decision 1
    1,280 ms
  • TypeSafe Jev 1.14
    Not measured
  • GPT-6 Luna
    Not measured
  • Gemini Decision 1
    3,131 ms
  • HeyGen Decision 1
  • HeyGen Decision 1 Turbo
  • Other models

What's next

HeyGen Decision 1 has already changed how we build decisions at HeyGen, and it has already saved thousands of hours of user wait time and LLM cost for us. We look forward to seeing what HeyGen Decision 1 can do for you.