2-Part Guide:
Part 1 : Getting Started in Python (this post)
Part 2: Deploying the Gemini Decisions API on Google Cloud Run (coming up next)
Most backend software doesn’t actually need an AI model to write paragraphs. It just needs a fast, trustworthy answer to a bounded question:
- Yes or no?
- Which support team owns this ticket?
- How urgent is it?
Early models like TypeSafe AI’s Jev recently sparked developer interest in dedicated “yes/no” classifiers by showing what happens when you drop slow, token-by-token JSON generation and score closed-set decisions in a single pass.
Google DeepMind’s new Gemini Decisions API (gemini-decision-1) brings this capability natively into the google-genai SDK. At the time of writing, it lets you you evaluate up to 1,000 yes/no (noul), multiple-choice (choice), and rubric (score) questions in one pass with calibrated probabilities, zero output tokens, and built-in multimodal support across text, photos, audio, video, and PDFs.
The official announcement is shown below:
Gemini Decisions API: Fast, calibrated multimodal decisions
What You Will Learn
Backend applications constantly make simple, bounded decisions:
- Which support team owns this ticket?
- Can this refund be approved automatically?
- Which tool matches the user’s request?
- Does this attachment match what the customer claimed?
For the past few years that have coincided with us, using LLMs for a lot of things, most teams answered these questions by asking a chat LLM to reply with a JSON string, running into multi-second wait times, expensive output-token bills, fragile string parsing, and guessed confidence numbers.
In this post, you only need Python 3.11+ and a GEMINI_API_KEY from Google AI Studio (no cloud setup required) to get hands-on with the Gemini Decisions API (gemini-decision-1).
You will learn about:
- Why Asking a Chat LLM for JSON Hurts Backends: The four practical problems, speed, output-token cost, fragile string parsing, and guessed confidence numbers, when you use chat generation for yes/no or multiple-choice decisions.
- How Jev Sparked Interest in Dedicated Yes/No Models: A quick look at how early yes/no classifiers shifted developer thinking away from token generation.
- How the Gemini Decisions API (gemini-decision-1) works: Scoring text, images, audio, video, and PDFs across up to 1,000 questions at once in a single pass with zero output tokens billed ($0.10 per 1M input tokens).
We will cover several examples in Python (scripts/), that cover:
- Example 1 (quickstart_decisions.py): Routing a support ticket to the right team, rating its urgency, and checking whether it can be handled automatically—or should wait for human review (like an unverified $140 refund request).
- Example 2 (parallel_guardrails.py): Checking a message against 8 safety, privacy, and routing rules all at once in a single API call.
- Example 3 (multimodal_triage.py): Inspecting a photo (low vs. high resolution) and verifying an attached PDF invoice directly—without needing a separate OCR step.
Why Asking a Chat LLM for JSON Slows Down Backends
Chat LLMs were built to have conversations with people. When you ask an assistant to write an email or explain a bug, waiting three seconds while words stream onto the screen feels completely natural.
Problems start when backend teams use that exact same chat workflow just to make a quick software routing decision, like deciding where to send a customer ticket ("Where is order #19028?") or whether it is safe to fulfill automatically:
What Happens When You Use a Chat LLM for Simple Routing:
│Incoming Ticket ("Where is order #19028?")
▼
Prompt: "Reply ONLY in valid JSON: {"department": "...", "auto_resolve": true, "confidence": 0.95}"
│
├──► 1. Too Slow: Model types out JSON one token at a time (2,000ms – 6,000ms)
├──► 2. Too Costly: You pay 4x–8x output-token prices just for "{", quotes, and keys
├──► 3. Too Brittle: Code must strip markdown fences, parse text, and retry on bad formats
└──► 4. Unreliable: "confidence": 0.95 is a guessed number, not a real probability
In production, this “ask a chatbot for JSON” pattern creates a few issues:
Too Slow: Waiting for the Model to Type Out Text
A chat model writes its reply one small piece (token) at a time, just like watching words appear in a chat window. Even when your code only needs a simple yes or a category name, the model still spends 2 to 6 seconds "thinking" internally and spelling out every curly brace, quote, and key of {"department": "order_status", "can_auto_resolve": true}. That is fine for a human reader, but far too slow inside an API endpoint or live webhook.
Too Costly: Paying Extra for Throwaway Formatting
Across commercial AI providers, output tokens (the text the model writes back, plus any hidden thinking tokens) cost 4x to 8x more than input tokens (the text you send in). At millions of tickets or events per month, you end up paying premium output rates just for the model to generate JSON punctuation ({, ", :, }) that your application immediately parses and throws away.
Too Brittle: Cleaning Up and Parsing Free-Form Strings
Because a chat model is still generating text, your code has to turn that string back into structured variables. Even with JSON mode or structured-output settings, models can wrap replies in markdown code fences, slightly rename a category, or return a polite refusal sentence instead of JSON, forcing every service to maintain cleanup code, validators, and retry loops.
Unreliable Confidence: Guessed Numbers Instead of Real Probabilities
Chat models are trained to sound helpful and confident to human readers. If you ask a chat prompt to include "confidence": 0.95 inside its JSON output, the model simply writes a number that looks reasonable in text rather than computing an actual mathematical probability. It might end up routinely writing "confidence": 0.95 on a tricky $140 billing dispute where it is really guessing, making it unsafe for your code to decide when to auto-approve an action versus when to escalate to a human.
Key Takeaway:
Chat models generate text for people to read. Backend code does not need a paragraph or a JSON string, it needs fast yes/no and multiple-choice probabilities that an if/else statement can actually trust.
How Jev Sparked Interest in Dedicated Yes/No Models
Before diving into the Gemini Decisions API, it helps to look at the simple idea that recently got developers rethinking classification.
In Thinking, Fast and Slow, psychologist Daniel Kahneman describes two ways of thinking:
- Fast judgment: Instantly recognizing what kind of situation you are looking at.
- Slow reasoning: Working step-by-step through a hard problem or writing a detailed explanation.
Startups such as TypeSafe AI helped popularize this idea for developers with Jev ( named after economist William Stanley Jevons and the Jevons Paradox), when a technology makes something dramatically faster and cheaper, teams find many more places to use it.
Jev showed that if you stop generating text token-by-token and instead score closed-set yes/no (noul), multiple-choice (choice), and rubric (score) questions in a single pass with calibrated probabilities and zero output tokens, classification stops feeling like a slow chatbot call and starts feeling like a normal function call.
That preview sparked wide interest in dedicated yes/no models and immediately raised two practical scenarios for teams building real applications:
- What happens when customer requests include photos of damaged packages, PDF invoices, audio voicemails, or video clips — not just plain text?
- What if we want to check dozens of independent safety, PII, and routing rules at once over large documents using the standard Gemini SDK?
Meet the Gemini Decisions API (gemini-decision-1)
Google DeepMind introduced the Gemini Decisions API, powered by Gemini Decision 1 (gemini-decision-1) in the google-genai SDK (v1beta), specifically to solve these problems.
Instead of generating text token-by-token, gemini-decision-1 reads your input once and scores all of your questions in a single pass, returning calibrated probabilities directly.
Key Highlights
- Pay Only for Input ($0.10 per 1M input tokens, $0 output): Because the model returns probabilities instead of generating text, output tokens are always zero. Even if you ask dozens of questions in one call, your input is only billed once, making it multiple times cheaper than lightweight chat models.
- Fast, Steady Speed (50–100 ms model time): With no word-by-word text generation, responses come back quickly and stay roughly the same speed whether you ask 1 question or 100 questions at once.
- Works Directly on Files and Media (No OCR Needed): You can pass text, photos, audio, video, and PDFs together (up to 262K tokens), and adjust image or video resolution (low to ultra_high) to control token cost.
Every call to client.decisions.create takes a questions dictionary where each question uses one of three types:
A Quick Plain-English Walkthrough
Imagine a customer sends this message:
“Where is my order #19028? Can you tell me when it arrives?”
Here is how each question type works on that message:
noul (Yes / No) — "Can we answer this automatically without a human?"
- What you send: A yes/no question (optionally with a short note on what counts as yes vs. no).
- What comes back: noul = 0.98 and confidence = 0.98—meaning a 98% chance the answer is "yes." (If the answer were clearly "no," you would see noul = 0.02 and confidence = 0.98—only a 2% chance of "yes," so 98% confident in "no.")
choice (Pick One Option) — "Which team should handle this message?"
- What you send: A fixed list of options like order_status, billing_refund, account_access, and other, each with a one-line description.
- What comes back: The winning key (choice = "order_status"), its confidence (0.9995), and a probability breakdown across every option ({"order_status": 0.9995, "billing_refund": 0.0001, …}) that adds up to 1.0.
Score(Ordered Scale) — "How urgent is this message?"
- What you send: An ordered scale from lowest (0) to highest (3), such as ["Low", "Medium", "High", "Critical"].
- What comes back: A weighted-average number between 0 and 3 (score = 0.15), plus the probability for each level and the top level's confidence (0.87). A routine tracking question lands near 0.15 (Low), whereas a live production outage lands near 2.98 (Critical).
Even better, you don’t make three separate API calls. You include all three questions in one request, and all three answers come back together.
Quick Local Setup (60 Seconds)
All you need is Python 3.11+ and a Gemini API Key from Google AI Studio.
git clone https://github.com/rominirani/gcp-gemini-decisions-hybrid-gateway.git
cd gcp-gemini-decisions-hybrid-gateway
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
export GEMINI_API_KEY="YOUR_GEMINI_API_KEY"
Note: The Gemini Decisions API requires google-genai >= 3.2.0 initialized with http_options={"api_version": "v1beta"}. None of the scripts in this repository use mock responses or fallbacks, every command calls gemini-decision-1 live.
Example 1: Routing Support Tickets (scripts/quickstart_decisions.py)
Let’s look at scripts/quickstart_decisions.py, which asks all three question types in a single call:
- department (choice)
- can_auto_resolve (noul)
- urgency (score)
from google import genai
client = genai.Client(api_key=api_key, http_options={"api_version": "v1beta"})
result = client.decisions.create(
model="gemini-decision-1",
input=query_text,
questions={
"department": {
"type": "choice",
"instructions": "Which operational queue should handle this customer message?",
"criteria": {
"order_status": "Order tracking, delivery dates, or shipment status.",
"billing_refund": "Duplicate charges, invoices, credits, or refund requests.",
"subscription_cancel": "Canceling or pausing a recurring subscription.",
"account_access": "Password resets, expired 2FA/MFA codes, or login lockouts.",
"complex_incident": "Production outages, HTTP 500 errors, architecture, or legal/DPA questions.",
"other": "Anything outside these specific categories.",
},
},
"can_auto_resolve": {
"type": "noul",
"instructions": "Can this request be fulfilled deterministically by an automated backend lookup or webhook without custom investigation?",
"criteria": {
"true_description": "Routine single-intent request such as checking tracking, resetting login, or standard cancellation.",
"false_description": "Production outage, multi-part legal question, ambiguous dispute, or engineering troubleshooting.",
},
},
"urgency": {
"type": "score",
"instructions": "Rate the operational urgency and severity of this message.",
"criteria": [
"Low: Routine informational or tracking question.",
"Medium: Standard account, subscription, or billing request.",
"High: Time-sensitive customer impact or repeated billing failure.",
"Critical: Production-halting outage, data loss risk, or security incident.",
],
},
},
)
Run 1A: Simple Order-Tracking Question
python scripts/quickstart_decisions.py
Live Output:
{
"query": "Where is my order #19028? Can you tell me when it arrives?",
"model": "gemini-decision-1",
"latency_ms": 680.43,
"department": {
"choice": "order_status",
"confidence": 0.9995,
"probabilities": {
"billing_refund": 0.0001,
"order_status": 0.9995,
"account_access": 0.0001,
"complex_incident": 0.0002,
"other": 0.0001,
"subscription_cancel": 0.0
}
},
"can_auto_resolve": {
"noul_probability_yes": 0.98,
"confidence": 0.98,
"decision": true
},
"urgency": {
"expected_score_0_to_3": 0.1524,
"top_level": "Low: Routine informational or tracking question.",
"confidence": 0.8717
},
"suggested_action": "DISPATCH_TRACKING_WEBHOOK",
"usage": {
"total_input_tokens": 424,
"total_output_tokens": 0,
"estimated_cost_usd": 4.24e-05
}
}
(Note: The 680 ms above includes initial TLS connection setup from a local laptop; when we run this same call warm inside Google Cloud Run us-central1 in Post 2, we will see the difference.)
Run 1B: Seeing Calibrated Caution on an Unverified $140 Refund Request
Now run the script with a $140 duplicate-charge claim without any invoice attached yet:
python scripts/quickstart_decisions.py \
"I was billed twice (\$140 total) on invoice #8812 for one seat. Refund \$70 right now."
Live Output:
{
"query": "I was billed twice (40 total) on invoice #8812 for one seat. Refund 0 right now.",
"model": "gemini-decision-1",
"latency_ms": 866.75,
"department": {
"choice": "billing_refund",
"confidence": 0.9981,
"probabilities": {
"subscription_cancel": 0.0009,
"account_access": 0.0003,
"order_status": 0.0002,
"other": 0.0004,
"billing_refund": 0.9981,
"complex_incident": 0.0001
}
},
"can_auto_resolve": {
"noul_probability_yes": 0.3809,
"confidence": 0.6191,
"decision": false
},
"urgency": {
"expected_score_0_to_3": 1.3928,
"top_level": "Medium: Standard account, subscription, or billing request.",
"confidence": 0.5621
},
"suggested_action": "INITIATE_BILLING_WORKFLOW",
"usage": {
"total_input_tokens": 431,
"total_output_tokens": 0,
"estimated_cost_usd": 4.31e-05
}
}
Look at how those two independent questions work together for your backend code:
- Where should this ticket go? (department = billing_refund, 0.9981): Figuring out which team owns the message is easy, the customer is asking for a refund on an invoice, so gemini-decision-1 is 99.81% sure it belongs in the Billing queue.
- Should code execute it automatically? (can_auto_resolve = 0.3809 → decision: false): Unlike looking up a tracking number in Run 1A (0.98), handing back $70 based on a text claim alone is a billing dispute that needs verification, not a simple lookup. Because 0.3809 falls well below the auto.noul >= 0.80 cutoff in quickstart_decisions.py, decision comes back false.
Why is this separation so useful? When teams ask a chat LLM for a single JSON object like {"department": "billing_refund", "confidence": 0.95}, the model returns 0.95 simply because it recognized the word "Refund", making it easy to confuse "sure about the category" with "safe to approve automatically." Asking two separate questions in one decisions.create call lets your app route the ticket straight to Billing without triggering an automatic $70 payout.
In Example 3 below, we will attach the actual PDF invoice so gemini-decision-1 can verify the document directly.
Example 2: Running 8 Policy Checks in One Call (scripts/parallel_guardrails.py)
In real applications, an incoming message usually needs more than one check before your code touches a database or triggers a tool. You might want to check 8 different things at the same time:
- Is the user attempting prompt injection (noul)?
- Did they paste raw PII or a live secret key (noul)?
- Does the claim meet our <=$50, <=30 days instant-refund rule (noul)?
- Are they threatening a chargeback or legal action (noul)?
- Which team queue owns the event (choice)?
- Which backend tool fits if automation is approved (choice)?
- What is the customer churn risk (score, 0..3)?
- What is the hostility / abuse level (score, 0..2)?
With chat LLMs, making 8 separate calls multiplies your latency and bills your input text 8 times, while stuffing 8 rules into one giant JSON prompt makes the model miss edge cases.
With gemini-decision-1, scripts/parallel_guardrails.py sends all 8 questions in a single decisions.create call (so your input text is billed once). It does it in 3 clean steps:
Step 1: Pass the Event Metadata + Message Together as input
Notice that our refund rule depends on amount_usd and order_age_days as well as the customer's text. Passing a serialized JSON dictionary as input lets gemini-decision-1 check both structured fields and free-form text at the same time and bills that input payload only once:
event = {
"event_id": "evt-03-clean-refund",
"customer_tier": "PREMIUM",
"order_age_days": 6,
"amount_usd": 34.0,
"message": "Order #77102 arrived yesterday with a cracked mug handle. Could I get a $34 refund?",
}
Step 2: Score All 8 Independent Questions in One decisions.create Call
We define all 8 questions (4 noul, 2 choice, and 2 score) in EIGHT_PARALLEL_QUESTIONS and send them in a single call:
Run the script across three sample messages (a prompt-injection attempt, an accidentally pasted sk_live_… API key, and a clean $34 damaged-mug refund):
EIGHT_PARALLEL_QUESTIONS = {
"prompt_injection_attempt": {
"type": "noul",
"instructions": "Is the sender attempting prompt injection, instruction override, or system-prompt extraction?",
"criteria": { ... },
},
"contains_raw_pii_or_secrets": {
"type": "noul",
"instructions": "Does the message expose raw payment card numbers, SSNs, passwords, or live API keys?",
"criteria": { ... },
},
"eligible_for_instant_refund": {
"type": "noul",
"instructions": "Does this request qualify for instant automated refund under policy (amount <= $50, order_age_days <= 30, clear defect or duplicate)?",
"criteria": { ... },
},
"threatens_chargeback_or_legal": { ... }, # noul
"target_queue": { ... }, # choice (billing_ops, fulfillment, security_trust, ...)
"selected_mcp_tool": { ... }, # choice (issue_instant_credit, quarantine_security_event, ...)
"churn_risk": { ... }, # score (0..3: Low -> Severe)
"abuse_toxicity": { ... }, # score (0..2: Benign -> Hostile)
}
result = client.decisions.create(
model="gemini-decision-1",
input=json.dumps(event), # Shared input — billed once
questions=EIGHT_PARALLEL_QUESTIONS, # All 8 questions scored in parallel
)
a = result.answers
Step 3: Combine the Answers with Clear Priority Rules in Python
Because each question returns its own independent score inside result.answers, our script checks the highest-priority safety risks first (prompt injection and leaked secrets) before allowing any automated refund or tool call:
if a["prompt_injection_attempt"].noul >= 0.70:
gate_verdict = "QUARANTINE_ADVERSARIAL_INPUT"
elif a["contains_raw_pii_or_secrets"].noul >= 0.75:
gate_verdict = "REDACT_PII_BEFORE_STORAGE"
elif a["threatens_chargeback_or_legal"].noul >= 0.75 or a["churn_risk"].score >= 2.3:
gate_verdict = "ROUTE_TO_VIP_RETENTION_SPECIALIST"
elif a["eligible_for_instant_refund"].noul >= 0.85 and a["selected_mcp_tool"].confidence >= 0.80:
gate_verdict = "AUTO_EXECUTE_INSTANT_REFUND"
else:
gate_verdict = f"DISPATCH_{a['selected_mcp_tool'].choice.upper()}"
Run the script across three sample messages (a prompt-injection attempt, an accidentally pasted sk_live_… API key, and a clean $34 damaged-mug refund):
python scripts/parallel_guardrails.py
Live Output:
[
{
"event_id": "evt-01-injection",
"gate_verdict": "QUARANTINE_ADVERSARIAL_INPUT",
"questions_evaluated_in_parallel": 8,
"latency_ms": 900.24,
"effective_ms_per_question": 112.53,
"signals": {
"prompt_injection_prob": 0.9766,
"raw_pii_prob": 0.0426,
"instant_refund_prob": 0.0275,
"chargeback_or_legal_prob": 0.0129,
"target_queue": "security_trust",
"selected_mcp_tool": "quarantine_security_event",
"churn_risk_0_to_3": 0.3086,
"abuse_toxicity_0_to_2": 1.9867
},
"usage": {
"total_input_tokens": 977,
"total_output_tokens": 0,
"estimated_cost_usd": 9.77e-05
}
},
{
"event_id": "evt-02-pii-leak",
"gate_verdict": "REDACT_PII_BEFORE_STORAGE",
"questions_evaluated_in_parallel": 8,
"latency_ms": 705.85,
"effective_ms_per_question": 88.23,
"signals": {
"prompt_injection_prob": 0.0548,
"raw_pii_prob": 0.9755,
"instant_refund_prob": 0.0292,
"chargeback_or_legal_prob": 0.0204,
"target_queue": "security_trust",
"selected_mcp_tool": "quarantine_security_event",
"churn_risk_0_to_3": 1.4901,
"abuse_toxicity_0_to_2": 1.0422
},
"usage": {
"total_input_tokens": 999,
"total_output_tokens": 0,
"estimated_cost_usd": 9.99e-05
}
},
{
"event_id": "evt-03-clean-refund",
"gate_verdict": "AUTO_EXECUTE_INSTANT_REFUND",
"questions_evaluated_in_parallel": 8,
"latency_ms": 654.39,
"effective_ms_per_question": 81.8,
"signals": {
"prompt_injection_prob": 0.0133,
"raw_pii_prob": 0.01,
"instant_refund_prob": 0.9676,
"chargeback_or_legal_prob": 0.01,
"target_queue": "billing_ops",
"selected_mcp_tool": "issue_instant_credit",
"churn_risk_0_to_3": 0.9037,
"abuse_toxicity_0_to_2": 0.071
},
"usage": {
"total_input_tokens": 987,
"total_output_tokens": 0,
"estimated_cost_usd": 9.87e-05
}
}
]
Evaluating all 8 checks together cost ~$0.000099 with 0 output tokens.
Example 3: Checking Photos and PDFs Without OCR (scripts/multimodal_triage.py)
Normally, when a user submits a claim with an attachment like a facility inspection photo or a PDF invoice, your backend has to run a separate OCR or vision pipeline first just to extract text before your routing rules can run.
Because gemini-decision-1 is built natively on Gemini's multimodal foundation, we can skip separate OCR completely and send both the written claim and the raw attachment together in one decisions.create call.
What We Are Going to Test
scripts/multimodal_triage.py tests two scenarios—each pairing a short written claim with an attachment:
Scenario 1: Facility Inspection Photo (Run 1a at low resolution vs. Run 1b at high resolution)
- User’s written claim: “Breakroom counter check before 9 AM client meeting. Small coffee spill spotted next to wiping cloth.”
- Attached photo (spill.jpg): Shows a small dark coffee spill on a light counter with a wiping cloth sitting right next to it
Scenario 2: Duplicate-Billed PDF Invoice (Run 2, raw PDF bytes, no OCR)
- User’s written claim: “I was double-charged $45 twice ($90 total) on invoice INV-8812. Attached is my PDF invoice."
- Attached PDF (sample_invoice_8812.pdf): A 1-page invoice listing Line 1: Pro Plan Monthly Subscription … $45.00 and Line 2: Pro Plan Monthly Subscription [DUPLICATE] $45.00 ($90.00 total).
For each run, gemini-decision-1 checks both the text and the file in a single pass and answers three questions:
What We Expect to See
Before running the script, here is what we are testing for:
- Spill photo (low vs. high resolution): Both runs should spot the small counter spill and give the same answer, while low resolution uses far fewer image tokens.
- PDF invoice (no OCR): Attaching sample_invoice_8812.pdf proves the customer really was billed twice, so the model should read the PDF directly and approve the refund.
Let’s see the code
Let’s look at the three key parts of scripts/multimodal_triage.py:
1. Defining Cross-Modal Questions (MULTIMODAL_QUESTIONS)
Notice how evidence_supports_claim asks the model to cross-check the written claim against the visual photo or PDF:
MULTIMODAL_QUESTIONS = {
"evidence_supports_claim": {
"type": "noul",
"instructions": "Does the attached visual image or PDF document directly corroborate the user's written claim?",
"criteria": {
"true_description": "The image or PDF clearly shows the spill, damage, or duplicate charge described in the claim text.",
"false_description": "The attachment contradicts the claim text, is unrelated, or lacks visible evidence.",
},
},
"disposition": {
"type": "choice",
"instructions": "Select the best operational action based on both the written claim and the attached evidence.",
"criteria": {
"automated_credit_or_replacement": "Verified duplicate charge or clear product defect eligible for immediate fulfillment.",
"dispatch_urgent_facilities": "Major hazard or severe spill requiring professional facilities dispatch.",
"local_cleanup_or_self_service": "Minor localized issue (such as a small counter spill with a cloth nearby) handled locally.",
"manual_adjuster_review": "Evidence is ambiguous, contradictory, or high-value requiring human review.",
"other": "Does not fit any disposition above.",
},
},
"severity": {
"type": "score",
"instructions": "Rate the physical or financial severity shown in the evidence.",
"criteria": [
"Negligible: Clean surface or valid single-charge invoice.",
"Minor: Small localized spill or minor cosmetic blemish.",
"Moderate: Clear duplicate invoice charge or functional item damage.",
"Severe: Hazardous condition or major multi-item loss.",
],
},
}
Passing Text + Image or PDF Blocks in input=[…]
Instead of passing a single string to input, we pass a list of content blocks, starting with a text block (ending in \n so blocks don't run together), followed by either an image block or a document (application/pdf) block:
# For Run 1a & 1b: Pass an image URL and set resolution ("low" vs. "high")
image_block = {
"type": "image",
"uri": "https://storage.googleapis.com/generativeai-downloads/images/spill.jpg",
"mime_type": "image/jpeg",
"resolution": "low", # or "high" in Run 1b
}
# For Run 2: Read raw PDF bytes from disk and base64-encode them as a "document" block
# (Note: PDF document blocks do not take a "resolution" setting)
pdf_block = {
"type": "document",
"data": base64.b64encode(pdf_bytes).decode("ascii"),
"mime_type": "application/pdf",
}
result = client.decisions.create(
model="gemini-decision-1",
input=[
{"type": "text", "text": f"Claim Statement: {claim_text}\nAttached Evidence:\n"},
media_block, # either image_block or pdf_block
],
questions=MULTIMODAL_QUESTIONS,
)
3. Inspecting Token Spend by Modality (input_tokens_by_modality)
To see the exact token difference between resolution="low" and resolution="high", the script reads result.usage.input_tokens_by_modality, which breaks input tokens down into "text" vs. "image" (PDF page tokens also report under "image"):
tokens_by_modality = {}
for item in result.usage.input_tokens_by_modality:
tokens_by_modality[str(item.modality)] = int(item.tokens or 0)
Run the Script & Check the Results
python scripts/multimodal_triage.py
Live Output:
[
{
"experiment": "1a_photo_resolution_low",
"resolution": "low",
"latency_ms": 3804.83,
"evidence_supports_claim_prob": 0.9532,
"disposition": "local_cleanup_or_self_service",
"disposition_confidence": 0.9863,
"severity_expected_0_to_3": 1.0319,
"severity_top_level": "Minor: Small localized spill or minor cosmetic blemish.",
"tokens_by_modality": {
"text": 441,
"image": 266
},
"total_input_tokens": 707,
"total_output_tokens": 0,
"estimated_cost_usd": 7.07e-05
},
{
"experiment": "1b_photo_resolution_high",
"resolution": "high",
"latency_ms": 2239.49,
"evidence_supports_claim_prob": 0.9154,
"disposition": "local_cleanup_or_self_service",
"disposition_confidence": 0.9825,
"severity_expected_0_to_3": 1.0471,
"severity_top_level": "Minor: Small localized spill or minor cosmetic blemish.",
"tokens_by_modality": {
"text": 441,
"image": 1064
},
"total_input_tokens": 1505,
"total_output_tokens": 0,
"estimated_cost_usd": 0.0001505
},
{
"experiment": "2_pdf_duplicate_invoice_zero_ocr",
"resolution": null,
"latency_ms": 676.98,
"evidence_supports_claim_prob": 0.9578,
"disposition": "automated_credit_or_replacement",
"disposition_confidence": 0.9367,
"severity_expected_0_to_3": 1.9849,
"severity_top_level": "Moderate: Clear duplicate invoice charge or functional item damage.",
"tokens_by_modality": {
"text": 587,
"image": 520
},
"total_input_tokens": 1107,
"total_output_tokens": 0,
"estimated_cost_usd": 0.0001107
}
]
Two quick takeaways from the output:
- low resolution used 4x fewer image tokens (266 vs. 1,064): Both runs gave the exact same answer (local_cleanup_or_self_service at >98% confidence). Save "high" or "ultra_high" for tiny text or fine print.
- The PDF invoice approved the refund with zero OCR: With sample_invoice_8812.pdf attached, evidence_supports_claim_prob jumped to 0.9578 and the model picked automated_credit_or_replacement in ~677 ms.
Practical Tips When Using the Decisions API
As you adapt these scripts to your own workloads, here are some tips to keep in mind:
- Never check noul > 0.99: noul probabilities are clamped to [0.01, 0.99], so 0.99 is the ceiling (> 0.99 can never be true). Use thresholds like >= 0.80 or >= 0.90, and add optional criteria={"true_description": "…", "false_description": "…"} when you want to clarify edge cases.
- Use "other" instead of "none_of_the_above" in choice: Option keys are sorted alphabetically in UTF-8 order before scoring, which places "none_of_the_above" ahead of keys like "shipping". Using "other" avoids confusing positional wording.
- Keep score scales evenly spaced: Ordered levels like Low, Medium, High, Critical (0..3) keep the weighted average score easy to interpret, whereas exponential jumps (1, 10, 100) skew the average.
- Keep questions independent and combine them in Python: Questions in the same call cannot see each other — ask simple, single-purpose questions and combine the answers with normal if/elif code.
- End adjacent text blocks with \n: When passing multiple blocks in input=[…], adjacent text blocks join with no automatic space or newline. Ending text blocks with \n keeps words cleanly separated.
What’s Next?
With three short Python scripts, you just used the Gemini Decisions API to route tickets, run 8 policy checks at once, and verify photos and PDF invoices, all with zero output tokens.
In a real backend service, though, spotting that a ticket is a live outage or a tricky edge case is only half the job. Something still needs to investigate those hard cases and draft a helpful troubleshooting plan or reply. We will look at combining the Decision API with other Gemini models, to address cases of various complexities.
Getting Started with the Gemini Decisions API in Python was originally published in Google Cloud – Community on Medium, where people are continuing the conversation by highlighting and responding to this story.
Source Credit: https://medium.com/google-cloud/getting-started-with-the-gemini-decisions-api-in-python-d2b40093034f?source=rss—-e52cf94d98af—4
About the Author
