My last post compared Jev to BigQuery’s AI functions to check accuracy, cost, and latency, scaling from one to ten million rows. Google recently launched the Gemini Decisions API, which runs on a new Jev-like model, Gemini Decision 1. It also returns confidence scores and answers instead of text. So I decided to rerun the same tests using this new model.
In this post, I’ll compare the Decisions API against Jev and BigQuery’s AI functions on accuracy, speed, and cost up to 100,000 rows. I’ll then test what the Decisions API is really built for: asking multiple questions about an item at scale. While writing, I ran into some surprises — like an early run that scored a suspiciously high 93.7% because of how I bunched rows and how BigQuery ordered my table.
These are personal experiments on public data. They are not official Google benchmarks.
What’s the Decisions API?
Put simply, you send the Decisions API an input and a typed set of questions and it returns an answer for each one. The question types are eerily similar to what we found in Jev: noul (the probability something is true), choice (picks one category from a list), and score (assigns a score given a scale you define).
In my first Jev post, I built a table of 30 customer support tickets (e.g. billing issues, auth failures) and scored each one for whether it’s urgent, if it’s blocking work, and which team should own it. Here’s what a Decisions API request looks like for those tickets:
POST https://generativelanguage.googleapis.com/v1beta/decisions
{
"model": "gemini-decision-1",
"input": "Subject: Charged twice. Description: Our card was billed twice for November ...",
"questions": {
"route": {
"type": "choice",
"instructions": "Select the single team that should own the main issue ...",
"criteria": {
"billing": "Invoices, charges, refunds ...",
"identity": "Login, SSO, MFA ...",
"...": "..."
}
},
"urgent": {
"type": "noul",
"instructions": "Does the ticket establish a concrete need for prompt action ...?"
}
}
}
Some info about how I tested it:
- Pricing is only on input tokens. Input tokens cost $0.10 per million and answers don’t count as output tokens (no cost).
- One request can ask up to 1,000 questions about the same input.
- It’s only on the Gemini Developer API. BigQuery’s AI functions call Gemini models on Google Cloud, so they can’t call it directly, and there’s no batch mode. Instead, I reused the pattern from my first post — a BigQuery remote function that calls Cloud Run. (Remote functions are a great connector whenever BigQuery can’t call a model or API directly!)

To kick things off, I re-scored the 30 support tickets from my first post with one request per ticket. It took 2.6 seconds and identified the same four urgent, blocked tickets that Jev did.
How accurate is it?
I ran the Decisions API on the three text tests from my last post (IMDb movie reviews, AG News articles, and Stack Overflow questions), with the same rows, prompts, and categories, one row per request, through Cloud Run with up to four instances. The comparison is Jev and BigQuery’s AI functions in two modes. That includes standard mode (Gemini reads each row) and optimized mode (Gemini labels a sample of the data and trains a small model on embeddings to do the rest).

The Decisions API beat Jev on all three tasks. It also beat BigQuery’s standard mode on Stack Overflow, essentially tied it on IMDb, and trailed it on news topics (mostly on Sci/Tech articles only getting 72% right). You don’t have to pick just one classification method either. If you trust the Decisions API whenever its confidence is >=0.80 and only hand the uncertain rows to BigQuery standard mode, accuracy climbs to 89.20%, which beats standard mode on its own.
Note that safety filters blocked 234 of 100k IMDb reviews with an error. My adapter returned NULL for these rows.
Speed and cost as the rows grew

The Decisions API was fast for small queries (< 7 seconds for up to 1,000 rows), but it leveled off at about 270 rows per second: on average, 100,000-row runs took about 363 seconds. Scaling by increasing the Cloud Run instances from 4 to 16 only saved 24 seconds on that. The API was throttling me, sending back thousands of HTTP 429 (“too many requests”) errors that my adapter retried. That’s close to the 15,000 request per minute published limit. Jev finished the same 100,000 articles in 56 to 111 seconds.
For 100,000 news articles, the Decisions API cost about $2.06, compared with $4.10 for standard mode and $0.88 for Jev. Optimized mode is still the cheapest at scale — only about $0.08 — once you’ve generated the embeddings.
Packing many rows into one request
When testing Jev, I sent 100 rows per request. So I tried the same approach with the Decisions API: one input holding 100 articles, alongside one question per article. That run finished 100,000 articles in 115 seconds and scored 93.70%. It seemed too good to be true, so I took a look.
It turns out my BigQuery table was ordered by topic, so BigQuery passed rows to my remote function in the same order. Meaning that a typical batch of 100 articles was all Sports or all Business. To see how that affected accuracy, I ran the same 10,000 articles three separate ways:
- Batches of 100, sorted by topic as stored in the raw data
- Batches of 100, shuffled randomly
- One article per request

Sorted by topic, the Decisions API scored 93.49%. With random rows it scored 88.49%, and with one article per request just 87.97% — drastically lower than the sorted table. Jev, which also took 100 articles per request, barely moved: 87.0% random, 87.2% sorted.
Turns out this isn’t a bug — even though the API answers each question individually, each question reads the entire input. So an article’s label is influenced by the 99 adjacent rows. And randomizing the batches of 100 rows didn’t fix things either: reshuffling which articles landed together still changed about 5% of answers each run.
The API docs best practices actually address this. Unrelated content lowers accuracy, and the 99 adjacent articles in the batch are definitely unrelated. The API is designed to ask many questions about one item, not packing lots of items into one request. So stick to one item per request! Every other Decisions API metric in this post uses one row per request.
Many questions about one document
Since the Decisions API is meant for asking lots of questions about a single item, I wanted to test it on longer documents and PDFs too. I downloaded 100 arXiv research papers and asked 10 yes/no questions about each one. Things like “Does the paper link to source code?” and “Does the paper have exactly one author?”
My adapter combines each PDF with all 10 questions into a single Decisions API call, so you only pay for the PDF tokens once. Conversely, AI.IF only handles one question per call, so 10 questions means 10 PDF requests. To emulate a single-call option, I also tried the AI.GENERATE function with an output_schema to return all 10 fields at once. I skipped Jev here because it doesn’t accept PDFs.

Be careful how you structure the AI.GENERATE query. I first used (well, my agent first used) UNNEST([AI.GENERATE(…)]) AS g and read the output fields in the same SELECT. This ended with BigQuery calling the model 12 times(!) per paper (1,203 calls total, 165 seconds, $0.93) — basically once for each field I referenced (10 answers + 2 metadata fields). I refactored the query to save the whole result to a single column first, which cut things down to 101 total calls.
CREATE TABLE papers.rubric AS
SELECT paper_id,
AI.GENERATE(('Answer each yes/no question about this research paper. ...', pdf_ref),
output_schema => 'code_available BOOL, single_author BOOL, ...') AS answers
FROM papers.pdfs; -- pdf_ref is an ObjectRef column that points to each PDF in Cloud Storage
-- Then read answers.code_available, answers.single_author, and so on.

Most of the 10 questions I asked were subjective (like whether a paper discussed limitations), with no pre-labeled answer key. But I checked accuracy using two metrics I could easily verify: whether the PDF included a code link (e.g. GitHub, GitLab) and arXiv’s author count. The Decisions API was fastest and did the best on finding code links, while AI.GENERATE was cheapest, even though the Decisions API charges less per token.
Why? Well, the Decisions API used about 40,000 tokens per paper — almost triple AI.GENERATE. Standard Gemini calls appear to charge only for rendered page images, while the Decisions API usage counted both the page image and the extracted text. On the BigQuery side, the 10 AI.IF calls totaled nearly 8x the cost of one AI.GENERATE call. So if you need multiple answers per row, consider bundling them into one AI.GENERATE statement using the code sample above.
Try it yourself
As before, the best tool for your data might not be the one that scores best here. Try a sample on your data, compare the answers with labels you trust, and be cognizant of time and price as you scale up.
Wonder how these tools handle multimodal data? That’s what I’m testing next: photos, audio, video, and trickier documents. Here’s a sneak preview — can you tell which is which?

If you aren’t sure, don’t worry. Even birders mix these two up and the models were also confused by them. This post showed that text classification accuracy was generally close between Jev, the Decisions API, and BigQuery. But I found a 20+ point gap between the models with a sample of 1,000 photos of different types of sparrows. In the next post, I’ll share which sparrow is which and compare how the Decisions API, Jev, and BigQuery handle multimodal data like photos, audio, video, and more difficult PDFs (and where each stops working altogether).
If you’d like to try this yourself, the GitHub repo has the Cloud Run code I used for both Jev and Decisions API. You can also read the docs for Decisions API, AI.IF and AI.GENERATE.
Let me know what kind of data you’re thinking about classifying!
—
Note on AI use: AI was used to brainstorm article structure and run queries. Author wrote the text.

Calling Google’s Decisions API from BigQuery was originally published in Google Cloud – Community on Medium, where people are continuing the conversation by highlighting and responding to this story.
Source Credit: https://medium.com/google-cloud/calling-googles-decisions-api-from-bigquery-8ce7762c8907?source=rss—-e52cf94d98af—4
