Tying together the latest Gemini model family with the toughest QA tester in the house
Hey there! Over the past few months, most of my writing has focused on the practical, grown-up side of software engineering: automating morning bug triage with Antigravity 2.0 scheduled tasks, passing ISO 27001 audits with local OWASP scans, and wiring Gemini models into the Microsoft Agent Framework.
Recently, as a Google Developer Expert (GDE) for AI/ML and Cloud, I was given the opportunity to preview Gemini 4 Argon.
I will follow up soon with a dedicated, in-depth analysis of Gemini 4 Argon’s core capabilities and benchmarks. For this first piece, however, I wanted to do something different: use Gemini 4 Argon as my pair programmer to take a wild idea from scratch to playable, and tie together all the recent models across the Gemini family (structured reasoning, fast image generation, expressive neural TTS, and audio transcription) in a single application.
To keep the test honest, I picked a stakeholder with zero patience for latency, bugs, or excuses: my four-year-old daughter, Marina.
At home, one of our favourite pretend games is cooking “magic soup” together. One of us is the hungry guest, and the other is the chef throwing ridiculous imaginary ingredients into a pot (a monster truck, a smelly sock, a hammerhead shark, or Dad’s tie). We stir the imaginary pot while chanting “MIX, MIX, MIX!”, take a big spoonful, and act out an over-the-top reaction: freezing into an ice cube, breathing fire, or floating like a balloon.
I wanted to bring our pretend kitchen to life on screen so we could play it together using a webcam and our voices. Along the way, we stress-tested Gemini 4 Argon across real-time Canvas physics, Web Audio synthesis, MediaPipe hand tracking, and the late-2026 Gemini 3.x runtime stack.
Here is how we built Silly Chef: The Magic Cauldron, the surprising SDK quirks we uncovered, and what happens when your four-year-old Lead Designer decides to change the product roadmap mid-sprint.
https://medium.com/media/3c9711566d198d5b606a1563225e3e4f/href
📦 Source Code: The full code, Canvas physics setup, and Gemini pipelines are available on GitHub at https://github.com/Tetsuoryuu/ToddlerGames/.
Let’s dive in.
Designing for a 4-Year-Old: Why Traditional UI Fails
When I opened Antigravity, switched my model selector to Gemini 4 Argon, and started brainstorming the architecture, the first problem we had to solve had nothing to do with AI. It was human ergonomics.
Standard web games fail four-year-olds in two ways:
- Fine-Motor Friction: A toddler cannot reliably use a mouse to drag and drop objects, click small buttons, or read text menus.
- Rigid Failure States: Timers, “game over” screens, or missed jumps cause immediate frustration.
To make the game feel like our living-room pretend play, we settled on three core design rules:
- The AR Mirror Stage: Instead of controlling an avatar with keys, the game mirrors the webcam feed (scaleX(-1)) inside an illustrated kitchen. Using @mediapipe/tasks-vision (HandLandmarker) with a lightweight frame-differencing motion fallback, Marina's real hands appear on screen trailing sparkles, acting as both magic wands and wooden spoons.
- Voice-First Creation: To summon a dinner guest or invent an ingredient, she just shouts it out loud. Everything generated during a session is saved to disk (pantry.json and PNG stickers in public/pantry/) and placed on visual shelves on the left and right of the screen. If she wants to play with a favourite character again without speaking, she can simply reach out and tap it on the shelf.
- Zero-Failure “Toddler-Assist” Physics: In src/game/physics.ts, swatting a floating ingredient bubble from any angle applies an assisted velocity vector toward the cauldron mouth. Once an ingredient enters the lower half of the screen, a magnetic funnel gently pulls it toward the center of the pot. And if a bubble misses the pot and hits the bottom of the screen, it never disappears or counts as a miss; it bounces back up in an elastic arc aimed right back at the cauldron rim.
Building the Multimodal Game Loop (And Cutting Latency from 9.8s to 2.4s)
Notice the distinction in how the models are used here: I used Gemini 4 Argon inside Antigravity to architect, write, and debug the codebase, while the game itself orchestrates four specialized models from the Gemini 3.x family at runtime via the @google/genai SDK (v2.28.0).
One of the first things I noticed during the initial scaffolding was that even a brand-new coding model can occasionally default to older model IDs from its training data unless you explicitly ask it to review the tech stack. After nudging Argon to audit our runtime models against the current late-2026 lineup, we put together a benchmark script to measure end-to-end latency for spawning a custom ingredient.
Our first working prototype ran sequentially: it generated the structured JSON metadata with gemini-3.8-flash (~4.8s) and then generated the cartoon sticker with gemini-nano-banana-2.1 (~4.9s).
Total wait time: 9.78 seconds.
If you have ever stood next to a four-year-old who just shouted “Sparkly Dragon!” at a screen, you know that ten seconds might as well be ten years. By second four, she is already asking if the computer is broken.
By switching the JSON brain to gemini-3.5-flash-lite, switching live sticker generation to gemini-3.1-flash-lite-image, and executing both in parallel with Promise.all, we cut the total /api/spawn-ingredient time down to ~2.45 seconds:
- Fast Brain & Structured JSON (gemini-3.8-flash to gemini-3.5-flash-lite): Dropped structured JSON metadata generation from ~4.8s to ~600ms.
- Live Die-Cut Stickers (gemini-nano-banana-2.1 to gemini-3.1-flash-lite-image): Combined with Promise.all, this cut total ingredient spawn latency from 9.78s down to ~2.45s.
- Expressive Multilingual TTS (gemini-3.8-flash-lite-tts): Supports inline vocal tags (<laugh>, <gasp>, <sigh>), distinct prebuilt character voices (Puck, Kore, Fenrir, Charon), and native multi-language speech.
- Toddler Speech Fallback (gemini-3.5-transcribe): Runs alongside the browser's Web Speech API to catch soft or high-pitched toddler speech across languages when the browser recognizer drops out.
And how do you hide the remaining 2.4 seconds when summoning a brand-new character? You turn the loading state into gameplay.
Instead of showing a spinner, the game spawns a shimmering Magic Cloud at the top of the webcam mirror and asks the child to “Wave your hands on the Magic Cloud to wake up your friend!” By the time she finishes waving her arms to charge the cloud with sparkles, the PNG sticker and voice greeting are already sitting in memory, ready to pop out.
A Multilingual Playground: Practicing Italian and Portuguese
One of the best side effects of tying the whole game loop to the Gemini model family is that the game is naturally multilingual.
Because gemini-3.5-flash-lite, gemini-3.5-transcribe, and gemini-3.8-flash-lite-tts handle multiple languages natively without needing separate voice packs or translation pipelines, the game can be played in any language. Even better, gemini-3.8-flash-lite-tts keeps the same expressive character personas and inline vocal tags (<laugh>, <giggle>, <gasp>) regardless of the language it is speaking.
For our family, this turned a weekend project into something genuinely special: Marina can use the game to practice her Italian and Portuguese alongside English. Whether she shouts an ingredient in English, Italian, or Portuguese, the brain model understands what she meant, generates the matching cartoon sticker, and has the hungry guest react in that language.
Under the Hood: 5 Technical Quirks That Caught Us by Surprise
Building a multimodal app that juggles real-time STT, structured JSON, image generation, and neural TTS in a tight loop exposed several non-obvious quirks in the SDK and model APIs. If you are building interactive apps on the @google/genai stack, keep these in mind.
[PAIN POINT] 1. thinkingConfig: { thinkingBudget: 0 } Slowed Down gemini-3.5-flash-lite
To squeeze every millisecond out of our JSON calls, we initially passed thinkingConfig: { thinkingBudget: 0 } into our generateStructuredJson helper, assuming it would disable reasoning overhead across all models.
Instead, our logs showed /api/validate-speech taking ~2.1s.
When we inspected the network trace, we found the culprit: while gemini-3.8-flash accepts thinkingConfig, gemini-3.5-flash-lite is a non-thinking model and rejects thinkingConfig with 400 INVALID_ARGUMENT. Because our helper caught the error and gracefully fell back to gemini-3.8-flash, every single request was failing on flash-lite, paying the network penalty of a 400 error, and running on the heavier model anyway.
Removing thinkingConfig when calling gemini-3.5-flash-lite immediately dropped our validation and metadata calls from 2.1s to 600ms.
[PAIN POINT] 2. Double WAV Headers (RIFF) in gemini-3.8-flash-lite-tts
In earlier Gemini 2.5 TTS versions, the API returned raw, headerless 24kHz 16-bit mono PCM bytes, which meant you had to prepend a 44-byte RIFF/WAVE header on your server before the browser could play it.
When we upgraded to gemini-3.8-flash-lite-tts, we noticed a tiny audible click at the very start of every voice line. It turns out gemini-3.8-flash-lite-tts returns a complete audio/wav binary that already begins with a RIFF header. By prepending our own 44-byte header, we were forcing the browser to play the original RIFF….WAVEfmt ASCII bytes as audio samples!
A simple four-byte check in server/gemini.ts solved it:
const rawBuffer = Buffer.from(base64Audio, 'base64');
// gemini-3.8-flash-lite-tts already includes a RIFF/WAVE header
const wavBase64 =
rawBuffer.subarray(0, 4).toString('ascii') === 'RIFF'
? base64Audio
: pcm24kToWavBase64(rawBuffer);
[PAIN POINT] 3. models.generateContent vs. interactions.create Latency for Images
The @google/genai SDK currently offers two surfaces that can invoke image generation on gemini-3.1-flash-lite-image: the newer ai.interactions.create endpoint and the standard ai.models.generateContent call with responseModalities: ['IMAGE'].
In our benchmarks, ai.interactions.create consistently took ~4.9s, whereas calling ai.models.generateContent with responseModalities: ['IMAGE'] (which skips text token generation entirely) returned the PNG in ~2.4s to 3.0s.
const response = await ai.models.generateContent({
model: 'gemini-3.1-flash-lite-image',
contents: fullPrompt,
config: {
responseModalities: ['IMAGE'],
},
});
[PAIN POINT] 4. Reading Transcripts from gemini-3.5-transcribe
If you call the convenience getter response.text on a gemini-3.5-transcribe response, the SDK logs a warning: there are non-text parts audioTranscription in the response, returning concatenation of all text parts.
To extract the transcription cleanly without console noise, inspect the candidate parts directly and read part.audioTranscription?.text:
const parts = response.candidates?.[0]?.content?.parts || [];
const transcript = parts
.map((p) => p.audioTranscription?.text || p.text || '')
.join(' ')
.trim();
[Bonus Tip] 5. Flood-Fill Chroma-Keying and the “16:9 White Strip” Trap
If you ask an image generation model for a “transparent background”, you will very often get a grey-and-white checkerboard pattern baked into the PNG.
Instead of fighting the prompt, we instruct gemini-3.1-flash-lite-image to render a "cute 2D cartoon die-cut sticker with a thick white border isolated on a solid pure white (#FFFFFF) background". On the client (src/game/stickerUtils.ts), we run a 4-edge flood-fill pass on an offscreen canvas that turns the outer #FFFFFF background transparent while preserving the thick white sticker border around the character.
Even then, we hit a subtle visual bug on our side shelves: gemini-3.1-flash-lite-image outputs 16:9 landscape images where the centered sticker only occupies about 25% of the total width. When we initially bound the shelf cards directly to the raw PNGs, each square card rendered a tiny 22-pixel sticker inside a wide horizontal white strip.
We updated removeOuterBackgroundAndCrop() in src/game/stickerUtils.ts so that right after the flood-fill transparency pass, it scans all remaining opaque pixels (alpha > 25) for their tight (minX, minY, maxX, maxY) bounding box and scales the cropped subject to fill 95% of a 512×512 square canvas. That gave us crisp, oversized 3D toy medallions both on the shelves and inside the floating bubbles.
Playtesting with a 4-Year-Old: Where Theory Meets Reality
Once the core loop was running, I called Marina over for the first live playtest. Watching a four-year-old interact with a voice-and-camera app will humble you faster than any production incident.
If you look at our local data/pantry.json file from those early sessions, it reads like a comedy log of everything we got wrong on day one:
- Before we muted the microphone during TTS playback, the game greeted her with “Hi there, Little Chef! What is your name?”, heard its own speaker output, and proudly welcomed “Chef There Little”.
- When I tested the mic myself, browser speech recognition decided that “Saverio” was clearly “Chef Celerio”.
- Best of all, when the hungry puppy character asked Marina what she wanted to put in the soup, she looked at the screen, paused, and said: “Umm… great… wait, you say it!”
Standard Web Speech API (rec.continuous = false) heard the pause after "umm", fired onresult, and spawned a floating bubble containing a custom sticker for "You Say It" (right next to "Chatter Word Bubble" and "Dad's Silly Tie").
Here is how Gemini 4 Argon and I fixed the toddler UX across four rapid iterations.
1. The Two-Stage Toddler Intent Filter (src/audio/speechInput.ts)
Adults speak to voice assistants in clean commands. Four-year-olds think out loud. They stall, giggle, repeat the prompt, or ask their parents to pick for them.
We switched SpeechRecognition to continuous = true with a patient 25-second auto-extending window, and built a two-stage filter before accepting any voice input:
- Stage 1 (0ms Regex Filter): A local check (isPureFillerOrHesitation) immediately catches pure filler and meta-chatter ("umm", "wait", "great", "you say it", "I don't know"). Instead of closing the mic, it keeps listening and updates the UI banner to: "Heard 'wait' – take your time, I'm still listening!"
- Stage 2 (~600ms Semantic Filter): Once a real phrase comes through, /api/validate-speech asks gemini-3.5-flash-lite whether the utterance actually contains a guest character or food/silly object for the current phase, stripping out surrounding filler (so "umm wait rainbow marshmallow please" cleanly extracts "Rainbow Marshmallow").
2. Speculative Prefetching (Because Comedy Needs 0ms Timing)
During our second playtest, we hit another snag: when Marina pushed the giant magic spoon into the guest’s mouth to taste the soup, there was a three-second pause while /api/evaluate-soup and /api/tts ran on the server. In slapstick comedy, a three-second delay between eating spicy soup and breathing fire ruins the joke.
We eliminated the wait using speculative execution (src/main.ts and src/audio/speechOutput.ts):
- While an ingredient bubble is still floating: The moment the sticker spawns, we immediately call speechOut.prefetch(ingredient.characterComment) in the background. When she swats the bubble into the cauldron, the splash reaction audio plays in 0ms straight from the in-memory Map cache.
- While she is stirring the cauldron: As soon as the third ingredient splashes into the pot, the child has to stir the soup for several seconds while chanting “MIX, MIX, MIX!”. In the background, preEvaluateSoup() immediately evaluates the 3-ingredient recipe and prefetches the WAV audio for both the taste-test reaction and the finale. When the spoon touches the character's mouth, the animation and voice fire on the exact frame of contact.
3. Monotonic speakToken and the 950ms Spoon Grace Period
Fixing latency exposed a classic race condition: “The soup is ready, I serve it to the guest before the narrator finishes telling me what to do, and both voices talk over each other.”
Two things were happening at once:
First, during the stirring phase, her hands were already waving right over the cauldron rim. On frame one of the FEEDING_SPOON phase, the spoon popped out of the pot right into her moving hand and launched into the character's mouth before the narrator even finished saying "The magic soup is ready!" We fixed this in src/game/physics.ts by adding a 950ms entrance grace period (spoonReady) while the spoon rises and pulses with a golden aura, giving her time to see it and intentionally push it toward the character.
Second, if an earlier speechOut.speak() call was still awaiting a network prefetch('/api/tts') when a new game phase called speechOut.stop(), the in-flight promise would resolve half a second later and start playing a second Audio element on top of the active voice.
We solved this in src/audio/speechOutput.ts with a monotonic speakToken counter:
export class SpeechOutputEngine {
private speakToken = 0;
async speak(text: string, options: SpeakOptions = {}): Promise<boolean> {
this.stop();
const myToken = ++this.speakToken;
const wavDataUrl = await this.prefetch(text, options);
// If stop() or another speak() was called while awaiting network TTS, abort!
if (myToken !== this.speakToken) {
return false;
}
// Play audio...
}
stop(): void {
this.speakToken++;
if (this.currentAudio) {
this.currentAudio.pause();
this.currentAudio = null;
}
}
}
The callers in src/main.ts checks const completed = await this.speechOut.speak(…); if (!completed) return;, guaranteeing that interrupted speech never triggers stale follow-up prompts.
4. Killing Developer Text Clutter for a Pure “Toybox” UI
As developers, our instinct is to put text labels under every icon, status subtitles under every header, and model badges in the top bar.
Watching Marina play made it obvious that all that text was dead weight. A four-year-old does not read UI labels; they just shrink the artwork and block her reflection in the AR mirror. Plus, removing English text labels makes switching between English, Italian, and Portuguese completely frictionless.
We stripped out the text banners under the floating bubbles, the spoon, and the shelf cards, replacing them with 108×108 glossy 3D sticker medallions (+150% artwork area), visual progress pills, and a tiny collapsible corner toggle for the parent keyboard fallback.
Scope Creep from a 4-Year-Old Stakeholder: “A Flying Train That Shoots!”
Once the cauldron game was polished (complete with a 24-node coupled-spring wave solver for the soup surface and 3.5x scaled characters for full-body webcam play across the room), my stakeholder changed the product roadmap.
Marina looked at me and asked: “Can we have a Flying Train… maybe a Flying Train that shoots?!”
When a four-year-old asks for “a flying train that shoots”, they are not asking for a stressful arcade shooter. They want big cause-and-effect, silly projectiles, and funny visual transformations.
This is where Gemini 4 Argon really showed its strength. Because Argon held the entire context of our existing architecture (the MediaPipe hand tracker, the patient toddler speech validator, the speakToken audio engine, the pantry.json store, and the 2.4-second gemini-3.1-flash-lite-image sticker pipeline), I asked it how we could translate her idea into a wholesome second game mode that reused our engine.
Within a single session, we turned the app into a two-mode arcade where she can switch between Magic Soup and Flying Train at any time:
- Silly Voice Cannon & Zero-Wait Hot-Swapping: The Flying Train has a brass Smokestack Cannon displaying a glowing medallion of the active ammo right on the barrel. When she shouts an ammo idea (“Giggly Grapes!”, “Hammerhead Shark!”, “Giggly Rubber Leg!”), the cannon immediately loads a glowing Mystery Ball (0ms wait) so she can start firing right away. As soon as gemini-3.1-flash-lite-image finishes generating the die-cut sticker in the background, physics.upgradeTrainAmmo() hot-swaps the active cannon ammo and any projectiles already flying through the air.
- Cross-Game Toybox: Every character summoned in the Magic Kitchen (like Killy the Bubbly Killer Whale or Bip-Bop the Robot) can hop into the train’s cab window as her Co-Pilot, and every ingredient invented for soup is waiting on the train’s ammo shelf.
- Cheering Up Grumpy Storm Clouds: Moving her hands on the webcam steers the train up and down the sky while firing silly ammo at rumbling grey Grumpy Clouds. Each cloud has a 3-star Cheer Meter: hit one turns it surprised lavender, hit two gives it blushing cheeks, and hit three transforms it into a smiling Happy Rainbow Cotton-Candy Cloud before a Giant Golden Pinata Cloud arrives for a bubble-popping finale.
[NOTE] Even in mode two, live playtesting caught a hilarious bug: on our first run, the clouds vanished almost immediately into a wall of 50 bubbles. Argon traced it in seconds to two root causes in src/game/physics.ts: our "touch a cloud directly with your hand" check was calling fireTrainProjectile() at 60 FPS with no cooldown, and both physics.ts and main.ts were calling spawnBurpBubbles() simultaneously when the boss cloud burst. Adding a 540ms hit cooldown, a two-pass semi-translucent cloud shader (globalAlpha = 0.72 so she can clearly see her reflection behind the clouds), and a 2-cloud active sky window made the pacing click immediately.
https://medium.com/media/9075f964a4dbd0b363db7377de1b0ce4/href
Reflections on Gemini 4 Argon (And What Is Next)
As I mentioned at the start, I will follow up this post with a dedicated, deep-dive article analyzing Gemini 4 Argon’s technical capabilities, reasoning behavior, and benchmarks in detail.
For this project, using Argon to glue together the recent technologies of the Gemini family into a real-time game highlighted two things:
- Translating Human Symptoms into Systems Fixes: I never had to tell Argon how to fix the audio race condition, the 16:9 sticker cropping, or the cloud physics pacing. I could describe what happened in plain parent observations (“the reaction to the food is too delayed”, “clouds are too dense and immediately become bubbles”, “wait until you hear an actual food ingredient and not interjections like umm, wait, you say it”) and it traced the symptom across the Express backend, the Web Audio state machine, and the 60 FPS Canvas loop in one pass.
- Full-Stack Cohesion: Writing custom 2D Canvas shaders, Hooke’s law spring physics, flood-fill image processing, procedural Web Audio sound effects, and multimodal Gemini API orchestration usually requires a lot of context-switching. Argon kept all of those layers consistent without breaking existing features when we bolted on a whole second game mode.
Key Takeaways
- Match the Runtime Model to the Job: In the 2026 Gemini 3.x lineup, pairing gemini-3.5-flash-lite (~600ms) with gemini-3.1-flash-lite-image inside Promise.all cut our interactive asset generation from nearly 10 seconds to 2.45s.
- Multilingual by Default: Combining Gemini’s language models with gemini-3.8-flash-lite-tts and a label-free visual UI lets kids play in any language, which has been a fun way for Marina to practice her Italian and Portuguese.
- Watch Out for Model-Specific Config Quirks: Do not pass thinkingConfig to non-thinking models like gemini-3.5-flash-lite (it triggers a 400 INVALID_ARGUMENT), and check for native RIFF WAV headers on gemini-3.8-flash-lite-tts before wrapping audio bytes.
- Design Around Latency: Between the Magic Cloud waving ritual, mid-air sticker hot-swapping, and speculative TTS prefetching during the stirring phase, a four-year-old never feels like she is waiting on an API call.
Stay tuned for the follow-up deep dive on Gemini 4 Argon! As always, if you have questions, spot a bug, or want to share what you are building with the latest Gemini models, my DMs are open on Twitter (@TetsuoRyuu), BlueSky (@tetsuoryuu.bsky.social), and LinkedIn (@saverioterracciano). And if you are around London, keep an eye on GDG Cloud London for our upcoming community sessions!
Now, if you will excuse me, Marina is calling me back to the living room. Apparently, Bip-Bop the Robot needs a bowl of Hammerhead Shark and Giggly Rubber Leg soup, and we have a Flying Train to catch.
Happy coding!
Previewing Gemini 4 Argon by Building an AR Game for My 4-Year-Old was originally published in Google Cloud – Community on Medium, where people are continuing the conversation by highlighting and responding to this story.
Source Credit: https://medium.com/google-cloud/previewing-gemini-4-argon-by-building-an-ar-game-for-my-4-year-old-baf198896cd8?source=rss—-e52cf94d98af—4
About the Author
