How to Build a Production-Grade RAG with ADK & Vertex AI RAG Engine via the Agent Starter Pack | Google Cloud

Retrieval-Augmented Generation (RAG) has become the foundation for enterprise-grade generative AI systems powering intelligent assistants, document search engines, and private knowledge bases. However, while building a RAG prototype is straightforward, turning it into a scalable, observable, and production-ready system requires a robust infrastructure and strong architecture backbone.

To accelerate the journey from concept to production, Google released the Agent Starter Pack, a pre-configured template that provides everything needed to build, test, and deploy production-grade agents on Google Cloud. Agent Starter Pack is designed precisely to bridge the gap between experimentation and production. It provides developers with a template foundation so they can focus on what truly matters: business logic, prompts, and tools, while the Starter Pack handles the rest:

• Deployment & Operations: API serving, Infrastructure-as-Code, and CI/CD pipelines.
• Observability: Centralized logging, tracing, and monitoring dashboards.
• Evaluation: Seamless integration with Vertex AI Evaluation for continuous quality assessment.
• Security: Built-in GCP IAM, data privacy, and compliance best practices.
• Data & UI: Connectors for data storage, vector databases, and ready-to-use UI playgrounds.

Agent Starter Pack is Google’s production blueprint for agentic AI systems, helping teams transition from an idea to a scalable, deployable RAG application in weeks, not months.

In this article, we’ll dive into how to set up, configure, and deploy a RAG app inside the Agent Engine using the Agent Starter Pack, with Google ADK and the Vertex AI RAG Engine.

High-Level Architecture

Press enter or click to view image in full size

In this article, we’ll narrow our focus to four core pillars of a production-grade RAG pipeline within the above architecture.

1. LLM Orchestration — Google ADK,
2. LLM — Gemini,
3. Deployment — Vertex AI Agent Engine, and
4. Retrieval & Grounding — Vertex AI RAG Engine

Vertex AI Agent Engine

Vertex AI Agent Engine, part of the Google Cloud Vertex AI Platform, offers a suite of services that enable developers to deploy, manage, and scale AI agents in production environments. It abstracts away the underlying infrastructure, allowing you to focus on building intelligent applications instead of managing runtime resources.

Agent Engine Services:

• Runtime: Deploy and scale agents with a managed runtime, customizable containers, built-in security, and integrated observability.
• Quality and Evaluation (Preview): Evaluate and optimize agent performance using the integrated Generative AI Evaluation service and Gemini model runs.
• Example Store (Preview): Store and dynamically retrieve few-shot examples to enhance agent accuracy and contextual relevance.
• Sessions (Preview): Persist user-agent interactions to maintain conversational context across sessions.
• Memory Bank (Preview): Store and recall information from sessions to enable personalized, memory-aware interactions.
• Code Execution (Preview): Execute custom code securely within an isolated, managed sandbox environment.

Press enter or click to view image in full size

Vertex AI RAG Engine

Vertex AI RAG Engine is also a core component of the Google Cloud Vertex AI Platform, designed to power Retrieval-Augmented Generation (RAG) workflows. It provides a fully managed vector store, seamless integration with Gemini models, and supports real-time retrieval and grounding. By enhancing large language model (LLM) outputs with relevant, contextually retrieved data, the RAG Engine enables developers to build intelligent, data-aware AI applications that deliver accurate, enterprise-grade responses. RAG Engine supports multiple vector databases and, by default, uses RagManagedDB backed by a Google Cloud Spanner instance

[ Basic Tier = 100 processing units and Scaled Tier = 1000 processing units for production workloads]

Implementation

Let us implement the RAG application and deploy it into Vertex AI using the Agent Starter pack.

Pre-Requisites

1. Python 3.10+ installed
2. GOOGLE_API_KEY https://aistudio.google.com/app/apikey

Step 1: Install the UV package manager

Ensure UV is installed.

# On macOS and Linux.
curl -LsSf https://astral.sh/uv/install.sh | sh

Step 2: Install Agent Starter pack using UVX

Authenticate with your GCP Project and create the rag agent demo

#Authenticate GCP login
gcloud auth login
#Set default GCP Project ID
gcloud config set project YOUR_PROJECT_ID

#Agent Starter Pack using rag Template identifier 
uvx agent-starter-pack create rag-agent-demo -a adk@rag

The “-a” flag specifies the Template identifier to use.

Here , we use adk@rag template, which fetches RAG template from google/adk-samples