Introducing BigQuery managed and SQL-native inference for open models

BigQuery provides access to a variety of LLMs for text and embedding generation, including Google’s Gemini models, Google-managed models from partners like Anthropic and Mistral. Using Gemini models and Google-managed partner models in BigQuery is simple — just create the model with the foundation model name and run inference directly in SQL queries. Today, we are bringing this same simplicity and power to any model you may choose from Hugging Face or Vertex AI Model Garden.

A SQL-native workflow with automated management

With the launch of managed third-party generative AI inference in BigQuery (Preview), you can now run open models using just two SQL statements. 

This new capability delivers four key benefits:

1. Simplified deployment: Deploy open models using a single CREATE MODEL SQL statement with the model id string (e.g., google/gemma-3-1b-it). BigQuery automatically provisions the compute resources with default configurations.
2. Automated resource management: BigQuery automatically releases idle compute resources, preventing unintended costs. You can configure idle time via endpoint_idle_ttl.
3. Granular resource control: You can customize backend computing resources (like machine types and min/max replicas) directly within your CREATE MODEL statement to meet your performance and cost needs.
4. Unified SQL interface: The entire workflow — from model creation and inference to cost management and cleanup — is managed directly in BigQuery using SQL.

How it works: A practical example

Let’s take a look at the process of creating and utilizing an open model.

Step 1: Create a BigQuery managed open model
To use an open model from Hugging Face or Vertex AI Model Garden, use a CREATE MODEL statement along with the open model ID. It typically takes a few minutes for the query to complete, depending on the model size and machine types.

Hugging Face models
Specify the option hugging_face_model_id in the format of provider_name/model_name. For example, sentence-transformers/all-MiniLM-L6-v2 .