New BigQuery gen AI functions for better data analysis

Next gen text and structured generation functions in GA 

The next generation of BigQuery gen AI functions — AI.GENERATE and AI.GENERATE_TABLE, previously in preview, are now in GA. With these new functions, BigQuery’s generative AI inference capabilities let you:

• Analyze any type of data: The new functions accept any type of input — text, images, video, audios and documents.

• Accomplish most major AI/ML tasks: Provide the prompt of what you desire the LLM to do and perform extraction, translation, summarization, sentiment analysis etc. tasks with ease.

• Use AI anywhere in your SQL: These functions are fully composable and can be placed anywhere standard SQL functions can go: in the SELECT statement, WHERE clause, and ORDER BY clause, allowing for sophisticated and flexible data processing.

• Generate structured output: Convert your unstructured data to structured insights by specifying your desired output_schema.

AI.GENERATE is excellent for free-form text generation, which is useful for a wide range of generic LLM tasks such as summarization, translation, sentiment analysis, and more, all from a simple user prompt. 

Additionally, AI.GENERATE can also generate structured output. By using the output_schema parameter, you can define the names and types of output fields, making the results immediately parseable and ready for use in downstream applications.

Furthermore, by specifying descriptive output field names like “sentiment” or “summarize_in_one_sentence”, AI.GENERATE can accomplish multiple AI tasks with a single function call, returning the results in multiple, easily consumable columns.

We illustrate this below using two examples. The first example uses text data in the bigquery-public-data.bbc_news.fulltext table. A single AI.GENERATE call simultaneously performs five tasks: 1) key entity extraction; 2) topic modeling; 3) sentiment analysis; 4) translation; and 5) summarization.