7.10.1 Run MySQL HeatWave Chat

When you run MySQL HeatWave Chat, it automatically loads the default LLM.

By default, MySQL HeatWave Chat searches for an answer to a query across all ingested documents by automatically discovering available vector stores, and returns the answer along with relevant citations. You can limit the scope of search to specific document collections available in certain vector stores or specify documents to include in the search.

As of MySQL 9.2.1, MySQL HeatWave Chat lets you use your own embedding tables for context retrieval. And, it uses only the name of the embedding model used to embed the input query to find relevant tables. In earlier versions of MySQL, MySQL HeatWave Chat uses only inbuilt vector store tables for context retrieval. And, it uses both the language specified for generating the output and the name of the embedding model used to embed the input query to find relevant tables containing information in the same language and vector embeddings from the sameembedding model.

If you do not have a vector store or an embedding table set up, then MySQL HeatWave Chat uses information available in public data sources to generate a response for your query.

This topic contains the following sections:

Before You Begin

Review the MySQL HeatWave GenAI requirements.
To extend the vector search functionality and ask specific questions about the information available in your proprietary documents stored in the vector store, complete the steps to set up a vector store.
In this topic, the HEATWAVE_CHAT routine uses the vector store table demo_embeddings created in the section Ingesting Files Using the URI with Asynchronous Load for context retrieval.
To use your own embedding table for context retrieval, create a table that satisfies the following criteria:
- The table must contain the following columns:
  - A string column containing the text segments.
  - A vector column containing the vector embeddings of the text segments.
- A comment on the vector column must specify the name of the embedding model used to generate the vector embeddings.
- The vector embeddings in your embedding table must be from an embedding model supported by MySQL HeatWave. To view the list of available embedding models, see MySQL HeatWave In-Database Embedding Models and OCI Generative AI Service Embedding Models.
  Following is an example of a valid embedding table that can be used for context retrieval:
```
mysql> CREATE TABLE demo_table (id INT AUTO_INCREMENT, demo_text TEXT, primary key (id));
mysql> INSERT INTO demo_table (demo_text) VALUES('What is MySQL?');
mysql> INSERT INTO demo_table (demo_text) VALUES('What is HeatWave?');
mysql> INSERT INTO demo_table (demo_text) VALUES('What is HeatWave GenAI?');
mysql> CALL sys.ML_EMBED_TABLE('demo_schema.demo_table.demo_text', 'demo_schema.demo_table.demo_embedding',
JSON_OBJECT('model_id', 'all_minilm_l12_v2'));
```
  To learn how to generate vector embeddings and embedding tables using MySQL HeatWave GenAI, see Generating Vector Embeddings.
If you want to use both inbuilt vector store tables and your own embedding tables for context retrieval, your embedding table must satisfy the following additional requirements:
- Since the inbuilt vector store tables set up using Asynchronous Load or Auto Parallel Load use predefined column names, the column names in your embedding tables must match the predefined inbuilt vector store table column names as given below:
  - segment: name of the mandatory string column containing the text segments.
  - segment_embedding: name of the mandatory vector column containing the vector embeddings of the text segments.
  - document_name: name of the optional column containing the document names. This column can be of any data type supported by MySQL HeatWave.
  - document_id: name of the optional integer column containing the document IDs.
  - metadata: name of the optional JSON column containing metadata for the table.
  - segment_number: name of the optional integer column containing segment number.
- The vector embeddings in your embedding table must be from the same embedding model as the vector store table.

Run the Chat

To run MySQL HeatWave Chat, perform the following steps:

To delete previous chat output and state, if any, reset the @chat_options variable:
```
mysql> SET @chat_options=NULL;
```
Note
Ensure that you use the name @chat_options for the variable. The HEATWAVE_CHAT routine reserves this variable for specifying and saving various chat parameter settings.
Optionally, set the @chat_options variable in the following scenarios:
- To use a language other than English, set the language model option:
```
mysql> SET @chat_options = JSON_OBJECT("model_options", JSON_OBJECT("language", "Language"));
```
  Replace Language with the two-letter ISO 639-1 code for the language you want to use. Default language is en, which is English. To view the list of supported languages, see Languages.
  The language parameter is available as of MySQL 9.0.1-u1.
  For example, to use French set language to fr:
```
mysql> SET @chat_options = JSON_OBJECT("model_options", JSON_OBJECT("language", "fr"));
```
  This resets the @chat_options variable, and specifies the language for the chat.
- To use your own embedding tables for context retrieval, change the column names used by the HEATWAVE_CHAT routine to filter tables by setting the vector_store_columns parameter:
```
mysql> SET @chat_options = JSON_OBJECT(
  "vector_store_columns", JSON_OBJECT("segment", "TextSegmentColumnName", "segment_embedding", "VectorEmbeddingColumnName"), 
  "embed_model_id", "EmbeddingModelName"
);
```
  Replace the following:
  - TextSegmentColumnName: the name of the embedding table column that contains the text segments in natural language. If multiple tables contain a string column with the same name, they are all used for context retrieval. Default value is segment.
  - VectorEmbeddingColumnName: the name of the embedding table column that contains vector embeddings of the natural-language text segments. If multiple tables contain a vector column with the same name which contain embeddings from the specified embedding model, they are all used for context retrieval. Default value is segment_embedding.
  - EmbeddingModelName: the name of the embedding model to use to generate the vector embeddings for the input query. The routine uses this embedding model name to find tables generated using the same model for context retrieval. By default, the routine uses minilm if the output language is set to English and multilingual-e5-small if the output language is set to a language other than English.
  By default, the routine uses all the predefined vector store column names to filter tables for context retrieval.
  For example:
```
mysql> SET @chat_options = JSON_OBJECT(
  "vector_store_columns", JSON_OBJECT("segment", "demo_text", "segment_embedding", "demo_embeddings"), 
  "embed_model_id", "all_minilm_l12_v2"
);
```
  This resets the @chat_options variable to specify the column names used for filtering tables for context retrieval. In this example, all embedding tables containing a string column demo_text and a vector column demo_embeddings which contains vector embeddings from all_minilm_l12_v2 are used for context retrieval.
  However, since the inbuilt vector store tables use predefined column names, if you change a column name used for filtering tables to any value other than the default value, the inbuilt vector store tables are filtered out and are not used for context retrieval.
- As of MySQL 9.3.0, to use vector store tables created in earlier versions of MySQL for context retrieval, set the embedding model name to all_minilm_l12_v2:
```
mysql> SET @chat_options = JSON_OBJECT("embed_model_id", "all_minilm_l12_v2");
```
  This is because as of MySQL 9.3.0, MySQL HeatWave Chat uses vector store and embedding tables created using multilingual-e5-small for context retrieval by default.

Then, add your query to MySQL HeatWave Chat by using the HEATWAVE_CHAT routine:

mysql> CALL sys.HEATWAVE_CHAT("YourQuery");

For example:

mysql> CALL sys.HEATWAVE_CHAT("What is HeatWave AutoML?");

The output looks similar to the following:

+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| response                                                                                                                                                                                                                      |
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| HeatWave AutoML is a feature of MySQL HeatWave that makes it easy to use machine learning, allowing users to create optimized machine learning models for predictions and explanations without having to leave the database. |
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

Repeat this step to ask follow-up questions using the HEATWAVE_CHAT routine:

mysql> CALL sys.HEATWAVE_CHAT("What learning algorithms does it use?");

The output looks similar to the following:

+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| response                                                                                                                                                                                                                                                                                       |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| HeatWave AutoML uses a variety of machine learning algorithms, including decision trees, random forests, neural networks, and support vector machines (SVMs). The specific algorithm used depends on the characteristics of the data being analyzed and the goals of the model being created. |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

What's Next

Learn how to View Chat Session Details.