5.9.1 Running GenAI Chat

When you run GenAI Chat, it automatically loads the llama3.2-3b-instruct-v1 LLM.

By default, GenAI Chat searches for an answer to a query across all ingested documents by automatically discovering available vector stores, and returns the answer along with relevant citations. You can limit the scope of search to specific document collections available in certain vector stores or specify documents to include in the search.

GenAI Chat also lets you use your own embedding tables for context retrieval. And, it uses only the name of the embedding model used to embed the input query to find relevant tables.

If you do not have a vector store or an embedding table set up, then GenAI Chat uses information available in public data sources to generate a response for your query.

This topic contains the following sections:

Before You Begin

Review the GenAI requirements.
To extend the vector search functionality and ask specific questions about the information available in your proprietary documents stored in the vector store, complete the steps to set up a vector store.
In this topic, the HEATWAVE_CHAT routine uses the vector store table demo_embeddings created in the section Ingesting Files into a Vector Store for context retrieval.
To use your own embedding table for context retrieval, create a table that satisfies the following criteria:
- The table must contain the following columns:
  - A string column containing the text segments.
  - A vector column containing the vector embeddings of the text segments.
- A comment on the vector column must specify the name of the embedding model used to generate the vector embeddings.
- The vector embeddings in your embedding table must be from an embedding model supported by GenAI. To view the list of available embedding models, see In-Database Embedding Model.
  Following is an example of a valid embedding table that can be used for context retrieval:
```
mysql> CREATE TABLE demo_table (id INT AUTO_INCREMENT, demo_text TEXT, primary key (id));
mysql> INSERT INTO demo_table (demo_text) VALUES('What is MySQL?');
mysql> INSERT INTO demo_table (demo_text) VALUES('What is HeatWave?');
mysql> INSERT INTO demo_table (demo_text) VALUES('What is HeatWave GenAI?');
mysql> CALL sys.ML_EMBED_TABLE('demo_schema.demo_table.demo_text', 'demo_schema.demo_table.demo_embedding',
JSON_OBJECT('model_id', 'all_minilm_l12_v2'));
```
  To learn how to generate vector embeddings and embedding tables, see Generating Vector Embeddings.
If you want to use both inbuilt vector store tables and your own embedding tables for context retrieval, your embedding table must satisfy the following additional requirements:
- Since the inbuilt vector store tables, use predefined column names, the column names in your embedding tables must match the predefined inbuilt vector store table column names as given below:
  - segment: name of the mandatory string column containing the text segments.
  - segment_embedding: name of the mandatory vector column containing the vector embeddings of the text segments.
  - document_name: name of the optional column containing the document names. This column can be of any data type supported by MySQL.
  - document_id: name of the optional integer column containing the document IDs.
  - metadata: name of the optional JSON column containing metadata for the table.
  - segment_number: name of the optional integer column containing segment number.
- The vector embeddings in your embedding table must be from the same embedding model as the vector store table.

Running the Chat

To run GenAI Chat, perform the following steps:

Optionally, to speed up vector processing, load the vector store or embedding tables that you want use with GenAI Chat in MySQL AI Engine:
```
mysql> ALTER TABLE TableName SECONDARY_LOAD;
```
Replace TableName with the name of the vector store table.
For example:
```
mysql> ALTER TABLE demo_db.demo_embeddings SECONDARY_LOAD;
```
This accelerates processing of vector distance function used to compare vector embeddings and generate relevant output later in this section.
To delete previous chat output and state, if any, reset the @chat_options variable:
```
mysql> SET @chat_options=NULL;
```
Note
Ensure that you use the name @chat_options for the variable. The HEATWAVE_CHAT routine reserves this variable for specifying and saving various chat parameter settings.
Optionally, set the @chat_options variable in the following scenarios:
- To use a language other than English, set the language model option:
```
mysql> SET @chat_options = JSON_OBJECT("model_options", JSON_OBJECT("language", "Language"));
```
  Replace Language with the two-letter ISO 639-1 code for the language you want to use. Default language is en, which is English. To view the list of supported languages, see Languages.
  For example, to use French set language to fr:
```
mysql> SET @chat_options = JSON_OBJECT("model_options", JSON_OBJECT("language", "fr"));
```
  This resets the @chat_options variable, and specifies the language for the chat.
- To use your own embedding tables for context retrieval, change the column names used by the HEATWAVE_CHAT routine to filter tables by setting the vector_store_columns parameter:
```
mysql> SET @chat_options = JSON_OBJECT(
  "vector_store_columns", JSON_OBJECT("segment", "TextSegmentColumnName", "segment_embedding", "VectorEmbeddingColumnName"), 
  "embed_model_id", "EmbeddingModelName"
);
```
  Replace the following:
  - TextSegmentColumnName: the name of the embedding table column that contains the text segments in natural language. If multiple tables contain a string column with the same name, they are all used for context retrieval. Default value is segment.
  - VectorEmbeddingColumnName: the name of the embedding table column that contains vector embeddings of the natural-language text segments. If multiple tables contain a vector column with the same name which contain embeddings from the specified embedding model, they are all used for context retrieval. Default value is segment_embedding.
  - EmbeddingModelName: the name of the embedding model to use to generate the vector embeddings for the input query. The routine uses this embedding model name to find tables generated using the same model for context retrieval. By default, the routine uses minilm if the output language is set to English and multilingual-e5-small if the output language is set to a language other than English.
  By default, the routine uses all the predefined vector store column names to filter tables for context retrieval.
  For example:
```
mysql> SET @chat_options = JSON_OBJECT(
  "vector_store_columns", JSON_OBJECT("segment", "demo_text", "segment_embedding", "demo_embeddings"), 
  "embed_model_id", "all_minilm_l12_v2"
);
```
  This resets the @chat_options variable to specify the column names used for filtering tables for context retrieval. In this example, all embedding tables containing a string column demo_text and a vector column demo_embeddings which contains vector embeddings from all_minilm_l12_v2 are used for context retrieval.
  However, since the inbuilt vector store tables use predefined column names, if you change a column name used for filtering tables to any value other than the default value, the inbuilt vector store tables are filtered out and are not used for context retrieval.

Then, add your query to GenAI Chat by using the HEATWAVE_CHAT routine:

CALL sys.HEATWAVE_CHAT("YourQuery");

For example:

mysql> CALL sys.HEATWAVE_CHAT("What is HeatWave AutoML?");

The output looks similar to the following:

| 
HeatWave AutoML is an automated machine learning (AutoML) platform that uses a combination of human-in-the-loop and autoML techniques to help users build, train, and deploy machine learning models without extensive technical expertise.

Here's a brief overview:

**Key Features:**

1. **Automated Model Selection**: HeatWave AutoML allows users to select the best-performing model for their specific problem from a range of pre-trained models.
2. **Hyperparameter Tuning**: The platform automatically tunes hyperparameters for the selected model, ensuring optimal performance and minimizing overfitting.
3. **Data Preprocessing**: HeatWave handles data preprocessing tasks such as feature engineering, normalization, and feature selection.
4. **Model Training**: The platform trains the selected model on the user's dataset and provides real-time feedback and guidance throughout the process.
5. **Model Deployment**: Once a model is trained, HeatWave AutoML deploys it to a cloud-based environment for easy integration with various applications.

**Benefits:**

1. **Reduced Time-to-Insight**: Automates the entire machine learning workflow, saving users time and effort.
2. **Improved Model Performance**: HeatWave's automated process ensures that models are optimized for performance and accuracy.
3. **Increased Collaboration |

Repeat this step to ask follow-up questions using the HEATWAVE_CHAT routine:

mysql> CALL sys.HEATWAVE_CHAT("What learning algorithms does it use?");

The output looks similar to the following:

| 
HeatWave is an AutoML (Automated Machine Learning) platform that uses a combination of various machine learning algorithms to automate the process of building, training, and deploying machine learning models. While I couldn't find specific information on the exact algorithms used by HeatWave, I can provide some general insights into the types of algorithms that are commonly used in AutoML platforms like HeatWave.

HeatWave is built on top of several popular open-source libraries and frameworks, including:

1. **Scikit-learn**: A widely-used Python library for machine learning that provides a variety of algorithms for classification, regression, clustering, and other tasks.
2. **TensorFlow**: An open-source machine learning framework developed by Google that provides tools for building and training neural networks.
3. **PyTorch**: Another popular open-source machine learning framework that provides a dynamic computation graph and automatic differentiation.

Some common machine learning algorithms used in AutoML platforms like HeatWave include:

1. **Random Forest**: An ensemble method that combines multiple decision trees to improve the accuracy and robustness of predictions.
2. **Gradient Boosting**: A type of ensemble method that uses gradient descent to optimize the weights of individual decision trees.
3. **Support Vector Machines (SVMs)**: A supervised learning algorithm |

What's Next

Learn how to View Chat Session Details.