The SQL Dialogs natural language processing model supports queries that translate to the basic SQL clauses: SELECT, FROM, WHERE, GROUP BY, HAVING, ORDER BY and LIMIT.

A query can involve up to 3 different entities. That is, it can have up to 2 joins. Queries that involve 3 joins might resolve to correct results depending on the use case. For 4 or more joins, you'll have to add custom training data to ensure the skill gives the correct results.

SQL Dialogs doesn't support the more complex queries that involve sub-queries and SET operators (INTERSECT, UNION, EXCEPT, and NONE). It also doesn't support non-English queries, queries that request a yes or no response, queries with pronouns, and follow-up queries. To learn about these and other SQL Dialogs query limitations, see Troubleshooting SQL Queries.

For some of the queries that aren't supported, you might be able to resolve the issue by using database views or creating virtual attributes as described in Add a Custom Attribute.

Below is a table that describes the types of queries that SQL Dialogs supports. It uses the following database values:

Employee Table

Department Table

Here are the types of queries that SQL Dialogs supports:

To connect to an Oracle database, follow these steps:

1. In Digital Assistant, click to open the side menu, click Settings, click Additional Services, and click the Data tab.

2. Click + Add Service.

3. In the New Data Service dialog, provide this basic information:

   Field Name	Description
   Database Type	Select Oracle.
   Name	A unique name for the service.
   Data Service Description	An optional description of the data service integration such as a description of the database or the purpose.
   User Name	Ask your database administrator for the user name and password that gives access to the tables that the skill developers need to create the composite entities for their SQL Dialog skill as described in Create Query Entities to Model the Data Service.
   Password	The user's password. Note that for Oracle Digital Assistant integration, a password must be at least 14 characters and no more than 30 characters, and it must contain at least one upper case character, one lowercase character and one number. It also can't start with a digit.

4. Click Continue to configure end-user authentication if your data service is configured for role-based access. Here is a description of the fields on that page:

   Field Name	Description
   End-User Authentication is required	Select this option if your data service is configured for role-based access.
   Authentication Service	Select an authentication services that you configured in Settings > Authentication Services.
   End-User Identifier	Select the type of end-user identifier.
   Custom Expression	If the selected end-user identifier type is custom, enter a FreeMarker expression to a user profile variable which represents the end user identifier. See Expressions for OICD Profile Claims for more information and examples.

5. Click Continue to add the connection details.

6. On the Connection Details page, select Basic or Cloud Wallet Connection for the connection type.

   • For single-node databases and RAC-enabled databases, you need to select Basic.
   • If you are connecting to an ATP database, you must select Cloud Wallet Connection.

7. If the connection type is Basic, enter these values, which you can get from the database administrator:

   Field Name	Description
   Use TLS	Move this switch to the ON position if you want to use TLS (Transport Layer Security) to secure the connection. Note If you are using a private CA certificate to connect to the database, this option needs to be switched on.
   Host Name	Enter the host for the data service. Leave out the https:// prefix. For example: example.com.
   Port	The port that allows client connections to the database.
   Service Identifier	Do one of the following: Select SID and enter the Oracle system identifier of the database instance. Select Service Name and enter the service name for the database.
   Private Endpoint	This option only appears if you have private endpoints configured in your Digital Assistant instance. If you are connecting to a private endpoint to access the service, switch the Private endpoint toggle to the ON position and then select from a list of private endpoints that are associated with the instance. (Using a private endpoint enables you to access a service that is not accessible directly from the public Internet. See Private Endpoint for the details.)

8. If the connection type is Cloud Wallet Connection, enter these values, which you can get from the database administrator:

   Field Name	Description
   Wallet File	Find and select the Cloud Wallet file that contains the client credentials or drag and drop it into the field.
   Wallet Password	Enter the password that was provided when the wallet file was downloaded. Note that for Oracle Digital Assistant integration, a wallet password must be at least 15 characters and no more than 30 characters, and it must contain at least one upper case character, one lowercase character, one special character, and one number. It also can't start with a digit.
   Service	Select the name of the database service. Be sure to select a service that has a sufficiently high service concurrency so that queries don't take longer than 30 seconds (at which point they time out). The service names with the suffixes _tp and _tpurgent are generally the most suitable choices here. You can read more about these considerations at Database Service Names for Autonomous Transaction Processing and Autonomous JSON Database and Service Concurrency.
   Private Endpoint	This option only appears if you have private endpoints configured in your Digital Assistant instance. If you are connecting to a private endpoint to access the service, switch the Private endpoint toggle to the ON position and then select from a list of private endpoints that are associated with the instance. (Using a private endpoint enables you to access a service that is not accessible directly from the public Internet. See Private Endpoint for the details.)

9. On the Advanced Properties page, if you need a private CA certificate to connect to the database, switch the Use Private Trust toggle to the ON position and fill in the rest of required fields.

   Field Name	Description
   Use Private Trust	If you are using a private CA certificate to connect to the database, switch this toggle to the ON position. This switch is only enabled if you have selected Use TLS on the Connection Details page.
   Certificates Resource	Select Self Managed.
   Choose PEM File and Paste PEM Text	Select one of these options for providing the certificate.

10. Click Add Service.

    You now can import the database schema into a skill to create query entities, which enable users to query the database using natural language.

1. In Digital Assistant, click to open the side menu, click Settings, click Additional Services, and click the Data tab.

2. Click + Add Service.

3. In the New Data Service dialog, provide this basic information:

   Field Name	Description
   Database Type	Select MySQL.
   Name	A unique name for the service.
   Data Service Description	An optional description of the data service integration such as a description of the database or the purpose.
   Authentication Type	Your database administrator will tell you whether to select Default, Kerberos, or OS.
   User Name	Ask your database administrator for the user name and password that gives access to the tables that the skill developers need to create the composite entities for their SQL Dialog skill as described in Create Query Entities to Model the Data Service.
   Password	The user's password. Note that for Oracle Digital Assistant integration, a password must be at least 14 characters and no more than 30 characters, and it must contain at least one upper case character, one lowercase character and one number. It also can't start with a digit.

4. Click Continue to configure the connection details listed in this table:

   Field Name	Description
   Use TLS	Move this switch to the ON position if you want to use TLS (Transport Layer Security) to secure the connection. Note If you are using a private CA certificate to connect to the database, this option needs to be switched on.
   Host Name	Enter the host for the data service. Leave out the https:// prefix. For example: example.com.
   Port	The port that allows client connections to the database.
   Database	The name of the database.
   Private Endpoint	This option only appears if you have private endpoints configured in your Digital Assistant instance. If you are connecting to a private endpoint to access the service, switch the Private endpoint toggle to the ON position and then select from a list of private endpoints that are associated with the instance. (Using a private endpoint enables you to access a service that is not accessible directly from the public Internet. See Private Endpoint for the details.)

5. On the Advanced Properties page, if you need a private CA certificate to connect to the database, switch the Use Private Trust toggle to the ON position and fill in the rest of required fields.

   Field Name	Description
   Use Private Trust	If you are using a private CA certificate to connect to the database, switch this toggle to the ON position. This switch is only enabled if you have selected Use TLS on the Connection Details page.
   Certificates Resource	Select Self Managed.
   Choose PEM File and Paste PEM Text	Select one of these options for providing the certificate.

6. Click Add Service.

   You now can import the database schema into a skill to create query entities, which enable users to query the database using natural language.

To help a SQL Dialogs skill associate natural language phrases with the underlying data structure (physical model), start by taking the identified utterances that the skill should handle (the training corpus), and analyzing them to discover the different ways that end users refer to the entities and attributes.

For example, suppose that you have these utterances in your training corpus:

• Show me the invoices with outstanding balances greater than zero.

• What's the amount due for reference 12656?

Here, you see that people use "outstanding balance" and "amount due" to refer to the amount_remaining column. You also see that "reference" is one way people refer to invoice_num.

In addition to the training corpus, you also might want to crowdsource utterances from your target users to get more phrases and analyze those as well.

After you compile your list of the ways people refer to the entities and attributes, pick which term you want to use for the primary name of each entity and attribute. They should be names that are closest to the most common usages. When you choose the name, consider that the out-of-the-box NLP model won't likely understand domain-specific relationships. For example, it won't automatically understand that invoice number and reference refer to the same thing. Because invoice number is commonly used and also is closest to other commonly used terms such as invoice no and bill number, you would make it the primary name.

Treat the rest of the terms as synonyms. In the above example, you would add reference, invoice no, and bill number to the synonym list.

Note that the primary name is the default for the result column headers and labels but you can change that in the Presentation tab.

Using your list, you create the training data in the Entities page.

• To set the entity's Primary Name and Synonyms, open the entity's Configuration tab and expand Natural Language.

• To set the attribute's Primary Name and Synonyms, open the attribute's Natural Language tab.

You can improve the natural language parser's accuracy by associating attributes with value lists or dynamic entities. This helps the parser identify an attribute based on its known values. You use Referenced Entity on the attribute's General Information tab to associate the attribute with the reference entity's values. For value list entities, you can automatically create the entity, import the data service's values, and associate it as a referenced entity all in one step.

When deciding whether to use a value list or a dynamic entity to store the values, consider whether the entity is open or closed.

• An open list is one that is infinite or dynamic (or both). For open lists, consider creating and maintaining a dynamic entity instead of a value list. If you choose to use a value list, then you should curate the list to make sure that it at least contains the most commonly used values. For example, for a vendors list that most likely grows over time, you'll want the list to include your most frequently used vendors. This is because queries about a vendor without using the word "vendor", such as "show the summary flag for Seven Corporation", won't match if that value isn't in the value list. You thus increase the frequency of correct resolutions by at least including the most frequently used values.

• A closed list is a static finite list. These are ideal for value list entities.

For both value lists and dynamic entities, add both singular and plural (when applicable) versions of synonyms for each entity value to indicate the ways that end users will refer to the value.

Synonyms are especially important when the list contains values that end users don't typically use. Take, for example, this list of valid payment statuses. Users will be much more likely to use words like paid, unpaid, and partially paid than to use Y, N, and P. Adding these words as synonyms helps to insure that the NLP recognizes that the users are referring to the payment status attribute.

When a term describes more than one entity value, then add that term as a synonym to each one. For example, both N and P indicate that the invoice is unpaid. If you add "unpaid" as a synonym for both statuses, then "show unpaid invoices" will retrieve invoices with a payment_status value of N or P.

For dynamic entities, you create the entity and then use Referenced Entity on the attribute's General Information tab to associate the attribute with the list.

For value lists, you can create a value list from the data service and associate with an entity by following these steps:

1. On the Entities page, edit the attribute and go to the General Information tab.

2. Select Entity from the Type drop-down list.

3. Click If the desired entity doesn't exist, you can generate a value-list entity based on the background mapping by clicking here. The value list is created and populated from the data service, and the Referenced Entity points to the new value list.

4. (Optional) To increase the chances of user input matching a value from the list, open the value list entity and switch Fuzzy Match to on. Otherwise it uses strict matching, meaning that the user input must be an exact match to the values and synonyms. For example, “cars” won’t match “car”.

   Fuzzy matching uses word stemming to identify attributes from query. For example "pound" matches "pounds", "hold" matches "on hold", "needed approval" matches "needs approval", and "rent-lease" matches "rent lease".

   Note that fuzzy matching doesn't work for "_" and "?". Also, partial matching doesn't work. For example, "Seven" doesn't match "Seven Corporation". If you need to enable matching for such strings, add them to the synonyms list.

5. Click Apply to save your changes.

If an attribute's values must match a certain pattern, then you can help the natural language parser identify that attribute's values by associating the attribute with a regular expression entity.

This can be helpful when all values must follow a specific pattern and the set of valid values is too large for a value list or is infinite. For example, for an ip_address attribute, you could associate it with a regular expression entity named IpAddress, which has the regular expression (\\d{1,2}|(0|1)\\d{2}|2[0-4]\\d|25[0-5]).

Note that, similar to value lists, if more than one attribute in the model can be associated with the same regular expression, the users will need to provide sufficient context in their query distinguish between the two (or more) attributes.

To associate an attribute with a regular expression:

1. On the Entities page, click + Add Entity, select Regular Expression from the Type drop down, enter the regular expression, and click Create.

   Ensure that you craft the regular expression in a way that prevents the natural language parser from association non-related values with the attribute.

2. Select the entity with the attribute that you want to associate with the regular expression, edit the attribute and go to the General Information tab.

3. Select Entity from the Type drop-down.

4. From the Referenced Entity drop-down, select the regular expression entity.

5. Click Apply to save your changes.

As an AI trainer, you'll encounter natural languages utterances that the skill can't translate to OMRQL. For example, the model may not be able to handle domain-specific synonyms that don't seem to be closely related to the primary name. Another example is when the model is not able to distinguish between two similar entities. When this happens, you can use training data to teach the skill how to correctly parse the utterance into OMRQL.

Adding to the training data is often referred to as custom training. You use custom training to teach the model to associate words and phrases with attributes, entities, and OMRQL keywords in the context of a full utterance by mapping the utterance to OMRQL.

For each scenario that you are fixing, start with 20 utterances and add more as needed. Because too many examples might cause the model to over predict attributes and operators, you should focus on a smaller set of diverse utterances rather than a large set of similar, lesser quality ones. Note that there is a limit of 120 utterances per skill.

All the values in the OMRQL statement must exactly match the database value and format. Take, for example, the utterance "who is the employee whose name is Jones". If the database values for the name attribute are all capital letters, then the name value must also be all capital letters. That is "SELECT * FROM Emp WHERE name = 'JONES'".

When the utterance that you are mapping uses a synonym for the actual database value, then that synonym must be defined for the value in a value list, and the OMRQL must use the actual database value. For example, if the utterance is "show the department whose location is the big apple", then "big apple" must be defined in the dept_loc value list as a synonym for the value "NEW YORK", and the OMRQL must be "SELECT * FROM Dept WHERE loc = 'NEW YORK'".

You can add utterances that contain absolute dates, such as "invoices due on 5 Jan 2022", but don't use utterances with relative dates or dates without the year. For example, if the utterance is "invoices due today", then today's date would be hard-coded into the OMRQL as SELECT * FROM invoices WHERE due_date = '2022-01-01'. Also, if you use a relative date such as "today", then you may get a error that relative dates aren't supported.

Here are some best practices for custom training utterances:

• Balance the number of utterances: Some of the more complex scenarios may need more utterances than the simple ones, but try to balance the number of utterances per scenario.

• Balance the training of similar attributes and entities: If you have two similar attributes, and you need to provide custom training data for one of them, then you also should provide the same amount of training data for the other. When the training data concentrates only on one of the similar attributes, then the model might over predict that attribute over its counterpart. The same is true for similar entities. For example, payment currency and invoice currency are similar attributes. If payment currency is over-represented in the training data, the model might predict payment currency even when the utterance asks for invoice currency.

  When you need to teach the model how to distinguish between two similar or closely-related attributes, balance the weighting of importance by providing half the utterances for one attribute and half the utterances for the other.

  Vary the utterances that refer to these similar attributes. For example, here are contrasting pairs of utterances to help the model distinguish between amount_remaining and amount_paid:

  • tell me the amount remaining for approved invoices
  • show us the amount paid for approved invoices
  • view total amount due to be paid to vendor AAD
  • calculate the total amount that was paid to vendor AAD
  • what is the amount due on invoices to vendor AAD
  • list the paid amount on invoices to vendor AAD

• Balance the training of values that match primary names or synonyms: Say, for example, that your model has a manager attribute and "manager" is also a value for the employee job attribute. If you want to add "How many managers" to the training data, then you should balance this training data with utterances that use the manager attribute, such as "Who is the manager of employee Adam Smith", as well as utterances that use the manager job, such as "Show all managers". That way, the model can learn to differentiate between the two usages. If you don't include examples for both types of usage, then the skill might over predict one usage over the other.

• Diversify phrases: The best practices for diverse phrasing for custom data are similar to those for intent utterances:

  • Use full sentences.

  • Use different verbs. For example: view, list, show, tell, and see.

  • Use various synonyms and paraphrases in addition to the entity or attribute name.

  • Use different pronouns. For example: show me, can we see, tell us, I want.

  • Vary the sentence structure. For example, put the attribute value near the beginning, middle, and end of the sentences.

  • If you have utterances with an aggregation, such as AVG, then also add utterances with other operators as well.

  • If possible, use different clauses, such as group by and where clauses with AND and OR conditions.

• Diversify Values: When you use more than one value in your scenario's utterances, the model is better able to recognize different values. Include values with different word lengths. Include some values with special characters such as '/' and "-". Include a few values with special keywords such as 'and'.

• Include a mix of known and unknown values. For value-list attributes, use a representative set of attribute values (but not all) to train that value-list matches are important signals. Also, for value lists that aren't closed lists, include values that aren't in the value list to teach it to also associate particular phrasings with the attribute.

To add a mapped utterance to the training data:

1. If the Train button has a red badge, click , and train using Trainer Tm.

2. In the Entities page, go to the Dataset tab and click Query Entities.

3. Click the Training Data tab.

4. Click Add Utterance.

   The Create Utterance dialog displays.

5. Enter the utterance and click Continue.

   The dialog displays the OMRQL query for the utterance. If it can't translate the utterance into the query, the query will be blank.

   Note that if the skill hasn't been trained, it can't translate the utterance into an OMRQL query.

6. Review the query and correct it if it's wrong.

   For OMRQL keywords and examples, see OMRQL Reference.

7. Click Done to add the mapped utterance to the training data.

If your skill has intents or is in a DA, then, just like with intents, your skill needs utterances to help it route SQL queries to the SQL Dialogs conversation. In addition to the autocomplete suggestions, training data, and handcrafted routing data, the routing mechanism uses generated routing utterances that you create from the Generated Routing Data tab on the Query Entities Dataset. The generated utterances represent a broad coverage of questions about all the query entities in the logical model.

To generate routing data:

1. If the Train button has a red badge, click and train using Trainer TM.

2. In the Entities page, go to the Dataset tab and click Query Entities.

3. Click the Generated Routing Data tab.

4. Click Generate.

   The Generate routing data dialogue displays.

5. In the Select entities field, select All. The first time you generate the routing data, you must generate data for all the entities. After you generate the initial set, you can come back and generate for specific entities if there is a need.

6. Click Generate.

   The skill generates 100 utterances, which reflect questions that the logical model can answer.

   
   
   
   
   
   

7. Review the generated data and edit any that need refining.

   Tip:

   The utterance is not editable if it has been approved. If you want to change an approved utterance, unapprove it, edit it, and then approve it again.

8. Delete entries where needed and approve the rest.

   The approved utterances are added to the combined routing data. If you edited an utterance, then its routing subtype in the Combined Routing Data tab is Refined. Otherwise it is Synthetic.

If there are valid SQL queries that the DA or skill is not routing to the SQL conversation, then you need to add those utterances to the routing data from the Combined Routing Data tab in the Query Entities Dataset page.

To add handcrafted routing data:

1. If the Train button has a red badge, click and train using Trainer TM.

2. In the Entities page, go to the Dataset tab and click Query Entities.

3. Click the Combined Routing Data tab.

4. Click Add Utterance.

   The Create Utterance dialogue displays.

5. Type the utterance and then click outside the text box.

6. Click Continue.

7. Review the OMRQL query to verify that its results would answer the query. If it doesn't, correct the query and then click Reinterpret. See OMRQL Reference for the OMRQL query keywords.

8. Click Done.

   The utterance is added to the data with the routing subtype set to Handcrafted.

The skill can display the entity's results as a table, a form, or a table form (where you can expand a row to see more details in form mode). You use the layout conversion fields on the entity's Presentation tab to configure when the results should be displayed in each mode.

By default, the skill displays each row in the response as a form unless the number of rows exceeds a threshold that you specify for Use form layout for this number of rows or less. Here are examples of a response in form mode and table mode:

In the case where the number of columns exceeds a threshold, the skill displays a table form. With a table form, only the specified number of columns are displayed and the user can expand the form to see the other attributes. Use Switch to table form layout when number of columns exceeds this number to specify the threshold. Here's an example of a table form layout for the column threshold of 2.

By default, in form mode, the skill displays all the result attributes one below the other. To save room, you can set Number of Horizontal Sections in Form Layout to 2 to display two columns of attributes.

By default, the skill uses the query entity's name for the results title, but you can use the Display Name on the Presentation tab to set a different title.

Note that after you set the display name, you can't clear the field.

You can specify a default sort order for the skill to use whenever the user's utterance doesn't specify one. To set the default, go to the entity's General tab, click Add Attribute Order, select an attribute and select its order (Ascending or Descending). You can continue clicking Add Attribute Order to add more attributes to the sort order.

If the utterance doesn't name any attributes, then you probably want the results to include some essential fields. You can use Default Attributes in the entity's Presentation tab to specify these fields. For example, for an invoices entity, you might want to display invoice_num, invoice_date, and invoice_amount when no attributes are named.

Note that you can't add attributes of type query entity to the default attributes list.

When an utterance identifies specific attributes, you might want the result to include not only the requested attributes, but also some context. For example, if someone enters "show invoice amounts", the data won't make sense if it only shows the invoice_amount values, and not some identifying context like invoice_num. Use Minimum Attributes on the entity's Presentation tab to identify the minimum attributes.

You cannot add attributes of type query entity to the minimum attributes list.

Use the Maximum number of rows per page on the entity's Presentation tab to set how many rows to display at once.

The user can click buttons to page through the results.

You can add buttons and links to a query entity's results at both the global level and the row level. A row action appears in each row, and a global action appears below the results.

For example, for an employee entity, you could add a global action that links to the company's employee search page. At the row level, you could add an action for a common follow-up query, such as a query about the employee's department.

You add actions from the entity's Presentation tab. If you have more than one action, you can indicate the sequence in which the actions appear. For QUERY action types, you'll need to provide an OMRQL query. For URL action types, you'll need to set the URL.

For row level follow-up actions, you can use ${row.attributeName} to reference each row’s attribute values. For example, select * from Emp WHERE dept.loc = "${row.loc}". At run time, each row’s button will have a different value for the query. This syntax is only available for row level actions.

You can optionally restrict when the action appears. For example, you might have a row action to show an employee's direct reports, which should appear only if the employee's job is manager. To do that, switch Visibility Expression to On and provide a FreeMarker expression, such as ${row.job = 'MANAGER'}.

You can add your own custom attributes to display additional information, such as derived or calculated values.

1. From the Attributes tab on the entity page, click + Add Attribute, and provide a canonical name and type.

2. On the Natural Language tab, provide a primary name and optionally add synonyms.

3. On the Backend Mapping tab, select SQL Expression and add the expression.

If the expression references a column, use the column name from the physical model (database schema) and prepend ${alias}. For example, for an invoices entity, you might add an amount_to_pay attribute with the expression ${alias}invoice_amount + ${alias}discount_taken where:

• invoice_amount and discount_taken are existing physical column names in the invoices table.
• The new derived column amount_to_pay is the sum of values from the invoice_amount and discount_taken physical columns.

You can use this table to determine what type to use for the attribute:

If you'd like the skill to dynamically change the way that the skill presents SQL query results, you can add data query event handlers to a custom component package, add the package to the skill as a custom component service, and then associate your entities with their specific handlers from the entity Presentation tabs. The skill triggers an entity's data query event when that query entity is the first named entity in the FROM clause (the root entity).

For example, you can dynamically add a row count to the header text, add a row to the table to show a sum, or determine when to show or hide an attribute.

To learn how to build data query event handlers, see Writing SQL Query Event Handlers.

The table below outlines general limitations in SQL Dialogs that you should be aware of. These limitations don't have workarounds.

The skill's Insights page provides several metrics you can use to measure how well your SQL Dialog skill is performing and to determine where to make improvements.

As a business analyst, you might be interested in these data query metrics on the Overview tab:

• Performance: Conversations Trend by Status (Line) shows the number of conversations over a given time period and whether the traffic is tending up, down, or sideways.

• The ratio between Correct Queries and Incorrect Queries indicates how satisfied the bot users are with the accuracy of translating utterances to SQL queries.

• The ratio between Completed and Incomplete conversations shows the extent to which technical issues impact the users' experiences.

• The ratio between Total Conversations and Unresolved (OOD/OOS) Queries helps measure the extent to which the skill meets the end users expectations.

• Both Conversations Trend by Type and the ratio between Total Conversations and Data Queries Conversations show the proportion of utterances that are SQL queries.

• Data Query Entities show which entities are queried the most.

As an AI trainer, you can examine the user messages on the Conversations tab to discover areas for improvement. For example, you can review these user messages:

• Type: Intent, Outcome: Incomplete user messages indicate problems with translating the utterance to an SQL query. Often, you can fix these issues by adding synonyms or, for more complex queries, adding mapped utterances to the query entities dataset. Note that you also can see these messages by selecting System Handled Errors from the Errors drop-down list.

• Type: Intent, Intent: unresolvedIntent user messages indicate both out of scope utterances and utterances that the skill doesn't recognize as a data query utterance. For the utterances that are valid data queries but the skill doesn't recognize as such, you often can fix the problems by adding synonyms or mapping the utterances to OMRQL in the query dataset.

• Type: Data Query, Entities shows the user messages by query entity.

• Type: Data Query, Outcome: Incorrect shows the messages that the users thought returned incorrect results. You should verify that the results are incorrect, and, if so, add synonyms, value lists, and query dataset entries as appropriate.

As an AI trainer, it is good practice to create batch tests to ensure that improving one area doesn't negatively impact another. You can also use batch tests to ensure that changes to the logical model don't have adverse effects on custom training or routing to SQL conversations.

Just as with batch testing for intent utterances, you might want to set aside about 20% of the real-world queries that you gathered to use for query batch testing. You can run the batch test periodically as well as after you make changes to the logical model, custom training, or routing data.

Each test case must belong to a test suite, so before you create a test case, you may want to first create a test suite that reflects some aspect of query testing, such as failure testing, in-domain testing, or out-of-domain testing. We provide a suite called Default Test Suite. You can assign test cases to this test suite if you haven't yet created any others.

You can add a test case to a batch test in two ways:

• After you test an utterance from the Query Tester, you can select a test suite from the Save as Test Case drop-down to save it to that suite.

• You can click + Test Case on the Test Suites tab in the batch tester.

To access the batch tester:

1. On the Entities page, click Test Queries.

   The Query Tester opens.

2. Click Go to Test Cases.

   On the Test Suites tab, you select a test suite and either run all test cases or select and run specific cases. The results are shown on the Test Results page. It takes some time for the tests to complete. You know the run has completed when In Progress shows 0.

With the exception of linking entities, the OMRQL components are similar to SQL. Instead of an SQL JOIN, you use a pair of link attributes to link one entity to another. Link attribute have primary names and synonyms that define the relationship between the entities. For example an employee/department attribute link with a 1-1 relationship can have a primary name "department" and synonyms "works in", "belongs to", and "team". A department/employees attribute link with a 1-many relationship can have a primary name "employees" and synonyms "members", and "workers".

Besides the typical primary key/foreign key link attributes you also can have these types of link attributes:

• Multiple link attributes from one entity to another that define multiple semantic relationships.
• A link attribute from an entity to itself that implies a self join.
• A link attribute for an intersection table due to a many-to-many join

The asterisk (*) is used in conjunction with ORDER BY when a user asks to order something without specifying what to order by. For example, "show the top 10 employees" (unclear what attribute we need to order by). The backend then replaces the * with a default, pre-specified attribute (measure_by).