Preparation is the Key to Success

7 Preparation is the Key to Success

Let's go over the types of preparation you should to before diving into development of your digital assistant project.

A quote attributed to Abraham Lincoln reads, "Give me six hours to cut a tree, and I'll spend the first four sharpening the ax." Digital assistant developers are certainly not forest workers, and developing digital assistants is not a physically demanding job either.

What forestry and building digital assistants have in common, however, is that preparation is a critical success factor. So, let's take a look at the prep work that is involved in starting a digital assistant project.

Description of the illustration planning-cdx.png

Firstly, you need to identify a project and goals that fit the conversational channel and that would allow you to benefit from having a digital assistant.

You then break the problem down into manageable use cases. Each of those partitions can then be implemented iteratively one-by-one until you have built the final product. Each use case has its own set of user messages that reflect how you think users would request that conversation.

A digital assistant is not human but should have human-like features that users will like. These human-like features include the definition of the conversation style your digital assistant should follow, which is best visualized as a persona for which you define a background story and CV.

When designing user interaction flows, it helps to know about the medium you want to expose your digital assistant on, such as web, mobile, Facebook, Slack, just to name a few. The medium you expose your digital assistant on is referred to as channel. You can support multiple channels.

CDX Workshop

To help with the preparational steps described in this section, consider participating in a conversation design experience (CDX) workshop. In a CDX workshop, you bring people from your project team and people from the intended user group together to discuss use cases and the way your digital assistant should be in terms of personality, voice, language, and attitude.

A CDX workshop can help you with the following fundamental aspects of a digital assistant project:

Identify the problem to be solved with a digital assistant and the corresponding use cases.
Identify the target audience and develop a bot persona that suits that audience in terms of voice, tone, and educational and cultural background.
Work through the existing user journey and identify where and how it can be improved with a digital assistant.
Identify the best channel or channels for reaching your users, also keeping in mind the capabilities of the different channels.
Model the dialog between the bot and user.
Identify the backend systems that have the data required to support the conversations.

You can run such a workshop yourself or with the help of your contact at Oracle.

Identify Good Digital Assistant Use Cases

Digital assistants don't replace web and mobile applications: they complement them. So, before you start building a digital assistant, ask yourself "why?" Why are you building a digital assistant and how would you like to benefit from it? We're not saying you shouldn't be building digital assistants. Rather, we want to make sure that you are building digital assistants that have their use cases within their sweet spot.

The sweet spot of digital assistants lies in their ease of use: the use of natural language conversations for user interaction with the application interface and the ability to extract information from the user message, which then leads to an overall reduction in the number of prompts displayed to a user.

Years ago, when mobile application development was a new skill to learn, the Oracle User Experience team recommended that a mobile task should take no more than 3 minutes. We don't have a corresponding recommendation for digital assistants. However, based on our experience, we are sure that it should be well below that, perhaps in the neighborhood of 60 seconds.

So, when defining a digital assistant use case, pay attention to the amount of user interaction required to get a task done. If you find that the interaction is very long, then the use case is either not suitable for digital assistants or you need to consider shortening interactions, e.g., by incorporating structured data entry forms into a conversation.

As a general recommendation, find use cases that are neither too large nor too complex (and break down bigger use cases into smaller ones). Especially if the digital assistant you want to create is your first, "simple" might already be too complex.

Define Digital Assistant Success

As with anything else you want to achieve, you need to set goals. For digital assistants, it's important to set realistic goals that will also benefit your company. For example, moving a web application to a conversational channel may not be a good goal unless the web application can be broken down into realistic digital assistant use cases and the digital assistant works better than the web application. In most cases, "moving from ... to ..." projects are unsuccessful because users and the development team simply copy the functionality and appearance of the application that they are moving away from.

Success has to be measured. For example, a digital assistant is a smart way to automate self-service, so it can offer a real benefit. However, in order to properly assess whether there is any benefit, you need to define what success means and how it can be measured for each of your digital assistant use cases. Here are some example criteria:

"We want 60% of all incoming student requests for books from our library to be resolved by the digital assistant to reduce the workload for our librarians."
"70% of our current IT operations, such as resetting passwords and access and authorization requests, should be handled by the digital assistant."
"All customer inquiries to buy new real estate should be pre-qualified and forwarded by a digital assistant so that our real estate agents can increase the number of on-site visits by 30%."
"The expense digital assistant should reduce the time between submitting an expense report by an employee and the reimbursement of expenses by the company to 3 days."

Identify What the Digital Assistant Should Not Do

Conversational use cases define the tasks that a bot should handle. However, there may be reasonable and related use cases within a domain that you don't want the digital assistant to handle but want it to be aware of.

In-Domain but Out-of-Scope

"In-domain but out-of-scope" use cases define tasks for which a digital assistant is not set up (yet) or is not allowed to handle due to company policies or legal restrictions. When a user requests such a task, a digital assistant should however understand the request and advise the user.

Suppose that in a digital assistant used by a bank, the digital assistant is not authorized to approve loan applications. If, however, the digital assistant understands what it is not supposed to do, it could advise the user and possibly connect them to a human agent.

Not Suitable for the Channel

Though it does not process them, a digital assistant should be aware of cases that are not suitable for the conversational channel the digital assistant is using.

Channels are constrained by the design of the party that owns their platform (unless you create a custom channel, in which case you are also responsible for developing the client that displays bot responses and user input controls). Those constraints might include a lack of rich user interface components and layouts, a reduced canvas size, a limited number of items that can be added as buttons or select items in a list of values or action menu, and more.

So, if a use case is not suited for the conversation channel due to its user interface or data requirements, building a web application may be a better choice to follow up with. Nevertheless, the digital assistant needs to know about the use case and that it does not support it because "no" as an answer is better than no answer at all.

Shape Your Conversational Mindset

Having a conversation is a native skill for humans and no special training is required to get a conversation going. For digital assistants, we need to identify the principles of a good conversation (which we might otherwise take for granted) and then incorporate them into our bot conversation design. If you don't shape your conversational mindset, you run the risk of creating your digital assistant as a command line or web application in disguise and not using any of the conversational features that make digital assistants so great.

So how do you shape your conversational mindset? Here are some options:

Take part in a conversation design experience (CDX) workshop to help define your digital assistant.

One exercise to try during the workshop is to have two people sit back to back on chairs. (When sitting back to back, you better simulate a digital assistant conversation, since you eliminate non-verbal cues.) One person acts as a digital assistant, the other as a user. Give them a use case and see how the conversation develops, how it progresses, and also where it gets stuck. If the conversation gets stuck, observe how the two persons resolve to a state where the conversation can be resumed. Take notes of the conversation, put them on sticky notes, and add them to a wall to discuss with the group.
Go to a pub or restaurant to observe people or the friends you are with. Don't stalk them; just watch them order food and drinks.
- How many people order food by mentioning the number of the meal on the menu? How many people mention the name of a meal as it is written on the menu? How many people order by mentioning the ingredients of a meal? How many people reference a previous person's order by saying "the same for me"?
- How does the bartender or waiter acknowledge or confirm your orders (if they do at all)?
- What does a bartender or waiter do if they did not understand the order due to a noisy environment?
- Does the bartender or waiter encourage you to order more? If so, how does he do it?
- How does the bartender or waiter support you with your order?
- Does the bartender or waiter ask how it is going or how you are doing while you are enjoying your food and drink? If so, why do you think they are doing this?

Pay attention to the details and maybe even take notes. All of what you notice and note is conversational style and practice.

Restaurant employees represent the restaurant itself and are usually the first impression a restaurant makes. The more professional the staff in a restaurant, the more likely it is that they have been trained in dealing with customers. They have learned the do’s and don'ts and how to deal with difficult customers. What do you think psychology has to do with food and drink sales and customer returns?

When you see an analogy with digital assistants, you understand why conversation design is important.

Define a Digital Assistant Persona

The digital assistant persona gives conversation designers and bot message writers a more concrete and representational framework for building a consistent user bot experience. In crafting the engagement between the digital assistant and user they might ask "how would <DIGITAL_ASSISTANT_NAME> handle this situation".

Though digital assistants are not humans, they can be given human characteristics. So, when you define a persona for your digital assistant, you are not only creating an avatar and a name to represent your brand or company, you're also creating a backstory that will serve as a handrail that your development team can use when designing user interactions and messages. The backstory should include all the things you might like to know about it if the digital assistant was a real person: name, age, schools visited, region the person was born in, region the person currently lives in, hobbies, marital status, the music, the food and the movies the person likes, the humor she has, the accent that she speaks in and much more.

If you design a digital assistant that will be accessed from users in different countries around the globe, you’ll want to consider cultural and regional differences. Though your digital assistant should be the same persona no matter from where it is accessed, it is okay – and sometimes a necessity – to adapt to regional and cultural specialities. But keep in mind that your digital assistant is there to serve and please them all.

Identify the Team Roles You Need for Bot Development

Digital assistants don't build themselves. They require a diverse set of skills such as backend integration, system administration, and programming. In addition, they require roles that are very specific to bots: conversation designer, conversation message writer, and model designer. While the latter three do not necessarily have to be represented by full-time positions on the team, those assigned to the positions must become experts at those roles.

Coversation Designer

It’s the job of the conversation designer to translate a technical process or problem into a conversation that the user perceives as natural, that is truly conversational and intuitive to use, and that keeps the interactions between the user and the digital assistant short.

After completing the CDX workshop, you should have a clear idea of the use cases and conversations a bot should have with users. The next step is to design these conversations so that users who start a conversation can complete the conversation successfully, regardless of how much experience they have with conversational channels or whether they are first-time or returning users. It’s also important to plan for mistakes as there is always the possibility that the user or the digital assistant will not get things right.

The role of a conversation designer also is to understand the psychology of human conversations and the human motivation that leads into completion of a task. For each of your identified use cases, the conversation designer needs to outline the conversation for the “happy path” (in other words, the flow you expect users to follow) as well as conversations dealing with errors and objections.

The biggest mistake we see in the development of digital assistants is that developers give digital assistants a web-oriented design that uses buttons and menus more than a conversational approach. Therefore, it is important not only to have a conversation designer to design the conversations, but also to continuously enforce the use of a conversational style throughout the digital assistant development project.

One aspect of good conversation design is employing active listening to make users feel heard. Have you ever had a conversation where you had the impression that the other was not listening? So, can you remember how it made you feel? How would it have been if the other person had given you feedback in between, for example by repeating part of your last statement?

Active listening is an important part of a conversation, especially when other modes of communication, such as facial expressions and body language, are missing. The lack of feedback is daunting and, in the event of a digital assistant interaction, increases the likelihood of incomplete digital assistant conversations.

Conversation designers work closely with conversation message writers.

Conversation Message Writer

When all you have are words, make them count. Conversation message writers create bot messages that mirror the digital assistant personality, guide users to do things right, ensure consistency in the digital assistant's tone and voice, and interact with users for a great user experience.

For example, when crafting bot prompts, you need to make sure you provide context, guidance, and motivation. Consider the example below for a prompt to provide the reason for an expense in an expense reporting conversation:

"Please briefly describe the reason for the expense"

The message is clear, it doesn’t provide useful context. Providing context is important because it helps ensure the user knows what the conversation is about and, just as importantly, it indicates to the user whether the digital assistant understood the initial request. So, let's have a look at the improved prompt below:

" Sure, let's create the expense report for you. Please briefly describe the reason for the expense"

This version provides the context that is missing from the first version, and the user gets confirmation that the request for filing an expense has been understood.

The prompt can be further improved by setting the expectation of how long it will take the user. We mentioned that digital assistant conversations should be short, but the user doesn’t necessarily know that that’s our intention. So, what about telling her?

"Sure, let's create the expenses report for you. With just a few questions, I will assist you to get your costs reimbursed. I promise it won't take long. So, first briefly describe the reason for the expense."

The revised prompt above contains two more bits of information. It firstly mentions that the bot will ask a few questions and, secondly, it reassures that the process won't take long.

Note:

Conversation message writers may not be developers, and therefore may not be familiar with the development environment that they use to create digital assistants. It is recommended that messages and prompts be stored in resource bundles that can be exported for external editing by the message writer or that provide a single editing environment for the message writer. Select the resource bundle keys names so that the message writer can easily identify which messages are related to the same conversation.

Model Designer

A badly trained digital assistant won't do anybody any good. So it’s important to have a model designer to focus on this aspect of bot development.

Model designers identify intents and entities to be required for a use case. They also help finding good utterances for training and testing the model. In Oracle Digital Assistant, model designers don't need to be data scientists, but they should have a good feel for how to build utterances for intents that don't overlap with the utterances of other intents. Besides collecting and curating utterances, model designers maintain and enhance models over time. This includes re-running tests and comparing the results with the previously-documented baseline results.

Break Down a Big Problem Into Small Ones

enables you to use skills to break down large use cases into many small ones. A skill is an individual digital assistant that is configured together with others in a digital assistant. Using machine learning, the digital assistant then determines the skill to which a request needs to be forwarded based on the content of a user message.

Partitioning a large use case into multiple skills allows you to incrementally increase the functionality of your digital assistant, eases development, and allows you to clearly separate concerns.

Use Case: Break Expense Functionality Into Multiple Skills

For example, an expense reporting use case may be broken up into the following functionality:

create, update, and cancel expense reports
find expense reports
check status of a report
human agent integration
frequently asked questions
small talk

The skills to partition this into could be:

a skill to handle the user welcome message
a skill to report new expenses, check expense status, and cancel or update expenses
a skill to transfer users to a human agent for help
a skill handling frequently asked questions
a skill for dealing with small talk

Each skill processes one or more conversations for which it has intents defined. Therefore, intents are a next level in breaking down a larger problem into many small ones. For example, the skill handling finding, updating and cancelling of expense reports would have at least three conversations defined for which it creates intents:

Find expense report
Update expense report
Cancel expense report

Because the update and cancel expense reports use cases must query an existing expense report before they can apply an operation, they can reuse parts of the conversation to find expense reports.

A valid question that may arise is whether, because of their similarity, the three use cases should be served from a single intent rather than three. The conversation could then differentiate the user request based on keywords in the message such as “cancel”, “update” or “find”, for which an entity could be built to extract the contained keyword:

"I want to update my recent expense report"
"I want to cancel my recent expense report"
"I want to find my recent expense report"

Of course, it sounds feasible to do and a couple of years back we probably would have recommended to do so. However, conversational AI (Artificial Intelligence) has improved greatly over time and is now in a position to distinguish between the use cases when trained accordingly. This not only allows you to associate different entities to extract information from user messages, it also is more mature if you consider user messages like those shown below:

"I’d like to update my recent expense report"
"I need to apply a change to my recent expense report"
"I want to add changes to my recent expense report"
"My recent expense report requires modifications"
"I need to correct my recent expense report"

Splitting Up Intents

Even with individual intents you can split them up into several intents if the use case of an intent is linguistically too broad. That is, if an intent was designed to handle multiple conversations for which you would have to use entities to determine the specific conversation to have with a user.

At the end of the previous example, there is a list of messages, all of which regard updating an existing expense report. However, notice the different ways in which the "update" operation is referred to. Therefore, trying to identify keywords in a user message can be a difficult task and at the same time error prone. You will get much better results here if the intent tells your bot what a user wants.

Sometimes it can even be beneficial to create two intents that eventually lead to the same user conversation. Think about how people can request a product return:

"I want to return the shirt I bought"
"The shirt I bought does not fit"

While the first utterance expresses the user's intent to return a product, the second describes a problem that implicitly indicates that the user wants to return it. To clearly separate the two motivations and to have it easier to curate and quality test utterances you use for the intent training, having two intents pointing to the same conversation flow could make sense.

A CDX workshop will help you identify the partitions into which your digital assistant use case can be broken down.

Prepare for Failure

As with everything new, your first release of a digital assistant will not be your best version of it. Like a toddler that first needs to learn how to crawl, then how walk, and eventually how to run, you will:

Experience situations that you did not consider in your design.
See messages logged that you as a human understand but the digital assistant did not get right.

Observe users struggling with conversations you thought were easy to have.
Notice users who abandon conversations.

As you need to have a way to measure success, you also need to have a plan for improving your digital assistant. Such a plan includes the frequency with which you monitor your digital assistant, add new features and bug fixes, and release new versions.

Oracle Digital Assistant includes Insights, an analytics feature that reports performance indicators for the skills and the digital assistant, that allows you to detect broken conversation paths and messages that the bot could not map to a task. In addition, Digital Assistant allows you to re-deploy new versions of a digital assistant without users experiencing a downtime.

The bitterness of failure also comes from overselling what a digital assistant can do. Manage expectations to be realistic when you start out with your digital assistant project. Your bot will eventually perform well and achieve the success goals you set, but it won't be in its first version.

Small Talk in Digital Assistant Conversations

A user's attention rate when working with bots is not endless. Soon, users may get sidetracked from the idea of testing what else your digital assistant can do or how to crack it. Therefore, beyond the business-related function design, you require functionality that is not in the domain of the business you built the digital assistant for.

Dealing with small talk is a common requirement for a digital assistant. Users can ask the bot if it is human, if it wants to go out on a date, what the weather is like, if it knows a good joke to share, and the like. The digital assistant doesn’t have to answer these queries perfectly but it should be able to handle the most common ones.

For example, when asked for the weather, a digital assistant that sells flowers could answer: "I don't have a window in my data center, but I’ve been told the weather is good enough for ordering flowers. So let me help you with your flower orders." So, thanks to small talk, the digital assistant understands the request and deals with it such that the user gets reminded of what the purpose of the bot is. Small talk, if handled well, is a win-win situation for the user and the bot.

Checklist of Preparation Steps

☑ Did you spend enough time sharpening the ax?
☑ Run a CDX workshop with stakeholders and project team members.
☑ Identify use cases that are within the sweet spot for digital assistants.
☑ Identify use cases that are in-domain but out-of-scope for a digital assistant.
☑ Define what success means for your digital assistant and how it can be measured for your use cases.
☑ Partition large use cases into smaller use cases.
☑ For each use case, identify the conversations to have.
☑ Define the goals and measures of success. How do you measure conversational success? Where is the business value in conversational?
☑ Check if you have the right skills on the team: conversation designer, NLP trainer, product management, Oracle Digital Assistant developer, backend integration developer.
☑ Define a bot persona.
☑ Write the backstory for the bot.
☑ Consider how to handle small talk and frequently asked questions.
☑ Prepare a plan for dealing with failure.

Learn More

Oracle TechExchange article: Decide how to partition your digital assistant into multiple skills
Tutorial: Use Oracle Digital Assistant Insights