This chapter covers the following topics:
Oracle Advanced Outbound Telephony provides the following failover and load balancing actions:
More than one dial server configured within the same server group.
One dial server goes down.
On the next agent request, Oracle Advanced Outbound Telephony automatically logs in the agent to the functioning dial server and begins to service the agent through the new dial server.
One or more dial server configured within the same server group.
UWQ server goes down and then comes back up.
If the agents do not shut down their UWQ Clients (Forms desktops), then when the UWQ server comes back up, the agents will automatically (without having to make a request) be logged into the same UWQ server and dial server as they were before the UWQ server went down.
If the agents shut down their UWQ Clients (Forms desktops), then they will need to log back in when the UWQ server comes back up. The agents will be logged into the same UWQ server and dial server as they were before the UWQ server went down.
Another physical server box is configured within ICSM and has been designated as a warm hardware backup; one or more dial servers are configured in the server group.
The dial server is not responding and appears to be down; ICSM is not getting a heartbeat from server box.
ICSM will start another instance of the dial server on the warm backup server box. On the next agent request, Oracle Advanced Outbound Telephony will automatically login the agent to the other dial server and begins to service the agent through the new dial server.
Another physical server box is configured within ICSM, and has been designated as a warm hardware backup; one or more dial servers are configured in the server group.
The dial server is not responding and appears to be down (possibly due to a network glitch), then the dial server seems to become available again and reappears.
ICSM will start another instance of the dial server on the warm backup server box. On the next agent request, Oracle Advanced Outbound Telephony will automatically login the agent to the duplicated dial server and begin to service the agent through the duplicated dial server. When the machine running the original dial server appears on the network again, the original dial server will recognize that a duplicate dial server (same ID) is running and will automatically shut itself down.
Each dial server maintains and calculates the Oracle Advanced Outbound Telephony runtime parameter - Major Load Factor (MLF). The calculation is based on the number of agent logins. New agents will be logged into servers with lower MLF. Agents will be logged in at a grainularity of ten - meaning Oracle Advanced Outbound Telephony will determine which server currently has the lowest MLF and the next ten agents who log in will be placed on that server. After the tenth agent logs in, Oracle Advanced Outbound Telephony will again evaluate each server's MLF.
There are a number of tools you can use to diagnose problems with Oracle Advanced Outbound Telephony, including:
Using Oracle ICSM to Diagnose Issues
With Oracle Interaction Center Server Manager (ICSM), you can:
Starting and Stopping Servers With ICSM
Follow the steps below to start or stop your servers with Oracle ICSM.
Self-Service Login URL
Call Center HTML Administration
Click ICSM > Server Groups.
The Server Group List page appears.
Click a server group name.
The Server Group Details page appears, displaying a summary view table of all servers in that group.
From the list to the left of the Submit button:
Select Start to start the selected server
Select Stop to stop the selected server.
Click Submit.
It may take several minutes to start or stop the server processes. To view the current status during startup or shutdown, refresh the browser several times.
As the server processes are starting up, "Starting" is displayed in the Stop/Start column. As the server processes are shutting down up, “Stopping" is displayed in the Stop/Start column. The status of each server process is displayed in the Servers area in the Status column.
Setting Server Log Levels With ICSM
You can use the database logging function to obtain information about when servers are started or stopped, and the reasons why they started or stopped.
HTML Login URL
Call Center HTML Administration
Click ICSM > Server Groups.
The Server Group List page appears, displaying a summary view table of any existing server groups.
From the Server Group Name column, click the server group that contains the server on which you want to set log levels.
The Server Group Details page appears for the selected server group, displaying a summary view table of any servers in that server group.
From the Server Name column, click the server on which you want to set log levels.
The Server Details page appears for the selected server.
From the Server Details page, click the Parameters hyperlink.
The Server Details page refreshes to display a summary view table for parameter settings.
In the Parameter Value field for OAO_LOG_LEVEL, type one of the following levels:
error - shows server errors.
warn - shows server warnings and errors
info - shows events, warnings, and errors
all - shows all server activity
The log level can be changed while the server is running, but you must wait for the time period specified in the OAO_ REFRESH_INTERVAL parameter before the information in the log begins reflecting the new log level.
Click Update.
Click Done.
Setting Trace File Options With ICSM
You can use the server trace function to obtain detailed information about server activity. By default the trace function is not activated.
Use the steps below to activate the trace function.
HTML Login URL
Call Center HTML Administration
From the Oracle Call Center Technology page, click the ICSM tab.
The Server Group List page appears, displaying a summary view table of any existing server groups.
From the Server Group Name column, click the server group that contains the server on which you want to run the tracing function.
The Server Group Details page appears for the selected server group, displaying a summary view table of any servers in that server group.
From the Server Name column, click the server on which you want to run the tracing function.
The Server Details page appears for the selected server.
From the Server Details page, click the Advanced hyperlink.
The Server Details page refreshes to display a Java Options field, a Server Arguments field, and a summary view table of any existing trace files for the selected server.
In the Java Options field, type the following:
-ms128m -mx256m -DAFLOG_MODULE=iec% -DAFLOG_ENABLED=FALSE -DAFLOG_LEVEL=STATEMENT -DAFLOG_FILENAME=prs.log
The –ms and –mx VM parameters should be set depending on the number of agents you expect to service. These are the minimum and maximum values for the Virtual Machine. These values cannot be changed dynamically; you will need to restart the server for changes to take effect.
Oracle recommends you set these parameters to -ms128m -mx256m. Refer to the Before You Begin section of this guide, for information on what values to enter for these parameters.
The -DAFLOG_LEVEL parameter can have values (in increasing order of severity):
STATEMENT
PROCEDURE
EVENT
EXCEPTION
ERROR
UNEXPECTED
In the Server Arguments field, type the following:
-AFLOG_ENABLED TRUE
Note: You will need to bounce the server to start or stop the tracing process.
Click Update.
Click Done.
Checking Server Log Messages With ICSM
Follow the steps below to check the server log messages.
HTML Login URL
CRM HTML Administration
Click the Settings tab.
Click the System subtab.
From the side panel, click Debug Logging > Messages.
The Debug Log Viewer page appears.
In the Module field, type iec% to only see Oracle Advanced Outbound Telephony logs.
Checking Server Parameter Settings With ICSM
Follow the steps below to check server parameter settings.
HTML Login URL
Call Center HTML Administration
Click ICSM > Server Groups.
The Server Group List page appears, displaying a summary view table of any existing server groups.
From the Server Group Name column, click the server group that contains the server on which you want to set log levels.
The Server Group Details page appears for the selected server group, displaying a summary view table of any servers in that server group.
From the Server Name column, click the server on which you want to set log levels.
The Server Details page appears for the selected server.
From the Server Details page, click the Parameters hyperlink.
The Server Details page refreshes to display a summary view table of the parameter settings for the selected server.
Verify that the parameter settings are correct.
For a detailed list of the necessary parameters for Oracle Advanced Outbound Telephony servers, refer to the Implementing the Oracle Advanced Outbound Telephony Servers section.
Using the Oracle Advanced Outbound Telephony HTML Admin Console to Diagnose Issues
You can use the Oracle Advanced Outbound Telephony HTML Administration console to:
Checking Campaign Activity Parameters in Oracle Advanced Outbound Telephony
Follow the steps below to check campaign activity parameters in the Oracle Advanced Outbound Telephony HTML Admin console.
HTML Login URL
Oracle Advanced Outbound Telephony Administrator
From the View field, select My Campaign Activities to check the parameters for a campaign activity you created in Oracle Marketing.
From the Name column of the summary view table on the Campaign Activities page, click the name of the campaign activity you want to check.
Note: If you do not see the desired campaign activity displayed, check Oracle Marketing to ensure the campaign activity is configured properly.
The Call Center Campaign Activity Details page appears, displaying a Call Center Parameters area.
In the Calendar field, verify the correct the user defined calendar is associated with the selected campaign activity (if you want one to be).
Do you want the CTI Disabled checkbox selected?
Checking this box disables computer telephony for the selected campaign activity. With CTI disabled, only manual dialing is allowed.
Is the On Hold checkbox selected?
Selecting the On Hold checkbox will place on hold a campaign activity with a status of Active.
Once you have made changes, click Update.
You return to the Call Center Campaign Activity Details page and a confirmation note is displayed, stating that the configuration was successful.
Checking Target Group Parameters in Oracle Advanced Outbound Telephony
Follow the steps below to check target group parameters in the Oracle Advanced Outbound Telephony HTML Admin console.
HTML Login URL
Oracle Advanced Outbound Telephony Administrator
From the View field, select My Campaign Activities.
From the Name column of the summary view table on the Campaign Activities page, click the name of the campaign activity containing the target group you want to check.
The Call Center Campaign Activity Details page appears, displaying a Call Center Parameters area.
From the side navigation bar, click Campaign Activity Target Groups.
The Interaction Center Campaign Activity Target Groups page appears, displaying a summary view table of the target group associated to the selected campaign activity.
From the Name column of the summary view table, click the name of the target group for which you want to check parameters.
The Interaction Center Campaign Activity Target Group Details page appears.
From the drop-down list in the Dialing Method field, verify your dialing method selection or select to default to the campaign activity's dialing method.
From the drop-down list in the Target Group Priority field, check the priority assigned to the target group.
Note: Priorities determine the relative priority of a target group or subset in comparison to other target groups and subsets.
Note: Highest through Medium level priorities are preemptive; all target groups with priorities in this range will be exhausted in priority sequence before the next priority level is used. Low and Lowest priorities are used in round-robin fashion; each of these target groups will release its quantum of records, then the next target group in the same priority level will be used.
Note: At this time, Oracle Marketing does not support multiple target groups.
From the drop-down list in the Recycling Algorithm field, verify you have selected the recycling algorithm you want to use for the target group.
From the drop-down list in the Record Filter field, check that you have selected the record filter rule you want to use with the target group.
From the drop-down list in the Validation Rule field, check that you have selected the validation rule you want to use with the target group.
From the drop-down list in the Calendar field, verify that you have selected the user defined calendar you want to use with the target group (if you want to use one).
From the drop-down list in the Release Strategy field, verify you have selected the correct release strategy to use with the target group.
Note: The quantum release strategy supports proportional calling across all target groups and subsets associated with the campaign strategy and it is considered the baseline strategy for an Oracle Advanced Outbound Telephony service. This means that when Oracle Advanced Outbound Telephony is distributing customer records to agents, it releases a quantum number of records from each target group or subset before it goes on to the next target group or subset in the campaign.
Note: The Quota Release Strategy stops releasing records from a target group or subset when a pre-defined limit has been reached. This strategy disburses records from a target group or subset based on the quota that is assigned to a target group or subset.
Check the number of records you set as your limit in either the Quantum or Quota field.
Note: If you selected Quota in step 11, you must select the hour and minute reset period from the drop-down list in each respective field. These fields specify the hours and minutes before the quota value is reset to zero (0).
Once you have made your changes, click Update.
The Interaction Center Campaign Activity Target Group Details pages updates to reflect the new dialing parameters and displays a confirmation stating that the dialing parameters have been configured successfully.
Using the Oracle Marketing HTML Admin Console to Diagnose Issues
You can use the Oracle Marketing HTML Administration console to:
Checking Campaign Activity and Target Group Parameters
Follow the steps below to check campaign activity and target group parameters in Oracle Marketing.
HTML Login URL
Oracle Marketing Super User
Click the Campaign tab.
The Campaigns page appears, displaying a summary view table of all existing campaigns.
From the table, select the campaign containing the campaign activity in which you want to view parameters.
The Campaign Details page appears.
From the side navigation panel, click Execution.
The Campaign Details - Campaign Activity page appears, displaying a summary view table of all campaign activities associated with the selected campaign.
From the summary view table, click the campaign activity in which you want to view parameters.
The Campaign Activity – Details page appears.
Verify the campaign activity parameters.
For more information on campaign activity, refer to the Oracle Marketing User Guide.
From the side navigation panel, click Execution > Target Group.
The Campaign Activity Details - Target Group page appears.
Verify the target group parameters.
For more information on target group parameters, refer to the Configuring an Oracle Marketing List, and Configuring an Oracle Marketing Target Group topics in this guide.
Checking the Agent to Campaign Activity Assignment
Follow the steps below to check the agent to campaign activity assignment in Oracle Marketing.
HTML Login URL
Oracle Marketing Super User
From the Oracle Marketing homepage, click the Administration tab.
Click the Sales subtab.
From the side panel under Call Center, click the Agent Assignments hyperlink.
The Resource Assignment window appears.
In the Campaign Name field, type the name or a partial name of the appropriate campaign activity. You can use the % sign to substitute for missing characters.
Click Go.
From the list, select the appropriate campaign activity and click OK.
The campaign activity name populates the Campaign Name field. and the Current Resource Assignments table refreshes to display all the agents currently assigned to the selected campaign activity.
Verify that the correct agents are assigned.
Using Java Console Logs to Diagnose Issues
The Java Console is a pop-up window that you can configure to appear whenever you launch an Oracle Forms session.
You can use the Java console logs to track application trace messages, to verify server information from the agent side, and to diagnose problems when dealing with multiple servers.
The following are the most commonly experienced errors with Oracle Advanced Outbound Telephony:
Servers Won't Start
Validation Failed
Validation Doesn't Fail, but no Records are Returned
Can't See Oracle Advanced Outbound Telephony Media Node
No Campaign Activities Appear
Queue Count Doesn't Update
Central Server Isn't Pulling Records
No More Callable Records for an Agent
Agent Cannot Make Progressive Calls
Oracle Telephony Manager Cannot Get Media Items From Oracle Advanced Outbound Telephony
Oracle Telephony Manager Received two Media Items for the Same Oracle Advanced Outbound Telephony Call
Media Items Were Not Closed
Oracle Advanced Outbound Telephony Fails in the Dial Server, but no "Enabler Related Errors" are Generated
Target Group Stuck in Validating Status
No Longer Seeing IEO Log Messages
No database connection.
Check the .dbc file. Make sure that NUM_CONNECTIONS in dbc file is set high enough.
ICSM setup is incorrect.
Check server arguments (ICSM - Advanced link).
Server or plugin parameters are incorrect.
Verify the server parameters are correct
Ensure the values set in ICSM (Advanced link) are correct
Ensure Java arguments set to at least “-ms256m mx256m”
Ensure server node is running (ICSM).
Start the server at the local node using a local script.
Make sure your Central Server and Dial Server(s) are running and set your trace level to All.
Make sure your Oracle Marketing Campaign and Campaign Activity are Active.
Make sure your Oracle Marketing Target Group has a status of Locked.
Make sure the Interaction Center Ready column of your Oracle Advanced Outbound Telephony Campaign Activity is set to Yes.
Make sure your Oracle Advanced Outbound Telephony Campaign Activity Target Group parameters are correctly configured.
Oracle Marketing Source Type does not exist or Oracle Marketing Source Type does not have required columns
Check the logs files for the following error messages:
Error messages titled ‘Validation Error'.
“ORA-00942: table or view does not exist”
“Source type view XXX has not been created in Oracle Marketing”
“ORA-00904: invalid column name”
“Source type view XXX is missing one or more columns.”
Create or update the Oracle Marketing list source type. Ensure all the required fields are present.
Insufficient Table Space
Check the log files the following error message:
“ORA-01654: unable to extend index IEC.IEC_G_RETURN_ENTRIES_N3 by 8195 in tablespace IECX”
Ask your DBA to extend the table space specified in the error message.
Validation Doesn't Fail, but no Records are Returned
Missing Time Zone Information
Missing country code or area code
Phone number contains invalid characters
Check time zone and time in the user record
Check country code/area code in user record
Check user phone numbers
Add missing data/correct error data
Create validation rules to insert time zone
Create validation rules to add country code/area code
Can't See Oracle Advanced Outbound Telephony Media Node
The agent's profile options are not set correctly
The profile settings for the agent.
Self-Service Login URL (Forms).
Select System Administrator responsibility.
Click Profile > System.
Type agent's user ID and click Find.
Ensure the following profile options are set:
“IEU: Queue Order: Oracle Advanced Outbound Telephony” – set to “1”
“IEU: Queue: Oracle Advanced Outbound Telephony” – set to “Yes”
The agent, or group to which the agent belongs, is not assigned to a campaign activity
Ensure that the agent (or group that the agent belongs to) is assigned to a campaign activity
The concurrent manager has not been started
Ensure the concurrent manager is running
The concurrent manager is not running.
Ensure the concurrent manager has been started.
Note: The queue count only changes when the concurrent manager is running. The count only reflects changes in:
the number of records that are set to “Do Not Use” (by recycling).
the number of records that are no longer calendar callable (fall outside the callable time parameters as defined in the calling calendar).
Central Server Isn't Pulling Records
Server Time Zone is not configured.
Check the log messages for any error messages with title “Calendar Error”.
Configure Server Time Zone in the Oracle Advanced Outbound Telephony Administration console.
Country calendar does not exist.
Check the country calendar in the Oracle Advanced Outbound Telephony Administration console.
Create the country calendar.
No More Callable Records for an Agent
Check if the correct campaign shows up on the UWQ desktop for this agent.
Check the campaign record count.
Check if the status of the campaign is active.
Ensure the status of the corresponding target group is “Executing”.
Check the Campaign Record Details report to see if there are any records currently available.
Check the Timezone report to see if there are any callable records at this time.
Check if the Central Server is running. Wait for at least 2 * Retrieve interval and see if this happens again.
Check if a calling calendar has been assigned to the campaign.
Check if a calling calendar has been assigned to the target group.
Check if the release strategy and the corresponding value has been set.
Check Quantum - Quantum value and Quota - Quota value.
Check if the database server time zone has been set.
Check the refresh and recover parameter values on the central server.
Agent Cannot Make Progressive Calls
Middleware configuration error.
CCC is not running.
Check for these errors in dial server trace file:
“no middleware”
“cannot connect to proxy server …”
“cannot connect to cti server…”
“enabler related error …”
Check middleware IP address, port number, CTI server ID, PBX name, etc.
Ensure Call Center Connectors is running.
Oracle Telephony Manager Cannot Get Media Items From Oracle Advanced Outbound Telephony
Middleware configuration error.
If middleware is CT_CONNECT, make sure "enable server application data" in CT_CONNECT configuration is selected.
Oracle Telephony Manager Received Two Media Items for the Same Oracle Advanced Outbound Telephony Call
Middleware configuration error.
If middleware is CT_CONNECT, make sure "enable server application data" in CT_CONNECT configuration is selected.
Middleware configuration error.
If middleware is CT_CONNECT, make sure "enable server application data" in CT_CONNECT configuration is selected.
Oracle Advanced Outbound Telephony Fails in the Dial Server, but no "Enabler Related Errors" Are Generated
Examine the Call Center Connectors log (log level must be 4). Determine if the number actually dialed has proper "prefix", (external access prefix, long distance prefix, etc...).
If no prefix, check middleware configuration, domestic dialing prefix, international dialing prefix, outgoing prefix, site area code, site country code, site overlay, site internal number length, site local number maximum length, etc...
This topic group contains information to help you resolve common problems with the AO Central Server.
The following topics are included:
No database connection
Check the .dbc file, make sure that NUM_CONNECTIONS in dbc file is high enough.
ICSM setup is incorrectly
Check server arguments.
Server or plugin parameters are incorrect.
Check if the parameters have been added
Check if the values set in ICSM (Advanced link) are correct
Make sure the server node is running (ICSM). If the server is not running, try to start the server again from ICSM
Try to start the server at the local node using a local script.
DB Server Time Zone is incorrect.
Correct the DB Server Time Zone using the AO Administrator utility, under the Administration Tab.
Another OCS is running.
Use ICSM to check/change server status.
AOCS Starts but Does not get Records
Make sure the activity has been marked for Telemarketing.
Make sure the activity has a status of 'LOCKED'.
Make sure the target group has been generated for the activity using lists that have been generated using one of the two supported source types.
Make sure the target group has a status of 'LOCKED' in order to validate target group.
Make sure that the list is executing.
This topic group contains information to help you resolve common problems with the AO Dial Server.
The following topics are included:
No database connection
Check the dbc file
ICSM setup is incorrect
Check server arguments
Server parameters are incorrect
Another Dial Server is running with the same server id
Use ICSM to check/change server status
Make sure that the agent is assigned the System Profile for AO
Use the System Administrator Responsibility, and click System, then Profile. Set the AO profile parameters to Yes.
Make sure that the agent is assigned to one of the running campaign activities.
Make sure that there are valid executing campaign activities.
Check if the agent/campaign activity association is set.
Check if the campaign activity/target group is in the correct state and that the calendar is callable.
Check the number of available records in this target group.
Records may not be in cache, check if Central Server has been started; check the execution interval of the Refresh plugin.
Wait and test again.
Make sure that the correct extension was provided.
Make sure that the middlewares in your server group are configured correctly (use the Call Center HTML Administrator responsibility, "Middleware" tab).
Check the IP address and the port of Telephony Adapter Server.
Make sure that the middleware parameters are set for Oracle Advanced Outbound Telephony use (including setting the "Use with Oracle Advanced Outbound Telephony" parameter to Yes.
Make sure that the Telephony Adapter Server is up.
Receiving Too Few Records for Target Group
Check to ensure there is a country calendar defined for each country code used in the target group.
Check to ensure each target groups source type matches the source type defined for the target group.
Check to ensure the source type defined for the target group matches the source type defined for the target groups.
If list validation fails, you should check the AO central server log files (via Oracle Call Center Technology Admin - ICSM tab). If you see a message indicating that the database view is missing, you must go into the Oracle Marketing Administration console and select data source types for Organization Contacts and Person List, then click Update.
Make sure your Central Server and Dial Server(s) are running and set your trace level to All.
Make sure your Oracle Marketing campaign and campaign activity are Active.
Make sure your Oracle Marketing Target Group has a status of Locked.
Make sure the Interaction Center Ready column of your Oracle Advanced Outbound Telephony Campaign Activity is set to Yes.
Make sure your Oracle Advanced Outbound Telephony Campaign Activity Target Group parameters are correctly configured.
Oracle Marketing source types do not exist or do not have the required columns
Use the steps below to diagnose this problem:
HTML Login URL
Call Center HTML Administration
Click the ICSM tab.
The Server Group List page appears, displaying a summary view table os any existing server groups.
From the top bar, click Logs.
The Summary Report for Servers page appears, displaying a summary view table of all the servers in the group.
From the list of servers, locate the appropriate server and click the hyperlink to display the log error messages.
The Detail Report page appears, displaying a summary view table of all the error messages for the selected server.
In the Title column of the summary view table, search for error messages titled "Validation Error".
Click the Record ID hyperlink to display the error message in the Message Description text box.
Check for the following error messages:
ORA-00942: table or view does not exist
Source type view XXX has not been created in Oracle Marketing
Ora-00904: invalid column name
Source type view XXX is missing one or more columns
If you locate these error messages, you must create or update the Oracle Marketing Source type view. Follow the steps in the Defining List Source Types topic to do so.
Target Group Stuck in Validating Status
If a target group stays in the 'Validating' status for an extended period of time, it is likely the case that validation has failed, but was unable to update the status to "Failed Validation". This usually occurs when there is a tablespace issue and the error message could not be logged. After the tablespace is increased, the target group will automatically resume validation.
Check:
Check your table space sizes.
Although the exact tablespace configuration will depend on what applications are running in the database and needs to be reviewed by a dba, Oracle recommends the following sizes for a sytem configured exclusively for AO:
TEMP (500 MB)
USER_DATA and USER_IDX (300 MB)
No Longer Seeing IEO Log Messages
If you stop seeing new error messages (beyond a specific date) in the IEO Log Viewer, it is very likely that you need to extend your tablespace. This is especially true if you are running into other errors that cannot be explained since other parts of the application may be failing, but are unable to log the errors due to the tablespace issues.
Check:
Check your table space sizes.
Although the exact tablespace configuration will depend on what applications are running in the database and needs to be reviewed by a dba, Oracle recommends the following sizes for a sytem configured exclusively for AO:
TEMP (500 MB)
USER_DATA and USER_IDX (300 MB)