This appendix contains troubleshooting guidelines for managing a replication environment.
This appendix contains these topics:
If you think a database link is not functioning properly, then you can drop and re-create it using the Oracle Enterprise Manager Console, SQL*Plus, or another tool.
Make sure that the database link name is the same as the global name of the target database.
Make sure that the scheduled interval is what you want.
Make sure that the scheduled interval is not shorter than the required execution time.
If you used a connection qualifier in a database link to a given site, then the other sites that link to that site must have the same connection qualifier. For example, suppose you create a database link as follows:
CREATE DATABASE LINK dbs1.world@myethernet CONNECT TO repadmin
IDENTIFIED BY secret USING 'connect_string_myethernet'
All the sites, whether masters or materialized views, associated with dbs1.world@myethernet
must include myethernet
as the connection qualifier.
See Also:
Oracle Database Administrator's Guide for more information database links and connection qualifiersProblems can arise in a multimaster replication system. The following sections discuss some problems and ways to solve them:
If you add a new master site to a master group, and the appropriate objects are not created at the new site, then try the following:
Ensure that the necessary private database links exist between the new master site and the existing master sites. If you used the Replication Management tool's Setup Wizard to setup your sites, then you should not have any problems. You must have links both to the new site from each existing site, and from the new site to each existing site.
Make sure that the administrative requests at all sites have completed successfully. If requests have not been executed yet, then you can manually execute pending administrative requests to complete the operation immediately.
If you create a new master group object or alter the definition of a master group object at the master definition site and the modification is not propagated to a master site, then first ensure that the administrative requests at all sites have completed successfully. If requests are pending execution, then you can manually execute them to complete the operation immediately.
When you execute DDL statements through the replication API, Oracle executes the statements on behalf of the user who submits the DDL. When a DDL statement applies to an object in a schema other than the submitter's schema, the submitter needs appropriate privileges to execute the statement. In addition, the statement must explicitly name the schema. For example, assume that you supply the following as the ddl_text
parameter to the DBMS_REPCAT.CREATE_MASTER_ REPOBJECT
procedure:
CREATE TABLE oe.new_employees AS SELECT * FROM hr.employees WHERE ...;
Because each table name contains a schema name, this statement works whether the replication administrator is oe
, hr
, or another user, as long as the administrator has the required privileges.
Note:
Qualify the name of every schema object with the appropriate schema.If you make an update to your data at a master site, and that change is not asynchronously propagated to the other sites in your replication environment, then try the following:
Use the Replication Management tool in the Oracle Enterprise Manager Console to check whether the corresponding deferred transaction has been pushed to the destination. If not, then you can also check to see how much longer it will be before the scheduled link pushes the queue to the destination site. If you do not want to wait for the next scheduled push across a link, then you can execute deferred transaction manually.
If a scheduled link's interval has passed and corresponding deferred transactions have not been pushed, then check the corresponding job for the link.
Even after propagating a deferred transaction to a destination, it might not execute because of an error. Check the DEFERROR
data dictionary view at the destination site for errors.
See Also:
Oracle Database Advanced Replication Management API Reference for information about modifying tables without replicating the modifications, which might be necessary when you need to manually synchronize the data in replicated tablesIf you receive the deferred_rpc_quiesce
exception when you attempt to modify a replicated table, then the master group to which your replicated object belongs is quiescing or quiesced. To proceed, your replication administrator must resume replication activity for the master group.
A single update statement applied to a replicated table can update zero or more rows. The update statement causes zero or more update requests to be queued for deferred execution, one for each row updated. This distinction is important when constraints are involved, because Oracle effectively performs constraint checking at the end of each statement. While a bulk update might not violate a uniqueness constraint, for example, some equivalent sequence of individual updates might violate uniqueness.
If the ordering of updates is important, then update one row at a time in an appropriate order. This lets you define the order of update requests in the deferred transactions queue.
If you add an object such as a package, procedure, or view to a master group, then the status of the object must be valid. If the status of an object is invalid, then recompile the object or drop and re-create the object before adding it to a master group. Check the DBA_REPOBJECT
data dictionary view for the status of replication objects.
When you generate replication support for a table, Oracle activates an internal trigger at the local site. EXECUTE
privileges for most of the packages involved with replication, such as DBMS_REPCAT
and DBMS_DEFER
, need to be granted to replication administrators and users that own replicated objects. The Replication Management tool's Setup Wizard and the DBMS_REPCAT_ADMIN
package both perform the grants needed by the replication administrators for many typical replication scenarios. When the owner of a replicated object is not a replication administrator, however, you must explicitly grant EXECUTE
privilege on DBMS_DEFER
to the object owner.
If you discover an unexpected unresolved conflict, and you were mixing procedural and row-level replication on a table, then carefully review the procedure to ensure that the replicated procedure did not cause the conflict. Complete the following checks:
Ensure that ordering conflicts between procedural and row-level updates are not possible.
Check if the replicated procedure locks the table in EXCLUSIVE
mode before performing updates or uses some other mechanism of avoiding conflicts with row-level updates.
Check that row-level replication is disabled at the start of the replicated procedure and reenabled at the end.
Ensure that row-level replication is reenabled even if exceptions occur when the procedure executes.
Check to be sure that the replicated procedure executed at all master sites.
You should perform similar checks on any replicated triggers that you have defined on replicated tables.
If deferred transactions at a site are not being pushed to their destinations, then the following sections explain some possible causes for the problem:
When you create a scheduled link, Oracle adds a corresponding job to the site's job queue. If you have scheduled a link to push deferred transactions at a periodic interval, and you encounter a problem, then you should first be certain that you are not experiencing a problem with the job queue.
When you use synchronous replication, Oracle uses a distributed transaction to ensure that the transaction has been properly committed at the remote site. Distributed transactions use two-phase commit. Asynchronous replication does not use two-phase commit.
See Also:
Oracle Database Administrator's Guide for information on diagnosing problems with distributed transactionsIf you notice that transactions are not being pushed to a given remote site, then you might have a problem with how you have specified the destination for the transaction. When you create a scheduled link, you must provide the full database link name.
Having the wrong view definitions can lead to erroneous deferred transaction behavior. The DEFCALLDEST
and DEFTRANDEST
views are defined differently in catdefer.sql
and catrepc.sql
. The definitions in catrepc.sql
should be used whenever replication is used. If catdefer.sql
is ever (re)loaded, then ensure that the view definitions in catrepc.sql
are subsequently loaded.
There are a number of problems that might happen with materialized view sites in a replication system. The following sections discuss some problems and ways to troubleshoot them:
If you unsuccessfully attempt to create a new object at a materialized view site, then try the following:
For an updatable materialized view, check that the associated master table or master materialized view has a materialized view log.
Make sure that you have the necessary privileges to create the object. For a materialized view, you need SELECT
privilege on the master table or master materialized view and its materialized view log. See "Assign Privileges" for more information.
If you are trying to add an existing materialized view to a materialized view group, then try re-creating the materialized view when you add it to the group.
If you are trying to create a fast refresh primary key or subquery materialized view, then make sure that the materialized view log on the master table or master materialized view logs primary keys.
If you are trying to create a fast refresh rowid materialized view, then make sure that the materialized view log on the master table logs rowids.
Check if the materialized view log has the required columns added for subquery materialized views. See "Logging Columns in the Materialized View Log" for information.
Check if the materialized view log exists for all tables that are involved in a fast refresh materialized view. If the materialized view contains a subquery, then each table referenced in the subquery should have a materialized view log.
If you receive and error stating that Oracle is unable to initialize the extent in the temporary tablespace when you try to instantiate a deployment template offline, then you might need to adjust the datafile for the temporary database so that it auto extends.
For example, issue the following statement to adjust the datafile:
ALTER DATABASE TEMPFILE '/u02/oracle/rbdb1/temp.dbf' AUTOEXTEND ON NEXT 10M;
After you have made this adjustment, instantiate the deployment template offline at the materialized view site.
The following sections explain several common materialized view refresh problems.
Several common factors can prevent the automatic refresh of a group of materialized views:
The lack of a job queue process at the materialized view database
An intervening network or server failure
An intervening server shutdown
When a materialized view refresh group is experiencing problems, ensure that none of the preceding situations is preventing Oracle from completing group refreshes.
When Oracle fails to refresh a group automatically, the group remains due for its refresh to complete. Oracle will retry an automatic refresh of a group with the following behavior:
Oracle retries the group refresh first one minute later, then two minutes later, four minutes later, and so on, with the retry interval doubling with each failed attempt to refresh the group.
Oracle does not allow the retry interval to exceed the refresh interval itself.
Oracle retries the automatic refresh up to sixteen times.
If after 16 attempts to refresh a refresh group Oracle continues to encounter errors, then Oracle considers the group broken. The General page of the Refresh Group property sheet in Schema Manager indicates when a refresh group is broken. You can also query the BROKEN
column of the USER_REFRESH
and USER_REFRESH_CHILDREN
data dictionary views to see the current status of a refresh group.
The errors causing Oracle to consider a materialized view refresh group broken are recorded in a trace file. After you correct the problems preventing a refresh group from refreshing successfully, you must refresh the group manually. Oracle then resets the broken flag so that automatic refreshes can happen again.
See Also:
The name of the materialized view trace file is of the form jn, where n is operating system specific. See the Oracle documentation for your operating system for the name on your system.In some cases, a materialized view log for a master table or master materialized view might be purged during the creation of a materialized view at a new materialized view site. When this happens, you might encounter the following errors:
ORA-12004 REFRESH FAST cannot be used for materialized view materialized_view_name ORA-12034 materialized view log on materialized_view_name younger than last refresh
See Also:
"Avoiding Problems When Adding a New Materialized View Site" for a complete description of how to avoid this problem.If you encounter a situation where Oracle continually refreshes a group of materialized views, then check the group's refresh interval. Oracle evaluates a group's automatic refresh interval before starting the refresh. If a group's refresh interval is less than the amount of time it takes to refresh all materialized views in the group, then Oracle continually starts a group refresh each time the job queue process checks the queue of outstanding jobs.
If a materialized view log at a master site or master materialized view site is growing too large, then check to see whether a network or site failure has prevented the master site or master materialized view site from becoming aware that a materialized view has been dropped. You might need to purge part of the materialized view log or unregister the unused materialized view site.
See Also:
Oracle Database Advanced Replication Management API Reference for more information about managing materialized view logsIf you have a problem refreshing a materialized view, then try the following:
Check the NEXT_DATE
value in the DBA_REFRESH_CHILDREN
view to determine if the refresh has been scheduled.
If the refresh interval has passed, then check the DBA_REFRESH
view for the associated job number for the materialized view refresh and then diagnose the problem with job queues.
Check if there are job queue process running. Check the JOB_QUEUE_PROCESSES
initialization parameter, query the DBA_JOBS_RUNNING
view, and use your operating system to check if the job queue processes are still running.
You also might encounter an error if you attempt to define a master detail relationship between two materialized views. You should define master detail relationships only on the master tables by using declarative referential integrity constraints. The related materialized views should then be placed in the same refresh group to preserve this relationship. However, you can define deferred (or deferrable) constraints on materialized views.
If there are any outstanding conflicts recorded at the master site or master materialized view site for the materialized views, then you can only refresh the materialized views by setting the parameter REFRESH_AFTER_ERRORS
to true
. This parameter can be set when you create or alter a materialized view refresh group. There is a corresponding parameter for the Replication Management tool's property sheets.
Materialized views in the same refresh groups have their rows updated in a single transaction. Such a transaction can be very large, requiring either a large rollback segment at the materialized view site, with the rollback segment specified to be used during refresh, or more frequent refreshes to reduce the transaction size.
If Oracle error ORA-12004
occurs, then the master site or master materialized view site might have run out of rollback segments when trying to maintain the materialized view log, or the materialized view log might be out of date. For example, the materialized view log might have been purged or re-created.
Complete refreshes of a single materialized view internally use the TRUNCATE
feature to increase speed and reduce rollback segment requirements. However, until the materialized view refresh is complete, users might temporarily see no data in the materialized view. Refreshes of multiple materialized views (for example, refresh groups) do not use the TRUNCATE
feature.
Reorganization of the master table (for example, to reclaim system resources) should TRUNCATE
the master table to force rowid materialized views to do complete refreshes. Otherwise, the materialized views have incorrect references to master table rowids. You use the BEGIN_TABLE_REORGANIZATION
and END_TABLE_REORGANIZATION
procedures in the DBMS_MVIEW
package to reorganize a master table. See the Oracle Database Advanced Replication Management API Reference for more information.
If while refreshing you see an ORA-942
(table or view does not exist), then check your database links and make sure you still have the required privileges on the master table or master materialized view and the materialized view log.
If a fast refresh was succeeding but then fails, then check whether:
The materialized view log was truncated, purged, or dropped.
You still have the required privileges on the materialized view log.
If a force refresh takes an inordinately long time, then check if the materialized view log used by the refresh has been dropped.
If the materialized view was created with BUILD
DEFERRED
, and its first fast refresh fails, then make sure a previous complete refresh was done successfully before checking for other problems.
See Also:
Oracle Database Advanced Replication Management API Reference for information about managing materialized view logs