Note:

Set up an Oracle Data Guard at the Pluggable Database Level

Introduction

Oracle Data Guard protects individual pluggable databases (PDBs) rather than the whole container database (CDB). This means an Oracle Data Guard per Pluggable Database (DG PDB) configuration will have two primary CDBs instead of one primary CDB and one standby CDB. Each CDB will contain PDBs open read-write (eventually protected by a target PDB on the remote CDB) and mounted PDBs protecting the corresponding PDBs in the remote CDB.

We will learn how to set up a data guard at the PDB level from CDB1 to CDB2 and conversely while both the CDBs are in read-write mode. Further, we will see switchover and failover at the PDB level without any outage at either the container level or the rest of the PDBs in the container.

This tutorial describes how to use DG PDB, an Oracle Data Guard for PDBs, rather than CDBs.

Note: DG PDB is not supported with the Oracle Cloud Infrastructure (OCI) tooling on Oracle Base Database Service.

Traditional per-CDB Architecture

image

Oracle Data Guard per PDB (DG PDB) Architecture

image

Features and Characteristics

Use Cases

Objectives

Prerequisite

Task 1: Set up a Data Guard at PDB Level from CDB1 to CDB2

Primary PDB: CDB1_PDB1 AT CDB1_ASH

Standby PDB: CDB1_PDB1 AT CDB2_SJ

Environment:

OCI Region Ashburn (ASH) San Jose (SJ)
Oracle Base Database DB System DB System
Oracle Version 23.3.0.23 23.3.0.23
Hostname ashvmdb1 sjvmdb2
DB Unique Name CDB1_ASH CDB2_SJ
CDB Name CDB1 CDB2
Primary PDB CDB1_PDB1, CDB1_PDB2 CDB2_PDB3, CDB2_PDB4
Standby PDB CDB2_PDB3 CDB1_PDB1
VCN/Subnet (DEFAULT) VCN_ASH/public subnet-VCN_ASH VCN_SJ/public subnet-VCN_SJ
Private IP 10.0.0.192 192.168.0.202

Task 1.1: Prepare the Databases

  1. Create two CDBs and two PDBs through the OCI Console, both of which are Oracle Database 23ai databases that are Transparent data encryption (TDE) encrypted.

  2. Enable flash logging and force logging on both databases.

  3. Configure log archive destination to Fast Recovery Area (FRA).

  4. Enable Oracle Data Guard broker.

Provision CDB1:

col FORCE_LOGGING for a5
set lines 300 pages 100
select name,open_mode,LOG_MODE,FORCE_LOGGING,FLASHBACK_ON,DB_UNIQUE_NAME from gv$database;

NAME OPEN_MODE LOG_MODE FORCE FLASHBACK_ON DB_UNIQUE_NAME
--------- -------------------- ------------ ----- ------------------ ------------------------------
CDB1 READ WRITE ARCHIVELOG YES NO CDB1_ASH

SQL> show pdbs
CON_ID CON_NAME OPEN MODE RESTRICTED
---------- ------------------------------ ---------- ----------
2 PDB$SEED READ ONLY NO
3 CDB1_PDB1 READ WRITE NO
4 CDB1_PDB2 READ WRITE NO

SQL> ALTER DATABASE FORCE LOGGING;
SQL> ALTER DATABASE FLASHBACK ON;
SQL> alter system set log_archive_dest_1='LOCATION=USE_DB_RECOVERY_FILE_DEST VALID_FOR=(ALL_LOGFILES,ALL_ROLES)' scope=both sid='\*';
SQL> ALTER SYSTEM SET DG_BROKER_CONFIG_FILE1='+DATA/CDB1_ash/PARAMETERFILE/dr1CDB1.dat' scope=both sid='\*';
SQL> ALTER SYSTEM SET DG_BROKER_CONFIG_FILE2='+DATA/CDB1_ash/PARAMETERFILE/dr2CDB1.dat' scope=both sid='\*';
SQL> alter system set standby_file_management = AUTO scope = both;
SQL> ALTER SYSTEM SET dg_broker_start=TRUE scope=both sid='\*';

Provision CDB2:

col FORCE_LOGGING for a5
set lines 300 pages 100
select name,open_mode,LOG_MODE,FORCE_LOGGING,FLASHBACK_ON,DB_UNIQUE_NAME from gv$database;

NAME OPEN_MODE LOG_MODE FORCE FLASHBACK_ON DB_UNIQUE_NAME
--------- -------------------- ------------ ----- ------------------ ------------------------------
CDB2 READ WRITE ARCHIVELOG YES NO CDB2_SJ

SQL> show pdbs
CON_ID CON_NAME OPEN MODE RESTRICTED
---------- ------------------------------ ---------- ----------
2 PDB$SEED READ ONLY NO
4 CDB2_PDB3 READ WRITE NO
5 CDB2_PDB4 READ WRITE NO

SQL> ALTER DATABASE FORCE LOGGING;
SQL> ALTER DATABASE FLASHBACK ON;
SQL> alter system set log_archive_dest_1='LOCATION=USE_DB_RECOVERY_FILE_DEST VALID_FOR=(ALL_LOGFILES,ALL_ROLES)' scope=both sid='\*';
SQL> ALTER SYSTEM SET DG_BROKER_CONFIG_FILE1='+DATA/CDB2_sj/PARAMETERFILE/dr1CDB2.dat' scope=both sid='\*';
SQL> ALTER SYSTEM SET DG_BROKER_CONFIG_FILE2='+DATA/CDB2_sj/PARAMETERFILE/dr2CDB2.dat' scope=both sid='\*';
SQL> ALTER SYSTEM SET dg_broker_start=TRUE scope=both sid='\*';
SQL> alter system set standby_file_management = AUTO scope = both;

Task 1.2: Establish a Transparent Network Substrate (TNS) Connection between both CDBs

  1. Validate the connection between both the database nodes.

    curl -sv telnet://10.0.0.192:1521
curl -sv telnet://192.168.0.202:1521

  2. Configure the network configuration files on each host to facilitate connections to the source and target databases.

    $vi $ORACLE_HOME/network/admin/tnsnames.ora

CDB1_ASH =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = 10.0.0.192)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = CDB1_ASH.sub02090909270.vcnash.oraclevcn.com)
)
)

CDB2_SJ =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.0.202)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = CDB2_SJ.sub02090543360.vcnsj.oraclevcn.com)
)
)

Task 1.3: Establish a Passwordless Connection using a Wallet

  1. On each host, create a directory to contain the wallet and secure the wallet by setting appropriate protection on the directory.

    $ mkdir -p $ORACLE_HOME/dbs/wallets
$ chmod -R 700 $ORACLE_HOME/dbs/wallets

  2. Create wallets that contain the credentials required to create and manage the DG PDB configuration.

    $ mkstore -wrl $ORACLE_HOME/dbs/wallets/dgpdb -create
Oracle Secret Store Tool Release 23.0.0.0.0 - Production
Version 23.0.0.0.0
Copyright (c) 2004, 2022, Oracle and/or its affiliates. All rights reserved.

Enter password:
Enter password again:


[oracle@ashCDB1 ~]$ mkstore -wrl $ORACLE_HOME/dbs/wallets/dgpdb -createCredential CDB1_ash 'sys'
Oracle Secret Store Tool Release 23.0.0.0.0 - Production
Version 23.0.0.0.0
Copyright (c) 2004, 2023, Oracle and/or its affiliates. All rights reserved.

Your secret/Password is missing in the command line
Enter your secret/Password:
Re-enter your secret/Password:
Enter wallet password:
[oracle@ashCDB1 ~]$ mkstore -wrl $ORACLE_HOME/dbs/wallets/dgpdb -createCredential CDB2_sj 'sys'
Oracle Secret Store Tool Release 23.0.0.0.0 - Production
Version 23.0.0.0.0
Copyright (c) 2004, 2023, Oracle and/or its affiliates. All rights reserved.

Your secret/Password is missing in the command line
Enter your secret/Password:
Re-enter your secret/Password:
Enter wallet password:
[oracle@ashCDB1 ~]$ mkstore -wrl $ORACLE_HOME/dbs/wallets/dgpdb -listCredential
Oracle Secret Store Tool Release 23.0.0.0.0 - Production
Version 23.0.0.0.0
Copyright (c) 2004, 2023, Oracle and/or its affiliates. All rights reserved.

Enter wallet password:
List credential (index: connect_string username)
2: CDB2_sj sys
1: CDB1_ash sys

  3. Validate passwordless connection on both databases. In this tutorial, we have used the same password during database provisioning.

    • Configure sqlnet.ora to use the wallet.

    • Now, we have to add the wallet on the sqlnet.ora configuration that Oracle will use.

    vi $ORACLE_HOME/network/admin/sqlnet.ora

NAMES.DIRECTORY_PATH= (TNSNAMES, ONAMES, HOSTNAME)
WALLET_LOCATION =
    (SOURCE =
    (METHOD = FILE)
    (METHOD_DATA =
        (DIRECTORY = /u01/app/oracle/product/23.0.0.0/dbhome_1/dbs/wallets/dgpdb)
    )
)
SQLNET.WALLET_OVERRIDE = TRUE

Stop and restart each database and Listener to configure redo transport to use the wallet. Because there are no static services configured, use OS authentication on each host to do this.

Validate that client connections to the source and target container databases are now possible using the wallets by issuing the following commands on each host:

sqlplus /@CDB1_ash as sysdba
sqlplus /@CDB2_sj as sysdba

Task 1.4: Create Source and Target Database Oracle Data Guard Broker Configuration

Connect to the source and target container database using the wallet and create a configuration using Oracle Data Guard Command-Line Interface (DGMGRL). This is the same as for a normal Oracle Data Guard configuration. Only difference is, that we have to do this for both CDBs, as both are primary databases.

Task 1.5: Establish the Connection Between Configurations

This task is different from a normal Oracle Data Guard configuration. Normally after adding the primary database, we add the standby database but now we will use a different syntax. Using DGMGRL, connect to the source container database and establish a connection with the target database.

[oracle@ashCDB1 ~]$ dgmgrl /@CDB1_ash

DGMGRL> add configuration CDB2_sj connect identifier is CDB2_sj;
DGMGRL> show configuration verbose;
DGMGRL> show configuration verbose CDB2_sj;
DGMGRL> enable configuration all;

Task 1.6: Prepare the Databases for DG PDB

  1. Connect to each container database and open the PDBs if they are not already open.

  2. The Oracle Data Guard broker EDIT CONFIGURATION PREPARE DGPDB command assumes that the source container database and target container database configurations have been configured and are enabled.

  3. The command prompts will open to enter a password for the DGPDB_INT account for each of the container databases and then configure the internal structures required to provide Oracle Data Guard protection or change roles for a PDB.

    SQL> ALTER PLUGGABLE DATABASE ALL OPEN;
SQL> show pdbs

dgmgrl /@CDB1_ash
DGMGRL> EDIT CONFIGURATION PREPARE DGPDB;
Enter password for DGPDB_INT account at CDB1_ASH:
Enter password for DGPDB_INT account at CDB2_SJ:
Prepared Data Guard for Pluggable Database at CDB2_SJ.
Prepared Data Guard for Pluggable Database at CDB1_ASH.
DGMGRL>

    Note: For ths sake of simplicity, I used the same password as DB or TDE password but password can be diffrent.

Task 1.7: Configure Oracle Data Guard Protection for the Source PDB

  1. Configure Oracle Data Guard PDB level protection for the source PDB by configuring the target PDB in the container database.

  2. Run the ADD PLUGGABLE DATABASE command to create the target PDB in the target container database.

    Primary PDB: CDB1_PDB1 AT CDB1_ASH

    Standby PDB: CDB1_PDB1 AT CDB2_SJ

    On Target Container:

    SQL> show pdbs

CON_ID CON_NAME OPEN MODE RESTRICTED
---------- ------------------------------ ---------- ----------
2 PDB$SEED READ ONLY NO
4 CDB2_PDB3 READ WRITE NO
5 CDB2_PDB4 READ WRITE NO


dgmgrl /@CDB1_ash
DGMGRL> ADD PLUGGABLE DATABASE CDB1_PDB1 AT CDB2_sj SOURCE IS CDB1_PDB1 AT CDB1_ash PDBFILENAMECONVERT IS "'+DATA/CDB1_ash','+DATA/CDB2_sj'" 'keystore IDENTIFIED BY "WelC0me_123#"';
Pluggable Database "CDB1_PDB1" added
DGMGRL>

SQL> show pdbs
CON_ID CON_NAME OPEN MODE RESTRICTED
---------- ------------------------------ ---------- ----------
2 PDB$SEED READ ONLY NO
3 CDB1_PDB1 MOUNTED
4 CDB2_PDB3 READ WRITE NO
5 CDB2_PDB4 READ WRITE NO

Task 1.8: Copy the Datafiles to the Destination

Instantiate the target PDB by copying all files associated with the source PDB to the target PDB location.

  1. Identify a list of files on the source PDB.

    SQL> select FILE#,NAME from v$datafile where con_id=3;

FILE# NAME                                                                                                     CON_ID
---------- ---------------------------------------------------------------------------------------------------- ----------
8 +DATA/CDB1_ASH/10F14C2387F73F42E063C000000A77F3/DATAFILE/system.273.1160475741                          3
9 +DATA/CDB1_ASH/10F14C2387F73F42E063C000000A77F3/DATAFILE/sysaux.270.1160475747                          3
10 +DATA/CDB1_ASH/10F14C2387F73F42E063C000000A77F3/DATAFILE/undotbs1.271.1160475753                        3
12 +DATA/CDB1_ASH/10F14C2387F73F42E063C000000A77F3/DATAFILE/users.274.1160475857                           3

  2. Copy PDB datafiles on the target container.

    [oracle@sjCDB2 ~]$ rman target sys/WelC0me_123#@CDB1_ash auxiliary sys/WelC0me_123#@CDB2_sj

Recovery Manager: Release 23.0.0.0.0 - Production on Fri Feb 9 12:41:14 2024
Version 23.3.0.23.09

Copyright (c) 1982, 2023, Oracle and/or its affiliates.  All rights reserved.

connected to target database: CDB1 (DBID=3764657260)
connected to auxiliary database: CDB2 (DBID=3648579145)

RMAN> run {
allocate channel ch1 type disk;
backup as copy reuse datafile 8,9,10,12 auxiliary format NEW;
}
run {
2> allocate channel ch1 type disk;
3> backup as copy reuse datafile 8,9,10,12 auxiliary format NEW;
4> }
using target database control file instead of recovery catalog
allocated channel: ch1
channel ch1: SID=38 device type=DISK

Starting backup at 09-FEB-24
channel ch1: starting datafile copy
input datafile file number=00009 name=+DATA/CDB1_ASH/10F14C2387F73F42E063C000000A77F3/DATAFILE/sysaux.270.1160475747
output file name=+DATA/CDB2_SJ/10F14C2387F73F42E063C000000A77F3/DATAFILE/sysaux.271.1160484127 tag=TAG20240209T124201
channel ch1: datafile copy complete, elapsed time: 00:00:08
channel ch1: starting datafile copy
input datafile file number=00008 name=+DATA/CDB1_ASH/10F14C2387F73F42E063C000000A77F3/DATAFILE/system.273.1160475741
output file name=+DATA/CDB2_SJ/10F14C2387F73F42E063C000000A77F3/DATAFILE/system.272.1160484135 tag=TAG20240209T124201
channel ch1: datafile copy complete, elapsed time: 00:00:08
channel ch1: starting datafile copy
input datafile file number=00010 name=+DATA/CDB1_ASH/10F14C2387F73F42E063C000000A77F3/DATAFILE/undotbs1.271.1160475753
output file name=+DATA/CDB2_SJ/10F14C2387F73F42E063C000000A77F3/DATAFILE/undotbs1.274.1160484145 tag=TAG20240209T124201
channel ch1: datafile copy complete, elapsed time: 00:00:04
channel ch1: starting datafile copy
input datafile file number=00012 name=+DATA/CDB1_ASH/10F14C2387F73F42E063C000000A77F3/DATAFILE/users.274.1160475857
output file name=+DATA/CDB2_SJ/10F14C2387F73F42E063C000000A77F3/DATAFILE/users.275.1160484149 tag=TAG20240209T124201
channel ch1: datafile copy complete, elapsed time: 00:00:04
Finished backup at 09-FEB-24
released channel: ch1

  3. Check datafiles on the target PDB.

    SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
        2 PDB$SEED                      READ ONLY  NO
        3 CDB1_PDB1                     MOUNTED
        4 CDB2_PDB3                     READ WRITE NO
        5 CDB2_PDB4                     READ WRITE NO
SQL> alter session set container=CDB1_PDB1;
Session altered.

SQL>  select name from v$datafile;

NAME
--------------------------------------------------------------------------------
+DATA/MUST_RENAME_THIS_DATAFILE_8.4294967295.4294967295
+DATA/MUST_RENAME_THIS_DATAFILE_9.4294967295.4294967295
+DATA/MUST_RENAME_THIS_DATAFILE_10.4294967295.4294967295
+DATA/MUST_RENAME_THIS_DATAFILE_12.4294967295.4294967295

  4. Rename datafiles. The target filename can be obtained from the RMAN output logs.

    alter database rename file '+DATA/MUST_RENAME_THIS_DATAFILE_8.4294967295.4294967295' to '+DATA/CDB2_SJ/10F14C2387F73F42E063C000000A77F3/DATAFILE/system.272.1160484135';
alter database rename file '+DATA/MUST_RENAME_THIS_DATAFILE_9.4294967295.4294967295' to '+DATA/CDB2_SJ/10F14C2387F73F42E063C000000A77F3/DATAFILE/sysaux.271.1160484127';
alter database rename file '+DATA/MUST_RENAME_THIS_DATAFILE_10.4294967295.4294967295' to '+DATA/CDB2_SJ/10F14C2387F73F42E063C000000A77F3/DATAFILE/undotbs1.274.1160484145';
alter database rename file '+DATA/MUST_RENAME_THIS_DATAFILE_12.4294967295.4294967295' to '+DATA/CDB2_SJ/10F14C2387F73F42E063C000000A77F3/DATAFILE/users.275.1160484149';

SQL>  select name from v$datafile;

NAME
---------------------------------------------------------------------------------------
+DATA/CDB2_SJ/10F14C2387F73F42E063C000000A77F3/DATAFILE/system.272.1160484135
+DATA/CDB2_SJ/10F14C2387F73F42E063C000000A77F3/DATAFILE/sysaux.271.1160484127
+DATA/CDB2_SJ/10F14C2387F73F42E063C000000A77F3/DATAFILE/undotbs1.274.1160484145
+DATA/CDB2_SJ/10F14C2387F73F42E063C000000A77F3/DATAFILE/users.275.1160484149

Task 1.9: Bring over the TDE Keys

We need to copy the wallet key for both CDB and PDB. Copy the source CDB key into the target CDB and source PDB key into target PDB for which we are setting up Data Guard.

  1. Check wallet on source container or PDB.

    SET LINESIZE 200
COLUMN wrl_parameter FORMAT A39
select * from v$encryption_wallet;

  2. Export the encryption keys (CDB and PDB) from the source database.

    SQL> ADMINISTER KEY MANAGEMENT EXPORT ENCRYPTION KEYS WITH SECRET "export_secret" TO '/tmp/CDB1_key' FORCE KEYSTORE IDENTIFIED BY "WelC0me_123#";
SQL> alter session set container=CDB1_PDB1;
SQL> ADMINISTER KEY MANAGEMENT EXPORT ENCRYPTION KEYS WITH SECRET "export_secret" TO '/tmp/CDB1_PDB1_key' FORCE KEYSTORE IDENTIFIED BY "WelC0me_123#";

  3. Copy the encryption key on target server and import it into the target database (CDB and PDB respectively).

    [opc@sjCDB2 tmp]$ chmod 777 CDB1_key CDB1_PDB1_key
SQL> ADMINISTER KEY MANAGEMENT IMPORT KEYS WITH SECRET "export_secret" FROM '/tmp/CDB1_key' force keystore IDENTIFIED BY "WelC0me_123#" WITH BACKUP;
SQL> alter session set container=CDB1_PDB1;
SQL> ADMINISTER KEY MANAGEMENT IMPORT KEYS WITH SECRET "export_secret" FROM '/tmp/CDB1_PDB1_key' force keystore IDENTIFIED BY "WelC0me_123#" WITH BACKUP;

  4. Validate the imported key.

    select * from v$encryption_keys order by creation_time;

Task 1.10: Add the Standby Redo Logs (SRLs) to the PDB and Validate

Create the SRLs from a standby PDB to receive foreign source redo. This operation is required only once for the whole CDB and does not require a different set of SRLs per PDB.

sqlplus /@CDB2_sj as sysdba

SQL> alter session set container=CDB1_PDB1;
SQL> show pdbs
CON_ID CON_NAME OPEN MODE RESTRICTED
--------- ------------------------------ ---------- ----------
4 CDB1_PDB1 MOUNTED

SQL> select group#,thread#,bytes from v$standby_log;
no rows selected

SQL> ALTER DATABASE ADD STANDBY LOGFILE;
Database altered.

SQL> ALTER DATABASE ADD STANDBY LOGFILE;
Database altered.

SQL> ALTER DATABASE ADD STANDBY LOGFILE;
Database altered.

SQL> select group#,thread#,bytes from v$standby_log;
GROUP# THREAD# BYTES
--------- ---------- ----------
1 0 104857600
2 0 104857600
3 0 104857600

Task 1.11: Change the State of Standby to APPLY-ON

Connect to DGMGRL from any database and initiate redo transport from the source PDB to the target PDB.

DGMGRL> EDIT PLUGGABLE DATABASE CDB1_PDB1 AT CDB2_sj SET STATE='APPLY-OFF';
DGMGRL> EDIT PLUGGABLE DATABASE CDB1_PDB1 AT CDB2_sj SET STATE='APPLY-ON';
DGMGRL> enable configuration all;

Perform Log switch on source database:
--------------------------------------
SQL> ALTER SYSTEM ARCHIVE LOG CURRENT;
SQL> ALTER SYSTEM ARCHIVE LOG CURRENT;

DGMGRL> show pluggable database CDB1_PDB1 at CDB2_sj;
Pluggable database - CDB1_PDB1 at CDB2_sj
Data Guard Role: Physical Standby
Con_ID: 3
Source: con_id 3 at CDB1_ASH
Transport Lag: 32 seconds (computed 21 seconds ago)
Apply Lag: 32 seconds (computed 21 seconds ago)
Intended State: APPLY-ON
Apply State: Running
Apply Instance: CDB2
Average Apply Rate: 1592 KByte/s
Real Time Query: OFF
Pluggable Database Status:
SUCCESS


DGMGRL> SHOW ALL PLUGGABLE DATABASE AT CDB1_ASH;
PDB Name PDB ID Data Guard Role Data Guard Partner
CDB1_PDB1 3 Primary CDB1_PDB1 (con_id 3) at CDB2_sj
CDB1_PDB2 4 None None

DGMGRL> SHOW ALL PLUGGABLE DATABASE AT CDB2_sj;
PDB Name PDB ID Data Guard Role Data Guard Partner
CDB1_PDB1 3 Physical Standby CDB1_PDB1 (con_id 3) at CDB1_ash
CDB2_PDB3 4 None None
CDB2_PDB4 5 None None

DGMGRL> VALIDATE PLUGGABLE DATABASE CDB1_PDB1 at CDB2_sj;
Ready for Switchover: NO
Data Guard Role: Physical Standby
Apply State: Waiting for Redo Data
Standby Redo Log Files: 3
Source: CDB1_PDB1 (con_id 3) at CDB1_ASH


sqlplus /@CDB2_sj as sysdba

SQL> show pdbs
CON_ID CON_NAME OPEN MODE RESTRICTED
--------- ------------------------------ ---------- ----------
2 PDB$SEED READ ONLY NO
3 CDB1_PDB1 MOUNTED
4 CDB2_PDB3 READ WRITE NO
5 CDB2_PDB4 READ WRITE NO

Task 2: (Optional) Set up Oracle Data Guard at PDB Level from CDB2 to CDB1

This is an optional task if we wish to configure Oracle Data Guard from CDB2 to CDB1 as well i.e. bi-directional.

Primary PDB: CDB2_PDB3 AT CDB2_SJ

Standby PDB: CDB2_PDB3 AT CDB1_ASH

Note: We just need to change the source and target CDB or PDB and repeat Task 1.7 to 1.11.

  1. Configure Oracle Data Guard protection for the source PDB.

    dgmgrl /@CDB2_SJ
DGMGRL> ADD PLUGGABLE DATABASE CDB2_PDB3 AT CDB1_ASH SOURCE IS CDB2_PDB3 AT CDB2_sj PDBFILENAMECONVERT IS "'+DATA/CDB2_sj','+DATA/CDB1_ash'" 'keystore IDENTIFIED BY "WelC0me_123#"';

  2. Copy the datafiles to the destination.

    [oracle@sjCDB2 ~\]$ rman target sys/WelC0me_123#@CDB2_sj auxiliary sys/WelC0me_123#@CDB1_ash

  3. Bring over the TDE keys.

    Export / Import CDB/PDB key from CDB2 to CDB1

  4. Add the standby redo logs for DG PDB and validate.

  5. Change the state of standby to APPLY-ON.

    DGMGRL> SHOW ALL PLUGGABLE DATABASE AT CDB1_ash;
PDB Name PDB ID Data Guard Role Data Guard Partner
CDB1_PDB1 3 Primary CDB1_PDB1 (con_id 3) at CDB2_sj
VMDB1_PDB2 4 None None
VMDB2_PDB3 6 Physical Standby VMDB2_PDB3 (con_id 4) at CDB2_sj

DGMGRL> SHOW ALL PLUGGABLE DATABASE AT CDB2_sj;
PDB Name PDB ID Data Guard Role Data Guard Partner
CDB1_PDB1 3 Physical Standby CDB1_PDB1 (con_id 3) at CDB1_ash
VMDB2_PDB3 4 Primary VMDB2_PDB3 (con_id 6) at CDB1_ash
VMDB2_PDB4 5 None None
DGMGRL>

Task 3: Switchover the Pluggable Database

Reverse the roles of the source PDB and its designated target PDB using Oracle Data Guard broker DGMGRL. No effect at the container level.

  1. Perform a switchover.

    [oracle@ashCDB1 ~]$ dgmgrl /@CDB1_ash
DGMGRL> VALIDATE PLUGGABLE DATABASE CDB1_PDB1 at CDB2_sj;

Ready for Switchover:    NO

Data Guard Role:         Physical Standby
Apply State:             Not Running
Standby Redo Log Files:  3
Source:                  CDB1_PDB1 (con_id 3) at CDB1_ASH

DGMGRL> SWITCHOVER TO PLUGGABLE DATABASE CDB1_PDB1 at CDB2_sj;
Performing switchover NOW, please wait...
Switchover succeeded, new primary is "CDB1_pdb1"
DGMGRL>

  2. Open new PDB to open mode.

    sqlplus /@CDB2_sj
SQL> alter pluggable database CDB1_PDB1 open;
Pluggable database altered.

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
        2 PDB$SEED                      READ ONLY  NO
        3 CDB1_PDB1                     READ WRITE NO
        4 CDB2_PDB3                     READ WRITE NO
        5 CDB2_PDB4                     READ WRITE NO

DGMGRL>  show pluggable database CDB1_PDB1 at CDB2_SJ;
Pluggable database - CDB1_PDB1 at CDB2_sj

Data Guard Role:     Primary
Con_ID:              3
Active Target:       con_id 3 at CDB1_ASH

Pluggable Database Status:
SUCCESS

DGMGRL> show pluggable database CDB1_PDB1 at CDB1_ASH;

Pluggable database - CDB1_PDB1 at CDB1_ash

Data Guard Role:     Physical Standby
Con_ID:              3
Source:              con_id 3 at CDB2_SJ
Transport Lag:       1 minute (computed 59 seconds ago)
Apply Lag:           1 minute 35 seconds (computed 59 seconds ago)
Intended State:      APPLY-ON
Apply State:         Not Running

Pluggable Database Status:
ORA-16766: Redo Apply is stopped.
ORA-28374: typed master key not found in wallet

  3. Export the encryption keys (CDB only) from new source to new target.

    SQL> ADMINISTER KEY MANAGEMENT EXPORT ENCRYPTION KEYS WITH SECRET "export_secret" TO '/tmp/CDB2_CDB_key' FORCE KEYSTORE IDENTIFIED BY "WelC0me_123#";
keystore altered.

Import them into the target database
[opc@sjCDB2 tmp]$ chmod 777 mydbkey mydbkeypdb1

SQL> ADMINISTER KEY MANAGEMENT IMPORT KEYS WITH SECRET "export_secret" FROM '/tmp/CDB2_CDB_key' force keystore IDENTIFIED BY "WelC0me_123#" WITH BACKUP;
keystore altered.

  4. New standby redolog.

    DGMGRL> EDIT PLUGGABLE DATABASE CDB2_PDB2 AT CDB2_sj SET STATE='APPLY-OFF';
Succeeded.

sqlplus /@CDB1_ash

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED

        2 PDB$SEED                      READ ONLY  NO
        3 CDB1_PDB1                     MOUNT NO
        4 CDB1_PDB2                     READ WRITE NO

SQL> alter session set container=CDB1_PDB1;
Session altered.

SQL> select group#,thread#,bytes from v$standby_log;
no rows selected

SQL> ALTER DATABASE ADD STANDBY LOGFILE;
Database altered.
SQL> /
Database altered.
SQL> /
Database altered.

SQL> select group#,thread#,bytes from v$standby_log;
GROUP# THREAD# BYTES

        1          0  104857600
        2          0  104857600
        3          0  104857600

DGMGRL> EDIT PLUGGABLE DATABASE CDB1_PDB1 AT CDB1_ash SET STATE='APPLY-OFF';
Succeeded.
DGMGRL> EDIT PLUGGABLE DATABASE CDB1_PDB1 AT CDB1_ash SET STATE='APPLY-ON';
Succeeded.
DGMGRL>

Task 4: Failover the Pluggable Database

During a failover, the specified target PDB is changed to the source role. If the the original source container database and other PDBs are all functioning correctly, then PDB must be reinstated as a target PDB before any redo from the new source PDB can be applied.

$dgmgrl /@CDB1_ash

DGMGRL> show all pluggable database at CDB1_ash;
PDB Name PDB ID Data Guard Role Data Guard Partner
CDB1_PDB1 3 Primary CDB1_PDB1 (con_id 3) at CDB2_sj
VMDB1_PDB2 4 None None
VMDB2_PDB3 6 Physical Standby VMDB2_PDB3 (con_id 4) at CDB2_sj

DGMGRL> FAILOVER TO PLUGGABLE DATABASE CDB1_PDB1 at CDB2_sj;
Performing failover NOW, please wait...
Failover succeeded, new primary is "CDB1_PDB1".
DGMGRL>

DGMGRL> show pluggable database CDB1_PDB1 at CDB2_sj
Pluggable database - CDB1_PDB1 at CDB2_sj
Data Guard Role: Primary
Con_ID: 3
Active Target: con_id 3 at CDB1_ASH needs to be reinstated
Pluggable Database Status:

DGM-17450: not protected

DGMGRL> show pluggable database CDB1_PDB1 at CDB1_ash;
Pluggable database - CDB1_PDB1 at CDB1_ash
Data Guard Role: Physical Standby
Con_ID: 3
Source: (unknown)
Pluggable Database Status:

ORA-16661: The standby database must be reinstated.

DGMGRL> EDIT PLUGGABLE DATABASE CDB1_PDB1 AT CDB1_ash SET STATE='APPLY-OFF';
Succeeded.
DGMGRL> EDIT PLUGGABLE DATABASE CDB1_PDB1 AT CDB1_ash SET STATE='APPLY-ON';
Succeeded.

DGMGRL> show pluggable database CDB1_PDB1 at CDB1_ash;
Pluggable database - CDB1_PDB1 at CDB1_ash
Data Guard Role: Physical Standby
Con_ID: 3
Source: con_id 3 at CDB2_SJ
Transport Lag: 5 minutes 48 seconds (computed 21 seconds ago)
Apply Lag: (unknown)
Intended State: APPLY-ON
Apply State: Running
Apply Instance: VMDB1
Average Apply Rate: (unknown)
Real Time Query: OFF
Pluggable Database Status:
SUCCESS

Task 5: Remove a DG PDB Configuration

Run the following command to remove the configuration.

[oracle@ashCDB1 ~\]$ dgmgrl /@CDB1_ash

DGMGRL> REMOVE PLUGGABLE DATABASE CDB2_PDB3 AT CDB1_ASH REMOVE DATAFILES;
Pluggable Database 'CDB1_PDB1' removed.

DGMGRL> SHOW PLUGGABLE DATABASE CDB2_PDB3 AT CDB1_ASH;
No pluggable databases at database 'CDB2_SJ'

DGMGRL> REMOVE CONFIGURATION CDB1_ASH;
Succeeded.

DGMGRL> SHOW CONFIGURATION;
Configuration - Boston
Protection Mode: MaxPerformance
Members:
CDB1_ASH - Primary database
Fast-Start Failover: Disabled
Configuration Status:
SUCCESS (status updated 25 seconds ago)

Acknowledgments

More Learning Resources

Explore other labs on docs.oracle.com/learn or access more free learning content on the Oracle Learning YouTube channel. Additionally, visit education.oracle.com/learning-explorer to become an Oracle Learning Explorer.

For product documentation, visit Oracle Help Center.