Replication with Warm Standby
Maintain a continuously updated standby database system in a separate region using PostgreSQL Replication with Warm Standby.
Replication with Warm Standby is a disaster recovery tool that enables you to maintain a continuously updated standby database system in a separate region. By implementing this method, you can reduce recovery time, safeguard your data, and ensure business continuity when unexpected disruptions occur. Replication with Warm Standby supports disaster recovery with Recovery Point Objective (RPO) SLO both in the same region and cross region which enables data protection against unexpected events. Warm standby in the same region can enable further horizontal scaling as a single database system supports only 8 nodes.
Replication with Warm Standby enhances the following capabilities
- Disaster recovery and business continuity: By regularly replicating data from primary to secondary region, you can rapidly rebuild applications in another region if a region-wide disaster occurs.
- Migration and expansion: You can easily migrate and expand your applications to another region.
- Simplifying Compliance: Organizations subject to regulatory requirements for high availability or geographic distribution, such as financial or government entities, can meet compliance standards more easily.
Database System Roles
If replication with warm standby isn't set up, then all existing database systems are treated as standalone, with no replication. Each standalone database system operates independently and provides both read and write access (Read/Write).
When you set up a warm standby database, that database acts in the warm standby role. The associated standalone database acts in the primary database system role. Following is more information about these roles.
- Primary database system role:
- A primary database system allows data updates through its primary endpoint and replication with warm standby setup would have one database system in this role.
- Supports read and write operations (Read/Write access).
- Continuously streams write-ahead logs (WAL) to the standby database system for replication.
- Warm standby database system role:
- The warm standby database system is configured under cross-region replication (CRR) as the replica/DR system.
- Runs in read-only mode to ensure data consistency (read-only access).
- Continuously receives WAL updates from the primary.
- Can be promoted to primary system during a disaster, but otherwise is limited to read queries (such as reporting, analytics, or read-scale operations).
For information about configuring extensions, see Configurations and Extensions.
Prerequisites
Note and follow these constraints and best practices for setting up and configuring the primary and warm standby database systems.
Shape Requirements
You're allowed to select any heterogeneous shapes or input/output operations per second (IOPS) between the primary and warm standby databases. However, we recommend configuring them with the same shape and performance to ensure a seamless failover to the standby database system with a comparable performance profile in the event of a disaster.
IAM
To use Oracle Cloud Infrastructure services, you must have the appropriate security access through IAM policies. If you receive "permission denied" or "unauthorized" errors, ensure you have the required group membership and compartment access.
To gain access to use Replication with Warm Standby, create the following policy:
Allow group <group_name> to manage { POSTGRES_DB_SYSTEM_REPLICATE } in [ tenancy | compartment <compartment_name> | compartment id <compartment_ocid> ]
If you already have a policy granting management of postgres-db-systems at the database level, this policy isn't needed, because POSTGRES_DB_SYSTEM_REPLICATE is already included in the broader postgres-db-systems permissions.
Here is an example of the broader policy:
Allow group <group_name> to manage postgres-db-systems in [ tenancy | compartment <compartment_name> | compartment id <compartment_ocid> ]
Administrator Credentials
Users and roles are replicated from the primary database to the warm standby database. The warm standby database can't have dedicated users or roles.
The admin username for the warm standby database is the same as the admin username defined during the creation of the primary database system.
Failover
Automatic failover isn't supported. You must perform failover manually by promoting the warm standby database system.
Recovery Point Objective (RPO)
Use Recovery Point Objective (RPO) to help limit potential data loss when using replication with warm standby. Enable or disable RPO enforcement depending on customer requirements, such as availability and data protection needs.
RPO is the amount of data, measured in time, that can be lost in the event of a disaster. RPO is calculated as the wall clock time difference between the timestamp of the most recent write on the database and the oldest WAL record that hasn't yet been replicated.
Following is more information about RPO.
-
Write-ahead logs (WAL) in the primary region are asynchronously replicated to the warm standby in the secondary region.
-
RPO tracks the replication lag, which represents how far the standby is behind the primary.
- Supported and default values for the enforcement threshold:
- Supported range: 300 seconds (5 minutes) to 10,800 seconds (3 hours).
- Default value: 300 seconds (5 minutes).
- When RPO enforcement is enabled:
- OCI Database with PostgreSQL continuously evaluates RPO while committing a transaction.
-
When replication lag exceeds the configured RPO value:
-
The primary database system automatically switches to read-only mode.
-
This switch prevents further data changes and excessive data loss.
-
The warm standby is allowed to catch up, after which normal operations resume.
-
-
When RPO enforcement is disabled:
-
The database system continues normal operations even if replication lag exceeds the configured RPO.
-
The primary database doesn't switch to read-only mode.
-
Supported Database Versions for Warm Standby
The warm standby feature requires both the primary and warm standby database systems having database version of 14 or higher.
Configurations and Extensions
Extensions installed on the primary database system must always be same or subset of those applied on the warm standby database system.
You can create configurations for the warm standby database system in the replica region. These configurations aren't automatically replicated from the primary to the warm standby database systems as part of Replication with Warm Standby setup.
Setting Up Replication with Warm Standby
Replication with Warm Standby provides resilience by maintaining a warm standby database in a secondary OCI region. This standby is continuously updated from the primary in the source region and can be promoted manually in the event of a failure.
This section describes the tasks required to set up this environment.
Creating the Primary Database System
Creating the Warm Standby Database System
Verifying the Role Assignment
List Replica Database Systems
In the Oracle Cloud Console, the associated replica database systems are displayed on the Primary Database System details page, where users can view the list of replicas linked to the primary database system along with their corresponding regions and identifiers.
Converting a Warm Standby Database System to Standalone
The primary database system that had been associated with it changes automatically converts to standalone status as it's no longer part of a Replication with Warm Standby configuration.
Converting a Standalone Database System to Warm Standby
Open the details page of the converted warm standby database system and confirm the DB system role value is Warm standby.
Open the details page for the database system you specified as the primary and confirm the DB system role value is Primary. Select Replica DB systems to open the Replica DB system panel and ensure the associated warm standby database you converted is listed.
Switching Roles Between Primary and Warm Standby Database Systems
Disaster Recovery
This section describes the actions to take when a primary region outage occurs and how to restore normal operations after the region is recovered.
In this disaster recovery scenario, the primary database system is in a source region such as PHX. The primary database system has Read/Write access. The warm standby database system is a replica region such as LHR. The warm standby database system has Read-only access, and is always synchronized with the primary database system using cross region replication.
Convert the Warm Standby to Standalone
For instructions, see Converting a Standalone Database System to Warm Standby.
Convert the Original Primary to Warm Standby
After the source region is restored, the original primary database (PHX) remains a standalone database system.
Update the role of the original primary database system (PHX) to a warm standby database system as described in Converting a Standalone Database System to Warm Standby. Specify the original warm standby database server (LHR, now converted to standalone) as the primary database system.
Now the primary database system is LHR with Read/Write access, and the warm standby database server is PHX with Read-only access.
Reestablish the Original Database Setup
Use the switchover function to return the current primary database system (LHR) back to the original primary (PHX) as described in Switching Roles Between Primary and Warm Standby Database Systems. After the switchover occurs, the updated primary is PHX and the updated warm standby is LHR. You can resume normal database operations with the primary in the source region and the warm standby in the replica region.
Applying RPO
You can enable RPO for a primary database system and specify the desired RPO value in the following ways.
- When setting up Replication with Warm Standby
Applying RPO at setup ensures that RPO enforcement is applied from the start and that any potential data loss remains within the configured threshold. If the configured RPO is exceeded, the primary database system might temporarily transition to read-only mode until the standby catches up.
To enable RPO at setup, use the Choose primary database system panel. For more information, see Creating the Warm Standby Database System.
- After setting up Replication with Warm Standby
To enable (or disable) RPO after setup:
- Open the primary database's details page (as described in Getting a Database System's Details).
- Select Edit management policy from the Actions menu.
- In the Edit management policy panel, enable or disable RPO enforcement and configure the RPO value within the supported range.
- Save your changes.
Your changes are immediately applied.
Metrics
Use metrics to monitor replication and RPO status for both your primary and warm standby database systems.
From the database system's details page, select Metrics to display the Metrics page. Following are included metrics.
Primary Database Metrics
- Current RPO (seconds): Current RPO of the primary database system. Displays the current replication lag in seconds, indicating how far the standby is behind the primary.
- Current transaction mode: Current transaction mode of the primary database system.
1: Read/Write0: Read-onlyA value of
0indicates that the database system has turned into read-only mode due to RPO enforcement.
Warm Standby Database Metrics
- Last transaction replay epoch: Displays the time when last replayed transaction's write-ahead logs record was generated on the primary.
- Last transaction replay lag: Displays the time lag in milliseconds between the epoch at which the last transaction was replayed on a warm standby database system and the epoch at which the same transaction was committed on the primary database system.
CLI Support
You can use CLI commands to perform Replication with Warm Standby tasks.
Create the Primary Database System
Switch to the region where you want your primary database system to reside and create the database using the CLI as described in Creating a Database System.
Create Warm Standby Database System
Switch to the region where you want the warm standby database system to reside. Use the following command to create the warm standby.
Update source database system details like sourceType and primaryDbSystemId with the appropriate values. For example:
oci raw-request --http-method POST --auth security_token --config-file ~/.oci/config --profile DEFAULT --target-uri 'https://postgresql.us-phoenix-1.oci.oraclecloud.com/20220915/dbSystems' \
--request-body '{
"displayName": "Warm Standby Database",
"description": "Test db",
"compartmentId": "ocid.compartment.oc1..exampleuniqueID",
"dbVersion": "14",
"storageDetails": {
"systemType": "OCI_OPTIMIZED_STORAGE",
"isRegionallyDurable": false,
"availabilityDomain": "gXfg:UK-LONDON-1-AD-2"
},
"shape": "PostgreSQL.VM.Standard.E5.Flex",
"instanceOcpuCount": 2,
"instanceCount": 1,
"instanceMemorySizeInGBs": 32,
"networkDetails": {
"subnetId": "ocid1.subnet..exampleuniqueID"
},
"source": {
"sourceType": "DB_SYSTEM",
"primaryDbSystemId": "ocid1.postgresqldbsystem..exampleuniqueID"
}
}'
List Replica Database Systems
Use the following command to list warm standby database systems associated with the primary database system.
oci raw-request --http-method GET --target-uri 'https://postgresql.us-phoenix-1.oci.oraclecloud.com/20220915/dbSystems/<dbSystemId>/replicas' --region us-phoenix-1 --profile DEFAULT --config-file ~/.oci/config --auth security_token
where <dbSystemId> is the OCID of the primary database system.
Converting a Warm Standby Database System to Standalone
Use the following command to convert a warm standby database system to standalone.
oci raw-request --http-method POST --target-uri 'https://postgresql.us-phoenix-1.oci.oraclecloud.com/20220915/dbSystems/<warm_standby>/actions/changeRoleToStandalone' --region us-phoenix-1 --request-body '{"changeMode":"[immediately | "Replay Pending Updates"]"}' --profile DEFAULT --config-file ~/.oci/config --auth security_token
where <warm_standby> is the warm standby database system.
Converting a Standalone Database System to Warm Standby
Use the following command to convert a standalone database system to warm standby.
oci raw-request --http-method POST --target-uri 'https://postgresql.us-phoenix-1.oci.oraclecloud.com/20220915/dbSystems/<warm_standby>/actions/changeRoleToReplica' --region us-phoenix-1 --request-body '{"primaryDbSystemId":"<primary>"}' --profile DEFAULT --config-file ~/.oci/config --auth security_token
where <warm_standby> is the warm standby database system and <primary> is the primary database system.
Updating RPO
Use the following command to update the replication configuration (RPO settings) for a database system.
oci raw-request --http-method PUT --target-uri 'https://postgresql.us-phoenix-1.oci.oraclecloud.com/20220915/dbSystems/<dbSystemId>' --region us-phoenix-1 --request-body '{"replicationConfig": {"isRpoEnforced": true, "rpoInSeconds": 300}}' --profile DEFAULT --config-file ~/.oci/config --auth security_token
where <dbSystemId> is the OCID of the database system.
-
isRpoEnforced: Specifies whether RPO enforcement is enabled or disabled. -
rpoInSeconds: Specifies the RPO value (in seconds) to enforce when enabled. The value must be between 300 and 10,800 seconds.
Switchover
Use the following command to perform a switchover between a standalone database system and a warm standby.
oci raw-request --http-method POST --target-uri 'https://postgresql.us-phoenix-1.oci.oraclecloud.com/20220915/dbSystems/<primary>/actions/switchover' --region us-phoenix-1 --request-body
'{"replicaDbSystemId":"<warm-standby>"}' --profile DEFAULT --config-file ~/.oci/config --auth security_token
where <primary> is the primary database system and <warm_standby> is the warm standby database system.
Restrictions
Restore Database Systems
Restore Database system isn't supported for Replication with Warm Standby database systems (primary & warm standby).