NAT on DRG - Part 2 - Overlapping CIDRs between multiple remote VCNs

Introduction

In NAT on DRG - Part 1 - Overlapping CIDRs between two local VCNs two VCNs with the same CIDR were attached to one Dynamic Routing Gateway (DRG) in one region. In practice overlapping networks are often further apart: a production environment in one Oracle Cloud Infrastructure (OCI) region and a copy of it in another region, or two business units that each built their own landing zone in their own region with the same address plan.

Regions are connected with a remote peering connection (RPC) between two DRGs. An RPC normally refuses to be useful when both sides advertise the same CIDR, because neither DRG can tell local from remote. NAT on DRG solves this by translating the addresses on the RPC attachment, so that each region is known to the other one under a unique “shadow” CIDR.

Region VCN Real subnet CIDR Shadow CIDR seen by the other region
Canada Southeast (Montreal), YUL uc2-vcn-yul 10.0.1.0/24 192.168.10.0/24
Canada Southeast (Toronto), YYZ uc2-vcn-yyz 10.0.1.0/24 192.168.20.0/24

The difference with Part 1 is the place where the NAT policy sits. In Part 1 the policies were associated with the VCN attachments. In this tutorial they are associated with the RPC attachments, and the VCN attachments have no policy at all. This has a useful side effect: the translation applies to everything that crosses the RPC, no matter which VCN in the region it comes from, and VCNs that do not overlap are not affected.

[!NOTE] A NAT rule describes traffic entering the DRG through the attachment that holds the policy. For an RPC attachment that is traffic arriving from the other region. For traffic leaving through that same attachment the DRG applies the reverse translation automatically.

This tutorial shows two ways to configure the rules, and both are tested:

image

Other Use Cases

This tutorial is part of a series of five. Every part solves a different overlap scenario with the same building blocks.

Diagram

Overlapping subnet CIDRs in different OCI regions resolved with stateless 1:1 NAT on DRG (This scenario maintains the translation for all VCNs in the region, including non-overlapping ones)

image

The diagram shows the same topology twice, once for each rule option. Everything except the NAT rules is identical:

Prerequisites

[!NOTE] The names, CIDRs and IP addresses in this tutorial come from the lab in the screenshots. You can use your own, as long as the shadow CIDRs do not overlap with anything else in your network and have the same prefix length as the CIDR they replace.

Steps

Step 01: Review the network in Montreal (YUL)

We start in the first region. The VCN here is an ordinary VCN with one public subnet. The only thing that matters for this tutorial is its address space, which is identical to the VCN in the other region.

image

image

The internet gateway is only here so that you can reach the test instance over SSH.

image

The instances in Montreal reach Toronto on its shadow CIDR 192.168.20.0/24, so that is the destination that goes to the DRG.

image

The test is a ping, so the security list must allow ICMP. A ping from Toronto arrives with a source address in a shadow range, not in 10.0.1.0/24.

The lab allows ICMP from all three private ranges. That is broader than this tutorial needs. The minimum is one ICMP rule per VCN, with the shadow CIDR of the other region as source: 192.168.20.0/24 in Montreal and 192.168.10.0/24 in Toronto.

image

image

Step 02: Create the DRG, the RPC and the DRG route tables in Montreal

Each region has its own DRG. The DRG in Montreal gets two attachments: one for the VCN and one for the remote peering connection to Toronto. Each attachment gets its own DRG route table. The autogenerated tables do not help here: the shadow CIDR of Toronto is not advertised by any attachment, so the route to it has to be a static route. Two small tables with one static route each keep the routing explicit.

The screenshots in this step show the end result. The DRG route tables uc2-drgrt-vcn-yul and uc2-drgrt-rpc-yul are not created automatically. Create them on the Routing tab of the DRG, add the static route that is shown for each of them, and select them as the DRG route table of the VCN attachment and of the RPC attachment. The static route to the RPC can only be added after the RPC exists.

image

image

The DRG route table of the VCN attachment handles traffic that comes from the VCN. It sends the shadow CIDR of Toronto to the RPC.

image

image

The DRG route table of the RPC attachment handles traffic that arrives from Toronto. The destination in this table is the real CIDR 10.0.1.0/24, because the NAT policy on the RPC attachment has already translated the destination by the time the DRG does its route lookup.

image

image

Step 03: Create the DRG NAT policy in Montreal

The NAT policy is created first and filled with rules later. A policy without rules translates nothing, so it is safe to associate it with the RPC attachment already.

image

image

Step 04: Review the network in Toronto (YYZ)

Toronto is the mirror image of Montreal. The VCN has the same CIDR, and the route table points to the shadow CIDR of Montreal.

image

image

image

image

image

image

Step 05: Create the DRG, the RPC and the DRG route tables in Toronto

The DRG in Toronto is built in the same way, and at the end of this step the two RPCs are connected to each other.

As in Montreal, the DRG route tables uc2-drgrt-vcn-yyz and uc2-drgrt-rpc-yyz are not created automatically. Create them, add the static route that is shown for each of them, and select them on the two attachments.

image

image

image

image

image

Now the two regions can be connected. You do this from one side only.

image

Step 06: Create the DRG NAT policy in Toronto

Toronto gets its own policy, also associated with the RPC attachment.

image

image

Both regions are now fully prepared. The routing is in place and both RPC attachments have a NAT policy. The only thing missing is the rules, and that is where the two options differ.

Step 07: Configure the NAT rules, option 1: a single rule in one region

In the first option all the translation is done in Montreal. The policy in Montreal gets one rule that translates source and destination, and the policy in Toronto stays empty.

The rule sits on the RPC attachment in Montreal, so it is written for traffic that arrives from Toronto:

  Original Translated Meaning
Source 10.0.1.0/24 192.168.20.0/24 The real addresses of Toronto are presented in Montreal as 192.168.20.0/24.
Destination 192.168.10.0/24 10.0.1.0/24 The shadow CIDR of Montreal is replaced with the real CIDR of Montreal.

image

image

This is what happens to a ping from uc2-inst-yyz in Toronto to uc2-inst-yul in Montreal:

Where Source Destination What happens
Leaves uc2-inst-yyz 10.0.1.152 192.168.10.121 The route table uc2-rt-yyz sends 192.168.10.0/24 to the DRG.
DRG in Toronto 10.0.1.152 192.168.10.121 uc2-drgrt-vcn-yyz sends 192.168.10.0/24 to the RPC. No rules, so nothing is translated.
Enters the DRG in Montreal on the RPC attachment 192.168.20.152 10.0.1.121 The rule in uc2-policy-yul translates source and destination.
Route lookup in uc2-drgrt-rpc-yul 192.168.20.152 10.0.1.121 10.0.1.0/24 points to uc2-att-vcn-yul.

And this is the reply:

Where Source Destination What happens
Leaves uc2-inst-yul 10.0.1.121 192.168.20.152 The route table uc2-rt-yul sends 192.168.20.0/24 to the DRG.
Route lookup in uc2-drgrt-vcn-yul 10.0.1.121 192.168.20.152 192.168.20.0/24 points to the RPC.
Leaves the DRG in Montreal on the RPC attachment 192.168.10.121 10.0.1.152 The DRG applies the reverse of the rule in uc2-policy-yul.
DRG in Toronto 192.168.10.121 10.0.1.152 uc2-drgrt-rpc-yyz sends 10.0.1.0/24 to uc2-att-vcn-yyz.

The last octet of an address is preserved by the 1:1 translation, which tells you which address to ping:

Instance Real IP address Address to use from the other region
uc2-inst-yul 10.0.1.121 192.168.10.121
uc2-inst-yyz 10.0.1.152 192.168.20.152

image

image

[!NOTE] With this option the whole NAT configuration lives in one region. That is practical when you only manage one side of the peering, for example when the other region belongs to another team or another tenancy.

Step 08: Configure the NAT rules, option 2: a rule in each region

In the second option each region is responsible for its own shadow CIDR. Both policies get one rule, and both rules only translate the destination.

Each rule is again written for traffic that arrives from the other region:

Policy Original destination Translated destination Meaning
uc2-policy-yul 192.168.10.0/24 10.0.1.0/24 Traffic for the shadow CIDR of Montreal is delivered to the real CIDR of Montreal.
uc2-policy-yyz 192.168.20.0/24 10.0.1.0/24 Traffic for the shadow CIDR of Toronto is delivered to the real CIDR of Toronto.

There is no source NAT rule, and none is needed. The reverse translation that the DRG applies to traffic leaving through the RPC attachment does that work: when a packet from 10.0.1.121 leaves Montreal, the reverse of the Montreal rule turns its source into 192.168.10.121.

image

image

This is what happens to a ping from uc2-inst-yul in Montreal to uc2-inst-yyz in Toronto:

Where Source Destination What happens
Leaves uc2-inst-yul 10.0.1.121 192.168.20.152 The route table uc2-rt-yul sends 192.168.20.0/24 to the DRG.
Route lookup in uc2-drgrt-vcn-yul 10.0.1.121 192.168.20.152 192.168.20.0/24 points to the RPC.
Leaves the DRG in Montreal on the RPC attachment 192.168.10.121 192.168.20.152 The DRG applies the reverse of the rule in uc2-policy-yul to the source.
Enters the DRG in Toronto on the RPC attachment 192.168.10.121 10.0.1.152 The rule in uc2-policy-yyz translates the destination.
Route lookup in uc2-drgrt-rpc-yyz 192.168.10.121 10.0.1.152 10.0.1.0/24 points to uc2-att-vcn-yyz.

Between the two regions the packet carries shadow addresses only. The real 10.0.1.x addresses never leave their own region. The reply follows the same steps in the other direction.

image

image

[!NOTE] Use one option or the other, not both at the same time. Option 2 is the cleaner design when you manage both regions: each region owns the translation of its own CIDR, the rules are identical in shape, and only shadow addresses travel between the regions.

[!NOTE] NAT on DRG does not translate DNS. If your applications find each other by name, the DNS records that the other region uses must point to the shadow addresses, not to the real ones.

Conclusion

Two VCNs with the same subnet CIDR, in two different regions, now communicate privately over a remote peering connection. Neither VCN was re-addressed and there is no firewall or NAT appliance in the path. The translation is done by the DRGs on the RPC attachments.

Placing the policy on the RPC attachment instead of the VCN attachment changes where things happen. The DRG route table of the VCN attachment routes on the shadow CIDR, because nothing has been translated yet when the packet comes in from the VCN. The DRG route table of the RPC attachment routes on the real CIDR, because the packet has been translated on its way in. And because the policy sits on the link between the regions, it covers every VCN behind the DRG without touching the VCN attachments.

The two rule options lead to the same result for the instances. A single rule in one region keeps all the configuration in one place, and a destination NAT rule in each region gives a symmetrical design in which the real addresses stay inside their own region. In NAT on DRG - Part 3 - Overlapping CIDRs between OCI and On-Premisesthe remote side is no longer an OCI region but an on-premises network behind an IPSec tunnel.

Acknowledgments

More Learning Resources

Explore other labs on docs.oracle.com/learn or access more free learning content on the Oracle Learning YouTube channel. Additionally, visit education.oracle.com/learning-explorer to become an Oracle Learning Explorer.

For product documentation, visit Oracle Help Center.