NAT on DRG - Part 4 - Shared services VCN

Introduction

A shared services VCN is a very common design in Oracle Cloud Infrastructure (OCI). One central VCN hosts the services everybody needs, such as DNS, monitoring, backup, a jump host or a licence server, and many other VCNs consume those services through a Dynamic Routing Gateway (DRG).

The design works well until two consumers use the same IP range. A service provider that connects customers to a shared platform runs into this almost immediately, because the provider does not control the addressing of its customers. The same happens inside a large organisation where business units built their VCNs independently. When two consumers both use 10.0.1.0/24, the shared services VCN cannot send a reply back: the address 10.0.1.13 could be in either VCN.

NAT on DRG solves this on the DRG itself. Each consumer VCN gets a DRG NAT policy on its attachment that translates its source addresses to a unique “shadow” CIDR. The shared services VCN only ever sees the shadow CIDRs, so every consumer is unique from its point of view.

VCN Role Real subnet CIDR CIDR seen by the shared services VCN
uc4-vcn-c1 Customer 1 10.0.1.0/24 192.168.10.0/24
uc4-vcn-c2 Customer 2 10.0.1.0/24 192.168.20.0/24
uc4-vcn-shared Shared services 10.50.1.0/24 Not translated

Compared to the earlier parts of this series there are two differences worth pointing out:

[!NOTE] A NAT rule describes traffic entering the DRG through the attachment that holds the policy. For traffic in the opposite direction, leaving the DRG through that same attachment, the DRG applies the reverse translation automatically. You do not create a second rule for the return path.

image

Other Use Cases

This tutorial is part of a series of five. Every part solves a different overlap scenario with the same building blocks.

Diagram

Multiple VCNs must access a common shared service, with potential overlap between organizations

image

The diagram shows every object you configure in this tutorial:

Prerequisites

[!NOTE] The names, CIDRs and IP addresses in this tutorial come from the lab in the screenshots. You can use your own, as long as the shadow CIDRs do not overlap with anything else in your network and have the same prefix length as the CIDR they replace.

Steps

Step 01: Review the three VCNs

We start with the three VCNs. Two of them belong to customers who picked the same address space, and the third one hosts the shared services with a range that nobody else uses.

image

image

image

image

Step 02: Review the subnets

Both customer VCNs have a single public subnet with the CIDR 10.0.1.0/24. This is the range that the NAT rules translate later. The shared services VCN has one subnet, 10.50.1.0/24, which is not translated.

image

image

Step 03: Review the internet gateways

The internet gateways have nothing to do with NAT on DRG. They are only here so that you can reach the test instances over SSH on their public IP addresses.

image

image

image

Step 04: Configure the security lists

The test in this tutorial is a ping, so the security lists must allow ICMP. The shared services VCN is the interesting one. It never sees a packet from 10.0.1.0/24. Traffic from the customers arrives with a source address in 192.168.10.0/24 or 192.168.20.0/24, so those are the ranges its security rules must allow. In a real deployment this is useful: you can write a separate security rule per customer, even though the customers use the same real addresses.

The lab allows ICMP from all three private ranges. That is broader than this tutorial needs. The shared services VCN only needs ICMP from the two shadow CIDRs 192.168.10.0/24 and 192.168.20.0/24. The customer VCNs only need ICMP from 10.50.0.0/16, and only for pings that start in the shared services VCN.

image

Step 05: Create the DRG and attach the three VCNs

The DRG is the router between the three VCNs and the place where the translation happens. All three VCNs are attached to the same DRG, each with its own VCN attachment. The route tables and the NAT policies in the next steps are all bound to an attachment, not to the DRG as a whole.

image

[!NOTE] We come back to this Attachments tab in Step 09. At that point every attachment also shows its own DRG route table, and the customer attachments show their DRG NAT policy.

Step 06: Create a DRG route table for each attachment

A DRG route table decides where a packet goes after it has entered the DRG through an attachment. By default all VCN attachments share one autogenerated route table that is filled with the CIDRs of the attached VCNs. With two customers that advertise the same 10.0.1.0/24, the DRG keeps only one of the two routes in that table and marks the other one as a conflict. That conflict is not what breaks this design, because nothing here is ever routed to 10.0.1.0/24. The problem is the way back: replies from the shared services VCN are addressed to the shadow CIDRs, and no attachment advertises those, so the autogenerated table has no route for them. A shared table would also give every attachment the same routes, while the customers should only see the shared services VCN. Each attachment therefore gets its own DRG route table with static routes.

There are two things to notice in these tables:

image

image

image

Step 07: Configure the VCN route tables

The instances must know which destinations are reachable through the DRG. The customers talk to the real addresses of the shared services VCN, so they route 10.50.0.0/16 to the DRG. The shared services VCN talks to the customers on their shadow addresses, so it routes both shadow CIDRs to the DRG.

image

image

image

Step 08: Create the DRG NAT policies and rules

This is the step the whole tutorial is about. We create one DRG NAT policy per customer, and each policy gets one rule. The rule only translates the source address:

Policy Original source Translated source Meaning
uc4-policy-c1 10.0.1.0/24 192.168.10.0/24 Customer 1 presents itself as 192.168.10.0/24.
uc4-policy-c2 10.0.1.0/24 192.168.20.0/24 Customer 2 presents itself as 192.168.20.0/24.

The destination is left alone. The customers address the shared services on their real 10.50.x.x addresses, and those are unique, so there is nothing to translate.

image

image

image

[!NOTE] The two rules look almost identical, and they both match the same original source 10.0.1.0/24. That is not a conflict. Each policy is bound to its own attachment, and a packet is only checked against the policy of the attachment it crosses.

Step 09: Associate the NAT policies with the customer attachments

A policy becomes active for traffic when it is associated with a DRG attachment. Each attachment can have one NAT policy. We associate the two policies with the two customer attachments. The attachment of the shared services VCN gets no policy.

image

image

You can verify the complete configuration from the DRG side as well.

image

image

image

image

Step 10: Test the connectivity

Everything is in place, so it is time to send some packets. The last octet of an address is preserved by the 1:1 translation, which tells you under which address the shared services VCN knows each customer instance:

Instance Real IP address Address seen by the shared services VCN
uc4-inst-c1 10.0.1.13 192.168.10.13
uc4-inst-c2 10.0.1.171 192.168.20.171
uc4-inst-shared 10.50.1.51 Not translated

This is what happens to a ping from uc4-inst-c1 to uc4-inst-shared:

Where Source Destination What happens
Leaves uc4-inst-c1 10.0.1.13 10.50.1.51 The route table uc4-rt-c1 sends 10.50.0.0/16 to the DRG.
Enters the DRG on uc4-att-c1 192.168.10.13 10.50.1.51 The rule in uc4-policy-c1 translates the source.
Route lookup in uc4-drgrt-c1 192.168.10.13 10.50.1.51 10.50.0.0/16 points to uc4-att-shared.
Arrives at uc4-inst-shared 192.168.10.13 10.50.1.51 The instance sees a ping from the shadow address of customer 1.

And this is the reply:

Where Source Destination What happens
Leaves uc4-inst-shared 10.50.1.51 192.168.10.13 The route table uc4-rt-shared sends 192.168.10.0/24 to the DRG.
Enters the DRG on uc4-att-shared 10.50.1.51 192.168.10.13 No NAT policy on this attachment, so nothing is translated.
Route lookup in uc4-drgrt-shared 10.50.1.51 192.168.10.13 192.168.10.0/24 points to uc4-att-c1.
Leaves the DRG on uc4-att-c1 10.50.1.51 10.0.1.13 The DRG applies the reverse of the rule in uc4-policy-c1.

First we test from the customers to the shared services.

image

image

The shared services VCN can also start the conversation. It reaches each customer on its shadow address, and the DRG translates the destination back to the real address when the packet leaves through the customer attachment.

image

image

[!NOTE] NAT on DRG does not translate DNS. If the shared services need to find the customer systems by name, the DNS records they use must point to the shadow addresses, not to the real ones.

Conclusion

Two customers with exactly the same subnet CIDR now use the same shared services VCN, and the shared services can tell them apart. The only thing that was added to a normal hub-and-spoke design is a source NAT rule on each customer attachment. The shared services VCN itself needs no NAT policy and no knowledge of the real customer addresses.

The design also scales in a predictable way. Adding a third customer means one more VCN attachment, one more NAT policy with a new shadow CIDR, one more DRG route table that points to the shared services VCN, and one more route in uc4-drgrt-shared and uc4-rt-shared. Nothing changes for the existing customers, and because the customer route tables only contain the shared services VCN, the customers remain isolated from each other.

Part 1 translated between two VCNs on the same DRG, Part 2 moved the translation to a remote peering connection between regions, Part 3 applied it to an IPSec tunnel towards on-premises, and this part used source NAT only to make overlapping consumers unique for a shared service. NAT on DRG - Part 5 - Overlapping CIDRs between OCI and On-Premises with BGP repeats the on-premises use case with BGP dynamic routing. In all of them the method is the same: write the rule for traffic entering the attachment, route on the address after translation, and make sure every network knows the way to the shadow CIDRs it has to reach.

Acknowledgments

More Learning Resources

Explore other labs on docs.oracle.com/learn or access more free learning content on the Oracle Learning YouTube channel. Additionally, visit education.oracle.com/learning-explorer to become an Oracle Learning Explorer.

For product documentation, visit Oracle Help Center.