Dynamic Routing Gateways (DRGs)

Working with DRGs and DRG attachments

When creating a DRG, you must specify the compartment where you want the DRG to reside. Placing the DRG in a compartment helps to limit access control. If you're not sure which compartment to use, put the DRG in the same compartment as a VCN you use regularly. For more information, see Access Control.

You might optionally assign a friendly name to the DRG. It doesn't have to be unique, and you can change it later. Oracle automatically assigns the DRG a unique identifier called an Oracle Cloud ID (OCID). For more information, see Resource Identifiers.

To use a DRG, it must be attached to other network resources. In the API, the process of attaching creates a DrgAttachment object with its own OCID. The DrgAttachment has a type field which denotes the type of object being attached to the DRG. The type field can be set to one of the following values:

• VCN
• VIRTUAL_CIRCUIT
• IPSEC_TUNNEL
• REMOTE_PEERING_CONNECTION

To attach a VCN to a DRG, use the CreateDrgAttachment operation or the console to explicitly create the DRG attachment object. Attachments for virtual circuits, IPSec tunnels, and remote peering connections are created (and deleted) automatically on your behalf when you create (or delete) the network object.

Working with DRG Route Tables and Route Distributions

A packet entering a DRG is routed using rules in the DRG route table assigned to that attachment. You can assign the same route table to multiple DRG attachments or create a dedicated route table for each attachment depending on the routing policies you want.

When you create a DRG, two default route tables are created for you: one for VCN attachments and one for all other attachments. When a route table is set as the default route table for an attachment type, the table is assigned to newly created attachments of that type unless an alternate table is explicitly specified. Route tables specified as the default for any type cannot be deleted. Ensure that a route table is not currently set as a default route table for an attachment type before trying to delete it.

A VCN attachment has two route tables: One DRG routing table for traffic entering the DRG, and one VCN routing table for traffic entering the VCN. The DRG route table exists in the DRG and is used to route packets entering the DRG through the attachment. The VCN route table is used to route packets entering the VCN through the attachment. If a VCN routing table is not defined, a hidden implicit table always provides connectivity to all subnets in the VCN.

Dynamic Route Import Distributions

A distribution is a list of declarative statements that contain a match criteria (such as an OCID or an attachment type) and an action. You can use route distributions to specify how routes get imported from or exported to a DRG attachment.

DRG route tables contain both static and dynamic routes. Static routes are inserted into tables using the API, while dynamic routes are imported from attachments and inserted using an import route distribution. When a statement's criteria matches on an attachment, the routes associated with the network object being attached to the DRG are dynamically imported into the DRG route table assigned to the containing distribution. If the statement is removed from the distribution, the routes are withdrawn from the DRG route tables. Statements in a route distribution are evaluated in priority order: the lowest number has the highest priority. The order in which statements are evaluated doesn't affect the preference set for the routes they import.

When building route distribution statements in the console, you can create a statement whose match type is "Match All". In the API, encode a "match all" statement by setting the match criteria to the empty list.

How do dynamic routes arrive at an attachment?

Routes to your on-premises networks are advertised from the CPE to IPSec tunnel and virtual circuit attachments using BGP. Routes are dynamically propagated from an RPC attachment on one DRG to an RPC attachment on another DRG through connected RPCs. Dynamic routes in a VCN include either all of the subnet CIDRs or all of the VCN CIDRs and all static route CIDRs configured on the VCN route table associated with the DRG attachment. The vcnRouteType property of the VCN attachment determines whether the subnet CIDRs or VCN CIDRs are propagated into the VCN attachment. The default behavior is to import the subnet CIDRs, but this behavior can be modified when creating or updating the VCN attachment.

Dynamic Route Export Distributions

When an attachment is assigned to a DRG route table, the contents of that table can be dynamically exported to the attachment. If the default export route distribution is assigned to an attachment, the entire contents of the attachment's assigned DRG route table are dynamically exported to the attachment. If you want to disable dynamic route exports to an attachment, use the API operation removeExportDrgRouteDistribution to set the attachment's exportDrgRouteDistributionId field to NULL. Dynamic route export to VCN attachments is not supported.

Route propagation restrictions

Routes imported from an IPSec tunnel or virtual circuit are never exported to other IPSec tunnel or virtual circuit attachments, regardless of how the export route distribution is configured. In a similar vein, packets which enter a DRG through an IPSec tunnel or virtual circuit attachment can never leave through an IPSec tunnel or virtual circuit attachment. Packets are dropped if routing is configured such that packets originating from IPSec tunnel or virtual circuit attachments are sent to IPSec tunnel or virtual circuit attachments.

ECMP

Equal-cost multi-path routing (ECMP) is a feature which allows flow-based load balancing of network traffic over multiple FastConnect virtual circuits or multiple IPSec tunnels (but not a mix of circuit types) using BGP. ECMP allows active-active load balancing and failover of network traffic between a maximum of eight circuits.

Oracle utilizes the protocol, destination IP, source IP, destination port, and source port to distinguish flows for load balancing purposes using a consistent and deterministic algorithm. Therefore, multiple flows are necessary to utilize all available bandwidth.

ECMP is off by default and can be enabled on a per-route table basis. Oracle only considers routes with identical route preference as eligible for ECMP forwarding. See Route Conflicts for more.

Route Source

DRG routes originate as either static routes or as dynamic routes from VCN, IPSec tunnel, FastConnect virtual circuit, or RPC attachments. This origin defines their source, which is an immutable characteristic of the route. In the API, the source is referred to as the routeProvenance of a DrgRouteRule.

Routes are propagated between DRGs using RPC attachments.

Routes with a source of IPSEC_TUNNEL or VIRTUAL_CIRCUIT are not exported to IPSec tunnel or virtual circuit attachments, regardless of the attachment's export distribution.

Routing a Subnet's Traffic to a DRG

The basic routing scenario sends traffic from a subnet in the VCN to the DRG. For example, if you're sending traffic from the subnet to your on-premises network, you set up a rule in the subnet's route table. The rule's destination CIDR is the CIDR for the on-premises network (or a subnet within), and the rule's target is the DRG. For more information, see VCN Route Tables.

Required IAM Policy

Peering VCNs using a DRG requires specific IAM permissions. See IAM policies related to DRG peering for details on the permissions needed.

DRG versions

DRGs created before May 17, 2021 use the legacy software, and can be upgraded to the most recent version. DRGs created after that have the upgraded features by default.

The following summarizes the difference between an upgraded and legacy DRG:

A legacy DRG:

• Has no programmable route tables. It has a default routing behavior where all traffic is forwarded from on-premises to an associated VCN and from the VCN to on-premises.
• Can attach to a single VCN. The DRG can only be used for remote VCN peering using an RPC.
• Can attach FastConnect or Site-to-Site VPN, or both. You can only reach resources in the local region using these connections.
• Can support an RPC connection with a remote DRG-VCN pair in the same tenancy. See Remote VCN Peering using a Legacy DRG to learn about remote peering using a legacy DRG.

An upgraded DRG:

• Has two route tables by default, and more can be added later.
• Can have many VCNs attached to it within the same region. Local VCN to VCN traffic can pass through a mutually connected DRG instead of an LPG. See Peering VCNs in the same region through a DRG for details.
• Can attach to on-premises using FastConnect orSite-to-Site VPN, or both. You can reach resources in both local and remote regions using these connections.
• Supports an RPC connection with a DRG/VCN pair in the same tenancy or another tenancy. See Peering VCNs in different regions through a DRG to learn about remote peering using an upgraded DRG.

The rest of this article has recently been updated to reflect the capabilities of an upgraded DRG, as have the common networking scenarios. Remote VCN Peering using a Legacy DRG is the only routing or peering scenario specific to a legacy DRG.

Before you upgrade a DRG

To take advantage of enhanced DRG features, upgrade your DRG. The DRG upgrade process is automated, but you must have the required permissions to initiate an upgrade.

Upgrading a DRG is a one-way process with no option to roll back to a legacy DRG after the upgrade process has been initiated.

Expect there to be a traffic outage for all the DRG's attachments during the upgrade process. Each attachment is updated in turn, forcing each upgrading attachment into a provisioning state where it will no longer forward traffic. If you have redundant connections, traffic will failover to other circuits while one is undergoing upgrade. If you only have one circuit, it could go down for about 20 minutes. Any existing BGP sessions for your on-premises connections (FastConnect or Site-to-Site VPN) are also reset while the attachment and any Site-to-Site VPN IPSec tunnels are also brought offline.

For example, if your DRG has two FastConnect virtual circuit attachments, one virtual circuit is upgraded first, causing it to drop connectivity while traffic can pass over the other virtual circuit. After that update has finished, the upgrade process upgrades the second virtual circuit attachment and the completed virtual circuit is brought back online. Expect the upgrade process to last up to 30 minutes per on-premises attachment.

Remote peering connections do not need to be reset like virtual circuit or IPSec tunnel connections do, and do not take the same length of time to upgrade.

After the DRG upgrade process has completed, any Site-to-Site VPN IPSec tunnels are brought back online and all BGP sessions for FastConnect and Site-to-Site VPN are re-established. By default, the upgraded DRG has two autogenerated DRG route tables and import route distributions enabled for your attachments. These resources are designed for backward compatibility with your legacy DRG and allow for all previous communication to resume in the same manner as before the upgrade without any additional user intervention.

For step-by-step instructions on how to upgrade your DRG refer to Upgrading a DRG.

Scenarios

We have provided some detailed networking scenarios to help you understand the role of a DRG in the Networking service and how the components work together in general.

Route Conflicts

If two routes with identical CIDRs are observed the same DRG route table, the DRG resolves the conflict using the following criteria:

1. Static routes always have higher preference than dynamic routes.

   Note
   
   You cannot manually specify two static routes with the same CIDR in the same DRG route table, but it's possible for more than one route with the same CIDR to be imported dynamically.
2. Conflicts between dynamic routes are resolved by first analyzing the route's AS Path:
   • Routes with a route source of VCN or STATIC always have an empty AS Path.
   • Routes with a route source of IPSEC_TUNNEL or VIRTUAL_CIRCUIT will have populated AS Path data (see Routing Details for Connections to Your On-Premises Network for more details).
3. Otherwise, the attachment type that imported the route is evaluated according to the following priority based on the attachment type:
   1. VCN: the DRG makes an arbitrary but stable selection.
   2. VIRTUAL_CIRCUIT: If ECMP is disabled, the DRG makes an arbitrary but stable selection. If ECMP is enabled, all routes will be added to the route table and the DRG makes routing choices using ECMP. The maximum supported ECMP width inside a DRG is 8.
   3. IPSEC_TUNNEL: If ECMP is disabled, the DRG makes an arbitrary but stable selection.If ECMP is enabled, all routes will be added to the route table and the DRG makes routing choices using ECMP. The maximum supported ECMP width inside a DRG is 8.

   4. REMOTE_PEERING_CONNECTION: The DRG will choose the route with the lowest network distance.

      If two routes have identical network distances, the DRG selects the route with the highest priority route source (STATIC > VCN > VIRTUAL_CIRCUIT> IPSEC_TUNNEL).

      If two routes have the same route source, the DRG makes an arbitrary but stable selection.

4. If conflicting routes are imported from attachments of the same type, the conflict is resolved differently depending on the attachment type:
   1. VCN attachments: If identical CIDRs are imported from two VCN attachments, only one is selected using an arbitrary but stable decision procedure.
   2. VIRTUAL_CIRCUIT and IPSEC_TUNNEL attachments: If multiple routes with the same CIDR and different AS_PATH lengths are imported into a DRG route table, the route with the lowest AS_PATH length is selected. Otherwise, one route is chosen using an arbitrary but stable decision procedure.
   3. RPC attachments: If identical CIDRs are imported from two RPC attachments, one of them is chosen using an arbitrary stable decision procedure.

You can observe the results of conflict resolution by listing the contents of the route table. Deprecated routes are marked with the "conflict" status.

Limitations

Some functions might appear to be possible based on the structure of the resource interaction model, but the following functions are not currently allowed:

1. Explicit creation or deletion of RPC, IPSec tunnel, or virtual circuit attachments
2. Static routes in DRG route tables with next-hop of IPSec tunnel or virtual circuit attachments
3. Use of non-default export route distributions
4. Dynamic route export to VCN attachments
5. Propagating a route through more than 3 DRG route tables
6. Routes propagating via RPC through more than 4 DRGs