Site-to-Site VPN Troubleshooting

Create a service request Ask the community

This topic covers the most common troubleshooting issues for Site-to-Site VPN. Some suggestions assume that you are a network engineer with access to your CPE device's configuration.

Log Messages

Viewing log messages generated for various operational aspects of Site-to-Site VPN can be a valuable aid in troubleshooting many of the issues presented during operation. Enabling and accessing the Site-to-Site VPN log messages can be done via Site-to-Site VPN or the Logging service.

Tunnel Flapping

Interesting traffic at all times: In general, Oracle recommends having interesting traffic running through the IPSec tunnels at all times if your CPE supports it. Certain Cisco ASA versions require the SLA monitor to be configured, which keeps interesting traffic running through the IPSec tunnels. For more information, see the section for "IP SLA Configuration" in the Cisco ASA policy-based configuration template.

Multiple IPSEC Connections: You can use two IPSec connections for redundancy. If both IPSec connections have only a default route (0.0.0.0/0) configured, traffic will route to either of those connections because Oracle uses asymmetric routing. If you want one IPSec connection as primary and another one as backup, configure more-specific routes for the primary connection and less-specific routes (or the default route of 0.0.0.0/0) on the backup connection.

Local IKE identifier: Some CPE platforms do not allow you to change the local IKE identifier. If you cannot, you must change the remote IKE ID in the Oracle Console to match your CPE's local IKE ID. You can provide the value either when you set up the IPSec connection, or later, by editing the IPSec connection. Oracle expects the value to be either an IP address or a fully qualified domain name (FQDN) such as cpe.example.com. For instructions, see Changing the CPE IKE Identifier That Oracle Uses.

Maximum Transmission Unit (MTU): The standard internet MTU size is 1500 bytes. For more information on how to determine your MTU please see Overview of MTU.

CPE Configuration

Local IKE identifier: Some CPE platforms do not allow you to change the local IKE identifier. If you cannot, you must change the remote IKE ID in the Oracle Console to match your CPE's local IKE ID. You can provide the value either when you set up the IPSec connection, or later, by editing the IPSec connection. Oracle expects the value to be either an IP address or a fully qualified domain name (FQDN) such as cpe.example.com. For instructions, see Changing the CPE IKE Identifier That Oracle Uses.

Cisco ASA: Policy Based: Oracle recommends using a route-based configuration to avoid interoperability issues and to achieve tunnel redundancy with a single Cisco ASA device.

The Cisco ASA does not support route-based configuration for software versions older than 9.7.1. For the best results, if your device allows it, Oracle recommends that you upgrade to a software version that supports route-based configuration.

With policy-based configuration, you can configure only a single tunnel between your Cisco ASA and your dynamic routing gateway (DRG).

Multiple Tunnels If you have multiple tunnels up simultaneously, ensure that your CPE is configured to handle traffic coming from your VCN on any of the tunnels. For example, you need to disable ICMP inspection, configure TCP state bypass, and so on. For more details about the appropriate configuration, contact your CPE vendor's support.

Encryption Domain Issues

The Oracle VPN headends use route-based tunnels, but can work with policy-based tunnels with some caveats. See Encryption domains for policy-based tunnels for full details.

Stateful security list rules: If you're using stateful security list rules (for TCP, UDP, or ICMP traffic), you don't need to ensure that your security list has an explicit rule to allow ICMP type 3 code 4 messages because the Networking service tracks the connections and automatically allows those messages. Stateless rules require an explicit ingress security list rule for ICMP type 3 code 4 messages. Confirm that the instance firewalls are set up correctly.

General Site-to-Site VPN Issues

IPSec tunnel is DOWN

Check these items:

IPSec tunnel is UP, but no traffic is passing through

Check these items:

  • Phase 2 (IPSec) configuration: Confirm that the phase 2 (IPSec) parameters are configured correctly on your CPE device. See the configuration appropriate for your CPE device:

  • VCN security lists: Ensure you've set up the VCN security lists to allow the desired traffic (both ingress and egress rules). Note that the VCN's default security list does not allow ping traffic (ICMP type 8 and ICMP type 0). You must add the appropriate ingress and egress rules to allow ping traffic.
  • Firewall rules: Ensure that your firewall rules allow both ingress and egress traffic with the Oracle VPN headend IPs and the VCN CIDR block.
  • Asymmetric routing: Oracle uses asymmetric routing across the multiple tunnels that make up the IPSec connection. Even if you configure one tunnel as primary and another as backup, traffic from your VCN to your on-premises network can use any tunnel that is "up" on your device. Configure your firewalls accordingly. Otherwise, ping tests or application traffic across the connection will not reliably work.
  • Cisco ASA: Do not use the originate-only option with an Oracle Site-to-Site VPN IPSec tunnel. It causes the tunnel's traffic to be inconsistently blackholed. The command is only for tunnels between two Cisco devices. Here's an example of the command that you should NOT use for the IPSec tunnels: crypto map <map name> <sequence number> set connection-type originate-only
IPSec tunnel is UP, but traffic is passing in only one direction

Check these items:

  • Asymmetric routing: Oracle uses asymmetric routing across the multiple tunnels that make up the IPSec connection. Even if you configure one tunnel as primary and another as backup, traffic from your VCN to your on-premises network can use any tunnel that is "up" on your device. Configure your firewalls accordingly. Otherwise, ping tests or application traffic across the connection will not reliably work.
  • Single tunnel preferred: If you want to use only one of the tunnels, ensure that you have the proper policy or routing in place on the CPE to prefer that tunnel.
  • Multiple IPSec connections: If you have multiple IPSec connections with Oracle, make sure to specify more specific static routes for the preferred IPSec connection.
  • VCN security lists: Ensure that your VCN security lists allow traffic in both directions (ingress and egress).
  • Firewall rules: Ensure that your firewall rules allow traffic in both directions with the Oracle VPN headend IPs and the VCN CIDR block.

Troubleshooting Site-to-Site VPN with a Policy-Based Configuration

IPSec tunnel is DOWN

Check these items:

  • Basic configuration: The IPSec tunnel consists of both phase-1 (ISAKMP) and phase-2 (IPSec) configuration. Confirm that both are configured correctly on your CPE device. See the configuration appropriate for your CPE device:

  • Local and remote proxy IDs: If you're using a policy-based configuration, check if your CPE is configured with more than one pair of local and remote proxy IDs (subnets). The Oracle VPN router supports only one pair on older connections. If your CPE has more than one pair, update the configuration to include only one pair, and choose one of the following two options:
    Option Local Proxy ID Remote Proxy ID
    1 ANY (or 0.0.0.0/0) ANY (or 0.0.0.0/0)
    2 On-premises CIDR (an aggregate that covers all the subnets of interest) VCN's CIDR

    Connections created after October 2020 in many regions are created using Site-to-Site VPN v2, which can support multiple encryption domains.

  • NAT device: If the CPE is behind a NAT device, the CPE IKE identifier configured on your CPE might not match the CPE IKE identifier Oracle is using (the public IP address of your CPE). If your CPE does not support setting the CPE IKE identifier on your end, you can provide Oracle with your CPE IKE identifier in the Oracle Console. For more information, see Overview of Site-to-Site VPN Components.
  • Cisco ASA: Do not use the originate-only option with an Oracle Site-to-Site VPN IPSec tunnel. It causes the tunnel's traffic to be inconsistently blackholed. The command is only for tunnels between two Cisco devices. Here's an example of the command that you should NOT use for the IPSec tunnels: crypto map <map name> <sequence number> set connection-type originate-only
IPSec tunnel is UP but keeps flapping

Check these items:

  • Initiation of connection: Ensure that your CPE device is initiating the connection.
  • Local and remote proxy IDs: If you're using a policy-based configuration, check if your CPE is configured with more than one pair of local and remote proxy IDs (subnets). The Oracle VPN router supports only one pair on older connections. If your CPE has more than one pair, update the configuration to include only one pair, and choose one of the following two options:
    Option Local Proxy ID Remote Proxy ID
    1 ANY (or 0.0.0.0/0) ANY (or 0.0.0.0/0)
    2 On-premises CIDR (an aggregate that covers all the subnets of interest) VCN's CIDR

    Connections created after October 2020 in many regions are created using Site-to-Site VPN v2, which can support multiple encryption domains.

  • Interesting traffic at all times: In general, Oracle recommends having interesting traffic running through the IPSec tunnels at all times if your CPE supports it. Certain Cisco ASA versions require the SLA monitor to be configured, which keeps interesting traffic running through the IPSec tunnels. For more information, see the section for "IP SLA Configuration" in the Cisco ASA policy-based configuration template.
IPSec tunnel is UP but traffic is unsteady

Check these items:

  • Local and remote proxy IDs: If you're using a policy-based configuration, check if your CPE is configured with more than one pair of local and remote proxy IDs (subnets). The Oracle VPN router supports only one pair on older connections. If your CPE has more than one pair, update the configuration to include only one pair, and choose one of the following two options:
    Option Local Proxy ID Remote Proxy ID
    1 ANY (or 0.0.0.0/0) ANY (or 0.0.0.0/0)
    2 On-premises CIDR (an aggregate that covers all the subnets of interest) VCN's CIDR

    Connections created after October 2020 in many regions are created using Site-to-Site VPN v2, which can support multiple encryption domains.

  • Interesting traffic at all times: In general, Oracle recommends having interesting traffic running through the IPSec tunnels at all times if your CPE supports it. Certain Cisco ASA versions require the SLA monitor to be configured, which keeps interesting traffic running through the IPSec tunnels. For more information, see the section for "IP SLA Configuration" in the Cisco ASA policy-based configuration template.

IPSec tunnel is only partially UP

Diagram showing multiple encryption domains and how to determine their number.

If you had a configuration similar to the example above and only configured three of the six possible IPv4 encryption domains on the CPE side, the link would be listed in a "Partial UP" state since all possible encryption domains are always created on the DRG side.

BGP Session Troubleshooting for Site-to-Site VPN

BGP status is DOWN

Check these items:

  • IPSec status: For the BGP session to be up, the IPSec tunnel itself must be up.
  • BGP address: Verify that both ends of the tunnel are configured with the correct BGP peering IP address.
  • ASN: Verify that both ends of the tunnel are configured with the correct BGP local ASN and Oracle BGP ASN. Oracle's BGP ASN for the commercial cloud is 31898. For the Government Cloud, see Oracle's BGP ASN.
  • MD5: Verify that MD5 authentication is disabled or not configured on your CPE device. Site-to-Site VPN does not support MD5 authentication.
  • Firewalls: Verify that your on-premises firewall or access control lists are not blocking the following ports:

    • TCP port 179 (BGP)
    • UDP port 500 (IKE)
    • IP protocol port 50 (ESP)

    If your CPE device's firewall is blocking TCP port 179 (BGP), the BGP neighborship state will always be down. Traffic cannot flow through the tunnel because the CPE device and Oracle router do not have any routes.

BGP status is flapping

Check these items:

  • IPSec status: For the BGP session to be up and not flapping, the IPSec tunnel itself must be up and not flapping.
  • Maximum prefixes: Verify that you are advertising no more than 2000 prefixes. If you're advertising more, BGP won't be established.
BGP status is UP, but no traffic is passing through

Check these items:

  • VCN security lists: Ensure you've set up the VCN security lists to allow the desired traffic (both ingress and egress rules). Note that the VCN's default security list does not allow ping traffic (ICMP type 8 and ICMP type 0). You must add the appropriate ingress and egress rules to allow ping traffic.
  • Correct routes on both ends: Verify that you have received the correct VCN routes from Oracle and the CPE device is using those routes. Likewise, verify that you are advertising the correct on-premises network routes over the Site-to-Site VPN, and the VCN route tables use those routes.
BGP status is UP, but traffic is passing in only one direction

Check these items:

  • VCN security lists: Ensure that your VCN security lists allow traffic in both directions (ingress and egress).
  • Firewalls: Verify that your on-premises firewall or access control lists are not blocking traffic to or from the Oracle end.
  • Asymmetric routing: Oracle uses asymmetric routing. If you have multiple IPSec connections, ensure that your CPE device is configured for asymmetric route processing.
  • Redundant connections: If you have redundant IPSec connections, ensure that they're both advertising the same routes.

Troubleshooting Redundant IPSec connections

Remember these important notes:

  • FastConnect uses BGP dynamic routing. Site-to-Site VPN IPSec connections can use either static routing or BGP, or a combination.
  • For important details about routing and preferred routes when using redundant connections, see Routing for Site-to-Site VPN.
  • You can use two IPSec connections for redundancy. If both IPSec connections have only a default route (0.0.0.0/0) configured, traffic will route to either of those connections because Oracle uses asymmetric routing. If you want one IPSec connection as primary and another one as backup, configure more-specific routes for the primary connection and less-specific routes (or the default route of 0.0.0.0/0) on the backup connection.
IPSec and FastConnect are both set up, but traffic is only passing through IPSec

Ensure that you use more specific routes for the connection you want as primary. If you're using the same routes for both IPSec and FastConnect, see the discussion of routing preferences in Routing for Site-to-Site VPN.

Two on-premises data centers each have an IPSec connection to Oracle, but only one is passing traffic

Verify that both IPSec connections are up and ensure that you have asymmetric route processing enabled on the CPE.

If both IPSec connections have only a default route (0.0.0.0/0) configured, traffic will route to either of those connections because Oracle uses asymmetric routing. If you want one IPSec connection as primary and another one as backup, configure more-specific routes for the primary connection and less-specific routes (or the default route of 0.0.0.0/0) on the backup connection.

For more information about this type of setup, see Example Layout with Multiple Geographic Areas.